Compare commits

..
Author SHA1 Message Date
ZakiandClaude Opus 4.6 3a8d4e0104 fix: mask master key in stdout output and consolidate tests
- Mask the generated SECRETS_MASTER_KEY in stdout using mask_api_key()
  to avoid leaking the full key in CI/Docker logs
- Consolidate two overlapping regression tests into one

Addresses review feedback on PR #673.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-07 10:56:05 -08:00
ZakiandClaude Opus 4.6 e1ffd30d37 fix(setup): initialize secrets crypto in env-var mode (#666)
When the user chose "Environment variable" in Step 2 (Security), the
wizard generated a master key but never initialized self.secrets_crypto,
causing subsequent API key saves in Step 3 to fail silently.

Three fixes:
- Initialize SecretsCrypto from the generated key (matching keychain path)
- Store the key hex in secrets_master_key_hex for write_bootstrap_env to
  persist to ~/.ironclaw/.env automatically
- Fix misleading message (shell profiles don't work, only .env files)

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-07 10:40:49 -08:00
148 changed files with 1876 additions and 14874 deletions
-13
View File
@@ -5,19 +5,6 @@ DATABASE_POOL_SIZE=10
# LLM Provider
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil
# LLM_REQUEST_TIMEOUT_SECS=120 # Increase for local LLMs (Ollama, vLLM, LM Studio)
# === Anthropic Direct ===
# Two auth modes:
# 1. API key: Set ANTHROPIC_API_KEY (from console.anthropic.com/settings/keys)
# 2. OAuth token: Set ANTHROPIC_OAUTH_TOKEN (from `claude login`)
# OAuth tokens use Authorization: Bearer instead of x-api-key header.
# ANTHROPIC_API_KEY=sk-ant-...
# ANTHROPIC_OAUTH_TOKEN=sk-ant-oat01-... # from `claude login` credentials
# ANTHROPIC_MODEL=claude-sonnet-4-20250514
# === OpenAI Direct ===
# OPENAI_API_KEY=sk-...
# === NEAR AI (Chat Completions API) ===
# Two auth modes:
-99
View File
@@ -1,99 +0,0 @@
name: Claude Code Review
on:
pull_request:
types: [opened, labeled]
permissions:
contents: read
pull-requests: write
issues: write
id-token: write
concurrency:
group: claude-review-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
review:
name: Claude Code Review
if: contains(github.event.pull_request.labels.*.name, 'staging-promotion')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Run Claude Code review
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
claude_args: "--max-turns 50 --model claude-haiku-4-5-20251001 --allowedTools 'Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh search:*),Bash(git blame:*),Bash(git log:*),Bash(git diff:*)'"
prompt: |
Code review this pull request. Follow these steps precisely:
1. Use a Haiku agent to find relevant CLAUDE.md files: the root CLAUDE.md
and any CLAUDE.md files in directories whose files this PR modifies.
2. Use a Haiku agent to summarize the PR change (use `gh pr diff`).
3. Launch 4 parallel agents to review the change independently. Each agent should
read the PR diff with `gh pr diff` and the full source files for changed
code, then return a list of issues found:
Agent 1 — Security & Safety
Check for: command injection, path traversal, SSRF, XSS, auth bypass,
secrets in logs, .unwrap()/.expect() in production code (not tests),
race conditions, TOCTOU, unsafe blocks, panics in async, unbounded allocations.
Agent 2 — Architecture & Patterns
Check for: extensible design (traits/enums over nested conditionals),
clean abstractions, proper error types (thiserror), CLAUDE.md compliance,
type-driven design over stringly-typed code, DRY violations.
Agent 3 — Bug Scan
Shallow diff-only scan for obvious bugs: logic errors, off-by-one,
missing error handling, division by zero, incorrect return values.
Ignore nitpicks and likely false positives. Do NOT read extra context
beyond the diff — focus only on the changes.
Agent 4 — Performance & Production
Check for: blocking in async, N+1 queries, unbounded loops, missing
timeouts, resource leaks (file handles, connections), large allocations
in hot paths.
4. For each issue found, launch a parallel Haiku agent to:
a. Assign a severity:
- CRITICAL: security vulns, panics in prod (.unwrap/.expect), data exfiltration, race conditions
- HIGH: logic bugs, missing error handling, breaking API/schema changes
- MEDIUM: missing tests, unnecessary complexity, performance issues
- LOW: documentation gaps, naming suggestions
b. Score confidence 0-100 (give this rubric verbatim):
0: False positive, doesn't stand up to scrutiny, or pre-existing issue.
25: Might be real, but may be false positive. Stylistic issues not in CLAUDE.md.
50: Real issue but nitpick or rare in practice. Not very important.
75: Verified real issue, will be hit in practice. Directly impacts functionality
or explicitly mentioned in CLAUDE.md.
100: Certain, confirmed, will happen frequently. Evidence directly confirms.
5. Post a single comment on the PR using `gh pr comment` with this format.
If no issues were found, post "No issues found." instead:
### Code review
Found N issues:
1. [SEVERITY:CONFIDENCE] <brief description>
<permalink to file:line using full SHA, eg https://github.com/owner/repo/blob/abc123def/src/file.rs#L10-L15>
Example: [CRITICAL:92] `.unwrap()` can panic in production when config is missing
You MUST use the full git SHA in links (not HEAD or branch name).
Provide 1 line of context before and after each linked range.
Notes:
- Use `gh` for all GitHub interactions, not web fetch
- Do NOT check build signal or attempt to build/test the code
- Ignore pre-existing issues not introduced by this PR
- Ignore issues a linter/compiler would catch (formatting, imports, types)
-28
View File
@@ -1,31 +1,3 @@
# Code Coverage Workflow
#
# This workflow runs test coverage analysis and uploads reports to Codecov.
# Coverage reports help identify untested code paths and maintain code quality.
#
# What it does:
# - Runs unit and integration tests with coverage instrumentation
# - Runs E2E tests with coverage instrumentation
# - Uploads coverage reports to Codecov (https://codecov.io/gh/nearai/ironclaw)
#
# Viewing coverage reports:
# - PRs automatically get coverage comments showing changes in coverage
# - Visit https://codecov.io/gh/nearai/ironclaw for detailed coverage reports
# - Coverage reports are generated for three configurations:
# 1. all-features: Full feature set
# 2. default: Default features
# 3. libsql-only: Minimal libSQL-only configuration
# - E2E coverage tracks end-to-end test coverage separately
#
# Coverage files:
# - Unit/integration: lcov.info (uploaded to Codecov with "unit" flag)
# - E2E: e2e-coverage.info (uploaded to Codecov with "e2e" flag)
#
# Requirements:
# - Uses cargo-llvm-cov for coverage instrumentation
# - Requires PostgreSQL for integration tests (pgvector/pgvector:pg16)
# - E2E tests require Python 3.12 and Playwright
name: Code Coverage
on:
push:
-1
View File
@@ -1,6 +1,5 @@
name: E2E Tests
on:
workflow_call:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
-473
View File
@@ -1,473 +0,0 @@
name: Staging CI (Batched)
on:
schedule:
- cron: "0 * * * *" # Every 60 minutes
workflow_dispatch:
inputs:
force:
description: "Force run even if no new commits"
type: boolean
default: false
skip_claude_gate:
description: "Skip Claude review gate (bypass blocking findings)"
type: boolean
default: false
permissions:
contents: write
issues: write
pull-requests: write
checks: read
concurrency:
group: staging-ci
cancel-in-progress: false # Let running suites finish
jobs:
# ── Check for new commits ──────────────────────────────────────
check-changes:
name: Check for new commits
runs-on: ubuntu-latest
outputs:
has_changes: ${{ steps.check.outputs.has_changes }}
current_head: ${{ steps.check.outputs.current_head }}
diff_range: ${{ steps.check.outputs.diff_range }}
steps:
- uses: actions/checkout@v6
with:
ref: staging
fetch-depth: 0
fetch-tags: true
- name: Check for changes since last tested
id: check
env:
FORCE_RUN: ${{ inputs.force }}
run: |
CURRENT_HEAD=$(git rev-parse HEAD)
echo "current_head=${CURRENT_HEAD}" >> "$GITHUB_OUTPUT"
if git rev-parse staging-tested >/dev/null 2>&1; then
LAST_TESTED=$(git rev-parse staging-tested)
else
LAST_TESTED=""
fi
DIFF_RANGE=""
if [ -n "$LAST_TESTED" ] && [ "$LAST_TESTED" = "$CURRENT_HEAD" ]; then
echo "No new commits since last tested (${CURRENT_HEAD})"
HAS_CHANGES=false
else
HAS_CHANGES=true
if [ -n "$LAST_TESTED" ]; then
COMMIT_COUNT=$(git rev-list --count "${LAST_TESTED}..HEAD")
echo "Found ${COMMIT_COUNT} new commit(s) since last tested"
DIFF_RANGE="${LAST_TESTED}..${CURRENT_HEAD}"
else
git fetch origin main
MERGE_BASE=$(git merge-base origin/main HEAD)
echo "First run -- reviewing from merge-base ${MERGE_BASE}"
DIFF_RANGE="${MERGE_BASE}..${CURRENT_HEAD}"
fi
fi
# Force override from workflow_dispatch
if [ "$FORCE_RUN" = "true" ]; then
echo "Force run requested"
HAS_CHANGES=true
if [ -z "$DIFF_RANGE" ]; then
DIFF_RANGE="${CURRENT_HEAD}..${CURRENT_HEAD}"
fi
fi
echo "has_changes=${HAS_CHANGES}" >> "$GITHUB_OUTPUT"
echo "diff_range=${DIFF_RANGE}" >> "$GITHUB_OUTPUT"
# ── Run full test suite ──────────────────────────────────────────
tests:
name: Test Suite
needs: check-changes
if: needs.check-changes.outputs.has_changes == 'true'
uses: ./.github/workflows/test.yml
# ── Run E2E browser tests ────────────────────────────────────────
e2e:
name: E2E Browser Tests
needs: check-changes
if: needs.check-changes.outputs.has_changes == 'true'
uses: ./.github/workflows/e2e.yml
# ── Create promotion PR (triggers claude-review.yml on the PR) ──
create-promotion-pr:
name: Create Promotion PR
needs: check-changes
if: needs.check-changes.outputs.has_changes == 'true'
runs-on: ubuntu-latest
outputs:
pr_number: ${{ steps.create-pr.outputs.pr_number }}
promotion_branch: ${{ steps.branch.outputs.branch }}
steps:
- uses: actions/checkout@v6
with:
ref: staging
fetch-depth: 0
- name: Generate GitHub App token
id: app-token
continue-on-error: true
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.GH_RELEASES_MANAGER_APP_ID }}
private-key: ${{ secrets.GH_RELEASES_MANAGER_APP_PRIVATE_KEY }}
- name: Set token
id: token
run: |
if [ -n "${{ steps.app-token.outputs.token }}" ]; then
echo "token=${{ steps.app-token.outputs.token }}" >> "$GITHUB_OUTPUT"
else
echo "token=${{ github.token }}" >> "$GITHUB_OUTPUT"
fi
- name: Check if staging is ahead of main
id: ahead-check
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
run: |
git fetch origin main
AHEAD=$(git rev-list --count origin/main..origin/staging)
echo "commits_ahead=${AHEAD}" >> "$GITHUB_OUTPUT"
if [ "$AHEAD" -eq 0 ]; then
echo "Staging is not ahead of main. Nothing to promote."
else
echo "Staging is ${AHEAD} commits ahead of main."
fi
- name: Create promotion branch
id: branch
if: steps.ahead-check.outputs.commits_ahead != '0'
run: |
SHORT_SHA=$(echo "${{ needs.check-changes.outputs.current_head }}" | cut -c1-8)
BRANCH="staging-promote/${SHORT_SHA}-${{ github.run_id }}"
git checkout -b "$BRANCH"
git push origin "$BRANCH"
echo "branch=${BRANCH}" >> "$GITHUB_OUTPUT"
echo "Created promotion branch: ${BRANCH}"
- name: Find base branch
id: find-base
if: steps.ahead-check.outputs.commits_ahead != '0'
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
run: |
# Find the newest open promotion PR with a staging-promote/* head branch
LATEST=$(gh pr list --label staging-promotion --state open \
--json headRefName,createdAt \
--jq '[.[] | select(.headRefName | startswith("staging-promote/"))] | sort_by(.createdAt) | last | .headRefName // empty')
if [ -n "$LATEST" ]; then
echo "base=${LATEST}" >> "$GITHUB_OUTPUT"
echo "Chaining onto existing promotion branch: ${LATEST}"
else
echo "base=main" >> "$GITHUB_OUTPUT"
echo "No existing promotion PR — targeting main"
fi
- name: Create promotion PR
id: create-pr
if: steps.ahead-check.outputs.commits_ahead != '0'
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
run: |
RANGE="${{ needs.check-changes.outputs.diff_range }}"
TIMESTAMP=$(date -u +"%Y-%m-%d %H:%M UTC")
BRANCH="${{ steps.branch.outputs.branch }}"
BASE="${{ steps.find-base.outputs.base }}"
PR_URL=$(gh pr create \
--base "$BASE" \
--head "$BRANCH" \
--title "chore: promote staging to main (${TIMESTAMP})" \
--body "## Auto-promotion from staging CI
**Batch range:** \`${RANGE}\`
**Promotion branch:** \`${BRANCH}\`
**Base:** \`${BASE}\`
**Triggered by:** Staging CI batch at ${TIMESTAMP}
Waiting for gates:
- Tests: pending
- E2E: pending
- Claude Code review: pending (will post comments on this PR)
---
*Auto-created by staging-ci workflow*" \
--label "staging-promotion")
PR_NUM=$(echo "$PR_URL" | grep -oE '[0-9]+$')
echo "pr_number=${PR_NUM}" >> "$GITHUB_OUTPUT"
echo "Created promotion PR #${PR_NUM}"
# ── Gate: wait for review, process findings, merge or block ─────
gate:
name: Staging Gate
needs: [check-changes, tests, e2e, create-promotion-pr]
if: >
always() &&
needs.check-changes.outputs.has_changes == 'true' &&
needs.tests.result == 'success' &&
needs.e2e.result == 'success' &&
needs.create-promotion-pr.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 25
outputs:
gate_passed: ${{ steps.evaluate.outputs.passed }}
steps:
- uses: actions/checkout@v6
with:
ref: staging
fetch-depth: 1
- name: Generate GitHub App token
id: app-token
continue-on-error: true
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.GH_RELEASES_MANAGER_APP_ID }}
private-key: ${{ secrets.GH_RELEASES_MANAGER_APP_PRIVATE_KEY }}
- name: Set token
id: token
run: |
if [ -n "${{ steps.app-token.outputs.token }}" ]; then
echo "token=${{ steps.app-token.outputs.token }}" >> "$GITHUB_OUTPUT"
else
echo "token=${{ github.token }}" >> "$GITHUB_OUTPUT"
fi
- name: Wait for Claude review job
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
PR_NUMBER: ${{ needs.create-promotion-pr.outputs.pr_number }}
REPO: ${{ github.repository }}
run: |
if [ -z "$PR_NUMBER" ]; then
echo "No PR number — skipping wait"
exit 0
fi
PR_SHA=$(gh pr view "$PR_NUMBER" --json headRefOid --jq '.headRefOid' || echo "")
if [ -z "$PR_SHA" ]; then
echo "::warning::Could not get PR head SHA"
exit 0
fi
echo "Polling for Claude Code Review job on PR #${PR_NUMBER} (SHA: ${PR_SHA})..."
TIMEOUT=1200 # 20 minutes
ELAPSED=0
INTERVAL=30
while [ "$ELAPSED" -lt "$TIMEOUT" ]; do
STATUS=$(gh api "repos/${REPO}/commits/${PR_SHA}/check-runs" \
--jq '[.check_runs[] | select(.name == "Claude Code Review") | .conclusion // .status] | first // "pending"' 2>/dev/null || echo "pending")
if [ "$STATUS" = "success" ] || [ "$STATUS" = "failure" ] || [ "$STATUS" = "cancelled" ]; then
echo "Claude review job completed with status: ${STATUS} (${ELAPSED}s)"
exit 0
fi
echo "Claude review status: ${STATUS} (${ELAPSED}s elapsed)"
sleep "$INTERVAL"
ELAPSED=$((ELAPSED + INTERVAL))
done
echo "::warning::Claude review job not completed after ${TIMEOUT}s"
- name: Process Claude review comments and create issues
id: process-findings
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
PR_NUMBER: ${{ needs.create-promotion-pr.outputs.pr_number }}
REPO: ${{ github.repository }}
run: |
HAS_BLOCKING=false
ISSUES_CREATED=0
if [ -z "$PR_NUMBER" ]; then
echo "No PR — skipping finding processing"
echo "has_blocking=false" >> "$GITHUB_OUTPUT"
exit 0
fi
# Check for "No issues found" first (clean pass)
NO_ISSUES=$(gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" \
--jq '[.[] | select(.user.login == "claude[bot]") | select(.body | test("No issues found"))] | length' 2>/dev/null || echo "0")
if [ "$NO_ISSUES" -gt 0 ]; then
echo "Claude review found no issues — gate passes"
echo "has_blocking=false" >> "$GITHUB_OUTPUT"
exit 0
fi
# Get the last Claude comment that contains findings
JQ_FILTER='[.[] | select(.user.login == "claude[bot]") | select(.body | test("Found [0-9]+ issue"))] | last'
BODY=$(gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" \
--jq "${JQ_FILTER} | .body // empty" 2>/dev/null || echo "")
COMMENT_URL=$(gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" \
--jq "${JQ_FILTER} | .html_url // empty" 2>/dev/null || echo "")
if [ -z "$BODY" ]; then
echo "::warning::No Claude review comment found for PR #${PR_NUMBER} — treating as blocking"
echo "has_blocking=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# Parse [SEVERITY:CONFIDENCE] tags from each numbered finding
# Matrix: CRITICAL always→issue, ≥80→block. HIGH ≥50→issue. MEDIUM ≥80→issue. LOW ≥80→issue.
# Use process substitution so variables propagate to parent shell
while read -r line; do
TAG=$(echo "$line" | grep -oE '^\[(CRITICAL|HIGH|MEDIUM|LOW):[0-9]+\]')
SEVERITY=$(echo "$TAG" | sed 's/\[\(.*\):\(.*\)\]/\1/')
CONFIDENCE=$(echo "$TAG" | sed 's/\[\(.*\):\(.*\)\]/\2/')
DESC=$(echo "$line" | sed "s/\[${SEVERITY}:${CONFIDENCE}\] *//" | head -1)
echo "Found: [${SEVERITY}:${CONFIDENCE}] ${DESC}"
# Check if blocking (CRITICAL ≥80)
if [ "$SEVERITY" = "CRITICAL" ] && [ "$CONFIDENCE" -ge 80 ]; then
HAS_BLOCKING=true
fi
# Determine if this should create an issue
CREATE_ISSUE=false
case "$SEVERITY" in
CRITICAL) CREATE_ISSUE=true ;;
HIGH) [ "$CONFIDENCE" -ge 50 ] && CREATE_ISSUE=true ;;
MEDIUM) [ "$CONFIDENCE" -ge 80 ] && CREATE_ISSUE=true ;;
LOW) [ "$CONFIDENCE" -ge 80 ] && CREATE_ISSUE=true ;;
esac
if [ "$CREATE_ISSUE" = "true" ]; then
case "$SEVERITY" in
CRITICAL) LABELS="bug,risk: high,staging-ci-review" ;;
HIGH) LABELS="bug,risk: medium,staging-ci-review" ;;
MEDIUM) LABELS="risk: medium,staging-ci-review" ;;
LOW) LABELS="risk: low,staging-ci-review" ;;
esac
TITLE=$(echo "$DESC" | cut -c1-80)
{
echo "## [${SEVERITY}:${CONFIDENCE}] Issue Found by Staging CI Review"
echo ""
echo "**Severity:** ${SEVERITY}"
echo "**Confidence:** ${CONFIDENCE}/100"
echo "**PR comment:** ${COMMENT_URL}"
echo ""
echo "### Description"
echo "$DESC"
echo ""
echo "---"
echo "*Auto-created by staging-ci Claude Code review*"
} > /tmp/issue-body.md
if gh issue create \
--title "[${SEVERITY}] ${TITLE}" \
--body-file /tmp/issue-body.md \
--label "${LABELS}"; then
ISSUES_CREATED=$((ISSUES_CREATED + 1))
else
echo "::warning::Failed to create issue for ${SEVERITY} finding"
fi
fi
done < <(echo "$BODY" | grep -oE '\[(CRITICAL|HIGH|MEDIUM|LOW):[0-9]+\].*')
echo "Created ${ISSUES_CREATED} issues"
echo "has_blocking=${HAS_BLOCKING}" >> "$GITHUB_OUTPUT"
- name: Evaluate gate
id: evaluate
env:
PR_NUMBER: ${{ needs.create-promotion-pr.outputs.pr_number }}
SKIP_GATE: ${{ inputs.skip_claude_gate }}
HAS_BLOCKING: ${{ steps.process-findings.outputs.has_blocking }}
run: |
SKIP_INPUT="$SKIP_GATE"
if [ "$HAS_BLOCKING" = "true" ]; then
echo "::warning::Claude review found blocking issues (CRITICAL ≥80 confidence)"
if [ "$SKIP_INPUT" = "true" ]; then
echo "::warning::Gate overridden by skip_claude_gate workflow input"
echo "passed=true" >> "$GITHUB_OUTPUT"
else
echo "::error::Blocking promotion due to CRITICAL findings (≥80 confidence)"
echo "::error::PR #${PR_NUMBER} left open with review comments"
echo "passed=false" >> "$GITHUB_OUTPUT"
exit 1
fi
else
echo "No blocking findings. Gate passed."
echo "passed=true" >> "$GITHUB_OUTPUT"
fi
- name: Merge promotion PR
id: merge
if: steps.evaluate.outputs.passed == 'true'
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
PR_NUMBER: ${{ needs.create-promotion-pr.outputs.pr_number }}
run: |
if [ -n "$PR_NUMBER" ]; then
echo "Merging promotion PR #${PR_NUMBER}"
# Do NOT use --delete-branch: deleting a promotion branch closes
# any chained PRs that use it as their base (verified in ironclaw-ci-test).
# Stale promotion branches are cleaned up separately.
gh pr merge "$PR_NUMBER" --merge
echo "merged=true" >> "$GITHUB_OUTPUT"
fi
# ── Update tested tag (always, so next batch covers only new commits) ──
update-tag:
name: Update staging-tested tag
needs: [check-changes, tests, e2e, create-promotion-pr, gate]
if: >
always() &&
needs.check-changes.outputs.has_changes == 'true' &&
needs.tests.result == 'success' &&
needs.e2e.result == 'success' &&
needs.create-promotion-pr.result == 'success'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
ref: staging
fetch-depth: 1
- name: Update staging-tested tag
run: |
git tag -f staging-tested "${{ needs.check-changes.outputs.current_head }}"
git push origin staging-tested --force
echo "Updated staging-tested tag to ${{ needs.check-changes.outputs.current_head }}"
# ── Report ───────────────────────────────────────────────────────
report:
name: Staging CI Summary
needs: [check-changes, tests, e2e, create-promotion-pr, gate, update-tag]
if: always() && needs.check-changes.outputs.has_changes == 'true'
runs-on: ubuntu-latest
steps:
- name: Summary
run: |
echo "## Staging CI Batch Results" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "| Check | Result |" >> "$GITHUB_STEP_SUMMARY"
echo "|-------|--------|" >> "$GITHUB_STEP_SUMMARY"
echo "| Tests | ${{ needs.tests.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "| E2E | ${{ needs.e2e.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "| Promotion PR | ${{ needs.create-promotion-pr.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "| Gate | ${{ needs.gate.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "| Tag Updated | ${{ needs.update-tag.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "Range: ${{ needs.check-changes.outputs.diff_range }}" >> "$GITHUB_STEP_SUMMARY"
PR_NUM="${{ needs.create-promotion-pr.outputs.pr_number }}"
if [ -n "$PR_NUM" ]; then
echo "Promotion PR: #${PR_NUM}" >> "$GITHUB_STEP_SUMMARY"
fi
+7 -30
View File
@@ -1,6 +1,5 @@
name: Run Tests
on:
workflow_call:
pull_request:
push:
branches:
@@ -39,9 +38,6 @@ jobs:
telegram-tests:
name: Telegram Channel Tests
if: >
github.event_name == 'push' ||
(github.event_name == 'pull_request' && github.base_ref != 'staging')
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -54,9 +50,6 @@ jobs:
windows-build:
name: Windows Build (${{ matrix.name }})
if: >
github.event_name == 'push' ||
(github.event_name == 'pull_request' && github.base_ref != 'staging')
runs-on: windows-latest
strategy:
fail-fast: false
@@ -81,9 +74,6 @@ jobs:
wasm-wit-compat:
name: WASM WIT Compatibility
if: >
github.event_name == 'push' ||
(github.event_name == 'pull_request' && github.base_ref != 'staging')
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -104,9 +94,6 @@ jobs:
docker-build:
name: Docker Build
if: >
github.event_name == 'push' ||
(github.event_name == 'pull_request' && github.base_ref != 'staging')
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -136,22 +123,12 @@ jobs:
needs: [tests, telegram-tests, wasm-wit-compat, docker-build, windows-build, version-check]
steps:
- run: |
# Unit tests must always pass
if [[ "${{ needs.tests.result }}" != "success" ]]; then
echo "Unit tests failed"
if [[ "${{ needs.tests.result }}" != "success" || "${{ needs.telegram-tests.result }}" != "success" || "${{ needs.wasm-wit-compat.result }}" != "success" || "${{ needs.docker-build.result }}" != "success" || "${{ needs.windows-build.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
# version-check only runs on PRs, so skip/success are both acceptable
if [[ "${{ needs.version-check.result }}" == "failure" ]]; then
echo "Version bump check failed"
exit 1
fi
# Gated jobs: must pass on promotion PRs / push, skipped on developer PRs
for job in telegram-tests wasm-wit-compat docker-build windows-build version-check; do
case "$job" in
telegram-tests) result="${{ needs.telegram-tests.result }}" ;;
wasm-wit-compat) result="${{ needs.wasm-wit-compat.result }}" ;;
docker-build) result="${{ needs.docker-build.result }}" ;;
windows-build) result="${{ needs.windows-build.result }}" ;;
version-check) result="${{ needs.version-check.result }}" ;;
esac
if [[ "$result" == "failure" || "$result" == "cancelled" ]]; then
echo "$job failed"
exit 1
fi
done
+1 -4
View File
@@ -4,9 +4,8 @@
.env.*
!.env.example
# Claude Code worktrees and lock files
# Claude Code worktrees
.claude/worktrees/
.claude/scheduled_tasks.lock
# Sidecar tool data
.sidecar/
@@ -23,5 +22,3 @@ bench-results/
# WASM build artifacts (loaded from disk, not bundled)
*.wasm
# Traces
trace_*.json
-4
View File
@@ -7,10 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- AWS Bedrock LLM provider via native Converse API with IAM and SSO auth support (feature-gated: `--features bedrock`)
## [0.16.1](https://github.com/nearai/ironclaw/compare/v0.16.0...v0.16.1) - 2026-03-06
### Fixed
+1 -25
View File
@@ -387,27 +387,12 @@ Dead code behind the wrong `#[cfg]` gate will only show up when building with a
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Transaction safety:** Multi-step database operations (INSERT+INSERT, UPDATE+DELETE, read-then-write) MUST be wrapped in a transaction. Never assume sequential calls are atomic. Before committing DB code, ask: "If this crashes between step N and N+1, is the database consistent?" If not, wrap in a transaction. This applies to both postgres and libsql backends.
**UTF-8 string safety:** Never use byte-index slicing (`&s[..n]`) on user-supplied or external strings — it panics on multi-byte characters. Use `is_char_boundary()` to walk backwards from the desired length, or iterate with `char_indices()`. Grep for `[..` in changed files to catch violations.
**Case-insensitive comparisons:** When comparing user-supplied strings (file paths, media types, extension names), always normalize to lowercase first with `.to_ascii_lowercase()`. On case-insensitive filesystems (macOS, Windows), path comparisons must be case-insensitive. File extension checks (`.png`, `.jpg`) and media type checks (`image/jpeg`) are common offenders.
**Decorator/wrapper trait delegation:** When adding a new method to `LlmProvider` (or any trait with decorator wrappers), you MUST update ALL wrapper types to delegate to their inner provider. Grep for `impl LlmProvider for` to find all implementations. Add a test that exercises the method through the full provider chain (`build_provider_chain()`), not just the base impl.
**Sensitive data in logs & events:** Tool parameters and outputs MUST be redacted before logging or broadcasting via SSE/WebSocket. Use `redact_params()` before any `tracing::info!`, `JobEvent`, or SSE emission that includes tool call data. Never log raw parameters from tool calls.
**Test temporary files:** Use the `tempfile` crate for test directories/files. Never hardcode `/tmp/...` paths — they collide in parallel test runs and break on non-Unix platforms.
**Trust boundaries in multi-process architecture:** Data from worker containers is untrusted. The orchestrator MUST validate: tool domain (never execute `Container`-domain tools on the host), nesting depth (server-side tracking, not client-supplied), and parameter sensitivity (redact before logging/broadcasting).
**Mechanical verification before committing:** Run these checks on changed files before committing:
- `cargo clippy --all --benches --tests --examples --all-features` -- zero warnings
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
- Fix commits must include regression tests (enforced by `commit-msg` hook; bypass with `[skip-regression-check]`)
- Run `scripts/pre-commit-safety.sh` to catch UTF-8, case-sensitivity, hardcoded /tmp, and logging issues
## Configuration
@@ -484,13 +469,6 @@ SKILLS_AUTO_DISCOVER=true # Scan skill directories on startup
TINFOIL_API_KEY=... # Required when LLM_BACKEND=tinfoil
TINFOIL_MODEL=kimi-k2-5 # Default model
# AWS Bedrock (native Converse API, requires --features bedrock)
# LLM_BACKEND=bedrock
# BEDROCK_REGION=us-east-1 # AWS region
# BEDROCK_MODEL=anthropic.claude-opus-4-6-v1 # Required model ID
# BEDROCK_CROSS_REGION=us # Cross-region prefix (us/eu/apac/global)
# AWS_PROFILE=my-profile # Named profile (SSO/assume-role)
# Tunnel (public internet exposure for webhooks)
TUNNEL_URL=https://abc123.ngrok.io # Static public URL (manual tunnel)
# Or use a managed tunnel provider:
@@ -507,9 +485,7 @@ OBSERVABILITY_BACKEND=none # none/noop (default) or log
### LLM Providers
Backends: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, `tinfoil`, `bedrock` (requires `--features bedrock`) — set via `LLM_BACKEND`. See [src/llm/CLAUDE.md](src/llm/CLAUDE.md) for per-provider auth and configuration details.
**AWS Bedrock** -- Uses the native Converse API via `aws-sdk-bedrockruntime`. Requires `--features bedrock` at build time (not included in default features due to heavy AWS SDK dependencies). Supports standard AWS auth methods: IAM credentials (`AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`), SSO profiles (`AWS_PROFILE`), and instance roles. Configure with `BEDROCK_REGION` (default: `us-east-1`), `BEDROCK_MODEL` (required, e.g., `anthropic.claude-opus-4-6-v1`), and `BEDROCK_CROSS_REGION` (optional: `us`, `eu`, `apac`, `global` for cross-region inference profiles). The SDK credential chain resolves auth automatically from the environment.
Backends: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, `tinfoil` — set via `LLM_BACKEND`. See [src/llm/CLAUDE.md](src/llm/CLAUDE.md) for per-provider auth and configuration details.
## Database
Generated
+23 -707
View File
File diff suppressed because it is too large Load Diff
+1 -9
View File
@@ -56,7 +56,7 @@ rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication", "remote", "tls"] }
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication"] }
# Error handling
thiserror = "2"
@@ -73,8 +73,6 @@ toml = "0.8"
# Core types
uuid = { version = "1", features = ["v4", "v5", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
chrono-tz = "0.10"
iana-time-zone = "0.1"
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "maths"] }
rust_decimal_macros = "1"
@@ -142,11 +140,6 @@ subtle = "2" # Constant-time comparisons for token validation
# Multi-provider LLM support
rig-core = "0.30"
# AWS Bedrock (native Converse API, opt-in via --features bedrock)
aws-config = { version = "1", features = ["behavior-version-latest"], optional = true }
aws-sdk-bedrockruntime = { version = "1", optional = true }
aws-smithy-types = { version = "1", optional = true }
# Docker sandbox
bollard = "0.18"
@@ -208,7 +201,6 @@ postgres = [
libsql = ["dep:libsql"]
integration = []
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
bedrock = ["dep:aws-config", "dep:aws-sdk-bedrockruntime", "dep:aws-smithy-types"]
[[test]]
name = "html_to_markdown"
+4 -8
View File
@@ -39,7 +39,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Network modes (loopback/LAN/remote) | ✅ | 🚧 | HTTP only |
| OpenAI-compatible HTTP API | ✅ | ✅ | /v1/chat/completions, per-request `model` override |
| Canvas hosting | ✅ | ❌ | Agent-driven UI |
| Gateway lock (PID-based) | ✅ | | `fs4` flock-based, acquired in `main.rs` before agent startup |
| Gateway lock (PID-based) | ✅ | | |
| launchd/systemd integration | ✅ | ❌ | |
| Bonjour/mDNS discovery | ✅ | ❌ | |
| Tailscale integration | ✅ | ❌ | |
@@ -215,13 +215,9 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6 |
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy |
| AWS Bedrock | ✅ | ✅ | - | Native Converse API via aws-sdk-bedrockruntime (requires `--features bedrock`) |
| Google Gemini | ✅ | | P3 | Via `gemini` adapter |
| io.net | ✅ | | P3 | Via `ionet` adapter |
| Mistral | ✅ | ✅ | P3 | Via `mistral` adapter |
| Yandex AI Studio | ✅ | ✅ | P3 | Via `yandex` adapter |
| Cloudflare Workers AI | ✅ | ✅ | P3 | Via `cloudflare` adapter |
| NVIDIA API | ✅ | ✅ | P3 | Via `nvidia` adapter and `providers.json` |
| AWS Bedrock | ✅ | ❌ | P3 | |
| Google Gemini | ✅ | | P3 | |
| NVIDIA API | ✅ | | P3 | New provider |
| OpenRouter | ✅ | ✅ | - | Via OpenAI-compatible provider (RigAdapter) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (IronClaw-only) |
| OpenAI-compatible | ❌ | ✅ | - | Generic OpenAI-compatible endpoint (RigAdapter) |
-5
View File
@@ -14,11 +14,6 @@
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
</p>
<p align="center">
<a href="README.md">English</a> |
<a href="README.zh-CN.md">简体中文</a>
</p>
<p align="center">
<a href="#philosophy">Philosophy</a> •
<a href="#features">Features</a> •
-319
View File
@@ -1,319 +0,0 @@
<p align="center">
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
<p align="center">
<strong>安全可靠的个人 AI 助手,始终站在你这边</strong>
</p>
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
</p>
<p align="center">
<a href="README.md">English</a> |
<a href="README.zh-CN.md">简体中文</a>
</p>
<p align="center">
<a href="#设计理念">设计理念</a> •
<a href="#功能特性">功能特性</a> •
<a href="#安装">安装</a> •
<a href="#配置">配置</a> •
<a href="#安全机制">安全机制</a> •
<a href="#系统架构">系统架构</a>
</p>
---
## 设计理念
IronClaw 基于一个简单的原则:**你的 AI 助手应该为你服务,而不是与你为敌。**
在 AI 系统对数据处理日益不透明、与企业利益捆绑的今天,IronClaw 选择了一条不同的路:
- **数据归你所有** — 所有信息存储在本地,加密保护,始终在你掌控之下
- **透明至上** — 完全开源,可审计,没有隐藏的遥测或数据收集
- **自主扩展** — 随时构建新工具,无需等待供应商更新
- **纵深防御** — 多层安全机制抵御提示注入和数据泄露
IronClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还是工作。
## 功能特性
### 安全优先
- **WASM 沙箱** — 不受信任的工具在隔离的 WebAssembly 容器中运行,采用基于能力的权限模型
- **凭据保护** — 密钥永远不会暴露给工具;在宿主边界注入并进行泄露检测
- **提示注入防御** — 模式检测、内容清理和策略执行
- **端点白名单** — HTTP 请求仅限于明确批准的主机和路径
### 随时可用
- **多渠道接入** — REPL、HTTP webhook、WASM 渠道(Telegram、Slack)和 Web 网关
- **Docker 沙箱** — 隔离的容器执行,支持每任务令牌和编排器/工作器模式
- **Web 网关** — 浏览器 UI,支持实时 SSE/WebSocket 流式传输
- **定时任务** — Cron 调度、事件触发器、Webhook 处理器,实现后台自动化
- **心跳系统** — 主动后台执行,用于监控和维护任务
- **并行任务** — 使用隔离上下文同时处理多个请求
- **自修复** — 自动检测并恢复卡住的操作
### 自主扩展
- **动态工具构建** — 描述你的需求,IronClaw 会将其构建为 WASM 工具
- **MCP 协议** — 连接模型上下文协议(Model Context Protocol)服务器以获取额外能力
- **插件架构** — 无需重启即可加载新的 WASM 工具和渠道
### 持久记忆
- **混合搜索** — 全文搜索 + 向量搜索,采用倒数排名融合(Reciprocal Rank Fusion
- **工作空间文件系统** — 灵活的基于路径的存储,用于笔记、日志和上下文
- **身份文件** — 跨会话保持一致的个性和偏好设置
## 安装
### 前置要求
- Rust 1.85+
- PostgreSQL 15+,需安装 [pgvector](https://github.com/pgvector/pgvector) 扩展
- NEAR AI 账户(通过设置向导进行身份验证)
## 下载或编译
访问 [Releases 页面](https://github.com/nearai/ironclaw/releases/) 查看最新版本。
<details>
<summary>通过 Windows 安装程序安装 (Windows)</summary>
下载 [Windows 安装程序](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi) 并运行。
</details>
<details>
<summary>通过 PowerShell 脚本安装 (Windows)</summary>
```sh
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
```
</details>
<details>
<summary>通过 Shell 脚本安装 (macOS、Linux、Windows/WSL)</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
```
</details>
<details>
<summary>通过 Homebrew 安装 (macOS/Linux)</summary>
```sh
brew install ironclaw
```
</details>
<details>
<summary>从源码编译 (Windows、Linux、macOS 上使用 Cargo)</summary>
确保你已安装 [Rust](https://rustup.rs)。
```bash
# 克隆仓库
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
# 编译
cargo build --release
# 运行测试
cargo test
```
如需进行**完整发布构建**(修改了渠道源码后),先运行 `./scripts/build-all.sh` 重新编译渠道。
</details>
### 数据库设置
```bash
# 创建数据库
createdb ironclaw
# 启用 pgvector 扩展
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## 配置
运行设置向导来配置 IronClaw:
```bash
ironclaw onboard
```
向导将引导你完成数据库连接、NEAR AI 身份验证(通过浏览器 OAuth)和密钥加密(使用系统钥匙串)。设置会保存在数据库中;引导变量(如 `DATABASE_URL``LLM_BACKEND`)写入 `~/.ironclaw/.env`,以便在数据库连接前可用。
### 替代 LLM 提供商
IronClaw 默认使用 NEAR AI,但兼容任何 OpenAI 兼容的端点。
常用选项包括 **OpenRouter**300+ 模型)、**Together AI**、**Fireworks AI**、**Ollama**(本地部署)以及自托管服务器如 **vLLM****LiteLLM**
在向导中选择 *"OpenAI-compatible"*,或直接设置环境变量:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
详见 [docs/LLM_PROVIDERS.md](docs/LLM_PROVIDERS.md) 获取完整的提供商指南。
## 安全机制
IronClaw 实现了纵深防御策略来保护你的数据并防止滥用。
### WASM 沙箱
所有不受信任的工具都在隔离的 WebAssembly 容器中运行:
- **基于能力的权限** — 明确授权 HTTP、密钥、工具调用等能力
- **端点白名单** — HTTP 请求仅限已批准的主机和路径
- **凭据注入** — 密钥在宿主边界注入,永远不会暴露给 WASM 代码
- **泄露检测** — 扫描请求和响应以防止密钥外泄
- **速率限制** — 每个工具独立的请求限制,防止滥用
- **资源限制** — 内存、CPU 和执行时间约束
```
WASM ──► 白名单 ──► 泄露扫描 ──► 凭据 ──► 执行 ──► 泄露扫描 ──► WASM
验证器 (请求) 注入器 请求 (响应)
```
### 提示注入防御
外部内容需通过多个安全层:
- 基于模式的注入尝试检测
- 内容清理和转义
- 带严重级别的策略规则(阻止/警告/审核/清理)
- 工具输出包装,确保安全的 LLM 上下文注入
### 数据保护
- 所有数据存储在本地 PostgreSQL 数据库中
- 密钥使用 AES-256-GCM 加密
- 无遥测、无分析、无数据共享
- 所有工具执行的完整审计日志
## 系统架构
```
┌────────────────────────────────────────────────────────────────┐
│ 渠道 │
│ ┌──────┐ ┌──────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ REPL │ │ HTTP │ │ WASM 渠道 │ │ Web 网关 │ │
│ └──┬───┘ └──┬───┘ └──────┬──────┘ │ (SSE + WS) │ │
│ │ │ │ └──────┬──────┘ │
│ └─────────┴──────────────┴────────────────┘ │
│ │ │
│ ┌─────────▼─────────┐ │
│ │ 代理循环 │ 意图路由 │
│ └────┬──────────┬───┘ │
│ │ │ │
│ ┌──────────▼────┐ ┌──▼───────────────┐ │
│ │ 调度器 │ │ 定时任务引擎 │ │
│ │ (并行任务) │ │(cron, 事件, wh) │ │
│ └──────┬────────┘ └────────┬─────────┘ │
│ │ │ │
│ ┌─────────────┼────────────────────┘ │
│ │ │ │
│ ┌───▼─────┐ ┌────▼────────────────┐ │
│ │ 本地 │ │ 编排器 │ │
│ │ 工作器 │ │ ┌───────────────┐ │ │
│ │(进程内) │ │ │ Docker 沙箱 │ │ │
│ └───┬─────┘ │ │ 容器 │ │ │
│ │ │ │ ┌───────────┐ │ │ │
│ │ │ │ │工作器/CC │ │ │ │
│ │ │ │ └───────────┘ │ │ │
│ │ │ └───────────────┘ │ │
│ │ └─────────┬───────────┘ │
│ └──────────────────┤ │
│ │ │
│ ┌───────────▼──────────┐ │
│ │ 工具注册表 │ │
│ │ 内置、MCP、WASM │ │
│ └──────────────────────┘ │
└────────────────────────────────────────────────────────────────┘
```
### 核心组件
| 组件 | 用途 |
|------|------|
| **代理循环** | 主消息处理和任务协调 |
| **路由器** | 分类用户意图(命令、查询、任务) |
| **调度器** | 管理带优先级的并行任务执行 |
| **工作器** | 执行包含 LLM 推理和工具调用的任务 |
| **编排器** | 容器生命周期、LLM 代理、每任务认证 |
| **Web 网关** | 浏览器 UI,含聊天、记忆、任务、日志、扩展、定时任务 |
| **定时任务引擎** | 定时(cron)和响应式(事件、webhook)后台任务 |
| **工作空间** | 带混合搜索的持久记忆 |
| **安全层** | 提示注入防御和内容清理 |
## 使用方式
```bash
# 首次设置(配置数据库、认证等)
ironclaw onboard
# 启动交互式 REPL
cargo run
# 启用调试日志
RUST_LOG=ironclaw=debug cargo run
```
## 开发
```bash
# 格式化代码
cargo fmt
# 代码检查
cargo clippy --all --benches --tests --examples --all-features
# 运行测试
createdb ironclaw_test
cargo test
# 运行指定测试
cargo test test_name
```
- **Telegram 渠道**:参见 [docs/TELEGRAM_SETUP.md](docs/TELEGRAM_SETUP.md) 了解设置和私信配对。
- **修改渠道源码**:在 `cargo build` 之前运行 `./channels-src/telegram/build.sh` 以便打包更新后的 WASM。
## OpenClaw 传承
IronClaw 是受 [OpenClaw](https://github.com/openclaw/openclaw) 启发的 Rust 重新实现。参见 [FEATURE_PARITY.md](FEATURE_PARITY.md) 了解完整的功能追踪矩阵。
主要差异:
- **Rust vs TypeScript** — 原生性能、内存安全、单一二进制文件
- **WASM 沙箱 vs Docker** — 轻量级、基于能力的安全机制
- **PostgreSQL vs SQLite** — 生产级持久化存储
- **安全优先设计** — 多层防御、凭据保护
## 许可证
可选择以下任一许可证:
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE))
- MIT License ([LICENSE-MIT](LICENSE-MIT))
+1 -1
View File
@@ -267,7 +267,7 @@ dependencies = [
[[package]]
name = "slack-channel"
version = "0.2.1"
version = "0.1.0"
dependencies = [
"hex",
"hmac",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "slack-channel"
version = "0.2.1"
version = "0.2.0"
edition = "2021"
description = "Slack Events API channel for IronClaw"
license = "MIT OR Apache-2.0"
-104
View File
@@ -357,108 +357,10 @@ fn extract_slack_attachments(files: &Option<Vec<SlackFile>>) -> Vec<InboundAttac
.collect()
}
/// Download a file from Slack using the url_private endpoint.
///
/// Slack file downloads require Bearer auth with the bot token, which is
/// injected by the host credential system via `channel_host::http_request`.
fn download_slack_file(url: &str) -> Result<Vec<u8>, String> {
let headers = serde_json::json!({});
let result = channel_host::http_request("GET", url, &headers.to_string(), None, None);
let response = result.map_err(|e| format!("Slack file download failed: {}", e))?;
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!(
"Slack file download returned {}: {}",
response.status, body_str
));
}
Ok(response.body)
}
/// Download file bytes and store them via the host for processing.
///
/// Downloads all file types (images, documents, etc.) so the host-side
/// middleware can process them (vision pipeline for images, text extraction
/// for documents, transcription for audio, etc.).
/// Maximum file size to download (20 MB). Files larger than this are skipped
/// to avoid excessive memory use and slow downloads in the WASM runtime.
const MAX_DOWNLOAD_SIZE_BYTES: u64 = 20 * 1024 * 1024;
fn download_and_store_slack_files(attachments: &[InboundAttachment]) {
for att in attachments {
let Some(ref url) = att.source_url else {
continue;
};
// Skip files that exceed the size limit
if let Some(size) = att.size_bytes {
if size > MAX_DOWNLOAD_SIZE_BYTES {
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"Skipping Slack file download: {} bytes exceeds {} MB limit (id={})",
size,
MAX_DOWNLOAD_SIZE_BYTES / (1024 * 1024),
att.id
),
);
continue;
}
}
match download_slack_file(url) {
Ok(bytes) => {
// Post-download size guard: metadata size_bytes is optional,
// so a file with no size info could bypass the pre-download check.
if bytes.len() as u64 > MAX_DOWNLOAD_SIZE_BYTES {
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"Discarding Slack file after download: {} bytes exceeds {} MB limit (id={})",
bytes.len(),
MAX_DOWNLOAD_SIZE_BYTES / (1024 * 1024),
att.id
),
);
continue;
}
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Downloaded Slack file: {} bytes, mime={}",
bytes.len(),
att.mime_type
),
);
if let Err(e) = channel_host::store_attachment_data(&att.id, &bytes) {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to store Slack file data: {}", e),
);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to download Slack file: {}", e),
);
}
}
}
}
/// Handle a Slack event and emit message if applicable.
fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Option<String>) {
let attachments = extract_slack_attachments(&event.files);
// Download and store file attachments for host-side processing
download_and_store_slack_files(&attachments);
match event.event_type.as_str() {
// Direct mention of the bot (always in a channel, not a DM)
"app_mention" => {
@@ -820,10 +722,4 @@ mod tests {
let event: SlackEvent = serde_json::from_str(json).unwrap();
assert!(event.files.is_none());
}
#[test]
fn test_max_download_size_constant() {
// Verify the constant is 20 MB
assert_eq!(MAX_DOWNLOAD_SIZE_BYTES, 20 * 1024 * 1024);
}
}
+1 -1
View File
@@ -212,7 +212,7 @@ dependencies = [
[[package]]
name = "telegram-channel"
version = "0.2.1"
version = "0.2.0"
dependencies = [
"serde",
"serde_json",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "telegram-channel"
version = "0.2.1"
version = "0.2.0"
edition = "2021"
description = "Telegram Bot API channel for IronClaw"
license = "MIT OR Apache-2.0"
+5 -57
View File
@@ -878,6 +878,10 @@ fn send_message(
// Voice File Download
// ============================================================================
/// Download a voice file from Telegram by file_id.
///
/// 1. Call getFile to get the file_path.
/// 2. Download the file bytes from /file/bot{TOKEN}/{file_path}.
/// Percent-encode a string for safe use as a URL query parameter value.
fn percent_encode(s: &str) -> String {
let mut out = String::with_capacity(s.len());
@@ -894,10 +898,6 @@ fn percent_encode(s: &str) -> String {
out
}
/// Maximum file size to download (20 MB). Files larger than this are discarded
/// to avoid excessive memory use and slow downloads in the WASM runtime.
const MAX_DOWNLOAD_SIZE_BYTES: u64 = 20 * 1024 * 1024;
fn download_telegram_file(file_id: &str) -> Result<Vec<u8>, String> {
// Reject file_id containing curly braces to prevent credential placeholder injection
if file_id.contains('{') || file_id.contains('}') {
@@ -965,16 +965,6 @@ fn download_telegram_file(file_id: &str) -> Result<Vec<u8>, String> {
));
}
// Post-download size guard: Telegram metadata file_size is optional,
// so enforce the limit on actual downloaded bytes.
if response.body.len() as u64 > MAX_DOWNLOAD_SIZE_BYTES {
return Err(format!(
"Downloaded file exceeds {} MB limit ({} bytes)",
MAX_DOWNLOAD_SIZE_BYTES / (1024 * 1024),
response.body.len()
));
}
Ok(response.body)
}
@@ -1545,39 +1535,6 @@ fn download_and_store_voice(attachments: &[InboundAttachment]) {
}
}
/// Download image file bytes and store them via the host for the vision pipeline.
///
/// Separated from `extract_attachments` so that function stays pure (no host
/// calls) and remains testable in native unit tests.
fn download_and_store_images(attachments: &[InboundAttachment]) {
for att in attachments {
if !att.mime_type.starts_with("image/") {
continue;
}
match download_telegram_file(&att.id) {
Ok(bytes) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!("Downloaded image file: {} bytes", bytes.len()),
);
if let Err(e) = channel_host::store_attachment_data(&att.id, &bytes) {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to store image data: {}", e),
);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to download image file: {}", e),
);
}
}
}
}
/// Returns true if the attachment should be downloaded for document text extraction.
///
/// Excludes voice (handled by transcription), image (vision pipeline),
@@ -1651,9 +1608,6 @@ fn handle_message(message: TelegramMessage) {
// Download and store voice attachments for host-side transcription
download_and_store_voice(&attachments);
// Download and store image attachments for host-side vision pipeline
download_and_store_images(&attachments);
// Download and store document attachments for host-side text extraction
download_and_store_documents(&mut attachments);
@@ -1727,7 +1681,7 @@ fn handle_message(message: TelegramMessage) {
let username_opt = from.username.as_deref();
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&id_str)
|| username_opt.is_some_and(|u| allowed.contains(&u.to_string()));
|| username_opt.map_or(false, |u| allowed.contains(&u.to_string()));
if !is_allowed {
if is_private && dm_policy == "pairing" {
@@ -2651,10 +2605,4 @@ mod tests {
assert!(!is_downloadable_document(&make("audio/mpeg", Some("song.mp3"))));
assert!(!is_downloadable_document(&make("video/mp4", Some("clip.mp4"))));
}
#[test]
fn test_max_download_size_constant() {
// Verify the constant is 20 MB, matching the Slack channel limit
assert_eq!(MAX_DOWNLOAD_SIZE_BYTES, 20 * 1024 * 1024);
}
}
@@ -1,17 +1,9 @@
{
"version": "0.2.2",
"version": "0.2.0",
"wit_version": "0.3.0",
"type": "channel",
"name": "telegram",
"description": "Telegram Bot API channel for receiving and responding to Telegram messages",
"auth": {
"secret_name": "telegram_bot_token",
"display_name": "Telegram",
"instructions": "Get your bot token from @BotFather on Telegram (https://t.me/BotFather). Send /newbot or /token to get it.",
"setup_url": "https://t.me/BotFather",
"token_hint": "Looks like 123456789:AABBccDDeeFFgg...",
"env_var": "TELEGRAM_BOT_TOKEN"
},
"setup": {
"required_secrets": [
{
+1 -1
View File
@@ -3,7 +3,7 @@ services:
postgres:
image: pgvector/pgvector:pg16
ports:
- "127.0.0.1:5432:5432"
- "5432:5432"
environment:
POSTGRES_DB: ironclaw
POSTGRES_USER: ironclaw
-55
View File
@@ -11,13 +11,7 @@ configurations.
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Google Gemini | `gemini` | `GEMINI_API_KEY` | Gemini models |
| io.net | `ionet` | `IONET_API_KEY` | Intelligence API |
| Mistral | `mistral` | `MISTRAL_API_KEY` | Mistral models |
| Yandex AI Studio | `yandex` | `YANDEX_API_KEY` | YandexGPT models |
| Cloudflare Workers AI | `cloudflare` | `CLOUDFLARE_API_KEY` | Access to Workers AI |
| Ollama | `ollama` | No | Local inference |
| AWS Bedrock | `bedrock` | AWS credentials | Native Converse API |
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
@@ -74,55 +68,6 @@ Pull a model first: `ollama pull llama3.2`
---
## AWS Bedrock (requires `--features bedrock`)
Uses the native AWS Converse API via `aws-sdk-bedrockruntime`. Supports standard AWS
authentication methods: IAM credentials, SSO profiles, and instance roles.
> **Build prerequisite:** The `aws-lc-sys` crate (transitive dependency via AWS SDK)
> requires **CMake** to compile. Install it before building with `--features bedrock`:
> - macOS: `brew install cmake`
> - Ubuntu/Debian: `sudo apt install cmake`
> - Fedora: `sudo dnf install cmake`
### With AWS credentials (IAM, SSO, instance roles)
```env
LLM_BACKEND=bedrock
BEDROCK_MODEL=anthropic.claude-opus-4-6-v1
BEDROCK_REGION=us-east-1
BEDROCK_CROSS_REGION=us
# AWS_PROFILE=my-sso-profile # optional, for named profiles
```
The AWS SDK credential chain automatically resolves credentials from environment
variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`), shared credentials file
(`~/.aws/credentials`), SSO profiles, and EC2/ECS instance roles.
### Cross-region inference
Set `BEDROCK_CROSS_REGION` to route requests across AWS regions for capacity:
| Prefix | Routing |
|---|---|
| `us` | US regions (us-east-1, us-east-2, us-west-2) |
| `eu` | European regions |
| `apac` | Asia-Pacific regions |
| `global` | All commercial AWS regions |
| _(unset)_ | Single-region only |
### Popular Bedrock model IDs
| Model | ID |
|---|---|
| Claude Opus 4.6 | `anthropic.claude-opus-4-6-v1` |
| Claude Sonnet 4.5 | `anthropic.claude-sonnet-4-5-20250929-v1:0` |
| Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` |
| Amazon Nova Pro | `amazon.nova-pro-v1:0` |
| Llama 4 Maverick | `meta.llama4-maverick-17b-instruct-v1:0` |
---
## OpenAI-Compatible Endpoints
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
@@ -1,13 +0,0 @@
-- Partial unique indexes to prevent duplicate singleton conversations.
-- These guard against TOCTOU races in get_or_create_routine_conversation
-- and get_or_create_heartbeat_conversation.
-- One routine conversation per user per routine_id.
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_routine
ON conversations (user_id, (metadata->>'routine_id'))
WHERE metadata->>'routine_id' IS NOT NULL;
-- One heartbeat conversation per user.
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_heartbeat
ON conversations (user_id)
WHERE metadata->>'thread_type' = 'heartbeat';
+11 -141
View File
@@ -1,9 +1,7 @@
[
{
"id": "openai",
"aliases": [
"open_ai"
],
"aliases": ["open_ai"],
"protocol": "open_ai_completions",
"api_key_env": "OPENAI_API_KEY",
"api_key_required": true,
@@ -21,9 +19,7 @@
},
{
"id": "anthropic",
"aliases": [
"claude"
],
"aliases": ["claude"],
"protocol": "anthropic",
"api_key_env": "ANTHROPIC_API_KEY",
"api_key_required": true,
@@ -56,10 +52,7 @@
},
{
"id": "openai_compatible",
"aliases": [
"openai-compatible",
"compatible"
],
"aliases": ["openai-compatible", "compatible"],
"protocol": "open_ai_completions",
"base_url_env": "LLM_BASE_URL",
"base_url_required": true,
@@ -96,9 +89,7 @@
},
{
"id": "openrouter",
"aliases": [
"open_router"
],
"aliases": ["open_router"],
"protocol": "open_ai_completions",
"default_base_url": "https://openrouter.ai/api/v1",
"api_key_env": "OPENROUTER_API_KEY",
@@ -135,10 +126,7 @@
},
{
"id": "nvidia",
"aliases": [
"nvidia_nim",
"nim"
],
"aliases": ["nvidia_nim", "nim"],
"protocol": "open_ai_completions",
"default_base_url": "https://integrate.api.nvidia.com/v1",
"api_key_env": "NVIDIA_API_KEY",
@@ -156,10 +144,7 @@
},
{
"id": "venice",
"aliases": [
"venice_ai",
"veniceai"
],
"aliases": ["venice_ai", "veniceai"],
"protocol": "open_ai_completions",
"default_base_url": "https://api.venice.ai/api/v1",
"api_key_env": "VENICE_API_KEY",
@@ -177,10 +162,7 @@
},
{
"id": "together",
"aliases": [
"together_ai",
"togetherai"
],
"aliases": ["together_ai", "togetherai"],
"protocol": "open_ai_completions",
"default_base_url": "https://api.together.xyz/v1",
"api_key_env": "TOGETHER_API_KEY",
@@ -198,9 +180,7 @@
},
{
"id": "fireworks",
"aliases": [
"fireworks_ai"
],
"aliases": ["fireworks_ai"],
"protocol": "open_ai_completions",
"default_base_url": "https://api.fireworks.ai/inference/v1",
"api_key_env": "FIREWORKS_API_KEY",
@@ -218,9 +198,7 @@
},
{
"id": "deepseek",
"aliases": [
"deep_seek"
],
"aliases": ["deep_seek"],
"protocol": "open_ai_completions",
"default_base_url": "https://api.deepseek.com/v1",
"api_key_env": "DEEPSEEK_API_KEY",
@@ -256,9 +234,7 @@
},
{
"id": "sambanova",
"aliases": [
"samba_nova"
],
"aliases": ["samba_nova"],
"protocol": "open_ai_completions",
"default_base_url": "https://api.sambanova.ai/v1",
"api_key_env": "SAMBANOVA_API_KEY",
@@ -273,111 +249,5 @@
"display_name": "SambaNova",
"can_list_models": false
}
},
{
"id": "gemini",
"aliases": [
"google_gemini",
"google"
],
"protocol": "open_ai_completions",
"default_base_url": "https://generativelanguage.googleapis.com/v1beta/openai",
"api_key_env": "GEMINI_API_KEY",
"api_key_required": true,
"model_env": "GEMINI_MODEL",
"default_model": "gemini-2.5-flash",
"description": "Google Gemini (via OpenAI-compatible endpoint)",
"setup": {
"kind": "api_key",
"secret_name": "llm_gemini_api_key",
"key_url": "https://aistudio.google.com/app/apikey",
"display_name": "Google Gemini",
"can_list_models": true
}
},
{
"id": "ionet",
"aliases": [
"io_net",
"io.net"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.intelligence.io.solutions/api/v1",
"api_key_env": "IONET_API_KEY",
"api_key_required": true,
"model_env": "IONET_MODEL",
"default_model": "deepseek-coder-v2-instruct",
"description": "io.net Intelligence API",
"setup": {
"kind": "api_key",
"secret_name": "llm_ionet_api_key",
"key_url": "https://cloud.io.net/intelligence",
"display_name": "io.net",
"can_list_models": true
}
},
{
"id": "mistral",
"aliases": [
"mistral_ai",
"mistralai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.mistral.ai/v1",
"api_key_env": "MISTRAL_API_KEY",
"api_key_required": true,
"model_env": "MISTRAL_MODEL",
"default_model": "mistral-large-latest",
"description": "Mistral AI API",
"setup": {
"kind": "api_key",
"secret_name": "llm_mistral_api_key",
"key_url": "https://console.mistral.ai/api-keys",
"display_name": "Mistral",
"can_list_models": true
}
},
{
"id": "yandex",
"aliases": [
"yandex_ai_studio",
"yandexgpt",
"yandex_gpt"
],
"protocol": "open_ai_completions",
"default_base_url": "https://ai.api.cloud.yandex.net/v1",
"api_key_env": "YANDEX_API_KEY",
"api_key_required": true,
"model_env": "YANDEX_MODEL",
"extra_headers_env": "YANDEX_EXTRA_HEADERS",
"default_model": "yandexgpt-lite",
"description": "Yandex AI Studio (YandexGPT)",
"setup": {
"kind": "api_key",
"secret_name": "llm_yandex_api_key",
"key_url": "https://aistudio.yandex.ru/platform/folders/",
"display_name": "Yandex AI Studio",
"can_list_models": true
}
},
{
"id": "cloudflare",
"aliases": [
"cloudflare_ai",
"cf_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "CLOUDFLARE_API_KEY",
"api_key_required": true,
"base_url_env": "CLOUDFLARE_BASE_URL",
"model_env": "CLOUDFLARE_MODEL",
"default_model": "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
"description": "Cloudflare Workers AI",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_cloudflare_api_key",
"display_name": "Cloudflare Workers AI",
"can_list_models": false
}
}
]
]
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-wasm32-wasip2.tar.gz",
"sha256": "030707431717bca3411a48f311c6ab5f92a45c747de26cafe4f6e3e23a8b3b2d"
"sha256": null
}
},
"auth_summary": {
+2 -2
View File
@@ -2,7 +2,7 @@
"name": "slack",
"display_name": "Slack Channel",
"kind": "channel",
"version": "0.2.1",
"version": "0.2.0",
"wit_version": "0.3.0",
"description": "Talk to your agent in Slack",
"keywords": [
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": "6ed36077b67ac70a041f06f760f93ba79b33269885413c3c3f2c8c87ee60807e"
"sha256": null
}
},
"auth_summary": {
+2 -2
View File
@@ -2,7 +2,7 @@
"name": "telegram",
"display_name": "Telegram Channel",
"kind": "channel",
"version": "0.2.2",
"version": "0.2.0",
"wit_version": "0.3.0",
"description": "Talk to your agent through a Telegram bot",
"keywords": [
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": "98c86895a9c4b0a1e19fe8a47f1ccbfe7e972e112b05e584bc897130dc32283a"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/whatsapp-wasm32-wasip2.tar.gz",
"sha256": "bd35cad18d87292ea8d2f52db9b514ed9f814a414de910f59073d475c26c4c14"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -20,7 +20,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/github-wasm32-wasip2.tar.gz",
"sha256": "6fcd32719a4ff15641a4b50fff8984686550f0c491dce60518f4126857d0c544"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-wasm32-wasip2.tar.gz",
"sha256": "023da7000b17568bf0e64b2e5013c8a042b2f323c85f1632339231c73d500e39"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-calendar-wasm32-wasip2.tar.gz",
"sha256": "fc42277b65881d6e9bcc5403dc54c7f5b3ddeaaaf04617fce2c5da05d76325f0"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-docs-wasm32-wasip2.tar.gz",
"sha256": "385c04abd1e6b8011ccc330e1f4bd7ce58577e488959b51594aa04eb26cbe7cc"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-drive-wasm32-wasip2.tar.gz",
"sha256": "1b107d575a5d52cc8c76d9a681802190f4373fb485f7f54f445533f097fa37c0"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-sheets-wasm32-wasip2.tar.gz",
"sha256": "c4f6b1e8c5126ac2c8a4b98e4283a3afa32223d2488fc3c3a609758c0c9beb90"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -18,7 +18,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-slides-wasm32-wasip2.tar.gz",
"sha256": "7110b8565340c888e51f99e9c013bf4de8f8a7f7b33bace00eb8fc47831ff20b"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -18,7 +18,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-tool-wasm32-wasip2.tar.gz",
"sha256": "6ed36077b67ac70a041f06f760f93ba79b33269885413c3c3f2c8c87ee60807e"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-mtproto-wasm32-wasip2.tar.gz",
"sha256": "98c86895a9c4b0a1e19fe8a47f1ccbfe7e972e112b05e584bc897130dc32283a"
"sha256": null
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-wasm32-wasip2.tar.gz",
"sha256": "66cb2b9b00652385e9f30f17c74902b9222c17c53e9d3bd1ef42f5cab705bcf6"
"sha256": null
}
},
"auth_summary": {
+2 -4
View File
@@ -51,11 +51,9 @@ echo "[6/6] Installing git hooks..."
HOOKS_DIR=$(git rev-parse --git-path hooks 2>/dev/null) || true
if [ -n "$HOOKS_DIR" ]; then
mkdir -p "$HOOKS_DIR"
SCRIPTS_ABS="$(cd "$(dirname "$0")" && pwd)"
ln -sf "$SCRIPTS_ABS/commit-msg-regression.sh" "$HOOKS_DIR/commit-msg"
SCRIPT_ABS="$(cd "$(dirname "$0")" && pwd)/commit-msg-regression.sh"
ln -sf "$SCRIPT_ABS" "$HOOKS_DIR/commit-msg"
echo " commit-msg hook installed (regression test enforcement)"
ln -sf "$SCRIPTS_ABS/pre-commit-safety.sh" "$HOOKS_DIR/pre-commit"
echo " pre-commit hook installed (UTF-8, case-sensitivity, /tmp, redaction checks)"
else
echo " Skipped: not a git repository"
fi
-136
View File
@@ -1,136 +0,0 @@
#!/usr/bin/env bash
# Pre-commit safety checks for common issues caught by AI code reviewers.
#
# Can be run standalone: bash scripts/pre-commit-safety.sh
# Or installed as a git pre-commit hook via dev-setup.sh.
#
# Checks staged .rs files for:
# 1. Unsafe UTF-8 byte slicing (panics on multi-byte chars)
# 2. Case-sensitive file extension comparisons
# 3. Hardcoded /tmp paths in tests (flaky in parallel runs)
# 4. Tool parameters logged without redaction (secret leaks)
# 5. Multi-step DB operations without transaction wrapping
#
# Suppress individual lines with an inline "// safety: <reason>" comment.
set -euo pipefail
# Determine a suitable base ref for standalone diffs.
resolve_base_ref() {
local candidates=(
"@{upstream}"
"origin/HEAD"
"origin/main"
"origin/master"
"main"
"master"
)
for ref in "${candidates[@]}"; do
if git rev-parse --verify --quiet "$ref" >/dev/null 2>&1; then
echo "$ref"
return 0
fi
done
echo "pre-commit-safety: could not determine a base Git ref for diff (tried: ${candidates[*]})." >&2
echo "pre-commit-safety: ensure your repository has an upstream or a local main/master branch." >&2
exit 1
}
# Support both pre-commit hook (staged files) and standalone (all changed vs base)
if git diff --cached --quiet 2>/dev/null; then
# No staged changes -- compare working tree against a resolved base ref
BASE_REF="$(resolve_base_ref)"
DIFF_OUTPUT=$(git diff "$BASE_REF" -- '*.rs' 2>/dev/null || true)
else
DIFF_OUTPUT=$(git diff --cached -U0 -- '*.rs' 2>/dev/null || true)
fi
# Early exit if there are no relevant .rs changes
if [ -z "$DIFF_OUTPUT" ]; then
exit 0
fi
WARNINGS=0
warn() {
if [ "$WARNINGS" -eq 0 ]; then
echo ""
echo "=== Pre-commit Safety Checks ==="
echo ""
fi
WARNINGS=$((WARNINGS + 1))
echo " [$1] $2"
}
# 1. Unsafe UTF-8 byte slicing: &s[..N] or &s[..some_var] on strings
# Safe patterns: is_char_boundary, char_indices, // safety:
if echo "$DIFF_OUTPUT" | grep -nE '^\+' | grep -E '\[\.\..*\]' | grep -vE 'is_char_boundary|char_indices|// safety:|as_bytes|Vec<|&\[u8\]|\[u8\]|bytes\(\)|&bytes' | head -3 | grep -q .; then
warn "UTF8" "Possible unsafe byte-index string slicing. Use is_char_boundary() or char_indices()."
echo "$DIFF_OUTPUT" | grep -nE '^\+' | grep -E '\[\.\..*\]' | grep -vE 'is_char_boundary|char_indices|// safety:|as_bytes|Vec<|&\[u8\]|\[u8\]|bytes\(\)|&bytes' | head -3 | sed 's/^/ /'
fi
# 2. Case-sensitive file extension checks
# Match: .ends_with(".png") without prior to_lowercase
if echo "$DIFF_OUTPUT" | grep -nE '^\+.*ends_with\("\.([pP][nN][gG]|[jJ][pP][eE]?[gG]|[gG][iI][fF]|[wW][eE][bB][pP]|[mM][dD])"\)' | grep -vE 'to_lowercase|to_ascii_lowercase|// safety:' | head -3 | grep -q .; then
warn "CASE" "Case-sensitive file extension comparison. Normalize to lowercase first."
echo "$DIFF_OUTPUT" | grep -nE '^\+.*ends_with\("\.([pP][nN][gG]|[jJ][pP][eE]?[gG]|[gG][iI][fF]|[wW][eE][bB][pP]|[mM][dD])"\)' | grep -vE 'to_lowercase|to_ascii_lowercase|// safety:' | head -3 | sed 's/^/ /'
fi
# 3. Hardcoded /tmp paths in test files
if echo "$DIFF_OUTPUT" | grep -nE '^\+.*"/tmp/' | grep -vE 'tempfile|tempdir|// safety:' | head -3 | grep -q .; then
warn "TMPDIR" "Hardcoded /tmp path. Use tempfile::tempdir() for parallel-safe tests."
echo "$DIFF_OUTPUT" | grep -nE '^\+.*"/tmp/' | grep -vE 'tempfile|tempdir|// safety:' | head -3 | sed 's/^/ /'
fi
# 4. Logging tool parameters without redaction
if echo "$DIFF_OUTPUT" | grep -nE '^\+.*tracing::(info|debug|warn|error).*param' | grep -vE 'redact|// safety:' | head -3 | grep -q .; then
warn "REDACT" "Logging tool parameters without redaction. Use redact_params() first."
echo "$DIFF_OUTPUT" | grep -nE '^\+.*tracing::(info|debug|warn|error).*param' | grep -vE 'redact|// safety:' | head -3 | sed 's/^/ /'
fi
# 5. Multi-step DB operations without transaction
# Uses -W (function context) to reduce false positives from existing transactions.
# Suppressible with "// safety:" in the hunk.
DIFF_W_OUTPUT=$(git diff --cached -W -- '*.rs' 2>/dev/null || git diff "$(resolve_base_ref)" -W -- '*.rs' 2>/dev/null || true)
if [ -n "$DIFF_W_OUTPUT" ]; then
HUNK_COUNT=$(echo "$DIFF_W_OUTPUT" | awk '
/^@@/ {
if (count >= 2 && !has_tx && !has_safety) found++
count=0; has_tx=0; has_safety=0
}
/^\+.*\.(execute|query)\(/ { count++ }
/^\+.*(transaction|\.tx\.|\.begin\()/ { has_tx=1 }
/ .*(transaction|\.tx\.|\.begin\()/ { has_tx=1 }
/\/\/ safety:/ { has_safety=1 }
END {
if (count >= 2 && !has_tx && !has_safety) found++
print found+0
}
')
if [ "$HUNK_COUNT" -gt 0 ]; then
warn "TX" "Multiple DB operations in same function without transaction. Wrap in a transaction for atomicity."
echo "$DIFF_W_OUTPUT" | awk '
/^@@/ {
if (count >= 2 && !has_tx && !has_safety) { print buf }
buf=""; count=0; has_tx=0; has_safety=0
}
/^\+.*\.(execute|query)\(/ { count++ }
/^\+.*(transaction|\.tx\.|\.begin\()/ { has_tx=1 }
/ .*(transaction|\.tx\.|\.begin\()/ { has_tx=1 }
/\/\/ safety:/ { has_safety=1 }
{ buf = buf "\n" $0 }
END {
if (count >= 2 && !has_tx && !has_safety) { print buf }
}
' | grep -E '^\+.*\.(execute|query)\(' | head -4 | sed 's/^/ /'
fi
fi
if [ "$WARNINGS" -gt 0 ]; then
echo ""
echo "Found $WARNINGS potential issue(s). Fix them or add '// safety: <reason>' to suppress."
echo ""
exit 1
fi
-54
View File
@@ -1,54 +0,0 @@
---
name: review-checklist
version: 0.1.0
description: Pre-merge review checklist based on recurring AI reviewer feedback patterns
activation:
patterns:
- "review.*checklist"
- "ready to merge"
- "pre-merge check"
- "check.*before.*merge"
keywords:
- review
- checklist
- merge
- pre-merge
max_context_tokens: 1500
---
# Pre-Merge Review Checklist
Before merging, verify these items. They represent the most common issues caught by automated code reviewers (Copilot, Gemini) on IronClaw PRs.
## Database Operations
- [ ] Multi-step DB operations are wrapped in transactions (INSERT+INSERT, UPDATE+DELETE, read-modify-write)
- [ ] Both postgres AND libsql backends updated for any new Database trait methods
- [ ] Migrations are atomic (SQL execution + version recording in same transaction)
## Security & Data Safety
- [ ] Tool parameters are redacted via `redact_params()` before logging or SSE/WebSocket broadcast
- [ ] URL validation resolves DNS before checking for private/loopback IPs (anti-SSRF via DNS rebinding)
- [ ] Destructive tools have `requires_approval()` returning `Always` or `UnlessAutoApproved`
- [ ] Data from worker containers is treated as untrusted (tool domain checks, server-side nesting depth)
- [ ] No secrets or credentials in error messages, logs, or SSE events
## String Safety
- [ ] No byte-index slicing (`&s[..n]`) on external/user strings -- use `is_char_boundary()` or `char_indices()`
- [ ] File extension and media type comparisons are case-insensitive (`.to_ascii_lowercase()` before matching)
- [ ] Path comparisons are case-insensitive where needed (macOS/Windows filesystems)
## Trait Wrappers & Decorator Chain
- [ ] New `LlmProvider` trait methods are delegated in ALL wrapper types (grep `impl LlmProvider for`)
- [ ] New trait methods are tested through the full decorator/provider chain, not just the base impl
- [ ] Default trait method implementations are intentional -- wrappers that silently return defaults are bugs
## Tests
- [ ] Temporary files/dirs use `tempfile` crate, no hardcoded `/tmp/` paths
- [ ] Tests don't mutate global statics without synchronization (use per-test state or `serial_test`)
- [ ] Tests don't make real network requests (use mocks, stubs, or RFC 5737 TEST-NET IPs like 192.0.2.1)
- [ ] Test names and comments match actual test behavior and assertions
## Comments & Documentation
- [ ] Code comments match actual behavior (especially route paths, tool names, function semantics)
- [ ] Spec/README files updated if module behavior changed
- [ ] Error messages are clear and non-redundant (don't nest tool name inside tool error that already contains it)
+2 -30
View File
@@ -96,9 +96,6 @@ pub struct Agent {
pub(super) heartbeat_config: Option<HeartbeatConfig>,
pub(super) hygiene_config: Option<crate::config::HygieneConfig>,
pub(super) routine_config: Option<RoutineConfig>,
/// Optional slot to expose the routine engine to the gateway for manual triggering.
pub(super) routine_engine_slot:
Option<Arc<tokio::sync::RwLock<Option<Arc<crate::agent::routine_engine::RoutineEngine>>>>>,
}
impl Agent {
@@ -151,18 +148,9 @@ impl Agent {
heartbeat_config,
hygiene_config,
routine_config,
routine_engine_slot: None,
}
}
/// Set the routine engine slot for exposing the engine to the gateway.
pub fn set_routine_engine_slot(
&mut self,
slot: Arc<tokio::sync::RwLock<Option<Arc<crate::agent::routine_engine::RoutineEngine>>>>,
) {
self.routine_engine_slot = Some(slot);
}
// Convenience accessors
/// Get the scheduler (for external wiring, e.g. CreateJobTool).
@@ -354,19 +342,8 @@ impl Agent {
let heartbeat_handle = if let Some(ref hb_config) = self.heartbeat_config {
if hb_config.enabled {
if let Some(workspace) = self.workspace() {
let mut config = AgentHeartbeatConfig::default()
let config = AgentHeartbeatConfig::default()
.with_interval(std::time::Duration::from_secs(hb_config.interval_secs));
config.quiet_hours_start = hb_config.quiet_hours_start;
config.quiet_hours_end = hb_config.quiet_hours_end;
config.timezone = hb_config
.timezone
.clone()
.or_else(|| Some(self.config.default_timezone.clone()));
if let (Some(user), Some(channel)) =
(&hb_config.notify_user, &hb_config.notify_channel)
{
config = config.with_notify(user, channel);
}
// Set up notification channel
let (notify_tx, mut notify_rx) =
@@ -417,8 +394,8 @@ impl Agent {
hygiene,
workspace.clone(),
self.cheap_llm().clone(),
self.safety().clone(),
Some(notify_tx),
self.store().map(Arc::clone),
))
} else {
tracing::warn!("Heartbeat enabled but no workspace available");
@@ -509,11 +486,6 @@ impl Agent {
// SAFETY: self is consumed by run(), we can smuggle the engine in
// via a local to use in the message loop below.
// Expose engine to gateway for manual triggering
if let Some(ref slot) = self.routine_engine_slot {
*slot.write().await = Some(Arc::clone(&engine));
}
tracing::info!(
"Routines enabled: cron ticker every {}s, max {} concurrent",
rt_config.cron_check_interval_secs,
+7 -48
View File
@@ -345,6 +345,7 @@ impl Agent {
crate::workspace::hygiene::HygieneConfig::default(),
workspace.clone(),
self.llm().clone(),
self.safety().clone(),
);
match runner.check_heartbeat().await {
@@ -405,7 +406,7 @@ impl Agent {
.with_max_tokens(512)
.with_temperature(0.3);
let reasoning = Reasoning::new(self.llm().clone());
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
match reasoning.complete(request).await {
Ok((text, _usage)) => Ok(SubmissionResult::response(format!(
"Thread Summary:\n\n{}",
@@ -453,7 +454,7 @@ impl Agent {
.with_max_tokens(512)
.with_temperature(0.5);
let reasoning = Reasoning::new(self.llm().clone());
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
match reasoning.complete(request).await {
Ok((text, _usage)) => Ok(SubmissionResult::response(format!(
"Suggested Next Steps:\n\n{}",
@@ -662,14 +663,10 @@ impl Agent {
}
match self.llm().set_model(requested) {
Ok(()) => {
// Persist the model choice so it survives restarts.
self.persist_selected_model(requested).await;
Ok(SubmissionResult::response(format!(
"Switched model to: {}",
requested
)))
}
Ok(()) => Ok(SubmissionResult::response(format!(
"Switched model to: {}",
requested
))),
Err(e) => Ok(SubmissionResult::error(format!(
"Failed to switch model: {}",
e
@@ -825,42 +822,4 @@ impl Agent {
_ => Ok(None),
}
}
/// Persist the selected model to the settings store (DB and/or TOML config).
///
/// Best-effort: logs warnings on failure but does not propagate errors,
/// since the in-memory model switch already succeeded.
async fn persist_selected_model(&self, model: &str) {
// 1. Persist to DB if available.
if let Some(store) = self.store() {
let value = serde_json::Value::String(model.to_string());
if let Err(e) = store.set_setting("default", "selected_model", &value).await {
tracing::warn!("Failed to persist model to DB: {}", e);
}
}
// 2. Update TOML config file if it exists (sync I/O in spawn_blocking).
let model_owned = model.to_string();
if let Err(e) = tokio::task::spawn_blocking(move || {
let toml_path = crate::settings::Settings::default_toml_path();
match crate::settings::Settings::load_toml(&toml_path) {
Ok(Some(mut settings)) => {
settings.selected_model = Some(model_owned);
if let Err(e) = settings.save_toml(&toml_path) {
tracing::warn!("Failed to persist model to config.toml: {}", e);
}
}
Ok(None) => {
// No config file on disk; nothing to update.
}
Err(e) => {
tracing::warn!("Failed to load config.toml for model persistence: {}", e);
}
}
})
.await
{
tracing::warn!("Model TOML persistence task failed: {}", e);
}
}
}
+37 -112
View File
@@ -13,6 +13,7 @@ use crate::agent::context_monitor::{CompactionStrategy, ContextBreakdown};
use crate::agent::session::Thread;
use crate::error::Error;
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider, Reasoning};
use crate::safety::SafetyLayer;
use crate::workspace::Workspace;
/// Result of a compaction operation.
@@ -33,12 +34,13 @@ pub struct CompactionResult {
/// Compacts conversation context to stay within limits.
pub struct ContextCompactor {
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
}
impl ContextCompactor {
/// Create a new context compactor.
pub fn new(llm: Arc<dyn LlmProvider>) -> Self {
Self { llm }
pub fn new(llm: Arc<dyn LlmProvider>, safety: Arc<SafetyLayer>) -> Self {
Self { llm, safety }
}
/// Compact a thread's context using the given strategy.
@@ -103,26 +105,27 @@ impl ContextCompactor {
// Generate summary
let summary = self.generate_summary(&to_summarize).await?;
// Write to workspace if available.
// If archival fails, preserve turns to avoid context loss.
let (summary_written, turns_removed) = if let Some(ws) = workspace {
// Write to workspace if available
let summary_written = if let Some(ws) = workspace {
match self.write_summary_to_workspace(ws, &summary).await {
Ok(()) => {
thread.truncate_turns(keep_recent);
(true, turns_to_remove)
}
Ok(()) => true,
Err(e) => {
tracing::warn!("Compaction summary write failed (turns preserved): {}", e);
(false, 0)
tracing::warn!(
"Compaction summary write failed (turns will still be truncated): {}",
e
);
false
}
}
} else {
thread.truncate_turns(keep_recent);
(false, turns_to_remove)
false
};
// Truncate thread
thread.truncate_turns(keep_recent);
Ok(CompactionPartial {
turns_removed,
turns_removed: turns_to_remove,
summary_written,
summary: Some(summary),
})
@@ -164,20 +167,23 @@ impl ContextCompactor {
// Format turns for storage
let content = format_turns_for_storage(old_turns);
// Write to workspace. If archival fails, preserve turns.
let (written, turns_removed) = match self.write_context_to_workspace(ws, &content).await {
Ok(()) => {
thread.truncate_turns(keep_recent);
(true, turns_to_remove)
}
// Write to workspace
let written = match self.write_context_to_workspace(ws, &content).await {
Ok(()) => true,
Err(e) => {
tracing::warn!("Compaction context write failed (turns preserved): {}", e);
(false, 0)
tracing::warn!(
"Compaction context write failed (turns will still be truncated): {}",
e
);
false
}
};
// Truncate
thread.truncate_turns(keep_recent);
Ok(CompactionPartial {
turns_removed,
turns_removed: turns_to_remove,
summary_written: written,
summary: None,
})
@@ -227,7 +233,7 @@ Be brief but capture all important details. Use bullet points."#,
.with_max_tokens(1024)
.with_temperature(0.3);
let reasoning = Reasoning::new(self.llm.clone());
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let (text, _) = reasoning.complete(request).await?;
Ok(text)
}
@@ -340,11 +346,17 @@ mod tests {
// === QA Plan - Compaction strategy tests ===
use crate::agent::context_monitor::CompactionStrategy;
use crate::config::SafetyConfig;
use crate::safety::SafetyLayer;
use crate::testing::StubLlm;
/// Helper: build a `ContextCompactor` with the given `StubLlm`.
fn make_compactor(llm: Arc<StubLlm>) -> ContextCompactor {
ContextCompactor::new(llm)
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
ContextCompactor::new(llm, safety)
}
/// Helper: build a thread with `n` completed turns.
@@ -358,19 +370,6 @@ mod tests {
thread
}
#[cfg(feature = "libsql")]
async fn make_unmigrated_workspace() -> crate::workspace::Workspace {
use crate::db::Database;
use crate::db::libsql::LibSqlBackend;
// Intentionally skip migrations so workspace append operations fail.
let backend = LibSqlBackend::new_memory()
.await
.expect("should create in-memory libsql backend");
let db: Arc<dyn Database> = Arc::new(backend);
crate::workspace::Workspace::new_with_db("compaction-test", db)
}
// ------------------------------------------------------------------
// 1. compact_truncate keeps last N turns
// ------------------------------------------------------------------
@@ -569,43 +568,6 @@ mod tests {
assert_eq!(llm.calls(), 0);
}
#[cfg(feature = "libsql")]
#[tokio::test]
async fn test_compact_with_summary_preserves_turns_when_workspace_write_fails() {
let llm = Arc::new(StubLlm::new("summary"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(8);
let original_inputs: Vec<String> =
thread.turns.iter().map(|t| t.user_input.clone()).collect();
let workspace = make_unmigrated_workspace().await;
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 3 },
Some(&workspace),
)
.await
.expect("compact should succeed even when workspace write fails");
// On archival failure, no turns should be removed.
assert_eq!(thread.turns.len(), 8);
assert_eq!(
thread
.turns
.iter()
.map(|t| t.user_input.as_str())
.collect::<Vec<_>>(),
original_inputs
.iter()
.map(|s| s.as_str())
.collect::<Vec<_>>()
);
assert_eq!(result.turns_removed, 0);
assert!(!result.summary_written);
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 7. compact_to_workspace without workspace falls back to truncation
// ------------------------------------------------------------------
@@ -654,43 +616,6 @@ mod tests {
assert_eq!(result.turns_removed, 0);
}
#[cfg(feature = "libsql")]
#[tokio::test]
async fn test_compact_to_workspace_preserves_turns_when_workspace_write_fails() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(20);
let original_inputs: Vec<String> =
thread.turns.iter().map(|t| t.user_input.clone()).collect();
let workspace = make_unmigrated_workspace().await;
let result = compactor
.compact(
&mut thread,
CompactionStrategy::MoveToWorkspace,
Some(&workspace),
)
.await
.expect("compact should succeed even when workspace write fails");
// On archival failure, no turns should be removed.
assert_eq!(thread.turns.len(), 20);
assert_eq!(
thread
.turns
.iter()
.map(|t| t.user_input.as_str())
.collect::<Vec<_>>(),
original_inputs
.iter()
.map(|s| s.as_str())
.collect::<Vec<_>>()
);
assert_eq!(result.turns_removed, 0);
assert!(!result.summary_written);
assert_eq!(llm.calls(), 0);
}
// ------------------------------------------------------------------
// 9. format_turns_for_storage includes tool calls
// ------------------------------------------------------------------
+7 -42
View File
@@ -131,12 +131,10 @@ impl CostGuard {
// Check hourly rate
if let Some(limit) = self.config.max_actions_per_hour {
let mut window = self.action_window.lock().await;
// checked_sub avoids panic when system uptime < 1 hour (Windows)
if let Some(cutoff) = Instant::now().checked_sub(std::time::Duration::from_secs(3600)) {
// Drain expired entries
while window.front().is_some_and(|t| *t < cutoff) {
window.pop_front();
}
let cutoff = Instant::now() - std::time::Duration::from_secs(3600);
// Drain expired entries
while window.front().is_some_and(|t| *t < cutoff) {
window.pop_front();
}
let count = window.len() as u64;
if count >= limit {
@@ -262,11 +260,9 @@ impl CostGuard {
/// Number of actions in the current hourly window.
pub async fn actions_this_hour(&self) -> u64 {
let mut window = self.action_window.lock().await;
// checked_sub avoids panic when system uptime < 1 hour (Windows)
if let Some(cutoff) = Instant::now().checked_sub(std::time::Duration::from_secs(3600)) {
while window.front().is_some_and(|t| *t < cutoff) {
window.pop_front();
}
let cutoff = Instant::now() - std::time::Duration::from_secs(3600);
while window.front().is_some_and(|t| *t < cutoff) {
window.pop_front();
}
window.len() as u64
}
@@ -625,35 +621,4 @@ mod tests {
"surcharge should be 100% of input cost for 1h cache writes"
);
}
/// Regression test for #657: Instant::now() - Duration panics on Windows
/// when system uptime is less than the subtracted duration.
#[tokio::test]
async fn test_checked_sub_no_panic_on_fresh_guard() {
// A fresh CostGuard with rate limits should not panic even if
// checked_sub returns None (simulating short uptime).
let guard = CostGuard::new(CostGuardConfig {
max_cost_per_day_cents: None,
max_actions_per_hour: Some(100),
});
// These must not panic regardless of system uptime
assert!(guard.check_allowed().await.is_ok());
assert_eq!(guard.actions_this_hour().await, 0);
// Record some actions and verify again
guard
.record_llm_call("gpt-4o", 10, 10, 0, 0, Decimal::ONE, Decimal::ONE, None)
.await;
assert!(guard.check_allowed().await.is_ok());
assert_eq!(guard.actions_this_hour().await, 1);
}
/// Verify that checked_sub itself behaves as expected for the pattern we use.
#[test]
fn test_instant_checked_sub_returns_none_for_overflow() {
// Duration::MAX will always exceed uptime, so checked_sub must return None
let result = Instant::now().checked_sub(std::time::Duration::MAX);
assert!(result.is_none());
}
}
+31 -128
View File
@@ -50,18 +50,8 @@ impl Agent {
// Load workspace system prompt (identity files: AGENTS.md, SOUL.md, etc.)
// In group chats, MEMORY.md is excluded to prevent leaking personal context.
// Resolve the user's timezone
let user_tz = crate::timezone::resolve_timezone(
message.timezone.as_deref(),
None, // user setting lookup can be added later
&self.config.default_timezone,
);
let system_prompt = if let Some(ws) = self.workspace() {
match ws
.system_prompt_for_context_tz(is_group_chat, user_tz)
.await
{
match ws.system_prompt_for_context(is_group_chat).await {
Ok(prompt) if !prompt.is_empty() => Some(prompt),
Ok(_) => None,
Err(e) => {
@@ -113,7 +103,7 @@ impl Agent {
None
};
let mut reasoning = Reasoning::new(self.llm().clone())
let mut reasoning = Reasoning::new(self.llm().clone(), self.safety().clone())
.with_channel(message.channel.clone())
.with_model_name(self.llm().active_model_name())
.with_group_chat(is_group_chat);
@@ -140,7 +130,6 @@ impl Agent {
let mut job_ctx =
JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
job_ctx.http_interceptor = self.deps.http_interceptor.clone();
job_ctx.user_timezone = user_tz.name().to_string();
// Build system prompts once for this turn. Two variants: with tools
// (normal iterations) and without (force_text final iteration).
@@ -681,53 +670,8 @@ impl Agent {
.into())
});
// Detect image generation sentinel in tool output
// (only from image tools — avoids parsing all tool outputs)
let is_image_sentinel = if let Ok(ref output) = tool_result
&& matches!(tc.name.as_str(), "image_generate" | "image_edit")
{
if let Ok(sentinel) =
serde_json::from_str::<serde_json::Value>(output)
&& sentinel.get("type").and_then(|v| v.as_str())
== Some("image_generated")
{
let data_url = sentinel
.get("data")
.and_then(|v| v.as_str())
.unwrap_or_default()
.to_string();
let path = sentinel
.get("path")
.and_then(|v| v.as_str())
.map(String::from);
// Skip broadcasting if data_url is empty to avoid
// sending a broken ImageGenerated SSE event.
if data_url.is_empty() {
tracing::warn!(
"Image generation sentinel has empty data URL, skipping broadcast"
);
} else {
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ImageGenerated { data_url, path },
&message.metadata,
)
.await;
}
true
} else {
false
}
} else {
false
};
// Send ToolResult preview (skip for image sentinels to avoid
// broadcasting multi-MB base64 data as a preview)
if !is_image_sentinel
&& let Ok(ref output) = tool_result
// Send ToolResult preview
if let Ok(ref output) = tool_result
&& !output.is_empty()
{
let _ = self
@@ -743,6 +687,23 @@ impl Agent {
.await;
}
// Record result in thread
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id)
&& let Some(turn) = thread.last_turn_mut()
{
match &tool_result {
Ok(output) => {
turn.record_tool_result(serde_json::json!(output));
}
Err(e) => {
turn.record_tool_error(e.to_string());
}
}
}
}
// Check for auth awaiting — defer the return
// until all results are recorded.
if deferred_auth.is_none()
@@ -782,7 +743,6 @@ impl Agent {
}
// Sanitize and add tool result to context
let is_tool_error = tool_result.is_err();
let result_content = match tool_result {
Ok(output) => {
let sanitized =
@@ -796,23 +756,6 @@ impl Agent {
Err(e) => format!("Tool '{}' failed: {}", tc.name, e),
};
// Record sanitized result in thread so messages()
// and persist_tool_calls() use cleaned content.
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id)
&& let Some(turn) = thread.last_turn_mut()
{
if is_tool_error {
turn.record_tool_error(result_content.clone());
} else {
turn.record_tool_result(serde_json::json!(
result_content
));
}
}
}
context_messages.push(ChatMessage::tool_result(
&tc.id,
&tc.name,
@@ -842,7 +785,6 @@ impl Agent {
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
deferred_tool_calls: tool_calls[approval_idx + 1..].to_vec(),
user_timezone: Some(user_tz.name().to_string()),
};
return Ok(AgenticLoopResult::NeedApproval { pending });
@@ -1204,7 +1146,6 @@ mod tests {
max_actions_per_hour: None,
max_tool_iterations: 50,
auto_approve_tools: false,
default_timezone: "UTC".to_string(),
},
deps,
Arc::new(ChannelManager::new()),
@@ -1307,7 +1248,6 @@ mod tests {
arguments: serde_json::json!({"message": "done"}),
},
],
user_timezone: None,
};
let json = serde_json::to_string(&pending).expect("serialize");
@@ -1655,8 +1595,12 @@ mod tests {
use crate::testing::StubLlm;
let stub = Arc::new(StubLlm::failing_non_transient("ctx-bomb"));
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(stub.clone());
let reasoning = Reasoning::new(stub.clone(), safety);
// Build a fat context with lots of history.
let messages = vec![
@@ -1766,7 +1710,11 @@ mod tests {
use crate::llm::{Reasoning, ReasoningContext, RespondResult, ToolDefinition};
let provider = Arc::new(AlwaysToolCallProvider);
let reasoning = Reasoning::new(provider);
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(provider, safety);
let tool_def = ToolDefinition {
name: "echo".to_string(),
@@ -1952,7 +1900,6 @@ mod tests {
max_actions_per_hour: None,
max_tool_iterations,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
},
deps,
Arc::new(ChannelManager::new()),
@@ -2068,7 +2015,6 @@ mod tests {
max_actions_per_hour: None,
max_tool_iterations: max_iter,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
},
deps,
Arc::new(ChannelManager::new()),
@@ -2170,47 +2116,4 @@ mod tests {
"Error should include the underlying reason, got: {formatted}"
);
}
#[test]
fn test_image_sentinel_empty_data_url_should_be_skipped() {
// Regression: unwrap_or_default() on missing "data" field produces an empty
// string. Broadcasting an empty data_url would send a broken SSE event.
let sentinel = serde_json::json!({
"type": "image_generated",
"path": "/tmp/image.png"
// "data" field is missing
});
let data_url = sentinel
.get("data")
.and_then(|v| v.as_str())
.unwrap_or_default()
.to_string();
assert!(
data_url.is_empty(),
"Missing 'data' field should produce empty string"
);
// The fix: empty data_url means we skip broadcasting
}
#[test]
fn test_image_sentinel_present_data_url_is_valid() {
let sentinel = serde_json::json!({
"type": "image_generated",
"data": "data:image/png;base64,abc123",
"path": "/tmp/image.png"
});
let data_url = sentinel
.get("data")
.and_then(|v| v.as_str())
.unwrap_or_default()
.to_string();
assert!(
!data_url.is_empty(),
"Present 'data' field should produce non-empty string"
);
}
}
+8 -168
View File
@@ -29,8 +29,8 @@ use std::time::Duration;
use tokio::sync::mpsc;
use crate::channels::OutgoingResponse;
use crate::db::Database;
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider, Reasoning};
use crate::safety::SafetyLayer;
use crate::workspace::Workspace;
use crate::workspace::hygiene::HygieneConfig;
@@ -47,12 +47,6 @@ pub struct HeartbeatConfig {
pub notify_user_id: Option<String>,
/// Channel to notify on heartbeat findings.
pub notify_channel: Option<String>,
/// Hour (0-23) when quiet hours start.
pub quiet_hours_start: Option<u32>,
/// Hour (0-23) when quiet hours end.
pub quiet_hours_end: Option<u32>,
/// Timezone for quiet hours evaluation (IANA name).
pub timezone: Option<String>,
}
impl Default for HeartbeatConfig {
@@ -63,9 +57,6 @@ impl Default for HeartbeatConfig {
max_failures: 3,
notify_user_id: None,
notify_channel: None,
quiet_hours_start: None,
quiet_hours_end: None,
timezone: None,
}
}
}
@@ -83,26 +74,6 @@ impl HeartbeatConfig {
self
}
/// Check whether the current time falls within configured quiet hours.
pub fn is_quiet_hours(&self) -> bool {
use chrono::Timelike;
let (Some(start), Some(end)) = (self.quiet_hours_start, self.quiet_hours_end) else {
return false;
};
let tz = self
.timezone
.as_deref()
.and_then(crate::timezone::parse_timezone)
.unwrap_or(chrono_tz::UTC);
let now_hour = crate::timezone::now_in_tz(tz).hour();
if start <= end {
now_hour >= start && now_hour < end
} else {
// Wraps midnight, e.g. 22..06
now_hour >= start || now_hour < end
}
}
/// Set the notification target.
pub fn with_notify(mut self, user_id: impl Into<String>, channel: impl Into<String>) -> Self {
self.notify_user_id = Some(user_id.into());
@@ -130,8 +101,8 @@ pub struct HeartbeatRunner {
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
response_tx: Option<mpsc::Sender<OutgoingResponse>>,
store: Option<Arc<dyn Database>>,
consecutive_failures: u32,
}
@@ -142,14 +113,15 @@ impl HeartbeatRunner {
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
) -> Self {
Self {
config,
hygiene_config,
workspace,
llm,
safety,
response_tx: None,
store: None,
consecutive_failures: 0,
}
}
@@ -160,12 +132,6 @@ impl HeartbeatRunner {
self
}
/// Set the database store for persistent heartbeat conversations.
pub fn with_store(mut self, store: Arc<dyn Database>) -> Self {
self.store = Some(store);
self
}
/// Run the heartbeat loop.
///
/// This runs forever, checking periodically based on the configured interval.
@@ -187,12 +153,6 @@ impl HeartbeatRunner {
loop {
interval.tick().await;
// Skip during quiet hours
if self.config.is_quiet_hours() {
tracing::debug!("Heartbeat skipped: quiet hours");
continue;
}
// Run memory hygiene in the background so it never delays the
// heartbeat checklist. Failures are logged inside run_if_due.
let hygiene_workspace = Arc::clone(&self.workspace);
@@ -303,7 +263,7 @@ impl HeartbeatRunner {
.with_max_tokens(max_tokens)
.with_temperature(0.3);
let reasoning = Reasoning::new(self.llm.clone());
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let (content, _usage) = match reasoning.complete(request).await {
Ok(r) => r,
Err(e) => return HeartbeatResult::Failed(format!("LLM call failed: {}", e)),
@@ -332,32 +292,9 @@ impl HeartbeatRunner {
return;
};
let user_id = self.config.notify_user_id.as_deref().unwrap_or("default");
// Persist to heartbeat conversation and get thread_id
let thread_id = if let Some(ref store) = self.store {
match store.get_or_create_heartbeat_conversation(user_id).await {
Ok(conv_id) => {
if let Err(e) = store
.add_conversation_message(conv_id, "assistant", message)
.await
{
tracing::error!("Failed to persist heartbeat message: {}", e);
}
Some(conv_id.to_string())
}
Err(e) => {
tracing::error!("Failed to get heartbeat conversation: {}", e);
None
}
}
} else {
None
};
let response = OutgoingResponse {
content: format!("🔔 *Heartbeat Alert*\n\n{}", message),
thread_id,
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "heartbeat",
@@ -417,16 +354,13 @@ pub fn spawn_heartbeat(
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
response_tx: Option<mpsc::Sender<OutgoingResponse>>,
store: Option<Arc<dyn Database>>,
) -> tokio::task::JoinHandle<()> {
let mut runner = HeartbeatRunner::new(config, hygiene_config, workspace, llm);
let mut runner = HeartbeatRunner::new(config, hygiene_config, workspace, llm, safety);
if let Some(tx) = response_tx {
runner = runner.with_response_channel(tx);
}
if let Some(s) = store {
runner = runner.with_store(s);
}
tokio::spawn(async move {
runner.run().await;
@@ -561,98 +495,4 @@ mod tests {
let content = "<!-- comment -->\nActual task here";
assert!(!is_effectively_empty(content));
}
// ==================== quiet hours ====================
#[test]
fn test_quiet_hours_inside() {
use chrono::{Timelike, Utc};
let now_utc = Utc::now();
let hour = now_utc.hour();
let start = hour;
let end = (hour + 1) % 24;
let config = HeartbeatConfig {
quiet_hours_start: Some(start),
quiet_hours_end: Some(end),
timezone: Some("UTC".to_string()),
..HeartbeatConfig::default()
};
// Current UTC hour is inside [start, end) by construction
assert!(config.is_quiet_hours());
}
#[test]
fn test_quiet_hours_outside() {
use chrono::{Timelike, Utc};
let now_utc = Utc::now();
let hour = now_utc.hour();
let start = (hour + 1) % 24;
let end = (hour + 2) % 24;
let config = HeartbeatConfig {
quiet_hours_start: Some(start),
quiet_hours_end: Some(end),
timezone: Some("UTC".to_string()),
..HeartbeatConfig::default()
};
// Current UTC hour is outside [start, end) by construction
assert!(!config.is_quiet_hours());
}
#[test]
fn test_quiet_hours_wraparound_excludes_now() {
use chrono::{Timelike, Utc};
let now_utc = Utc::now();
let hour = now_utc.hour();
// Window covers all hours except the current one
let start = (hour + 1) % 24;
let end = hour;
let config = HeartbeatConfig {
quiet_hours_start: Some(start),
quiet_hours_end: Some(end),
timezone: Some("UTC".to_string()),
..HeartbeatConfig::default()
};
assert!(!config.is_quiet_hours());
}
#[test]
fn test_quiet_hours_none_configured() {
let config = HeartbeatConfig::default();
assert!(!config.is_quiet_hours());
}
#[test]
fn test_quiet_hours_same_start_end() {
let config = HeartbeatConfig {
quiet_hours_start: Some(10),
quiet_hours_end: Some(10),
timezone: Some("UTC".to_string()),
..HeartbeatConfig::default()
};
// start == end means zero-width window, should be false
assert!(!config.is_quiet_hours());
}
#[test]
fn test_spawn_heartbeat_accepts_store_param() {
// Regression: spawn_heartbeat must accept an optional Database store
// for persisting heartbeat notifications to a dedicated conversation.
// Compile-time check: the 7th parameter is `Option<Arc<dyn Database>>`.
#[allow(clippy::type_complexity)]
let _fn_ptr: fn(
HeartbeatConfig,
HygieneConfig,
Arc<crate::workspace::Workspace>,
Arc<dyn crate::llm::LlmProvider>,
Option<tokio::sync::mpsc::Sender<crate::channels::OutgoingResponse>>,
Option<Arc<dyn crate::db::Database>>,
) -> tokio::task::JoinHandle<()> = spawn_heartbeat;
let _ = _fn_ptr;
}
}
+9 -87
View File
@@ -57,11 +57,7 @@ pub struct Routine {
#[serde(tag = "type", rename_all = "snake_case")]
pub enum Trigger {
/// Fire on a cron schedule (e.g. "0 9 * * MON-FRI" or "every 2h").
Cron {
schedule: String,
#[serde(default)]
timezone: Option<String>,
},
Cron { schedule: String },
/// Fire when a channel message matches a pattern.
Event {
/// Optional channel filter (e.g. "telegram", "slack").
@@ -103,21 +99,7 @@ impl Trigger {
field: "schedule".into(),
})?
.to_string();
let timezone = config
.get("timezone")
.and_then(|v| v.as_str())
.and_then(|tz| {
if crate::timezone::parse_timezone(tz).is_some() {
Some(tz.to_string())
} else {
tracing::warn!(
"Ignoring invalid timezone '{}' from DB for cron trigger",
tz
);
None
}
});
Ok(Trigger::Cron { schedule, timezone })
Ok(Trigger::Cron { schedule })
}
"event" => {
let pattern = config
@@ -155,10 +137,7 @@ impl Trigger {
/// Serialize trigger-specific config to JSON for DB storage.
pub fn to_config_json(&self) -> serde_json::Value {
match self {
Trigger::Cron { schedule, timezone } => serde_json::json!({
"schedule": schedule,
"timezone": timezone,
}),
Trigger::Cron { schedule } => serde_json::json!({ "schedule": schedule }),
Trigger::Event { channel, pattern } => serde_json::json!({
"pattern": pattern,
"channel": channel,
@@ -436,25 +415,12 @@ pub fn content_hash(content: &str) -> u64 {
}
/// Parse a cron expression and compute the next fire time from now.
///
/// When `timezone` is provided and valid, the schedule is evaluated in that
/// timezone and the result is converted back to UTC. Otherwise UTC is used.
pub fn next_cron_fire(
schedule: &str,
timezone: Option<&str>,
) -> Result<Option<DateTime<Utc>>, RoutineError> {
pub fn next_cron_fire(schedule: &str) -> Result<Option<DateTime<Utc>>, RoutineError> {
let cron_schedule =
cron::Schedule::from_str(schedule).map_err(|e| RoutineError::InvalidCron {
reason: e.to_string(),
})?;
if let Some(tz) = timezone.and_then(crate::timezone::parse_timezone) {
Ok(cron_schedule
.upcoming(tz)
.next()
.map(|dt| dt.with_timezone(&Utc)))
} else {
Ok(cron_schedule.upcoming(Utc).next())
}
Ok(cron_schedule.upcoming(Utc).next())
}
#[cfg(test)]
@@ -467,11 +433,10 @@ mod tests {
fn test_trigger_roundtrip() {
let trigger = Trigger::Cron {
schedule: "0 9 * * MON-FRI".to_string(),
timezone: None,
};
let json = trigger.to_config_json();
let parsed = Trigger::from_db("cron", json).expect("parse cron");
assert!(matches!(parsed, Trigger::Cron { schedule, .. } if schedule == "0 9 * * MON-FRI"));
assert!(matches!(parsed, Trigger::Cron { schedule } if schedule == "0 9 * * MON-FRI"));
}
#[test]
@@ -544,58 +509,16 @@ mod tests {
#[test]
fn test_next_cron_fire_valid() {
// Every minute should always have a next fire
let next = next_cron_fire("* * * * * *", None).expect("valid cron");
let next = next_cron_fire("* * * * * *").expect("valid cron");
assert!(next.is_some());
}
#[test]
fn test_next_cron_fire_invalid() {
let result = next_cron_fire("not a cron", None);
let result = next_cron_fire("not a cron");
assert!(result.is_err());
}
#[test]
fn test_trigger_cron_timezone_roundtrip() {
let trigger = Trigger::Cron {
schedule: "0 9 * * MON-FRI".to_string(),
timezone: Some("America/New_York".to_string()),
};
let json = trigger.to_config_json();
let parsed = Trigger::from_db("cron", json).expect("parse cron");
assert!(matches!(parsed, Trigger::Cron { schedule, timezone }
if schedule == "0 9 * * MON-FRI"
&& timezone.as_deref() == Some("America/New_York")));
}
#[test]
fn test_trigger_cron_no_timezone_backward_compat() {
let json = serde_json::json!({"schedule": "0 9 * * *"});
let parsed = Trigger::from_db("cron", json).expect("parse cron");
assert!(matches!(parsed, Trigger::Cron { timezone, .. } if timezone.is_none()));
}
#[test]
fn test_trigger_cron_invalid_timezone_coerced_to_none() {
let json = serde_json::json!({"schedule": "0 9 * * *", "timezone": "Fake/Zone"});
let parsed = Trigger::from_db("cron", json).expect("parse cron");
assert!(
matches!(parsed, Trigger::Cron { timezone, .. } if timezone.is_none()),
"invalid timezone should be coerced to None"
);
}
#[test]
fn test_next_cron_fire_with_timezone() {
let next_utc = next_cron_fire("0 0 9 * * * *", None)
.expect("valid cron")
.expect("has next");
let next_est = next_cron_fire("0 0 9 * * * *", Some("America/New_York"))
.expect("valid cron")
.expect("has next");
// EST is UTC-5 (or EDT UTC-4), so the UTC result should differ
assert_ne!(next_utc, next_est, "timezone should shift the fire time");
}
#[test]
fn test_guardrails_default() {
let g = RoutineGuardrails::default();
@@ -608,8 +531,7 @@ mod tests {
fn test_trigger_type_tag() {
assert_eq!(
Trigger::Cron {
schedule: String::new(),
timezone: None,
schedule: String::new()
}
.type_tag(),
"cron"
+15 -60
View File
@@ -170,7 +170,7 @@ impl RoutineEngine {
continue;
}
let detail = if let Trigger::Cron { ref schedule, .. } = routine.trigger {
let detail = if let Trigger::Cron { ref schedule } = routine.trigger {
Some(schedule.clone())
} else {
None
@@ -184,11 +184,7 @@ impl RoutineEngine {
///
/// Bypasses cooldown checks (those only apply to cron/event triggers).
/// Still enforces enabled check and concurrent run limit.
pub async fn fire_manual(
&self,
routine_id: Uuid,
user_id: Option<&str>,
) -> Result<Uuid, RoutineError> {
pub async fn fire_manual(&self, routine_id: Uuid) -> Result<Uuid, RoutineError> {
let routine = self
.store
.get_routine(routine_id)
@@ -198,13 +194,6 @@ impl RoutineEngine {
})?
.ok_or(RoutineError::NotFound { id: routine_id })?;
// Enforce ownership when a user_id is provided (gateway calls).
if let Some(uid) = user_id
&& routine.user_id != uid
{
return Err(RoutineError::NotAuthorized { id: routine_id });
}
if !routine.enabled {
return Err(RoutineError::Disabled {
name: routine.name.clone(),
@@ -380,12 +369,8 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
// Update routine runtime state
let now = Utc::now();
let next_fire = if let Trigger::Cron {
ref schedule,
ref timezone,
} = routine.trigger
{
next_cron_fire(schedule, timezone.as_deref()).unwrap_or(None)
let next_fire = if let Trigger::Cron { ref schedule } = routine.trigger {
next_cron_fire(schedule).unwrap_or(None)
} else {
None
};
@@ -411,39 +396,6 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
tracing::error!(routine = %routine.name, "Failed to update runtime state: {}", e);
}
// Persist routine result to its dedicated conversation thread
let thread_id = match ctx
.store
.get_or_create_routine_conversation(routine.id, &routine.name, &routine.user_id)
.await
{
Ok(conv_id) => {
tracing::debug!(
routine = %routine.name,
routine_id = %routine.id,
conversation_id = %conv_id,
"Resolved routine conversation thread"
);
// Record the run result as a conversation message
let msg = match (&summary, status) {
(Some(s), _) => format!("[{}] {}: {}", run.trigger_type, status, s),
(None, _) => format!("[{}] {}", run.trigger_type, status),
};
if let Err(e) = ctx
.store
.add_conversation_message(conv_id, "assistant", &msg)
.await
{
tracing::error!(routine = %routine.name, "Failed to persist routine message: {}", e);
}
Some(conv_id.to_string())
}
Err(e) => {
tracing::error!(routine = %routine.name, "Failed to get routine conversation: {}", e);
None
}
};
// Send notifications based on config
send_notification(
&ctx.notify_tx,
@@ -451,7 +403,6 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
&routine.name,
status,
summary.as_deref(),
thread_id.as_deref(),
)
.await;
}
@@ -492,13 +443,18 @@ async fn execute_full_job(
reason: "scheduler not available".to_string(),
})?;
let mut metadata = serde_json::json!({ "max_iterations": max_iterations });
// Carry the routine's notify config in job metadata so the message tool
// can resolve channel/target per-job without global state mutation.
// Set the message tool's default channel/target from the routine's notify config
// so the LLM can send results without triggering cross-channel approval.
// TODO: This mutates shared global state and can race with concurrent jobs.
// Move notify config into JobContext metadata and apply per-job instead.
if let Some(channel) = &routine.notify.channel {
metadata["notify_channel"] = serde_json::json!(channel);
scheduler
.tools()
.set_message_tool_context(Some(channel.clone()), Some(routine.notify.user.clone()))
.await;
}
metadata["notify_user"] = serde_json::json!(&routine.notify.user);
let metadata = serde_json::json!({ "max_iterations": max_iterations });
// Build approval context: UnlessAutoApproved tools are auto-approved for routines;
// Always tools require explicit listing in tool_permissions.
@@ -655,7 +611,6 @@ async fn send_notification(
routine_name: &str,
status: RunStatus,
summary: Option<&str>,
thread_id: Option<&str>,
) {
let should_notify = match status {
RunStatus::Ok => notify.on_success,
@@ -682,7 +637,7 @@ async fn send_notification(
let response = OutgoingResponse {
content: message,
thread_id: thread_id.map(String::from),
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "routine",
+11 -324
View File
@@ -16,7 +16,6 @@ use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::channels::web::util::truncate_preview;
use crate::llm::{ChatMessage, ToolCall};
/// A session containing one or more threads.
@@ -165,10 +164,6 @@ pub struct PendingApproval {
/// executed yet when approval was requested.
#[serde(default)]
pub deferred_tool_calls: Vec<ToolCall>,
/// User timezone at the time the approval was requested, so it persists
/// through the approval flow even if the approval message lacks timezone.
#[serde(default)]
pub user_timezone: Option<String>,
}
/// A conversation thread within a session.
@@ -321,13 +316,7 @@ impl Thread {
}
}
/// Get all messages for context building, including tool call history.
///
/// Emits the full LLM-compatible message sequence per turn:
/// `user → [assistant_with_tool_calls → tool_result*] → assistant`
///
/// This ensures the LLM sees prior tool executions and won't re-attempt
/// completed actions in subsequent turns.
/// Get all messages for context building.
pub fn messages(&self) -> Vec<ChatMessage> {
let mut messages = Vec::new();
for turn in &self.turns {
@@ -339,42 +328,6 @@ impl Thread {
turn.image_content_parts.clone(),
));
}
if !turn.tool_calls.is_empty() {
// Build ToolCall objects with synthetic stable IDs
let tool_calls: Vec<ToolCall> = turn
.tool_calls
.iter()
.enumerate()
.map(|(i, tc)| ToolCall {
id: format!("turn{}_{}", turn.turn_number, i),
name: tc.name.clone(),
arguments: tc.parameters.clone(),
})
.collect();
// Assistant message declaring the tool calls (no text content)
messages.push(ChatMessage::assistant_with_tool_calls(None, tool_calls));
// Individual tool result messages, truncated to limit context size.
for (i, tc) in turn.tool_calls.iter().enumerate() {
let call_id = format!("turn{}_{}", turn.turn_number, i);
let content = if let Some(ref err) = tc.error {
// .error already contains the full error text;
// pass through without wrapping to avoid double-prefix.
truncate_preview(err, 1000)
} else if let Some(ref res) = tc.result {
let raw = match res {
serde_json::Value::String(s) => s.clone(),
other => other.to_string(),
};
truncate_preview(&raw, 1000)
} else {
"OK".to_string()
};
messages.push(ChatMessage::tool_result(call_id, &tc.name, content));
}
}
if let Some(ref response) = turn.response {
messages.push(ChatMessage::assistant(response));
}
@@ -396,16 +349,13 @@ impl Thread {
/// Restore thread state from a checkpoint's messages.
///
/// Clears existing turns and rebuilds from the message sequence.
/// Handles the full message pattern including tool messages:
/// `user → [assistant_with_tool_calls → tool_result*] → assistant`
///
/// Also supports the legacy pattern (user/assistant pairs only) for
/// backward compatibility with old checkpoint data.
/// Clears existing turns and rebuilds from message pairs.
/// Messages should alternate: user, assistant, user, assistant...
pub fn restore_from_messages(&mut self, messages: Vec<ChatMessage>) {
self.turns.clear();
self.state = ThreadState::Idle;
// Messages alternate: user, assistant, user, assistant...
let mut iter = messages.into_iter().peekable();
let mut turn_number = 0;
@@ -413,58 +363,18 @@ impl Thread {
if msg.role == crate::llm::Role::User {
let mut turn = Turn::new(turn_number, &msg.content);
// Consume tool call sequences (assistant_with_tool_calls + tool_results).
// A single turn may contain multiple rounds of tool calls, so we
// track the cumulative base index into turn.tool_calls.
while let Some(next) = iter.peek() {
if next.role == crate::llm::Role::Assistant && next.tool_calls.is_some() {
let call_base_idx = turn.tool_calls.len();
if let Some(assistant_msg) = iter.next()
&& let Some(ref tcs) = assistant_msg.tool_calls
{
for tc in tcs {
turn.record_tool_call(&tc.name, tc.arguments.clone());
}
}
// Consume the corresponding tool_result messages,
// indexing relative to this batch's base offset.
let mut pos = 0;
while let Some(tr) = iter.peek() {
if tr.role != crate::llm::Role::Tool {
break;
}
if let Some(tool_msg) = iter.next() {
let idx = call_base_idx + pos;
if idx < turn.tool_calls.len() {
// Store as result — the error/success distinction
// is for the live turn only; restored context just
// needs the content the LLM originally saw.
turn.tool_calls[idx].result =
Some(serde_json::Value::String(tool_msg.content.clone()));
}
}
pos += 1;
}
} else {
break;
// Check if next is assistant response
if let Some(next) = iter.peek()
&& next.role == crate::llm::Role::Assistant
{
// iter.next() is guaranteed Some after a successful peek()
if let Some(response) = iter.next() {
turn.complete(&response.content);
}
}
// Check if next is the final assistant response for this turn
let is_final_assistant = iter.peek().is_some_and(|n| {
n.role == crate::llm::Role::Assistant && n.tool_calls.is_none()
});
if is_final_assistant && let Some(response) = iter.next() {
turn.complete(&response.content);
}
self.turns.push(turn);
turn_number += 1;
} else {
// Skip non-user messages that aren't anchored to a turn
continue;
}
}
@@ -1066,7 +976,6 @@ mod tests {
tool_call_id: "call_123".to_string(),
context_messages: vec![ChatMessage::user("do it")],
deferred_tool_calls: vec![],
user_timezone: None,
};
thread.await_approval(approval);
@@ -1092,7 +1001,6 @@ mod tests {
tool_call_id: "call_456".to_string(),
context_messages: vec![],
deferred_tool_calls: vec![],
user_timezone: None,
};
thread.await_approval(approval);
@@ -1121,225 +1029,4 @@ mod tests {
ThreadState::Processing
);
}
// Regression tests for #568: tool call history must survive hydration.
#[test]
fn test_messages_includes_tool_calls() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("Search for X");
{
let turn = thread.turns.last_mut().unwrap();
turn.record_tool_call("memory_search", serde_json::json!({"query": "X"}));
turn.record_tool_result(serde_json::json!("Found X in doc.md"));
}
thread.complete_turn("I found X in doc.md.");
let messages = thread.messages();
// user + assistant_with_tool_calls + tool_result + assistant = 4
assert_eq!(messages.len(), 4);
assert_eq!(messages[0].role, crate::llm::Role::User);
assert_eq!(messages[0].content, "Search for X");
assert_eq!(messages[1].role, crate::llm::Role::Assistant);
assert!(messages[1].tool_calls.is_some());
let tcs = messages[1].tool_calls.as_ref().unwrap();
assert_eq!(tcs.len(), 1);
assert_eq!(tcs[0].name, "memory_search");
assert_eq!(messages[2].role, crate::llm::Role::Tool);
assert!(messages[2].content.contains("Found X"));
assert_eq!(messages[3].role, crate::llm::Role::Assistant);
assert_eq!(messages[3].content, "I found X in doc.md.");
}
#[test]
fn test_messages_multiple_tool_calls_per_turn() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("Do two things");
{
let turn = thread.turns.last_mut().unwrap();
turn.record_tool_call("echo", serde_json::json!({"msg": "a"}));
turn.record_tool_result(serde_json::json!("a"));
turn.record_tool_call("time", serde_json::json!({}));
turn.record_tool_error("timeout");
}
thread.complete_turn("Done.");
let messages = thread.messages();
// user + assistant_with_calls(2) + tool_result + tool_result + assistant = 5
assert_eq!(messages.len(), 5);
let tcs = messages[1].tool_calls.as_ref().unwrap();
assert_eq!(tcs.len(), 2);
// First tool: success
assert_eq!(messages[2].content, "a");
// Second tool: error (passed through directly, no wrapping)
assert!(messages[3].content.contains("timeout"));
}
#[test]
fn test_restore_from_messages_with_tool_calls() {
let mut thread = Thread::new(Uuid::new_v4());
// Build a message sequence with tool calls
let tc = ToolCall {
id: "call_0".to_string(),
name: "search".to_string(),
arguments: serde_json::json!({"q": "test"}),
};
let messages = vec![
ChatMessage::user("Find test"),
ChatMessage::assistant_with_tool_calls(None, vec![tc]),
ChatMessage::tool_result("call_0", "search", "result: found"),
ChatMessage::assistant("Found it."),
];
thread.restore_from_messages(messages);
assert_eq!(thread.turns.len(), 1);
let turn = &thread.turns[0];
assert_eq!(turn.user_input, "Find test");
assert_eq!(turn.tool_calls.len(), 1);
assert_eq!(turn.tool_calls[0].name, "search");
assert_eq!(
turn.tool_calls[0].result,
Some(serde_json::Value::String("result: found".to_string()))
);
assert_eq!(turn.response, Some("Found it.".to_string()));
}
#[test]
fn test_restore_from_messages_with_tool_error() {
let mut thread = Thread::new(Uuid::new_v4());
let tc = ToolCall {
id: "call_0".to_string(),
name: "http".to_string(),
arguments: serde_json::json!({}),
};
let messages = vec![
ChatMessage::user("Fetch URL"),
ChatMessage::assistant_with_tool_calls(None, vec![tc]),
ChatMessage::tool_result("call_0", "http", "Error: timeout"),
ChatMessage::assistant("The request timed out."),
];
thread.restore_from_messages(messages);
// restore_from_messages stores all tool content as result (not error),
// because it can't reliably distinguish errors from results that happen
// to start with "Error: ". The content is preserved for LLM context.
let turn = &thread.turns[0];
assert_eq!(
turn.tool_calls[0].result,
Some(serde_json::Value::String("Error: timeout".to_string()))
);
}
#[test]
fn test_messages_round_trip_with_tools() {
// Build a thread with tool calls, get messages(), restore, get messages() again
// The two message sequences should be equivalent.
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("Do search");
{
let turn = thread.turns.last_mut().unwrap();
turn.record_tool_call("search", serde_json::json!({"q": "test"}));
turn.record_tool_result(serde_json::json!("found"));
}
thread.complete_turn("Here are results.");
let messages_original = thread.messages();
// Restore into a new thread
let mut thread2 = Thread::new(Uuid::new_v4());
thread2.restore_from_messages(messages_original.clone());
let messages_restored = thread2.messages();
// Same number of messages
assert_eq!(messages_original.len(), messages_restored.len());
// Same roles
for (orig, rest) in messages_original.iter().zip(messages_restored.iter()) {
assert_eq!(orig.role, rest.role);
}
// Same final response
assert_eq!(
messages_original.last().unwrap().content,
messages_restored.last().unwrap().content
);
}
#[test]
fn test_restore_multi_stage_tool_calls() {
let mut thread = Thread::new(Uuid::new_v4());
let tc1 = ToolCall {
id: "call_a".to_string(),
name: "search".to_string(),
arguments: serde_json::json!({"q": "data"}),
};
let tc2 = ToolCall {
id: "call_b".to_string(),
name: "write".to_string(),
arguments: serde_json::json!({"path": "out.txt"}),
};
let messages = vec![
ChatMessage::user("Find and save"),
ChatMessage::assistant_with_tool_calls(None, vec![tc1]),
ChatMessage::tool_result("call_a", "search", "found data"),
ChatMessage::assistant_with_tool_calls(None, vec![tc2]),
ChatMessage::tool_result("call_b", "write", "written"),
ChatMessage::assistant("Done, saved to out.txt"),
];
thread.restore_from_messages(messages);
assert_eq!(thread.turns.len(), 1);
let turn = &thread.turns[0];
assert_eq!(turn.tool_calls.len(), 2);
assert_eq!(turn.tool_calls[0].name, "search");
assert_eq!(turn.tool_calls[1].name, "write");
assert_eq!(
turn.tool_calls[0].result,
Some(serde_json::Value::String("found data".to_string()))
);
assert_eq!(
turn.tool_calls[1].result,
Some(serde_json::Value::String("written".to_string()))
);
assert_eq!(turn.response, Some("Done, saved to out.txt".to_string()));
}
#[test]
fn test_messages_truncates_large_tool_results() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("Read big file");
{
let turn = thread.turns.last_mut().unwrap();
turn.record_tool_call("read_file", serde_json::json!({"path": "big.txt"}));
let big_result = "x".repeat(2000);
turn.record_tool_result(serde_json::json!(big_result));
}
thread.complete_turn("Here's the file content.");
let messages = thread.messages();
let tool_result_content = &messages[2].content;
assert!(
tool_result_content.len() <= 1010,
"Tool result should be truncated, got {} chars",
tool_result_content.len()
);
assert!(tool_result_content.ends_with("..."));
}
}
+60 -305
View File
@@ -20,7 +20,7 @@ use crate::channels::web::util::truncate_preview;
use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
use crate::llm::{ChatMessage, ToolCall};
use crate::llm::ChatMessage;
use crate::tools::redact_params;
impl Agent {
@@ -66,7 +66,16 @@ impl Agent {
.await
.unwrap_or_default();
msg_count = db_messages.len();
chat_messages = rebuild_chat_messages_from_db(&db_messages);
chat_messages = db_messages
.iter()
.filter_map(|m| match m.role.as_str() {
"user" => Some(ChatMessage::user(&m.content)),
"assistant" => Some(ChatMessage::assistant(&m.content)),
// tool_calls rows are UI metadata (tool name + preview),
// not part of the LLM conversation context.
_ => None,
})
.collect();
} else {
msg_count = 0;
}
@@ -221,7 +230,7 @@ impl Agent {
)
.await;
let compactor = ContextCompactor::new(self.llm().clone());
let compactor = ContextCompactor::new(self.llm().clone(), self.safety().clone());
if let Err(e) = compactor
.compact(thread, strategy, self.workspace().map(|w| w.as_ref()))
.await
@@ -331,10 +340,10 @@ impl Agent {
};
thread.complete_turn(&response);
let (turn_number, tool_calls) = thread
let tool_calls = thread
.turns
.last()
.map(|t| (t.turn_number, t.tool_calls.clone()))
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
let _ = self
.channels
@@ -346,7 +355,7 @@ impl Agent {
.await;
// Persist tool calls then assistant response (user message already persisted at turn start)
self.persist_tool_calls(thread_id, &message.user_id, turn_number, &tool_calls)
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
@@ -455,7 +464,6 @@ impl Agent {
&self,
thread_id: Uuid,
user_id: &str,
turn_number: usize,
tool_calls: &[crate::agent::session::TurnToolCall],
) {
if tool_calls.is_empty() {
@@ -469,24 +477,14 @@ impl Agent {
let summaries: Vec<serde_json::Value> = tool_calls
.iter()
.enumerate()
.map(|(i, tc)| {
let mut obj = serde_json::json!({
"name": tc.name,
"call_id": format!("turn{}_{}", turn_number, i),
});
.map(|tc| {
let mut obj = serde_json::json!({ "name": tc.name });
if let Some(ref result) = tc.result {
let preview = match result {
serde_json::Value::String(s) => truncate_preview(s, 500),
other => truncate_preview(&other.to_string(), 500),
};
obj["result_preview"] = serde_json::Value::String(preview);
// Store full result (truncated to ~1000 chars) for LLM context rebuild
let full_result = match result {
serde_json::Value::String(s) => truncate_preview(s, 1000),
other => truncate_preview(&other.to_string(), 1000),
};
obj["result"] = serde_json::Value::String(full_result);
}
if let Some(ref error) = tc.error {
obj["error"] = serde_json::Value::String(truncate_preview(error, 200));
@@ -629,7 +627,7 @@ impl Agent {
crate::agent::context_monitor::CompactionStrategy::Summarize { keep_recent: 5 },
);
let compactor = ContextCompactor::new(self.llm().clone());
let compactor = ContextCompactor::new(self.llm().clone(), self.safety().clone());
match compactor
.compact(thread, strategy, self.workspace().map(|w| w.as_ref()))
.await
@@ -748,16 +746,6 @@ impl Agent {
let mut job_ctx =
JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
job_ctx.http_interceptor = self.deps.http_interceptor.clone();
// Prefer a valid timezone from the approval message, fall back to the
// resolved timezone stored when the approval was originally requested.
let tz_candidate = message
.timezone
.as_deref()
.filter(|tz| crate::timezone::parse_timezone(tz).is_some())
.or(pending.user_timezone.as_deref());
if let Some(tz) = tz_candidate {
job_ctx.user_timezone = tz.to_string();
}
let _ = self
.channels
@@ -809,33 +797,19 @@ impl Agent {
let mut context_messages = pending.context_messages;
let deferred_tool_calls = pending.deferred_tool_calls;
// Sanitize tool result, then record the cleaned version in the
// thread. Must happen before auth intercept check which may return early.
let is_tool_error = tool_result.is_err();
let result_content = match &tool_result {
Ok(output) => {
let sanitized = self
.safety()
.sanitize_tool_output(&pending.tool_name, output);
self.safety().wrap_for_llm(
&pending.tool_name,
&sanitized.content,
sanitized.was_modified,
)
}
Err(e) => format!("Error: {}", e),
};
// Record sanitized result in thread
// Record result in thread
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id)
&& let Some(turn) = thread.last_turn_mut()
{
if is_tool_error {
turn.record_tool_error(result_content.clone());
} else {
turn.record_tool_result(serde_json::json!(result_content));
match &tool_result {
Ok(output) => {
turn.record_tool_result(serde_json::json!(output));
}
Err(e) => {
turn.record_tool_error(e.to_string());
}
}
}
}
@@ -857,6 +831,21 @@ impl Agent {
return Ok(SubmissionResult::response(instructions));
}
// Add tool result to context
let result_content = match tool_result {
Ok(output) => {
let sanitized = self
.safety()
.sanitize_tool_output(&pending.tool_name, &output);
self.safety().wrap_for_llm(
&pending.tool_name,
&sanitized.content,
sanitized.was_modified,
)
}
Err(e) => format!("Error: {}", e),
};
context_messages.push(ChatMessage::tool_result(
&pending.tool_call_id,
&pending.tool_name,
@@ -1061,31 +1050,15 @@ impl Agent {
.await;
}
// Sanitize first, then record the cleaned version in thread.
// Must happen before auth detection which may set deferred_auth.
let is_deferred_error = deferred_result.is_err();
let deferred_content = match &deferred_result {
Ok(output) => {
let sanitized = self.safety().sanitize_tool_output(&tc.name, output);
self.safety().wrap_for_llm(
&tc.name,
&sanitized.content,
sanitized.was_modified,
)
}
Err(e) => format!("Error: {}", e),
};
// Record sanitized result in thread
// Record in thread
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id)
&& let Some(turn) = thread.last_turn_mut()
{
if is_deferred_error {
turn.record_tool_error(deferred_content.clone());
} else {
turn.record_tool_result(serde_json::json!(deferred_content));
match &deferred_result {
Ok(output) => turn.record_tool_result(serde_json::json!(output)),
Err(e) => turn.record_tool_error(e.to_string()),
}
}
}
@@ -1107,6 +1080,18 @@ impl Agent {
deferred_auth = Some(instructions);
}
let deferred_content = match deferred_result {
Ok(output) => {
let sanitized = self.safety().sanitize_tool_output(&tc.name, &output);
self.safety().wrap_for_llm(
&tc.name,
&sanitized.content,
sanitized.was_modified,
)
}
Err(e) => format!("Error: {}", e),
};
context_messages.push(ChatMessage::tool_result(&tc.id, &tc.name, deferred_content));
}
@@ -1126,8 +1111,6 @@ impl Agent {
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
deferred_tool_calls: deferred_tool_calls[approval_idx + 1..].to_vec(),
// Carry forward the resolved timezone from the original pending approval
user_timezone: pending.user_timezone.clone(),
};
let request_id = new_pending.request_id;
@@ -1174,13 +1157,13 @@ impl Agent {
match result {
Ok(AgenticLoopResult::Response(response)) => {
thread.complete_turn(&response);
let (turn_number, tool_calls) = thread
let tool_calls = thread
.turns
.last()
.map(|t| (t.turn_number, t.tool_calls.clone()))
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
// User message already persisted at turn start; save tool calls then assistant response
self.persist_tool_calls(thread_id, &message.user_id, turn_number, &tool_calls)
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
@@ -1495,231 +1478,3 @@ impl Agent {
}
}
}
/// Rebuild full LLM-compatible `ChatMessage` sequence from DB messages.
///
/// Parses `role="tool_calls"` rows to reconstruct `assistant_with_tool_calls`
/// and `tool_result` messages so that the LLM sees the complete tool execution
/// history on thread hydration. Falls back gracefully for legacy rows that
/// lack the enriched fields (`call_id`, `parameters`, `result`).
fn rebuild_chat_messages_from_db(
db_messages: &[crate::history::ConversationMessage],
) -> Vec<ChatMessage> {
let mut result = Vec::new();
for msg in db_messages {
match msg.role.as_str() {
"user" => result.push(ChatMessage::user(&msg.content)),
"assistant" => result.push(ChatMessage::assistant(&msg.content)),
"tool_calls" => {
// Try to parse the enriched JSON and rebuild tool messages.
if let Ok(calls) = serde_json::from_str::<Vec<serde_json::Value>>(&msg.content) {
if calls.is_empty() {
continue;
}
// Check if this is an enriched row (has call_id) or legacy
let has_call_id = calls
.first()
.and_then(|c| c.get("call_id"))
.and_then(|v| v.as_str())
.is_some();
if has_call_id {
// Build assistant_with_tool_calls + tool_result messages
let tool_calls: Vec<ToolCall> = calls
.iter()
.map(|c| ToolCall {
id: c["call_id"].as_str().unwrap_or("call_0").to_string(),
name: c["name"].as_str().unwrap_or("unknown").to_string(),
arguments: c
.get("parameters")
.cloned()
.unwrap_or(serde_json::json!({})),
})
.collect();
// The assistant text for tool_calls is always None here;
// the final assistant response comes as a separate
// "assistant" row after this tool_calls row.
result.push(ChatMessage::assistant_with_tool_calls(None, tool_calls));
// Emit tool_result messages for each call
for c in &calls {
let call_id = c["call_id"].as_str().unwrap_or("call_0").to_string();
let name = c["name"].as_str().unwrap_or("unknown").to_string();
let content = if let Some(err) = c.get("error").and_then(|v| v.as_str())
{
format!("Error: {}", err)
} else if let Some(res) = c.get("result").and_then(|v| v.as_str()) {
res.to_string()
} else if let Some(preview) =
c.get("result_preview").and_then(|v| v.as_str())
{
preview.to_string()
} else {
"OK".to_string()
};
result.push(ChatMessage::tool_result(call_id, name, content));
}
}
// Legacy rows without call_id: skip (will appear as
// simple user/assistant pairs, same as before this fix).
}
}
_ => {} // Skip unknown roles
}
}
result
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_rebuild_chat_messages_user_assistant_only() {
let messages = vec![
make_db_msg("user", "Hello"),
make_db_msg("assistant", "Hi there!"),
];
let result = rebuild_chat_messages_from_db(&messages);
assert_eq!(result.len(), 2);
assert_eq!(result[0].role, crate::llm::Role::User);
assert_eq!(result[1].role, crate::llm::Role::Assistant);
}
#[test]
fn test_rebuild_chat_messages_with_enriched_tool_calls() {
let tool_json = serde_json::json!([
{
"name": "memory_search",
"call_id": "call_0",
"parameters": {"query": "test"},
"result": "Found 3 results",
"result_preview": "Found 3 re..."
},
{
"name": "echo",
"call_id": "call_1",
"parameters": {"message": "hi"},
"error": "timeout"
}
]);
let messages = vec![
make_db_msg("user", "Search for test"),
make_db_msg("tool_calls", &tool_json.to_string()),
make_db_msg("assistant", "I found some results."),
];
let result = rebuild_chat_messages_from_db(&messages);
// user + assistant_with_tool_calls + tool_result*2 + assistant
assert_eq!(result.len(), 5);
// user
assert_eq!(result[0].role, crate::llm::Role::User);
// assistant with tool_calls
assert_eq!(result[1].role, crate::llm::Role::Assistant);
assert!(result[1].tool_calls.is_some());
let tcs = result[1].tool_calls.as_ref().unwrap();
assert_eq!(tcs.len(), 2);
assert_eq!(tcs[0].name, "memory_search");
assert_eq!(tcs[0].id, "call_0");
assert_eq!(tcs[1].name, "echo");
// tool results
assert_eq!(result[2].role, crate::llm::Role::Tool);
assert_eq!(result[2].tool_call_id, Some("call_0".to_string()));
assert!(result[2].content.contains("Found 3 results"));
assert_eq!(result[3].role, crate::llm::Role::Tool);
assert_eq!(result[3].tool_call_id, Some("call_1".to_string()));
assert!(result[3].content.contains("Error: timeout"));
// final assistant
assert_eq!(result[4].role, crate::llm::Role::Assistant);
assert_eq!(result[4].content, "I found some results.");
}
#[test]
fn test_rebuild_chat_messages_legacy_tool_calls_skipped() {
// Legacy format: no call_id field
let tool_json = serde_json::json!([
{"name": "echo", "result_preview": "hello"}
]);
let messages = vec![
make_db_msg("user", "Hi"),
make_db_msg("tool_calls", &tool_json.to_string()),
make_db_msg("assistant", "Done"),
];
let result = rebuild_chat_messages_from_db(&messages);
// Legacy rows are skipped, only user + assistant
assert_eq!(result.len(), 2);
assert_eq!(result[0].role, crate::llm::Role::User);
assert_eq!(result[1].role, crate::llm::Role::Assistant);
}
#[test]
fn test_rebuild_chat_messages_empty() {
let result = rebuild_chat_messages_from_db(&[]);
assert!(result.is_empty());
}
#[test]
fn test_rebuild_chat_messages_malformed_tool_calls_json() {
let messages = vec![
make_db_msg("user", "Hi"),
make_db_msg("tool_calls", "not valid json"),
make_db_msg("assistant", "Done"),
];
let result = rebuild_chat_messages_from_db(&messages);
// Malformed JSON is silently skipped
assert_eq!(result.len(), 2);
}
#[test]
fn test_rebuild_chat_messages_multi_turn_with_tools() {
let tool_json_1 = serde_json::json!([
{"name": "search", "call_id": "call_0", "parameters": {}, "result": "found it"}
]);
let tool_json_2 = serde_json::json!([
{"name": "write", "call_id": "call_0", "parameters": {"path": "a.txt"}, "result": "ok"}
]);
let messages = vec![
make_db_msg("user", "Find X"),
make_db_msg("tool_calls", &tool_json_1.to_string()),
make_db_msg("assistant", "Found X"),
make_db_msg("user", "Write it"),
make_db_msg("tool_calls", &tool_json_2.to_string()),
make_db_msg("assistant", "Written"),
];
let result = rebuild_chat_messages_from_db(&messages);
// Turn 1: user + assistant_with_calls + tool_result + assistant = 4
// Turn 2: user + assistant_with_calls + tool_result + assistant = 4
assert_eq!(result.len(), 8);
// Verify turn boundaries
assert_eq!(result[0].content, "Find X");
assert!(result[1].tool_calls.is_some());
assert_eq!(result[2].role, crate::llm::Role::Tool);
assert_eq!(result[3].content, "Found X");
assert_eq!(result[4].content, "Write it");
assert!(result[5].tool_calls.is_some());
assert_eq!(result[6].role, crate::llm::Role::Tool);
assert_eq!(result[7].content, "Written");
}
fn make_db_msg(role: &str, content: &str) -> crate::history::ConversationMessage {
crate::history::ConversationMessage {
id: uuid::Uuid::new_v4(),
role: role.to_string(),
content: content.to_string(),
created_at: chrono::Utc::now(),
}
}
}
+32 -63
View File
@@ -15,8 +15,7 @@ use crate::db::Database;
use crate::error::Error;
use crate::hooks::HookRegistry;
use crate::llm::{
ActionPlan, ChatMessage, LlmProvider, Reasoning, ReasoningContext, RespondResult, ToolCall,
ToolSelection,
ActionPlan, ChatMessage, LlmProvider, Reasoning, ReasoningContext, RespondResult, ToolSelection,
};
use crate::safety::SafetyLayer;
use crate::tools::rate_limiter::RateLimitResult;
@@ -212,7 +211,7 @@ impl Worker {
let job_ctx = self.context_manager().get_context(self.job_id).await?;
// Create reasoning engine
let reasoning = Reasoning::new(self.llm().clone());
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
// Build initial reasoning context (tool definitions refreshed each iteration in execution_loop)
let mut reason_ctx = ReasoningContext::new().with_job(&job_ctx.description);
@@ -577,54 +576,37 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
}
}
}
} else {
} else if selections.len() == 1 {
consecutive_tool_intent_nudges = 0;
// Single tool: execute directly
let selection = &selections[0];
tracing::debug!(
"Job {} selecting tool: {} - {}",
self.job_id,
selection.tool_name,
selection.reasoning
);
// Record the assistant tool_calls message so that tool_result
// messages have a matching parent (prevents orphaned rewrites).
let tool_calls: Vec<ToolCall> = selections
.iter()
.map(|s| ToolCall {
id: s.tool_call_id.clone(),
name: s.tool_name.clone(),
arguments: s.parameters.clone(),
})
.collect();
reason_ctx
.messages
.push(ChatMessage::assistant_with_tool_calls(None, tool_calls));
let result = self
.execute_tool(&selection.tool_name, &selection.parameters)
.await;
if selections.len() == 1 {
// Single tool: execute directly
let selection = &selections[0];
tracing::debug!(
"Job {} selecting tool: {} - {}",
self.job_id,
selection.tool_name,
selection.reasoning
);
self.process_tool_result(reason_ctx, selection, result)
.await?;
} else {
// Multiple tools: execute in parallel
tracing::debug!(
"Job {} executing {} tools in parallel",
self.job_id,
selections.len()
);
let result = self
.execute_tool(&selection.tool_name, &selection.parameters)
.await;
let results = self.execute_tools_parallel(&selections).await;
self.process_tool_result(reason_ctx, selection, result)
// Process all results
for (selection, result) in selections.iter().zip(results) {
self.process_tool_result(reason_ctx, selection, result.result)
.await?;
} else {
// Multiple tools: execute in parallel
tracing::debug!(
"Job {} executing {} tools in parallel",
self.job_id,
selections.len()
);
let results = self.execute_tools_parallel(&selections).await;
// Process all results
for (selection, result) in selections.iter().zip(results) {
self.process_tool_result(reason_ctx, selection, result.result)
.await?;
}
}
}
@@ -1105,6 +1087,11 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
action.reasoning
);
// Execute the planned tool
let result = self
.execute_tool(&action.tool_name, &action.parameters)
.await;
// Create a synthetic ToolSelection for process_tool_result.
// Plan actions don't originate from an LLM tool_call response so
// there is no real tool_call_id; generate a unique one.
@@ -1116,24 +1103,6 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tool_call_id: format!("plan_{}_{}", self.job_id, i),
};
// Record the assistant tool_calls message so that the tool_result
// has a matching parent (prevents orphaned rewrites).
reason_ctx
.messages
.push(ChatMessage::assistant_with_tool_calls(
None,
vec![ToolCall {
id: selection.tool_call_id.clone(),
name: selection.tool_name.clone(),
arguments: selection.parameters.clone(),
}],
));
// Execute the planned tool
let result = self
.execute_tool(&action.tool_name, &action.parameters)
.await;
// Process the result
let completed = self
.process_tool_result(reason_ctx, &selection, result)
+94 -248
View File
@@ -21,7 +21,7 @@ use crate::secrets::SecretsStore;
use crate::skills::SkillRegistry;
use crate::skills::catalog::SkillCatalog;
use crate::tools::ToolRegistry;
use crate::tools::mcp::{McpProcessManager, McpSessionManager};
use crate::tools::mcp::McpSessionManager;
use crate::tools::wasm::SharedCredentialRegistry;
use crate::tools::wasm::WasmToolRuntime;
use crate::workspace::{EmbeddingProvider, Workspace};
@@ -41,7 +41,6 @@ pub struct AppComponents {
pub workspace: Option<Arc<Workspace>>,
pub extension_manager: Option<Arc<ExtensionManager>>,
pub mcp_session_manager: Arc<McpSessionManager>,
pub mcp_process_manager: Arc<McpProcessManager>,
pub wasm_tool_runtime: Option<Arc<WasmToolRuntime>>,
pub log_broadcaster: Arc<LogBroadcaster>,
pub context_manager: Arc<ContextManager>,
@@ -245,31 +244,11 @@ impl AppBuilder {
let master_key = match self.config.secrets.master_key() {
Some(k) => k,
None => {
// No secrets DB available, but we can still load tokens from
// OS credential stores (e.g., Anthropic OAuth via Claude Code's
// macOS Keychain / Linux ~/.claude/.credentials.json).
crate::config::inject_os_credentials();
// Consume unused handles
#[cfg(feature = "libsql")]
{
self.libsql_db.take();
}
// Re-resolve only the LLM config with OS credentials.
let store: Option<&(dyn crate::db::SettingsStore + Sync)> =
self.db.as_ref().map(|db| db.as_ref() as _);
let toml_path = self.toml_path.as_deref();
if let Err(e) = self
.config
.re_resolve_llm(store, "default", toml_path)
.await
{
tracing::warn!(
"Failed to re-resolve LLM config after OS credential injection: {e}"
);
}
return Ok(());
}
};
@@ -312,16 +291,18 @@ impl AppBuilder {
// Inject LLM API keys from encrypted storage
crate::config::inject_llm_keys_from_secrets(secrets.as_ref(), "default").await;
// Re-resolve only the LLM config with newly available keys.
let store: Option<&(dyn crate::db::SettingsStore + Sync)> =
self.db.as_ref().map(|db| db.as_ref() as _);
let toml_path = self.toml_path.as_deref();
if let Err(e) = self
.config
.re_resolve_llm(store, "default", toml_path)
.await
{
tracing::warn!("Failed to re-resolve LLM config after secret injection: {e}");
// Re-resolve config with newly available keys
if let Some(ref db) = self.db {
let toml_path = self.toml_path.as_deref();
match Config::from_db_with_toml(db.as_ref(), "default", toml_path).await {
Ok(refreshed) => {
self.config = refreshed;
tracing::debug!("LlmConfig re-resolved after secret injection");
}
Err(e) => {
tracing::warn!("Failed to re-resolve config after secret injection: {}", e);
}
}
}
}
@@ -334,7 +315,7 @@ impl AppBuilder {
/// Delegates to `build_provider_chain` which applies all decorators
/// (retry, smart routing, failover, circuit breaker, response cache).
#[allow(clippy::type_complexity)]
pub async fn init_llm(
pub fn init_llm(
&self,
) -> Result<
(
@@ -345,7 +326,7 @@ impl AppBuilder {
anyhow::Error,
> {
let (llm, cheap_llm, recording_handle) =
crate::llm::build_provider_chain(&self.config.llm, self.session.clone()).await?;
crate::llm::build_provider_chain(&self.config.llm, self.session.clone())?;
Ok((llm, cheap_llm, recording_handle))
}
@@ -400,55 +381,16 @@ impl AppBuilder {
None
};
// Register image/vision tools if we have a workspace and LLM API credentials
if workspace.is_some() {
let (api_base, api_key_opt) = if let Some(ref provider) = self.config.llm.provider {
(
provider.base_url.clone(),
provider.api_key.as_ref().map(|s| {
use secrecy::ExposeSecret;
s.expose_secret().to_string()
}),
)
} else {
(
self.config.llm.nearai.base_url.clone(),
self.config.llm.nearai.api_key.as_ref().map(|s| {
use secrecy::ExposeSecret;
s.expose_secret().to_string()
}),
)
};
if let Some(api_key) = api_key_opt {
// Check for image generation models
let model_name = self
.config
.llm
.provider
.as_ref()
.map(|p| p.model.clone())
.unwrap_or_else(|| self.config.llm.nearai.model.clone());
let models = vec![model_name.clone()];
let gen_model = crate::llm::image_models::suggest_image_model(&models)
.unwrap_or("flux-1.1-pro")
.to_string();
tools.register_image_tools(api_base.clone(), api_key.clone(), gen_model, None);
// Check for vision models
let vision_model = crate::llm::vision_models::suggest_vision_model(&models)
.unwrap_or(&model_name)
.to_string();
tools.register_vision_tools(api_base, api_key, vision_model, None);
}
}
// Register builder tool if enabled
if self.config.builder.enabled
&& (self.config.agent.allow_local_tools || !self.config.sandbox.enabled)
{
tools
.register_builder_tool(llm.clone(), Some(self.config.builder.to_builder_config()))
.register_builder_tool(
llm.clone(),
safety.clone(),
Some(self.config.builder.to_builder_config()),
)
.await;
tracing::info!("Builder mode enabled");
}
@@ -464,7 +406,6 @@ impl AppBuilder {
) -> Result<
(
Arc<McpSessionManager>,
Arc<McpProcessManager>,
Option<Arc<WasmToolRuntime>>,
Option<Arc<ExtensionManager>>,
Vec<crate::extensions::RegistryEntry>,
@@ -472,13 +413,10 @@ impl AppBuilder {
),
anyhow::Error,
> {
use crate::tools::mcp::{
McpClient, McpTransport, config::load_mcp_servers_from_db, is_authenticated,
};
use crate::tools::mcp::{McpClient, config::load_mcp_servers_from_db, is_authenticated};
use crate::tools::wasm::{WasmToolLoader, load_dev_tools};
let mcp_session_manager = Arc::new(McpSessionManager::new());
let mcp_process_manager = Arc::new(McpProcessManager::new());
// Create WASM tool runtime eagerly so extensions installed after startup
// (e.g. via the web UI) can still be activated. The tools directory is only
@@ -554,175 +492,97 @@ impl AppBuilder {
let db = self.db.clone();
let tools = Arc::clone(tools);
let mcp_sm = Arc::clone(&mcp_session_manager);
let pm = Arc::clone(&mcp_process_manager);
async move {
let servers_result = if let Some(ref d) = db {
load_mcp_servers_from_db(d.as_ref(), "default").await
} else {
crate::tools::mcp::config::load_mcp_servers().await
};
match servers_result {
Ok(servers) => {
let enabled: Vec<_> = servers.enabled_servers().cloned().collect();
if !enabled.is_empty() {
tracing::info!("Loading {} configured MCP server(s)...", enabled.len());
}
if let Some(ref secrets) = secrets_store {
let servers_result = if let Some(ref d) = db {
load_mcp_servers_from_db(d.as_ref(), "default").await
} else {
crate::tools::mcp::config::load_mcp_servers().await
};
match servers_result {
Ok(servers) => {
let enabled: Vec<_> = servers.enabled_servers().cloned().collect();
if !enabled.is_empty() {
tracing::info!(
"Loading {} configured MCP server(s)...",
enabled.len()
);
}
let mut join_set = tokio::task::JoinSet::new();
for server in enabled {
let mcp_sm = Arc::clone(&mcp_sm);
let secrets = secrets_store.clone();
let tools = Arc::clone(&tools);
let pm = Arc::clone(&pm);
let mut join_set = tokio::task::JoinSet::new();
for server in enabled {
let mcp_sm = Arc::clone(&mcp_sm);
let secrets = Arc::clone(secrets);
let tools = Arc::clone(&tools);
join_set.spawn(async move {
let server_name = server.name.clone();
join_set.spawn(async move {
let server_name = server.name.clone();
let has_tokens =
is_authenticated(&server, &secrets, "default").await;
let client: McpClient = match server.effective_transport() {
crate::tools::mcp::config::EffectiveTransport::Stdio {
command,
args,
env,
} => {
match pm
.spawn_stdio(
&server_name,
command,
args.to_vec(),
env.clone(),
)
.await
{
Ok(transport) => McpClient::new_with_transport(
&server_name,
transport as Arc<dyn McpTransport>,
None,
secrets,
"default",
Some(server),
),
Err(e) => {
tracing::warn!(
"Failed to spawn stdio MCP server '{}': {}",
server_name,
e
);
return;
}
}
}
#[cfg(unix)]
crate::tools::mcp::config::EffectiveTransport::Unix {
socket_path,
} => {
match crate::tools::mcp::unix_transport::UnixMcpTransport::connect(
&server_name,
socket_path,
let client = if has_tokens || server.requires_auth() {
McpClient::new_authenticated(
server, mcp_sm, secrets, "default",
)
.await
{
Ok(transport) => McpClient::new_with_transport(
&server_name,
Arc::new(transport) as Arc<dyn McpTransport>,
None,
secrets,
"default",
Some(server),
),
Err(e) => {
tracing::warn!(
"Failed to connect to Unix MCP server '{}': {}",
server_name,
e
);
return;
}
}
}
#[cfg(not(unix))]
crate::tools::mcp::config::EffectiveTransport::Unix { .. } => {
tracing::warn!(
"Unix socket transport is not supported on this platform (server '{}')",
server_name
);
return;
}
crate::tools::mcp::config::EffectiveTransport::Http => {
if let Some(ref secrets) = secrets {
let has_tokens =
is_authenticated(&server, secrets, "default")
.await;
} else {
McpClient::new_with_name(&server_name, &server.url)
};
if has_tokens || server.requires_auth() {
McpClient::new_authenticated(
server,
Arc::clone(&mcp_sm),
Arc::clone(secrets),
"default",
)
} else {
McpClient::new_with_config(server)
}
} else {
McpClient::new_with_config(server)
}
}
};
match client.list_tools().await {
Ok(mcp_tools) => {
let tool_count = mcp_tools.len();
match client.create_tools().await {
Ok(tool_impls) => {
for tool in tool_impls {
tools.register(tool).await;
match client.list_tools().await {
Ok(mcp_tools) => {
let tool_count = mcp_tools.len();
match client.create_tools().await {
Ok(tool_impls) => {
for tool in tool_impls {
tools.register(tool).await;
}
tracing::info!(
"Loaded {} tools from MCP server '{}'",
tool_count,
server_name
);
}
tracing::info!(
"Loaded {} tools from MCP server '{}'",
tool_count,
Err(e) => {
tracing::warn!(
"Failed to create tools from MCP server '{}': {}",
server_name,
e
);
}
}
}
Err(e) => {
let err_str = e.to_string();
if err_str.contains("401")
|| err_str.contains("authentication")
{
tracing::warn!(
"MCP server '{}' requires authentication. \
Run: ironclaw mcp auth {}",
server_name,
server_name
);
}
Err(e) => {
} else {
tracing::warn!(
"Failed to create tools from MCP server '{}': {}",
"Failed to connect to MCP server '{}': {}",
server_name,
e
);
}
}
}
Err(e) => {
let err_str = e.to_string();
if err_str.contains("401")
|| err_str.contains("authentication")
{
tracing::warn!(
"MCP server '{}' requires authentication. \
Run: ironclaw mcp auth {}",
server_name,
server_name
);
} else {
tracing::warn!(
"Failed to connect to MCP server '{}': {}",
server_name,
e
);
}
}
}
});
}
});
}
while let Some(result) = join_set.join_next().await {
if let Err(e) = result {
tracing::warn!("MCP server loading task panicked: {}", e);
while let Some(result) = join_set.join_next().await {
if let Err(e) = result {
tracing::warn!("MCP server loading task panicked: {}", e);
}
}
}
}
Err(e) => {
tracing::debug!("No MCP servers configured ({})", e);
Err(e) => {
tracing::debug!("No MCP servers configured ({})", e);
}
}
}
}
@@ -793,7 +653,6 @@ impl AppBuilder {
Ok((
mcp_session_manager,
mcp_process_manager,
wasm_tool_runtime,
extension_manager,
catalog_entries,
@@ -806,21 +665,10 @@ impl AppBuilder {
self.init_database().await?;
self.init_secrets().await?;
// Post-init validation: if a non-nearai backend was selected but
// credentials were never resolved (deferred resolution found no keys),
// fail early with a clear error instead of a confusing runtime failure.
if self.config.llm.backend != "nearai" && self.config.llm.provider.is_none() {
let backend = &self.config.llm.backend;
anyhow::bail!(
"LLM_BACKEND={backend} is configured but no credentials were found. \
Set the appropriate API key environment variable or run the setup wizard."
);
}
let (llm, cheap_llm, recording_handle) = if let Some(llm) = self.llm_override.take() {
(llm, None, None)
} else {
self.init_llm().await?
self.init_llm()?
};
let (safety, tools, embeddings, workspace) = self.init_tools(&llm).await?;
@@ -829,7 +677,6 @@ impl AppBuilder {
let (
mcp_session_manager,
mcp_process_manager,
wasm_tool_runtime,
extension_manager,
catalog_entries,
@@ -927,7 +774,6 @@ impl AppBuilder {
workspace,
extension_manager,
mcp_session_manager,
mcp_process_manager,
wasm_tool_runtime,
log_broadcaster: self.log_broadcaster,
context_manager,
-251
View File
@@ -414,103 +414,10 @@ pub enum MigrationError {
Io(String),
}
// ── PID Lock ──────────────────────────────────────────────────────────────
/// Path to the PID lock file: `~/.ironclaw/ironclaw.pid`.
pub fn pid_lock_path() -> PathBuf {
ironclaw_base_dir().join("ironclaw.pid")
}
/// A PID-based lock that prevents multiple IronClaw instances from running
/// simultaneously.
///
/// Uses `fs4::try_lock_exclusive()` for atomic locking (no TOCTOU race),
/// then writes the current PID into the locked file for diagnostics.
/// The OS-level lock is held for the lifetime of this struct and
/// automatically released on drop (along with the PID file cleanup).
#[derive(Debug)]
pub struct PidLock {
path: PathBuf,
/// Held open to maintain the OS-level exclusive lock.
_file: std::fs::File,
}
/// Errors from PID lock acquisition.
#[derive(Debug, thiserror::Error)]
pub enum PidLockError {
#[error("Another IronClaw instance is already running (PID {pid})")]
AlreadyRunning { pid: u32 },
#[error("Failed to acquire PID lock: {0}")]
Io(#[from] std::io::Error),
}
impl PidLock {
/// Try to acquire the PID lock.
///
/// Uses an exclusive file lock (`flock`/`LockFileEx`) so that two
/// concurrent processes cannot both acquire the lock — no TOCTOU race.
/// If the lock file exists but the holding process is gone (stale),
/// the lock is reclaimed automatically by the OS.
pub fn acquire() -> Result<Self, PidLockError> {
Self::acquire_at(pid_lock_path())
}
/// Acquire at a specific path (for testing).
fn acquire_at(path: PathBuf) -> Result<Self, PidLockError> {
use fs4::FileExt;
use std::fs::OpenOptions;
use std::io::Write;
// Ensure parent directory exists
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
// Open (or create) the lock file
let mut file = OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(false)
.open(&path)?;
// Try non-blocking exclusive lock — if another process holds it,
// this fails immediately instead of blocking.
if let Err(e) = file.try_lock_exclusive() {
if e.kind() == std::io::ErrorKind::WouldBlock {
// Lock held by another process — read its PID for the error message
let pid = std::fs::read_to_string(&path)
.ok()
.and_then(|s| s.trim().parse::<u32>().ok())
.unwrap_or(0);
return Err(PidLockError::AlreadyRunning { pid });
}
// Other errors (permissions, unsupported filesystem, etc.)
return Err(PidLockError::Io(e));
}
// We hold the exclusive lock — write our PID
file.set_len(0)?; // truncate
write!(file, "{}", std::process::id())?;
Ok(PidLock { path, _file: file })
}
}
impl Drop for PidLock {
fn drop(&mut self) {
// Remove the PID file; the OS-level lock is released when _file is dropped.
let _ = std::fs::remove_file(&self.path);
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::process::Command;
use std::sync::Mutex;
use std::thread;
use std::time::{Duration, Instant};
use tempfile::tempdir;
static ENV_MUTEX: Mutex<()> = Mutex::new(());
@@ -1079,162 +986,4 @@ INJECTED="pwned"#;
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
// ── PID Lock tests ───────────────────────────────────────────────
#[test]
fn test_pid_lock_acquire_and_drop() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// Acquire lock
let lock = PidLock::acquire_at(pid_path.clone()).unwrap();
assert!(pid_path.exists());
// PID file should contain our PID
let contents = std::fs::read_to_string(&pid_path).unwrap();
assert_eq!(contents.trim().parse::<u32>().unwrap(), std::process::id());
// Drop should remove the file
drop(lock);
assert!(!pid_path.exists());
}
#[test]
fn test_pid_lock_rejects_second_acquire() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// First lock succeeds
let _lock1 = PidLock::acquire_at(pid_path.clone()).unwrap();
// Second acquire on same file must fail (exclusive flock held)
let result = PidLock::acquire_at(pid_path.clone());
assert!(result.is_err());
match result.unwrap_err() {
PidLockError::AlreadyRunning { pid } => {
assert_eq!(pid, std::process::id());
}
other => panic!("expected AlreadyRunning, got: {}", other),
}
}
#[test]
fn test_pid_lock_reclaims_after_drop() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// Acquire and release
let lock = PidLock::acquire_at(pid_path.clone()).unwrap();
drop(lock);
// Should succeed — OS lock was released on drop
let lock2 = PidLock::acquire_at(pid_path).unwrap();
drop(lock2);
}
#[test]
fn test_pid_lock_reclaims_stale_file_without_flock() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// Write a stale PID file manually (no flock held)
std::fs::write(&pid_path, "4294967294").unwrap();
// Should succeed because no OS lock is held on the file
let lock = PidLock::acquire_at(pid_path.clone()).unwrap();
let contents = std::fs::read_to_string(&pid_path).unwrap();
assert_eq!(contents.trim().parse::<u32>().unwrap(), std::process::id());
drop(lock);
}
#[test]
fn test_pid_lock_handles_corrupt_pid_file() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// Write garbage (no flock held)
std::fs::write(&pid_path, "not-a-number").unwrap();
// Should succeed — no OS lock held, file is reclaimed
let lock = PidLock::acquire_at(pid_path).unwrap();
drop(lock);
}
#[test]
fn test_pid_lock_creates_parent_dirs() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("nested").join("deep").join("ironclaw.pid");
let lock = PidLock::acquire_at(pid_path.clone()).unwrap();
assert!(pid_path.exists());
drop(lock);
}
#[test]
fn test_pid_lock_child_helper_holds_lock() {
if std::env::var("IRONCLAW_PID_LOCK_CHILD").ok().as_deref() != Some("1") {
return;
}
let pid_path = PathBuf::from(
std::env::var("IRONCLAW_PID_LOCK_PATH").expect("IRONCLAW_PID_LOCK_PATH missing"),
);
let hold_ms = std::env::var("IRONCLAW_PID_LOCK_HOLD_MS")
.ok()
.and_then(|s| s.parse::<u64>().ok())
.unwrap_or(3000);
let _lock = PidLock::acquire_at(pid_path).expect("child failed to acquire pid lock");
thread::sleep(Duration::from_millis(hold_ms));
}
#[test]
fn test_pid_lock_rejects_lock_held_by_other_process() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
let current_exe = std::env::current_exe().unwrap();
let mut child = Command::new(current_exe)
.args([
"--exact",
"bootstrap::tests::test_pid_lock_child_helper_holds_lock",
"--nocapture",
"--test-threads=1",
])
.env("IRONCLAW_PID_LOCK_CHILD", "1")
.env("IRONCLAW_PID_LOCK_PATH", pid_path.display().to_string())
.env("IRONCLAW_PID_LOCK_HOLD_MS", "3000")
.spawn()
.unwrap();
let started = Instant::now();
while started.elapsed() < Duration::from_secs(2) {
if pid_path.exists() {
break;
}
if let Some(status) = child.try_wait().unwrap() {
panic!("child exited before acquiring lock: {}", status);
}
thread::sleep(Duration::from_millis(20));
}
assert!(
pid_path.exists(),
"child did not create lock file in time: {}",
pid_path.display()
);
let result = PidLock::acquire_at(pid_path.clone());
match result.unwrap_err() {
PidLockError::AlreadyRunning { .. } => {}
other => panic!("expected AlreadyRunning, got: {}", other),
}
let status = child.wait().unwrap();
assert!(status.success(), "child process failed: {}", status);
// After the child exits, lock should be released and reacquirable.
let lock = PidLock::acquire_at(pid_path).unwrap();
drop(lock);
}
}
-22
View File
@@ -79,8 +79,6 @@ pub struct IncomingMessage {
pub received_at: DateTime<Utc>,
/// Channel-specific metadata.
pub metadata: serde_json::Value,
/// IANA timezone string from the client (e.g. "America/New_York").
pub timezone: Option<String>,
/// File or media attachments on this message.
pub attachments: Vec<IncomingAttachment>,
}
@@ -101,7 +99,6 @@ impl IncomingMessage {
thread_id: None,
received_at: Utc::now(),
metadata: serde_json::Value::Null,
timezone: None,
attachments: Vec::new(),
}
}
@@ -124,12 +121,6 @@ impl IncomingMessage {
self
}
/// Set the client timezone.
pub fn with_timezone(mut self, tz: impl Into<String>) -> Self {
self.timezone = Some(tz.into());
self
}
/// Set attachments.
pub fn with_attachments(mut self, attachments: Vec<IncomingAttachment>) -> Self {
self.attachments = attachments;
@@ -231,13 +222,6 @@ pub enum StatusUpdate {
success: bool,
message: String,
},
/// An image was generated by a tool.
ImageGenerated {
/// Base64 data URL of the generated image.
data_url: String,
/// Optional workspace path where the image was saved.
path: Option<String>,
},
}
impl StatusUpdate {
@@ -470,10 +454,4 @@ mod tests {
panic!("expected ToolCompleted variant");
}
}
#[test]
fn test_incoming_message_with_timezone() {
let msg = IncomingMessage::new("test", "user1", "hello").with_timezone("America/New_York");
assert_eq!(msg.timezone.as_deref(), Some("America/New_York"));
}
}
+6 -126
View File
@@ -17,9 +17,7 @@ use tokio::sync::{RwLock, mpsc, oneshot};
use tokio_stream::wrappers::ReceiverStream;
use uuid::Uuid;
use crate::channels::{
AttachmentKind, Channel, IncomingAttachment, IncomingMessage, MessageStream, OutgoingResponse,
};
use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse};
use crate::config::HttpConfig;
use crate::error::ChannelError;
@@ -48,9 +46,8 @@ struct RateLimitState {
request_count: u32,
}
/// Maximum JSON body size for webhook requests (15 MB, to support base64 image attachments
/// with ~33% overhead from base64 encoding).
const MAX_BODY_BYTES: usize = 15 * 1024 * 1024;
/// Maximum JSON body size for webhook requests (64 KB).
const MAX_BODY_BYTES: usize = 64 * 1024;
/// Maximum number of pending wait-for-response requests.
const MAX_PENDING_RESPONSES: usize = 100;
@@ -118,34 +115,8 @@ struct WebhookRequest {
/// Whether to wait for a synchronous response.
#[serde(default)]
wait_for_response: bool,
/// Optional file attachments (base64-encoded).
#[serde(default)]
attachments: Vec<AttachmentData>,
}
/// A file attachment in a webhook request.
#[derive(Debug, Deserialize)]
struct AttachmentData {
/// MIME type (e.g. "image/png", "application/pdf").
mime_type: String,
/// Optional filename.
#[serde(default)]
filename: Option<String>,
/// Base64-encoded file data.
#[serde(default)]
data_base64: Option<String>,
/// URL to fetch the file from (not downloaded server-side for SSRF prevention).
#[serde(default)]
url: Option<String>,
}
/// Maximum size per attachment (5 MB decoded).
const MAX_ATTACHMENT_BYTES: usize = 5 * 1024 * 1024;
/// Maximum total attachment size (10 MB decoded).
const MAX_TOTAL_ATTACHMENT_BYTES: usize = 10 * 1024 * 1024;
/// Maximum number of attachments per request.
const MAX_ATTACHMENTS: usize = 5;
#[derive(Debug, Serialize)]
struct WebhookResponse {
/// Message ID assigned to this request.
@@ -240,106 +211,15 @@ async fn webhook_handler(
);
}
// Validate and decode attachments
let attachments = if !req.attachments.is_empty() {
if req.attachments.len() > MAX_ATTACHMENTS {
return (
StatusCode::BAD_REQUEST,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some(format!("Too many attachments (max {})", MAX_ATTACHMENTS)),
}),
);
}
let mut decoded_attachments = Vec::new();
let mut total_bytes: usize = 0;
for att in &req.attachments {
if let Some(ref b64) = att.data_base64 {
use base64::Engine;
let data = match base64::engine::general_purpose::STANDARD.decode(b64) {
Ok(d) => d,
Err(_) => {
return (
StatusCode::BAD_REQUEST,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Invalid base64 in attachment".to_string()),
}),
);
}
};
if data.len() > MAX_ATTACHMENT_BYTES {
return (
StatusCode::PAYLOAD_TOO_LARGE,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some(format!(
"Attachment too large (max {} bytes)",
MAX_ATTACHMENT_BYTES
)),
}),
);
}
total_bytes += data.len();
if total_bytes > MAX_TOTAL_ATTACHMENT_BYTES {
return (
StatusCode::PAYLOAD_TOO_LARGE,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Total attachment size exceeds limit".to_string()),
}),
);
}
decoded_attachments.push(IncomingAttachment {
id: Uuid::new_v4().to_string(),
kind: AttachmentKind::from_mime_type(&att.mime_type),
mime_type: att.mime_type.clone(),
filename: att.filename.clone(),
size_bytes: Some(data.len() as u64),
source_url: None,
storage_key: None,
extracted_text: None,
data,
duration_secs: None,
});
} else if let Some(ref url) = att.url {
// URL-only attachment: set source_url but don't download (SSRF prevention)
decoded_attachments.push(IncomingAttachment {
id: Uuid::new_v4().to_string(),
kind: AttachmentKind::from_mime_type(&att.mime_type),
mime_type: att.mime_type.clone(),
filename: att.filename.clone(),
size_bytes: None,
source_url: Some(url.clone()),
storage_key: None,
extracted_text: None,
data: Vec::new(),
duration_secs: None,
});
}
}
decoded_attachments
} else {
Vec::new()
};
let mut msg = IncomingMessage::new("http", &state.user_id, &req.content).with_metadata(
let msg = IncomingMessage::new("http", &state.user_id, &req.content).with_metadata(
serde_json::json!({
"wait_for_response": req.wait_for_response,
}),
);
if !attachments.is_empty() {
msg = msg.with_attachments(attachments);
}
if let Some(thread_id) = &req.thread_id {
msg = msg.with_thread(thread_id);
let msg = msg.with_thread(thread_id);
return process_message(state, msg, req.wait_for_response).await;
}
process_message(state, msg, req.wait_for_response).await
+9 -57
View File
@@ -18,7 +18,7 @@
//! - `Esc` - Interrupt current operation
use std::borrow::Cow;
use std::io::{self, IsTerminal, Write};
use std::io::{self, Write};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
@@ -297,15 +297,10 @@ impl Channel for ReplChannel {
let esc_interrupt_triggered_for_thread = Arc::new(AtomicBool::new(false));
std::thread::spawn(move || {
let sys_tz = crate::timezone::detect_system_timezone().name().to_string();
// Single message mode: send it and return
if let Some(msg) = single_message {
let incoming = IncomingMessage::new("repl", "default", &msg).with_timezone(&sys_tz);
let incoming = IncomingMessage::new("repl", "default", &msg);
let _ = tx.blocking_send(incoming);
// Ensure the agent exits after handling exactly one turn in -m mode,
// even when other channels (gateway/http) are enabled.
let _ = tx.blocking_send(IncomingMessage::new("repl", "default", "/quit"));
return;
}
@@ -366,8 +361,7 @@ impl Channel for ReplChannel {
"/quit" | "/exit" => {
// Forward shutdown command so the agent loop exits even
// when other channels (e.g. web gateway) are still active.
let msg = IncomingMessage::new("repl", "default", "/quit")
.with_timezone(&sys_tz);
let msg = IncomingMessage::new("repl", "default", "/quit");
let _ = tx.blocking_send(msg);
break;
}
@@ -388,8 +382,7 @@ impl Channel for ReplChannel {
_ => {}
}
let msg =
IncomingMessage::new("repl", "default", line).with_timezone(&sys_tz);
let msg = IncomingMessage::new("repl", "default", line);
if tx.blocking_send(msg).is_err() {
break;
}
@@ -397,29 +390,21 @@ impl Channel for ReplChannel {
Err(ReadlineError::Interrupted) => {
if esc_interrupt_triggered_for_thread.swap(false, Ordering::Relaxed) {
// Esc: interrupt current operation and keep REPL open.
let msg = IncomingMessage::new("repl", "default", "/interrupt")
.with_timezone(&sys_tz);
let msg = IncomingMessage::new("repl", "default", "/interrupt");
if tx.blocking_send(msg).is_err() {
break;
}
} else {
// Ctrl+C (VINTR): request graceful shutdown.
let msg = IncomingMessage::new("repl", "default", "/quit")
.with_timezone(&sys_tz);
let msg = IncomingMessage::new("repl", "default", "/quit");
let _ = tx.blocking_send(msg);
break;
}
}
Err(ReadlineError::Eof) => {
// Ctrl+D in interactive mode: graceful shutdown.
// In daemon mode (stdin = /dev/null, no TTY), EOF arrives
// immediately — just drop the REPL thread silently so other
// channels (gateway, telegram, …) keep running.
if std::io::stdin().is_terminal() {
let msg = IncomingMessage::new("repl", "default", "/quit")
.with_timezone(&sys_tz);
let _ = tx.blocking_send(msg);
}
// Ctrl+D: send /quit so the agent loop runs graceful shutdown
let msg = IncomingMessage::new("repl", "default", "/quit");
let _ = tx.blocking_send(msg);
break;
}
Err(e) => {
@@ -600,13 +585,6 @@ impl Channel for ReplChannel {
eprintln!("\x1b[31m {extension_name}: {message}\x1b[0m");
}
}
StatusUpdate::ImageGenerated { path, .. } => {
if let Some(ref p) = path {
eprintln!("\x1b[36m [image] {p}\x1b[0m");
} else {
eprintln!("\x1b[36m [image generated]\x1b[0m");
}
}
}
Ok(())
}
@@ -636,29 +614,3 @@ impl Channel for ReplChannel {
Ok(())
}
}
#[cfg(test)]
mod tests {
use futures::StreamExt;
use super::*;
#[tokio::test]
async fn single_message_mode_sends_message_then_quit() {
let repl = ReplChannel::with_message("hi".to_string());
let mut stream = repl.start().await.expect("repl start should succeed");
let first = stream.next().await.expect("first message missing");
assert_eq!(first.channel, "repl");
assert_eq!(first.content, "hi");
let second = stream.next().await.expect("quit message missing");
assert_eq!(second.channel, "repl");
assert_eq!(second.content, "/quit");
assert!(
stream.next().await.is_none(),
"stream should end after /quit"
);
}
}
-8
View File
@@ -2809,14 +2809,6 @@ fn status_to_wit(status: &StatusUpdate, metadata: &serde_json::Value) -> wit_cha
),
metadata_json,
},
StatusUpdate::ImageGenerated { path, .. } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::Status,
message: match path {
Some(p) => format!("[image] {}", p),
None => "[image generated]".to_string(),
},
metadata_json,
},
}
}
+33 -38
View File
@@ -426,7 +426,7 @@ pub async fn chat_threads_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if let Ok(summaries) = store
.list_conversations_all_channels(&state.user_id, 50)
.list_conversations_with_preview(&state.user_id, "gateway", 50)
.await
{
let mut assistant_thread = None;
@@ -441,7 +441,6 @@ pub async fn chat_threads_handler(
updated_at: s.last_activity.to_rfc3339(),
title: s.title.clone(),
thread_type: s.thread_type.clone(),
channel: Some(s.channel.clone()),
};
if s.id == assistant_id {
@@ -461,7 +460,6 @@ pub async fn chat_threads_handler(
updated_at: chrono::Utc::now().to_rfc3339(),
title: None,
thread_type: Some("assistant".to_string()),
channel: Some("gateway".to_string()),
});
}
@@ -474,10 +472,9 @@ pub async fn chat_threads_handler(
}
// Fallback: in-memory only (no assistant thread without DB)
let mut sorted_threads: Vec<_> = sess.threads.values().collect();
sorted_threads.sort_by(|a, b| b.updated_at.cmp(&a.updated_at));
let threads: Vec<ThreadInfo> = sorted_threads
.into_iter()
let threads: Vec<ThreadInfo> = sess
.threads
.values()
.map(|t| ThreadInfo {
id: t.id,
state: format!("{:?}", t.state),
@@ -486,7 +483,6 @@ pub async fn chat_threads_handler(
updated_at: t.updated_at.to_rfc3339(),
title: None,
thread_type: None,
channel: Some("gateway".to_string()),
})
.collect();
@@ -506,39 +502,38 @@ pub async fn chat_new_thread_handler(
))?;
let session = session_manager.get_or_create_session(&state.user_id).await;
let (thread_id, info) = {
let mut sess = session.lock().await;
let thread = sess.create_thread();
let id = thread.id;
let info = ThreadInfo {
id: thread.id,
state: format!("{:?}", thread.state),
turn_count: thread.turns.len(),
created_at: thread.created_at.to_rfc3339(),
updated_at: thread.updated_at.to_rfc3339(),
title: None,
thread_type: Some("thread".to_string()),
channel: Some("gateway".to_string()),
};
(id, info)
let mut sess = session.lock().await;
let thread = sess.create_thread();
let thread_id = thread.id;
let info = ThreadInfo {
id: thread.id,
state: format!("{:?}", thread.state),
turn_count: thread.turns.len(),
created_at: thread.created_at.to_rfc3339(),
updated_at: thread.updated_at.to_rfc3339(),
title: None,
thread_type: Some("thread".to_string()),
};
// Persist the empty conversation row with thread_type metadata synchronously
// so that the subsequent loadThreads() call from the frontend sees it.
// Persist the empty conversation row with thread_type metadata
if let Some(ref store) = state.store {
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &state.user_id, None)
.await
{
tracing::warn!("Failed to persist new thread: {}", e);
}
let metadata_val = serde_json::json!("thread");
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "thread_type", &metadata_val)
.await
{
tracing::warn!("Failed to set thread_type metadata: {}", e);
}
let store = Arc::clone(store);
let user_id = state.user_id.clone();
tokio::spawn(async move {
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &user_id, None)
.await
{
tracing::warn!("Failed to persist new thread: {}", e);
}
let metadata_val = serde_json::json!("thread");
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "thread_type", &metadata_val)
.await
{
tracing::warn!("Failed to set thread_type metadata: {}", e);
}
});
}
Ok(Json(info))
+43 -24
View File
@@ -10,9 +10,9 @@ use axum::{
use serde::Deserialize;
use uuid::Uuid;
use crate::channels::IncomingMessage;
use crate::channels::web::server::GatewayState;
use crate::channels::web::types::*;
use crate::error::RoutineError;
pub async fn routines_list_handler(
State(state): State<Arc<GatewayState>>,
@@ -133,27 +133,56 @@ pub async fn routines_trigger_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
// Clone the Arc out of the lock to avoid holding the RwLock across .await.
let engine = {
let guard = state.routine_engine.read().await;
guard.as_ref().cloned().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Routine engine not available".to_string(),
))?
};
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let routine_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid routine ID".to_string()))?;
let run_id = engine
.fire_manual(routine_id, Some(&state.user_id))
let routine = store
.get_routine(routine_id)
.await
.map_err(|e| (routine_error_status(&e), e.to_string()))?;
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Routine not found".to_string()))?;
if routine.user_id != state.user_id {
return Err((StatusCode::FORBIDDEN, "Access denied".to_string()));
}
// Send the routine prompt through the message pipeline as a manual trigger.
let prompt = match &routine.action {
crate::agent::routine::RoutineAction::Lightweight { prompt, .. } => prompt.clone(),
crate::agent::routine::RoutineAction::FullJob {
title, description, ..
} => format!("{}: {}", title, description),
};
let content = format!("[routine:{}] {}", routine.name, prompt);
let thread_id = format!(
"routine-{}-{}",
routine_id,
chrono::Utc::now().timestamp_millis()
);
let msg = IncomingMessage::new("gateway", &state.user_id, content).with_thread(thread_id);
let tx_guard = state.msg_tx.read().await;
let tx = tx_guard.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Channel not started".to_string(),
))?;
tx.send(msg).await.map_err(|_| {
(
StatusCode::INTERNAL_SERVER_ERROR,
"Channel closed".to_string(),
)
})?;
Ok(Json(serde_json::json!({
"status": "triggered",
"routine_id": routine_id,
"run_id": run_id,
})))
}
@@ -264,7 +293,7 @@ pub async fn routines_runs_handler(
/// Convert a Routine to the trimmed RoutineInfo for list display.
fn routine_to_info(r: &crate::agent::routine::Routine) -> RoutineInfo {
let (trigger_type, trigger_summary) = match &r.trigger {
crate::agent::routine::Trigger::Cron { schedule, .. } => {
crate::agent::routine::Trigger::Cron { schedule } => {
("cron".to_string(), format!("cron: {}", schedule))
}
crate::agent::routine::Trigger::Event {
@@ -308,13 +337,3 @@ fn routine_to_info(r: &crate::agent::routine::Routine) -> RoutineInfo {
status: status.to_string(),
}
}
/// Map `RoutineError` variants to appropriate HTTP status codes.
fn routine_error_status(err: &RoutineError) -> StatusCode {
match err {
RoutineError::NotFound { .. } => StatusCode::NOT_FOUND,
RoutineError::NotAuthorized { .. } => StatusCode::FORBIDDEN,
RoutineError::Disabled { .. } | RoutineError::MaxConcurrent { .. } => StatusCode::CONFLICT,
_ => StatusCode::INTERNAL_SERVER_ERROR,
}
}
+2 -26
View File
@@ -99,7 +99,6 @@ impl GatewayChannel {
chat_rate_limiter: server::RateLimiter::new(30, 60),
registry_entries: Vec::new(),
cost_guard: None,
routine_engine: Arc::new(tokio::sync::RwLock::new(None)),
startup_time: std::time::Instant::now(),
});
@@ -135,7 +134,6 @@ impl GatewayChannel {
chat_rate_limiter: server::RateLimiter::new(30, 60),
registry_entries: self.state.registry_entries.clone(),
cost_guard: self.state.cost_guard.clone(),
routine_engine: Arc::clone(&self.state.routine_engine),
startup_time: self.state.startup_time,
};
mutate(&mut new_state);
@@ -283,15 +281,7 @@ impl Channel for GatewayChannel {
msg: &IncomingMessage,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let thread_id = match &msg.thread_id {
Some(tid) => tid.clone(),
None => {
tracing::warn!(
"Gateway respond with no thread_id — skipping (clients would drop it)"
);
return Ok(());
}
};
let thread_id = msg.thread_id.clone().unwrap_or_default();
self.state.sse.broadcast(SseEvent::Response {
content: response.content,
@@ -386,11 +376,6 @@ impl Channel for GatewayChannel {
success,
message,
},
StatusUpdate::ImageGenerated { data_url, path } => SseEvent::ImageGenerated {
data_url,
path,
thread_id,
},
};
self.state.sse.broadcast(event);
@@ -402,18 +387,9 @@ impl Channel for GatewayChannel {
_user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let thread_id = match response.thread_id {
Some(tid) => tid,
None => {
tracing::warn!(
"Gateway broadcast with no thread_id — skipping (clients would drop it)"
);
return Ok(());
}
};
self.state.sse.broadcast(SseEvent::Response {
content: response.content,
thread_id,
thread_id: String::new(),
});
Ok(())
}
+71 -141
View File
@@ -57,10 +57,6 @@ pub type PromptQueue = Arc<
>,
>;
/// Slot for the routine engine, filled at runtime after the agent starts.
pub type RoutineEngineSlot =
Arc<tokio::sync::RwLock<Option<Arc<crate::agent::routine_engine::RoutineEngine>>>>;
/// Simple sliding-window rate limiter.
///
/// Tracks the number of requests in the current window. Resets when the window expires.
@@ -169,8 +165,6 @@ pub struct GatewayState {
pub registry_entries: Vec<crate::extensions::RegistryEntry>,
/// Cost guard for token/cost tracking.
pub cost_guard: Option<Arc<crate::agent::cost_guard::CostGuard>>,
/// Routine engine slot for manual routine triggering (filled at runtime).
pub routine_engine: RoutineEngineSlot,
/// Server startup time for uptime calculation.
pub startup_time: std::time::Instant,
}
@@ -351,7 +345,7 @@ pub async fn start_server(
.merge(statics)
.merge(projects)
.merge(protected)
.layer(DefaultBodyLimit::max(10 * 1024 * 1024)) // 10 MB max request body (image uploads)
.layer(DefaultBodyLimit::max(1024 * 1024)) // 1 MB max request body
.layer(cors)
.layer(SetResponseHeaderLayer::if_not_present(
header::X_CONTENT_TYPE_OPTIONS,
@@ -608,59 +602,8 @@ async fn oauth_callback_handler(
// --- Chat handlers ---
/// Convert web gateway `ImageData` to `IncomingAttachment` objects.
pub(crate) fn images_to_attachments(
images: &[ImageData],
) -> Vec<crate::channels::IncomingAttachment> {
use base64::Engine;
images
.iter()
.enumerate()
.filter_map(|(i, img)| {
if !img.media_type.starts_with("image/") {
tracing::warn!(
"Skipping image {i}: invalid media type '{}' (must start with 'image/')",
img.media_type
);
return None;
}
let data = match base64::engine::general_purpose::STANDARD.decode(&img.data) {
Ok(d) => d,
Err(e) => {
tracing::warn!("Skipping image {i}: invalid base64 data: {e}");
return None;
}
};
Some(crate::channels::IncomingAttachment {
id: format!("web-image-{i}"),
kind: crate::channels::AttachmentKind::Image,
mime_type: img.media_type.clone(),
filename: Some(format!("image-{i}.{}", mime_to_ext(&img.media_type))),
size_bytes: Some(data.len() as u64),
source_url: None,
storage_key: None,
extracted_text: None,
data,
duration_secs: None,
})
})
.collect()
}
/// Map MIME type to file extension.
fn mime_to_ext(mime: &str) -> &str {
match mime {
"image/png" => "png",
"image/gif" => "gif",
"image/webp" => "webp",
"image/svg+xml" => "svg",
_ => "jpg",
}
}
async fn chat_send_handler(
State(state): State<Arc<GatewayState>>,
headers: axum::http::HeaderMap,
Json(req): Json<SendMessageRequest>,
) -> Result<(StatusCode, Json<SendMessageResponse>), (StatusCode, String)> {
tracing::debug!(
@@ -677,32 +620,17 @@ async fn chat_send_handler(
}
let mut msg = IncomingMessage::new("gateway", &state.user_id, &req.content);
// Prefer timezone from JSON body, fall back to X-Timezone header
let tz = req
.timezone
.as_deref()
.or_else(|| headers.get("X-Timezone").and_then(|v| v.to_str().ok()));
if let Some(tz) = tz {
msg = msg.with_timezone(tz);
}
if let Some(ref thread_id) = req.thread_id {
msg = msg.with_thread(thread_id);
msg = msg.with_metadata(serde_json::json!({"thread_id": thread_id}));
}
// Convert uploaded images to IncomingAttachments
if !req.images.is_empty() {
let attachments = images_to_attachments(&req.images);
msg = msg.with_attachments(attachments);
}
let msg_id = msg.id;
tracing::debug!(
"[chat_send_handler] Created message id={}, content={:?}, images={}",
"[chat_send_handler] Created message id={}, content={:?}",
msg_id,
req.content,
req.images.len()
req.content
);
let tx_guard = state.msg_tx.read().await;
@@ -1109,7 +1037,7 @@ async fn chat_threads_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if let Ok(summaries) = store
.list_conversations_all_channels(&state.user_id, 50)
.list_conversations_with_preview(&state.user_id, "gateway", 50)
.await
{
let mut assistant_thread = None;
@@ -1124,7 +1052,6 @@ async fn chat_threads_handler(
updated_at: s.last_activity.to_rfc3339(),
title: s.title.clone(),
thread_type: s.thread_type.clone(),
channel: Some(s.channel.clone()),
};
if s.id == assistant_id {
@@ -1144,7 +1071,6 @@ async fn chat_threads_handler(
updated_at: chrono::Utc::now().to_rfc3339(),
title: None,
thread_type: Some("assistant".to_string()),
channel: Some("gateway".to_string()),
});
}
@@ -1157,10 +1083,9 @@ async fn chat_threads_handler(
}
// Fallback: in-memory only (no assistant thread without DB)
let mut sorted_threads: Vec<_> = sess.threads.values().collect();
sorted_threads.sort_by(|a, b| b.updated_at.cmp(&a.updated_at));
let threads: Vec<ThreadInfo> = sorted_threads
.into_iter()
let threads: Vec<ThreadInfo> = sess
.threads
.values()
.map(|t| ThreadInfo {
id: t.id,
state: format!("{:?}", t.state),
@@ -1169,7 +1094,6 @@ async fn chat_threads_handler(
updated_at: t.updated_at.to_rfc3339(),
title: None,
thread_type: None,
channel: Some("gateway".to_string()),
})
.collect();
@@ -1189,39 +1113,38 @@ async fn chat_new_thread_handler(
))?;
let session = session_manager.get_or_create_session(&state.user_id).await;
let (thread_id, info) = {
let mut sess = session.lock().await;
let thread = sess.create_thread();
let id = thread.id;
let info = ThreadInfo {
id: thread.id,
state: format!("{:?}", thread.state),
turn_count: thread.turns.len(),
created_at: thread.created_at.to_rfc3339(),
updated_at: thread.updated_at.to_rfc3339(),
title: None,
thread_type: Some("thread".to_string()),
channel: Some("gateway".to_string()),
};
(id, info)
let mut sess = session.lock().await;
let thread = sess.create_thread();
let thread_id = thread.id;
let info = ThreadInfo {
id: thread.id,
state: format!("{:?}", thread.state),
turn_count: thread.turns.len(),
created_at: thread.created_at.to_rfc3339(),
updated_at: thread.updated_at.to_rfc3339(),
title: None,
thread_type: Some("thread".to_string()),
};
// Persist the empty conversation row with thread_type metadata synchronously
// so that the subsequent loadThreads() call from the frontend sees it.
// Persist the empty conversation row with thread_type metadata
if let Some(ref store) = state.store {
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &state.user_id, None)
.await
{
tracing::warn!("Failed to persist new thread: {}", e);
}
let metadata_val = serde_json::json!("thread");
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "thread_type", &metadata_val)
.await
{
tracing::warn!("Failed to set thread_type metadata: {}", e);
}
let store = Arc::clone(store);
let user_id = state.user_id.clone();
tokio::spawn(async move {
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &user_id, None)
.await
{
tracing::warn!("Failed to persist new thread: {}", e);
}
let metadata_val = serde_json::json!("thread");
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "thread_type", &metadata_val)
.await
{
tracing::warn!("Failed to set thread_type metadata: {}", e);
}
});
}
Ok(Json(info))
@@ -2042,35 +1965,47 @@ async fn routines_trigger_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let engine = {
let guard = state.routine_engine.read().await;
guard.as_ref().cloned().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Routine engine not available".to_string(),
))?
};
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let routine_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid routine ID".to_string()))?;
let run_id = engine
.fire_manual(routine_id, Some(&state.user_id))
let routine = store
.get_routine(routine_id)
.await
.map_err(|e| {
let status = match &e {
crate::error::RoutineError::NotFound { .. } => StatusCode::NOT_FOUND,
crate::error::RoutineError::NotAuthorized { .. } => StatusCode::FORBIDDEN,
crate::error::RoutineError::Disabled { .. }
| crate::error::RoutineError::MaxConcurrent { .. } => StatusCode::CONFLICT,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
(status, e.to_string())
})?;
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Routine not found".to_string()))?;
// Send the routine prompt through the message pipeline as a manual trigger.
let prompt = match &routine.action {
crate::agent::routine::RoutineAction::Lightweight { prompt, .. } => prompt.clone(),
crate::agent::routine::RoutineAction::FullJob {
title, description, ..
} => format!("{}: {}", title, description),
};
let content = format!("[routine:{}] {}", routine.name, prompt);
let msg = IncomingMessage::new("gateway", &state.user_id, content);
let tx_guard = state.msg_tx.read().await;
let tx = tx_guard.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Channel not started".to_string(),
))?;
tx.send(msg).await.map_err(|_| {
(
StatusCode::INTERNAL_SERVER_ERROR,
"Channel closed".to_string(),
)
})?;
Ok(Json(serde_json::json!({
"status": "triggered",
"routine_id": routine_id,
"run_id": run_id,
})))
}
@@ -2181,7 +2116,7 @@ async fn routines_runs_handler(
/// Convert a Routine to the trimmed RoutineInfo for list display.
fn routine_to_info(r: &crate::agent::routine::Routine) -> RoutineInfo {
let (trigger_type, trigger_summary) = match &r.trigger {
crate::agent::routine::Trigger::Cron { schedule, .. } => {
crate::agent::routine::Trigger::Cron { schedule } => {
("cron".to_string(), format!("cron: {}", schedule))
}
crate::agent::routine::Trigger::Event {
@@ -2528,7 +2463,6 @@ mod tests {
chat_rate_limiter: RateLimiter::new(30, 60),
registry_entries: vec![],
cost_guard: None,
routine_engine: Arc::new(tokio::sync::RwLock::new(None)),
startup_time: std::time::Instant::now(),
})
}
@@ -2686,9 +2620,7 @@ mod tests {
secrets,
sse_sender: None,
gateway_token: None,
created_at: std::time::Instant::now()
.checked_sub(std::time::Duration::from_secs(600))
.expect("System uptime is too low to run expired flow test"),
created_at: std::time::Instant::now() - std::time::Duration::from_secs(600),
};
ext_mgr
@@ -2795,9 +2727,7 @@ mod tests {
sse_sender: None,
gateway_token: None,
// Expired — handler will reject after lookup (no network I/O)
created_at: std::time::Instant::now()
.checked_sub(std::time::Duration::from_secs(600))
.expect("System uptime is too low to run expired flow test"),
created_at: std::time::Instant::now() - std::time::Duration::from_secs(600),
};
ext_mgr
-1
View File
@@ -142,7 +142,6 @@ impl SseManager {
SseEvent::JobStatus { .. } => "job_status",
SseEvent::JobResult { .. } => "job_result",
SseEvent::Heartbeat => "heartbeat",
SseEvent::ImageGenerated { .. } => "image_generated",
SseEvent::ExtensionStatus { .. } => "extension_status",
};
Ok(Event::default().event(event_type).data(data))
+49 -270
View File
@@ -5,7 +5,6 @@ let eventSource = null;
let logEventSource = null;
let currentTab = 'chat';
let currentThreadId = null;
let currentThreadIsReadOnly = false;
let assistantThreadId = null;
let hasMore = false;
let oldestTimestamp = null;
@@ -14,11 +13,8 @@ let sseHasConnectedBefore = false;
let jobEvents = new Map(); // job_id -> Array of events
let jobListRefreshTimer = null;
let pairingPollInterval = null;
let unreadThreads = new Map(); // thread_id -> unread count
let _loadThreadsTimer = null;
const JOB_EVENTS_CAP = 500;
const MEMORY_SEARCH_QUERY_MAX_LENGTH = 100;
let stagedImages = [];
// --- Slash Commands ---
@@ -182,7 +178,6 @@ function confirmRestart() {
body: {
content: '/restart',
thread_id: currentThreadId,
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
},
})
.then((response) => {
@@ -225,6 +220,23 @@ function updateRestartButtonVisibility() {
}
}
function startGatewayStatusPolling() {
fetchGatewayStatus();
// Poll every 5 seconds
setInterval(fetchGatewayStatus, 5000);
}
function fetchGatewayStatus() {
apiFetch('/api/gateway/status')
.then((data) => {
restartEnabled = data.restart_enabled || false;
updateRestartButtonVisibility();
})
.catch((err) => {
console.warn('[gateway status] Failed to fetch:', err);
});
}
// --- SSE ---
function connectSSE() {
@@ -261,13 +273,7 @@ function connectSSE() {
eventSource.addEventListener('response', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) {
if (data.thread_id) {
unreadThreads.set(data.thread_id, (unreadThreads.get(data.thread_id) || 0) + 1);
debouncedLoadThreads();
}
return;
}
if (!isCurrentThread(data.thread_id)) return;
finalizeActivityGroup();
addMessage('assistant', data.content);
enableChatInput();
@@ -282,10 +288,7 @@ function connectSSE() {
eventSource.addEventListener('thinking', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) {
if (data.thread_id) debouncedLoadThreads();
return;
}
if (!isCurrentThread(data.thread_id)) return;
showActivityThinking(data.message);
});
@@ -321,10 +324,7 @@ function connectSSE() {
eventSource.addEventListener('status', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) {
if (data.thread_id) debouncedLoadThreads();
return;
}
if (!isCurrentThread(data.thread_id)) return;
// "Done" and "Awaiting approval" are terminal signals from the agent:
// the agentic loop finished, so re-enable input as a safety net in case
// the response SSE event is empty or lost.
@@ -373,12 +373,6 @@ function connectSSE() {
if (currentTab === 'extensions') loadExtensions();
});
eventSource.addEventListener('image_generated', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return;
addGeneratedImage(data.data_url, data.path);
});
eventSource.addEventListener('error', (e) => {
if (e.data) {
const data = JSON.parse(e.data);
@@ -420,9 +414,9 @@ function connectSSE() {
}
// Check if an SSE event belongs to the currently viewed thread.
// Events without a thread_id are dropped (prevents notification leaking).
// Events without a thread_id (legacy) are always shown.
function isCurrentThread(threadId) {
if (!threadId) return false;
if (!threadId) return true;
if (!currentThreadId) return true;
return threadId === currentThreadId;
}
@@ -436,135 +430,23 @@ function sendMessage() {
return;
}
const content = input.value.trim();
if (!content && stagedImages.length === 0) return;
if (!content) return;
addMessage('user', content || '(images attached)');
addMessage('user', content);
input.value = '';
autoResizeTextarea(input);
input.focus();
const body = { content, thread_id: currentThreadId || undefined, timezone: Intl.DateTimeFormat().resolvedOptions().timeZone };
if (stagedImages.length > 0) {
body.images = stagedImages.map(img => ({ media_type: img.media_type, data: img.data }));
stagedImages = [];
renderImagePreviews();
}
apiFetch('/api/chat/send', {
method: 'POST',
body: body,
body: { content, thread_id: currentThreadId || undefined },
}).catch((err) => {
addMessage('system', 'Failed to send: ' + err.message);
});
}
function enableChatInput() {
if (currentThreadIsReadOnly) return;
const input = document.getElementById('chat-input');
const btn = document.getElementById('send-btn');
if (input) {
input.disabled = false;
input.placeholder = 'Message or / for commands...';
}
if (btn) btn.disabled = false;
}
// --- Image Upload ---
function renderImagePreviews() {
const strip = document.getElementById('image-preview-strip');
strip.innerHTML = '';
stagedImages.forEach((img, idx) => {
const container = document.createElement('div');
container.className = 'image-preview-container';
const preview = document.createElement('img');
preview.className = 'image-preview';
preview.src = img.dataUrl;
preview.alt = 'Attached image';
const removeBtn = document.createElement('button');
removeBtn.className = 'image-preview-remove';
removeBtn.textContent = '\u00d7';
removeBtn.addEventListener('click', () => {
stagedImages.splice(idx, 1);
renderImagePreviews();
});
container.appendChild(preview);
container.appendChild(removeBtn);
strip.appendChild(container);
});
}
const MAX_IMAGE_SIZE_BYTES = 5 * 1024 * 1024; // 5 MB per image
const MAX_STAGED_IMAGES = 5;
function handleImageFiles(files) {
Array.from(files).forEach(file => {
if (!file.type.startsWith('image/')) return;
if (file.size > MAX_IMAGE_SIZE_BYTES) {
alert(`Image "${file.name}" exceeds 5 MB limit (${(file.size / 1024 / 1024).toFixed(1)} MB)`);
return;
}
if (stagedImages.length >= MAX_STAGED_IMAGES) {
alert(`Maximum ${MAX_STAGED_IMAGES} images allowed per message`);
return;
}
const reader = new FileReader();
reader.onload = function(e) {
const dataUrl = e.target.result;
const commaIdx = dataUrl.indexOf(',');
const meta = dataUrl.substring(0, commaIdx); // e.g. "data:image/png;base64"
const base64 = dataUrl.substring(commaIdx + 1);
const mediaType = meta.replace('data:', '').replace(';base64', '');
stagedImages.push({ media_type: mediaType, data: base64, dataUrl: dataUrl });
renderImagePreviews();
};
reader.readAsDataURL(file);
});
}
document.getElementById('attach-btn').addEventListener('click', () => {
document.getElementById('image-file-input').click();
});
document.getElementById('image-file-input').addEventListener('change', (e) => {
handleImageFiles(e.target.files);
e.target.value = '';
});
document.getElementById('chat-input').addEventListener('paste', (e) => {
const items = (e.clipboardData || e.originalEvent.clipboardData).items;
for (let i = 0; i < items.length; i++) {
if (items[i].kind === 'file' && items[i].type.startsWith('image/')) {
const file = items[i].getAsFile();
if (file) handleImageFiles([file]);
}
}
});
function addGeneratedImage(dataUrl, path) {
const container = document.getElementById('chat-messages');
const card = document.createElement('div');
card.className = 'generated-image-card';
const img = document.createElement('img');
img.className = 'generated-image';
img.src = dataUrl;
img.alt = 'Generated image';
card.appendChild(img);
if (path) {
const pathLabel = document.createElement('div');
pathLabel.className = 'generated-image-path';
pathLabel.textContent = path;
card.appendChild(pathLabel);
}
container.appendChild(card);
container.scrollTop = container.scrollHeight;
// no-op: input and send button are always enabled
}
// --- Slash Autocomplete ---
@@ -659,13 +541,6 @@ function sendApprovalAction(requestId, action) {
function renderMarkdown(text) {
if (typeof marked !== 'undefined') {
// Escape raw HTML error pages instead of rendering them as markup.
// Only triggers when the text *starts with* a doctype or <html> tag
// (after optional whitespace), so normal messages that mention HTML
// tags in prose or code fences are not affected. See #263.
if (/^\s*<!doctype\s/i.test(text) || /^\s*<html[\s>]/i.test(text)) {
return escapeHtml(text);
}
let html = marked.parse(text);
// Sanitize HTML output to prevent XSS from tool output or LLM responses.
html = sanitizeRenderedHtml(html);
@@ -1259,9 +1134,7 @@ function loadHistory(before) {
// Fresh load: clear and render
container.innerHTML = '';
for (const turn of data.turns) {
if (turn.user_input) {
addMessage('user', turn.user_input);
}
addMessage('user', turn.user_input);
if (turn.tool_calls && turn.tool_calls.length > 0) {
addToolCallsSummary(turn.tool_calls);
}
@@ -1283,10 +1156,8 @@ function loadHistory(before) {
const savedHeight = container.scrollHeight;
const fragment = document.createDocumentFragment();
for (const turn of data.turns) {
if (turn.user_input) {
const userDiv = createMessageElement('user', turn.user_input);
fragment.appendChild(userDiv);
}
const userDiv = createMessageElement('user', turn.user_input);
fragment.appendChild(userDiv);
if (turn.tool_calls && turn.tool_calls.length > 0) {
fragment.appendChild(createToolCallsSummaryElement(turn.tool_calls));
}
@@ -1385,37 +1256,6 @@ function removeScrollSpinner() {
// --- Threads ---
function threadTitle(thread) {
if (thread.title) return thread.title;
const ch = thread.channel || 'gateway';
if (thread.thread_type === 'heartbeat') return 'Heartbeat Alerts';
if (thread.thread_type === 'routine') return 'Routine';
if (ch !== 'gateway') return ch.charAt(0).toUpperCase() + ch.slice(1);
if (thread.turn_count === 0) return 'New chat';
return thread.id.substring(0, 8);
}
function relativeTime(isoStr) {
if (!isoStr) return '';
const diff = Date.now() - new Date(isoStr).getTime();
const mins = Math.floor(diff / 60000);
if (mins < 1) return 'now';
if (mins < 60) return mins + 'm ago';
const hrs = Math.floor(mins / 60);
if (hrs < 24) return hrs + 'h ago';
const days = Math.floor(hrs / 24);
return days + 'd ago';
}
function isReadOnlyChannel(channel) {
return channel && channel !== 'gateway' && channel !== 'routine' && channel !== 'heartbeat';
}
function debouncedLoadThreads() {
if (_loadThreadsTimer) clearTimeout(_loadThreadsTimer);
_loadThreadsTimer = setTimeout(() => { _loadThreadsTimer = null; loadThreads(); }, 500);
}
function loadThreads() {
apiFetch('/api/chat/threads').then((data) => {
// Pinned assistant thread
@@ -1424,13 +1264,9 @@ function loadThreads() {
const el = document.getElementById('assistant-thread');
const isActive = currentThreadId === assistantThreadId;
el.className = 'assistant-item' + (isActive ? ' active' : '');
const labelEl = document.getElementById('assistant-label');
if (labelEl) {
const at = data.assistant_thread;
labelEl.textContent = 'Assistant';
}
const meta = document.getElementById('assistant-meta');
meta.textContent = relativeTime(data.assistant_thread.updated_at);
const count = data.assistant_thread.turn_count || 0;
meta.textContent = count > 0 ? count + ' turns' : '';
}
// Regular threads
@@ -1439,38 +1275,16 @@ function loadThreads() {
const threads = data.threads || [];
for (const thread of threads) {
const item = document.createElement('div');
const isActive = thread.id === currentThreadId;
item.className = 'thread-item' + (isActive ? ' active' : '');
// Channel badge for non-gateway threads
const ch = thread.channel || 'gateway';
if (ch !== 'gateway') {
const badge = document.createElement('span');
badge.className = 'thread-badge thread-badge-' + ch;
badge.textContent = ch;
item.appendChild(badge);
}
item.className = 'thread-item' + (thread.id === currentThreadId ? ' active' : '');
const label = document.createElement('span');
label.className = 'thread-label';
label.textContent = threadTitle(thread);
label.title = (thread.title || '') + ' (' + thread.id + ')';
label.textContent = thread.title || thread.id.substring(0, 8);
label.title = thread.title ? thread.title + ' (' + thread.id + ')' : thread.id;
item.appendChild(label);
const meta = document.createElement('span');
meta.className = 'thread-meta';
meta.textContent = relativeTime(thread.updated_at);
meta.textContent = (thread.turn_count || 0) + ' turns';
item.appendChild(meta);
// Unread dot
const unread = unreadThreads.get(thread.id) || 0;
if (unread > 0 && !isActive) {
const dot = document.createElement('span');
dot.className = 'thread-unread';
dot.textContent = unread > 9 ? '9+' : String(unread);
item.appendChild(dot);
}
item.addEventListener('click', () => switchThread(thread.id));
list.appendChild(item);
}
@@ -1480,36 +1294,17 @@ function loadThreads() {
switchToAssistant();
}
// Enable/disable chat input based on channel type
// Enable chat input once a thread is available
if (currentThreadId) {
const currentThread = threads.find(t => t.id === currentThreadId);
const ch = currentThread ? currentThread.channel : 'gateway';
currentThreadIsReadOnly = isReadOnlyChannel(ch);
if (currentThreadIsReadOnly) {
disableChatInputReadOnly();
} else {
enableChatInput();
}
enableChatInput();
}
}).catch(() => {});
}
function disableChatInputReadOnly() {
const input = document.getElementById('chat-input');
const btn = document.getElementById('send-btn');
if (input) {
input.disabled = true;
input.placeholder = 'Read-only thread (external channel)';
}
if (btn) btn.disabled = true;
}
function switchToAssistant() {
if (!assistantThreadId) return;
finalizeActivityGroup();
currentThreadId = assistantThreadId;
currentThreadIsReadOnly = false;
unreadThreads.delete(assistantThreadId);
hasMore = false;
oldestTimestamp = null;
loadHistory();
@@ -1519,7 +1314,6 @@ function switchToAssistant() {
function switchThread(threadId) {
finalizeActivityGroup();
currentThreadId = threadId;
unreadThreads.delete(threadId);
hasMore = false;
oldestTimestamp = null;
loadHistory();
@@ -1576,7 +1370,7 @@ chatInput.addEventListener('keydown', (e) => {
}
}
if (e.key === 'Enter' && !e.shiftKey && !e.isComposing) {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
hideSlashAutocomplete();
sendMessage();
@@ -3511,10 +3305,6 @@ function shortModelName(model) {
function fetchGatewayStatus() {
apiFetch('/api/gateway/status').then(function(data) {
// Update restart button visibility
restartEnabled = data.restart_enabled || false;
updateRestartButtonVisibility();
var popover = document.getElementById('gateway-popover');
var html = '';
@@ -3580,15 +3370,10 @@ let teeReportCache = null;
let teeReportLoading = false;
function teeApiBase() {
var hostname = window.location.hostname;
// Skip IP addresses (IPv4 and IPv6) and localhost
if (hostname === "localhost" || /^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$/.test(hostname) || hostname.indexOf(":") !== -1) {
return null;
}
var parts = hostname.split(".");
if (parts.length < 2) return null;
var domain = parts.slice(1).join(".");
return window.location.protocol + "//api." + domain;
var parts = window.location.hostname.split('.');
if (parts.length < 2) return null;
var domain = parts.slice(1).join('.');
return window.location.protocol + '//api.' + domain;
}
function teeInstanceName() {
@@ -3599,19 +3384,13 @@ function checkTeeStatus() {
var base = teeApiBase();
if (!base) return;
var name = teeInstanceName();
try {
fetch(base + '/instances/' + encodeURIComponent(name) + '/attestation').then(function(res) {
if (!res.ok) throw new Error(res.status);
return res.json();
}).then(function(data) {
teeInfo = data;
document.getElementById('tee-shield').style.display = 'flex';
}).catch(function(err) {
console.warn('Failed to fetch TEE attestation:', err);
});
} catch (e) {
console.warn("Failed to check TEE status:", e);
}
fetch(base + '/instances/' + encodeURIComponent(name) + '/attestation').then(function(res) {
if (!res.ok) throw new Error(res.status);
return res.json();
}).then(function(data) {
teeInfo = data;
document.getElementById('tee-shield').style.display = 'flex';
}).catch(function() {});
}
function fetchTeeReport() {
+3 -6
View File
@@ -99,7 +99,7 @@
<span id="sse-status">Connected</span>
<div class="gateway-popover" id="gateway-popover"></div>
</div>
<button class="restart-btn" id="restart-btn" onclick="triggerRestart()" title="Gracefully restart the process" style="display: none;">
<button class="restart-btn" id="restart-btn" onclick="triggerRestart()" title="Gracefully restart the process">
<svg id="restart-icon" width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M23 4v6h-6"></path>
<path d="M1 20v-6h6"></path>
@@ -113,12 +113,12 @@
<div class="tab-panel active" id="tab-chat">
<div class="thread-sidebar" id="thread-sidebar">
<div class="thread-sidebar-header">
<span>Threads</span>
<button class="thread-new-btn" onclick="createNewThread()" title="New thread (Ctrl/Cmd+N)">+</button>
<div class="spacer"></div>
<button class="thread-toggle-btn" id="thread-toggle-btn" onclick="toggleThreadSidebar()" title="Toggle sidebar">&laquo;</button>
</div>
<div class="assistant-item" id="assistant-thread" onclick="switchToAssistant()">
<span class="assistant-label" id="assistant-label">Assistant</span>
<span class="assistant-label">Assistant</span>
<span class="assistant-meta" id="assistant-meta"></span>
</div>
<div class="threads-section-header">
@@ -130,10 +130,7 @@
<div class="chat-messages" id="chat-messages"></div>
<div id="slash-autocomplete" class="slash-autocomplete" style="display:none"></div>
<div class="chat-input">
<div id="image-preview-strip" class="image-preview-strip"></div>
<textarea id="chat-input" placeholder="Message or / for commands..." rows="1"></textarea>
<input type="file" id="image-file-input" accept="image/*" multiple style="display:none">
<button id="attach-btn" class="attach-btn" title="Attach images" aria-label="Attach images">&#x1F4CE;</button>
<button id="send-btn" onclick="sendMessage()">Send</button>
</div>
</div>
+23 -153
View File
@@ -159,7 +159,7 @@ body {
flex-shrink: 0;
}
.tab-bar button:not(.status-logs-btn):not(.restart-btn) {
.tab-bar button:not(.status-logs-btn) {
padding: 10px 20px;
background: none;
border: none;
@@ -171,11 +171,11 @@ body {
transition: color 0.2s, border-color 0.2s;
}
.tab-bar button:not(.status-logs-btn):not(.restart-btn):hover {
.tab-bar button:not(.status-logs-btn):hover {
color: var(--text);
}
.tab-bar button:not(.status-logs-btn):not(.restart-btn).active {
.tab-bar button:not(.status-logs-btn).active {
color: var(--accent);
border-bottom-color: var(--accent);
}
@@ -260,42 +260,42 @@ body {
}
/* Restart Button */
.tab-bar .restart-btn {
.restart-btn {
display: flex;
align-items: center;
gap: 0.375rem;
margin: 0.375rem;
padding: 0.25rem 0.75rem;
border-radius: 0.5rem;
font-size: 0.8rem;
border: 1px solid #00d894;
border: 1px solid;
border-color: #00d894;
color: #00d894;
background-color: transparent;
cursor: pointer;
transition: color 150ms, background-color 150ms, border-color 150ms;
}
.tab-bar .restart-btn:hover:not(:disabled) {
.restart-btn:hover:not(:disabled) {
background-color: rgba(0, 216, 148, 0.1);
}
.tab-bar .restart-btn:disabled {
.restart-btn:disabled {
border-color: #333;
color: #666;
cursor: not-allowed;
}
.tab-bar .restart-btn:disabled:hover {
.restart-btn:disabled:hover {
background-color: transparent;
}
.tab-bar .restart-btn svg {
.restart-btn svg {
flex-shrink: 0;
width: 13px;
height: 13px;
}
.tab-bar .restart-btn svg.spinning {
.restart-btn svg.spinning {
animation: spin-icon 1s linear infinite;
}
@@ -1272,7 +1272,6 @@ body {
/* Chat input */
.chat-input {
display: flex;
flex-wrap: wrap;
padding: 12px 16px max(12px, env(safe-area-inset-bottom)) 16px;
gap: 8px;
background: var(--bg-secondary);
@@ -3075,7 +3074,7 @@ mark {
}
.thread-sidebar {
width: 240px;
width: 200px;
background: var(--bg-secondary);
border-right: 1px solid var(--border);
display: flex;
@@ -3083,8 +3082,6 @@ mark {
flex-shrink: 0;
transition: width 0.2s ease;
overflow: hidden;
padding: 6px;
gap: 2px;
}
.thread-sidebar.collapsed {
@@ -3102,7 +3099,8 @@ mark {
.thread-sidebar-header {
display: flex;
align-items: center;
padding: 10px 10px;
padding: 10px 12px;
border-bottom: 1px solid var(--border);
font-size: 13px;
font-weight: 600;
gap: 8px;
@@ -3136,22 +3134,21 @@ mark {
display: flex;
align-items: center;
justify-content: space-between;
padding: 12px 14px;
padding: 10px 12px;
cursor: pointer;
font-size: 13px;
font-weight: 600;
color: var(--text);
background: var(--bg-tertiary);
border-radius: var(--radius);
margin-bottom: 2px;
border-bottom: 1px solid var(--border);
background: var(--bg-secondary);
}
.assistant-item:hover {
background: rgba(255, 255, 255, 0.06);
background: var(--bg-tertiary);
}
.assistant-item.active {
background: rgba(52, 211, 153, 0.1);
background: rgba(52, 211, 153, 0.08);
color: var(--accent);
border-left: 2px solid var(--accent);
}
@@ -3169,7 +3166,7 @@ mark {
}
.threads-section-header {
padding: 10px 10px 4px;
padding: 8px 12px 4px;
font-size: 11px;
font-weight: 500;
text-transform: uppercase;
@@ -3199,11 +3196,11 @@ mark {
display: flex;
align-items: center;
justify-content: space-between;
padding: 10px 14px;
padding: 8px 12px;
cursor: pointer;
font-size: 13px;
color: var(--text-secondary);
border-radius: var(--radius);
border-bottom: 1px solid rgba(255, 255, 255, 0.03);
}
.thread-item:hover {
@@ -3225,43 +3222,6 @@ mark {
.thread-meta {
font-size: 11px;
color: var(--text-secondary);
flex-shrink: 0;
}
.thread-badge {
display: inline-block;
font-size: 9px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.5px;
padding: 1px 5px;
border-radius: 3px;
background: rgba(255, 255, 255, 0.08);
color: var(--text-secondary);
margin-right: 6px;
flex-shrink: 0;
}
.thread-badge-routine { background: rgba(52, 211, 153, 0.15); color: var(--accent); }
.thread-badge-heartbeat { background: rgba(245, 166, 35, 0.15); color: var(--warning); }
.thread-badge-telegram { background: rgba(0, 136, 204, 0.15); color: #0088cc; }
.thread-badge-signal { background: rgba(59, 118, 240, 0.15); color: #3b76f0; }
.thread-badge-slack { background: rgba(74, 21, 75, 0.15); color: #e01e5a; }
.thread-unread {
display: inline-flex;
align-items: center;
justify-content: center;
min-width: 16px;
height: 16px;
font-size: 10px;
font-weight: 700;
background: var(--accent);
color: var(--bg);
border-radius: 8px;
padding: 0 4px;
margin-left: auto;
flex-shrink: 0;
}
/* --- Memory editing --- */
@@ -3660,7 +3620,7 @@ mark {
left: 0;
top: 0;
bottom: 0;
width: 240px;
width: 200px;
z-index: 50;
}
@@ -3762,93 +3722,3 @@ mark {
text-overflow: ellipsis;
white-space: nowrap;
}
/* Image Upload */
.attach-btn {
background: none;
border: none;
cursor: pointer;
font-size: 1.2em;
padding: 8px;
align-self: flex-end;
color: var(--text-secondary);
transition: color 0.2s;
min-height: 40px;
display: flex;
align-items: center;
justify-content: center;
}
.attach-btn:hover {
color: var(--text);
}
.image-preview-strip {
display: flex;
flex-direction: row;
gap: 8px;
padding: 4px;
overflow-x: auto;
min-height: 0;
width: 100%;
}
.image-preview-strip:empty {
display: none;
}
.image-preview-container {
position: relative;
display: inline-block;
flex-shrink: 0;
}
.image-preview {
width: 60px;
height: 60px;
border-radius: 6px;
object-fit: cover;
display: block;
}
.image-preview-remove {
position: absolute;
top: -6px;
right: -6px;
width: 18px;
height: 18px;
border-radius: 50%;
background: var(--danger);
color: #fff;
border: none;
font-size: 12px;
line-height: 18px;
text-align: center;
cursor: pointer;
padding: 0;
}
.image-preview-remove:hover {
background: #c33;
}
/* Generated Image */
.generated-image-card {
max-width: 512px;
margin: 8px 0;
border-radius: 8px;
overflow: hidden;
border: 1px solid var(--border);
}
.generated-image {
max-width: 100%;
display: block;
}
.generated-image-path {
font-size: 12px;
color: var(--text-secondary);
padding: 4px 8px;
background: var(--bg-secondary);
}
-1
View File
@@ -84,7 +84,6 @@ impl TestGatewayBuilder {
chat_rate_limiter: RateLimiter::new(30, 60),
registry_entries: Vec::new(),
cost_guard: None,
routine_engine: Arc::new(tokio::sync::RwLock::new(None)),
startup_time: std::time::Instant::now(),
})
}
+2 -72
View File
@@ -5,23 +5,10 @@ use uuid::Uuid;
// --- Chat ---
/// Base64-encoded image data sent from the web frontend.
#[derive(Debug, Clone, Deserialize)]
pub struct ImageData {
/// MIME type (e.g., "image/png", "image/jpeg").
pub media_type: String,
/// Base64-encoded image data (without data: URL prefix).
pub data: String,
}
#[derive(Debug, Deserialize)]
pub struct SendMessageRequest {
pub content: String,
pub thread_id: Option<String>,
pub timezone: Option<String>,
/// Optional images attached to the message.
#[serde(default)]
pub images: Vec<ImageData>,
}
#[derive(Debug, Serialize)]
@@ -41,8 +28,6 @@ pub struct ThreadInfo {
pub title: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub thread_type: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub channel: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -232,16 +217,6 @@ pub enum SseEvent {
session_id: Option<String>,
},
/// An image was generated by a tool.
#[serde(rename = "image_generated")]
ImageGenerated {
data_url: String,
#[serde(skip_serializing_if = "Option::is_none")]
path: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
thread_id: Option<String>,
},
/// Extension activation status change (WASM channels).
#[serde(rename = "extension_status")]
ExtensionStatus {
@@ -636,10 +611,6 @@ pub enum WsClientMessage {
Message {
content: String,
thread_id: Option<String>,
timezone: Option<String>,
/// Optional images attached to the message.
#[serde(default)]
images: Vec<ImageData>,
},
/// Approve or deny a pending tool execution.
#[serde(rename = "approval")]
@@ -706,7 +677,6 @@ impl WsServerMessage {
SseEvent::JobToolResult { .. } => "job_tool_result",
SseEvent::JobStatus { .. } => "job_status",
SseEvent::JobResult { .. } => "job_result",
SseEvent::ImageGenerated { .. } => "image_generated",
SseEvent::ExtensionStatus { .. } => "extension_status",
};
let data = serde_json::to_value(event).unwrap_or(serde_json::Value::Null);
@@ -826,9 +796,7 @@ mod tests {
let json = r#"{"type":"message","content":"hello","thread_id":"t1"}"#;
let msg: WsClientMessage = serde_json::from_str(json).unwrap();
match msg {
WsClientMessage::Message {
content, thread_id, ..
} => {
WsClientMessage::Message { content, thread_id } => {
assert_eq!(content, "hello");
assert_eq!(thread_id.as_deref(), Some("t1"));
}
@@ -841,9 +809,7 @@ mod tests {
let json = r#"{"type":"message","content":"hi"}"#;
let msg: WsClientMessage = serde_json::from_str(json).unwrap();
match msg {
WsClientMessage::Message {
content, thread_id, ..
} => {
WsClientMessage::Message { content, thread_id } => {
assert_eq!(content, "hi");
assert!(thread_id.is_none());
}
@@ -1097,40 +1063,4 @@ mod tests {
let req: AuthCancelRequest = serde_json::from_str(json).unwrap();
assert_eq!(req.extension_name, "telegram");
}
// ---- ThreadInfo channel field tests ----
#[test]
fn test_thread_info_channel_serialized() {
let info = ThreadInfo {
id: Uuid::nil(),
state: "Idle".to_string(),
turn_count: 0,
created_at: "2026-01-01T00:00:00Z".to_string(),
updated_at: "2026-01-01T00:00:00Z".to_string(),
title: None,
thread_type: None,
channel: Some("telegram".to_string()),
};
let json = serde_json::to_string(&info).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["channel"], "telegram");
}
#[test]
fn test_thread_info_channel_omitted_when_none() {
let info = ThreadInfo {
id: Uuid::nil(),
state: "Idle".to_string(),
turn_count: 0,
created_at: "2026-01-01T00:00:00Z".to_string(),
updated_at: "2026-01-01T00:00:00Z".to_string(),
title: None,
thread_type: None,
channel: None,
};
let json = serde_json::to_string(&info).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert!(parsed.get("channel").is_none());
}
}
+1 -75
View File
@@ -3,10 +3,6 @@
use crate::channels::web::types::{ToolCallInfo, TurnInfo};
/// Truncate a string to at most `max_bytes` bytes at a char boundary, appending "...".
///
/// If the input is wrapped in `<tool_output …>…</tool_output>` and truncation
/// removes the closing tag, the tag is re-appended so downstream XML parsers
/// never see an unclosed element.
pub fn truncate_preview(s: &str, max_bytes: usize) -> String {
if s.len() <= max_bytes {
return s.to_string();
@@ -16,14 +12,7 @@ pub fn truncate_preview(s: &str, max_bytes: usize) -> String {
while end > 0 && !s.is_char_boundary(end) {
end -= 1;
}
let mut result = format!("{}...", &s[..end]);
// Re-close <tool_output> if truncation cut through the closing tag.
if s.starts_with("<tool_output") && !result.ends_with("</tool_output>") {
result.push_str("\n</tool_output>");
}
result
format!("{}...", &s[..end])
}
/// Build TurnInfo pairs from flat DB messages (user/tool_calls/assistant triples).
@@ -94,19 +83,6 @@ pub fn build_turns_from_db_messages(
turns.push(turn);
turn_number += 1;
} else if msg.role == "assistant" {
// Standalone assistant message (e.g. routine output, heartbeat)
// with no preceding user message — render as a turn with empty input.
turns.push(TurnInfo {
turn_number,
user_input: String::new(),
response: Some(msg.content.clone()),
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: Some(msg.created_at.to_rfc3339()),
tool_calls: Vec::new(),
});
turn_number += 1;
}
}
@@ -173,33 +149,6 @@ mod tests {
assert_eq!(truncate_preview("hello", 0), "...");
}
#[test]
fn test_truncate_preview_closes_tool_output_tag() {
let s = "<tool_output name=\"search\" sanitized=\"true\">\nSome very long content here\n</tool_output>";
// Truncate so it cuts before the closing tag
let result = truncate_preview(s, 60);
assert!(result.ends_with("</tool_output>"));
assert!(result.contains("..."));
}
#[test]
fn test_truncate_preview_no_extra_close_when_intact() {
let s = "<tool_output name=\"echo\" sanitized=\"false\">\nshort\n</tool_output>";
// The string is short enough not to be truncated
let result = truncate_preview(s, 500);
assert_eq!(result, s);
// Should not have a duplicate closing tag
assert_eq!(result.matches("</tool_output>").count(), 1);
}
#[test]
fn test_truncate_preview_non_xml_unaffected() {
let s = "Just a plain long string that gets truncated";
let result = truncate_preview(s, 10);
assert_eq!(result, "Just a pla...");
assert!(!result.contains("</tool_output>"));
}
// ---- build_turns_from_db_messages tests ----
fn make_msg(role: &str, content: &str, offset_ms: i64) -> crate::history::ConversationMessage {
@@ -271,29 +220,6 @@ mod tests {
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_standalone_assistant_messages() {
// Routine conversations only have assistant messages (no user messages).
let messages = vec![
make_msg("assistant", "Routine executed: all checks passed", 0),
make_msg("assistant", "Routine executed: found 2 issues", 5000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 2);
// Standalone assistant messages should have empty user_input
assert_eq!(turns[0].user_input, "");
assert_eq!(
turns[0].response.as_deref(),
Some("Routine executed: all checks passed")
);
assert_eq!(turns[0].state, "Completed");
assert_eq!(turns[1].user_input, "");
assert_eq!(
turns[1].response.as_deref(),
Some("Routine executed: found 2 issues")
);
}
#[test]
fn test_build_turns_backward_compatible() {
let messages = vec![
+1 -20
View File
@@ -156,26 +156,12 @@ async fn handle_client_message(
direct_tx: &mpsc::Sender<WsServerMessage>,
) {
match msg {
WsClientMessage::Message {
content,
thread_id,
timezone,
images,
} => {
WsClientMessage::Message { content, thread_id } => {
let mut incoming = IncomingMessage::new("gateway", user_id, &content);
if let Some(ref tz) = timezone {
incoming = incoming.with_timezone(tz);
}
if let Some(ref tid) = thread_id {
incoming = incoming.with_thread(tid);
}
// Convert uploaded images to IncomingAttachments
if !images.is_empty() {
let attachments = crate::channels::web::server::images_to_attachments(&images);
incoming = incoming.with_attachments(attachments);
}
let tx_guard = state.msg_tx.read().await;
if let Some(ref tx) = *tx_guard {
if tx.send(incoming).await.is_err() {
@@ -363,8 +349,6 @@ mod tests {
WsClientMessage::Message {
content: "hello agent".to_string(),
thread_id: Some("t1".to_string()),
timezone: None,
images: Vec::new(),
},
&state,
"user1",
@@ -389,8 +373,6 @@ mod tests {
WsClientMessage::Message {
content: "hello".to_string(),
thread_id: None,
timezone: None,
images: Vec::new(),
},
&state,
"user1",
@@ -511,7 +493,6 @@ mod tests {
chat_rate_limiter: crate::channels::web::server::RateLimiter::new(30, 60),
registry_entries: Vec::new(),
cost_guard: None,
routine_engine: Arc::new(tokio::sync::RwLock::new(None)),
startup_time: std::time::Instant::now(),
}
}
+121 -243
View File
@@ -2,79 +2,52 @@
//!
//! Commands for adding, removing, authenticating, and testing MCP servers.
use std::collections::HashMap;
use std::io::Write;
use std::sync::Arc;
use clap::{Args, Subcommand};
use clap::Subcommand;
use crate::config::Config;
use crate::db::Database;
#[cfg(feature = "postgres")]
use crate::secrets::PostgresSecretsStore;
use crate::secrets::{SecretsCrypto, SecretsStore};
use crate::tools::mcp::{
McpClient, McpServerConfig, McpSessionManager, OAuthConfig,
auth::{authorize_mcp_server, is_authenticated},
config::{self, EffectiveTransport, McpServersFile},
config::{self, McpServersFile},
};
/// Arguments for the `mcp add` subcommand.
#[derive(Args, Debug, Clone)]
pub struct McpAddArgs {
/// Server name (e.g., "notion", "github")
pub name: String,
/// Server URL (e.g., "https://mcp.notion.com") -- required for http transport
pub url: Option<String>,
/// Transport type: http (default), stdio, unix
#[arg(long, default_value = "http")]
pub transport: String,
/// Command to run (stdio transport)
#[arg(long)]
pub command: Option<String>,
/// Command arguments (stdio transport, can be repeated)
#[arg(long = "arg", num_args = 1..)]
pub cmd_args: Vec<String>,
/// Environment variables (stdio transport, KEY=VALUE format, can be repeated)
#[arg(long = "env", value_parser = parse_env_var)]
pub env: Vec<(String, String)>,
/// Unix socket path (unix transport)
#[arg(long)]
pub socket: Option<String>,
/// Custom HTTP headers (KEY:VALUE format, can be repeated)
#[arg(long = "header", value_parser = parse_header)]
pub headers: Vec<(String, String)>,
/// OAuth client ID (if authentication is required)
#[arg(long)]
pub client_id: Option<String>,
/// OAuth authorization URL (optional, can be discovered)
#[arg(long)]
pub auth_url: Option<String>,
/// OAuth token URL (optional, can be discovered)
#[arg(long)]
pub token_url: Option<String>,
/// Scopes to request (comma-separated)
#[arg(long)]
pub scopes: Option<String>,
/// Server description
#[arg(long)]
pub description: Option<String>,
}
#[derive(Subcommand, Debug, Clone)]
pub enum McpCommand {
/// Add an MCP server
Add(Box<McpAddArgs>),
Add {
/// Server name (e.g., "notion", "github")
name: String,
/// Server URL (e.g., "https://mcp.notion.com")
url: String,
/// OAuth client ID (if authentication is required)
#[arg(long)]
client_id: Option<String>,
/// OAuth authorization URL (optional, can be discovered)
#[arg(long)]
auth_url: Option<String>,
/// OAuth token URL (optional, can be discovered)
#[arg(long)]
token_url: Option<String>,
/// Scopes to request (comma-separated)
#[arg(long)]
scopes: Option<String>,
/// Server description
#[arg(long)]
description: Option<String>,
},
/// Remove an MCP server
Remove {
@@ -124,24 +97,29 @@ pub enum McpCommand {
},
}
fn parse_header(s: &str) -> Result<(String, String), String> {
let pos = s
.find(':')
.ok_or_else(|| format!("invalid header format '{}', expected KEY:VALUE", s))?;
Ok((s[..pos].trim().to_string(), s[pos + 1..].trim().to_string()))
}
fn parse_env_var(s: &str) -> Result<(String, String), String> {
let pos = s
.find('=')
.ok_or_else(|| format!("invalid env var format '{}', expected KEY=VALUE", s))?;
Ok((s[..pos].to_string(), s[pos + 1..].to_string()))
}
/// Run an MCP command.
pub async fn run_mcp_command(cmd: McpCommand) -> anyhow::Result<()> {
match cmd {
McpCommand::Add(args) => add_server(*args).await,
McpCommand::Add {
name,
url,
client_id,
auth_url,
token_url,
scopes,
description,
} => {
add_server(
name,
url,
client_id,
auth_url,
token_url,
scopes,
description,
)
.await
}
McpCommand::Remove { name } => remove_server(name).await,
McpCommand::List { verbose } => list_servers(verbose).await,
McpCommand::Auth { name, user } => auth_server(name, user).await,
@@ -155,58 +133,16 @@ pub async fn run_mcp_command(cmd: McpCommand) -> anyhow::Result<()> {
}
/// Add a new MCP server.
async fn add_server(args: McpAddArgs) -> anyhow::Result<()> {
let McpAddArgs {
name,
url,
transport,
command,
cmd_args,
env,
socket,
headers,
client_id,
auth_url,
token_url,
scopes,
description,
} = args;
let transport_lower = transport.to_lowercase();
let mut config = match transport_lower.as_str() {
"stdio" => {
let cmd = command
.clone()
.ok_or_else(|| anyhow::anyhow!("--command is required for stdio transport"))?;
let env_map: HashMap<String, String> = env.into_iter().collect();
McpServerConfig::new_stdio(&name, &cmd, cmd_args.clone(), env_map)
}
"unix" => {
let socket_path = socket
.clone()
.ok_or_else(|| anyhow::anyhow!("--socket is required for unix transport"))?;
McpServerConfig::new_unix(&name, &socket_path)
}
"http" => {
let url_val = url
.as_deref()
.ok_or_else(|| anyhow::anyhow!("URL is required for http transport"))?;
McpServerConfig::new(&name, url_val)
}
other => {
anyhow::bail!(
"Unknown transport type '{}'. Supported: http, stdio, unix",
other
);
}
};
// Apply headers if any
if !headers.is_empty() {
let headers_map: HashMap<String, String> = headers.into_iter().collect();
config = config.with_headers(headers_map);
}
async fn add_server(
name: String,
url: String,
client_id: Option<String>,
auth_url: Option<String>,
token_url: Option<String>,
scopes: Option<String>,
description: Option<String>,
) -> anyhow::Result<()> {
let mut config = McpServerConfig::new(&name, &url);
if let Some(desc) = description {
config = config.with_description(desc);
@@ -215,12 +151,8 @@ async fn add_server(args: McpAddArgs) -> anyhow::Result<()> {
// Track if auth is required
let requires_auth = client_id.is_some();
// Set up OAuth if client_id is provided (HTTP transport only)
// Set up OAuth if client_id is provided
if let Some(client_id) = client_id {
if transport_lower != "http" {
anyhow::bail!("OAuth authentication is only supported with http transport");
}
let mut oauth = OAuthConfig::new(client_id);
if let (Some(auth), Some(token)) = (auth_url, token_url) {
@@ -249,24 +181,7 @@ async fn add_server(args: McpAddArgs) -> anyhow::Result<()> {
println!();
println!(" ✓ Added MCP server '{}'", name);
match transport_lower.as_str() {
"stdio" => {
println!(
" Transport: stdio (command: {})",
command.as_deref().unwrap_or("")
);
}
"unix" => {
println!(
" Transport: unix (socket: {})",
socket.as_deref().unwrap_or("")
);
}
_ => {
println!(" URL: {}", url.as_deref().unwrap_or(""));
}
}
println!(" URL: {}", url);
if requires_auth {
println!();
@@ -321,40 +236,9 @@ async fn list_servers(verbose: bool) -> anyhow::Result<()> {
""
};
let effective = server.effective_transport();
let transport_label = match &effective {
EffectiveTransport::Http => "http".to_string(),
EffectiveTransport::Stdio { command, .. } => {
format!("stdio ({})", command)
}
EffectiveTransport::Unix { socket_path } => {
format!("unix ({})", socket_path)
}
};
if verbose {
println!(" {} {}{}", status, server.name, auth_status);
println!(" Transport: {}", transport_label);
match &effective {
EffectiveTransport::Http => {
println!(" URL: {}", server.url);
}
EffectiveTransport::Stdio { command, args, env } => {
println!(" Command: {}", command);
if !args.is_empty() {
println!(" Args: {}", args.join(", "));
}
if !env.is_empty() {
// Only print env var names, not values (may contain secrets).
let env_keys: Vec<&str> = env.keys().map(|k| k.as_str()).collect();
println!(" Env: {}", env_keys.join(", "));
}
}
EffectiveTransport::Unix { socket_path } => {
println!(" Socket: {}", socket_path);
}
}
println!(" URL: {}", server.url);
if let Some(ref desc) = server.description {
println!(" Description: {}", desc);
}
@@ -364,27 +248,11 @@ async fn list_servers(verbose: bool) -> anyhow::Result<()> {
println!(" Scopes: {}", oauth.scopes.join(", "));
}
}
if !server.headers.is_empty() {
let header_keys: Vec<&String> = server.headers.keys().collect();
println!(
" Headers: {}",
header_keys
.iter()
.map(|k| k.as_str())
.collect::<Vec<_>>()
.join(", ")
);
}
println!();
} else {
let display = match &effective {
EffectiveTransport::Http => server.url.clone(),
EffectiveTransport::Stdio { command, .. } => command.to_string(),
EffectiveTransport::Unix { socket_path } => socket_path.to_string(),
};
println!(
" {} {} - {} [{}]{}",
status, server.name, display, transport_label, auth_status
" {} {} - {}{}",
status, server.name, server.url, auth_status
);
}
}
@@ -506,7 +374,7 @@ async fn test_server(name: String, user_id: String) -> anyhow::Result<()> {
return Ok(());
} else {
// No OAuth and no tokens - try unauthenticated
McpClient::new_with_config(server.clone())
McpClient::new_with_name(&server.name, &server.url)
};
// Test connection
@@ -636,9 +504,61 @@ async fn get_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Syn
)
})?;
let crypto = Arc::new(SecretsCrypto::new(master_key.clone())?);
let crypto = SecretsCrypto::new(master_key.clone())?;
Ok(crate::db::create_secrets_store(&config.database, crypto).await?)
#[cfg(feature = "postgres")]
{
let store = crate::history::Store::new(&config.database).await?;
store.run_migrations().await?;
Ok(Arc::new(PostgresSecretsStore::new(
store.pool(),
Arc::new(crypto),
)))
}
#[cfg(all(feature = "libsql", not(feature = "postgres")))]
{
use crate::db::Database as _;
use crate::db::libsql::LibSqlBackend;
use secrecy::ExposeSecret as _;
let default_path = crate::config::default_libsql_path();
let db_path = config
.database
.libsql_path
.as_deref()
.unwrap_or(&default_path);
let backend = if let Some(ref url) = config.database.libsql_url {
let token = config.database.libsql_auth_token.as_ref().ok_or_else(|| {
anyhow::anyhow!("LIBSQL_AUTH_TOKEN is required when LIBSQL_URL is set")
})?;
LibSqlBackend::new_remote_replica(db_path, url, token.expose_secret())
.await
.map_err(|e| anyhow::anyhow!("{}", e))?
} else {
LibSqlBackend::new_local(db_path)
.await
.map_err(|e| anyhow::anyhow!("{}", e))?
};
backend
.run_migrations()
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new(
backend.shared_db(),
Arc::new(crypto),
)))
}
#[cfg(not(any(feature = "postgres", feature = "libsql")))]
{
let _ = crypto;
anyhow::bail!(
"No database backend available for secrets. Enable 'postgres' or 'libsql' feature."
);
}
}
#[cfg(test)]
@@ -659,46 +579,4 @@ mod tests {
TestCli::command().debug_assert();
}
#[test]
fn test_parse_header_valid() {
let result = parse_header("Authorization: Bearer token123").unwrap();
assert_eq!(result.0, "Authorization");
assert_eq!(result.1, "Bearer token123");
}
#[test]
fn test_parse_header_no_spaces() {
let result = parse_header("X-Api-Key:abc123").unwrap();
assert_eq!(result.0, "X-Api-Key");
assert_eq!(result.1, "abc123");
}
#[test]
fn test_parse_header_invalid() {
let result = parse_header("no-colon-here");
assert!(result.is_err());
assert!(result.unwrap_err().contains("invalid header format"));
}
#[test]
fn test_parse_env_var_valid() {
let result = parse_env_var("NODE_ENV=production").unwrap();
assert_eq!(result.0, "NODE_ENV");
assert_eq!(result.1, "production");
}
#[test]
fn test_parse_env_var_with_equals_in_value() {
let result = parse_env_var("KEY=value=with=equals").unwrap();
assert_eq!(result.0, "KEY");
assert_eq!(result.1, "value=with=equals");
}
#[test]
fn test_parse_env_var_invalid() {
let result = parse_env_var("no-equals-here");
assert!(result.is_err());
assert!(result.unwrap_err().contains("invalid env var format"));
}
}
+1 -1
View File
@@ -132,7 +132,7 @@ pub enum Command {
about = "Manage MCP servers",
long_about = "Add, auth, list, or test MCP servers.\nExample: ironclaw mcp add notion https://mcp.notion.com"
)]
Mcp(Box<McpCommand>),
Mcp(McpCommand),
/// Query and manage workspace memory
#[command(
+1 -123
View File
@@ -8,35 +8,9 @@ use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
use crate::settings::Settings;
/// Load settings from JSON and TOML config files, matching the runtime
/// priority: TOML overlay > settings.json > defaults.
///
/// This mirrors the loading chain in `Config::from_env_with_toml()` but
/// without resolving the full `Config` (which requires async + secrets).
fn load_settings() -> Settings {
load_settings_from(&Settings::default_path(), &Settings::default_toml_path())
}
/// Inner implementation with injectable paths (testable).
fn load_settings_from(json_path: &std::path::Path, toml_path: &std::path::Path) -> Settings {
let mut settings = Settings::load_from(json_path);
match Settings::load_toml(toml_path) {
Ok(Some(toml_settings)) => {
settings.merge_from(&toml_settings);
}
Ok(None) => {} // File not found — fine for default path
Err(e) => {
eprintln!("Warning: failed to parse {}: {}", toml_path.display(), e);
}
}
settings
}
/// Run the status command, printing system health info.
pub async fn run_status_command() -> anyhow::Result<()> {
let settings = load_settings();
let settings = Settings::default();
println!("IronClaw Status");
println!("===============\n");
@@ -235,99 +209,3 @@ fn default_tools_dir() -> PathBuf {
fn default_channels_dir() -> PathBuf {
ironclaw_base_dir().join("channels")
}
#[cfg(test)]
mod tests {
use super::load_settings_from;
/// Regression test for #354: load_settings_from must read config.toml.
#[test]
fn reads_toml_heartbeat_enabled() {
let dir = tempfile::tempdir().expect("tempdir");
let json_path = dir.path().join("settings.json");
let toml_path = dir.path().join("config.toml");
// No JSON file — only TOML
std::fs::write(
&toml_path,
"[heartbeat]\nenabled = true\ninterval_secs = 600",
)
.expect("write toml");
let settings = load_settings_from(&json_path, &toml_path);
assert!(settings.heartbeat.enabled);
assert_eq!(settings.heartbeat.interval_secs, 600);
}
/// Without any config files, defaults are returned.
#[test]
fn defaults_without_config_files() {
let dir = tempfile::tempdir().expect("tempdir");
let settings = load_settings_from(
&dir.path().join("nonexistent.json"),
&dir.path().join("nonexistent.toml"),
);
assert!(!settings.heartbeat.enabled);
}
/// settings.json is respected.
#[test]
fn reads_json_heartbeat_enabled() {
let dir = tempfile::tempdir().expect("tempdir");
let json_path = dir.path().join("settings.json");
let toml_path = dir.path().join("nonexistent.toml");
std::fs::write(
&json_path,
r#"{"heartbeat":{"enabled":true,"interval_secs":900}}"#,
)
.expect("write json");
let settings = load_settings_from(&json_path, &toml_path);
assert!(settings.heartbeat.enabled);
assert_eq!(settings.heartbeat.interval_secs, 900);
}
/// TOML overlay wins over JSON settings.
#[test]
fn toml_overlay_wins_over_json() {
let dir = tempfile::tempdir().expect("tempdir");
let json_path = dir.path().join("settings.json");
let toml_path = dir.path().join("config.toml");
std::fs::write(
&json_path,
r#"{"heartbeat":{"enabled":false,"interval_secs":100}}"#,
)
.expect("write json");
std::fs::write(
&toml_path,
"[heartbeat]\nenabled = true\ninterval_secs = 200",
)
.expect("write toml");
let settings = load_settings_from(&json_path, &toml_path);
assert!(settings.heartbeat.enabled);
assert_eq!(settings.heartbeat.interval_secs, 200);
}
/// Invalid TOML is warned but doesn't crash; falls back to JSON/defaults.
#[test]
fn invalid_toml_falls_back_gracefully() {
let dir = tempfile::tempdir().expect("tempdir");
let json_path = dir.path().join("settings.json");
let toml_path = dir.path().join("config.toml");
std::fs::write(
&json_path,
r#"{"heartbeat":{"enabled":true,"interval_secs":500}}"#,
)
.expect("write json");
std::fs::write(&toml_path, "this is not valid toml [[[").expect("write bad toml");
let settings = load_settings_from(&json_path, &toml_path);
// Should fall back to JSON values, not crash
assert!(settings.heartbeat.enabled);
assert_eq!(settings.heartbeat.interval_secs, 500);
}
}
+56 -2
View File
@@ -11,6 +11,10 @@ use tokio::fs;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::Config;
#[allow(unused_imports)]
use crate::db::Database;
#[cfg(feature = "postgres")]
use crate::secrets::PostgresSecretsStore;
use crate::secrets::{CreateSecretParams, SecretsCrypto, SecretsStore};
use crate::tools::wasm::{CapabilitiesFile, compute_binary_hash};
@@ -559,9 +563,59 @@ async fn init_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Sy
)
})?;
let crypto = Arc::new(SecretsCrypto::new(master_key.clone())?);
let crypto = SecretsCrypto::new(master_key.clone())?;
Ok(crate::db::create_secrets_store(&config.database, crypto).await?)
let store: Arc<dyn SecretsStore + Send + Sync> = {
#[cfg(feature = "postgres")]
{
let store = crate::history::Store::new(&config.database).await?;
store.run_migrations().await?;
Arc::new(PostgresSecretsStore::new(store.pool(), Arc::new(crypto)))
}
#[cfg(all(feature = "libsql", not(feature = "postgres")))]
{
use crate::db::Database as _;
use crate::db::libsql::LibSqlBackend;
use secrecy::ExposeSecret as _;
let default_path = crate::config::default_libsql_path();
let db_path = config
.database
.libsql_path
.as_deref()
.unwrap_or(&default_path);
let backend = if let Some(ref url) = config.database.libsql_url {
let token = config.database.libsql_auth_token.as_ref().ok_or_else(|| {
anyhow::anyhow!("LIBSQL_AUTH_TOKEN is required when LIBSQL_URL is set")
})?;
LibSqlBackend::new_remote_replica(db_path, url, token.expose_secret())
.await
.map_err(|e| anyhow::anyhow!("{}", e))?
} else {
LibSqlBackend::new_local(db_path)
.await
.map_err(|e| anyhow::anyhow!("{}", e))?
};
backend
.run_migrations()
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
Arc::new(crate::secrets::LibSqlSecretsStore::new(
backend.shared_db(),
Arc::new(crypto),
))
}
#[cfg(not(any(feature = "postgres", feature = "libsql")))]
{
let _ = crypto;
anyhow::bail!(
"No database backend available for secrets. Enable 'postgres' or 'libsql' feature."
);
}
};
Ok(store)
}
/// Configure authentication for a tool.
-37
View File
@@ -27,8 +27,6 @@ pub struct AgentConfig {
pub max_tool_iterations: usize,
/// When true, skip tool approval checks entirely. For benchmarks/CI.
pub auto_approve_tools: bool,
/// Default timezone for new sessions (IANA name, e.g. "America/New_York").
pub default_timezone: String,
}
impl AgentConfig {
@@ -49,7 +47,6 @@ impl AgentConfig {
max_actions_per_hour: None,
max_tool_iterations: 10,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
}
}
@@ -92,40 +89,6 @@ impl AgentConfig {
"AGENT_AUTO_APPROVE_TOOLS",
settings.agent.auto_approve_tools,
)?,
default_timezone: {
let tz: String = parse_optional_env(
"DEFAULT_TIMEZONE",
settings.agent.default_timezone.clone(),
)?;
if crate::timezone::parse_timezone(&tz).is_none() {
return Err(ConfigError::InvalidValue {
key: "DEFAULT_TIMEZONE".into(),
message: format!("invalid IANA timezone: '{tz}'"),
});
}
tz
},
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_default_timezone_rejects_invalid() {
let mut settings = Settings::default();
settings.agent.default_timezone = "Fake/Zone".to_string();
let result = AgentConfig::resolve(&settings);
assert!(result.is_err(), "invalid IANA timezone should be rejected");
}
#[test]
fn test_default_timezone_accepts_valid() {
let settings = Settings::default(); // default is "UTC"
let config = AgentConfig::resolve(&settings).expect("resolve");
assert_eq!(config.default_timezone, "UTC");
}
}
+1 -102
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::{optional_env, parse_bool_env, parse_option_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -13,12 +13,6 @@ pub struct HeartbeatConfig {
pub notify_channel: Option<String>,
/// User ID to notify on heartbeat findings.
pub notify_user: Option<String>,
/// Hour (0-23) when quiet hours start.
pub quiet_hours_start: Option<u32>,
/// Hour (0-23) when quiet hours end.
pub quiet_hours_end: Option<u32>,
/// Timezone for quiet hours evaluation (IANA name).
pub timezone: Option<String>,
}
impl Default for HeartbeatConfig {
@@ -28,9 +22,6 @@ impl Default for HeartbeatConfig {
interval_secs: 1800, // 30 minutes
notify_channel: None,
notify_user: None,
quiet_hours_start: None,
quiet_hours_end: None,
timezone: None,
}
}
}
@@ -47,98 +38,6 @@ impl HeartbeatConfig {
.or_else(|| settings.heartbeat.notify_channel.clone()),
notify_user: optional_env("HEARTBEAT_NOTIFY_USER")?
.or_else(|| settings.heartbeat.notify_user.clone()),
quiet_hours_start: parse_option_env::<u32>("HEARTBEAT_QUIET_START")?
.or(settings.heartbeat.quiet_hours_start)
.map(|h| {
if h > 23 {
return Err(ConfigError::InvalidValue {
key: "HEARTBEAT_QUIET_START".into(),
message: "must be 0-23".into(),
});
}
Ok(h)
})
.transpose()?,
quiet_hours_end: parse_option_env::<u32>("HEARTBEAT_QUIET_END")?
.or(settings.heartbeat.quiet_hours_end)
.map(|h| {
if h > 23 {
return Err(ConfigError::InvalidValue {
key: "HEARTBEAT_QUIET_END".into(),
message: "must be 0-23".into(),
});
}
Ok(h)
})
.transpose()?,
timezone: {
let tz = optional_env("HEARTBEAT_TIMEZONE")?
.or_else(|| settings.heartbeat.timezone.clone());
if let Some(ref tz_str) = tz
&& crate::timezone::parse_timezone(tz_str).is_none()
{
return Err(ConfigError::InvalidValue {
key: "HEARTBEAT_TIMEZONE".into(),
message: format!("invalid IANA timezone: '{tz_str}'"),
});
}
tz
},
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_quiet_hours_settings_fallback() {
// When env vars are not set, settings values should be used
let mut settings = Settings::default();
settings.heartbeat.quiet_hours_start = Some(22);
settings.heartbeat.quiet_hours_end = Some(6);
let config = HeartbeatConfig::resolve(&settings).expect("resolve");
assert_eq!(config.quiet_hours_start, Some(22));
assert_eq!(config.quiet_hours_end, Some(6));
}
#[test]
fn test_quiet_hours_rejects_invalid_hour() {
let mut settings = Settings::default();
settings.heartbeat.quiet_hours_start = Some(24);
let result = HeartbeatConfig::resolve(&settings);
assert!(result.is_err());
}
#[test]
fn test_quiet_hours_accepts_boundary_values() {
let mut settings = Settings::default();
settings.heartbeat.quiet_hours_start = Some(0);
settings.heartbeat.quiet_hours_end = Some(23);
let config = HeartbeatConfig::resolve(&settings).expect("resolve");
assert_eq!(config.quiet_hours_start, Some(0));
assert_eq!(config.quiet_hours_end, Some(23));
}
#[test]
fn test_heartbeat_timezone_rejects_invalid() {
let mut settings = Settings::default();
settings.heartbeat.timezone = Some("Fake/Zone".to_string());
let result = HeartbeatConfig::resolve(&settings);
assert!(result.is_err(), "invalid IANA timezone should be rejected");
}
#[test]
fn test_heartbeat_timezone_accepts_valid() {
let mut settings = Settings::default();
settings.heartbeat.timezone = Some("America/New_York".to_string());
let config = HeartbeatConfig::resolve(&settings).expect("resolve");
assert_eq!(config.timezone.as_deref(), Some("America/New_York"));
}
}
+2 -7
View File
@@ -25,13 +25,8 @@ pub(crate) fn optional_env(key: &str) -> Result<Option<String>, ConfigError> {
}
// Fall back to thread-safe overlay (secrets injected from DB)
if let Some(val) = INJECTED_VARS
.lock()
.unwrap_or_else(|p| p.into_inner())
.get(key)
.cloned()
{
return Ok(Some(val));
if let Some(val) = INJECTED_VARS.get().and_then(|map| map.get(key)) {
return Ok(Some(val.clone()));
}
Ok(None)
+10 -237
View File
@@ -9,13 +9,6 @@ use crate::llm::registry::{ProviderProtocol, ProviderRegistry};
use crate::llm::session::SessionConfig;
use crate::settings::Settings;
/// Sentinel value used as `api_key` when only an OAuth token is present.
///
/// When we only have an OAuth token the provider factory in `llm/mod.rs`
/// checks for this value and routes to `AnthropicOAuthProvider`, so this
/// placeholder is never sent over the wire.
pub const OAUTH_PLACEHOLDER: &str = "oauth-placeholder";
/// Prompt cache retention policy for Anthropic.
///
/// Controls Anthropic's automatic prompt caching via a top-level
@@ -73,7 +66,6 @@ pub struct RegistryProviderConfig {
/// Provider identifier (e.g., "groq", "openai", "tinfoil").
pub provider_id: String,
/// API key (optional for some providers like Ollama).
/// For Anthropic OAuth, this is set to `OAUTH_PLACEHOLDER`.
pub api_key: Option<SecretString>,
/// Base URL for the API endpoint.
pub base_url: String,
@@ -81,22 +73,6 @@ pub struct RegistryProviderConfig {
pub model: String,
/// Extra HTTP headers injected into every request.
pub extra_headers: Vec<(String, String)>,
/// OAuth token for providers that support Bearer auth (e.g. Anthropic via `claude login`).
/// When set, the provider factory routes to the OAuth-specific provider implementation.
pub oauth_token: Option<SecretString>,
}
/// Configuration for AWS Bedrock (native Converse API).
#[derive(Debug, Clone)]
pub struct BedrockConfig {
/// AWS region (e.g. "us-east-1").
pub region: String,
/// Bedrock model ID (e.g. "anthropic.claude-opus-4-6-v1").
pub model: String,
/// Cross-region inference prefix: "us", "eu", "apac", "global", or None.
pub cross_region: Option<String>,
/// AWS named profile (for SSO / assume-role workflows).
pub profile: Option<String>,
}
/// LLM provider configuration.
@@ -114,14 +90,8 @@ pub struct LlmConfig {
/// NEAR AI config (always populated, also used for embeddings).
pub nearai: NearAiConfig,
/// Resolved provider config for registry-based providers.
/// `None` when backend is "nearai" or "bedrock".
/// `None` when backend is "nearai".
pub provider: Option<RegistryProviderConfig>,
/// AWS Bedrock config (populated when backend=bedrock, requires --features bedrock).
pub bedrock: Option<BedrockConfig>,
/// HTTP request timeout in seconds for LLM API calls.
/// Default: 120. Increase for local LLMs (Ollama, vLLM, LM Studio) that
/// need more time for prompt evaluation on consumer hardware.
pub request_timeout_secs: u64,
}
/// NEAR AI configuration.
@@ -184,8 +154,6 @@ impl LlmConfig {
smart_routing_cascade: false,
},
provider: None,
bedrock: None,
request_timeout_secs: 120,
}
}
@@ -216,10 +184,8 @@ impl LlmConfig {
let backend_lower = backend.to_lowercase();
let is_nearai =
backend_lower == "nearai" || backend_lower == "near_ai" || backend_lower == "near";
let is_bedrock =
backend_lower == "bedrock" || backend_lower == "aws_bedrock" || backend_lower == "aws";
if !is_nearai && !is_bedrock && registry.find(&backend_lower).is_none() {
if !is_nearai && registry.find(&backend_lower).is_none() {
tracing::warn!(
"Unknown LLM backend '{}'. Will attempt as openai_compatible fallback.",
backend
@@ -266,8 +232,8 @@ impl LlmConfig {
smart_routing_cascade: parse_optional_env("SMART_ROUTING_CASCADE", true)?,
};
// Resolve registry provider config (for non-NearAI, non-Bedrock backends)
let provider = if is_nearai || is_bedrock {
// Resolve registry provider config (for non-NearAI backends)
let provider = if is_nearai {
None
} else {
Some(Self::resolve_registry_provider(
@@ -277,50 +243,9 @@ impl LlmConfig {
)?)
};
let bedrock = if is_bedrock {
let explicit_region =
optional_env("BEDROCK_REGION")?.or_else(|| settings.bedrock_region.clone());
if explicit_region.is_none() {
tracing::info!("BEDROCK_REGION not set, defaulting to us-east-1");
}
let region = explicit_region.unwrap_or_else(|| "us-east-1".to_string());
let model = optional_env("BEDROCK_MODEL")?
.or_else(|| settings.selected_model.clone())
.ok_or_else(|| ConfigError::MissingRequired {
key: "BEDROCK_MODEL".to_string(),
hint: "Set BEDROCK_MODEL when LLM_BACKEND=bedrock".to_string(),
})?;
let cross_region = optional_env("BEDROCK_CROSS_REGION")?
.or_else(|| settings.bedrock_cross_region.clone());
if let Some(ref cr) = cross_region
&& !matches!(cr.as_str(), "us" | "eu" | "apac" | "global")
{
return Err(ConfigError::InvalidValue {
key: "BEDROCK_CROSS_REGION".to_string(),
message: format!(
"'{}' is not valid, expected one of: us, eu, apac, global",
cr
),
});
}
let profile = optional_env("AWS_PROFILE")?.or_else(|| settings.bedrock_profile.clone());
Some(BedrockConfig {
region,
model,
cross_region,
profile,
})
} else {
None
};
let request_timeout_secs = parse_optional_env("LLM_REQUEST_TIMEOUT_SECS", 120)?;
Ok(Self {
backend: if is_nearai {
"nearai".to_string()
} else if is_bedrock {
"bedrock".to_string()
} else if let Some(ref p) = provider {
p.provider_id.clone()
} else {
@@ -329,8 +254,6 @@ impl LlmConfig {
session,
nearai,
provider,
bedrock,
request_timeout_secs,
})
}
@@ -443,22 +366,6 @@ impl LlmConfig {
Vec::new()
};
// Resolve OAuth token (Anthropic-specific: `claude login` flow).
// Only check for OAuth token when the provider is actually Anthropic.
let oauth_token = if canonical_id == "anthropic" {
optional_env("ANTHROPIC_OAUTH_TOKEN")?.map(SecretString::from)
} else {
None
};
let api_key = if api_key.is_none() && oauth_token.is_some() {
// OAuth token present but no API key: use a placeholder so the
// config block is populated. The provider factory will route to
// the OAuth provider instead of rig-core's x-api-key client.
Some(SecretString::from(OAUTH_PLACEHOLDER.to_string()))
} else {
api_key
};
Ok(RegistryProviderConfig {
protocol,
provider_id: canonical_id.to_string(),
@@ -466,7 +373,6 @@ impl LlmConfig {
base_url,
model,
extra_headers,
oauth_token,
})
}
}
@@ -771,6 +677,8 @@ mod tests {
#[test]
fn backend_alias_normalized_to_canonical_id() {
// When the user sets LLM_BACKEND to an alias (e.g., "open_ai"),
// LlmConfig.backend should resolve to the canonical ID ("openai").
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_openai_compatible_env();
// SAFETY: Under ENV_MUTEX.
@@ -797,6 +705,8 @@ mod tests {
#[test]
fn unknown_backend_falls_back_to_openai_compatible() {
// An unrecognized LLM_BACKEND should fall back to the openai_compatible
// provider definition instead of erroring.
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_openai_compatible_env();
// SAFETY: Under ENV_MUTEX.
@@ -807,6 +717,7 @@ mod tests {
let settings = Settings::default();
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
// Falls back to openai_compatible since "some_custom_provider" is unknown
assert_eq!(cfg.backend, "openai_compatible");
let provider = cfg.provider.expect("should have provider config");
assert_eq!(provider.provider_id, "openai_compatible");
@@ -848,6 +759,7 @@ mod tests {
#[test]
fn base_url_resolution_priority() {
// Env var > settings > registry default
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_openai_compatible_env();
@@ -888,119 +800,6 @@ mod tests {
}
}
// ── OAuth resolution tests ──────────────────────────────────────
/// Clear all Anthropic-related env vars.
fn clear_anthropic_env() {
// SAFETY: Only called under ENV_MUTEX in tests.
unsafe {
std::env::remove_var("LLM_BACKEND");
std::env::remove_var("ANTHROPIC_API_KEY");
std::env::remove_var("ANTHROPIC_OAUTH_TOKEN");
std::env::remove_var("ANTHROPIC_MODEL");
std::env::remove_var("ANTHROPIC_BASE_URL");
}
}
#[test]
fn anthropic_oauth_token_sets_placeholder_api_key() {
use secrecy::ExposeSecret;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_anthropic_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("ANTHROPIC_OAUTH_TOKEN", "sk-ant-oat01-test-token");
}
let settings = Settings {
llm_backend: Some("anthropic".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let provider = cfg.provider.expect("provider config should be present");
assert_eq!(
provider
.api_key
.as_ref()
.map(|k| k.expose_secret().to_string()),
Some(OAUTH_PLACEHOLDER.to_string()),
"api_key should be the OAuth placeholder when only OAuth token is set"
);
assert!(
provider.oauth_token.is_some(),
"oauth_token should be populated"
);
assert_eq!(
provider.oauth_token.as_ref().unwrap().expose_secret(),
"sk-ant-oat01-test-token"
);
clear_anthropic_env();
}
#[test]
fn anthropic_api_key_takes_priority_over_oauth() {
use secrecy::ExposeSecret;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_anthropic_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("ANTHROPIC_API_KEY", "sk-ant-real-key");
std::env::set_var("ANTHROPIC_OAUTH_TOKEN", "sk-ant-oat01-test-token");
}
let settings = Settings {
llm_backend: Some("anthropic".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let provider = cfg.provider.expect("provider config should be present");
assert_eq!(
provider
.api_key
.as_ref()
.map(|k| k.expose_secret().to_string()),
Some("sk-ant-real-key".to_string()),
"real API key should take priority over OAuth placeholder"
);
assert!(
provider.oauth_token.is_some(),
"oauth_token should still be populated"
);
clear_anthropic_env();
}
#[test]
fn non_anthropic_provider_has_no_oauth_token() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_anthropic_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("ANTHROPIC_OAUTH_TOKEN", "sk-ant-oat01-test-token");
}
let settings = Settings {
llm_backend: Some("openai".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let provider = cfg.provider.expect("provider config should be present");
assert!(
provider.oauth_token.is_none(),
"non-Anthropic providers should not pick up ANTHROPIC_OAUTH_TOKEN"
);
clear_anthropic_env();
}
// ── Cache retention tests ───────────────────────────────────────
#[test]
fn cache_retention_from_str_primary_values() {
assert_eq!(
@@ -1082,30 +881,4 @@ mod tests {
assert_eq!(parsed, variant, "round-trip failed for {s}");
}
}
#[test]
fn test_request_timeout_defaults_to_120() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::remove_var("LLM_REQUEST_TIMEOUT_SECS");
}
let config = LlmConfig::resolve(&Settings::default()).expect("resolve");
assert_eq!(config.request_timeout_secs, 120);
}
#[test]
fn test_request_timeout_configurable() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("LLM_REQUEST_TIMEOUT_SECS", "300");
}
let config = LlmConfig::resolve(&Settings::default()).expect("resolve");
assert_eq!(config.request_timeout_secs, 300);
// SAFETY: Cleanup
unsafe {
std::env::remove_var("LLM_REQUEST_TIMEOUT_SECS");
}
}
}
+6 -102
View File
@@ -13,7 +13,7 @@ mod embeddings;
mod heartbeat;
pub(crate) mod helpers;
mod hygiene;
pub(crate) mod llm;
mod llm;
mod routines;
mod safety;
mod sandbox;
@@ -24,7 +24,7 @@ mod tunnel;
mod wasm;
use std::collections::HashMap;
use std::sync::{LazyLock, Mutex};
use std::sync::OnceLock;
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -37,9 +37,7 @@ pub use self::database::{DatabaseBackend, DatabaseConfig, SslMode, default_libsq
pub use self::embeddings::EmbeddingsConfig;
pub use self::heartbeat::HeartbeatConfig;
pub use self::hygiene::HygieneConfig;
pub use self::llm::{
BedrockConfig, CacheRetention, LlmConfig, NearAiConfig, RegistryProviderConfig,
};
pub use self::llm::{CacheRetention, LlmConfig, NearAiConfig, RegistryProviderConfig};
pub use self::routines::RoutineConfig;
pub use self::safety::SafetyConfig;
pub use self::sandbox::{ClaudeCodeConfig, SandboxModeConfig};
@@ -55,12 +53,7 @@ pub use crate::llm::session::SessionConfig;
/// Used by `inject_llm_keys_from_secrets()` to make API keys available to
/// `optional_env()` without unsafe `set_var` calls. `optional_env()` checks
/// real env vars first, then falls back to this overlay.
///
/// Uses `Mutex<HashMap>` instead of `OnceLock` so that both
/// `inject_os_credentials()` and `inject_llm_keys_from_secrets()` can merge
/// their data. Whichever runs first initialises the map; the second merges in.
static INJECTED_VARS: LazyLock<Mutex<HashMap<String, String>>> =
LazyLock::new(|| Mutex::new(HashMap::new()));
static INJECTED_VARS: OnceLock<HashMap<String, String>> = OnceLock::new();
/// Main configuration for the agent.
#[derive(Debug, Clone)]
@@ -259,32 +252,6 @@ impl Config {
Ok(())
}
/// Re-resolve only the LLM config after credential injection.
///
/// Called by `AppBuilder::init_secrets()` after injecting API keys into
/// the env overlay. Only rebuilds `self.llm` — all other config fields
/// are unaffected, preserving values from the initial config load (or
/// from `Config::for_testing()` in test mode).
pub async fn re_resolve_llm(
&mut self,
store: Option<&(dyn crate::db::SettingsStore + Sync)>,
user_id: &str,
toml_path: Option<&std::path::Path>,
) -> Result<(), ConfigError> {
let settings = if let Some(store) = store {
let mut s = match store.get_all_settings(user_id).await {
Ok(map) => Settings::from_db_map(&map),
Err(_) => Settings::default(),
};
Self::apply_toml_overlay(&mut s, toml_path)?;
s
} else {
Settings::default()
};
self.llm = LlmConfig::resolve(&settings)?;
Ok(())
}
/// Build config from settings (shared by from_env and from_db).
async fn build(settings: &Settings) -> Result<Self, ConfigError> {
Ok(Self {
@@ -318,9 +285,6 @@ impl Config {
/// env-var-first resolution in `LlmConfig::resolve()`. Keys in the overlay
/// are read by `optional_env()` before falling back to `std::env::var()`,
/// so explicit env vars always win.
///
/// Also loads tokens from OS credential stores (macOS Keychain, Linux
/// credentials files) which don't require the secrets DB.
pub async fn inject_llm_keys_from_secrets(
secrets: &dyn crate::secrets::SecretsStore,
user_id: &str,
@@ -328,10 +292,7 @@ pub async fn inject_llm_keys_from_secrets(
// Static mappings for well-known providers.
// The registry's setup hints define secret_name -> env_var mappings,
// so new providers added to providers.json get injection automatically.
let mut mappings: Vec<(&str, &str)> = vec![
("llm_nearai_api_key", "NEARAI_API_KEY"),
("llm_anthropic_oauth_token", "ANTHROPIC_OAUTH_TOKEN"),
];
let mut mappings: Vec<(&str, &str)> = vec![("llm_nearai_api_key", "NEARAI_API_KEY")];
// Dynamically discover secret->env mappings from the provider registry.
// Uses selectable() which deduplicates user overrides correctly.
@@ -370,62 +331,5 @@ pub async fn inject_llm_keys_from_secrets(
}
}
inject_os_credential_store_tokens(&mut injected);
merge_injected_vars(injected);
}
/// Load tokens from OS credential stores (no DB required).
///
/// Called unconditionally during startup — even when the encrypted secrets DB
/// is unavailable (no master key, no DB connection). This ensures OAuth tokens
/// from `claude login` (macOS Keychain / Linux credentials.json)
/// are available for config resolution.
pub fn inject_os_credentials() {
let mut injected = HashMap::new();
inject_os_credential_store_tokens(&mut injected);
merge_injected_vars(injected);
}
/// Merge new entries into the global injected-vars overlay.
///
/// New keys are inserted; existing keys are overwritten (later callers win,
/// e.g. fresh OS credential store tokens override stale DB copies).
fn merge_injected_vars(new_entries: HashMap<String, String>) {
if new_entries.is_empty() {
return;
}
match INJECTED_VARS.lock() {
Ok(mut map) => map.extend(new_entries),
Err(poisoned) => poisoned.into_inner().extend(new_entries),
}
}
/// Inject a single key-value pair into the overlay.
///
/// Used by the setup wizard to make credentials available to `optional_env()`
/// without calling `unsafe { std::env::set_var }`.
pub fn inject_single_var(key: &str, value: &str) {
match INJECTED_VARS.lock() {
Ok(mut map) => {
map.insert(key.to_string(), value.to_string());
}
Err(poisoned) => {
poisoned
.into_inner()
.insert(key.to_string(), value.to_string());
}
}
}
/// Shared helper: extract tokens from OS credential stores into the overlay map.
fn inject_os_credential_store_tokens(injected: &mut HashMap<String, String>) {
// Try the OS credential store for a fresh Anthropic OAuth token.
// Tokens from `claude login` expire in 8-12h, so the DB copy may be stale.
// A fresh extraction from macOS Keychain / Linux credentials.json wins
// over the (possibly expired) copy stored in the encrypted secrets DB.
if let Some(fresh) = crate::config::ClaudeCodeConfig::extract_oauth_token() {
injected.insert("ANTHROPIC_OAUTH_TOKEN".to_string(), fresh);
tracing::debug!("Refreshed ANTHROPIC_OAUTH_TOKEN from OS credential store");
}
let _ = INJECTED_VARS.set(injected);
}
+5 -16
View File
@@ -233,14 +233,9 @@ impl ClaudeCodeConfig {
/// Expected shape: `{"claudeAiOauth": {"accessToken": "sk-ant-oat01-..."}}`
fn parse_oauth_access_token(json: &str) -> Option<String> {
let creds: serde_json::Value = serde_json::from_str(json).ok()?;
let token = creds["claudeAiOauth"]["accessToken"].as_str()?;
// Validate that the token looks like a real OAuth token before using it.
// Claude CLI tokens start with "sk-ant-oat".
if !token.starts_with("sk-ant-oat") {
tracing::debug!("Ignoring credential store token with unexpected prefix");
return None;
}
Some(token.to_string())
creds["claudeAiOauth"]["accessToken"]
.as_str()
.map(String::from)
}
#[cfg(test)]
@@ -406,14 +401,14 @@ mod tests {
fn parse_oauth_token_nested_extra_fields() {
let json = r#"{
"claudeAiOauth": {
"accessToken": "sk-ant-oat01-real-token",
"accessToken": "sk-ant-real-token",
"refreshToken": "rt-abc",
"expiresAt": 1700000000
}
}"#;
assert_eq!(
parse_oauth_access_token(json),
Some("sk-ant-oat01-real-token".to_string())
Some("sk-ant-real-token".to_string())
);
}
@@ -423,12 +418,6 @@ mod tests {
assert_eq!(parse_oauth_access_token(json), None);
}
#[test]
fn parse_oauth_token_rejects_invalid_prefix() {
let json = r#"{"claudeAiOauth": {"accessToken": "not-an-oauth-token"}}"#;
assert_eq!(parse_oauth_access_token(json), None);
}
// ── default_claude_code_allowed_tools ───────────────────────────
#[test]
-9
View File
@@ -164,8 +164,6 @@ pub struct JobContext {
/// previous results by ID via `$tool_call_id` parameter syntax.
#[serde(skip)]
pub tool_output_stash: Arc<tokio::sync::RwLock<HashMap<String, String>>>,
/// User's preferred timezone (IANA name, e.g. "America/New_York"). Defaults to "UTC".
pub user_timezone: String,
}
impl JobContext {
@@ -205,16 +203,9 @@ impl JobContext {
http_interceptor: None,
metadata: serde_json::Value::Null,
tool_output_stash: Arc::new(tokio::sync::RwLock::new(HashMap::new())),
user_timezone: "UTC".to_string(),
}
}
/// Set the user timezone on this context.
pub fn with_timezone(mut self, tz: impl Into<String>) -> Self {
self.user_timezone = tz.into();
self
}
/// Transition to a new state.
pub fn transition_to(
&mut self,
+11 -346
View File
@@ -20,10 +20,9 @@ impl ConversationStore for LibSqlBackend {
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
conn.execute(
"INSERT INTO conversations (id, channel, user_id, thread_id, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), channel, user_id, opt_text(thread_id), now],
"INSERT INTO conversations (id, channel, user_id, thread_id) VALUES (?1, ?2, ?3, ?4)",
params![id.to_string(), channel, user_id, opt_text(thread_id)],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
@@ -72,8 +71,8 @@ impl ConversationStore for LibSqlBackend {
let now = fmt_ts(&Utc::now());
conn.execute(
r#"
INSERT INTO conversations (id, channel, user_id, thread_id, started_at, last_activity)
VALUES (?1, ?2, ?3, ?4, ?5, ?5)
INSERT INTO conversations (id, channel, user_id, thread_id)
VALUES (?1, ?2, ?3, ?4)
ON CONFLICT (id) DO UPDATE SET last_activity = ?5
"#,
params![id.to_string(), channel, user_id, opt_text(thread_id), now],
@@ -98,7 +97,6 @@ impl ConversationStore for LibSqlBackend {
c.started_at,
c.last_activity,
c.metadata,
c.channel,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT substr(m2.content, 1, 100)
FROM conversation_messages m2
@@ -108,7 +106,7 @@ impl ConversationStore for LibSqlBackend {
) AS title
FROM conversations c
WHERE c.user_id = ?1 AND c.channel = ?2
ORDER BY datetime(c.last_activity) DESC
ORDER BY c.last_activity DESC
LIMIT ?3
"#,
params![user_id, channel, limit],
@@ -127,13 +125,6 @@ impl ConversationStore for LibSqlBackend {
.get("thread_type")
.and_then(|v| v.as_str())
.map(String::from);
let sql_title = get_opt_text(&row, 6);
let title = sql_title.or_else(|| {
metadata
.get("routine_name")
.and_then(|v| v.as_str())
.map(String::from)
});
results.push(ConversationSummary {
id: row
.get::<String>(0)
@@ -142,213 +133,14 @@ impl ConversationStore for LibSqlBackend {
.unwrap_or_default(),
started_at: get_ts(&row, 1),
last_activity: get_ts(&row, 2),
message_count: get_i64(&row, 5),
title,
message_count: get_i64(&row, 4),
title: get_opt_text(&row, 5),
thread_type,
channel: get_text(&row, 4),
});
}
Ok(results)
}
async fn list_conversations_all_channels(
&self,
user_id: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
r#"
SELECT
c.id,
c.started_at,
c.last_activity,
c.metadata,
c.channel,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT substr(m2.content, 1, 100)
FROM conversation_messages m2
WHERE m2.conversation_id = c.id AND m2.role = 'user'
ORDER BY m2.created_at ASC, m2.rowid ASC
LIMIT 1
) AS title
FROM conversations c
WHERE c.user_id = ?1
ORDER BY datetime(c.last_activity) DESC
LIMIT ?2
"#,
params![user_id, limit],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let mut results = Vec::new();
while let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let metadata = get_json(&row, 3);
let thread_type = metadata
.get("thread_type")
.and_then(|v| v.as_str())
.map(String::from);
let sql_title = get_opt_text(&row, 6);
let title = sql_title.or_else(|| {
metadata
.get("routine_name")
.and_then(|v| v.as_str())
.map(String::from)
});
results.push(ConversationSummary {
id: row
.get::<String>(0)
.unwrap_or_default()
.parse()
.unwrap_or_default(),
started_at: get_ts(&row, 1),
last_activity: get_ts(&row, 2),
message_count: get_i64(&row, 5),
title,
thread_type,
channel: get_text(&row, 4),
});
}
Ok(results)
}
/// Uses BEGIN IMMEDIATE to serialize concurrent writers and prevent
/// duplicate routine conversations (TOCTOU race).
async fn get_or_create_routine_conversation(
&self,
routine_id: Uuid,
routine_name: &str,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
let rid = routine_id.to_string();
conn.execute("BEGIN IMMEDIATE", params![])
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let result: Result<Uuid, DatabaseError> = async {
let mut rows = conn
.query(
r#"
SELECT id FROM conversations
WHERE user_id = ?1 AND json_extract(metadata, '$.routine_id') = ?2
LIMIT 1
"#,
params![user_id, rid],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
if let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let id_str: String = row.get(0).unwrap_or_default();
return id_str
.parse()
.map_err(|_| DatabaseError::Serialization("Invalid UUID".to_string()));
}
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
let metadata = serde_json::json!({
"thread_type": "routine",
"routine_id": routine_id.to_string(),
"routine_name": routine_name,
});
conn.execute(
"INSERT INTO conversations (id, channel, user_id, metadata, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), "routine", user_id, metadata.to_string(), now],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
Ok(id)
}
.await;
match &result {
Ok(_) => {
conn.execute("COMMIT", params![])
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
}
Err(_) => {
let _ = conn.execute("ROLLBACK", params![]).await;
}
}
result
}
/// Uses BEGIN IMMEDIATE to serialize concurrent writers and prevent
/// duplicate heartbeat conversations (TOCTOU race).
async fn get_or_create_heartbeat_conversation(
&self,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
conn.execute("BEGIN IMMEDIATE", params![])
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let result: Result<Uuid, DatabaseError> = async {
let mut rows = conn
.query(
r#"
SELECT id FROM conversations
WHERE user_id = ?1 AND json_extract(metadata, '$.thread_type') = 'heartbeat'
LIMIT 1
"#,
params![user_id],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
if let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let id_str: String = row.get(0).unwrap_or_default();
return id_str
.parse()
.map_err(|_| DatabaseError::Serialization("Invalid UUID".to_string()));
}
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
let metadata = serde_json::json!({ "thread_type": "heartbeat" });
conn.execute(
"INSERT INTO conversations (id, channel, user_id, metadata, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), "heartbeat", user_id, metadata.to_string(), now],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
Ok(id)
}
.await;
match &result {
Ok(_) => {
conn.execute("COMMIT", params![])
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
}
Err(_) => {
let _ = conn.execute("ROLLBACK", params![]).await;
}
}
result
}
async fn get_or_create_assistant_conversation(
&self,
user_id: &str,
@@ -382,11 +174,10 @@ impl ConversationStore for LibSqlBackend {
// Create new
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
let metadata = serde_json::json!({"thread_type": "assistant", "title": "Assistant"});
conn.execute(
"INSERT INTO conversations (id, channel, user_id, metadata, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), channel, user_id, metadata.to_string(), now],
"INSERT INTO conversations (id, channel, user_id, metadata) VALUES (?1, ?2, ?3, ?4)",
params![id.to_string(), channel, user_id, metadata.to_string()],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
@@ -401,10 +192,9 @@ impl ConversationStore for LibSqlBackend {
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
conn.execute(
"INSERT INTO conversations (id, channel, user_id, metadata, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), channel, user_id, metadata.to_string(), now],
"INSERT INTO conversations (id, channel, user_id, metadata) VALUES (?1, ?2, ?3, ?4)",
params![id.to_string(), channel, user_id, metadata.to_string()],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
@@ -563,128 +353,3 @@ impl ConversationStore for LibSqlBackend {
Ok(found.is_some())
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::db::Database;
#[tokio::test]
async fn test_get_or_create_routine_conversation_is_idempotent() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_routine_conv.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
let routine_id = Uuid::new_v4();
let user_id = "test_user";
// First call — creates the conversation
let id1 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
// Second call — should return the SAME conversation
let id2 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
assert_eq!(id1, id2, "Expected same conversation ID on repeated calls");
// Third call — still the same
let id3 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
assert_eq!(id1, id3);
// Different routine_id should get a different conversation
let other_routine_id = Uuid::new_v4();
let id4 = backend
.get_or_create_routine_conversation(other_routine_id, "other-routine", user_id)
.await
.unwrap();
assert_ne!(
id1, id4,
"Different routines should get different conversations"
);
}
#[tokio::test]
async fn test_routine_conversation_persists_across_messages() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_routine_persist.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
let routine_id = Uuid::new_v4();
let user_id = "test_user";
// First invocation: create conversation and add a message
let id1 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
backend
.add_conversation_message(id1, "assistant", "[cron] Completed: all good")
.await
.unwrap();
// Second invocation: should find existing conversation
let id2 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
assert_eq!(id1, id2, "Second invocation should reuse same conversation");
backend
.add_conversation_message(id2, "assistant", "[cron] Completed: still good")
.await
.unwrap();
// Verify only one routine conversation exists (not two)
let convs = backend
.list_conversations_all_channels(user_id, 50)
.await
.unwrap();
let routine_convs: Vec<_> = convs.iter().filter(|c| c.channel == "routine").collect();
assert_eq!(
routine_convs.len(),
1,
"Should have exactly 1 routine conversation, found {}",
routine_convs.len()
);
}
#[tokio::test]
async fn test_get_or_create_heartbeat_conversation_is_idempotent() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_heartbeat_conv.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
let user_id = "test_user";
let id1 = backend
.get_or_create_heartbeat_conversation(user_id)
.await
.unwrap();
let id2 = backend
.get_or_create_heartbeat_conversation(user_id)
.await
.unwrap();
assert_eq!(
id1, id2,
"Expected same heartbeat conversation on repeated calls"
);
}
}
-3
View File
@@ -121,9 +121,6 @@ impl JobStore for LibSqlBackend {
tool_output_stash: std::sync::Arc::new(tokio::sync::RwLock::new(
std::collections::HashMap::new(),
)),
// TODO(#661): persist user_timezone in agent_jobs table so
// background/routine jobs retain the session's timezone context.
user_timezone: "UTC".to_string(),
}))
}
None => Ok(None),
+9 -107
View File
@@ -118,37 +118,15 @@ impl LibSqlBackend {
/// Sets `PRAGMA busy_timeout = 5000` on every connection so concurrent
/// writers wait up to 5 seconds instead of failing instantly with
/// "database is locked".
///
/// Retries up to 3 times with exponential backoff to handle transient
/// "unable to open database file" errors from concurrent connection
/// creation (e.g. cron ticker vs main thread).
pub async fn connect(&self) -> Result<Connection, DatabaseError> {
let mut last_err = None;
for attempt in 0..3u32 {
match self.db.connect() {
Ok(conn) => {
conn.query("PRAGMA busy_timeout = 5000", ())
.await
.map_err(|e| {
DatabaseError::Pool(format!("Failed to set busy_timeout: {}", e))
})?;
return Ok(conn);
}
Err(e) => {
last_err = Some(e);
if attempt < 2 {
tokio::time::sleep(std::time::Duration::from_millis(
50 * 2u64.pow(attempt),
))
.await;
}
}
}
}
Err(DatabaseError::Pool(format!(
"Failed to create connection after 3 attempts: {}",
last_err.map(|e| e.to_string()).unwrap_or_default()
)))
let conn = self
.db
.connect()
.map_err(|e| DatabaseError::Pool(format!("Failed to create connection: {}", e)))?;
conn.query("PRAGMA busy_timeout = 5000", ())
.await
.map_err(|e| DatabaseError::Pool(format!("Failed to set busy_timeout: {}", e)))?;
Ok(conn)
}
}
@@ -169,18 +147,10 @@ pub(crate) fn parse_timestamp(s: &str) -> Result<DateTime<Utc>, String> {
}
// Naive with fractional seconds (legacy or SQLite datetime() output)
if let Ok(ndt) = NaiveDateTime::parse_from_str(s, "%Y-%m-%d %H:%M:%S%.f") {
tracing::warn!(
timestamp = %s,
"parsed naive timestamp without timezone; assuming UTC for backward compatibility"
);
return Ok(ndt.and_utc());
}
// Naive without fractional seconds (legacy format)
if let Ok(ndt) = NaiveDateTime::parse_from_str(s, "%Y-%m-%d %H:%M:%S") {
tracing::warn!(
timestamp = %s,
"parsed naive timestamp without timezone; assuming UTC for backward compatibility"
);
return Ok(ndt.and_utc());
}
Err(format!("unparseable timestamp: {:?}", s))
@@ -410,47 +380,8 @@ pub(crate) fn row_to_routine_run_libsql(row: &libsql::Row) -> Result<RoutineRun,
#[cfg(test)]
mod tests {
use chrono::{TimeZone, Utc};
use crate::db::Database;
use crate::db::libsql::{LibSqlBackend, parse_timestamp};
#[test]
fn test_parse_timestamp_accepts_rfc3339_and_legacy_naive_formats() {
let expected = Utc.with_ymd_and_hms(2026, 3, 7, 12, 34, 56).unwrap();
let with_millis = parse_timestamp("2026-03-07T12:34:56.789Z").unwrap();
assert_eq!(with_millis, expected + chrono::Duration::milliseconds(789));
let naive_with_millis = parse_timestamp("2026-03-07 12:34:56.789").unwrap();
assert_eq!(
naive_with_millis,
expected + chrono::Duration::milliseconds(789)
);
let naive_without_millis = parse_timestamp("2026-03-07 12:34:56").unwrap();
assert_eq!(naive_without_millis, expected);
}
#[tokio::test]
async fn test_libsql_now_format_is_rfc3339_and_parseable() {
let backend = LibSqlBackend::new_memory().await.unwrap();
backend.run_migrations().await.unwrap();
let conn = backend.connect().await.unwrap();
let mut rows = conn
.query("SELECT strftime('%Y-%m-%dT%H:%M:%fZ', 'now')", ())
.await
.unwrap();
let row = rows.next().await.unwrap().unwrap();
let ts: String = row.get(0).unwrap();
let parsed = parse_timestamp(&ts).unwrap();
assert_eq!(
ts,
parsed.to_rfc3339_opts(chrono::SecondsFormat::Millis, true)
);
}
use crate::db::libsql::LibSqlBackend;
#[tokio::test]
async fn test_wal_mode_after_migrations() {
@@ -528,33 +459,4 @@ mod tests {
let count: i64 = row.get(0).unwrap();
assert_eq!(count, 20);
}
#[tokio::test]
async fn test_connect_retry_succeeds_on_valid_db() {
// Verify connect() works with retry logic on a file-backed DB
// (exercises the retry path even though transient failures are hard
// to reproduce deterministically).
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_retry.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
// Multiple concurrent connect() calls should all succeed
let mut handles = Vec::new();
for _ in 0..10 {
let b = LibSqlBackend {
db: backend.shared_db(),
};
handles.push(tokio::spawn(async move { b.connect().await }));
}
for handle in handles {
let result = handle.await.unwrap();
assert!(
result.is_ok(),
"concurrent connect failed: {:?}",
result.err()
);
}
}
}
+55 -64
View File
@@ -26,7 +26,7 @@ pub const SCHEMA: &str = r#"
CREATE TABLE IF NOT EXISTS _migrations (
version INTEGER PRIMARY KEY,
name TEXT NOT NULL,
applied_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
applied_at TEXT NOT NULL DEFAULT (datetime('now'))
);
-- ==================== Conversations ====================
@@ -36,8 +36,8 @@ CREATE TABLE IF NOT EXISTS conversations (
channel TEXT NOT NULL,
user_id TEXT NOT NULL,
thread_id TEXT,
started_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
last_activity TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
started_at TEXT NOT NULL DEFAULT (datetime('now')),
last_activity TEXT NOT NULL DEFAULT (datetime('now')),
metadata TEXT NOT NULL DEFAULT '{}'
);
@@ -45,21 +45,12 @@ CREATE INDEX IF NOT EXISTS idx_conversations_channel ON conversations(channel);
CREATE INDEX IF NOT EXISTS idx_conversations_user ON conversations(user_id);
CREATE INDEX IF NOT EXISTS idx_conversations_last_activity ON conversations(last_activity);
-- Partial unique indexes to prevent duplicate singleton conversations.
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_routine
ON conversations (user_id, json_extract(metadata, '$.routine_id'))
WHERE json_extract(metadata, '$.routine_id') IS NOT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_heartbeat
ON conversations (user_id)
WHERE json_extract(metadata, '$.thread_type') = 'heartbeat';
CREATE TABLE IF NOT EXISTS conversation_messages (
id TEXT PRIMARY KEY,
conversation_id TEXT NOT NULL REFERENCES conversations(id) ON DELETE CASCADE,
role TEXT NOT NULL,
content TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_conversation_messages_conversation
@@ -91,7 +82,7 @@ CREATE TABLE IF NOT EXISTS agent_jobs (
failure_reason TEXT,
stuck_since TEXT,
repair_attempts INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
started_at TEXT,
completed_at TEXT
);
@@ -116,7 +107,7 @@ CREATE TABLE IF NOT EXISTS job_actions (
duration_ms INTEGER,
success INTEGER NOT NULL,
error_message TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE(job_id, sequence_num)
);
@@ -137,8 +128,8 @@ CREATE TABLE IF NOT EXISTS dynamic_tools (
failure_count INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
status TEXT NOT NULL DEFAULT 'active',
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_dynamic_tools_status ON dynamic_tools(status);
@@ -156,7 +147,7 @@ CREATE TABLE IF NOT EXISTS llm_calls (
output_tokens INTEGER NOT NULL,
cost TEXT NOT NULL,
purpose TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_llm_calls_job ON llm_calls(job_id);
@@ -176,7 +167,7 @@ CREATE TABLE IF NOT EXISTS estimation_snapshots (
actual_time_secs INTEGER,
estimated_value TEXT NOT NULL,
actual_value TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_estimation_category ON estimation_snapshots(category);
@@ -192,7 +183,7 @@ CREATE TABLE IF NOT EXISTS repair_attempts (
action_taken TEXT NOT NULL,
success INTEGER NOT NULL,
error_message TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_repair_attempts_target ON repair_attempts(target_type, target_id);
@@ -206,8 +197,8 @@ CREATE TABLE IF NOT EXISTS memory_documents (
agent_id TEXT,
path TEXT NOT NULL,
content TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
metadata TEXT NOT NULL DEFAULT '{}',
UNIQUE (user_id, agent_id, path)
);
@@ -222,7 +213,7 @@ CREATE TRIGGER IF NOT EXISTS update_memory_documents_updated_at
FOR EACH ROW
WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE memory_documents SET updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now') WHERE id = NEW.id;
UPDATE memory_documents SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- ==================== Workspace: Memory Chunks ====================
@@ -234,7 +225,7 @@ CREATE TABLE IF NOT EXISTS memory_chunks (
chunk_index INTEGER NOT NULL,
content TEXT NOT NULL,
embedding BLOB,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (document_id, chunk_index)
);
@@ -296,8 +287,8 @@ CREATE TABLE IF NOT EXISTS secrets (
expires_at TEXT,
last_used_at TEXT,
usage_count INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (user_id, name)
);
@@ -318,8 +309,8 @@ CREATE TABLE IF NOT EXISTS wasm_tools (
source_url TEXT,
trust_level TEXT NOT NULL DEFAULT 'user',
status TEXT NOT NULL DEFAULT 'active',
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (user_id, name, version)
);
@@ -340,8 +331,8 @@ CREATE TABLE IF NOT EXISTS wasm_channels (
binary_hash BLOB NOT NULL,
capabilities_json TEXT NOT NULL DEFAULT '{}',
status TEXT NOT NULL DEFAULT 'active',
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (user_id, name)
);
@@ -359,8 +350,8 @@ CREATE TABLE IF NOT EXISTS tool_capabilities (
max_response_body_bytes INTEGER NOT NULL DEFAULT 10485760,
workspace_read_prefixes TEXT NOT NULL DEFAULT '[]',
http_timeout_secs INTEGER NOT NULL DEFAULT 30,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (wasm_tool_id)
);
@@ -373,7 +364,7 @@ CREATE TABLE IF NOT EXISTS leak_detection_patterns (
severity TEXT NOT NULL DEFAULT 'high',
action TEXT NOT NULL DEFAULT 'block',
enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
-- ==================== Rate Limit State ====================
@@ -382,9 +373,9 @@ CREATE TABLE IF NOT EXISTS tool_rate_limit_state (
id TEXT PRIMARY KEY,
wasm_tool_id TEXT NOT NULL REFERENCES wasm_tools(id) ON DELETE CASCADE,
user_id TEXT NOT NULL,
minute_window_start TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
minute_window_start TEXT NOT NULL DEFAULT (datetime('now')),
minute_count INTEGER NOT NULL DEFAULT 0,
hour_window_start TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
hour_window_start TEXT NOT NULL DEFAULT (datetime('now')),
hour_count INTEGER NOT NULL DEFAULT 0,
UNIQUE (wasm_tool_id, user_id)
);
@@ -400,7 +391,7 @@ CREATE TABLE IF NOT EXISTS secret_usage_log (
target_path TEXT,
success INTEGER NOT NULL,
error_message TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_secret_usage_user ON secret_usage_log(user_id);
@@ -415,7 +406,7 @@ CREATE TABLE IF NOT EXISTS leak_detection_events (
source TEXT NOT NULL,
action_taken TEXT NOT NULL,
context_preview TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
-- ==================== Tool Failures ====================
@@ -425,8 +416,8 @@ CREATE TABLE IF NOT EXISTS tool_failures (
tool_name TEXT NOT NULL UNIQUE,
error_message TEXT,
error_count INTEGER DEFAULT 1,
first_failure TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
last_failure TEXT DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
first_failure TEXT DEFAULT (datetime('now')),
last_failure TEXT DEFAULT (datetime('now')),
last_build_result TEXT,
repaired_at TEXT,
repair_attempts INTEGER DEFAULT 0
@@ -441,7 +432,7 @@ CREATE TABLE IF NOT EXISTS job_events (
job_id TEXT NOT NULL REFERENCES agent_jobs(id),
event_type TEXT NOT NULL,
data TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_job_events_job ON job_events(job_id, id);
@@ -471,8 +462,8 @@ CREATE TABLE IF NOT EXISTS routines (
next_fire_at TEXT,
run_count INTEGER NOT NULL DEFAULT 0,
consecutive_failures INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (user_id, name)
);
@@ -485,13 +476,13 @@ CREATE TABLE IF NOT EXISTS routine_runs (
routine_id TEXT NOT NULL REFERENCES routines(id) ON DELETE CASCADE,
trigger_type TEXT NOT NULL,
trigger_detail TEXT,
started_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
started_at TEXT NOT NULL DEFAULT (datetime('now')),
completed_at TEXT,
status TEXT NOT NULL DEFAULT 'running',
result_summary TEXT,
tokens_used INTEGER,
job_id TEXT REFERENCES agent_jobs(id),
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_routine_runs_routine ON routine_runs(routine_id);
@@ -502,7 +493,7 @@ CREATE TABLE IF NOT EXISTS settings (
user_id TEXT NOT NULL,
key TEXT NOT NULL,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (user_id, key)
);
@@ -558,24 +549,24 @@ CREATE INDEX IF NOT EXISTS idx_heartbeat_next_run ON heartbeat_state(next_run);
-- Pre-populate leak detection patterns (matches PostgreSQL V2 migration).
INSERT OR IGNORE INTO leak_detection_patterns (id, name, pattern, severity, action, enabled, created_at) VALUES
('550e8400-e29b-41d4-a716-446655440001', 'openai_api_key', 'sk-(?:proj-)?[a-zA-Z0-9]{20,}(?:T3BlbkFJ[a-zA-Z0-9_-]*)?', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440002', 'anthropic_api_key', 'sk-ant-api[a-zA-Z0-9_-]{90,}', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440003', 'aws_access_key', 'AKIA[0-9A-Z]{16}', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440004', 'aws_secret_key', '(?<![A-Za-z0-9/+=])[A-Za-z0-9/+=]{40}(?![A-Za-z0-9/+=])', 'high', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440005', 'github_token', 'gh[pousr]_[A-Za-z0-9_]{36,}', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440006', 'github_fine_grained_pat', 'github_pat_[a-zA-Z0-9]{22}_[a-zA-Z0-9]{59}', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440007', 'stripe_api_key', 'sk_(?:live|test)_[a-zA-Z0-9]{24,}', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440008', 'nearai_session', 'sess_[a-zA-Z0-9]{32,}', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440009', 'bearer_token', 'Bearer\s+[a-zA-Z0-9_-]{20,}', 'high', 'redact', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-44665544000a', 'pem_private_key', '-----BEGIN\s+(?:RSA\s+)?PRIVATE\s+KEY-----', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-44665544000b', 'ssh_private_key', '-----BEGIN\s+(?:OPENSSH|EC|DSA)\s+PRIVATE\s+KEY-----', 'critical', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-44665544000c', 'google_api_key', 'AIza[0-9A-Za-z_-]{35}', 'high', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-44665544000d', 'slack_token', 'xox[baprs]-[0-9a-zA-Z-]{10,}', 'high', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-44665544000e', 'discord_token', '[MN][A-Za-z\d]{23,}\.[\w-]{6}\.[\w-]{27}', 'high', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-44665544000f', 'twilio_api_key', 'SK[a-fA-F0-9]{32}', 'high', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440010', 'sendgrid_api_key', 'SG\.[a-zA-Z0-9_-]{22}\.[a-zA-Z0-9_-]{43}', 'high', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440011', 'mailchimp_api_key', '[a-f0-9]{32}-us[0-9]{1,2}', 'medium', 'block', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
('550e8400-e29b-41d4-a716-446655440012', 'high_entropy_hex', '(?<![a-fA-F0-9])[a-fA-F0-9]{64}(?![a-fA-F0-9])', 'medium', 'warn', 1, strftime('%Y-%m-%dT%H:%M:%fZ', 'now'));
('550e8400-e29b-41d4-a716-446655440001', 'openai_api_key', 'sk-(?:proj-)?[a-zA-Z0-9]{20,}(?:T3BlbkFJ[a-zA-Z0-9_-]*)?', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440002', 'anthropic_api_key', 'sk-ant-api[a-zA-Z0-9_-]{90,}', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440003', 'aws_access_key', 'AKIA[0-9A-Z]{16}', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440004', 'aws_secret_key', '(?<![A-Za-z0-9/+=])[A-Za-z0-9/+=]{40}(?![A-Za-z0-9/+=])', 'high', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440005', 'github_token', 'gh[pousr]_[A-Za-z0-9_]{36,}', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440006', 'github_fine_grained_pat', 'github_pat_[a-zA-Z0-9]{22}_[a-zA-Z0-9]{59}', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440007', 'stripe_api_key', 'sk_(?:live|test)_[a-zA-Z0-9]{24,}', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440008', 'nearai_session', 'sess_[a-zA-Z0-9]{32,}', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440009', 'bearer_token', 'Bearer\s+[a-zA-Z0-9_-]{20,}', 'high', 'redact', 1, datetime('now')),
('550e8400-e29b-41d4-a716-44665544000a', 'pem_private_key', '-----BEGIN\s+(?:RSA\s+)?PRIVATE\s+KEY-----', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-44665544000b', 'ssh_private_key', '-----BEGIN\s+(?:OPENSSH|EC|DSA)\s+PRIVATE\s+KEY-----', 'critical', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-44665544000c', 'google_api_key', 'AIza[0-9A-Za-z_-]{35}', 'high', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-44665544000d', 'slack_token', 'xox[baprs]-[0-9a-zA-Z-]{10,}', 'high', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-44665544000e', 'discord_token', '[MN][A-Za-z\d]{23,}\.[\w-]{6}\.[\w-]{27}', 'high', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-44665544000f', 'twilio_api_key', 'SK[a-fA-F0-9]{32}', 'high', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440010', 'sendgrid_api_key', 'SG\.[a-zA-Z0-9_-]{22}\.[a-zA-Z0-9_-]{43}', 'high', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440011', 'mailchimp_api_key', '[a-f0-9]{32}-us[0-9]{1,2}', 'medium', 'block', 1, datetime('now')),
('550e8400-e29b-41d4-a716-446655440012', 'high_entropy_hex', '(?<![a-fA-F0-9])[a-fA-F0-9]{64}(?![a-fA-F0-9])', 'medium', 'warn', 1, datetime('now'));
"#;
@@ -613,7 +604,7 @@ CREATE TABLE IF NOT EXISTS memory_chunks_new (
chunk_index INTEGER NOT NULL,
content TEXT NOT NULL,
embedding BLOB,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (document_id, chunk_index)
);
-116
View File
@@ -91,64 +91,6 @@ pub async fn connect_from_config(
}
}
/// Create a secrets store from database and secrets configuration.
///
/// This is the shared factory for CLI commands and other call sites that need
/// a `SecretsStore` without going through the full `AppBuilder`. Mirrors the
/// pattern of [`connect_from_config`] but returns a secrets-specific store.
pub async fn create_secrets_store(
config: &crate::config::DatabaseConfig,
crypto: Arc<crate::secrets::SecretsCrypto>,
) -> Result<Arc<dyn crate::secrets::SecretsStore + Send + Sync>, DatabaseError> {
match config.backend {
#[cfg(feature = "libsql")]
crate::config::DatabaseBackend::LibSql => {
use secrecy::ExposeSecret as _;
let default_path = crate::config::default_libsql_path();
let db_path = config.libsql_path.as_deref().unwrap_or(&default_path);
let backend = if let Some(ref url) = config.libsql_url {
let token = config.libsql_auth_token.as_ref().ok_or_else(|| {
DatabaseError::Pool(
"LIBSQL_AUTH_TOKEN required when LIBSQL_URL is set".to_string(),
)
})?;
libsql::LibSqlBackend::new_remote_replica(db_path, url, token.expose_secret())
.await
.map_err(|e| DatabaseError::Pool(e.to_string()))?
} else {
libsql::LibSqlBackend::new_local(db_path)
.await
.map_err(|e| DatabaseError::Pool(e.to_string()))?
};
backend.run_migrations().await?;
Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new(
backend.shared_db(),
crypto,
)))
}
#[cfg(feature = "postgres")]
_ => {
let pg = postgres::PgBackend::new(config)
.await
.map_err(|e| DatabaseError::Pool(e.to_string()))?;
pg.run_migrations().await?;
Ok(Arc::new(crate::secrets::PostgresSecretsStore::new(
pg.pool(),
crypto,
)))
}
#[cfg(not(feature = "postgres"))]
_ => Err(DatabaseError::Pool(
"No database backend available for secrets. Enable 'postgres' or 'libsql' feature."
.to_string(),
)),
}
}
// ==================== Sub-traits ====================
//
// Each sub-trait groups related persistence methods. The `Database` supertrait
@@ -183,21 +125,6 @@ pub trait ConversationStore: Send + Sync {
channel: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError>;
async fn list_conversations_all_channels(
&self,
user_id: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError>;
async fn get_or_create_routine_conversation(
&self,
routine_id: Uuid,
routine_name: &str,
user_id: &str,
) -> Result<Uuid, DatabaseError>;
async fn get_or_create_heartbeat_conversation(
&self,
user_id: &str,
) -> Result<Uuid, DatabaseError>;
async fn get_or_create_assistant_conversation(
&self,
user_id: &str,
@@ -493,46 +420,3 @@ pub trait Database:
/// Run schema migrations for this backend.
async fn run_migrations(&self) -> Result<(), DatabaseError>;
}
#[cfg(test)]
mod tests {
use super::*;
/// Regression test: `create_secrets_store` selects the correct backend at
/// runtime based on `DatabaseConfig`, not at compile time. Previously the
/// CLI duplicated this logic with compile-time `#[cfg]` gates that always
/// chose postgres when both features were enabled (PR #209).
#[cfg(feature = "libsql")]
#[tokio::test]
async fn test_create_secrets_store_libsql_backend() {
use secrecy::SecretString;
let tmp = tempfile::tempdir().unwrap();
let db_path = tmp.path().join("test.db");
let config = crate::config::DatabaseConfig {
backend: crate::config::DatabaseBackend::LibSql,
libsql_path: Some(db_path),
libsql_url: None,
libsql_auth_token: None,
url: SecretString::from("unused://libsql".to_string()),
pool_size: 1,
ssl_mode: crate::config::SslMode::default(),
};
let master_key = SecretString::from("a]".repeat(16));
let crypto = Arc::new(crate::secrets::SecretsCrypto::new(master_key).unwrap());
let store = create_secrets_store(&config, crypto).await;
assert!(
store.is_ok(),
"create_secrets_store should succeed for libsql backend"
);
// Verify basic operation works
let store = store.unwrap();
let exists = store.exists("test_user", "nonexistent_secret").await;
assert!(exists.is_ok());
assert!(!exists.unwrap());
}
}
-30
View File
@@ -116,36 +116,6 @@ impl ConversationStore for PgBackend {
.await
}
async fn list_conversations_all_channels(
&self,
user_id: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError> {
self.store
.list_conversations_all_channels(user_id, limit)
.await
}
async fn get_or_create_routine_conversation(
&self,
routine_id: Uuid,
routine_name: &str,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
self.store
.get_or_create_routine_conversation(routine_id, routine_name, user_id)
.await
}
async fn get_or_create_heartbeat_conversation(
&self,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
self.store
.get_or_create_heartbeat_conversation(user_id)
.await
}
async fn get_or_create_assistant_conversation(
&self,
user_id: &str,
-3
View File
@@ -401,9 +401,6 @@ pub enum RoutineError {
#[error("Routine not found: {id}")]
NotFound { id: Uuid },
#[error("Not authorized to trigger routine {id}")]
NotAuthorized { id: Uuid },
#[error("Routine {name} at max concurrent runs")]
MaxConcurrent { name: String },
+227 -123
View File
@@ -1,10 +1,13 @@
//! Success evaluation for jobs.
use std::sync::Arc;
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use crate::context::{ActionRecord, JobContext};
use crate::error::EvaluationError;
use crate::llm::LlmProvider;
/// Result of evaluating job success.
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -61,132 +64,233 @@ pub trait SuccessEvaluator: Send + Sync {
) -> Result<EvaluationResult, EvaluationError>;
}
/// Rule-based success evaluator.
pub struct RuleBasedEvaluator {
/// Minimum success rate for actions.
min_action_success_rate: f64,
/// Maximum allowed failures.
max_failures: u32,
}
impl RuleBasedEvaluator {
/// Create a new rule-based evaluator.
pub fn new() -> Self {
Self {
min_action_success_rate: 0.8,
max_failures: 3,
}
}
/// Set minimum action success rate.
#[allow(dead_code)] // Public API for configuring evaluation threshold
pub fn with_min_success_rate(mut self, rate: f64) -> Self {
self.min_action_success_rate = rate;
self
}
/// Set maximum failures.
#[allow(dead_code)] // Public API for configuring failure tolerance
pub fn with_max_failures(mut self, max: u32) -> Self {
self.max_failures = max;
self
}
}
impl Default for RuleBasedEvaluator {
fn default() -> Self {
Self::new()
}
}
#[async_trait]
impl SuccessEvaluator for RuleBasedEvaluator {
async fn evaluate(
&self,
job: &JobContext,
actions: &[ActionRecord],
_output: Option<&str>,
) -> Result<EvaluationResult, EvaluationError> {
let mut issues = Vec::new();
// Check if there were any actions
if actions.is_empty() {
return Ok(EvaluationResult::failure(
"No actions were taken",
vec!["No actions recorded".to_string()],
));
}
// Calculate action success rate
let successful = actions.iter().filter(|a| a.success).count();
let total = actions.len();
let success_rate = successful as f64 / total as f64;
if success_rate < self.min_action_success_rate {
issues.push(format!(
"Action success rate {:.1}% below threshold {:.1}%",
success_rate * 100.0,
self.min_action_success_rate * 100.0
));
}
// Count failures
let failures = actions.iter().filter(|a| !a.success).count() as u32;
if failures > self.max_failures {
issues.push(format!(
"Too many failures: {} (max {})",
failures, self.max_failures
));
}
// Check for critical errors
for action in actions.iter().filter(|a| !a.success) {
if let Some(ref error) = action.error
&& (error.to_lowercase().contains("critical")
|| error.to_lowercase().contains("fatal"))
{
issues.push(format!("Critical error in {}: {}", action.tool_name, error));
}
}
// Check job state
if job.state != crate::context::JobState::Completed
&& job.state != crate::context::JobState::Submitted
{
issues.push(format!("Job not in completed state: {:?}", job.state));
}
// Calculate quality score
let quality_score = if issues.is_empty() {
let base_score = (success_rate * 80.0) as u32;
let completion_bonus = if job.state == crate::context::JobState::Completed {
20
} else {
0
};
(base_score + completion_bonus).min(100)
} else {
((success_rate * 50.0) as u32).min(50)
};
if issues.is_empty() {
Ok(EvaluationResult::success(
format!(
"Job completed successfully with {}/{} actions succeeding ({:.1}%)",
successful,
total,
success_rate * 100.0
),
quality_score,
))
} else {
Ok(EvaluationResult {
success: false,
confidence: 0.85,
reasoning: format!("Job had {} issues", issues.len()),
issues,
suggestions: vec![
"Review failed actions for common patterns".to_string(),
"Consider adjusting retry logic".to_string(),
],
quality_score,
})
}
}
}
/// LLM-based success evaluator for more nuanced evaluation.
pub struct LlmEvaluator {
llm: Arc<dyn LlmProvider>,
}
impl LlmEvaluator {
/// Create a new LLM-based evaluator.
#[allow(dead_code)] // Public API for LLM-based evaluation
pub fn new(llm: Arc<dyn LlmProvider>) -> Self {
Self { llm }
}
}
#[async_trait]
impl SuccessEvaluator for LlmEvaluator {
async fn evaluate(
&self,
job: &JobContext,
actions: &[ActionRecord],
output: Option<&str>,
) -> Result<EvaluationResult, EvaluationError> {
// Build evaluation prompt
let actions_summary: Vec<String> = actions
.iter()
.map(|a| {
format!(
"- {}: {} ({})",
a.tool_name,
if a.success { "success" } else { "failed" },
a.error.as_deref().unwrap_or("ok")
)
})
.collect();
let prompt = format!(
r#"Evaluate if this job was completed successfully.
Job: {}
Description: {}
State: {:?}
Actions taken:
{}
{}
Respond in JSON format:
{{
"success": true/false,
"confidence": 0.0-1.0,
"reasoning": "...",
"issues": ["..."],
"suggestions": ["..."],
"quality_score": 0-100
}}"#,
job.title,
job.description,
job.state,
actions_summary.join("\n"),
output
.map(|o| format!("Output:\n{}", o))
.unwrap_or_default()
);
let request =
crate::llm::CompletionRequest::new(vec![crate::llm::ChatMessage::user(prompt)])
.with_max_tokens(1024)
.with_temperature(0.1);
let response = self
.llm
.complete(request)
.await
.map_err(|e| EvaluationError::Failed {
job_id: job.job_id,
reason: e.to_string(),
})?;
// Parse the response
let result: EvaluationResult =
serde_json::from_str(&response.content).map_err(|e| EvaluationError::Failed {
job_id: job.job_id,
reason: format!("Failed to parse LLM evaluation: {}", e),
})?;
Ok(result)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::context::{ActionRecord, JobContext};
use crate::error::EvaluationError;
/// Rule-based success evaluator (test-only; no production callers).
struct RuleBasedEvaluator {
min_action_success_rate: f64,
max_failures: u32,
}
impl RuleBasedEvaluator {
fn new() -> Self {
Self {
min_action_success_rate: 0.8,
max_failures: 3,
}
}
fn with_min_success_rate(mut self, rate: f64) -> Self {
self.min_action_success_rate = rate;
self
}
fn with_max_failures(mut self, max: u32) -> Self {
self.max_failures = max;
self
}
}
impl Default for RuleBasedEvaluator {
fn default() -> Self {
Self::new()
}
}
#[async_trait::async_trait]
impl SuccessEvaluator for RuleBasedEvaluator {
async fn evaluate(
&self,
job: &JobContext,
actions: &[ActionRecord],
_output: Option<&str>,
) -> Result<EvaluationResult, EvaluationError> {
let mut issues = Vec::new();
if actions.is_empty() {
return Ok(EvaluationResult::failure(
"No actions were taken",
vec!["No actions recorded".to_string()],
));
}
let successful = actions.iter().filter(|a| a.success).count();
let total = actions.len();
let success_rate = successful as f64 / total as f64;
if success_rate < self.min_action_success_rate {
issues.push(format!(
"Action success rate {:.1}% below threshold {:.1}%",
success_rate * 100.0,
self.min_action_success_rate * 100.0
));
}
let failures = actions.iter().filter(|a| !a.success).count() as u32;
if failures > self.max_failures {
issues.push(format!(
"Too many failures: {} (max {})",
failures, self.max_failures
));
}
for action in actions.iter().filter(|a| !a.success) {
if let Some(ref error) = action.error
&& (error.to_lowercase().contains("critical")
|| error.to_lowercase().contains("fatal"))
{
issues.push(format!("Critical error in {}: {}", action.tool_name, error));
}
}
if job.state != crate::context::JobState::Completed
&& job.state != crate::context::JobState::Submitted
{
issues.push(format!("Job not in completed state: {:?}", job.state));
}
let quality_score = if issues.is_empty() {
let base_score = (success_rate * 80.0) as u32;
let completion_bonus = if job.state == crate::context::JobState::Completed {
20
} else {
0
};
(base_score + completion_bonus).min(100)
} else {
((success_rate * 50.0) as u32).min(50)
};
if issues.is_empty() {
Ok(EvaluationResult::success(
format!(
"Job completed successfully with {}/{} actions succeeding ({:.1}%)",
successful,
total,
success_rate * 100.0
),
quality_score,
))
} else {
Ok(EvaluationResult {
success: false,
confidence: 0.85,
reasoning: format!("Job had {} issues", issues.len()),
issues,
suggestions: vec![
"Review failed actions for common patterns".to_string(),
"Consider adjusting retry logic".to_string(),
],
quality_score,
})
}
}
}
use crate::context::JobContext;
#[tokio::test]
async fn test_rule_based_evaluator_success() {
+33 -2
View File
@@ -1405,6 +1405,38 @@ impl ExtensionManager {
Ok(())
}
#[allow(dead_code)] // Used by upcoming hot-activation flow
async fn install_bundled_channel_from_artifacts(
&self,
name: &str,
) -> Result<InstallResult, ExtensionError> {
// Check if already installed
let channel_wasm = self.wasm_channels_dir.join(format!("{}.wasm", name));
if channel_wasm.exists() {
return Err(ExtensionError::AlreadyInstalled(name.to_string()));
}
crate::channels::wasm::install_bundled_channel(name, &self.wasm_channels_dir, false)
.await
.map_err(ExtensionError::InstallFailed)?;
tracing::info!(
"Installed bundled channel '{}' to {}",
name,
self.wasm_channels_dir.display()
);
Ok(InstallResult {
name: name.to_string(),
kind: ExtensionKind::WasmChannel,
message: format!(
"Channel '{}' installed. \
Run tool_auth('{}') to configure authentication, then activate.",
name, name,
),
})
}
/// Install a WASM extension from local build artifacts (WasmBuildable source).
///
/// Resolves the build directory (relative to `CARGO_MANIFEST_DIR` or absolute),
@@ -1596,7 +1628,6 @@ impl ExtensionManager {
&metadata.scopes_supported,
Some(&pkce),
&std::collections::HashMap::new(),
None,
);
// Store pending auth for later callback handling
@@ -2477,7 +2508,7 @@ impl ExtensionManager {
&self.user_id,
)
} else {
McpClient::new_with_config(server.clone())
McpClient::new_with_name(&server.name, &server.url)
};
// Try to list and create tools
+1 -207
View File
@@ -241,9 +241,6 @@ impl Store {
tool_output_stash: std::sync::Arc::new(tokio::sync::RwLock::new(
std::collections::HashMap::new(),
)),
// TODO(#661): persist user_timezone in agent_jobs table so
// background/routine jobs retain the session's timezone context.
user_timezone: "UTC".to_string(),
}))
}
None => Ok(None),
@@ -1380,8 +1377,6 @@ pub struct ConversationSummary {
pub last_activity: DateTime<Utc>,
/// Thread type extracted from metadata (e.g. "assistant", "thread").
pub thread_type: Option<String>,
/// Channel that owns this conversation (e.g. "gateway", "telegram", "routine").
pub channel: String,
}
/// A single message in a conversation.
@@ -1434,7 +1429,6 @@ impl Store {
c.started_at,
c.last_activity,
c.metadata,
c.channel,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT LEFT(m2.content, 100)
FROM conversation_messages m2
@@ -1459,181 +1453,18 @@ impl Store {
.get("thread_type")
.and_then(|v| v.as_str())
.map(String::from);
let sql_title: Option<String> = r.get("title");
let title = sql_title.or_else(|| {
metadata
.get("routine_name")
.and_then(|v| v.as_str())
.map(String::from)
});
ConversationSummary {
id: r.get("id"),
title,
title: r.get("title"),
message_count: r.get("message_count"),
started_at: r.get("started_at"),
last_activity: r.get("last_activity"),
thread_type,
channel: r.get("channel"),
}
})
.collect())
}
/// List conversations across all channels with a title derived from the first user message.
pub async fn list_conversations_all_channels(
&self,
user_id: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError> {
let conn = self.conn().await?;
let rows = conn
.query(
r#"
SELECT
c.id,
c.started_at,
c.last_activity,
c.metadata,
c.channel,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT LEFT(m2.content, 100)
FROM conversation_messages m2
WHERE m2.conversation_id = c.id AND m2.role = 'user'
ORDER BY m2.created_at ASC
LIMIT 1
) AS title
FROM conversations c
WHERE c.user_id = $1
ORDER BY c.last_activity DESC
LIMIT $2
"#,
&[&user_id, &limit],
)
.await?;
Ok(rows
.iter()
.map(|r| {
let metadata: serde_json::Value = r.get("metadata");
let thread_type = metadata
.get("thread_type")
.and_then(|v| v.as_str())
.map(String::from);
// For routine/heartbeat threads, derive title from metadata
// since they may have no user messages.
let sql_title: Option<String> = r.get("title");
let title = sql_title.or_else(|| {
metadata
.get("routine_name")
.and_then(|v| v.as_str())
.map(String::from)
});
ConversationSummary {
id: r.get("id"),
title,
message_count: r.get("message_count"),
started_at: r.get("started_at"),
last_activity: r.get("last_activity"),
thread_type,
channel: r.get("channel"),
}
})
.collect())
}
/// Get or create a persistent conversation for a routine.
///
/// Looks for a conversation where `metadata->>'routine_id' = routine_id`.
/// Creates one if it doesn't exist. Uses INSERT ON CONFLICT to avoid
/// TOCTOU races under concurrent routine executions.
pub async fn get_or_create_routine_conversation(
&self,
routine_id: Uuid,
routine_name: &str,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
let conn = self.conn().await?;
let rid = routine_id.to_string();
// Attempt insert first; the partial unique index
// uq_conv_routine(user_id, (metadata->>'routine_id')) prevents duplicates.
let new_id = Uuid::new_v4();
let metadata = serde_json::json!({
"thread_type": "routine",
"routine_id": routine_id.to_string(),
"routine_name": routine_name,
});
conn.execute(
r#"
INSERT INTO conversations (id, channel, user_id, metadata)
VALUES ($1, 'routine', $2, $3)
ON CONFLICT (user_id, (metadata->>'routine_id'))
WHERE metadata->>'routine_id' IS NOT NULL
DO NOTHING
"#,
&[&new_id, &user_id, &metadata],
)
.await?;
// Select back — always returns the winner.
let row = conn
.query_one(
r#"
SELECT id FROM conversations
WHERE user_id = $1 AND metadata->>'routine_id' = $2
LIMIT 1
"#,
&[&user_id, &rid],
)
.await?;
Ok(row.get("id"))
}
/// Get or create the singleton heartbeat conversation for a user.
///
/// Looks for a conversation where `metadata->>'thread_type' = 'heartbeat'`.
/// Creates one if it doesn't exist. Uses INSERT ON CONFLICT to avoid
/// TOCTOU races under concurrent heartbeat sends.
pub async fn get_or_create_heartbeat_conversation(
&self,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
let conn = self.conn().await?;
// Attempt insert; the partial unique index
// uq_conv_heartbeat(user_id) prevents duplicates.
let new_id = Uuid::new_v4();
let metadata = serde_json::json!({
"thread_type": "heartbeat",
});
conn.execute(
r#"
INSERT INTO conversations (id, channel, user_id, metadata)
VALUES ($1, 'heartbeat', $2, $3)
ON CONFLICT (user_id)
WHERE metadata->>'thread_type' = 'heartbeat'
DO NOTHING
"#,
&[&new_id, &user_id, &metadata],
)
.await?;
// Select back — always returns the winner.
let row = conn
.query_one(
r#"
SELECT id FROM conversations
WHERE user_id = $1 AND metadata->>'thread_type' = 'heartbeat'
LIMIT 1
"#,
&[&user_id],
)
.await?;
Ok(row.get("id"))
}
/// Get or create the singleton "assistant" conversation for a user+channel.
///
/// Looks for a conversation where `metadata->>'thread_type' = 'assistant'`.
@@ -2097,40 +1928,3 @@ impl Store {
Ok(count > 0)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_conversation_summary_has_channel_field() {
// Regression: ConversationSummary must include a `channel` field
// so the gateway can distinguish thread origins.
let summary = ConversationSummary {
id: Uuid::nil(),
title: Some("Hello".to_string()),
message_count: 1,
started_at: Utc::now(),
last_activity: Utc::now(),
thread_type: Some("thread".to_string()),
channel: "telegram".to_string(),
};
assert_eq!(summary.channel, "telegram");
}
#[test]
fn test_conversation_summary_channel_various_values() {
for ch in ["gateway", "routine", "heartbeat", "telegram", "signal"] {
let summary = ConversationSummary {
id: Uuid::nil(),
title: None,
message_count: 0,
started_at: Utc::now(),
last_activity: Utc::now(),
thread_type: None,
channel: ch.to_string(),
};
assert_eq!(summary.channel, ch);
}
}
}
-1
View File
@@ -66,7 +66,6 @@ pub mod service;
pub mod settings;
pub mod setup;
pub mod skills;
pub mod timezone;
pub mod tools;
pub mod tracing_fmt;
pub mod transcription;
-641
View File
@@ -1,641 +0,0 @@
//! Anthropic OAuth provider (direct HTTP, `Authorization: Bearer`).
//!
//! This provider exists because the `rig-core` Anthropic client hardcodes the
//! `x-api-key` header, which is rejected by Anthropic's OAuth tokens from
//! `claude login`. OAuth tokens require `Authorization: Bearer <token>` instead.
//!
//! Pattern follows `nearai_chat.rs`: direct HTTP calls via `reqwest::Client`.
use async_trait::async_trait;
use reqwest::Client;
use rust_decimal::Decimal;
use secrecy::{ExposeSecret, SecretString};
use serde::{Deserialize, Serialize};
use crate::config::RegistryProviderConfig;
use crate::error::LlmError;
use crate::llm::costs;
use crate::llm::provider::{
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, Role, ToolCall,
ToolCompletionRequest, ToolCompletionResponse,
};
const ANTHROPIC_API_URL: &str = "https://api.anthropic.com/v1/messages";
/// OAuth beta requires 2023-06-01; the 2024-10-22 version is not valid with the beta flag.
const ANTHROPIC_API_VERSION: &str = "2023-06-01";
/// Required beta flag to enable OAuth Bearer auth on api.anthropic.com.
/// Without this header, the API returns 401 "OAuth authentication is currently not supported."
const ANTHROPIC_OAUTH_BETA: &str = "oauth-2025-04-20";
const DEFAULT_MAX_TOKENS: u32 = 8192;
/// Anthropic provider using OAuth Bearer authentication.
pub struct AnthropicOAuthProvider {
client: Client,
token: SecretString,
model: String,
base_url: Option<String>,
active_model: std::sync::RwLock<String>,
}
impl AnthropicOAuthProvider {
pub fn new(config: &RegistryProviderConfig) -> Result<Self, LlmError> {
let token = config
.oauth_token
.clone()
.ok_or_else(|| LlmError::AuthFailed {
provider: "anthropic_oauth".to_string(),
})?;
let client = Client::builder()
.timeout(std::time::Duration::from_secs(120))
.build()
.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: format!("Failed to build HTTP client: {}", e),
})?;
let active_model = std::sync::RwLock::new(config.model.clone());
let base_url = if config.base_url.is_empty() {
None
} else {
Some(config.base_url.clone())
};
Ok(Self {
client,
token,
model: config.model.clone(),
base_url,
active_model,
})
}
fn api_url(&self) -> String {
if let Some(ref base) = self.base_url {
let base = base.trim_end_matches('/');
format!("{}/v1/messages", base)
} else {
ANTHROPIC_API_URL.to_string()
}
}
async fn send_request<R: for<'de> Deserialize<'de>>(
&self,
body: &AnthropicRequest,
) -> Result<R, LlmError> {
let url = self.api_url();
tracing::debug!("Sending request to Anthropic OAuth: {}", url);
let response = self
.client
.post(&url)
.bearer_auth(self.token.expose_secret())
.header("anthropic-version", ANTHROPIC_API_VERSION)
.header("anthropic-beta", ANTHROPIC_OAUTH_BETA)
.header("Content-Type", "application/json")
.json(body)
.send()
.await
.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: e.to_string(),
})?;
let status = response.status();
if !status.is_success() {
// Parse Retry-After header before consuming the body.
let retry_after = response
.headers()
.get("retry-after")
.and_then(|v| v.to_str().ok())
.and_then(|v| v.parse::<u64>().ok())
.map(std::time::Duration::from_secs);
let response_text = response
.text()
.await
.unwrap_or_else(|e| format!("(failed to read error body: {e})"));
if status.as_u16() == 401 {
// OAuth tokens from `claude login` expire in ~8-12h. Attempt
// to re-extract a fresh token from the OS credential store
// (macOS Keychain / Linux credentials file) before giving up.
if let Some(fresh) = crate::config::ClaudeCodeConfig::extract_oauth_token() {
let fresh_token = SecretString::from(fresh);
// Retry once with the refreshed token
let retry = self
.client
.post(&url)
.bearer_auth(fresh_token.expose_secret())
.header("anthropic-version", ANTHROPIC_API_VERSION)
.header("anthropic-beta", ANTHROPIC_OAUTH_BETA)
.header("Content-Type", "application/json")
.json(body)
.send()
.await
.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: e.to_string(),
})?;
if retry.status().is_success() {
let text = retry.text().await.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: format!("Failed to read response body: {}", e),
})?;
return serde_json::from_str(&text).map_err(|e| {
let truncated = crate::agent::truncate_for_preview(&text, 512);
LlmError::InvalidResponse {
provider: "anthropic_oauth".to_string(),
reason: format!("JSON parse error: {}. Raw: {}", e, truncated),
}
});
}
tracing::warn!(
"Anthropic OAuth 401 retry with refreshed token also failed ({})",
retry.status()
);
}
return Err(LlmError::AuthFailed {
provider: "anthropic_oauth".to_string(),
});
}
if status.as_u16() == 429 {
return Err(LlmError::RateLimited {
provider: "anthropic_oauth".to_string(),
retry_after,
});
}
let truncated = crate::agent::truncate_for_preview(&response_text, 512);
return Err(LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: format!("HTTP {}: {}", status, truncated),
});
}
let response_text = response.text().await.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: format!("Failed to read response body: {}", e),
})?;
tracing::debug!(
"Anthropic OAuth response: status={}, bytes={}",
status,
response_text.len()
);
serde_json::from_str(&response_text).map_err(|e| {
let truncated = crate::agent::truncate_for_preview(&response_text, 512);
LlmError::InvalidResponse {
provider: "anthropic_oauth".to_string(),
reason: format!("JSON parse error: {}. Raw: {}", e, truncated),
}
})
}
}
#[async_trait]
impl LlmProvider for AnthropicOAuthProvider {
async fn complete(&self, req: CompletionRequest) -> Result<CompletionResponse, LlmError> {
let model = req.model.unwrap_or_else(|| self.active_model_name());
let (system, messages) = convert_messages(req.messages);
let request = AnthropicRequest {
model,
messages,
system,
max_tokens: req.max_tokens.unwrap_or(DEFAULT_MAX_TOKENS),
temperature: req.temperature,
tools: None,
tool_choice: None,
};
let response: AnthropicResponse = self.send_request(&request).await?;
let (content, _tool_calls) = extract_response_content(&response);
let finish_reason = match response.stop_reason.as_deref() {
Some("end_turn") | Some("stop") => FinishReason::Stop,
Some("max_tokens") => FinishReason::Length,
Some("tool_use") => FinishReason::ToolUse,
_ => FinishReason::Unknown,
};
Ok(CompletionResponse {
content: content.unwrap_or_default(),
finish_reason,
input_tokens: response.usage.input_tokens,
output_tokens: response.usage.output_tokens,
cache_creation_input_tokens: response.usage.cache_creation_input_tokens,
cache_read_input_tokens: response.usage.cache_read_input_tokens,
})
}
async fn complete_with_tools(
&self,
req: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
let model = req.model.unwrap_or_else(|| self.active_model_name());
let (system, messages) = convert_messages(req.messages);
let tools: Vec<AnthropicTool> = req
.tools
.into_iter()
.map(|t| AnthropicTool {
name: t.name,
description: t.description,
input_schema: t.parameters,
})
.collect();
// Map tool_choice from OpenAI format to Anthropic format
let tool_choice = req.tool_choice.map(|tc| match tc.as_str() {
"auto" => AnthropicToolChoice {
choice_type: "auto".to_string(),
name: None,
},
"required" => AnthropicToolChoice {
choice_type: "any".to_string(),
name: None,
},
"none" => AnthropicToolChoice {
choice_type: "none".to_string(),
name: None,
},
specific => AnthropicToolChoice {
choice_type: "tool".to_string(),
name: Some(specific.to_string()),
},
});
let request = AnthropicRequest {
model,
messages,
system,
max_tokens: req.max_tokens.unwrap_or(DEFAULT_MAX_TOKENS),
temperature: req.temperature,
tools: if tools.is_empty() { None } else { Some(tools) },
tool_choice,
};
let response: AnthropicResponse = self.send_request(&request).await?;
let (content, tool_calls) = extract_response_content(&response);
let finish_reason = match response.stop_reason.as_deref() {
Some("end_turn") | Some("stop") => FinishReason::Stop,
Some("max_tokens") => FinishReason::Length,
Some("tool_use") => FinishReason::ToolUse,
_ => {
if !tool_calls.is_empty() {
FinishReason::ToolUse
} else {
FinishReason::Unknown
}
}
};
Ok(ToolCompletionResponse {
content,
tool_calls,
finish_reason,
input_tokens: response.usage.input_tokens,
output_tokens: response.usage.output_tokens,
cache_creation_input_tokens: response.usage.cache_creation_input_tokens,
cache_read_input_tokens: response.usage.cache_read_input_tokens,
})
}
fn model_name(&self) -> &str {
&self.model
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
let model = self.active_model_name();
costs::model_cost(&model).unwrap_or_else(costs::default_cost)
}
fn active_model_name(&self) -> String {
match self.active_model.read() {
Ok(guard) => guard.clone(),
Err(poisoned) => poisoned.into_inner().clone(),
}
}
fn set_model(&self, model: &str) -> Result<(), LlmError> {
match self.active_model.write() {
Ok(mut guard) => {
*guard = model.to_string();
}
Err(poisoned) => {
*poisoned.into_inner() = model.to_string();
}
}
Ok(())
}
}
// --- Anthropic Messages API types ---
#[derive(Debug, Serialize)]
struct AnthropicRequest {
model: String,
messages: Vec<AnthropicMessage>,
#[serde(skip_serializing_if = "Option::is_none")]
system: Option<String>,
max_tokens: u32,
#[serde(skip_serializing_if = "Option::is_none")]
temperature: Option<f32>,
#[serde(skip_serializing_if = "Option::is_none")]
tools: Option<Vec<AnthropicTool>>,
#[serde(skip_serializing_if = "Option::is_none")]
tool_choice: Option<AnthropicToolChoice>,
}
#[derive(Debug, Serialize)]
struct AnthropicMessage {
role: String,
content: AnthropicContent,
}
/// Anthropic content can be a simple string or a list of content blocks.
#[derive(Debug, Serialize)]
#[serde(untagged)]
enum AnthropicContent {
Text(String),
Blocks(Vec<AnthropicContentBlock>),
}
#[derive(Debug, Serialize)]
#[serde(tag = "type")]
enum AnthropicContentBlock {
#[serde(rename = "text")]
Text { text: String },
#[serde(rename = "tool_use")]
ToolUse {
id: String,
name: String,
input: serde_json::Value,
},
#[serde(rename = "tool_result")]
ToolResult {
tool_use_id: String,
content: String,
},
}
#[derive(Debug, Serialize)]
struct AnthropicTool {
name: String,
description: String,
input_schema: serde_json::Value,
}
#[derive(Debug, Serialize)]
struct AnthropicToolChoice {
#[serde(rename = "type")]
choice_type: String,
#[serde(skip_serializing_if = "Option::is_none")]
name: Option<String>,
}
#[derive(Debug, Deserialize)]
struct AnthropicResponse {
content: Vec<AnthropicResponseBlock>,
#[serde(default)]
stop_reason: Option<String>,
usage: AnthropicUsage,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type")]
enum AnthropicResponseBlock {
#[serde(rename = "text")]
Text { text: String },
#[serde(rename = "tool_use")]
ToolUse {
id: String,
name: String,
input: serde_json::Value,
},
}
#[derive(Debug, Deserialize)]
struct AnthropicUsage {
#[serde(default)]
input_tokens: u32,
#[serde(default)]
output_tokens: u32,
#[serde(default)]
cache_creation_input_tokens: u32,
#[serde(default)]
cache_read_input_tokens: u32,
}
/// Convert ChatMessage list to Anthropic format.
///
/// Extracts system messages to the top-level `system` parameter (Anthropic
/// doesn't allow system messages in the `messages` array). Tool-call/tool-result
/// pairs are converted to content blocks.
fn convert_messages(messages: Vec<ChatMessage>) -> (Option<String>, Vec<AnthropicMessage>) {
let mut system_parts: Vec<String> = Vec::new();
let mut anthropic_msgs: Vec<AnthropicMessage> = Vec::new();
for msg in messages {
match msg.role {
Role::System => {
if !msg.content.is_empty() {
system_parts.push(msg.content);
}
}
Role::User => {
anthropic_msgs.push(AnthropicMessage {
role: "user".to_string(),
content: AnthropicContent::Text(msg.content),
});
}
Role::Assistant => {
if let Some(tool_calls) = msg.tool_calls {
// Assistant message with tool calls → content blocks
let mut blocks: Vec<AnthropicContentBlock> = Vec::new();
if !msg.content.is_empty() {
blocks.push(AnthropicContentBlock::Text { text: msg.content });
}
for tc in tool_calls {
blocks.push(AnthropicContentBlock::ToolUse {
id: tc.id,
name: tc.name,
input: tc.arguments,
});
}
anthropic_msgs.push(AnthropicMessage {
role: "assistant".to_string(),
content: AnthropicContent::Blocks(blocks),
});
} else {
anthropic_msgs.push(AnthropicMessage {
role: "assistant".to_string(),
content: AnthropicContent::Text(msg.content),
});
}
}
Role::Tool => {
let Some(tool_call_id) = msg.tool_call_id else {
tracing::warn!("Skipping Tool message without tool_call_id");
continue;
};
// Tool results go into a user message with tool_result blocks
let block = AnthropicContentBlock::ToolResult {
tool_use_id: tool_call_id,
content: msg.content,
};
// If the last message is already a user message with blocks,
// append to it (Anthropic requires consecutive tool results
// in one user message).
if let Some(last) = anthropic_msgs.last_mut()
&& last.role == "user"
&& let AnthropicContent::Blocks(ref mut blocks) = last.content
{
blocks.push(block);
continue;
}
anthropic_msgs.push(AnthropicMessage {
role: "user".to_string(),
content: AnthropicContent::Blocks(vec![block]),
});
}
}
}
let system = if system_parts.is_empty() {
None
} else {
Some(system_parts.join("\n\n"))
};
(system, anthropic_msgs)
}
/// Extract text content and tool calls from an Anthropic response.
fn extract_response_content(response: &AnthropicResponse) -> (Option<String>, Vec<ToolCall>) {
let mut text_parts: Vec<String> = Vec::new();
let mut tool_calls: Vec<ToolCall> = Vec::new();
for block in &response.content {
match block {
AnthropicResponseBlock::Text { text } => {
text_parts.push(text.clone());
}
AnthropicResponseBlock::ToolUse { id, name, input } => {
tool_calls.push(ToolCall {
id: id.clone(),
name: name.clone(),
arguments: input.clone(),
});
}
}
}
let content = if text_parts.is_empty() {
None
} else {
Some(text_parts.join(""))
};
(content, tool_calls)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_convert_messages_extracts_system() {
let messages = vec![
ChatMessage::system("You are helpful."),
ChatMessage::user("Hello"),
];
let (system, msgs) = convert_messages(messages);
assert_eq!(system, Some("You are helpful.".to_string()));
assert_eq!(msgs.len(), 1);
assert_eq!(msgs[0].role, "user");
}
#[test]
fn test_convert_messages_multiple_systems() {
let messages = vec![
ChatMessage::system("System 1"),
ChatMessage::system("System 2"),
ChatMessage::user("Hello"),
];
let (system, msgs) = convert_messages(messages);
assert_eq!(system, Some("System 1\n\nSystem 2".to_string()));
assert_eq!(msgs.len(), 1);
}
#[test]
fn test_convert_messages_tool_calls() {
let tool_calls = vec![ToolCall {
id: "call_1".to_string(),
name: "search".to_string(),
arguments: serde_json::json!({"q": "test"}),
}];
let messages = vec![
ChatMessage::user("Search for test"),
ChatMessage::assistant_with_tool_calls(Some("Let me search.".to_string()), tool_calls),
ChatMessage::tool_result("call_1", "search", "found it"),
];
let (system, msgs) = convert_messages(messages);
assert!(system.is_none());
assert_eq!(msgs.len(), 3);
assert_eq!(msgs[0].role, "user");
assert_eq!(msgs[1].role, "assistant");
// Tool result should be a user message
assert_eq!(msgs[2].role, "user");
}
#[test]
fn test_extract_response_text_only() {
let response = AnthropicResponse {
content: vec![AnthropicResponseBlock::Text {
text: "Hello!".to_string(),
}],
stop_reason: Some("end_turn".to_string()),
usage: AnthropicUsage {
input_tokens: 10,
output_tokens: 5,
cache_creation_input_tokens: 0,
cache_read_input_tokens: 0,
},
};
let (content, tool_calls) = extract_response_content(&response);
assert_eq!(content, Some("Hello!".to_string()));
assert!(tool_calls.is_empty());
}
#[test]
fn test_extract_response_with_tool_use() {
let response = AnthropicResponse {
content: vec![
AnthropicResponseBlock::Text {
text: "Let me search.".to_string(),
},
AnthropicResponseBlock::ToolUse {
id: "call_1".to_string(),
name: "search".to_string(),
input: serde_json::json!({"q": "test"}),
},
],
stop_reason: Some("tool_use".to_string()),
usage: AnthropicUsage {
input_tokens: 20,
output_tokens: 15,
cache_creation_input_tokens: 0,
cache_read_input_tokens: 0,
},
};
let (content, tool_calls) = extract_response_content(&response);
assert_eq!(content, Some("Let me search.".to_string()));
assert_eq!(tool_calls.len(), 1);
assert_eq!(tool_calls[0].name, "search");
}
}
-1148
View File
File diff suppressed because it is too large Load Diff
-95
View File
@@ -1,95 +0,0 @@
//! Image generation model detection utilities.
/// Known image generation model families.
const IMAGE_GEN_PATTERNS: &[&str] = &[
"flux",
"dall-e",
"dalle",
"stable-diffusion",
"sdxl",
"imagen",
"midjourney",
"ideogram",
"playground",
];
/// Check if a model name indicates an image generation model.
pub fn is_image_generation_model(model: &str) -> bool {
let lower = model.to_lowercase();
IMAGE_GEN_PATTERNS.iter().any(|p| lower.contains(p))
}
/// Suggest the best image generation model from a list of available models.
///
/// Priority: FLUX > DALL-E > Stable Diffusion > others.
pub fn suggest_image_model(models: &[String]) -> Option<&str> {
let priorities: &[&str] = &[
"flux",
"dall-e",
"dalle",
"stable-diffusion",
"sdxl",
"imagen",
];
for priority in priorities {
if let Some(model) = models.iter().find(|m| m.to_lowercase().contains(priority)) {
return Some(model);
}
}
// Fall back to any image gen model
models.iter().find_map(|m| {
if is_image_generation_model(m) {
Some(m.as_str())
} else {
None
}
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn detects_flux_models() {
assert!(is_image_generation_model(
"black-forest-labs/FLUX.1-schnell"
));
assert!(is_image_generation_model("flux-pro"));
}
#[test]
fn detects_dalle_models() {
assert!(is_image_generation_model("dall-e-3"));
assert!(is_image_generation_model("dalle-3"));
}
#[test]
fn rejects_non_image_models() {
assert!(!is_image_generation_model("gpt-4o"));
assert!(!is_image_generation_model("claude-3-sonnet"));
assert!(!is_image_generation_model("llama-3.1-70b"));
}
#[test]
fn suggests_flux_first() {
let models = vec![
"gpt-4o".to_string(),
"dall-e-3".to_string(),
"flux-pro".to_string(),
];
assert_eq!(suggest_image_model(&models), Some("flux-pro"));
}
#[test]
fn suggests_dalle_without_flux() {
let models = vec!["gpt-4o".to_string(), "dall-e-3".to_string()];
assert_eq!(suggest_image_model(&models), Some("dall-e-3"));
}
#[test]
fn returns_none_when_no_image_models() {
let models = vec!["gpt-4o".to_string(), "claude-3-sonnet".to_string()];
assert_eq!(suggest_image_model(&models), None);
}
}
+7 -83
View File
@@ -6,11 +6,7 @@
//! - **Anthropic**: Direct API access with your own key
//! - **Ollama**: Local model inference
//! - **OpenAI-compatible**: Any endpoint that speaks the OpenAI API
//! - **AWS Bedrock**: Native Converse API via aws-sdk-bedrockruntime
mod anthropic_oauth;
#[cfg(feature = "bedrock")]
mod bedrock;
pub mod circuit_breaker;
pub mod costs;
pub mod failover;
@@ -25,9 +21,6 @@ mod rig_adapter;
pub mod session;
pub mod smart_routing;
pub mod image_models;
pub mod vision_models;
pub use circuit_breaker::{CircuitBreakerConfig, CircuitBreakerProvider};
pub use failover::{CooldownConfig, FailoverProvider};
pub use nearai_chat::{ModelInfo, NearAiChatProvider};
@@ -60,29 +53,12 @@ use crate::error::LlmError;
///
/// - NearAI backend: Uses session manager for authentication
/// - Registry providers: Looked up by protocol and constructed generically
pub async fn create_llm_provider(
pub fn create_llm_provider(
config: &LlmConfig,
session: Arc<SessionManager>,
) -> Result<Arc<dyn LlmProvider>, LlmError> {
let timeout = config.request_timeout_secs;
if config.backend == "nearai" || config.backend == "near_ai" || config.backend == "near" {
return create_llm_provider_with_config(&config.nearai, session, timeout);
}
// Bedrock uses a native AWS SDK, not the rig-core registry
if config.backend == "bedrock" {
#[cfg(feature = "bedrock")]
{
return create_bedrock_provider(config).await;
}
#[cfg(not(feature = "bedrock"))]
{
return Err(LlmError::RequestFailed {
provider: "bedrock".to_string(),
reason: "Bedrock support not compiled. Rebuild with --features bedrock".to_string(),
});
}
return create_llm_provider_with_config(&config.nearai, session);
}
let reg_config = config
@@ -102,7 +78,6 @@ pub async fn create_llm_provider(
pub fn create_llm_provider_with_config(
config: &NearAiConfig,
session: Arc<SessionManager>,
request_timeout_secs: u64,
) -> Result<Arc<dyn LlmProvider>, LlmError> {
let auth_mode = if config.api_key.is_some() {
"API key"
@@ -113,14 +88,9 @@ pub fn create_llm_provider_with_config(
model = %config.model,
base_url = %config.base_url,
auth = auth_mode,
timeout_secs = request_timeout_secs,
"Using NEAR AI (Chat Completions API)"
);
Ok(Arc::new(NearAiChatProvider::new_with_timeout(
config.clone(),
session,
request_timeout_secs,
)?))
Ok(Arc::new(NearAiChatProvider::new(config.clone(), session)?))
}
/// Create a provider from a registry-resolved config.
@@ -138,24 +108,6 @@ fn create_registry_provider(
}
}
#[cfg(feature = "bedrock")]
async fn create_bedrock_provider(config: &LlmConfig) -> Result<Arc<dyn LlmProvider>, LlmError> {
let br = config
.bedrock
.as_ref()
.ok_or_else(|| LlmError::AuthFailed {
provider: "bedrock".to_string(),
})?;
let provider = bedrock::BedrockProvider::new(br).await?;
tracing::info!(
"Using AWS Bedrock (Converse API, region: {}, model: {})",
br.region,
provider.active_model_name(),
);
Ok(Arc::new(provider))
}
fn create_openai_compat_from_registry(
config: &RegistryProviderConfig,
) -> Result<Arc<dyn LlmProvider>, LlmError> {
@@ -226,24 +178,6 @@ fn create_openai_compat_from_registry(
fn create_anthropic_from_registry(
config: &RegistryProviderConfig,
) -> Result<Arc<dyn LlmProvider>, LlmError> {
// Route to OAuth provider when an OAuth token is present and no real API
// key was provided. When both are set, the API key takes priority (standard
// x-api-key auth via rig-core).
let api_key_is_placeholder = config
.api_key
.as_ref()
.is_some_and(|k| k.expose_secret() == crate::config::llm::OAUTH_PLACEHOLDER);
if config.oauth_token.is_some() && (config.api_key.is_none() || api_key_is_placeholder) {
tracing::info!(
provider = %config.provider_id,
model = %config.model,
base_url = if config.base_url.is_empty() { "default" } else { &config.base_url },
"Using Anthropic OAuth API"
);
let provider = anthropic_oauth::AnthropicOAuthProvider::new(config)?;
return Ok(Arc::new(provider));
}
use crate::config::CacheRetention;
use crate::config::helpers::optional_env;
use rig::providers::anthropic;
@@ -380,7 +314,7 @@ pub fn create_cheap_llm_provider(
/// This is the single source of truth for provider chain construction,
/// called by both `main.rs` and `app.rs`.
#[allow(clippy::type_complexity)]
pub async fn build_provider_chain(
pub fn build_provider_chain(
config: &LlmConfig,
session: Arc<SessionManager>,
) -> Result<
@@ -391,7 +325,7 @@ pub async fn build_provider_chain(
),
LlmError,
> {
let llm = create_llm_provider(config, session.clone()).await?;
let llm = create_llm_provider(config, session.clone())?;
tracing::info!("LLM provider initialized: {}", llm.model_name());
// 1. Retry
@@ -412,11 +346,7 @@ pub async fn build_provider_chain(
let llm: Arc<dyn LlmProvider> = if let Some(ref cheap_model) = config.nearai.cheap_model {
let mut cheap_config = config.nearai.clone();
cheap_config.model = cheap_model.clone();
let cheap = create_llm_provider_with_config(
&cheap_config,
session.clone(),
config.request_timeout_secs,
)?;
let cheap = create_llm_provider_with_config(&cheap_config, session.clone())?;
let cheap: Arc<dyn LlmProvider> = if retry_config.max_retries > 0 {
Arc::new(RetryProvider::new(cheap, retry_config.clone()))
} else {
@@ -448,11 +378,7 @@ pub async fn build_provider_chain(
}
let mut fallback_config = config.nearai.clone();
fallback_config.model = fallback_model.clone();
let fallback = create_llm_provider_with_config(
&fallback_config,
session.clone(),
config.request_timeout_secs,
)?;
let fallback = create_llm_provider_with_config(&fallback_config, session.clone())?;
tracing::info!(
primary = %llm.model_name(),
fallback = %fallback.model_name(),
@@ -558,8 +484,6 @@ mod tests {
session: SessionConfig::default(),
nearai: test_nearai_config(),
provider: None,
bedrock: None,
request_timeout_secs: 120,
}
}
+4 -15
View File
@@ -58,28 +58,17 @@ impl NearAiChatProvider {
/// By default this enables tool-message flattening for compatibility with
/// providers that reject `role: "tool"` messages.
pub fn new(config: NearAiConfig, session: Arc<SessionManager>) -> Result<Self, LlmError> {
Self::new_with_options(config, session, true, 120)
Self::new_with_flatten(config, session, true)
}
/// Create a new provider with a custom request timeout.
pub fn new_with_timeout(
config: NearAiConfig,
session: Arc<SessionManager>,
request_timeout_secs: u64,
) -> Result<Self, LlmError> {
Self::new_with_options(config, session, true, request_timeout_secs)
}
/// Create a chat completions provider with configurable tool-message flattening
/// and request timeout.
pub fn new_with_options(
/// Create a chat completions provider with configurable tool-message flattening.
pub fn new_with_flatten(
config: NearAiConfig,
session: Arc<SessionManager>,
flatten_tool_messages: bool,
request_timeout_secs: u64,
) -> Result<Self, LlmError> {
let client = Client::builder()
.timeout(std::time::Duration::from_secs(request_timeout_secs))
.timeout(std::time::Duration::from_secs(120))
.build()
.map_err(|e| LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
-62
View File
@@ -522,66 +522,4 @@ mod tests {
assert_eq!(messages[3].role, Role::User); // call_2 orphaned
assert_eq!(messages[4].role, Role::User); // call_3 orphaned
}
/// Regression: worker's select_tools/execute_plan now emit
/// assistant_with_tool_calls before tool_result messages.
/// Verify sanitize_tool_messages preserves all tool_results when
/// each has a matching assistant tool_call.
#[test]
fn test_sanitize_preserves_tool_results_with_matching_assistant() {
let tc1 = ToolCall {
id: "call_sel_1".to_string(),
name: "search".to_string(),
arguments: serde_json::json!({"q": "test"}),
};
let tc2 = ToolCall {
id: "call_sel_2".to_string(),
name: "http".to_string(),
arguments: serde_json::json!({"url": "https://example.com"}),
};
let mut messages = vec![
ChatMessage::system("You are a helpful assistant."),
ChatMessage::assistant_with_tool_calls(None, vec![tc1, tc2]),
ChatMessage::tool_result("call_sel_1", "search", "found 3 results"),
ChatMessage::tool_result("call_sel_2", "http", "200 OK"),
];
sanitize_tool_messages(&mut messages);
// All tool_results must keep Role::Tool -- none should be rewritten.
assert_eq!(messages[2].role, Role::Tool);
assert_eq!(messages[2].tool_call_id, Some("call_sel_1".to_string()));
assert_eq!(messages[2].content, "found 3 results");
assert_eq!(messages[3].role, Role::Tool);
assert_eq!(messages[3].tool_call_id, Some("call_sel_2".to_string()));
assert_eq!(messages[3].content, "200 OK");
}
/// Regression: the OLD buggy worker code pushed tool_result messages
/// without a preceding assistant_with_tool_calls, causing
/// sanitize_tool_messages to rewrite them as orphaned user messages.
/// This test reproduces that buggy sequence and confirms the rewrite.
#[test]
fn test_sanitize_rewrites_orphaned_tool_results() {
let mut messages = vec![
ChatMessage::system("You are a helpful assistant."),
// No assistant_with_tool_calls -- mimics the old bug.
ChatMessage::tool_result("call_bug_1", "search", "found 3 results"),
ChatMessage::tool_result("call_bug_2", "http", "200 OK"),
];
sanitize_tool_messages(&mut messages);
// Both tool_results must be rewritten to Role::User.
assert_eq!(messages[1].role, Role::User);
assert!(messages[1].content.contains("[Tool `search` returned:"));
assert!(messages[1].content.contains("found 3 results"));
assert!(messages[1].tool_call_id.is_none());
assert!(messages[1].name.is_none());
assert_eq!(messages[2].role, Role::User);
assert!(messages[2].content.contains("[Tool `http` returned:"));
assert!(messages[2].content.contains("200 OK"));
assert!(messages[2].tool_call_id.is_none());
assert!(messages[2].name.is_none());
}
}
+15 -88
View File
@@ -8,9 +8,9 @@ use serde::{Deserialize, Serialize};
use crate::error::LlmError;
use crate::llm::{
ChatMessage, CompletionRequest, LlmProvider, Role, ToolCall, ToolCompletionRequest,
ToolDefinition,
ChatMessage, CompletionRequest, LlmProvider, ToolCall, ToolCompletionRequest, ToolDefinition,
};
use crate::safety::SafetyLayer;
/// Token the agent returns when it has nothing to say (e.g. in group chats).
/// The dispatcher should check for this and suppress the message.
@@ -342,6 +342,8 @@ pub struct RespondOutput {
/// Reasoning engine for the agent.
pub struct Reasoning {
llm: Arc<dyn LlmProvider>,
#[allow(dead_code)] // Will be used for sanitizing tool outputs
safety: Arc<SafetyLayer>,
/// Optional workspace for loading identity/system prompts.
workspace_system_prompt: Option<String>,
/// Optional skill context block to inject into system prompt.
@@ -359,9 +361,10 @@ pub struct Reasoning {
impl Reasoning {
/// Create a new reasoning engine.
pub fn new(llm: Arc<dyn LlmProvider>) -> Self {
pub fn new(llm: Arc<dyn LlmProvider>, safety: Arc<SafetyLayer>) -> Self {
Self {
llm,
safety,
workspace_system_prompt: None,
skill_context: None,
channel: None,
@@ -457,15 +460,8 @@ impl Reasoning {
pub async fn plan(&self, context: &ReasoningContext) -> Result<ActionPlan, LlmError> {
let system_prompt = self.build_planning_prompt(context);
let system_prompt = merge_system_messages(system_prompt, &context.messages);
let mut messages = vec![ChatMessage::system(system_prompt)];
messages.extend(
context
.messages
.iter()
.filter(|m| m.role != Role::System)
.cloned(),
);
messages.extend(context.messages.clone());
if let Some(ref job) = context.job_description {
messages.push(ChatMessage::user(format!(
@@ -616,15 +612,8 @@ Respond in JSON format:
None => self.build_system_prompt_with_tools(&context.available_tools),
};
let system_prompt = merge_system_messages(system_prompt, &context.messages);
let mut messages = vec![ChatMessage::system(system_prompt)];
messages.extend(
context
.messages
.iter()
.filter(|m| m.role != Role::System)
.cloned(),
);
messages.extend(context.messages.clone());
let effective_tools = if context.force_text {
Vec::new()
@@ -1037,22 +1026,6 @@ pub struct SuccessEvaluation {
pub suggestions: Vec<String>,
}
/// Merge the reasoning method's system prompt with any system messages already
/// present in the conversation context. Strict LLM providers (e.g. Qwen)
/// reject conversations with system messages that are not at the very
/// beginning, so we concatenate all system content into a single prompt.
fn merge_system_messages(primary: String, context_messages: &[ChatMessage]) -> String {
let extra: Vec<&str> = context_messages
.iter()
.filter(|m| m.role == Role::System)
.map(|m| m.content.as_str())
.collect();
if extra.is_empty() {
return primary;
}
format!("{}\n\n---\n\n{}", primary, extra.join("\n\n"))
}
/// Extract JSON from text that might contain other content.
fn extract_json(text: &str) -> Option<&str> {
// Find the first { and last } to extract JSON
@@ -2113,9 +2086,15 @@ That's my plan."#;
// ---- System prompt building tests (issue #565) ----
fn make_test_reasoning() -> Reasoning {
use crate::config::SafetyConfig;
use crate::safety::SafetyLayer;
use crate::testing::StubLlm;
let llm = Arc::new(StubLlm::new("test"));
Reasoning::new(llm)
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
Reasoning::new(llm, safety)
}
#[test]
@@ -2219,58 +2198,6 @@ That's my plan."#;
assert!(cleaned.contains("Here are the results."));
}
// ---- merge_system_messages: duplicate system message regression (Bug #597) ----
#[test]
fn test_merge_system_messages_no_system_in_context() {
let messages = vec![
ChatMessage::user("Hello"),
ChatMessage::assistant("Hi there"),
];
let result = merge_system_messages("primary prompt".into(), &messages);
assert_eq!(result, "primary prompt");
}
#[test]
fn test_merge_system_messages_merges_worker_system() {
let messages = vec![
ChatMessage::system("You are an autonomous agent working on a job.\n\nJob: Test Job"),
ChatMessage::user("Do the thing"),
];
let result = merge_system_messages("planning prompt".into(), &messages);
assert!(
result.contains("planning prompt"),
"must contain the primary prompt"
);
assert!(
result.contains("autonomous agent"),
"must contain worker system text"
);
assert!(
result.contains("Test Job"),
"must contain job description from worker system message"
);
}
#[test]
fn test_merge_system_messages_multiple_system() {
let messages = vec![
ChatMessage::system("First system instruction"),
ChatMessage::system("Second system instruction"),
ChatMessage::user("Hello"),
];
let result = merge_system_messages("primary".into(), &messages);
assert!(result.contains("primary"), "must contain primary prompt");
assert!(
result.contains("First system instruction"),
"must contain first system message"
);
assert!(
result.contains("Second system instruction"),
"must contain second system message"
);
}
#[test]
fn test_system_prompt_without_tools_omits_tools_section() {
let reasoning = make_test_reasoning();

Some files were not shown because too many files have changed in this diff Show More