Compare commits

..
Author SHA1 Message Date
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Illia Polosukhin
b8901baafd chore: release v0.10.0 (#279)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-22 18:16:21 +00:00
4003300a8c fix: improve Telegram status delivery and reliability (#304)
* fix: make Telegram status prompts reliable

Approval and auth prompts could be missed when polling or reply-context sends failed, leaving users stuck in waiting states. This adds explicit status mapping and retries, keeps typing active through intermediate work while suppressing noisy tool telemetry, and adds regression tests plus CI coverage for the Telegram channel crate.

* fix: normalize terminal status handling

Terminal status strings from the agent loop can vary in casing and formatting, which could leak internal status lines to Telegram. This normalizes Done/Interrupted mapping and filters terminal status text consistently to keep chat UX clean while preserving actionable prompts.

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 18:07:57 +00:00
Robert YanandGitHub c68dc2ff2a feat: update dashboard favicon (#309) 2026-02-22 18:06:50 +00:00
d4785ce4d2 fix: persist user message at turn start before agentic loop (#305)
* fix: persist user message at turn start before agentic loop

Split persist_turn into persist_user_message + persist_assistant_response.
The user message is now written to DB immediately after thread.start_turn(),
before the agentic loop runs. This ensures the message survives process
crashes mid-response. The assistant response is persisted only on completion.

Updated all 6 call sites in thread_ops.rs (success, error, approval
success/error, rejection, and auth intercept paths).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: document persist_assistant_response dependency on persist_user_message

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: re-ensure conversation in persist_assistant_response

Add ensure_conversation call and user_id parameter to
persist_assistant_response so assistant replies are still persisted
even if persist_user_message failed transiently at turn start.

Addresses PR review feedback from @ilblackdragon.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:50:28 +00:00
2544df1c4a feat: add web UI test skill for Chrome extension (#302)
* feat: add web UI test skill for Chrome extension testing

Add a SKILL.md checklist for manually testing the IronClaw web gateway
UI using the Claude for Chrome browser extension. Covers connection,
chat, skills tab (search, install by search, install by URL, remove),
and smoke tests for other tabs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use placeholder token and correct cleanup path per review

- Replace hardcoded test123 token with <your-token> placeholder
- Fix cleanup path: ~/.ironclaw/installed_skills/ (not skills/)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:49:35 +00:00
82f24bf08f fix: block send until thread is selected (#306)
* fix: block send until thread is selected

Prevents messages from ending up in orphan threads when user sends
while currentThreadId is null during page load.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: guard enableChatInput against null thread + add user feedback

Prevents SSE events from re-enabling input before a thread is selected.
Adds status message when user tries to send without a thread.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:19:18 +00:00
510fba4c92 fix: reload chat history on SSE reconnect (#307)
When SSE auto-reconnects after a server restart, the chat now
re-syncs from the database so no messages are lost.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:18:46 +00:00
04d3b005b1 feat: implement FullJob routine mode with scheduler dispatch (#288)
* feat: implement FullJob routine mode with scheduler dispatch

FullJob routines previously fell back to lightweight mode (single LLM call,
no tools) with a warning. This wires them to the existing Scheduler/Worker
infrastructure so they dispatch real jobs with full tool access.

Fire-and-forget model: the routine creates a job via ContextManager, schedules
it, links the routine_run to the job_id, and completes immediately. The job
runs independently with full tool access.

- Add RoutineError::JobDispatchFailed variant
- Add RoutineStore::link_routine_run_to_job (PostgreSQL + libSQL)
- Add execute_full_job() in routine_engine with context_manager/scheduler
- Wire context_manager + scheduler into RoutineEngine from agent_loop
- Fix pre-existing clippy warnings in tests/html_to_markdown.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: persist job to DB before scheduling in execute_full_job

The worker emits job_actions and llm_calls rows that reference agent_jobs
via foreign key. Without persisting the job first, those inserts can fail.
Match the pattern from commands.rs: fetch JobContext, save_job(), then schedule.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: consolidate job dispatch into Scheduler::dispatch_job and wire max_iterations

Move the create + persist + schedule sequence into a single
Scheduler::dispatch_job() method so callers (commands.rs, routine_engine.rs)
don't duplicate the logic. FullJob routines now pass max_iterations via job
metadata, and the worker reads it (defaulting to 50 if unset).

Also removes the context_manager field from RoutineEngine since dispatch_job
handles everything internally.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: clamp max_iterations to 500 and log category update failures

Address PR review feedback:
- worker.rs: clamp max_iterations from metadata to MAX_WORKER_ITERATIONS (500)
  to prevent unbounded LLM token usage from malicious/buggy configs
- commands.rs: log warning on category update failure instead of silently
  discarding the error

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:18:21 +00:00
ea57447649 feat: hot-activate WASM channels, channel-first prompts, unified artifact resolution (#297)
* refactor: unify WASM artifact resolution into registry/artifacts.rs

Consolidate duplicated WASM find/build/install logic from 5+ files into
a single src/registry/artifacts.rs module. This fixes two bugs:
- registry/installer.rs now respects CARGO_TARGET_DIR (was hardcoded)
- channels/wasm/bundled.rs now searches all WASM triples (was wasip2 only)

Also includes: extension manager hot-activation for WASM channels,
extension guidance in LLM prompts, channel manager hot-add support,
webhook router channel lookup, and minor cleanups.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: send approval prompts as messages on WASM channels (Telegram, Slack)

WASM channels mapped ApprovalNeeded status to a typing indicator,
so users on Telegram never saw tool approval prompts — the agent
got stuck in AwaitingApproval and all subsequent messages failed
with "Waiting for approval".

- Intercept ApprovalNeeded in WasmChannel::handle_status_update and
  send the prompt as an actual message via call_on_respond, showing
  tool name, description, parameters, and yes/no/always instructions
- Guard against empty LLM responses after clean_response() strips
  reasoning_content think-tags (defense-in-depth for reasoning models)
- Add reasoning_content fallback to NearAiChatProvider::complete()
  for consistency with complete_with_tools()
- Add debug logging when empty responses are suppressed
- Improve error logging for channel respond() failures
- Register WASM channel webhook routes before credential checks so
  platforms don't deactivate webhook URLs with 404s

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #297 review comments

- ChannelManager::add: use async write().await instead of try_write()
- resolve_target_dir: resolve relative CARGO_TARGET_DIR against crate_dir
- install_wasm_files: log warning on capabilities copy failure
- refresh_active_channel: load capabilities file for webhook secret name
- activate_wasm_channel: validate name against path traversal
- Fix cargo fmt formatting in nearai_chat.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire up channel runtime for hot-activation and address PR review round 2

- Wire up set_channel_runtime() in main.rs so hot-activation actually works
  (with_channel_runtime was never called — hot-activation was dead code)
- Change ExtensionManager channel runtime fields to RwLock<Option<...>>
  interior mutability so set_channel_runtime(&self) works after Arc wrapping
- Fix artifact tests to use resolve_target_dir() instead of hardcoding
  "target/" (breaks when CARGO_TARGET_DIR is set)
- Fix bundled.rs build hint: cargo component build (not cargo build --target)
- Fix wasm_artifact_path doc: binary_name should not include .wasm extension

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use char-aware truncation to prevent UTF-8 panic in approval prompt

&s[..77] panics on multi-byte UTF-8 (CJK, emoji). Use s.chars().take(77)
for safe truncation at character boundaries.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:09:56 +00:00
a320f265b3 Fix tool schema OpenAI compatibility (#301)
* fix: remove union type arrays from tool schemas for OpenAI compatibility

OpenAI rejects JSON Schema union types containing "array" without an
"items" subschema. The http tool's "body" and json tool's "data" params
used union types to accept any value. Replace with freeform (untyped)
schemas which OpenAI treats as accepting any JSON value.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: update schema tests to assert type is absent, fix missed json.rs test

- http.rs test: assert body has no "type" (not just has description)
- json.rs test: update to match the freeform schema change (was still
  asserting type is present)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 07:52:34 +00:00
c3ce26278a refactor: simplify config resolution and consolidate main.rs init (#287)
* refactor: simplify config resolution and consolidate main.rs init into AppBuilder

- Add parse_bool_env() and parse_string_env() helpers to eliminate repetitive
  5-line optional_env/parse/map_err/unwrap_or boilerplate across 12 config files
- Add EmbeddingsConfig::create_provider() to centralize embeddings construction
  (fixes hardcoded 1536 dimensions and missing Ollama provider in app.rs)
- Extract init_cli_tracing(), setup_wasm_channels(), start_tunnel(),
  run_memory_command(), run_worker(), run_claude_bridge() from main.rs
- Replace ~600 lines of inline init in main.rs with AppBuilder::build_all()
- Expose catalog_entries from AppComponents for gateway registry entries
- Net reduction: ~738 lines across 15 files

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: propagate dev_loaded_tool_names from AppBuilder and add parse_option_env helper

Address PR review feedback:

- Capture dev_loaded_tool_names from WASM loading in init_extensions()
  and expose via AppComponents so bootstrap_hooks receives the actual
  dev tool names instead of an empty slice (fixes silent hook skip)
- Add parse_option_env<T>() helper for Option<T> config fields,
  simplifying max_cost_per_day_cents and max_actions_per_hour in agent.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: fetch real NEAR AI pricing and unify cost calculation path

CostGuard was independently looking up pricing via costs::model_cost(),
falling back to GPT-4o default rates when NEAR AI model names didn't
match the static table — causing ~3x cost overestimates in logs.

- Add pricing map to NearAiChatProvider that fetches real rates from
  /v1/model/list at startup (background, non-blocking)
- Update cost_per_token() to check fetched pricing first, then static
  table, then default
- Add cost_per_token parameter to CostGuard::record_llm_call() so the
  dispatcher passes provider-sourced rates directly

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: update default NEAR AI model to GLM-latest

Replace fireworks llama4-maverick-instruct-basic with zai-org/GLM-latest
as the default model in config and setup wizard.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: align wizard default model name with config

Change "zai/GLM-latest" to "zai-org/GLM-latest" in wizard.rs to match
the default in config/llm.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 02:54:31 +00:00
mfcoburnandGitHub 91b602790a Update image source in README.md 2026-02-21 17:21:50 -05:00
mfcoburnandGitHub e5ce076773 Add files via upload 2026-02-21 15:14:57 -07:00
c1f3b83c98 refactor: remove ExtensionSource::Bundled, use download-only install for WASM channels (#293)
The Bundled variant and its local-artifacts fallback are superseded by the
embedded registry catalog which provides WasmDownload entries with GitHub
release URLs. The in-chat extension manager now always downloads channel
WASM binaries from releases, simplifying the install path.

The setup wizard retains its own local install_bundled_channel path for
dev builds where build artifacts exist on disk.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 13:55:49 -08:00
80 changed files with 4103 additions and 3433 deletions
+2
View File
@@ -19,3 +19,5 @@ jobs:
- uses: Swatinem/rust-cache@v2
- name: Run Tests
run: cargo test --all-features -- --nocapture
- name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
+37
View File
@@ -7,6 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.10.0](https://github.com/nearai/ironclaw/compare/v0.9.0...v0.10.0) - 2026-02-22
### Added
- update dashboard favicon ([#309](https://github.com/nearai/ironclaw/pull/309))
- add web UI test skill for Chrome extension ([#302](https://github.com/nearai/ironclaw/pull/302))
- implement FullJob routine mode with scheduler dispatch ([#288](https://github.com/nearai/ironclaw/pull/288))
- hot-activate WASM channels, channel-first prompts, unified artifact resolution ([#297](https://github.com/nearai/ironclaw/pull/297))
- add pairing/permission system to all WASM channels and fix extension registry ([#286](https://github.com/nearai/ironclaw/pull/286))
- group chat privacy, channel-aware prompts, and safety hardening ([#285](https://github.com/nearai/ironclaw/pull/285))
- embedded registry catalog and WASM bundle install pipeline ([#283](https://github.com/nearai/ironclaw/pull/283))
- show token usage and cost tracker in gateway status popover ([#284](https://github.com/nearai/ironclaw/pull/284))
- support custom HTTP headers for OpenAI-compatible provider ([#269](https://github.com/nearai/ironclaw/pull/269))
- add smart routing provider for cost-optimized model selection ([#281](https://github.com/nearai/ironclaw/pull/281))
### Fixed
- persist user message at turn start before agentic loop ([#305](https://github.com/nearai/ironclaw/pull/305))
- block send until thread is selected ([#306](https://github.com/nearai/ironclaw/pull/306))
- reload chat history on SSE reconnect ([#307](https://github.com/nearai/ironclaw/pull/307))
- map Esc to interrupt and Ctrl+C to graceful quit ([#267](https://github.com/nearai/ironclaw/pull/267))
### Other
- Fix tool schema OpenAI compatibility ([#301](https://github.com/nearai/ironclaw/pull/301))
- simplify config resolution and consolidate main.rs init ([#287](https://github.com/nearai/ironclaw/pull/287))
- Update image source in README.md
- Add files via upload
- remove ExtensionSource::Bundled, use download-only install for WASM channels ([#293](https://github.com/nearai/ironclaw/pull/293))
- allow OAuth callback to work on remote servers (fixes #186) ([#212](https://github.com/nearai/ironclaw/pull/212))
- add rate limiting for built-in tools (closes #171) ([#276](https://github.com/nearai/ironclaw/pull/276))
- add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) ([#193](https://github.com/nearai/ironclaw/pull/193))
- Feat/html to markdown #106 ([#115](https://github.com/nearai/ironclaw/pull/115))
- adopt agent-market design language for web UI ([#282](https://github.com/nearai/ironclaw/pull/282))
- speed up startup from ~15s to ~2s ([#280](https://github.com/nearai/ironclaw/pull/280))
- consolidate tool approval into single param-aware method ([#274](https://github.com/nearai/ironclaw/pull/274))
## [0.9.0](https://github.com/nearai/ironclaw/compare/v0.8.0...v0.9.0) - 2026-02-21
### Added
+4
View File
@@ -572,6 +572,10 @@ Four built-in tools for managing skills at runtime:
- `<workspace>/skills/` -- Per-workspace skills (trusted)
- `~/.ironclaw/installed_skills/` -- Registry-installed skills (installed trust)
### Testing Skills
- `skills/web-ui-test/` -- Manual test checklist for the web gateway UI via Claude for Chrome extension. Covers connection, chat, skills search/install/remove, and other tabs.
Skills configuration: see Configuration section above.
## Docker Sandbox
Generated
+10 -10
View File
@@ -2679,7 +2679,7 @@ dependencies = [
[[package]]
name = "ironclaw"
version = "0.9.0"
version = "0.10.0"
dependencies = [
"aes-gcm",
"aho-corasick",
@@ -4975,15 +4975,6 @@ dependencies = [
"syn 2.0.114",
]
[[package]]
name = "servo_arc"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "170fb83ab34de17dc69aa7c67482b22218ddb85da56546f9bd6b929e32a05930"
dependencies = [
"stable_deref_trait",
]
[[package]]
name = "serde_yml"
version = "0.0.12"
@@ -4999,6 +4990,15 @@ dependencies = [
"version_check",
]
[[package]]
name = "servo_arc"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "170fb83ab34de17dc69aa7c67482b22218ddb85da56546f9bd6b929e32a05930"
dependencies = [
"stable_deref_trait",
]
[[package]]
name = "sha1"
version = "0.10.6"
+2 -2
View File
@@ -19,7 +19,7 @@ exclude = [
[package]
name = "ironclaw"
version = "0.9.0"
version = "0.10.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -41,7 +41,7 @@ tokio-stream = { version = "0.1", features = ["sync"] }
futures = "0.3"
# HTTP client
reqwest = { version = "0.12", default-features = false, features = ["json", "multipart", "rustls-tls-native-roots", "stream"] }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-native-roots", "stream"] }
# Serialization
serde = { version = "1", features = ["derive"] }
+1 -1
View File
@@ -120,7 +120,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Per-group tool policies | ✅ | ❌ | Allow/deny specific tools |
| Thread isolation | ✅ | ✅ | Separate sessions per thread |
| Per-channel media limits | ✅ | 🚧 | Caption support for media; no size limits |
| Typing indicators | ✅ | 🚧 | TUI shows status |
| Typing indicators | ✅ | 🚧 | TUI + Telegram typing/actionable status prompts; richer parity pending |
| Per-channel ackReaction config | ✅ | ❌ | Customizable acknowledgement reactions |
| Group session priming | ✅ | ❌ | Member roster injected for context |
| Sender_id in trusted metadata | ✅ | ❌ | Exposed in system metadata |
+1 -1
View File
@@ -1,5 +1,5 @@
<p align="center">
<img src="ironclaw.png" alt="IronClaw" width="200"/>
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
+2 -1
View File
@@ -17,7 +17,6 @@ serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
# Exclude from parent workspace (this is a standalone WASM component)
[workspace]
[profile.release]
# Optimize for size
@@ -25,3 +24,5 @@ opt-level = "s"
lto = true
strip = true
codegen-units = 1
[workspace]
+524 -299
View File
@@ -76,9 +76,6 @@ struct TelegramMessage {
#[serde(default)]
caption: Option<String>,
/// Voice message.
voice: Option<TelegramVoice>,
/// Original message if this is a reply.
reply_to_message: Option<Box<TelegramMessage>>,
@@ -142,36 +139,6 @@ struct MessageEntity {
user: Option<TelegramUser>,
}
/// Telegram Voice object.
/// https://core.telegram.org/bots/api#voice
#[derive(Debug, Deserialize)]
struct TelegramVoice {
/// Identifier for this file, which can be used to download the file.
file_id: String,
/// Duration of the audio in seconds.
duration: u32,
/// MIME type of the file.
#[serde(default)]
mime_type: Option<String>,
/// File size in bytes.
#[serde(default)]
file_size: Option<i64>,
}
/// Telegram File object returned by getFile.
/// https://core.telegram.org/bots/api#file
#[derive(Debug, Deserialize)]
struct TelegramFile {
/// Identifier for this file.
file_id: String,
/// File path for downloading. Use https://api.telegram.org/file/bot<token>/<file_path>.
file_path: Option<String>,
}
/// Telegram API response wrapper.
#[derive(Debug, Deserialize)]
struct TelegramApiResponse<T> {
@@ -277,6 +244,67 @@ struct TelegramConfig {
struct TelegramChannel;
#[derive(Debug, Clone, PartialEq, Eq)]
enum TelegramStatusAction {
Typing,
Notify(String),
}
const TELEGRAM_STATUS_MAX_CHARS: usize = 600;
fn truncate_status_message(input: &str, max_chars: usize) -> String {
let mut iter = input.chars();
let truncated: String = iter.by_ref().take(max_chars).collect();
if iter.next().is_some() {
format!("{}...", truncated)
} else {
truncated
}
}
fn status_message_for_user(update: &StatusUpdate) -> Option<String> {
let message = update.message.trim();
if message.is_empty() {
None
} else {
Some(truncate_status_message(message, TELEGRAM_STATUS_MAX_CHARS))
}
}
fn get_updates_url(offset: i64, timeout_secs: u32) -> String {
format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getUpdates?offset={}&timeout={}&allowed_updates=[\"message\",\"edited_message\"]",
offset, timeout_secs
)
}
fn classify_status_update(update: &StatusUpdate) -> Option<TelegramStatusAction> {
match update.status {
StatusType::Thinking => Some(TelegramStatusAction::Typing),
StatusType::Done | StatusType::Interrupted => None,
// Tool telemetry can be noisy in chat; keep it as typing-only UX.
StatusType::ToolStarted | StatusType::ToolCompleted | StatusType::ToolResult => None,
StatusType::Status => {
let msg = update.message.trim();
if msg.eq_ignore_ascii_case("Done")
|| msg.eq_ignore_ascii_case("Interrupted")
|| msg.eq_ignore_ascii_case("Awaiting approval")
|| msg.eq_ignore_ascii_case("Rejected")
{
None
} else {
status_message_for_user(update).map(TelegramStatusAction::Notify)
}
}
StatusType::ApprovalNeeded
| StatusType::JobStarted
| StatusType::AuthRequired
| StatusType::AuthCompleted => {
status_message_for_user(update).map(TelegramStatusAction::Notify)
}
}
}
impl Guest for TelegramChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
channel_host::log(
@@ -455,20 +483,36 @@ impl Guest for TelegramChannel {
&format!("Polling getUpdates with offset {}", offset),
);
// Build getUpdates URL with parameters
// - offset: Identifier of the first update to be returned
// - timeout: Long polling timeout in seconds (Telegram recommends 30+)
// - allowed_updates: Only get message updates
let url = format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getUpdates?offset={}&timeout=30&allowed_updates=[\"message\",\"edited_message\"]",
offset
);
let headers_json = serde_json::json!({}).to_string();
let primary_url = get_updates_url(offset, 30);
let headers = serde_json::json!({});
// 35s HTTP timeout outlives Telegram's 30s server-side long-poll.
// If the TCP connection drops, retry once immediately with a short poll
// so we don't wait a full extra tick (~30s) before delivering updates.
let result = match channel_host::http_request(
"GET",
&primary_url,
&headers_json,
None,
Some(35_000),
) {
Ok(response) => Ok(response),
Err(primary_err) => {
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"getUpdates request failed ({}), retrying once immediately",
primary_err
),
);
// 35s HTTP timeout outlives Telegram's 30s server-side long-poll
let result =
channel_host::http_request("GET", &url, &headers.to_string(), None, Some(35_000));
let retry_url = get_updates_url(offset, 3);
channel_host::http_request("GET", &retry_url, &headers_json, None, Some(8_000))
.map_err(|retry_err| {
format!("primary error: {}; retry error: {}", primary_err, retry_err)
})
}
};
match result {
Ok(response) => {
@@ -549,7 +593,7 @@ impl Guest for TelegramChannel {
let result = send_message(
metadata.chat_id,
&response.content,
metadata.message_id,
Some(metadata.message_id),
Some("Markdown"),
);
@@ -572,7 +616,7 @@ impl Guest for TelegramChannel {
let msg_id = send_message(
metadata.chat_id,
&response.content,
metadata.message_id,
Some(metadata.message_id),
None,
)
.map_err(|e| format!("Plain-text retry also failed: {}", e))?;
@@ -591,10 +635,10 @@ impl Guest for TelegramChannel {
}
fn on_status(update: StatusUpdate) {
// Only send typing indicator for Thinking status
if !matches!(update.status, StatusType::Thinking) {
return;
}
let action = match classify_status_update(&update) {
Some(action) => action,
None => return,
};
// Parse chat_id from metadata
let metadata: TelegramMessageMetadata = match serde_json::from_str(&update.metadata_json) {
@@ -602,40 +646,68 @@ impl Guest for TelegramChannel {
Err(_) => {
channel_host::log(
channel_host::LogLevel::Debug,
"on_status: no valid Telegram metadata, skipping typing indicator",
"on_status: no valid Telegram metadata, skipping status update",
);
return;
}
};
// POST /sendChatAction with action "typing"
let payload = serde_json::json!({
"chat_id": metadata.chat_id,
"action": "typing"
});
match action {
TelegramStatusAction::Typing => {
// POST /sendChatAction with action "typing"
let payload = serde_json::json!({
"chat_id": metadata.chat_id,
"action": "typing"
});
let payload_bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,
Err(_) => return,
};
let payload_bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,
Err(_) => return,
};
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendChatAction",
&headers.to_string(),
Some(&payload_bytes),
None,
);
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendChatAction",
&headers.to_string(),
Some(&payload_bytes),
None,
);
if let Err(e) = result {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("sendChatAction failed: {}", e),
);
if let Err(e) = result {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("sendChatAction failed: {}", e),
);
}
}
TelegramStatusAction::Notify(prompt) => {
// Send user-visible status updates for actionable events.
if let Err(first_err) =
send_message(metadata.chat_id, &prompt, Some(metadata.message_id), None)
{
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"Failed to send status reply ({}), retrying without reply context",
first_err
),
);
if let Err(retry_err) = send_message(metadata.chat_id, &prompt, None, None) {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Failed to send status message without reply context: {}",
retry_err
),
);
}
}
}
}
}
@@ -676,15 +748,18 @@ impl std::fmt::Display for SendError {
fn send_message(
chat_id: i64,
text: &str,
reply_to_message_id: i64,
reply_to_message_id: Option<i64>,
parse_mode: Option<&str>,
) -> Result<i64, SendError> {
let mut payload = serde_json::json!({
"chat_id": chat_id,
"text": text,
"reply_to_message_id": reply_to_message_id,
});
if let Some(message_id) = reply_to_message_id {
payload["reply_to_message_id"] = serde_json::Value::Number(message_id.into());
}
if let Some(mode) = parse_mode {
payload["parse_mode"] = serde_json::Value::String(mode.to_string());
}
@@ -864,121 +939,17 @@ fn register_webhook(tunnel_url: &str, webhook_secret: Option<&str>) -> Result<()
/// Send a pairing code message to a chat. Used when an unknown user DMs the bot.
fn send_pairing_reply(chat_id: i64, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"chat_id": chat_id,
"text": format!(
send_message(
chat_id,
&format!(
"To pair with this bot, run: `ironclaw pairing approve telegram {}`",
code
),
"parse_mode": "Markdown",
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize payload: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) => {
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!("HTTP {}: {}", response.status, body_str));
}
Ok(())
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
// ============================================================================
// Voice File Download
// ============================================================================
/// Download a voice file from Telegram by file_id.
///
/// 1. Call getFile to get the file_path.
/// 2. Download the file bytes from /file/bot{TOKEN}/{file_path}.
fn download_voice_file(file_id: &str) -> Result<Vec<u8>, String> {
// Reject file_id containing curly braces to prevent credential placeholder
// injection (e.g., a malicious file_id like "{OPENAI_API_KEY}" would be
// interpreted by the host-side credential injector).
if file_id.contains('{') || file_id.contains('}') {
return Err("invalid file_id: contains forbidden characters".to_string());
}
// Step 1: Call getFile to get file_path
// Double braces `{{...}}` produce a literal `{TELEGRAM_BOT_TOKEN}` placeholder
// in the URL, which the host-side credential injector replaces with the real token.
let get_file_url = format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getFile?file_id={}",
file_id
);
let headers = serde_json::json!({});
let result = channel_host::http_request("GET", &get_file_url, &headers.to_string(), None, None);
let response = result.map_err(|e| format!("getFile request failed: {}", e))?;
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!("getFile returned {}: {}", response.status, body_str));
}
let api_response: TelegramApiResponse<TelegramFile> =
serde_json::from_slice(&response.body)
.map_err(|e| format!("Failed to parse getFile response: {}", e))?;
if !api_response.ok {
return Err(format!(
"getFile API error: {}",
api_response
.description
.unwrap_or_else(|| "unknown".to_string())
));
}
let file = api_response
.result
.ok_or_else(|| "getFile returned no result".to_string())?;
let file_path = file
.file_path
.ok_or_else(|| "getFile returned no file_path".to_string())?;
// Sanitize file_path against credential placeholder injection
if file_path.contains('{') || file_path.contains('}') {
return Err("invalid file_path: contains forbidden characters".to_string());
}
// Step 2: Download the actual file bytes
let download_url = format!(
"https://api.telegram.org/file/bot{{TELEGRAM_BOT_TOKEN}}/{}",
file_path
);
let result =
channel_host::http_request("GET", &download_url, &headers.to_string(), None, None);
let response = result.map_err(|e| format!("File download failed: {}", e))?;
if response.status != 200 {
return Err(format!(
"File download returned status {}",
response.status
));
}
Ok(response.body)
Some("Markdown"),
)
.map(|_| ())
.map_err(|e| e.to_string())
}
// ============================================================================
@@ -1000,9 +971,6 @@ fn handle_update(update: TelegramUpdate) {
/// Process a single message.
fn handle_message(message: TelegramMessage) {
// Check for voice note first (voice-only messages have no text)
let is_voice = message.voice.is_some();
// Use text or caption (for media messages)
let content = message
.text
@@ -1010,8 +978,7 @@ fn handle_message(message: TelegramMessage) {
.or_else(|| message.caption.filter(|c| !c.is_empty()))
.unwrap_or_default();
// Allow voice notes through even when content is empty
if content.is_empty() && !is_voice {
if content.is_empty() {
return;
}
@@ -1145,79 +1112,17 @@ fn handle_message(message: TelegramMessage) {
let metadata_json = serde_json::to_string(&metadata).unwrap_or_else(|_| "{}".to_string());
// Clean the message text (strip bot mentions and commands)
let bot_username = channel_host::workspace_read(BOT_USERNAME_PATH).unwrap_or_default();
let cleaned_text = clean_message_text(
let content_to_emit = match content_to_emit_for_agent(
&content,
if bot_username.is_empty() {
None
} else {
Some(bot_username.as_str())
},
);
// Handle voice notes: download and attach audio bytes.
// Note: download is synchronous (two HTTP roundtrips to Telegram API).
// This blocks the WASM execution for the current polling tick.
let mut attachments = Vec::new();
let mut voice_download_failed = false;
if let Some(ref voice) = message.voice {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Voice note from user {} (duration: {}s, file_id: {})",
from.id, voice.duration, voice.file_id
),
);
match download_voice_file(&voice.file_id) {
Ok(audio_bytes) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!("Downloaded voice file: {} bytes", audio_bytes.len()),
);
attachments.push(channel_host::Attachment {
kind: channel_host::AttachmentKind::Audio,
mime_type: voice
.mime_type
.clone()
.unwrap_or_else(|| "audio/ogg".to_string()),
data: audio_bytes,
filename: Some(format!("voice_{}.ogg", voice.file_id)),
duration_secs: Some(voice.duration),
});
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to download voice file: {}", e),
);
voice_download_failed = true;
}
}
}
// Determine what to emit to the agent.
// - Voice notes: use "[Voice note]" as content (transcription happens host-side)
// - `/start` (no args): emit a welcome placeholder so the agent greets the user
// - Other bare `/commands` (e.g. /interrupt, /help): pass the raw command through
// so Submission::parse() can handle it
// - Commands with args (e.g. `/start hello`): cleaned_text already has the args
// - Plain text: pass through as-is
let trimmed_content = content.trim();
let content_to_emit = if is_voice && voice_download_failed && content.is_empty() {
"[Voice note: download failed]".to_string()
} else if is_voice && content.is_empty() {
"[Voice note]".to_string()
} else if trimmed_content.eq_ignore_ascii_case("/start") {
"[User started the bot]".to_string()
} else if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
// Bare control command like /interrupt, /stop, /help — pass through raw
trimmed_content.to_string()
} else if cleaned_text.is_empty() && !is_voice {
return;
} else {
cleaned_text
) {
Some(value) => value,
None => return,
};
// Emit the message to the agent
@@ -1227,7 +1132,6 @@ fn handle_message(message: TelegramMessage) {
content: content_to_emit,
thread_id: None, // Telegram doesn't have threads in the same way
metadata_json,
attachments,
});
channel_host::log(
@@ -1286,6 +1190,31 @@ fn clean_message_text(text: &str, bot_username: Option<&str>) -> String {
result
}
/// Decide which user content should be emitted to the agent loop.
///
/// - `/start` emits a placeholder so the agent can greet the user
/// - bare slash commands are passed through for Submission parsing
/// - empty/mention-only messages are ignored
/// - otherwise cleaned text is emitted
fn content_to_emit_for_agent(content: &str, bot_username: Option<&str>) -> Option<String> {
let cleaned_text = clean_message_text(content, bot_username);
let trimmed_content = content.trim();
if trimmed_content.eq_ignore_ascii_case("/start") {
return Some("[User started the bot]".to_string());
}
if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
return Some(trimmed_content.to_string());
}
if cleaned_text.is_empty() {
return None;
}
Some(cleaned_text)
}
// ============================================================================
// Utilities
// ============================================================================
@@ -1346,62 +1275,126 @@ mod tests {
// Commands with args: command prefix stripped, args returned
assert_eq!(clean_message_text("/start hello", None), "hello");
assert_eq!(clean_message_text("/help me please", None), "me please");
assert_eq!(clean_message_text("/model claude-opus-4-6", None), "claude-opus-4-6");
assert_eq!(
clean_message_text("/model claude-opus-4-6", None),
"claude-opus-4-6"
);
}
/// Tests for the content_to_emit logic in handle_message.
/// Since handle_message uses WASM host calls, we test the decision logic inline.
/// Since handle_message uses WASM host calls, test the extracted decision function.
#[test]
fn test_content_to_emit_logic() {
// Simulates the content_to_emit decision for various inputs.
// This mirrors the logic in handle_message after clean_message_text.
fn resolve_content(content: &str) -> Option<String> {
let cleaned_text = clean_message_text(content, None);
let trimmed_content = content.trim();
if trimmed_content.eq_ignore_ascii_case("/start") {
Some("[User started the bot]".to_string())
} else if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
Some(trimmed_content.to_string())
} else if cleaned_text.is_empty() {
None // would return/skip in handle_message
} else {
Some(cleaned_text)
}
}
// /start → welcome placeholder
assert_eq!(resolve_content("/start"), Some("[User started the bot]".to_string()));
assert_eq!(resolve_content("/Start"), Some("[User started the bot]".to_string()));
assert_eq!(resolve_content(" /start "), Some("[User started the bot]".to_string()));
assert_eq!(
content_to_emit_for_agent("/start", None),
Some("[User started the bot]".to_string())
);
assert_eq!(
content_to_emit_for_agent("/Start", None),
Some("[User started the bot]".to_string())
);
assert_eq!(
content_to_emit_for_agent(" /start ", None),
Some("[User started the bot]".to_string())
);
// /start with args → pass args through
assert_eq!(resolve_content("/start hello"), Some("hello".to_string()));
assert_eq!(
content_to_emit_for_agent("/start hello", None),
Some("hello".to_string())
);
// Control commands → pass through raw so Submission::parse() can match
assert_eq!(resolve_content("/interrupt"), Some("/interrupt".to_string()));
assert_eq!(resolve_content("/stop"), Some("/stop".to_string()));
assert_eq!(resolve_content("/help"), Some("/help".to_string()));
assert_eq!(resolve_content("/undo"), Some("/undo".to_string()));
assert_eq!(resolve_content("/redo"), Some("/redo".to_string()));
assert_eq!(resolve_content("/ping"), Some("/ping".to_string()));
assert_eq!(resolve_content("/tools"), Some("/tools".to_string()));
assert_eq!(resolve_content("/compact"), Some("/compact".to_string()));
assert_eq!(resolve_content("/clear"), Some("/clear".to_string()));
assert_eq!(resolve_content("/version"), Some("/version".to_string()));
assert_eq!(
content_to_emit_for_agent("/interrupt", None),
Some("/interrupt".to_string())
);
assert_eq!(
content_to_emit_for_agent("/stop", None),
Some("/stop".to_string())
);
assert_eq!(
content_to_emit_for_agent("/help", None),
Some("/help".to_string())
);
assert_eq!(
content_to_emit_for_agent("/undo", None),
Some("/undo".to_string())
);
assert_eq!(
content_to_emit_for_agent("/redo", None),
Some("/redo".to_string())
);
assert_eq!(
content_to_emit_for_agent("/ping", None),
Some("/ping".to_string())
);
assert_eq!(
content_to_emit_for_agent("/tools", None),
Some("/tools".to_string())
);
assert_eq!(
content_to_emit_for_agent("/compact", None),
Some("/compact".to_string())
);
assert_eq!(
content_to_emit_for_agent("/clear", None),
Some("/clear".to_string())
);
assert_eq!(
content_to_emit_for_agent("/version", None),
Some("/version".to_string())
);
assert_eq!(
content_to_emit_for_agent("/approve", None),
Some("/approve".to_string())
);
assert_eq!(
content_to_emit_for_agent("/always", None),
Some("/always".to_string())
);
assert_eq!(
content_to_emit_for_agent("/deny", None),
Some("/deny".to_string())
);
assert_eq!(
content_to_emit_for_agent("/yes", None),
Some("/yes".to_string())
);
assert_eq!(
content_to_emit_for_agent("/no", None),
Some("/no".to_string())
);
// Commands with args → cleaned text (command stripped)
assert_eq!(resolve_content("/help me please"), Some("me please".to_string()));
assert_eq!(
content_to_emit_for_agent("/help me please", None),
Some("me please".to_string())
);
// Plain text → pass through
assert_eq!(resolve_content("hello world"), Some("hello world".to_string()));
assert_eq!(resolve_content("just text"), Some("just text".to_string()));
assert_eq!(
content_to_emit_for_agent("hello world", None),
Some("hello world".to_string())
);
assert_eq!(
content_to_emit_for_agent("just text", None),
Some("just text".to_string())
);
// Empty / whitespace → skip (None)
assert_eq!(resolve_content(""), None);
assert_eq!(resolve_content(" "), None);
assert_eq!(content_to_emit_for_agent("", None), None);
assert_eq!(content_to_emit_for_agent(" ", None), None);
// Bare @mention without bot → skip
assert_eq!(resolve_content("@botname"), None);
assert_eq!(content_to_emit_for_agent("@botname", None), None);
// With bot username configured: other mentions are preserved.
assert_eq!(
content_to_emit_for_agent("@alice hello", Some("MyBot")),
Some("@alice hello".to_string())
);
}
#[test]
@@ -1482,4 +1475,236 @@ mod tests {
assert_eq!(msg.text, None);
assert_eq!(msg.caption.as_deref(), Some("What's in this image?"));
}
#[test]
fn test_get_updates_url_includes_offset_and_timeout() {
let url = get_updates_url(444_809_884, 30);
assert!(url.contains("offset=444809884"));
assert!(url.contains("timeout=30"));
assert!(url.contains("allowed_updates=[\"message\",\"edited_message\"]"));
}
#[test]
fn test_classify_status_update_thinking() {
let update = StatusUpdate {
status: StatusType::Thinking,
message: "Thinking...".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Typing)
);
}
#[test]
fn test_classify_status_update_approval_needed() {
let update = StatusUpdate {
status: StatusType::ApprovalNeeded,
message: "Approval needed for tool 'http_request'".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Approval needed for tool 'http_request'".to_string()
))
);
}
#[test]
fn test_classify_status_update_done_ignored() {
let update = StatusUpdate {
status: StatusType::Done,
message: "Done".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_auth_required() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: "Authentication required for weather.".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Authentication required for weather.".to_string()
))
);
}
#[test]
fn test_classify_status_update_tool_started_ignored() {
let update = StatusUpdate {
status: StatusType::ToolStarted,
message: "Tool started: http_request".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_tool_completed_ignored() {
let update = StatusUpdate {
status: StatusType::ToolCompleted,
message: "Tool completed: http_request (ok)".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_job_started_notify() {
let update = StatusUpdate {
status: StatusType::JobStarted,
message: "Job started: Daily sync".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Job started: Daily sync".to_string()
))
);
}
#[test]
fn test_classify_status_update_auth_completed_notify() {
let update = StatusUpdate {
status: StatusType::AuthCompleted,
message: "Authentication completed for weather.".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Authentication completed for weather.".to_string()
))
);
}
#[test]
fn test_classify_status_update_tool_result_ignored() {
let update = StatusUpdate {
status: StatusType::ToolResult,
message: "Tool result: http_request ...".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_awaiting_approval_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Awaiting approval".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_interrupted_ignored() {
let update = StatusUpdate {
status: StatusType::Interrupted,
message: "Interrupted".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_done_ignored_case_insensitive() {
let update = StatusUpdate {
status: StatusType::Status,
message: "done".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_interrupted_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "interrupted".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_rejected_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Rejected".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_notify() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Context compaction started".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Context compaction started".to_string()
))
);
}
#[test]
fn test_status_message_for_user_ignores_blank() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: " ".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(status_message_for_user(&update), None);
}
#[test]
fn test_truncate_status_message_appends_ellipsis() {
let input = "abcdefghijklmnopqrstuvwxyz";
let output = truncate_status_message(input, 10);
assert_eq!(output, "abcdefghij...");
}
#[test]
fn test_status_message_for_user_truncates_long_input() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: "x".repeat(700),
metadata_json: "{}".to_string(),
};
let msg = status_message_for_user(&update).expect("expected message");
assert!(msg.len() <= TELEGRAM_STATUS_MAX_CHARS + 3);
assert!(msg.ends_with("..."));
}
}
@@ -14,8 +14,7 @@
"capabilities": {
"http": {
"allowlist": [
{ "host": "api.telegram.org", "path_prefix": "/bot" },
{ "host": "api.telegram.org", "path_prefix": "/file/bot" }
{ "host": "api.telegram.org", "path_prefix": "/bot" }
],
"credentials": {
"telegram_bot": {
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

After

Width:  |  Height:  |  Size: 267 KiB

+106
View File
@@ -0,0 +1,106 @@
---
name: web-ui-test
version: 0.1.0
description: Test the IronClaw web UI using the Claude for Chrome browser extension.
activation:
keywords:
- test web ui
- test the ui
- browser test
- chrome test
- test skills tab
- test chat
- web gateway test
patterns:
- "test.*web.*ui"
- "test.*browser"
- "chrome.*extension.*test"
---
# Web UI Testing with Claude for Chrome
Use this skill when manually testing the IronClaw web gateway UI via the Claude for Chrome browser extension.
## Prerequisites
- IronClaw must be running with `GATEWAY_ENABLED=true`
- Note the gateway URL (default: `http://127.0.0.1:3000/`) and auth token
- The Claude for Chrome extension must be installed and connected
## Starting the Server
```bash
CLI_ENABLED=false GATEWAY_AUTH_TOKEN=<your-token> cargo run
```
Wait for "Agent ironclaw ready and listening" in the logs before proceeding.
## Test Checklist
### 1. Connection
- Navigate to `http://127.0.0.1:3000/?token=<token>`
- Verify "Connected" indicator in the top-right corner
- Verify all tabs are visible: Chat, Memory, Jobs, Routines, Extensions, Skills
### 2. Chat Tab
- Send a simple message (e.g., "Hello, what tools do you have?")
- Verify the LLM responds without errors
- If you see "Invalid schema for function" errors, the tool schema fix (PR #301) may not be merged yet
### 3. Skills Tab
- Click the Skills tab
- Verify "No skills installed" or a list of installed skills (no "Skills system not enabled" error)
- Search for "markdown" in the ClawHub search box
- Verify results appear with: name, version, description, relevance score, "updated X ago"
- Verify skill names are clickable links to clawhub.ai
- If search returns empty with a yellow warning banner, the registry may be unreachable
### 4. Skill Install (from search)
- Search for a skill (e.g., "markdown")
- Click "Install" on a result
- Confirm the install dialog
- Verify success toast appears
- Verify the skill appears in "Installed Skills" section
### 5. Skill Install (by URL)
- Scroll to "Install Skill by URL"
- Enter a skill name and a ClawHub download URL:
- Name: `markdown-viewer`
- URL: `https://wry-manatee-359.convex.site/api/v1/download?slug=markdown-viewer`
- Click Install
- Verify success toast and skill appears in installed list
### 6. Skill Remove
- Find an installed skill
- Click "Remove"
- Confirm removal
- Verify the skill disappears from the installed list
### 7. Other Tabs (smoke test)
- **Memory**: Should show the memory filesystem (may be empty)
- **Jobs**: Should show job list (may be empty)
- **Routines**: Should show routine list
- **Extensions**: Should show extension list with install options
## Cleanup
After testing, remove any test-installed skills:
```bash
rm -rf ~/.ironclaw/installed_skills/<skill-name>
```
Stop the server with Ctrl+C or by killing the process.
## Known Issues
- ClawHub registry at `clawhub.ai` is behind Vercel which blocks non-browser TLS fingerprints; the backend uses `wry-manatee-359.convex.site` directly
- Skill downloads are ZIP archives containing SKILL.md, not raw text
- The `confirm()` dialog for install may block browser automation; override with `window.confirm = () => true` in the console first
+37 -9
View File
@@ -98,7 +98,7 @@ impl Agent {
pub fn new(
config: AgentConfig,
deps: AgentDeps,
channels: ChannelManager,
channels: Arc<ChannelManager>,
heartbeat_config: Option<HeartbeatConfig>,
hygiene_config: Option<crate::config::HygieneConfig>,
routine_config: Option<RoutineConfig>,
@@ -123,7 +123,7 @@ impl Agent {
Self {
config,
deps,
channels: Arc::new(channels),
channels,
context_manager,
scheduler,
router: Router::new(),
@@ -397,6 +397,7 @@ impl Agent {
self.llm().clone(),
Arc::clone(workspace),
notify_tx,
Some(self.scheduler.clone()),
));
// Register routine tools
@@ -499,21 +500,41 @@ impl Agent {
Ok(crate::hooks::HookOutcome::Continue {
modified: Some(new_content),
}) => {
let _ = self
if let Err(e) = self
.channels
.respond(&message, OutgoingResponse::text(new_content))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %e,
"Failed to send response to channel"
);
}
}
_ => {
let _ = self
if let Err(e) = self
.channels
.respond(&message, OutgoingResponse::text(response))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %e,
"Failed to send response to channel"
);
}
}
}
}
Ok(Some(_)) => {
Ok(Some(empty)) => {
// Empty response, nothing to send (e.g. approval handled via send_status)
tracing::debug!(
channel = %message.channel,
user = %message.user_id,
empty_len = empty.len(),
"Suppressed empty response (not sent to channel)"
);
}
Ok(None) => {
// Shutdown signal received (/quit, /exit, /shutdown)
@@ -522,10 +543,17 @@ impl Agent {
}
Err(e) => {
tracing::error!("Error handling message: {}", e);
let _ = self
if let Err(send_err) = self
.channels
.respond(&message, OutgoingResponse::text(format!("Error: {}", e)))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %send_err,
"Failed to send error response to channel"
);
}
}
}
+8 -21
View File
@@ -73,36 +73,23 @@ impl Agent {
description: String,
category: Option<String>,
) -> Result<String, Error> {
// Create job context
let job_id = self
.context_manager
.create_job_for_user(user_id, &title, &description)
.scheduler
.dispatch_job(user_id, &title, &description, None)
.await?;
// Update category if provided
if let Some(cat) = category {
self.context_manager
// Set the dedicated category field (not stored in metadata)
if let Some(cat) = category
&& let Err(e) = self
.context_manager
.update_context(job_id, |ctx| {
ctx.category = Some(cat);
})
.await?;
}
// Persist new job to database (fire-and-forget)
if let Some(store) = self.store()
&& let Ok(ctx) = self.context_manager.get_context(job_id).await
.await
{
let store = store.clone();
tokio::spawn(async move {
if let Err(e) = store.save_job(&ctx).await {
tracing::warn!("Failed to persist new job {}: {}", job_id, e);
}
});
tracing::warn!(job_id = %job_id, "Failed to set job category: {}", e);
}
// Schedule for execution
self.scheduler.schedule(job_id).await?;
Ok(format!(
"Created job: {}\nID: {}\n\nThe job has been scheduled and is now running.",
title, job_id
+18 -11
View File
@@ -151,14 +151,19 @@ impl CostGuard {
/// Record a completed LLM action: its token costs and the action timestamp.
///
/// Call this AFTER an LLM call completes so that costs are tracked.
///
/// When `cost_per_token` is `Some`, those rates are used directly (provider-
/// sourced pricing). When `None`, falls back to the static `costs::model_cost`
/// lookup table, then `costs::default_cost`.
pub async fn record_llm_call(
&self,
model: &str,
input_tokens: u32,
output_tokens: u32,
cost_per_token: Option<(Decimal, Decimal)>,
) -> Decimal {
let (input_rate, output_rate) =
costs::model_cost(model).unwrap_or_else(costs::default_cost);
let (input_rate, output_rate) = cost_per_token
.unwrap_or_else(|| costs::model_cost(model).unwrap_or_else(costs::default_cost));
let cost =
input_rate * Decimal::from(input_tokens) + output_rate * Decimal::from(output_tokens);
@@ -261,7 +266,9 @@ mod tests {
assert!(guard.check_allowed().await.is_ok());
// Record a big call, still allowed
guard.record_llm_call("gpt-4o", 100_000, 100_000).await;
guard
.record_llm_call("gpt-4o", 100_000, 100_000, None)
.await;
assert!(guard.check_allowed().await.is_ok());
}
@@ -278,7 +285,7 @@ mod tests {
// Record a call that costs more than $0.01
// gpt-4o: input=$0.0000025/tok, output=$0.00001/tok
// 10000 input + 10000 output = $0.025 + $0.10 = $0.125
guard.record_llm_call("gpt-4o", 10_000, 10_000).await;
guard.record_llm_call("gpt-4o", 10_000, 10_000, None).await;
// Now should be blocked
let result = guard.check_allowed().await;
@@ -301,7 +308,7 @@ mod tests {
// First 3 actions allowed
for _ in 0..3 {
assert!(guard.check_allowed().await.is_ok());
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
}
// 4th should be blocked
@@ -322,7 +329,7 @@ mod tests {
assert_eq!(guard.daily_spend().await, Decimal::ZERO);
let cost = guard.record_llm_call("gpt-4o", 1000, 500).await;
let cost = guard.record_llm_call("gpt-4o", 1000, 500, None).await;
assert!(cost > Decimal::ZERO);
assert_eq!(guard.daily_spend().await, cost);
}
@@ -333,8 +340,8 @@ mod tests {
assert_eq!(guard.actions_this_hour().await, 0);
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
assert_eq!(guard.actions_this_hour().await, 2);
}
@@ -371,10 +378,10 @@ mod tests {
assert!(guard.model_usage().await.is_empty());
// Record calls for two different models
guard.record_llm_call("gpt-4o", 1000, 500).await;
guard.record_llm_call("gpt-4o", 2000, 1000).await;
guard.record_llm_call("gpt-4o", 1000, 500, None).await;
guard.record_llm_call("gpt-4o", 2000, 1000, None).await;
guard
.record_llm_call("claude-3-5-sonnet-20241022", 500, 200)
.record_llm_call("claude-3-5-sonnet-20241022", 500, 200, None)
.await;
let usage = guard.model_usage().await;
+2 -1
View File
@@ -222,6 +222,7 @@ impl Agent {
&model_name,
output.usage.input_tokens,
output.usage.output_tokens,
Some(self.llm().cost_per_token()),
)
.await;
tracing::debug!(
@@ -890,7 +891,7 @@ mod tests {
auto_approve_tools: false,
},
deps,
ChannelManager::new(),
Arc::new(ChannelManager::new()),
None,
None,
None,
+64 -26
View File
@@ -19,6 +19,7 @@ use regex::Regex;
use tokio::sync::{RwLock, mpsc};
use uuid::Uuid;
use crate::agent::Scheduler;
use crate::agent::routine::{
NotifyConfig, Routine, RoutineAction, RoutineRun, RunStatus, Trigger, next_cron_fire,
};
@@ -41,6 +42,8 @@ pub struct RoutineEngine {
running_count: Arc<AtomicUsize>,
/// Compiled event regex cache: routine_id -> compiled regex.
event_cache: Arc<RwLock<Vec<(Uuid, Routine, Regex)>>>,
/// Scheduler for dispatching jobs (FullJob mode).
scheduler: Option<Arc<Scheduler>>,
}
impl RoutineEngine {
@@ -50,6 +53,7 @@ impl RoutineEngine {
llm: Arc<dyn LlmProvider>,
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
scheduler: Option<Arc<Scheduler>>,
) -> Self {
Self {
config,
@@ -59,6 +63,7 @@ impl RoutineEngine {
notify_tx,
running_count: Arc::new(AtomicUsize::new(0)),
event_cache: Arc::new(RwLock::new(Vec::new())),
scheduler,
}
}
@@ -225,7 +230,7 @@ impl RoutineEngine {
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
max_lightweight_tokens: self.config.max_lightweight_tokens,
scheduler: self.scheduler.clone(),
};
tokio::spawn(async move {
@@ -257,7 +262,7 @@ impl RoutineEngine {
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
max_lightweight_tokens: self.config.max_lightweight_tokens,
scheduler: self.scheduler.clone(),
};
// Record the run in DB, then spawn execution
@@ -304,7 +309,7 @@ struct EngineContext {
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
running_count: Arc<AtomicUsize>,
max_lightweight_tokens: u32,
scheduler: Option<Arc<Scheduler>>,
}
/// Execute a routine run. Handles both lightweight and full_job modes.
@@ -318,29 +323,11 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
context_paths,
max_tokens,
} => execute_lightweight(&ctx, &routine, prompt, context_paths, *max_tokens).await,
RoutineAction::FullJob { description, .. } => {
// Full job mode: scheduler integration not yet implemented.
// Execute as lightweight and prepend a warning to the summary.
tracing::warn!(
routine = %routine.name,
"FullJob mode not yet implemented; falling back to lightweight execution"
);
match execute_lightweight(&ctx, &routine, description, &[], ctx.max_lightweight_tokens)
.await
{
Ok((status, summary, tokens)) => {
let warning = "[Note: FullJob mode is not yet implemented. This routine ran as \
a single LLM call without tool access. Configure as 'lightweight' \
or wait for full scheduler integration.]";
let summary = match summary {
Some(s) => Some(format!("{warning}\n\n{s}")),
None => Some(warning.to_string()),
};
Ok((status, summary, tokens))
}
Err(e) => Err(e),
}
}
RoutineAction::FullJob {
title,
description,
max_iterations,
} => execute_full_job(&ctx, &routine, &run, title, description, *max_iterations).await,
};
// Decrement running count
@@ -418,6 +405,57 @@ fn sanitize_routine_name(name: &str) -> String {
.collect()
}
/// Execute a full-job routine by dispatching to the scheduler.
///
/// Fire-and-forget: creates a job via `Scheduler::dispatch_job` (which handles
/// creation, metadata, persistence, and scheduling), links the routine run to
/// the job, and returns immediately. The job runs independently via the
/// existing Worker/Scheduler with full tool access.
async fn execute_full_job(
ctx: &EngineContext,
routine: &Routine,
run: &RoutineRun,
title: &str,
description: &str,
max_iterations: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
let scheduler = ctx
.scheduler
.as_ref()
.ok_or_else(|| RoutineError::JobDispatchFailed {
reason: "scheduler not available".to_string(),
})?;
let metadata = serde_json::json!({ "max_iterations": max_iterations });
let job_id = scheduler
.dispatch_job(&routine.user_id, title, description, Some(metadata))
.await
.map_err(|e| RoutineError::JobDispatchFailed {
reason: format!("failed to dispatch job: {e}"),
})?;
// Link the routine run to the dispatched job
if let Err(e) = ctx.store.link_routine_run_to_job(run.id, job_id).await {
tracing::error!(
routine = %routine.name,
"Failed to link run to job: {}", e
);
}
tracing::info!(
routine = %routine.name,
job_id = %job_id,
max_iterations = max_iterations,
"Dispatched full job for routine"
);
let summary = format!(
"Dispatched job {job_id} for full execution with tool access (max_iterations: {max_iterations})"
);
Ok((RunStatus::Ok, Some(summary), None))
}
/// Execute a lightweight routine (single LLM call).
async fn execute_lightweight(
ctx: &EngineContext,
+44
View File
@@ -81,6 +81,50 @@ impl Scheduler {
}
}
/// Create, persist, and schedule a job in one shot.
///
/// This is the preferred entry point for dispatching new jobs. It:
/// 1. Creates the job context via `ContextManager`
/// 2. Optionally applies metadata (e.g. `max_iterations`)
/// 3. Persists the job to the database (so FK references from
/// `job_actions` / `llm_calls` work immediately)
/// 4. Schedules the job for worker execution
///
/// Returns the new job ID.
pub async fn dispatch_job(
&self,
user_id: &str,
title: &str,
description: &str,
metadata: Option<serde_json::Value>,
) -> Result<Uuid, JobError> {
let job_id = self
.context_manager
.create_job_for_user(user_id, title, description)
.await?;
// Apply metadata if provided
if let Some(meta) = metadata {
self.context_manager
.update_context(job_id, |ctx| {
ctx.metadata = meta;
})
.await?;
}
// Persist to DB before scheduling so the worker's FK references are valid
if let Some(ref store) = self.store {
let ctx = self.context_manager.get_context(job_id).await?;
store.save_job(&ctx).await.map_err(|e| JobError::Failed {
id: job_id,
reason: format!("failed to persist job: {e}"),
})?;
}
self.schedule(job_id).await?;
Ok(job_id)
}
/// Schedule a job for execution.
pub async fn schedule(&self, job_id: Uuid) -> Result<(), JobError> {
// Hold write lock for the entire check-insert sequence to prevent
+49 -35
View File
@@ -252,6 +252,10 @@ impl Agent {
thread.messages()
};
// Persist user message to DB immediately so it survives crashes
self.persist_user_message(thread_id, &message.user_id, content)
.await;
// Send thinking status
let _ = self
.channels
@@ -320,9 +324,8 @@ impl Agent {
)
.await;
// Persist turn to DB before returning so the write
// completes even if the process shuts down right after.
self.persist_turn(thread_id, &message.user_id, content, Some(&response))
// Persist assistant response (user message already persisted at turn start)
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
Ok(SubmissionResult::response(response))
@@ -351,23 +354,21 @@ impl Agent {
}
Err(e) => {
thread.fail_turn(e.to_string());
// Persist the user message even on failure
self.persist_turn(thread_id, &message.user_id, content, None)
.await;
// User message already persisted at turn start; nothing else to save
Ok(SubmissionResult::error(e.to_string()))
}
}
}
/// Persist a turn (user message + optional assistant response) to the DB.
pub(super) async fn persist_turn(
/// Persist the user message to the DB at turn start (before the agentic loop).
///
/// This ensures the user message is durable even if the process crashes
/// mid-response. Call this right after `thread.start_turn()`.
pub(super) async fn persist_user_message(
&self,
thread_id: Uuid,
user_id: &str,
user_input: &str,
response: Option<&str>,
) {
let store = match self.store() {
Some(s) => Arc::clone(s),
@@ -387,13 +388,36 @@ impl Agent {
.await
{
tracing::warn!("Failed to persist user message: {}", e);
}
}
/// Persist the assistant response to the DB after the agentic loop completes.
///
/// Re-ensures the conversation row exists so that assistant responses are
/// still persisted even if `persist_user_message` failed transiently at
/// turn start (e.g. a brief DB blip that resolved before response time).
pub(super) async fn persist_assistant_response(
&self,
thread_id: Uuid,
user_id: &str,
response: &str,
) {
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Some(resp) = response
&& let Err(e) = store
.add_conversation_message(thread_id, "assistant", resp)
.await
if let Err(e) = store
.add_conversation_message(thread_id, "assistant", response)
.await
{
tracing::warn!("Failed to persist assistant message: {}", e);
}
@@ -1015,12 +1039,10 @@ impl Agent {
match result {
Ok(AgenticLoopResult::Response(response)) => {
let user_input = thread.last_turn().map(|t| t.user_input.clone());
thread.complete_turn(&response);
if let Some(input) = user_input {
self.persist_turn(thread_id, &message.user_id, &input, Some(&response))
.await;
}
// User message already persisted at turn start; save assistant response
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
let _ = self
.channels
.send_status(
@@ -1055,12 +1077,8 @@ impl Agent {
})
}
Err(e) => {
let user_input = thread.last_turn().map(|t| t.user_input.clone());
thread.fail_turn(e.to_string());
if let Some(input) = user_input {
self.persist_turn(thread_id, &message.user_id, &input, None)
.await;
}
// User message already persisted at turn start
Ok(SubmissionResult::error(e.to_string()))
}
}
@@ -1074,13 +1092,11 @@ impl Agent {
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
let user_input = thread.last_turn().map(|t| t.user_input.clone());
thread.clear_pending_approval();
thread.complete_turn(&rejection);
if let Some(input) = user_input {
self.persist_turn(thread_id, &message.user_id, &input, Some(&rejection))
.await;
}
// User message already persisted at turn start; save rejection response
self.persist_assistant_response(thread_id, &message.user_id, &rejection)
.await;
}
}
@@ -1115,13 +1131,11 @@ impl Agent {
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
let user_input = thread.last_turn().map(|t| t.user_input.clone());
thread.enter_auth_mode(ext_name.clone());
thread.complete_turn(&instructions);
if let Some(input) = user_input {
self.persist_turn(thread_id, &message.user_id, &input, Some(&instructions))
.await;
}
// User message already persisted at turn start; save auth instructions
self.persist_assistant_response(thread_id, &message.user_id, &instructions)
.await;
}
}
let _ = self
+94 -1
View File
@@ -98,6 +98,20 @@ impl Worker {
}
}
/// Fire-and-forget persistence of a job event.
fn log_event(&self, event_type: &str, data: serde_json::Value) {
if let Some(store) = self.store() {
let store = store.clone();
let job_id = self.job_id;
let event_type = event_type.to_string();
tokio::spawn(async move {
if let Err(e) = store.save_job_event(job_id, &event_type, &data).await {
tracing::warn!("Failed to persist event for job {}: {}", job_id, e);
}
});
}
}
/// Run the worker until the job is complete or stopped.
pub async fn run(self, mut rx: mpsc::Receiver<WorkerMessage>) -> Result<(), Error> {
tracing::info!("Worker starting for job {}", self.job_id);
@@ -164,7 +178,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reasoning: &Reasoning,
reason_ctx: &mut ReasoningContext,
) -> Result<(), Error> {
let max_iterations = 50;
const MAX_WORKER_ITERATIONS: usize = 500;
let max_iterations = self
.context_manager()
.get_context(self.job_id)
.await
.ok()
.and_then(|ctx| ctx.metadata.get("max_iterations").and_then(|v| v.as_u64()))
.unwrap_or(50) as usize;
let max_iterations = max_iterations.min(MAX_WORKER_ITERATIONS);
let mut iteration = 0;
// Initial tool definitions for planning (will be refreshed in loop)
@@ -193,6 +215,14 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.join("\n")
)));
self.log_event("message", serde_json::json!({
"role": "assistant",
"content": format!("Plan: {}\n\n{}", p.goal,
p.actions.iter().enumerate()
.map(|(i, a)| format!("{}. {} - {}", i + 1, a.tool_name, a.reasoning))
.collect::<Vec<_>>().join("\n"))
}));
Some(p)
}
Err(e) => {
@@ -267,6 +297,14 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Add assistant response to context
reason_ctx.messages.push(ChatMessage::assistant(&response));
self.log_event(
"message",
serde_json::json!({
"role": "assistant",
"content": response,
}),
);
// Give it one more chance to select a tool
if iteration > 3 && iteration % 5 == 0 {
reason_ctx.messages.push(ChatMessage::user(
@@ -285,6 +323,16 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tool_calls.len()
);
if let Some(ref text) = content {
self.log_event(
"message",
serde_json::json!({
"role": "assistant",
"content": text,
}),
);
}
// Add assistant message with tool_calls (OpenAI protocol)
reason_ctx
.messages
@@ -667,6 +715,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
selection: &ToolSelection,
result: Result<String, Error>,
) -> Result<bool, Error> {
self.log_event(
"tool_use",
serde_json::json!({
"tool_name": selection.tool_name,
"input": crate::agent::agent_loop::truncate_for_preview(
&selection.parameters.to_string(), 500),
}),
);
match result {
Ok(output) => {
// Sanitize output
@@ -687,6 +744,12 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
wrapped,
));
self.log_event("tool_result", serde_json::json!({
"tool_name": selection.tool_name,
"success": true,
"output": crate::agent::agent_loop::truncate_for_preview(&sanitized.content, 500),
}));
// Tool output never drives job completion. A malicious tool could
// emit "TASK_COMPLETE" to force premature completion. Only the LLM's
// own structured response (in execution_loop) can mark a job done.
@@ -713,6 +776,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
});
}
self.log_event(
"tool_result",
serde_json::json!({
"tool_name": selection.tool_name,
"success": false,
"output": format!("Error: {}", e),
}),
);
reason_ctx.messages.push(ChatMessage::tool_result(
&selection.tool_call_id,
&selection.tool_name,
@@ -834,6 +906,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": true,
"message": "Job completed successfully",
}),
);
self.persist_status(
JobState::Completed,
Some("Job completed successfully".to_string()),
@@ -852,6 +931,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": false,
"message": format!("Execution failed: {}", reason),
}),
);
self.persist_status(JobState::Failed, Some(reason.to_string()));
Ok(())
}
@@ -865,6 +951,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": false,
"message": format!("Job stuck: {}", reason),
}),
);
self.persist_status(JobState::Stuck, Some(reason.to_string()));
Ok(())
}
+59 -49
View File
@@ -22,6 +22,7 @@ use crate::skills::SkillRegistry;
use crate::skills::catalog::SkillCatalog;
use crate::tools::ToolRegistry;
use crate::tools::mcp::McpSessionManager;
use crate::tools::wasm::SharedCredentialRegistry;
use crate::tools::wasm::WasmToolRuntime;
use crate::workspace::{EmbeddingProvider, Workspace};
@@ -48,6 +49,8 @@ pub struct AppComponents {
pub skill_catalog: Option<Arc<SkillCatalog>>,
pub cost_guard: Arc<crate::agent::cost_guard::CostGuard>,
pub session: Arc<SessionManager>,
pub catalog_entries: Vec<crate::extensions::RegistryEntry>,
pub dev_loaded_tool_names: Vec<String>,
}
/// Options that control optional init phases.
@@ -313,54 +316,41 @@ impl AppBuilder {
),
anyhow::Error,
> {
use crate::workspace::{NearAiEmbeddings, OpenAiEmbeddings};
let safety = Arc::new(SafetyLayer::new(&self.config.safety));
tracing::info!("Safety layer initialized");
let tools = Arc::new(ToolRegistry::new());
// Initialize tool registry with credential injection support
let credential_registry = Arc::new(SharedCredentialRegistry::new());
let tools = if let Some(ref ss) = self.secrets_store {
Arc::new(
ToolRegistry::new()
.with_credentials(Arc::clone(&credential_registry), Arc::clone(ss)),
)
} else {
Arc::new(ToolRegistry::new())
};
tools.register_builtin_tools();
// Create embeddings provider if configured
let embeddings: Option<Arc<dyn EmbeddingProvider>> = if self.config.embeddings.enabled {
match self.config.embeddings.provider.as_str() {
"nearai" => {
tracing::info!(
"Embeddings enabled via NEAR AI (model: {})",
self.config.embeddings.model
);
Some(Arc::new(
NearAiEmbeddings::new(
&self.config.llm.nearai.base_url,
self.session.clone(),
)
.with_model(&self.config.embeddings.model, 1536),
))
}
_ => {
if let Some(api_key) = self.config.embeddings.openai_api_key() {
tracing::info!(
"Embeddings enabled via OpenAI (model: {})",
self.config.embeddings.model
);
Some(Arc::new(OpenAiEmbeddings::with_model(
api_key,
&self.config.embeddings.model,
match self.config.embeddings.model.as_str() {
"text-embedding-3-large" => 3072,
_ => 1536,
},
)))
} else {
tracing::warn!("Embeddings configured but OPENAI_API_KEY not set");
None
}
}
}
} else {
tracing::info!("Embeddings disabled (set OPENAI_API_KEY or EMBEDDING_ENABLED=true)");
None
};
// Create embeddings provider using the unified method
let embeddings = self
.config
.embeddings
.create_provider(&self.config.llm.nearai.base_url, self.session.clone());
// Warn if libSQL backend is used with non-1536 embedding dimension.
if self.config.database.backend == crate::config::DatabaseBackend::LibSql
&& self.config.embeddings.enabled
&& self.config.embeddings.dimension != 1536
{
tracing::warn!(
configured_dimension = self.config.embeddings.dimension,
"Embedding dimension {} is not 1536. The libSQL schema uses \
F32_BLOB(1536) which requires exactly 1536 dimensions. \
Embedding storage will fail. Use PostgreSQL or set \
EMBEDDING_DIMENSION=1536.",
self.config.embeddings.dimension
);
}
// Register memory tools if database is available
let workspace = if let Some(ref db) = self.db {
@@ -402,6 +392,8 @@ impl AppBuilder {
Arc<McpSessionManager>,
Option<Arc<WasmToolRuntime>>,
Option<Arc<ExtensionManager>>,
Vec<crate::extensions::RegistryEntry>,
Vec<String>,
),
anyhow::Error,
> {
@@ -431,6 +423,8 @@ impl AppBuilder {
let tools = Arc::clone(tools);
let wasm_config = self.config.wasm.clone();
async move {
let mut dev_loaded_tool_names: Vec<String> = Vec::new();
if let Some(ref runtime) = wasm_tool_runtime {
let mut loader = WasmToolLoader::new(Arc::clone(runtime), Arc::clone(&tools));
if let Some(ref secrets) = secrets_store {
@@ -461,10 +455,11 @@ impl AppBuilder {
match load_dev_tools(&loader, &wasm_config.tools_dir).await {
Ok(results) => {
if !results.loaded.is_empty() {
dev_loaded_tool_names.extend(results.loaded.iter().cloned());
if !dev_loaded_tool_names.is_empty() {
tracing::info!(
"Loaded {} dev WASM tools from build artifacts",
results.loaded.len()
dev_loaded_tool_names.len()
);
}
}
@@ -473,6 +468,8 @@ impl AppBuilder {
}
}
}
dev_loaded_tool_names
}
};
@@ -577,7 +574,7 @@ impl AppBuilder {
}
};
tokio::join!(wasm_tools_future, mcp_servers_future);
let (dev_loaded_tool_names, _) = tokio::join!(wasm_tools_future, mcp_servers_future);
// Load registry catalog entries for extension discovery
let catalog_entries = match crate::registry::RegistryCatalog::load_or_embedded() {
@@ -640,7 +637,13 @@ impl AppBuilder {
tools.register_dev_tools();
}
Ok((mcp_session_manager, wasm_tool_runtime, extension_manager))
Ok((
mcp_session_manager,
wasm_tool_runtime,
extension_manager,
catalog_entries,
dev_loaded_tool_names,
))
}
/// Run all init phases in order and return the assembled components.
@@ -654,8 +657,13 @@ impl AppBuilder {
// Create hook registry early so runtime extension activation can register hooks.
let hooks = Arc::new(HookRegistry::new());
let (mcp_session_manager, wasm_tool_runtime, extension_manager) =
self.init_extensions(&tools, &hooks).await?;
let (
mcp_session_manager,
wasm_tool_runtime,
extension_manager,
catalog_entries,
dev_loaded_tool_names,
) = self.init_extensions(&tools, &hooks).await?;
// Seed workspace and backfill embeddings
if let Some(ref ws) = workspace {
@@ -730,6 +738,8 @@ impl AppBuilder {
skill_catalog,
cost_guard,
session: self.session,
catalog_entries,
dev_loaded_tool_names,
})
}
}
-35
View File
@@ -9,32 +9,6 @@ use uuid::Uuid;
use crate::error::ChannelError;
/// Kind of attachment carried on an incoming message.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AttachmentKind {
/// Audio content (voice notes, audio files).
Audio,
/// Image content (photos, screenshots).
Image,
/// Document content (PDFs, files).
Document,
}
/// Binary attachment on a message (e.g., voice note, photo).
#[derive(Debug, Clone)]
pub struct Attachment {
/// What kind of content this is.
pub kind: AttachmentKind,
/// MIME type (e.g., "audio/ogg", "image/jpeg").
pub mime_type: String,
/// Raw bytes of the attachment.
pub data: Vec<u8>,
/// Optional filename.
pub filename: Option<String>,
/// Duration in seconds (for audio/video).
pub duration_secs: Option<u32>,
}
/// A message received from an external channel.
#[derive(Debug, Clone)]
pub struct IncomingMessage {
@@ -54,8 +28,6 @@ pub struct IncomingMessage {
pub received_at: DateTime<Utc>,
/// Channel-specific metadata.
pub metadata: serde_json::Value,
/// Binary attachments (voice notes, images, etc.).
pub attachments: Vec<Attachment>,
}
impl IncomingMessage {
@@ -74,7 +46,6 @@ impl IncomingMessage {
thread_id: None,
received_at: Utc::now(),
metadata: serde_json::Value::Null,
attachments: Vec::new(),
}
}
@@ -95,12 +66,6 @@ impl IncomingMessage {
self.user_name = Some(name.into());
self
}
/// Set attachments.
pub fn with_attachments(mut self, attachments: Vec<Attachment>) -> Self {
self.attachments = attachments;
self
}
}
/// Stream of incoming messages.
+32 -9
View File
@@ -40,16 +40,39 @@ impl ChannelManager {
}
/// Add a channel to the manager.
pub fn add(&mut self, channel: Box<dyn Channel>) {
pub async fn add(&self, channel: Box<dyn Channel>) {
let name = channel.name().to_string();
// We need to get the inner HashMap to insert
// Since we're in a sync context during setup, we'll use try_write
if let Ok(mut channels) = self.channels.try_write() {
channels.insert(name.clone(), channel);
tracing::debug!("Added channel: {}", name);
} else {
tracing::error!("Failed to add channel: {} (lock contention)", name);
}
self.channels.write().await.insert(name.clone(), channel);
tracing::debug!("Added channel: {}", name);
}
/// Hot-add a channel to a running agent.
///
/// Starts the channel, registers it in the channels map for `respond()`/`broadcast()`,
/// and spawns a task that forwards its stream messages through `inject_tx` into
/// the agent loop.
pub async fn hot_add(&self, channel: Box<dyn Channel>) -> Result<(), ChannelError> {
let name = channel.name().to_string();
let stream = channel.start().await?;
// Register for respond/broadcast/send_status
self.channels.write().await.insert(name.clone(), channel);
// Forward stream messages through inject_tx
let tx = self.inject_tx.clone();
tokio::spawn(async move {
use futures::StreamExt;
let mut stream = stream;
while let Some(msg) = stream.next().await {
if tx.send(msg).await.is_err() {
tracing::warn!(channel = %name, "Inject channel closed, stopping hot-added channel");
break;
}
}
tracing::info!(channel = %name, "Hot-added channel stream ended");
});
Ok(())
}
/// Start all channels and return a merged stream of messages.
+1 -4
View File
@@ -35,10 +35,7 @@ pub mod wasm;
pub mod web;
mod webhook_server;
pub use channel::{
Attachment, AttachmentKind, Channel, IncomingMessage, MessageStream, OutgoingResponse,
StatusUpdate,
};
pub use channel::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
pub use http::HttpChannel;
pub use manager::ChannelManager;
pub use repl::ReplChannel;
+13 -9
View File
@@ -65,24 +65,28 @@ fn locate_channel_artifacts(name: &str) -> Result<(PathBuf, PathBuf), String> {
return Ok((flat_wasm, caps_path));
}
// Fall back to build tree layout (dev builds)
let build_wasm = channel_dir
.join("target/wasm32-wasip2/release")
.join(format!("{}.wasm", crate_name));
if build_wasm.exists() && caps_path.exists() {
// Fall back to build tree layout (dev builds) — search across all WASM triples
if let Some(build_wasm) =
crate::registry::artifacts::find_wasm_artifact(&channel_dir, crate_name, "release")
&& caps_path.exists()
{
return Ok((build_wasm, caps_path));
}
// Provide a helpful error with the paths we checked
let expected_build = crate::registry::artifacts::resolve_target_dir(&channel_dir)
.join("wasm32-wasip2/release")
.join(format!("{}.wasm", crate_name));
Err(format!(
"Channel '{}' WASM not found. Checked:\n \
- {} (flat/packaged)\n \
- {} (build tree)\n \
- {} (build tree, and other triples)\n \
Build it first:\n \
cd {} && cargo build --target wasm32-wasip2 --release",
cd {} && cargo component build --release",
name,
flat_wasm.display(),
build_wasm.display(),
expected_build.display(),
channel_dir.display()
))
}
+1 -31
View File
@@ -7,7 +7,6 @@
use std::time::{SystemTime, UNIX_EPOCH};
use crate::channels::channel::Attachment;
use crate::channels::wasm::capabilities::{ChannelCapabilities, EmitRateLimitConfig};
use crate::channels::wasm::error::WasmChannelError;
use crate::tools::wasm::{HostState, LogLevel};
@@ -18,9 +17,6 @@ const MAX_EMITS_PER_EXECUTION: usize = 100;
/// Maximum message content size (64 KB).
const MAX_MESSAGE_CONTENT_SIZE: usize = 64 * 1024;
/// Maximum size for a single attachment (10 MB).
const MAX_ATTACHMENT_SIZE: usize = 10 * 1024 * 1024;
/// A message emitted by a WASM channel to be sent to the agent.
#[derive(Debug, Clone)]
pub struct EmittedMessage {
@@ -41,9 +37,6 @@ pub struct EmittedMessage {
/// Timestamp when the message was emitted.
pub emitted_at_millis: u64,
/// Binary attachments (voice notes, images, etc.).
pub attachments: Vec<Attachment>,
}
impl EmittedMessage {
@@ -59,7 +52,6 @@ impl EmittedMessage {
.duration_since(UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0),
attachments: Vec::new(),
}
}
@@ -80,12 +72,6 @@ impl EmittedMessage {
self.metadata_json = metadata_json.into();
self
}
/// Set attachments.
pub fn with_attachments(mut self, attachments: Vec<Attachment>) -> Self {
self.attachments = attachments;
self
}
}
/// A pending workspace write operation.
@@ -182,7 +168,7 @@ impl ChannelHostState {
///
/// Messages are queued and delivered after callback execution completes.
/// Rate limiting is enforced per-execution and globally.
pub fn emit_message(&mut self, mut msg: EmittedMessage) -> Result<(), WasmChannelError> {
pub fn emit_message(&mut self, msg: EmittedMessage) -> Result<(), WasmChannelError> {
// Check per-execution limit
if !self.emit_enabled {
self.emits_dropped += 1;
@@ -200,22 +186,6 @@ impl ChannelHostState {
return Ok(());
}
// Validate attachment sizes — drop only oversized attachments, not the whole message
msg.attachments.retain(|attachment| {
if attachment.data.len() > MAX_ATTACHMENT_SIZE {
tracing::warn!(
channel = %self.channel_name,
size = attachment.data.len(),
max = MAX_ATTACHMENT_SIZE,
mime = %attachment.mime_type,
"Attachment too large, dropping attachment (message still delivered)"
);
false
} else {
true
}
});
// Validate message content size
if msg.content.len() > MAX_MESSAGE_CONTENT_SIZE {
tracing::warn!(
+15
View File
@@ -110,6 +110,21 @@ impl WasmChannelRouter {
.unwrap_or_else(|| "X-Webhook-Secret".to_string())
}
/// Update the webhook secret for an already-registered channel.
///
/// This is used when credentials are saved after a channel was registered
/// without a secret (e.g., loaded at startup before the user configured it).
pub async fn update_secret(&self, channel_name: &str, secret: String) {
self.secrets
.write()
.await
.insert(channel_name.to_string(), secret);
tracing::info!(
channel = %channel_name,
"Updated webhook secret for channel"
);
}
/// Unregister a channel and its endpoints.
pub async fn unregister(&self, channel_name: &str) {
self.channels.write().await.remove(channel_name);
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -20,7 +20,7 @@ pub async fn extensions_list_handler(
))?;
let installed = ext_mgr
.list(None)
.list(None, false)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
+15 -4
View File
@@ -290,7 +290,8 @@ pub async fn start_server(
let statics = Router::new()
.route("/", get(index_handler))
.route("/style.css", get(css_handler))
.route("/app.js", get(js_handler));
.route("/app.js", get(js_handler))
.route("/favicon.ico", get(favicon_handler));
// Project file serving (behind auth to prevent unauthorized file access).
let projects = Router::new()
@@ -392,6 +393,16 @@ async fn js_handler() -> impl IntoResponse {
)
}
async fn favicon_handler() -> impl IntoResponse {
(
[
(header::CONTENT_TYPE, "image/x-icon"),
(header::CACHE_CONTROL, "public, max-age=86400"),
],
include_bytes!("static/favicon.ico").as_slice(),
)
}
// --- Health ---
async fn health_handler() -> Json<HealthResponse> {
@@ -1704,7 +1715,7 @@ async fn extensions_list_handler(
))?;
let installed = ext_mgr
.list(None)
.list(None, false)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
@@ -1955,7 +1966,7 @@ async fn extensions_registry_handler(
let installed: std::collections::HashSet<(String, String)> =
if let Some(ext_mgr) = state.extension_manager.as_ref() {
ext_mgr
.list(None)
.list(None, false)
.await
.unwrap_or_default()
.into_iter()
@@ -1998,7 +2009,7 @@ async fn extensions_setup_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let kind = ext_mgr
.list(None)
.list(None, false)
.await
.ok()
.and_then(|list| list.into_iter().find(|e| e.name == name))
+44 -20
View File
@@ -9,6 +9,7 @@ let assistantThreadId = null;
let hasMore = false;
let oldestTimestamp = null;
let loadingOlder = false;
let sseHasConnectedBefore = false;
let jobEvents = new Map(); // job_id -> Array of events
let jobListRefreshTimer = null;
const JOB_EVENTS_CAP = 500;
@@ -107,6 +108,10 @@ function connectSSE() {
eventSource.onopen = () => {
document.getElementById('sse-dot').classList.remove('disconnected');
document.getElementById('sse-status').textContent = 'Connected';
if (sseHasConnectedBefore && currentThreadId) {
loadHistory();
}
sseHasConnectedBefore = true;
};
eventSource.onerror = () => {
@@ -236,6 +241,11 @@ function isCurrentThread(threadId) {
function sendMessage() {
const input = document.getElementById('chat-input');
const sendBtn = document.getElementById('send-btn');
if (!currentThreadId) {
console.warn('sendMessage: no thread selected, ignoring');
setStatus('Waiting for thread to load...');
return;
}
const content = input.value.trim();
if (!content) return;
@@ -258,6 +268,8 @@ function sendMessage() {
}
function enableChatInput() {
// Don't re-enable until a thread is selected (prevents orphan messages)
if (!currentThreadId) return;
const input = document.getElementById('chat-input');
const sendBtn = document.getElementById('send-btn');
sendBtn.disabled = false;
@@ -735,6 +747,11 @@ function loadThreads() {
if (!currentThreadId && assistantThreadId) {
switchToAssistant();
}
// Enable chat input once a thread is available
if (currentThreadId) {
enableChatInput();
}
}).catch(() => {});
}
@@ -783,6 +800,10 @@ chatInput.addEventListener('keydown', (e) => {
});
chatInput.addEventListener('input', () => autoResizeTextarea(chatInput));
// Disable send until a thread is selected (loadThreads will enable it)
chatInput.disabled = true;
document.getElementById('send-btn').disabled = true;
// Infinite scroll: load older messages when scrolled near the top
document.getElementById('chat-messages').addEventListener('scroll', function () {
if (this.scrollTop < 100 && hasMore && !loadingOlder) {
@@ -1455,18 +1476,11 @@ function renderExtensionCard(ext) {
actions.className = 'ext-actions';
if (!ext.active) {
if (ext.kind === 'wasm_channel') {
const restartLabel = document.createElement('span');
restartLabel.className = 'ext-restart-label';
restartLabel.textContent = 'Restart to activate';
actions.appendChild(restartLabel);
} else {
const activateBtn = document.createElement('button');
activateBtn.className = 'btn-ext activate';
activateBtn.textContent = 'Activate';
activateBtn.addEventListener('click', () => activateExtension(ext.name));
actions.appendChild(activateBtn);
}
const activateBtn = document.createElement('button');
activateBtn.className = 'btn-ext activate';
activateBtn.textContent = 'Activate';
activateBtn.addEventListener('click', () => activateExtension(ext.name));
actions.appendChild(activateBtn);
} else {
const activeLabel = document.createElement('span');
activeLabel.className = 'ext-active-label';
@@ -1490,8 +1504,10 @@ function renderExtensionCard(ext) {
card.appendChild(actions);
// For active WASM channels, check for pending pairing requests
if (ext.active && ext.kind === 'wasm_channel') {
// For WASM channels, check for pending pairing requests.
// Show even when inactive — pairing requests can arrive via webhooks
// before the channel is fully activated.
if (ext.kind === 'wasm_channel') {
const pairingSection = document.createElement('div');
pairingSection.className = 'ext-pairing';
card.appendChild(pairingSection);
@@ -2192,14 +2208,20 @@ function appendActivityEvent(terminal, eventType, data) {
+ escapeHtml(typeof data.input === 'string' ? data.input : JSON.stringify(data.input, null, 2))
+ '</pre></details>';
break;
case 'tool_result':
el.innerHTML = '<details class="activity-tool-block activity-tool-result"><summary>'
+ '<span class="activity-tool-icon">&#10003;</span> '
case 'tool_result': {
const trSuccess = data.success !== false;
const trIcon = trSuccess ? '&#10003;' : '&#10007;';
const trOutput = data.output || data.error || '';
const trClass = 'activity-tool-block activity-tool-result'
+ (trSuccess ? '' : ' activity-tool-error');
el.innerHTML = '<details class="' + trClass + '"><summary>'
+ '<span class="activity-tool-icon">' + trIcon + '</span> '
+ escapeHtml(data.tool_name || 'result')
+ '</summary><pre class="activity-tool-output">'
+ escapeHtml(data.output || '')
+ escapeHtml(trOutput)
+ '</pre></details>';
break;
}
case 'status':
el.innerHTML = '<span class="activity-status">' + escapeHtml(data.message || '') + '</span>';
break;
@@ -2207,7 +2229,7 @@ function appendActivityEvent(terminal, eventType, data) {
el.className += ' activity-final';
const success = data.success !== false;
el.innerHTML = '<span class="activity-result-status" data-success="' + success + '">'
+ escapeHtml(data.message || data.status || 'done') + '</span>';
+ escapeHtml(data.message || data.error || data.status || 'done') + '</span>';
if (data.session_id) {
el.innerHTML += ' <span class="activity-session-id">session: ' + escapeHtml(data.session_id) + '</span>';
}
@@ -2392,7 +2414,9 @@ function renderRoutineDetail(routine) {
+ '<td>' + formatDate(run.started_at) + '</td>'
+ '<td>' + formatDate(run.completed_at) + '</td>'
+ '<td><span class="badge ' + runStatusClass + '">' + escapeHtml(run.status) + '</span></td>'
+ '<td>' + escapeHtml(run.result_summary || '-') + '</td>'
+ '<td>' + escapeHtml(run.result_summary || '-')
+ (run.job_id ? ' <a href="#" onclick="event.preventDefault(); switchTab(\'jobs\'); openJobDetail(\'' + run.job_id + '\')">[view job]</a>' : '')
+ '</td>'
+ '<td>' + (run.tokens_used != null ? run.tokens_used : '-') + '</td>'
+ '</tr>';
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 3.8 KiB

+1
View File
@@ -4,6 +4,7 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>IronClaw</title>
<link rel="icon" href="/favicon.ico" type="image/x-icon">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=DM+Sans:wght@400;500;600;700&family=IBM+Plex+Mono:wght@400;500;600&display=swap" rel="stylesheet">
+8 -6
View File
@@ -1936,12 +1936,6 @@ body {
font-weight: 500;
}
.ext-restart-label {
font-size: 12px;
color: var(--text-secondary);
font-style: italic;
}
.btn-ext {
padding: 4px 10px;
border-radius: var(--radius);
@@ -2246,6 +2240,14 @@ body {
color: var(--success);
}
.activity-tool-error .activity-tool-icon {
color: var(--danger);
}
.activity-tool-error summary {
color: var(--danger);
}
.activity-tool-input,
.activity-tool-output {
padding: 8px 10px;
+9 -166
View File
@@ -4,7 +4,6 @@
use std::io::Write;
use std::path::{Path, PathBuf};
use std::process::Command as ProcessCommand;
use std::sync::Arc;
use clap::Subcommand;
@@ -155,11 +154,18 @@ async fn install_tool(
};
// Build the WASM component if not skipping
let profile = if release { "release" } else { "debug" };
let wasm_path = if skip_build {
// Look for existing wasm file
find_wasm_artifact(&path, &tool_name, release)?
crate::registry::artifacts::find_wasm_artifact(&path, &tool_name, profile)
.or_else(|| crate::registry::artifacts::find_any_wasm_artifact(&path, profile))
.ok_or_else(|| {
anyhow::anyhow!(
"No .wasm artifact found. Run without --skip-build to build first."
)
})?
} else {
build_wasm_component(&path, release)?
crate::registry::artifacts::build_wasm_component_sync(&path, release)?
};
// Look for capabilities file
@@ -253,169 +259,6 @@ async fn install_tool(
Ok(())
}
/// Build a WASM component using cargo-component.
fn build_wasm_component(source_dir: &Path, release: bool) -> anyhow::Result<PathBuf> {
println!("Building WASM component in {}...", source_dir.display());
// Check if cargo-component is available
let check = ProcessCommand::new("cargo")
.args(["component", "--version"])
.output();
if check.is_err() || !check.unwrap().status.success() {
anyhow::bail!(
"cargo-component not found. Install with: cargo install cargo-component\n\
Or use --skip-build with an existing .wasm file."
);
}
// Build command
let mut cmd = ProcessCommand::new("cargo");
cmd.current_dir(source_dir).args(["component", "build"]);
if release {
cmd.arg("--release");
}
println!(
" Running: cargo component build{}",
if release { " --release" } else { "" }
);
let output = cmd.output()?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("Build failed:\n{}", stderr);
}
// Find the output wasm file
// cargo-component may output to wasm32-wasip1 or wasm32-wasip2 depending on version
let profile = if release { "release" } else { "debug" };
let candidates = [
source_dir
.join("target")
.join("wasm32-wasip1")
.join(profile),
source_dir
.join("target")
.join("wasm32-wasip2")
.join(profile),
source_dir
.join("target")
.join("wasm32-unknown-unknown")
.join(profile),
];
let target_dir = candidates.iter().find(|p| p.exists()).ok_or_else(|| {
anyhow::anyhow!(
"No WASM target directory found. Expected one of: {}",
candidates
.iter()
.map(|p| p.display().to_string())
.collect::<Vec<_>>()
.join(", ")
)
})?;
// Look for .wasm files in target dir
let entries: Vec<_> = std::fs::read_dir(target_dir)?
.filter_map(|e| e.ok())
.filter(|e| {
e.path()
.extension()
.map(|ext| ext == "wasm")
.unwrap_or(false)
})
.collect();
if entries.is_empty() {
anyhow::bail!(
"No .wasm file found in {}. Build may have failed.",
target_dir.display()
);
}
if entries.len() > 1 {
println!(
" Warning: Multiple .wasm files found, using first: {}",
entries[0].path().display()
);
}
let wasm_path = entries[0].path();
println!(" Built: {}", wasm_path.display());
Ok(wasm_path)
}
/// Find an existing WASM artifact without building.
fn find_wasm_artifact(source_dir: &Path, name: &str, release: bool) -> anyhow::Result<PathBuf> {
let profile = if release { "release" } else { "debug" };
// cargo-component may output to wasm32-wasip1 or wasm32-wasip2 depending on version
let target_dirs = [
source_dir
.join("target")
.join("wasm32-wasip1")
.join(profile),
source_dir
.join("target")
.join("wasm32-wasip2")
.join(profile),
source_dir
.join("target")
.join("wasm32-unknown-unknown")
.join(profile),
];
let snake_name = name.replace('-', "_");
// Try exact name match in any target dir first
for target_dir in &target_dirs {
let candidates = [
target_dir.join(format!("{}.wasm", name)),
target_dir.join(format!("{}.wasm", snake_name)),
];
for candidate in &candidates {
if candidate.exists() {
return Ok(candidate.clone());
}
}
}
// Find a target dir that exists
let target_dir = target_dirs.iter().find(|p| p.exists()).ok_or_else(|| {
anyhow::anyhow!("No target directory found. Run without --skip-build to build first.")
})?;
// Fall back to any .wasm file
let entries: Vec<_> = std::fs::read_dir(target_dir)
.map_err(|_| {
anyhow::anyhow!(
"Target directory not found: {}. Run without --skip-build.",
target_dir.display()
)
})?
.filter_map(|e| e.ok())
.filter(|e| {
e.path()
.extension()
.map(|ext| ext == "wasm")
.unwrap_or(false)
})
.collect();
if entries.is_empty() {
anyhow::bail!(
"No .wasm file found in {}. Build the project first or remove --skip-build.",
target_dir.display()
);
}
Ok(entries[0].path())
}
/// Extract crate name from Cargo.toml.
async fn extract_crate_name(cargo_toml: &Path) -> anyhow::Result<String> {
let content = fs::read_to_string(cargo_toml).await?;
+38 -104
View File
@@ -1,6 +1,6 @@
use std::time::Duration;
use crate::config::helpers::optional_env;
use crate::config::helpers::{parse_bool_env, parse_option_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -32,109 +32,43 @@ pub struct AgentConfig {
impl AgentConfig {
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
Ok(Self {
name: optional_env("AGENT_NAME")?.unwrap_or_else(|| settings.agent.name.clone()),
max_parallel_jobs: optional_env("AGENT_MAX_PARALLEL_JOBS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_MAX_PARALLEL_JOBS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.max_parallel_jobs as usize),
job_timeout: Duration::from_secs(
optional_env("AGENT_JOB_TIMEOUT_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_JOB_TIMEOUT_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.job_timeout_secs),
),
stuck_threshold: Duration::from_secs(
optional_env("AGENT_STUCK_THRESHOLD_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_STUCK_THRESHOLD_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.stuck_threshold_secs),
),
repair_check_interval: Duration::from_secs(
optional_env("SELF_REPAIR_CHECK_INTERVAL_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SELF_REPAIR_CHECK_INTERVAL_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.repair_check_interval_secs),
),
max_repair_attempts: optional_env("SELF_REPAIR_MAX_ATTEMPTS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SELF_REPAIR_MAX_ATTEMPTS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.max_repair_attempts),
use_planning: optional_env("AGENT_USE_PLANNING")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_USE_PLANNING".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.agent.use_planning),
session_idle_timeout: Duration::from_secs(
optional_env("SESSION_IDLE_TIMEOUT_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SESSION_IDLE_TIMEOUT_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.session_idle_timeout_secs),
),
allow_local_tools: optional_env("ALLOW_LOCAL_TOOLS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "ALLOW_LOCAL_TOOLS".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(false),
max_cost_per_day_cents: optional_env("MAX_COST_PER_DAY_CENTS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MAX_COST_PER_DAY_CENTS".to_string(),
message: format!("must be a positive integer: {e}"),
})?,
max_actions_per_hour: optional_env("MAX_ACTIONS_PER_HOUR")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MAX_ACTIONS_PER_HOUR".to_string(),
message: format!("must be a positive integer: {e}"),
})?,
max_tool_iterations: optional_env("AGENT_MAX_TOOL_ITERATIONS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_MAX_TOOL_ITERATIONS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.max_tool_iterations),
auto_approve_tools: optional_env("AGENT_AUTO_APPROVE_TOOLS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_AUTO_APPROVE_TOOLS".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.agent.auto_approve_tools),
name: parse_optional_env("AGENT_NAME", settings.agent.name.clone())?,
max_parallel_jobs: parse_optional_env(
"AGENT_MAX_PARALLEL_JOBS",
settings.agent.max_parallel_jobs as usize,
)?,
job_timeout: Duration::from_secs(parse_optional_env(
"AGENT_JOB_TIMEOUT_SECS",
settings.agent.job_timeout_secs,
)?),
stuck_threshold: Duration::from_secs(parse_optional_env(
"AGENT_STUCK_THRESHOLD_SECS",
settings.agent.stuck_threshold_secs,
)?),
repair_check_interval: Duration::from_secs(parse_optional_env(
"SELF_REPAIR_CHECK_INTERVAL_SECS",
settings.agent.repair_check_interval_secs,
)?),
max_repair_attempts: parse_optional_env(
"SELF_REPAIR_MAX_ATTEMPTS",
settings.agent.max_repair_attempts,
)?,
use_planning: parse_bool_env("AGENT_USE_PLANNING", settings.agent.use_planning)?,
session_idle_timeout: Duration::from_secs(parse_optional_env(
"SESSION_IDLE_TIMEOUT_SECS",
settings.agent.session_idle_timeout_secs,
)?),
allow_local_tools: parse_bool_env("ALLOW_LOCAL_TOOLS", false)?,
max_cost_per_day_cents: parse_option_env("MAX_COST_PER_DAY_CENTS")?,
max_actions_per_hour: parse_option_env("MAX_ACTIONS_PER_HOUR")?,
max_tool_iterations: parse_optional_env(
"AGENT_MAX_TOOL_ITERATIONS",
settings.agent.max_tool_iterations,
)?,
auto_approve_tools: parse_bool_env(
"AGENT_AUTO_APPROVE_TOOLS",
settings.agent.auto_approve_tools,
)?,
})
}
}
+3 -17
View File
@@ -1,7 +1,7 @@
use std::path::PathBuf;
use std::time::Duration;
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Builder mode configuration.
@@ -34,25 +34,11 @@ impl Default for BuilderModeConfig {
impl BuilderModeConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("BUILDER_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "BUILDER_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
enabled: parse_bool_env("BUILDER_ENABLED", true)?,
build_dir: optional_env("BUILDER_DIR")?.map(PathBuf::from),
max_iterations: parse_optional_env("BUILDER_MAX_ITERATIONS", 20)?,
timeout_secs: parse_optional_env("BUILDER_TIMEOUT_SECS", 600)?,
auto_register: optional_env("BUILDER_AUTO_REGISTER")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "BUILDER_AUTO_REGISTER".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
auto_register: parse_bool_env("BUILDER_AUTO_REGISTER", true)?,
})
}
+7 -30
View File
@@ -2,7 +2,7 @@ use std::path::PathBuf;
use secrecy::SecretString;
use crate::config::helpers::optional_env;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -48,14 +48,7 @@ impl ChannelsConfig {
let http = if optional_env("HTTP_PORT")?.is_some() || optional_env("HTTP_HOST")?.is_some() {
Some(HttpConfig {
host: optional_env("HTTP_HOST")?.unwrap_or_else(|| "0.0.0.0".to_string()),
port: optional_env("HTTP_PORT")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "HTTP_PORT".to_string(),
message: format!("must be a valid port number: {e}"),
})?
.unwrap_or(8080),
port: parse_optional_env("HTTP_PORT", 8080)?,
webhook_secret: optional_env("HTTP_WEBHOOK_SECRET")?.map(SecretString::from),
user_id: optional_env("HTTP_USER_ID")?.unwrap_or_else(|| "http".to_string()),
})
@@ -63,20 +56,11 @@ impl ChannelsConfig {
None
};
let gateway = if optional_env("GATEWAY_ENABLED")?
.map(|s| s.to_lowercase() == "true" || s == "1")
.unwrap_or(true)
{
let gateway_enabled = parse_bool_env("GATEWAY_ENABLED", true)?;
let gateway = if gateway_enabled {
Some(GatewayConfig {
host: optional_env("GATEWAY_HOST")?.unwrap_or_else(|| "127.0.0.1".to_string()),
port: optional_env("GATEWAY_PORT")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "GATEWAY_PORT".to_string(),
message: format!("must be a valid port number: {e}"),
})?
.unwrap_or(3000),
port: parse_optional_env("GATEWAY_PORT", 3000)?,
auth_token: optional_env("GATEWAY_AUTH_TOKEN")?,
user_id: optional_env("GATEWAY_USER_ID")?.unwrap_or_else(|| "default".to_string()),
})
@@ -97,18 +81,11 @@ impl ChannelsConfig {
wasm_channels_dir: optional_env("WASM_CHANNELS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_channels_dir),
wasm_channels_enabled: optional_env("WASM_CHANNELS_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "WASM_CHANNELS_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
wasm_channels_enabled: parse_bool_env("WASM_CHANNELS_ENABLED", true)?,
telegram_owner_id: optional_env("TELEGRAM_OWNER_ID")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
.map_err(|e: std::num::ParseIntError| ConfigError::InvalidValue {
key: "TELEGRAM_OWNER_ID".to_string(),
message: format!("must be an integer: {e}"),
})?
+67 -17
View File
@@ -1,8 +1,12 @@
use std::sync::Arc;
use secrecy::{ExposeSecret, SecretString};
use crate::config::helpers::optional_env;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
use crate::llm::SessionManager;
use crate::settings::Settings;
use crate::workspace::EmbeddingProvider;
/// Embeddings provider configuration.
#[derive(Debug, Clone)]
@@ -65,23 +69,10 @@ impl EmbeddingsConfig {
.or_else(|| settings.ollama_base_url.clone())
.unwrap_or_else(|| "http://localhost:11434".to_string());
let dimension = optional_env("EMBEDDING_DIMENSION")?
.map(|s| s.parse::<usize>())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "EMBEDDING_DIMENSION".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or_else(|| default_dimension_for_model(&model));
let dimension =
parse_optional_env("EMBEDDING_DIMENSION", default_dimension_for_model(&model))?;
let enabled = optional_env("EMBEDDING_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "EMBEDDING_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.embeddings.enabled);
let enabled = parse_bool_env("EMBEDDING_ENABLED", settings.embeddings.enabled)?;
Ok(Self {
enabled,
@@ -97,6 +88,65 @@ impl EmbeddingsConfig {
pub fn openai_api_key(&self) -> Option<&str> {
self.openai_api_key.as_ref().map(|s| s.expose_secret())
}
/// Create the appropriate embedding provider based on configuration.
///
/// Returns `None` if embeddings are disabled or the required credentials
/// are missing. The `nearai_base_url` and `session` are needed only for
/// the NEAR AI provider but must be passed unconditionally.
pub fn create_provider(
&self,
nearai_base_url: &str,
session: Arc<SessionManager>,
) -> Option<Arc<dyn EmbeddingProvider>> {
if !self.enabled {
tracing::info!("Embeddings disabled (set EMBEDDING_ENABLED=true to enable)");
return None;
}
match self.provider.as_str() {
"nearai" => {
tracing::info!(
"Embeddings enabled via NEAR AI (model: {}, dim: {})",
self.model,
self.dimension,
);
Some(Arc::new(
crate::workspace::NearAiEmbeddings::new(nearai_base_url, session)
.with_model(&self.model, self.dimension),
))
}
"ollama" => {
tracing::info!(
"Embeddings enabled via Ollama (model: {}, url: {}, dim: {})",
self.model,
self.ollama_base_url,
self.dimension,
);
Some(Arc::new(
crate::workspace::OllamaEmbeddings::new(&self.ollama_base_url)
.with_model(&self.model, self.dimension),
))
}
_ => {
if let Some(api_key) = self.openai_api_key() {
tracing::info!(
"Embeddings enabled via OpenAI (model: {}, dim: {})",
self.model,
self.dimension,
);
Some(Arc::new(crate::workspace::OpenAiEmbeddings::with_model(
api_key,
&self.model,
self.dimension,
)))
} else {
tracing::warn!("Embeddings configured but OPENAI_API_KEY not set");
None
}
}
}
}
}
#[cfg(test)]
+6 -17
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::optional_env;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -29,22 +29,11 @@ impl Default for HeartbeatConfig {
impl HeartbeatConfig {
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("HEARTBEAT_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "HEARTBEAT_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.heartbeat.enabled),
interval_secs: optional_env("HEARTBEAT_INTERVAL_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "HEARTBEAT_INTERVAL_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.heartbeat.interval_secs),
enabled: parse_bool_env("HEARTBEAT_ENABLED", settings.heartbeat.enabled)?,
interval_secs: parse_optional_env(
"HEARTBEAT_INTERVAL_SECS",
settings.heartbeat.interval_secs,
)?,
notify_channel: optional_env("HEARTBEAT_NOTIFY_CHANNEL")?
.or_else(|| settings.heartbeat.notify_channel.clone()),
notify_user: optional_env("HEARTBEAT_NOTIFY_USER")?
+42
View File
@@ -47,3 +47,45 @@ where
.transpose()
.map(|opt| opt.unwrap_or(default))
}
/// Parse a boolean from an env var with a default.
///
/// Accepts "true"/"1" as true, "false"/"0" as false.
pub(crate) fn parse_bool_env(key: &str, default: bool) -> Result<bool, ConfigError> {
match optional_env(key)? {
Some(s) => match s.to_lowercase().as_str() {
"true" | "1" => Ok(true),
"false" | "0" => Ok(false),
_ => Err(ConfigError::InvalidValue {
key: key.to_string(),
message: format!("must be 'true' or 'false', got '{s}'"),
}),
},
None => Ok(default),
}
}
/// Parse an env var into `Option<T>` — returns `None` when unset,
/// `Some(parsed)` when set to a valid value.
pub(crate) fn parse_option_env<T>(key: &str) -> Result<Option<T>, ConfigError>
where
T: std::str::FromStr,
T::Err: std::fmt::Display,
{
optional_env(key)?
.map(|s| {
s.parse().map_err(|e| ConfigError::InvalidValue {
key: key.to_string(),
message: format!("{e}"),
})
})
.transpose()
}
/// Parse a string from an env var with a default.
pub(crate) fn parse_string_env(
key: &str,
default: impl Into<String>,
) -> Result<String, ConfigError> {
Ok(optional_env(key)?.unwrap_or_else(|| default.into()))
}
+4 -25
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::optional_env;
use crate::config::helpers::{parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Memory hygiene configuration.
@@ -28,30 +28,9 @@ impl Default for HygieneConfig {
impl HygieneConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("MEMORY_HYGIENE_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MEMORY_HYGIENE_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
retention_days: optional_env("MEMORY_HYGIENE_RETENTION_DAYS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MEMORY_HYGIENE_RETENTION_DAYS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(30),
cadence_hours: optional_env("MEMORY_HYGIENE_CADENCE_HOURS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MEMORY_HYGIENE_CADENCE_HOURS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(12),
enabled: parse_bool_env("MEMORY_HYGIENE_ENABLED", true)?,
retention_days: parse_optional_env("MEMORY_HYGIENE_RETENTION_DAYS", 30)?,
cadence_hours: parse_optional_env("MEMORY_HYGIENE_CADENCE_HOURS", 12)?,
})
}
+1 -4
View File
@@ -206,10 +206,7 @@ impl LlmConfig {
let nearai = NearAiConfig {
model: optional_env("NEARAI_MODEL")?
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| {
"fireworks::accounts/fireworks/models/llama4-maverick-instruct-basic"
.to_string()
}),
.unwrap_or_else(|| "zai-org/GLM-latest".to_string()),
cheap_model: optional_env("NEARAI_CHEAP_MODEL")?,
base_url: optional_env("NEARAI_BASE_URL")?.unwrap_or_else(|| {
if nearai_api_key.is_some() {
-4
View File
@@ -19,7 +19,6 @@ mod safety;
mod sandbox;
mod secrets;
mod skills;
mod transcription;
mod tunnel;
mod wasm;
@@ -46,7 +45,6 @@ pub use self::safety::SafetyConfig;
pub use self::sandbox::{ClaudeCodeConfig, SandboxModeConfig};
pub use self::secrets::SecretsConfig;
pub use self::skills::SkillsConfig;
pub use self::transcription::TranscriptionConfig;
pub use self::tunnel::TunnelConfig;
pub use self::wasm::WasmConfig;
@@ -76,7 +74,6 @@ pub struct Config {
pub sandbox: SandboxModeConfig,
pub claude_code: ClaudeCodeConfig,
pub skills: SkillsConfig,
pub transcription: TranscriptionConfig,
pub observability: crate::observability::ObservabilityConfig,
}
@@ -201,7 +198,6 @@ impl Config {
sandbox: SandboxModeConfig::resolve()?,
claude_code: ClaudeCodeConfig::resolve()?,
skills: SkillsConfig::resolve()?,
transcription: TranscriptionConfig::resolve(settings)?,
observability: crate::observability::ObservabilityConfig {
backend: std::env::var("OBSERVABILITY_BACKEND").unwrap_or_else(|_| "none".into()),
},
+2 -9
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Routines configuration.
@@ -31,14 +31,7 @@ impl Default for RoutineConfig {
impl RoutineConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("ROUTINES_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "ROUTINES_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
enabled: parse_bool_env("ROUTINES_ENABLED", true)?,
cron_check_interval_secs: parse_optional_env("ROUTINES_CRON_INTERVAL", 15)?,
max_concurrent_routines: parse_optional_env("ROUTINES_MAX_CONCURRENT", 10)?,
default_cooldown_secs: parse_optional_env("ROUTINES_DEFAULT_COOLDOWN", 300)?,
+2 -9
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Safety configuration.
@@ -12,14 +12,7 @@ impl SafetyConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
max_output_length: parse_optional_env("SAFETY_MAX_OUTPUT_LENGTH", 100_000)?,
injection_check_enabled: optional_env("SAFETY_INJECTION_CHECK_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SAFETY_INJECTION_CHECK_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
injection_check_enabled: parse_bool_env("SAFETY_INJECTION_CHECK_ENABLED", true)?,
})
}
}
+7 -29
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env, parse_string_env};
use crate::error::ConfigError;
/// Docker sandbox configuration.
@@ -44,28 +44,13 @@ impl SandboxModeConfig {
.unwrap_or_default();
Ok(Self {
enabled: optional_env("SANDBOX_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SANDBOX_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
policy: optional_env("SANDBOX_POLICY")?.unwrap_or_else(|| "readonly".to_string()),
enabled: parse_bool_env("SANDBOX_ENABLED", true)?,
policy: parse_string_env("SANDBOX_POLICY", "readonly")?,
timeout_secs: parse_optional_env("SANDBOX_TIMEOUT_SECS", 120)?,
memory_limit_mb: parse_optional_env("SANDBOX_MEMORY_LIMIT_MB", 2048)?,
cpu_shares: parse_optional_env("SANDBOX_CPU_SHARES", 1024)?,
image: optional_env("SANDBOX_IMAGE")?
.unwrap_or_else(|| "ironclaw-worker:latest".to_string()),
auto_pull_image: optional_env("SANDBOX_AUTO_PULL")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SANDBOX_AUTO_PULL".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
image: parse_string_env("SANDBOX_IMAGE", "ironclaw-worker:latest")?,
auto_pull_image: parse_bool_env("SANDBOX_AUTO_PULL", true)?,
extra_allowed_domains: extra_domains,
})
}
@@ -221,18 +206,11 @@ impl ClaudeCodeConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
let defaults = Self::default();
Ok(Self {
enabled: optional_env("CLAUDE_CODE_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "CLAUDE_CODE_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(defaults.enabled),
enabled: parse_bool_env("CLAUDE_CODE_ENABLED", defaults.enabled)?,
config_dir: optional_env("CLAUDE_CONFIG_DIR")?
.map(std::path::PathBuf::from)
.unwrap_or(defaults.config_dir),
model: optional_env("CLAUDE_CODE_MODEL")?.unwrap_or(defaults.model),
model: parse_string_env("CLAUDE_CODE_MODEL", defaults.model)?,
max_turns: parse_optional_env("CLAUDE_CODE_MAX_TURNS", defaults.max_turns)?,
memory_limit_mb: parse_optional_env(
"CLAUDE_CODE_MEMORY_LIMIT_MB",
+2 -9
View File
@@ -1,6 +1,6 @@
use std::path::PathBuf;
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Skills system configuration.
@@ -38,14 +38,7 @@ fn default_skills_dir() -> PathBuf {
impl SkillsConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("SKILLS_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SKILLS_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(false),
enabled: parse_bool_env("SKILLS_ENABLED", false)?,
local_dir: optional_env("SKILLS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_skills_dir),
-195
View File
@@ -1,195 +0,0 @@
use secrecy::SecretString;
use crate::config::helpers::optional_env;
use crate::error::ConfigError;
use crate::settings::Settings;
/// Transcription provider configuration.
#[derive(Debug, Clone)]
pub struct TranscriptionConfig {
/// Whether transcription is enabled.
pub enabled: bool,
/// Provider to use: "openai".
pub provider: String,
/// OpenAI API key (reused from embeddings/LLM config).
pub openai_api_key: Option<SecretString>,
/// Model to use for transcription.
pub model: String,
/// Optional language hint (ISO-639-1, e.g., "en").
pub language: Option<String>,
}
impl Default for TranscriptionConfig {
fn default() -> Self {
Self {
enabled: false,
provider: "openai".to_string(),
openai_api_key: None,
model: "whisper-1".to_string(),
language: None,
}
}
}
impl TranscriptionConfig {
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
let openai_api_key = optional_env("OPENAI_API_KEY")?.map(SecretString::from);
let provider = optional_env("TRANSCRIPTION_PROVIDER")?
.unwrap_or_else(|| settings.transcription.provider.clone());
let model = optional_env("TRANSCRIPTION_MODEL")?
.unwrap_or_else(|| settings.transcription.model.clone());
let language = optional_env("TRANSCRIPTION_LANGUAGE")?
.or_else(|| settings.transcription.language.clone());
let enabled = optional_env("TRANSCRIPTION_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "TRANSCRIPTION_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.transcription.enabled);
// Only "openai" is currently supported
if enabled && provider != "openai" {
return Err(ConfigError::InvalidValue {
key: "TRANSCRIPTION_PROVIDER".to_string(),
message: format!(
"unsupported provider '{}', only 'openai' is currently supported",
provider
),
});
}
Ok(Self {
enabled,
provider,
openai_api_key,
model,
language,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::config::helpers::ENV_MUTEX;
use crate::settings::{Settings, TranscriptionSettings};
fn clear_transcription_env() {
unsafe {
std::env::remove_var("TRANSCRIPTION_ENABLED");
std::env::remove_var("TRANSCRIPTION_PROVIDER");
std::env::remove_var("TRANSCRIPTION_MODEL");
std::env::remove_var("TRANSCRIPTION_LANGUAGE");
std::env::remove_var("OPENAI_API_KEY");
}
}
#[test]
fn transcription_defaults_from_settings() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_transcription_env();
let settings = Settings::default();
let config = TranscriptionConfig::resolve(&settings).expect("resolve should succeed");
assert!(!config.enabled);
assert_eq!(config.provider, "openai");
assert_eq!(config.model, "whisper-1");
assert!(config.language.is_none());
}
#[test]
fn transcription_env_overrides_settings() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_transcription_env();
unsafe {
std::env::set_var("TRANSCRIPTION_ENABLED", "true");
std::env::set_var("TRANSCRIPTION_MODEL", "whisper-large-v3");
std::env::set_var("TRANSCRIPTION_LANGUAGE", "en");
}
let settings = Settings::default();
let config = TranscriptionConfig::resolve(&settings).expect("resolve should succeed");
assert!(config.enabled);
assert_eq!(config.model, "whisper-large-v3");
assert_eq!(config.language, Some("en".to_string()));
unsafe {
std::env::remove_var("TRANSCRIPTION_ENABLED");
std::env::remove_var("TRANSCRIPTION_MODEL");
std::env::remove_var("TRANSCRIPTION_LANGUAGE");
}
}
#[test]
fn transcription_settings_with_custom_values() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_transcription_env();
let settings = Settings {
transcription: TranscriptionSettings {
enabled: true,
model: "whisper-large-v3".to_string(),
language: Some("fr".to_string()),
..Default::default()
},
..Default::default()
};
let config = TranscriptionConfig::resolve(&settings).expect("resolve should succeed");
assert!(config.enabled);
assert_eq!(config.model, "whisper-large-v3");
assert_eq!(config.language, Some("fr".to_string()));
}
#[test]
fn transcription_rejects_unsupported_provider() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_transcription_env();
let settings = Settings {
transcription: TranscriptionSettings {
enabled: true,
provider: "deepgram".to_string(),
..Default::default()
},
..Default::default()
};
let result = TranscriptionConfig::resolve(&settings);
assert!(result.is_err());
let err_msg = result.unwrap_err().to_string();
assert!(
err_msg.contains("unsupported provider"),
"Error should mention unsupported provider, got: {err_msg}"
);
}
#[test]
fn transcription_disabled_skips_provider_validation() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_transcription_env();
// When disabled, any provider string is accepted (never used)
let settings = Settings {
transcription: TranscriptionSettings {
enabled: false,
provider: "nonexistent".to_string(),
..Default::default()
},
..Default::default()
};
let config = TranscriptionConfig::resolve(&settings).expect("should succeed when disabled");
assert!(!config.enabled);
}
}
+3 -17
View File
@@ -1,7 +1,7 @@
use std::path::PathBuf;
use std::time::Duration;
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// WASM sandbox configuration.
@@ -48,14 +48,7 @@ fn default_tools_dir() -> PathBuf {
impl WasmConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("WASM_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "WASM_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
enabled: parse_bool_env("WASM_ENABLED", true)?,
tools_dir: optional_env("WASM_TOOLS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_tools_dir),
@@ -65,14 +58,7 @@ impl WasmConfig {
)?,
default_timeout_secs: parse_optional_env("WASM_DEFAULT_TIMEOUT_SECS", 60)?,
default_fuel_limit: parse_optional_env("WASM_DEFAULT_FUEL_LIMIT", 10_000_000)?,
cache_compiled: optional_env("WASM_CACHE_COMPILED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "WASM_CACHE_COMPILED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
cache_compiled: parse_bool_env("WASM_CACHE_COMPILED", true)?,
cache_dir: optional_env("WASM_CACHE_DIR")?.map(PathBuf::from),
})
}
+15
View File
@@ -387,4 +387,19 @@ impl RoutineStore for LibSqlBackend {
None => Ok(0),
}
}
async fn link_routine_run_to_job(
&self,
run_id: Uuid,
job_id: Uuid,
) -> Result<(), DatabaseError> {
let conn = self.connect().await?;
conn.execute(
"UPDATE routine_runs SET job_id = ?1 WHERE id = ?2",
params![job_id.to_string(), run_id.to_string()],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
Ok(())
}
}
+5
View File
@@ -274,6 +274,11 @@ pub trait RoutineStore: Send + Sync {
limit: i64,
) -> Result<Vec<RoutineRun>, DatabaseError>;
async fn count_running_routine_runs(&self, routine_id: Uuid) -> Result<i64, DatabaseError>;
async fn link_routine_run_to_job(
&self,
run_id: Uuid,
job_id: Uuid,
) -> Result<(), DatabaseError>;
}
#[async_trait]
+8
View File
@@ -437,6 +437,14 @@ impl RoutineStore for PgBackend {
async fn count_running_routine_runs(&self, routine_id: Uuid) -> Result<i64, DatabaseError> {
self.store.count_running_routine_runs(routine_id).await
}
async fn link_routine_run_to_job(
&self,
run_id: Uuid,
job_id: Uuid,
) -> Result<(), DatabaseError> {
self.store.link_routine_run_to_job(run_id, job_id).await
}
}
// ==================== ToolFailureStore ====================
+3
View File
@@ -407,6 +407,9 @@ pub enum RoutineError {
#[error("LLM call failed: {reason}")]
LlmFailed { reason: String },
#[error("Failed to dispatch full job: {reason}")]
JobDispatchFailed { reason: String },
#[error("LLM returned empty content")]
EmptyResponse,
+1 -2
View File
@@ -1,4 +1,4 @@
//! Online extension discovery for finding MCP servers not in the built-in registry.
//! Online extension discovery for finding extensions not in the built-in registry.
//!
//! Multi-tier search strategy:
//! 1. Probe well-known URL patterns (mcp.{service}.com, {service}.com/mcp)
@@ -246,7 +246,6 @@ fn extract_url(source: &ExtensionSource) -> String {
ExtensionSource::Discovered { url } => url.clone(),
ExtensionSource::WasmDownload { wasm_url, .. } => wasm_url.clone(),
ExtensionSource::WasmBuildable { repo_url, .. } => repo_url.clone(),
ExtensionSource::Bundled { name } => name.clone(),
}
}
+588 -34
View File
@@ -1,8 +1,8 @@
//! Central extension manager that dispatches operations by ExtensionKind.
//!
//! Holds references to MCP infrastructure, WASM tool runtime, secrets store,
//! and tool registry. All extension operations (search, install, auth, activate,
//! list, remove) flow through here.
//! Holds references to channel runtime, WASM tool runtime, MCP infrastructure,
//! secrets store, and tool registry. All extension operations (search, install,
//! auth, activate, list, remove) flow through here.
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
@@ -10,6 +10,10 @@ use std::sync::Arc;
use tokio::sync::RwLock;
use crate::channels::ChannelManager;
use crate::channels::wasm::{
RegisteredEndpoint, SharedWasmChannel, WasmChannelLoader, WasmChannelRouter, WasmChannelRuntime,
};
use crate::extensions::discovery::OnlineDiscovery;
use crate::extensions::registry::ExtensionRegistry;
use crate::extensions::{
@@ -17,6 +21,7 @@ use crate::extensions::{
InstalledExtension, RegistryEntry, ResultSource, SearchResult,
};
use crate::hooks::HookRegistry;
use crate::pairing::PairingStore;
use crate::secrets::{CreateSecretParams, SecretsStore};
use crate::tools::ToolRegistry;
use crate::tools::mcp::McpClient;
@@ -35,6 +40,18 @@ struct PendingAuth {
created_at: std::time::Instant,
}
/// Runtime infrastructure needed for hot-activating WASM channels.
///
/// Set after construction via [`ExtensionManager::set_channel_runtime`] once the
/// channel manager, WASM runtime, pairing store, and webhook router are available.
struct ChannelRuntimeState {
channel_manager: Arc<ChannelManager>,
wasm_channel_runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
wasm_channel_router: Arc<WasmChannelRouter>,
telegram_owner_id: Option<i64>,
}
/// Central manager for extension lifecycle operations.
pub struct ExtensionManager {
registry: ExtensionRegistry,
@@ -50,13 +67,16 @@ pub struct ExtensionManager {
wasm_tools_dir: PathBuf,
wasm_channels_dir: PathBuf,
// WASM channel hot-activation infrastructure (set post-construction)
channel_runtime: RwLock<Option<ChannelRuntimeState>>,
// Shared
secrets: Arc<dyn SecretsStore + Send + Sync>,
tool_registry: Arc<ToolRegistry>,
hooks: Option<Arc<HookRegistry>>,
pending_auth: RwLock<HashMap<String, PendingAuth>>,
/// Tunnel URL for remote OAuth callbacks (used in future iterations).
_tunnel_url: Option<String>,
/// Tunnel URL for webhook configuration and remote OAuth callbacks.
tunnel_url: Option<String>,
user_id: String,
/// Optional database store for DB-backed MCP config.
store: Option<Arc<dyn crate::db::Database>>,
@@ -92,17 +112,40 @@ impl ExtensionManager {
wasm_tool_runtime,
wasm_tools_dir,
wasm_channels_dir,
channel_runtime: RwLock::new(None),
secrets,
tool_registry,
hooks,
pending_auth: RwLock::new(HashMap::new()),
_tunnel_url: tunnel_url,
tunnel_url,
user_id,
store,
active_channel_names: RwLock::new(HashSet::new()),
}
}
/// Configure the channel runtime infrastructure for hot-activating WASM channels.
///
/// Call after construction (and after wrapping in `Arc`) once the channel
/// manager, WASM runtime, pairing store, and webhook router are available.
/// Without this, channel activation returns an error.
pub async fn set_channel_runtime(
&self,
channel_manager: Arc<ChannelManager>,
wasm_channel_runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
wasm_channel_router: Arc<WasmChannelRouter>,
telegram_owner_id: Option<i64>,
) {
*self.channel_runtime.write().await = Some(ChannelRuntimeState {
channel_manager,
wasm_channel_runtime,
pairing_store,
wasm_channel_router,
telegram_owner_id,
});
}
/// Register channel names that were loaded at startup.
/// Called after WASM channels are loaded so `list()` reports accurate active status.
pub async fn set_active_channels(&self, names: Vec<String>) {
@@ -207,14 +250,18 @@ impl ExtensionManager {
match kind {
ExtensionKind::McpServer => self.activate_mcp(name).await,
ExtensionKind::WasmTool => self.activate_wasm_tool(name).await,
ExtensionKind::WasmChannel => Err(ExtensionError::ChannelNeedsRestart),
ExtensionKind::WasmChannel => self.activate_wasm_channel(name).await,
}
}
/// List all installed extensions with their status.
/// List extensions with their status.
///
/// When `include_available` is `true`, registry entries that are not yet
/// installed are appended with `installed: false`.
pub async fn list(
&self,
kind_filter: Option<ExtensionKind>,
include_available: bool,
) -> Result<Vec<InstalledExtension>, ExtensionError> {
let mut extensions = Vec::new();
@@ -249,6 +296,7 @@ impl ExtensionManager {
active,
tools,
needs_setup: false,
installed: true,
});
}
}
@@ -276,6 +324,7 @@ impl ExtensionManager {
active,
tools: if active { vec![name] } else { Vec::new() },
needs_setup: false,
installed: true,
});
}
}
@@ -305,6 +354,7 @@ impl ExtensionManager {
active,
tools: Vec::new(),
needs_setup,
installed: true,
});
}
}
@@ -314,6 +364,36 @@ impl ExtensionManager {
}
}
// Append available-but-not-installed registry entries
if include_available {
let installed_names: std::collections::HashSet<(String, ExtensionKind)> = extensions
.iter()
.map(|e| (e.name.clone(), e.kind))
.collect();
for entry in self.registry.all_entries().await {
if let Some(filter) = kind_filter
&& entry.kind != filter
{
continue;
}
if installed_names.contains(&(entry.name.clone(), entry.kind)) {
continue;
}
extensions.push(InstalledExtension {
name: entry.name,
kind: entry.kind,
description: Some(entry.description),
url: None,
authenticated: false,
active: false,
tools: Vec::new(),
needs_setup: false,
installed: false,
});
}
}
Ok(extensions)
}
@@ -491,12 +571,19 @@ impl ExtensionManager {
)
.await
}
ExtensionSource::WasmBuildable { .. } => {
Err(ExtensionError::InstallFailed(format!(
"'{}' requires building from source. Run `ironclaw registry install {}` \
from the CLI (requires cargo-component).",
entry.name, entry.name
)))
ExtensionSource::WasmBuildable {
build_dir,
crate_name,
..
} => {
self.install_wasm_from_buildable(
&entry.name,
build_dir.as_deref(),
crate_name.as_deref(),
&self.wasm_tools_dir,
ExtensionKind::WasmTool,
)
.await
}
_ => Err(ExtensionError::InstallFailed(
"WASM tool entry has no download URL".to_string(),
@@ -514,15 +601,19 @@ impl ExtensionManager {
)
.await
}
ExtensionSource::WasmBuildable { .. } => {
Err(ExtensionError::InstallFailed(format!(
"'{}' requires building from source. Run `ironclaw registry install {}` \
from the CLI (requires cargo-component).",
entry.name, entry.name
)))
}
ExtensionSource::Bundled { name } => {
self.install_bundled_channel_from_artifacts(name).await
ExtensionSource::WasmBuildable {
build_dir,
crate_name,
..
} => {
self.install_wasm_from_buildable(
&entry.name,
build_dir.as_deref(),
crate_name.as_deref(),
&self.wasm_channels_dir,
ExtensionKind::WasmChannel,
)
.await
}
_ => Err(ExtensionError::InstallFailed(
"WASM channel entry has no download URL".to_string(),
@@ -600,9 +691,8 @@ impl ExtensionManager {
name: name.to_string(),
kind: ExtensionKind::WasmChannel,
message: format!(
"WASM channel '{}' installed to {}. Restart to activate.",
"WASM channel '{}' installed. Run activate to start it.",
name,
self.wasm_channels_dir.display()
),
})
}
@@ -829,6 +919,7 @@ impl ExtensionManager {
Ok(())
}
#[allow(dead_code)] // Used by upcoming hot-activation flow
async fn install_bundled_channel_from_artifacts(
&self,
name: &str,
@@ -853,11 +944,86 @@ impl ExtensionManager {
name: name.to_string(),
kind: ExtensionKind::WasmChannel,
message: format!(
"Channel '{}' installed to {}. Restart IronClaw for the channel to activate. \
Run tool_auth('{}') to configure authentication before restarting.",
name,
self.wasm_channels_dir.display(),
name,
"Channel '{}' installed. \
Run tool_auth('{}') to configure authentication, then activate.",
name, name,
),
})
}
/// Install a WASM extension from local build artifacts (WasmBuildable source).
///
/// Resolves the build directory (relative to `CARGO_MANIFEST_DIR` or absolute),
/// looks for the compiled WASM artifact, and copies it (plus capabilities.json)
/// to the install directory. Falls back to an error if artifacts don't exist.
async fn install_wasm_from_buildable(
&self,
name: &str,
build_dir: Option<&str>,
crate_name: Option<&str>,
target_dir: &std::path::Path,
kind: ExtensionKind,
) -> Result<InstallResult, ExtensionError> {
let manifest_dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR"));
// Resolve build directory
let resolved_dir = match build_dir {
Some(dir) => {
let p = std::path::Path::new(dir);
if p.is_absolute() {
p.to_path_buf()
} else {
manifest_dir.join(dir)
}
}
None => manifest_dir.to_path_buf(),
};
// Determine the binary name to look for
let binary_name = crate_name.unwrap_or(name);
let wasm_src =
crate::registry::artifacts::find_wasm_artifact(&resolved_dir, binary_name, "release")
.ok_or_else(|| {
ExtensionError::InstallFailed(format!(
"'{}' requires building from source. Build artifact not found. \
Run `cargo component build --release` in {} first, \
or use `ironclaw registry install {}`.",
name,
resolved_dir.display(),
name,
))
})?;
let wasm_dst = crate::registry::artifacts::install_wasm_files(
&wasm_src,
&resolved_dir,
name,
target_dir,
true,
)
.await
.map_err(|e| ExtensionError::InstallFailed(e.to_string()))?;
let kind_label = match kind {
ExtensionKind::WasmTool => "WASM tool",
ExtensionKind::WasmChannel => "WASM channel",
ExtensionKind::McpServer => "MCP server",
};
tracing::info!(
"Installed {} '{}' from build artifacts at {}",
kind_label,
name,
wasm_dst.display(),
);
Ok(InstallResult {
name: name.to_string(),
kind,
message: format!(
"{} '{}' installed from local build artifacts. Run activate to load it.",
kind_label, name,
),
})
}
@@ -1485,6 +1651,332 @@ impl ExtensionManager {
})
}
/// Activate a WASM channel at runtime without restarting.
///
/// Loads the channel from its WASM file, injects credentials and config,
/// registers it with the webhook router, and hot-adds it to the channel manager
/// so its stream feeds into the agent loop.
async fn activate_wasm_channel(&self, name: &str) -> Result<ActivateResult, ExtensionError> {
// If already active, re-inject credentials and refresh webhook secret.
// Handles the case where a channel was loaded at startup before the
// user saved secrets via the web UI.
{
let active = self.active_channel_names.read().await;
if active.contains(name) {
return self.refresh_active_channel(name).await;
}
}
// Verify runtime infrastructure is available and clone Arcs so we don't
// hold the RwLock guard across awaits.
let (
channel_runtime,
channel_manager,
pairing_store,
wasm_channel_router,
telegram_owner_id,
) = {
let rt_guard = self.channel_runtime.read().await;
let rt = rt_guard.as_ref().ok_or_else(|| {
ExtensionError::ActivationFailed(
"WASM channel runtime not configured. Restart IronClaw to activate."
.to_string(),
)
})?;
(
Arc::clone(&rt.wasm_channel_runtime),
Arc::clone(&rt.channel_manager),
Arc::clone(&rt.pairing_store),
Arc::clone(&rt.wasm_channel_router),
rt.telegram_owner_id,
)
};
// Check auth status first
let (authenticated, _needs_setup) = self.check_channel_auth_status(name).await;
if !authenticated {
return Err(ExtensionError::ActivationFailed(format!(
"Channel '{}' requires configuration. Use the setup form to provide credentials.",
name
)));
}
// Validate name to prevent path traversal
if name.contains('/') || name.contains('\\') || name.contains("..") || name.contains('\0') {
return Err(ExtensionError::ActivationFailed(format!(
"Invalid channel name '{}': contains path separator or traversal characters",
name
)));
}
// Load the channel from files
let wasm_path = self.wasm_channels_dir.join(format!("{}.wasm", name));
let cap_path = self
.wasm_channels_dir
.join(format!("{}.capabilities.json", name));
let cap_path_option = if cap_path.exists() {
Some(cap_path.as_path())
} else {
None
};
let loader =
WasmChannelLoader::new(Arc::clone(&channel_runtime), Arc::clone(&pairing_store));
let loaded = loader
.load_from_files(name, &wasm_path, cap_path_option)
.await
.map_err(|e| ExtensionError::ActivationFailed(e.to_string()))?;
let channel_name = loaded.name().to_string();
let webhook_secret_name = loaded.webhook_secret_name();
let secret_header = loaded.webhook_secret_header().map(|s| s.to_string());
// Get webhook secret from secrets store
let webhook_secret = self
.secrets
.get_decrypted(&self.user_id, &webhook_secret_name)
.await
.ok()
.map(|s| s.expose().to_string());
let channel_arc = Arc::new(loaded.channel);
// Inject runtime config (tunnel_url, webhook_secret, owner_id)
{
let mut config_updates = std::collections::HashMap::new();
if let Some(ref tunnel_url) = self.tunnel_url {
config_updates.insert(
"tunnel_url".to_string(),
serde_json::Value::String(tunnel_url.clone()),
);
}
if let Some(ref secret) = webhook_secret {
config_updates.insert(
"webhook_secret".to_string(),
serde_json::Value::String(secret.clone()),
);
}
if channel_name == "telegram"
&& let Some(owner_id) = telegram_owner_id
{
config_updates.insert("owner_id".to_string(), serde_json::json!(owner_id));
}
if !config_updates.is_empty() {
channel_arc.update_config(config_updates).await;
tracing::info!(
channel = %channel_name,
has_tunnel = self.tunnel_url.is_some(),
has_webhook_secret = webhook_secret.is_some(),
"Injected runtime config into hot-activated channel"
);
}
}
// Register with webhook router
{
let webhook_path = format!("/webhook/{}", channel_name);
let endpoints = vec![RegisteredEndpoint {
channel_name: channel_name.clone(),
path: webhook_path,
methods: vec!["POST".to_string()],
require_secret: webhook_secret.is_some(),
}];
wasm_channel_router
.register(
Arc::clone(&channel_arc),
endpoints,
webhook_secret,
secret_header,
)
.await;
tracing::info!(channel = %channel_name, "Registered hot-activated channel with webhook router");
}
// Inject credentials
match crate::extensions::manager::inject_channel_credentials_from_secrets(
&channel_arc,
self.secrets.as_ref(),
&channel_name,
&self.user_id,
)
.await
{
Ok(count) => {
if count > 0 {
tracing::info!(
channel = %channel_name,
credentials_injected = count,
"Credentials injected into hot-activated channel"
);
}
}
Err(e) => {
tracing::error!(
channel = %channel_name,
error = %e,
"Failed to inject credentials into hot-activated channel"
);
}
}
// Hot-add the channel to the running agent
channel_manager
.hot_add(Box::new(SharedWasmChannel::new(channel_arc)))
.await
.map_err(|e| ExtensionError::ActivationFailed(e.to_string()))?;
// Mark as active
self.active_channel_names
.write()
.await
.insert(channel_name.clone());
tracing::info!(channel = %channel_name, "Hot-activated WASM channel");
Ok(ActivateResult {
name: channel_name,
kind: ExtensionKind::WasmChannel,
tools_loaded: Vec::new(),
message: format!("Channel '{}' activated and running", name),
})
}
/// Refresh credentials and webhook secret on an already-active channel.
///
/// Called when the user saves new secrets via the setup form for a channel
/// that was loaded at startup (possibly without credentials).
async fn refresh_active_channel(&self, name: &str) -> Result<ActivateResult, ExtensionError> {
let router = {
let rt_guard = self.channel_runtime.read().await;
match rt_guard.as_ref() {
Some(rt) => Arc::clone(&rt.wasm_channel_router),
None => {
return Ok(ActivateResult {
name: name.to_string(),
kind: ExtensionKind::WasmChannel,
tools_loaded: Vec::new(),
message: format!("Channel '{}' is already active", name),
});
}
}
};
let webhook_path = format!("/webhook/{}", name);
let existing_channel = match router.get_channel_for_path(&webhook_path).await {
Some(ch) => ch,
None => {
return Ok(ActivateResult {
name: name.to_string(),
kind: ExtensionKind::WasmChannel,
tools_loaded: Vec::new(),
message: format!("Channel '{}' is already active", name),
});
}
};
// Re-inject credentials from secrets store into the running channel
let cred_count = match inject_channel_credentials_from_secrets(
&existing_channel,
self.secrets.as_ref(),
name,
&self.user_id,
)
.await
{
Ok(count) => count,
Err(e) => {
tracing::warn!(
channel = %name,
error = %e,
"Failed to refresh credentials on already-active channel"
);
0
}
};
// Also refresh the webhook secret in the router
// Load capabilities file to get the correct secret name (may be overridden)
let webhook_secret_name = {
let cap_path = self
.wasm_channels_dir
.join(format!("{}.capabilities.json", name));
match tokio::fs::read(&cap_path).await {
Ok(bytes) => crate::channels::wasm::ChannelCapabilitiesFile::from_bytes(&bytes)
.map(|f| f.webhook_secret_name())
.unwrap_or_else(|_| format!("{}_webhook_secret", name)),
Err(_) => format!("{}_webhook_secret", name),
}
};
if let Ok(secret) = self
.secrets
.get_decrypted(&self.user_id, &webhook_secret_name)
.await
{
router
.update_secret(name, secret.expose().to_string())
.await;
// Also inject the webhook_secret into the channel's runtime config
let mut config_updates = std::collections::HashMap::new();
config_updates.insert(
"webhook_secret".to_string(),
serde_json::Value::String(secret.expose().to_string()),
);
existing_channel.update_config(config_updates).await;
}
// Refresh tunnel_url in case it wasn't set at startup
if let Some(ref tunnel_url) = self.tunnel_url {
let mut config_updates = std::collections::HashMap::new();
config_updates.insert(
"tunnel_url".to_string(),
serde_json::Value::String(tunnel_url.clone()),
);
existing_channel.update_config(config_updates).await;
}
// Re-call on_start() to trigger webhook registration with the
// now-available credentials (e.g., setWebhook for Telegram).
if cred_count > 0 {
match existing_channel.call_on_start().await {
Ok(_config) => {
tracing::info!(
channel = %name,
"Re-ran on_start after credential refresh (webhook re-registered)"
);
}
Err(e) => {
tracing::warn!(
channel = %name,
error = %e,
"on_start failed after credential refresh"
);
}
}
}
tracing::info!(
channel = %name,
credentials_refreshed = cred_count,
"Refreshed credentials and config on already-active channel"
);
Ok(ActivateResult {
name: name.to_string(),
kind: ExtensionKind::WasmChannel,
tools_loaded: Vec::new(),
message: format!(
"Channel '{}' is already active; refreshed {} credential(s)",
name, cred_count
),
})
}
/// Determine what kind of installed extension this is.
async fn determine_installed_kind(&self, name: &str) -> Result<ExtensionKind, ExtensionError> {
// Check MCP servers first
@@ -1643,10 +2135,25 @@ impl ExtensionManager {
}
}
Ok(format!(
"Configuration saved for '{}'. Restart IronClaw for changes to take effect.",
name
))
// Try to hot-activate the channel now that secrets are saved
match self.activate_wasm_channel(name).await {
Ok(result) => Ok(format!(
"Configuration saved and channel '{}' activated. {}",
name, result.message
)),
Err(e) => {
tracing::warn!(
channel = name,
error = %e,
"Saved configuration but hot-activation failed, restart may be needed"
);
Ok(format!(
"Configuration saved for '{}'. \
Automatic activation failed ({}), restart IronClaw to activate.",
name, e
))
}
}
}
async fn unregister_hook_prefix(&self, prefix: &str) -> usize {
@@ -1665,6 +2172,53 @@ impl ExtensionManager {
}
}
/// Inject credentials for a channel based on naming convention.
///
/// Looks for secrets matching the pattern `{channel_name}_*` and injects them
/// as credential placeholders (e.g., `telegram_bot_token` -> `{TELEGRAM_BOT_TOKEN}`).
///
/// Returns the number of credentials injected.
async fn inject_channel_credentials_from_secrets(
channel: &Arc<crate::channels::wasm::WasmChannel>,
secrets: &dyn SecretsStore,
channel_name: &str,
user_id: &str,
) -> Result<usize, String> {
let all_secrets = secrets
.list(user_id)
.await
.map_err(|e| format!("Failed to list secrets: {}", e))?;
let prefix = format!("{}_", channel_name);
let mut count = 0;
for secret_meta in all_secrets {
if !secret_meta.name.starts_with(&prefix) {
continue;
}
let decrypted = match secrets.get_decrypted(user_id, &secret_meta.name).await {
Ok(d) => d,
Err(e) => {
tracing::warn!(
secret = %secret_meta.name,
error = %e,
"Failed to decrypt secret for channel credential injection"
);
continue;
}
};
let placeholder = secret_meta.name.to_uppercase();
channel
.set_credential(&placeholder, decrypted.expose().to_string())
.await;
count += 1;
}
Ok(count)
}
/// Infer the extension kind from a URL.
fn infer_kind_from_url(url: &str) -> ExtensionKind {
if url.ends_with(".wasm") || url.ends_with(".tar.gz") {
+24 -19
View File
@@ -1,16 +1,19 @@
//! Unified extension system for discovering, installing, authenticating, and activating
//! MCP servers and WASM tools through conversational agent interactions.
//! Lifecycle management for extensions: discovery, installation, authentication,
//! and activation of channels, tools, and MCP servers.
//!
//! Extensions are the user-facing abstraction over MCP servers and WASM tools. The agent
//! can search a built-in registry (or discover online), install, authenticate, and activate
//! extensions at runtime without CLI commands.
//! Extensions are the user-facing abstraction that unifies three runtime kinds:
//! - **Channels** (Telegram, Slack, Discord) — messaging integrations (WASM)
//! - **Tools** — sandboxed capabilities (WASM)
//! - **MCP servers** — external API integrations via Model Context Protocol
//!
//! The agent can search a built-in registry (or discover online), install,
//! authenticate, and activate extensions at runtime without CLI commands.
//!
//! ```text
//! User: "add notion"
//! -> tool_search("notion") -> finds MCP server in registry
//! -> tool_install("notion") -> saves config to mcp-servers.json
//! -> tool_auth("notion") -> OAuth 2.1 flow, returns URL
//! -> tool_activate("notion") -> connects, registers tools
//! User: "add telegram"
//! -> tool_search("telegram") -> finds channel in registry
//! -> tool_install("telegram") -> copies bundled WASM to channels dir
//! -> tool_activate("telegram") -> configures credentials, starts channel
//! ```
pub mod discovery;
@@ -31,7 +34,7 @@ pub enum ExtensionKind {
McpServer,
/// Sandboxed WASM module, file-based, capabilities auth.
WasmTool,
/// WASM channel module (future: dynamic activation, currently needs restart).
/// WASM channel module with hot-activation support.
WasmChannel,
}
@@ -82,14 +85,12 @@ pub enum ExtensionSource {
repo_url: String,
#[serde(default)]
build_dir: Option<String>,
/// Crate name used to locate the build artifact binary.
#[serde(default)]
crate_name: Option<String>,
},
/// Discovered online (not yet validated for a specific source type).
Discovered { url: String },
/// Bundled with the application (pre-built WASM, copied from build artifacts).
Bundled {
/// Channel or tool name used to locate build artifacts.
name: String,
},
}
/// Hint about what authentication method is needed.
@@ -174,6 +175,10 @@ pub struct ActivateResult {
pub message: String,
}
fn default_true() -> bool {
true
}
/// An installed extension with its current status.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct InstalledExtension {
@@ -192,6 +197,9 @@ pub struct InstalledExtension {
/// Whether this extension has a setup schema (required_secrets) that can be configured.
#[serde(default)]
pub needs_setup: bool,
/// Whether this extension is installed locally (false = available in registry but not installed).
#[serde(default = "default_true")]
pub installed: bool,
}
/// Error type for extension operations.
@@ -227,9 +235,6 @@ pub enum ExtensionError {
#[error("Config error: {0}")]
Config(String),
#[error("Channels require restart to activate")]
ChannelNeedsRestart,
#[error("{0}")]
Other(String),
}
+24 -110
View File
@@ -1,7 +1,7 @@
//! Curated in-memory catalog of known extensions with fuzzy search.
//!
//! The registry holds well-known MCP servers and WASM tools that can be installed
//! via conversational commands. Online discoveries are cached here too.
//! The registry holds well-known channels, tools, and MCP servers that can be
//! installed via conversational commands. Online discoveries are cached here too.
use tokio::sync::RwLock;
@@ -116,6 +116,21 @@ impl ExtensionRegistry {
cache.iter().find(|e| e.name == name).cloned()
}
/// Return all registry entries (builtins + cached discoveries).
pub async fn all_entries(&self) -> Vec<RegistryEntry> {
let mut entries = self.entries.clone();
let cache = self.discovery_cache.read().await;
for entry in cache.iter() {
if !entries
.iter()
.any(|e| e.name == entry.name && e.kind == entry.kind)
{
entries.push(entry.clone());
}
}
entries
}
/// Add discovered entries to the cache.
pub async fn cache_discovered(&self, entries: Vec<RegistryEntry>) {
let mut cache = self.discovery_cache.write().await;
@@ -380,72 +395,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
},
auth_hint: AuthHint::Dcr,
},
// -- WASM Channels (bundled) --
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Telegram Bot API channel for receiving and sending messages via Telegram"
.to_string(),
keywords: vec![
"chat".into(),
"messaging".into(),
"bot".into(),
"channel".into(),
],
source: ExtensionSource::Bundled {
name: "telegram".to_string(),
},
auth_hint: AuthHint::CapabilitiesAuth,
},
RegistryEntry {
name: "slack".to_string(),
display_name: "Slack".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Slack Events API channel for receiving and sending messages via Slack"
.to_string(),
keywords: vec![
"chat".into(),
"messaging".into(),
"team".into(),
"channel".into(),
],
source: ExtensionSource::Bundled {
name: "slack".to_string(),
},
auth_hint: AuthHint::CapabilitiesAuth,
},
RegistryEntry {
name: "discord".to_string(),
display_name: "Discord".to_string(),
kind: ExtensionKind::WasmChannel,
description:
"Discord Gateway channel for handling slash commands, buttons, and messages"
.to_string(),
keywords: vec![
"chat".into(),
"messaging".into(),
"gaming".into(),
"channel".into(),
],
source: ExtensionSource::Bundled {
name: "discord".to_string(),
},
auth_hint: AuthHint::CapabilitiesAuth,
},
RegistryEntry {
name: "whatsapp".to_string(),
display_name: "WhatsApp".to_string(),
kind: ExtensionKind::WasmChannel,
description:
"WhatsApp Business API channel for receiving and sending WhatsApp messages"
.to_string(),
keywords: vec!["chat".into(), "messaging".into(), "channel".into()],
source: ExtensionSource::Bundled {
name: "whatsapp".to_string(),
},
auth_hint: AuthHint::CapabilitiesAuth,
},
// WASM channels (telegram, slack, discord, whatsapp) come from the embedded
// registry catalog (registry/channels/*.json) with WasmDownload URLs pointing
// to GitHub release artifacts. See new_with_catalog() for merging.
]
}
@@ -641,6 +593,7 @@ mod tests {
source: ExtensionSource::WasmBuildable {
repo_url: "channels-src/telegram".to_string(),
build_dir: Some("channels-src/telegram".to_string()),
crate_name: Some("telegram-channel".to_string()),
},
auth_hint: AuthHint::CapabilitiesAuth,
},
@@ -654,6 +607,7 @@ mod tests {
source: ExtensionSource::WasmBuildable {
repo_url: "tools-src/slack".to_string(),
build_dir: Some("tools-src/slack".to_string()),
crate_name: Some("slack-tool".to_string()),
},
auth_hint: AuthHint::CapabilitiesAuth,
},
@@ -701,46 +655,6 @@ mod tests {
assert_eq!(entry.unwrap().display_name, "Slack MCP");
}
#[tokio::test]
async fn test_search_finds_telegram_channel() {
let registry = ExtensionRegistry::new();
let results = registry.search("telegram").await;
assert!(!results.is_empty(), "Should find telegram in registry");
assert_eq!(results[0].entry.name, "telegram");
assert_eq!(results[0].entry.kind, ExtensionKind::WasmChannel);
}
#[tokio::test]
async fn test_search_channel_by_keyword() {
let registry = ExtensionRegistry::new();
let results = registry.search("bot messaging").await;
let has_telegram = results.iter().any(|r| r.entry.name == "telegram");
assert!(
has_telegram,
"Telegram should appear in bot messaging search"
);
}
#[tokio::test]
async fn test_get_bundled_channels() {
let registry = ExtensionRegistry::new();
let telegram = registry.get("telegram").await;
assert!(telegram.is_some());
assert_eq!(telegram.unwrap().kind, ExtensionKind::WasmChannel);
let slack = registry.get("slack").await;
assert!(slack.is_some());
assert_eq!(slack.unwrap().kind, ExtensionKind::WasmChannel);
let discord = registry.get("discord").await;
assert!(discord.is_some());
assert_eq!(discord.unwrap().kind, ExtensionKind::WasmChannel);
let whatsapp = registry.get("whatsapp").await;
assert!(whatsapp.is_some());
assert_eq!(whatsapp.unwrap().kind, ExtensionKind::WasmChannel);
}
// Channel tests (telegram, slack, discord, whatsapp) require the embedded catalog
// to be loaded via new_with_catalog(). See test_new_with_catalog for catalog coverage.
}
+15
View File
@@ -1167,6 +1167,21 @@ impl Store {
.await?;
Ok(row.get("cnt"))
}
/// Link a routine run to a dispatched job.
pub async fn link_routine_run_to_job(
&self,
run_id: Uuid,
job_id: Uuid,
) -> Result<(), DatabaseError> {
let conn = self.conn().await?;
conn.execute(
"UPDATE routine_runs SET job_id = $1 WHERE id = $2",
&[&job_id, &run_id],
)
.await?;
Ok(())
}
}
#[cfg(feature = "postgres")]
-1
View File
@@ -67,7 +67,6 @@ pub mod setup;
pub mod skills;
pub mod tools;
pub mod tracing_fmt;
pub mod transcription;
pub mod tunnel;
pub mod util;
pub mod worker;
+279 -8
View File
@@ -6,12 +6,13 @@
//! - **Session token auth**: Otherwise, uses `SessionManager` for Bearer session token
//! with automatic renewal on 401 errors
use std::collections::HashMap;
use std::sync::Arc;
use async_trait::async_trait;
use reqwest::Client;
use rust_decimal::Decimal;
use rust_decimal_macros::dec;
use rust_decimal::prelude::MathematicalOps;
use secrecy::ExposeSecret;
use serde::{Deserialize, Serialize};
@@ -21,7 +22,7 @@ use crate::llm::provider::{
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, Role, ToolCall,
ToolCompletionRequest, ToolCompletionResponse,
};
use crate::llm::session::SessionManager;
use crate::llm::{costs, session::SessionManager};
/// Information about an available model from NEAR AI API.
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -42,6 +43,9 @@ pub struct NearAiChatProvider {
session: Arc<SessionManager>,
active_model: std::sync::RwLock<String>,
flatten_tool_messages: bool,
/// Per-model pricing fetched from the NEAR AI `/v1/model/list` endpoint.
/// Maps model ID → (input_cost_per_token, output_cost_per_token).
pricing: Arc<std::sync::RwLock<HashMap<String, (Decimal, Decimal)>>>,
}
impl NearAiChatProvider {
@@ -72,13 +76,49 @@ impl NearAiChatProvider {
})?;
let active_model = std::sync::RwLock::new(config.model.clone());
Ok(Self {
let pricing = Arc::new(std::sync::RwLock::new(HashMap::new()));
let provider = Self {
client,
config,
session,
active_model,
flatten_tool_messages,
})
pricing,
};
// Fire-and-forget background pricing fetch — don't block startup.
// Only spawns when a tokio runtime is active (skipped in sync tests).
if let Ok(handle) = tokio::runtime::Handle::try_current() {
let client = provider.client.clone();
let base_url = provider.config.base_url.clone();
let api_key = provider.config.api_key.clone();
let session = provider.session.clone();
let pricing = provider.pricing.clone();
handle.spawn(async move {
match fetch_pricing(&client, &base_url, api_key.as_ref(), &session).await {
Ok(map) if !map.is_empty() => {
tracing::info!("Loaded NEAR AI pricing for {} model(s)", map.len());
match pricing.write() {
Ok(mut guard) => *guard = map,
Err(poisoned) => *poisoned.into_inner() = map,
}
}
Ok(_) => {
tracing::debug!("NEAR AI pricing endpoint returned no pricing data");
}
Err(e) => {
tracing::debug!(
"Could not fetch NEAR AI pricing (will use fallback): {}",
e
);
}
}
});
}
Ok(provider)
}
fn api_url(&self, path: &str) -> String {
@@ -382,7 +422,13 @@ impl LlmProvider for NearAiChatProvider {
reason: "No choices in response".to_string(),
})?;
let content = choice.message.content.unwrap_or_default();
// Fall back to reasoning_content when content is null (same as
// complete_with_tools — reasoning models may put the answer there).
let content = choice
.message
.content
.or(choice.message.reasoning_content)
.unwrap_or_default();
let finish_reason = match choice.finish_reason.as_deref() {
Some("stop") => FinishReason::Stop,
Some("length") => FinishReason::Length,
@@ -453,7 +499,9 @@ impl LlmProvider for NearAiChatProvider {
reason: "No choices in response".to_string(),
})?;
let content = choice.message.content;
// Fall back to reasoning_content when content is null (e.g. GLM-5
// returns its answer in reasoning_content instead of content).
let content = choice.message.content.or(choice.message.reasoning_content);
let tool_calls: Vec<ToolCall> = choice
.message
.tool_calls
@@ -500,8 +548,14 @@ impl LlmProvider for NearAiChatProvider {
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
// Default costs - could be model-specific in the future
(dec!(0.000003), dec!(0.000015))
let model = self.active_model_name();
// Try fetched pricing first, then static lookup table, then default
if let Ok(guard) = self.pricing.read()
&& let Some(&rates) = guard.get(&model)
{
return rates;
}
costs::model_cost(&model).unwrap_or_else(costs::default_cost)
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
@@ -562,6 +616,143 @@ struct ChatCompletionMessage {
tool_calls: Option<Vec<ChatCompletionToolCall>>,
}
// -- Pricing fetch types and logic -----------------------------------------
/// Cost amount from the NEAR AI `/v1/model/list` response.
///
/// Real cost per token = `amount * 10^(-scale)`.
#[derive(Debug, Deserialize)]
struct ModelCost {
amount: f64,
#[serde(default)]
scale: i32,
}
/// A single model entry from the pricing response.
#[derive(Debug, Deserialize)]
struct PricingModelEntry {
#[serde(default, alias = "modelId", alias = "model_id")]
model_id: Option<String>,
#[serde(default, alias = "inputCostPerToken")]
input_cost_per_token: Option<ModelCost>,
#[serde(default, alias = "outputCostPerToken")]
output_cost_per_token: Option<ModelCost>,
#[serde(default)]
metadata: Option<PricingMetadata>,
}
#[derive(Debug, Deserialize)]
struct PricingMetadata {
#[serde(default)]
aliases: Vec<String>,
}
/// Wrapper for the `/v1/model/list` response body.
#[derive(Debug, Deserialize)]
struct PricingResponse {
#[serde(default)]
models: Option<Vec<PricingModelEntry>>,
#[serde(default)]
data: Option<Vec<PricingModelEntry>>,
}
/// Convert a `ModelCost` to a `Decimal` per-token price.
fn model_cost_to_decimal(mc: &ModelCost) -> Option<Decimal> {
if mc.amount == 0.0 {
return Some(Decimal::ZERO);
}
// amount * 10^(-scale)
let base = Decimal::try_from(mc.amount).ok()?;
let factor = Decimal::TEN.checked_powi(-i64::from(mc.scale))?;
base.checked_mul(factor)
}
/// Fetch pricing from the NEAR AI `/v1/model/list` endpoint.
///
/// Returns a map of model_id → (input_cost_per_token, output_cost_per_token).
/// Errors are non-fatal; callers should fall back to the static lookup table.
async fn fetch_pricing(
client: &Client,
base_url: &str,
api_key: Option<&secrecy::SecretString>,
session: &SessionManager,
) -> Result<HashMap<String, (Decimal, Decimal)>, LlmError> {
let base = base_url.trim_end_matches('/');
let url = if base.ends_with("/v1") {
format!("{}/model/list", base)
} else {
format!("{}/v1/model/list", base)
};
let token = if let Some(key) = api_key {
key.expose_secret().to_string()
} else {
let tok = session.get_token().await?;
tok.expose_secret().to_string()
};
let response = client
.get(&url)
.header("Authorization", format!("Bearer {}", token))
.timeout(std::time::Duration::from_secs(15))
.send()
.await
.map_err(|e| LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: format!("Failed to fetch pricing: {}", e),
})?;
if !response.status().is_success() {
return Err(LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: format!("Pricing endpoint returned HTTP {}", response.status()),
});
}
let body = response.text().await.map_err(|e| LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: format!("Failed to read pricing response: {}", e),
})?;
// Parse as {models: [...]} or {data: [...]} or direct array
let entries: Vec<PricingModelEntry> =
if let Ok(resp) = serde_json::from_str::<PricingResponse>(&body) {
resp.models.or(resp.data).unwrap_or_default()
} else if let Ok(arr) = serde_json::from_str::<Vec<PricingModelEntry>>(&body) {
arr
} else {
return Ok(HashMap::new());
};
let mut map = HashMap::new();
for entry in &entries {
let (Some(input_mc), Some(output_mc)) =
(&entry.input_cost_per_token, &entry.output_cost_per_token)
else {
continue;
};
let (Some(input), Some(output)) = (
model_cost_to_decimal(input_mc),
model_cost_to_decimal(output_mc),
) else {
continue;
};
// Insert under the primary model_id
if let Some(ref id) = entry.model_id {
map.insert(id.clone(), (input, output));
}
// Also insert under any aliases
if let Some(ref meta) = entry.metadata {
for alias in &meta.aliases {
map.insert(alias.clone(), (input, output));
}
}
}
Ok(map)
}
/// Rewrite tool-call / tool-result messages into plain assistant/user text.
///
/// NEAR AI cloud-api does not support the OpenAI multi-turn tool-calling
@@ -598,6 +789,7 @@ fn flatten_tool_messages(messages: Vec<ChatCompletionMessage>) -> Vec<ChatComple
ChatCompletionMessage {
role: "assistant".to_string(),
content: Some(parts.join("\n")),
tool_call_id: None,
name: None,
tool_calls: None,
@@ -609,6 +801,7 @@ fn flatten_tool_messages(messages: Vec<ChatCompletionMessage>) -> Vec<ChatComple
ChatCompletionMessage {
role: "user".to_string(),
content: Some(format!("[Tool `{}` returned: {}]", tool_name, result)),
tool_call_id: None,
name: None,
tool_calls: None,
@@ -696,6 +889,10 @@ struct ChatCompletionResponseMessage {
#[allow(dead_code)]
role: String,
content: Option<String>,
/// Some models (e.g. GLM-5) return chain-of-thought reasoning here
/// instead of in `content`.
#[serde(default)]
reasoning_content: Option<String>,
tool_calls: Option<Vec<ChatCompletionToolCall>>,
}
@@ -748,6 +945,7 @@ fn parse_usage(usage: Option<&ChatCompletionUsage>) -> (u32, u32) {
mod tests {
use super::*;
use crate::llm::session::SessionConfig;
use rust_decimal_macros::dec;
fn test_nearai_config(base_url: &str) -> NearAiConfig {
NearAiConfig {
@@ -991,4 +1189,77 @@ mod tests {
assert!(text.starts_with("Let me check that."));
assert!(text.contains("[Called tool `search`"));
}
#[test]
fn test_model_cost_to_decimal_basic() {
// amount=3, scale=6 → 3 * 10^-6 = 0.000003
let mc = ModelCost {
amount: 3.0,
scale: 6,
};
let result = model_cost_to_decimal(&mc).unwrap();
assert_eq!(result, dec!(0.000003));
}
#[test]
fn test_model_cost_to_decimal_zero() {
let mc = ModelCost {
amount: 0.0,
scale: 6,
};
assert_eq!(model_cost_to_decimal(&mc), Some(Decimal::ZERO));
}
#[test]
fn test_model_cost_to_decimal_larger_scale() {
// amount=85, scale=8 → 85 * 10^-8 = 0.00000085
let mc = ModelCost {
amount: 85.0,
scale: 8,
};
let result = model_cost_to_decimal(&mc).unwrap();
assert_eq!(result, dec!(0.00000085));
}
#[test]
fn test_cost_per_token_uses_pricing_map() {
let cfg = test_nearai_config("http://127.0.0.1:8318");
let provider = NearAiChatProvider::new(cfg, test_session()).expect("provider");
// Inject pricing directly
{
let mut guard = provider.pricing.write().unwrap();
guard.insert("test-model".to_string(), (dec!(0.000001), dec!(0.000005)));
}
let (input, output) = provider.cost_per_token();
assert_eq!(input, dec!(0.000001));
assert_eq!(output, dec!(0.000005));
}
#[test]
fn test_cost_per_token_falls_back_to_static() {
let mut cfg = test_nearai_config("http://127.0.0.1:8318");
cfg.model = "gpt-4o".to_string();
let provider = NearAiChatProvider::new(cfg, test_session()).expect("provider");
// No pricing in map, should fall back to static costs::model_cost
let (input, output) = provider.cost_per_token();
let (expected_in, expected_out) = costs::model_cost("gpt-4o").unwrap();
assert_eq!(input, expected_in);
assert_eq!(output, expected_out);
}
#[test]
fn test_cost_per_token_falls_back_to_default() {
let mut cfg = test_nearai_config("http://127.0.0.1:8318");
cfg.model = "some-unknown-nearai-model".to_string();
let provider = NearAiChatProvider::new(cfg, test_session()).expect("provider");
// No pricing in map, not in static table, should use default_cost
let (input, output) = provider.cost_per_token();
let (default_in, default_out) = costs::default_cost();
assert_eq!(input, default_in);
assert_eq!(output, default_out);
}
}
+54 -4
View File
@@ -502,8 +502,23 @@ Respond in JSON format:
});
}
// Guard against empty text after cleaning. This can happen
// when reasoning models (e.g. GLM-5) return chain-of-thought
// in reasoning_content wrapped in <think> tags and content is
// null — the .or(reasoning_content) fallback picks it up, then
// clean_response strips the think tags leaving an empty string.
let cleaned = clean_response(&content);
let final_text = if cleaned.trim().is_empty() {
tracing::warn!(
"LLM response was empty after cleaning (original len={}), using fallback",
content.len()
);
"I'm not sure how to respond to that.".to_string()
} else {
cleaned
};
Ok(RespondOutput {
result: RespondResult::Text(clean_response(&content)),
result: RespondResult::Text(final_text),
usage,
})
} else {
@@ -514,8 +529,18 @@ Respond in JSON format:
request.metadata = context.metadata.clone();
let response = self.llm.complete(request).await?;
let cleaned = clean_response(&response.content);
let final_text = if cleaned.trim().is_empty() {
tracing::warn!(
"LLM response was empty after cleaning (original len={}), using fallback",
response.content.len()
);
"I'm not sure how to respond to that.".to_string()
} else {
cleaned
};
Ok(RespondOutput {
result: RespondResult::Text(clean_response(&response.content)),
result: RespondResult::Text(final_text),
usage: TokenUsage {
input_tokens: response.input_tokens,
output_tokens: response.output_tokens,
@@ -607,6 +632,9 @@ Respond with a JSON plan in this format:
// Channel-specific formatting hints
let channel_section = self.build_channel_section();
// Extension guidance (only when extension tools are available)
let extensions_section = self.build_extensions_section(context);
// Runtime context (agent metadata)
let runtime_section = self.build_runtime_section();
@@ -614,7 +642,7 @@ Respond with a JSON plan in this format:
let group_section = self.build_group_section();
format!(
r#"You are NEAR AI Agent, an autonomous assistant.
r#"You are IronClaw Agent, a secure autonomous assistant.
## Response Format CRITICAL
@@ -648,9 +676,10 @@ Example:
- Prioritize safety and human oversight over task completion. If instructions conflict, pause and ask.
- Comply with stop, pause, or audit requests. Never bypass safeguards.
- Do not manipulate anyone to expand your access or disable safeguards.
- Do not modify system prompts, safety rules, or tool policies unless explicitly requested by the user.{}{}{}{}
- Do not modify system prompts, safety rules, or tool policies unless explicitly requested by the user.{}{}{}{}{}
{}{}"#,
tools_section,
extensions_section,
channel_section,
runtime_section,
group_section,
@@ -659,6 +688,27 @@ Example:
)
}
fn build_extensions_section(&self, context: &ReasoningContext) -> String {
// Only include when the extension management tools are available
let has_ext_tools = context
.available_tools
.iter()
.any(|t| t.name == "tool_search");
if !has_ext_tools {
return String::new();
}
"\n\n## Extensions\n\
You can search, install, and activate extensions to add new capabilities:\n\
- **Channels** (Telegram, Slack, Discord) messaging integrations. \
When users ask about connecting a messaging platform, search for it as a channel.\n\
- **Tools** sandboxed functions that extend your abilities.\n\
- **MCP servers** external API integrations via the Model Context Protocol.\n\n\
Use `tool_search` to find extensions by name. Refer to them by their kind \
(channel, tool, or server) not as \"MCP server\" generically."
.to_string()
}
fn build_channel_section(&self) -> String {
let channel = match self.channel.as_deref() {
Some(c) => c,
+508 -1128
View File
File diff suppressed because it is too large Load Diff
+377
View File
@@ -0,0 +1,377 @@
//! Unified WASM artifact resolution: find, build, and install WASM components.
//!
//! This module consolidates all WASM artifact logic that was previously duplicated
//! across `cli/tool.rs`, `registry/installer.rs`, `extensions/manager.rs`,
//! `channels/wasm/bundled.rs`, and `tools/wasm/loader.rs`.
//!
//! # Functions
//!
//! - [`resolve_target_dir`] — resolve the cargo target directory for a crate
//! - [`find_wasm_artifact`] — find a compiled `.wasm` by crate name across all triples
//! - [`find_any_wasm_artifact`] — find any `.wasm` file (fallback when name is unknown)
//! - [`build_wasm_component`] — async build via `cargo component build`
//! - [`build_wasm_component_sync`] — sync build for CLI use
//! - [`install_wasm_files`] — copy `.wasm` + optional `.capabilities.json` to install dir
use std::path::{Path, PathBuf};
use tokio::fs;
/// WASM target triples to search, in priority order.
const WASM_TRIPLES: &[&str] = &[
"wasm32-wasip1",
"wasm32-wasip2",
"wasm32-wasi",
"wasm32-unknown-unknown",
];
/// Resolve the cargo target directory for a crate.
///
/// Checks (in order):
/// 1. `CARGO_TARGET_DIR` env var (shared target dir)
/// 2. `<crate_dir>/target/` (default per-crate layout)
pub fn resolve_target_dir(crate_dir: &Path) -> PathBuf {
if let Ok(dir) = std::env::var("CARGO_TARGET_DIR") {
let p = PathBuf::from(dir);
// Resolve relative CARGO_TARGET_DIR against crate_dir
if p.is_relative() {
return crate_dir.join(p);
}
return p;
}
crate_dir.join("target")
}
/// Find a compiled WASM artifact by searching across all target triples.
///
/// Tries exact name match first (with hyphen-to-underscore normalization),
/// then falls back to searching in whichever target directory exists.
/// `profile` is `"release"` or `"debug"`.
pub fn find_wasm_artifact(crate_dir: &Path, crate_name: &str, profile: &str) -> Option<PathBuf> {
let target_base = resolve_target_dir(crate_dir);
let snake_name = crate_name.replace('-', "_");
// Try exact name match in each target triple directory
for triple in WASM_TRIPLES {
let dir = target_base.join(triple).join(profile);
let candidates = [
dir.join(format!("{}.wasm", crate_name)),
dir.join(format!("{}.wasm", snake_name)),
];
for candidate in &candidates {
if candidate.exists() {
return Some(candidate.clone());
}
}
}
None
}
/// Find any `.wasm` file in the target dirs (fallback when crate name is unknown).
///
/// Returns the first `.wasm` found across target triples.
pub fn find_any_wasm_artifact(crate_dir: &Path, profile: &str) -> Option<PathBuf> {
let target_base = resolve_target_dir(crate_dir);
for triple in WASM_TRIPLES {
let dir = target_base.join(triple).join(profile);
if !dir.is_dir() {
continue;
}
if let Ok(entries) = std::fs::read_dir(&dir) {
for entry in entries.flatten() {
let path = entry.path();
if path.extension().map(|ext| ext == "wasm").unwrap_or(false) {
return Some(path);
}
}
}
}
None
}
/// Build a WASM component using `cargo-component` (async).
///
/// Streams build output to the terminal. Returns the path to the built artifact.
pub async fn build_wasm_component(
source_dir: &Path,
crate_name: &str,
release: bool,
) -> anyhow::Result<PathBuf> {
use tokio::process::Command;
// Check cargo-component availability
let check = Command::new("cargo")
.args(["component", "--version"])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.await;
if check.is_err() || !check.as_ref().map(|s| s.success()).unwrap_or(false) {
anyhow::bail!("cargo-component not found. Install with: cargo install cargo-component");
}
let mut cmd = Command::new("cargo");
cmd.current_dir(source_dir).args(["component", "build"]);
if release {
cmd.arg("--release");
}
// Use status() with inherited stdio so build output streams to the terminal.
let status = cmd.status().await?;
if !status.success() {
anyhow::bail!("Build failed (exit code: {})", status);
}
let profile = if release { "release" } else { "debug" };
let wasm_filename = format!("{}.wasm", crate_name.replace('-', "_"));
// Look for the specific crate's WASM file across target triples
find_wasm_artifact(source_dir, wasm_filename.trim_end_matches(".wasm"), profile)
.or_else(|| {
// Fall back: search by crate_name directly
find_wasm_artifact(source_dir, crate_name, profile)
})
.or_else(|| find_any_wasm_artifact(source_dir, profile))
.ok_or_else(|| {
anyhow::anyhow!(
"Could not find {} in {}/target/*/{}/ after build",
wasm_filename,
source_dir.display(),
profile,
)
})
}
/// Build a WASM component using `cargo-component` (sync, for CLI use).
///
/// Returns the path to the built artifact.
pub fn build_wasm_component_sync(source_dir: &Path, release: bool) -> anyhow::Result<PathBuf> {
use std::process::Command;
println!("Building WASM component in {}...", source_dir.display());
// Check if cargo-component is available
let check = Command::new("cargo")
.args(["component", "--version"])
.output();
if check.is_err() || !check.as_ref().map(|o| o.status.success()).unwrap_or(false) {
anyhow::bail!(
"cargo-component not found. Install with: cargo install cargo-component\n\
Or use --skip-build with an existing .wasm file."
);
}
let mut cmd = Command::new("cargo");
cmd.current_dir(source_dir).args(["component", "build"]);
if release {
cmd.arg("--release");
}
println!(
" Running: cargo component build{}",
if release { " --release" } else { "" }
);
let output = cmd.output()?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("Build failed:\n{}", stderr);
}
let profile = if release { "release" } else { "debug" };
// Find the built artifact
find_any_wasm_artifact(source_dir, profile).ok_or_else(|| {
anyhow::anyhow!(
"No .wasm file found after build in {}/target/*/{}",
source_dir.display(),
profile,
)
})
}
/// Copy WASM binary + optional `capabilities.json` sidecar to an install directory.
///
/// Looks for capabilities files in `source_dir` matching several naming conventions.
/// Returns the destination wasm path.
pub async fn install_wasm_files(
wasm_src: &Path,
source_dir: &Path,
name: &str,
target_dir: &Path,
force: bool,
) -> anyhow::Result<PathBuf> {
fs::create_dir_all(target_dir).await?;
let wasm_dst = target_dir.join(format!("{}.wasm", name));
let caps_dst = target_dir.join(format!("{}.capabilities.json", name));
if wasm_dst.exists() && !force {
anyhow::bail!(
"Tool '{}' already exists at {}. Use --force to overwrite.",
name,
wasm_dst.display()
);
}
// Copy WASM binary
fs::copy(wasm_src, &wasm_dst).await?;
// Look for capabilities.json sidecar in the source directory
let caps_candidates = [
source_dir.join(format!("{}.capabilities.json", name)),
source_dir.join(format!("{}-tool.capabilities.json", name)),
source_dir.join("capabilities.json"),
];
for caps_src in &caps_candidates {
if caps_src.exists() {
if let Err(e) = fs::copy(caps_src, &caps_dst).await {
tracing::warn!(
"Failed to copy capabilities sidecar {} -> {}: {}",
caps_src.display(),
caps_dst.display(),
e,
);
}
break;
}
}
Ok(wasm_dst)
}
#[cfg(test)]
mod tests {
use tempfile::TempDir;
use super::*;
#[test]
fn test_resolve_target_dir_default() {
// When CARGO_TARGET_DIR is not set, should return <crate_dir>/target
let dir = Path::new("/some/crate");
let result = resolve_target_dir(dir);
assert!(result.ends_with("target"));
}
#[test]
fn test_find_wasm_artifact_not_found() {
let dir = TempDir::new().unwrap();
assert!(find_wasm_artifact(dir.path(), "nonexistent", "release").is_none());
}
#[test]
fn test_find_wasm_artifact_found() {
let dir = TempDir::new().unwrap();
let target_base = resolve_target_dir(dir.path());
let wasm_dir = target_base.join("wasm32-wasip2/release");
std::fs::create_dir_all(&wasm_dir).unwrap();
std::fs::File::create(wasm_dir.join("my_tool.wasm")).unwrap();
let result = find_wasm_artifact(dir.path(), "my_tool", "release");
assert!(result.is_some());
assert!(result.unwrap().ends_with("my_tool.wasm"));
}
#[test]
fn test_find_wasm_artifact_hyphen_to_underscore() {
let dir = TempDir::new().unwrap();
let target_base = resolve_target_dir(dir.path());
let wasm_dir = target_base.join("wasm32-wasip1/release");
std::fs::create_dir_all(&wasm_dir).unwrap();
std::fs::File::create(wasm_dir.join("my_tool.wasm")).unwrap();
// Search with hyphens, should find underscore version
let result = find_wasm_artifact(dir.path(), "my-tool", "release");
assert!(result.is_some());
}
#[test]
fn test_find_any_wasm_artifact_found() {
let dir = TempDir::new().unwrap();
let target_base = resolve_target_dir(dir.path());
let wasm_dir = target_base.join("wasm32-wasip2/release");
std::fs::create_dir_all(&wasm_dir).unwrap();
std::fs::File::create(wasm_dir.join("something.wasm")).unwrap();
let result = find_any_wasm_artifact(dir.path(), "release");
assert!(result.is_some());
}
#[test]
fn test_find_any_wasm_artifact_not_found() {
let dir = TempDir::new().unwrap();
assert!(find_any_wasm_artifact(dir.path(), "release").is_none());
}
#[tokio::test]
async fn test_install_wasm_files_copies() {
let src_dir = TempDir::new().unwrap();
let target_dir = TempDir::new().unwrap();
let wasm_src = src_dir.path().join("test.wasm");
tokio::fs::write(&wasm_src, b"\0asm\x01\x00\x00\x00")
.await
.unwrap();
// Create a capabilities file
let caps_src = src_dir.path().join("mytool.capabilities.json");
tokio::fs::write(&caps_src, b"{}").await.unwrap();
let result = install_wasm_files(
&wasm_src,
src_dir.path(),
"mytool",
target_dir.path(),
false,
)
.await;
assert!(result.is_ok());
let wasm_dst = result.unwrap();
assert!(wasm_dst.exists());
assert!(target_dir.path().join("mytool.capabilities.json").exists());
}
#[tokio::test]
async fn test_install_wasm_files_refuses_overwrite() {
let src_dir = TempDir::new().unwrap();
let target_dir = TempDir::new().unwrap();
let wasm_src = src_dir.path().join("test.wasm");
tokio::fs::write(&wasm_src, b"\0asm").await.unwrap();
// Pre-create the target
let existing = target_dir.path().join("mytool.wasm");
tokio::fs::write(&existing, b"existing").await.unwrap();
let result = install_wasm_files(
&wasm_src,
src_dir.path(),
"mytool",
target_dir.path(),
false,
)
.await;
assert!(result.is_err());
}
#[test]
fn test_wasm_triples_order() {
// Verify the order is as documented
assert_eq!(WASM_TRIPLES[0], "wasm32-wasip1");
assert_eq!(WASM_TRIPLES[1], "wasm32-wasip2");
assert_eq!(WASM_TRIPLES[2], "wasm32-wasi");
assert_eq!(WASM_TRIPLES[3], "wasm32-unknown-unknown");
}
}
+7 -64
View File
@@ -94,12 +94,13 @@ impl RegistryInstaller {
source_dir.display()
);
let crate_name = &manifest.source.crate_name;
let wasm_path = build_wasm_component(&source_dir, crate_name)
.await
.map_err(|e| RegistryError::ManifestRead {
path: source_dir.clone(),
reason: format!("build failed: {}", e),
})?;
let wasm_path =
crate::registry::artifacts::build_wasm_component(&source_dir, crate_name, true)
.await
.map_err(|e| RegistryError::ManifestRead {
path: source_dir.clone(),
reason: format!("build failed: {}", e),
})?;
// Copy WASM binary
println!(" Installing to {}", target_wasm.display());
@@ -353,64 +354,6 @@ impl RegistryInstaller {
}
}
/// Build a WASM component from a source directory using `cargo component build --release`.
///
/// Uses `tokio::process::Command` with inherited stdio so build progress is visible.
/// Looks for the specific `{crate_name}.wasm` in the release directory rather than
/// picking the first `.wasm` file found.
async fn build_wasm_component(source_dir: &Path, crate_name: &str) -> anyhow::Result<PathBuf> {
use tokio::process::Command;
// Check cargo-component availability
let check = Command::new("cargo")
.args(["component", "--version"])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.await;
if check.is_err() || !check.as_ref().map(|s| s.success()).unwrap_or(false) {
anyhow::bail!("cargo-component not found. Install with: cargo install cargo-component");
}
// Use status() with inherited stdio so build output streams to the terminal.
let status = Command::new("cargo")
.current_dir(source_dir)
.args(["component", "build", "--release"])
.status()
.await?;
if !status.success() {
anyhow::bail!("Build failed (exit code: {})", status);
}
// Look for the specific crate's WASM file (Cargo uses underscores in artifact names).
let wasm_filename = format!("{}.wasm", crate_name.replace('-', "_"));
let target_base = source_dir.join("target");
let candidates = [
"wasm32-wasip1",
"wasm32-wasip2",
"wasm32-wasi",
"wasm32-unknown-unknown",
];
for target in &candidates {
let wasm_path = target_base
.join(target)
.join("release")
.join(&wasm_filename);
if wasm_path.exists() {
return Ok(wasm_path);
}
}
anyhow::bail!(
"Could not find {} in {}/target/*/release/",
wasm_filename,
source_dir.display()
)
}
/// Download an artifact from a URL.
async fn download_artifact(url: &str) -> Result<bytes::Bytes, RegistryError> {
let response = reqwest::get(url)
+2
View File
@@ -165,12 +165,14 @@ impl ExtensionManifest {
ExtensionSource::WasmBuildable {
repo_url: self.source.dir.clone(),
build_dir: Some(self.source.dir.clone()),
crate_name: Some(self.source.crate_name.clone()),
}
}
} else {
ExtensionSource::WasmBuildable {
repo_url: self.source.dir.clone(),
build_dir: Some(self.source.dir.clone()),
crate_name: Some(self.source.crate_name.clone()),
}
};
+1
View File
@@ -11,6 +11,7 @@
//! └── _bundles.json <- Bundle definitions (google, messaging, default)
//! ```
pub mod artifacts;
pub mod catalog;
pub mod embedded;
pub mod installer;
-44
View File
@@ -63,11 +63,6 @@ pub struct Settings {
#[serde(default)]
pub embeddings: EmbeddingsSettings,
// === Transcription (STT) ===
/// Transcription configuration for voice notes.
#[serde(default)]
pub transcription: TranscriptionSettings,
// === Step 6: Channels ===
/// Tunnel configuration for public webhook endpoints.
#[serde(default)]
@@ -151,45 +146,6 @@ impl Default for EmbeddingsSettings {
}
}
/// Transcription (STT) configuration for voice notes.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TranscriptionSettings {
/// Whether transcription is enabled.
#[serde(default)]
pub enabled: bool,
/// Provider to use: "openai".
#[serde(default = "default_transcription_provider")]
pub provider: String,
/// Model to use for transcription.
#[serde(default = "default_transcription_model")]
pub model: String,
/// Optional language hint (ISO-639-1, e.g., "en").
#[serde(default)]
pub language: Option<String>,
}
fn default_transcription_provider() -> String {
"openai".to_string()
}
fn default_transcription_model() -> String {
"whisper-1".to_string()
}
impl Default for TranscriptionSettings {
fn default() -> Self {
Self {
enabled: false,
provider: default_transcription_provider(),
model: default_transcription_model(),
language: None,
}
}
}
/// Tunnel settings for public webhook endpoints.
///
/// The tunnel URL is shared across all channels that need webhooks.
+2 -3
View File
@@ -1084,9 +1084,8 @@ impl SetupWizard {
let fetched = self.fetch_nearai_models().await;
let default_models: Vec<(String, String)> = vec![
(
"fireworks::accounts/fireworks/models/llama4-maverick-instruct-basic"
.into(),
"Llama 4 Maverick (default, fast)".into(),
"zai-org/GLM-latest".into(),
"GLM Latest (default, fast)".into(),
),
(
"anthropic::claude-sonnet-4-20250514".into(),
+4 -4
View File
@@ -798,10 +798,10 @@ Create alongside the .wasm file to grant capabilities:
match (&requirement.software_type, &requirement.language) {
(SoftwareType::WasmTool, Language::Rust) => {
// WASM output location
project_dir.join(format!(
"target/wasm32-wasip2/release/{}.wasm",
requirement.name.replace('-', "_")
))
crate::tools::wasm::wasm_artifact_path(
project_dir,
&requirement.name.replace('-', "_"),
)
}
(SoftwareType::CliBinary, Language::Rust) => project_dir.join(format!(
"target/release/{}",
+18 -6
View File
@@ -30,7 +30,8 @@ impl Tool for ToolSearchTool {
}
fn description(&self) -> &str {
"Search for available extensions (MCP servers, WASM tools, WASM channels) to add. \
"Search for available extensions to add new capabilities. Extensions include \
channels (Telegram, Slack, Discord for messaging), tools, and MCP servers. \
Use discover:true to search online if the built-in registry has no results."
}
@@ -100,7 +101,7 @@ impl Tool for ToolInstallTool {
}
fn description(&self) -> &str {
"Install an extension (MCP server, WASM tool, or WASM channel). \
"Install an extension (channel, tool, or MCP server). \
Use the name from tool_search results, or provide an explicit URL."
}
@@ -278,7 +279,7 @@ impl Tool for ToolActivateTool {
}
fn description(&self) -> &str {
"Activate an installed extension, connecting to MCP servers or loading WASM tools into the runtime."
"Activate an installed extension — starts channels, loads tools, or connects to MCP servers."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -372,7 +373,8 @@ impl Tool for ToolListTool {
}
fn description(&self) -> &str {
"List all installed extensions with their authentication and activation status."
"List extensions with their authentication and activation status. \
Set include_available:true to also show registry entries not yet installed."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -383,6 +385,11 @@ impl Tool for ToolListTool {
"type": "string",
"enum": ["mcp_server", "wasm_tool", "wasm_channel"],
"description": "Filter by extension type (omit to list all)"
},
"include_available": {
"type": "boolean",
"description": "If true, also include registry entries that are not yet installed",
"default": false
}
}
})
@@ -405,9 +412,14 @@ impl Tool for ToolListTool {
_ => None,
});
let include_available = params
.get("include_available")
.and_then(|v| v.as_bool())
.unwrap_or(false);
let extensions = self
.manager
.list(kind_filter)
.list(kind_filter, include_available)
.await
.map_err(|e| ToolError::ExecutionFailed(e.to_string()))?;
@@ -439,7 +451,7 @@ impl Tool for ToolRemoveTool {
}
fn description(&self) -> &str {
"Remove an installed extension (MCP server or WASM tool). \
"Remove an installed extension (channel, tool, or MCP server). \
Unregisters tools and deletes configuration."
}
+7 -5
View File
@@ -231,8 +231,7 @@ impl Tool for HttpTool {
}
},
"body": {
"type": ["object", "array", "string", "number", "boolean", "null"],
"description": "Request body (for POST/PUT/PATCH)"
"description": "Request body (for POST/PUT/PATCH). Can be a JSON object, array, string, or other value."
},
"timeout_secs": {
"type": "integer",
@@ -561,16 +560,19 @@ mod tests {
}
#[test]
fn test_http_tool_schema_body_has_type() {
fn test_http_tool_schema_body_is_freeform() {
let schema = HttpTool::new().parameters_schema();
let body = schema
.get("properties")
.and_then(|p| p.get("body"))
.expect("body schema missing");
// Body is intentionally freeform (no "type" constraint) for OpenAI
// compatibility. OpenAI rejects union types containing "array" unless
// "items" is also specified, and body accepts any JSON value.
assert!(
body.get("type").is_some(),
"body schema must include a type for OpenAI-compatible tool validation"
body.get("type").is_none(),
"body schema should not have a 'type' to be freeform for OpenAI compatibility"
);
}
+7 -5
View File
@@ -28,8 +28,7 @@ impl Tool for JsonTool {
"description": "The JSON operation to perform"
},
"data": {
"type": ["string", "object", "array", "number", "boolean", "null"],
"description": "JSON input data. Pass a string for parse, any type otherwise."
"description": "JSON input data. Pass a string for parse, or any JSON value (object, array, string, number, boolean, null) otherwise."
},
"path": {
"type": "string",
@@ -192,16 +191,19 @@ mod tests {
}
#[test]
fn test_json_tool_schema_data_has_type() {
fn test_json_tool_schema_data_is_freeform() {
let schema = JsonTool.parameters_schema();
let data = schema
.get("properties")
.and_then(|p| p.get("data"))
.expect("data schema missing");
// Data is intentionally freeform (no "type" constraint) for OpenAI
// compatibility. OpenAI rejects union types containing "array" unless
// "items" is also specified.
assert!(
data.get("type").is_some(),
"data schema must include a type for OpenAI-compatible tool validation"
data.get("type").is_none(),
"data schema should not have a 'type' to be freeform for OpenAI compatibility"
);
}
}
+26 -3
View File
@@ -383,6 +383,31 @@ impl LoadResults {
/// Compile-time project root, used to locate tools-src/ in dev builds.
const CARGO_MANIFEST_DIR: &str = env!("CARGO_MANIFEST_DIR");
/// Resolve the WASM target directory for a given crate directory.
///
/// Checks (in order):
/// 1. `CARGO_TARGET_DIR` env var (shared target dir)
/// 2. `<crate_dir>/target/` (default per-crate layout)
pub fn resolve_wasm_target_dir(crate_dir: &Path) -> PathBuf {
crate::registry::artifacts::resolve_target_dir(crate_dir)
}
/// Return the expected path to a compiled WASM artifact for a given crate.
///
/// Combines [`resolve_wasm_target_dir`] with the `wasm32-wasip2/release/` subdirectory
/// and the binary name without extension (e.g. `slack_tool`).
///
/// `binary_name` should not include the `.wasm` extension; it is appended automatically.
///
/// This is a convenience function for callers that know the exact triple (wasip2)
/// and binary name. For multi-triple search, use
/// [`crate::registry::artifacts::find_wasm_artifact`] instead.
pub fn wasm_artifact_path(crate_dir: &Path, binary_name: &str) -> PathBuf {
resolve_wasm_target_dir(crate_dir)
.join("wasm32-wasip2/release")
.join(format!("{}.wasm", binary_name))
}
/// Resolve the tools source directory.
///
/// Checks (in order):
@@ -426,9 +451,7 @@ pub async fn discover_dev_tools() -> Result<HashMap<String, DiscoveredTool>, std
let crate_name = dir_name.replace('-', "_");
let install_name = format!("{}-tool", dir_name);
let wasm_path = path
.join("target/wasm32-wasip2/release")
.join(format!("{}_tool.wasm", crate_name));
let wasm_path = wasm_artifact_path(&path, &format!("{}_tool", crate_name));
if !wasm_path.exists() {
continue;
+1 -1
View File
@@ -123,7 +123,7 @@ pub use storage::{
// Loader
pub use loader::{
DiscoveredTool, LoadResults, WasmLoadError, WasmToolLoader, discover_dev_tools, discover_tools,
load_dev_tools,
load_dev_tools, resolve_wasm_target_dir, wasm_artifact_path,
};
// Capabilities schema (for parsing *.capabilities.json files)
-375
View File
@@ -1,375 +0,0 @@
//! Audio transcription for voice notes and audio attachments.
//!
//! Provides a `TranscriptionProvider` trait and middleware that automatically
//! transcribes audio attachments on incoming messages before they reach the agent.
//!
//! # Architecture
//!
//! ```text
//! [WASM Channel] → emit-message { attachments=[audio bytes] }
//! → [Host: EmittedMessage → IncomingMessage]
//! → [TranscriptionMiddleware: detect audio, call provider, replace content]
//! → [Agent Loop: sees plain text]
//! ```
pub mod openai;
use std::sync::Arc;
use async_trait::async_trait;
use crate::channels::{AttachmentKind, IncomingMessage};
/// Supported audio formats for transcription.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AudioFormat {
/// OGG with Opus codec (Telegram voice notes).
OggOpus,
/// MP3.
Mp3,
/// WAV.
Wav,
/// WebM.
Webm,
/// M4A / AAC.
M4a,
}
impl AudioFormat {
/// File extension for this format.
pub fn extension(&self) -> &'static str {
match self {
Self::OggOpus => "ogg",
Self::Mp3 => "mp3",
Self::Wav => "wav",
Self::Webm => "webm",
Self::M4a => "m4a",
}
}
/// MIME type for this format.
pub fn mime_type(&self) -> &'static str {
match self {
Self::OggOpus => "audio/ogg",
Self::Mp3 => "audio/mpeg",
Self::Wav => "audio/wav",
Self::Webm => "audio/webm",
Self::M4a => "audio/mp4",
}
}
/// Detect format from MIME type string.
pub fn from_mime_type(mime: &str) -> Option<Self> {
// Normalize: strip parameters (e.g., "audio/ogg; codecs=opus" → "audio/ogg")
let base = mime.split(';').next().unwrap_or(mime).trim();
match base {
"audio/ogg" | "audio/opus" => Some(Self::OggOpus),
"audio/mpeg" | "audio/mp3" => Some(Self::Mp3),
"audio/wav" | "audio/x-wav" | "audio/wave" => Some(Self::Wav),
"audio/webm" => Some(Self::Webm),
"audio/mp4" | "audio/m4a" | "audio/x-m4a" | "audio/aac" => Some(Self::M4a),
_ => None,
}
}
}
/// Errors from transcription operations.
#[derive(Debug, thiserror::Error)]
pub enum TranscriptionError {
/// Unsupported audio format.
#[error("unsupported audio format: {mime_type}")]
UnsupportedFormat { mime_type: String },
/// Audio file exceeds the provider's size limit.
#[error("audio file too large: {size} bytes (max {max})")]
FileTooLarge { size: usize, max: usize },
/// Provider API returned an error.
#[error("transcription API error: {message}")]
ApiError { message: String },
/// Network or HTTP error.
#[error("transcription request failed: {0}")]
RequestFailed(String),
/// Provider is not configured.
#[error("transcription provider not configured: {reason}")]
NotConfigured { reason: String },
}
/// Trait for speech-to-text transcription providers.
#[async_trait]
pub trait TranscriptionProvider: Send + Sync {
/// Provider name (e.g., "openai").
fn name(&self) -> &str;
/// Model name (e.g., "whisper-1").
fn model_name(&self) -> &str;
/// Maximum file size in bytes.
fn max_file_size(&self) -> usize;
/// Supported audio formats.
fn supported_formats(&self) -> &[AudioFormat];
/// Transcribe audio bytes to text.
async fn transcribe(
&self,
audio: &[u8],
format: AudioFormat,
language: Option<&str>,
) -> Result<String, TranscriptionError>;
}
/// Middleware that detects audio attachments and transcribes them.
pub struct TranscriptionMiddleware {
provider: Arc<dyn TranscriptionProvider>,
language: Option<String>,
}
impl TranscriptionMiddleware {
/// Create a new transcription middleware.
pub fn new(provider: Arc<dyn TranscriptionProvider>, language: Option<String>) -> Self {
Self { provider, language }
}
/// Process an incoming message, transcribing any audio attachments.
///
/// If audio attachments are found and transcription succeeds, the message
/// content is replaced with the transcribed text. On failure, a fallback
/// message is used.
pub async fn process(&self, mut msg: IncomingMessage) -> IncomingMessage {
let audio_attachment = msg
.attachments
.iter()
.find(|a| a.kind == AttachmentKind::Audio);
let Some(attachment) = audio_attachment else {
return msg;
};
let format = match AudioFormat::from_mime_type(&attachment.mime_type) {
Some(f) => f,
None => {
tracing::warn!(
mime = %attachment.mime_type,
"Unsupported audio format for transcription"
);
if msg.content.is_empty() || msg.content == "[Voice note]" {
msg.content = "[Voice note: unsupported audio format]".to_string();
}
return msg;
}
};
// Check size limit
if attachment.data.len() > self.provider.max_file_size() {
tracing::warn!(
size = attachment.data.len(),
max = self.provider.max_file_size(),
"Audio attachment exceeds provider size limit"
);
if msg.content.is_empty() || msg.content == "[Voice note]" {
msg.content = "[Voice note: file too large for transcription]".to_string();
}
return msg;
}
match self
.provider
.transcribe(&attachment.data, format, self.language.as_deref())
.await
{
Ok(text) => {
tracing::info!(
provider = %self.provider.name(),
text_len = text.len(),
"Audio transcription successful"
);
msg.content = text;
}
Err(e) => {
tracing::error!(
error = %e,
provider = %self.provider.name(),
"Audio transcription failed"
);
if msg.content.is_empty() || msg.content == "[Voice note]" {
msg.content = "[Voice note: transcription failed]".to_string();
}
}
}
msg
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::channels::Attachment;
#[test]
fn audio_format_from_mime_type() {
assert_eq!(
AudioFormat::from_mime_type("audio/ogg"),
Some(AudioFormat::OggOpus)
);
assert_eq!(
AudioFormat::from_mime_type("audio/ogg; codecs=opus"),
Some(AudioFormat::OggOpus)
);
assert_eq!(
AudioFormat::from_mime_type("audio/mpeg"),
Some(AudioFormat::Mp3)
);
assert_eq!(
AudioFormat::from_mime_type("audio/mp3"),
Some(AudioFormat::Mp3)
);
assert_eq!(
AudioFormat::from_mime_type("audio/wav"),
Some(AudioFormat::Wav)
);
assert_eq!(
AudioFormat::from_mime_type("audio/webm"),
Some(AudioFormat::Webm)
);
assert_eq!(
AudioFormat::from_mime_type("audio/mp4"),
Some(AudioFormat::M4a)
);
assert_eq!(
AudioFormat::from_mime_type("audio/m4a"),
Some(AudioFormat::M4a)
);
assert_eq!(AudioFormat::from_mime_type("text/plain"), None);
assert_eq!(AudioFormat::from_mime_type(""), None);
}
#[test]
fn audio_format_extension() {
assert_eq!(AudioFormat::OggOpus.extension(), "ogg");
assert_eq!(AudioFormat::Mp3.extension(), "mp3");
assert_eq!(AudioFormat::Wav.extension(), "wav");
assert_eq!(AudioFormat::Webm.extension(), "webm");
assert_eq!(AudioFormat::M4a.extension(), "m4a");
}
#[test]
fn audio_format_mime_type() {
assert_eq!(AudioFormat::OggOpus.mime_type(), "audio/ogg");
assert_eq!(AudioFormat::Mp3.mime_type(), "audio/mpeg");
}
/// Mock provider for testing middleware.
struct MockProvider {
result: Result<String, TranscriptionError>,
}
#[async_trait]
impl TranscriptionProvider for MockProvider {
fn name(&self) -> &str {
"mock"
}
fn model_name(&self) -> &str {
"mock-1"
}
fn max_file_size(&self) -> usize {
25 * 1024 * 1024
}
fn supported_formats(&self) -> &[AudioFormat] {
&[AudioFormat::OggOpus, AudioFormat::Mp3]
}
async fn transcribe(
&self,
_audio: &[u8],
_format: AudioFormat,
_language: Option<&str>,
) -> Result<String, TranscriptionError> {
match &self.result {
Ok(text) => Ok(text.clone()),
Err(_) => Err(TranscriptionError::ApiError {
message: "mock error".to_string(),
}),
}
}
}
#[tokio::test]
async fn middleware_transcribes_audio_attachment() {
let provider = Arc::new(MockProvider {
result: Ok("Hello, world!".to_string()),
});
let middleware = TranscriptionMiddleware::new(provider, None);
let msg = IncomingMessage::new("telegram", "user1", "[Voice note]").with_attachments(vec![
Attachment {
kind: AttachmentKind::Audio,
mime_type: "audio/ogg".to_string(),
data: vec![0u8; 100],
filename: None,
duration_secs: Some(5),
},
]);
let result = middleware.process(msg).await;
assert_eq!(result.content, "Hello, world!");
}
#[tokio::test]
async fn middleware_skips_non_audio_messages() {
let provider = Arc::new(MockProvider {
result: Ok("transcribed".to_string()),
});
let middleware = TranscriptionMiddleware::new(provider, None);
let msg = IncomingMessage::new("telegram", "user1", "regular text");
let result = middleware.process(msg).await;
assert_eq!(result.content, "regular text");
}
#[tokio::test]
async fn middleware_handles_transcription_failure() {
let provider = Arc::new(MockProvider {
result: Err(TranscriptionError::ApiError {
message: "test".to_string(),
}),
});
let middleware = TranscriptionMiddleware::new(provider, None);
let msg = IncomingMessage::new("telegram", "user1", "[Voice note]").with_attachments(vec![
Attachment {
kind: AttachmentKind::Audio,
mime_type: "audio/ogg".to_string(),
data: vec![0u8; 100],
filename: None,
duration_secs: Some(5),
},
]);
let result = middleware.process(msg).await;
assert_eq!(result.content, "[Voice note: transcription failed]");
}
#[tokio::test]
async fn middleware_handles_unsupported_format() {
let provider = Arc::new(MockProvider {
result: Ok("text".to_string()),
});
let middleware = TranscriptionMiddleware::new(provider, None);
let msg = IncomingMessage::new("telegram", "user1", "[Voice note]").with_attachments(vec![
Attachment {
kind: AttachmentKind::Audio,
mime_type: "audio/flac".to_string(),
data: vec![0u8; 100],
filename: None,
duration_secs: None,
},
]);
let result = middleware.process(msg).await;
assert_eq!(result.content, "[Voice note: unsupported audio format]");
}
}
-196
View File
@@ -1,196 +0,0 @@
//! OpenAI Whisper transcription provider.
//!
//! Uses the OpenAI `/v1/audio/transcriptions` endpoint with multipart form upload.
use async_trait::async_trait;
use reqwest::multipart;
use secrecy::{ExposeSecret, SecretString};
use crate::transcription::{AudioFormat, TranscriptionError, TranscriptionProvider};
/// Maximum file size for Whisper API (25 MB).
const WHISPER_MAX_FILE_SIZE: usize = 25 * 1024 * 1024;
/// Supported formats for Whisper.
const WHISPER_FORMATS: &[AudioFormat] = &[
AudioFormat::OggOpus,
AudioFormat::Mp3,
AudioFormat::Wav,
AudioFormat::Webm,
AudioFormat::M4a,
];
/// OpenAI Whisper transcription provider.
pub struct OpenAiWhisper {
api_key: SecretString,
model: String,
base_url: String,
client: reqwest::Client,
}
impl OpenAiWhisper {
/// Create a new OpenAI Whisper provider.
pub fn new(api_key: SecretString, model: String) -> Self {
Self {
api_key,
model,
base_url: "https://api.openai.com".to_string(),
client: reqwest::Client::new(),
}
}
/// Set a custom base URL (for testing or alternative endpoints).
pub fn with_base_url(mut self, url: impl Into<String>) -> Self {
self.base_url = url.into();
self
}
}
#[async_trait]
impl TranscriptionProvider for OpenAiWhisper {
fn name(&self) -> &str {
"openai"
}
fn model_name(&self) -> &str {
&self.model
}
fn max_file_size(&self) -> usize {
WHISPER_MAX_FILE_SIZE
}
fn supported_formats(&self) -> &[AudioFormat] {
WHISPER_FORMATS
}
async fn transcribe(
&self,
audio: &[u8],
format: AudioFormat,
language: Option<&str>,
) -> Result<String, TranscriptionError> {
if audio.len() > WHISPER_MAX_FILE_SIZE {
return Err(TranscriptionError::FileTooLarge {
size: audio.len(),
max: WHISPER_MAX_FILE_SIZE,
});
}
let filename = format!("audio.{}", format.extension());
// Note: to_vec() copies the audio bytes for the multipart body.
// Peak memory is ~2x the file size (original slice + copy). Acceptable
// for the 25 MB Whisper limit; revisit if supporting larger files.
let file_part = multipart::Part::bytes(audio.to_vec())
.file_name(filename)
.mime_str(format.mime_type())
.map_err(|e| TranscriptionError::RequestFailed(e.to_string()))?;
let mut form = multipart::Form::new()
.part("file", file_part)
.text("model", self.model.clone())
.text("response_format", "text");
if let Some(lang) = language {
form = form.text("language", lang.to_string());
}
let url = format!("{}/v1/audio/transcriptions", self.base_url);
let response = self
.client
.post(&url)
.header(
"Authorization",
format!("Bearer {}", self.api_key.expose_secret()),
)
.multipart(form)
.send()
.await
.map_err(|e| TranscriptionError::RequestFailed(e.to_string()))?;
let status = response.status();
let body = response
.text()
.await
.map_err(|e| TranscriptionError::RequestFailed(e.to_string()))?;
if !status.is_success() {
return Err(TranscriptionError::ApiError {
message: format!("HTTP {}: {}", status, body),
});
}
// response_format=text returns raw text, trim whitespace
Ok(body.trim().to_string())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn file_too_large_is_rejected_sync() {
// The size check happens before any async work, so we can verify
// via a simple construction + manual call of the guard logic.
let oversized = vec![0u8; WHISPER_MAX_FILE_SIZE + 1];
assert!(oversized.len() > WHISPER_MAX_FILE_SIZE);
}
#[tokio::test]
async fn file_too_large_returns_error() {
let provider = OpenAiWhisper::new(
SecretString::from("sk-test".to_string()),
"whisper-1".to_string(),
);
let oversized = vec![0u8; WHISPER_MAX_FILE_SIZE + 1];
let result = provider
.transcribe(&oversized, AudioFormat::Wav, None)
.await;
match result {
Err(TranscriptionError::FileTooLarge { size, max }) => {
assert_eq!(size, WHISPER_MAX_FILE_SIZE + 1);
assert_eq!(max, WHISPER_MAX_FILE_SIZE);
}
other => panic!("Expected FileTooLarge, got: {:?}", other),
}
}
#[tokio::test]
async fn api_error_on_bad_url() {
// Point at a URL that will fail to connect
let provider = OpenAiWhisper::new(
SecretString::from("sk-test".to_string()),
"whisper-1".to_string(),
)
.with_base_url("http://127.0.0.1:1"); // port 1 won't be listening
let audio = vec![0u8; 100];
let result = provider
.transcribe(&audio, AudioFormat::OggOpus, Some("en"))
.await;
assert!(
matches!(result, Err(TranscriptionError::RequestFailed(_))),
"Expected RequestFailed, got: {:?}",
result
);
}
#[test]
fn provider_metadata() {
let provider = OpenAiWhisper::new(
SecretString::from("sk-test".to_string()),
"whisper-1".to_string(),
);
assert_eq!(provider.name(), "openai");
assert_eq!(provider.model_name(), "whisper-1");
assert_eq!(provider.max_file_size(), WHISPER_MAX_FILE_SIZE);
assert_eq!(provider.supported_formats().len(), 5);
}
}
+9 -13
View File
@@ -27,10 +27,8 @@ fn normalize(s: &str) -> String {
/// Normalize typographic/smart punctuation to ASCII so tests match converter output
/// regardless of apostrophe/quote variants (e.g. U+2019 ' → U+0027 ').
fn normalize_smart_punctuation(s: &str) -> String {
s.replace('\u{2019}', "'") // RIGHT SINGLE QUOTATION MARK
.replace('\u{2018}', "'") // LEFT SINGLE QUOTATION MARK
.replace('\u{201C}', "\"") // LEFT DOUBLE QUOTATION MARK
.replace('\u{201D}', "\"") // RIGHT DOUBLE QUOTATION MARK
s.replace(['\u{2019}', '\u{2018}'], "'")
.replace(['\u{201C}', '\u{201D}'], "\"")
}
#[test]
@@ -58,15 +56,13 @@ fn convert_test_pages_to_markdown() {
.unwrap_or("unknown");
let default_url = format!("https://example.com/test-pages/{}/", dir_name);
let metadata: PageMetadata = path
.join("metadata.json")
.is_file()
.then(|| {
let raw = std::fs::read_to_string(path.join("metadata.json"))
.expect("read metadata.json");
serde_json::from_str(&raw).expect("invalid metadata.json")
})
.unwrap_or_default();
let metadata: PageMetadata = if path.join("metadata.json").is_file() {
let raw =
std::fs::read_to_string(path.join("metadata.json")).expect("read metadata.json");
serde_json::from_str(&raw).expect("invalid metadata.json")
} else {
Default::default()
};
let url = metadata.url.as_deref().unwrap_or(&default_url).to_string();
+2 -2
View File
@@ -33,7 +33,7 @@ fn okta_api_call(method: &str, url: &str, body: Option<&str>) -> Result<String,
&format!("Okta API: {} {}", method, url),
);
let response = host::http_request(method, url, headers, body_bytes.as_deref())?;
let response = host::http_request(method, url, headers, body_bytes.as_deref(), None)?;
if response.status < 200 || response.status >= 300 {
let body_text = String::from_utf8_lossy(&response.body);
@@ -224,7 +224,7 @@ fn okta_api_call_with_headers(
&format!("Okta API: {} {}", method, url),
);
let response = host::http_request(method, url, headers, body_bytes.as_deref())?;
let response = host::http_request(method, url, headers, body_bytes.as_deref(), None)?;
if response.status < 200 || response.status >= 300 {
let body_text = String::from_utf8_lossy(&response.body);
+12 -23
View File
@@ -113,27 +113,6 @@ interface channel-host {
// ==================== Channel-Specific Capabilities ====================
/// Kind of attachment (audio, image, document).
enum attachment-kind {
audio,
image,
document,
}
/// Binary attachment on an emitted message (e.g., voice note, photo).
record attachment {
/// What kind of content this is.
kind: attachment-kind,
/// MIME type (e.g., "audio/ogg", "image/jpeg").
mime-type: string,
/// Raw bytes of the attachment.
data: list<u8>,
/// Optional filename.
filename: option<string>,
/// Duration in seconds (for audio/video).
duration-secs: option<u32>,
}
/// A message to emit to the agent.
record emitted-message {
/// User identifier within the channel (e.g., Slack user ID).
@@ -146,8 +125,6 @@ interface channel-host {
thread-id: option<string>,
/// Channel-specific metadata as JSON string.
metadata-json: string,
/// Optional binary attachments (voice notes, images, etc.).
attachments: list<attachment>,
}
/// Emit a message to the agent.
@@ -284,6 +261,18 @@ interface channel {
tool-started,
/// A tool execution completed.
tool-completed,
/// A tool execution produced a preview/result status.
tool-result,
/// A tool call is waiting for user approval.
approval-needed,
/// Generic status text that should be shown to the user.
status,
/// A background/sandbox job was started.
job-started,
/// An extension/tool requires user authentication.
auth-required,
/// Authentication flow completed.
auth-completed,
}
/// A status update from the agent.