gitops/optimclaw
1
0
Fork 0
mirror of https://github.com/outbackdingo/optimclaw.git synced 2026-09-03 01:59:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: remove Responses API, consolidate to Chat Completions (#272)

Browse Source 
* fix: strip reasoning from LLM responses and persist assistant messages reliably

- Filter out `type: "reasoning"` output items from NEAR AI Responses API
  parsing so chain-of-thought never reaches the UI (nearai.rs)
- Rewrite clean_response with regex-based tag stripping that is
  code-aware (preserves tags inside fenced blocks and inline backticks),
  supports 9+ tag names (think, thought, reasoning, reflection, etc.),
  handles <final> extraction, pipe-delimited tags, and case/whitespace
  tolerance (reasoning.rs)
- Add Reasoning::complete() helper so all non-agentic LLM call sites
  (summarize, suggest, heartbeat, compaction) get automatic response
  cleaning; thread SafetyLayer through to those callers
- Change persist_turn from fire-and-forget tokio::spawn to awaited async
  so both user and assistant messages are written before returning,
  preventing data loss on shutdown/restart
- Pass input_count through seed_response_chain so response chaining
  delta calculation is accurate after thread hydration on restart
- Make NearAiResponse.usage optional and preserve response_id in alt
  response path for chaining continuity
- Persist session token to DB during onboarding wizard so runtime
  loads it without legacy-key fallback; suppress spurious warning on
  fresh installs
- Fix dev tool double-registration when builder already registers them
- Load dotenv/ironclaw env for doctor and status subcommands
- Reduce startup log noise (demote info→debug for skills, remove
  redundant info lines)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Nudge to not loop over tools continuesly

* refactor: remove Responses API, consolidate NEAR AI to Chat Completions only

The Responses API provider (nearai.rs, 1278 lines) added significant complexity
(response chaining state machine, delta message calculation, previous_response_id
persistence) for marginal benefit. This consolidates to the Chat Completions API
only, upgrading NearAiChatProvider with dual auth (session token + API key) and
401 retry for session token renewal.

- Delete src/llm/nearai.rs (Responses API provider)
- Upgrade nearai_chat.rs with SessionManager, dual auth, flexible list_models
- Remove response_id from CompletionResponse and ToolCompletionResponse
- Remove seed_response_chain/get_response_chain_id from LlmProvider trait
- Remove response chain persistence from agent (thread_ops, session)
- Remove NearAiApiMode enum and NEARAI_API_MODE config
- Clean up all wrapper providers (retry, circuit_breaker, failover, cache)
- Update documentation (CLAUDE.md, .env.example)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: runtime log level control via gateway UI and URL parameter

Add server-side log level switching using tracing_subscriber::reload::Layer
so the EnvFilter can be swapped at runtime without restarting. Expose via
GET/PUT /api/logs/level endpoints, a "Server: LEVEL" dropdown in the logs
toolbar, and a ?log_level=debug URL parameter for one-click activation.

Also applies cargo fmt to pre-existing files (llm/, tests/).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
Illia Polosukhin
2026-02-20 20:43:32 +00:00
committed by GitHub
co-authored by Claude Opus 4.6
parent 7df356c109
commit 448383cfb0
38 changed files with 1331 additions and 1785 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/llm/circuit_breaker.rs
-8
View File
@@ -296,14 +296,6 @@ impl LlmProvider for CircuitBreakerProvider {
self.inner.set_model(model)
}
fn seed_response_chain(&self, thread_id: &str, response_id: String) {
self.inner.seed_response_chain(thread_id, response_id)
}
fn get_response_chain_id(&self, thread_id: &str) -> Option<String> {
self.inner.get_response_chain_id(thread_id)
}
fn calculate_cost(&self, input_tokens: u32, output_tokens: u32) -> Decimal {
self.inner.calculate_cost(input_tokens, output_tokens)
}
src/llm/failover.rs
-13
View File
@@ -359,15 +359,6 @@ impl LlmProvider for FailoverProvider {
.await
}
fn seed_response_chain(&self, thread_id: &str, response_id: String) {
self.providers[self.last_used.load(Ordering::Relaxed)]
.seed_response_chain(thread_id, response_id);
}
fn get_response_chain_id(&self, thread_id: &str) -> Option<String> {
self.providers[self.last_used.load(Ordering::Relaxed)].get_response_chain_id(thread_id)
}
fn calculate_cost(&self, input_tokens: u32, output_tokens: u32) -> Decimal {
self.providers[self.last_used.load(Ordering::Relaxed)]
.calculate_cost(input_tokens, output_tokens)
@@ -413,7 +404,6 @@ mod tests {
input_tokens: 10,
output_tokens: 5,
finish_reason: FinishReason::Stop,
response_id: None,
}))),
tool_complete_result: Mutex::new(Some(Ok(ToolCompletionResponse {
content: Some(content.to_string()),
@@ -421,7 +411,6 @@ mod tests {
input_tokens: 10,
output_tokens: 5,
finish_reason: FinishReason::Stop,
response_id: None,
}))),
}
}
@@ -803,7 +792,6 @@ mod tests {
input_tokens: 10,
output_tokens: 5,
finish_reason: FinishReason::Stop,
response_id: None,
})
}
@@ -829,7 +817,6 @@ mod tests {
input_tokens: 10,
output_tokens: 5,
finish_reason: FinishReason::Stop,
response_id: None,
})
}
src/llm/mod.rs
+21 -34
View File
@@ -1,7 +1,7 @@
//! LLM integration for the agent.
//!
//! Supports multiple backends:
//! - **NEAR AI** (default): Session-based or API key auth via NEAR AI proxy
//! - **NEAR AI** (default): Session token or API key auth via Chat Completions API
//! - **OpenAI**: Direct API access with your own key
//! - **Anthropic**: Direct API access with your own key
//! - **Ollama**: Local model inference
@@ -10,7 +10,6 @@
pub mod circuit_breaker;
pub mod costs;
pub mod failover;
mod nearai;
mod nearai_chat;
mod provider;
mod reasoning;
@@ -21,8 +20,7 @@ pub mod session;
pub use circuit_breaker::{CircuitBreakerConfig, CircuitBreakerProvider};
pub use failover::{CooldownConfig, FailoverProvider};
pub use nearai::{ModelInfo, NearAiProvider};
pub use nearai_chat::NearAiChatProvider;
pub use nearai_chat::{ModelInfo, NearAiChatProvider};
pub use provider::{
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, ModelMetadata,
Role, ToolCall, ToolCompletionRequest, ToolCompletionResponse, ToolDefinition, ToolResult,
@@ -41,7 +39,7 @@ use std::sync::Arc;
use rig::client::CompletionClient;
use secrecy::ExposeSecret;
use crate::config::{LlmBackend, LlmConfig, NearAiApiMode, NearAiConfig};
use crate::config::{LlmBackend, LlmConfig, NearAiConfig};
use crate::error::LlmError;
/// Create an LLM provider based on configuration.
@@ -71,24 +69,18 @@ pub fn create_llm_provider_with_config(
config: &NearAiConfig,
session: Arc<SessionManager>,
) -> Result<Arc<dyn LlmProvider>, LlmError> {
match config.api_mode {
NearAiApiMode::Responses => {
tracing::info!(
model = %config.model,
base_url = %config.base_url,
"Using NEAR AI Chat (Responses API, session token auth)"
);
Ok(Arc::new(NearAiProvider::new(config.clone(), session)?))
}
NearAiApiMode::ChatCompletions => {
tracing::info!(
model = %config.model,
base_url = %config.base_url,
"Using NEAR AI Cloud (Chat Completions API, API key auth)"
);
Ok(Arc::new(NearAiChatProvider::new(config.clone())?))
}
}
let auth_mode = if config.api_key.is_some() {
"API key"
} else {
"session token"
};
tracing::info!(
model = %config.model,
base_url = %config.base_url,
auth = auth_mode,
"Using NEAR AI (Chat Completions API)"
);
Ok(Arc::new(NearAiChatProvider::new(config.clone(), session)?))
}
fn create_openai_provider(config: &LlmConfig) -> Result<Arc<dyn LlmProvider>, LlmError> {
@@ -254,7 +246,7 @@ fn create_openai_compatible_provider(config: &LlmConfig) -> Result<Arc<dyn LlmPr
/// Create a cheap/fast LLM provider for lightweight tasks (heartbeat, routing, evaluation).
///
/// Uses `NEARAI_CHEAP_MODEL` if set, otherwise falls back to the main provider.
/// Currently only supports NEAR AI backends (Responses and ChatCompletions modes).
/// Currently only supports NEAR AI backend.
pub fn create_cheap_llm_provider(
config: &LlmConfig,
session: Arc<SessionManager>,
@@ -275,20 +267,16 @@ pub fn create_cheap_llm_provider(
let mut cheap_config = config.nearai.clone();
cheap_config.model = cheap_model.clone();
tracing::info!("Cheap LLM provider: {}", cheap_model);
match cheap_config.api_mode {
NearAiApiMode::Responses => Ok(Some(Arc::new(NearAiProvider::new(cheap_config, session)?))),
NearAiApiMode::ChatCompletions => {
Ok(Some(Arc::new(NearAiChatProvider::new(cheap_config)?)))
}
}
Ok(Some(Arc::new(NearAiChatProvider::new(
cheap_config,
session,
)?)))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::config::{LlmBackend, NearAiApiMode, NearAiConfig};
use crate::config::{LlmBackend, NearAiConfig};
use std::path::PathBuf;
fn test_nearai_config() -> NearAiConfig {
@@ -298,7 +286,6 @@ mod tests {
base_url: "https://api.near.ai".to_string(),
auth_base_url: "https://private.near.ai".to_string(),
session_path: PathBuf::from("/tmp/test-session.json"),
api_mode: NearAiApiMode::Responses,
api_key: None,
fallback_model: None,
max_retries: 3,
src/llm/nearai.rs
-1205
View File
File diff suppressed because it is too large Load Diff
src/llm/nearai_chat.rs
+199 -61
View File
@@ -1,8 +1,12 @@
//! NEAR AI Cloud provider implementation (Chat Completions API).
//! NEAR AI provider implementation (Chat Completions API).
//!
//! This provider uses the NEAR AI Cloud API (`cloud-api.near.ai`) which
//! exposes an OpenAI-compatible chat completions endpoint with API key
//! authentication.
//! This provider uses the OpenAI-compatible Chat Completions endpoint with
//! dual auth support:
//! - **API key auth**: When `NEARAI_API_KEY` is set, uses Bearer API key
//! - **Session token auth**: Otherwise, uses `SessionManager` for Bearer session token
//! with automatic renewal on 401 errors
use std::sync::Arc;
use async_trait::async_trait;
use reqwest::Client;
@@ -14,38 +18,51 @@ use serde::{Deserialize, Serialize};
use crate::config::NearAiConfig;
use crate::error::LlmError;
use crate::llm::provider::{
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, ModelMetadata,
Role, ToolCall, ToolCompletionRequest, ToolCompletionResponse,
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, Role, ToolCall,
ToolCompletionRequest, ToolCompletionResponse,
};
use crate::llm::session::SessionManager;
/// NEAR AI Cloud provider (Chat Completions API, API key auth).
/// Information about an available model from NEAR AI API.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ModelInfo {
/// Model identifier.
#[serde(alias = "id", alias = "model")]
pub name: String,
/// Optional provider name.
#[serde(default)]
pub provider: Option<String>,
}
/// NEAR AI provider (Chat Completions API, dual auth).
pub struct NearAiChatProvider {
client: Client,
config: NearAiConfig,
/// Session manager for session token auth (used when no API key is set).
session: Arc<SessionManager>,
active_model: std::sync::RwLock<String>,
flatten_tool_messages: bool,
}
impl NearAiChatProvider {
/// Create a new NEAR AI Cloud provider with API key auth.
/// Create a new NEAR AI Chat Completions provider.
///
/// Auth mode is determined by `config.api_key`:
/// - If set, uses Bearer API key auth
/// - If not set, uses session token auth via `SessionManager`
///
/// By default this enables tool-message flattening for compatibility with
/// providers that reject `role: "tool"` messages.
pub fn new(config: NearAiConfig) -> Result<Self, LlmError> {
Self::new_with_flatten(config, true)
pub fn new(config: NearAiConfig, session: Arc<SessionManager>) -> Result<Self, LlmError> {
Self::new_with_flatten(config, session, true)
}
/// Create a chat completions provider with configurable tool-message flattening.
pub fn new_with_flatten(
config: NearAiConfig,
session: Arc<SessionManager>,
flatten_tool_messages: bool,
) -> Result<Self, LlmError> {
if config.api_key.is_none() {
return Err(LlmError::AuthFailed {
provider: "nearai_chat".to_string(),
});
}
let client = Client::builder()
.timeout(std::time::Duration::from_secs(120))
.build()
@@ -58,6 +75,7 @@ impl NearAiChatProvider {
Ok(Self {
client,
config,
session,
active_model,
flatten_tool_messages,
})
@@ -74,23 +92,50 @@ impl NearAiChatProvider {
}
}
fn api_key(&self) -> String {
self.config
.api_key
.as_ref()
.map(|k| k.expose_secret().to_string())
.unwrap_or_default()
/// Returns true if using API key auth, false if session token auth.
fn uses_api_key(&self) -> bool {
self.config.api_key.is_some()
}
/// Resolve the Bearer token for the current auth mode.
async fn resolve_bearer_token(&self) -> Result<String, LlmError> {
if let Some(ref api_key) = self.config.api_key {
Ok(api_key.expose_secret().to_string())
} else {
let token = self.session.get_token().await?;
Ok(token.expose_secret().to_string())
}
}
/// Send a single request to the chat completions API.
///
/// Does not retry internally — retries are handled by the external
/// For session token auth, handles 401 by calling `session.handle_auth_failure()`
/// and retrying once.
///
/// Does not retry on other errors — retries are handled by the external
/// `RetryProvider` wrapper in the composition chain.
async fn send_request<T: Serialize, R: for<'de> Deserialize<'de>>(
&self,
body: &T,
) -> Result<R, LlmError> {
match self.send_request_inner(body).await {
Ok(result) => Ok(result),
Err(LlmError::SessionExpired { .. }) if !self.uses_api_key() => {
// Session expired, attempt renewal and retry once
self.session.handle_auth_failure().await?;
self.send_request_inner(body).await
}
Err(e) => Err(e),
}
}
/// Inner request implementation (single attempt).
async fn send_request_inner<T: Serialize, R: for<'de> Deserialize<'de>>(
&self,
body: &T,
) -> Result<R, LlmError> {
let url = self.api_url("chat/completions");
let token = self.resolve_bearer_token().await?;
tracing::debug!("Sending request to NEAR AI Chat: {}", url);
@@ -103,7 +148,7 @@ impl NearAiChatProvider {
let response = self
.client
.post(&url)
.header("Authorization", format!("Bearer {}", self.api_key()))
.header("Authorization", format!("Bearer {}", token))
.header("Content-Type", "application/json")
.json(body)
.send()
@@ -126,6 +171,17 @@ impl NearAiChatProvider {
let status_code = status.as_u16();
if status_code == 401 {
// For session token auth, distinguish session expired from plain auth failure
if !self.uses_api_key() {
let lower = response_text.to_lowercase();
let is_session_expired = lower.contains("session")
&& (lower.contains("expired") || lower.contains("invalid"));
if is_session_expired {
return Err(LlmError::SessionExpired {
provider: "nearai_chat".to_string(),
});
}
}
return Err(LlmError::AuthFailed {
provider: "nearai_chat".to_string(),
});
@@ -154,14 +210,31 @@ impl NearAiChatProvider {
})
}
/// Fetch available models with full metadata from the `/v1/models` endpoint.
async fn fetch_models(&self) -> Result<Vec<ApiModelEntry>, LlmError> {
/// Fetch available models from the NEAR AI API.
///
/// Handles session renewal on 401 (same pattern as `send_request`).
/// Supports multiple response formats: `{models: [...]}`, `{data: [...]}`, and plain array.
pub async fn list_models_full(&self) -> Result<Vec<ModelInfo>, LlmError> {
match self.list_models_inner().await {
Ok(models) => Ok(models),
Err(LlmError::SessionExpired { .. }) if !self.uses_api_key() => {
self.session.handle_auth_failure().await?;
self.list_models_inner().await
}
Err(e) => Err(e),
}
}
async fn list_models_inner(&self) -> Result<Vec<ModelInfo>, LlmError> {
let url = self.api_url("models");
let token = self.resolve_bearer_token().await?;
tracing::debug!("Fetching models from: {}", url);
let response = self
.client
.get(&url)
.header("Authorization", format!("Bearer {}", self.api_key()))
.header("Authorization", format!("Bearer {}", token))
.send()
.await
.map_err(|e| LlmError::RequestFailed {
@@ -176,6 +249,11 @@ impl NearAiChatProvider {
})?;
if !status.is_success() {
if status.as_u16() == 401 && !self.uses_api_key() {
return Err(LlmError::SessionExpired {
provider: "nearai_chat".to_string(),
});
}
let truncated = crate::agent::truncate_for_preview(&response_text, 512);
return Err(LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
@@ -183,29 +261,97 @@ impl NearAiChatProvider {
});
}
// Flexible model entry parsing -- handle various field names
#[derive(Deserialize)]
struct ModelsResponse {
data: Vec<ApiModelEntry>,
struct ModelMetadataInner {
#[serde(default)]
name: Option<String>,
#[serde(default, alias = "modelName", alias = "model_name")]
model_name: Option<String>,
}
let resp: ModelsResponse =
serde_json::from_str(&response_text).map_err(|e| LlmError::InvalidResponse {
provider: "nearai_chat".to_string(),
reason: format!("JSON parse error: {}", e),
})?;
#[derive(Deserialize)]
struct ModelEntry {
#[serde(default)]
name: Option<String>,
#[serde(default)]
id: Option<String>,
#[serde(default)]
model: Option<String>,
#[serde(default, alias = "modelName", alias = "model_name")]
model_name: Option<String>,
#[serde(default, alias = "modelId", alias = "model_id")]
model_id: Option<String>,
#[serde(default)]
metadata: Option<ModelMetadataInner>,
}
Ok(resp.data)
impl ModelEntry {
fn get_name(&self) -> Option<String> {
self.name
.clone()
.or_else(|| self.id.clone())
.or_else(|| self.model.clone())
.or_else(|| self.model_name.clone())
.or_else(|| self.model_id.clone())
.or_else(|| self.metadata.as_ref().and_then(|m| m.name.clone()))
.or_else(|| self.metadata.as_ref().and_then(|m| m.model_name.clone()))
}
}
#[derive(Deserialize)]
struct ModelsResponse {
#[serde(default)]
models: Option<Vec<ModelEntry>>,
#[serde(default)]
data: Option<Vec<ModelEntry>>,
}
// Try {models: [...]} or {data: [...]} format
if let Ok(resp) = serde_json::from_str::<ModelsResponse>(&response_text)
&& let Some(entries) = resp.models.or(resp.data)
{
let models: Vec<ModelInfo> = entries
.into_iter()
.filter_map(|e| {
e.get_name().map(|name| ModelInfo {
name,
provider: None,
})
})
.collect();
if !models.is_empty() {
return Ok(models);
}
}
// Try direct array format
if let Ok(entries) = serde_json::from_str::<Vec<ModelEntry>>(&response_text) {
let models: Vec<ModelInfo> = entries
.into_iter()
.filter_map(|e| {
e.get_name().map(|name| ModelInfo {
name,
provider: None,
})
})
.collect();
if !models.is_empty() {
return Ok(models);
}
}
// Couldn't find model names in response
Err(LlmError::InvalidResponse {
provider: "nearai_chat".to_string(),
reason: format!(
"No model names found in response: {}",
&response_text[..response_text.len().min(300)]
),
})
}
}
/// Model entry as returned by the `/v1/models` API.
#[derive(Debug, Deserialize)]
struct ApiModelEntry {
id: String,
#[serde(default)]
context_length: Option<u32>,
}
#[async_trait]
impl LlmProvider for NearAiChatProvider {
async fn complete(&self, req: CompletionRequest) -> Result<CompletionResponse, LlmError> {
@@ -252,7 +398,6 @@ impl LlmProvider for NearAiChatProvider {
finish_reason,
input_tokens,
output_tokens,
response_id: None,
})
}
@@ -347,7 +492,6 @@ impl LlmProvider for NearAiChatProvider {
finish_reason,
input_tokens,
output_tokens,
response_id: None,
})
}
@@ -361,18 +505,8 @@ impl LlmProvider for NearAiChatProvider {
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
let models = self.fetch_models().await?;
Ok(models.into_iter().map(|m| m.id).collect())
}
async fn model_metadata(&self) -> Result<ModelMetadata, LlmError> {
let active = self.active_model_name();
let models = self.fetch_models().await?;
let current = models.iter().find(|m| m.id == active);
Ok(ModelMetadata {
id: active,
context_length: current.and_then(|m| m.context_length),
})
let models = self.list_models_full().await?;
Ok(models.into_iter().map(|m| m.name).collect())
}
fn active_model_name(&self) -> String {
@@ -613,6 +747,7 @@ fn parse_usage(usage: Option<&ChatCompletionUsage>) -> (u32, u32) {
#[cfg(test)]
mod tests {
use super::*;
use crate::llm::session::SessionConfig;
fn test_nearai_config(base_url: &str) -> NearAiConfig {
NearAiConfig {
@@ -620,7 +755,6 @@ mod tests {
base_url: base_url.to_string(),
auth_base_url: "https://private.near.ai".to_string(),
session_path: std::path::PathBuf::from("/tmp/session.json"),
api_mode: crate::config::NearAiApiMode::ChatCompletions,
api_key: Some(secrecy::SecretString::from("test-key".to_string())),
cheap_model: None,
fallback_model: None,
@@ -635,18 +769,22 @@ mod tests {
}
}
fn test_session() -> Arc<SessionManager> {
Arc::new(SessionManager::new(SessionConfig::default()))
}
#[test]
fn test_api_url_with_base_without_v1() {
let mut cfg = test_nearai_config("http://127.0.0.1:8318");
let provider = NearAiChatProvider::new(cfg.clone()).expect("provider");
let provider = NearAiChatProvider::new(cfg.clone(), test_session()).expect("provider");
assert_eq!(
provider.api_url("chat/completions"),
"http://127.0.0.1:8318/v1/chat/completions"
);
cfg.base_url = "http://127.0.0.1:8318/".to_string();
let provider = NearAiChatProvider::new(cfg).expect("provider");
let provider = NearAiChatProvider::new(cfg, test_session()).expect("provider");
assert_eq!(
provider.api_url("/chat/completions"),
"http://127.0.0.1:8318/v1/chat/completions"
@@ -657,7 +795,7 @@ mod tests {
fn test_api_url_with_base_already_v1() {
let cfg = test_nearai_config("http://127.0.0.1:8318/v1");
let provider = NearAiChatProvider::new(cfg).expect("provider");
let provider = NearAiChatProvider::new(cfg, test_session()).expect("provider");
assert_eq!(
provider.api_url("chat/completions"),
"http://127.0.0.1:8318/v1/chat/completions"
src/llm/provider.rs
-18
View File
@@ -153,8 +153,6 @@ pub struct CompletionResponse {
pub input_tokens: u32,
pub output_tokens: u32,
pub finish_reason: FinishReason,
/// Provider-specific response ID (e.g. for NEAR AI response chaining).
pub response_id: Option<String>,
}
/// Why the completion finished.
@@ -256,8 +254,6 @@ pub struct ToolCompletionResponse {
pub input_tokens: u32,
pub output_tokens: u32,
pub finish_reason: FinishReason,
/// Provider-specific response ID (e.g. for NEAR AI response chaining).
pub response_id: Option<String>,
}
/// Metadata about a model returned by the provider's API.
@@ -327,20 +323,6 @@ pub trait LlmProvider: Send + Sync {
})
}
/// Seed a response chain for a thread (e.g. restoring from DB).
///
/// Providers that support response chaining (e.g. NEAR AI `previous_response_id`)
/// store this so subsequent calls send only delta messages.
fn seed_response_chain(&self, _thread_id: &str, _response_id: String) {}
/// Get the last response chain ID for a thread.
///
/// Returns `None` if the provider doesn't support chaining or has no
/// stored state for this thread.
fn get_response_chain_id(&self, _thread_id: &str) -> Option<String> {
None
}
/// Calculate cost for a completion.
fn calculate_cost(&self, input_tokens: u32, output_tokens: u32) -> Decimal {
let (input_cost, output_cost) = self.cost_per_token();
src/llm/reasoning.rs
+687 -159
View File
File diff suppressed because it is too large Load Diff
src/llm/response_cache.rs
-8
View File
@@ -228,14 +228,6 @@ impl LlmProvider for CachedProvider {
fn set_model(&self, model: &str) -> Result<(), LlmError> {
self.inner.set_model(model)
}
fn seed_response_chain(&self, thread_id: &str, response_id: String) {
self.inner.seed_response_chain(thread_id, response_id);
}
fn get_response_chain_id(&self, thread_id: &str) -> Option<String> {
self.inner.get_response_chain_id(thread_id)
}
}
#[cfg(test)]
src/llm/retry.rs
-8
View File
@@ -210,14 +210,6 @@ impl LlmProvider for RetryProvider {
self.inner.set_model(model)
}
fn seed_response_chain(&self, thread_id: &str, response_id: String) {
self.inner.seed_response_chain(thread_id, response_id)
}
fn get_response_chain_id(&self, thread_id: &str) -> Option<String> {
self.inner.get_response_chain_id(thread_id)
}
fn calculate_cost(&self, input_tokens: u32, output_tokens: u32) -> Decimal {
self.inner.calculate_cost(input_tokens, output_tokens)
}
src/llm/rig_adapter.rs
-2
View File
@@ -445,7 +445,6 @@ where
input_tokens: saturate_u32(response.usage.input_tokens),
output_tokens: saturate_u32(response.usage.output_tokens),
finish_reason: finish,
response_id: None,
})
}
@@ -511,7 +510,6 @@ where
input_tokens: saturate_u32(response.usage.input_tokens),
output_tokens: saturate_u32(response.usage.output_tokens),
finish_reason: finish,
response_id: None,
})
}
src/llm/session.rs
+19 -9
View File
@@ -513,20 +513,30 @@ impl SessionManager {
})? {
value
} else {
tracing::warn!(
"nearai.session_token missing; falling back to legacy nearai.session for backwards compatibility"
);
store
// Try the legacy key. Only warn if it actually exists (real
// backwards-compat migration). When neither key is present
// (fresh install), just return the "No session in DB" error.
let legacy = store
.get_setting(&user_id, "nearai.session")
.await
.map_err(|e| LlmError::SessionRenewalFailed {
provider: "nearai".to_string(),
reason: format!("DB query failed: {}", e),
})?
.ok_or(LlmError::SessionRenewalFailed {
provider: "nearai".to_string(),
reason: "No session in DB".to_string(),
})?
})?;
match legacy {
Some(value) => {
tracing::warn!(
"nearai.session_token missing; falling back to legacy nearai.session for backwards compatibility"
);
value
}
None => {
return Err(LlmError::SessionRenewalFailed {
provider: "nearai".to_string(),
reason: "No session in DB".to_string(),
});
}
}
};
let session: SessionData =