feat: add AWS Bedrock LLM provider via native Converse API (#713)

* feat: add AWS Bedrock LLM provider via native Converse API * fix: use JSON parsing for tool result error detection instead of brittle substring matching * refactor: extract duplicated inference config builder into helper function * fix: address review feedback — safe casts, input validation, and tests - Safe u32→i32 cast for max_tokens using try_from with clamp - Remove brittle string-based error detection fallback for tool results - Validate BEDROCK_CROSS_REGION against allowed values (us/eu/apac/global) - Validate message list is non-empty before Converse API call - Log when using default us-east-1 region - Update llm_backend doc comment to list all backends - Add tests for build_inference_config and empty message handling * fix: persist AWS_PROFILE for Bedrock named profile auth The wizard collected the profile name but only printed a hint to set it manually. Now it saves to settings and writes AWS_PROFILE to the bootstrap .env, consistent with how BEDROCK_REGION and other Bedrock settings are persisted. * feat: gate AWS Bedrock behind optional `bedrock` feature flag The AWS SDK dependencies (aws-config, aws-sdk-bedrockruntime, aws-smithy-types) require cmake and a C compiler to build aws-lc-sys. Gate them behind an opt-in `bedrock` feature flag so default builds are unaffected. Build with: cargo build --features bedrock All config, settings, and wizard code stays unconditional (no AWS deps) so users can configure Bedrock even without the feature compiled — they get a clear error at startup directing them to rebuild. * fix: address review feedback and adapt Bedrock provider to registry architecture (takeover #345) - Resolve merge conflicts with main's registry-based provider system - Add missing cache_creation_input_tokens/cache_read_input_tokens fields - Add missing content_parts field in test ChatMessage - Fix string literal type mismatches in wizard env_vars (.to_string()) - Remove non-functional bearer token auth (AWS_BEARER_TOKEN_BEDROCK) from wizard and documentation per reviewer feedback from @zmanian and @serrrfirat - Remove stale BEDROCK_ACCESS_KEY proxy entry from provider table - Update Bedrock provider to use is_bedrock string check (LlmBackend enum removed) - Add bedrock_profile fallback from settings in config resolution [skip-regression-check] Co-Authored-By: cgorski <[email protected]> Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use main's Cargo.lock as base to preserve dependency versions Regenerating Cargo.lock from scratch caused transitive dependency version drift that broke the html_to_markdown fixture test in CI. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: bedrock config bugs — spurious warning, alias normalization, profile fallback - Move is_bedrock check before unknown-backend warning to prevent spurious "unknown backend" log for bedrock users - Normalize backend aliases ("aws", "aws_bedrock") to "bedrock" so the provider factory matches correctly - Add settings.bedrock_profile fallback for AWS_PROFILE, consistent with region and cross_region resolution [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address Copilot review feedback — bearer token cleanup, stop_sequences, model dedup - Remove stale bearer token refs from setup README and CHANGELOG - Remove dead bedrock_api_key secret injection mapping - Pass stop_sequences through to Bedrock InferenceConfiguration - Remove "API key" from wizard menu description (bearer token removed) - Skip duplicate LLM_MODEL write for bedrock backend in wizard - Fix cargo fmt formatting [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review feedback — async new(), remove LiteLLM entry, wizard fixes - Remove dead LiteLLM-based bedrock entry from providers.json (native Converse API intercepts before registry lookup) - Make BedrockProvider::new() async to avoid block_in_place panic in current_thread runtimes; propagate async to create_llm_provider, build_provider_chain, and init_llm - Document CMake build prerequisite in docs/LLM_PROVIDERS.md - Clear bedrock_profile when user selects "default credentials" in wizard - Fix selected_model clearing to match established pattern (conditional on provider switch, not unconditional) - Add regression tests for bedrock model preservation and profile clearing Addresses review feedback from @zmanian on PR #713. Streaming support tracked in #741. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address remaining review comments — CLAUDE.md backends, wizard UX - Add `bedrock` to CLAUDE.md inline backend list (#10) - Skip full setup re-run when keeping existing Bedrock config (#11) - Clear stale bedrock_profile on empty named-profile input (#12) - Add regression test for empty profile clearing Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Chris Gorski <[email protected]> Co-authored-by: cgorski <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-08-30 08:17:53 +00:00 · 2026-03-09 07:10:25 +00:00
parent 30d81fcdee
commit d73e35cfb0
16 changed files with 2076 additions and 44 deletions
@@ -86,6 +86,19 @@ pub struct RegistryProviderConfig {
    pub oauth_token: Option<SecretString>,
 }

+/// Configuration for AWS Bedrock (native Converse API).
+#[derive(Debug, Clone)]
+pub struct BedrockConfig {
+    /// AWS region (e.g. "us-east-1").
+    pub region: String,
+    /// Bedrock model ID (e.g. "anthropic.claude-opus-4-6-v1").
+    pub model: String,
+    /// Cross-region inference prefix: "us", "eu", "apac", "global", or None.
+    pub cross_region: Option<String>,
+    /// AWS named profile (for SSO / assume-role workflows).
+    pub profile: Option<String>,
+}
+
 /// LLM provider configuration.
 ///
 /// NearAI remains the default backend with its own config struct (session auth).
@@ -101,8 +114,10 @@ pub struct LlmConfig {
    /// NEAR AI config (always populated, also used for embeddings).
    pub nearai: NearAiConfig,
    /// Resolved provider config for registry-based providers.
-    /// `None` when backend is "nearai".
+    /// `None` when backend is "nearai" or "bedrock".
    pub provider: Option<RegistryProviderConfig>,
+    /// AWS Bedrock config (populated when backend=bedrock, requires --features bedrock).
+    pub bedrock: Option<BedrockConfig>,
    /// HTTP request timeout in seconds for LLM API calls.
    /// Default: 120. Increase for local LLMs (Ollama, vLLM, LM Studio) that
    /// need more time for prompt evaluation on consumer hardware.
@@ -169,6 +184,7 @@ impl LlmConfig {
                smart_routing_cascade: false,
            },
            provider: None,
+            bedrock: None,
            request_timeout_secs: 120,
        }
    }
@@ -200,8 +216,10 @@ impl LlmConfig {
        let backend_lower = backend.to_lowercase();
        let is_nearai =
            backend_lower == "nearai" || backend_lower == "near_ai" || backend_lower == "near";
+        let is_bedrock =
+            backend_lower == "bedrock" || backend_lower == "aws_bedrock" || backend_lower == "aws";

-        if !is_nearai && registry.find(&backend_lower).is_none() {
+        if !is_nearai && !is_bedrock && registry.find(&backend_lower).is_none() {
            tracing::warn!(
                "Unknown LLM backend '{}'. Will attempt as openai_compatible fallback.",
                backend
@@ -248,8 +266,8 @@ impl LlmConfig {
            smart_routing_cascade: parse_optional_env("SMART_ROUTING_CASCADE", true)?,
        };

-        // Resolve registry provider config (for non-NearAI backends)
-        let provider = if is_nearai {
+        // Resolve registry provider config (for non-NearAI, non-Bedrock backends)
+        let provider = if is_nearai || is_bedrock {
            None
        } else {
            Some(Self::resolve_registry_provider(
@@ -259,11 +277,50 @@ impl LlmConfig {
            )?)
        };

+        let bedrock = if is_bedrock {
+            let explicit_region =
+                optional_env("BEDROCK_REGION")?.or_else(|| settings.bedrock_region.clone());
+            if explicit_region.is_none() {
+                tracing::info!("BEDROCK_REGION not set, defaulting to us-east-1");
+            }
+            let region = explicit_region.unwrap_or_else(|| "us-east-1".to_string());
+            let model = optional_env("BEDROCK_MODEL")?
+                .or_else(|| settings.selected_model.clone())
+                .ok_or_else(|| ConfigError::MissingRequired {
+                    key: "BEDROCK_MODEL".to_string(),
+                    hint: "Set BEDROCK_MODEL when LLM_BACKEND=bedrock".to_string(),
+                })?;
+            let cross_region = optional_env("BEDROCK_CROSS_REGION")?
+                .or_else(|| settings.bedrock_cross_region.clone());
+            if let Some(ref cr) = cross_region
+                && !matches!(cr.as_str(), "us" | "eu" | "apac" | "global")
+            {
+                return Err(ConfigError::InvalidValue {
+                    key: "BEDROCK_CROSS_REGION".to_string(),
+                    message: format!(
+                        "'{}' is not valid, expected one of: us, eu, apac, global",
+                        cr
+                    ),
+                });
+            }
+            let profile = optional_env("AWS_PROFILE")?.or_else(|| settings.bedrock_profile.clone());
+            Some(BedrockConfig {
+                region,
+                model,
+                cross_region,
+                profile,
+            })
+        } else {
+            None
+        };
+
        let request_timeout_secs = parse_optional_env("LLM_REQUEST_TIMEOUT_SECS", 120)?;

        Ok(Self {
            backend: if is_nearai {
                "nearai".to_string()
+            } else if is_bedrock {
+                "bedrock".to_string()
            } else if let Some(ref p) = provider {
                p.provider_id.clone()
            } else {
@@ -272,6 +329,7 @@ impl LlmConfig {
            session,
            nearai,
            provider,
+            bedrock,
            request_timeout_secs,
        })
    }
@@ -37,7 +37,9 @@ pub use self::database::{DatabaseBackend, DatabaseConfig, SslMode, default_libsq
 pub use self::embeddings::EmbeddingsConfig;
 pub use self::heartbeat::HeartbeatConfig;
 pub use self::hygiene::HygieneConfig;
-pub use self::llm::{CacheRetention, LlmConfig, NearAiConfig, RegistryProviderConfig};
+pub use self::llm::{
+    BedrockConfig, CacheRetention, LlmConfig, NearAiConfig, RegistryProviderConfig,
+};
 pub use self::routines::RoutineConfig;
 pub use self::safety::SafetyConfig;
 pub use self::sandbox::{ClaudeCodeConfig, SandboxModeConfig};