mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
feat: add AWS Bedrock LLM provider via native Converse API (#713)
* feat: add AWS Bedrock LLM provider via native Converse API * fix: use JSON parsing for tool result error detection instead of brittle substring matching * refactor: extract duplicated inference config builder into helper function * fix: address review feedback — safe casts, input validation, and tests - Safe u32→i32 cast for max_tokens using try_from with clamp - Remove brittle string-based error detection fallback for tool results - Validate BEDROCK_CROSS_REGION against allowed values (us/eu/apac/global) - Validate message list is non-empty before Converse API call - Log when using default us-east-1 region - Update llm_backend doc comment to list all backends - Add tests for build_inference_config and empty message handling * fix: persist AWS_PROFILE for Bedrock named profile auth The wizard collected the profile name but only printed a hint to set it manually. Now it saves to settings and writes AWS_PROFILE to the bootstrap .env, consistent with how BEDROCK_REGION and other Bedrock settings are persisted. * feat: gate AWS Bedrock behind optional `bedrock` feature flag The AWS SDK dependencies (aws-config, aws-sdk-bedrockruntime, aws-smithy-types) require cmake and a C compiler to build aws-lc-sys. Gate them behind an opt-in `bedrock` feature flag so default builds are unaffected. Build with: cargo build --features bedrock All config, settings, and wizard code stays unconditional (no AWS deps) so users can configure Bedrock even without the feature compiled — they get a clear error at startup directing them to rebuild. * fix: address review feedback and adapt Bedrock provider to registry architecture (takeover #345) - Resolve merge conflicts with main's registry-based provider system - Add missing cache_creation_input_tokens/cache_read_input_tokens fields - Add missing content_parts field in test ChatMessage - Fix string literal type mismatches in wizard env_vars (.to_string()) - Remove non-functional bearer token auth (AWS_BEARER_TOKEN_BEDROCK) from wizard and documentation per reviewer feedback from @zmanian and @serrrfirat - Remove stale BEDROCK_ACCESS_KEY proxy entry from provider table - Update Bedrock provider to use is_bedrock string check (LlmBackend enum removed) - Add bedrock_profile fallback from settings in config resolution [skip-regression-check] Co-Authored-By: cgorski <[email protected]> Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use main's Cargo.lock as base to preserve dependency versions Regenerating Cargo.lock from scratch caused transitive dependency version drift that broke the html_to_markdown fixture test in CI. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: bedrock config bugs — spurious warning, alias normalization, profile fallback - Move is_bedrock check before unknown-backend warning to prevent spurious "unknown backend" log for bedrock users - Normalize backend aliases ("aws", "aws_bedrock") to "bedrock" so the provider factory matches correctly - Add settings.bedrock_profile fallback for AWS_PROFILE, consistent with region and cross_region resolution [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address Copilot review feedback — bearer token cleanup, stop_sequences, model dedup - Remove stale bearer token refs from setup README and CHANGELOG - Remove dead bedrock_api_key secret injection mapping - Pass stop_sequences through to Bedrock InferenceConfiguration - Remove "API key" from wizard menu description (bearer token removed) - Skip duplicate LLM_MODEL write for bedrock backend in wizard - Fix cargo fmt formatting [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review feedback — async new(), remove LiteLLM entry, wizard fixes - Remove dead LiteLLM-based bedrock entry from providers.json (native Converse API intercepts before registry lookup) - Make BedrockProvider::new() async to avoid block_in_place panic in current_thread runtimes; propagate async to create_llm_provider, build_provider_chain, and init_llm - Document CMake build prerequisite in docs/LLM_PROVIDERS.md - Clear bedrock_profile when user selects "default credentials" in wizard - Fix selected_model clearing to match established pattern (conditional on provider switch, not unconditional) - Add regression tests for bedrock model preservation and profile clearing Addresses review feedback from @zmanian on PR #713. Streaming support tracked in #741. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address remaining review comments — CLAUDE.md backends, wizard UX - Add `bedrock` to CLAUDE.md inline backend list (#10) - Skip full setup re-run when keeping existing Bedrock config (#11) - Clear stale bedrock_profile on empty named-profile input (#12) - Add regression test for empty profile clearing Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Chris Gorski <[email protected]> Co-authored-by: cgorski <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
Illia Polosukhin
co-authored by
Chris Gorski
cgorski
Claude Opus 4.6
Chris Gorski
cgorski
Claude Opus 4.6
parent
30d81fcdee
commit
d73e35cfb0
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- AWS Bedrock LLM provider via native Converse API with IAM and SSO auth support (feature-gated: `--features bedrock`)
|
||||
|
||||
## [0.16.1](https://github.com/nearai/ironclaw/compare/v0.16.0...v0.16.1) - 2026-03-06
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -484,6 +484,13 @@ SKILLS_AUTO_DISCOVER=true # Scan skill directories on startup
|
||||
TINFOIL_API_KEY=... # Required when LLM_BACKEND=tinfoil
|
||||
TINFOIL_MODEL=kimi-k2-5 # Default model
|
||||
|
||||
# AWS Bedrock (native Converse API, requires --features bedrock)
|
||||
# LLM_BACKEND=bedrock
|
||||
# BEDROCK_REGION=us-east-1 # AWS region
|
||||
# BEDROCK_MODEL=anthropic.claude-opus-4-6-v1 # Required model ID
|
||||
# BEDROCK_CROSS_REGION=us # Cross-region prefix (us/eu/apac/global)
|
||||
# AWS_PROFILE=my-profile # Named profile (SSO/assume-role)
|
||||
|
||||
# Tunnel (public internet exposure for webhooks)
|
||||
TUNNEL_URL=https://abc123.ngrok.io # Static public URL (manual tunnel)
|
||||
# Or use a managed tunnel provider:
|
||||
@@ -500,7 +507,9 @@ OBSERVABILITY_BACKEND=none # none/noop (default) or log
|
||||
|
||||
### LLM Providers
|
||||
|
||||
Backends: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, `tinfoil` — set via `LLM_BACKEND`. See [src/llm/CLAUDE.md](src/llm/CLAUDE.md) for per-provider auth and configuration details.
|
||||
Backends: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, `tinfoil`, `bedrock` (requires `--features bedrock`) — set via `LLM_BACKEND`. See [src/llm/CLAUDE.md](src/llm/CLAUDE.md) for per-provider auth and configuration details.
|
||||
|
||||
**AWS Bedrock** -- Uses the native Converse API via `aws-sdk-bedrockruntime`. Requires `--features bedrock` at build time (not included in default features due to heavy AWS SDK dependencies). Supports standard AWS auth methods: IAM credentials (`AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`), SSO profiles (`AWS_PROFILE`), and instance roles. Configure with `BEDROCK_REGION` (default: `us-east-1`), `BEDROCK_MODEL` (required, e.g., `anthropic.claude-opus-4-6-v1`), and `BEDROCK_CROSS_REGION` (optional: `us`, `eu`, `apac`, `global` for cross-region inference profiles). The SDK credential chain resolves auth automatically from the environment.
|
||||
|
||||
## Database
|
||||
|
||||
|
||||
Generated
+533
@@ -392,6 +392,412 @@ version = "1.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
|
||||
|
||||
[[package]]
|
||||
name = "aws-config"
|
||||
version = "1.8.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "11493b0bad143270fb8ad284a096dd529ba91924c5409adeac856cc1bf047dbc"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-runtime",
|
||||
"aws-sdk-sso",
|
||||
"aws-sdk-ssooidc",
|
||||
"aws-sdk-sts",
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-http",
|
||||
"aws-smithy-json",
|
||||
"aws-smithy-runtime",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"aws-types",
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"hex",
|
||||
"http 1.4.0",
|
||||
"sha1",
|
||||
"time",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"url",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-credential-types"
|
||||
version = "1.2.14"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8f20799b373a1be121fe3005fba0c2090af9411573878f224df44b42727fcaf7"
|
||||
dependencies = [
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-lc-rs"
|
||||
version = "1.16.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "94bffc006df10ac2a68c83692d734a465f8ee6c5b384d8545a636f81d858f4bf"
|
||||
dependencies = [
|
||||
"aws-lc-sys",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-lc-sys"
|
||||
version = "0.38.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4321e568ed89bb5a7d291a7f37997c2c0df89809d7b6d12062c81ddb54aa782e"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"cmake",
|
||||
"dunce",
|
||||
"fs_extra",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-runtime"
|
||||
version = "1.7.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5fc0651c57e384202e47153c1260b84a9936e19803d747615edf199dc3b98d17"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-sigv4",
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-eventstream",
|
||||
"aws-smithy-http",
|
||||
"aws-smithy-runtime",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"aws-types",
|
||||
"bytes",
|
||||
"bytes-utils",
|
||||
"fastrand",
|
||||
"http 1.4.0",
|
||||
"http-body 1.0.1",
|
||||
"percent-encoding",
|
||||
"pin-project-lite",
|
||||
"tracing",
|
||||
"uuid",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-sdk-bedrockruntime"
|
||||
version = "1.127.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7dcd5ccbed3bd50d342077d3f731de46d9608340386c87d07566c4c507891eda"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-runtime",
|
||||
"aws-sigv4",
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-eventstream",
|
||||
"aws-smithy-http",
|
||||
"aws-smithy-json",
|
||||
"aws-smithy-observability",
|
||||
"aws-smithy-runtime",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"aws-types",
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"http-body-util",
|
||||
"regex-lite",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-sdk-sso"
|
||||
version = "1.96.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f64a6eded248c6b453966e915d32aeddb48ea63ad17932682774eb026fbef5b1"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-runtime",
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-http",
|
||||
"aws-smithy-json",
|
||||
"aws-smithy-observability",
|
||||
"aws-smithy-runtime",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"aws-types",
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"regex-lite",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-sdk-ssooidc"
|
||||
version = "1.98.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "db96d720d3c622fcbe08bae1c4b04a72ce6257d8b0584cb5418da00ae20a344f"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-runtime",
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-http",
|
||||
"aws-smithy-json",
|
||||
"aws-smithy-observability",
|
||||
"aws-smithy-runtime",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"aws-types",
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"regex-lite",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-sdk-sts"
|
||||
version = "1.100.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fafbdda43b93f57f699c5dfe8328db590b967b8a820a13ccdd6687355dfcc7ca"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-runtime",
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-http",
|
||||
"aws-smithy-json",
|
||||
"aws-smithy-observability",
|
||||
"aws-smithy-query",
|
||||
"aws-smithy-runtime",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"aws-smithy-xml",
|
||||
"aws-types",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"regex-lite",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-sigv4"
|
||||
version = "1.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b0b660013a6683ab23797778e21f1f854744fdf05f68204b4cca4c8c04b5d1f4"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-smithy-eventstream",
|
||||
"aws-smithy-http",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"bytes",
|
||||
"form_urlencoded",
|
||||
"hex",
|
||||
"hmac",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"percent-encoding",
|
||||
"sha2",
|
||||
"time",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-async"
|
||||
version = "1.2.14"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2ffcaf626bdda484571968400c326a244598634dc75fd451325a54ad1a59acfc"
|
||||
dependencies = [
|
||||
"futures-util",
|
||||
"pin-project-lite",
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-eventstream"
|
||||
version = "0.60.20"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "faf09d74e5e32f76b8762da505a3cd59303e367a664ca67295387baa8c1d7548"
|
||||
dependencies = [
|
||||
"aws-smithy-types",
|
||||
"bytes",
|
||||
"crc32fast",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-http"
|
||||
version = "0.63.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ba1ab2dc1c2c3749ead27180d333c42f11be8b0e934058fb4b2258ee8dbe5231"
|
||||
dependencies = [
|
||||
"aws-smithy-eventstream",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"bytes",
|
||||
"bytes-utils",
|
||||
"futures-core",
|
||||
"futures-util",
|
||||
"http 1.4.0",
|
||||
"http-body 1.0.1",
|
||||
"http-body-util",
|
||||
"percent-encoding",
|
||||
"pin-project-lite",
|
||||
"pin-utils",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-http-client"
|
||||
version = "1.1.12"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6a2f165a7feee6f263028b899d0a181987f4fa7179a6411a32a439fba7c5f769"
|
||||
dependencies = [
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"h2 0.3.27",
|
||||
"h2 0.4.13",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"http-body 0.4.6",
|
||||
"hyper 0.14.32",
|
||||
"hyper 1.8.1",
|
||||
"hyper-rustls 0.24.2",
|
||||
"hyper-rustls 0.27.7",
|
||||
"hyper-util",
|
||||
"pin-project-lite",
|
||||
"rustls 0.21.12",
|
||||
"rustls 0.23.37",
|
||||
"rustls-native-certs 0.8.3",
|
||||
"rustls-pki-types",
|
||||
"tokio",
|
||||
"tokio-rustls 0.26.4",
|
||||
"tower 0.5.3",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-json"
|
||||
version = "0.62.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9648b0bb82a2eedd844052c6ad2a1a822d1f8e3adee5fbf668366717e428856a"
|
||||
dependencies = [
|
||||
"aws-smithy-types",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-observability"
|
||||
version = "0.2.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a06c2315d173edbf1920da8ba3a7189695827002e4c0fc961973ab1c54abca9c"
|
||||
dependencies = [
|
||||
"aws-smithy-runtime-api",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-query"
|
||||
version = "0.60.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1a56d79744fb3edb5d722ef79d86081e121d3b9422cb209eb03aea6aa4f21ebd"
|
||||
dependencies = [
|
||||
"aws-smithy-types",
|
||||
"urlencoding",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-runtime"
|
||||
version = "1.10.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "028999056d2d2fd58a697232f9eec4a643cf73a71cf327690a7edad1d2af2110"
|
||||
dependencies = [
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-http",
|
||||
"aws-smithy-http-client",
|
||||
"aws-smithy-observability",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"http-body 0.4.6",
|
||||
"http-body 1.0.1",
|
||||
"http-body-util",
|
||||
"pin-project-lite",
|
||||
"pin-utils",
|
||||
"tokio",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-runtime-api"
|
||||
version = "1.11.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "876ab3c9c29791ba4ba02b780a3049e21ec63dabda09268b175272c3733a79e6"
|
||||
dependencies = [
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-types",
|
||||
"bytes",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"pin-project-lite",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-types"
|
||||
version = "1.4.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d2b1117b3b2bbe166d11199b540ceed0d0f7676e36e7b962b5a437a9971eac75"
|
||||
dependencies = [
|
||||
"base64-simd",
|
||||
"bytes",
|
||||
"bytes-utils",
|
||||
"futures-core",
|
||||
"http 0.2.12",
|
||||
"http 1.4.0",
|
||||
"http-body 0.4.6",
|
||||
"http-body 1.0.1",
|
||||
"http-body-util",
|
||||
"itoa",
|
||||
"num-integer",
|
||||
"pin-project-lite",
|
||||
"pin-utils",
|
||||
"ryu",
|
||||
"serde",
|
||||
"time",
|
||||
"tokio",
|
||||
"tokio-util",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-xml"
|
||||
version = "0.60.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0ce02add1aa3677d022f8adf81dcbe3046a95f17a1b1e8979c145cd21d3d22b3"
|
||||
dependencies = [
|
||||
"xmlparser",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-types"
|
||||
version = "1.3.14"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "47c8323699dd9b3c8d5b3c13051ae9cdef58fd179957c882f8374dd8725962d9"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"rustc_version",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "axum"
|
||||
version = "0.6.20"
|
||||
@@ -504,6 +910,16 @@ version = "0.22.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
|
||||
|
||||
[[package]]
|
||||
name = "base64-simd"
|
||||
version = "0.8.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "339abbe78e73178762e23bea9dfd08e697eb3f3301cd4be981c0f78ba5859195"
|
||||
dependencies = [
|
||||
"outref",
|
||||
"vsimd",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "base64ct"
|
||||
version = "1.8.3"
|
||||
@@ -730,6 +1146,16 @@ dependencies = [
|
||||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bytes-utils"
|
||||
version = "0.1.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7dafe3a8757b027e2be6e4e5601ed563c55989fcf1546e933c66c8eb3a058d35"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"either",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cap-fs-ext"
|
||||
version = "3.4.5"
|
||||
@@ -953,6 +1379,15 @@ dependencies = [
|
||||
"error-code",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cmake"
|
||||
version = "0.1.57"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "75443c44cd6b379beb8c5b45d85d0773baf31cce901fe7bb252f4eff3008ef7d"
|
||||
dependencies = [
|
||||
"cc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cobs"
|
||||
version = "0.3.0"
|
||||
@@ -1708,6 +2143,12 @@ dependencies = [
|
||||
"dtoa",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dunce"
|
||||
version = "1.0.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813"
|
||||
|
||||
[[package]]
|
||||
name = "dyn-clone"
|
||||
version = "1.0.20"
|
||||
@@ -2021,6 +2462,12 @@ dependencies = [
|
||||
"windows-sys 0.48.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fs_extra"
|
||||
version = "1.3.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c"
|
||||
|
||||
[[package]]
|
||||
name = "funty"
|
||||
version = "2.0.0"
|
||||
@@ -2568,6 +3015,21 @@ dependencies = [
|
||||
"winapi",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "hyper-rustls"
|
||||
version = "0.24.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ec3efd23720e2049821a693cbc7e65ea87c72f1c58ff2f9522ff332b1491e590"
|
||||
dependencies = [
|
||||
"futures-util",
|
||||
"http 0.2.12",
|
||||
"hyper 0.14.32",
|
||||
"log",
|
||||
"rustls 0.21.12",
|
||||
"tokio",
|
||||
"tokio-rustls 0.24.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "hyper-rustls"
|
||||
version = "0.25.0"
|
||||
@@ -2894,6 +3356,9 @@ dependencies = [
|
||||
"aho-corasick",
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
"aws-config",
|
||||
"aws-sdk-bedrockruntime",
|
||||
"aws-smithy-types",
|
||||
"axum 0.8.8",
|
||||
"base64 0.22.1",
|
||||
"blake3",
|
||||
@@ -3833,6 +4298,12 @@ dependencies = [
|
||||
"pin-project-lite",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "outref"
|
||||
version = "0.5.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1a80800c0488c3a21695ea981a54918fbb37abf04f4d0720c453632255e2ff0e"
|
||||
|
||||
[[package]]
|
||||
name = "parking"
|
||||
version = "2.2.1"
|
||||
@@ -4660,6 +5131,12 @@ dependencies = [
|
||||
"regex-syntax",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "regex-lite"
|
||||
version = "0.1.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cab834c73d247e67f4fae452806d17d3c7501756d98c8808d7c9c7aa7d18f973"
|
||||
|
||||
[[package]]
|
||||
name = "regex-syntax"
|
||||
version = "0.8.10"
|
||||
@@ -4888,6 +5365,18 @@ dependencies = [
|
||||
"rustix 1.1.4",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustls"
|
||||
version = "0.21.12"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3f56a14d1f48b391359b22f731fd4bd7e43c97f3c50eee276f3aa09c94784d3e"
|
||||
dependencies = [
|
||||
"log",
|
||||
"ring",
|
||||
"rustls-webpki 0.101.7",
|
||||
"sct",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustls"
|
||||
version = "0.22.4"
|
||||
@@ -4908,6 +5397,7 @@ version = "0.23.37"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4"
|
||||
dependencies = [
|
||||
"aws-lc-rs",
|
||||
"once_cell",
|
||||
"ring",
|
||||
"rustls-pki-types",
|
||||
@@ -4960,6 +5450,16 @@ dependencies = [
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustls-webpki"
|
||||
version = "0.101.7"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8b6275d1ee7a1cd780b64aca7726599a1dbc893b1e64144529e55c3c2f745765"
|
||||
dependencies = [
|
||||
"ring",
|
||||
"untrusted",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rustls-webpki"
|
||||
version = "0.102.8"
|
||||
@@ -4977,6 +5477,7 @@ version = "0.103.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d7df23109aa6c1567d1c575b9952556388da57401e4ace1d15f79eedad0d8f53"
|
||||
dependencies = [
|
||||
"aws-lc-rs",
|
||||
"ring",
|
||||
"rustls-pki-types",
|
||||
"untrusted",
|
||||
@@ -5104,6 +5605,16 @@ dependencies = [
|
||||
"tendril 0.4.3",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sct"
|
||||
version = "0.7.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "da046153aa2352493d6cb7da4b6e5c0c057d8a1d0a9aa8560baffdd945acd414"
|
||||
dependencies = [
|
||||
"ring",
|
||||
"untrusted",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "seahash"
|
||||
version = "4.1.0"
|
||||
@@ -6060,6 +6571,16 @@ dependencies = [
|
||||
"x509-cert",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tokio-rustls"
|
||||
version = "0.24.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c28327cf380ac148141087fbfb9de9d7bd4e84ab5d2c28fbc911d753de8a7081"
|
||||
dependencies = [
|
||||
"rustls 0.21.12",
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tokio-rustls"
|
||||
version = "0.25.0"
|
||||
@@ -6697,6 +7218,12 @@ version = "0.9.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
|
||||
|
||||
[[package]]
|
||||
name = "vsimd"
|
||||
version = "0.8.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5c3082ca00d5a5ef149bb8b555a72ae84c9c59f7250f013ac822ac2e49b19c64"
|
||||
|
||||
[[package]]
|
||||
name = "walkdir"
|
||||
version = "2.5.0"
|
||||
@@ -7893,6 +8420,12 @@ dependencies = [
|
||||
"windows-sys 0.59.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "xmlparser"
|
||||
version = "0.13.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "66fee0b777b0f5ac1c69bb06d361268faafa61cd4682ae064a171c16c433e9e4"
|
||||
|
||||
[[package]]
|
||||
name = "yansi"
|
||||
version = "1.0.1"
|
||||
|
||||
@@ -142,6 +142,11 @@ subtle = "2" # Constant-time comparisons for token validation
|
||||
# Multi-provider LLM support
|
||||
rig-core = "0.30"
|
||||
|
||||
# AWS Bedrock (native Converse API, opt-in via --features bedrock)
|
||||
aws-config = { version = "1", features = ["behavior-version-latest"], optional = true }
|
||||
aws-sdk-bedrockruntime = { version = "1", optional = true }
|
||||
aws-smithy-types = { version = "1", optional = true }
|
||||
|
||||
# Docker sandbox
|
||||
bollard = "0.18"
|
||||
|
||||
@@ -203,6 +208,7 @@ postgres = [
|
||||
libsql = ["dep:libsql"]
|
||||
integration = []
|
||||
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
|
||||
bedrock = ["dep:aws-config", "dep:aws-sdk-bedrockruntime", "dep:aws-smithy-types"]
|
||||
|
||||
[[test]]
|
||||
name = "html_to_markdown"
|
||||
|
||||
+1
-1
@@ -215,7 +215,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
|
||||
| NEAR AI | ✅ | ✅ | - | Primary provider |
|
||||
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6 |
|
||||
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy |
|
||||
| AWS Bedrock | ✅ | ✅ | P3 | Via `openai_compatible` adapter (e.g. LiteLLM) |
|
||||
| AWS Bedrock | ✅ | ✅ | - | Native Converse API via aws-sdk-bedrockruntime (requires `--features bedrock`) |
|
||||
| Google Gemini | ✅ | ✅ | P3 | Via `gemini` adapter |
|
||||
| io.net | ✅ | ✅ | P3 | Via `ionet` adapter |
|
||||
| Mistral | ✅ | ✅ | P3 | Via `mistral` adapter |
|
||||
|
||||
+50
-1
@@ -12,12 +12,12 @@ configurations.
|
||||
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
|
||||
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
|
||||
| Google Gemini | `gemini` | `GEMINI_API_KEY` | Gemini models |
|
||||
| AWS Bedrock | `bedrock` | `BEDROCK_ACCESS_KEY` | Requires OpenAI proxy (e.g. LiteLLM) |
|
||||
| io.net | `ionet` | `IONET_API_KEY` | Intelligence API |
|
||||
| Mistral | `mistral` | `MISTRAL_API_KEY` | Mistral models |
|
||||
| Yandex AI Studio | `yandex` | `YANDEX_API_KEY` | YandexGPT models |
|
||||
| Cloudflare Workers AI | `cloudflare` | `CLOUDFLARE_API_KEY` | Access to Workers AI |
|
||||
| Ollama | `ollama` | No | Local inference |
|
||||
| AWS Bedrock | `bedrock` | AWS credentials | Native Converse API |
|
||||
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
|
||||
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
|
||||
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
|
||||
@@ -74,6 +74,55 @@ Pull a model first: `ollama pull llama3.2`
|
||||
|
||||
---
|
||||
|
||||
## AWS Bedrock (requires `--features bedrock`)
|
||||
|
||||
Uses the native AWS Converse API via `aws-sdk-bedrockruntime`. Supports standard AWS
|
||||
authentication methods: IAM credentials, SSO profiles, and instance roles.
|
||||
|
||||
> **Build prerequisite:** The `aws-lc-sys` crate (transitive dependency via AWS SDK)
|
||||
> requires **CMake** to compile. Install it before building with `--features bedrock`:
|
||||
> - macOS: `brew install cmake`
|
||||
> - Ubuntu/Debian: `sudo apt install cmake`
|
||||
> - Fedora: `sudo dnf install cmake`
|
||||
|
||||
### With AWS credentials (IAM, SSO, instance roles)
|
||||
|
||||
```env
|
||||
LLM_BACKEND=bedrock
|
||||
BEDROCK_MODEL=anthropic.claude-opus-4-6-v1
|
||||
BEDROCK_REGION=us-east-1
|
||||
BEDROCK_CROSS_REGION=us
|
||||
# AWS_PROFILE=my-sso-profile # optional, for named profiles
|
||||
```
|
||||
|
||||
The AWS SDK credential chain automatically resolves credentials from environment
|
||||
variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`), shared credentials file
|
||||
(`~/.aws/credentials`), SSO profiles, and EC2/ECS instance roles.
|
||||
|
||||
### Cross-region inference
|
||||
|
||||
Set `BEDROCK_CROSS_REGION` to route requests across AWS regions for capacity:
|
||||
|
||||
| Prefix | Routing |
|
||||
|---|---|
|
||||
| `us` | US regions (us-east-1, us-east-2, us-west-2) |
|
||||
| `eu` | European regions |
|
||||
| `apac` | Asia-Pacific regions |
|
||||
| `global` | All commercial AWS regions |
|
||||
| _(unset)_ | Single-region only |
|
||||
|
||||
### Popular Bedrock model IDs
|
||||
|
||||
| Model | ID |
|
||||
|---|---|
|
||||
| Claude Opus 4.6 | `anthropic.claude-opus-4-6-v1` |
|
||||
| Claude Sonnet 4.5 | `anthropic.claude-sonnet-4-5-20250929-v1:0` |
|
||||
| Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` |
|
||||
| Amazon Nova Pro | `amazon.nova-pro-v1:0` |
|
||||
| Llama 4 Maverick | `meta.llama4-maverick-17b-instruct-v1:0` |
|
||||
|
||||
---
|
||||
|
||||
## OpenAI-Compatible Endpoints
|
||||
|
||||
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
|
||||
|
||||
@@ -295,26 +295,6 @@
|
||||
"can_list_models": true
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "bedrock",
|
||||
"aliases": [
|
||||
"aws_bedrock",
|
||||
"aws"
|
||||
],
|
||||
"protocol": "open_ai_completions",
|
||||
"api_key_env": "BEDROCK_ACCESS_KEY",
|
||||
"api_key_required": false,
|
||||
"base_url_env": "BEDROCK_BASE_URL",
|
||||
"model_env": "BEDROCK_MODEL",
|
||||
"default_model": "anthropic.claude-3-5-sonnet-20241022-v2:0",
|
||||
"description": "AWS Bedrock (requires LiteLLM or OpenAI-compatible proxy)",
|
||||
"setup": {
|
||||
"kind": "open_ai_compatible",
|
||||
"secret_name": "llm_bedrock_api_key",
|
||||
"display_name": "AWS Bedrock",
|
||||
"can_list_models": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "ionet",
|
||||
"aliases": [
|
||||
|
||||
+3
-3
@@ -334,7 +334,7 @@ impl AppBuilder {
|
||||
/// Delegates to `build_provider_chain` which applies all decorators
|
||||
/// (retry, smart routing, failover, circuit breaker, response cache).
|
||||
#[allow(clippy::type_complexity)]
|
||||
pub fn init_llm(
|
||||
pub async fn init_llm(
|
||||
&self,
|
||||
) -> Result<
|
||||
(
|
||||
@@ -345,7 +345,7 @@ impl AppBuilder {
|
||||
anyhow::Error,
|
||||
> {
|
||||
let (llm, cheap_llm, recording_handle) =
|
||||
crate::llm::build_provider_chain(&self.config.llm, self.session.clone())?;
|
||||
crate::llm::build_provider_chain(&self.config.llm, self.session.clone()).await?;
|
||||
Ok((llm, cheap_llm, recording_handle))
|
||||
}
|
||||
|
||||
@@ -820,7 +820,7 @@ impl AppBuilder {
|
||||
let (llm, cheap_llm, recording_handle) = if let Some(llm) = self.llm_override.take() {
|
||||
(llm, None, None)
|
||||
} else {
|
||||
self.init_llm()?
|
||||
self.init_llm().await?
|
||||
};
|
||||
let (safety, tools, embeddings, workspace) = self.init_tools(&llm).await?;
|
||||
|
||||
|
||||
+62
-4
@@ -86,6 +86,19 @@ pub struct RegistryProviderConfig {
|
||||
pub oauth_token: Option<SecretString>,
|
||||
}
|
||||
|
||||
/// Configuration for AWS Bedrock (native Converse API).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct BedrockConfig {
|
||||
/// AWS region (e.g. "us-east-1").
|
||||
pub region: String,
|
||||
/// Bedrock model ID (e.g. "anthropic.claude-opus-4-6-v1").
|
||||
pub model: String,
|
||||
/// Cross-region inference prefix: "us", "eu", "apac", "global", or None.
|
||||
pub cross_region: Option<String>,
|
||||
/// AWS named profile (for SSO / assume-role workflows).
|
||||
pub profile: Option<String>,
|
||||
}
|
||||
|
||||
/// LLM provider configuration.
|
||||
///
|
||||
/// NearAI remains the default backend with its own config struct (session auth).
|
||||
@@ -101,8 +114,10 @@ pub struct LlmConfig {
|
||||
/// NEAR AI config (always populated, also used for embeddings).
|
||||
pub nearai: NearAiConfig,
|
||||
/// Resolved provider config for registry-based providers.
|
||||
/// `None` when backend is "nearai".
|
||||
/// `None` when backend is "nearai" or "bedrock".
|
||||
pub provider: Option<RegistryProviderConfig>,
|
||||
/// AWS Bedrock config (populated when backend=bedrock, requires --features bedrock).
|
||||
pub bedrock: Option<BedrockConfig>,
|
||||
/// HTTP request timeout in seconds for LLM API calls.
|
||||
/// Default: 120. Increase for local LLMs (Ollama, vLLM, LM Studio) that
|
||||
/// need more time for prompt evaluation on consumer hardware.
|
||||
@@ -169,6 +184,7 @@ impl LlmConfig {
|
||||
smart_routing_cascade: false,
|
||||
},
|
||||
provider: None,
|
||||
bedrock: None,
|
||||
request_timeout_secs: 120,
|
||||
}
|
||||
}
|
||||
@@ -200,8 +216,10 @@ impl LlmConfig {
|
||||
let backend_lower = backend.to_lowercase();
|
||||
let is_nearai =
|
||||
backend_lower == "nearai" || backend_lower == "near_ai" || backend_lower == "near";
|
||||
let is_bedrock =
|
||||
backend_lower == "bedrock" || backend_lower == "aws_bedrock" || backend_lower == "aws";
|
||||
|
||||
if !is_nearai && registry.find(&backend_lower).is_none() {
|
||||
if !is_nearai && !is_bedrock && registry.find(&backend_lower).is_none() {
|
||||
tracing::warn!(
|
||||
"Unknown LLM backend '{}'. Will attempt as openai_compatible fallback.",
|
||||
backend
|
||||
@@ -248,8 +266,8 @@ impl LlmConfig {
|
||||
smart_routing_cascade: parse_optional_env("SMART_ROUTING_CASCADE", true)?,
|
||||
};
|
||||
|
||||
// Resolve registry provider config (for non-NearAI backends)
|
||||
let provider = if is_nearai {
|
||||
// Resolve registry provider config (for non-NearAI, non-Bedrock backends)
|
||||
let provider = if is_nearai || is_bedrock {
|
||||
None
|
||||
} else {
|
||||
Some(Self::resolve_registry_provider(
|
||||
@@ -259,11 +277,50 @@ impl LlmConfig {
|
||||
)?)
|
||||
};
|
||||
|
||||
let bedrock = if is_bedrock {
|
||||
let explicit_region =
|
||||
optional_env("BEDROCK_REGION")?.or_else(|| settings.bedrock_region.clone());
|
||||
if explicit_region.is_none() {
|
||||
tracing::info!("BEDROCK_REGION not set, defaulting to us-east-1");
|
||||
}
|
||||
let region = explicit_region.unwrap_or_else(|| "us-east-1".to_string());
|
||||
let model = optional_env("BEDROCK_MODEL")?
|
||||
.or_else(|| settings.selected_model.clone())
|
||||
.ok_or_else(|| ConfigError::MissingRequired {
|
||||
key: "BEDROCK_MODEL".to_string(),
|
||||
hint: "Set BEDROCK_MODEL when LLM_BACKEND=bedrock".to_string(),
|
||||
})?;
|
||||
let cross_region = optional_env("BEDROCK_CROSS_REGION")?
|
||||
.or_else(|| settings.bedrock_cross_region.clone());
|
||||
if let Some(ref cr) = cross_region
|
||||
&& !matches!(cr.as_str(), "us" | "eu" | "apac" | "global")
|
||||
{
|
||||
return Err(ConfigError::InvalidValue {
|
||||
key: "BEDROCK_CROSS_REGION".to_string(),
|
||||
message: format!(
|
||||
"'{}' is not valid, expected one of: us, eu, apac, global",
|
||||
cr
|
||||
),
|
||||
});
|
||||
}
|
||||
let profile = optional_env("AWS_PROFILE")?.or_else(|| settings.bedrock_profile.clone());
|
||||
Some(BedrockConfig {
|
||||
region,
|
||||
model,
|
||||
cross_region,
|
||||
profile,
|
||||
})
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let request_timeout_secs = parse_optional_env("LLM_REQUEST_TIMEOUT_SECS", 120)?;
|
||||
|
||||
Ok(Self {
|
||||
backend: if is_nearai {
|
||||
"nearai".to_string()
|
||||
} else if is_bedrock {
|
||||
"bedrock".to_string()
|
||||
} else if let Some(ref p) = provider {
|
||||
p.provider_id.clone()
|
||||
} else {
|
||||
@@ -272,6 +329,7 @@ impl LlmConfig {
|
||||
session,
|
||||
nearai,
|
||||
provider,
|
||||
bedrock,
|
||||
request_timeout_secs,
|
||||
})
|
||||
}
|
||||
|
||||
+3
-1
@@ -37,7 +37,9 @@ pub use self::database::{DatabaseBackend, DatabaseConfig, SslMode, default_libsq
|
||||
pub use self::embeddings::EmbeddingsConfig;
|
||||
pub use self::heartbeat::HeartbeatConfig;
|
||||
pub use self::hygiene::HygieneConfig;
|
||||
pub use self::llm::{CacheRetention, LlmConfig, NearAiConfig, RegistryProviderConfig};
|
||||
pub use self::llm::{
|
||||
BedrockConfig, CacheRetention, LlmConfig, NearAiConfig, RegistryProviderConfig,
|
||||
};
|
||||
pub use self::routines::RoutineConfig;
|
||||
pub use self::safety::SafetyConfig;
|
||||
pub use self::sandbox::{ClaudeCodeConfig, SandboxModeConfig};
|
||||
|
||||
+1148
File diff suppressed because it is too large
Load Diff
+40
-3
@@ -6,8 +6,11 @@
|
||||
//! - **Anthropic**: Direct API access with your own key
|
||||
//! - **Ollama**: Local model inference
|
||||
//! - **OpenAI-compatible**: Any endpoint that speaks the OpenAI API
|
||||
//! - **AWS Bedrock**: Native Converse API via aws-sdk-bedrockruntime
|
||||
|
||||
mod anthropic_oauth;
|
||||
#[cfg(feature = "bedrock")]
|
||||
mod bedrock;
|
||||
pub mod circuit_breaker;
|
||||
pub mod costs;
|
||||
pub mod failover;
|
||||
@@ -57,7 +60,7 @@ use crate::error::LlmError;
|
||||
///
|
||||
/// - NearAI backend: Uses session manager for authentication
|
||||
/// - Registry providers: Looked up by protocol and constructed generically
|
||||
pub fn create_llm_provider(
|
||||
pub async fn create_llm_provider(
|
||||
config: &LlmConfig,
|
||||
session: Arc<SessionManager>,
|
||||
) -> Result<Arc<dyn LlmProvider>, LlmError> {
|
||||
@@ -67,6 +70,21 @@ pub fn create_llm_provider(
|
||||
return create_llm_provider_with_config(&config.nearai, session, timeout);
|
||||
}
|
||||
|
||||
// Bedrock uses a native AWS SDK, not the rig-core registry
|
||||
if config.backend == "bedrock" {
|
||||
#[cfg(feature = "bedrock")]
|
||||
{
|
||||
return create_bedrock_provider(config).await;
|
||||
}
|
||||
#[cfg(not(feature = "bedrock"))]
|
||||
{
|
||||
return Err(LlmError::RequestFailed {
|
||||
provider: "bedrock".to_string(),
|
||||
reason: "Bedrock support not compiled. Rebuild with --features bedrock".to_string(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
let reg_config = config
|
||||
.provider
|
||||
.as_ref()
|
||||
@@ -120,6 +138,24 @@ fn create_registry_provider(
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "bedrock")]
|
||||
async fn create_bedrock_provider(config: &LlmConfig) -> Result<Arc<dyn LlmProvider>, LlmError> {
|
||||
let br = config
|
||||
.bedrock
|
||||
.as_ref()
|
||||
.ok_or_else(|| LlmError::AuthFailed {
|
||||
provider: "bedrock".to_string(),
|
||||
})?;
|
||||
|
||||
let provider = bedrock::BedrockProvider::new(br).await?;
|
||||
tracing::info!(
|
||||
"Using AWS Bedrock (Converse API, region: {}, model: {})",
|
||||
br.region,
|
||||
provider.active_model_name(),
|
||||
);
|
||||
Ok(Arc::new(provider))
|
||||
}
|
||||
|
||||
fn create_openai_compat_from_registry(
|
||||
config: &RegistryProviderConfig,
|
||||
) -> Result<Arc<dyn LlmProvider>, LlmError> {
|
||||
@@ -344,7 +380,7 @@ pub fn create_cheap_llm_provider(
|
||||
/// This is the single source of truth for provider chain construction,
|
||||
/// called by both `main.rs` and `app.rs`.
|
||||
#[allow(clippy::type_complexity)]
|
||||
pub fn build_provider_chain(
|
||||
pub async fn build_provider_chain(
|
||||
config: &LlmConfig,
|
||||
session: Arc<SessionManager>,
|
||||
) -> Result<
|
||||
@@ -355,7 +391,7 @@ pub fn build_provider_chain(
|
||||
),
|
||||
LlmError,
|
||||
> {
|
||||
let llm = create_llm_provider(config, session.clone())?;
|
||||
let llm = create_llm_provider(config, session.clone()).await?;
|
||||
tracing::info!("LLM provider initialized: {}", llm.model_name());
|
||||
|
||||
// 1. Retry
|
||||
@@ -522,6 +558,7 @@ mod tests {
|
||||
session: SessionConfig::default(),
|
||||
nearai: test_nearai_config(),
|
||||
provider: None,
|
||||
bedrock: None,
|
||||
request_timeout_secs: 120,
|
||||
}
|
||||
}
|
||||
|
||||
+13
-1
@@ -47,7 +47,7 @@ pub struct Settings {
|
||||
pub secrets_master_key_hex: Option<String>,
|
||||
|
||||
// === Step 3: Inference Provider ===
|
||||
/// LLM backend: "nearai", "anthropic", "openai", "ollama", "openai_compatible".
|
||||
/// LLM backend: "nearai", "anthropic", "openai", "ollama", "openai_compatible", "tinfoil", "bedrock".
|
||||
#[serde(default)]
|
||||
pub llm_backend: Option<String>,
|
||||
|
||||
@@ -59,6 +59,18 @@ pub struct Settings {
|
||||
#[serde(default)]
|
||||
pub openai_compatible_base_url: Option<String>,
|
||||
|
||||
/// Bedrock region (when llm_backend = "bedrock").
|
||||
#[serde(default)]
|
||||
pub bedrock_region: Option<String>,
|
||||
|
||||
/// Bedrock cross-region inference prefix (when llm_backend = "bedrock").
|
||||
#[serde(default)]
|
||||
pub bedrock_cross_region: Option<String>,
|
||||
|
||||
/// AWS profile name for Bedrock (when llm_backend = "bedrock").
|
||||
#[serde(default)]
|
||||
pub bedrock_profile: Option<String>,
|
||||
|
||||
// === Step 4: Model Selection ===
|
||||
/// Currently selected model.
|
||||
#[serde(default)]
|
||||
|
||||
+2
-1
@@ -174,6 +174,7 @@ env-var mode or skipped secrets.
|
||||
| Ollama | None | - | - |
|
||||
| OpenRouter¹ | API key | `llm_compatible_api_key` | `LLM_API_KEY` |
|
||||
| OpenAI-compatible¹ | Optional API key | `llm_compatible_api_key` | `LLM_API_KEY` |
|
||||
| AWS Bedrock | AWS credentials (IAM, SSO, instance roles) | - | - |
|
||||
|
||||
¹ OpenRouter and OpenAI-compatible share the same secret name and env var because
|
||||
OpenRouter is stored as `llm_backend = "openai_compatible"` under the hood.
|
||||
@@ -479,7 +480,7 @@ pub struct Settings {
|
||||
pub secrets_master_key_source: KeySource, // Keychain | Env | None
|
||||
|
||||
// Step 3: Inference
|
||||
pub llm_backend: Option<String>, // "nearai" | "anthropic" | "openai" | "ollama" | "openai_compatible"
|
||||
pub llm_backend: Option<String>, // "nearai" | "anthropic" | "openai" | "ollama" | "openai_compatible" | "bedrock"
|
||||
pub ollama_base_url: Option<String>,
|
||||
pub openai_compatible_base_url: Option<String>,
|
||||
|
||||
|
||||
+198
-7
@@ -827,9 +827,16 @@ impl SetupWizard {
|
||||
print_info(&format!("Current provider: {}", display));
|
||||
println!();
|
||||
|
||||
let is_known = current == "nearai" || registry.is_known(¤t);
|
||||
let is_known =
|
||||
current == "nearai" || current == "bedrock" || registry.is_known(¤t);
|
||||
|
||||
if is_known && confirm("Keep current provider?", true).map_err(SetupError::Io)? {
|
||||
if current == "bedrock" {
|
||||
// Keeping the existing Bedrock config — no need to re-run
|
||||
// the full setup flow (region, auth, cross-region).
|
||||
print_info("Keeping existing AWS Bedrock configuration.");
|
||||
return Ok(());
|
||||
}
|
||||
return self.run_provider_setup(¤t, ®istry).await;
|
||||
}
|
||||
|
||||
@@ -844,10 +851,10 @@ impl SetupWizard {
|
||||
print_info("Select your inference provider:");
|
||||
println!();
|
||||
|
||||
// Build menu: NearAI first, then all registry providers with setup hints
|
||||
// Build menu: NearAI first, then all registry providers with setup hints, then Bedrock
|
||||
let selectable = registry.selectable();
|
||||
let mut options: Vec<String> = Vec::with_capacity(1 + selectable.len());
|
||||
let mut provider_ids: Vec<String> = Vec::with_capacity(1 + selectable.len());
|
||||
let mut options: Vec<String> = Vec::with_capacity(2 + selectable.len());
|
||||
let mut provider_ids: Vec<String> = Vec::with_capacity(2 + selectable.len());
|
||||
|
||||
options.push("NEAR AI - multi-model access via NEAR account".to_string());
|
||||
provider_ids.push("nearai".to_string());
|
||||
@@ -865,11 +872,19 @@ impl SetupWizard {
|
||||
provider_ids.push(def.id.clone());
|
||||
}
|
||||
|
||||
// Bedrock is a special case (native AWS SDK, not registry-based)
|
||||
options.push("AWS Bedrock - Claude & other models via AWS (IAM, SSO)".to_string());
|
||||
provider_ids.push("bedrock".to_string());
|
||||
|
||||
let option_refs: Vec<&str> = options.iter().map(|s| s.as_str()).collect();
|
||||
let choice = select_one("Provider:", &option_refs).map_err(SetupError::Io)?;
|
||||
let selected_id = &provider_ids[choice];
|
||||
|
||||
self.run_provider_setup(selected_id, ®istry).await?;
|
||||
if selected_id == "bedrock" {
|
||||
self.setup_bedrock().await?;
|
||||
} else {
|
||||
self.run_provider_setup(selected_id, ®istry).await?;
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -1230,6 +1245,95 @@ impl SetupWizard {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// AWS Bedrock provider setup: region, auth, and cross-region config.
|
||||
async fn setup_bedrock(&mut self) -> Result<(), SetupError> {
|
||||
if self.settings.llm_backend.as_deref() != Some("bedrock") {
|
||||
self.settings.selected_model = None;
|
||||
}
|
||||
self.settings.llm_backend = Some("bedrock".to_string());
|
||||
|
||||
// Region
|
||||
let default_region = self
|
||||
.settings
|
||||
.bedrock_region
|
||||
.as_deref()
|
||||
.unwrap_or("us-east-1");
|
||||
|
||||
let region_input =
|
||||
optional_input("AWS region", Some(&format!("default: {}", default_region)))
|
||||
.map_err(SetupError::Io)?;
|
||||
|
||||
let region = region_input.unwrap_or_else(|| default_region.to_string());
|
||||
self.settings.bedrock_region = Some(region.clone());
|
||||
|
||||
// Auth method
|
||||
print_info("Select authentication method:");
|
||||
println!();
|
||||
let auth_options = &[
|
||||
"AWS default credentials (env vars, ~/.aws/credentials, IAM roles)",
|
||||
"AWS named profile (SSO / assume-role)",
|
||||
];
|
||||
let auth_choice = select_one("Auth:", auth_options).map_err(SetupError::Io)?;
|
||||
|
||||
match auth_choice {
|
||||
0 => {
|
||||
// Default AWS credentials — clear any stale named profile
|
||||
self.settings.bedrock_profile = None;
|
||||
print_info(
|
||||
"Using default AWS credential chain (env vars, ~/.aws/credentials, IAM roles).",
|
||||
);
|
||||
}
|
||||
1 => {
|
||||
// Named profile
|
||||
let profile =
|
||||
input("AWS profile name (from ~/.aws/config)").map_err(SetupError::Io)?;
|
||||
if profile.trim().is_empty() {
|
||||
// Empty input clears any previously configured profile
|
||||
self.settings.bedrock_profile = None;
|
||||
print_info("AWS profile cleared; using default AWS credential chain instead.");
|
||||
} else {
|
||||
self.settings.bedrock_profile = Some(profile.clone());
|
||||
print_success(&format!("AWS profile '{}' saved", profile));
|
||||
}
|
||||
}
|
||||
_ => return Err(SetupError::Config("Invalid auth selection".to_string())),
|
||||
}
|
||||
|
||||
self.setup_bedrock_cross_region()
|
||||
}
|
||||
|
||||
/// Bedrock cross-region inference prefix selection (sub-step of setup_bedrock).
|
||||
fn setup_bedrock_cross_region(&mut self) -> Result<(), SetupError> {
|
||||
print_info("Cross-region inference routes requests across AWS regions for capacity:");
|
||||
println!();
|
||||
let cross_options = &[
|
||||
"us - route within US regions (recommended for us-east-1)",
|
||||
"global - route to any AWS region worldwide",
|
||||
"eu - route within European regions",
|
||||
"apac - route within Asia-Pacific regions",
|
||||
"none - single-region only (no cross-region routing)",
|
||||
];
|
||||
let cross_choice = select_one("Cross-region:", cross_options).map_err(SetupError::Io)?;
|
||||
|
||||
let cross_region = match cross_choice {
|
||||
0 => Some("us".to_string()),
|
||||
1 => Some("global".to_string()),
|
||||
2 => Some("eu".to_string()),
|
||||
3 => Some("apac".to_string()),
|
||||
4 => None,
|
||||
_ => None,
|
||||
};
|
||||
self.settings.bedrock_cross_region = cross_region;
|
||||
|
||||
let region = self
|
||||
.settings
|
||||
.bedrock_region
|
||||
.as_deref()
|
||||
.unwrap_or("us-east-1");
|
||||
print_success(&format!("AWS Bedrock configured (region: {})", region));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Generic OpenAI-compatible setup: base URL + optional API key.
|
||||
async fn setup_openai_compatible_generic(
|
||||
&mut self,
|
||||
@@ -1412,6 +1516,14 @@ impl SetupWizard {
|
||||
self.settings.selected_model = Some(model_id.clone());
|
||||
print_success(&format!("Selected {}", model_id));
|
||||
}
|
||||
} else if backend == "bedrock" {
|
||||
let model_id = input("Bedrock model ID (e.g., anthropic.claude-opus-4-6-v1)")
|
||||
.map_err(SetupError::Io)?;
|
||||
if model_id.is_empty() {
|
||||
return Err(SetupError::Config("Model ID is required".to_string()));
|
||||
}
|
||||
self.settings.selected_model = Some(model_id.clone());
|
||||
print_success(&format!("Selected {}", model_id));
|
||||
} else {
|
||||
// Unknown provider, manual entry
|
||||
let model_id = input("Model name (e.g., meta-llama/Llama-3-8b-chat-hf)")
|
||||
@@ -1495,10 +1607,11 @@ impl SetupWizard {
|
||||
smart_routing_cascade: true,
|
||||
},
|
||||
provider: None,
|
||||
bedrock: None,
|
||||
request_timeout_secs: 120,
|
||||
};
|
||||
|
||||
match create_llm_provider(&config, session) {
|
||||
match create_llm_provider(&config, session).await {
|
||||
Ok(provider) => match provider.list_models().await {
|
||||
Ok(models) => models,
|
||||
Err(e) => {
|
||||
@@ -2315,12 +2428,29 @@ impl SetupWizard {
|
||||
if let Some(ref url) = self.settings.ollama_base_url {
|
||||
env_vars.push(("OLLAMA_BASE_URL".to_string(), url.clone()));
|
||||
}
|
||||
if let Some(ref region) = self.settings.bedrock_region {
|
||||
env_vars.push(("BEDROCK_REGION".to_string(), region.clone()));
|
||||
}
|
||||
if self.settings.llm_backend.as_deref() == Some("bedrock") {
|
||||
if let Some(ref model) = self.settings.selected_model {
|
||||
env_vars.push(("BEDROCK_MODEL".to_string(), model.clone()));
|
||||
}
|
||||
if let Some(ref cross) = self.settings.bedrock_cross_region {
|
||||
env_vars.push(("BEDROCK_CROSS_REGION".to_string(), cross.clone()));
|
||||
}
|
||||
if let Some(ref profile) = self.settings.bedrock_profile {
|
||||
env_vars.push(("AWS_PROFILE".to_string(), profile.clone()));
|
||||
}
|
||||
}
|
||||
|
||||
// Model name: same chicken-and-egg — Config::from_env() resolves the
|
||||
// model before the DB is connected, so we must persist it to .env.
|
||||
// Write the backend-specific env var so the correct resolution path
|
||||
// picks it up (looked up from the provider registry).
|
||||
if let Some(ref model) = self.settings.selected_model {
|
||||
// Bedrock model is already written above as BEDROCK_MODEL, skip here.
|
||||
if self.settings.llm_backend.as_deref() != Some("bedrock")
|
||||
&& let Some(ref model) = self.settings.selected_model
|
||||
{
|
||||
let backend_str = self.settings.llm_backend.as_deref().unwrap_or("nearai");
|
||||
let model_env = registry.model_env_var(backend_str);
|
||||
env_vars.push((model_env.to_string(), model.clone()));
|
||||
@@ -2605,6 +2735,7 @@ impl SetupWizard {
|
||||
"openai" => "OpenAI",
|
||||
"ollama" => "Ollama",
|
||||
"openai_compatible" => "OpenAI-compatible",
|
||||
"bedrock" => "AWS Bedrock",
|
||||
other => other,
|
||||
};
|
||||
println!(" Provider: {}", display);
|
||||
@@ -3569,6 +3700,66 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// Regression: Bedrock setup_bedrock() should preserve selected_model
|
||||
/// when re-entering the same provider (matches pattern from #600).
|
||||
#[test]
|
||||
fn test_bedrock_same_provider_preserves_model() {
|
||||
let mut wizard = SetupWizard::new();
|
||||
wizard.settings.llm_backend = Some("bedrock".to_string());
|
||||
wizard.settings.selected_model = Some("anthropic.claude-opus-4-6-v1".to_string());
|
||||
|
||||
// Simulate the conditional clearing logic from setup_bedrock()
|
||||
if wizard.settings.llm_backend.as_deref() != Some("bedrock") {
|
||||
wizard.settings.selected_model = None;
|
||||
}
|
||||
wizard.settings.llm_backend = Some("bedrock".to_string());
|
||||
|
||||
assert_eq!(
|
||||
wizard.settings.selected_model.as_deref(),
|
||||
Some("anthropic.claude-opus-4-6-v1"),
|
||||
"bedrock model should be preserved when re-selecting bedrock"
|
||||
);
|
||||
}
|
||||
|
||||
/// Regression: switching from another provider to bedrock must clear
|
||||
/// selected_model, and choosing "default credentials" must clear
|
||||
/// bedrock_profile.
|
||||
#[test]
|
||||
fn test_bedrock_clears_stale_profile_on_default_creds() {
|
||||
let mut wizard = SetupWizard::new();
|
||||
wizard.settings.llm_backend = Some("bedrock".to_string());
|
||||
wizard.settings.bedrock_profile = Some("old-sso-profile".to_string());
|
||||
|
||||
// Simulate auth_choice == 0 (default credentials) clearing the profile
|
||||
wizard.settings.bedrock_profile = None;
|
||||
|
||||
assert!(
|
||||
wizard.settings.bedrock_profile.is_none(),
|
||||
"bedrock_profile should be cleared when selecting default credentials"
|
||||
);
|
||||
}
|
||||
|
||||
/// Regression: empty profile input in named-profile auth should clear
|
||||
/// any previously configured profile instead of leaving it stale.
|
||||
#[test]
|
||||
fn test_bedrock_empty_profile_clears_existing() {
|
||||
let mut wizard = SetupWizard::new();
|
||||
wizard.settings.bedrock_profile = Some("old-profile".to_string());
|
||||
|
||||
// Simulate auth_choice == 1 with empty input
|
||||
let profile = "".to_string();
|
||||
if profile.trim().is_empty() {
|
||||
wizard.settings.bedrock_profile = None;
|
||||
} else {
|
||||
wizard.settings.bedrock_profile = Some(profile);
|
||||
}
|
||||
|
||||
assert!(
|
||||
wizard.settings.bedrock_profile.is_none(),
|
||||
"empty profile input should clear existing bedrock_profile"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_run_provider_setup_no_setup_hint() {
|
||||
// A provider with setup: None should not error. It should set the
|
||||
|
||||
@@ -84,7 +84,9 @@ async fn test_heartbeat_end_to_end() {
|
||||
|
||||
// 5. Create LLM provider
|
||||
let session = create_session_manager(config.llm.session.clone()).await;
|
||||
let llm = create_llm_provider(&config.llm, session).expect("Failed to create LLM provider");
|
||||
let llm = create_llm_provider(&config.llm, session)
|
||||
.await
|
||||
.expect("Failed to create LLM provider");
|
||||
println!("[5/6] LLM provider created (model: {})", llm.model_name());
|
||||
|
||||
// 6. Run heartbeat check
|
||||
|
||||
Reference in New Issue
Block a user