feat: add AWS Bedrock LLM provider via native Converse API (#713)

* feat: add AWS Bedrock LLM provider via native Converse API

* fix: use JSON parsing for tool result error detection instead of brittle substring matching

* refactor: extract duplicated inference config builder into helper function

* fix: address review feedback — safe casts, input validation, and tests

- Safe u32→i32 cast for max_tokens using try_from with clamp
- Remove brittle string-based error detection fallback for tool results
- Validate BEDROCK_CROSS_REGION against allowed values (us/eu/apac/global)
- Validate message list is non-empty before Converse API call
- Log when using default us-east-1 region
- Update llm_backend doc comment to list all backends
- Add tests for build_inference_config and empty message handling

* fix: persist AWS_PROFILE for Bedrock named profile auth

The wizard collected the profile name but only printed a hint to set
it manually. Now it saves to settings and writes AWS_PROFILE to the
bootstrap .env, consistent with how BEDROCK_REGION and other Bedrock
settings are persisted.

* feat: gate AWS Bedrock behind optional `bedrock` feature flag

The AWS SDK dependencies (aws-config, aws-sdk-bedrockruntime,
aws-smithy-types) require cmake and a C compiler to build aws-lc-sys.
Gate them behind an opt-in `bedrock` feature flag so default builds
are unaffected.

Build with: cargo build --features bedrock
All config, settings, and wizard code stays unconditional (no AWS deps)
so users can configure Bedrock even without the feature compiled — they
get a clear error at startup directing them to rebuild.

* fix: address review feedback and adapt Bedrock provider to registry architecture (takeover #345)

- Resolve merge conflicts with main's registry-based provider system
- Add missing cache_creation_input_tokens/cache_read_input_tokens fields
- Add missing content_parts field in test ChatMessage
- Fix string literal type mismatches in wizard env_vars (.to_string())
- Remove non-functional bearer token auth (AWS_BEARER_TOKEN_BEDROCK) from
  wizard and documentation per reviewer feedback from @zmanian and @serrrfirat
- Remove stale BEDROCK_ACCESS_KEY proxy entry from provider table
- Update Bedrock provider to use is_bedrock string check (LlmBackend enum removed)
- Add bedrock_profile fallback from settings in config resolution

[skip-regression-check]

Co-Authored-By: cgorski <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use main's Cargo.lock as base to preserve dependency versions

Regenerating Cargo.lock from scratch caused transitive dependency version
drift that broke the html_to_markdown fixture test in CI.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: bedrock config bugs — spurious warning, alias normalization, profile fallback

- Move is_bedrock check before unknown-backend warning to prevent
  spurious "unknown backend" log for bedrock users
- Normalize backend aliases ("aws", "aws_bedrock") to "bedrock" so
  the provider factory matches correctly
- Add settings.bedrock_profile fallback for AWS_PROFILE, consistent
  with region and cross_region resolution

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address Copilot review feedback — bearer token cleanup, stop_sequences, model dedup

- Remove stale bearer token refs from setup README and CHANGELOG
- Remove dead bedrock_api_key secret injection mapping
- Pass stop_sequences through to Bedrock InferenceConfiguration
- Remove "API key" from wizard menu description (bearer token removed)
- Skip duplicate LLM_MODEL write for bedrock backend in wizard
- Fix cargo fmt formatting

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review feedback — async new(), remove LiteLLM entry, wizard fixes

- Remove dead LiteLLM-based bedrock entry from providers.json (native
  Converse API intercepts before registry lookup)
- Make BedrockProvider::new() async to avoid block_in_place panic in
  current_thread runtimes; propagate async to create_llm_provider,
  build_provider_chain, and init_llm
- Document CMake build prerequisite in docs/LLM_PROVIDERS.md
- Clear bedrock_profile when user selects "default credentials" in wizard
- Fix selected_model clearing to match established pattern (conditional
  on provider switch, not unconditional)
- Add regression tests for bedrock model preservation and profile clearing

Addresses review feedback from @zmanian on PR #713.
Streaming support tracked in #741.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address remaining review comments — CLAUDE.md backends, wizard UX

- Add `bedrock` to CLAUDE.md inline backend list (#10)
- Skip full setup re-run when keeping existing Bedrock config (#11)
- Clear stale bedrock_profile on empty named-profile input (#12)
- Add regression test for empty profile clearing

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Chris Gorski <[email protected]>
Co-authored-by: cgorski <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
Illia Polosukhin
2026-03-09 07:10:25 +00:00
committed by GitHub
co-authored by Chris Gorski cgorski Claude Opus 4.6
parent 30d81fcdee
commit d73e35cfb0
16 changed files with 2076 additions and 44 deletions
+4
View File
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
### Added
- AWS Bedrock LLM provider via native Converse API with IAM and SSO auth support (feature-gated: `--features bedrock`)
## [0.16.1](https://github.com/nearai/ironclaw/compare/v0.16.0...v0.16.1) - 2026-03-06 ## [0.16.1](https://github.com/nearai/ironclaw/compare/v0.16.0...v0.16.1) - 2026-03-06
### Fixed ### Fixed
+10 -1
View File
@@ -484,6 +484,13 @@ SKILLS_AUTO_DISCOVER=true # Scan skill directories on startup
TINFOIL_API_KEY=... # Required when LLM_BACKEND=tinfoil TINFOIL_API_KEY=... # Required when LLM_BACKEND=tinfoil
TINFOIL_MODEL=kimi-k2-5 # Default model TINFOIL_MODEL=kimi-k2-5 # Default model
# AWS Bedrock (native Converse API, requires --features bedrock)
# LLM_BACKEND=bedrock
# BEDROCK_REGION=us-east-1 # AWS region
# BEDROCK_MODEL=anthropic.claude-opus-4-6-v1 # Required model ID
# BEDROCK_CROSS_REGION=us # Cross-region prefix (us/eu/apac/global)
# AWS_PROFILE=my-profile # Named profile (SSO/assume-role)
# Tunnel (public internet exposure for webhooks) # Tunnel (public internet exposure for webhooks)
TUNNEL_URL=https://abc123.ngrok.io # Static public URL (manual tunnel) TUNNEL_URL=https://abc123.ngrok.io # Static public URL (manual tunnel)
# Or use a managed tunnel provider: # Or use a managed tunnel provider:
@@ -500,7 +507,9 @@ OBSERVABILITY_BACKEND=none # none/noop (default) or log
### LLM Providers ### LLM Providers
Backends: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, `tinfoil` — set via `LLM_BACKEND`. See [src/llm/CLAUDE.md](src/llm/CLAUDE.md) for per-provider auth and configuration details. Backends: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, `tinfoil`, `bedrock` (requires `--features bedrock`) — set via `LLM_BACKEND`. See [src/llm/CLAUDE.md](src/llm/CLAUDE.md) for per-provider auth and configuration details.
**AWS Bedrock** -- Uses the native Converse API via `aws-sdk-bedrockruntime`. Requires `--features bedrock` at build time (not included in default features due to heavy AWS SDK dependencies). Supports standard AWS auth methods: IAM credentials (`AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`), SSO profiles (`AWS_PROFILE`), and instance roles. Configure with `BEDROCK_REGION` (default: `us-east-1`), `BEDROCK_MODEL` (required, e.g., `anthropic.claude-opus-4-6-v1`), and `BEDROCK_CROSS_REGION` (optional: `us`, `eu`, `apac`, `global` for cross-region inference profiles). The SDK credential chain resolves auth automatically from the environment.
## Database ## Database
Generated
+533
View File
@@ -392,6 +392,412 @@ version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8" checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "aws-config"
version = "1.8.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "11493b0bad143270fb8ad284a096dd529ba91924c5409adeac856cc1bf047dbc"
dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-sdk-sso",
"aws-sdk-ssooidc",
"aws-sdk-sts",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
"aws-types",
"bytes",
"fastrand",
"hex",
"http 1.4.0",
"sha1",
"time",
"tokio",
"tracing",
"url",
"zeroize",
]
[[package]]
name = "aws-credential-types"
version = "1.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f20799b373a1be121fe3005fba0c2090af9411573878f224df44b42727fcaf7"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
"aws-smithy-types",
"zeroize",
]
[[package]]
name = "aws-lc-rs"
version = "1.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94bffc006df10ac2a68c83692d734a465f8ee6c5b384d8545a636f81d858f4bf"
dependencies = [
"aws-lc-sys",
"zeroize",
]
[[package]]
name = "aws-lc-sys"
version = "0.38.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4321e568ed89bb5a7d291a7f37997c2c0df89809d7b6d12062c81ddb54aa782e"
dependencies = [
"cc",
"cmake",
"dunce",
"fs_extra",
]
[[package]]
name = "aws-runtime"
version = "1.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5fc0651c57e384202e47153c1260b84a9936e19803d747615edf199dc3b98d17"
dependencies = [
"aws-credential-types",
"aws-sigv4",
"aws-smithy-async",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
"aws-types",
"bytes",
"bytes-utils",
"fastrand",
"http 1.4.0",
"http-body 1.0.1",
"percent-encoding",
"pin-project-lite",
"tracing",
"uuid",
]
[[package]]
name = "aws-sdk-bedrockruntime"
version = "1.127.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7dcd5ccbed3bd50d342077d3f731de46d9608340386c87d07566c4c507891eda"
dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-sigv4",
"aws-smithy-async",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
"aws-types",
"bytes",
"fastrand",
"http 0.2.12",
"http 1.4.0",
"http-body-util",
"regex-lite",
"tracing",
]
[[package]]
name = "aws-sdk-sso"
version = "1.96.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f64a6eded248c6b453966e915d32aeddb48ea63ad17932682774eb026fbef5b1"
dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
"aws-types",
"bytes",
"fastrand",
"http 0.2.12",
"http 1.4.0",
"regex-lite",
"tracing",
]
[[package]]
name = "aws-sdk-ssooidc"
version = "1.98.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db96d720d3c622fcbe08bae1c4b04a72ce6257d8b0584cb5418da00ae20a344f"
dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
"aws-types",
"bytes",
"fastrand",
"http 0.2.12",
"http 1.4.0",
"regex-lite",
"tracing",
]
[[package]]
name = "aws-sdk-sts"
version = "1.100.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fafbdda43b93f57f699c5dfe8328db590b967b8a820a13ccdd6687355dfcc7ca"
dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-query",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
"aws-smithy-xml",
"aws-types",
"fastrand",
"http 0.2.12",
"http 1.4.0",
"regex-lite",
"tracing",
]
[[package]]
name = "aws-sigv4"
version = "1.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0b660013a6683ab23797778e21f1f854744fdf05f68204b4cca4c8c04b5d1f4"
dependencies = [
"aws-credential-types",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-runtime-api",
"aws-smithy-types",
"bytes",
"form_urlencoded",
"hex",
"hmac",
"http 0.2.12",
"http 1.4.0",
"percent-encoding",
"sha2",
"time",
"tracing",
]
[[package]]
name = "aws-smithy-async"
version = "1.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2ffcaf626bdda484571968400c326a244598634dc75fd451325a54ad1a59acfc"
dependencies = [
"futures-util",
"pin-project-lite",
"tokio",
]
[[package]]
name = "aws-smithy-eventstream"
version = "0.60.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf09d74e5e32f76b8762da505a3cd59303e367a664ca67295387baa8c1d7548"
dependencies = [
"aws-smithy-types",
"bytes",
"crc32fast",
]
[[package]]
name = "aws-smithy-http"
version = "0.63.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba1ab2dc1c2c3749ead27180d333c42f11be8b0e934058fb4b2258ee8dbe5231"
dependencies = [
"aws-smithy-eventstream",
"aws-smithy-runtime-api",
"aws-smithy-types",
"bytes",
"bytes-utils",
"futures-core",
"futures-util",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"percent-encoding",
"pin-project-lite",
"pin-utils",
"tracing",
]
[[package]]
name = "aws-smithy-http-client"
version = "1.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a2f165a7feee6f263028b899d0a181987f4fa7179a6411a32a439fba7c5f769"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
"aws-smithy-types",
"h2 0.3.27",
"h2 0.4.13",
"http 0.2.12",
"http 1.4.0",
"http-body 0.4.6",
"hyper 0.14.32",
"hyper 1.8.1",
"hyper-rustls 0.24.2",
"hyper-rustls 0.27.7",
"hyper-util",
"pin-project-lite",
"rustls 0.21.12",
"rustls 0.23.37",
"rustls-native-certs 0.8.3",
"rustls-pki-types",
"tokio",
"tokio-rustls 0.26.4",
"tower 0.5.3",
"tracing",
]
[[package]]
name = "aws-smithy-json"
version = "0.62.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9648b0bb82a2eedd844052c6ad2a1a822d1f8e3adee5fbf668366717e428856a"
dependencies = [
"aws-smithy-types",
]
[[package]]
name = "aws-smithy-observability"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a06c2315d173edbf1920da8ba3a7189695827002e4c0fc961973ab1c54abca9c"
dependencies = [
"aws-smithy-runtime-api",
]
[[package]]
name = "aws-smithy-query"
version = "0.60.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a56d79744fb3edb5d722ef79d86081e121d3b9422cb209eb03aea6aa4f21ebd"
dependencies = [
"aws-smithy-types",
"urlencoding",
]
[[package]]
name = "aws-smithy-runtime"
version = "1.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "028999056d2d2fd58a697232f9eec4a643cf73a71cf327690a7edad1d2af2110"
dependencies = [
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http-client",
"aws-smithy-observability",
"aws-smithy-runtime-api",
"aws-smithy-types",
"bytes",
"fastrand",
"http 0.2.12",
"http 1.4.0",
"http-body 0.4.6",
"http-body 1.0.1",
"http-body-util",
"pin-project-lite",
"pin-utils",
"tokio",
"tracing",
]
[[package]]
name = "aws-smithy-runtime-api"
version = "1.11.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "876ab3c9c29791ba4ba02b780a3049e21ec63dabda09268b175272c3733a79e6"
dependencies = [
"aws-smithy-async",
"aws-smithy-types",
"bytes",
"http 0.2.12",
"http 1.4.0",
"pin-project-lite",
"tokio",
"tracing",
"zeroize",
]
[[package]]
name = "aws-smithy-types"
version = "1.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2b1117b3b2bbe166d11199b540ceed0d0f7676e36e7b962b5a437a9971eac75"
dependencies = [
"base64-simd",
"bytes",
"bytes-utils",
"futures-core",
"http 0.2.12",
"http 1.4.0",
"http-body 0.4.6",
"http-body 1.0.1",
"http-body-util",
"itoa",
"num-integer",
"pin-project-lite",
"pin-utils",
"ryu",
"serde",
"time",
"tokio",
"tokio-util",
]
[[package]]
name = "aws-smithy-xml"
version = "0.60.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ce02add1aa3677d022f8adf81dcbe3046a95f17a1b1e8979c145cd21d3d22b3"
dependencies = [
"xmlparser",
]
[[package]]
name = "aws-types"
version = "1.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "47c8323699dd9b3c8d5b3c13051ae9cdef58fd179957c882f8374dd8725962d9"
dependencies = [
"aws-credential-types",
"aws-smithy-async",
"aws-smithy-runtime-api",
"aws-smithy-types",
"rustc_version",
"tracing",
]
[[package]] [[package]]
name = "axum" name = "axum"
version = "0.6.20" version = "0.6.20"
@@ -504,6 +910,16 @@ version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "base64-simd"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "339abbe78e73178762e23bea9dfd08e697eb3f3301cd4be981c0f78ba5859195"
dependencies = [
"outref",
"vsimd",
]
[[package]] [[package]]
name = "base64ct" name = "base64ct"
version = "1.8.3" version = "1.8.3"
@@ -730,6 +1146,16 @@ dependencies = [
"serde", "serde",
] ]
[[package]]
name = "bytes-utils"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7dafe3a8757b027e2be6e4e5601ed563c55989fcf1546e933c66c8eb3a058d35"
dependencies = [
"bytes",
"either",
]
[[package]] [[package]]
name = "cap-fs-ext" name = "cap-fs-ext"
version = "3.4.5" version = "3.4.5"
@@ -953,6 +1379,15 @@ dependencies = [
"error-code", "error-code",
] ]
[[package]]
name = "cmake"
version = "0.1.57"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75443c44cd6b379beb8c5b45d85d0773baf31cce901fe7bb252f4eff3008ef7d"
dependencies = [
"cc",
]
[[package]] [[package]]
name = "cobs" name = "cobs"
version = "0.3.0" version = "0.3.0"
@@ -1708,6 +2143,12 @@ dependencies = [
"dtoa", "dtoa",
] ]
[[package]]
name = "dunce"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813"
[[package]] [[package]]
name = "dyn-clone" name = "dyn-clone"
version = "1.0.20" version = "1.0.20"
@@ -2021,6 +2462,12 @@ dependencies = [
"windows-sys 0.48.0", "windows-sys 0.48.0",
] ]
[[package]]
name = "fs_extra"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c"
[[package]] [[package]]
name = "funty" name = "funty"
version = "2.0.0" version = "2.0.0"
@@ -2568,6 +3015,21 @@ dependencies = [
"winapi", "winapi",
] ]
[[package]]
name = "hyper-rustls"
version = "0.24.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec3efd23720e2049821a693cbc7e65ea87c72f1c58ff2f9522ff332b1491e590"
dependencies = [
"futures-util",
"http 0.2.12",
"hyper 0.14.32",
"log",
"rustls 0.21.12",
"tokio",
"tokio-rustls 0.24.1",
]
[[package]] [[package]]
name = "hyper-rustls" name = "hyper-rustls"
version = "0.25.0" version = "0.25.0"
@@ -2894,6 +3356,9 @@ dependencies = [
"aho-corasick", "aho-corasick",
"anyhow", "anyhow",
"async-trait", "async-trait",
"aws-config",
"aws-sdk-bedrockruntime",
"aws-smithy-types",
"axum 0.8.8", "axum 0.8.8",
"base64 0.22.1", "base64 0.22.1",
"blake3", "blake3",
@@ -3833,6 +4298,12 @@ dependencies = [
"pin-project-lite", "pin-project-lite",
] ]
[[package]]
name = "outref"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a80800c0488c3a21695ea981a54918fbb37abf04f4d0720c453632255e2ff0e"
[[package]] [[package]]
name = "parking" name = "parking"
version = "2.2.1" version = "2.2.1"
@@ -4660,6 +5131,12 @@ dependencies = [
"regex-syntax", "regex-syntax",
] ]
[[package]]
name = "regex-lite"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cab834c73d247e67f4fae452806d17d3c7501756d98c8808d7c9c7aa7d18f973"
[[package]] [[package]]
name = "regex-syntax" name = "regex-syntax"
version = "0.8.10" version = "0.8.10"
@@ -4888,6 +5365,18 @@ dependencies = [
"rustix 1.1.4", "rustix 1.1.4",
] ]
[[package]]
name = "rustls"
version = "0.21.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f56a14d1f48b391359b22f731fd4bd7e43c97f3c50eee276f3aa09c94784d3e"
dependencies = [
"log",
"ring",
"rustls-webpki 0.101.7",
"sct",
]
[[package]] [[package]]
name = "rustls" name = "rustls"
version = "0.22.4" version = "0.22.4"
@@ -4908,6 +5397,7 @@ version = "0.23.37"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4" checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4"
dependencies = [ dependencies = [
"aws-lc-rs",
"once_cell", "once_cell",
"ring", "ring",
"rustls-pki-types", "rustls-pki-types",
@@ -4960,6 +5450,16 @@ dependencies = [
"zeroize", "zeroize",
] ]
[[package]]
name = "rustls-webpki"
version = "0.101.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b6275d1ee7a1cd780b64aca7726599a1dbc893b1e64144529e55c3c2f745765"
dependencies = [
"ring",
"untrusted",
]
[[package]] [[package]]
name = "rustls-webpki" name = "rustls-webpki"
version = "0.102.8" version = "0.102.8"
@@ -4977,6 +5477,7 @@ version = "0.103.9"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7df23109aa6c1567d1c575b9952556388da57401e4ace1d15f79eedad0d8f53" checksum = "d7df23109aa6c1567d1c575b9952556388da57401e4ace1d15f79eedad0d8f53"
dependencies = [ dependencies = [
"aws-lc-rs",
"ring", "ring",
"rustls-pki-types", "rustls-pki-types",
"untrusted", "untrusted",
@@ -5104,6 +5605,16 @@ dependencies = [
"tendril 0.4.3", "tendril 0.4.3",
] ]
[[package]]
name = "sct"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da046153aa2352493d6cb7da4b6e5c0c057d8a1d0a9aa8560baffdd945acd414"
dependencies = [
"ring",
"untrusted",
]
[[package]] [[package]]
name = "seahash" name = "seahash"
version = "4.1.0" version = "4.1.0"
@@ -6060,6 +6571,16 @@ dependencies = [
"x509-cert", "x509-cert",
] ]
[[package]]
name = "tokio-rustls"
version = "0.24.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c28327cf380ac148141087fbfb9de9d7bd4e84ab5d2c28fbc911d753de8a7081"
dependencies = [
"rustls 0.21.12",
"tokio",
]
[[package]] [[package]]
name = "tokio-rustls" name = "tokio-rustls"
version = "0.25.0" version = "0.25.0"
@@ -6697,6 +7218,12 @@ version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "vsimd"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c3082ca00d5a5ef149bb8b555a72ae84c9c59f7250f013ac822ac2e49b19c64"
[[package]] [[package]]
name = "walkdir" name = "walkdir"
version = "2.5.0" version = "2.5.0"
@@ -7893,6 +8420,12 @@ dependencies = [
"windows-sys 0.59.0", "windows-sys 0.59.0",
] ]
[[package]]
name = "xmlparser"
version = "0.13.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "66fee0b777b0f5ac1c69bb06d361268faafa61cd4682ae064a171c16c433e9e4"
[[package]] [[package]]
name = "yansi" name = "yansi"
version = "1.0.1" version = "1.0.1"
+6
View File
@@ -142,6 +142,11 @@ subtle = "2" # Constant-time comparisons for token validation
# Multi-provider LLM support # Multi-provider LLM support
rig-core = "0.30" rig-core = "0.30"
# AWS Bedrock (native Converse API, opt-in via --features bedrock)
aws-config = { version = "1", features = ["behavior-version-latest"], optional = true }
aws-sdk-bedrockruntime = { version = "1", optional = true }
aws-smithy-types = { version = "1", optional = true }
# Docker sandbox # Docker sandbox
bollard = "0.18" bollard = "0.18"
@@ -203,6 +208,7 @@ postgres = [
libsql = ["dep:libsql"] libsql = ["dep:libsql"]
integration = [] integration = []
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"] html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
bedrock = ["dep:aws-config", "dep:aws-sdk-bedrockruntime", "dep:aws-smithy-types"]
[[test]] [[test]]
name = "html_to_markdown" name = "html_to_markdown"
+1 -1
View File
@@ -215,7 +215,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| NEAR AI | ✅ | ✅ | - | Primary provider | | NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6 | | Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6 |
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy | | OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy |
| AWS Bedrock | ✅ | ✅ | P3 | Via `openai_compatible` adapter (e.g. LiteLLM) | | AWS Bedrock | ✅ | ✅ | - | Native Converse API via aws-sdk-bedrockruntime (requires `--features bedrock`) |
| Google Gemini | ✅ | ✅ | P3 | Via `gemini` adapter | | Google Gemini | ✅ | ✅ | P3 | Via `gemini` adapter |
| io.net | ✅ | ✅ | P3 | Via `ionet` adapter | | io.net | ✅ | ✅ | P3 | Via `ionet` adapter |
| Mistral | ✅ | ✅ | P3 | Via `mistral` adapter | | Mistral | ✅ | ✅ | P3 | Via `mistral` adapter |
+50 -1
View File
@@ -12,12 +12,12 @@ configurations.
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models | | Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models | | OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Google Gemini | `gemini` | `GEMINI_API_KEY` | Gemini models | | Google Gemini | `gemini` | `GEMINI_API_KEY` | Gemini models |
| AWS Bedrock | `bedrock` | `BEDROCK_ACCESS_KEY` | Requires OpenAI proxy (e.g. LiteLLM) |
| io.net | `ionet` | `IONET_API_KEY` | Intelligence API | | io.net | `ionet` | `IONET_API_KEY` | Intelligence API |
| Mistral | `mistral` | `MISTRAL_API_KEY` | Mistral models | | Mistral | `mistral` | `MISTRAL_API_KEY` | Mistral models |
| Yandex AI Studio | `yandex` | `YANDEX_API_KEY` | YandexGPT models | | Yandex AI Studio | `yandex` | `YANDEX_API_KEY` | YandexGPT models |
| Cloudflare Workers AI | `cloudflare` | `CLOUDFLARE_API_KEY` | Access to Workers AI | | Cloudflare Workers AI | `cloudflare` | `CLOUDFLARE_API_KEY` | Access to Workers AI |
| Ollama | `ollama` | No | Local inference | | Ollama | `ollama` | No | Local inference |
| AWS Bedrock | `bedrock` | AWS credentials | Native Converse API |
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models | | OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference | | Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference | | Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
@@ -74,6 +74,55 @@ Pull a model first: `ollama pull llama3.2`
--- ---
## AWS Bedrock (requires `--features bedrock`)
Uses the native AWS Converse API via `aws-sdk-bedrockruntime`. Supports standard AWS
authentication methods: IAM credentials, SSO profiles, and instance roles.
> **Build prerequisite:** The `aws-lc-sys` crate (transitive dependency via AWS SDK)
> requires **CMake** to compile. Install it before building with `--features bedrock`:
> - macOS: `brew install cmake`
> - Ubuntu/Debian: `sudo apt install cmake`
> - Fedora: `sudo dnf install cmake`
### With AWS credentials (IAM, SSO, instance roles)
```env
LLM_BACKEND=bedrock
BEDROCK_MODEL=anthropic.claude-opus-4-6-v1
BEDROCK_REGION=us-east-1
BEDROCK_CROSS_REGION=us
# AWS_PROFILE=my-sso-profile # optional, for named profiles
```
The AWS SDK credential chain automatically resolves credentials from environment
variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`), shared credentials file
(`~/.aws/credentials`), SSO profiles, and EC2/ECS instance roles.
### Cross-region inference
Set `BEDROCK_CROSS_REGION` to route requests across AWS regions for capacity:
| Prefix | Routing |
|---|---|
| `us` | US regions (us-east-1, us-east-2, us-west-2) |
| `eu` | European regions |
| `apac` | Asia-Pacific regions |
| `global` | All commercial AWS regions |
| _(unset)_ | Single-region only |
### Popular Bedrock model IDs
| Model | ID |
|---|---|
| Claude Opus 4.6 | `anthropic.claude-opus-4-6-v1` |
| Claude Sonnet 4.5 | `anthropic.claude-sonnet-4-5-20250929-v1:0` |
| Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` |
| Amazon Nova Pro | `amazon.nova-pro-v1:0` |
| Llama 4 Maverick | `meta.llama4-maverick-17b-instruct-v1:0` |
---
## OpenAI-Compatible Endpoints ## OpenAI-Compatible Endpoints
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
-20
View File
@@ -295,26 +295,6 @@
"can_list_models": true "can_list_models": true
} }
}, },
{
"id": "bedrock",
"aliases": [
"aws_bedrock",
"aws"
],
"protocol": "open_ai_completions",
"api_key_env": "BEDROCK_ACCESS_KEY",
"api_key_required": false,
"base_url_env": "BEDROCK_BASE_URL",
"model_env": "BEDROCK_MODEL",
"default_model": "anthropic.claude-3-5-sonnet-20241022-v2:0",
"description": "AWS Bedrock (requires LiteLLM or OpenAI-compatible proxy)",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_bedrock_api_key",
"display_name": "AWS Bedrock",
"can_list_models": false
}
},
{ {
"id": "ionet", "id": "ionet",
"aliases": [ "aliases": [
+3 -3
View File
@@ -334,7 +334,7 @@ impl AppBuilder {
/// Delegates to `build_provider_chain` which applies all decorators /// Delegates to `build_provider_chain` which applies all decorators
/// (retry, smart routing, failover, circuit breaker, response cache). /// (retry, smart routing, failover, circuit breaker, response cache).
#[allow(clippy::type_complexity)] #[allow(clippy::type_complexity)]
pub fn init_llm( pub async fn init_llm(
&self, &self,
) -> Result< ) -> Result<
( (
@@ -345,7 +345,7 @@ impl AppBuilder {
anyhow::Error, anyhow::Error,
> { > {
let (llm, cheap_llm, recording_handle) = let (llm, cheap_llm, recording_handle) =
crate::llm::build_provider_chain(&self.config.llm, self.session.clone())?; crate::llm::build_provider_chain(&self.config.llm, self.session.clone()).await?;
Ok((llm, cheap_llm, recording_handle)) Ok((llm, cheap_llm, recording_handle))
} }
@@ -820,7 +820,7 @@ impl AppBuilder {
let (llm, cheap_llm, recording_handle) = if let Some(llm) = self.llm_override.take() { let (llm, cheap_llm, recording_handle) = if let Some(llm) = self.llm_override.take() {
(llm, None, None) (llm, None, None)
} else { } else {
self.init_llm()? self.init_llm().await?
}; };
let (safety, tools, embeddings, workspace) = self.init_tools(&llm).await?; let (safety, tools, embeddings, workspace) = self.init_tools(&llm).await?;
+62 -4
View File
@@ -86,6 +86,19 @@ pub struct RegistryProviderConfig {
pub oauth_token: Option<SecretString>, pub oauth_token: Option<SecretString>,
} }
/// Configuration for AWS Bedrock (native Converse API).
#[derive(Debug, Clone)]
pub struct BedrockConfig {
/// AWS region (e.g. "us-east-1").
pub region: String,
/// Bedrock model ID (e.g. "anthropic.claude-opus-4-6-v1").
pub model: String,
/// Cross-region inference prefix: "us", "eu", "apac", "global", or None.
pub cross_region: Option<String>,
/// AWS named profile (for SSO / assume-role workflows).
pub profile: Option<String>,
}
/// LLM provider configuration. /// LLM provider configuration.
/// ///
/// NearAI remains the default backend with its own config struct (session auth). /// NearAI remains the default backend with its own config struct (session auth).
@@ -101,8 +114,10 @@ pub struct LlmConfig {
/// NEAR AI config (always populated, also used for embeddings). /// NEAR AI config (always populated, also used for embeddings).
pub nearai: NearAiConfig, pub nearai: NearAiConfig,
/// Resolved provider config for registry-based providers. /// Resolved provider config for registry-based providers.
/// `None` when backend is "nearai". /// `None` when backend is "nearai" or "bedrock".
pub provider: Option<RegistryProviderConfig>, pub provider: Option<RegistryProviderConfig>,
/// AWS Bedrock config (populated when backend=bedrock, requires --features bedrock).
pub bedrock: Option<BedrockConfig>,
/// HTTP request timeout in seconds for LLM API calls. /// HTTP request timeout in seconds for LLM API calls.
/// Default: 120. Increase for local LLMs (Ollama, vLLM, LM Studio) that /// Default: 120. Increase for local LLMs (Ollama, vLLM, LM Studio) that
/// need more time for prompt evaluation on consumer hardware. /// need more time for prompt evaluation on consumer hardware.
@@ -169,6 +184,7 @@ impl LlmConfig {
smart_routing_cascade: false, smart_routing_cascade: false,
}, },
provider: None, provider: None,
bedrock: None,
request_timeout_secs: 120, request_timeout_secs: 120,
} }
} }
@@ -200,8 +216,10 @@ impl LlmConfig {
let backend_lower = backend.to_lowercase(); let backend_lower = backend.to_lowercase();
let is_nearai = let is_nearai =
backend_lower == "nearai" || backend_lower == "near_ai" || backend_lower == "near"; backend_lower == "nearai" || backend_lower == "near_ai" || backend_lower == "near";
let is_bedrock =
backend_lower == "bedrock" || backend_lower == "aws_bedrock" || backend_lower == "aws";
if !is_nearai && registry.find(&backend_lower).is_none() { if !is_nearai && !is_bedrock && registry.find(&backend_lower).is_none() {
tracing::warn!( tracing::warn!(
"Unknown LLM backend '{}'. Will attempt as openai_compatible fallback.", "Unknown LLM backend '{}'. Will attempt as openai_compatible fallback.",
backend backend
@@ -248,8 +266,8 @@ impl LlmConfig {
smart_routing_cascade: parse_optional_env("SMART_ROUTING_CASCADE", true)?, smart_routing_cascade: parse_optional_env("SMART_ROUTING_CASCADE", true)?,
}; };
// Resolve registry provider config (for non-NearAI backends) // Resolve registry provider config (for non-NearAI, non-Bedrock backends)
let provider = if is_nearai { let provider = if is_nearai || is_bedrock {
None None
} else { } else {
Some(Self::resolve_registry_provider( Some(Self::resolve_registry_provider(
@@ -259,11 +277,50 @@ impl LlmConfig {
)?) )?)
}; };
let bedrock = if is_bedrock {
let explicit_region =
optional_env("BEDROCK_REGION")?.or_else(|| settings.bedrock_region.clone());
if explicit_region.is_none() {
tracing::info!("BEDROCK_REGION not set, defaulting to us-east-1");
}
let region = explicit_region.unwrap_or_else(|| "us-east-1".to_string());
let model = optional_env("BEDROCK_MODEL")?
.or_else(|| settings.selected_model.clone())
.ok_or_else(|| ConfigError::MissingRequired {
key: "BEDROCK_MODEL".to_string(),
hint: "Set BEDROCK_MODEL when LLM_BACKEND=bedrock".to_string(),
})?;
let cross_region = optional_env("BEDROCK_CROSS_REGION")?
.or_else(|| settings.bedrock_cross_region.clone());
if let Some(ref cr) = cross_region
&& !matches!(cr.as_str(), "us" | "eu" | "apac" | "global")
{
return Err(ConfigError::InvalidValue {
key: "BEDROCK_CROSS_REGION".to_string(),
message: format!(
"'{}' is not valid, expected one of: us, eu, apac, global",
cr
),
});
}
let profile = optional_env("AWS_PROFILE")?.or_else(|| settings.bedrock_profile.clone());
Some(BedrockConfig {
region,
model,
cross_region,
profile,
})
} else {
None
};
let request_timeout_secs = parse_optional_env("LLM_REQUEST_TIMEOUT_SECS", 120)?; let request_timeout_secs = parse_optional_env("LLM_REQUEST_TIMEOUT_SECS", 120)?;
Ok(Self { Ok(Self {
backend: if is_nearai { backend: if is_nearai {
"nearai".to_string() "nearai".to_string()
} else if is_bedrock {
"bedrock".to_string()
} else if let Some(ref p) = provider { } else if let Some(ref p) = provider {
p.provider_id.clone() p.provider_id.clone()
} else { } else {
@@ -272,6 +329,7 @@ impl LlmConfig {
session, session,
nearai, nearai,
provider, provider,
bedrock,
request_timeout_secs, request_timeout_secs,
}) })
} }
+3 -1
View File
@@ -37,7 +37,9 @@ pub use self::database::{DatabaseBackend, DatabaseConfig, SslMode, default_libsq
pub use self::embeddings::EmbeddingsConfig; pub use self::embeddings::EmbeddingsConfig;
pub use self::heartbeat::HeartbeatConfig; pub use self::heartbeat::HeartbeatConfig;
pub use self::hygiene::HygieneConfig; pub use self::hygiene::HygieneConfig;
pub use self::llm::{CacheRetention, LlmConfig, NearAiConfig, RegistryProviderConfig}; pub use self::llm::{
BedrockConfig, CacheRetention, LlmConfig, NearAiConfig, RegistryProviderConfig,
};
pub use self::routines::RoutineConfig; pub use self::routines::RoutineConfig;
pub use self::safety::SafetyConfig; pub use self::safety::SafetyConfig;
pub use self::sandbox::{ClaudeCodeConfig, SandboxModeConfig}; pub use self::sandbox::{ClaudeCodeConfig, SandboxModeConfig};
+1148
View File
File diff suppressed because it is too large Load Diff
+40 -3
View File
@@ -6,8 +6,11 @@
//! - **Anthropic**: Direct API access with your own key //! - **Anthropic**: Direct API access with your own key
//! - **Ollama**: Local model inference //! - **Ollama**: Local model inference
//! - **OpenAI-compatible**: Any endpoint that speaks the OpenAI API //! - **OpenAI-compatible**: Any endpoint that speaks the OpenAI API
//! - **AWS Bedrock**: Native Converse API via aws-sdk-bedrockruntime
mod anthropic_oauth; mod anthropic_oauth;
#[cfg(feature = "bedrock")]
mod bedrock;
pub mod circuit_breaker; pub mod circuit_breaker;
pub mod costs; pub mod costs;
pub mod failover; pub mod failover;
@@ -57,7 +60,7 @@ use crate::error::LlmError;
/// ///
/// - NearAI backend: Uses session manager for authentication /// - NearAI backend: Uses session manager for authentication
/// - Registry providers: Looked up by protocol and constructed generically /// - Registry providers: Looked up by protocol and constructed generically
pub fn create_llm_provider( pub async fn create_llm_provider(
config: &LlmConfig, config: &LlmConfig,
session: Arc<SessionManager>, session: Arc<SessionManager>,
) -> Result<Arc<dyn LlmProvider>, LlmError> { ) -> Result<Arc<dyn LlmProvider>, LlmError> {
@@ -67,6 +70,21 @@ pub fn create_llm_provider(
return create_llm_provider_with_config(&config.nearai, session, timeout); return create_llm_provider_with_config(&config.nearai, session, timeout);
} }
// Bedrock uses a native AWS SDK, not the rig-core registry
if config.backend == "bedrock" {
#[cfg(feature = "bedrock")]
{
return create_bedrock_provider(config).await;
}
#[cfg(not(feature = "bedrock"))]
{
return Err(LlmError::RequestFailed {
provider: "bedrock".to_string(),
reason: "Bedrock support not compiled. Rebuild with --features bedrock".to_string(),
});
}
}
let reg_config = config let reg_config = config
.provider .provider
.as_ref() .as_ref()
@@ -120,6 +138,24 @@ fn create_registry_provider(
} }
} }
#[cfg(feature = "bedrock")]
async fn create_bedrock_provider(config: &LlmConfig) -> Result<Arc<dyn LlmProvider>, LlmError> {
let br = config
.bedrock
.as_ref()
.ok_or_else(|| LlmError::AuthFailed {
provider: "bedrock".to_string(),
})?;
let provider = bedrock::BedrockProvider::new(br).await?;
tracing::info!(
"Using AWS Bedrock (Converse API, region: {}, model: {})",
br.region,
provider.active_model_name(),
);
Ok(Arc::new(provider))
}
fn create_openai_compat_from_registry( fn create_openai_compat_from_registry(
config: &RegistryProviderConfig, config: &RegistryProviderConfig,
) -> Result<Arc<dyn LlmProvider>, LlmError> { ) -> Result<Arc<dyn LlmProvider>, LlmError> {
@@ -344,7 +380,7 @@ pub fn create_cheap_llm_provider(
/// This is the single source of truth for provider chain construction, /// This is the single source of truth for provider chain construction,
/// called by both `main.rs` and `app.rs`. /// called by both `main.rs` and `app.rs`.
#[allow(clippy::type_complexity)] #[allow(clippy::type_complexity)]
pub fn build_provider_chain( pub async fn build_provider_chain(
config: &LlmConfig, config: &LlmConfig,
session: Arc<SessionManager>, session: Arc<SessionManager>,
) -> Result< ) -> Result<
@@ -355,7 +391,7 @@ pub fn build_provider_chain(
), ),
LlmError, LlmError,
> { > {
let llm = create_llm_provider(config, session.clone())?; let llm = create_llm_provider(config, session.clone()).await?;
tracing::info!("LLM provider initialized: {}", llm.model_name()); tracing::info!("LLM provider initialized: {}", llm.model_name());
// 1. Retry // 1. Retry
@@ -522,6 +558,7 @@ mod tests {
session: SessionConfig::default(), session: SessionConfig::default(),
nearai: test_nearai_config(), nearai: test_nearai_config(),
provider: None, provider: None,
bedrock: None,
request_timeout_secs: 120, request_timeout_secs: 120,
} }
} }
+13 -1
View File
@@ -47,7 +47,7 @@ pub struct Settings {
pub secrets_master_key_hex: Option<String>, pub secrets_master_key_hex: Option<String>,
// === Step 3: Inference Provider === // === Step 3: Inference Provider ===
/// LLM backend: "nearai", "anthropic", "openai", "ollama", "openai_compatible". /// LLM backend: "nearai", "anthropic", "openai", "ollama", "openai_compatible", "tinfoil", "bedrock".
#[serde(default)] #[serde(default)]
pub llm_backend: Option<String>, pub llm_backend: Option<String>,
@@ -59,6 +59,18 @@ pub struct Settings {
#[serde(default)] #[serde(default)]
pub openai_compatible_base_url: Option<String>, pub openai_compatible_base_url: Option<String>,
/// Bedrock region (when llm_backend = "bedrock").
#[serde(default)]
pub bedrock_region: Option<String>,
/// Bedrock cross-region inference prefix (when llm_backend = "bedrock").
#[serde(default)]
pub bedrock_cross_region: Option<String>,
/// AWS profile name for Bedrock (when llm_backend = "bedrock").
#[serde(default)]
pub bedrock_profile: Option<String>,
// === Step 4: Model Selection === // === Step 4: Model Selection ===
/// Currently selected model. /// Currently selected model.
#[serde(default)] #[serde(default)]
+2 -1
View File
@@ -174,6 +174,7 @@ env-var mode or skipped secrets.
| Ollama | None | - | - | | Ollama | None | - | - |
| OpenRouter¹ | API key | `llm_compatible_api_key` | `LLM_API_KEY` | | OpenRouter¹ | API key | `llm_compatible_api_key` | `LLM_API_KEY` |
| OpenAI-compatible¹ | Optional API key | `llm_compatible_api_key` | `LLM_API_KEY` | | OpenAI-compatible¹ | Optional API key | `llm_compatible_api_key` | `LLM_API_KEY` |
| AWS Bedrock | AWS credentials (IAM, SSO, instance roles) | - | - |
¹ OpenRouter and OpenAI-compatible share the same secret name and env var because ¹ OpenRouter and OpenAI-compatible share the same secret name and env var because
OpenRouter is stored as `llm_backend = "openai_compatible"` under the hood. OpenRouter is stored as `llm_backend = "openai_compatible"` under the hood.
@@ -479,7 +480,7 @@ pub struct Settings {
pub secrets_master_key_source: KeySource, // Keychain | Env | None pub secrets_master_key_source: KeySource, // Keychain | Env | None
// Step 3: Inference // Step 3: Inference
pub llm_backend: Option<String>, // "nearai" | "anthropic" | "openai" | "ollama" | "openai_compatible" pub llm_backend: Option<String>, // "nearai" | "anthropic" | "openai" | "ollama" | "openai_compatible" | "bedrock"
pub ollama_base_url: Option<String>, pub ollama_base_url: Option<String>,
pub openai_compatible_base_url: Option<String>, pub openai_compatible_base_url: Option<String>,
+198 -7
View File
@@ -827,9 +827,16 @@ impl SetupWizard {
print_info(&format!("Current provider: {}", display)); print_info(&format!("Current provider: {}", display));
println!(); println!();
let is_known = current == "nearai" || registry.is_known(&current); let is_known =
current == "nearai" || current == "bedrock" || registry.is_known(&current);
if is_known && confirm("Keep current provider?", true).map_err(SetupError::Io)? { if is_known && confirm("Keep current provider?", true).map_err(SetupError::Io)? {
if current == "bedrock" {
// Keeping the existing Bedrock config — no need to re-run
// the full setup flow (region, auth, cross-region).
print_info("Keeping existing AWS Bedrock configuration.");
return Ok(());
}
return self.run_provider_setup(&current, &registry).await; return self.run_provider_setup(&current, &registry).await;
} }
@@ -844,10 +851,10 @@ impl SetupWizard {
print_info("Select your inference provider:"); print_info("Select your inference provider:");
println!(); println!();
// Build menu: NearAI first, then all registry providers with setup hints // Build menu: NearAI first, then all registry providers with setup hints, then Bedrock
let selectable = registry.selectable(); let selectable = registry.selectable();
let mut options: Vec<String> = Vec::with_capacity(1 + selectable.len()); let mut options: Vec<String> = Vec::with_capacity(2 + selectable.len());
let mut provider_ids: Vec<String> = Vec::with_capacity(1 + selectable.len()); let mut provider_ids: Vec<String> = Vec::with_capacity(2 + selectable.len());
options.push("NEAR AI - multi-model access via NEAR account".to_string()); options.push("NEAR AI - multi-model access via NEAR account".to_string());
provider_ids.push("nearai".to_string()); provider_ids.push("nearai".to_string());
@@ -865,11 +872,19 @@ impl SetupWizard {
provider_ids.push(def.id.clone()); provider_ids.push(def.id.clone());
} }
// Bedrock is a special case (native AWS SDK, not registry-based)
options.push("AWS Bedrock - Claude & other models via AWS (IAM, SSO)".to_string());
provider_ids.push("bedrock".to_string());
let option_refs: Vec<&str> = options.iter().map(|s| s.as_str()).collect(); let option_refs: Vec<&str> = options.iter().map(|s| s.as_str()).collect();
let choice = select_one("Provider:", &option_refs).map_err(SetupError::Io)?; let choice = select_one("Provider:", &option_refs).map_err(SetupError::Io)?;
let selected_id = &provider_ids[choice]; let selected_id = &provider_ids[choice];
self.run_provider_setup(selected_id, &registry).await?; if selected_id == "bedrock" {
self.setup_bedrock().await?;
} else {
self.run_provider_setup(selected_id, &registry).await?;
}
Ok(()) Ok(())
} }
@@ -1230,6 +1245,95 @@ impl SetupWizard {
Ok(()) Ok(())
} }
/// AWS Bedrock provider setup: region, auth, and cross-region config.
async fn setup_bedrock(&mut self) -> Result<(), SetupError> {
if self.settings.llm_backend.as_deref() != Some("bedrock") {
self.settings.selected_model = None;
}
self.settings.llm_backend = Some("bedrock".to_string());
// Region
let default_region = self
.settings
.bedrock_region
.as_deref()
.unwrap_or("us-east-1");
let region_input =
optional_input("AWS region", Some(&format!("default: {}", default_region)))
.map_err(SetupError::Io)?;
let region = region_input.unwrap_or_else(|| default_region.to_string());
self.settings.bedrock_region = Some(region.clone());
// Auth method
print_info("Select authentication method:");
println!();
let auth_options = &[
"AWS default credentials (env vars, ~/.aws/credentials, IAM roles)",
"AWS named profile (SSO / assume-role)",
];
let auth_choice = select_one("Auth:", auth_options).map_err(SetupError::Io)?;
match auth_choice {
0 => {
// Default AWS credentials — clear any stale named profile
self.settings.bedrock_profile = None;
print_info(
"Using default AWS credential chain (env vars, ~/.aws/credentials, IAM roles).",
);
}
1 => {
// Named profile
let profile =
input("AWS profile name (from ~/.aws/config)").map_err(SetupError::Io)?;
if profile.trim().is_empty() {
// Empty input clears any previously configured profile
self.settings.bedrock_profile = None;
print_info("AWS profile cleared; using default AWS credential chain instead.");
} else {
self.settings.bedrock_profile = Some(profile.clone());
print_success(&format!("AWS profile '{}' saved", profile));
}
}
_ => return Err(SetupError::Config("Invalid auth selection".to_string())),
}
self.setup_bedrock_cross_region()
}
/// Bedrock cross-region inference prefix selection (sub-step of setup_bedrock).
fn setup_bedrock_cross_region(&mut self) -> Result<(), SetupError> {
print_info("Cross-region inference routes requests across AWS regions for capacity:");
println!();
let cross_options = &[
"us - route within US regions (recommended for us-east-1)",
"global - route to any AWS region worldwide",
"eu - route within European regions",
"apac - route within Asia-Pacific regions",
"none - single-region only (no cross-region routing)",
];
let cross_choice = select_one("Cross-region:", cross_options).map_err(SetupError::Io)?;
let cross_region = match cross_choice {
0 => Some("us".to_string()),
1 => Some("global".to_string()),
2 => Some("eu".to_string()),
3 => Some("apac".to_string()),
4 => None,
_ => None,
};
self.settings.bedrock_cross_region = cross_region;
let region = self
.settings
.bedrock_region
.as_deref()
.unwrap_or("us-east-1");
print_success(&format!("AWS Bedrock configured (region: {})", region));
Ok(())
}
/// Generic OpenAI-compatible setup: base URL + optional API key. /// Generic OpenAI-compatible setup: base URL + optional API key.
async fn setup_openai_compatible_generic( async fn setup_openai_compatible_generic(
&mut self, &mut self,
@@ -1412,6 +1516,14 @@ impl SetupWizard {
self.settings.selected_model = Some(model_id.clone()); self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id)); print_success(&format!("Selected {}", model_id));
} }
} else if backend == "bedrock" {
let model_id = input("Bedrock model ID (e.g., anthropic.claude-opus-4-6-v1)")
.map_err(SetupError::Io)?;
if model_id.is_empty() {
return Err(SetupError::Config("Model ID is required".to_string()));
}
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
} else { } else {
// Unknown provider, manual entry // Unknown provider, manual entry
let model_id = input("Model name (e.g., meta-llama/Llama-3-8b-chat-hf)") let model_id = input("Model name (e.g., meta-llama/Llama-3-8b-chat-hf)")
@@ -1495,10 +1607,11 @@ impl SetupWizard {
smart_routing_cascade: true, smart_routing_cascade: true,
}, },
provider: None, provider: None,
bedrock: None,
request_timeout_secs: 120, request_timeout_secs: 120,
}; };
match create_llm_provider(&config, session) { match create_llm_provider(&config, session).await {
Ok(provider) => match provider.list_models().await { Ok(provider) => match provider.list_models().await {
Ok(models) => models, Ok(models) => models,
Err(e) => { Err(e) => {
@@ -2315,12 +2428,29 @@ impl SetupWizard {
if let Some(ref url) = self.settings.ollama_base_url { if let Some(ref url) = self.settings.ollama_base_url {
env_vars.push(("OLLAMA_BASE_URL".to_string(), url.clone())); env_vars.push(("OLLAMA_BASE_URL".to_string(), url.clone()));
} }
if let Some(ref region) = self.settings.bedrock_region {
env_vars.push(("BEDROCK_REGION".to_string(), region.clone()));
}
if self.settings.llm_backend.as_deref() == Some("bedrock") {
if let Some(ref model) = self.settings.selected_model {
env_vars.push(("BEDROCK_MODEL".to_string(), model.clone()));
}
if let Some(ref cross) = self.settings.bedrock_cross_region {
env_vars.push(("BEDROCK_CROSS_REGION".to_string(), cross.clone()));
}
if let Some(ref profile) = self.settings.bedrock_profile {
env_vars.push(("AWS_PROFILE".to_string(), profile.clone()));
}
}
// Model name: same chicken-and-egg — Config::from_env() resolves the // Model name: same chicken-and-egg — Config::from_env() resolves the
// model before the DB is connected, so we must persist it to .env. // model before the DB is connected, so we must persist it to .env.
// Write the backend-specific env var so the correct resolution path // Write the backend-specific env var so the correct resolution path
// picks it up (looked up from the provider registry). // picks it up (looked up from the provider registry).
if let Some(ref model) = self.settings.selected_model { // Bedrock model is already written above as BEDROCK_MODEL, skip here.
if self.settings.llm_backend.as_deref() != Some("bedrock")
&& let Some(ref model) = self.settings.selected_model
{
let backend_str = self.settings.llm_backend.as_deref().unwrap_or("nearai"); let backend_str = self.settings.llm_backend.as_deref().unwrap_or("nearai");
let model_env = registry.model_env_var(backend_str); let model_env = registry.model_env_var(backend_str);
env_vars.push((model_env.to_string(), model.clone())); env_vars.push((model_env.to_string(), model.clone()));
@@ -2605,6 +2735,7 @@ impl SetupWizard {
"openai" => "OpenAI", "openai" => "OpenAI",
"ollama" => "Ollama", "ollama" => "Ollama",
"openai_compatible" => "OpenAI-compatible", "openai_compatible" => "OpenAI-compatible",
"bedrock" => "AWS Bedrock",
other => other, other => other,
}; };
println!(" Provider: {}", display); println!(" Provider: {}", display);
@@ -3569,6 +3700,66 @@ mod tests {
); );
} }
/// Regression: Bedrock setup_bedrock() should preserve selected_model
/// when re-entering the same provider (matches pattern from #600).
#[test]
fn test_bedrock_same_provider_preserves_model() {
let mut wizard = SetupWizard::new();
wizard.settings.llm_backend = Some("bedrock".to_string());
wizard.settings.selected_model = Some("anthropic.claude-opus-4-6-v1".to_string());
// Simulate the conditional clearing logic from setup_bedrock()
if wizard.settings.llm_backend.as_deref() != Some("bedrock") {
wizard.settings.selected_model = None;
}
wizard.settings.llm_backend = Some("bedrock".to_string());
assert_eq!(
wizard.settings.selected_model.as_deref(),
Some("anthropic.claude-opus-4-6-v1"),
"bedrock model should be preserved when re-selecting bedrock"
);
}
/// Regression: switching from another provider to bedrock must clear
/// selected_model, and choosing "default credentials" must clear
/// bedrock_profile.
#[test]
fn test_bedrock_clears_stale_profile_on_default_creds() {
let mut wizard = SetupWizard::new();
wizard.settings.llm_backend = Some("bedrock".to_string());
wizard.settings.bedrock_profile = Some("old-sso-profile".to_string());
// Simulate auth_choice == 0 (default credentials) clearing the profile
wizard.settings.bedrock_profile = None;
assert!(
wizard.settings.bedrock_profile.is_none(),
"bedrock_profile should be cleared when selecting default credentials"
);
}
/// Regression: empty profile input in named-profile auth should clear
/// any previously configured profile instead of leaving it stale.
#[test]
fn test_bedrock_empty_profile_clears_existing() {
let mut wizard = SetupWizard::new();
wizard.settings.bedrock_profile = Some("old-profile".to_string());
// Simulate auth_choice == 1 with empty input
let profile = "".to_string();
if profile.trim().is_empty() {
wizard.settings.bedrock_profile = None;
} else {
wizard.settings.bedrock_profile = Some(profile);
}
assert!(
wizard.settings.bedrock_profile.is_none(),
"empty profile input should clear existing bedrock_profile"
);
}
#[tokio::test] #[tokio::test]
async fn test_run_provider_setup_no_setup_hint() { async fn test_run_provider_setup_no_setup_hint() {
// A provider with setup: None should not error. It should set the // A provider with setup: None should not error. It should set the
+3 -1
View File
@@ -84,7 +84,9 @@ async fn test_heartbeat_end_to_end() {
// 5. Create LLM provider // 5. Create LLM provider
let session = create_session_manager(config.llm.session.clone()).await; let session = create_session_manager(config.llm.session.clone()).await;
let llm = create_llm_provider(&config.llm, session).expect("Failed to create LLM provider"); let llm = create_llm_provider(&config.llm, session)
.await
.expect("Failed to create LLM provider");
println!("[5/6] LLM provider created (model: {})", llm.model_name()); println!("[5/6] LLM provider created (model: {})", llm.model_name());
// 6. Run heartbeat check // 6. Run heartbeat check