gitops/optimclaw

Fork 0

mirror of https://github.com/outbackdingo/optimclaw.git synced 2026-08-25 14:53:34 +00:00

Files

T

d73e35cfb0 feat: add AWS Bedrock LLM provider via native Converse API (#713 )

* feat: add AWS Bedrock LLM provider via native Converse API

* fix: use JSON parsing for tool result error detection instead of brittle substring matching

* refactor: extract duplicated inference config builder into helper function

* fix: address review feedback — safe casts, input validation, and tests

- Safe u32→i32 cast for max_tokens using try_from with clamp
- Remove brittle string-based error detection fallback for tool results
- Validate BEDROCK_CROSS_REGION against allowed values (us/eu/apac/global)
- Validate message list is non-empty before Converse API call
- Log when using default us-east-1 region
- Update llm_backend doc comment to list all backends
- Add tests for build_inference_config and empty message handling

* fix: persist AWS_PROFILE for Bedrock named profile auth

The wizard collected the profile name but only printed a hint to set
it manually. Now it saves to settings and writes AWS_PROFILE to the
bootstrap .env, consistent with how BEDROCK_REGION and other Bedrock
settings are persisted.

* feat: gate AWS Bedrock behind optional `bedrock` feature flag

The AWS SDK dependencies (aws-config, aws-sdk-bedrockruntime,
aws-smithy-types) require cmake and a C compiler to build aws-lc-sys.
Gate them behind an opt-in `bedrock` feature flag so default builds
are unaffected.

Build with: cargo build --features bedrock
All config, settings, and wizard code stays unconditional (no AWS deps)
so users can configure Bedrock even without the feature compiled — they
get a clear error at startup directing them to rebuild.

* fix: address review feedback and adapt Bedrock provider to registry architecture (takeover #345)

- Resolve merge conflicts with main's registry-based provider system
- Add missing cache_creation_input_tokens/cache_read_input_tokens fields
- Add missing content_parts field in test ChatMessage
- Fix string literal type mismatches in wizard env_vars (.to_string())
- Remove non-functional bearer token auth (AWS_BEARER_TOKEN_BEDROCK) from
  wizard and documentation per reviewer feedback from @zmanian and @serrrfirat
- Remove stale BEDROCK_ACCESS_KEY proxy entry from provider table
- Update Bedrock provider to use is_bedrock string check (LlmBackend enum removed)
- Add bedrock_profile fallback from settings in config resolution

[skip-regression-check]

Co-Authored-By: cgorski <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use main's Cargo.lock as base to preserve dependency versions

Regenerating Cargo.lock from scratch caused transitive dependency version
drift that broke the html_to_markdown fixture test in CI.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: bedrock config bugs — spurious warning, alias normalization, profile fallback

- Move is_bedrock check before unknown-backend warning to prevent
  spurious "unknown backend" log for bedrock users
- Normalize backend aliases ("aws", "aws_bedrock") to "bedrock" so
  the provider factory matches correctly
- Add settings.bedrock_profile fallback for AWS_PROFILE, consistent
  with region and cross_region resolution

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address Copilot review feedback — bearer token cleanup, stop_sequences, model dedup

- Remove stale bearer token refs from setup README and CHANGELOG
- Remove dead bedrock_api_key secret injection mapping
- Pass stop_sequences through to Bedrock InferenceConfiguration
- Remove "API key" from wizard menu description (bearer token removed)
- Skip duplicate LLM_MODEL write for bedrock backend in wizard
- Fix cargo fmt formatting

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review feedback — async new(), remove LiteLLM entry, wizard fixes

- Remove dead LiteLLM-based bedrock entry from providers.json (native
  Converse API intercepts before registry lookup)
- Make BedrockProvider::new() async to avoid block_in_place panic in
  current_thread runtimes; propagate async to create_llm_provider,
  build_provider_chain, and init_llm
- Document CMake build prerequisite in docs/LLM_PROVIDERS.md
- Clear bedrock_profile when user selects "default credentials" in wizard
- Fix selected_model clearing to match established pattern (conditional
  on provider switch, not unconditional)
- Add regression tests for bedrock model preservation and profile clearing

Addresses review feedback from @zmanian on PR #713.
Streaming support tracked in #741.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address remaining review comments — CLAUDE.md backends, wizard UX

- Add `bedrock` to CLAUDE.md inline backend list (#10)
- Skip full setup re-run when keeping existing Bedrock config (#11)
- Clear stale bedrock_profile on empty named-profile input (#12)
- Add regression test for empty profile clearing

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Chris Gorski <[email protected]>
Co-authored-by: cgorski <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>

2026-03-09 07:10:25 +00:00

6.3 KiB

Raw Blame History

LLM Provider Configuration

IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible endpoint as well as Anthropic and Ollama directly. This guide covers the most common configurations.

Provider Overview

Provider	Backend value	Requires API key	Notes
NEAR AI	`nearai`	OAuth (browser)	Default; multi-model
Anthropic	`anthropic`	`ANTHROPIC_API_KEY`	Claude models
OpenAI	`openai`	`OPENAI_API_KEY`	GPT models
Google Gemini	`gemini`	`GEMINI_API_KEY`	Gemini models
io.net	`ionet`	`IONET_API_KEY`	Intelligence API
Mistral	`mistral`	`MISTRAL_API_KEY`	Mistral models
Yandex AI Studio	`yandex`	`YANDEX_API_KEY`	YandexGPT models
Cloudflare Workers AI	`cloudflare`	`CLOUDFLARE_API_KEY`	Access to Workers AI
Ollama	`ollama`	No	Local inference
AWS Bedrock	`bedrock`	AWS credentials	Native Converse API
OpenRouter	`openai_compatible`	`LLM_API_KEY`	300+ models
Together AI	`openai_compatible`	`LLM_API_KEY`	Fast inference
Fireworks AI	`openai_compatible`	`LLM_API_KEY`	Fast inference
vLLM / LiteLLM	`openai_compatible`	Optional	Self-hosted
LM Studio	`openai_compatible`	No	Local GUI

NEAR AI (default)

No additional configuration required. On first run, ironclaw onboard opens a browser for OAuth authentication. Credentials are saved to ~/.ironclaw/session.json.

NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai

Anthropic (Claude)

LLM_BACKEND=anthropic
ANTHROPIC_API_KEY=sk-ant-...

Popular models: claude-sonnet-4-20250514, claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022

OpenAI (GPT)

LLM_BACKEND=openai
OPENAI_API_KEY=sk-...

Popular models: gpt-4o, gpt-4o-mini, o3-mini

Ollama (local)

Install Ollama from ollama.com, pull a model, then:

LLM_BACKEND=ollama
OLLAMA_MODEL=llama3.2
# OLLAMA_BASE_URL=http://localhost:11434   # default

Pull a model first: ollama pull llama3.2

AWS Bedrock (requires `--features bedrock`)

Uses the native AWS Converse API via aws-sdk-bedrockruntime. Supports standard AWS authentication methods: IAM credentials, SSO profiles, and instance roles.

Build prerequisite: The aws-lc-sys crate (transitive dependency via AWS SDK) requires CMake to compile. Install it before building with --features bedrock:

macOS: brew install cmake

Ubuntu/Debian: sudo apt install cmake

Fedora: sudo dnf install cmake

With AWS credentials (IAM, SSO, instance roles)

LLM_BACKEND=bedrock
BEDROCK_MODEL=anthropic.claude-opus-4-6-v1
BEDROCK_REGION=us-east-1
BEDROCK_CROSS_REGION=us
# AWS_PROFILE=my-sso-profile   # optional, for named profiles

The AWS SDK credential chain automatically resolves credentials from environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), shared credentials file (~/.aws/credentials), SSO profiles, and EC2/ECS instance roles.

Cross-region inference

Set BEDROCK_CROSS_REGION to route requests across AWS regions for capacity:

Prefix	Routing
`us`	US regions (us-east-1, us-east-2, us-west-2)
`eu`	European regions
`apac`	Asia-Pacific regions
`global`	All commercial AWS regions
(unset)	Single-region only

Popular Bedrock model IDs

Model	ID
Claude Opus 4.6	`anthropic.claude-opus-4-6-v1`
Claude Sonnet 4.5	`anthropic.claude-sonnet-4-5-20250929-v1:0`
Claude Haiku 4.5	`anthropic.claude-haiku-4-5-20251001-v1:0`
Amazon Nova Pro	`amazon.nova-pro-v1:0`
Llama 4 Maverick	`meta.llama4-maverick-17b-instruct-v1:0`

OpenAI-Compatible Endpoints

All providers below use LLM_BACKEND=openai_compatible. Set LLM_BASE_URL to the provider's OpenAI-compatible endpoint and LLM_API_KEY to your API key.

OpenRouter

OpenRouter routes to 300+ models from a single API key.

LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4

Popular OpenRouter model IDs:

Model	ID
Claude Sonnet 4	`anthropic/claude-sonnet-4`
GPT-4o	`openai/gpt-4o`
Llama 4 Maverick	`meta-llama/llama-4-maverick`
Gemini 2.0 Flash	`google/gemini-2.0-flash-001`
Mistral Small	`mistralai/mistral-small-3.1-24b-instruct`

Browse all models at openrouter.ai/models.

Together AI

Together AI provides fast inference for open-source models.

LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.together.xyz/v1
LLM_API_KEY=...
LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo

Popular Together AI model IDs:

Model	ID
Llama 3.3 70B	`meta-llama/Llama-3.3-70B-Instruct-Turbo`
DeepSeek R1	`deepseek-ai/DeepSeek-R1`
Qwen 2.5 72B	`Qwen/Qwen2.5-72B-Instruct-Turbo`

Fireworks AI

Fireworks AI offers fast inference with compound AI system support.

LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.fireworks.ai/inference/v1
LLM_API_KEY=fw_...
LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic

vLLM / LiteLLM (self-hosted)

For self-hosted inference servers:

LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:8000/v1
LLM_API_KEY=token-abc123        # set to any string if auth is not configured
LLM_MODEL=meta-llama/Llama-3.1-8B-Instruct

LiteLLM proxy (forwards to any backend, including Bedrock, Vertex, Azure):

LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:4000/v1
LLM_API_KEY=sk-...
LLM_MODEL=gpt-4o                 # as configured in litellm config.yaml

LM Studio (local GUI)

Start LM Studio's local server, then:

LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:1234/v1
LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
# LLM_API_KEY is not required for LM Studio

Using the Setup Wizard

Instead of editing .env manually, run the onboarding wizard:

ironclaw onboard

Select "OpenAI-compatible" for OpenRouter, Together AI, Fireworks, vLLM, LiteLLM, or LM Studio. You will be prompted for the base URL and (optionally) an API key. The model name is configured in the following step.

6.3 KiB Raw Blame History