mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
3124ab2b7f
- Add docs/LLM_PROVIDERS.md with setup instructions for all supported providers - Expand .env.example with Together AI and Fireworks AI example configs - Add "Alternative LLM Providers" section to README with quickstart snippet Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: firat.sertgoz <[email protected]>
173 lines
4.3 KiB
Markdown
173 lines
4.3 KiB
Markdown
# LLM Provider Configuration
|
|
|
|
IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible
|
|
endpoint as well as Anthropic and Ollama directly. This guide covers the most common
|
|
configurations.
|
|
|
|
## Provider Overview
|
|
|
|
| Provider | Backend value | Requires API key | Notes |
|
|
|---|---|---|---|
|
|
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
|
|
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
|
|
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
|
|
| Ollama | `ollama` | No | Local inference |
|
|
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
|
|
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
|
|
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
|
|
| vLLM / LiteLLM | `openai_compatible` | Optional | Self-hosted |
|
|
| LM Studio | `openai_compatible` | No | Local GUI |
|
|
|
|
---
|
|
|
|
## NEAR AI (default)
|
|
|
|
No additional configuration required. On first run, `ironclaw onboard` opens a browser
|
|
for OAuth authentication. Credentials are saved to `~/.ironclaw/session.json`.
|
|
|
|
```env
|
|
NEARAI_MODEL=claude-3-5-sonnet-20241022
|
|
NEARAI_BASE_URL=https://private.near.ai
|
|
```
|
|
|
|
---
|
|
|
|
## Anthropic (Claude)
|
|
|
|
```env
|
|
LLM_BACKEND=anthropic
|
|
ANTHROPIC_API_KEY=sk-ant-...
|
|
```
|
|
|
|
Popular models: `claude-sonnet-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
|
|
|
|
---
|
|
|
|
## OpenAI (GPT)
|
|
|
|
```env
|
|
LLM_BACKEND=openai
|
|
OPENAI_API_KEY=sk-...
|
|
```
|
|
|
|
Popular models: `gpt-4o`, `gpt-4o-mini`, `o3-mini`
|
|
|
|
---
|
|
|
|
## Ollama (local)
|
|
|
|
Install Ollama from [ollama.com](https://ollama.com), pull a model, then:
|
|
|
|
```env
|
|
LLM_BACKEND=ollama
|
|
OLLAMA_MODEL=llama3.2
|
|
# OLLAMA_BASE_URL=http://localhost:11434 # default
|
|
```
|
|
|
|
Pull a model first: `ollama pull llama3.2`
|
|
|
|
---
|
|
|
|
## OpenAI-Compatible Endpoints
|
|
|
|
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
|
|
provider's OpenAI-compatible endpoint and `LLM_API_KEY` to your API key.
|
|
|
|
### OpenRouter
|
|
|
|
[OpenRouter](https://openrouter.ai) routes to 300+ models from a single API key.
|
|
|
|
```env
|
|
LLM_BACKEND=openai_compatible
|
|
LLM_BASE_URL=https://openrouter.ai/api/v1
|
|
LLM_API_KEY=sk-or-...
|
|
LLM_MODEL=anthropic/claude-sonnet-4
|
|
```
|
|
|
|
Popular OpenRouter model IDs:
|
|
|
|
| Model | ID |
|
|
|---|---|
|
|
| Claude Sonnet 4 | `anthropic/claude-sonnet-4` |
|
|
| GPT-4o | `openai/gpt-4o` |
|
|
| Llama 4 Maverick | `meta-llama/llama-4-maverick` |
|
|
| Gemini 2.0 Flash | `google/gemini-2.0-flash-001` |
|
|
| Mistral Small | `mistralai/mistral-small-3.1-24b-instruct` |
|
|
|
|
Browse all models at [openrouter.ai/models](https://openrouter.ai/models).
|
|
|
|
### Together AI
|
|
|
|
[Together AI](https://www.together.ai) provides fast inference for open-source models.
|
|
|
|
```env
|
|
LLM_BACKEND=openai_compatible
|
|
LLM_BASE_URL=https://api.together.xyz/v1
|
|
LLM_API_KEY=...
|
|
LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
|
|
```
|
|
|
|
Popular Together AI model IDs:
|
|
|
|
| Model | ID |
|
|
|---|---|
|
|
| Llama 3.3 70B | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
|
|
| DeepSeek R1 | `deepseek-ai/DeepSeek-R1` |
|
|
| Qwen 2.5 72B | `Qwen/Qwen2.5-72B-Instruct-Turbo` |
|
|
|
|
### Fireworks AI
|
|
|
|
[Fireworks AI](https://fireworks.ai) offers fast inference with compound AI system support.
|
|
|
|
```env
|
|
LLM_BACKEND=openai_compatible
|
|
LLM_BASE_URL=https://api.fireworks.ai/inference/v1
|
|
LLM_API_KEY=fw_...
|
|
LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
|
|
```
|
|
|
|
### vLLM / LiteLLM (self-hosted)
|
|
|
|
For self-hosted inference servers:
|
|
|
|
```env
|
|
LLM_BACKEND=openai_compatible
|
|
LLM_BASE_URL=http://localhost:8000/v1
|
|
LLM_API_KEY=token-abc123 # set to any string if auth is not configured
|
|
LLM_MODEL=meta-llama/Llama-3.1-8B-Instruct
|
|
```
|
|
|
|
LiteLLM proxy (forwards to any backend, including Bedrock, Vertex, Azure):
|
|
|
|
```env
|
|
LLM_BACKEND=openai_compatible
|
|
LLM_BASE_URL=http://localhost:4000/v1
|
|
LLM_API_KEY=sk-...
|
|
LLM_MODEL=gpt-4o # as configured in litellm config.yaml
|
|
```
|
|
|
|
### LM Studio (local GUI)
|
|
|
|
Start LM Studio's local server, then:
|
|
|
|
```env
|
|
LLM_BACKEND=openai_compatible
|
|
LLM_BASE_URL=http://localhost:1234/v1
|
|
LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
|
|
# LLM_API_KEY is not required for LM Studio
|
|
```
|
|
|
|
---
|
|
|
|
## Using the Setup Wizard
|
|
|
|
Instead of editing `.env` manually, run the onboarding wizard:
|
|
|
|
```bash
|
|
ironclaw onboard
|
|
```
|
|
|
|
Select **"OpenAI-compatible"** for OpenRouter, Together AI, Fireworks, vLLM, LiteLLM,
|
|
or LM Studio. You will be prompted for the base URL and (optionally) an API key.
|
|
The model name is configured in the following step.
|