mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
docs: add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) (#193)
- Add docs/LLM_PROVIDERS.md with setup instructions for all supported providers - Expand .env.example with Together AI and Fireworks AI example configs - Add "Alternative LLM Providers" section to README with quickstart snippet Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: firat.sertgoz <[email protected]>
This commit is contained in:
Nitanshu Lokhande
co-authored by
Claude Sonnet 4.6
firat.sertgoz
Claude Sonnet 4.6
firat.sertgoz
parent
dbd3e0807f
commit
3124ab2b7f
+16
-2
@@ -33,13 +33,27 @@ NEARAI_AUTH_URL=https://private.near.ai
|
||||
# LLM_BASE_URL=http://localhost:1234/v1
|
||||
# LLM_API_KEY=sk-... # optional for local servers
|
||||
|
||||
# === OpenRouter (via OpenAI-compatible) ===
|
||||
# LLM_MODEL=anthropic/claude-sonnet-4
|
||||
# === OpenRouter (300+ models via OpenAI-compatible) ===
|
||||
# LLM_MODEL=anthropic/claude-sonnet-4 # see openrouter.ai/models for IDs
|
||||
# LLM_BACKEND=openai_compatible
|
||||
# LLM_BASE_URL=https://openrouter.ai/api/v1
|
||||
# LLM_API_KEY=sk-or-...
|
||||
# LLM_EXTRA_HEADERS=HTTP-Referer:https://myapp.com,X-Title:MyApp
|
||||
|
||||
# === Together AI (via OpenAI-compatible) ===
|
||||
# LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
|
||||
# LLM_BACKEND=openai_compatible
|
||||
# LLM_BASE_URL=https://api.together.xyz/v1
|
||||
# LLM_API_KEY=...
|
||||
|
||||
# === Fireworks AI (via OpenAI-compatible) ===
|
||||
# LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
|
||||
# LLM_BACKEND=openai_compatible
|
||||
# LLM_BASE_URL=https://api.fireworks.ai/inference/v1
|
||||
# LLM_API_KEY=fw_...
|
||||
|
||||
# For full provider setup guide see docs/LLM_PROVIDERS.md
|
||||
|
||||
|
||||
# Channel Configuration
|
||||
# CLI is always enabled
|
||||
|
||||
@@ -143,6 +143,23 @@ and secrets encryption (using your system keychain). Settings are persisted in t
|
||||
connected database; bootstrap variables (e.g. `DATABASE_URL`, `LLM_BACKEND`) are
|
||||
written to `~/.ironclaw/.env` so they are available before the database connects.
|
||||
|
||||
### Alternative LLM Providers
|
||||
|
||||
IronClaw defaults to NEAR AI but works with any OpenAI-compatible endpoint.
|
||||
Popular options include **OpenRouter** (300+ models), **Together AI**, **Fireworks AI**,
|
||||
**Ollama** (local), and self-hosted servers like **vLLM** or **LiteLLM**.
|
||||
|
||||
Select *"OpenAI-compatible"* in the wizard, or set environment variables directly:
|
||||
|
||||
```env
|
||||
LLM_BACKEND=openai_compatible
|
||||
LLM_BASE_URL=https://openrouter.ai/api/v1
|
||||
LLM_API_KEY=sk-or-...
|
||||
LLM_MODEL=anthropic/claude-sonnet-4
|
||||
```
|
||||
|
||||
See [docs/LLM_PROVIDERS.md](docs/LLM_PROVIDERS.md) for a full provider guide.
|
||||
|
||||
## Security
|
||||
|
||||
IronClaw implements defense in depth to protect your data and prevent misuse.
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
# LLM Provider Configuration
|
||||
|
||||
IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible
|
||||
endpoint as well as Anthropic and Ollama directly. This guide covers the most common
|
||||
configurations.
|
||||
|
||||
## Provider Overview
|
||||
|
||||
| Provider | Backend value | Requires API key | Notes |
|
||||
|---|---|---|---|
|
||||
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
|
||||
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
|
||||
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
|
||||
| Ollama | `ollama` | No | Local inference |
|
||||
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
|
||||
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
|
||||
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
|
||||
| vLLM / LiteLLM | `openai_compatible` | Optional | Self-hosted |
|
||||
| LM Studio | `openai_compatible` | No | Local GUI |
|
||||
|
||||
---
|
||||
|
||||
## NEAR AI (default)
|
||||
|
||||
No additional configuration required. On first run, `ironclaw onboard` opens a browser
|
||||
for OAuth authentication. Credentials are saved to `~/.ironclaw/session.json`.
|
||||
|
||||
```env
|
||||
NEARAI_MODEL=claude-3-5-sonnet-20241022
|
||||
NEARAI_BASE_URL=https://private.near.ai
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Anthropic (Claude)
|
||||
|
||||
```env
|
||||
LLM_BACKEND=anthropic
|
||||
ANTHROPIC_API_KEY=sk-ant-...
|
||||
```
|
||||
|
||||
Popular models: `claude-sonnet-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
|
||||
|
||||
---
|
||||
|
||||
## OpenAI (GPT)
|
||||
|
||||
```env
|
||||
LLM_BACKEND=openai
|
||||
OPENAI_API_KEY=sk-...
|
||||
```
|
||||
|
||||
Popular models: `gpt-4o`, `gpt-4o-mini`, `o3-mini`
|
||||
|
||||
---
|
||||
|
||||
## Ollama (local)
|
||||
|
||||
Install Ollama from [ollama.com](https://ollama.com), pull a model, then:
|
||||
|
||||
```env
|
||||
LLM_BACKEND=ollama
|
||||
OLLAMA_MODEL=llama3.2
|
||||
# OLLAMA_BASE_URL=http://localhost:11434 # default
|
||||
```
|
||||
|
||||
Pull a model first: `ollama pull llama3.2`
|
||||
|
||||
---
|
||||
|
||||
## OpenAI-Compatible Endpoints
|
||||
|
||||
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
|
||||
provider's OpenAI-compatible endpoint and `LLM_API_KEY` to your API key.
|
||||
|
||||
### OpenRouter
|
||||
|
||||
[OpenRouter](https://openrouter.ai) routes to 300+ models from a single API key.
|
||||
|
||||
```env
|
||||
LLM_BACKEND=openai_compatible
|
||||
LLM_BASE_URL=https://openrouter.ai/api/v1
|
||||
LLM_API_KEY=sk-or-...
|
||||
LLM_MODEL=anthropic/claude-sonnet-4
|
||||
```
|
||||
|
||||
Popular OpenRouter model IDs:
|
||||
|
||||
| Model | ID |
|
||||
|---|---|
|
||||
| Claude Sonnet 4 | `anthropic/claude-sonnet-4` |
|
||||
| GPT-4o | `openai/gpt-4o` |
|
||||
| Llama 4 Maverick | `meta-llama/llama-4-maverick` |
|
||||
| Gemini 2.0 Flash | `google/gemini-2.0-flash-001` |
|
||||
| Mistral Small | `mistralai/mistral-small-3.1-24b-instruct` |
|
||||
|
||||
Browse all models at [openrouter.ai/models](https://openrouter.ai/models).
|
||||
|
||||
### Together AI
|
||||
|
||||
[Together AI](https://www.together.ai) provides fast inference for open-source models.
|
||||
|
||||
```env
|
||||
LLM_BACKEND=openai_compatible
|
||||
LLM_BASE_URL=https://api.together.xyz/v1
|
||||
LLM_API_KEY=...
|
||||
LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
|
||||
```
|
||||
|
||||
Popular Together AI model IDs:
|
||||
|
||||
| Model | ID |
|
||||
|---|---|
|
||||
| Llama 3.3 70B | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
|
||||
| DeepSeek R1 | `deepseek-ai/DeepSeek-R1` |
|
||||
| Qwen 2.5 72B | `Qwen/Qwen2.5-72B-Instruct-Turbo` |
|
||||
|
||||
### Fireworks AI
|
||||
|
||||
[Fireworks AI](https://fireworks.ai) offers fast inference with compound AI system support.
|
||||
|
||||
```env
|
||||
LLM_BACKEND=openai_compatible
|
||||
LLM_BASE_URL=https://api.fireworks.ai/inference/v1
|
||||
LLM_API_KEY=fw_...
|
||||
LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
|
||||
```
|
||||
|
||||
### vLLM / LiteLLM (self-hosted)
|
||||
|
||||
For self-hosted inference servers:
|
||||
|
||||
```env
|
||||
LLM_BACKEND=openai_compatible
|
||||
LLM_BASE_URL=http://localhost:8000/v1
|
||||
LLM_API_KEY=token-abc123 # set to any string if auth is not configured
|
||||
LLM_MODEL=meta-llama/Llama-3.1-8B-Instruct
|
||||
```
|
||||
|
||||
LiteLLM proxy (forwards to any backend, including Bedrock, Vertex, Azure):
|
||||
|
||||
```env
|
||||
LLM_BACKEND=openai_compatible
|
||||
LLM_BASE_URL=http://localhost:4000/v1
|
||||
LLM_API_KEY=sk-...
|
||||
LLM_MODEL=gpt-4o # as configured in litellm config.yaml
|
||||
```
|
||||
|
||||
### LM Studio (local GUI)
|
||||
|
||||
Start LM Studio's local server, then:
|
||||
|
||||
```env
|
||||
LLM_BACKEND=openai_compatible
|
||||
LLM_BASE_URL=http://localhost:1234/v1
|
||||
LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
|
||||
# LLM_API_KEY is not required for LM Studio
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Using the Setup Wizard
|
||||
|
||||
Instead of editing `.env` manually, run the onboarding wizard:
|
||||
|
||||
```bash
|
||||
ironclaw onboard
|
||||
```
|
||||
|
||||
Select **"OpenAI-compatible"** for OpenRouter, Together AI, Fireworks, vLLM, LiteLLM,
|
||||
or LM Studio. You will be prompted for the base URL and (optionally) an API key.
|
||||
The model name is configured in the following step.
|
||||
Reference in New Issue
Block a user