Artem
8452102454
- Implement function calling support (functionDeclarations, functionResponse) - Add functionCall SSE parsing and empty stream retry support - Add generationConfig (temperature, maxOutputTokens) - Add thinkingConfig for Gemini 3 and thinking models - Add toolConfig (functionCallingConfig.mode) - Fix .expect() panics with .ok_or_else() - Restrict oauth credentials file permissions to 0600 - Update docs and FEATURE_PARITY.md - Update wizard to current Gemini 3.1 and 2.5 models
5.9 KiB
LLM Provider Configuration
IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible endpoint as well as Anthropic, Ollama, and Google Gemini directly. This guide covers the most common configurations.
Provider Overview
| Provider | Backend value | Requires API key | Notes |
|---|---|---|---|
| NEAR AI | nearai |
OAuth (browser) | Default; multi-model |
| Anthropic | anthropic |
ANTHROPIC_API_KEY |
Claude models |
| OpenAI | openai |
OPENAI_API_KEY |
GPT models |
| Google Gemini | gemini_oauth |
OAuth (browser) | Gemini models; function calling |
| Ollama | ollama |
No | Local inference |
| OpenRouter | openai_compatible |
LLM_API_KEY |
300+ models |
| Together AI | openai_compatible |
LLM_API_KEY |
Fast inference |
| Fireworks AI | openai_compatible |
LLM_API_KEY |
Fast inference |
| vLLM / LiteLLM | openai_compatible |
Optional | Self-hosted |
| LM Studio | openai_compatible |
No | Local GUI |
NEAR AI (default)
No additional configuration required. On first run, ironclaw onboard opens a browser
for OAuth authentication. Credentials are saved to ~/.ironclaw/session.json.
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai
Anthropic (Claude)
LLM_BACKEND=anthropic
ANTHROPIC_API_KEY=sk-ant-...
Popular models: claude-sonnet-4-20250514, claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022
OpenAI (GPT)
LLM_BACKEND=openai
OPENAI_API_KEY=sk-...
Popular models: gpt-4o, gpt-4o-mini, o3-mini
Google Gemini (OAuth)
Uses Google OAuth with PKCE (S256) for authentication — no API key required.
On first run, a browser opens for Google account login. Credentials (including
refresh token) are saved to ~/.gemini/oauth_creds.json with 0600 permissions.
LLM_BACKEND=gemini_oauth
GEMINI_MODEL=gemini-2.5-flash
Supported features
| Feature | Status | Notes |
|---|---|---|
| Function calling | ✅ | functionDeclarations / functionCall / functionResponse |
generationConfig |
✅ | temperature, maxOutputTokens passed from request |
thinkingConfig |
✅ | includeThoughts: true for gemini-3/thinking models |
toolConfig |
✅ | functionCallingConfig.mode: AUTO/ANY/NONE |
| SSE streaming | ✅ | Cloud Code API with streamGenerateContent?alt=sse |
| Token refresh | ✅ | Automatic via refresh token |
Popular models
| Model | ID | Notes |
|---|---|---|
| Gemini 3.1 Pro | gemini-3.1-pro-preview |
Latest, strongest reasoning |
| Gemini 3 Flash | gemini-3-flash-preview |
Fast preview with thinkingLevel |
| Gemini 2.5 Pro | gemini-2.5-pro |
Stable, strong reasoning |
| Gemini 2.5 Flash | gemini-2.5-flash |
Fast, good quality |
| Gemini 2.5 Flash Lite | gemini-2.5-flash-lite |
Fastest, lightweight |
Cloud Code API vs standard API
Models containing preview or gemini-3 in the name route through the
Cloud Code API (cloudcode-pa.googleapis.com) which supports SSE streaming
and project-scoped access. Other models use the standard Generative Language
API (generativelanguage.googleapis.com).
Ollama (local)
Install Ollama from ollama.com, pull a model, then:
LLM_BACKEND=ollama
OLLAMA_MODEL=llama3.2
# OLLAMA_BASE_URL=http://localhost:11434 # default
Pull a model first: ollama pull llama3.2
OpenAI-Compatible Endpoints
All providers below use LLM_BACKEND=openai_compatible. Set LLM_BASE_URL to the
provider's OpenAI-compatible endpoint and LLM_API_KEY to your API key.
OpenRouter
OpenRouter routes to 300+ models from a single API key.
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
Popular OpenRouter model IDs:
| Model | ID |
|---|---|
| Claude Sonnet 4 | anthropic/claude-sonnet-4 |
| GPT-4o | openai/gpt-4o |
| Llama 4 Maverick | meta-llama/llama-4-maverick |
| Gemini 2.0 Flash | google/gemini-2.0-flash-001 |
| Mistral Small | mistralai/mistral-small-3.1-24b-instruct |
Browse all models at openrouter.ai/models.
Together AI
Together AI provides fast inference for open-source models.
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.together.xyz/v1
LLM_API_KEY=...
LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
Popular Together AI model IDs:
| Model | ID |
|---|---|
| Llama 3.3 70B | meta-llama/Llama-3.3-70B-Instruct-Turbo |
| DeepSeek R1 | deepseek-ai/DeepSeek-R1 |
| Qwen 2.5 72B | Qwen/Qwen2.5-72B-Instruct-Turbo |
Fireworks AI
Fireworks AI offers fast inference with compound AI system support.
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.fireworks.ai/inference/v1
LLM_API_KEY=fw_...
LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
vLLM / LiteLLM (self-hosted)
For self-hosted inference servers:
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:8000/v1
LLM_API_KEY=token-abc123 # set to any string if auth is not configured
LLM_MODEL=meta-llama/Llama-3.1-8B-Instruct
LiteLLM proxy (forwards to any backend, including Bedrock, Vertex, Azure):
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:4000/v1
LLM_API_KEY=sk-...
LLM_MODEL=gpt-4o # as configured in litellm config.yaml
LM Studio (local GUI)
Start LM Studio's local server, then:
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:1234/v1
LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
# LLM_API_KEY is not required for LM Studio
Using the Setup Wizard
Instead of editing .env manually, run the onboarding wizard:
ironclaw onboard
Select "OpenAI-compatible" for OpenRouter, Together AI, Fireworks, vLLM, LiteLLM, or LM Studio. You will be prompted for the base URL and (optionally) an API key. The model name is configured in the following step.