feat: Reuse Codex CLI OAuth tokens for ChatGPT backend LLM calls (#693)

* feat: add Codex auth.json token reuse for LLM authentication

When LLM_USE_CODEX_AUTH=true, IronClaw reads the Codex CLI's auth.json
(default ~/.codex/auth.json) and extracts the API key or OAuth access
token. This lets IronClaw piggyback on a Codex login without
implementing its own OAuth flow.

New env vars:
  - LLM_USE_CODEX_AUTH: enable Codex auth fallback (default: false)
  - CODEX_AUTH_PATH: override path to auth.json

* fix: handle ChatGPT auth mode correctly

Switch base_url to chatgpt.com/backend-api/codex when auth.json
contains ChatGPT OAuth tokens. The access_token is a JWT that only
works against the private ChatGPT backend, not the public OpenAI API.

Refactored codex_auth.rs to return CodexCredentials (token +
is_chatgpt_mode) instead of just a string key.

* fix: Codex auth takes highest priority over secrets store

When LLM_USE_CODEX_AUTH=true, Codex credentials are now loaded before
checking env vars or the secrets store overlay. Previously the secrets
store key (injected during onboarding) would shadow the Codex token.

* feat: Responses API provider for ChatGPT backend

- New CodexChatGptProvider speaks the Responses API protocol
- Auto-detects model from /models endpoint (gpt-4o -> gpt-5.2-codex)
- Adds store=false (required by ChatGPT backend)
- Error handling with timeout for HTTP 400 responses
- Message format translation: Chat Completions -> Responses API
- SSE response parsing for text, tool calls, and usage stats
- 7 unit tests for message conversion and SSE parsing

* fix: SSE parser uses item_id instead of call_id for tool call deltas

The Responses API sends function_call_arguments.delta events with
item_id (e.g. fc_...) not call_id (e.g. call_...). The parser now
keys pending tool calls by item_id from output_item.added and
tracks call_id separately for result matching.

* fix: strip empty string values from tool call arguments

gpt-5.2-codex fills optional tool parameters with empty strings
(e.g. timestamp: ""), which IronClaw's tool validation rejects.
Strip them before passing to tool execution.

* fix: prevent apiKey mode fallback to ChatGPT token

When auth_mode is explicitly 'apiKey' but the key is missing/empty,
do not fall through to check for a ChatGPT access_token. This prevents
returning credentials with is_chatgpt_mode: true and routing to the
wrong LLM provider.

* refactor: reuse single reqwest::Client across model discovery and LLM calls

Create Client once in with_auto_model, pass &Client to
fetch_default_model, and move it into the provider struct.
Eliminates the redundant Client::new() that wasted a connection pool.

* fix: bump client_version to 1.0.0 to unlock gpt-5.3-codex and gpt-5.4

The /models endpoint gates newer models behind client_version.
Version 0.1.0 only returns up to gpt-5.2-codex, while 1.0.0+
also returns gpt-5.3-codex and gpt-5.4.

* feat: user-configured LLM_MODEL takes priority over auto-detection

Fetch the full model list from /models endpoint. If LLM_MODEL is set,
validate it against the supported list and warn with available models
if not found. If LLM_MODEL is not set, auto-detect the highest-priority
model. Also bumps client_version to 1.0.0 to unlock gpt-5.3/5.4.

* fix: add 10s timeout to model discovery HTTP request

Prevents startup from blocking indefinitely if chatgpt.com
is slow or unreachable. Uses reqwest per-request timeout.

* docs: add private API warning for ChatGPT backend endpoint

The chatgpt.com/backend-api/codex endpoint is private and
undocumented. Add warning in module docs and a runtime log
on first use to inform users of potential ToS implications.

* feat: implement OAuth 401 token refresh for Codex ChatGPT provider

On HTTP 401, if a refresh_token is available, the provider now
automatically refreshes the access token via auth.openai.com/oauth/token
(same protocol as Codex CLI) and retries the request once. Refreshed
tokens are persisted back to auth.json.

Changes:
- codex_auth: read refresh_token, add refresh_access_token() and
  persist_refreshed_tokens()
- codex_chatgpt: RwLock for api_key, 401 detection + retry in
  send_request, send_http_request helper
- config/llm: thread refresh_token/auth_path through RegistryProviderConfig
- llm/mod: pass refresh params to with_auto_model

* refactor: lazy model detection via OnceCell, remove block_in_place

Model is no longer resolved during provider construction. Instead,
resolve_model() uses tokio::sync::OnceCell to lazily fetch from
/models on the first LLM call. This eliminates the block_in_place
+ block_on workaround in create_codex_chatgpt_from_registry.

- with_auto_model (async) -> with_lazy_model (sync constructor)
- resolve_model() added with OnceCell-based lazy init
- build_request_body takes model as parameter
- model_name() returns resolved or configured_model as fallback

* feat: support multimodal content (images) in Codex ChatGPT provider

message_to_input_items now checks content_parts for user messages.
ContentPart::Text maps to input_text and ContentPart::ImageUrl maps
to input_image, matching the Responses API format used by Codex CLI.
Falls back to plain text when content_parts is empty.

Also updates client_version to 0.111.0 for /models endpoint.

Adds test: test_message_conversion_user_with_image

* refactor: move codex_auth module from src/ to src/llm/

codex_auth is only used by the LLM layer (codex_chatgpt provider
and config/llm). Moving it under src/llm/ reflects its actual scope.

- Remove pub mod codex_auth from lib.rs
- Add pub mod codex_auth to llm/mod.rs
- Update imports: super::codex_auth, crate::llm::codex_auth

* Fix codex provider style issues

* Use SecretString throughout codex auth refresh flow

* Use SecretString for codex access tokens

* Reuse provider client for codex token refresh

* Stream Codex SSE responses incrementally

* Fix Windows clippy and SQLite test linkage

* Trigger checks after regression skip label

* Tighten codex auth module handling
This commit is contained in:
ZeroTrust
2026-03-16 07:43:45 +00:00
committed by GitHub
parent 81724cad93
commit 1b59eb6b39
11 changed files with 1429 additions and 21 deletions
+49 -19
View File
@@ -9,7 +9,6 @@ use crate::llm::config::*;
use crate::llm::registry::{ProviderProtocol, ProviderRegistry};
use crate::llm::session::SessionConfig;
use crate::settings::Settings;
impl LlmConfig {
/// Create a test-friendly config without reading env vars.
#[cfg(feature = "libsql")]
@@ -241,8 +240,30 @@ impl LlmConfig {
)
};
// Resolve API key from env
let api_key = if let Some(env_var) = api_key_env {
// Codex auth.json override: when LLM_USE_CODEX_AUTH=true,
// credentials from the Codex CLI's auth.json take highest priority
// (over env vars AND secrets store). In ChatGPT mode, the base URL
// is also overridden to the private ChatGPT backend endpoint.
let mut codex_base_url_override: Option<String> = None;
let codex_creds = if parse_optional_env("LLM_USE_CODEX_AUTH", false)? {
let path = optional_env("CODEX_AUTH_PATH")?
.map(std::path::PathBuf::from)
.unwrap_or_else(crate::llm::codex_auth::default_codex_auth_path);
crate::llm::codex_auth::load_codex_credentials(&path)
} else {
None
};
let codex_refresh_token = codex_creds.as_ref().and_then(|c| c.refresh_token.clone());
let codex_auth_path = codex_creds.as_ref().and_then(|c| c.auth_path.clone());
let api_key = if let Some(creds) = codex_creds {
if creds.is_chatgpt_mode {
codex_base_url_override = Some(creds.base_url().to_string());
}
Some(creds.token)
} else if let Some(env_var) = api_key_env {
// Resolve API key from env (including secrets store overlay)
optional_env(env_var)?.map(SecretString::from)
} else {
None
@@ -259,22 +280,28 @@ impl LlmConfig {
}
}
// Resolve base URL: env var > settings (backward compat) > registry default
let base_url = if let Some(env_var) = base_url_env {
optional_env(env_var)?
} else {
None
}
.or_else(|| {
// Backward compat: check legacy settings fields
match backend {
"ollama" => settings.ollama_base_url.clone(),
"openai_compatible" | "openrouter" => settings.openai_compatible_base_url.clone(),
_ => None,
}
})
.or_else(|| default_base_url.map(String::from))
.unwrap_or_default();
// Resolve base URL: codex override > env var > settings (backward compat) > registry default
let is_codex_chatgpt = codex_base_url_override.is_some();
let base_url = codex_base_url_override
.or_else(|| {
if let Some(env_var) = base_url_env {
optional_env(env_var).ok().flatten()
} else {
None
}
})
.or_else(|| {
// Backward compat: check legacy settings fields
match backend {
"ollama" => settings.ollama_base_url.clone(),
"openai_compatible" | "openrouter" => {
settings.openai_compatible_base_url.clone()
}
_ => None,
}
})
.or_else(|| default_base_url.map(String::from))
.unwrap_or_default();
if base_url_required
&& base_url.is_empty()
@@ -340,6 +367,9 @@ impl LlmConfig {
model,
extra_headers,
oauth_token,
is_codex_chatgpt,
refresh_token: codex_refresh_token,
auth_path: codex_auth_path,
cache_retention,
unsupported_params,
})