Track per-provider failure state with lock-free atomics and temporarily
skip providers that have repeatedly failed with retryable errors. This
reduces latency when a provider is known to be down, instead of
wasting time on every request trying all providers sequentially.
- Add CooldownConfig (duration + threshold) and ProviderCooldown (atomics)
- Rewrite try_providers() to skip cooled-down providers, with a safety
net that always tries the oldest-cooled provider if all are down
- Add 2 env vars: LLM_FAILOVER_COOLDOWN_SECS, LLM_FAILOVER_THRESHOLD
- Add MultiCallMockProvider and 7 new test cases
- Mark "Cooldown management" as complete in FEATURE_PARITY.md
Co-authored-by: Claude Opus 4.6 <[email protected]>
* feat: add multi-provider LLM failover
Add FailoverProvider that wraps multiple LlmProvider instances and
tries each in sequence on transient failures. Non-retryable errors
(auth, context length, model not available) propagate immediately.
- New `FailoverProvider` with generic `try_providers` helper
- `is_retryable()` classifies transient errors (request failed,
rate limited, invalid response, session renewal, HTTP, IO)
- Configurable via `NEARAI_FALLBACK_MODEL` env var
- Returns `Result` from constructor (no panics in production)
- Updates FEATURE_PARITY.md: failover chains ✅, cooldown ❌
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: track last-used provider for accurate cost/model reporting
After failover, model_name() and cost_per_token() now reflect the
provider that actually handled the request, not always the primary.
Also corrects is_retryable() docs to list ModelNotAvailable as retryable.
Addresses PR #28 review comments.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* feat: add retry with exponential backoff for LLM providers
Add retry logic with exponential backoff and jitter to both NearAiProvider
and NearAiChatProvider for transient errors (HTTP 429, 500, 502, 503, 504).
Extract shared retry helpers (is_retryable_status, retry_backoff_delay)
into src/llm/retry.rs so both providers reuse the same logic.
Configurable via NEARAI_MAX_RETRIES env var (default: 3).
* docs: clarify max_retries means N retries, not N total attempts
* warn when fallback model equals primary model
* fix: saturating_mul in backoff delay, dedupe to_lowercase allocation
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
