feat: add cooldown management to FailoverProvider (#114)

Track per-provider failure state with lock-free atomics and temporarily
skip providers that have repeatedly failed with retryable errors. This
reduces latency when a provider is known to be down, instead of
wasting time on every request trying all providers sequentially.

- Add CooldownConfig (duration + threshold) and ProviderCooldown (atomics)
- Rewrite try_providers() to skip cooled-down providers, with a safety
  net that always tries the oldest-cooled provider if all are down
- Add 2 env vars: LLM_FAILOVER_COOLDOWN_SECS, LLM_FAILOVER_THRESHOLD
- Add MultiCallMockProvider and 7 new test cases
- Mark "Cooldown management" as complete in FEATURE_PARITY.md

Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
alexthebuildr
2026-02-17 08:00:32 +00:00
committed by GitHub
co-authored by Claude Opus 4.6
parent dfa105539b
commit 68a1851c19
6 changed files with 568 additions and 12 deletions
+2
View File
@@ -1022,6 +1022,8 @@ impl SetupWizard {
api_key: None,
fallback_model: None,
max_retries: 3,
failover_cooldown_secs: 300,
failover_cooldown_threshold: 3,
},
openai: None,
anthropic: None,