feat: support per-request model override in /v1/chat/completions (#103)

* feat: support per-request model override for /v1/chat/completions

- add optional model override to completion request types\n- forward request model through gateway, worker, and orchestrator proxy paths\n- use request model in NEAR AI providers with fallback to active model\n- replace model-mismatch integration test with override propagation checks\n- update FEATURE_PARITY.md note for OpenAI-compatible API behavior\n\nRefs #49

* Wire gateway OpenAI-compatible routes to active LLM provider

* Validate OpenAI model name length before streaming

* Address PR103 review feedback on model override and validation

* Report effective model in OpenAI-compatible responses

* Use async mutexes in OpenAI compatibility integration tests

* fix tests for per-request model field in response cache

* fix formatting and clippy lint after main merge

* Fix model override reporting and cache correctness

---------

Co-authored-by: Illia Polosukhin <[email protected]>
This commit is contained in:
Raahim Salman
2026-02-19 21:45:37 +00:00
committed by GitHub
co-authored by Illia Polosukhin
parent 89fdd81420
commit ccf60055f4
13 changed files with 519 additions and 62 deletions
+2
View File
@@ -143,6 +143,7 @@ async fn llm_complete(
) -> Result<Json<ProxyCompletionResponse>, StatusCode> {
let completion_req = CompletionRequest {
messages: req.messages,
model: req.model,
max_tokens: req.max_tokens,
temperature: req.temperature,
stop_sequences: req.stop_sequences,
@@ -170,6 +171,7 @@ async fn llm_complete_with_tools(
let tool_req = ToolCompletionRequest {
messages: req.messages,
tools: req.tools,
model: req.model,
max_tokens: req.max_tokens,
temperature: req.temperature,
tool_choice: req.tool_choice,