Skip to content

fix: finish chat responses when the model stream fails - #1445

Open
tuanzirwar wants to merge 1 commit into
khoj-ai:masterfrom
tuanzirwar:codex/finish-failed-model-streams
Open

tuanzirwar wants to merge 1 commit into
khoj-ai:masterfrom
tuanzirwar:codex/finish-failed-model-streams

Conversation

@tuanzirwar

Copy link
Copy Markdown

Refs #1362 (provider failure handling, not its additional retry/model-switch suggestions).

Provider creation and iteration errors currently bypass the normal event_generator() completion path. An offline model or a broken response stream can omit the terminal events and leave the disconnect-monitor task running.

Protect both response creation and iteration, emit a user-visible failure through the existing message protocol, and finish with END_LLM_RESPONSE/END_RESPONSE. Close the provider iterator and stop the monitor in finally, including consumer shutdown/cancellation. Preserve partial visible text, but do not save failed output as a successfully completed conversation. No retry or automatic model switching is introduced.

Validation: 18 scoped API-generator tests pass across HTTP/WebSocket, stream=True/False, failure before the first chunk, mid-response failure, creation failure, normal completion, early close and cancellation. Tests check terminal event ordering, monitor completion and save behavior. Original-code failures are recorded separately. A real local TCP HTTP flow through the actual API generator and OpenAI SDK exercises a refused provider connection, truncated chunked SSE after partial text, and a complete successful response. Failures terminate visibly without a successful save; the normal response saves once. Ruff 0.12.3 lint/format and diff checks pass.

AI-assisted implementation and automated checks. Database adapters are isolated; PostgreSQL, a real Ollama server and the complete production application were not deployed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant