Repository navigation
fix: finish chat responses when the model stream fails - #1445
Open
tuanzirwar wants to merge 1 commit into
Open
tuanzirwar wants to merge 1 commit into
tuanzirwar wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1362 (provider failure handling, not its additional retry/model-switch suggestions).
Provider creation and iteration errors currently bypass the normal
event_generator()completion path. An offline model or a broken response stream can omit the terminal events and leave the disconnect-monitor task running.Protect both response creation and iteration, emit a user-visible failure through the existing message protocol, and finish with END_LLM_RESPONSE/END_RESPONSE. Close the provider iterator and stop the monitor in
finally, including consumer shutdown/cancellation. Preserve partial visible text, but do not save failed output as a successfully completed conversation. No retry or automatic model switching is introduced.Validation: 18 scoped API-generator tests pass across HTTP/WebSocket, stream=True/False, failure before the first chunk, mid-response failure, creation failure, normal completion, early close and cancellation. Tests check terminal event ordering, monitor completion and save behavior. Original-code failures are recorded separately. A real local TCP HTTP flow through the actual API generator and OpenAI SDK exercises a refused provider connection, truncated chunked SSE after partial text, and a complete successful response. Failures terminate visibly without a successful save; the normal response saves once. Ruff 0.12.3 lint/format and diff checks pass.
AI-assisted implementation and automated checks. Database adapters are isolated; PostgreSQL, a real Ollama server and the complete production application were not deployed.