luulam/9router
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(stream): report aborts after HTTP 200 in-band instead of closing silently

Browse Source 
A stream that stalled or lost its upstream was closed with no terminal frame
at all, so clients saw "200 OK, a few chunks, then nothing" and could not tell
a truncated reply from a finished one. The Responses passthrough path already
synthesized response.failed; every other client format got nothing.

The watchdog now hands its reason ("stream stall timeout" or "upstream
connection lost") to onAbortTerminal, and buildStreamErrorBytes frames it per
client format: OpenAI-compatible clients get data: {"error":{...}} followed by
data: [DONE], Anthropic clients get `event: error`. The error frame always
precedes [DONE] (openai-python raises APIError on any data payload carrying an
error key), and no synthetic finish_reason is ever emitted — a truncated
stream must not look like a clean stop.

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
decolua
2026-09-16 20:04:10 +07:00
parent 5c399b6406
commit 9300121366
4 changed files with 111 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
open-sse/handlers/chatCore/streamingHandler.js
View File

@@ -3,8 +3,9 @@ import { needsTranslation } from "../../translator/index.js";
import { createSSETransformStreamWithLogger, createPassthroughStreamWithLogger } from "../../utils/stream.js";
import { pipeWithDisconnect } from "../../utils/streamHandler.js";
import { PROVIDERS } from "../../config/providers.js";
import { STREAM_STALL_TIMEOUT_MS } from "../../config/runtimeConfig.js";
import { HTTP_STATUS, STREAM_STALL_TIMEOUT_MS } from "../../config/runtimeConfig.js";
import { buildAbortedResponsesTerminalBytes } from "../../utils/responsesStreamHelpers.js";
import { buildStreamErrorBytes } from "../../utils/streamHelpers.js";
import { buildRequestDetail, extractRequestConfig, saveUsageStats, formatDoneLine } from "./requestDetail.js";
import { saveRequestDetail } from "@/lib/usageDb.js";
import { SSE_HEADERS_CORS as SSE_HEADERS } from "../../utils/sseConstants.js";
@@ -81,9 +82,14 @@ export async function handleStreamingResponse({ providerResponse, provider, mode
const transformStream = buildTransformStream({ provider, sourceFormat, targetFormat, userAgent, reqLogger, toolNameMap, customToolNames, model, connectionId, body, onStreamComplete, apiKey, credentials });
// Responses passthrough: synthesize response.failed + [DONE] if the stream aborts/stalls before a terminal event
// Terminal bytes when the stream aborts after HTTP 200 was already sent, so the
// client sees a real error instead of a silently truncated stream.
// Responses passthrough keeps its own response.failed shape; every other client
// format gets the OpenAI error frame + [DONE], or `event: error` for Claude.
const isResponsesPassthrough = sourceFormat === FORMATS.OPENAI_RESPONSES && targetFormat === FORMATS.OPENAI_RESPONSES;
const onAbortTerminal = isResponsesPassthrough ? buildAbortedResponsesTerminalBytes : null;
const onAbortTerminal = isResponsesPassthrough
? buildAbortedResponsesTerminalBytes
: (message) => buildStreamErrorBytes(HTTP_STATUS.GATEWAY_TIMEOUT, message, sourceFormat);
const stallTimeoutMs = PROVIDERS[provider]?.stallTimeoutMs || STREAM_STALL_TIMEOUT_MS;
const transformedBody = pipeWithDisconnect(providerResponse, transformStream, streamController, onAbortTerminal, stallTimeoutMs);