luulam/9router
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(providers): self-hosted OpenAI-compatible STT, TTS and embedding providers

Browse Source 
Add Self-hosted STT/TTS/Embedding providers that read baseUrl per connection
instead of a fixed registry endpoint, so 9Router can point at whisper.cpp,
faster-whisper, Kokoro-FastAPI, llama-server, vLLM, Infinity, and similar
OpenAI-compatible local servers.

Self-hosted Embedding refuses to run without a baseUrl rather than falling
back to api.openai.com like openaiCompatNode does, since that fallback would
silently send input text and the API key to OpenAI under a provider named
"Self-hosted". Also fixes embeddingsCore to catch adapter build errors as a
400 instead of letting them escape uncaught, and bounds the upstream fetch
with FETCH_CONNECT_TIMEOUT_MS to avoid hanging forever on a dead endpoint.

Self-hosted TTS treats a bare model value as the model rather than the voice,
since the generic OpenAI TTS convention (bare = voice) is backwards for a
provider where the model is the variable part.
This commit is contained in:
omar-nahhas
2026-08-05 13:33:28 +07:00
committed by decolua
parent b480892952
commit fe547f4dc0
12 changed files with 387 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
open-sse/handlers/embeddingsCore.js
View File

@@ -1,5 +1,5 @@
import { createErrorResult, parseUpstreamError, formatProviderError } from "../utils/error.js";
import { HTTP_STATUS } from "../config/runtimeConfig.js";
import { HTTP_STATUS, FETCH_CONNECT_TIMEOUT_MS } from "../config/runtimeConfig.js";
import { getExecutor } from "../executors/index.js";
import { refreshWithRetry } from "../services/tokenRefresh.js";
import { getEmbeddingAdapter } from "./embeddingProviders/index.js";
@@ -38,13 +38,24 @@ export async function handleEmbeddingsCore({
}
const ctx = { input };
const url = adapter.buildUrl(model, credentials, ctx);
const headers = adapter.buildHeaders(credentials, ctx);
const requestBody = adapter.buildBody(model, {
input,
encoding_format: body.encoding_format || "float",
dimensions: body.dimensions,
});
// buildUrl/buildHeaders/buildBody were called bare. An adapter that rejects a
// misconfigured connection — selfhosted-embedding throws when no baseUrl is set
// rather than silently falling back to api.openai.com — would have escaped this
// function uncaught, surfacing as a 500 or a request that never settles. A
// configuration mistake is a 400 with the reason in it.
let url, headers, requestBody;
try {
url = adapter.buildUrl(model, credentials, ctx);
headers = adapter.buildHeaders(credentials, ctx);
requestBody = adapter.buildBody(model, {
input,
encoding_format: body.encoding_format || "float",
dimensions: body.dimensions,
});
} catch (error) {
log?.debug?.("EMBEDDINGS", `Request build failed: ${error.message}`);
return createErrorResult(HTTP_STATUS.BAD_REQUEST, `[${provider}/${model}] ${error.message}`);
}
log?.debug?.("EMBEDDINGS", `${provider.toUpperCase()} | ${model} | input_type=${Array.isArray(input) ? `array[${input.length}]` : "string"}`);
@@ -54,6 +65,9 @@ export async function handleEmbeddingsCore({
method: "POST",
headers,
body: JSON.stringify(requestBody),
...(typeof AbortSignal?.timeout === "function"
? { signal: AbortSignal.timeout(FETCH_CONNECT_TIMEOUT_MS) }
: {}),
});
} catch (error) {
const errMsg = formatProviderError(error, provider, model, HTTP_STATUS.BAD_GATEWAY);