feat(providers): self-hosted OpenAI-compatible STT, TTS and embedding providers

Add Self-hosted STT/TTS/Embedding providers that read baseUrl per connection
instead of a fixed registry endpoint, so 9Router can point at whisper.cpp,
faster-whisper, Kokoro-FastAPI, llama-server, vLLM, Infinity, and similar
OpenAI-compatible local servers.

Self-hosted Embedding refuses to run without a baseUrl rather than falling
back to api.openai.com like openaiCompatNode does, since that fallback would
silently send input text and the API key to OpenAI under a provider named
"Self-hosted". Also fixes embeddingsCore to catch adapter build errors as a
400 instead of letting them escape uncaught, and bounds the upstream fetch
with FETCH_CONNECT_TIMEOUT_MS to avoid hanging forever on a dead endpoint.

Self-hosted TTS treats a bare model value as the model rather than the voice,
since the generic OpenAI TTS convention (bare = voice) is backwards for a
provider where the model is the variable part.
This commit is contained in:
omar-nahhas
2026-08-05 13:33:28 +07:00
committed by decolua
parent b480892952
commit fe547f4dc0
12 changed files with 387 additions and 9 deletions

View File

@@ -117,6 +117,9 @@ import p113 from "./morph.js";
// import p104 from "./windsurf.js";
import p115 from "./poolside.js";
import p116 from "./tokenrouter.js";
import p117 from "./selfhosted-stt.js";
import p118 from "./selfhosted-tts.js";
import p119 from "./selfhosted-embedding.js";
export default [
p0,
@@ -235,4 +238,7 @@ export default [
// p104, // windsurf — hidden, no tool calling
p115,
p116,
p117,
p118,
p119,
];

View File

@@ -0,0 +1,73 @@
// Self-hosted, OpenAI-compatible embeddings (llama.cpp / llama-server, vLLM,
// Infinity, text-embeddings-inference, ...) — the embeddings counterpart of
// selfhosted-stt and selfhosted-tts.
//
// Routing a self-hosted embeddings server already WORKS today, via a custom
// provider node: getEmbeddingAdapter() matches `openai-compatible-*` and
// `custom-embedding-*` and returns openaiCompatNode, whose buildUrl reads
// creds.providerSpecificData.baseUrl. What is missing is a first-class provider,
// and the gap is visible rather than functional:
//
// /v1/embeddings on such a node -> 200, correct vectors
// the Embedding page in the dashboard -> the node is not listed at all
//
// The page renders getProvidersByKind("embedding") plus provider nodes filtered
// to `type === "custom-embedding"`. A node created as `openai-compatible` — the
// natural choice when ONE endpoint serves chat and embeddings behind the same
// front door — satisfies neither, so a working self-hosted embeddings endpoint is
// invisible on the page whose job is to show embeddings providers. Diagnosed on a
// deployment serving Qwen3-Embedding-8B at 4096 dimensions through exactly that
// shape (2026-08-04).
//
// Declaring it as a provider with serviceKinds: ["embedding"] puts it on the page
// beside Voyage, Jina and the rest, and keeps the per-connection baseUrl that
// makes self-hosting possible at all.
//
// authType is "apikey" rather than "none" for the same reason as the STT and TTS
// entries: it is what gives the connection a credentials record, and
// providerSpecificData.baseUrl lives there. Local servers ignore the key itself;
// any non-empty value works.
export default {
id: "selfhosted-embedding",
priority: 50,
hasFree: true,
alias: "selfhosted-embedding",
display: {
name: "Self-hosted Embedding",
icon: "cloud",
color: "#ffffffff",
textIcon: "SE",
website: "https://github.com/ggml-org/llama.cpp",
},
category: "apikey",
auth: {
apiKey: {
// Note the /v1: the adapter appends "/embeddings" to whatever it is given,
// so a bare http://host:8080 resolves to http://host:8080/embeddings and
// misses the OpenAI route entirely. Give it the OpenAI base, the same value
// an OpenAI client would use. A trailing /embeddings is tolerated.
text: "Set providerSpecificData.baseUrl to the OpenAI base URL, e.g. http://host:8080/v1 — /embeddings is appended. The API key is not checked by local servers; any value works.",
},
},
// A self-hosted server serves whatever model it was started with, so the id
// here is a placeholder for the UI: the request passes `model` straight
// through, and llama-server ignores an unknown value rather than rejecting it.
// Dimensions are deliberately NOT declared — they are a property of the loaded
// weights, and asserting a number here would be a guess that silently
// contradicts the server.
models: [
{ id: "embedding", name: "Self-hosted embedding model", kind: "embedding" },
],
serviceKinds: ["embedding"],
embeddingConfig: {
// Declared for shape-consistency with the other embedding providers, and
// read by the UI — but NOT by the request path. openaiCompatNode resolves the
// URL purely from creds.providerSpecificData.baseUrl (falling back to
// api.openai.com), so unlike a fixed cloud provider this baseUrl never
// reaches the wire. Stated plainly because a reader would otherwise
// reasonably assume it is the default endpoint.
baseUrl: "http://localhost:8080/v1/embeddings",
authType: "apikey",
authHeader: "bearer",
},
};

View File

@@ -0,0 +1,48 @@
// Self-hosted, OpenAI-compatible speech-to-text (whisper.cpp, faster-whisper,
// Speaches, vLLM-served Whisper, ...).
//
// Every other STT provider here is a named cloud service with a fixed endpoint.
// This one exists so a locally-served /v1/audio/transcriptions can be used at
// all: set the connection's providerSpecificData.baseUrl to the full URL of the
// endpoint, exactly as the custom embedding providers already work.
//
// sttCore dispatches on `format`; anything that is not one of the five named
// cloud shapes falls through to transcribeOpenAICompatible, which POSTs the
// standard multipart body (file, model, and optional language / prompt /
// response_format / temperature). That is precisely what whisper.cpp's OpenAI
// endpoint accepts.
//
// authType is "apikey" rather than "none" so the connection carries a
// credentials record — which is where providerSpecificData.baseUrl lives. Local
// servers ignore the key itself; any non-empty value works.
export default {
id: "selfhosted-stt",
priority: 50,
hasFree: true,
alias: "selfhosted-stt",
display: {
name: "Self-hosted STT",
icon: "cloud",
color: "#ffffffff",
textIcon: "ST",
website: "https://github.com/ggml-org/whisper.cpp",
},
category: "apikey",
auth: {
apiKey: {
text: "Set providerSpecificData.baseUrl to the full transcriptions URL, e.g. http://host:8080/v1/audio/transcriptions. The API key is not checked by local servers; any value works.",
},
},
models: [
{ id: "whisper-1", name: "Whisper (self-hosted)", params: ["language", "response_format", "temperature", "prompt"], kind: "stt" },
],
serviceKinds: ["stt"],
sttConfig: {
// Overridden per connection by providerSpecificData.baseUrl; this default
// only makes the provider usable out of the box on a same-host deployment.
baseUrl: "http://localhost:8080/v1/audio/transcriptions",
authType: "apikey",
authHeader: "bearer",
format: "openai",
},
};

View File

@@ -0,0 +1,44 @@
// Self-hosted, OpenAI-compatible text-to-speech (Kokoro-FastAPI, openedai-speech,
// vLLM-served TTS, ...) — the TTS counterpart of selfhosted-stt.
//
// Every other self-hostable TTS provider here (coqui, tortoise) carries a FIXED
// localhost baseUrl in its registry entry and `authType: "none"`, and the generic
// dispatcher reads `ttsConfig.baseUrl` from that entry rather than from the
// connection. So there was no way to point TTS at a server on another host.
//
// `authType: "apikey"` is what makes the override possible at all: it gives the
// connection a credentials record, which is where providerSpecificData.baseUrl
// lives. Local servers ignore the key; any non-empty value works.
export default {
id: "selfhosted-tts",
priority: 50,
hasFree: true,
alias: "selfhosted-tts",
display: {
name: "Self-hosted TTS",
icon: "cloud",
color: "#ffffffff",
textIcon: "TT",
website: "https://github.com/remsky/Kokoro-FastAPI",
},
category: "apikey",
auth: {
apiKey: {
text: "Set providerSpecificData.baseUrl to the server root, e.g. http://host:8080 — /v1/audio/speech is appended. The API key is not checked by local servers; any value works.",
},
},
// Voice is selected as "<model>/<voice>", the same convention the OpenAI TTS
// adapter uses, so existing clients need no special casing.
models: [
{ id: "kokoro", name: "Kokoro (self-hosted)", params: ["voice", "response_format", "speed"], kind: "tts" },
],
serviceKinds: ["tts"],
ttsConfig: {
// Overridden per connection by providerSpecificData.baseUrl; this default
// only makes the provider usable on a same-host deployment.
baseUrl: "http://localhost:8880",
defaultModel: "kokoro",
authType: "apikey",
format: "openai-speech",
},
};