feat(providers): self-hosted OpenAI-compatible STT, TTS and embedding providers
Add Self-hosted STT/TTS/Embedding providers that read baseUrl per connection instead of a fixed registry endpoint, so 9Router can point at whisper.cpp, faster-whisper, Kokoro-FastAPI, llama-server, vLLM, Infinity, and similar OpenAI-compatible local servers. Self-hosted Embedding refuses to run without a baseUrl rather than falling back to api.openai.com like openaiCompatNode does, since that fallback would silently send input text and the API key to OpenAI under a provider named "Self-hosted". Also fixes embeddingsCore to catch adapter build errors as a 400 instead of letting them escape uncaught, and bounds the upstream fetch with FETCH_CONNECT_TIMEOUT_MS to avoid hanging forever on a dead endpoint. Self-hosted TTS treats a bare model value as the model rather than the voice, since the generic OpenAI TTS convention (bare = voice) is backwards for a provider where the model is the variable part.
This commit is contained in:
40
README.md
40
README.md
@@ -467,6 +467,46 @@ Default URLs:
|
||||
<p><i>...and 20+ more providers including Nebius, Chutes, Hyperbolic, and custom OpenAI/Anthropic compatible endpoints</i></p>
|
||||
</div>
|
||||
|
||||
### 🏠 Self-hosted Providers
|
||||
|
||||
For speech and embeddings served from **your own** machine — whisper.cpp,
|
||||
faster-whisper, Speaches, Kokoro-FastAPI, openedai-speech, llama.cpp/llama-server,
|
||||
vLLM, Infinity, text-embeddings-inference, or anything else that speaks the OpenAI
|
||||
shape.
|
||||
|
||||
| Provider | Endpoint used | Typical server |
|
||||
| --- | --- | --- |
|
||||
| **Self-hosted STT** | `/v1/audio/transcriptions` | whisper.cpp, faster-whisper |
|
||||
| **Self-hosted TTS** | `/v1/audio/speech` | Kokoro-FastAPI, openedai-speech |
|
||||
| **Self-hosted Embedding** | `/v1/embeddings` | llama-server, vLLM, Infinity |
|
||||
|
||||
Every other speech provider is a named cloud service with a fixed endpoint. These
|
||||
three read their address from **each connection**, so one provider can front
|
||||
several machines and load-balance across them like any other.
|
||||
|
||||
Set it on the connection as `providerSpecificData.baseUrl`:
|
||||
|
||||
| Provider | Give it | Result |
|
||||
| --- | --- | --- |
|
||||
| Self-hosted STT | the full URL — `http://host:8080/v1/audio/transcriptions` | used as-is |
|
||||
| Self-hosted TTS | the server root — `http://host:8880` | `+ /v1/audio/speech` |
|
||||
| Self-hosted Embedding | the **OpenAI base**, `/v1` included — `http://host:8080/v1` | `+ /embeddings` |
|
||||
|
||||
> **Mind the `/v1` on embeddings.** The adapter appends `/embeddings`, so
|
||||
> `http://host:8080` resolves to `http://host:8080/embeddings` and misses the
|
||||
> OpenAI route — llama-server answers **501**. Give it the same base URL an OpenAI
|
||||
> client would use. A full `.../v1/embeddings` is also accepted, so a value pasted
|
||||
> from a `curl` example works too.
|
||||
|
||||
The API key is not checked by most local servers, but the field must be non-empty:
|
||||
it is what gives the connection a credentials record, and `baseUrl` lives there.
|
||||
Any placeholder works.
|
||||
|
||||
Self-hosted Embedding has **no cloud fallback by design** — a connection saved
|
||||
without a `baseUrl` is reported as a configuration error rather than quietly
|
||||
falling back to `api.openai.com`, which would send your input text and API key to
|
||||
a third party under a provider named "Self-hosted".
|
||||
|
||||
---
|
||||
|
||||
## 💡 Key Features
|
||||
|
||||
Reference in New Issue
Block a user