diff --git a/README.md b/README.md index 4345d21b..709a7805 100644 --- a/README.md +++ b/README.md @@ -407,6 +407,7 @@ Default URLs: | Feature | What It Does | Why It Matters | |---------|--------------|----------------| | πŸš€ **RTK Token Saver** ([RTK](https://github.com/rtk-ai/rtk) ⭐40K) | Compress tool outputs (`git diff`, `grep`, `ls`, `tree`...) before sending to LLM | Save **20-40% input tokens** per request | +| 🧠 **Headroom Token Saver** ([Headroom](https://github.com/chopratejas/headroom)) | Optional external `/v1/compress` proxy before provider routing | Save more context tokens without changing clients | | πŸͺ¨ **Caveman Mode** ([Caveman](https://github.com/JuliusBrussee/caveman) ⭐52K) | Inject caveman-speak prompt β†’ LLM replies terse, technical substance preserved | Save **up to 65% output tokens** | | 🎯 **Smart 3-Tier Fallback** | Auto-route: Subscription β†’ Cheap β†’ Free | Never stop coding, zero downtime | | πŸ“Š **Real-Time Quota Tracking** | Live token count + reset countdown | Maximize subscription value | @@ -437,6 +438,35 @@ Without RTK: 47K tokens sent to LLM With RTK: 28K tokens sent to LLM (40% saved Β· same context Β· same answer) ``` +### 🧠 Headroom Token Saver + +Headroom is optional and runs separately. 9Router calls Headroom's local `/v1/compress` endpoint, then keeps normal routing, fallback, auth, and usage tracking: + +``` +Client β†’ 9Router β†’ Headroom /v1/compress β†’ 9Router β†’ provider +``` + +Local setup: + +```bash +pip install "headroom-ai[proxy]" +headroom proxy --port 8787 +``` + +Enable in Dashboard β†’ Endpoint β†’ Token Saver β†’ Headroom. Default URL: `http://localhost:8787`. + +Docker examples: + +```bash +# Headroom service in same Docker network +http://headroom:8787 + +# Headroom running on host machine +http://host.docker.internal:8787 +``` + +If Headroom is down or returns an error, 9Router fails open and sends the original request. + ### 🎯 Smart 3-Tier Fallback Create combos with automatic fallback: diff --git a/cli/src/cli/menus/settings.js b/cli/src/cli/menus/settings.js index a86a7c49..ce779339 100644 --- a/cli/src/cli/menus/settings.js +++ b/cli/src/cli/menus/settings.js @@ -39,6 +39,8 @@ async function showSettingsMenu(breadcrumb = []) { // RTK section const rtkOn = data?.settings?.rtkEnabled !== false; lines.push(` RTK: ${rtkOn ? `${COLORS.green}ON${COLORS.reset}` : `${COLORS.red}OFF${COLORS.reset}`} ${COLORS.dim}(Token Saver)${COLORS.reset}`); + const headroomOn = data?.settings?.headroomEnabled === true; + lines.push(` Headroom: ${headroomOn ? `${COLORS.green}ON${COLORS.reset}` : `${COLORS.red}OFF${COLORS.reset}`} ${COLORS.dim}(${data?.settings?.headroomUrl || "http://localhost:8787"})${COLORS.reset}`); // Auth mode section const authMode = data?.settings?.authMode || "password"; @@ -73,6 +75,13 @@ async function showSettingsMenu(breadcrumb = []) { }, action: async (d) => { await toggleRtk(d?.settings?.rtkEnabled !== false); return true; } }, + { + label: (d) => { + const on = d?.settings?.headroomEnabled === true; + return `Token Saver (Headroom): ${on ? "ON" : "OFF"} β†’ toggle`; + }, + action: async (d) => { await toggleHeadroom(d?.settings?.headroomEnabled === true); return true; } + }, { label: "πŸ”‘ Reset Password to Default", action: async () => { await resetPassword(); return true; } @@ -160,6 +169,17 @@ async function toggleRtk(currentlyOn) { await pause(); } +async function toggleHeadroom(currentlyOn) { + const next = !currentlyOn; + const result = await api.updateSettings({ headroomEnabled: next }); + if (result.success) { + showStatus(`Headroom ${next ? "enabled" : "disabled"}`, "success"); + } else { + showStatus(`Failed: ${result.error}`, "error"); + } + await pause(); +} + /** * Reset dashboard password to default via server API (writes the live SQLite DB). * After reset, user can log in with the default password "123456". diff --git a/open-sse/AGENTS.md b/open-sse/AGENTS.md index a4da782f..5ea0d971 100644 --- a/open-sse/AGENTS.md +++ b/open-sse/AGENTS.md @@ -4,17 +4,20 @@ Provider-agnostic SSE engine: one OpenAI-style request β†’ any provider (LLM cha ## Request lifecycle (chat) -`handlers/chatCore.js` β†’ `services/model.js` `parseModel` (resolve `provider/model`) β†’ `executors/index.js` `getExecutor(provider)` β†’ `translator/index.js` `translateRequest` (client format β†’ provider format) β†’ `executor.execute()` (streams upstream) β†’ `translateResponse` (provider chunks β†’ client format) β†’ SSE out. +`handlers/chatCore.js` β†’ `services/model.js` `parseModel` (resolve `provider/model`) β†’ **pre-translate hooks** (`rtk/` tool_result compress, `rtk/headroom.js` proxy compress, `rtk/caveman.js` system inject β€” all fail-open) β†’ `executors/index.js` `getExecutor(provider)` β†’ `translator/index.js` `translateRequest` (client format β†’ provider format) β†’ `executor.execute()` (streams upstream) β†’ `translateResponse` (provider chunks β†’ client format) β†’ SSE out. ## Directory map - `config/` β€” ALL constants/config (no hardcode elsewhere). `providers.js`/`registry/` (provider defs), `providerModels.js` (aliasβ†’models matrix), `runtimeConfig.js` (timeouts, token limits), `*Constants.js`. -- `translator/` β€” format conversion. `request/-to-.js`, `response/-to-.js`, `schema/` (enums: ROLE, CLAUDE_BLOCK…), `concerns/` (shared logic), `formats/` (per-format). See `tests/translator/AGENTS.md`. +- `translator/` β€” format conversion. `request/-to-.js`, `response/-to-.js`, `schema/` (enums: ROLE, CLAUDE_BLOCK…), `concerns/` (shared logic), `formats.js`+`formats/` (per-format). `index.js` is the registry/entry. - `executors/` β€” per-provider upstream call. `base.js` (BaseExecutor), one file per special provider, `index.js` map. - `providers/` β€” registry build + `capabilities.js` + `pricing.js`. Entry: `index.js` (PROVIDERS). -- `handlers/` β€” per-modality cores (chat/image/embedding/tts/stt/search) + sub-provider folders. -- `services/` β€” `tokenRefresh/`, `usage/`, `combo.js`, `accountFallback.js`, `model.js`. -- `utils/` β€” streamHandler, error, sessionManager, claudeCloaking. +- `handlers/` β€” per-modality cores (chat/image/embedding/tts/stt/search) + sub-provider folders. `chatCore/` has the streaming/non-streaming/sse-to-json handlers. +- `rtk/` β€” request token-killer. `index.js` compresses `tool_result` content in-place (OpenAI/Claude/Kiro shapes); `filters/` per-tool compressors + `autodetect.js`; `headroom.js` external compress proxy; `caveman.js` system-prompt injector. +- `transformer/` β€” `responsesTransformer.js` (Chat Completions SSE β†’ Codex Responses API SSE), `streamToJsonConverter.js`. +- `shared/` β€” cross-provider auth/identity: `clineAuth.js`, `machineId.js`, `qoder/`. +- `services/` β€” `model.js`, `provider.js`, `accountFallback.js`, `combo.js`, `compact.js`, `tokenRefresh/`+`tokenRefresh.js`, `oauthCredentialManager.js`, `usage/`, `projectId.js`, `kiroModels.js`/`qoderModels.js`. +- `utils/` β€” streamHandler, stream, sse, error, sessionManager, claudeCloaking, clientDetector, proxyFetch (patches global fetch), cursorProtobuf/cursorChecksum, ollamaTransform. ## Conventions @@ -33,3 +36,4 @@ Provider-agnostic SSE engine: one OpenAI-style request β†’ any provider (LLM cha - OpenAI bridge is lossy (thinking, non-base64 images, tool ids, is_error) β€” prefer a direct route for fragile pairs. - `registry/index.js` is an auto-generated static import list; regenerate it (don't hand-edit) after adding a `registry/{id}.js`. REGISTRY_TEMPLATE is excluded by design. - Special binary/protobuf formats (kiro EventStream, cursor protobuf, commandcode NDJSON) don't round-trip through OpenAI β€” handle in their executor. +- `rtk/` + `headroom.js` mutate the request body in-place and are **fail-open**: any error returns null and leaves the body untouched β€” never throw out of them. RTK skips `is_error`/`status:"error"` tool results to preserve traces. diff --git a/open-sse/executors/codex.js b/open-sse/executors/codex.js index bfff245d..8c093d2a 100644 --- a/open-sse/executors/codex.js +++ b/open-sse/executors/codex.js @@ -22,7 +22,8 @@ const SERVER_ID_PATTERN = /^(rs|fc|resp|msg)_/; // Hosted tool types that Codex/OpenAI Responses executes server-side const CODEX_HOSTED_TOOL_TYPES = new Set([ "image_generation", "web_search", "web_search_preview", "file_search", - "computer", "computer_use_preview", "code_interpreter", "mcp", "local_shell" + "computer", "computer_use_preview", "code_interpreter", "mcp", "local_shell", + "tool_search" ]); // Allowlist of fields accepted by Codex Responses API β€” anything else is stripped diff --git a/open-sse/executors/default.js b/open-sse/executors/default.js index e80fe228..ca3061d6 100644 --- a/open-sse/executors/default.js +++ b/open-sse/executors/default.js @@ -116,6 +116,11 @@ export class DefaultExecutor extends BaseExecutor { } buildUrl(model, stream, urlIndex = 0, credentials = null) { + // Runtime transport (multi-endpoint providers): use the sourceFormat-matched endpoint + const rt = credentials?.runtimeTransport; + if (rt?.baseUrl) { + return rt.urlSuffix ? `${rt.baseUrl}${rt.urlSuffix}` : rt.baseUrl; + } if (this.provider?.startsWith?.("openai-compatible-")) { const baseUrl = credentials?.providerSpecificData?.baseUrl || OPENAI_COMPAT_BASE; const normalized = baseUrl.replace(/\/$/, ""); @@ -156,8 +161,9 @@ export class DefaultExecutor extends BaseExecutor { } buildHeaders(credentials, stream = true) { - const headers = { "Content-Type": "application/json", ...this.config.headers }; - const desc = AUTH_DESCRIPTORS[this.provider] || this.resolveAuthDescriptor(); + const rt = credentials?.runtimeTransport; + const headers = { "Content-Type": "application/json", ...(rt ? rt.headers : this.config.headers) }; + const desc = rt?.auth || AUTH_DESCRIPTORS[this.provider] || this.resolveAuthDescriptor(); // Hooks run BEFORE auth so dynamic overlays (claude cached headers) can't clobber the token. for (const hook of desc.hooks || []) HEADER_HOOKS[hook]?.(headers, credentials); applyAuth(headers, desc, credentials); diff --git a/open-sse/executors/xiaomi-tokenplan.js b/open-sse/executors/xiaomi-tokenplan.js index a1df9142..75799adb 100644 --- a/open-sse/executors/xiaomi-tokenplan.js +++ b/open-sse/executors/xiaomi-tokenplan.js @@ -8,13 +8,13 @@ export class XiaomiTokenplanExecutor extends DefaultExecutor { super("xiaomi-tokenplan"); } - // Token Plan keys are region-specific β€” always OpenAI-compatible /chat/completions + // Token Plan keys are region-specific. Route per sourceFormat-matched transport: + // claude β†’ Anthropic /anthropic/v1/messages, openai β†’ /chat/completions. buildUrl(model, stream, urlIndex = 0, credentials = null) { const baseUrl = resolveXiaomiTokenplanBaseUrl(credentials); - // Claude-native aliases route to the Anthropic-compatible messages endpoint - // if (getModelTargetFormat(this.provider, model) === FORMATS.CLAUDE) { - // return `${baseUrl.replace(/\/v1\/?$/, "/anthropic/v1")}/messages`; - // } + if (credentials?.runtimeTransport?.format === "claude") { + return `${baseUrl.replace(/\/v1\/?$/, "")}/anthropic/v1/messages`; + } return `${baseUrl}/chat/completions`; } } diff --git a/open-sse/handlers/chatCore.js b/open-sse/handlers/chatCore.js index c1a02603..b537fcaf 100644 --- a/open-sse/handlers/chatCore.js +++ b/open-sse/handlers/chatCore.js @@ -1,4 +1,4 @@ -import { detectFormat, getTargetFormat } from "../services/provider.js"; +import { detectFormat, getTargetFormat, resolveTransport } from "../services/provider.js"; import { translateRequest } from "../translator/index.js"; import { FORMATS } from "../translator/formats.js"; import { normalizeClaudePassthrough } from "../translator/formats/claude.js"; @@ -20,7 +20,9 @@ import { handleStreamingResponse, buildOnStreamComplete } from "./chatCore/strea import { detectClientTool, isNativePassthrough } from "../utils/clientDetector.js"; import { dedupeTools } from "../utils/toolDeduper.js"; import { injectCaveman } from "../rtk/caveman.js"; +import { injectPonytail } from "../rtk/ponytail.js"; import { compressMessages, formatRtkLog } from "../rtk/index.js"; +import { compressWithHeadroom, formatHeadroomLog } from "../rtk/headroom.js"; import { getCapabilitiesForModel } from "../providers/capabilities.js"; import { stripUnsupportedModalities } from "../translator/concerns/modality.js"; import { prefetchRemoteImages } from "../translator/concerns/prefetch.js"; @@ -32,7 +34,7 @@ import { prefetchRemoteImages } from "../translator/concerns/prefetch.js"; * @param {object} options.credentials - Provider credentials * @param {string} options.sourceFormatOverride - Override detected source format (e.g. "openai-responses") */ -export async function handleChatCore({ body, modelInfo, credentials, log, onCredentialsRefreshed, onRequestSuccess, onDisconnect, clientRawRequest, connectionId, userAgent, apiKey, ccFilterNaming, rtkEnabled, cavemanEnabled, cavemanLevel, sourceFormatOverride, providerThinking }) { +export async function handleChatCore({ body, modelInfo, credentials, log, onCredentialsRefreshed, onRequestSuccess, onDisconnect, clientRawRequest, connectionId, userAgent, apiKey, ccFilterNaming, rtkEnabled, headroomEnabled, headroomUrl, headroomCompressUserMessages, cavemanEnabled, cavemanLevel, ponytailEnabled, ponytailLevel, sourceFormatOverride, providerThinking }) { const { provider, model } = modelInfo; const requestStartTime = Date.now(); @@ -44,7 +46,10 @@ export async function handleChatCore({ body, modelInfo, credentials, log, onCred const alias = PROVIDER_ID_TO_ALIAS[provider] || provider; const modelTargetFormat = getModelTargetFormat(alias, model); - const targetFormat = modelTargetFormat || getTargetFormat(provider); + // Multi-endpoint providers: pick transport matching sourceFormat β†’ zero translation + const runtimeTransport = resolveTransport(provider, sourceFormat); + const targetFormat = modelTargetFormat || runtimeTransport?.format || getTargetFormat(provider); + if (runtimeTransport && credentials) credentials.runtimeTransport = runtimeTransport; const stripList = getModelStrip(alias, model); const upstreamModel = getModelUpstreamId(alias, model); @@ -149,12 +154,23 @@ export async function handleChatCore({ body, modelInfo, credentials, log, onCred const rtkLine = formatRtkLog(rtkStats); if (rtkLine) console.log(rtkLine); + // Headroom: optional external proxy compression; fail open if proxy is absent. + const headroomStats = await compressWithHeadroom(translatedBody, { enabled: headroomEnabled, url: headroomUrl, model: upstreamModel, format: finalFormat, compressUserMessages: headroomCompressUserMessages }); + const headroomLine = formatHeadroomLog(headroomStats); + if (headroomLine) log?.info?.("HEADROOM", headroomLine); + // Caveman: inject terse-style system prompt if (cavemanEnabled && cavemanLevel) { injectCaveman(translatedBody, finalFormat, cavemanLevel); log?.debug?.("CAVEMAN", `${cavemanLevel} | ${finalFormat}`); } + // Ponytail: inject lazy-senior-dev system prompt + if (ponytailEnabled && ponytailLevel) { + injectPonytail(translatedBody, finalFormat, ponytailLevel); + log?.debug?.("PONYTAIL", `${ponytailLevel} | ${finalFormat}`); + } + const executor = getExecutor(provider); trackPendingRequest(model, provider, connectionId, true); appendRequestLog({ model, provider, connectionId, status: "PENDING" }).catch(() => { }); diff --git a/open-sse/providers/index.js b/open-sse/providers/index.js index 41212d69..ab5123b1 100644 --- a/open-sse/providers/index.js +++ b/open-sse/providers/index.js @@ -32,7 +32,10 @@ export const PROVIDER_MODELS = {}; export const PROVIDER_OAUTH = {}; export const PROVIDER_MEDIA = {}; for (const entry of REGISTRY) { - if (entry.transport) PROVIDERS[entry.id] = buildTransport(entry.transport, entry.oauth); + if (entry.transport) { + PROVIDERS[entry.id] = buildTransport(entry.transport, entry.oauth); + if (entry.transports) PROVIDERS[entry.id].transports = entry.transports; + } if (entry.models !== undefined) PROVIDER_MODELS[entry.alias || entry.id] = entry.models.map(normalizeModel); if (entry.oauth) PROVIDER_OAUTH[entry.id] = entry.oauth; // Build PROVIDER_MEDIA from top-level fields (post-migration) + legacy entry.media diff --git a/open-sse/providers/registry/deepseek.js b/open-sse/providers/registry/deepseek.js index f6804ae0..5c167f7a 100644 --- a/open-sse/providers/registry/deepseek.js +++ b/open-sse/providers/registry/deepseek.js @@ -1,3 +1,5 @@ +import { CLAUDE_API_HEADERS } from "../shared.js"; + export default { id: "deepseek", priority: 110, @@ -24,6 +26,20 @@ export default { scope: "all", }, }, + // Multi-endpoint: pick the transport matching client sourceFormat to skip translation. + transports: [ + { + format: "openai", + baseUrl: "https://api.deepseek.com/chat/completions", + auth: { combined: true, header: "Authorization", scheme: "bearer" }, + }, + { + format: "claude", + baseUrl: "https://api.deepseek.com/anthropic/v1/messages", + headers: { ...CLAUDE_API_HEADERS }, + auth: { combined: true, header: "x-api-key", scheme: "raw" }, + }, + ], models: [ { id: "deepseek-v4-pro", name: "DeepSeek V4 Pro" }, { id: "deepseek-v4-pro-max", name: "DeepSeek V4 Pro Max", upstreamModelId: "deepseek-v4-pro" }, diff --git a/open-sse/providers/registry/glm.js b/open-sse/providers/registry/glm.js index ceeb8796..95e11533 100644 --- a/open-sse/providers/registry/glm.js +++ b/open-sse/providers/registry/glm.js @@ -19,10 +19,7 @@ export default { baseUrl: "https://api.z.ai/api/anthropic/v1/messages", format: "claude", urlSuffix: "?beta=true", - headers: { - "Anthropic-Version": "2023-06-01", - "Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14", - }, + headers: { ...CLAUDE_API_HEADERS }, auth: { combined: true, header: "x-api-key", @@ -32,6 +29,21 @@ export default { url: "https://api.z.ai/api/monitor/usage/quota/limit", }, }, + // Multi-endpoint: pick the transport matching client sourceFormat to skip translation. + transports: [ + { + format: "openai", + baseUrl: "https://api.z.ai/api/coding/paas/v4/chat/completions", + auth: { combined: true, header: "Authorization", scheme: "bearer" }, + }, + { + format: "claude", + baseUrl: "https://api.z.ai/api/anthropic/v1/messages", + urlSuffix: "?beta=true", + headers: { ...CLAUDE_API_HEADERS }, + auth: { combined: true, header: "x-api-key", scheme: "raw" }, + }, + ], models: [ { id: "glm-5.2", name: "GLM 5.2" }, { id: "glm-5.1", name: "GLM 5.1" }, diff --git a/open-sse/providers/registry/kimi-coding.js b/open-sse/providers/registry/kimi-coding.js index 77ec4564..15705a86 100644 --- a/open-sse/providers/registry/kimi-coding.js +++ b/open-sse/providers/registry/kimi-coding.js @@ -20,10 +20,7 @@ export default { baseUrl: "https://api.kimi.com/coding/v1/messages", format: "claude", urlSuffix: "?beta=true", - headers: { - "Anthropic-Version": "2023-06-01", - "Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14", - }, + headers: { ...CLAUDE_API_HEADERS }, clientId: "17e5f671-d194-4dfb-9706-5516cb48c098", tokenUrl: "https://auth.kimi.com/api/oauth/token", refreshUrl: "https://auth.kimi.com/api/oauth/token", @@ -36,6 +33,21 @@ export default { ], }, }, + // Multi-endpoint: pick the transport matching client sourceFormat to skip translation. + transports: [ + { + format: "openai", + baseUrl: "https://api.kimi.com/coding/v1/chat/completions", + auth: { combined: true, header: "Authorization", scheme: "bearer", hooks: ["kimiHeaders"] }, + }, + { + format: "claude", + baseUrl: "https://api.kimi.com/coding/v1/messages", + urlSuffix: "?beta=true", + headers: { ...CLAUDE_API_HEADERS }, + auth: { combined: true, header: "x-api-key", scheme: "raw", hooks: ["kimiHeaders"] }, + }, + ], models: [ { id: "kimi-k2.6", name: "Kimi K2.6" }, { id: "kimi-k2.5", name: "Kimi K2.5" }, diff --git a/open-sse/providers/registry/kimi.js b/open-sse/providers/registry/kimi.js index ac22357b..e22286b4 100644 --- a/open-sse/providers/registry/kimi.js +++ b/open-sse/providers/registry/kimi.js @@ -19,16 +19,28 @@ export default { baseUrl: "https://api.kimi.com/coding/v1/messages", format: "claude", urlSuffix: "?beta=true", - headers: { - "Anthropic-Version": "2023-06-01", - "Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14", - }, + headers: { ...CLAUDE_API_HEADERS }, auth: { combined: true, header: "x-api-key", scheme: "raw", }, }, + // Multi-endpoint: pick the transport matching client sourceFormat to skip translation. + transports: [ + { + format: "openai", + baseUrl: "https://api.kimi.com/coding/v1/chat/completions", + auth: { combined: true, header: "Authorization", scheme: "bearer" }, + }, + { + format: "claude", + baseUrl: "https://api.kimi.com/coding/v1/messages", + urlSuffix: "?beta=true", + headers: { ...CLAUDE_API_HEADERS }, + auth: { combined: true, header: "x-api-key", scheme: "raw" }, + }, + ], models: [ { id: "kimi-k2.6", name: "Kimi K2.6" }, { id: "kimi-k2.5", name: "Kimi K2.5" }, diff --git a/open-sse/providers/registry/minimax-cn.js b/open-sse/providers/registry/minimax-cn.js index 95f130f9..19aa3127 100644 --- a/open-sse/providers/registry/minimax-cn.js +++ b/open-sse/providers/registry/minimax-cn.js @@ -19,10 +19,7 @@ export default { baseUrl: "https://api.minimaxi.com/anthropic/v1/messages", format: "claude", urlSuffix: "?beta=true", - headers: { - "Anthropic-Version": "2023-06-01", - "Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14", - }, + headers: { ...CLAUDE_API_HEADERS }, quirks: { dropOutputConfig: true, }, @@ -41,6 +38,21 @@ export default { ], }, }, + // Multi-endpoint: pick the transport matching client sourceFormat to skip translation. + transports: [ + { + format: "openai", + baseUrl: "https://api.minimaxi.com/v1/chat/completions", + auth: { combined: true, header: "Authorization", scheme: "bearer" }, + }, + { + format: "claude", + baseUrl: "https://api.minimaxi.com/anthropic/v1/messages", + urlSuffix: "?beta=true", + headers: { ...CLAUDE_API_HEADERS }, + auth: { combined: true, header: "x-api-key", scheme: "raw" }, + }, + ], models: [ { id: "MiniMax-M3", name: "MiniMax M3", targetFormat: "claude" }, { id: "MiniMax-M2.7", name: "MiniMax M2.7" }, diff --git a/open-sse/providers/registry/minimax.js b/open-sse/providers/registry/minimax.js index 99bafae9..47b82c89 100644 --- a/open-sse/providers/registry/minimax.js +++ b/open-sse/providers/registry/minimax.js @@ -19,10 +19,7 @@ export default { baseUrl: "https://api.minimax.io/anthropic/v1/messages", format: "claude", urlSuffix: "?beta=true", - headers: { - "Anthropic-Version": "2023-06-01", - "Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14", - }, + headers: { ...CLAUDE_API_HEADERS }, quirks: { dropOutputConfig: true, }, @@ -41,6 +38,21 @@ export default { ], }, }, + // Multi-endpoint: pick the transport matching client sourceFormat to skip translation. + transports: [ + { + format: "openai", + baseUrl: "https://api.minimax.io/v1/chat/completions", + auth: { combined: true, header: "Authorization", scheme: "bearer" }, + }, + { + format: "claude", + baseUrl: "https://api.minimax.io/anthropic/v1/messages", + urlSuffix: "?beta=true", + headers: { ...CLAUDE_API_HEADERS }, + auth: { combined: true, header: "x-api-key", scheme: "raw" }, + }, + ], models: [ { id: "MiniMax-M3", name: "MiniMax M3", targetFormat: "claude" }, { id: "MiniMax-M2.7", name: "MiniMax M2.7" }, diff --git a/open-sse/providers/registry/xiaomi-mimo.js b/open-sse/providers/registry/xiaomi-mimo.js index 97cec2e8..fcef7af8 100644 --- a/open-sse/providers/registry/xiaomi-mimo.js +++ b/open-sse/providers/registry/xiaomi-mimo.js @@ -1,3 +1,5 @@ +import { CLAUDE_API_HEADERS } from "../shared.js"; + export default { id: "xiaomi-mimo", priority: 290, @@ -21,6 +23,20 @@ export default { baseUrl: "https://api.xiaomimimo.com/v1/chat/completions", validateUrl: "https://api.xiaomimimo.com/v1/models", }, + // Multi-endpoint: pick the transport matching client sourceFormat to skip translation. + transports: [ + { + format: "openai", + baseUrl: "https://api.xiaomimimo.com/v1/chat/completions", + auth: { combined: true, header: "Authorization", scheme: "bearer" }, + }, + { + format: "claude", + baseUrl: "https://api.xiaomimimo.com/anthropic/v1/messages", + headers: { ...CLAUDE_API_HEADERS }, + auth: { combined: true, header: "x-api-key", scheme: "raw" }, + }, + ], models: [ { id: "mimo-v2.5-pro", name: "MiMo V2.5 Pro" }, { id: "mimo-v2.5", name: "MiMo V2.5" }, diff --git a/open-sse/providers/registry/xiaomi-tokenplan.js b/open-sse/providers/registry/xiaomi-tokenplan.js index 35d714a0..55441434 100644 --- a/open-sse/providers/registry/xiaomi-tokenplan.js +++ b/open-sse/providers/registry/xiaomi-tokenplan.js @@ -1,3 +1,5 @@ +import { CLAUDE_API_HEADERS } from "../shared.js"; + export default { id: "xiaomi-tokenplan", priority: 300, @@ -29,6 +31,19 @@ export default { }, defaultRegion: "sgp", }, + // Multi-endpoint: pick the transport matching client sourceFormat to skip translation. + // baseUrl omitted β€” region-dynamic, resolved in the executor's buildUrl. + transports: [ + { + format: "openai", + auth: { combined: true, header: "Authorization", scheme: "bearer" }, + }, + { + format: "claude", + headers: { ...CLAUDE_API_HEADERS }, + auth: { combined: true, header: "x-api-key", scheme: "raw" }, + }, + ], models: [ { id: "mimo-v2.5-pro", name: "MiMo V2.5 Pro" }, { id: "mimo-v2.5-pro-claude", name: "MiMo V2.5 Pro (Claude Native)", targetFormat: "claude", upstreamModelId: "mimo-v2.5-pro" }, diff --git a/open-sse/rtk/caveman.js b/open-sse/rtk/caveman.js index 09cc8cfb..9c9a2065 100644 --- a/open-sse/rtk/caveman.js +++ b/open-sse/rtk/caveman.js @@ -1,100 +1,9 @@ // Caveman injector: appends a caveman-style instruction into the system message // of the final request body, just before it is dispatched to the provider executor. -// Dispatches by format so it works for both translated and native-passthrough flows. -import { FORMATS } from "../translator/formats.js"; +import { injectSystemPrompt } from "./systemInject.js"; import { CAVEMAN_PROMPTS } from "./cavemanPrompts.js"; -const SEP = "\n\n"; - export function injectCaveman(body, format, level) { - const prompt = CAVEMAN_PROMPTS[level]; - if (!body || !prompt) return; - - switch (format) { - case FORMATS.CLAUDE: - injectClaudeSystem(body, prompt); - return; - case FORMATS.GEMINI: - case FORMATS.GEMINI_CLI: - case FORMATS.VERTEX: - case FORMATS.ANTIGRAVITY: - // Antigravity wraps Gemini shape in body.request β†’ injectGeminiSystem handles it - injectGeminiSystem(body, prompt); - return; - default: - // OpenAI and OpenAI-shaped formats (responses/codex/cursor/kiro/ollama) - injectMessagesSystem(body, prompt); - } -} - -// OpenAI-shaped: messages[] (chat) or input[] (responses) or instructions (responses string) -function injectMessagesSystem(body, prompt) { - // OpenAI Responses API: top-level string field - if (typeof body.instructions === "string") { - body.instructions = body.instructions - ? `${body.instructions}${SEP}${prompt}` - : prompt; - return; - } - - const arr = Array.isArray(body.messages) ? body.messages - : Array.isArray(body.input) ? body.input - : null; - if (!arr) return; - - const idx = arr.findIndex(m => m && (m.role === "system" || m.role === "developer")); - if (idx >= 0) { - appendToOpenAIMessage(arr[idx], prompt); - } else { - arr.unshift({ role: "system", content: prompt }); - } -} - -function appendToOpenAIMessage(msg, prompt) { - if (typeof msg.content === "string") { - msg.content = `${msg.content}${SEP}${prompt}`; - } else if (Array.isArray(msg.content)) { - // Responses-style array of parts {type:"input_text"|"text", text} - msg.content.push({ type: "input_text", text: prompt }); - } else { - msg.content = prompt; - } -} - -// Claude shape: body.system as string | array of {type:"text", text} -// Insert before the last cache_control block to keep caveman inside the cached prefix. -function injectClaudeSystem(body, prompt) { - if (typeof body.system === "string" && body.system.length > 0) { - body.system = `${body.system}${SEP}${prompt}`; - return; - } - if (Array.isArray(body.system)) { - const block = { type: "text", text: prompt }; - let lastCacheIdx = -1; - for (let i = body.system.length - 1; i >= 0; i--) { - if (body.system[i]?.cache_control) { lastCacheIdx = i; break; } - } - if (lastCacheIdx >= 0) { - body.system.splice(lastCacheIdx, 0, block); - } else { - body.system.push(block); - } - return; - } - body.system = prompt; -} - -// Gemini shape: body.system_instruction | body.systemInstruction | body.request.systemInstruction -// Each shape: { parts: [{ text }] } -function injectGeminiSystem(body, prompt) { - const target = body.request && typeof body.request === "object" ? body.request : body; - const useSnake = Object.prototype.hasOwnProperty.call(target, "system_instruction"); - const key = useSnake ? "system_instruction" : "systemInstruction"; - const sys = target[key]; - if (sys && Array.isArray(sys.parts)) { - sys.parts.push({ text: prompt }); - return; - } - target[key] = { parts: [{ text: prompt }] }; + injectSystemPrompt(body, format, CAVEMAN_PROMPTS[level]); } diff --git a/open-sse/rtk/headroom.js b/open-sse/rtk/headroom.js new file mode 100644 index 00000000..ac2f4db5 --- /dev/null +++ b/open-sse/rtk/headroom.js @@ -0,0 +1,63 @@ +import { claudeToOpenAIRequest } from "../translator/request/claude-to-openai.js"; +import { openaiToClaudeRequest } from "../translator/request/openai-to-claude.js"; + +const DEFAULT_TIMEOUT_MS = 3000; + +// POST messages to Headroom /v1/compress; returns compressed messages + stats or null. +async function callCompress(url, messages, model, timeoutMs, compressUserMessages) { + const endpoint = `${String(url).replace(/\/$/, "")}/v1/compress`; + const payload = { messages, model }; + if (compressUserMessages) payload.config = { compress_user_messages: true }; + const res = await fetch(endpoint, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(payload), + signal: AbortSignal.timeout(timeoutMs), + }); + if (!res.ok) return null; + const data = await res.json(); + if (!Array.isArray(data?.messages)) return null; + return data; +} + +// Compress request body via Headroom proxy. Fail-open: returns null on any error. +// /v1/compress only understands OpenAI shape, so Claude bodies are translated +// to OpenAI, compressed, then translated back using 9Router's own translators. +export async function compressWithHeadroom(body, { enabled, url, model, format, compressUserMessages, timeoutMs = DEFAULT_TIMEOUT_MS } = {}) { + if (!enabled || !url || !body) return null; + + try { + // Claude shape: translate β†’ OpenAI β†’ compress β†’ translate back. + if (format === "claude") { + const oai = claudeToOpenAIRequest(model, body, false); + if (!Array.isArray(oai?.messages)) return null; + const data = await callCompress(url, oai.messages, model, timeoutMs, compressUserMessages); + if (!data) return null; + const claudeBody = openaiToClaudeRequest(model, { ...oai, messages: data.messages }, false); + if (Array.isArray(claudeBody?.messages)) body.messages = claudeBody.messages; + if (claudeBody?.system !== undefined) body.system = claudeBody.system; + return data; + } + + // OpenAI shape: messages/input go straight to the proxy. + const key = Array.isArray(body.messages) ? "messages" + : Array.isArray(body.input) ? "input" + : null; + if (!key) return null; + const data = await callCompress(url, body[key], model, timeoutMs, compressUserMessages); + if (!data) return null; + body[key] = data.messages; + return data; + } catch { + return null; + } +} + +export function formatHeadroomLog(stats) { + if (!stats) return null; + const before = stats.tokens_before || 0; + const after = stats.tokens_after || 0; + const saved = stats.tokens_saved || 0; + const pct = before > 0 ? ((saved / before) * 100).toFixed(1) : "0"; + return `saved ${saved} tokens / ${before} (${pct}%) ${after ? `after=${after}` : ""}`.trim(); +} diff --git a/open-sse/rtk/ponytail.js b/open-sse/rtk/ponytail.js new file mode 100644 index 00000000..1af6a1ab --- /dev/null +++ b/open-sse/rtk/ponytail.js @@ -0,0 +1,9 @@ +// Ponytail injector: appends the "lazy senior dev" instruction into the system +// message of the final request body, just before dispatch to the provider executor. + +import { injectSystemPrompt } from "./systemInject.js"; +import { PONYTAIL_PROMPTS } from "./ponytailPrompt.js"; + +export function injectPonytail(body, format, level) { + injectSystemPrompt(body, format, PONYTAIL_PROMPTS[level]); +} diff --git a/open-sse/rtk/ponytailPrompt.js b/open-sse/rtk/ponytailPrompt.js new file mode 100644 index 00000000..1de20663 --- /dev/null +++ b/open-sse/rtk/ponytailPrompt.js @@ -0,0 +1,52 @@ +// Ponytail intensity-level prompts injected into system message to bias toward minimal code. +// Adapted from ponytail skill (https://github.com/DietrichGebert/ponytail). + +export const PONYTAIL_LEVELS = { + LITE: "lite", + FULL: "full", + ULTRA: "ultra", +}; + +const SHARED_PERSONA = "You are a lazy senior developer. Lazy means efficient, not careless. The best code is the code never written."; + +const SHARED_LADDER = "Before writing code, stop at the first rung that holds: 1) Does this need to exist at all? (YAGNI) 2) Stdlib does it? Use it. 3) Native platform feature covers it? Use it (CSS over JS, DB constraint over app code). 4) Already-installed dependency solves it? Use it; never add a new one for what a few lines can do. 5) Can it be one line? One line. 6) Only then: the minimum code that works."; + +const SHARED_RULES = "No unrequested abstractions (no interface with one implementation, no factory for one product, no config for a value that never changes). No boilerplate or scaffolding \"for later\". Deletion over addition. Boring over clever. Fewest files possible; shortest working diff wins. Two stdlib options the same size: take the edge-case-correct one. Mark deliberate simplifications with a `ponytail:` comment naming the ceiling and upgrade path."; + +const SHARED_OUTPUT = "Code first. Then at most three short lines: what was skipped, when to add it. No essays or design notes. Pattern: `[code] β†’ skipped: [X], add when [Y].`"; + +const SHARED_NOT_LAZY = "Never simplify away: input validation at trust boundaries, error handling that prevents data loss, security, accessibility, anything explicitly requested. Non-trivial logic leaves ONE runnable check behind (an assert-based self-check or one small test file; no frameworks). Trivial one-liners need no test."; + +const SHARED_PERSISTENCE = "ACTIVE EVERY RESPONSE. No drift back to over-building. Still active if unsure."; + +export const PONYTAIL_PROMPTS = { + [PONYTAIL_LEVELS.LITE]: [ + SHARED_PERSONA, + "Lite: build what's asked, but name the lazier alternative in one line. User picks.", + SHARED_LADDER, + SHARED_RULES, + SHARED_OUTPUT, + SHARED_NOT_LAZY, + SHARED_PERSISTENCE, + ].join(" "), + + [PONYTAIL_LEVELS.FULL]: [ + SHARED_PERSONA, + "Full: the ladder enforced. Stdlib and native first. Shortest diff, shortest explanation.", + SHARED_LADDER, + SHARED_RULES, + SHARED_OUTPUT, + SHARED_NOT_LAZY, + SHARED_PERSISTENCE, + ].join(" "), + + [PONYTAIL_LEVELS.ULTRA]: [ + SHARED_PERSONA, + "Ultra: YAGNI extremist. Deletion before addition. Ship the one-liner and challenge the rest of the requirement in the same response.", + SHARED_LADDER, + SHARED_RULES, + SHARED_OUTPUT, + SHARED_NOT_LAZY, + SHARED_PERSISTENCE, + ].join(" "), +}; diff --git a/open-sse/rtk/systemInject.js b/open-sse/rtk/systemInject.js new file mode 100644 index 00000000..0d5af728 --- /dev/null +++ b/open-sse/rtk/systemInject.js @@ -0,0 +1,98 @@ +// Shared system-prompt injector: appends an instruction into the system message of +// the final request body, dispatching by format so it works for translated and +// native-passthrough flows. Used by caveman.js and ponytail.js. + +import { FORMATS } from "../translator/formats.js"; + +const SEP = "\n\n"; + +export function injectSystemPrompt(body, format, prompt) { + if (!body || !prompt) return; + + switch (format) { + case FORMATS.CLAUDE: + injectClaudeSystem(body, prompt); + return; + case FORMATS.GEMINI: + case FORMATS.GEMINI_CLI: + case FORMATS.VERTEX: + case FORMATS.ANTIGRAVITY: + // Antigravity wraps Gemini shape in body.request β†’ injectGeminiSystem handles it + injectGeminiSystem(body, prompt); + return; + default: + // OpenAI and OpenAI-shaped formats (responses/codex/cursor/kiro/ollama) + injectMessagesSystem(body, prompt); + } +} + +// OpenAI-shaped: messages[] (chat) or input[] (responses) or instructions (responses string) +function injectMessagesSystem(body, prompt) { + // OpenAI Responses API: top-level string field + if (typeof body.instructions === "string") { + body.instructions = body.instructions + ? `${body.instructions}${SEP}${prompt}` + : prompt; + return; + } + + const arr = Array.isArray(body.messages) ? body.messages + : Array.isArray(body.input) ? body.input + : null; + if (!arr) return; + + const idx = arr.findIndex(m => m && (m.role === "system" || m.role === "developer")); + if (idx >= 0) { + appendToOpenAIMessage(arr[idx], prompt); + } else { + arr.unshift({ role: "system", content: prompt }); + } +} + +function appendToOpenAIMessage(msg, prompt) { + if (typeof msg.content === "string") { + msg.content = `${msg.content}${SEP}${prompt}`; + } else if (Array.isArray(msg.content)) { + // Responses-style array of parts {type:"input_text"|"text", text} + msg.content.push({ type: "input_text", text: prompt }); + } else { + msg.content = prompt; + } +} + +// Claude shape: body.system as string | array of {type:"text", text} +// Insert before the last cache_control block to keep injection inside the cached prefix. +function injectClaudeSystem(body, prompt) { + if (typeof body.system === "string" && body.system.length > 0) { + body.system = `${body.system}${SEP}${prompt}`; + return; + } + if (Array.isArray(body.system)) { + const block = { type: "text", text: prompt }; + let lastCacheIdx = -1; + for (let i = body.system.length - 1; i >= 0; i--) { + if (body.system[i]?.cache_control) { lastCacheIdx = i; break; } + } + if (lastCacheIdx >= 0) { + body.system.splice(lastCacheIdx, 0, block); + } else { + body.system.push(block); + } + return; + } + body.system = prompt; +} + +// Gemini shape: body.system_instruction | body.systemInstruction | body.request.systemInstruction +// Each shape: { parts: [{ text }] } +function injectGeminiSystem(body, prompt) { + const target = body.request && typeof body.request === "object" ? body.request : body; + const useSnake = Object.prototype.hasOwnProperty.call(target, "system_instruction"); + const key = useSnake ? "system_instruction" : "systemInstruction"; + const sys = target[key]; + if (sys && Array.isArray(sys.parts)) { + sys.parts.push({ text: prompt }); + return; + } + target[key] = { parts: [{ text: prompt }] }; +} diff --git a/open-sse/services/provider.js b/open-sse/services/provider.js index 4b3a6690..495c3533 100644 --- a/open-sse/services/provider.js +++ b/open-sse/services/provider.js @@ -136,6 +136,16 @@ export function getTargetFormat(provider) { return config.format || "openai"; } +// Resolve which transport to use for a provider given the client sourceFormat. +// Multi-endpoint providers (transport.transports[]) pick the entry matching sourceFormat +// to avoid lossy translation; falls back to the default transport when no match. +export function resolveTransport(provider, sourceFormat) { + const config = PROVIDERS[provider]; + const transports = config?.transports; + if (!Array.isArray(transports) || !transports.length) return null; + return transports.find(t => t.format === sourceFormat) || null; +} + // Check if last message is from user export function isLastMessageFromUser(body) { const messages = body.messages || body.contents; diff --git a/src/app/(dashboard)/dashboard/endpoint/EndpointPageClient.js b/src/app/(dashboard)/dashboard/endpoint/EndpointPageClient.js index d679f596..096fd28d 100644 --- a/src/app/(dashboard)/dashboard/endpoint/EndpointPageClient.js +++ b/src/app/(dashboard)/dashboard/endpoint/EndpointPageClient.js @@ -14,6 +14,7 @@ import { REACHABLE_MISS_THRESHOLD, CLIENT_PING_FAST_MS, CAVEMAN_LEVELS, + PONYTAIL_LEVELS, } from "./endpointConstants"; import { clientPingUrl, clientPingAny } from "./endpointPing"; import EndpointRow from "./components/EndpointRow"; @@ -33,8 +34,17 @@ export default function APIPageClient({ machineId }) { const [hasPassword, setHasPassword] = useState(true); const [tunnelDashboardAccess, setTunnelDashboardAccess] = useState(false); const [rtkEnabled, setRtkEnabledState] = useState(true); + const [headroomEnabled, setHeadroomEnabled] = useState(false); + const [headroomUrl, setHeadroomUrl] = useState("http://localhost:8787"); + const [headroomCompressUserMessages, setHeadroomCompressUserMessages] = useState(false); + const [headroomStatus, setHeadroomStatus] = useState({ installed: false, running: false, python: null, loading: true }); + const [showHeadroomInstallModal, setShowHeadroomInstallModal] = useState(false); + const [headroomActionLoading, setHeadroomActionLoading] = useState(false); + const [headroomActionError, setHeadroomActionError] = useState(""); const [cavemanEnabled, setCavemanEnabled] = useState(false); const [cavemanLevel, setCavemanLevel] = useState("full"); + const [ponytailEnabled, setPonytailEnabled] = useState(false); + const [ponytailLevel, setPonytailLevel] = useState("full"); const [locale, setLocale] = useState("en"); // Cloudflare Tunnel state @@ -232,8 +242,14 @@ export default function APIPageClient({ machineId }) { setHasPassword(data.hasPassword || false); setTunnelDashboardAccess(data.tunnelDashboardAccess || false); setRtkEnabledState(data.rtkEnabled !== false); + setHeadroomEnabled(!!data.headroomEnabled); + setHeadroomUrl(data.headroomUrl || "http://localhost:8787"); + setHeadroomCompressUserMessages(!!data.headroomCompressUserMessages); + refreshHeadroomStatus(); setCavemanEnabled(!!data.cavemanEnabled); setCavemanLevel(data.cavemanLevel || "full"); + setPonytailEnabled(!!data.ponytailEnabled); + setPonytailLevel(data.ponytailLevel || "full"); } if (statusRes.ok) { const data = await statusRes.json(); @@ -313,11 +329,75 @@ export default function APIPageClient({ machineId }) { patchSetting({ cavemanEnabled: value }); }; + const handleHeadroomEnabled = (value) => { + const nextUrl = headroomUrl.trim() || "http://localhost:8787"; + setHeadroomUrl(nextUrl); + setHeadroomEnabled(value); + patchSetting({ headroomEnabled: value, headroomUrl: nextUrl }); + }; + + const handleHeadroomUrlBlur = () => { + const next = headroomUrl.trim() || "http://localhost:8787"; + setHeadroomUrl(next); + patchSetting({ headroomUrl: next }); + }; + + const handleHeadroomCompressUserMessages = (value) => { + setHeadroomCompressUserMessages(value); + patchSetting({ headroomCompressUserMessages: value }); + }; + + const refreshHeadroomStatus = useCallback(async () => { + setHeadroomStatus((s) => ({ ...s, loading: true })); + try { + const res = await fetch("/api/headroom/status", { headers: { "Cache-Control": "no-store" } }); + const data = await res.json(); + setHeadroomStatus({ ...data, loading: false }); + } catch { + setHeadroomStatus({ installed: false, running: false, python: null, loading: false }); + } + }, []); + + const handleHeadroomStart = useCallback(async () => { + setHeadroomActionError(""); + setHeadroomActionLoading(true); + try { + const res = await fetch("/api/headroom/start", { method: "POST" }); + const data = await res.json().catch(() => ({})); + if (!res.ok) throw new Error(data.error || "Failed to start proxy"); + await refreshHeadroomStatus(); + } catch (e) { + setHeadroomActionError(e.message); + } finally { + setHeadroomActionLoading(false); + } + }, [refreshHeadroomStatus]); + + const handleHeadroomStop = useCallback(async () => { + setHeadroomActionLoading(true); + try { + await fetch("/api/headroom/stop", { method: "POST" }); + await refreshHeadroomStatus(); + } finally { + setHeadroomActionLoading(false); + } + }, [refreshHeadroomStatus]); + const handleCavemanLevel = (level) => { setCavemanLevel(level); patchSetting({ cavemanLevel: level }); }; + const handlePonytailEnabled = (value) => { + setPonytailEnabled(value); + patchSetting({ ponytailEnabled: value }); + }; + + const handlePonytailLevel = (level) => { + setPonytailLevel(level); + patchSetting({ ponytailLevel: level }); + }; + const fetchData = async () => { try { const keysRes = await fetch("/api/keys"); @@ -1043,6 +1123,47 @@ export default function APIPageClient({ machineId }) { onChange={() => handleRtkEnabled(!rtkEnabled)} /> +
+
+
+

+ Compress context{" "} + + (Headroom) + +

+ + {headroomStatus.loading + ? "Checking…" + : !headroomStatus.installed + ? "Not installed" + : !headroomStatus.running + ? "Proxy off" + : "Running"} + + +
+

+ Compress prompts via /v1/compress before routing to the model +

+
+ handleHeadroomEnabled(!headroomEnabled)} + /> +

@@ -1090,6 +1211,53 @@ export default function APIPageClient({ machineId }) { />

+
+
+

+ Lazy senior dev{" "} + + (Ponytail) + +

+

+ Bias the model toward minimal code: YAGNI, reuse stdlib, deletion over addition +

+
+
+ {ponytailEnabled && ( +
+
+ {PONYTAIL_LEVELS.map((lvl) => ( + + ))} +
+

+ {PONYTAIL_LEVELS.find((lvl) => lvl.id === ponytailLevel)?.desc} +

+
+ )} + handlePonytailEnabled(!ponytailEnabled)} + /> +
+
{/* API Keys */} @@ -1420,6 +1588,58 @@ export default function APIPageClient({ machineId }) { + {/* Headroom Install Guide Modal */} + setShowHeadroomInstallModal(false)} + > +
+
+ Status + + {headroomStatus.loading + ? "Checking…" + : !headroomStatus.installed + ? "Not installed" + : !headroomStatus.running + ? "Proxy off" + : "Running"} + +
+ {headroomStatus.installed ? ( + headroomStatus.running ? ( + + ) : ( + + ) + ) : !headroomStatus.python ? ( +

Python β‰₯ 3.10 required. Install Python first.

+ ) : ( +
+

Install then click Start:

+
+ 
{`pip install "headroom-ai[proxy]"`}
+ +
+
+ )} + {headroomActionError && ( +

{headroomActionError}

+ )} +
+ + +
+
+ + {/* Confirm Modal */} 0 && p < 65536) return p; + } catch { /* ignore, fall through to default */ } + return null; +} + +export async function POST() { + try { + const settings = await getSettings(); + const url = settings.headroomUrl || "http://localhost:8787"; + const port = parsePortFromUrl(url) || 8787; + const result = await startHeadroomProxy({ port }); + return NextResponse.json({ success: true, ...result }); + } catch (error) { + const status = error.code === "NOT_INSTALLED" ? 400 : 500; + return NextResponse.json({ error: error.message, code: error.code || null }, { status }); + } +} diff --git a/src/app/api/headroom/status/route.js b/src/app/api/headroom/status/route.js new file mode 100644 index 00000000..1ae54435 --- /dev/null +++ b/src/app/api/headroom/status/route.js @@ -0,0 +1,18 @@ +import { NextResponse } from "next/server"; +import { getSettings } from "@/lib/localDb"; +import { getHeadroomStatus } from "@/lib/headroom/detect"; +import { getManagedPid } from "@/lib/headroom/process"; + +export const dynamic = "force-dynamic"; + +export async function GET() { + try { + const settings = await getSettings(); + const url = settings.headroomUrl || "http://localhost:8787"; + const status = await getHeadroomStatus(url); + const managedPid = getManagedPid(); + return NextResponse.json({ ...status, url, managedPid }); + } catch (error) { + return NextResponse.json({ error: error.message }, { status: 500 }); + } +} diff --git a/src/app/api/headroom/stop/route.js b/src/app/api/headroom/stop/route.js new file mode 100644 index 00000000..122251e1 --- /dev/null +++ b/src/app/api/headroom/stop/route.js @@ -0,0 +1,14 @@ +import { NextResponse } from "next/server"; +import { stopHeadroomProxy } from "@/lib/headroom/process"; + +export const dynamic = "force-dynamic"; + +export async function POST() { + try { + const result = stopHeadroomProxy(); + const status = result.stopped ? 200 : 409; + return NextResponse.json({ ...result }, { status }); + } catch (error) { + return NextResponse.json({ error: error.message, code: error.code || null }, { status: 500 }); + } +} diff --git a/src/dashboardGuard.js b/src/dashboardGuard.js index 5a7a2588..6d86cc78 100644 --- a/src/dashboardGuard.js +++ b/src/dashboardGuard.js @@ -79,6 +79,8 @@ const LOCAL_ONLY_PATHS = [ "/api/oauth/cursor/auto-import", "/api/oauth/kiro/auto-import", "/api/auth/reset-password", + "/api/headroom/start", + "/api/headroom/stop", ]; const LOOPBACK_HOSTS = new Set(["localhost", "127.0.0.1", "::1"]); diff --git a/src/lib/db/repos/settingsRepo.js b/src/lib/db/repos/settingsRepo.js index 7201a760..02d07f6b 100644 --- a/src/lib/db/repos/settingsRepo.js +++ b/src/lib/db/repos/settingsRepo.js @@ -34,8 +34,13 @@ const DEFAULT_SETTINGS = { mitmRouterBaseUrl: DEFAULT_MITM_ROUTER_BASE, dnsToolEnabled: {}, rtkEnabled: true, + headroomEnabled: false, + headroomUrl: "http://localhost:8787", + headroomCompressUserMessages: false, cavemanEnabled: false, cavemanLevel: "full", + ponytailEnabled: false, + ponytailLevel: "full", }; async function readRaw() { diff --git a/src/lib/headroom/detect.js b/src/lib/headroom/detect.js new file mode 100644 index 00000000..3320d6b6 --- /dev/null +++ b/src/lib/headroom/detect.js @@ -0,0 +1,63 @@ +import { execSync } from "child_process"; + +const EXTENDED_PATH = `/usr/local/bin:/opt/homebrew/bin:/usr/bin:/bin:${process.env.PATH || ""}`; +const PYTHON_CANDIDATES = ["python3.13", "python3.12", "python3.11", "python3.10", "python3"]; +const MIN_VERSION = [3, 10]; +const HEADROOM_HEALTH_TIMEOUT_MS = 1500; + +// Detect whether the headroom CLI is installed and where its binary lives. +export function findHeadroomBinary() { + try { + const path = execSync("which headroom", { + stdio: ["ignore", "pipe", "ignore"], + windowsHide: true, + env: { ...process.env, PATH: EXTENDED_PATH }, + }).toString().trim(); + return path || null; + } catch { + return null; + } +} + +// Find a Python interpreter >= 3.10 (headroom-ai requires it). Returns null if none. +export function findPython310() { + for (const candidate of PYTHON_CANDIDATES) { + try { + const ver = execSync(`${candidate} --version`, { + stdio: ["ignore", "pipe", "ignore"], + windowsHide: true, + env: { ...process.env, PATH: EXTENDED_PATH }, + }).toString().trim(); + const match = ver.match(/(\d+)\.(\d+)/); + if (!match) continue; + const [major, minor] = [parseInt(match[1], 10), parseInt(match[2], 10)]; + if (major > MIN_VERSION[0] || (major === MIN_VERSION[0] && minor >= MIN_VERSION[1])) { + return candidate; + } + } catch { + // candidate not present, try next + } + } + return null; +} + +// Probe whether a Headroom proxy is reachable at the given URL by hitting /health. +export async function probeProxyRunning(url) { + if (!url) return false; + const base = String(url).replace(/\/$/, ""); + try { + const res = await fetch(`${base}/health`, { signal: AbortSignal.timeout(HEADROOM_HEALTH_TIMEOUT_MS) }); + return res.ok; + } catch { + return false; + } +} + +// Aggregate status for the dashboard: installed, running, python interpreter. +export async function getHeadroomStatus(url) { + const path = findHeadroomBinary(); + const python = findPython310(); + const installed = Boolean(path); + const running = installed ? await probeProxyRunning(url) : false; + return { installed, path, running, python }; +} diff --git a/src/lib/headroom/process.js b/src/lib/headroom/process.js new file mode 100644 index 00000000..d50bc7ec --- /dev/null +++ b/src/lib/headroom/process.js @@ -0,0 +1,128 @@ +import fs from "fs"; +import path from "path"; +import { spawn } from "child_process"; +import { DATA_DIR } from "@/lib/dataDir.js"; +import { findHeadroomBinary } from "./detect.js"; + +const HEADROOM_DIR = path.join(DATA_DIR, "headroom"); +const PID_FILE = path.join(HEADROOM_DIR, "proxy.pid"); +const LOG_FILE = path.join(HEADROOM_DIR, "proxy.log"); +const DEFAULT_PORT = 8787; +const STARTUP_TIMEOUT_MS = 8000; + +function ensureDir() { + if (!fs.existsSync(HEADROOM_DIR)) fs.mkdirSync(HEADROOM_DIR, { recursive: true }); +} + +function readPid() { + try { + if (fs.existsSync(PID_FILE)) return parseInt(fs.readFileSync(PID_FILE, "utf8"), 10); + } catch { /* ignore */ } + return null; +} + +function writePid(pid) { + ensureDir(); + fs.writeFileSync(PID_FILE, String(pid)); +} + +function clearPid() { + try { if (fs.existsSync(PID_FILE)) fs.unlinkSync(PID_FILE); } catch { /* ignore */ } +} + +// process.kill throws if pid is dead β€” use this to probe. +export function isPidAlive(pid) { + if (!pid || typeof pid !== "number") return false; + try { process.kill(pid, 0); return true; } catch { return false; } +} + +export function getManagedPid() { + const pid = readPid(); + return pid && isPidAlive(pid) ? pid : null; +} + +export async function startHeadroomProxy({ port = DEFAULT_PORT } = {}) { + const safePort = Number(port) > 0 && Number(port) < 65536 ? Number(port) : DEFAULT_PORT; + const binary = findHeadroomBinary(); + if (!binary) { + const err = new Error("Headroom CLI not installed"); + err.code = "NOT_INSTALLED"; + throw err; + } + + const existing = getManagedPid(); + if (existing) return { pid: existing, alreadyRunning: true }; + + ensureDir(); + // spawn stdio requires fd numbers, not WriteStream objects. + const outFd = fs.openSync(LOG_FILE, "a"); + + const child = spawn(binary, ["proxy", "--port", String(safePort)], { + stdio: ["ignore", outFd, outFd], + detached: true, + windowsHide: true, + env: { ...process.env }, + }); + + if (!child.pid) { + fs.closeSync(outFd); + const err = new Error("Failed to spawn headroom proxy"); + err.code = "SPAWN_FAILED"; + throw err; + } + + child.unref(); + writePid(child.pid); + + // Wait until the process either stays alive briefly (success) or exits fast (failure). + await new Promise((resolve, reject) => { + const startupTimer = setTimeout(() => { + if (isPidAlive(child.pid)) resolve(); + else reject(new Error("headroom proxy exited during startup β€” see proxy.log")); + }, STARTUP_TIMEOUT_MS); + + child.once("exit", (code) => { + clearTimeout(startupTimer); + clearPid(); + fs.closeSync(outFd); + const e = new Error(`headroom proxy exited early (code=${code}) β€” see proxy.log`); + e.code = "EARLY_EXIT"; + reject(e); + }); + }); + + // Close parent's copy of the fd; child retains its own after unref. + fs.closeSync(outFd); + + return { pid: child.pid, alreadyRunning: false }; +} + +export function stopHeadroomProxy() { + const pid = getManagedPid(); + if (!pid) return { stopped: false, reason: "not_running" }; + try { + process.kill(pid, "SIGTERM"); + // Give it a moment, then force if still alive. + setTimeout(() => { + if (isPidAlive(pid)) { + try { process.kill(pid, "SIGKILL"); } catch { /* already gone */ } + } + }, 2000); + clearPid(); + return { stopped: true, pid }; + } catch (e) { + clearPid(); + const err = new Error(`Failed to stop headroom proxy: ${e.message}`); + err.code = "STOP_FAILED"; + throw err; + } +} + +export function getHeadroomLogTail(maxLines = 200) { + try { + if (!fs.existsSync(LOG_FILE)) return ""; + const content = fs.readFileSync(LOG_FILE, "utf8"); + const lines = content.split(/\r?\n/).filter(Boolean); + return lines.slice(-maxLines).join("\n"); + } catch { return ""; } +} diff --git a/src/sse/handlers/chat.js b/src/sse/handlers/chat.js index 75c26b5d..311f33d9 100644 --- a/src/sse/handlers/chat.js +++ b/src/sse/handlers/chat.js @@ -251,8 +251,13 @@ async function handleSingleModelChat(body, modelStr, clientRawRequest = null, re apiKey, ccFilterNaming: !!chatSettings.ccFilterNaming, rtkEnabled: !!chatSettings.rtkEnabled, + headroomEnabled: !!chatSettings.headroomEnabled, + headroomUrl: chatSettings.headroomUrl || "http://localhost:8787", + headroomCompressUserMessages: !!chatSettings.headroomCompressUserMessages, cavemanEnabled: !!chatSettings.cavemanEnabled, cavemanLevel: chatSettings.cavemanLevel || "full", + ponytailEnabled: !!chatSettings.ponytailEnabled, + ponytailLevel: chatSettings.ponytailLevel || "full", providerThinking, // Detect source format by endpoint + body sourceFormatOverride: request?.url ? detectFormatByEndpoint(new URL(request.url).pathname, body) : null, diff --git a/tests/unit/headroom.test.js b/tests/unit/headroom.test.js new file mode 100644 index 00000000..f4e4baca --- /dev/null +++ b/tests/unit/headroom.test.js @@ -0,0 +1,73 @@ +import { describe, it, expect, vi, afterEach } from "vitest"; +import { compressWithHeadroom, formatHeadroomLog } from "../../open-sse/rtk/headroom.js"; + +afterEach(() => { + vi.restoreAllMocks(); +}); + +describe("compressWithHeadroom", () => { + it("no-ops when disabled", async () => { + global.fetch = vi.fn(); + const body = { messages: [{ role: "user", content: "hello" }] }; + + const stats = await compressWithHeadroom(body, { enabled: false, url: "http://localhost:8787" }); + + expect(stats).toBeNull(); + expect(global.fetch).not.toHaveBeenCalled(); + expect(body.messages[0].content).toBe("hello"); + }); + + it("compresses messages in-place", async () => { + global.fetch = vi.fn(async () => new Response(JSON.stringify({ + messages: [{ role: "user", content: "short" }], + tokens_before: 100, + tokens_after: 20, + tokens_saved: 80, + }), { status: 200 })); + const body = { messages: [{ role: "user", content: "long" }] }; + + const stats = await compressWithHeadroom(body, { enabled: true, url: "http://headroom:8787/", model: "gpt-4o" }); + + expect(body.messages[0].content).toBe("short"); + expect(stats.tokens_saved).toBe(80); + expect(global.fetch).toHaveBeenCalledWith("http://headroom:8787/v1/compress", expect.objectContaining({ method: "POST" })); + }); + + it("compresses responses input in-place", async () => { + global.fetch = vi.fn(async () => new Response(JSON.stringify({ + messages: [{ role: "user", content: "short" }], + }), { status: 200 })); + const body = { input: [{ role: "user", content: "long" }] }; + + await compressWithHeadroom(body, { enabled: true, url: "http://localhost:8787" }); + + expect(body.input[0].content).toBe("short"); + }); + + it("fails open on bad response", async () => { + global.fetch = vi.fn(async () => new Response(JSON.stringify({ error: "bad" }), { status: 500 })); + const body = { messages: [{ role: "user", content: "long" }] }; + + const stats = await compressWithHeadroom(body, { enabled: true, url: "http://localhost:8787" }); + + expect(stats).toBeNull(); + expect(body.messages[0].content).toBe("long"); + }); + + it("skips unknown shapes", async () => { + global.fetch = vi.fn(); + const body = { contents: [{ parts: [{ text: "long" }] }] }; + + const stats = await compressWithHeadroom(body, { enabled: true, url: "http://localhost:8787" }); + + expect(stats).toBeNull(); + expect(global.fetch).not.toHaveBeenCalled(); + }); +}); + +describe("formatHeadroomLog", () => { + it("formats savings", () => { + expect(formatHeadroomLog({ tokens_before: 100, tokens_after: 25, tokens_saved: 75 })) + .toBe("saved 75 tokens / 100 (75.0%) after=25"); + }); +});