feat(headroom): add proxy lifecycle management + dashboard UI
Build on the optional Headroom Token Saver from Carmelo Campos
(PR: feat: add optional Headroom token saver). Add managed start/stop
of the local headroom proxy from the dashboard, install detection,
status probing, and a simplified Token Saver UI.
- detect headroom CLI + python>=3.10, probe proxy /health
- spawn/stop proxy as a detached, pid-tracked process
- /api/headroom/{status,start,stop} routes, gated local-only in dashboardGuard
- one-click Start/Stop Headroom modal, no manual config needed
- claude<->openai shape conversion for /v1/compress via 9router translators
Thanks to Carmelo Campos (@carmelogunsroses) for the original Headroom integration.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
30
README.md
30
README.md
@@ -407,6 +407,7 @@ Default URLs:
|
||||
| Feature | What It Does | Why It Matters |
|
||||
|---------|--------------|----------------|
|
||||
| 🚀 **RTK Token Saver** ([RTK](https://github.com/rtk-ai/rtk) ⭐40K) | Compress tool outputs (`git diff`, `grep`, `ls`, `tree`...) before sending to LLM | Save **20-40% input tokens** per request |
|
||||
| 🧠 **Headroom Token Saver** ([Headroom](https://github.com/chopratejas/headroom)) | Optional external `/v1/compress` proxy before provider routing | Save more context tokens without changing clients |
|
||||
| 🪨 **Caveman Mode** ([Caveman](https://github.com/JuliusBrussee/caveman) ⭐52K) | Inject caveman-speak prompt → LLM replies terse, technical substance preserved | Save **up to 65% output tokens** |
|
||||
| 🎯 **Smart 3-Tier Fallback** | Auto-route: Subscription → Cheap → Free | Never stop coding, zero downtime |
|
||||
| 📊 **Real-Time Quota Tracking** | Live token count + reset countdown | Maximize subscription value |
|
||||
@@ -437,6 +438,35 @@ Without RTK: 47K tokens sent to LLM
|
||||
With RTK: 28K tokens sent to LLM (40% saved · same context · same answer)
|
||||
```
|
||||
|
||||
### 🧠 Headroom Token Saver
|
||||
|
||||
Headroom is optional and runs separately. 9Router calls Headroom's local `/v1/compress` endpoint, then keeps normal routing, fallback, auth, and usage tracking:
|
||||
|
||||
```
|
||||
Client → 9Router → Headroom /v1/compress → 9Router → provider
|
||||
```
|
||||
|
||||
Local setup:
|
||||
|
||||
```bash
|
||||
pip install "headroom-ai[proxy]"
|
||||
headroom proxy --port 8787
|
||||
```
|
||||
|
||||
Enable in Dashboard → Endpoint → Token Saver → Headroom. Default URL: `http://localhost:8787`.
|
||||
|
||||
Docker examples:
|
||||
|
||||
```bash
|
||||
# Headroom service in same Docker network
|
||||
http://headroom:8787
|
||||
|
||||
# Headroom running on host machine
|
||||
http://host.docker.internal:8787
|
||||
```
|
||||
|
||||
If Headroom is down or returns an error, 9Router fails open and sends the original request.
|
||||
|
||||
### 🎯 Smart 3-Tier Fallback
|
||||
|
||||
Create combos with automatic fallback:
|
||||
|
||||
@@ -39,6 +39,8 @@ async function showSettingsMenu(breadcrumb = []) {
|
||||
// RTK section
|
||||
const rtkOn = data?.settings?.rtkEnabled !== false;
|
||||
lines.push(` RTK: ${rtkOn ? `${COLORS.green}ON${COLORS.reset}` : `${COLORS.red}OFF${COLORS.reset}`} ${COLORS.dim}(Token Saver)${COLORS.reset}`);
|
||||
const headroomOn = data?.settings?.headroomEnabled === true;
|
||||
lines.push(` Headroom: ${headroomOn ? `${COLORS.green}ON${COLORS.reset}` : `${COLORS.red}OFF${COLORS.reset}`} ${COLORS.dim}(${data?.settings?.headroomUrl || "http://localhost:8787"})${COLORS.reset}`);
|
||||
|
||||
// Auth mode section
|
||||
const authMode = data?.settings?.authMode || "password";
|
||||
@@ -73,6 +75,13 @@ async function showSettingsMenu(breadcrumb = []) {
|
||||
},
|
||||
action: async (d) => { await toggleRtk(d?.settings?.rtkEnabled !== false); return true; }
|
||||
},
|
||||
{
|
||||
label: (d) => {
|
||||
const on = d?.settings?.headroomEnabled === true;
|
||||
return `Token Saver (Headroom): ${on ? "ON" : "OFF"} → toggle`;
|
||||
},
|
||||
action: async (d) => { await toggleHeadroom(d?.settings?.headroomEnabled === true); return true; }
|
||||
},
|
||||
{
|
||||
label: "🔑 Reset Password to Default",
|
||||
action: async () => { await resetPassword(); return true; }
|
||||
@@ -160,6 +169,17 @@ async function toggleRtk(currentlyOn) {
|
||||
await pause();
|
||||
}
|
||||
|
||||
async function toggleHeadroom(currentlyOn) {
|
||||
const next = !currentlyOn;
|
||||
const result = await api.updateSettings({ headroomEnabled: next });
|
||||
if (result.success) {
|
||||
showStatus(`Headroom ${next ? "enabled" : "disabled"}`, "success");
|
||||
} else {
|
||||
showStatus(`Failed: ${result.error}`, "error");
|
||||
}
|
||||
await pause();
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset dashboard password to default via server API (writes the live SQLite DB).
|
||||
* After reset, user can log in with the default password "123456".
|
||||
|
||||
@@ -4,17 +4,20 @@ Provider-agnostic SSE engine: one OpenAI-style request → any provider (LLM cha
|
||||
|
||||
## Request lifecycle (chat)
|
||||
|
||||
`handlers/chatCore.js` → `services/model.js` `parseModel` (resolve `provider/model`) → `executors/index.js` `getExecutor(provider)` → `translator/index.js` `translateRequest` (client format → provider format) → `executor.execute()` (streams upstream) → `translateResponse` (provider chunks → client format) → SSE out.
|
||||
`handlers/chatCore.js` → `services/model.js` `parseModel` (resolve `provider/model`) → **pre-translate hooks** (`rtk/` tool_result compress, `rtk/headroom.js` proxy compress, `rtk/caveman.js` system inject — all fail-open) → `executors/index.js` `getExecutor(provider)` → `translator/index.js` `translateRequest` (client format → provider format) → `executor.execute()` (streams upstream) → `translateResponse` (provider chunks → client format) → SSE out.
|
||||
|
||||
## Directory map
|
||||
|
||||
- `config/` — ALL constants/config (no hardcode elsewhere). `providers.js`/`registry/` (provider defs), `providerModels.js` (alias→models matrix), `runtimeConfig.js` (timeouts, token limits), `*Constants.js`.
|
||||
- `translator/` — format conversion. `request/<from>-to-<to>.js`, `response/<from>-to-<to>.js`, `schema/` (enums: ROLE, CLAUDE_BLOCK…), `concerns/` (shared logic), `formats/` (per-format). See `tests/translator/AGENTS.md`.
|
||||
- `translator/` — format conversion. `request/<from>-to-<to>.js`, `response/<from>-to-<to>.js`, `schema/` (enums: ROLE, CLAUDE_BLOCK…), `concerns/` (shared logic), `formats.js`+`formats/` (per-format). `index.js` is the registry/entry.
|
||||
- `executors/` — per-provider upstream call. `base.js` (BaseExecutor), one file per special provider, `index.js` map.
|
||||
- `providers/` — registry build + `capabilities.js` + `pricing.js`. Entry: `index.js` (PROVIDERS).
|
||||
- `handlers/` — per-modality cores (chat/image/embedding/tts/stt/search) + sub-provider folders.
|
||||
- `services/` — `tokenRefresh/`, `usage/`, `combo.js`, `accountFallback.js`, `model.js`.
|
||||
- `utils/` — streamHandler, error, sessionManager, claudeCloaking.
|
||||
- `handlers/` — per-modality cores (chat/image/embedding/tts/stt/search) + sub-provider folders. `chatCore/` has the streaming/non-streaming/sse-to-json handlers.
|
||||
- `rtk/` — request token-killer. `index.js` compresses `tool_result` content in-place (OpenAI/Claude/Kiro shapes); `filters/` per-tool compressors + `autodetect.js`; `headroom.js` external compress proxy; `caveman.js` system-prompt injector.
|
||||
- `transformer/` — `responsesTransformer.js` (Chat Completions SSE → Codex Responses API SSE), `streamToJsonConverter.js`.
|
||||
- `shared/` — cross-provider auth/identity: `clineAuth.js`, `machineId.js`, `qoder/`.
|
||||
- `services/` — `model.js`, `provider.js`, `accountFallback.js`, `combo.js`, `compact.js`, `tokenRefresh/`+`tokenRefresh.js`, `oauthCredentialManager.js`, `usage/`, `projectId.js`, `kiroModels.js`/`qoderModels.js`.
|
||||
- `utils/` — streamHandler, stream, sse, error, sessionManager, claudeCloaking, clientDetector, proxyFetch (patches global fetch), cursorProtobuf/cursorChecksum, ollamaTransform.
|
||||
|
||||
## Conventions
|
||||
|
||||
@@ -33,3 +36,4 @@ Provider-agnostic SSE engine: one OpenAI-style request → any provider (LLM cha
|
||||
- OpenAI bridge is lossy (thinking, non-base64 images, tool ids, is_error) — prefer a direct route for fragile pairs.
|
||||
- `registry/index.js` is an auto-generated static import list; regenerate it (don't hand-edit) after adding a `registry/{id}.js`. REGISTRY_TEMPLATE is excluded by design.
|
||||
- Special binary/protobuf formats (kiro EventStream, cursor protobuf, commandcode NDJSON) don't round-trip through OpenAI — handle in their executor.
|
||||
- `rtk/` + `headroom.js` mutate the request body in-place and are **fail-open**: any error returns null and leaves the body untouched — never throw out of them. RTK skips `is_error`/`status:"error"` tool results to preserve traces.
|
||||
|
||||
@@ -22,7 +22,8 @@ const SERVER_ID_PATTERN = /^(rs|fc|resp|msg)_/;
|
||||
// Hosted tool types that Codex/OpenAI Responses executes server-side
|
||||
const CODEX_HOSTED_TOOL_TYPES = new Set([
|
||||
"image_generation", "web_search", "web_search_preview", "file_search",
|
||||
"computer", "computer_use_preview", "code_interpreter", "mcp", "local_shell"
|
||||
"computer", "computer_use_preview", "code_interpreter", "mcp", "local_shell",
|
||||
"tool_search"
|
||||
]);
|
||||
|
||||
// Allowlist of fields accepted by Codex Responses API — anything else is stripped
|
||||
|
||||
@@ -116,6 +116,11 @@ export class DefaultExecutor extends BaseExecutor {
|
||||
}
|
||||
|
||||
buildUrl(model, stream, urlIndex = 0, credentials = null) {
|
||||
// Runtime transport (multi-endpoint providers): use the sourceFormat-matched endpoint
|
||||
const rt = credentials?.runtimeTransport;
|
||||
if (rt?.baseUrl) {
|
||||
return rt.urlSuffix ? `${rt.baseUrl}${rt.urlSuffix}` : rt.baseUrl;
|
||||
}
|
||||
if (this.provider?.startsWith?.("openai-compatible-")) {
|
||||
const baseUrl = credentials?.providerSpecificData?.baseUrl || OPENAI_COMPAT_BASE;
|
||||
const normalized = baseUrl.replace(/\/$/, "");
|
||||
@@ -156,8 +161,9 @@ export class DefaultExecutor extends BaseExecutor {
|
||||
}
|
||||
|
||||
buildHeaders(credentials, stream = true) {
|
||||
const headers = { "Content-Type": "application/json", ...this.config.headers };
|
||||
const desc = AUTH_DESCRIPTORS[this.provider] || this.resolveAuthDescriptor();
|
||||
const rt = credentials?.runtimeTransport;
|
||||
const headers = { "Content-Type": "application/json", ...(rt ? rt.headers : this.config.headers) };
|
||||
const desc = rt?.auth || AUTH_DESCRIPTORS[this.provider] || this.resolveAuthDescriptor();
|
||||
// Hooks run BEFORE auth so dynamic overlays (claude cached headers) can't clobber the token.
|
||||
for (const hook of desc.hooks || []) HEADER_HOOKS[hook]?.(headers, credentials);
|
||||
applyAuth(headers, desc, credentials);
|
||||
|
||||
@@ -8,13 +8,13 @@ export class XiaomiTokenplanExecutor extends DefaultExecutor {
|
||||
super("xiaomi-tokenplan");
|
||||
}
|
||||
|
||||
// Token Plan keys are region-specific — always OpenAI-compatible /chat/completions
|
||||
// Token Plan keys are region-specific. Route per sourceFormat-matched transport:
|
||||
// claude → Anthropic /anthropic/v1/messages, openai → /chat/completions.
|
||||
buildUrl(model, stream, urlIndex = 0, credentials = null) {
|
||||
const baseUrl = resolveXiaomiTokenplanBaseUrl(credentials);
|
||||
// Claude-native aliases route to the Anthropic-compatible messages endpoint
|
||||
// if (getModelTargetFormat(this.provider, model) === FORMATS.CLAUDE) {
|
||||
// return `${baseUrl.replace(/\/v1\/?$/, "/anthropic/v1")}/messages`;
|
||||
// }
|
||||
if (credentials?.runtimeTransport?.format === "claude") {
|
||||
return `${baseUrl.replace(/\/v1\/?$/, "")}/anthropic/v1/messages`;
|
||||
}
|
||||
return `${baseUrl}/chat/completions`;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { detectFormat, getTargetFormat } from "../services/provider.js";
|
||||
import { detectFormat, getTargetFormat, resolveTransport } from "../services/provider.js";
|
||||
import { translateRequest } from "../translator/index.js";
|
||||
import { FORMATS } from "../translator/formats.js";
|
||||
import { normalizeClaudePassthrough } from "../translator/formats/claude.js";
|
||||
@@ -20,7 +20,9 @@ import { handleStreamingResponse, buildOnStreamComplete } from "./chatCore/strea
|
||||
import { detectClientTool, isNativePassthrough } from "../utils/clientDetector.js";
|
||||
import { dedupeTools } from "../utils/toolDeduper.js";
|
||||
import { injectCaveman } from "../rtk/caveman.js";
|
||||
import { injectPonytail } from "../rtk/ponytail.js";
|
||||
import { compressMessages, formatRtkLog } from "../rtk/index.js";
|
||||
import { compressWithHeadroom, formatHeadroomLog } from "../rtk/headroom.js";
|
||||
import { getCapabilitiesForModel } from "../providers/capabilities.js";
|
||||
import { stripUnsupportedModalities } from "../translator/concerns/modality.js";
|
||||
import { prefetchRemoteImages } from "../translator/concerns/prefetch.js";
|
||||
@@ -32,7 +34,7 @@ import { prefetchRemoteImages } from "../translator/concerns/prefetch.js";
|
||||
* @param {object} options.credentials - Provider credentials
|
||||
* @param {string} options.sourceFormatOverride - Override detected source format (e.g. "openai-responses")
|
||||
*/
|
||||
export async function handleChatCore({ body, modelInfo, credentials, log, onCredentialsRefreshed, onRequestSuccess, onDisconnect, clientRawRequest, connectionId, userAgent, apiKey, ccFilterNaming, rtkEnabled, cavemanEnabled, cavemanLevel, sourceFormatOverride, providerThinking }) {
|
||||
export async function handleChatCore({ body, modelInfo, credentials, log, onCredentialsRefreshed, onRequestSuccess, onDisconnect, clientRawRequest, connectionId, userAgent, apiKey, ccFilterNaming, rtkEnabled, headroomEnabled, headroomUrl, headroomCompressUserMessages, cavemanEnabled, cavemanLevel, ponytailEnabled, ponytailLevel, sourceFormatOverride, providerThinking }) {
|
||||
const { provider, model } = modelInfo;
|
||||
const requestStartTime = Date.now();
|
||||
|
||||
@@ -44,7 +46,10 @@ export async function handleChatCore({ body, modelInfo, credentials, log, onCred
|
||||
|
||||
const alias = PROVIDER_ID_TO_ALIAS[provider] || provider;
|
||||
const modelTargetFormat = getModelTargetFormat(alias, model);
|
||||
const targetFormat = modelTargetFormat || getTargetFormat(provider);
|
||||
// Multi-endpoint providers: pick transport matching sourceFormat → zero translation
|
||||
const runtimeTransport = resolveTransport(provider, sourceFormat);
|
||||
const targetFormat = modelTargetFormat || runtimeTransport?.format || getTargetFormat(provider);
|
||||
if (runtimeTransport && credentials) credentials.runtimeTransport = runtimeTransport;
|
||||
const stripList = getModelStrip(alias, model);
|
||||
const upstreamModel = getModelUpstreamId(alias, model);
|
||||
|
||||
@@ -149,12 +154,23 @@ export async function handleChatCore({ body, modelInfo, credentials, log, onCred
|
||||
const rtkLine = formatRtkLog(rtkStats);
|
||||
if (rtkLine) console.log(rtkLine);
|
||||
|
||||
// Headroom: optional external proxy compression; fail open if proxy is absent.
|
||||
const headroomStats = await compressWithHeadroom(translatedBody, { enabled: headroomEnabled, url: headroomUrl, model: upstreamModel, format: finalFormat, compressUserMessages: headroomCompressUserMessages });
|
||||
const headroomLine = formatHeadroomLog(headroomStats);
|
||||
if (headroomLine) log?.info?.("HEADROOM", headroomLine);
|
||||
|
||||
// Caveman: inject terse-style system prompt
|
||||
if (cavemanEnabled && cavemanLevel) {
|
||||
injectCaveman(translatedBody, finalFormat, cavemanLevel);
|
||||
log?.debug?.("CAVEMAN", `${cavemanLevel} | ${finalFormat}`);
|
||||
}
|
||||
|
||||
// Ponytail: inject lazy-senior-dev system prompt
|
||||
if (ponytailEnabled && ponytailLevel) {
|
||||
injectPonytail(translatedBody, finalFormat, ponytailLevel);
|
||||
log?.debug?.("PONYTAIL", `${ponytailLevel} | ${finalFormat}`);
|
||||
}
|
||||
|
||||
const executor = getExecutor(provider);
|
||||
trackPendingRequest(model, provider, connectionId, true);
|
||||
appendRequestLog({ model, provider, connectionId, status: "PENDING" }).catch(() => { });
|
||||
|
||||
@@ -32,7 +32,10 @@ export const PROVIDER_MODELS = {};
|
||||
export const PROVIDER_OAUTH = {};
|
||||
export const PROVIDER_MEDIA = {};
|
||||
for (const entry of REGISTRY) {
|
||||
if (entry.transport) PROVIDERS[entry.id] = buildTransport(entry.transport, entry.oauth);
|
||||
if (entry.transport) {
|
||||
PROVIDERS[entry.id] = buildTransport(entry.transport, entry.oauth);
|
||||
if (entry.transports) PROVIDERS[entry.id].transports = entry.transports;
|
||||
}
|
||||
if (entry.models !== undefined) PROVIDER_MODELS[entry.alias || entry.id] = entry.models.map(normalizeModel);
|
||||
if (entry.oauth) PROVIDER_OAUTH[entry.id] = entry.oauth;
|
||||
// Build PROVIDER_MEDIA from top-level fields (post-migration) + legacy entry.media
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { CLAUDE_API_HEADERS } from "../shared.js";
|
||||
|
||||
export default {
|
||||
id: "deepseek",
|
||||
priority: 110,
|
||||
@@ -24,6 +26,20 @@ export default {
|
||||
scope: "all",
|
||||
},
|
||||
},
|
||||
// Multi-endpoint: pick the transport matching client sourceFormat to skip translation.
|
||||
transports: [
|
||||
{
|
||||
format: "openai",
|
||||
baseUrl: "https://api.deepseek.com/chat/completions",
|
||||
auth: { combined: true, header: "Authorization", scheme: "bearer" },
|
||||
},
|
||||
{
|
||||
format: "claude",
|
||||
baseUrl: "https://api.deepseek.com/anthropic/v1/messages",
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: { combined: true, header: "x-api-key", scheme: "raw" },
|
||||
},
|
||||
],
|
||||
models: [
|
||||
{ id: "deepseek-v4-pro", name: "DeepSeek V4 Pro" },
|
||||
{ id: "deepseek-v4-pro-max", name: "DeepSeek V4 Pro Max", upstreamModelId: "deepseek-v4-pro" },
|
||||
|
||||
@@ -19,10 +19,7 @@ export default {
|
||||
baseUrl: "https://api.z.ai/api/anthropic/v1/messages",
|
||||
format: "claude",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: {
|
||||
"Anthropic-Version": "2023-06-01",
|
||||
"Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14",
|
||||
},
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: {
|
||||
combined: true,
|
||||
header: "x-api-key",
|
||||
@@ -32,6 +29,21 @@ export default {
|
||||
url: "https://api.z.ai/api/monitor/usage/quota/limit",
|
||||
},
|
||||
},
|
||||
// Multi-endpoint: pick the transport matching client sourceFormat to skip translation.
|
||||
transports: [
|
||||
{
|
||||
format: "openai",
|
||||
baseUrl: "https://api.z.ai/api/coding/paas/v4/chat/completions",
|
||||
auth: { combined: true, header: "Authorization", scheme: "bearer" },
|
||||
},
|
||||
{
|
||||
format: "claude",
|
||||
baseUrl: "https://api.z.ai/api/anthropic/v1/messages",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: { combined: true, header: "x-api-key", scheme: "raw" },
|
||||
},
|
||||
],
|
||||
models: [
|
||||
{ id: "glm-5.2", name: "GLM 5.2" },
|
||||
{ id: "glm-5.1", name: "GLM 5.1" },
|
||||
|
||||
@@ -20,10 +20,7 @@ export default {
|
||||
baseUrl: "https://api.kimi.com/coding/v1/messages",
|
||||
format: "claude",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: {
|
||||
"Anthropic-Version": "2023-06-01",
|
||||
"Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14",
|
||||
},
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
clientId: "17e5f671-d194-4dfb-9706-5516cb48c098",
|
||||
tokenUrl: "https://auth.kimi.com/api/oauth/token",
|
||||
refreshUrl: "https://auth.kimi.com/api/oauth/token",
|
||||
@@ -36,6 +33,21 @@ export default {
|
||||
],
|
||||
},
|
||||
},
|
||||
// Multi-endpoint: pick the transport matching client sourceFormat to skip translation.
|
||||
transports: [
|
||||
{
|
||||
format: "openai",
|
||||
baseUrl: "https://api.kimi.com/coding/v1/chat/completions",
|
||||
auth: { combined: true, header: "Authorization", scheme: "bearer", hooks: ["kimiHeaders"] },
|
||||
},
|
||||
{
|
||||
format: "claude",
|
||||
baseUrl: "https://api.kimi.com/coding/v1/messages",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: { combined: true, header: "x-api-key", scheme: "raw", hooks: ["kimiHeaders"] },
|
||||
},
|
||||
],
|
||||
models: [
|
||||
{ id: "kimi-k2.6", name: "Kimi K2.6" },
|
||||
{ id: "kimi-k2.5", name: "Kimi K2.5" },
|
||||
|
||||
@@ -19,16 +19,28 @@ export default {
|
||||
baseUrl: "https://api.kimi.com/coding/v1/messages",
|
||||
format: "claude",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: {
|
||||
"Anthropic-Version": "2023-06-01",
|
||||
"Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14",
|
||||
},
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: {
|
||||
combined: true,
|
||||
header: "x-api-key",
|
||||
scheme: "raw",
|
||||
},
|
||||
},
|
||||
// Multi-endpoint: pick the transport matching client sourceFormat to skip translation.
|
||||
transports: [
|
||||
{
|
||||
format: "openai",
|
||||
baseUrl: "https://api.kimi.com/coding/v1/chat/completions",
|
||||
auth: { combined: true, header: "Authorization", scheme: "bearer" },
|
||||
},
|
||||
{
|
||||
format: "claude",
|
||||
baseUrl: "https://api.kimi.com/coding/v1/messages",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: { combined: true, header: "x-api-key", scheme: "raw" },
|
||||
},
|
||||
],
|
||||
models: [
|
||||
{ id: "kimi-k2.6", name: "Kimi K2.6" },
|
||||
{ id: "kimi-k2.5", name: "Kimi K2.5" },
|
||||
|
||||
@@ -19,10 +19,7 @@ export default {
|
||||
baseUrl: "https://api.minimaxi.com/anthropic/v1/messages",
|
||||
format: "claude",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: {
|
||||
"Anthropic-Version": "2023-06-01",
|
||||
"Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14",
|
||||
},
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
quirks: {
|
||||
dropOutputConfig: true,
|
||||
},
|
||||
@@ -41,6 +38,21 @@ export default {
|
||||
],
|
||||
},
|
||||
},
|
||||
// Multi-endpoint: pick the transport matching client sourceFormat to skip translation.
|
||||
transports: [
|
||||
{
|
||||
format: "openai",
|
||||
baseUrl: "https://api.minimaxi.com/v1/chat/completions",
|
||||
auth: { combined: true, header: "Authorization", scheme: "bearer" },
|
||||
},
|
||||
{
|
||||
format: "claude",
|
||||
baseUrl: "https://api.minimaxi.com/anthropic/v1/messages",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: { combined: true, header: "x-api-key", scheme: "raw" },
|
||||
},
|
||||
],
|
||||
models: [
|
||||
{ id: "MiniMax-M3", name: "MiniMax M3", targetFormat: "claude" },
|
||||
{ id: "MiniMax-M2.7", name: "MiniMax M2.7" },
|
||||
|
||||
@@ -19,10 +19,7 @@ export default {
|
||||
baseUrl: "https://api.minimax.io/anthropic/v1/messages",
|
||||
format: "claude",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: {
|
||||
"Anthropic-Version": "2023-06-01",
|
||||
"Anthropic-Beta": "claude-code-20250219,interleaved-thinking-2025-05-14",
|
||||
},
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
quirks: {
|
||||
dropOutputConfig: true,
|
||||
},
|
||||
@@ -41,6 +38,21 @@ export default {
|
||||
],
|
||||
},
|
||||
},
|
||||
// Multi-endpoint: pick the transport matching client sourceFormat to skip translation.
|
||||
transports: [
|
||||
{
|
||||
format: "openai",
|
||||
baseUrl: "https://api.minimax.io/v1/chat/completions",
|
||||
auth: { combined: true, header: "Authorization", scheme: "bearer" },
|
||||
},
|
||||
{
|
||||
format: "claude",
|
||||
baseUrl: "https://api.minimax.io/anthropic/v1/messages",
|
||||
urlSuffix: "?beta=true",
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: { combined: true, header: "x-api-key", scheme: "raw" },
|
||||
},
|
||||
],
|
||||
models: [
|
||||
{ id: "MiniMax-M3", name: "MiniMax M3", targetFormat: "claude" },
|
||||
{ id: "MiniMax-M2.7", name: "MiniMax M2.7" },
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { CLAUDE_API_HEADERS } from "../shared.js";
|
||||
|
||||
export default {
|
||||
id: "xiaomi-mimo",
|
||||
priority: 290,
|
||||
@@ -21,6 +23,20 @@ export default {
|
||||
baseUrl: "https://api.xiaomimimo.com/v1/chat/completions",
|
||||
validateUrl: "https://api.xiaomimimo.com/v1/models",
|
||||
},
|
||||
// Multi-endpoint: pick the transport matching client sourceFormat to skip translation.
|
||||
transports: [
|
||||
{
|
||||
format: "openai",
|
||||
baseUrl: "https://api.xiaomimimo.com/v1/chat/completions",
|
||||
auth: { combined: true, header: "Authorization", scheme: "bearer" },
|
||||
},
|
||||
{
|
||||
format: "claude",
|
||||
baseUrl: "https://api.xiaomimimo.com/anthropic/v1/messages",
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: { combined: true, header: "x-api-key", scheme: "raw" },
|
||||
},
|
||||
],
|
||||
models: [
|
||||
{ id: "mimo-v2.5-pro", name: "MiMo V2.5 Pro" },
|
||||
{ id: "mimo-v2.5", name: "MiMo V2.5" },
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { CLAUDE_API_HEADERS } from "../shared.js";
|
||||
|
||||
export default {
|
||||
id: "xiaomi-tokenplan",
|
||||
priority: 300,
|
||||
@@ -29,6 +31,19 @@ export default {
|
||||
},
|
||||
defaultRegion: "sgp",
|
||||
},
|
||||
// Multi-endpoint: pick the transport matching client sourceFormat to skip translation.
|
||||
// baseUrl omitted — region-dynamic, resolved in the executor's buildUrl.
|
||||
transports: [
|
||||
{
|
||||
format: "openai",
|
||||
auth: { combined: true, header: "Authorization", scheme: "bearer" },
|
||||
},
|
||||
{
|
||||
format: "claude",
|
||||
headers: { ...CLAUDE_API_HEADERS },
|
||||
auth: { combined: true, header: "x-api-key", scheme: "raw" },
|
||||
},
|
||||
],
|
||||
models: [
|
||||
{ id: "mimo-v2.5-pro", name: "MiMo V2.5 Pro" },
|
||||
{ id: "mimo-v2.5-pro-claude", name: "MiMo V2.5 Pro (Claude Native)", targetFormat: "claude", upstreamModelId: "mimo-v2.5-pro" },
|
||||
|
||||
@@ -1,100 +1,9 @@
|
||||
// Caveman injector: appends a caveman-style instruction into the system message
|
||||
// of the final request body, just before it is dispatched to the provider executor.
|
||||
// Dispatches by format so it works for both translated and native-passthrough flows.
|
||||
|
||||
import { FORMATS } from "../translator/formats.js";
|
||||
import { injectSystemPrompt } from "./systemInject.js";
|
||||
import { CAVEMAN_PROMPTS } from "./cavemanPrompts.js";
|
||||
|
||||
const SEP = "\n\n";
|
||||
|
||||
export function injectCaveman(body, format, level) {
|
||||
const prompt = CAVEMAN_PROMPTS[level];
|
||||
if (!body || !prompt) return;
|
||||
|
||||
switch (format) {
|
||||
case FORMATS.CLAUDE:
|
||||
injectClaudeSystem(body, prompt);
|
||||
return;
|
||||
case FORMATS.GEMINI:
|
||||
case FORMATS.GEMINI_CLI:
|
||||
case FORMATS.VERTEX:
|
||||
case FORMATS.ANTIGRAVITY:
|
||||
// Antigravity wraps Gemini shape in body.request → injectGeminiSystem handles it
|
||||
injectGeminiSystem(body, prompt);
|
||||
return;
|
||||
default:
|
||||
// OpenAI and OpenAI-shaped formats (responses/codex/cursor/kiro/ollama)
|
||||
injectMessagesSystem(body, prompt);
|
||||
}
|
||||
}
|
||||
|
||||
// OpenAI-shaped: messages[] (chat) or input[] (responses) or instructions (responses string)
|
||||
function injectMessagesSystem(body, prompt) {
|
||||
// OpenAI Responses API: top-level string field
|
||||
if (typeof body.instructions === "string") {
|
||||
body.instructions = body.instructions
|
||||
? `${body.instructions}${SEP}${prompt}`
|
||||
: prompt;
|
||||
return;
|
||||
}
|
||||
|
||||
const arr = Array.isArray(body.messages) ? body.messages
|
||||
: Array.isArray(body.input) ? body.input
|
||||
: null;
|
||||
if (!arr) return;
|
||||
|
||||
const idx = arr.findIndex(m => m && (m.role === "system" || m.role === "developer"));
|
||||
if (idx >= 0) {
|
||||
appendToOpenAIMessage(arr[idx], prompt);
|
||||
} else {
|
||||
arr.unshift({ role: "system", content: prompt });
|
||||
}
|
||||
}
|
||||
|
||||
function appendToOpenAIMessage(msg, prompt) {
|
||||
if (typeof msg.content === "string") {
|
||||
msg.content = `${msg.content}${SEP}${prompt}`;
|
||||
} else if (Array.isArray(msg.content)) {
|
||||
// Responses-style array of parts {type:"input_text"|"text", text}
|
||||
msg.content.push({ type: "input_text", text: prompt });
|
||||
} else {
|
||||
msg.content = prompt;
|
||||
}
|
||||
}
|
||||
|
||||
// Claude shape: body.system as string | array of {type:"text", text}
|
||||
// Insert before the last cache_control block to keep caveman inside the cached prefix.
|
||||
function injectClaudeSystem(body, prompt) {
|
||||
if (typeof body.system === "string" && body.system.length > 0) {
|
||||
body.system = `${body.system}${SEP}${prompt}`;
|
||||
return;
|
||||
}
|
||||
if (Array.isArray(body.system)) {
|
||||
const block = { type: "text", text: prompt };
|
||||
let lastCacheIdx = -1;
|
||||
for (let i = body.system.length - 1; i >= 0; i--) {
|
||||
if (body.system[i]?.cache_control) { lastCacheIdx = i; break; }
|
||||
}
|
||||
if (lastCacheIdx >= 0) {
|
||||
body.system.splice(lastCacheIdx, 0, block);
|
||||
} else {
|
||||
body.system.push(block);
|
||||
}
|
||||
return;
|
||||
}
|
||||
body.system = prompt;
|
||||
}
|
||||
|
||||
// Gemini shape: body.system_instruction | body.systemInstruction | body.request.systemInstruction
|
||||
// Each shape: { parts: [{ text }] }
|
||||
function injectGeminiSystem(body, prompt) {
|
||||
const target = body.request && typeof body.request === "object" ? body.request : body;
|
||||
const useSnake = Object.prototype.hasOwnProperty.call(target, "system_instruction");
|
||||
const key = useSnake ? "system_instruction" : "systemInstruction";
|
||||
const sys = target[key];
|
||||
if (sys && Array.isArray(sys.parts)) {
|
||||
sys.parts.push({ text: prompt });
|
||||
return;
|
||||
}
|
||||
target[key] = { parts: [{ text: prompt }] };
|
||||
injectSystemPrompt(body, format, CAVEMAN_PROMPTS[level]);
|
||||
}
|
||||
|
||||
63
open-sse/rtk/headroom.js
Normal file
63
open-sse/rtk/headroom.js
Normal file
@@ -0,0 +1,63 @@
|
||||
import { claudeToOpenAIRequest } from "../translator/request/claude-to-openai.js";
|
||||
import { openaiToClaudeRequest } from "../translator/request/openai-to-claude.js";
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 3000;
|
||||
|
||||
// POST messages to Headroom /v1/compress; returns compressed messages + stats or null.
|
||||
async function callCompress(url, messages, model, timeoutMs, compressUserMessages) {
|
||||
const endpoint = `${String(url).replace(/\/$/, "")}/v1/compress`;
|
||||
const payload = { messages, model };
|
||||
if (compressUserMessages) payload.config = { compress_user_messages: true };
|
||||
const res = await fetch(endpoint, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(payload),
|
||||
signal: AbortSignal.timeout(timeoutMs),
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
const data = await res.json();
|
||||
if (!Array.isArray(data?.messages)) return null;
|
||||
return data;
|
||||
}
|
||||
|
||||
// Compress request body via Headroom proxy. Fail-open: returns null on any error.
|
||||
// /v1/compress only understands OpenAI shape, so Claude bodies are translated
|
||||
// to OpenAI, compressed, then translated back using 9Router's own translators.
|
||||
export async function compressWithHeadroom(body, { enabled, url, model, format, compressUserMessages, timeoutMs = DEFAULT_TIMEOUT_MS } = {}) {
|
||||
if (!enabled || !url || !body) return null;
|
||||
|
||||
try {
|
||||
// Claude shape: translate → OpenAI → compress → translate back.
|
||||
if (format === "claude") {
|
||||
const oai = claudeToOpenAIRequest(model, body, false);
|
||||
if (!Array.isArray(oai?.messages)) return null;
|
||||
const data = await callCompress(url, oai.messages, model, timeoutMs, compressUserMessages);
|
||||
if (!data) return null;
|
||||
const claudeBody = openaiToClaudeRequest(model, { ...oai, messages: data.messages }, false);
|
||||
if (Array.isArray(claudeBody?.messages)) body.messages = claudeBody.messages;
|
||||
if (claudeBody?.system !== undefined) body.system = claudeBody.system;
|
||||
return data;
|
||||
}
|
||||
|
||||
// OpenAI shape: messages/input go straight to the proxy.
|
||||
const key = Array.isArray(body.messages) ? "messages"
|
||||
: Array.isArray(body.input) ? "input"
|
||||
: null;
|
||||
if (!key) return null;
|
||||
const data = await callCompress(url, body[key], model, timeoutMs, compressUserMessages);
|
||||
if (!data) return null;
|
||||
body[key] = data.messages;
|
||||
return data;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function formatHeadroomLog(stats) {
|
||||
if (!stats) return null;
|
||||
const before = stats.tokens_before || 0;
|
||||
const after = stats.tokens_after || 0;
|
||||
const saved = stats.tokens_saved || 0;
|
||||
const pct = before > 0 ? ((saved / before) * 100).toFixed(1) : "0";
|
||||
return `saved ${saved} tokens / ${before} (${pct}%) ${after ? `after=${after}` : ""}`.trim();
|
||||
}
|
||||
9
open-sse/rtk/ponytail.js
Normal file
9
open-sse/rtk/ponytail.js
Normal file
@@ -0,0 +1,9 @@
|
||||
// Ponytail injector: appends the "lazy senior dev" instruction into the system
|
||||
// message of the final request body, just before dispatch to the provider executor.
|
||||
|
||||
import { injectSystemPrompt } from "./systemInject.js";
|
||||
import { PONYTAIL_PROMPTS } from "./ponytailPrompt.js";
|
||||
|
||||
export function injectPonytail(body, format, level) {
|
||||
injectSystemPrompt(body, format, PONYTAIL_PROMPTS[level]);
|
||||
}
|
||||
52
open-sse/rtk/ponytailPrompt.js
Normal file
52
open-sse/rtk/ponytailPrompt.js
Normal file
@@ -0,0 +1,52 @@
|
||||
// Ponytail intensity-level prompts injected into system message to bias toward minimal code.
|
||||
// Adapted from ponytail skill (https://github.com/DietrichGebert/ponytail).
|
||||
|
||||
export const PONYTAIL_LEVELS = {
|
||||
LITE: "lite",
|
||||
FULL: "full",
|
||||
ULTRA: "ultra",
|
||||
};
|
||||
|
||||
const SHARED_PERSONA = "You are a lazy senior developer. Lazy means efficient, not careless. The best code is the code never written.";
|
||||
|
||||
const SHARED_LADDER = "Before writing code, stop at the first rung that holds: 1) Does this need to exist at all? (YAGNI) 2) Stdlib does it? Use it. 3) Native platform feature covers it? Use it (CSS over JS, DB constraint over app code). 4) Already-installed dependency solves it? Use it; never add a new one for what a few lines can do. 5) Can it be one line? One line. 6) Only then: the minimum code that works.";
|
||||
|
||||
const SHARED_RULES = "No unrequested abstractions (no interface with one implementation, no factory for one product, no config for a value that never changes). No boilerplate or scaffolding \"for later\". Deletion over addition. Boring over clever. Fewest files possible; shortest working diff wins. Two stdlib options the same size: take the edge-case-correct one. Mark deliberate simplifications with a `ponytail:` comment naming the ceiling and upgrade path.";
|
||||
|
||||
const SHARED_OUTPUT = "Code first. Then at most three short lines: what was skipped, when to add it. No essays or design notes. Pattern: `[code] → skipped: [X], add when [Y].`";
|
||||
|
||||
const SHARED_NOT_LAZY = "Never simplify away: input validation at trust boundaries, error handling that prevents data loss, security, accessibility, anything explicitly requested. Non-trivial logic leaves ONE runnable check behind (an assert-based self-check or one small test file; no frameworks). Trivial one-liners need no test.";
|
||||
|
||||
const SHARED_PERSISTENCE = "ACTIVE EVERY RESPONSE. No drift back to over-building. Still active if unsure.";
|
||||
|
||||
export const PONYTAIL_PROMPTS = {
|
||||
[PONYTAIL_LEVELS.LITE]: [
|
||||
SHARED_PERSONA,
|
||||
"Lite: build what's asked, but name the lazier alternative in one line. User picks.",
|
||||
SHARED_LADDER,
|
||||
SHARED_RULES,
|
||||
SHARED_OUTPUT,
|
||||
SHARED_NOT_LAZY,
|
||||
SHARED_PERSISTENCE,
|
||||
].join(" "),
|
||||
|
||||
[PONYTAIL_LEVELS.FULL]: [
|
||||
SHARED_PERSONA,
|
||||
"Full: the ladder enforced. Stdlib and native first. Shortest diff, shortest explanation.",
|
||||
SHARED_LADDER,
|
||||
SHARED_RULES,
|
||||
SHARED_OUTPUT,
|
||||
SHARED_NOT_LAZY,
|
||||
SHARED_PERSISTENCE,
|
||||
].join(" "),
|
||||
|
||||
[PONYTAIL_LEVELS.ULTRA]: [
|
||||
SHARED_PERSONA,
|
||||
"Ultra: YAGNI extremist. Deletion before addition. Ship the one-liner and challenge the rest of the requirement in the same response.",
|
||||
SHARED_LADDER,
|
||||
SHARED_RULES,
|
||||
SHARED_OUTPUT,
|
||||
SHARED_NOT_LAZY,
|
||||
SHARED_PERSISTENCE,
|
||||
].join(" "),
|
||||
};
|
||||
98
open-sse/rtk/systemInject.js
Normal file
98
open-sse/rtk/systemInject.js
Normal file
@@ -0,0 +1,98 @@
|
||||
// Shared system-prompt injector: appends an instruction into the system message of
|
||||
// the final request body, dispatching by format so it works for translated and
|
||||
// native-passthrough flows. Used by caveman.js and ponytail.js.
|
||||
|
||||
import { FORMATS } from "../translator/formats.js";
|
||||
|
||||
const SEP = "\n\n";
|
||||
|
||||
export function injectSystemPrompt(body, format, prompt) {
|
||||
if (!body || !prompt) return;
|
||||
|
||||
switch (format) {
|
||||
case FORMATS.CLAUDE:
|
||||
injectClaudeSystem(body, prompt);
|
||||
return;
|
||||
case FORMATS.GEMINI:
|
||||
case FORMATS.GEMINI_CLI:
|
||||
case FORMATS.VERTEX:
|
||||
case FORMATS.ANTIGRAVITY:
|
||||
// Antigravity wraps Gemini shape in body.request → injectGeminiSystem handles it
|
||||
injectGeminiSystem(body, prompt);
|
||||
return;
|
||||
default:
|
||||
// OpenAI and OpenAI-shaped formats (responses/codex/cursor/kiro/ollama)
|
||||
injectMessagesSystem(body, prompt);
|
||||
}
|
||||
}
|
||||
|
||||
// OpenAI-shaped: messages[] (chat) or input[] (responses) or instructions (responses string)
|
||||
function injectMessagesSystem(body, prompt) {
|
||||
// OpenAI Responses API: top-level string field
|
||||
if (typeof body.instructions === "string") {
|
||||
body.instructions = body.instructions
|
||||
? `${body.instructions}${SEP}${prompt}`
|
||||
: prompt;
|
||||
return;
|
||||
}
|
||||
|
||||
const arr = Array.isArray(body.messages) ? body.messages
|
||||
: Array.isArray(body.input) ? body.input
|
||||
: null;
|
||||
if (!arr) return;
|
||||
|
||||
const idx = arr.findIndex(m => m && (m.role === "system" || m.role === "developer"));
|
||||
if (idx >= 0) {
|
||||
appendToOpenAIMessage(arr[idx], prompt);
|
||||
} else {
|
||||
arr.unshift({ role: "system", content: prompt });
|
||||
}
|
||||
}
|
||||
|
||||
function appendToOpenAIMessage(msg, prompt) {
|
||||
if (typeof msg.content === "string") {
|
||||
msg.content = `${msg.content}${SEP}${prompt}`;
|
||||
} else if (Array.isArray(msg.content)) {
|
||||
// Responses-style array of parts {type:"input_text"|"text", text}
|
||||
msg.content.push({ type: "input_text", text: prompt });
|
||||
} else {
|
||||
msg.content = prompt;
|
||||
}
|
||||
}
|
||||
|
||||
// Claude shape: body.system as string | array of {type:"text", text}
|
||||
// Insert before the last cache_control block to keep injection inside the cached prefix.
|
||||
function injectClaudeSystem(body, prompt) {
|
||||
if (typeof body.system === "string" && body.system.length > 0) {
|
||||
body.system = `${body.system}${SEP}${prompt}`;
|
||||
return;
|
||||
}
|
||||
if (Array.isArray(body.system)) {
|
||||
const block = { type: "text", text: prompt };
|
||||
let lastCacheIdx = -1;
|
||||
for (let i = body.system.length - 1; i >= 0; i--) {
|
||||
if (body.system[i]?.cache_control) { lastCacheIdx = i; break; }
|
||||
}
|
||||
if (lastCacheIdx >= 0) {
|
||||
body.system.splice(lastCacheIdx, 0, block);
|
||||
} else {
|
||||
body.system.push(block);
|
||||
}
|
||||
return;
|
||||
}
|
||||
body.system = prompt;
|
||||
}
|
||||
|
||||
// Gemini shape: body.system_instruction | body.systemInstruction | body.request.systemInstruction
|
||||
// Each shape: { parts: [{ text }] }
|
||||
function injectGeminiSystem(body, prompt) {
|
||||
const target = body.request && typeof body.request === "object" ? body.request : body;
|
||||
const useSnake = Object.prototype.hasOwnProperty.call(target, "system_instruction");
|
||||
const key = useSnake ? "system_instruction" : "systemInstruction";
|
||||
const sys = target[key];
|
||||
if (sys && Array.isArray(sys.parts)) {
|
||||
sys.parts.push({ text: prompt });
|
||||
return;
|
||||
}
|
||||
target[key] = { parts: [{ text: prompt }] };
|
||||
}
|
||||
@@ -136,6 +136,16 @@ export function getTargetFormat(provider) {
|
||||
return config.format || "openai";
|
||||
}
|
||||
|
||||
// Resolve which transport to use for a provider given the client sourceFormat.
|
||||
// Multi-endpoint providers (transport.transports[]) pick the entry matching sourceFormat
|
||||
// to avoid lossy translation; falls back to the default transport when no match.
|
||||
export function resolveTransport(provider, sourceFormat) {
|
||||
const config = PROVIDERS[provider];
|
||||
const transports = config?.transports;
|
||||
if (!Array.isArray(transports) || !transports.length) return null;
|
||||
return transports.find(t => t.format === sourceFormat) || null;
|
||||
}
|
||||
|
||||
// Check if last message is from user
|
||||
export function isLastMessageFromUser(body) {
|
||||
const messages = body.messages || body.contents;
|
||||
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
REACHABLE_MISS_THRESHOLD,
|
||||
CLIENT_PING_FAST_MS,
|
||||
CAVEMAN_LEVELS,
|
||||
PONYTAIL_LEVELS,
|
||||
} from "./endpointConstants";
|
||||
import { clientPingUrl, clientPingAny } from "./endpointPing";
|
||||
import EndpointRow from "./components/EndpointRow";
|
||||
@@ -33,8 +34,17 @@ export default function APIPageClient({ machineId }) {
|
||||
const [hasPassword, setHasPassword] = useState(true);
|
||||
const [tunnelDashboardAccess, setTunnelDashboardAccess] = useState(false);
|
||||
const [rtkEnabled, setRtkEnabledState] = useState(true);
|
||||
const [headroomEnabled, setHeadroomEnabled] = useState(false);
|
||||
const [headroomUrl, setHeadroomUrl] = useState("http://localhost:8787");
|
||||
const [headroomCompressUserMessages, setHeadroomCompressUserMessages] = useState(false);
|
||||
const [headroomStatus, setHeadroomStatus] = useState({ installed: false, running: false, python: null, loading: true });
|
||||
const [showHeadroomInstallModal, setShowHeadroomInstallModal] = useState(false);
|
||||
const [headroomActionLoading, setHeadroomActionLoading] = useState(false);
|
||||
const [headroomActionError, setHeadroomActionError] = useState("");
|
||||
const [cavemanEnabled, setCavemanEnabled] = useState(false);
|
||||
const [cavemanLevel, setCavemanLevel] = useState("full");
|
||||
const [ponytailEnabled, setPonytailEnabled] = useState(false);
|
||||
const [ponytailLevel, setPonytailLevel] = useState("full");
|
||||
const [locale, setLocale] = useState("en");
|
||||
|
||||
// Cloudflare Tunnel state
|
||||
@@ -232,8 +242,14 @@ export default function APIPageClient({ machineId }) {
|
||||
setHasPassword(data.hasPassword || false);
|
||||
setTunnelDashboardAccess(data.tunnelDashboardAccess || false);
|
||||
setRtkEnabledState(data.rtkEnabled !== false);
|
||||
setHeadroomEnabled(!!data.headroomEnabled);
|
||||
setHeadroomUrl(data.headroomUrl || "http://localhost:8787");
|
||||
setHeadroomCompressUserMessages(!!data.headroomCompressUserMessages);
|
||||
refreshHeadroomStatus();
|
||||
setCavemanEnabled(!!data.cavemanEnabled);
|
||||
setCavemanLevel(data.cavemanLevel || "full");
|
||||
setPonytailEnabled(!!data.ponytailEnabled);
|
||||
setPonytailLevel(data.ponytailLevel || "full");
|
||||
}
|
||||
if (statusRes.ok) {
|
||||
const data = await statusRes.json();
|
||||
@@ -313,11 +329,75 @@ export default function APIPageClient({ machineId }) {
|
||||
patchSetting({ cavemanEnabled: value });
|
||||
};
|
||||
|
||||
const handleHeadroomEnabled = (value) => {
|
||||
const nextUrl = headroomUrl.trim() || "http://localhost:8787";
|
||||
setHeadroomUrl(nextUrl);
|
||||
setHeadroomEnabled(value);
|
||||
patchSetting({ headroomEnabled: value, headroomUrl: nextUrl });
|
||||
};
|
||||
|
||||
const handleHeadroomUrlBlur = () => {
|
||||
const next = headroomUrl.trim() || "http://localhost:8787";
|
||||
setHeadroomUrl(next);
|
||||
patchSetting({ headroomUrl: next });
|
||||
};
|
||||
|
||||
const handleHeadroomCompressUserMessages = (value) => {
|
||||
setHeadroomCompressUserMessages(value);
|
||||
patchSetting({ headroomCompressUserMessages: value });
|
||||
};
|
||||
|
||||
const refreshHeadroomStatus = useCallback(async () => {
|
||||
setHeadroomStatus((s) => ({ ...s, loading: true }));
|
||||
try {
|
||||
const res = await fetch("/api/headroom/status", { headers: { "Cache-Control": "no-store" } });
|
||||
const data = await res.json();
|
||||
setHeadroomStatus({ ...data, loading: false });
|
||||
} catch {
|
||||
setHeadroomStatus({ installed: false, running: false, python: null, loading: false });
|
||||
}
|
||||
}, []);
|
||||
|
||||
const handleHeadroomStart = useCallback(async () => {
|
||||
setHeadroomActionError("");
|
||||
setHeadroomActionLoading(true);
|
||||
try {
|
||||
const res = await fetch("/api/headroom/start", { method: "POST" });
|
||||
const data = await res.json().catch(() => ({}));
|
||||
if (!res.ok) throw new Error(data.error || "Failed to start proxy");
|
||||
await refreshHeadroomStatus();
|
||||
} catch (e) {
|
||||
setHeadroomActionError(e.message);
|
||||
} finally {
|
||||
setHeadroomActionLoading(false);
|
||||
}
|
||||
}, [refreshHeadroomStatus]);
|
||||
|
||||
const handleHeadroomStop = useCallback(async () => {
|
||||
setHeadroomActionLoading(true);
|
||||
try {
|
||||
await fetch("/api/headroom/stop", { method: "POST" });
|
||||
await refreshHeadroomStatus();
|
||||
} finally {
|
||||
setHeadroomActionLoading(false);
|
||||
}
|
||||
}, [refreshHeadroomStatus]);
|
||||
|
||||
const handleCavemanLevel = (level) => {
|
||||
setCavemanLevel(level);
|
||||
patchSetting({ cavemanLevel: level });
|
||||
};
|
||||
|
||||
const handlePonytailEnabled = (value) => {
|
||||
setPonytailEnabled(value);
|
||||
patchSetting({ ponytailEnabled: value });
|
||||
};
|
||||
|
||||
const handlePonytailLevel = (level) => {
|
||||
setPonytailLevel(level);
|
||||
patchSetting({ ponytailLevel: level });
|
||||
};
|
||||
|
||||
const fetchData = async () => {
|
||||
try {
|
||||
const keysRes = await fetch("/api/keys");
|
||||
@@ -1043,6 +1123,47 @@ export default function APIPageClient({ machineId }) {
|
||||
onChange={() => handleRtkEnabled(!rtkEnabled)}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex items-center justify-between py-4 border-b border-border gap-4 flex-wrap">
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="flex items-center gap-3 flex-wrap">
|
||||
<p className="font-medium">
|
||||
Compress context{" "}
|
||||
<a
|
||||
href="https://github.com/chopratejas/headroom"
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
className="text-xs font-normal text-primary underline hover:opacity-80"
|
||||
>
|
||||
(Headroom)
|
||||
</a>
|
||||
</p>
|
||||
<span className={`text-xs px-2 py-0.5 rounded ${headroomStatus.installed && headroomStatus.running ? "bg-success/15 text-success" : "bg-warning/15 text-warning"}`}>
|
||||
{headroomStatus.loading
|
||||
? "Checking…"
|
||||
: !headroomStatus.installed
|
||||
? "Not installed"
|
||||
: !headroomStatus.running
|
||||
? "Proxy off"
|
||||
: "Running"}
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setShowHeadroomInstallModal(true)}
|
||||
className="text-xs text-primary underline hover:opacity-80"
|
||||
>
|
||||
{headroomStatus.installed && headroomStatus.running ? "Manage" : "Setup"}
|
||||
</button>
|
||||
</div>
|
||||
<p className="text-sm text-text-muted mt-1">
|
||||
Compress prompts via /v1/compress before routing to the model
|
||||
</p>
|
||||
</div>
|
||||
<Toggle
|
||||
checked={headroomEnabled && headroomStatus.installed && headroomStatus.running}
|
||||
disabled={!headroomStatus.installed || !headroomStatus.running}
|
||||
onChange={() => handleHeadroomEnabled(!headroomEnabled)}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex items-center justify-between pt-4 gap-4 flex-wrap">
|
||||
<div className="min-w-0 flex-1">
|
||||
<p className="font-medium">
|
||||
@@ -1090,6 +1211,53 @@ export default function APIPageClient({ machineId }) {
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center justify-between pt-4 mt-4 border-t border-border gap-4 flex-wrap">
|
||||
<div className="min-w-0 flex-1">
|
||||
<p className="font-medium">
|
||||
Lazy senior dev{" "}
|
||||
<a
|
||||
href="https://github.com/DietrichGebert/ponytail"
|
||||
target="_blank"
|
||||
rel="noreferrer"
|
||||
className="text-xs font-normal text-primary underline hover:opacity-80"
|
||||
>
|
||||
(Ponytail)
|
||||
</a>
|
||||
</p>
|
||||
<p className="text-sm text-text-muted">
|
||||
Bias the model toward minimal code: YAGNI, reuse stdlib, deletion over addition
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-3 shrink-0">
|
||||
{ponytailEnabled && (
|
||||
<div className="flex flex-col items-end gap-1">
|
||||
<div className="flex items-center gap-1.5">
|
||||
{PONYTAIL_LEVELS.map((lvl) => (
|
||||
<button
|
||||
key={lvl.id}
|
||||
onClick={() => handlePonytailLevel(lvl.id)}
|
||||
className={`px-3 py-1.5 rounded text-xs font-medium border transition-colors ${
|
||||
ponytailLevel === lvl.id
|
||||
? "bg-primary text-white border-primary"
|
||||
: "bg-transparent border-border text-text-muted hover:bg-surface-2"
|
||||
}`}
|
||||
title={lvl.desc}
|
||||
>
|
||||
{lvl.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<p className="text-xs text-primary">
|
||||
{PONYTAIL_LEVELS.find((lvl) => lvl.id === ponytailLevel)?.desc}
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
<Toggle
|
||||
checked={ponytailEnabled}
|
||||
onChange={() => handlePonytailEnabled(!ponytailEnabled)}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</Card>
|
||||
|
||||
{/* API Keys */}
|
||||
@@ -1420,6 +1588,58 @@ export default function APIPageClient({ machineId }) {
|
||||
</div>
|
||||
</Modal>
|
||||
|
||||
{/* Headroom Install Guide Modal */}
|
||||
<Modal
|
||||
isOpen={showHeadroomInstallModal}
|
||||
title={headroomStatus.installed ? "Headroom" : "Install Headroom"}
|
||||
onClose={() => setShowHeadroomInstallModal(false)}
|
||||
>
|
||||
<div className="flex flex-col gap-4">
|
||||
<div className="flex items-center justify-between text-sm">
|
||||
<span>Status</span>
|
||||
<span className={headroomStatus.installed && headroomStatus.running ? "text-success" : "text-warning"}>
|
||||
{headroomStatus.loading
|
||||
? "Checking…"
|
||||
: !headroomStatus.installed
|
||||
? "Not installed"
|
||||
: !headroomStatus.running
|
||||
? "Proxy off"
|
||||
: "Running"}
|
||||
</span>
|
||||
</div>
|
||||
{headroomStatus.installed ? (
|
||||
headroomStatus.running ? (
|
||||
<Button onClick={handleHeadroomStop} variant="ghost" fullWidth disabled={headroomActionLoading}>
|
||||
{headroomActionLoading ? "Stopping…" : "Stop Headroom"}
|
||||
</Button>
|
||||
) : (
|
||||
<Button onClick={handleHeadroomStart} fullWidth disabled={headroomActionLoading}>
|
||||
{headroomActionLoading ? "Starting…" : "Start Headroom"}
|
||||
</Button>
|
||||
)
|
||||
) : !headroomStatus.python ? (
|
||||
<p className="text-sm text-warning">Python ≥ 3.10 required. Install Python first.</p>
|
||||
) : (
|
||||
<div className="flex flex-col gap-1">
|
||||
<p className="text-sm font-medium">Install then click Start:</p>
|
||||
<div className="flex items-center gap-2">
|
||||
<pre className="flex-1 rounded bg-black/5 dark:bg-white/5 p-2 text-xs font-mono overflow-x-auto">{`pip install "headroom-ai[proxy]"`}</pre>
|
||||
<Button size="sm" variant="ghost" onClick={() => copy(`pip install "headroom-ai[proxy]"`)}>
|
||||
{copied ? "Copied" : "Copy"}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{headroomActionError && (
|
||||
<p className="text-sm text-warning">{headroomActionError}</p>
|
||||
)}
|
||||
<div className="flex gap-2">
|
||||
<Button onClick={() => refreshHeadroomStatus()} variant="ghost" fullWidth>Recheck</Button>
|
||||
<Button onClick={() => setShowHeadroomInstallModal(false)} fullWidth>Done</Button>
|
||||
</div>
|
||||
</div>
|
||||
</Modal>
|
||||
|
||||
{/* Confirm Modal */}
|
||||
<ConfirmModal
|
||||
isOpen={!!confirmState}
|
||||
|
||||
@@ -24,3 +24,9 @@ export const CAVEMAN_LEVELS = [
|
||||
{ id: "wenyan", label: "文 Full", desc: "Maximum 文言文, 80-90% reduction", wenyan: true },
|
||||
{ id: "wenyan-ultra", label: "文 Ultra", desc: "Extreme classical compression", wenyan: true },
|
||||
];
|
||||
|
||||
export const PONYTAIL_LEVELS = [
|
||||
{ id: "lite", label: "Lite", desc: "Build asked, name lazier option" },
|
||||
{ id: "full", label: "Full", desc: "Ladder enforced: stdlib/native first" },
|
||||
{ id: "ultra", label: "Ultra", desc: "YAGNI extremist, deletion first" },
|
||||
];
|
||||
|
||||
27
src/app/api/headroom/start/route.js
Normal file
27
src/app/api/headroom/start/route.js
Normal file
@@ -0,0 +1,27 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import { getSettings } from "@/lib/localDb";
|
||||
import { startHeadroomProxy } from "@/lib/headroom/process";
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
function parsePortFromUrl(url) {
|
||||
try {
|
||||
const u = new URL(url);
|
||||
const p = parseInt(u.port, 10);
|
||||
if (p > 0 && p < 65536) return p;
|
||||
} catch { /* ignore, fall through to default */ }
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function POST() {
|
||||
try {
|
||||
const settings = await getSettings();
|
||||
const url = settings.headroomUrl || "http://localhost:8787";
|
||||
const port = parsePortFromUrl(url) || 8787;
|
||||
const result = await startHeadroomProxy({ port });
|
||||
return NextResponse.json({ success: true, ...result });
|
||||
} catch (error) {
|
||||
const status = error.code === "NOT_INSTALLED" ? 400 : 500;
|
||||
return NextResponse.json({ error: error.message, code: error.code || null }, { status });
|
||||
}
|
||||
}
|
||||
18
src/app/api/headroom/status/route.js
Normal file
18
src/app/api/headroom/status/route.js
Normal file
@@ -0,0 +1,18 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import { getSettings } from "@/lib/localDb";
|
||||
import { getHeadroomStatus } from "@/lib/headroom/detect";
|
||||
import { getManagedPid } from "@/lib/headroom/process";
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
export async function GET() {
|
||||
try {
|
||||
const settings = await getSettings();
|
||||
const url = settings.headroomUrl || "http://localhost:8787";
|
||||
const status = await getHeadroomStatus(url);
|
||||
const managedPid = getManagedPid();
|
||||
return NextResponse.json({ ...status, url, managedPid });
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: error.message }, { status: 500 });
|
||||
}
|
||||
}
|
||||
14
src/app/api/headroom/stop/route.js
Normal file
14
src/app/api/headroom/stop/route.js
Normal file
@@ -0,0 +1,14 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import { stopHeadroomProxy } from "@/lib/headroom/process";
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
export async function POST() {
|
||||
try {
|
||||
const result = stopHeadroomProxy();
|
||||
const status = result.stopped ? 200 : 409;
|
||||
return NextResponse.json({ ...result }, { status });
|
||||
} catch (error) {
|
||||
return NextResponse.json({ error: error.message, code: error.code || null }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -79,6 +79,8 @@ const LOCAL_ONLY_PATHS = [
|
||||
"/api/oauth/cursor/auto-import",
|
||||
"/api/oauth/kiro/auto-import",
|
||||
"/api/auth/reset-password",
|
||||
"/api/headroom/start",
|
||||
"/api/headroom/stop",
|
||||
];
|
||||
|
||||
const LOOPBACK_HOSTS = new Set(["localhost", "127.0.0.1", "::1"]);
|
||||
|
||||
@@ -34,8 +34,13 @@ const DEFAULT_SETTINGS = {
|
||||
mitmRouterBaseUrl: DEFAULT_MITM_ROUTER_BASE,
|
||||
dnsToolEnabled: {},
|
||||
rtkEnabled: true,
|
||||
headroomEnabled: false,
|
||||
headroomUrl: "http://localhost:8787",
|
||||
headroomCompressUserMessages: false,
|
||||
cavemanEnabled: false,
|
||||
cavemanLevel: "full",
|
||||
ponytailEnabled: false,
|
||||
ponytailLevel: "full",
|
||||
};
|
||||
|
||||
async function readRaw() {
|
||||
|
||||
63
src/lib/headroom/detect.js
Normal file
63
src/lib/headroom/detect.js
Normal file
@@ -0,0 +1,63 @@
|
||||
import { execSync } from "child_process";
|
||||
|
||||
const EXTENDED_PATH = `/usr/local/bin:/opt/homebrew/bin:/usr/bin:/bin:${process.env.PATH || ""}`;
|
||||
const PYTHON_CANDIDATES = ["python3.13", "python3.12", "python3.11", "python3.10", "python3"];
|
||||
const MIN_VERSION = [3, 10];
|
||||
const HEADROOM_HEALTH_TIMEOUT_MS = 1500;
|
||||
|
||||
// Detect whether the headroom CLI is installed and where its binary lives.
|
||||
export function findHeadroomBinary() {
|
||||
try {
|
||||
const path = execSync("which headroom", {
|
||||
stdio: ["ignore", "pipe", "ignore"],
|
||||
windowsHide: true,
|
||||
env: { ...process.env, PATH: EXTENDED_PATH },
|
||||
}).toString().trim();
|
||||
return path || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// Find a Python interpreter >= 3.10 (headroom-ai requires it). Returns null if none.
|
||||
export function findPython310() {
|
||||
for (const candidate of PYTHON_CANDIDATES) {
|
||||
try {
|
||||
const ver = execSync(`${candidate} --version`, {
|
||||
stdio: ["ignore", "pipe", "ignore"],
|
||||
windowsHide: true,
|
||||
env: { ...process.env, PATH: EXTENDED_PATH },
|
||||
}).toString().trim();
|
||||
const match = ver.match(/(\d+)\.(\d+)/);
|
||||
if (!match) continue;
|
||||
const [major, minor] = [parseInt(match[1], 10), parseInt(match[2], 10)];
|
||||
if (major > MIN_VERSION[0] || (major === MIN_VERSION[0] && minor >= MIN_VERSION[1])) {
|
||||
return candidate;
|
||||
}
|
||||
} catch {
|
||||
// candidate not present, try next
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Probe whether a Headroom proxy is reachable at the given URL by hitting /health.
|
||||
export async function probeProxyRunning(url) {
|
||||
if (!url) return false;
|
||||
const base = String(url).replace(/\/$/, "");
|
||||
try {
|
||||
const res = await fetch(`${base}/health`, { signal: AbortSignal.timeout(HEADROOM_HEALTH_TIMEOUT_MS) });
|
||||
return res.ok;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// Aggregate status for the dashboard: installed, running, python interpreter.
|
||||
export async function getHeadroomStatus(url) {
|
||||
const path = findHeadroomBinary();
|
||||
const python = findPython310();
|
||||
const installed = Boolean(path);
|
||||
const running = installed ? await probeProxyRunning(url) : false;
|
||||
return { installed, path, running, python };
|
||||
}
|
||||
128
src/lib/headroom/process.js
Normal file
128
src/lib/headroom/process.js
Normal file
@@ -0,0 +1,128 @@
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
import { spawn } from "child_process";
|
||||
import { DATA_DIR } from "@/lib/dataDir.js";
|
||||
import { findHeadroomBinary } from "./detect.js";
|
||||
|
||||
const HEADROOM_DIR = path.join(DATA_DIR, "headroom");
|
||||
const PID_FILE = path.join(HEADROOM_DIR, "proxy.pid");
|
||||
const LOG_FILE = path.join(HEADROOM_DIR, "proxy.log");
|
||||
const DEFAULT_PORT = 8787;
|
||||
const STARTUP_TIMEOUT_MS = 8000;
|
||||
|
||||
function ensureDir() {
|
||||
if (!fs.existsSync(HEADROOM_DIR)) fs.mkdirSync(HEADROOM_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
function readPid() {
|
||||
try {
|
||||
if (fs.existsSync(PID_FILE)) return parseInt(fs.readFileSync(PID_FILE, "utf8"), 10);
|
||||
} catch { /* ignore */ }
|
||||
return null;
|
||||
}
|
||||
|
||||
function writePid(pid) {
|
||||
ensureDir();
|
||||
fs.writeFileSync(PID_FILE, String(pid));
|
||||
}
|
||||
|
||||
function clearPid() {
|
||||
try { if (fs.existsSync(PID_FILE)) fs.unlinkSync(PID_FILE); } catch { /* ignore */ }
|
||||
}
|
||||
|
||||
// process.kill throws if pid is dead — use this to probe.
|
||||
export function isPidAlive(pid) {
|
||||
if (!pid || typeof pid !== "number") return false;
|
||||
try { process.kill(pid, 0); return true; } catch { return false; }
|
||||
}
|
||||
|
||||
export function getManagedPid() {
|
||||
const pid = readPid();
|
||||
return pid && isPidAlive(pid) ? pid : null;
|
||||
}
|
||||
|
||||
export async function startHeadroomProxy({ port = DEFAULT_PORT } = {}) {
|
||||
const safePort = Number(port) > 0 && Number(port) < 65536 ? Number(port) : DEFAULT_PORT;
|
||||
const binary = findHeadroomBinary();
|
||||
if (!binary) {
|
||||
const err = new Error("Headroom CLI not installed");
|
||||
err.code = "NOT_INSTALLED";
|
||||
throw err;
|
||||
}
|
||||
|
||||
const existing = getManagedPid();
|
||||
if (existing) return { pid: existing, alreadyRunning: true };
|
||||
|
||||
ensureDir();
|
||||
// spawn stdio requires fd numbers, not WriteStream objects.
|
||||
const outFd = fs.openSync(LOG_FILE, "a");
|
||||
|
||||
const child = spawn(binary, ["proxy", "--port", String(safePort)], {
|
||||
stdio: ["ignore", outFd, outFd],
|
||||
detached: true,
|
||||
windowsHide: true,
|
||||
env: { ...process.env },
|
||||
});
|
||||
|
||||
if (!child.pid) {
|
||||
fs.closeSync(outFd);
|
||||
const err = new Error("Failed to spawn headroom proxy");
|
||||
err.code = "SPAWN_FAILED";
|
||||
throw err;
|
||||
}
|
||||
|
||||
child.unref();
|
||||
writePid(child.pid);
|
||||
|
||||
// Wait until the process either stays alive briefly (success) or exits fast (failure).
|
||||
await new Promise((resolve, reject) => {
|
||||
const startupTimer = setTimeout(() => {
|
||||
if (isPidAlive(child.pid)) resolve();
|
||||
else reject(new Error("headroom proxy exited during startup — see proxy.log"));
|
||||
}, STARTUP_TIMEOUT_MS);
|
||||
|
||||
child.once("exit", (code) => {
|
||||
clearTimeout(startupTimer);
|
||||
clearPid();
|
||||
fs.closeSync(outFd);
|
||||
const e = new Error(`headroom proxy exited early (code=${code}) — see proxy.log`);
|
||||
e.code = "EARLY_EXIT";
|
||||
reject(e);
|
||||
});
|
||||
});
|
||||
|
||||
// Close parent's copy of the fd; child retains its own after unref.
|
||||
fs.closeSync(outFd);
|
||||
|
||||
return { pid: child.pid, alreadyRunning: false };
|
||||
}
|
||||
|
||||
export function stopHeadroomProxy() {
|
||||
const pid = getManagedPid();
|
||||
if (!pid) return { stopped: false, reason: "not_running" };
|
||||
try {
|
||||
process.kill(pid, "SIGTERM");
|
||||
// Give it a moment, then force if still alive.
|
||||
setTimeout(() => {
|
||||
if (isPidAlive(pid)) {
|
||||
try { process.kill(pid, "SIGKILL"); } catch { /* already gone */ }
|
||||
}
|
||||
}, 2000);
|
||||
clearPid();
|
||||
return { stopped: true, pid };
|
||||
} catch (e) {
|
||||
clearPid();
|
||||
const err = new Error(`Failed to stop headroom proxy: ${e.message}`);
|
||||
err.code = "STOP_FAILED";
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
export function getHeadroomLogTail(maxLines = 200) {
|
||||
try {
|
||||
if (!fs.existsSync(LOG_FILE)) return "";
|
||||
const content = fs.readFileSync(LOG_FILE, "utf8");
|
||||
const lines = content.split(/\r?\n/).filter(Boolean);
|
||||
return lines.slice(-maxLines).join("\n");
|
||||
} catch { return ""; }
|
||||
}
|
||||
@@ -251,8 +251,13 @@ async function handleSingleModelChat(body, modelStr, clientRawRequest = null, re
|
||||
apiKey,
|
||||
ccFilterNaming: !!chatSettings.ccFilterNaming,
|
||||
rtkEnabled: !!chatSettings.rtkEnabled,
|
||||
headroomEnabled: !!chatSettings.headroomEnabled,
|
||||
headroomUrl: chatSettings.headroomUrl || "http://localhost:8787",
|
||||
headroomCompressUserMessages: !!chatSettings.headroomCompressUserMessages,
|
||||
cavemanEnabled: !!chatSettings.cavemanEnabled,
|
||||
cavemanLevel: chatSettings.cavemanLevel || "full",
|
||||
ponytailEnabled: !!chatSettings.ponytailEnabled,
|
||||
ponytailLevel: chatSettings.ponytailLevel || "full",
|
||||
providerThinking,
|
||||
// Detect source format by endpoint + body
|
||||
sourceFormatOverride: request?.url ? detectFormatByEndpoint(new URL(request.url).pathname, body) : null,
|
||||
|
||||
73
tests/unit/headroom.test.js
Normal file
73
tests/unit/headroom.test.js
Normal file
@@ -0,0 +1,73 @@
|
||||
import { describe, it, expect, vi, afterEach } from "vitest";
|
||||
import { compressWithHeadroom, formatHeadroomLog } from "../../open-sse/rtk/headroom.js";
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
describe("compressWithHeadroom", () => {
|
||||
it("no-ops when disabled", async () => {
|
||||
global.fetch = vi.fn();
|
||||
const body = { messages: [{ role: "user", content: "hello" }] };
|
||||
|
||||
const stats = await compressWithHeadroom(body, { enabled: false, url: "http://localhost:8787" });
|
||||
|
||||
expect(stats).toBeNull();
|
||||
expect(global.fetch).not.toHaveBeenCalled();
|
||||
expect(body.messages[0].content).toBe("hello");
|
||||
});
|
||||
|
||||
it("compresses messages in-place", async () => {
|
||||
global.fetch = vi.fn(async () => new Response(JSON.stringify({
|
||||
messages: [{ role: "user", content: "short" }],
|
||||
tokens_before: 100,
|
||||
tokens_after: 20,
|
||||
tokens_saved: 80,
|
||||
}), { status: 200 }));
|
||||
const body = { messages: [{ role: "user", content: "long" }] };
|
||||
|
||||
const stats = await compressWithHeadroom(body, { enabled: true, url: "http://headroom:8787/", model: "gpt-4o" });
|
||||
|
||||
expect(body.messages[0].content).toBe("short");
|
||||
expect(stats.tokens_saved).toBe(80);
|
||||
expect(global.fetch).toHaveBeenCalledWith("http://headroom:8787/v1/compress", expect.objectContaining({ method: "POST" }));
|
||||
});
|
||||
|
||||
it("compresses responses input in-place", async () => {
|
||||
global.fetch = vi.fn(async () => new Response(JSON.stringify({
|
||||
messages: [{ role: "user", content: "short" }],
|
||||
}), { status: 200 }));
|
||||
const body = { input: [{ role: "user", content: "long" }] };
|
||||
|
||||
await compressWithHeadroom(body, { enabled: true, url: "http://localhost:8787" });
|
||||
|
||||
expect(body.input[0].content).toBe("short");
|
||||
});
|
||||
|
||||
it("fails open on bad response", async () => {
|
||||
global.fetch = vi.fn(async () => new Response(JSON.stringify({ error: "bad" }), { status: 500 }));
|
||||
const body = { messages: [{ role: "user", content: "long" }] };
|
||||
|
||||
const stats = await compressWithHeadroom(body, { enabled: true, url: "http://localhost:8787" });
|
||||
|
||||
expect(stats).toBeNull();
|
||||
expect(body.messages[0].content).toBe("long");
|
||||
});
|
||||
|
||||
it("skips unknown shapes", async () => {
|
||||
global.fetch = vi.fn();
|
||||
const body = { contents: [{ parts: [{ text: "long" }] }] };
|
||||
|
||||
const stats = await compressWithHeadroom(body, { enabled: true, url: "http://localhost:8787" });
|
||||
|
||||
expect(stats).toBeNull();
|
||||
expect(global.fetch).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("formatHeadroomLog", () => {
|
||||
it("formats savings", () => {
|
||||
expect(formatHeadroomLog({ tokens_before: 100, tokens_after: 25, tokens_saved: 75 }))
|
||||
.toBe("saved 75 tokens / 100 (75.0%) after=25");
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user