snower 993c6eb469 feat(headroom): make the compression request timeout configurable ...

The 3000 ms timeout on /v1/compress was fixed, so busy or slow machines
timed out often and sent the LLM an inconsistently compressed body,
hurting prompt caching. Add a headroomTimeoutMs setting, thread it from
the chat handler down to compressWithHeadroom, expose it in the Token
Saver dashboard, and normalize invalid values back to the 3000 ms default.

2026-08-28 16:34:33 +07:00

11 KiB

Raw Permalink Blame History

View Raw