Merge remote-tracking branch 'origin/master' into gitea/feature/end
Resolved conflicts taking origin/master (v0.5.55) as canonical, with local features re-applied: - runtime log level (LOG_LEVEL env + dashboard Settings → Logging, applied immediately and persisted across restarts) - free/noAuth provider enable/disable toggle via providerStrategies.enabled - parallel model testing (Test All Models / Test Selected Keys)
This commit is contained in:
BIN
docs/images/saml-admin-dashboard.png
Normal file
BIN
docs/images/saml-admin-dashboard.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 103 KiB |
BIN
docs/images/saml-login-screen.png
Normal file
BIN
docs/images/saml-login-screen.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
@@ -0,0 +1,328 @@
|
||||
# GPT-5.6 Codex Reasoning Overrides Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Preserve Codex-advertised Max and Ultra overrides for GPT-5.6 Sol and Terra, preserve Max for Luna, and convert Luna Ultra to Max without changing Kiro or generic OpenAI-format behavior.
|
||||
|
||||
**Architecture:** Keep the supported reasoning matrix in the existing `getThinkingLevels(provider, model)` resolver and reuse that result in both translation and Codex executor normalization. The dashboard already consumes this resolver, so no UI component change is required. Unsupported top-end levels remain safely normalized, with Luna Ultra selecting Luna's supported Max level.
|
||||
|
||||
**Tech Stack:** JavaScript ES modules, Next.js, Vitest, Codex Responses transport.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Apply the new overrides only to the OpenAI Codex provider (`codex`, exposed as `cx/`).
|
||||
- Sol and Terra support `max` and `ultra`; Luna supports `max` but not `ultra`.
|
||||
- Convert Luna `ultra` requests to `max` in both translated and native passthrough request paths.
|
||||
- Preserve existing Kiro and generic OpenAI-compatible normalization.
|
||||
- Do not add runtime model-catalog fetching, dependencies, pricing changes, or unrelated refactors.
|
||||
- Write each behavior test first and observe the expected failure before changing production code.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Provider-scoped GPT-5.6 level matrix
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/unit/thinking-levels-gpt56-sol.test.js`
|
||||
- Modify: `open-sse/providers/thinkingLevels.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `getThinkingLevels(provider, model)` and existing capability metadata.
|
||||
- Produces: `getThinkingLevels(provider, model): string[] | null` with Codex-only GPT-5.6 level overrides.
|
||||
|
||||
- [ ] **Step 1: Replace the Sol-only assertions with the complete behavior matrix**
|
||||
|
||||
Use literal expected arrays so each model/provider contract is independently checked:
|
||||
|
||||
```js
|
||||
it.each([
|
||||
["gpt-5.6-sol", ["none", "minimal", "low", "medium", "high", "xhigh", "max", "ultra"]],
|
||||
["gpt-5.6-terra", ["none", "minimal", "low", "medium", "high", "xhigh", "max", "ultra"]],
|
||||
["gpt-5.6-luna", ["none", "minimal", "low", "medium", "high", "xhigh", "max"]],
|
||||
["gpt-5.6-sol-review", ["none", "minimal", "low", "medium", "high", "xhigh", "max", "ultra"]],
|
||||
["gpt-5.6-terra-review", ["none", "minimal", "low", "medium", "high", "xhigh", "max", "ultra"]],
|
||||
["gpt-5.6-luna-review", ["none", "minimal", "low", "medium", "high", "xhigh", "max"]],
|
||||
])("returns Codex levels for %s", (model, expected) => {
|
||||
expect(getThinkingLevels("codex", model)).toEqual(expected);
|
||||
});
|
||||
|
||||
it("does not expose Codex-only GPT-5.6 overrides on Kiro", () => {
|
||||
expect(getThinkingLevels("kiro", "gpt-5.6-sol")).toEqual([
|
||||
"none", "minimal", "low", "medium", "high", "xhigh",
|
||||
]);
|
||||
});
|
||||
```
|
||||
|
||||
Keep the older Codex-model assertion to protect the existing `gpt-5.3-codex` behavior.
|
||||
|
||||
- [ ] **Step 2: Run the level test and verify it fails for the missing matrix/provider scoping**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npx vitest run tests/unit/thinking-levels-gpt56-sol.test.js
|
||||
```
|
||||
|
||||
Expected: FAIL because Sol lacks Ultra, Terra/Luna lack Max, and Kiro currently inherits Sol Max.
|
||||
|
||||
- [ ] **Step 3: Add provider-aware pattern matching and the three Codex model rules**
|
||||
|
||||
Update `PATTERN_THINKING` entries to accept an optional `provider` field and match it in `getThinkingLevels`:
|
||||
|
||||
```js
|
||||
const CODEX_GPT_5_6_LEVELS = ["none", "minimal", "low", "medium", "high", "xhigh", "max"];
|
||||
|
||||
const PATTERN_THINKING = [
|
||||
{ provider: "codex", pattern: "*gpt-5.6-sol*", levels: [...CODEX_GPT_5_6_LEVELS, "ultra"] },
|
||||
{ provider: "codex", pattern: "*gpt-5.6-terra*", levels: [...CODEX_GPT_5_6_LEVELS, "ultra"] },
|
||||
{ provider: "codex", pattern: "*gpt-5.6-luna*", levels: CODEX_GPT_5_6_LEVELS },
|
||||
{ pattern: "*codex*", levels: ["low", "medium", "high", "xhigh"] },
|
||||
];
|
||||
|
||||
const hit = PATTERN_THINKING.find((entry) =>
|
||||
(!entry.provider || entry.provider === provider) && matchPattern(entry.pattern, model)
|
||||
);
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Re-run the level test and verify it passes**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npx vitest run tests/unit/thinking-levels-gpt56-sol.test.js
|
||||
```
|
||||
|
||||
Expected: 1 test file passed with no failures.
|
||||
|
||||
- [ ] **Step 5: Commit the capability matrix**
|
||||
|
||||
```bash
|
||||
git add open-sse/providers/thinkingLevels.js tests/unit/thinking-levels-gpt56-sol.test.js
|
||||
git commit -m "feat(codex): expose GPT-5.6 reasoning overrides"
|
||||
```
|
||||
|
||||
### Task 2: Model-aware shared thinking translation
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/translator/thinking-unified.test.js`
|
||||
- Modify: `open-sse/translator/concerns/thinkingUnified.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `getThinkingLevels(provider, cleanModel): string[] | null` from Task 1.
|
||||
- Produces: `parseSuffix(model)` support for `ultra` and `applyThinking(...)` output that preserves supported Codex levels.
|
||||
|
||||
- [ ] **Step 1: Add failing suffix and translation tests**
|
||||
|
||||
Add a literal parser assertion:
|
||||
|
||||
```js
|
||||
expect(parseSuffix("gpt-5.6-sol(ultra)")).toEqual({
|
||||
cleanModel: "gpt-5.6-sol",
|
||||
override: { mode: "level", level: "ultra" },
|
||||
});
|
||||
```
|
||||
|
||||
Add table-driven Codex assertions using direct request fields:
|
||||
|
||||
```js
|
||||
it.each([
|
||||
["gpt-5.6-sol", "max", "max"],
|
||||
["gpt-5.6-sol", "ultra", "ultra"],
|
||||
["gpt-5.6-terra", "max", "max"],
|
||||
["gpt-5.6-terra", "ultra", "ultra"],
|
||||
["gpt-5.6-luna", "max", "max"],
|
||||
["gpt-5.6-luna", "ultra", "max"],
|
||||
])("normalizes Codex %s effort %s to %s", (model, effort, expected) => {
|
||||
const out = apply("openai-responses", model, { reasoning: { effort } }, "codex");
|
||||
expect(out.reasoning_effort).toBe(expected);
|
||||
});
|
||||
```
|
||||
|
||||
Add a parenthesized override assertion and Kiro isolation assertion:
|
||||
|
||||
```js
|
||||
expect(apply("openai-responses", "gpt-5.6-sol(ultra)", {}, "codex").reasoning_effort).toBe("ultra");
|
||||
expect(apply("openai", "gpt-5.6-sol", { reasoning_effort: "max" }, "kiro").reasoning_effort).toBe("xhigh");
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the translator test and verify it fails for Ultra parsing and preserved Max/Ultra**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npx vitest run tests/translator/thinking-unified.test.js
|
||||
```
|
||||
|
||||
Expected: FAIL because Ultra suffixes are ignored and OpenAI translation clamps Max to XHigh.
|
||||
|
||||
- [ ] **Step 3: Implement supported-level normalization in the shared translator**
|
||||
|
||||
Import `getThinkingLevels`. Recognize `ultra` explicitly in `parseSuffix` without adding it to the budget map. Resolve supported levels once in `applyThinking` and pass them to `applyFormat`.
|
||||
|
||||
Use this normalization rule for the OpenAI format:
|
||||
|
||||
```js
|
||||
function normalizeOpenAILevel(level, supportedLevels) {
|
||||
if (level !== "max" && level !== "ultra") return level;
|
||||
if (supportedLevels?.includes(level)) return level;
|
||||
if (level === "ultra" && supportedLevels?.includes("max")) return "max";
|
||||
return "xhigh";
|
||||
}
|
||||
```
|
||||
|
||||
Keep `none`, automatic effort, budget conversion, and every non-OpenAI format unchanged.
|
||||
|
||||
- [ ] **Step 4: Re-run the translator and generic OpenAI clamp tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npx vitest run tests/translator/thinking-unified.test.js tests/unit/thinking-effort-openai-max-clamp.test.js
|
||||
```
|
||||
|
||||
Expected: 2 test files passed; generic OpenAI Max still becomes XHigh.
|
||||
|
||||
- [ ] **Step 5: Commit shared translation support**
|
||||
|
||||
```bash
|
||||
git add open-sse/translator/concerns/thinkingUnified.js tests/translator/thinking-unified.test.js
|
||||
git commit -m "feat(codex): preserve supported reasoning efforts"
|
||||
```
|
||||
|
||||
### Task 3: Codex native passthrough normalization
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/unit/codex-fast-capacity.test.js`
|
||||
- Modify: `open-sse/executors/codex.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `getThinkingLevels("codex", upstreamModel): string[] | null` from Task 1.
|
||||
- Produces: `CodexExecutor.transformRequest(...)` payloads with model-supported upstream `reasoning.effort` values.
|
||||
|
||||
- [ ] **Step 1: Add failing Codex executor behavior tests**
|
||||
|
||||
Add a separate `describe("Codex reasoning normalization", ...)` block with real `transformRequest` calls:
|
||||
|
||||
```js
|
||||
it.each([
|
||||
["gpt-5.6-sol", "max", "max"],
|
||||
["gpt-5.6-sol", "ultra", "ultra"],
|
||||
["gpt-5.6-terra", "max", "max"],
|
||||
["gpt-5.6-terra", "ultra", "ultra"],
|
||||
["gpt-5.6-luna", "max", "max"],
|
||||
["gpt-5.6-luna", "ultra", "max"],
|
||||
])("normalizes %s effort %s to %s", (model, effort, expected) => {
|
||||
const body = new CodexExecutor().transformRequest(model, {
|
||||
model,
|
||||
input: "hi",
|
||||
reasoning: { effort },
|
||||
}, true, {});
|
||||
expect(body.reasoning.effort).toBe(expected);
|
||||
});
|
||||
|
||||
it("resolves review models before applying the reasoning matrix", () => {
|
||||
const body = new CodexExecutor().transformRequest("gpt-5.6-terra-review", {
|
||||
model: "gpt-5.6-terra-review",
|
||||
input: "hi",
|
||||
reasoning_effort: "ultra",
|
||||
}, true, {});
|
||||
expect(body.model).toBe("gpt-5.6-terra");
|
||||
expect(body.reasoning.effort).toBe("ultra");
|
||||
});
|
||||
```
|
||||
|
||||
Keep the existing GPT-5.5 Max-to-XHigh fast-tier test.
|
||||
|
||||
- [ ] **Step 2: Run the executor test and verify supported values fail by being clamped**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npx vitest run tests/unit/codex-fast-capacity.test.js
|
||||
```
|
||||
|
||||
Expected: FAIL because current normalization maps supported Max to XHigh and does not map Luna Ultra to Max.
|
||||
|
||||
- [ ] **Step 3: Make Codex normalization model-aware**
|
||||
|
||||
Import `getThinkingLevels` and replace the global Max clamp with:
|
||||
|
||||
```js
|
||||
function normalizeReasoningEffort(model, value) {
|
||||
const supportedLevels = getThinkingLevels("codex", model);
|
||||
if (supportedLevels?.includes(value)) return value;
|
||||
if (value === "ultra" && supportedLevels?.includes("max")) return "max";
|
||||
if (value === "max" || value === "ultra") return "xhigh";
|
||||
return value;
|
||||
}
|
||||
```
|
||||
|
||||
Call it only after `body.model` has resolved review aliases to their upstream base model. Pass `body.model` for both `reasoning_effort` and existing `reasoning.effort` request shapes.
|
||||
|
||||
- [ ] **Step 4: Re-run the executor and focused feature suites**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npx vitest run tests/unit/codex-fast-capacity.test.js tests/unit/thinking-levels-gpt56-sol.test.js tests/translator/thinking-unified.test.js tests/unit/thinking-effort-openai-max-clamp.test.js
|
||||
```
|
||||
|
||||
Expected: 4 test files passed with no failures.
|
||||
|
||||
- [ ] **Step 5: Commit native Codex normalization**
|
||||
|
||||
```bash
|
||||
git add open-sse/executors/codex.js tests/unit/codex-fast-capacity.test.js
|
||||
git commit -m "feat(codex): forward GPT-5.6 max and ultra efforts"
|
||||
```
|
||||
|
||||
### Task 4: Full verification and pull request
|
||||
|
||||
**Files:**
|
||||
- Verify all changed production, test, design, and plan files.
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: completed Tasks 1-3.
|
||||
- Produces: verified branch pushed to `origin` and a pull request targeting `decolua/9router:master`.
|
||||
|
||||
- [ ] **Step 1: Run all focused regression tests**
|
||||
|
||||
```bash
|
||||
npx vitest run tests/unit/thinking-levels-gpt56-sol.test.js tests/translator/thinking-unified.test.js tests/unit/thinking-effort-openai-max-clamp.test.js tests/unit/codex-fast-capacity.test.js
|
||||
```
|
||||
|
||||
Expected: all selected test files and tests pass.
|
||||
|
||||
- [ ] **Step 2: Run the complete unit test suite**
|
||||
|
||||
```bash
|
||||
npx vitest run tests/unit tests/translator
|
||||
```
|
||||
|
||||
Expected: all test files pass with zero failed tests.
|
||||
|
||||
- [ ] **Step 3: Run the production build**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
Expected: Next.js production build exits with status 0.
|
||||
|
||||
- [ ] **Step 4: Verify repository hygiene and requirement coverage**
|
||||
|
||||
```bash
|
||||
git diff --check upstream/master...HEAD
|
||||
git status --short --branch
|
||||
git log --oneline upstream/master..HEAD
|
||||
```
|
||||
|
||||
Expected: no whitespace errors, no uncommitted source changes, and only scoped feature commits.
|
||||
|
||||
- [ ] **Step 5: Push the feature branch and open the pull request**
|
||||
|
||||
```bash
|
||||
git push -u origin codex/gpt-5-6-reasoning-overrides
|
||||
gh pr create --repo decolua/9router --base master --head seakleangnhak:codex/gpt-5-6-reasoning-overrides --title "feat(codex): support GPT-5.6 Max and Ultra overrides" --body $'## Summary\n- expose Max and Ultra for Codex GPT-5.6 Sol and Terra\n- expose Max for Codex GPT-5.6 Luna and normalize Luna Ultra to Max\n- keep Kiro and generic OpenAI-compatible reasoning behavior unchanged\n\n## Verification\n- `npx vitest run tests/unit tests/translator`\n- `npm run build`'
|
||||
```
|
||||
|
||||
The pull request body must summarize the Codex-only support matrix, Luna Ultra-to-Max fallback, Kiro isolation, and fresh test/build evidence.
|
||||
@@ -0,0 +1,122 @@
|
||||
# GPT-5.6 Codex Reasoning Overrides Design
|
||||
|
||||
## Goal
|
||||
|
||||
Expose and preserve the reasoning levels currently advertised by the OpenAI
|
||||
Codex model catalog for GPT-5.6 Sol, Terra, and Luna when they are routed
|
||||
through the `codex` provider (`cx/`).
|
||||
|
||||
The supported override matrix is:
|
||||
|
||||
| Model family | Max | Ultra |
|
||||
| --- | --- | --- |
|
||||
| GPT-5.6 Sol | Yes | Yes |
|
||||
| GPT-5.6 Terra | Yes | Yes |
|
||||
| GPT-5.6 Luna | Yes | No |
|
||||
|
||||
The same matrix applies to 9router's virtual `-review` variants because they
|
||||
resolve to the corresponding upstream base model.
|
||||
|
||||
## Scope
|
||||
|
||||
This change is limited to OpenAI Codex (`cx/`) routes. Kiro (`kr/`) and other
|
||||
OpenAI-format providers retain their existing reasoning-level behavior even
|
||||
when they expose models with the same GPT-5.6 names.
|
||||
|
||||
The change covers the complete local request path:
|
||||
|
||||
1. The provider page advertises only the levels supported by each Codex model.
|
||||
2. A copied model suffix such as `gpt-5.6-sol(ultra)` is parsed as a reasoning
|
||||
override.
|
||||
3. The shared thinking translator preserves a supported Codex override while
|
||||
retaining the existing `xhigh` fallback for unsupported OpenAI levels.
|
||||
4. The Codex executor sends supported `max` and `ultra` values unchanged to the
|
||||
upstream Codex Responses endpoint.
|
||||
|
||||
## Current Behavior
|
||||
|
||||
`gpt-5.6-luna` and the other GPT-5.6 models already exist in the Codex model
|
||||
registry. The capability picker has a global Sol-only `max` pattern, which also
|
||||
affects providers such as Kiro unintentionally. The shared OpenAI translator
|
||||
and Codex executor then convert `max` to `xhigh`, so the advertised override is
|
||||
not preserved end to end. `ultra` is not recognized as a model suffix.
|
||||
|
||||
## Design
|
||||
|
||||
### Provider-scoped level resolution
|
||||
|
||||
Extend the existing model-pattern overrides in
|
||||
`open-sse/providers/thinkingLevels.js` with an optional provider constraint.
|
||||
Add three Codex-only GPT-5.6 patterns in most-specific order:
|
||||
|
||||
- Sol: existing levels plus `max` and `ultra`.
|
||||
- Terra: existing levels plus `max` and `ultra`.
|
||||
- Luna: existing levels plus `max`.
|
||||
|
||||
Matching remains wildcard-based so virtual `-review` variants inherit the
|
||||
base model's levels. Provider matching prevents these overrides from changing
|
||||
Kiro or other providers.
|
||||
|
||||
### Shared translation
|
||||
|
||||
Teach the suffix parser to recognize `ultra` as a discrete level without
|
||||
assigning it a synthetic token budget. When applying the OpenAI wire format,
|
||||
reuse the resolved per-provider model levels:
|
||||
|
||||
- Preserve `max` or `ultra` when the target provider/model explicitly supports
|
||||
the requested level.
|
||||
- Convert `ultra` to `max` for GPT-5.6 Luna, preserving the highest level Luna
|
||||
supports.
|
||||
- Convert other unsupported `max` or `ultra` requests to `xhigh`, preserving
|
||||
the existing safe fallback for generic OpenAI-compatible providers.
|
||||
- Leave all existing lower levels and `none` handling unchanged.
|
||||
|
||||
This keeps one capability source for the dashboard and translation behavior
|
||||
instead of duplicating the GPT-5.6 matrix.
|
||||
|
||||
### Codex executor
|
||||
|
||||
Make Codex reasoning normalization model-aware. After virtual review models
|
||||
are resolved to their upstream base model, preserve a requested level when
|
||||
the Codex capability resolver lists it. Continue converting unsupported
|
||||
`max` or `ultra` values to `xhigh`, except that Luna converts `ultra` to its
|
||||
supported `max` level.
|
||||
|
||||
Do not add `max` to the executor's legacy hyphen-suffix parser because
|
||||
`gpt-5.1-codex-max` is an actual model identifier. Dashboard overrides use the
|
||||
existing parenthesized suffix and the shared translator removes that suffix
|
||||
before executor dispatch.
|
||||
|
||||
## Error and Compatibility Behavior
|
||||
|
||||
- `cx/gpt-5.6-luna(ultra)` becomes `max` rather than sending an unsupported
|
||||
level upstream.
|
||||
- Non-GPT-5.6 Codex models retain their current supported levels and fallback
|
||||
behavior.
|
||||
- Kiro GPT-5.6 routes no longer inherit the Codex Sol-only picker override and
|
||||
continue using Kiro's existing effort normalization.
|
||||
- Direct request fields and parenthesized model overrides follow the same
|
||||
model-aware rules.
|
||||
|
||||
## Testing
|
||||
|
||||
Use test-driven development with focused unit coverage:
|
||||
|
||||
1. Level resolver tests for Sol, Terra, Luna, their review variants, an older
|
||||
Codex model, and Kiro isolation.
|
||||
2. Shared translator tests proving `max` and `ultra` survive only for supported
|
||||
Codex model/provider combinations, Luna `ultra` becomes `max`, and other
|
||||
unsupported combinations become `xhigh`.
|
||||
3. Codex executor tests proving native and translated request shapes preserve
|
||||
supported values after upstream model resolution.
|
||||
4. Existing thinking translation and Codex executor suites to guard generic
|
||||
OpenAI clamping and fast-tier behavior.
|
||||
5. Project lint/build checks in proportion to the changed JavaScript modules.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Runtime fetching or caching of the Codex model catalog.
|
||||
- Adding these levels to Kiro or another provider.
|
||||
- Changing model pricing, quotas, defaults, or service tiers.
|
||||
- Adding Codex Ultra's multi-agent orchestration behavior inside 9router;
|
||||
9router only forwards the catalog-advertised reasoning override.
|
||||
Reference in New Issue
Block a user