Dashboard › opencode-lore › Session 1Jof3Mogteq0
1Jof3Mogteq03odftStrict READ-ONLY security/privacy review in /home/byk/Code/opencode-lore-responses-accumulation-review-fix. Candidate rwunmtrq/3cbc99c3cf3844c67f5132411a5dcee3eda1723c, base 2cbf40060c67df229731cf1b9e34d5bf22ec2145, expected 3 files, diff hash 98d15668aca1ab833aec446154f7de6e0b0a7520a369531189a97eff250a98b6. No edits/VCS/server/network/prod DB. Verify immutable integrity. Attack sparse/extreme indices, normalization-added indices, shifted overflow, lifecycle-only private fields, terminal-only reasoning, output/callback/log leakage, usage manipulation/double accounting, failure rollback, identity collision bypass, high-cardinality CPU/memory, cancellation/backpressure/retries. Verify public sparse content compaction doesn't mutate private state; failure output empty; accepted crossing event retained privately; final done values authoritative. Assess fail-first/mutations supplied. Run focused tests if useful, exhaustive separate. Findings with exact file:line and PASS/CONCERN/MUST-FIX. End exactly MERGE or DO-NOT-MERGE.
<skill_content name="jj-guide"> # Skill: jj-guide # jj Guide for AI Agents Jujutsu (jj) is a Git-compatible VCS with mutable commits, automatic snapshotting, no staging area, and first-class conflicts. This skill teaches you how to use it safely from a non-interactive agent environment. If `.jj/` exists in the repo root, this is a jj repo. **Use `jj` commands, not `git`.** In a colocated repo (`.jj/` *and* `.git/`), git tools can read the state, but mutations should go through `jj` so the operation log stays consistent. ## Critical Rules - **NEVER** use interactive flags (`-i`, `--interactive`). TUI prompts hang in agent environments. This applies to `jj split -i`, `jj squash -i`, `jj commit -i`, `jj resolve`, `jj diffedit`, etc. - **ALWAYS** pass `-m "msg"` when describing/committing. Without `-m`, an editor opens and hangs. - **VERIFY** mutations with `jj st` and `jj log` after `squash`, `abandon`, `rebase`, `restore`, `commit`. ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix</path> <type>directory</type> <entries> .craft.yml .git .github/ .gitignore .jj/ .lore.md .oxfmtrc.json .oxlintrc.json AGENTS.md CHANGELOG.md codecov.yml LICENSE node_modules/ package.json packages/ patches/ pnpm-lock.yaml pnpm-workspace.yaml quality/ README.md scripts/ stryker.config.mjs supabase/ tsconfig.base.json tsconfig.json vitest.config.ts vitest.evals.config.ts vitest.mutation.config.ts (28 entries) </entries>
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/AGENTS.md /home/byk/Code/opencode-lore-responses-accumulation-review-fix/package.json /home/byk/Code/opencode-lore-responses-accumulation-review-fix/quality/REVIEW.md /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/package.json /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/website/package.json /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/pi/package.json /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/opencode/package.json /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/core/package.json
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/AGENTS.md</path> <type>file</type> <content> 1: <!-- This section is maintained by the coding agent via lore (https://github.com/BYK/loreai) --> 2: ## Long-term Knowledge 3: 4: For long-term knowledge entries managed by [lore](https://github.com/BYK/loreai) (gotchas, patterns, decisions, architecture), see [`.lore.md`](.lore.md) in the project root. 5: <!-- End lore-managed section --> 6: 7: ## Review 8: 9: See [`quality/REVIEW.md`](quality/REVIEW.md) for the review heuristics and workflow — regression-test discipline, adversarial-order state setup, fan-out registry coverage, recurring bug-class batteries, and the two-reviewer rule. Every non-trivial PR requires an adversarial correctness review before merge. 10: 11: # Project Overview 12: 13: Lore is a **three-tier memory architecture** for AI coding agents. ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/quality/REVIEW.md</path> <type>file</type> <content> 1: # Review Heuristics — Lore 2: 3: This document codifies the review bar for pull requests. It exists because 4: adversarial review has repeatedly caught correctness bugs that a green test 5: suite missed — skip/early-return branches, lifecycle sequences, and fan-out 6: registry gaps. Institutionalizing these patterns so they compound. 7: 8: The review workflow (§6) is referenced from AGENTS.md. 9: 10: --- 11: 12: ## 1. Regression-test discipline 13: 14: Every adversarial-review finding that surfaces a defect **must** land a 15: deterministic regression test in the same PR. The test: 16: - fails on the base branch (proving it reproduces the bug), 17: - passes on the fix (proving the fix works), 18: - drives the *real* precondition, not an artificially-constructed state (e.g. ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/package.json</path>
<type>file</type>
<content>
1: {
2: "name": "lore-monorepo",
3: "private": true,
4: "type": "module",
5: "license": "FSL-1.1-Apache-2.0",
6: "description": "Monorepo root for Lore — three-tier memory architecture",
7: "main": "./packages/opencode/src/index.ts",
8: "exports": {
9: ".": "./packages/opencode/src/index.ts"
10: },
11: "packageManager": "pnpm@10.28.0",
12: "volta": {
13: "node": "24.16.0",
14: "pnpm": "10.28.0"
15: },
16: "pnpm": {
17: "onlyBuiltDependencies": [
18: "esbuild",
19: "onnxruntime-node",
20: "sharp"
21: ],
22: "overrides": {
23: "basic-ftp": "5.3.1",
24: "ip-address": "10.2.0",
25: "fast-xml-parser": "5.7.3",
26: "@huggingface/hub": "2.11.0",
27: "onnxruntime-node": "1.27.0",
28: "yaml": ">=2.8.3",
29: "esbuild": ">=0.28.1",
30: "@opentelemetry/core": ...
The working copy has no changes. Working copy (@) : osyoktmo e0e1dde0 (empty) (no description set) Parent commit (@-): rwunmtrq 3cbc99c3 fix/responses-accumulation-order | fix(responses): complete accumulation boundaries Warning: These bookmarks have conflicts: fix/responses-private-references Hint: Use `jj bookmark list` to see details. Use `jj bookmark set <name> -r <rev>` to resolve.
rwunmtrqvptrsmozvynytrrxrroryloz 3cbc99c3cf3844c67f5132411a5dcee3eda1723c parents=ce88c614c0ae55d229cb605aa7c7369d918d3650 conflict=no empty=no desc=fix(responses): complete accumulation boundaries tklkqxytmpkzzryqlolnpvxouropxznp ce88c614c0ae55d229cb605aa7c7369d918d3650 parents=683dff02101f15198b18e3088a0b57dade290056 conflict=no empty=no desc=fix(responses): complete lifecycle reconciliation pxrvwsvvkmpoxnsqmrkwrnlqvvkoltwz 2cbf40060c67df229731cf1b9e34d5bf22ec2145 parents=e977b4b1da8f8197c3238466d41de0bccd8eb054 conflict=no empty=no desc=feat(cch): add seeds for Claude Code up to 2.1.272 (#1780)
rwunmtrqvptrsmoz 3cbc99c3cf3844c67f5132411a5dcee3eda1723c parents=ce88c614c0ae55d229cb605aa7c7369d918d3650 conflict=no desc=fix(responses): complete accumulation boundaries tklkqxytmpkzzryq ce88c614c0ae55d229cb605aa7c7369d918d3650 parents=683dff02101f15198b18e3088a0b57dade290056 conflict=no desc=fix(responses): complete lifecycle reconciliation kvpmnlwnznqxomuz 683dff02101f15198b18e3088a0b57dade290056 parents=df49b0cec904a3f432a0f7f29ea398840d62fe9e conflict=no desc=fix(responses): bound sparse lifecycle coordinates lplymwvllmznozzr df49b0cec904a3f432a0f7f29ea398840d62fe9e parents=2cbf40060c67df229731cf1b9e34d5bf22ec2145 conflict=no desc=fix(responses): preserve limit-crossing state pxrvwsvvkmpoxnsq 2cbf40060c67df229731cf1b9e34d5bf22ec2145 parents=e977b4b1da8f8197c3238466d41de0bccd8eb054 conflict=no desc=feat(cch): add seeds for Claude Code up to 2.1.272 (#1780)
packages/gateway/src/pipeline.ts packages/gateway/src/stream/openai-responses.ts packages/gateway/test/openai-responses-recall-aware-stream.test.ts
98d15668aca1ab833aec446154f7de6e0b0a7520a369531189a97eff250a98b6 -
packages/gateway/src/pipeline.ts | 654 ++++--- packages/gateway/src/stream/openai-responses.ts | 1 + ...st/openai-responses-recall-aware-stream.test.ts | 2049 +++++++++++++++++++++- 3 files changed, 2470 insertions(+), 234 deletions(-)
diff --git a/packages/gateway/src/pipeline.ts b/packages/gateway/src/pipeline.ts
index 004e03b3e2..7efc3562fe 100644
--- a/packages/gateway/src/pipeline.ts
+++ b/packages/gateway/src/pipeline.ts
@@ -7933,6 +7933,7 @@
acc: ResponsesAccState,
event: string,
parsed: Record<string, unknown>,
+ outputOffset = 0,
): ResponsesAccState | undefined => {
if (opts.validation !== "codex") return undefined;
let normalizationState = codexNormalizationStates.get(acc);
@@ -7940,11 +7941,21 @@
normalizationState = makeResponsesAccState();
codexNormalizationStates.set(acc, normalizationState);
}
+ // Tighten Codex's local exclusive ceiling by the continuation shift. This
+ // rejects an omitted index while the normalizer derives it, before either
+ // normalization or accumulation can retain an out-of-range item.
+ const localSparseCeiling = Math.max(
+ 0,
+ Math.min(
+ opts.maxSSEFrames ?? ...
diff --git a/packages/gateway/src/stream/openai-responses.ts b/packages/gateway/src/stream/openai-responses.ts
index 4c33c821b5..5623788876 100644
--- a/packages/gateway/src/stream/openai-responses.ts
+++ b/packages/gateway/src/stream/openai-responses.ts
@@ -824,6 +824,7 @@
if (item.type === "text") {
if (item.content) {
for (const part of item.content) {
+ if (!part) continue;
if (part.type === "output_text" && typeof part.text === "string") {
content.push({ type: "text", text: part.text });
} else {
+ test("accumulates the principal event that crosses the retained-state limit", async () => {
+ test("retains a sparse principal Codex function call resolved by the limit-crossing done event", async () => {
+ test("preserves an accepted sparse message content coordinate", async () => {
+ test("preserves sparse Codex content when done and terminal snapshots omit content", async () => {
+ test("accumulates the continuation event that crosses the retained-state limit", async () => {
+ test("retains a sparse continuation Codex function call resolved by the limit-crossing done event", async () => {
+ test("iteratively reconstructs a resource-limited continuation with 125500 content blocks", async () => {
+ test("keeps high-cardinality identity validation within a linear operation bound", async () => {
+ test("accounts for terminal Codex continuation usage once when candidate flush exceeds the transaction limit", async () => {
+ test("compacts sparse principal message content in a ...
Found 11 matches
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts:
Line 969: test("accumulates the principal event that crosses the retained-state limit", async () => {
Line 1187: test("retains a sparse principal Codex function call resolved by the limit-crossing done event", async () => {
Line 2610: test("preserves an accepted sparse message content coordinate", async () => {
Line 3153: test("preserves sparse Codex content when done and terminal snapshots omit content", async () => {
Line 7400: test("accumulates the continuation event that crosses the retained-state limit", async () => {
Line 7652: test("retains a sparse continuation Codex function call resolved by the limit-crossing done event", async () => {
Line 7748: test("iteratively reconstructs a resource-limited continuation with 125500 content blocks", async () => {
Line 7818: test("keeps high-cardinality ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
7850: signal: AbortSignal;
7851: }) => Promise<{
7852: anchorText: string;
7853: resultText: string;
7854: /** Private source coverage; never emitted to the client. */
7855: coverage?: readonly import("@loreai/core").RecallCoverage[];
7856: commit?: () => void;
7857: rollback?: () => void;
7858: }>;
7859: /** Streaming follow-up stage: build + forward + assert-SSE + reader. */
7860: runFollowUp: (ctx: {
7861: /** This is the one final continuation after the last allowed recall. ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
8490: }
8491: const lifecycles = lifecyclesFor(state);
8492: const lifecycle = lifecycles.get(outputIndex);
8493: if (lifecycle?.outputDone && event !== "response.output_item.added") {
8494: throw new Error(
8495: `Responses event after output_item.done for index ${outputIndex}`,
8496: );
8497: }
8498: if (event === "response.output_item.added") {
8499: if (state.rawItems.has(outputIndex)) {
8500: throw new Error(`duplicate Responses output_index ${outputIndex}`);
8501: }
8502: const item = parsed.item as Record<string, unknown> | undefined;
8503: const sparseCodexFunction =
8504: opts.validation === "codex" && item?.type === "function_call";
8505: if (
8506: !item ||
8507: typeof item.type !== "string" ||
8508: ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
9280: event === "response.completed" ||
9281: event === "response.done" ||
9282: event === "response.incomplete" ||
9283: event === "response.failed"
9284: ) {
9285: const response = parsed.response as Record<string, unknown> | undefined;
9286: if (acc.id && response?.id !== acc.id) {
9287: throw new Error("Responses terminal event changed response identity");
9288: }
9289: const status = response?.status;
9290: const terminalStatuses = new Set([
9291: "completed",
9292: "incomplete",
9293: "failed",
9294: "cancelled",
9295: ]);
9296: if (typeof status !== "string" || !terminalStatuses.has(status)) {
9297: throw new Error("Responses terminal event has nonterminal status");
9298: }
9299: if (
9300: (event === ...
Found 25 matches /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/pipeline.ts: Line 217: type ResponsesAccState, Line 252: DEFAULT_MAX_SSE_FRAMES, Line 7043: maxFrames: DEFAULT_MAX_SSE_FRAMES, Line 7569: maxFrames: DEFAULT_MAX_SSE_FRAMES, Line 7950: opts.maxSSEFrames ?? DEFAULT_MAX_SSE_FRAMES, Line 7951: DEFAULT_MAX_SSE_FRAMES, Line 8127: const maxSSEFrames = opts.maxSSEFrames ?? DEFAULT_MAX_SSE_FRAMES; Line 8128: const maxSparseIndex = Math.min(maxSSEFrames, DEFAULT_MAX_SSE_FRAMES); Line 15508: maxFrames: DEFAULT_MAX_SSE_FRAMES, /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai.ts: Line 27: DEFAULT_MAX_SSE_FRAMES, Line 194: maxFrames: opts.strict ? DEFAULT_MAX_SSE_FRAMES : undefined, Line 621: ? (opts.maxFrames ?? ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
520:
521: function completeCodexMessageSnapshot(
522: snapshot: Record<string, unknown>,
523: outputIndex: number,
524: terminalContentParts: ReadonlyMap<
525: string,
526: { kind: "text" | "refusal"; value: string }
527: >,
528: ): Record<string, unknown> {
529: if (snapshot.type !== "message" || snapshot.content !== undefined) {
530: return snapshot;
531: }
532: const finalizedContent = finalizedResponsesMessageContent(
533: outputIndex,
534: terminalContentParts,
535: ).map((part) =>
536: part.kind === "text"
537: ? { type: "output_text", text: part.value }
538: : { type: "refusal", refusal: part.value },
539: );
540: return finalizedContent.length > 0
541: ? { ...snapshot, content: finalizedContent }
542: : snapshot;
543: }
544:
545: /** Validated unsuccessful ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
1680: }
1681: }
1682: if (state.id && response.id !== undefined && response.id !== state.id) {
1683: throw new Error("malformed Responses terminal event");
1684: }
1685: if (
1686: state.model &&
1687: response.model !== undefined &&
1688: response.model !== state.model
1689: ) {
1690: throw new Error("malformed Responses terminal event");
1691: }
1692: if (
1693: event === "response.created" &&
1694: response.status !== undefined &&
1695: response.status !== "in_progress" &&
1696: response.status !== "queued"
1697: ) {
1698: malformedResponsesEvent();
1699: }
1700: if (
1701: event === "response.in_progress" &&
1702: response.status !== undefined &&
1703: response.status !== "in_progress"
1704: ) {
1705: ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
1380: outputIndex: number,
1381: itemId: string,
1382: kind: "text" | "tool_use",
1383: ): void {
1384: const item =
1385: kind === "text"
1386: ? { type: "message", id: itemId, role: "assistant" }
1387: : {
1388: type: "function_call",
1389: id: itemId,
1390: call_id: "",
1391: name: "",
1392: arguments: "",
1393: };
1394: applyResponsesEvent(state, "response.output_item.added", {
1395: output_index: outputIndex,
1396: item,
1397: });
1398: bindResponsesIdentity(state, state.itemIndexById, itemId, outputIndex);
1399: }
1400:
1401: function bindCodexItemId(
1402: state: ResponsesAccState,
1403: outputIndex: number,
1404: itemId: string,
1405: ): void {
1406: bindResponsesIdentity(state, state.itemIndexById, itemId, ...
Found 13 matches /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai-responses.ts: Line 1225: function nextResponsesOutputIndex( Line 1236: function findResponsesItemById( Line 1243: function bindResponsesIdentity( Line 1295: bindResponsesIdentity( Line 1398: bindResponsesIdentity(state, state.itemIndexById, itemId, outputIndex); Line 1406: bindResponsesIdentity(state, state.itemIndexById, itemId, outputIndex); Line 1448: bindResponsesIdentity( Line 1492: itemId !== undefined ? findResponsesItemById(state, itemId) : undefined; Line 1508: outputIndex = nextResponsesOutputIndex(state, maxSparseIndex); Line 1551: itemId !== undefined ? findResponsesItemById(state, itemId) : undefined; Line 1580: outputIndex ??= nextResponsesOutputIndex(state, maxSparseIndex); Line 1610: bindResponsesIdentity(state, state.itemIndexById, itemId, outputIndex); Line 1615: ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
1200: (typeof part.refusal === "string" && part.refusal.length > 0)),
1201: );
1202: }
1203: }
1204: if (
1205: event === "response.output_text.delta" ||
1206: event === "response.output_text.done" ||
1207: event === "response.refusal.delta" ||
1208: event === "response.refusal.done" ||
1209: event === "response.function_call_arguments.delta" ||
1210: event === "response.function_call_arguments.done"
1211: ) {
1212: return [parsed.delta, parsed.text, parsed.refusal, parsed.arguments].some(
1213: (value) => typeof value === "string" && value.length > 0,
1214: );
1215: }
1216: if (event.startsWith("response.reasoning")) {
1217: const part = isRecord(parsed.part) ? parsed.part : undefined;
1218: return [parsed.delta, parsed.text, parsed.summary_text, ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10220: let recallDetected = false;
10221: let resourceLimitedContinuationResponse: GatewayResponse | undefined;
10222: type PrincipalFailureCategory =
10223: | "principal_transport"
10224: | "principal_resource_limit"
10225: | "principal_protocol"
10226: | "principal_missing_terminal"
10227: | "principal_unexpected";
10228: let principalFailureCategory: PrincipalFailureCategory =
10229: "principal_unexpected";
10230: const classifyPrincipalFailure = (
10231: error: unknown,
10232: ): PrincipalFailureCategory => {
10233: if (error instanceof SSEStreamTransportError) {
10234: return "principal_transport";
10235: }
10236: if (error instanceof SSEStreamLimitError) {
10237: return ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10960: const chunk = encoder.encode(
10961: formatResponsesEvent(ce, cd),
10962: );
10963: if (
10964: contRecallIndices.size > 0 ||
10965: contUnresolvedToolIndices.size > 0
10966: ) {
10967: holdContinuation(chunk);
10968: } else {
10969: queueTransactional(chunk);
10970: }
10971: }
10972: continue;
10973: }
10974: if (cparsed.type !== ce) {
10975: throw new Error(
10976: ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/pipeline.ts</path> <type>file</type> <content> 11740: kind: error.kind, 11741: stage: "pre_output", 11742: outcome: "retry_exhausted", 11743: }); 11744: throw error; 11745: } 11746: 11747: currentPrincipalResponse = retryResponse; 11748: state = makeResponsesAccState(); 11749: syntheticIdentities.clear(); 11750: referenceIdentities.clear(); 11751: outputIdentities.clear(); 11752: recallIndices.clear(); 11753: unresolvedToolIndices.clear(); 11754: referenceIndices.clear(); 11755: recallDetected = false; 11756: principalFailureCategory = "principal_unexpected"; 11757: retainedStateBytes = retainedStateBaseline; 11758: ...
+ return Object.hasOwn(parsed, "output_index")
+ "response.reasoning_summary_text.delta",
+ "response.reasoning_summary_text.done",
+ "response.reasoning_summary_part.added",
+ "response.reasoning_summary_part.done",
+ "response.reasoning_text.delta",
+ "response.reasoning_text.done",
+type SparseCoordinateKind = "output" | "content" | "summary";
+ value = "private sparse coordinate value",
+ output_index: coordinate,
+ const reasoning = kind === "summary";
+ output_index: 0,
+ item: reasoning
+ ? { type: "reasoning", id: itemId, summary: [] }
+ reasoning
+ ? "response.reasoning_summary_text.delta"
+ output_index: 0,
+ ...(reasoning
+ ? { summary_index: coordinate }
+ : { content_index: coordinate }),
+ const privateMarker = `private-lifecycle-${event}-${suffix}`;
+ const reasoning = event.includes("reasoning");
+ output_index: 0,
+ item: reasoning
+ ? ...
@@ -11,7 +11,7 @@ @@ -31,6 +31,7 @@ @@ -44,6 +45,23 @@ @@ -173,6 +191,215 @@ @@ -739,6 +966,302 @@ @@ -1740,13 +2263,8 @@ @@ -1972,7 +2490,8 @@ @@ -2088,6 +2607,72 @@ @@ -2565,6 +3150,217 @@ @@ -4122,6 +4918,330 @@ @@ -4729,6 +5849,8 @@ @@ -4744,16 +5866,22 @@ @@ -6269,6 +7397,715 @@ @@ -6331,6 +8168,158 @@ @@ -8105,10 +10094,19 @@ @@ -8117,7 +10115,11 @@ @@ -8134,6 +10136,13 @@ @@ -8286,6 +10295,12 @@
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
1: /**
2: * Unit tests for `streamResponsesRecallAware` — the true-streaming,
3: * recall-aware OpenAI Responses (codex/ChatGPT) streamer.
4: *
5: * Regression for the "Provider response headers timed out after 10000ms" issue:
6: * the buffered `accumulateResponsesSSEStream` path withholds ALL client bytes
7: * until the entire (slow, reasoning-heavy) upstream completes, so opencode's
8: * 10s `ProviderHeaderTimeoutError` fired on ChatGPT sessions. This streamer
9: * forwards events live while transparently intercepting a `recall`
10: * function_call (emit marker, run follow-up, rebuild the terminal
11: * `response.completed`).
12: */
13: import { log } from "@loreai/core";
14: import { afterEach, describe, test, expect, vi } from "vitest";
15: import { streamResponsesRecallAware } from ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
930: runFollowUp: async () => {
931: throw new Error("should not run");
932: },
933: },
934: );
935: const out = await drain(client);
936: expect(out).toContain("response.failed");
937: expect(out).toContain(PUBLIC_GATEWAY_ERROR);
938: expect(out).not.toContain(PUBLIC_RECALL_ERROR);
939: expect(retries).toBe(0);
940: expect(errors).toEqual([
941: "openai-responses recall-aware stream failed category=principal_resource_limit",
942: ]);
943: });
944:
945: test("does not retry a pre-output principal resource limit", async () => {
946: let retries = 0;
947: const client = streamResponsesRecallAware(
948: streamFrom([created("resp_pre_output_limit", "gpt-5.6-terra")]),
949: {
950: maxStreamBytes: 1,
951: ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
2450: type: "image_generation_call",
2451: id: "image_terminal_null",
2452: status: "completed",
2453: result: null,
2454: details: { revised_prompt: null },
2455: };
2456: const terminalItem = {
2457: ...item,
2458: result: "base64-result",
2459: details: { revised_prompt: "cat" },
2460: };
2461: const client = streamResponsesRecallAware(
2462: streamFrom([
2463: created("resp_terminal_null", "gpt-5.6-terra"),
2464: sseEvent("response.output_item.added", {
2465: output_index: 0,
2466: item: { ...item, status: "generating" },
2467: }),
2468: sseEvent("response.output_item.done", {
2469: output_index: 0,
2470: item,
2471: }),
2472: sseEvent("response.completed", ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
4840: output_index: 0,
4841: item_id: "rs_summary_prefix",
4842: summary_index: 0,
4843: part: { type: "summary_text", text: "prefix" },
4844: }),
4845: sseEvent("response.output_item.done", {
4846: output_index: 0,
4847: item: {
4848: type: "reasoning",
4849: id: "rs_summary_prefix",
4850: summary: [{ type: "summary_text", text: "prefix" }],
4851: },
4852: }),
4853: completed("resp_summary_prefix"),
4854: ]),
4855: {
4856: onComplete: () => {},
4857: onRecall: async () => ({ anchorText: "", resultText: "" }),
4858: runFollowUp: async () => {
4859: throw new Error("should not run");
4860: },
4861: },
4862: );
4863: ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
7340: ];
7341: const continuationCreated = created(
7342: "resp_stream_chain_followup",
7343: "gpt-5.6-terra",
7344: );
7345: const followUp = streamFrom([
7346: continuationCreated,
7347: textItem(0, "answer"),
7348: completed("resp_stream_chain_followup"),
7349: ]);
7350: const client = streamResponsesRecallAware(streamFrom(principal), {
7351: maxStreamBytes:
7352: new TextEncoder().encode(principal.join("") + continuationCreated)
7353: .byteLength + 1,
7354: onComplete: () => {},
7355: onRecall: async () => ({
7356: anchorText: buildAnchor("architecture"),
7357: resultText: "results",
7358: }),
7359: runFollowUp: async () => ({ reader: followUp.body!.getReader() }),
7360: });
7361: ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
8360:
8361: const out = await drain(client);
8362: expect(out).toContain('"type":"refusal"');
8363: expect(out).toContain("cannot comply");
8364: expect(JSON.stringify(completedResponse)).toContain("cannot comply");
8365: });
8366:
8367: test("rejects an upstream item that collides with the synthetic anchor ID", async () => {
8368: let recalled = 0;
8369: const client = streamResponsesRecallAware(
8370: streamFrom([
8371: created("resp_collision", "gpt-5.6-terra"),
8372: recallCall(0, { query: "architecture" }),
8373: sseEvent("response.output_item.added", {
8374: output_index: 1,
8375: item: {
8376: type: "function_call",
8377: id: "msg_resp_collision_0",
8378: call_id: "call_read",
8379: ...
Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts:
Line 444: onComplete: () => {},
Line 460: test("finalizes when the client cancels immediately after a no-recall terminal", async () => {
Line 467: created("resp_cancel_terminal", "gpt-5.6-terra") +
Line 468: completed("resp_cancel_terminal", {
Line 478: cancel() {
Line 486: onComplete: () => completeCalls++,
Line 501: await reader.cancel();
Line 521: onComplete: () => {},
Line 550: onComplete: () => {},
Line 588: onComplete: () => {},
Line 602: test.each(["failed", "cancelled"])(
Line 613: onComplete: (response) => {
Line 670: onComplete: (response) => {
Line 727: onComplete: () => {},
Line 784: onComplete: () => {},
Line 835: onComplete: () => ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
430: status: "completed",
431: content: [{ type: "output_text", text }],
432: },
433: });
434:
435: describe("streamResponsesRecallAware", () => {
436: test("forwards events live with NO recall — no header hold-back", async () => {
437: const client = streamResponsesRecallAware(
438: streamFrom([
439: created("resp_plain", "gpt-5.6-terra"),
440: textItem(0, "hello world"),
441: completed("resp_plain", { input_tokens: 10, output_tokens: 3 }),
442: ]),
443: {
444: onComplete: () => {},
445: onRecall: async () => ({ anchorText: "", resultText: "" }),
446: runFollowUp: async () => {
447: throw new Error("should not be called");
448: },
449: },
450: );
451: const out = await drain(client);
452: // ...
Found 8 matches
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts:
Line 460: test("finalizes when the client cancels immediately after a no-recall terminal", async () => {
Line 1568: test("does not retry when the client cancels a stalled principal read", async () => {
Line 1598: test("cancels a retry response returned after foreground abort", async () => {
Line 7275: test("stops reading upstream while the client applies backpressure", async () => {
Line 9080: test("cancellation never waits for a non-settling recall callback", async () => {
Line 9111: test("cancels a reader returned late by an abort-ignoring follow-up setup", async () => {
Line 9150: test("does not retry when cancellation races a dropped continuation", async () => {
Line 9205: test("cancellation never waits for non-settling follow-up setup", async () => {
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
7180: {
7181: validation: "codex",
7182: maxDeferredBytes: 1024 * 1024,
7183: maxRetainedStateBytes: 8 * 1024 * 1024,
7184: onComplete: () => {},
7185: onRecall: async () => ({
7186: anchorText: buildAnchor("sparse limit"),
7187: resultText: "results",
7188: }),
7189: runFollowUp: async () => ({ reader: followUp.body!.getReader() }),
7190: },
7191: );
7192:
7193: expect(await drain(client)).toContain(PUBLIC_RECALL_ERROR);
7194: expect(failures).toEqual(["resource_limit"]);
7195: });
7196:
7197: test("charges a resolved sparse continuation candidate to the transactional spool", async () => {
7198: const failures: RecallContinuationFailureCategory[] = [];
7199: setRecallContinuationFailureHook((category) => ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
9020: rollbackObserved.resolve();
9021: },
9022: };
9023: },
9024: runFollowUp: async () => {
9025: throw new Error("should not run");
9026: },
9027: },
9028: );
9029: const reader = client.body!.getReader();
9030: await recallStarted;
9031: const cancelled = reader.cancel();
9032: resolveRecall?.();
9033: await cancelled;
9034: await rollbackObserved.promise;
9035: expect(committed).toBe(0);
9036: expect(rolledBack).toBe(1);
9037: });
9038:
9039: test("rolls back when the client disconnects after continuation delivery starts", async () => {
9040: let committed = 0;
9041: let completedCalls = 0;
9042: const rollbackObserved = Promise.withResolvers<void>();
9043: const client = ...
--- rwunmtrqvptrsmoz 3cbc99c3cf3844c67f5132411a5dcee3eda1723c fix(responses): complete accumulation boundaries --- tklkqxytmpkzzryq ce88c614c0ae55d229cb605aa7c7369d918d3650 fix(responses): complete lifecycle reconciliation --- kvpmnlwnznqxomuz 683dff02101f15198b18e3088a0b57dade290056 fix(responses): bound sparse lifecycle coordinates --- lplymwvllmznozzr df49b0cec904a3f432a0f7f29ea398840d62fe9e fix(responses): preserve limit-crossing state
packages/gateway/src/pipeline.ts | 363 ++++--- ...st/openai-responses-recall-aware-stream.test.ts | 1078 +++++++++++++++++++++- 2 files changed, 1320 insertions(+), 121 deletions(-) packages/gateway/src/pipeline.ts | 69 ++++- packages/gateway/src/stream/openai-responses.ts | 1 + ...est/openai-responses-recall-aware-stream.test.ts | 302 +++++++++++++++++++++- 3 files changed, 357 insertions(+), 15 deletions(-) packages/gateway/src/pipeline.ts | 87 ++++++ ...est/openai-responses-recall-aware-stream.test.ts | 304 ++++++++++++++++++++++ 2 files changed, 391 insertions(+), 0 deletions(-) packages/gateway/src/pipeline.ts | 155 +++--- ...est/openai-responses-recall-aware-stream.test.ts | 513 ++++++++++++++++++---- 2 files changed, 486 insertions(+), 182 deletions(-)
507801f745ae981c8b464b9ab792f32e0b673996554d96cbab9e4b9927eddbcc packages/gateway/src/pipeline.ts 46943c92c9dc972617ec8a48d56fb3b9fd47a8d00da7b61b3f15af149e65803b packages/gateway/src/stream/openai-responses.ts 1bfa67e70746b648b02b3eae3680b1724c6629339936b4ca7079227b236269df packages/gateway/test/openai-responses-recall-aware-stream.test.ts
RUN v4.1.8 /home/byk/Code/opencode-lore-responses-accumulation-review-fix
Test Files 1 passed (1)
Tests 262 passed (262)
Start at 17:54:57
Duration 19.18s (transform 8.34s, setup 1.01s, import 13.51s, tests 4.18s, environment 0ms)
Found 2 matches
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/core/src/log.ts:
Line 399: export function logFilePath(): string | undefined {
Line 497: export function error(...args: unknown[]): void {
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/core/src/log.ts</path>
<type>file</type>
<content>
430: }
431:
432: /** Append a single log line to the persistent log file. */
433: function writeToFile(level: string, message: string): void {
434: const path = logFilePath();
435: if (!path) return;
436:
437: // Periodic rotation check
438: if (++writeCount % ROTATION_CHECK_INTERVAL === 0) {
439: maybeRotate();
440: }
441:
442: const ts = new Date().toISOString();
443: const tag = level.toUpperCase().padEnd(5);
444: // Flatten multiline messages for clean tail -f output
445: const flat = message.replace(/\n/g, "\\n");
446: const line = `${ts} [${tag}] ${flat}\n`;
447:
448: let fd: number | undefined;
449: try {
450: fd = openLogFileForAppend(path);
451: writeFileSync(fd, line, "utf8");
452: } catch {
453: // Silently degrade — logging failure shouldn't crash the app
454: } finally {
455: if (fd !== ...
Found 4 matches
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/core/src/log.ts:
Line 150: function formatArgs(args: unknown[]): string {
Line 175: export function redactSensitiveLogText(value: string): string {
Line 206: function safeArgs(args: unknown[]): string[] {
Line 218: function sanitizedError(error: Error): Error {
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/core/src/log.ts</path>
<type>file</type>
<content>
120: // gateway `script/bundle.ts`). A module-level flag set by the plugin's core
121: // instance would NOT be seen by the gateway's bundled core instance, leaving
122: // the in-process gateway's own `[lore]` lines writing to the TUI. `globalThis`
123: // is the single object shared across every core instance in the main thread,
124: // so one `silenceStderr()` call silences them all.
125: const STDERR_SILENCED_KEY = "__loreStderrSilenced";
126:
127: function readStderrSilenced(): boolean {
128: return (globalThis as Record<string, unknown>)[STDERR_SILENCED_KEY] === true;
129: }
130:
131: /**
132: * Silence ALL stderr output from the logger (`info`/`warn`/`notice`/`error`),
133: * unconditionally — including when `LORE_DEBUG=1`. The file sink and the
134: * registered {@link LogSink} keep receiving everything, so nothing is lost. ...
No files found
Found 7 matches /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/usage-validation.ts: Line 204: export function validateResponsesUsage( /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/pipeline.ts: Line 235: validateResponsesUsage, Line 12710: const usage = validateResponsesUsage( /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/llm-adapter.ts: Line 83: validateResponsesUsage, Line 2685: const validatedUsage = validateResponsesUsage( /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai-responses.ts: Line 35: validateResponsesUsage, Line 763: const respUsage = validateResponsesUsage(
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/usage-validation.ts</path>
<type>file</type>
<content>
180: }
181: validateOpenAIDetails(
182: value.completion_tokens_details,
183: completion,
184: OPENAI_COMPLETION_DETAIL_FIELDS,
185: message,
186: );
187:
188: return value;
189: }
190:
191: function validateResponsesInputDetails(
192: value: unknown,
193: input: number | undefined,
194: message: string,
195: ): Record<string, unknown> | undefined {
196: const details = optionalDetails(value, message);
197: if (!details) return undefined;
198: const cached = tokenCount(details.cached_tokens, message);
199: const cacheWrite = tokenCount(details.cache_write_tokens, message);
200: validateSubset([cached, cacheWrite], input, message);
201: return details;
202: }
203:
204: export function validateResponsesUsage(
205: value: unknown,
206: message: string,
207: ): Record<string, unknown> | undefined ...
Found 10 matches
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts:
Line 3098: test("preserves principal finalized text when the Codex terminal item omits content", async () => {
Line 3113: text: "finalized text",
Line 3145: expect(output).toContain("finalized text");
Line 3149: text: "finalized text",
Line 3170: part: { type: "output_text", text: "sparse finalized text" },
Line 3176: text: "sparse finalized text",
Line 3182: part: { type: "output_text", text: "sparse finalized text" },
Line 3224: expect(output).toContain("sparse finalized text");
Line 3227: { type: "text", text: "sparse finalized text" },
Line 3236: text: "sparse finalized text",
Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/gateway-access-auth.e2e.test.ts:
Line 710: test("configured foreground auth replaces client auth and never forwards gateway access", () => {
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/compact-endpoint-tenant-isolation.e2e.test.ts:
Line 266: const summary = summaryFromResponse(endpoint, responseBody).replaceAll(
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/cch.test.ts:
Line 60: test("replaces cch=00000 with a 5-char hex hash", () => {
Line 148: text: "CCH_PLACEHOLDER = `cch=00000` (cch.ts:103); signBody() replaces it.",
Line 609: test("replaces client cch with a valid worker-signed cch", () => {
Line 621: test("replaces cc_version with worker version + computed suffix", () => {
Line 625: // Client's version should be replaced
...
The working copy has no changes. Working copy (@) : osyoktmo e0e1dde0 (empty) (no description set) Parent commit (@-): rwunmtrq 3cbc99c3 fix/responses-accumulation-order | fix(responses): complete accumulation boundaries Warning: These bookmarks have conflicts: fix/responses-private-references Hint: Use `jj bookmark list` to see details. Use `jj bookmark set <name> -r <rev>` to resolve. rwunmtrqvptrsmozvynytrrxrroryloz 3cbc99c3cf3844c67f5132411a5dcee3eda1723c parents=ce88c614c0ae55d229cb605aa7c7369d918d3650 conflict=no tklkqxytmpkzzryqlolnpvxouropxznp ce88c614c0ae55d229cb605aa7c7369d918d3650 parents=683dff02101f15198b18e3088a0b57dade290056 conflict=no
98d15668aca1ab833aec446154f7de6e0b0a7520a369531189a97eff250a98b6 - 507801f745ae981c8b464b9ab792f32e0b673996554d96cbab9e4b9927eddbcc packages/gateway/src/pipeline.ts 46943c92c9dc972617ec8a48d56fb3b9fd47a8d00da7b61b3f15af149e65803b packages/gateway/src/stream/openai-responses.ts 1bfa67e70746b648b02b3eae3680b1724c6629339936b4ca7079227b236269df packages/gateway/test/openai-responses-recall-aware-stream.test.ts
(no output)
Found 13 matches
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts:
Line 242: : "response.output_text.delta",
Line 412: sseEvent("response.output_text.delta", {
Line 2541: test("rejects an item_reference missing output_item.done", async () => {
Line 2969: sseEvent("response.output_text.delta", {
Line 3012: sseEvent("response.output_text.delta", {
Line 3241: "preserves sparse principal Codex content when item done is omitted and terminal output is %s",
Line 3299: "preserves shifted sparse continuation Codex content when item done is omitted and terminal output is %s",
Line 4275: sseEvent("response.output_text.delta", {
Line 4697: sseEvent("response.output_text.delta", {
Line 5365: sseEvent("response.output_text.delta", {
Line 6319: test("rejects a recall item missing output_item.done", async () => {
Line 7427: ...
<path>/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
10080: /event: response\.completed\ndata: (.+)/.exec(out)?.[1] ?? "{}",
10081: ) as { response?: { usage?: Record<string, unknown> } };
10082: expect(terminal.response?.usage).toEqual({
10083: input_tokens: 20,
10084: output_tokens: 5,
10085: total_tokens: 25,
10086: input_tokens_details: {
10087: cached_tokens: 7,
10088: cache_write_tokens: 3,
10089: },
10090: });
10091: });
10092:
10093: test("preserves response.incomplete from the continuation", async () => {
10094: const outcomes: boolean[] = [];
10095: let commits = 0;
10096: let rollbacks = 0;
10097: let rollbacksAtComplete: number | undefined;
10098: let completedResponse: GatewayResponse | undefined;
10099: const followUp = streamFrom([
10100: ...
Found 39 matches /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai.ts: Line 194: maxFrames: opts.strict ? DEFAULT_MAX_SSE_FRAMES : undefined, Line 195: maxEventBytes: opts.strict ? 4 * 1024 * 1024 : undefined, Line 484: maxFrames?: number; Line 620: maxFrames: opts.strict Line 621: ? (opts.maxFrames ?? DEFAULT_MAX_SSE_FRAMES) Line 622: : opts.maxFrames, /home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/src/stream/openai-responses.ts: Line 1882: maxFrames?: number; Line 1914: opts.maxFrames ?? DEFAULT_MAX_SSE_FRAMES, Line 1929: maxFrames: opts.validation Line 1930: ? (opts.maxFrames ?? DEFAULT_MAX_SSE_FRAMES) Line 1931: : opts.maxFrames, Line 2820: maxFrames: opts.strict ? DEFAULT_MAX_SSE_FRAMES : undefined, Line 2821: maxEventBytes: opts.strict ? ...
Found 1 matches
/home/byk/Code/opencode-lore-responses-accumulation-review-fix/packages/gateway/test/openai-responses-recall-aware-stream.test.ts:
Line 8890: throw new Error("postResponse failed");
The no-recall terminal path forwards failed provider events verbatim at packages/gateway/src/pipeline.ts:10616-10659. A hostile response.failed or failed/cancelled response.done can expose provider-controlled errors and output.
The existing test codifies the leak by expecting "provider failed" at packages/gateway/test/openai-responses-recall-aware-stream.test.ts:507-532.
Additionally, callback exceptions reach logs through pipeline.ts:9706-9719. packages/core/src/log.ts:150-155,175-203,497-504 preserves arbitrary error text except recognized credentials.
Required fixes:
These gaps predate the candidate but violate this review’s explicit privacy boundary.
safeEnqueue() waits for demand and invokes completion only after delivery (pipeline.ts:10149-10163). Both synthetic success delivery (pipeline.ts:11590-11639) and failure delivery (pipeline.ts:11878-11919) therefore own accounting completion.
Cancellation only rolls back and aborts (pipeline.ts:11934-11955); it never invokes finish(). The test at openai-responses-recall-aware-stream.test.ts:9039-9078 explicitly expects zero completion calls after a fully accumulated continuation terminal when the client cancels during delivery.
A client can stop reading or cancel after provider usage is known and prevent accounting entirely. Finalization must occur exactly once when the terminal or failure becomes authoritative, independently of client demand. ...