Dashboard › opencode-lore › Session 0QLZIxt9RELv
0QLZIxt9RELv1dxwHPerform a strict READ-ONLY independent security/privacy review in /home/byk/Code/opencode-lore-responses-projection-security. Do not edit files or mutate VCS. Candidate stable change zlsmzkntqzzxsqvyxzzpxltmrsprymww, commit a3fa555bdcabb376aa33278bdb76007269f1940a. Base literal 2cbf40060c67df229731cf1b9e34d5bf22ec2145. Expected plain diff SHA-256 a7e914bd8ffb704f8fba2798f9de95fd4668b7212a485304c1cf7c056c278a40; expected changed paths exactly packages/gateway/src/pipeline.ts and packages/gateway/test/openai-responses-recall-aware-stream.test.ts. Read AGENTS.md, quality/REVIEW.md, scripts, every changed line and relevant stream/accumulator/callback/persistence/logging paths. Verify identity/status/conflicts/scope/hash before and after. Attack hidden recall projection, sparse and reordered source indices, direct live frames vs dense terminal arrays, terminal-discovered recalls, synthetic anchor placement, opaque/unknown/unindexed/indexed extension fields carrying private query/result/coverage/identity/newlines, codex.rate_limits allowlist and private accumulation, created_at injection, malformed identity/status, retries, cancellation, backpressure, resource and CPU amplification, callback/log/telemetry/persistence leakage, exact-once behavior, and failed terminal envelopes. Confirm every event still reaches required private state before suppression where applicable, while unsupported frames never reach clients. Assess tests and mutation/fail-first discrimination; focused tests allowed, no exhaustive suite. Report PASS/CONCERN/MUST-FIX with exact current file:line evidence and residual risk. End exactly MERGE or DO-NOT-MERGE.
<skill_content name="jj-guide"> # Skill: jj-guide # jj Guide for AI Agents Jujutsu (jj) is a Git-compatible VCS with mutable commits, automatic snapshotting, no staging area, and first-class conflicts. This skill teaches you how to use it safely from a non-interactive agent environment. If `.jj/` exists in the repo root, this is a jj repo. **Use `jj` commands, not `git`.** In a colocated repo (`.jj/` *and* `.git/`), git tools can read the state, but mutations should go through `jj` so the operation log stays consistent. ## Critical Rules - **NEVER** use interactive flags (`-i`, `--interactive`). TUI prompts hang in agent environments. This applies to `jj split -i`, `jj squash -i`, `jj commit -i`, `jj resolve`, `jj diffedit`, etc. - **ALWAYS** pass `-m "msg"` when describing/committing. Without `-m`, an editor opens and hangs. - **VERIFY** mutations with `jj st` and `jj log` after `squash`, `abandon`, `rebase`, `restore`, `commit`. ...
<path>/home/byk/Code/opencode-lore-responses-projection-security</path> <type>directory</type> <entries> .craft.yml .git .github/ .gitignore .jj/ .lore.md .oxfmtrc.json .oxlintrc.json AGENTS.md CHANGELOG.md codecov.yml LICENSE node_modules/ package.json packages/ patches/ pnpm-lock.yaml pnpm-workspace.yaml quality/ README.md scripts/ stryker.config.mjs supabase/ tsconfig.base.json tsconfig.json vitest.config.ts vitest.evals.config.ts vitest.mutation.config.ts (28 entries) </entries>
The working copy has no changes. Working copy (@) : lnvrxqoq 884f9b00 (empty) (no description set) Parent commit (@-): zlsmzknt a3fa555b fix/responses-hidden-output* | fix(responses): harden hidden-output projection Warning: These bookmarks have conflicts: fix/responses-private-references Hint: Use `jj bookmark list` to see details. Use `jj bookmark set <name> -r <rev>` to resolve. lnvrxqoqsqutpvpxvnppskkmmmsztwus 884f9b007c89e71800dbebc00af2e22f0454e592 parents=a3fa555bdcabb376aa33278bdb76007269f1940a conflict=false empty=true description= zlsmzkntqzzxsqvyxzzpxltmrsprymww a3fa555bdcabb376aa33278bdb76007269f1940a parents=583020fad43e1481470aac2588251b8636d16251 conflict=false empty=false description=fix(responses): harden hidden-output projection pxrvwsvvkmpoxnsqmrkwrnlqvvkoltwz 2cbf40060c67df229731cf1b9e34d5bf22ec2145 parents=e977b4b1da8f8197c3238466d41de0bccd8eb054 conflict=false empty=false description=feat(cch): add seeds for Claude Code up to 2.1.272 ...
a7e914bd8ffb704f8fba2798f9de95fd4668b7212a485304c1cf7c056c278a40 - c4e0f41b3ce8f3cb96fc080925f58b36b6127799bb50918d47d683330431af02 - a7e914bd8ffb704f8fba2798f9de95fd4668b7212a485304c1cf7c056c278a40 - zlsmzkntqzzx a3fa555bdcabb376aa33278bdb76007269f1940a parents=583020fad43e1481470aac2588251b8636d16251 fix(responses): harden hidden-output projection ozrmvotwxzuy 583020fad43e1481470aac2588251b8636d16251 parents=ab979df5cddfec3952680e423e39564664e881fe fix(responses): stabilize sparse principal projection unznwpylspzq ab979df5cddfec3952680e423e39564664e881fe parents=2cbf40060c67df229731cf1b9e34d5bf22ec2145 fix(responses): project hidden output consistently pxrvwsvvkmpo 2cbf40060c67df229731cf1b9e34d5bf22ec2145 parents=e977b4b1da8f8197c3238466d41de0bccd8eb054 feat(cch): add seeds for Claude Code up to 2.1.272 (#1780)
<path>/home/byk/Code/opencode-lore-responses-projection-security/AGENTS.md</path> <type>file</type> <content> 1: <!-- This section is maintained by the coding agent via lore (https://github.com/BYK/loreai) --> 2: ## Long-term Knowledge 3: 4: For long-term knowledge entries managed by [lore](https://github.com/BYK/loreai) (gotchas, patterns, decisions, architecture), see [`.lore.md`](.lore.md) in the project root. 5: <!-- End lore-managed section --> 6: 7: ## Review 8: 9: See [`quality/REVIEW.md`](quality/REVIEW.md) for the review heuristics and workflow — regression-test discipline, adversarial-order state setup, fan-out registry coverage, recurring bug-class batteries, and the two-reviewer rule. Every non-trivial PR requires an adversarial correctness review before merge. 10: 11: # Project Overview 12: 13: Lore is a **three-tier memory architecture** for AI coding agents. ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/quality/REVIEW.md</path> <type>file</type> <content> 1: # Review Heuristics — Lore 2: 3: This document codifies the review bar for pull requests. It exists because 4: adversarial review has repeatedly caught correctness bugs that a green test 5: suite missed — skip/early-return branches, lifecycle sequences, and fan-out 6: registry gaps. Institutionalizing these patterns so they compound. 7: 8: The review workflow (§6) is referenced from AGENTS.md. 9: 10: --- 11: 12: ## 1. Regression-test discipline 13: 14: Every adversarial-review finding that surfaces a defect **must** land a 15: deterministic regression test in the same PR. The test: 16: - fails on the base branch (proving it reproduces the bug), 17: - passes on the fix (proving the fix works), 18: - drives the *real* precondition, not an artificially-constructed state (e.g. ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/scripts</path> <type>directory</type> <entries> bump-version.sh cch-scan.c check-cc-version.ts check-docs-links.mjs check-patches.ts check-preview-links.mjs check-social-meta.mjs extract-cch-seed.ts generate-config-docs.ts generate-env-docs.ts generate-release-checksums.mjs smoke-install.mjs (12 entries) </entries>
diff --git a/packages/gateway/src/pipeline.ts b/packages/gateway/src/pipeline.ts
index 004e03b3e2..9aa7944751 100644
--- a/packages/gateway/src/pipeline.ts
+++ b/packages/gateway/src/pipeline.ts
@@ -8057,6 +8057,21 @@
const sseInactivityMs = opts.sseInactivityMs ?? FOREGROUND_SSE_INACTIVITY_MS;
const maxPrincipalTransportRetries = 1;
const maxRecallContinuationTransportRetries = 1;
+ const isSemanticResponsesEvent = (event: string): boolean =>
+ event === "response.created" ||
+ event === "response.in_progress" ||
+ event === "response.completed" ||
+ event === "response.done" ||
+ event === "response.incomplete" ||
+ event === "response.failed" ||
+ event === "response.output_item.added" ||
+ event === "response.output_item.done" ||
+ event.startsWith("response.output_text.") ||
+ event.startsWith("response.refusal.") ||
+ event.startsWith("response.function_call_arguments.") ||
+ event.startsWith("response.content_part.") ||
+ ...
diff --git a/packages/gateway/test/openai-responses-recall-aware-stream.test.ts b/packages/gateway/test/openai-responses-recall-aware-stream.test.ts
index dca2a57d78..50a45d6d92 100644
--- a/packages/gateway/test/openai-responses-recall-aware-stream.test.ts
+++ b/packages/gateway/test/openai-responses-recall-aware-stream.test.ts
@@ -11,7 +11,7 @@
* `response.completed`).
*/
import { log } from "@loreai/core";
-import { afterEach, describe, test, expect } from "vitest";
+import { afterEach, describe, test, expect, vi } from "vitest";
import { streamResponsesRecallAware } from "../src/pipeline";
import {
setRecallContinuationFailureHook,
@@ -44,6 +44,10 @@
return `event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`;
}
+function rawSSEEvent(event: string, data: string): string {
+ return `event: ${event}\ndata: ${data}\n\n`;
+}
+
/** Build an upstream Responses SSE stream from ordered events. ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/package.json</path>
<type>file</type>
<content>
1: {
2: "name": "lore-monorepo",
3: "private": true,
4: "type": "module",
5: "license": "FSL-1.1-Apache-2.0",
6: "description": "Monorepo root for Lore — three-tier memory architecture",
7: "main": "./packages/opencode/src/index.ts",
8: "exports": {
9: ".": "./packages/opencode/src/index.ts"
10: },
11: "packageManager": "pnpm@10.28.0",
12: "volta": {
13: "node": "24.16.0",
14: "pnpm": "10.28.0"
15: },
16: "pnpm": {
17: "onlyBuiltDependencies": [
18: "esbuild",
19: "onnxruntime-node",
20: "sharp"
21: ],
22: "overrides": {
23: "basic-ftp": "5.3.1",
24: "ip-address": "10.2.0",
25: "fast-xml-parser": "5.7.3",
26: "@huggingface/hub": "2.11.0",
27: "onnxruntime-node": "1.27.0",
28: "yaml": ">=2.8.3",
29: "esbuild": ">=0.28.1",
30: "@opentelemetry/core": ">=2.8.0",
31: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/package.json</path>
<type>file</type>
<content>
1: {
2: "name": "@loreai/gateway",
3: "version": "0.40.0",
4: "type": "module",
5: "license": "FSL-1.1-Apache-2.0",
6: "description": "Lore as a transparent LLM proxy — context management for any AI coding client",
7: "main": "./dist/index.cjs",
8: "types": "./dist/index.d.cts",
9: "exports": {
10: ".": {
11: "types": "./dist/index.d.cts",
12: "bun": "./dist/index.bun.js",
13: "require": "./dist/index.cjs",
14: "default": "./dist/index.cjs"
15: }
16: },
17: "bin": {
18: "lore": "./dist/bin.cjs",
19: "lore-gateway": "./dist/bin.cjs"
20: },
21: "scripts": {
22: "typecheck": "tsc --noEmit",
23: "build": "tsx script/build.ts",
24: "bundle": "tsx script/bundle.ts",
25: "build:binary": "tsx script/build.ts --binary",
26: "build:binary:sea": "tsx script/build-binary-sea.ts",
27: ...
Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts:
Line 70: recordCacheUsage,
Line 687: let postResponseStartObserver: (() => void) | undefined;
Line 845: postResponseStartObserver = observer;
Line 932: log.error(
Line 1011: postResponseStartObserver = undefined;
Line 1114: log.error("streaming post-response drop cleanup failed:", error);
Line 1124: log.info("streaming post-response skipped during pipeline reset");
Line 1139: log.warn("streaming post-response queue full; dropping finalizer");
Line 1164: log.error("streaming post-response processing failed:", error);
Line 2141: * Build a compact, recall-by-id catalog of project knowledge titles (#917 "A").
Line 2722: log.warn(
Line 3345: log.info(
Line 3362: log.info(
Line 3634: log.warn("entity injection failed (non-fatal):", err);
Line 3649: ...
Found 50 matches
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts:
Line 205: applyResponsesEvent,
Line 206: finalizeResponsesAcc,
Line 207: formatResponsesEvent,
Line 8254: const contentPosition = finalizeResponsesAcc(acc).content.findIndex(
Line 8382: applyResponsesEvent(state, "response.output_item.added", {
Line 9154: applyResponsesEvent(acc, "response.output_item.added", {
Line 9445: applyResponsesEvent(acc, "response.output_item.done", {
Line 9627: output += formatResponsesEvent(
Line 9765: formatResponsesEvent(
Line 9779: formatResponsesEvent(
Line 9789: formatResponsesEvent(
Line 9799: formatResponsesEvent(
Line 9809: formatResponsesEvent(
Line 9819: formatResponsesEvent(
Line 9872: return formatResponsesEvent(
Line 10023: output += formatResponsesEvent(
Line 10348: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
7790: * The recall execution callback abstracts the pipeline-scope dependencies
7791: * (`executeRecall` + follow-up forwarding + recall store), so this function
7792: * stays a self-contained streamer in the Responses module.
7793: */
7794: export function streamResponsesRecallAware(
7795: upstreamResponse: Response,
7796: opts: {
7797: onComplete: (response: GatewayResponse, successful: boolean) => void;
7798: onTransactionReady?: (transaction: {
7799: commit: () => void;
7800: rollback: () => void;
7801: }) => void;
7802: sessionID?: string;
7803: /** Emergency ceiling for the request-owned recall chain. */
7804: maxRecallExecutions?: number;
7805: /** @deprecated Use `maxRecallExecutions`. */
7806: maxRecallDepth?: number;
7807: noStore?: boolean;
7808: maxDeferredBytes?: number;
7809: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
8490: ) {
8491: throw new Error("duplicate Responses item identity");
8492: }
8493: }
8494: const initialArguments =
8495: item.type === "function_call" && typeof item.arguments === "string"
8496: ? item.arguments
8497: : "";
8498: const newLifecycle: OutputLifecycle = {
8499: argumentDeltaSeen: initialArguments.length > 0,
8500: argumentDeltas: initialArguments,
8501: argumentsDone: item.type !== "function_call",
8502: outputDone: false,
8503: reasoning: new Map(),
8504: content: new Map(),
8505: };
8506: if (item.type === "message") {
8507: seedTextParts(
8508: item.content,
8509: newLifecycle.content,
8510: new Set(["output_text", "refusal"]),
8511: "message content",
8512: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
9290: item?.type === "reasoning" &&
9291: typeof item.encrypted_content === "string"
9292: ) {
9293: throw new Error(
9294: `Responses stream ended with provisional reasoning for index ${index}`,
9295: );
9296: }
9297: // Sparse Codex may omit output_item.done. Non-reasoning items and
9298: // reasoning without a string ciphertext envelope are safe to retain.
9299: }
9300: };
9301: const preserveStreamedReasoning = (
9302: acc: ResponsesAccState,
9303: outputIndex: number,
9304: ): void => {
9305: const raw = acc.rawItems.get(outputIndex);
9306: const lifecycle = lifecyclesFor(acc).get(outputIndex);
9307: if (raw?.type !== "reasoning" || !lifecycle?.reasoning.size) return;
9308: const summary = Array.isArray(raw.summary) ? [...raw.summary] : [];
9309: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10140: | "principal_missing_terminal"
10141: | "principal_unexpected";
10142: let principalFailureCategory: PrincipalFailureCategory =
10143: "principal_unexpected";
10144: const classifyPrincipalFailure = (
10145: error: unknown,
10146: ): PrincipalFailureCategory => {
10147: if (error instanceof SSEStreamTransportError) {
10148: return "principal_transport";
10149: }
10150: if (error instanceof SSEStreamLimitError) {
10151: return "principal_resource_limit";
10152: }
10153: return principalFailureCategory;
10154: };
10155: const principalTransportStage = () =>
10156: ordinaryToolEmitted
10157: ? ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10990: const bytes =
10991: contUnresolvedToolBytes.get(outputIndex) ?? 0;
10992: contUnresolvedToolBytes.delete(outputIndex);
10993: continuationRecallBytes += bytes;
10994: hiddenRecallBytes += bytes;
10995: if (
10996: continuationRecallBytes > maxDeferredBytes ||
10997: hiddenRecallBytes > maxHiddenRecallBytes
10998: ) {
10999: throw new RecallContinuationFailure("resource_limit");
11000: }
11001: };
11002: let contOtherTool = false;
11003: let continuationCompleted = false;
11004: let ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
11890: : continuationAttempted
11891: ? (continuationFailureCategory ?? "unexpected")
11892: : classifyPrincipalFailure(err);
11893: log.error(
11894: `openai-responses recall-aware stream failed${category ? ` category=${category}` : ""}`,
11895: );
11896: }
11897: if (!signal.aborted) {
11898: if (err instanceof RecallContinuationFailure) {
11899: reportContinuationFailure(err.category);
11900: } else if (continuationAttempted) {
11901: reportContinuationFailure(
11902: continuationFailureCategory ?? ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/stream/openai-responses.ts</path> <type>file</type> <content> 1: /** 2: * OpenAI Responses API SSE stream accumulator. 3: * 4: * Parses upstream Responses API streaming events and accumulates the full 5: * response into a `GatewayResponse`. The Responses API uses a different 6: * SSE event lifecycle than Anthropic: 7: * 8: * response.created → response.in_progress → 9: * response.output_item.added → response.output_text.delta (repeated) → 10: * response.output_item.done → response.function_call_arguments.delta → 11: * response.function_call_arguments.done → 12: * response.completed 13: * 14: * Reuses `parseSSEStream` from the Anthropic stream module since the 15: * underlying SSE wire format is the same. ...
Found 49 matches
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/worker-copilot-responses.test.ts:
Line 20: * That endpoint is `unsupported_api_for_model` on `/chat/completions` and ONLY
Line 44: created_at: 1785900000,
Line 194: test("effort off omits unsupported Copilot Responses reasoning none", async () => {
Line 400: test("unsupported_api_for_model rebuilds and retries via the alternate protocol", async () => {
Line 404: JSON.stringify({ error: { code: "unsupported_api_for_model" } }),
Line 445: JSON.stringify({ error: { code: "unsupported_api_for_model" } }),
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/responses-upstream-error-relay.test.ts:
Line 83: new Response("provider diagnostic: token=secret", {
Line 90: "x-provider-diagnostic": "token=secret",
Line 101: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
8120: id: null,
8121: ids: ["k:one", "k:two"],
8122: detailOffset: null,
8123: detailLimit: null,
8124: },
8125: expected: {
8126: query: "",
8127: scope: undefined,
8128: id: undefined,
8129: ids: ["k:one", "k:two"],
8130: detailOffset: undefined,
8131: detailLimit: undefined,
8132: },
8133: },
8134: ])(
8135: "accepts nullable strict $mode recall arguments",
8136: async ({ args, expected }) => {
8137: let seen:
8138: | {
8139: query: string;
8140: scope?: string;
8141: id?: string;
8142: ids?: string[];
8143: detailOffset?: number;
8144: detailLimit?: number;
8145: }
8146: | undefined;
8147: const ...
Found 10 matches
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/translate/types.ts:
Line 349: codexRateLimits?: Array<Record<string, unknown>>;
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/translate/openai-responses.ts:
Line 1281: for (const quota of resp.codexRateLimits ?? []) {
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/stream/openai-responses.ts:
Line 60: codexRateLimits?: Array<Record<string, unknown>>;
Line 602: (state.codexRateLimits ??= []).push(quota);
Line 897: ...(state.codexRateLimits
Line 898: ? { codexRateLimits: state.codexRateLimits }
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts:
Line 18717: if (currentResp.codexRateLimits?.length) {
Line 18718: continuationResp.codexRateLimits = [
Line 18719: ...currentResp.codexRateLimits,
Line 18720: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/translate/openai-responses.ts</path>
<type>file</type>
<content>
1240:
1241: function incompleteDetails(stopReason: string): { reason: string } {
1242: return {
1243: reason:
1244: stopReason === "content_filter" ? "content_filter" : "max_output_tokens",
1245: };
1246: }
1247:
1248: function buildOpenAIResponsesStreamResponse(resp: GatewayResponse): Response {
1249: const usage = resp.usage ?? ZERO_USAGE;
1250: const encoder = new TextEncoder();
1251:
1252: const stream = new ReadableStream({
1253: start(controller) {
1254: const respId = resp.id.startsWith("resp_") ? resp.id : `resp_${resp.id}`;
1255: const created = Math.floor(Date.now() / 1000);
1256:
1257: function emit(eventType: string, data: Record<string, unknown>) {
1258: controller.enqueue(
1259: encoder.encode(
1260: `event: ${eventType}\ndata: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
18650: );
18651: }
18652: log.error(
18653: `recall follow-up fetch failed (non-stream, depth=${recallDepth}) for session ${sessionState.sessionID.slice(0, 16)}`,
18654: );
18655: if (finalRecallRound) return failRecall("follow_up_failed");
18656: bufferedRecallDiagnostics.finish("failed");
18657: // Fall back to response with marker (no continuation)
18658: markerResp.usage = cumulativeUsage;
18659: finishBufferedResponse(markerResp);
18660: return nonStreamHttpResponse(
18661: shouldInjectWarning
18662: ? injectContextWarning(markerResp, warningText)
18663: : markerResp,
18664: req.protocol,
18665: req.stream,
18666: { "x-lore-recall-invoked": "true" },
18667: longContext,
18668: ...
Found 3 matches
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/stream/anthropic.ts:
Line 11: * All functions are pure (no side effects) except `parseSSEStream` which is
Line 150: export async function* parseSSEStream(
Line 1986: for await (const { event, data } of parseSSEStream(reader, {
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/stream/anthropic.ts</path>
<type>file</type>
<content>
120: "inactivity",
121: "SSE stream inactivity deadline exceeded",
122: );
123: void reader.cancel(inactivityError).catch(() => {});
124: reject(inactivityError);
125: }, opts.inactivityMs);
126: }),
127: );
128: }
129:
130: try {
131: const result = await Promise.race(reads);
132: if (inactivityError) throw inactivityError;
133: opts.signal?.throwIfAborted();
134: return result;
135: } finally {
136: if (onAbort) opts.signal?.removeEventListener("abort", onAbort);
137: if (inactivityTimer) clearTimeout(inactivityTimer);
138: }
139: }
140:
141: /**
142: * Parse an SSE byte stream into typed events. ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
2370: return finalizeResponsesAcc(state);
2371: }
2372:
2373: // ---------------------------------------------------------------------------
2374: // True pass-through streamer (Responses upstream → Responses client)
2375: // ---------------------------------------------------------------------------
2376:
2377: /**
2378: * Serialize a parsed SSE event back to wire form, preserving the original data
2379: * payload (multi-line `data:` payloads are re-prefixed per line so nothing is
2380: * dropped or re-serialized — `reasoning_summary`, content_part annotations,
2381: * etc. survive intact because we forward the original `data` string).
2382: */
2383: export function formatResponsesEvent(event: string, data: string): string {
2384: const dataLines = data
2385: .split("\n")
2386: .map((line) => `data: ${line}`)
2387: ...
Found 2 matches
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts:
Line 7794: export function streamResponsesRecallAware(
Line 18931: streamResponsesRecallAware(upstreamResponse, {
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
18850: dropStreamingFinalizer,
18851: true,
18852: requestCredentialFingerprint(req.rawHeaders, config) ?? undefined,
18853: );
18854: }
18855: function finishUnsuccessfulStreaming(resp: GatewayResponse): void {
18856: if (streamingFinalizerRegistered) return;
18857: streamingFinalizerRegistered = true;
18858: scheduleStreamingPostResponse(
18859: sessionState.sessionID,
18860: requestGeneration,
18861: async () => {
18862: await downstreamSettled;
18863: await new Promise<void>((resolve) => setImmediate(resolve));
18864: rollbackRecallPersistence();
18865: if (
18866: requestGeneration !== streamingPostResponseGeneration ||
18867: sessionSignal.aborted
18868: ) {
18869: dropStreamingFinalizer();
18870: return;
18871: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
18690: finishBufferedResponse(markerResp);
18691: return nonStreamHttpResponse(
18692: shouldInjectWarning
18693: ? injectContextWarning(markerResp, warningText)
18694: : markerResp,
18695: req.protocol,
18696: req.stream,
18697: { "x-lore-recall-invoked": "true" },
18698: longContext,
18699: );
18700: }
18701:
18702: const { continuation: continuationResp, followUp } = jsonFollowUp;
18703:
18704: // Accumulate usage from this iteration
18705: const contUsage = continuationResp.usage ?? ZERO_USAGE;
18706: const continuationStopReason = recallBudget.recordUsage(contUsage);
18707: Object.assign(
18708: cumulativeUsage,
18709: mergeRecallUsage(cumulativeUsage, contUsage),
18710: );
18711:
18712: ...
Found 11 matches
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/codex-usage-provisional.test.ts:
Line 22: type: "codex.rate_limits",
Line 48: `event: codex.rate_limits\ndata: ${JSON.stringify(quota)}\n\n${await response.text()}`,
Line 104: .filter((frame) => frame.startsWith("event: codex.rate_limits\n"))
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/codex-usage-metadata.test.ts:
Line 11: type: "codex.rate_limits",
Line 58: .filter((frame) => frame.startsWith("event: codex.rate_limits\n"))
Line 72: expect(body.indexOf("codex.rate_limits")).toBeLessThan(
Line 130: `${await upstream([]).text()}event: codex.rate_limits\ndata: ${JSON.stringify(limits)}\n\n`,
Line 169: type: "codex.rate_limits",
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/recall-buffered-transaction.test.ts:
Line 65: type: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/codex-usage-metadata.test.ts</path>
<type>file</type>
<content>
1: import { describe, expect, test } from "vitest";
2: import fc from "fast-check";
3: import {
4: accumulateResponsesSSEStream,
5: ResponsesTerminalError,
6: streamResponsesPassthrough,
7: } from "../src/stream/openai-responses";
8: import { buildOpenAIResponsesResponse } from "../src/translate/openai-responses";
9:
10: const limits = {
11: type: "codex.rate_limits",
12: plan_type: "pro",
13: rate_limits: {
14: primary: { used_percent: 12.5, window_minutes: 300, reset_at: 2000000000 },
15: secondary: {
16: used_percent: 75,
17: window_minutes: 10080,
18: reset_at: 2000100000,
19: },
20: },
21: credits: { has_credits: true, unlimited: false, balance: "12.34" },
22: };
23:
24: function upstream(
25: events: Record<string, unknown>[],
26: incomplete = false,
27: ): Response {
28: const ...
Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/browser-origin-data-plane.test.ts:
Line 287: data: JSON.stringify({ data: [], output: [] }),
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/gateway-access-auth.e2e.test.ts:
Line 404: output: [],
Line 466: output: [],
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/codex-usage-metadata.test.ts:
Line 38: output: [],
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/foreground-body-limit.test.ts:
Line 198: output: [],
Line 226: output: [],
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/worker-reasoning-parse.test.ts:
Line 270: output: [],
Line 277: output: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
5840: textItem(9, "continuation", "msg_sparse_continuation"),
5841: completed("resp_followup"),
5842: ]);
5843: const client = streamResponsesRecallAware(
5844: streamFrom([
5845: created("resp_collision", "gpt-5.6-terra"),
5846: recallCall(0, { query: "architecture" }),
5847: textItem(7, "principal", "msg_sparse_principal"),
5848: completed("resp_collision"),
5849: ]),
5850: {
5851: onComplete: () => {},
5852: onRecall: async () => ({
5853: anchorText: buildAnchor("architecture"),
5854: resultText: "results",
5855: }),
5856: runFollowUp: async () => ({ reader: followUp.body!.getReader() }),
5857: },
5858: );
5859:
5860: const out = await drain(client);
5861: const terminal = ...
RUN v4.1.8 /home/byk/Code/opencode-lore-responses-projection-security
Test Files 1 passed (1)
Tests 223 passed (223)
Start at 15:52:56
Duration 18.67s (transform 8.47s, setup 1.44s, import 13.03s, tests 3.66s, environment 0ms)
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
1: /**
2: * Unit tests for `streamResponsesRecallAware` — the true-streaming,
3: * recall-aware OpenAI Responses (codex/ChatGPT) streamer.
4: *
5: * Regression for the "Provider response headers timed out after 10000ms" issue:
6: * the buffered `accumulateResponsesSSEStream` path withholds ALL client bytes
7: * until the entire (slow, reasoning-heavy) upstream completes, so opencode's
8: * 10s `ProviderHeaderTimeoutError` fired on ChatGPT sessions. This streamer
9: * forwards events live while transparently intercepting a `recall`
10: * function_call (emit marker, run follow-up, rebuild the terminal
11: * `response.completed`).
12: */
13: import { log } from "@loreai/core";
14: import { afterEach, describe, test, expect, vi } from "vitest";
15: import { streamResponsesRecallAware } from "../src/pipeline";
16: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/core/test/setup.ts</path>
<type>file</type>
<content>
1: import { mkdtempSync, rmSync } from "node:fs";
2: import { join } from "node:path";
3: import { tmpdir } from "node:os";
4: import { afterAll, afterEach } from "vitest";
5: import { close, invalidateProjectIdCache } from "../src/db";
6: import { silenceStderr } from "../src/log";
7:
8: // Create an isolated temporary database for the entire test run.
9: // This prevents test fixtures from leaking into the live lore DB
10: // at ~/.local/share/lore/lore.db.
11: const tmp = mkdtempSync(join(tmpdir(), "lore-test-"));
12: process.env.LORE_DB_PATH = join(tmp, "test.db");
13:
14: // ---------------------------------------------------------------------------
15: // Block live network to models.dev during tests.
16: //
17: // `fetchModelData()` (gateway/src/worker-model.ts) hits
18: // https://models.dev/api.json to pull pricing/limits, and the gateway
19: // ...
error: unexpected argument '--no-graph' found tip: a similar argument exists: '--no-pager' Usage: jj show --stat --no-pager [REVSETS]... For more information, try '--help'.
Commit ID: ab979df5cddfec3952680e423e39564664e881fe
Change ID: unznwpylspzqykmnpxwnnswmnzpspxzq
Author : Burak Yigit Kaya <ben@byk.im> (2026-09-12 22:22:40)
Committer: Burak Yigit Kaya <ben@byk.im> (2026-09-15 22:20:26)
fix(responses): project hidden output consistently
packages/gateway/src/pipeline.ts | 247 ++++++++++---
...est/openai-responses-recall-aware-stream.test.ts | 395 ++++++++++++++++++++--
2 files changed, 581 insertions(+), 61 deletions(-)
Commit ID: 583020fad43e1481470aac2588251b8636d16251
Change ID: ozrmvotwxzuykyvutstpwpzukpptktxv
Author : Burak Yigit Kaya <ben@byk.im> (2026-09-16 13:14:24)
Committer: Burak Yigit Kaya <ben@byk.im> (2026-09-16 13:43:03)
fix(responses): stabilize sparse principal projection
packages/gateway/src/pipeline.ts | 101 +++++++++++++---------
...est/openai-responses-recall-aware-stream.test.ts | 51 +++++++++++
2 files changed, 112 insertions(+), 40 deletions(-)
Commit ID: ...
Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/stream/openai-responses.ts:
Line 605: case "response.created":
Line 606: case "response.in_progress": {
Line 615: case "response.output_item.added": {
Line 644: case "response.output_item.done": {
Line 659: case "response.output_text.delta": {
Line 671: case "response.output_text.done": {
Line 684: case "response.refusal.delta": {
Line 694: case "response.refusal.done": {
Line 704: case "response.function_call_arguments.delta": {
Line 716: case "response.function_call_arguments.done": {
Line 733: case "response.failed":
Line 734: case "response.done":
Line 735: case "response.incomplete":
Line 736: case "response.completed": {
Line 740: event === "response.failed" ||
Line 744: : event === "response.incomplete" || status === "incomplete"
Line ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
1620: }
1621: }
1622: parsed.output_index = outputIndex;
1623: }
1624:
1625: function validateCodexResponsesEvent(
1626: state: ResponsesAccState,
1627: event: string,
1628: parsed: Record<string, unknown>,
1629: maxSparseIndex: number,
1630: ): void {
1631: if (parsed.type !== undefined && parsed.type !== event) {
1632: malformedResponsesEvent();
1633: }
1634: if (
1635: parsed.output_index !== undefined &&
1636: (!Number.isSafeInteger(parsed.output_index) ||
1637: (parsed.output_index as number) < 0 ||
1638: (parsed.output_index as number) >= maxSparseIndex)
1639: ) {
1640: malformedResponsesEvent();
1641: }
1642: if (
1643: parsed.content_index !== undefined &&
1644: (!Number.isSafeInteger(parsed.content_index) ||
1645: (parsed.content_index as number) < 0 ||
1646: ...
RUN v4.1.8 /home/byk/Code/opencode-lore-responses-projection-security
Test Files 1 passed (1)
Tests 15 passed (15)
Start at 15:55:28
Duration 6.92s (transform 3.45s, setup 645ms, import 5.46s, tests 576ms, environment 0ms)
<path>/home/byk/Code/opencode-lore-responses-projection-security/vitest.config.ts</path>
<type>file</type>
<content>
1: import { defineConfig } from "vitest/config";
2: import path from "node:path";
3:
4: export default defineConfig({
5: // Alias @loreai/core and @loreai/gateway for test imports.
6: // MUST be at the top level of the vite config — putting this under
7: // `test.resolve.alias` does NOT work (vite's resolver is a top-level
8: // option, not a test.* option). The previous placement silently
9: // resolved to the stale dist build, masking real test failures.
10: resolve: {
11: alias: {
12: "@loreai/core": path.resolve(__dirname, "packages/core/src"),
13: "@loreai/gateway": path.resolve(__dirname, "packages/gateway/src"),
14: },
15: },
16: test: {
17: // Run all packages' tests
18: include: [
19: "packages/core/test/**/*.test.ts",
20: // Fast, pure unit tests for the eval suite's own logic (scorers,
21: // ...
Found 14 matches
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts:
Line 6899: const recallDiagnostics = createRecallDiagnostics(
Line 7216: recallDiagnostics.record(input, result, coverage);
Line 7406: recallDiagnostics.finish("completed");
Line 7480: recallDiagnostics.finish("failed");
Line 7519: recallDiagnostics.finish("failed");
Line 7657: recallDiagnostics.finish("completed");
Line 7671: recallDiagnostics.finish(streamSignal.aborted ? "aborted" : "failed");
Line 7719: recallDiagnostics.finish("aborted");
Line 7876: const recallDiagnostics = createRecallDiagnostics(!opts.noStore);
Line 9641: recallDiagnostics.finish(successful ? "completed" : "failed");
Line 9692: recallDiagnostics.record(input, result.resultText, result.coverage);
Line 10035: recallDiagnostics.finish("aborted");
Line 12028: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/recall-diagnostics.ts</path>
<type>file</type>
<content>
1: import { createHash } from "node:crypto";
2: import { log, type RecallCoverage } from "@loreai/core";
3:
4: /** Keep request-local diagnostic state bounded while observing chains beyond ten rounds. */
5: export const MAX_RECALL_DIAGNOSTIC_ROUNDS = 64;
6:
7: /** Request-local comparisons only. Fingerprints and recall content never leave this closure. */
8: export function createRecallDiagnostics(enabled = true) {
9: const inputs = new Set<string>();
10: const results = new Set<string>();
11: const pairs = new Set<string>();
12: const coverage = new Set<string>();
13: const started = performance.now();
14: let finished = false;
15: let rounds = 0;
16: let detailCalls = 0;
17: let emptyBodies = 0;
18: let resultBytes = 0;
19: const fingerprint = (value: string) =>
20: ...
Found 15 matches
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/translate/types.ts:
Line 346: rawOutputItems?: Array<Record<string, unknown>>;
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/translate/openai-responses.ts:
Line 1163: const output: Array<Record<string, unknown>> = resp.rawOutputItems
Line 1164: ? [...resp.rawOutputItems]
Line 1169: if (!resp.rawOutputItems) {
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/stream/openai-responses.ts:
Line 891: rawOutputItems: Array.from(state.rawItems.entries())
/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/recall.ts:
Line 1217: (resp.rawOutputItems?.some(hasResponsesRefusal) ?? false)
Line 1274: const rawPrefixBlocks: GatewayContentBlock[] = (resp.rawOutputItems ?? [])
Line 1721: for (const item of resp.rawOutputItems ?? ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
12760: } else if (item.type === "function_call") {
12761: let input: unknown = {};
12762: if (typeof item.arguments === "string") {
12763: try {
12764: input = JSON.parse(item.arguments);
12765: } catch {
12766: input = item.arguments;
12767: }
12768: }
12769: const id = asString(item.call_id ?? item.id);
12770: if (!id || identities.has(id)) {
12771: throw new Error("malformed Responses response tool identity");
12772: }
12773: identities.add(id);
12774: content.push({
12775: type: "tool_use",
12776: id,
12777: name: asString(item.name),
12778: input,
12779: });
12780: }
12781: }
12782: }
12783:
12784: // Map Responses API status to gateway stop reason
12785: ...
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
12890: }
12891: }
12892: if (parts.length === 0 && textBlocks[textIndex]) {
12893: content.push(textBlocks[textIndex++]);
12894: }
12895: continue;
12896: }
12897: if (raw.type === "function_call") {
12898: const toolUseId = asString(raw.call_id ?? raw.id);
12899: if (toolUseId === stopBeforeToolUseId) break;
12900: const replacement = replacements.get(toolUseId);
12901: if (replacement) {
12902: content.push({ type: "text", text: replacement });
12903: continue;
12904: }
12905: const block = response.content.find(
12906: (candidate): candidate is GatewayToolUseBlock =>
12907: candidate.type === "tool_use" && candidate.id === toolUseId,
12908: );
12909: if (block) content.push(block);
12910: continue;
12911: }
12912: ...
Found 3 matches /home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts: Line 13207: function postResponseForTenant( Line 13575: postResponseForTenant( Line 18825: const persisted = postResponseForTenant(
<path>/home/byk/Code/opencode-lore-responses-projection-security/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
13180: ): AnthropicUsage {
13181: const usageForSentry: AnthropicUsage = {
13182: input_tokens: usage.inputTokens,
13183: output_tokens: usage.outputTokens,
13184: cache_read_input_tokens: usage.cacheReadInputTokens,
13185: cache_creation_input_tokens: usage.cacheCreationInputTokens,
13186: };
13187: setSentryCacheContext(usage);
13188: emitCostMetric(
13189: model,
13190: usageForSentry,
13191: "conversation",
13192: resolvedConversationTTL,
13193: );
13194: recordConversationCost(
13195: sessionID,
13196: model,
13197: usageForSentry,
13198: resolvedConversationTTL,
13199: );
13200: return usageForSentry;
13201: }
13202:
13203: /**
13204: * Run after a successful response: calibrate, store temporal messages,
13205: * and schedule background work (distillation, curation). ...
...output truncated...
Full output saved to: /home/byk/.local/share/opencode/tool-output/tool_0aaf29f2d00147dE66K4m2TAhI
+ let next = position + 1;
+ next < visibleOutputSources.length;
+ next++
+ ) {
+ const index = visibleOutputSources[next];
+ if (index !== undefined) visibleOutputIndices.set(index, next);
+ }
+ visibleOutputIndices.set(sourceIndex, position);
+ return true;
+ };
+ const assertForwardedProjectionStable = (): void => {
+ for (const [
+ sourceIndex,
+ visibleIndex,
+ ] of forwardedVisibleOutputIndices) {
+ if (visibleOutputIndices.get(sourceIndex) !== visibleIndex) {
+ projectionInvalid = true;
+ throw new Error(
+ "Responses output changed a client-visible output_index",
+ );
+ }
+ }
+ };
+ const ...
<path>/home/byk/.local/share/opencode/tool-output/tool_0aaf29f2d00147dE66K4m2TAhI</path> <type>file</type> <content> 1: The working copy has no changes. 2: Working copy (@) : lnvrxqoq 884f9b00 (empty) (no description set) 3: Parent commit (@-): zlsmzknt a3fa555b fix/responses-hidden-output* | fix(responses): harden hidden-output projection 4: Warning: These bookmarks have conflicts: 5: fix/responses-private-references 6: Hint: Use `jj bookmark list` to see details. Use `jj bookmark set <name> -r <rev>` to resolve. ...
Candidate identity and scope matched:
zlsmzkntqzzxsqvyxzzpxltmrsprymwwa3fa555bdcabb376aa33278bdb76007269f1940a2cbf40060c67df229731cf1b9e34d5bf22ec2145a7e914bd8ffb704f8fba2798f9de95fd4668b7212a485304c1cf7c056c278a40packages/gateway/src/pipeline.tspackages/gateway/test/openai-responses-recall-aware-stream.test.tsfix/responses-private-references, exists outside the candidate range.The new filter classifies entire event-name prefixes as semantic:
packages/gateway/src/pipeline.ts:8060-8074Therefore an unsupported event such as response.output_text.private passes as semantic. Validation accepts the prefix and does not reject unknown suffixes or unknown payload fields. The payload is then copied wholesale during remapping:
packages/gateway/src/pipeline.ts:10008-10026packages/gateway/src/pipeline.ts:11309-11327packages/gateway/src/pipeline.ts:11729-11749Valid semantic events have the same problem: arbitrary fields attached to deltas, parts, items, created/in-progress events, reasoning, refusals, and function calls remain intact.
Non-semantic principal frames are also forwarded verbatim until recall has already been identified:
packages/gateway/src/pipeline.ts:10364-10390A provider can emit a private diagnostic or indexed extension first and reveal the recall later, including through terminal discovery. The earlier frame has already reached the client.
Raw item fields also survive terminal reconstruction:
packages/gateway/src/pipeline.ts:9910-9972packages/gateway/src/pipeline.ts:9872-9889On failures, the public envelope uses these raw items unless projectionInvalid is ...