Dashboard › opencode-lore › Session 07myZYLrTNUL
07myZYLrTNULOuRNBImplement all accepted recovery correctness/security findings in /home/byk/Code/opencode-lore-recall-recovery current additive child pqvltmkr over immutable spyyzvyy/3f3ae768. You may edit but do not commit/push/rebase/bookmark/server/production DB. Read AGENTS.md, quality/REVIEW.md, scripts; use apply_patch. TDD mandatory: deterministic real-path tests must fail on current parent behavior before source fixes, then pass; independently remove each relevant guard and prove its named test fails; restore exactly. Findings: (1) malformed non-JSON event: codex.rate_limits must never be generically forwarded in principal or continuation; suppress/reject before opaque forwarding and no sentinel/quota accounting escapes. (2) byte cap fixture must cross 16 KiB before 64 events with max-schema payloads; deletion of byte guard fails; include live principal+continuation request-wide case. (3) failed continuation retry must restore request-wide public quota ledger baseline so failed attempt quota updates never consume successful retry cap. (4) disguised SSE with application/json must receive recovery-specific strict successful-completion validation for OpenAI, Anthropic, Gemini; hostile text/role/failure/missing identity/model/usage gives fixed 502, rollback, no persistence. (5) OpenAI JSON successful recovery must require numeric prompt_tokens and completion_tokens; usage:{} rejected. (6) non-OK error body reads get a short dedicated bounded timeout; hostile read/cancel never settles but caller proceeds to recovery; cancellation fire-and-forget, detail empty, fixed log. (7) assertSuccessfulResponsesCompletion must use terminal type/status registry, not blanket completed; valid failed function_call and image_generation_call output items remain accepted in ordinary non-stream parsing, while recovery usability still rejects failed tool handoffs. ...
<skill_content name="jj-guide"> # Skill: jj-guide # jj Guide for AI Agents Jujutsu (jj) is a Git-compatible VCS with mutable commits, automatic snapshotting, no staging area, and first-class conflicts. This skill teaches you how to use it safely from a non-interactive agent environment. If `.jj/` exists in the repo root, this is a jj repo. **Use `jj` commands, not `git`.** In a colocated repo (`.jj/` *and* `.git/`), git tools can read the state, but mutations should go through `jj` so the operation log stays consistent. ## Critical Rules - **NEVER** use interactive flags (`-i`, `--interactive`). TUI prompts hang in agent environments. This applies to `jj split -i`, `jj squash -i`, `jj commit -i`, `jj resolve`, `jj diffedit`, etc. - **ALWAYS** pass `-m "msg"` when describing/committing. Without `-m`, an editor opens and hangs. - **VERIFY** mutations with `jj st` and `jj log` after `squash`, `abandon`, `rebase`, `restore`, `commit`. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/AGENTS.md</path> <type>file</type> <content> 1: <!-- This section is maintained by the coding agent via lore (https://github.com/BYK/loreai) --> 2: ## Long-term Knowledge 3: 4: For long-term knowledge entries managed by [lore](https://github.com/BYK/loreai) (gotchas, patterns, decisions, architecture), see [`.lore.md`](.lore.md) in the project root. 5: <!-- End lore-managed section --> 6: 7: ## Review 8: 9: See [`quality/REVIEW.md`](quality/REVIEW.md) for the review heuristics and workflow — regression-test discipline, adversarial-order state setup, fan-out registry coverage, recurring bug-class batteries, and the two-reviewer rule. Every non-trivial PR requires an adversarial correctness review before merge. 10: 11: # Project Overview 12: 13: Lore is a **three-tier memory architecture** for AI coding agents. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/quality/REVIEW.md</path> <type>file</type> <content> 1: # Review Heuristics — Lore 2: 3: This document codifies the review bar for pull requests. It exists because 4: adversarial review has repeatedly caught correctness bugs that a green test 5: suite missed — skip/early-return branches, lifecycle sequences, and fan-out 6: registry gaps. Institutionalizing these patterns so they compound. 7: 8: The review workflow (§6) is referenced from AGENTS.md. 9: 10: --- 11: 12: ## 1. Regression-test discipline 13: 14: Every adversarial-review finding that surfaces a defect **must** land a 15: deterministic regression test in the same PR. The test: 16: - fails on the base branch (proving it reproduces the bug), 17: - passes on the fix (proving the fix works), 18: - drives the *real* precondition, not an artificially-constructed state (e.g. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/scripts</path> <type>directory</type> <entries> bump-version.sh cch-scan.c check-cc-version.ts check-docs-links.mjs check-patches.ts check-preview-links.mjs check-social-meta.mjs extract-cch-seed.ts generate-config-docs.ts generate-env-docs.ts generate-release-checksums.mjs smoke-install.mjs (12 entries) </entries>
<path>/home/byk/Code/opencode-lore-recall-recovery/package.json</path>
<type>file</type>
<content>
1: {
2: "name": "lore-monorepo",
3: "private": true,
4: "type": "module",
5: "license": "FSL-1.1-Apache-2.0",
6: "description": "Monorepo root for Lore — three-tier memory architecture",
7: "main": "./packages/opencode/src/index.ts",
8: "exports": {
9: ".": "./packages/opencode/src/index.ts"
10: },
11: "packageManager": "pnpm@10.28.0",
12: "volta": {
13: "node": "24.16.0",
14: "pnpm": "10.28.0"
15: },
16: "pnpm": {
17: "onlyBuiltDependencies": [
18: "esbuild",
19: "onnxruntime-node",
20: "sharp"
21: ],
22: "overrides": {
23: "basic-ftp": "5.3.1",
24: "ip-address": "10.2.0",
25: "fast-xml-parser": "5.7.3",
26: "@huggingface/hub": "2.11.0",
27: "onnxruntime-node": "1.27.0",
28: "yaml": ">=2.8.3",
29: "esbuild": ">=0.28.1",
30: "@opentelemetry/core": ">=2.8.0",
31: ...
No files found
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/package.json</path>
<type>file</type>
<content>
1: {
2: "name": "@loreai/gateway",
3: "version": "0.40.0",
4: "type": "module",
5: "license": "FSL-1.1-Apache-2.0",
6: "description": "Lore as a transparent LLM proxy — context management for any AI coding client",
7: "main": "./dist/index.cjs",
8: "types": "./dist/index.d.cts",
9: "exports": {
10: ".": {
11: "types": "./dist/index.d.cts",
12: "bun": "./dist/index.bun.js",
13: "require": "./dist/index.cjs",
14: "default": "./dist/index.cjs"
15: }
16: },
17: "bin": {
18: "lore": "./dist/bin.cjs",
19: "lore-gateway": "./dist/bin.cjs"
20: },
21: "scripts": {
22: "typecheck": "tsc --noEmit",
23: "build": "tsx script/build.ts",
24: "bundle": "tsx script/bundle.ts",
25: "build:binary": "tsx script/build.ts --binary",
26: "build:binary:sea": "tsx script/build-binary-sea.ts",
27: "start": ...
<path>/home/byk/Code/opencode-lore-recall-recovery</path> <type>directory</type> <entries> .craft.yml .git .github/ .gitignore .jj/ .lore.md .oxfmtrc.json .oxlintrc.json AGENTS.md CHANGELOG.md codecov.yml LICENSE node_modules/ package.json packages/ patches/ pnpm-lock.yaml pnpm-workspace.yaml quality/ README.md scripts/ stryker.config.mjs supabase/ tsconfig.base.json tsconfig.json vitest.config.ts vitest.evals.config.ts vitest.mutation.config.ts (28 entries) </entries>
The working copy has no changes. Working copy (@) : pqvltmkr f4c25568 (empty) fix(recall): harden quota metadata Parent commit (@-): spyyzvyy 3f3ae768 fix/recall-failure-synthesis | fix(recall): validate recovery metadata Warning: These bookmarks have conflicts: fix/responses-private-references Hint: Use `jj bookmark list` to see details. Use `jj bookmark set <name> -r <rev>` to resolve. pqvltmkrxpyn f4c2556896c8 fix(recall): harden quota metadata spyyzvyyyqxs 3f3ae768fde5 fix(recall): validate recovery metadata
Found 90 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/codex-rate-limits.ts:
Line 1: export const MAX_CODEX_RATE_LIMIT_EVENTS = 64;
Line 2: export const MAX_CODEX_RATE_LIMIT_BYTES = 16 * 1024;
Line 17: function rateLimit(value: unknown): Record<string, unknown> | undefined {
Line 68: if (!isRecord(value) || value.type !== "codex.rate_limits") return undefined;
Line 69: const result: Record<string, unknown> = { type: "codex.rate_limits" };
Line 78: if (isRecord(value.rate_limits)) {
Line 81: const normalized = rateLimit(value.rate_limits[name]);
Line 85: result.rate_limits = normalizedLimits;
Line 102: if (events.length >= MAX_CODEX_RATE_LIMIT_EVENTS) return undefined;
Line 107: if (bytes + encoder.encode(encoded).byteLength > MAX_CODEX_RATE_LIMIT_BYTES) {
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/http-body.ts:
Line 42: export const MAX_HTTP_REQUEST_COMPRESSED_BYTES = 32 * 1024 * ...
Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/cli-bundle-smoke.test.ts:
Line 167: // a `Try: lore login` recovery command.
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/anthropic-recall-continuation-abort.test.ts:
Line 248: "Anthropic recall exhaustion recovery (native=%s)",
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/llm-adapter.test.ts:
Line 3272: // Data-policy 404 detection + :free auto-recovery
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/cache-warmer.test.ts:
Line 667: // A partial is a body-divergence DEFECT, not a recovery: it must NOT clear
Line 962: // model these hit the recovery branch and were ignored.
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/invariant-judge.test.ts:
Line 213: describe("invariant worker recovery", () => ...
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/worker-recovery.test.ts
spyyzvyyyqxs 3f3ae768fde5 fix(recall): validate recovery metadata mmmktlwmwuts b61b086186cb fix(recall): harden failure synthesis mvtnxtlyooxq 9258a29630b9 fix(recall): synthesize after recall failure pxrvwsvvkmpo 2cbf40060c67 feat(cch): add seeds for Claude Code up to 2.1.272 (#1780) xpzmsyowwmzs e977b4b1da8f feat(semantic-lint): lint small PRs holistically (#1778) owulpnqtvpkq 991d7553c728 refactor(installer): move installation into the standalone CLI (#1743) uwoyklqrsuux 1e3657a08fd8 fix(gateway): recover principal Responses transport failures (#1776) zystzmxonwzs 81514b3cb46c feat(semantic-lint): pass bounded PR context to judges (#1772) zuqrxwsvzwrx 8ef91360cc9f feat(cch): add seeds for Claude Code up to 2.1.270 (#1769) twstwootrpsp b764c13fcfed Fix OpenCode lifecycle regression and SIGTERM shutdown (#1768) xuuwzpqyytkt 11cfe10bae21 Dispose embedded Lore gateway with OpenCode (#1767) tkzvmrtrwnsr 425cdac58fba refactor(core): split embedding runtime, pools, and backfills (#1766)
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/codex-rate-limits.ts</path>
<type>file</type>
<content>
1: export const MAX_CODEX_RATE_LIMIT_EVENTS = 64;
2: export const MAX_CODEX_RATE_LIMIT_BYTES = 16 * 1024;
3:
4: const encoder = new TextEncoder();
5:
6: const CATEGORY = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/;
7: const DECIMAL = /^(?:0|[1-9][0-9]{0,23})(?:\.[0-9]{1,12})?$/;
8:
9: function isRecord(value: unknown): value is Record<string, unknown> {
10: return value !== null && typeof value === "object" && !Array.isArray(value);
11: }
12:
13: function category(value: unknown): string | undefined {
14: return typeof value === "string" && CATEGORY.test(value) ? value : undefined;
15: }
16:
17: function rateLimit(value: unknown): Record<string, unknown> | undefined {
18: if (!isRecord(value)) return undefined;
19: const result: Record<string, unknown> = {};
20: if (
21: typeof value.used_percent === "number" &&
22: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
220: },
221: };
222: for (const type of [
223: "function_call_output",
224: "computer_call_output",
225: "computer_tool_call_output",
226: "local_shell_call_output",
227: "shell_call_output",
228: "custom_tool_call_output",
229: "program_output",
230: "tool_search_output",
231: ]) {
232: OUTPUT_ITEM_STATUSES_BY_TYPE[type] = {
233: added: COMPLETED,
234: done: COMPLETED,
235: terminal: COMPLETED,
236: };
237: }
238:
239: export function isSupportedResponsesOutputItemType(
240: type: unknown,
241: ): type is string {
242: return typeof type === "string" && RESPONSES_OUTPUT_ITEM_TYPES.has(type);
243: }
244:
245: export function isValidResponsesOutputItemStatus(
246: type: unknown,
247: status: unknown,
248: phase: OutputItemStatusPhase,
249: ): boolean {
250: if (status === undefined) return true;
251: if ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts</path> <type>file</type> <content> 1: /** 2: * OpenAI Responses API SSE stream accumulator. 3: * 4: * Parses upstream Responses API streaming events and accumulates the full 5: * response into a `GatewayResponse`. The Responses API uses a different 6: * SSE event lifecycle than Anthropic: 7: * 8: * response.created → response.in_progress → 9: * response.output_item.added → response.output_text.delta (repeated) → 10: * response.output_item.done → response.function_call_arguments.delta → 11: * response.function_call_arguments.done → 12: * response.completed 13: * 14: * Reuses `parseSSEStream` from the Anthropic stream module since the 15: * underlying SSE wire format is the same. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/recall.ts</path>
<type>file</type>
<content>
1120: const diagnostic = new Error("gateway recall execution failed");
1121: diagnostic.name = "RecallExecutionError";
1122: log.error(diagnostic);
1123: return {
1124: result: "Recall search failed. The memory system encountered an error.",
1125: input: { query, scope, id, ids, detailOffset, detailLimit },
1126: coverage: [],
1127: };
1128: }
1129: }
1130:
1131: // ---------------------------------------------------------------------------
1132: // Follow-up request builder (Case 1: recall-only)
1133: // ---------------------------------------------------------------------------
1134:
1135: /** Wire protocol used for a recall follow-up upstream response. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/recall.ts</path>
<type>file</type>
<content>
1680: }
1681: assertJSONResponse(response);
1682: const continuation = await parseFollowUpAgainstAbort(
1683: response,
1684: () => ctx.parseJSON(response, effectiveProtocol, signal),
1685: signal,
1686: );
1687: return { ok: true, continuation, followUp };
1688: }
1689:
1690: /**
1691: * Build a `stream: true` follow-up, forward it, accumulate the SSE body into a
1692: * non-streaming continuation, and return it in the same shape as
1693: * `runRecallFollowUpJSON`.
1694: *
1695: * This is the follow-up path for backends that MANDATE streaming and reject a
1696: * non-streaming request — specifically Pi's `openai-codex` provider, whose
1697: * ChatGPT backend (`/backend-api/codex/responses`) returns
1698: * `400 {"detail":"Stream must be set to true"}` for `stream: false`. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/recall.ts</path>
<type>file</type>
<content>
1540: signal?.removeEventListener("abort", onAbort);
1541: }
1542: }
1543:
1544: // ---------------------------------------------------------------------------
1545: // Coupled build + consume — the ONLY entry points the pipeline should use
1546: // ---------------------------------------------------------------------------
1547:
1548: /** Result of a streaming recall follow-up: the SSE reader + the request sent. */
1549: export interface RecallFollowUpStreaming {
1550: ok: true;
1551: /** SSE reader for the continuation stream — pipe through parseSSEStream(). */
1552: reader: ReadableStreamDefaultReader<Uint8Array>;
1553: /** The follow-up request that was sent (for the next loop iteration). */
1554: followUp: GatewayRequest;
1555: }
1556:
1557: /** Result of a non-streaming recall follow-up: the parsed continuation. ...
Found 20 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/recall.ts:
Line 44: import { promiseAgainstAbort } from "./abort-race";
Line 1399: async function ensureSSEResponse(
Line 1425: const { done, value } = await promiseAgainstAbort(
Line 1474: async function readResponseTextLimited(
Line 1487: const { done, value } = await promiseAgainstAbort(
Line 1510: function assertJSONResponse(response: Response): void {
Line 1524: async function parseFollowUpAgainstAbort<T>(
Line 1538: return await promiseAgainstAbort(parse, signal);
Line 1599: const { response } = await promiseAgainstAbort(
Line 1613: detail = await readResponseTextLimited(response, 500, signal);
Line 1620: const sseResponse = await ensureSSEResponse(response, signal);
Line 1633: * structurally impossible and any divergence fails loud (see assertJSONResponse). ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/recall.ts</path>
<type>file</type>
<content>
1340: ): GatewayRequest {
1341: const followUp = buildRecallFollowUpRequest(
1342: originalReq,
1343: resp,
1344: recallResult,
1345: recallToolUseBlock,
1346: stream,
1347: );
1348: const messages = [...followUp.messages];
1349: const resultMessage = messages.at(-1);
1350: if (!resultMessage) throw new Error("recall recovery result message missing");
1351: messages[messages.length - 1] = {
1352: ...resultMessage,
1353: content: resultMessage.content.map((block) =>
1354: block.type === "tool_result" && block.toolUseId === recallToolUseBlock.id
1355: ? {
1356: ...block,
1357: content: [
1358: ...block.content,
1359: { type: "text", text: RECALL_RECOVERY_INSTRUCTION },
1360: ],
1361: }
1362: : block,
1363: ),
1364: };
1365:
1366: ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts: Line 361: export function cancelAndReleaseReader(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts</path>
<type>file</type>
<content>
340: throw new Error("unterminated SSE event at EOF");
341: }
342: let eventType = "message";
343: const dataLines: string[] = [];
344: for (const line of buffer.split(/\r\n|\r|\n/)) {
345: if (line.startsWith("event:")) {
346: eventType = line.slice(6).trim();
347: } else if (line.startsWith("data:")) {
348: dataLines.push(line.slice(5).trimStart());
349: }
350: }
351: if (dataLines.length > 0) {
352: yield { event: eventType, data: dataLines.join("\n") };
353: }
354: }
355: break;
356: }
357: }
358: }
359:
360: /** Cancel without awaiting hostile sources, and release the reader lock safely. */
361: export function cancelAndReleaseReader(
362: reader: ReadableStreamDefaultReader<Uint8Array>,
363: reason?: ...
Found 7 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/recall.ts: Line 1613: detail = await readResponseTextLimited(response, 500, signal); Line 1674: detail = await readResponseTextLimited(response, 500, signal); Line 1752: detail = await readResponseTextLimited(response, 500, signal); /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts: Line 1032: foregroundErrorBodyTimeoutMs = FOREGROUND_ERROR_BODY_TIMEOUT_MS; Line 11819: const FOREGROUND_ERROR_BODY_TIMEOUT_MS = 10_000; Line 11821: let foregroundErrorBodyTimeoutMs = FOREGROUND_ERROR_BODY_TIMEOUT_MS; Line 11825: foregroundErrorBodyTimeoutMs = timeoutMs ?? FOREGROUND_ERROR_BODY_TIMEOUT_MS;
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
980: // bounded drain timeout (rare in tests — incremental distill/curation seldom
981: // trigger in short runs). The point: a late `saveSessionTracking()` write
982: // must land in THIS process's DB, not leak into the next one's as a phantom
983: // row — the cross-harness contamination behind the #859 flake. See #885.
984: if (!opts?.fast) {
985: if (stopIdleScheduler) {
986: stopIdleScheduler();
987: stopIdleScheduler = null;
988: }
989: await drainBackground();
990: // Bound this drain too (Seer) — a stalled urgent distillation / curation
991: // chain must not hang the reset, matching drainBackground's guarantee.
992: await boundedSettle(inFlightBackground);
993: inFlightBackground.clear();
994: }
995: initialized = false;
996: maxActivePipelineRequests = DEFAULT_MAX_ACTIVE_PIPELINE_REQUESTS;
997: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
11770:
11771: pull() {
11772: resumeDemand?.();
11773: resumeDemand = undefined;
11774: },
11775: cancel() {
11776: recallDiagnostics.finish("aborted");
11777: resumeDemand?.();
11778: resumeDemand = undefined;
11779: cancelled = true;
11780: cleanupAbort();
11781: if (deferredTransaction) deferredTransaction.rollback();
11782: else rollbackTransaction();
11783: abortController.abort(
11784: new DOMException("Responses client disconnected", "AbortError"),
11785: );
11786: if (keepaliveTimer) clearTimeout(keepaliveTimer);
11787: if (activeReader) cancelAndReleaseReader(activeReader, signal.reason);
11788: else
11789: void currentPrincipalResponse.body
11790: ?.cancel(signal.reason)
11791: .catch(() => {});
11792: },
11793: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
12240: throw new Error("upstream Responses request did not complete");
12241: }
12242: for (const rawPart of parts) {
12243: if (
12244: !rawPart ||
12245: typeof rawPart !== "object" ||
12246: Array.isArray(rawPart) ||
12247: (rawPart as Record<string, unknown>).type !== partType ||
12248: typeof (rawPart as Record<string, unknown>).text !== "string"
12249: ) {
12250: throw new Error("upstream Responses request did not complete");
12251: }
12252: }
12253: }
12254: if (
12255: item.encrypted_content !== undefined &&
12256: item.encrypted_content !== null &&
12257: typeof item.encrypted_content !== "string"
12258: ) {
12259: throw new ...
Found 9 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts: Line 11976: export async function accumulateNonStreamResponse( Line 11986: requireValidCompletion = false, Line 12021: requireSuccessfulCompletion: requireValidCompletion, Line 12057: if (requireValidCompletion) Line 12062: if (!requireValidCompletion || error instanceof ResponsesTerminalError) Line 15656: accumulateNonStreamResponse( Line 15748: : await accumulateNonStreamResponse( Line 18459: accumulateNonStreamResponse( Line 19196: accumulateNonStreamResponse(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
15590: if (req.protocol === "openai") {
15591: return withLimits(
15592: translateAnthropicStreamToOpenAI(anthropicSSE, {
15593: strict: true,
15594: signal: abortScope.signal,
15595: }),
15596: );
15597: }
15598: if (req.protocol === "openai-responses") {
15599: return withLimits(
15600: translateAnthropicStreamToResponses(anthropicSSE, {
15601: strict: true,
15602: signal: abortScope.signal,
15603: }),
15604: );
15605: }
15606: if (req.protocol === "gemini") {
15607: return withLimits(
15608: translateAnthropicStreamToGemini(anthropicSSE, {
15609: strict: true,
15610: signal: abortScope.signal,
15611: }),
15612: );
15613: }
15614: }
15615: // ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
18410: finishBufferedResponse(markerResp);
18411: return nonStreamHttpResponse(
18412: shouldInjectWarning
18413: ? injectContextWarning(markerResp, warningText)
18414: : markerResp,
18415: req.protocol,
18416: req.stream,
18417: { "x-lore-recall-invoked": "true" },
18418: longContext,
18419: );
18420: }
18421:
18422: const mergeFailedContinuationMetadata = (error: unknown): void => {
18423: if (
18424: error instanceof ResponsesTerminalError ||
18425: error instanceof NonStreamCompletionError
18426: ) {
18427: Object.assign(
18428: cumulativeUsage,
18429: mergeRecallUsage(
18430: cumulativeUsage,
18431: error.response.usage ?? ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
18550: log.info(
18551: `recall (non-stream, depth=${recallDepth}, codex=${followUpRequiresStream}): executing follow-up for session ${sessionState.sessionID.slice(0, 16)}`,
18552: );
18553: const jsonRecallCtx = makeJSONRecallCtx(false);
18554: let jsonFollowUp: Awaited<ReturnType<typeof runRecallFollowUpJSON>>;
18555: try {
18556: jsonFollowUp = followUpRequiresStream
18557: ? await runRecallFollowUpStreamAccumulated(
18558: jsonRecallCtx,
18559: currentModifiedReq,
18560: currentResp,
18561: followUpResult,
18562: recallBlock,
18563: foregroundAbort.signal,
18564: finalRecallRound,
18565: )
18566: : await runRecallFollowUpJSON(
18567: jsonRecallCtx,
18568: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10940: hiddenRecallBytes > maxHiddenRecallBytes
10941: ) {
10942: throw new RecallContinuationFailure(
10943: "resource_limit",
10944: );
10945: }
10946: if (
10947: ce === "response.function_call_arguments.done" &&
10948: isContRecall
10949: ) {
10950: contRecallInputs.set(
10951: ci,
10952: parseRecallArguments(cparsed.arguments),
10953: );
10954: }
10955: if (isContUnresolvedTool && !isContRecall) ...
Found 34 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts:
Line 7900: const publicCodexRateLimits: Array<Record<string, unknown>> = [];
Line 10259: const acceptedCodexRateLimit = applyResponsesEvent(
Line 10264: const publicCodexRateLimit = acceptedCodexRateLimit
Line 10266: publicCodexRateLimits,
Line 10267: acceptedCodexRateLimit,
Line 10270: const publicData = publicCodexRateLimit
Line 10271: ? JSON.stringify(publicCodexRateLimit)
Line 10624: let continuationRetryBaseline = {
Line 10867: const acceptedCodexRateLimit = applyResponsesEvent(
Line 10872: const publicCodexRateLimit = acceptedCodexRateLimit
Line 10874: publicCodexRateLimits,
Line 10875: acceptedCodexRateLimit,
Line 10878: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
7860: id?: string;
7861: ids?: string[];
7862: detailOffset?: number;
7863: detailLimit?: number;
7864: outputIndex: number;
7865: toolUseId: string;
7866: /** Position in the normalized GatewayResponse content array. */
7867: contentPosition: number;
7868: /** The accumulated response INCLUDING the recall tool_use, so the caller
7869: * can build the follow-up request from the same accumulation the
7870: * streamer uses. */
7871: acc: GatewayResponse;
7872: signal: AbortSignal;
7873: }) => Promise<{
7874: anchorText: string;
7875: resultText: string;
7876: /** Private source coverage; never emitted to the client. */
7877: coverage?: readonly import("@loreai/core").RecallCoverage[];
7878: commit?: () => void;
7879: rollback?: () => void;
7880: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10200: (index, item) => {
10201: if (
10202: item.type !== "function_call" ||
10203: item.name !== RECALL_TOOL_NAME
10204: ) {
10205: return;
10206: }
10207: recallDetected = true;
10208: recallIndices.add(index);
10209: },
10210: );
10211: if (outputIndex !== undefined) {
10212: retainedStateBytes += encoder.encode(data).byteLength;
10213: if (retainedStateBytes > maxRetainedStateBytes) {
10214: throw new SSEStreamLimitError(
10215: "Responses retained state exceeded byte limit",
10216: );
10217: }
10218: const implicitItem = state.rawItems.get(outputIndex);
10219: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10090: principalReadFinished = false;
10091: if (!currentPrincipalResponse.body) {
10092: throw new Error("Upstream response has no body");
10093: }
10094: const reader = currentPrincipalResponse.body.getReader();
10095: principalReader = reader;
10096: activeReader = reader;
10097:
10098: // --- Recall interception state ---
10099: // `output_index` values whose item is a suppressed `recall` function_call.
10100: const parsedRecallInputs = new Map<number, RecallArguments>();
10101: // Ordered list of parsed recall invocations: { outputIndex, block }.
10102: const pendingRecalls: PendingResponsesRecall[] = [];
10103: // Whether any NON-recall function_call appeared (mixed-tools case). ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10580: queueTransactional(anchorChunk);
10581: for (const deferred of deferredEvents) {
10582: queueTransactional(deferred.chunk);
10583: }
10584: } else {
10585: queueTransactional(anchorChunk);
10586: for (const deferred of deferredEvents) {
10587: queueTransactional(deferred.chunk);
10588: }
10589: }
10590: deferredEvents.length = 0;
10591: deferredBytes = 0;
10592:
10593: if (
10594: !otherToolSeen &&
10595: recall === pendingRecalls[pendingRecalls.length - 1]
10596: ) {
10597: // Recall-only: run the streaming follow-up and pipe the
10598: // ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/codex-usage-metadata.test.ts</path>
<type>file</type>
<content>
1: import { describe, expect, test } from "vitest";
2: import fc from "fast-check";
3: import {
4: accumulateResponsesSSEStream,
5: ResponsesTerminalError,
6: streamResponsesPassthrough,
7: } from "../src/stream/openai-responses";
8: import { buildOpenAIResponsesResponse } from "../src/translate/openai-responses";
9:
10: const MAX_CODEX_RATE_LIMIT_EVENTS = 64;
11: const MAX_CODEX_RATE_LIMIT_BYTES = 16 * 1024;
12:
13: const limits = {
14: type: "codex.rate_limits",
15: plan_type: "pro",
16: rate_limits: {
17: primary: { used_percent: 12.5, window_minutes: 300, reset_at: 2000000000 },
18: secondary: {
19: used_percent: 75,
20: window_minutes: 10080,
21: reset_at: 2000100000,
22: },
23: },
24: credits: { has_credits: true, unlimited: false, balance: "12.34" },
25: };
26:
27: function upstream(
28: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
1: /**
2: * Unit tests for `streamResponsesRecallAware` — the true-streaming,
3: * recall-aware OpenAI Responses (codex/ChatGPT) streamer.
4: *
5: * Regression for the "Provider response headers timed out after 10000ms" issue:
6: * the buffered `accumulateResponsesSSEStream` path withholds ALL client bytes
7: * until the entire (slow, reasoning-heavy) upstream completes, so opencode's
8: * 10s `ProviderHeaderTimeoutError` fired on ChatGPT sessions. This streamer
9: * forwards events live while transparently intercepting a `recall`
10: * function_call (emit marker, run follow-up, rebuild the terminal
11: * `response.completed`).
12: */
13: import { log } from "@loreai/core";
14: import { afterEach, describe, test, expect } from "vitest";
15: import { streamResponsesRecallAware } from "../src/pipeline";
16: import {
17: ...
Found 19 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/llm-adapter.test.ts:
Line 3233: // Soft-pause the session so we stop retrying every turn.
Line 5511: // never be given MORE room — retrying would either shrink the budget or waste
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/worker-recovery.test.ts:
Line 140: } else controller.error(new Error("private upstream read failure"));
Line 204: test("adapter read failure is not classified as empty output", async () => {
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/cli-doctor-contract.test.ts:
Line 189: embeddings: { available: false, state: "retrying", detail: "broken" },
Line 211: embeddings: { available: false, state: "retrying", detail: "broken" },
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/websocket-upgrade.test.ts:
Line 22: * the socket (under Bun), both of which stop the client from ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
5870: expect(out.includes(continuationText)).toBe(true);
5871: expect(out).not.toContain(PUBLIC_RECALL_ERROR);
5872: });
5873:
5874: test("caps unindexed transactional continuation output at the retained-state limit", async () => {
5875: const failures: RecallContinuationFailureCategory[] = [];
5876: setRecallContinuationFailureHook((category) => failures.push(category));
5877: const unindexedEvent = `event: extension\ndata: ${"x".repeat(1024 * 1024)}\n\n`;
5878: const followUp = streamFrom([
5879: created("resp_transactional_limit", "gpt-5.6-terra"),
5880: unindexedEvent,
5881: textItem(0, "answer"),
5882: completed("resp_transactional_limit"),
5883: ]);
5884: const client = streamResponsesRecallAware(
5885: streamFrom([
5886: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/pipeline-streaming.test.ts</path>
<type>file</type>
<content>
260: return (
261: responsesEvent("response.created", {
262: response: { id, model: "gpt-5.6-sol", status: "in_progress" },
263: }) +
264: responsesEvent("response.incomplete", {
265: response: {
266: id,
267: model: "gpt-5.6-sol",
268: status: "incomplete",
269: incomplete_details: { reason: "max_output_tokens" },
270: output: [],
271: usage: { input_tokens: 1, output_tokens: 0 },
272: },
273: })
274: );
275: }
276:
277: function makeResponsesRequest(input: {
278: sessionHeaders: Record<string, string>;
279: messages?: GatewayRequest["messages"];
280: tools?: GatewayRequest["tools"];
281: }): GatewayRequest {
282: return {
283: protocol: "openai-responses",
284: model: "gpt-5.6-sol",
285: system: "You are a coding agent.",
286: messages: ...
Found 43 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/pipeline-streaming.test.ts:
Line 404: { headers: { "content-type": "application/json" } },
Line 464: { headers: { "content-type": "application/json" } },
Line 491: { headers: { "content-type": "application/json" } },
Line 566: { headers: { "content-type": "application/json" } },
Line 592: { headers: { "content-type": "application/json" } },
Line 623: { headers: { "content-type": "application/json" } },
Line 628: headers: { "content-type": "application/json" },
Line 662: { headers: { "content-type": "application/json" } },
Line 687: { headers: { "content-type": "application/json" } },
Line 725: { headers: { "content-type": "application/json" } },
Line 1429: "content-type": "application/json",
Line 1899: { headers: { "content-type": ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/pipeline-streaming.test.ts</path>
<type>file</type>
<content>
6750: tools: [],
6751: stream: true,
6752: maxTokens: 32,
6753: metadata: {},
6754: rawHeaders: { "x-lore-agent": "title" },
6755: },
6756: loadLocalConfig(),
6757: );
6758: await vi.advanceTimersByTimeAsync(300_000);
6759: const response = await pending;
6760: expect(response.status).toBe(502);
6761: await expect(response.text()).resolves.toContain(
6762: "Gateway request failed",
6763: );
6764: } finally {
6765: setUpstreamInterceptor(undefined);
6766: vi.useRealTimers();
6767: }
6768: });
6769:
6770: it.each(STALLED_META_CASES)(
6771: "caller abort settles a stalled $protocol meta body",
6772: async ({ protocol, model, provider, upstream, wire }) => {
6773: const source = ...
A packages/gateway/src/codex-rate-limits.ts M packages/gateway/src/pipeline.ts M packages/gateway/src/recall.ts M packages/gateway/src/stream/openai-responses.ts M packages/gateway/src/translate/openai-responses.ts M packages/gateway/test/codex-usage-metadata.test.ts M packages/gateway/test/openai-responses-recall-aware-stream.test.ts M packages/gateway/test/pipeline-streaming.test.ts M packages/gateway/test/recall-buffered-transaction.test.ts M packages/gateway/test/recall-exhaustion.test.ts M packages/gateway/test/recall.test.ts
Found 14 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall.test.ts:
Line 23: isUsableRecallContinuation,
Line 331: test("does not contain the old soft wording (regression guard)", () => {
Line 1648: test("throws on content-type mismatch (JSON instead of SSE) — #511 regression guard", async () => {
Line 1650: // instead of SSE. Without the assertSSEResponse guard, parseSSEStream
Line 1916: // Mirror of the streaming guard: if the upstream returns JSON instead of
Line 3556: expect(isUsableRecallContinuation(response)).toBe(true);
Line 3600: expect(isUsableRecallContinuation(response)).toBe(false);
Line 3631: expect(isUsableRecallContinuation(response)).toBe(expected);
Line 3632: expect(isUsableRecallContinuation({ ...response, content: [] })).toBe(
Line 3642: expect(isUsableRecallContinuation({ ...response, stopReason })).toBe(
Line 3646: isUsableRecallContinuation({ ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall.test.ts</path>
<type>file</type>
<content>
3500: const replaced = replaceRecallWithMarker(resp);
3501: expect(replaced.content).toHaveLength(3);
3502: expect(replaced.content[0].type).toBe("text");
3503: expect(replaced.content[1].type).toBe("text");
3504: expect((replaced.content[1] as { text: string }).text).toBe(
3505: buildRecallMarker("find config", "project"),
3506: );
3507: expect(replaced.content[2].type).toBe("tool_use");
3508: expect((replaced.content[2] as GatewayToolUseBlock).name).toBe("Read");
3509: });
3510:
3511: test("returns same content when no recall present", () => {
3512: const resp = makeResponse([
3513: { type: "text", text: "hello" },
3514: { type: "tool_use", id: "toolu_1", name: "Read", input: {} },
3515: ]);
3516:
3517: const replaced = replaceRecallWithMarker(resp);
3518: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/pipeline-streaming.test.ts</path>
<type>file</type>
<content>
1200: {
1201: role: "user",
1202: content: [
1203: {
1204: type: "tool_result",
1205: toolUseId: "call_read",
1206: content: [{ type: "text", text: toolOutput }],
1207: },
1208: ],
1209: },
1210: ],
1211: });
1212:
1213: try {
1214: const response = await handleRequest(
1215: request({ "x-session-affinity": "legacy-affinity-session" }),
1216: loadLocalConfig(),
1217: );
1218:
1219: const body = await response.text();
1220: order.push("eof1");
1221: const secondResponse = await handleRequest(
1222: request({
1223: "x-lore-session-id": "stable-lore-session",
1224: "x-session-affinity": "legacy-affinity-session",
1225: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/pipeline-streaming.test.ts</path>
<type>file</type>
<content>
4250: "x-session-affinity": alias,
4251: },
4252: messages: [
4253: {
4254: role: "user",
4255: content: [
4256: {
4257: type: "text",
4258: text: "continue\n[lore:session-id=secret-marker]\n[lore:project=/secret/path]",
4259: },
4260: ],
4261: },
4262: ],
4263: }),
4264: loadLocalConfig(),
4265: );
4266: await response.text();
4267: expect(forwardedBodies.at(-1)).not.toContain("lore:session-id");
4268: expect(forwardedBodies.at(-1)).not.toContain("lore:project");
4269: } finally {
4270: setUpstreamInterceptor(undefined);
4271: await resetPipelineState();
4272: }
4273: });
4274:
4275: it("sanitizes upstream errors ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/pipeline-streaming.test.ts</path>
<type>file</type>
<content>
1: /**
2: * Pipeline streaming-response coverage (Anthropic conversation turns).
3: *
4: * The harness's replay interceptor now emits Anthropic SSE for streaming
5: * requests (see test/helpers/replay.ts), so a `stream: true` turn exercises
6: * the streaming path end-to-end: buildStreamingResponse parses the upstream
7: * SSE, forwards it to the client, and accumulates in parallel for
8: * postResponse storage.
9: */
10: import { describe, it, expect, afterEach, vi } from "vitest";
11: import {
12: distillation,
13: db,
14: getDailyCostForDay,
15: ltm,
16: loadSessionTracking,
17: saveSessionTracking,
18: temporal,
19: } from "@loreai/core";
20: import * as Sentry from "@sentry/bun";
21: import type { Harness } from "./helpers/harness";
22: import { createHarness } from "./helpers/harness";
23: import type { FixtureEntry } from ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/anthropic.ts: Line 544: export function parseAnthropicResponseJSON(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/anthropic.ts</path>
<type>file</type>
<content>
520:
521: // Restore all metadata params (temperature, top_p, stop_sequences, etc.)
522: for (const [key, value] of Object.entries(req.metadata)) {
523: body[key] = value;
524: }
525:
526: return {
527: url: "/v1/messages",
528: headers,
529: body,
530: };
531: }
532:
533: // ---------------------------------------------------------------------------
534: // buildAnthropicNonStreamResponse
535: // ---------------------------------------------------------------------------
536:
537: /**
538: * Parse an Anthropic-format response JSON back into a `GatewayResponse`.
539: *
540: * This is the inverse of `buildAnthropicNonStreamResponse`. Used when the
541: * pipeline returns Anthropic-format JSON that needs to be translated to
542: * another protocol (OpenAI Chat Completions, OpenAI Responses API). ...
diff --git a/packages/gateway/src/codex-rate-limits.ts b/packages/gateway/src/codex-rate-limits.ts
new file mode 100644
index 0000000000..6df1ed7587
--- /dev/null
+++ b/packages/gateway/src/codex-rate-limits.ts
@@ -0,0 +1,120 @@
+export const MAX_CODEX_RATE_LIMIT_EVENTS = 64;
+export const MAX_CODEX_RATE_LIMIT_BYTES = 16 * 1024;
+
+const encoder = new TextEncoder();
+
+const CATEGORY = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/;
+const DECIMAL = /^(?:0|[1-9][0-9]{0,23})(?:\.[0-9]{1,12})?$/;
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+ return value !== null && typeof value === "object" && !Array.isArray(value);
+}
+
+function category(value: unknown): string | undefined {
+ return typeof value === "string" && CATEGORY.test(value) ? value : undefined;
+}
+
+function rateLimit(value: unknown): Record<string, unknown> | undefined {
+ if (!isRecord(value)) return undefined;
+ const result: Record<string, unknown> = {};
+ if (
+ typeof value.used_percent === ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts</path>
<type>file</type>
<content>
1: import { afterEach, describe, expect, test, vi } from "vitest";
2: import { db, ltm, loadSessionTracking, temporal } from "@loreai/core";
3: import * as core from "@loreai/core";
4: import { loadConfig } from "../src/config";
5: import { clearAllCosts, getSessionCosts } from "../src/cost-tracker";
6: import {
7: accumulateNonStreamResponse,
8: buildStreamingResponse,
9: getActiveSessions,
10: handleRequest,
11: resetPipelineState,
12: setRecallPersistenceCommitObserverForTest,
13: setUpstreamInterceptor,
14: streamingPostResponsePendingForTest,
15: } from "../src/pipeline";
16: import type { GatewayRequest } from "../src/translate/types";
17: import { parseAnthropicResponseJSON } from "../src/translate/anthropic";
18: import {
19: _resetForTest as resetWorkerHealth,
20: _setNowForTest as setWorkerHealthTime,
21: ...
Found 24 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai.ts:
Line 87: strict?: boolean;
Line 188: const validator = opts.strict ? new AnthropicSSEValidator() : null;
Line 192: requireEventTerminator: opts.strict,
Line 193: fatalUtf8: opts.strict,
Line 194: maxFrames: opts.strict ? DEFAULT_MAX_SSE_FRAMES : undefined,
Line 195: maxEventBytes: opts.strict ? 4 * 1024 * 1024 : undefined,
Line 196: maxTotalBytes: opts.strict ? 4 * 1024 * 1024 : undefined,
Line 400: if (opts.strict || opts.propagateErrors) {
Line 477: export async function accumulateOpenAISSEStream(
Line 482: strict?: boolean;
Line 618: requireEventTerminator: opts.strict,
Line 619: fatalUtf8: opts.strict,
Line 620: maxFrames: opts.strict
Line 623: maxTotalBytes: opts.strict ? 4 * 1024 * 1024 : undefined,
Line 635: if ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai.ts</path>
<type>file</type>
<content>
460: });
461: }
462:
463: /**
464: * Accumulate a streaming OpenAI Chat Completions SSE response into a
465: * GatewayResponse.
466: *
467: * Reads EVERY `data:` chunk and merges the incremental `choices[0].delta`
468: * fields (text + tool-call fragments) into a single response — so a
469: * multi-chunk stream is reconstructed faithfully. This is the correct reader
470: * for a non-streaming request whose provider replied with SSE anyway (the
471: * ChatGPT/Copilot backend, DeepSeek): taking only the last `data:` line would
472: * drop all but the final delta.
473: *
474: * OpenAI SSE chunk shape:
475: * data: {"id":"...","choices":[{"delta":{"content":"..."},"finish_reason":null}]}
476: */
477: export async function accumulateOpenAISSEStream(
478: upstreamResponse: Response,
479: opts: {
480: signal?: AbortSignal;
481: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai.ts</path>
<type>file</type>
<content>
877: return {
878: id,
879: model,
880: content,
881: stopReason,
882: usage: {
883: // prompt_tokens is inclusive of cache reads/writes; subtract them to
884: // match the gateway's disjoint token convention (see
885: // disjointOpenAIInputTokens in llm-adapter.ts). Inlined here to keep this
886: // leaf stream module free of a cross-module import.
887: inputTokens: Math.max(
888: 0,
889: inputTokens - (cachedTokens ?? 0) - (cacheWriteTokens ?? 0),
890: ),
891: outputTokens,
892: cacheReadInputTokens: cachedTokens,
893: cacheCreationInputTokens: cacheWriteTokens,
894: },
895: };
896: }
(End of file - total 896 lines)
</content>
Found 60 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts:
Line 9: * message_start -> content_block_start/delta/stop (repeated) -> message_delta -> message_stop
Line 81: opts: { signal?: AbortSignal; inactivityMs?: number } = {},
Line 152: opts: {
Line 398: /** Whether the stream has completed (message_stop received). */
Line 492: * Rewrite usage fields in a `message_start` or `message_delta` SSE event
Line 502: if (eventType === "message_start") {
Line 560: case "message_start":
Line 575: case "message_stop":
Line 800: * Build a synthetic `message_start` SSE event from a GatewayResponse.
Line 808: type: "message_start",
Line 832: return formatSSEEvent("message_start", JSON.stringify(message));
Line 839: * message_start -> content_block_start -> content_block_delta ->
Line 840: * content_block_stop -> message_delta -> message_stop
Line 853: // ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts</path>
<type>file</type>
<content>
1950: assertDone(): void {
1951: if (!this.terminalSeen) {
1952: throw new Error("missing Anthropic message_stop terminal");
1953: }
1954: }
1955: }
1956:
1957: /**
1958: * Consume an Anthropic SSE streaming Response and return the accumulated
1959: * GatewayResponse. Useful when the response needs to be translated to another
1960: * protocol format (e.g. OpenAI) after the pipeline produces Anthropic SSE.
1961: */
1962: export async function accumulateSSEResponse(
1963: response: Response,
1964: opts: {
1965: signal?: AbortSignal;
1966: stopAtTerminal?: boolean;
1967: strict?: boolean;
1968: inactivityMs?: number;
1969: maxFrames?: number;
1970: onSemanticContent?: () => void;
1971: } = {},
1972: ): Promise<GatewayResponse> {
1973: const accumulator = createStreamAccumulator();
1974: let messageStarted = ...
Found 2 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts:
Line 1728: function validateAnthropicMessageStart(parsed: Record<string, unknown>): void {
Line 1765: function validateAnthropicMessageDelta(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts</path>
<type>file</type>
<content>
1690: case "tool_use":
1691: if (
1692: typeof block.id !== "string" ||
1693: typeof block.name !== "string" ||
1694: !isRecord(block.input)
1695: ) {
1696: malformedAnthropicStream();
1697: }
1698: break;
1699: case "server_tool_use":
1700: if (
1701: typeof block.id !== "string" ||
1702: typeof block.name !== "string" ||
1703: block.input === undefined
1704: ) {
1705: malformedAnthropicStream();
1706: }
1707: break;
1708: case "container_upload":
1709: if (typeof block.file_id !== "string") malformedAnthropicStream();
1710: break;
1711: case "web_search_tool_result":
1712: case "web_fetch_tool_result":
1713: case "code_execution_tool_result":
1714: case "bash_code_execution_tool_result":
1715: case ...
Found 20 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/gemini.ts: Line 43: * concatenated; `functionCall` parts arrive complete; `usageMetadata` and Line 44: * `finishReason` appear on the final frame(s). Line 46: export async function accumulateGeminiSSEStream( Line 51: strict?: boolean; Line 65: let finishReason: unknown; Line 122: candidate.finishReason !== undefined && Line 123: candidate.finishReason !== null && Line 124: typeof candidate.finishReason !== "string" Line 179: parsed.usageMetadata, Line 226: finishReason = promptFeedback.blockReason; Line 255: if (first.finishReason != null) finishReason = first.finishReason; Line 257: // streamGenerateContent reports cumulative usageMetadata on the same Line 258: // candidate frame that carries finishReason. The API has no OpenAI-style Line 263: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/gemini.ts</path>
<type>file</type>
<content>
35: } from "./anthropic";
36: import { isRecord, validateGeminiUsageMetadata } from "../usage-validation";
37:
38: type GeminiPart = Record<string, unknown>;
39:
40: /**
41: * Accumulate an upstream Gemini SSE (`?alt=sse`) response into a
42: * `GatewayResponse`. Text parts arrive as deltas across frames and are
43: * concatenated; `functionCall` parts arrive complete; `usageMetadata` and
44: * `finishReason` appear on the final frame(s).
45: */
46: export async function accumulateGeminiSSEStream(
47: upstreamResponse: Response,
48: opts: {
49: signal?: AbortSignal;
50: stopAtTerminal?: boolean;
51: strict?: boolean;
52: inactivityMs?: number;
53: maxFrames?: number;
54: onSemanticContent?: () => void;
55: onValidatedEvent?: (event: string, data: string) => void | Promise<void>;
56: } = {},
57: ): Promise<GatewayResponse> ...
Found 3 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts:
Line 926: function providerResponse(
Line 1029: function responsesStream(response: Record<string, unknown>): Response {
Line 1295: function request(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts</path>
<type>file</type>
<content>
880: ),
881: );
882: const response = await handleRequest(req, config());
883: expect(await response.text()).toContain("Completed answer");
884: await settled();
885: expect(stateFor(alias).recallStore.size).toBe(0);
886: expect(ltm.transferCount(id)).toBe(0);
887: });
888:
889: const query =
890: "transactional glacier orchard telescope cobalt lantern mercury compass velvet island";
891: let productiveRecallIds: string[] | undefined;
892:
893: /**
894: * Terminal-recall fixtures start with a search then use distinct detail reads,
895: * so they remain productive. They use the minimum legal emergency cap to test
896: * finalization without coupling the test runtime to the production default. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts</path>
<type>file</type>
<content>
1070: { response },
1071: ),
1072: { headers: { "content-type": "text/event-stream" } },
1073: );
1074: }
1075:
1076: function anthropicStream(json: Record<string, unknown>): Response {
1077: const event = (type: string, data: object) =>
1078: `event: ${type}\ndata: ${JSON.stringify({ type, ...data })}\n\n`;
1079: const usage = json.usage as Record<string, unknown>;
1080: return new Response(
1081: event("message_start", {
1082: message: {
1083: ...json,
1084: content: [],
1085: stop_reason: null,
1086: stop_sequence: null,
1087: usage: { input_tokens: usage.input_tokens, output_tokens: 0 },
1088: },
1089: }) +
1090: (json.content as Array<Record<string, unknown>>)
1091: .map(
1092: (block, index) =>
1093: ...
No files found
Found 28 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/foreground-body-limit.test.ts: Line 58: const pending = accumulateNonStreamResponse( Line 172: const result = await accumulateNonStreamResponse( Line 187: const result = await accumulateNonStreamResponse(response, "openai"); Line 205: accumulateNonStreamResponse( Line 233: accumulateNonStreamResponse( Line 268: accumulateNonStreamResponse(response, "openai-responses"), Line 300: type: "image_generation_call", Line 306: const result = await accumulateNonStreamResponse( Line 352: accumulateNonStreamResponse(response, "openai-responses"), Line 364: accumulateNonStreamResponse(response, "openai-responses", true), /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts: Line 375: accumulateNonStreamResponse( Line 400: accumulateNonStreamResponse( Line 677: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/foreground-body-limit.test.ts</path>
<type>file</type>
<content>
1: import { describe, expect, test } from "vitest";
2: import {
3: accumulateNonStreamResponse,
4: readForegroundBody,
5: } from "../src/pipeline";
6:
7: function chunkedResponse(chunks: number, chunkBytes: number): Response {
8: const chunk = new Uint8Array(chunkBytes).fill(120);
9: let sent = 0;
10: return new Response(
11: new ReadableStream<Uint8Array>({
12: pull(controller) {
13: if (sent++ < chunks) controller.enqueue(chunk);
14: else controller.close();
15: },
16: }),
17: );
18: }
19:
20: describe("foreground response body limits", () => {
21: test.each([
22: [
23: "anthropic",
24: 'event: message_start\ndata: ...
Found 6 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/llm-adapter.ts: Line 63: validateGeminiFunctionCallIdentity, Line 2128: validateGeminiFunctionCallIdentity( /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts: Line 48: export function validateGeminiFunctionCallIdentity( Line 125: validateGeminiFunctionCallIdentity( /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/gemini.ts: Line 28: validateGeminiFunctionCallIdentity, Line 170: validateGeminiFunctionCallIdentity(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts</path>
<type>file</type>
<content>
1: /**
2: * Google Gemini (native `generateContent`) ↔ Gateway translation layer.
3: *
4: * Converts between Google's native Generative Language API
5: * (`POST /v1beta/models/{model}:generateContent` /
6: * `:streamGenerateContent?alt=sse`) and the gateway's internal
7: * `GatewayRequest`/`GatewayResponse` types.
8: *
9: * This is a DISTINCT wire format from OpenAI/Anthropic (it is NOT the OpenAI
10: * compatibility layer at `/v1beta/openai/...`). Key differences:
11: * - roles are `user` / `model` (not `assistant`);
12: * - the system prompt lives in `systemInstruction` (a `{parts:[{text}]}`);
13: * - tool calls are `functionCall` parts; modern Gemini responses may include
14: * a per-call `id`. ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts: Line 485: export function parseGeminiResponseJSON(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts</path>
<type>file</type>
<content>
470: if (typeof um.cachedContentTokenCount === "number") {
471: usage.cacheReadInputTokens = cached;
472: }
473: return usage;
474: }
475:
476: /** Parse Gemini `usageMetadata` into `GatewayUsage`. */
477: function parseGeminiUsage(json: Record<string, unknown>): GatewayUsage {
478: return geminiUsageFromMetadata(json.usageMetadata);
479: }
480:
481: /**
482: * Parse a Gemini `generateContent` response JSON into a `GatewayResponse`.
483: * Mirrors `parseAnthropicResponseJSON`, over `candidates[0].content.parts[]`.
484: */
485: export function parseGeminiResponseJSON(
486: json: Record<string, unknown>,
487: ): GatewayResponse {
488: const candidates = Array.isArray(json.candidates) ? json.candidates : [];
489: validateGeminiCandidateToolIdentities(
490: candidates,
491: "malformed Gemini response tool identity",
492: );
493: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/usage-validation.ts</path>
<type>file</type>
<content>
1: /** Runtime validation for provider-reported token usage. */
2:
3: export function isRecord(value: unknown): value is Record<string, unknown> {
4: return !!value && typeof value === "object" && !Array.isArray(value);
5: }
6:
7: function invalid(message: string): never {
8: throw new Error(message);
9: }
10:
11: function tokenCount(
12: value: unknown,
13: message: string,
14: allowNull = false,
15: ): number | undefined {
16: if (value === undefined || (allowNull && value === null)) return undefined;
17: if (!Number.isSafeInteger(value) || (value as number) < 0) invalid(message);
18: return value as number;
19: }
20:
21: /** Add validated token counts without allowing a safe-integer overflow. ...
I’ll add focused regressions first, run them against the unchanged parent behavior, then change production code. After the fixes pass, I’ll remove each guard one at a time and rerun its named regression.
Success. Updated the following files: M ../opencode-lore-recall-recovery/packages/gateway/test/codex-usage-metadata.test.ts M ../opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts
Resume the exact recovery correction in /home/byk/Code/opencode-lore-recall-recovery. Your last output was only a Lore context warning, not a result. Preserve and inspect all current edits first (jj st, diff); do not discard or restart work. Recover context with lore recall --project /home/byk/Code/opencode-lore --scope project --limit 15 --json or GET local /ui/search; do not use broken recall tool. Complete every blocker from the prior assignment: malformed non-JSON quota frames; real 16KiB byte-cap discrimination; failed-retry quota rollback; disguised SSE strict recovery validation across OpenAI/Anthropic/Gemini; OpenAI usage required counts; bounded hostile error-body reads with fire-and-forget cancellation; output-item type/status registry compatibility; strict-envelope guard matrices. Preserve fail-first and independent guard-removal evidence; restore exact tree. Run exact former foreground-body-limit failures, all new tests, full affected recovery matrix, gateway typecheck, format. No commit/push/rebase/bookmark/server/network/production DB. Return substantive counts, files, residual risks; never warning-only/empty.
<skill_content name="jj-guide"> # Skill: jj-guide # jj Guide for AI Agents Jujutsu (jj) is a Git-compatible VCS with mutable commits, automatic snapshotting, no staging area, and first-class conflicts. This skill teaches you how to use it safely from a non-interactive agent environment. If `.jj/` exists in the repo root, this is a jj repo. **Use `jj` commands, not `git`.** In a colocated repo (`.jj/` *and* `.git/`), git tools can read the state, but mutations should go through `jj` so the operation log stays consistent. ## Critical Rules - **NEVER** use interactive flags (`-i`, `--interactive`). TUI prompts hang in agent environments. This applies to `jj split -i`, `jj squash -i`, `jj commit -i`, `jj resolve`, `jj diffedit`, etc. - **ALWAYS** pass `-m "msg"` when describing/committing. Without `-m`, an editor opens and hangs. - **VERIFY** mutations with `jj st` and `jj log` after `squash`, `abandon`, `rebase`, `restore`, `commit`. ...
The required repo-setup skill is unavailable in this environment. I loaded the mandatory Jujutsu guide and will preserve the current workspace.
Working copy changes: M packages/gateway/test/codex-usage-metadata.test.ts M packages/gateway/test/openai-responses-recall-aware-stream.test.ts Working copy (@) : pqvltmkr ac80b275 fix(recall): harden quota metadata Parent commit (@-): spyyzvyy 3f3ae768 fix/recall-failure-synthesis | fix(recall): validate recovery metadata Warning: These bookmarks have conflicts: fix/responses-private-references Hint: Use `jj bookmark list` to see details. Use `jj bookmark set <name> -r <rev>` to resolve.
Modified regular file packages/gateway/test/codex-usage-metadata.test.ts:
...
6 6: streamResponsesPassthrough,
7 7: } from "../src/stream/openai-responses";
8 8: import { buildOpenAIResponsesResponse } from "../src/translate/openai-responses";
9 :
10 : const MAX_CODEX_RATE_LIMIT_EVENTS = 64;
11 : const MAX_CODEX_RATE_LIMIT_BYTES = 16 * 1024;
9: import {
10: MAX_CODEX_RATE_LIMIT_BYTES,
11: MAX_CODEX_RATE_LIMIT_EVENTS,
12: } from "../src/codex-rate-limits";
12 13:
13 14: const limits = {
14 15: type: "codex.rate_limits",
...
62 63: .map((frame) => JSON.parse(frame.split("\ndata: ")[1]));
63 64: }
64 65:
66: function maximalQuota(index: number): Record<string, unknown> {
67: const category = (prefix: string) =>
68: `${prefix}_${String(index).padStart(3, "0")}_${"x".repeat(64)}`.slice(
69: 0,
70: 64,
71: );
...
{
"error": "UsageError",
"code": 20,
"message": "Please provide a search query.",
"try": "lore recall <query>"
}
[lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-536181/vec0.so)
{
"query": "recovery correction quota strict envelope hostile error body",
"scope": "project",
"projectPath": "/home/byk/Code/opencode-lore",
"result": "## Recall Results\n\nFound 45 results, showing top 15 (score range: 0.065–0.048).\n\n### Strong Matches\n\n#### Knowledge\n- **Recall body-read privacy** (10h ago): Trap: logging a caught response-body read error looks diagnostic, but `Error.message` may contain hostile provider content and is persisted to logs and sinks. Fix: streaming, JSON, and accumulated-SSE paths log only `recall follow-up error body could not be read`; JSON/SSE mismatches start best-effort fire-and-forget body cancellation before rejecting. Cancellation may reject or never settle, so cleanup must never delay the caller. ↳ files: packages/gateway/src/recall.ts, packages/gateway/test/recall.test.ts\n- **Recovery accounting transaction** (10h ago): Chose ...
/home/byk/Code/opencode-lore-recall-recovery/package.json /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/package.json
<path>/home/byk/Code/opencode-lore-recall-recovery/package.json</path>
<type>file</type>
<content>
1: {
2: "name": "lore-monorepo",
3: "private": true,
4: "type": "module",
5: "license": "FSL-1.1-Apache-2.0",
6: "description": "Monorepo root for Lore — three-tier memory architecture",
7: "main": "./packages/opencode/src/index.ts",
8: "exports": {
9: ".": "./packages/opencode/src/index.ts"
10: },
11: "packageManager": "pnpm@10.28.0",
12: "volta": {
13: "node": "24.16.0",
14: "pnpm": "10.28.0"
15: },
16: "pnpm": {
17: "onlyBuiltDependencies": [
18: "esbuild",
19: "onnxruntime-node",
20: "sharp"
21: ],
22: "overrides": {
23: "basic-ftp": "5.3.1",
24: "ip-address": "10.2.0",
25: "fast-xml-parser": "5.7.3",
26: "@huggingface/hub": "2.11.0",
27: "onnxruntime-node": "1.27.0",
28: "yaml": ">=2.8.3",
29: "esbuild": ">=0.28.1",
30: "@opentelemetry/core": ">=2.8.0",
31: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/package.json</path>
<type>file</type>
<content>
1: {
2: "name": "@loreai/gateway",
3: "version": "0.40.0",
4: "type": "module",
5: "license": "FSL-1.1-Apache-2.0",
6: "description": "Lore as a transparent LLM proxy — context management for any AI coding client",
7: "main": "./dist/index.cjs",
8: "types": "./dist/index.d.cts",
9: "exports": {
10: ".": {
11: "types": "./dist/index.d.cts",
12: "bun": "./dist/index.bun.js",
13: "require": "./dist/index.cjs",
14: "default": "./dist/index.cjs"
15: }
16: },
17: "bin": {
18: "lore": "./dist/bin.cjs",
19: "lore-gateway": "./dist/bin.cjs"
20: },
21: "scripts": {
22: "typecheck": "tsc --noEmit",
23: "build": "tsx script/build.ts",
24: "bundle": "tsx script/bundle.ts",
25: "build:binary": "tsx script/build.ts --binary",
26: "build:binary:sea": "tsx script/build-binary-sea.ts",
27: "start": ...
...output truncated... Full output saved to: /home/byk/.local/share/opencode/tool-output/tool_0ac02531b001xbdHKkeo89idWM } <shell_metadata> shell tool terminated command after exceeding timeout 300000 ms. If this command is expected to take longer and is not waiting for interactive input, retry with a larger timeout value in milliseconds. </shell_metadata>
<path>/home/byk/.local/share/opencode/tool-output/tool_0ac02531b001xbdHKkeo89idWM</path>
<type>file</type>
<content>
1: [lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-539055/vec0.so)
2: {
3: "query": "malformed non-JSON quota frames failed retry quota rollback disguised SSE strict recovery OpenAI usage required counts hostile error body output item registry guard matrices",
4: "scope": "project",
5: "projectPath": "/home/byk/Code/opencode-lore",
6: "result": "## Recall Results\n\nFound 45 results, showing top 15 (score range: 0.063–0.052).\n\n### Strong Matches\n\n#### Distilled\n- Date: Sep 16, 2026 * 🟡 (20:11) User asked to implement all accepted recovery correctness/security findings in `/home/byk/Code/opencode-lore-recall-recovery`, working in additive child change `pqvltmkr` over immutable parent `spyyzvyy/3f3ae768`. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/quality/REVIEW.md</path> <type>file</type> <content> 1: # Review Heuristics — Lore 2: 3: This document codifies the review bar for pull requests. It exists because 4: adversarial review has repeatedly caught correctness bugs that a green test 5: suite missed — skip/early-return branches, lifecycle sequences, and fan-out 6: registry gaps. Institutionalizing these patterns so they compound. 7: 8: The review workflow (§6) is referenced from AGENTS.md. 9: 10: --- 11: 12: ## 1. Regression-test discipline 13: 14: Every adversarial-review finding that surfaces a defect **must** land a 15: deterministic regression test in the same PR. The test: 16: - fails on the base branch (proving it reproduces the bug), 17: - passes on the fix (proving the fix works), 18: - drives the *real* precondition, not an artificially-constructed state (e.g. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/codex-rate-limits.ts</path>
<type>file</type>
<content>
1: export const MAX_CODEX_RATE_LIMIT_EVENTS = 64;
2: export const MAX_CODEX_RATE_LIMIT_BYTES = 16 * 1024;
3:
4: const encoder = new TextEncoder();
5:
6: const CATEGORY = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/;
7: const DECIMAL = /^(?:0|[1-9][0-9]{0,23})(?:\.[0-9]{1,12})?$/;
8:
9: function isRecord(value: unknown): value is Record<string, unknown> {
10: return value !== null && typeof value === "object" && !Array.isArray(value);
11: }
12:
13: function category(value: unknown): string | undefined {
14: return typeof value === "string" && CATEGORY.test(value) ? value : undefined;
15: }
16:
17: function rateLimit(value: unknown): Record<string, unknown> | undefined {
18: if (!isRecord(value)) return undefined;
19: const result: Record<string, unknown> = {};
20: if (
21: typeof value.used_percent === "number" &&
22: ...
RUN v4.1.8 /home/byk/Code/opencode-lore-recall-recovery
❯ packages/gateway/test/openai-responses-recall-aware-stream.test.ts (209 tests | 3 failed | 206 skipped) 250ms
× suppresses malformed non-JSON principal Codex quota events 142ms
× suppresses malformed non-JSON continuation Codex quota events 34ms
× bounds Codex quota metadata by bytes across the principal and continuation 67ms
⎯⎯⎯⎯⎯⎯⎯ Failed Tests 3 ⎯⎯⎯⎯⎯⎯⎯
FAIL packages/gateway/test/openai-responses-recall-aware-stream.test.ts > streamResponsesRecallAware > suppresses malformed non-JSON principal Codex quota events
AssertionError: expected 'event: response.created\ndata: {"type…' not to contain 'private-principal-quota-sentinel'
- Expected
+ Received
- private-principal-quota-sentinel
+ event: response.created
+ data: {"type":"response.created","response":{"id":"resp_malformed_principal_quota","model":"gpt-5.6-terra"},"sequence_number":0}
+
+ event: codex.rate_limits
+ data: ...
Found 43 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall.test.ts:
Line 32: buildRecallRecoveryRequest,
Line 36: runRecallRecovery,
Line 215: describe("executeRecall malformed input", () => {
Line 234: id: "recall-malformed",
Line 239: "malformed-input",
Line 276: test("logs malformed recall input through a fixed error envelope", async () => {
Line 677: "rejects query-keyed entry with malformed context hash %s",
Line 902: "builds an isolated %s recovery request without recall or forced controls",
Line 923: const recovery = buildRecallRecoveryRequest(
Line 941: expect(recovery).not.toBe(request);
Line 942: expect(recovery.tools.map((tool) => tool.name)).toEqual(["Read"]);
Line 943: expect(recovery.metadata).not.toHaveProperty("tool_choice");
Line 944: expect(recovery.metadata).not.toHaveProperty("toolConfig");
Line 945: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall.test.ts</path>
<type>file</type>
<content>
1200: );
1201: expect(followUp.messages[0].content).not.toContainEqual(
1202: expect.objectContaining({
1203: type: "opaque",
1204: raw: expect.objectContaining({ type: "item_reference" }),
1205: }),
1206: );
1207: const wire = buildOpenAIResponsesUpstreamRequest(
1208: followUp,
1209: "https://api.openai.com",
1210: ).body as { input: Array<Record<string, unknown>> };
1211: expect(wire.input).toContainEqual({
1212: type: "reasoning",
1213: id: "rs_1",
1214: encrypted_content: "encrypted",
1215: });
1216: expect(wire.input).not.toContainEqual(
1217: expect.objectContaining({ type: "item_reference" }),
1218: );
1219: });
1220:
1221: test("excludes text blocks but keeps thinking blocks", () => {
1222: const req = makeRequest([
1223: { role: "user", ...
Found 17 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/recall.ts:
Line 1191: export function isUsableRecallContinuation(resp: GatewayResponse): boolean {
Line 1474: async function readResponseTextLimited(
Line 1510: function assertJSONResponse(response: Response): void {
Line 1580: * and any divergence fails loud (see assertSSEResponse).
Line 1613: detail = await readResponseTextLimited(response, 500, signal);
Line 1633: * structurally impossible and any divergence fails loud (see assertJSONResponse).
Line 1652: return runRecallJSONRequest(ctx, followUp, signal);
Line 1655: async function runRecallJSONRequest(
Line 1674: detail = await readResponseTextLimited(response, 500, signal);
Line 1681: assertJSONResponse(response);
Line 1705: * to be SSE — so a flag/consumer mismatch fails loud (see assertSSEResponse). ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/recall.ts</path>
<type>file</type>
<content>
1140: | "vertex"
1141: | "gemini";
1142:
1143: /**
1144: * Injected upstream dependencies for recall follow-up execution.
1145: *
1146: * Passed by the pipeline so `recall.ts` never imports `pipeline.ts`
1147: * (avoids a circular dependency). `forward` wraps `forwardToUpstream`
1148: * — callers should disable conversation caching on the follow-up;
1149: * `parseJSON` wraps `accumulateNonStreamResponse`.
1150: */
1151: export interface RecallFollowUpCtx {
1152: /** Forward a follow-up request upstream and return the raw response. */
1153: forward: (
1154: req: GatewayRequest,
1155: signal?: AbortSignal,
1156: ) => Promise<{ response: Response; effectiveProtocol: RecallProtocol }>;
1157: /** Parse a non-streaming (JSON) upstream response into a GatewayResponse. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/scripts</path> <type>directory</type> <entries> bump-version.sh cch-scan.c check-cc-version.ts check-docs-links.mjs check-patches.ts check-preview-links.mjs check-social-meta.mjs extract-cch-seed.ts generate-config-docs.ts generate-env-docs.ts generate-release-checksums.mjs smoke-install.mjs (12 entries) </entries>
Found 73 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts:
Line 216: assertSuccessfulResponsesCompletion,
Line 3860: * rendered and the agent loop exits with no auto-recovery.
Line 4552: // permanently strand the bucket data with no retry. Keeping the flag set
Line 4588: // next confident turn could never retry and the old rows would remain
Line 4706: `self-heal re-attribution failed (${fromPath} → ${toPath}); will retry on next confident turn:`,
Line 5813: // it in postResponse. This preserves retry continuity without authorizing
Line 6145: // Preserve retry continuity in memory, but do not authorize the learned
Line 6376: retry: (signal?: AbortSignal) => Promise<Response>;
Line 6734: return { response, retry: dispatch, serializedBody, effectiveProtocol };
Line 7634: if (!isUsableRecallContinuation(continuationResp))
Line 7845: retryPrincipal?: (input: {
Line ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
18200: log.warn(
18201: `prompt overflow: escalating to layer ${escalateLayer} for session ${sessionID.slice(0, 16)}` +
18202: ` (ratio=${overshootRatio.toFixed(2)})`,
18203: );
18204: }
18205:
18206: captureToolPairing400({
18207: status: upstreamResponse.status,
18208: errorBody,
18209: messages: modifiedReq.messages,
18210: layer: result.layer,
18211: model: req.model,
18212: sessionID,
18213: });
18214:
18215: genAiSpan.setStatus({
18216: code: 2,
18217: message: `HTTP ${upstreamResponse.status}`,
18218: });
18219: endGenAiSpan();
18220: return finishForeground(sanitizedUpstreamErrorResponse(upstreamResponse));
18221: }
18222:
18223: // Run the recall-interception loop over an already-accumulated
18224: // (internal Anthropic-format) GatewayResponse ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10550: detailLimit: recall.detailLimit,
10551: outputIndex: recall.outputIndex,
10552: toolUseId: recall.toolUseId,
10553: contentPosition,
10554: acc: recallAcc,
10555: signal,
10556: });
10557: } catch (error) {
10558: if (signal.aborted) throw error;
10559: if (error instanceof RecallContinuationFailure) throw error;
10560: throw new RecallContinuationFailure("recall_execution");
10561: }
10562: anchorTexts.push(executed.anchorText);
10563: if (executed.commit) pendingCommits.push(executed.commit);
10564: if (executed.rollback) {
10565: ...
Found 18 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts:
Line 7900: const publicCodexRateLimits: Array<Record<string, unknown>> = [];
Line 8034: let transactionProviderUsage: GatewayUsage = { ...ZERO_USAGE };
Line 10266: publicCodexRateLimits,
Line 10513: transactionProviderUsage = { ...ZERO_USAGE };
Line 10624: let continuationRetryBaseline = {
Line 10874: publicCodexRateLimits,
Line 11115: continuationRetryBaseline.transactionalEvents;
Line 11117: continuationRetryBaseline.transactionalBytes;
Line 11119: continuationRetryBaseline.retainedStateBytes;
Line 11121: continuationRetryBaseline.hiddenRecallBytes;
Line 11123: for (const identity of continuationRetryBaseline.outputIdentities) {
...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
7830: maxDeferredBytes?: number;
7831: maxHiddenRecallBytes?: number;
7832: maxRetainedStateBytes?: number;
7833: maxStreamBytes?: number;
7834: maxSSEFrames?: number;
7835: validation?: "public" | "codex";
7836: /** Caller abort combined with the stream's client-disconnect controller. */
7837: signal?: AbortSignal;
7838: /** Absolute request deadline inherited from the foreground abort scope. */
7839: recallDeadlineAt?: number;
7840: /**
7841: * Reissue the byte-stable principal request after a pre-output body-read
7842: * failure. The caller must retain the original transformed request, route,
7843: * credentials, and abort deadline.
7844: */
7845: retryPrincipal?: (input: {
7846: attempt: number;
7847: signal: AbortSignal;
7848: }) => Promise<Response>;
7849: /** Test-only ...
No files found
Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts:
Line 181: `event: codex.rate_limits\ndata: ${sentinel}\n\n`;
Line 189: return sseEvent("codex.rate_limits", {
Line 193: rate_limits: {
Line 270: test("suppresses malformed non-JSON principal Codex quota events", async () => {
Line 271: const sentinel = "private-principal-quota-sentinel";
Line 274: created("resp_malformed_principal_quota", "gpt-5.6-terra"),
Line 277: completed("resp_malformed_principal_quota", {
Line 295: expect(out).not.toContain("event: codex.rate_limits");
Line 298: test("suppresses malformed non-JSON continuation Codex quota events", async () => {
Line 299: const sentinel = "private-continuation-quota-sentinel";
Line 302: created("resp_malformed_continuation_quota", "gpt-5.6-terra"),
Line 303: recallCall(0, { query: "quota ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
6000: },
6001: );
6002:
6003: expect(await drain(client)).toContain(PUBLIC_RECALL_ERROR);
6004: expect(failures).toEqual(["resource_limit"]);
6005: });
6006:
6007: test("caps client-visible output across chained continuations", async () => {
6008: const failures: RecallContinuationFailureCategory[] = [];
6009: setRecallContinuationFailureHook((category) => failures.push(category));
6010: const firstEvent = `event: extension\ndata: ${"a".repeat(600 * 1024)}\n\n`;
6011: const secondEvent = `event: extension\ndata: ${"b".repeat(600 * 1024)}\n\n`;
6012: const firstFollowUp = streamFrom([
6013: created("resp_spool_first", "gpt-5.6-terra"),
6014: firstEvent,
6015: recallCall(
6016: 1,
6017: { query: "second spool" },
6018: "fc_spool_second",
6019: ...
Found 3 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts: Line 11976: export async function accumulateNonStreamResponse( Line 12021: requireSuccessfulCompletion: requireValidCompletion, Line 18472: requireSuccessfulCompletion: recovery,
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path> <type>file</type> <content> 11800: connection: "keep-alive", 11801: }, 11802: }); 11803: } 11804: 11805: /** 11806: * Accumulate a non-streaming upstream response into a GatewayResponse. 11807: * 11808: * Dispatches to the correct parser based on the upstream wire protocol: 11809: * - "anthropic": Anthropic Messages API format 11810: * - "openai": OpenAI Chat Completions API format 11811: * - "openai-responses": OpenAI Responses API format 11812: */ 11813: const MAX_FOREGROUND_RESPONSE_BYTES = 4 * 1024 * 1024; 11814: const MAX_FOREGROUND_ERROR_BYTES = 64 * 1024; 11815: const FOREGROUND_SSE_INACTIVITY_MS = 120_000; 11816: // A gateway-owned reason stays distinct from provider token-limit reasons and 11817: // maps to OpenCode's retryable `unknown` finish, preserving its agent loop. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
12345: }
12346:
12347: if (
12348: json.type !== "message" ||
12349: json.role !== "assistant" ||
12350: typeof json.id !== "string" ||
12351: !json.id ||
12352: typeof json.model !== "string" ||
12353: !json.model ||
12354: !Array.isArray(json.content) ||
12355: typeof json.stop_reason !== "string" ||
12356: !json.usage ||
12357: typeof json.usage !== "object" ||
12358: Array.isArray(json.usage)
12359: ) {
12360: throw new Error("upstream Anthropic request did not complete");
12361: }
12362: validateAnthropicUsage(json.usage, {
12363: message: "malformed Anthropic usage",
12364: required: true,
12365: requireInput: true,
12366: requireOutput: true,
12367: });
12368: if (
12369: ![
12370: "end_turn",
12371: "tool_use",
12372: "max_tokens",
12373: ...
Found 11 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/foreground-body-limit.test.ts:
Line 214: message: "upstream response did not complete",
Line 242: message: "upstream response did not complete",
Line 269: ).rejects.toThrow("upstream Responses request did not complete");
Line 353: ).rejects.toThrow("upstream Responses request did not complete");
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/worker-google-openai-path.test.ts:
Line 72: test("accumulates a MULTI-CHUNK SSE reply (mislabeled application/json) into the full text", async () => {
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/worker-codex-sse-path.test.ts:
Line 98: test("mislabeled SSE (application/json) → accumulates full text, does NOT throw", async () => {
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/gemini-translate.test.ts:
Line 609: test("non-stream → application/json; stream → ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/foreground-body-limit.test.ts</path>
<type>file</type>
<content>
1: import { describe, expect, test } from "vitest";
2: import {
3: accumulateNonStreamResponse,
4: readForegroundBody,
5: } from "../src/pipeline";
6:
7: function chunkedResponse(chunks: number, chunkBytes: number): Response {
8: const chunk = new Uint8Array(chunkBytes).fill(120);
9: let sent = 0;
10: return new Response(
11: new ReadableStream<Uint8Array>({
12: pull(controller) {
13: if (sent++ < chunks) controller.enqueue(chunk);
14: else controller.close();
15: },
16: }),
17: );
18: }
19:
20: describe("foreground response body limits", () => {
21: test.each([
22: [
23: "anthropic",
24: 'event: message_start\ndata: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts</path>
<type>file</type>
<content>
1: import { afterEach, describe, expect, test, vi } from "vitest";
2: import { db, ltm, loadSessionTracking, temporal } from "@loreai/core";
3: import * as core from "@loreai/core";
4: import { loadConfig } from "../src/config";
5: import { clearAllCosts, getSessionCosts } from "../src/cost-tracker";
6: import {
7: accumulateNonStreamResponse,
8: buildStreamingResponse,
9: getActiveSessions,
10: handleRequest,
11: resetPipelineState,
12: setRecallPersistenceCommitObserverForTest,
13: setUpstreamInterceptor,
14: streamingPostResponsePendingForTest,
15: } from "../src/pipeline";
16: import type { GatewayRequest } from "../src/translate/types";
17: import { parseAnthropicResponseJSON } from "../src/translate/anthropic";
18: import {
19: _resetForTest as resetWorkerHealth,
20: _setNowForTest as setWorkerHealthTime,
21: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts</path>
<type>file</type>
<content>
410: test.each([
411: "anthropic",
412: "vertex",
413: "gemini",
414: "openai-responses",
415: "codex",
416: ] as const)("rolls back malformed %s recovery synthesis", async (kind) => {
417: const knowledgeId = knowledge();
418: const alias = crypto.randomUUID();
419: const client =
420: kind === "gemini"
421: ? "gemini"
422: : kind === "openai-responses" || kind === "codex"
423: ? "openai-responses"
424: : "anthropic";
425: const req = request(client, alias, kind === "codex");
426: if (kind === "vertex") {
427: req.rawHeaders["x-lore-provider"] = "vertex";
428: req.rawHeaders["x-lore-upstream-url"] = "https://aiplatform.googleapis.com";
429: const cfg = config();
430: cfg.vertexProject = "test-vertex-project";
431: cfg.vertexRegion = "global";
432: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10170: continue;
10171: }
10172: if (parsed.type !== event) {
10173: throw new Error(`Responses payload type does not match ${event}`);
10174: }
10175: if (
10176: (event === "response.output_item.added" ||
10177: event === "response.output_item.done") &&
10178: (parsed.item as Record<string, unknown> | undefined)?.type ===
10179: "function_call" &&
10180: (parsed.item as Record<string, unknown>).name === RECALL_TOOL_NAME
10181: ) {
10182: recallDetected = true;
10183: }
10184: const normalizationState = normalizeCodexEvent(
10185: state,
10186: event,
10187: parsed,
10188: );
10189: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10080: };
10081: // Recall items are gateway-internal and must stay hidden on every exit,
10082: // including failures raised before marker replacement.
10083: const recallIndices = new Set<number>();
10084: const unresolvedToolIndices = new Set<number>();
10085: const referenceIndices = new Map<number, ReferenceLifecycle>();
10086:
10087: const retainedStateBaseline = retainedStateBytes;
10088: const hiddenRecallBaseline = hiddenRecallBytes;
10089: const runPrincipalAttempt = async (): Promise<void> => {
10090: principalReadFinished = false;
10091: if (!currentPrincipalResponse.body) {
10092: throw new Error("Upstream response has no body");
10093: }
10094: const reader = currentPrincipalResponse.body.getReader();
10095: ...
Found 12 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/llm-adapter.ts: Line 5205: : AbortSignal.timeout(options.candidateTimeoutMs); Line 5309: : AbortSignal.timeout(options.candidateTimeoutMs); /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/quota.ts: Line 221: signal: AbortSignal.timeout(QUOTA_FETCH_TIMEOUT_MS), /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts: Line 1032: foregroundErrorBodyTimeoutMs = FOREGROUND_ERROR_BODY_TIMEOUT_MS; Line 11819: const FOREGROUND_ERROR_BODY_TIMEOUT_MS = 10_000; Line 11821: let foregroundErrorBodyTimeoutMs = FOREGROUND_ERROR_BODY_TIMEOUT_MS; Line 11825: foregroundErrorBodyTimeoutMs = timeoutMs ?? FOREGROUND_ERROR_BODY_TIMEOUT_MS; Line 18158: AbortSignal.timeout(foregroundErrorBodyTimeoutMs), /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/cli/inventory.ts: Line 644: signal: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts</path> <type>file</type> <content> 1: /** 2: * OpenAI Responses API SSE stream accumulator. 3: * 4: * Parses upstream Responses API streaming events and accumulates the full 5: * response into a `GatewayResponse`. The Responses API uses a different 6: * SSE event lifecycle than Anthropic: 7: * 8: * response.created → response.in_progress → 9: * response.output_item.added → response.output_text.delta (repeated) → 10: * response.output_item.done → response.function_call_arguments.delta → 11: * response.function_call_arguments.done → 12: * response.completed 13: * 14: * Reuses `parseSSEStream` from the Anthropic stream module since the 15: * underlying SSE wire format is the same. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
70: }
71:
72: const created = (id: string, model: string) =>
73: sseEvent("response.created", { response: { id, model } });
74:
75: const recallCall = (
76: outputIndex: number,
77: args: Record<string, unknown>,
78: itemId = `fc_${outputIndex}`,
79: callId = `call_${outputIndex}`,
80: ) =>
81: sseEvent("response.output_item.added", {
82: output_index: outputIndex,
83: item: {
84: type: "function_call",
85: id: itemId,
86: call_id: callId,
87: name: "recall",
88: },
89: }) +
90: sseEvent("response.function_call_arguments.done", {
91: output_index: outputIndex,
92: item_id: itemId,
93: arguments: JSON.stringify(args),
94: }) +
95: sseEvent("response.output_item.done", {
96: output_index: outputIndex,
97: item: {
98: type: "function_call",
99: id: ...
Found 6 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts:
Line 889: const query =
Line 926: function providerResponse(
Line 1029: function responsesStream(response: Record<string, unknown>): Response {
Line 1295: function request(
Line 1627: function config() {
Line 1661: async function settled() {
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts</path>
<type>file</type>
<content>
880: ),
881: );
882: const response = await handleRequest(req, config());
883: expect(await response.text()).toContain("Completed answer");
884: await settled();
885: expect(stateFor(alias).recallStore.size).toBe(0);
886: expect(ltm.transferCount(id)).toBe(0);
887: });
888:
889: const query =
890: "transactional glacier orchard telescope cobalt lantern mercury compass velvet island";
891: let productiveRecallIds: string[] | undefined;
892:
893: /**
894: * Terminal-recall fixtures start with a search then use distinct detail reads,
895: * so they remain productive. They use the minimum legal emergency cap to test
896: * finalization without coupling the test runtime to the production default. ...
Found 9 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts:
Line 257: finishReason: "STOP",
Line 336: (value.candidates as Array<Record<string, unknown>>)[0].finishReason =
Line 469: finishReason: "STOP",
Line 486: finishReason: "FAILED",
Line 668: candidates: [{ content: { parts: [call, call] }, finishReason: "STOP" }],
Line 1029: function responsesStream(response: Record<string, unknown>): Response {
Line 1076: function anthropicStream(json: Record<string, unknown>): Response {
Line 1274: expect(received).not.toMatch(/"finish_reason":"|"finishReason":/);
Line 1375: : wire.includes('"finishReason":')
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts</path>
<type>file</type>
<content>
1330: "x-lore-provider":
1331: protocol === "anthropic"
1332: ? "anthropic"
1333: : protocol === "openai"
1334: ? "vllm"
1335: : "openai",
1336: "x-lore-upstream-url":
1337: protocol === "anthropic"
1338: ? "https://api.anthropic.com"
1339: : "https://api.openai.com/v1",
1340: },
1341: };
1342: }
1343:
1344: describe.each(["anthropic", "openai", "openai-responses", "gemini"] as const)(
1345: "native Anthropic recall transaction for %s client",
1346: (client) => {
1347: test.each([false, true])(
1348: "mixed handoff waits for EOF (cancel=%s)",
1349: async (cancel) => {
1350: const id = knowledge();
1351: const alias = crypto.randomUUID();
1352: const req = request(client, alias);
1353: req.stream ...
Found 50 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai.ts:
Line 33: import { safeTokenSum, validateOpenAIUsage } from "../usage-validation";
Line 87: strict?: boolean;
Line 115: usage?: Record<string, unknown>,
Line 130: if (usage) {
Line 131: chunk.usage = usage;
Line 188: const validator = opts.strict ? new AnthropicSSEValidator() : null;
Line 192: requireEventTerminator: opts.strict,
Line 193: fatalUtf8: opts.strict,
Line 194: maxFrames: opts.strict ? DEFAULT_MAX_SSE_FRAMES : undefined,
Line 195: maxEventBytes: opts.strict ? 4 * 1024 * 1024 : undefined,
Line 196: maxTotalBytes: opts.strict ? 4 * 1024 * 1024 : undefined,
Line 352: // Build usage from accumulator
Line 354: const ru = resp.usage ?? ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai.ts</path>
<type>file</type>
<content>
470: * for a non-streaming request whose provider replied with SSE anyway (the
471: * ChatGPT/Copilot backend, DeepSeek): taking only the last `data:` line would
472: * drop all but the final delta.
473: *
474: * OpenAI SSE chunk shape:
475: * data: {"id":"...","choices":[{"delta":{"content":"..."},"finish_reason":null}]}
476: */
477: export async function accumulateOpenAISSEStream(
478: upstreamResponse: Response,
479: opts: {
480: signal?: AbortSignal;
481: stopAtTerminal?: boolean;
482: strict?: boolean;
483: inactivityMs?: number;
484: maxFrames?: number;
485: onSemanticContent?: () => void;
486: consumeUntilDone?: boolean;
487: onValidatedEvent?: (event: string, data: string) => void | Promise<void>;
488: } = {},
489: ): Promise<GatewayResponse> {
490: let id = "";
491: let model = "";
492: let stopReason ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts: Line 361: export function cancelAndReleaseReader(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts</path>
<type>file</type>
<content>
330: `SSE event exceeded ${maxEventBytes} byte limit`,
331: );
332: }
333:
334: if (done) {
335: const buffer = bufferParts.join("");
336: // Flush any remaining partial block (shouldn't happen with well-formed SSE)
337: if (buffer.trim()) {
338: countFrame();
339: if (opts.requireEventTerminator) {
340: throw new Error("unterminated SSE event at EOF");
341: }
342: let eventType = "message";
343: const dataLines: string[] = [];
344: for (const line of buffer.split(/\r\n|\r|\n/)) {
345: if (line.startsWith("event:")) {
346: eventType = line.slice(6).trim();
347: } else if (line.startsWith("data:")) {
348: dataLines.push(line.slice(5).trimStart());
349: }
350: }
351: if (dataLines.length > 0) ...
Found 2 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts:
Line 11083: (ce !== "codex.rate_limits" || publicCodexRateLimit)
Line 11490: if (event === "codex.rate_limits" && !publicCodexRateLimit) {
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
11445: transactionSettled = true;
11446: pendingCommits.length = 0;
11447: transactionRollbacks.length = 0;
11448: transactionBaseline = undefined;
11449: } catch (error) {
11450: pendingCommits.length = 0;
11451: transaction.rollback();
11452: throw error;
11453: }
11454: },
11455: rollback: () => {
11456: if (transactionSettled) return;
11457: transactionSettled = true;
11458: pendingCommits.length = 0;
11459: rollbackTransaction();
11460: },
11461: };
11462: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
1: /**
2: * Unit tests for `streamResponsesRecallAware` — the true-streaming,
3: * recall-aware OpenAI Responses (codex/ChatGPT) streamer.
4: *
5: * Regression for the "Provider response headers timed out after 10000ms" issue:
6: * the buffered `accumulateResponsesSSEStream` path withholds ALL client bytes
7: * until the entire (slow, reasoning-heavy) upstream completes, so opencode's
8: * 10s `ProviderHeaderTimeoutError` fired on ChatGPT sessions. This streamer
9: * forwards events live while transparently intercepting a `recall`
10: * function_call (emit marker, run follow-up, rebuild the terminal
11: * `response.completed`).
12: */
13: import { log } from "@loreai/core";
14: import { afterEach, describe, test, expect } from "vitest";
15: import { streamResponsesRecallAware } from "../src/pipeline";
16: import {
17: ...
Found 12 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts:
Line 1808: /** Incremental strict validator shared by buffered and true-streaming paths. */
Line 1962: export async function accumulateSSEResponse(
Line 1967: strict?: boolean;
Line 1981: const strictValidator = opts.strict ? new AnthropicSSEValidator() : undefined;
Line 1989: requireEventTerminator: opts.strict,
Line 1990: fatalUtf8: opts.strict,
Line 1991: maxEventBytes: opts.strict ? undefined : Number.POSITIVE_INFINITY,
Line 1992: maxFrames: opts.strict
Line 1995: maxTotalBytes: opts.strict ? 4 * 1024 * 1024 : undefined,
Line 1997: strictValidator?.process(event, data);
Line 1998: if (opts.strict) {
Line 2182: if (opts.stopAtTerminal) strictValidator?.assertDone();
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts</path>
<type>file</type>
<content>
1960: * protocol format (e.g. OpenAI) after the pipeline produces Anthropic SSE.
1961: */
1962: export async function accumulateSSEResponse(
1963: response: Response,
1964: opts: {
1965: signal?: AbortSignal;
1966: stopAtTerminal?: boolean;
1967: strict?: boolean;
1968: inactivityMs?: number;
1969: maxFrames?: number;
1970: onSemanticContent?: () => void;
1971: } = {},
1972: ): Promise<GatewayResponse> {
1973: const accumulator = createStreamAccumulator();
1974: let messageStarted = false;
1975: let messageDeltaPhase = false;
1976: let terminalStopReason: string | null = null;
1977: const activeBlocks = new Set<number>();
1978: const seenBlocks = new Set<number>();
1979: const blockTypes = new Map<number, string>();
1980: const toolUseIds = new Set<string>();
1981: const strictValidator = opts.strict ? ...
Found 18 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/gemini.ts:
Line 46: export async function accumulateGeminiSSEStream(
Line 51: strict?: boolean;
Line 69: let terminalSeen = false;
Line 77: requireEventTerminator: opts.strict,
Line 78: fatalUtf8: opts.strict,
Line 79: maxFrames: opts.strict
Line 82: maxTotalBytes: opts.strict ? 4 * 1024 * 1024 : undefined,
Line 89: if (opts.strict) throw new Error("malformed Gemini stream event");
Line 94: if (opts.strict) throw new Error("malformed Gemini stream event");
Line 98: if (opts.strict) {
Line 198: opts.strict &&
Line 259: // legal usage-only frame after that terminal, so stopAtTerminal
Line 274: terminalSeen = true;
Line 282: if (opts.stopAtTerminal && !terminalSeen) {
Line 283: throw new Error("missing Gemini finishReason terminal");
Line 317: opts: { strict?: boolean; ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/gemini.ts</path>
<type>file</type>
<content>
40: /**
41: * Accumulate an upstream Gemini SSE (`?alt=sse`) response into a
42: * `GatewayResponse`. Text parts arrive as deltas across frames and are
43: * concatenated; `functionCall` parts arrive complete; `usageMetadata` and
44: * `finishReason` appear on the final frame(s).
45: */
46: export async function accumulateGeminiSSEStream(
47: upstreamResponse: Response,
48: opts: {
49: signal?: AbortSignal;
50: stopAtTerminal?: boolean;
51: strict?: boolean;
52: inactivityMs?: number;
53: maxFrames?: number;
54: onSemanticContent?: () => void;
55: onValidatedEvent?: (event: string, data: string) => void | Promise<void>;
56: } = {},
57: ): Promise<GatewayResponse> {
58: if (!upstreamResponse.body) {
59: throw new Error("Upstream response has no body");
60: }
61:
62: let textContent = "";
63: let thinkingContent = ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/gemini.ts</path>
<type>file</type>
<content>
286: const blocks: GatewayContentBlock[] = [];
287: if (thinkingContent)
288: blocks.push({ type: "thinking", thinking: thinkingContent });
289: if (textContent) blocks.push({ type: "text", text: textContent });
290: for (const tu of toolUses) {
291: blocks.push({
292: type: "tool_use",
293: id: tu.id,
294: name: tu.name,
295: input: tu.input,
296: });
297: }
298:
299: return {
300: id: responseId,
301: model,
302: content: blocks,
303: stopReason: mapGeminiFinishReason(finishReason, toolUses.length > 0),
304: usage,
305: };
306: }
307:
308: /**
309: * Translate an internal Anthropic SSE stream into a Gemini SSE `Response`.
310: *
311: * Buffers via the shared Anthropic stream accumulator, then emits a single
312: * aggregated Gemini `data: <json>\n\n` frame. ...
[lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-551180/vec0.so)
{
"query": "readResponseTextLimited short dedicated bounded timeout hostile cancellation never settles recovery",
"scope": "project",
"projectPath": "/home/byk/Code/opencode-lore",
"result": "## Recall Results\n\nFound 45 results, showing top 15 (score range: 0.063–0.050).\n\n### Strong Matches\n\n#### Knowledge\n- **Foreground cancellation and bounded hostile I/O cleanup** (2d ago): Race recall, follow-up setup, provider work, and every body read against the foreground abort/deadline; byte caps cannot bound a reader that never settles. On cancellation, return promptly and asynchronously roll back late results or cancel/release late readers without awaiting hostile cleanup. `readForegroundBody` must bound non-OK diagnostic reads by timeout and abort, always release the reader, sanitize content, preserve status, and clamp far-future HTTP-date `Retry-After` to 300 seconds. ...
Found 9 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts:
Line 943: parsed.sequence_number !== undefined &&
Line 944: (!Number.isSafeInteger(parsed.sequence_number) ||
Line 945: (parsed.sequence_number as number) < 0)
Line 1668: parsed.sequence_number !== undefined &&
Line 1669: (!Number.isSafeInteger(parsed.sequence_number) ||
Line 1670: (parsed.sequence_number as number) < 0)
Line 1767: export function normalizeCodexResponsesEvent(
Line 2010: if (opts.validation && parsed.sequence_number !== undefined) {
Line 2011: const sequenceNumber = parsed.sequence_number as number;
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts</path> <type>file</type> <content> 1740: if (typeof parsed.text !== "string") malformedResponsesEvent(); 1741: normalizeCodexDataEvent(state, event, parsed, maxSparseIndex); 1742: break; 1743: case "response.refusal.delta": 1744: if (typeof parsed.delta !== "string") malformedResponsesEvent(); 1745: normalizeCodexDataEvent(state, event, parsed, maxSparseIndex); 1746: break; 1747: case "response.refusal.done": 1748: if (typeof parsed.refusal !== "string") malformedResponsesEvent(); 1749: normalizeCodexDataEvent(state, event, parsed, maxSparseIndex); 1750: break; 1751: case "response.function_call_arguments.delta": 1752: if (typeof parsed.delta !== "string") malformedResponsesEvent(); 1753: normalizeCodexDataEvent(state, event, parsed, maxSparseIndex); 1754: break; 1755: case ...
Found 3 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts:
Line 612: export function applyResponsesEvent(
Line 618: case "codex.rate_limits": {
Line 2380: if (event !== "codex.rate_limits" || acceptedCodexRateLimit) {
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
590: stopReason: "end_turn",
591: usage: { inputTokens: 0, outputTokens: 0 },
592: items: new Map(),
593: rawItems: new Map(),
594: itemIndexById: new Map(),
595: callIndexById: new Map(),
596: effectiveToolIndexById: new Map(),
597: nextOutputIndex: 0,
598: activeTextItems: new Set(),
599: activeToolItems: new Set(),
600: unboundTextItems: new Set(),
601: unboundToolItems: new Set(),
602: textDoneItems: new Set(),
603: refusalDoneItems: new Set(),
604: argumentDoneItems: new Set(),
605: };
606: }
607:
608: /**
609: * Apply one parsed Responses SSE event to the accumulation state. Never touches
610: * I/O — safe to call while forwarding the same event verbatim to the client. ...
Found 13 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts:
Line 9838: function buildTerminal(res: GatewayResponse): string {
Line 11434: encoder.encode(buildTerminal(visibleResp)),
Line 18246: resp.codexRateLimits ?? [],
Line 18295: ? { codexRateLimits: cumulativeCodexRateLimits }
Line 18434: if (error.response.codexRateLimits?.length) {
Line 18435: for (const quota of error.response.codexRateLimits) {
Line 18496: if (recovered.codexRateLimits?.length) {
Line 18497: for (const quota of recovered.codexRateLimits) {
Line 18521: recovered.codexRateLimits = cumulativeCodexRateLimits;
Line 18620: if (continuationResp.codexRateLimits?.length) {
Line 18621: for (const quota of continuationResp.codexRateLimits) {
Line 18632: continuationResp.codexRateLimits = cumulativeCodexRateLimits;
Line 18654: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
9800: type: "response.output_text.done",
9801: item_id: itemId,
9802: output_index: outputIndex,
9803: content_index: 0,
9804: text,
9805: }),
9806: ) +
9807: formatResponsesEvent(
9808: "response.content_part.done",
9809: JSON.stringify({
9810: type: "response.content_part.done",
9811: item_id: itemId,
9812: output_index: outputIndex,
9813: content_index: 0,
9814: part: { type: "output_text", text, annotations: [] },
9815: }),
9816: ) +
9817: formatResponsesEvent(
9818: "response.output_item.done",
9819: JSON.stringify({
9820: type: "response.output_item.done",
9821: output_index: outputIndex,
9822: item: {
9823: type: "message",
9824: id: ...
Found 2 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/openai-responses.ts: Line 1283: resp.codexRateLimits ?? [], /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/types.ts: Line 349: codexRateLimits?: Array<Record<string, unknown>>;
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/openai-responses.ts</path>
<type>file</type>
<content>
1260: encoder.encode(
1261: `event: ${eventType}\ndata: ${JSON.stringify(data)}\n\n`,
1262: ),
1263: );
1264: }
1265:
1266: // response.created
1267: emit("response.created", {
1268: type: "response.created",
1269: response: {
1270: id: respId,
1271: object: "response",
1272: created_at: created,
1273: model: resp.model,
1274: status: "in_progress",
1275: output: [],
1276: usage: null,
1277: },
1278: });
1279:
1280: // Buffered Codex responses still carry subscription windows and credits.
1281: // These values are independent of the token usage Lore may rescale.
1282: for (const quota of sanitizeCodexRateLimitEvents(
1283: resp.codexRateLimits ?? ...
Found 3 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts: Line 7900: const publicCodexRateLimits: Array<Record<string, unknown>> = []; Line 10266: publicCodexRateLimits, Line 10874: publicCodexRateLimits,
Found 4 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts:
Line 1728: function validateAnthropicMessageStart(parsed: Record<string, unknown>): void {
Line 1757: validateAnthropicUsage(message.usage, {
Line 1765: function validateAnthropicMessageDelta(
Line 1799: validateAnthropicUsage(parsed.usage, {
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/anthropic.ts</path>
<type>file</type>
<content>
1710: break;
1711: case "web_search_tool_result":
1712: case "web_fetch_tool_result":
1713: case "code_execution_tool_result":
1714: case "bash_code_execution_tool_result":
1715: case "text_editor_code_execution_tool_result":
1716: case "tool_search_tool_result":
1717: if (typeof block.tool_use_id !== "string" || block.content == null) {
1718: malformedAnthropicStream();
1719: }
1720: break;
1721: case "fallback":
1722: // Server-side fallback emits only a start/stop boundary block.
1723: break;
1724: }
1725: return block.type;
1726: }
1727:
1728: function validateAnthropicMessageStart(parsed: Record<string, unknown>): void {
1729: if (!isRecord(parsed.message)) malformedAnthropicStream();
1730: const message = parsed.message;
1731: if (
1732: typeof message.id !== "string" ...
RUN v4.1.8 /home/byk/Code/opencode-lore-recall-recovery
❯ packages/gateway/test/foreground-body-limit.test.ts (24 tests | 2 failed | 19 skipped) 106ms
× accepts standard non-stream output item 'function_call' 11ms
× accepts standard non-stream output item 'image_generation_call' 1ms
Test Files 1 failed | 1 passed (2)
Tests 2 failed | 14 passed | 192 skipped (208)
Start at 21:22:49
Duration 7.87s (transform 6.80s, setup 1.24s, import 13.53s, tests 165ms, environment 1ms)
⎯⎯⎯⎯⎯⎯⎯ Failed Tests 2 ⎯⎯⎯⎯⎯⎯⎯
FAIL packages/gateway/test/foreground-body-limit.test.ts > foreground response body limits > accepts standard non-stream output item 'function_call'
FAIL packages/gateway/test/foreground-body-limit.test.ts > foreground response body limits > accepts standard non-stream output item 'image_generation_call'
Error: upstream Responses request did not complete
❯ assertSuccessfulResponsesCompletion packages/gateway/src/stream/openai-responses.ts:279:13
...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts: Line 16139: export function mergeRecallUsage(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path> <type>file</type> <content> 16110: } 16111: 16112: /** 16113: * Decide whether to skip post-idle compaction (PR2b). The unified cache-economics 16114: * strategy provides the INTENT (hold-warm → protect the warm prefix by skipping 16115: * compaction; cool-bust/cool-full-write → let it compact), but the cache must 16116: * ACTUALLY still be live (`cacheIsLive` — the `isCacheWarm` time check) — a stale 16117: * hold-warm strategy whose cache has expired must NOT skip compaction (the cache 16118: * is cold; compaction is free and reduces ongoing read cost). Non-confident 16119: * strategy → `cacheIsLive` alone (the legacy behavior, byte-identical). ...
Found 4 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts:
Line 8293: const mergeUsage = (target: GatewayUsage, source: GatewayUsage): void => {
Line 11213: mergeUsage(state.usage, contState.usage);
Line 11219: mergeUsage(transactionProviderUsage, contState.usage);
Line 11673: mergeUsage(failedResponse.usage, transactionProviderUsage);
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
8265: pending.push({
8266: outputIndex,
8267: contentPosition,
8268: query: input.query,
8269: scope: input.scope,
8270: id: input.id,
8271: ids: input.ids,
8272: detailOffset: input.detailOffset,
8273: detailLimit: input.detailLimit,
8274: toolUseId,
8275: });
8276: parsedInputs.delete(outputIndex);
8277: return true;
8278: };
8279:
8280: const addUsageTokens = (left: number, right: number): number => {
8281: const result = left + right;
8282: if (
8283: !Number.isSafeInteger(left) ||
8284: left < 0 ||
8285: !Number.isSafeInteger(right) ||
8286: right < 0 ||
8287: !Number.isSafeInteger(result)
8288: ) {
8289: throw new Error("Responses usage token overflow");
8290: }
8291: return result;
8292: };
8293: const mergeUsage = (target: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
35: };
36:
37: afterEach(() => {
38: setRecallContinuationFailureHook(undefined);
39: setPrincipalTransportFailureHook(undefined);
40: log.registerSink(silentLogSink);
41: });
42:
43: function sseEvent(event: string, data: unknown): string {
44: const payload =
45: data && typeof data === "object" && !Array.isArray(data)
46: ? { type: event, ...(data as Record<string, unknown>) }
47: : data;
48: return `event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`;
49: }
50:
51: /** Build an upstream Responses SSE stream from ordered events. */
52: function streamFrom(events: string[]): Response {
53: return new Response(events.join(""), {
54: status: 200,
55: headers: { "content-type": "text/event-stream" },
56: });
57: }
58:
59: /** Drain a client-facing Response into the full SSE text. ...
Found 2 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts: Line 1908: requireSuccessfulCompletion?: boolean; Line 2025: opts.requireSuccessfulCompletion &&
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
1880: }
1881: }
1882: }
1883: }
1884:
1885: /**
1886: * Accumulate an OpenAI Responses API SSE stream into a GatewayResponse.
1887: *
1888: * Consumes the upstream Response body and returns the accumulated result.
1889: */
1890: export async function accumulateResponsesSSEStream(
1891: response: Response,
1892: opts: {
1893: /** Omit to preserve the legacy tolerant accumulator behavior. */
1894: validation?: ResponsesValidationMode;
1895: stopAtTerminal?: boolean;
1896: signal?: AbortSignal;
1897: inactivityMs?: number;
1898: maxFrames?: number;
1899: onSemanticContent?: () => void;
1900: /** Called only after the event has passed strict validation and mutation. */
1901: onValidatedEvent?: (event: string, data: string) => void | Promise<void>;
1902: /** Passthrough clients must receive a ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/openai-responses-recall-aware-stream.test.ts</path>
<type>file</type>
<content>
40: log.registerSink(silentLogSink);
41: });
42:
43: function sseEvent(event: string, data: unknown): string {
44: const payload =
45: data && typeof data === "object" && !Array.isArray(data)
46: ? { type: event, ...(data as Record<string, unknown>) }
47: : data;
48: return `event: ${event}\ndata: ${JSON.stringify(payload)}\n\n`;
49: }
50:
51: /** Build an upstream Responses SSE stream from ordered events. */
52: function streamFrom(events: string[]): Response {
53: return new Response(events.join(""), {
54: status: 200,
55: headers: { "content-type": "text/event-stream" },
56: });
57: }
58:
59: /** Drain a client-facing Response into the full SSE text. */
60: async function drain(resp: Response): Promise<string> {
61: const reader = resp.body!.getReader();
62: const decoder = new ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts: Line 831: export function finalizeResponsesAcc(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
820: }
821: }
822: break;
823: }
824:
825: // Other events (response.content_part.*,
826: // response.reasoning_summary_*, etc.) — ignored for accumulation
827: }
828: }
829:
830: /** Build the final GatewayResponse from accumulated state. */
831: export function finalizeResponsesAcc(
832: state: ResponsesAccState,
833: ): GatewayResponse {
834: const content: GatewayContentBlock[] = [];
835: const sortedIndices = Array.from(state.items.keys()).sort((a, b) => a - b);
836:
837: for (const index of sortedIndices) {
838: const item = state.items.get(index);
839: if (!item) continue;
840: if (item.type === "text") {
841: if (item.content) {
842: for (const part of item.content) {
843: if (part.type === "output_text" && typeof part.text === "string") {
844: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/stream/openai-responses.ts</path>
<type>file</type>
<content>
888: content.push({
889: type: "tool_use",
890: id: item.callId || item.id,
891: name: item.name,
892: input,
893: });
894: }
895: }
896:
897: let stopReason = state.stopReason;
898: // If we saw tool_use, map stop reason accordingly
899: if (content.some((b) => b.type === "tool_use") && stopReason === "end_turn") {
900: stopReason = "tool_use";
901: }
902:
903: return {
904: id: state.id,
905: model: state.model,
906: content,
907: rawOutputItems: Array.from(state.rawItems.entries())
908: .sort(([a], [b]) => a - b)
909: .map(([, item]) => item)
910: .filter((item) => item.type !== "item_reference"),
911: stopReason,
912: usage: state.usage,
913: ...(state.codexRateLimits
914: ? { codexRateLimits: state.codexRateLimits }
915: : ...
Found 2 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts:
Line 9604: let nextSequenceNumber = 0;
Line 9627: JSON.stringify({ ...parsed, sequence_number: nextSequenceNumber++ }),
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
9580: outputIdentities.has(syntheticId) ||
9581: states.some(
9582: (acc) =>
9583: [...acc.items.values()].some(
9584: (item) =>
9585: item.id === syntheticId ||
9586: (item.type === "tool_use" && item.callId === syntheticId),
9587: ) ||
9588: [...acc.rawItems.values()].some(
9589: (item) => item.id === syntheticId || item.call_id === syntheticId,
9590: ),
9591: )
9592: ) {
9593: throw new Error("duplicate synthetic Responses item identity");
9594: }
9595: syntheticIdentities.add(syntheticId);
9596: };
9597:
9598: // --- Keepalive (same as streamResponsesPassthrough) ---
9599: const KEEPALIVE_INACTIVITY_MS = 30_000;
9600: const keepaliveComment = encoder.encode(`: keepalive\n\n`);
9601: let keepaliveTimer: ReturnType<typeof ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts: Line 9985: controller.enqueue(sequenceChunk(chunk));
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
9950: resumeDemand?.();
9951: resumeDemand = undefined;
9952: if (keepaliveTimer) clearTimeout(keepaliveTimer);
9953: keepaliveTimer = null;
9954: if (activeReader) cancelAndReleaseReader(activeReader, signal.reason);
9955: else
9956: void currentPrincipalResponse.body?.cancel(signal.reason).catch(() => {});
9957: };
9958: signal.addEventListener("abort", onStreamAbort, { once: true });
9959: if (signal.aborted) onStreamAbort();
9960: const stream = new ReadableStream<Uint8Array>({
9961: start(controller) {
9962: void (async () => {
9963: const waitForDemand = async (): Promise<void> => {
9964: while (
9965: !cancelled &&
9966: !signal.aborted &&
9967: (controller.desiredSize ?? ...
Found 3 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts:
Line 8049: transactionBaseline = undefined;
Line 10507: transactionBaseline = {
Line 11448: transactionBaseline = undefined;
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/pipeline.ts</path>
<type>file</type>
<content>
10475: ),
10476: otherToolSeen,
10477: () => {
10478: terminalDelivered = true;
10479: finish(
10480: finalResponse,
10481: state.terminalEvent === "response.completed",
10482: );
10483: },
10484: ))
10485: )
10486: break;
10487: cancelAndReleaseReader(reader, signal.reason);
10488: principalReader = null;
10489: clearKeepalive();
10490: safeClose();
10491: return;
10492: }
10493: if (state.terminalEvent === "response.failed") {
10494: throw new Error("recall principal returned response.failed");
10495: ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/usage-validation.ts</path>
<type>file</type>
<content>
1: /** Runtime validation for provider-reported token usage. */
2:
3: export function isRecord(value: unknown): value is Record<string, unknown> {
4: return !!value && typeof value === "object" && !Array.isArray(value);
5: }
6:
7: function invalid(message: string): never {
8: throw new Error(message);
9: }
10:
11: function tokenCount(
12: value: unknown,
13: message: string,
14: allowNull = false,
15: ): number | undefined {
16: if (value === undefined || (allowNull && value === null)) return undefined;
17: if (!Number.isSafeInteger(value) || (value as number) < 0) invalid(message);
18: return value as number;
19: }
20:
21: /** Add validated token counts without allowing a safe-integer overflow. ...
Found 3 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/anthropic.ts:
Line 22: import { validateAnthropicUsage } from "../usage-validation";
Line 544: export function parseAnthropicResponseJSON(
Line 597: const usage = validateAnthropicUsage(json.usage, {
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/anthropic.ts</path>
<type>file</type>
<content>
535: // ---------------------------------------------------------------------------
536:
537: /**
538: * Parse an Anthropic-format response JSON back into a `GatewayResponse`.
539: *
540: * This is the inverse of `buildAnthropicNonStreamResponse`. Used when the
541: * pipeline returns Anthropic-format JSON that needs to be translated to
542: * another protocol (OpenAI Chat Completions, OpenAI Responses API).
543: */
544: export function parseAnthropicResponseJSON(
545: json: Record<string, unknown>,
546: ): GatewayResponse {
547: const content: GatewayContentBlock[] = [];
548: const toolIdentities = new Set<string>();
549: const rawContent = json.content as Array<Record<string, unknown>> | undefined;
550: if (rawContent) {
551: for (const block of rawContent) {
552: switch (block.type) {
553: case "text":
554: ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall-buffered-transaction.test.ts: Line 313: name: "Gemini missing usage",
No files found
[lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-564256/vec0.so)
{
"query": "implement all accepted recovery correctness security findings additive child pqvltmkr immutable parent spyyzvyy",
"scope": "project",
"projectPath": "/home/byk/Code/opencode-lore",
"result": "## Recall Results\n\nFound 30 results, showing top 15 (score range: 0.066–0.038).\n\n### Strong Matches\n\n#### Knowledge\n- **Independent correctness security reviews** (22h ago): Every non-trivial PR requires two independent passes: adversarial correctness, then security/pentest for injection, credentials, privacy, protocol, authorization, and privilege escalation. A combined summary looks efficient but can let one discipline obscure the other; inspect actual source and tests, exercise break states, apply/revert mutants, property tests ×10, hash equivalence, frozen-array mutation, and stale caches. ...
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall.test.ts</path>
<type>file</type>
<content>
1: /**
2: * Unit tests for gateway recall interception helpers.
3: *
4: * Tests the pure functions in recall.ts:
5: * - Tool definition
6: * - Detection helpers (findRecallToolUse, hasRecallToolUse, hasOtherToolUse)
7: * - Follow-up request builder
8: * - Pending recall injection
9: * - Response stripping
10: */
11: import { describe, test, expect, vi } from "vitest";
12: import { log } from "@loreai/core";
13: import {
14: LORE_COMMIT_REMINDER,
15: accumulateOpenAINonStreamJSON,
16: accumulateResponsesNonStreamJSON,
17: loreMessagesToGateway,
18: responsesProvenanceContent,
19: responsesProvenanceByMessageId,
20: responsesAnchorContext,
21: } from "../src/pipeline";
22: import {
23: isUsableRecallContinuation,
24: RECALL_GATEWAY_TOOL,
25: RECALL_TOOL_NAME,
26: MAX_RECALL_DEPTH,
27: findRecallToolUse,
28: ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/test/recall.test.ts: Line 1484: if (mode === "timeout") vi.useRealTimers();
(no output) <shell_metadata> shell tool terminated command after exceeding timeout 900000 ms. If this command is expected to take longer and is not waiting for interactive input, retry with a larger timeout value in milliseconds. </shell_metadata>
<path>/tmp/opencode/recovery-recall.json</path>
<type>file</type>
<content>
1: [lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-572134/vec0.so)
2: {
3: "query": "failed continuation retries quota rollback malformed JSON quota frame non-settling error body strict recovery output registry",
4: "scope": "project",
5: "projectPath": "/home/byk/Code/opencode-lore",
6: "result": "## Recall Results\n\nFound 90 results, showing top 15 (score range: 0.061–0.045).\n\n### Strong Matches\n\n#### Knowledge\n- **Recovery accounting transaction** (11h ago): Chose request-owned cumulative usage and ordered Codex quota accumulators over mutating each response because failed-continuation metadata must survive recovery without escaping the rollback boundary. ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts: Line 416: export function mapGeminiFinishReason(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts</path> <type>file</type> <content> 405: 406: /** 407: * Map a Gemini `finishReason` + tool presence to an internal stop reason. 408: * 409: * Abnormal reasons (SAFETY, RECITATION, BLOCKLIST, PROHIBITED_CONTENT, SPII, 410: * MALFORMED_FUNCTION_CALL, OTHER, …) are preserved VERBATIM so a proxied client 411: * still sees the real block/filter signal instead of a laundered "STOP". 412: * `toGeminiFinishReason` echoes any such preserved value back on egress. Only 413: * the truly-normal reasons are normalized to the internal model. A block reason 414: * takes precedence over `hasToolCall` (a filtered turn is not a tool turn). ...
USAGE
lore recall [--project value] [--scope value] [--session value] [--limit value] [--json] <args>...
lore recall --help
Search local project memory, or the configured remote gateway when LORE_REMOTE_URL is set. Query words are joined with spaces. Use --scope session with --session to search one session.
FLAGS
[--project] Project directory (default: current directory)
[--scope] Search scope (all | session | project | knowledge)
[--session] Session ID (required with --scope session)
[--limit] Maximum results, from 1 to 50 (default: 10)
[--json] Output raw JSON instead of formatted markdown [default = false]
-h --help Print help information and exit
-- All subsequent inputs should be interpreted as arguments
ARGUMENTS
args... Search query
...output truncated... Full output saved to: /home/byk/.local/share/opencode/tool-output/tool_0ac406b5f001itm4IscjurxX0p }
<path>/home/byk/.local/share/opencode/tool-output/tool_0ac406b5f001itm4IscjurxX0p</path>
<type>file</type>
<content>
1: [lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-588511/vec0.so)
2: {
3: "query": "quota rollback retry continuation",
4: "scope": "project",
5: "projectPath": "/home/byk/Code/opencode-lore",
6: "result": "## Recall Results\n\nFound 45 results, showing top 15 (score range: 0.063–0.054).\n\n### Strong Matches\n\n#### Distilled\n- Date: Sep 16, 2026 * 🟡 (00:15) Recovery delivery/validation background audit returned only `[lore:context-warning] Lore could not retrieve more memory`; assistant retired it as having no evidence and proceeded from accepted review findings. * 🟡 (00:16) Added two fail-first regressions without production edits: `packages/gateway/test/recall-buffered-transaction.test.ts` tests ordered Codex quota preservation across accepted recall, failed continuation, and recovery; ...
[lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-591058/vec0.so)
{
"query": "malformed non-JSON Codex quota retry failed attempt exact findings",
"scope": "project",
"projectPath": "/home/byk/Code/opencode-lore",
"result": "## Recall Results\n\nFound 45 results, showing top 15 (score range: 0.066–0.049).\n\n### Strong Matches\n\n#### Knowledge\n- **Sparse Codex provider failures** (2d ago): Trap: requiring every recall-aware failed terminal to repeat response ID/model looks strict, but Codex may legally omit metadata already established by prior events; rejecting it before provider-failure classification causes false `principal_protocol` and drops valid usage. Fix: recognize failed/cancelled aliases first, resolve omitted identity from trusted accumulator state, then sanitize and account exactly once without executing recall. ↳ files: packages/gateway/src/pipeline.ts, packages/gateway/src/stream/openai-responses.ts, ...
[lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-593215/vec0.so)
{
"query": "Implement all accepted recovery correctness/security findings Findings",
"scope": "session",
"projectPath": "/home/byk/Code/opencode-lore",
"result": "No results found for this query."
}
[lore]sqlite-vec: native vector search enabled (v0.1.9, /tmp/lore-native/pid-595986/vec0.so)
{
"query": "strict-envelope guard discrimination Anthropic Vertex OpenAI Chat Responses Codex Gemini",
"scope": "project",
"projectPath": "/home/byk/Code/opencode-lore",
"result": "## Recall Results\n\nFound 45 results, showing top 15 (score range: 0.073–0.049).\n\n### Strong Matches\n\n#### Knowledge\n- **Always verify gateway changes with exhaustive, protocol-specific regression analysis** (2d ago): Before modifying gateway, streaming, recall, or continuation behavior, inspect the current implementation, adjacent tests, schemas, parsers, normalized types, builders, persistence, accumulators, retries, budgets, transactions, cancellation, cleanup, duplicated branches, and all callers. Trace item types, phase statuses, identities, indices, reconciliation, malformed events, terminal reconstruction, hidden data, and resource limits. ...
Found 1 matches /home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts: Line 485: export function parseGeminiResponseJSON(
<path>/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts</path>
<type>file</type>
<content>
480:
481: /**
482: * Parse a Gemini `generateContent` response JSON into a `GatewayResponse`.
483: * Mirrors `parseAnthropicResponseJSON`, over `candidates[0].content.parts[]`.
484: */
485: export function parseGeminiResponseJSON(
486: json: Record<string, unknown>,
487: ): GatewayResponse {
488: const candidates = Array.isArray(json.candidates) ? json.candidates : [];
489: validateGeminiCandidateToolIdentities(
490: candidates,
491: "malformed Gemini response tool identity",
492: );
493: // LIMITATION: the internal model holds a single response, so when a client
494: // requests `candidateCount > 1` only candidates[0] is surfaced. Multi-candidate
495: // fan-out is a documented follow-up, not currently supported.
496: const first = (candidates[0] ?? {}) as Record<string, unknown>;
497: const content = (first.content ?? ...
Found 2 matches
/home/byk/Code/opencode-lore-recall-recovery/packages/gateway/src/translate/gemini.ts:
Line 477: function parseGeminiUsage(json: Record<string, unknown>): GatewayUsage {
Line 529: usage: parseGeminiUsage(json),
Date: Sep 16, 2026 * 🔴 (20:26) User stated retry-budget invariant: a retry must “never be given MORE room — retrying would either shrink the budget or waste”; retries must not increase the available room. * 🟡 (20:25) `packages/gateway/test/codex-usage-metadata.test.ts` defines request-wide Codex quota metadata caps of `MAX_CODEX_RATE_LIMIT_EVENTS = 64` and `MAX_CODEX_RATE_LIMIT_BYTES = 16 * 102…
Date: Sep 16, 2026 * 🔴 (20:24) User stated recall follow-up invariant: the follow-up request’s `stream` flag must never diverge from how the continuation is consumed. * 🟡 (20:24) In `packages/gateway/src/pipeline.ts`, non-stream recall couples build, forward, content-type assertion, and parsing: Codex uses `runRecallFollowUpStreamAccumulated()` because `/backend-api/codex/responses` mandates st…
Date: Sep 16, 2026 * 🔴 (20:22) User stated cleanup invariant: best-effort stream cancellation must never delay the caller. * 🟡 (20:22) `packages/gateway/src/recall.ts` function `ensureSSEResponse()` accepts `text/event-stream`; when ChatGPT/Codex omits `Content-Type`, it tees the body and probes up to `4096` bytes or `64` chunks with a `10_000` ms timeout, ignoring a BOM, leading whitespace, an…
Date: Sep 16, 2026 * 🔴 (20:21) User stated architectural invariant: `packages/gateway/src/recall.ts` never imports `pipeline.ts`; `RecallFollowUpCtx` receives injected `forward`, `parseJSON`, and optional `parseSSE` dependencies to avoid a circular dependency. * 🔴 (20:21) User directed response rewriting to replace recall `tool_use` with marker text. * 🟡 (20:21) `packages/gateway/src/recall.ts…
Date: Sep 16, 2026 * 🟡 (20:11) User asked to implement all accepted recovery correctness/security findings in `/home/byk/Code/opencode-lore-recall-recovery`, working in additive child change `pqvltmkr` over immutable parent `spyyzvyy/3f3ae768`. * 🔴 [enforced-workflow] (20:11) User authorized file edits but prohibited commit, push, rebase, bookmark operations, server operations, and production D…