Session 0LKjSWssiW9B

/home/byk/Code/opencode-lore-embedding-runtime/AGENTS.md

No files found

/home/byk/Code/opencode-lore-embedding-runtime/package.json
/home/byk/Code/opencode-lore-embedding-runtime/packages/website/package.json
/home/byk/Code/opencode-lore-embedding-runtime/packages/pi/package.json
/home/byk/Code/opencode-lore-embedding-runtime/packages/opencode/package.json
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/package.json
/home/byk/Code/opencode-lore-embedding-runtime/packages/gateway/package.json

No files found

/home/byk/Code/opencode-lore-embedding-runtime/.git

<path>/home/byk/Code/opencode-lore-embedding-runtime/AGENTS.md</path>
<type>file</type>
<content>
1: <!-- This section is maintained by the coding agent via lore (https://github.com/BYK/loreai) -->
2: ## Long-term Knowledge
3: 
4: For long-term knowledge entries managed by [lore](https://github.com/BYK/loreai) (gotchas, patterns, decisions, architecture), see [`.lore.md`](.lore.md) in the project root.
5: <!-- End lore-managed section -->
6: 
7: ## Review
8: 
9: See [`quality/REVIEW.md`](quality/REVIEW.md) for the review heuristics and workflow — regression-test discipline, adversarial-order state setup, fan-out registry coverage, recurring bug-class batteries, and the two-reviewer rule. Every non-trivial PR requires an adversarial correctness review before merge.
10: 
11: # Project Overview
12: 
13: Lore is a **three-tier memory architecture** for AI coding agents. ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/package.json</path>
<type>file</type>
<content>
1: {
2:   "name": "lore-monorepo",
3:   "private": true,
4:   "type": "module",
5:   "license": "FSL-1.1-Apache-2.0",
6:   "description": "Monorepo root for Lore — three-tier memory architecture",
7:   "main": "./packages/opencode/src/index.ts",
8:   "exports": {
9:     ".": "./packages/opencode/src/index.ts"
10:   },
11:   "packageManager": "pnpm@10.28.0",
12:   "volta": {
13:     "node": "24.16.0",
14:     "pnpm": "10.28.0"
15:   },
16:   "pnpm": {
17:     "onlyBuiltDependencies": [
18:       "esbuild",
19:       "onnxruntime-node",
20:       "sharp"
21:     ],
22:     "overrides": {
23:       "basic-ftp": "5.3.1",
24:       "ip-address": "10.2.0",
25:       "fast-xml-parser": "5.7.3",
26:       "@huggingface/hub": "2.11.0",
27:       "onnxruntime-node": "1.27.0",
28:       "yaml": ">=2.8.3",
29:       "esbuild": ">=0.28.1",
30:       "@opentelemetry/core": ">=2.8.0",
31:       ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/.lore.md</path>
<type>file</type>
<content>
1: <!-- Managed by lore (https://github.com/BYK/loreai) — manual edits are imported on next session. -->
2: 
3: ## Long-term Knowledge
4: 
5: ### Architecture
6: 
7: <!-- lore:01a08126-4de0-7a17-acbb-36db061a13de -->
8: * **#1711 Responses ingress boundary**: Chose streaming only \`/v1/responses\` and \`/v1/codex/responses\` over converting every JSON ingress route, because their shared OpenAI item normalizer can process \`input\` incrementally. Anthropic, Chat, Gemini, and compaction routes require distinct full-protocol parsing or intentionally retain exact body text for fallback replay. This reduces compressed/decoded/raw-tree peak allocations, but deliberately does not solve #1710’s post-parse full-history semantic replay: normalized \`GatewayRequest.messages\` are still complete.
9: 
10: <!-- lore:01a0266e-04b2-791c-b2be-02d2f63d969a -->
11: * **captureRequestUpstream requestOrder ...

Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/temporal-embedding-queue.test.ts:
  Line 313:   test("persistent failures back off exponentially, cap retries, and recover once after durable progress", async () => {


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/curator-changed-entries.test.ts:
  Line 238:   test("runs duplicate cleanup, cap enforcement, and cursor persistence", async () => {


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/sync-data.test.ts:
  Line 571:     // captures — they live in sqlite_temp_master, not the persistent schema.


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/background-drain.test.ts:
  Line 11:   _persistEmbedCap,

  Line 87:     _persistEmbedCap(8192, 0); // trust cap as-is regardless of host free memory

  Line 117:     _persistEmbedCap(8192, 0);

  Line 179:     _persistEmbedCap(8192, 0);

  Line 207:     _persistEmbedCap(8192, ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/.git</path>
<type>file</type>
<content>
1: gitdir: /home/byk/Code/opencode-lore/.git/worktrees/opencode-lore-embedding-runtime

(End of file - total 1 lines)
</content>

No files found

<path>/home/byk/Code/opencode-lore</path>
<type>directory</type>
<entries>
.craft.yml
.git/
.github/
.gitignore
.jj/
.lore.md
.opencode/
.oxfmtrc.json
.oxlintrc.json
.plans/
.sentryclirc
.vendor-build/
.worktrees/
AGENTS.md
CHANGELOG.md
codecov.yml
LICENSE
local_cache/
node_modules/
package.json
packages/
patches/
pnpm-lock.yaml
pnpm-workspace.yaml
quality/
README.md
scripts/
stryker.config.mjs
supabase/
tsconfig.base.json
tsconfig.json
vitest.config.ts
vitest.evals.config.ts
vitest.mutation.config.ts

(34 entries)
</entries>

<skill_content name="jj-guide">
# Skill: jj-guide

# jj Guide for AI Agents

Jujutsu (jj) is a Git-compatible VCS with mutable commits, automatic snapshotting, no staging area, and first-class conflicts. This skill teaches you how to use it safely from a non-interactive agent environment.

If `.jj/` exists in the repo root, this is a jj repo. **Use `jj` commands, not `git`.** In a colocated repo (`.jj/` *and* `.git/`), git tools can read the state, but mutations should go through `jj` so the operation log stays consistent.

## Critical Rules

- **NEVER** use interactive flags (`-i`, `--interactive`). TUI prompts hang in agent environments. This applies to `jj split -i`, `jj squash -i`, `jj commit -i`, `jj resolve`, `jj diffedit`, etc.
- **ALWAYS** pass `-m "msg"` when describing/committing. Without `-m`, an editor opens and hangs.
- **VERIFY** mutations with `jj st` and `jj log` after `squash`, `abandon`, `rebase`, `restore`, `commit`. ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/quality/REVIEW.md</path>
<type>file</type>
<content>
1: # Review Heuristics — Lore
2: 
3: This document codifies the review bar for pull requests. It exists because
4: adversarial review has repeatedly caught correctness bugs that a green test
5: suite missed — skip/early-return branches, lifecycle sequences, and fan-out
6: registry gaps. Institutionalizing these patterns so they compound.
7: 
8: The review workflow (§6) is referenced from AGENTS.md.
9: 
10: ---
11: 
12: ## 1. Regression-test discipline
13: 
14: Every adversarial-review finding that surfaces a defect **must** land a
15: deterministic regression test in the same PR. The test:
16: - fails on the base branch (proving it reproduces the bug),
17: - passes on the fix (proving the fix works),
18: - drives the *real* precondition, not an artificially-constructed state (e.g. ...

tmzmtlsspstn 6a5b70c9ee77 fix(core): correct embedding runtime selection and admission
pxrvwsvvkmpo 2cbf40060c67 feat(cch): add seeds for Claude Code up to 2.1.272 (#1780)

packages/core/src/cgroup-memory.ts                 | 224 ++++++++++
packages/core/src/embedding/local.ts               | 146 +++----
packages/core/src/embedding/pool.ts                |  18 +-
packages/core/src/embedding-cap.ts                 |  77 ++-
packages/core/src/embedding-worker-types.ts        |  18 +-
packages/core/src/embedding-worker.ts              |  27 +-
packages/core/src/embedding.ts                     |   1 +
packages/core/src/ort-native.ts                    | 256 +++++++++++-
packages/core/test/cgroup-memory.test.ts           | 415 +++++++++++++++++++
packages/core/test/embedding-cap.test.ts           |  75 ++-
packages/core/test/embedding-pool-memory.test.ts   |  38 +-
packages/core/test/embedding-pool.test.ts          |  69 +++
packages/core/test/embedding-wasm-fallback.test.ts |  60 +-
packages/core/test/embedding-worker-stdio.test.ts  |  46 +-
packages/core/test/embedding-worker-types.test.ts  |  41 +
packages/core/test/ort-native.test.ts              | 478 ...

<path>/home/byk/Code/opencode-lore/.git/worktrees/opencode-lore-embedding-runtime/HEAD</path>
<type>file</type>
<content>
1: 2cbf40060c67df229731cf1b9e34d5bf22ec2145

(End of file - total 1 lines)
</content>

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-cap.ts</path>
<type>file</type>
<content>
1: /**
2:  * Adaptive token-cap math for local ONNX embedding inference.
3:  *
4:  * Local inference OOMs on long inputs: the O(L²) attention tensor for a long
5:  * sequence blows the WASM heap, and because WASM linear memory never shrinks,
6:  * an in-process retry cannot recover. We instead cap the input sequence length
7:  * up-front and adapt that cap to the host — start from a memory-aware estimate,
8:  * and on each OOM lower the cap ×0.7 and respawn the worker on a fresh heap.
9:  *
10:  * This module holds the pure, dependency-free math so it can be unit-tested in
11:  * isolation. The stateful pieces (persistence, worker lifecycle, telemetry)
12:  * live in embedding.ts.
13:  */
14: 
15: /** Floor token cap. ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/local.ts</path>
<type>file</type>
<content>
1: /** Local ONNX embedding worker lifecycle. */
2: 
3: import { freemem } from "node:os";
4: import { readLinuxCgroupMemoryHeadroomBytes } from "../cgroup-memory";
5: import { db } from "../db";
6: import * as log from "../log";
7: import { vendorModelInfo } from "../embedding-vendor";
8: import { nativeIntraOpThreads } from "../ort-native";
9: import {
10:   MIN_EMBED_TOKENS,
11:   MODEL_MAX_TOKENS,
12:   backoffEmbedCap,
13:   clampFreeToContainerLimit,
14:   memoryModelEmbedCap,
15:   reconcileEmbedCap,
16:   reprobeEmbedCap,
17:   resolveMemoryHeadroom,
18:   shouldReprobeEmbedCap,
19:   type PersistedEmbedCap,
20: } from "../embedding-cap";
21: import {
22:   EMBED_OOM_EXIT_CODE,
23:   isMissingLocalStackError,
24:   isWasmFatalError,
25:   type EmbedRequest,
26:   type WorkerInbound,
27:   type WorkerOutbound,
28:   type WorkerInitData,
29: } from ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/local.ts</path>
<type>file</type>
<content>
400:   stream.on("data", () => {});
401:   stream.on("error", () => {
402:     log.debug(`embedding worker ${source} stream error`);
403:   });
404:   stream.resume();
405: }
406: 
407: function drainEmbeddingWorkerOutput(
408:   worker: import("node:worker_threads").Worker,
409: ): void {
410:   drainEmbeddingWorkerStream(worker.stdout, "stdout");
411:   drainEmbeddingWorkerStream(worker.stderr, "stderr");
412: }
413: 
414: /** True iff the local provider has been probed and found broken. */
415: export function localProviderKnownUnavailable(): boolean {
416:   return localEmbeddingState.failureCause !== null;
417: }
418: 
419: /** Local embedding provider using @huggingface/transformers with nomic-embed-text-v1.5 by default. ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/local.ts</path>
<type>file</type>
<content>
940:         );
941:       }
942:     } else {
943:       // A potentially transient failure (e.g. a model read that raced a
944:       // concurrent writer during a multi-instance restart, momentary memory
945:       // pressure). Retry a FRESH worker after a cooldown rather than disabling
946:       // local embeddings for the whole process lifetime; only give up (latch)
947:       // once the retry budget is exhausted.
948:       localEmbeddingState.initFailures++;
949:       localEmbeddingState.initFailureGeneration++;
950:       if (localEmbeddingState.initFailures >= LOCAL_INIT_MAX_ATTEMPTS) {
951:         latchLocalProvider("transient-init-exhausted");
952:         localEmbeddingState.initRetryAt = 0;
953:         if (!localEmbeddingState.errorLogged) {
954:           localEmbeddingState.errorLogged = true;
955:           log.error(
956:             ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-cap.test.ts</path>
<type>file</type>
<content>
240:       expect(cap).toBeGreaterThanOrEqual(MIN_EMBED_TOKENS);
241:       expect(cap).toBeLessThanOrEqual(MODEL_MAX_TOKENS);
242:     }
243:   });
244: });
245: 
246: describe("EMBED_TOKEN_CEILING", () => {
247:   it("is a fixed ceiling below the model max, in the measured ~4900 range", () => {
248:     // Derived from the 4 GiB WASM MAXIMUM_MEMORY (measured on the built worker)
249:     // × 0.85 headroom, minus baseline, over K. Must sit below MODEL_MAX_TOKENS
250:     // (that's the whole point) and near the observed safe convergence (≤4962).
251:     expect(EMBED_TOKEN_CEILING).toBeLessThan(MODEL_MAX_TOKENS);
252:     expect(EMBED_TOKEN_CEILING).toBeGreaterThan(4000);
253:     expect(EMBED_TOKEN_CEILING).toBeLessThanOrEqual(5200);
254:   });
255: 
256:   it("is the binding ceiling once free RAM is large enough to hit it", () => {
257:     // Below the ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-oom-recovery.test.ts</path>
<type>file</type>
<content>
1: import { afterEach, beforeEach, describe, expect, it } from "vitest";
2: import { EventEmitter } from "node:events";
3: import type { Worker } from "node:worker_threads";
4: import {
5:   embed,
6:   isAvailable,
7:   LocalProviderUnavailableError,
8:   _persistEmbedCap,
9:   _readPersistedEmbedCap,
10:   _resetLocalProviderProbe,
11:   _restoreProvider,
12:   _saveAndClearProvider,
13:   _setConstrainedMemoryForTest,
14:   _setContainerFreeForTest,
15:   _setTestWorkerFactory,
16: } from "../src/embedding";
17: import {
18:   backoffEmbedCap,
19:   MIN_EMBED_TOKENS,
20:   EMBED_TOKEN_CEILING,
21: } from "../src/embedding-cap";
22: import { EMBED_OOM_EXIT_CODE } from "../src/embedding-worker-types";
23: import { isStderrSilenced, silenceStderr } from "../src/log";
24: 
25: // Exercises the stateful OOM-recovery lifecycle in LocalProvider that the ...

Found 45 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/background-drain.test.ts:
  Line 87:     _persistEmbedCap(8192, 0); // trust cap as-is regardless of host free memory

  Line 117:     _persistEmbedCap(8192, 0);

  Line 179:     _persistEmbedCap(8192, 0);

  Line 207:     _persistEmbedCap(8192, 0);

  Line 227:     _persistEmbedCap(8192, 0);

  Line 282:     _persistEmbedCap(8192, 0);

  Line 304:     _persistEmbedCap(8192, 0);


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-wasm-fallback.test.ts:
  Line 120:     _persistEmbedCap(8192, 0);

  Line 204:     _persistEmbedCap(8192, 0);

  Line 235:     _persistEmbedCap(8192, 0);

  Line 269:     _persistEmbedCap(8192, 0);


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-pool-memory.test.ts:
  Line 126:     _persistEmbedCap(8192, 0);

  Line 150:     _persistEmbedCap(8192, 0);

  Line 165:     _persistEmbedCap(8192, 0);

  Line 182:     ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-cap-persistence.test.ts</path>
<type>file</type>
<content>
1: import { describe, it, expect, beforeEach } from "vitest";
2: import { db } from "../src/db";
3: import { _persistEmbedCap, _readPersistedEmbedCap } from "../src/embedding";
4: 
5: // Mirrors EMBED_CAP_KV_KEY in embedding.ts (private). Only used to seed corrupt
6: // rows for the guard tests below.
7: const KV_KEY = "lore:embedding_cap";
8: 
9: function writeRaw(value: string): void {
10:   db()
11:     .query(
12:       "INSERT INTO kv_meta (key, value) VALUES (?, ?) ON CONFLICT(key) DO UPDATE SET value = ?",
13:     )
14:     .run(KV_KEY, value, value);
15: }
16: 
17: describe("embedding cap persistence", () => {
18:   beforeEach(() => {
19:     db().query("DELETE FROM kv_meta WHERE key = ?").run(KV_KEY);
20:   });
21: 
22:   it("returns null when no cap is persisted", () => {
23:     expect(_readPersistedEmbedCap()).toBeNull();
24:   ...

Found 49 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/vec0-orphan-maintenance.ts:
  Line 92:         connection.exec("SAVEPOINT vec0_idle_gc");


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/sync-data.ts:
  Line 1443: // BEGIN IMMEDIATE would nest). cross_project=0 only: global/cross-project rows were never


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/entities.ts:
  Line 210:   d.exec("BEGIN IMMEDIATE");

  Line 1419:   d.exec("BEGIN IMMEDIATE");


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/db.ts:
  Line 1840:   // performed by a JS SAVEPOINT step (applyRefFkDrop), special-cased in migrate() by

  Line 1873:   // SAVEPOINT step (applyScopeKeyEpochPk), special-cased in migrate() by

  Line 2092: // performed atomically by a JS SAVEPOINT step (applyRefFkDrop) instead of plain SQL,

  Line 2099: // ALTER a primary key, so it recreates the table; done as a JS SAVEPOINT step so a crash

  Line 2143: ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/db.ts</path>
<type>file</type>
<content>
4920:   const columns = Object.keys(row);
4921:   if (columns.length === 0) {
4922:     throw new Error(`runUpsert: no columns provided for table "${table}"`);
4923:   }
4924:   const values = columns.map((c) => row[c]);
4925:   const placeholders = columns.map(() => "?").join(", ");
4926: 
4927:   const conflict = new Set(conflictColumns);
4928:   const exclude = new Set(opts.excludeFromUpdate ?? []);
4929:   const updatable = columns.filter((c) => !conflict.has(c) && !exclude.has(c));
4930: 
4931:   const conflictClause = conflictColumns.join(", ");
4932:   const action =
4933:     updatable.length === 0
4934:       ? "DO NOTHING"
4935:       : `DO UPDATE SET ${updatable.map((c) => `${c} = excluded.${c}`).join(", ")}`;
4936: 
4937:   const sql = `INSERT INTO ${table} (${columns.join(", ")}) VALUES (${placeholders}) ON CONFLICT(${conflictClause}) ${action}`;
4938:   ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/backfill.ts</path>
<type>file</type>
<content>
45:   recallEmbedsInFlight,
46: } from "./runtime";
47: import {
48:   enqueueTemporalEmbedding,
49:   invalidateTemporalEmbedding,
50: } from "../temporal-embedding-admission";
51: import { TEMPORAL_EMBEDDING_MIN_CONTENT_LENGTH } from "../embedding-units";
52: 
53: // ---------------------------------------------------------------------------
54: // Config change detection
55: // ---------------------------------------------------------------------------
56: 
57: function configFingerprint(): string {
58:   const cfg = config().search.embeddings;
59:   return `${cfg.provider}:${cfg.model}:${cfg.dimensions}`;
60: }
61: 
62: const EMBEDDING_CONFIG_KEY = "lore:embedding_config";
63: 
64: /** Check if embedding config has changed since the last backfill. ...

Found 1 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/db.ts:
  Line 146: export function databaseInTransaction(conn: Database): boolean {

Found 39 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/cache-economics.test.ts:
  Line 116: describe("decideCacheStrategy — monotonicity (the crux: ongoing cost of staying large)", () => {

  Line 586:   test("per-token scale: smaller values produce smaller costs (monotonicity)", () => {


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/semantic-lint/prompt.ts:
  Line 20:  * code, added guards/enforcement, migrations that rewrite a known-bad shape

  Line 42: - "fixes": the change removes code that violated the invariant, OR adds a guard/enforcement for it, OR migrates a known-bad shape into a known-good one. The hunk clearly resolves a documented conflict. It is more than a topical refactor.


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/semantic-lint/check.ts:
  Line 761:    *   guard/enforcement, migrates a known-bad shape to a known-good one, ...

diff --git a/packages/core/src/embedding/local.ts b/packages/core/src/embedding/local.ts
index 5c5e7e600f..32f4307b4d 100644
--- a/packages/core/src/embedding/local.ts
+++ b/packages/core/src/embedding/local.ts
@@ -1,6 +1,7 @@
 /** Local ONNX embedding worker lifecycle. */
 
 import { freemem } from "node:os";
+import { readLinuxCgroupMemoryHeadroomBytes } from "../cgroup-memory";
 import { db } from "../db";
 import * as log from "../log";
 import { vendorModelInfo } from "../embedding-vendor";
@@ -13,6 +14,7 @@
   memoryModelEmbedCap,
   reconcileEmbedCap,
   reprobeEmbedCap,
+  resolveMemoryHeadroom,
   shouldReprobeEmbedCap,
   type PersistedEmbedCap,
 } from "../embedding-cap";
@@ -76,6 +78,11 @@
   testHostFreememBytes = bytes;
 }
 
+let testAvailableMemoryBytes: number | null = null;
+export function _setAvailableMemoryForTest(bytes: number | null): void {
+  testAvailableMemoryBytes = bytes;
+}
+
 /**
  * The process's cgroup memory LIMIT in bytes (not free-within-limit), or ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/pool.ts</path>
<type>file</type>
<content>
1: /** Host-owned queue and pool for local embedding workers. */
2: 
3: import { createHash } from "node:crypto";
4: import { freemem } from "node:os";
5: import { config } from "../config";
6: import * as log from "../log";
7: import {
8:   DEFAULT_MAX_EMBED_POOL,
9:   EMBED_POOL_ABS_MAX,
10:   PER_WORKER_MEM_BUDGET_BYTES,
11:   clampFreeToContainerLimit,
12: } from "../embedding-cap";
13: import {
14:   EmbeddingQueueCapacityError,
15:   EmbeddingRequestAbortedError,
16:   EmbeddingWorkerWatchdogError,
17:   type EmbeddingProvider,
18:   LocalProviderUnavailableError,
19:   WORKER_SHUTDOWN_TIMEOUT_MS,
20:   isRecallEmbed,
21: } from "./contract";
22: import {
23:   LocalProvider,
24:   availableMemoryHeadroom,
25:   clearLocalProviderLatch,
26:   constrainedMemoryLimit,
27:   localEmbeddingState,
28: } from "./local";
29: import { OwnedRetirements } from ...

Found 7 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/sync-data.ts:
  Line 160:    * RETURNINGs internally, and for a co-member wrap the returned row fails scope_keys_read (a

  Line 162:    * plain INSERT with return=minimal has no RETURNING → no read-back → it lands. A re-push that

  Line 278:       // the writing admin (scope_keys_read = own wrap only), so UPSERT's RETURNING 42501s. Push

  Line 279:       // via plain INSERT (no RETURNING); a 23505 on re-push means already-synced.

  Line 1252:       // json_valid guard: a single corrupt source_ids must not throw and abort the

  Line 1264:                       WHERE json_valid(source_ids) AND ${projectRemoteGate("project_id")}) d,

  Line 1536:          (SELECT source_ids FROM distillations WHERE project_id = ? AND json_valid(source_ids)) d,

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding.ts</path>
<type>file</type>
<content>
1: /**
2:  * Public embedding facade.
3:  *
4:  * Implementation is split by responsibility so lifecycle ownership, search,
5:  * document writes, and migrations/backfills can evolve independently without
6:  * changing the long-standing public API imported by core and gateway callers.
7:  */
8: 
9: export {
10:   EmbeddingAbortError,
11:   EmbeddingQueueCapacityError,
12:   EmbeddingRequestAbortedError,
13:   EmbeddingWorkerWatchdogError,
14:   LocalProviderUnavailableError,
15:   awaitWorkerShutdown,
16:   l2Normalize,
17: } from "./embedding/contract";
18: export type {
19:   EmbeddingAbortCode,
20:   EmbeddingAbortPhase,
21:   EmbeddingOperationOptions,
22:   EmbeddingProvider,
23:   ShutdownableWorker,
24: } from "./embedding/contract";
25: 
26: export {
27:   _getLocalInitRetryAtForTest,
28:   _markLocalProviderUnavailable,
29:   _persistEmbedCap,
30:   ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-pool-memory.test.ts</path>
<type>file</type>
<content>
1: import { afterEach, beforeEach, describe, expect, it } from "vitest";
2: import { EventEmitter } from "node:events";
3: import type { Worker } from "node:worker_threads";
4: import {
5:   embed,
6:   _persistEmbedCap,
7:   _resetLocalProviderProbe,
8:   _restoreProvider,
9:   _saveAndClearProvider,
10:   _setConstrainedMemoryForTest,
11:   _setContainerFreeForTest,
12:   _setAvailableMemoryForTest,
13:   _setEmbedPoolSizeForTest,
14:   _setPoolFreememForTest,
15:   _setTestWorkerFactory,
16: } from "../src/embedding";
17: import {
18:   MIN_EMBED_TOKENS,
19:   backoffEmbedCap,
20:   memoryModelEmbedCap,
21: } from "../src/embedding-cap";
22: import { EMBED_OOM_EXIT_CODE } from "../src/embedding-worker-types";
23: 
24: // Regression suite for the embedding-pool OOM that SIGKILLed the whole gateway
25: // (Onur's report): the pool grew to N workers, ...

error: unexpected argument '-L' found

  tip: to pass '-L' as a value, use '-- -L'

Usage: jj file annotate [OPTIONS] <PATH>

For more information, try '--help'.

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/setup.ts</path>
<type>file</type>
<content>
1: import { mkdtempSync, rmSync } from "node:fs";
2: import { join } from "node:path";
3: import { tmpdir } from "node:os";
4: import { afterAll, afterEach } from "vitest";
5: import { close, invalidateProjectIdCache } from "../src/db";
6: import { silenceStderr } from "../src/log";
7: 
8: // Create an isolated temporary database for the entire test run.
9: // This prevents test fixtures from leaking into the live lore DB
10: // at ~/.local/share/lore/lore.db.
11: const tmp = mkdtempSync(join(tmpdir(), "lore-test-"));
12: process.env.LORE_DB_PATH = join(tmp, "test.db");
13: 
14: // ---------------------------------------------------------------------------
15: // Block live network to models.dev during tests.
16: //
17: // `fetchModelData()` (gateway/src/worker-model.ts) hits
18: // https://models.dev/api.json to pull pricing/limits, and the gateway
19: // pre-warms it ...

Found 68 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/semantic-lint.test.ts:
  Line 1: import { execFileSync } from "node:child_process";

  Line 125:   execFileSync("git", ["init", "--quiet"], { cwd: repo });

  Line 126:   execFileSync("git", ["config", "user.email", "test@example.com"], {

  Line 129:   execFileSync("git", ["config", "user.name", "Lore Test"], { cwd: repo });

  Line 134:   execFileSync("git", ["add", "-A"], { cwd: repo });

  Line 135:   execFileSync(

  Line 148:   return execFileSync("git", ["rev-parse", "HEAD"], {


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/db.test.ts:
  Line 1009:       const external = new DatabaseSync(process.env.LORE_DB_PATH as string);

  Line 1053:     // which bypasses ensureProject()'s production-DB guard. If LORE_DB_PATH is

  Line 1056:     if (!process.env.LORE_DB_PATH) {

  Line 1058:         "test DB not isolated: LORE_DB_PATH must be set (see packages/core/test/setup.ts)",

  ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/semantic-token-cache.test.ts</path>
<type>file</type>
<content>
240:     pending.persist();
241:     expect(isCurrentDatabase(connection)).toBe(false);
242:     expect(
243:       db().query("SELECT COUNT(*) AS n FROM semantic_token_cache").get(),
244:     ).toEqual({ n: 0 });
245:   });
246: 
247:   it("clears pre-response derived data and rejects an outstanding publication", () => {
248:     fill();
249:     const pending = new SemanticTokenCache(scope);
250:     pending.count(visible, provenance);
251:     expect(
252:       db().query("SELECT COUNT(*) AS n FROM temporal_messages").get(),
253:     ).toEqual({ n: 0 });
254:     clearProject(scope.projectPath);
255:     expect(
256:       db().query("SELECT COUNT(*) AS n FROM semantic_token_cache").get(),
257:     ).toEqual({ n: 0 });
258:     pending.persist();
259:     expect(
260:       db().query("SELECT COUNT(*) AS n FROM semantic_token_cache").get(),
261:   ...

Found 51 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/vector-worker.ts:
  Line 61:   setTimeout(() => process.exit(1), 0);

  Line 153:       setTimeout(() => process.exit(0), 0);


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/vector-pool.ts:
  Line 504:       // or a pending request delays process exit by up to the timeout on

  Line 827:  *  `vector-worker.ts` to flush its reader `close()` + `process.exit(0)`. That

  Line 883:  *      run its `reader.db.close()` flush + `process.exit(0)`;


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/temporal-embedding-queue.ts:
  Line 40:   startedAt: number;

  Line 344:     startedAt: performance.now(),

  Line 430:           // A restarted scheduler can share the old, cancelled provider call.

  Line 451:           `temporal embedding scheduler drain failed: reason=${reason} stage=${diagnostics.stage} elapsed_ms=${Math.round(performance.now() - diagnostics.startedAt)} ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
730:   let normalized: { tolist(): number[][]; data: Float32Array; dims: number[] };
731:   if (truncate) {
732:     // layer_norm → slice → L2 normalize
733:     normalized = layerNorm(output, [fullDim])
734:       .slice(null, [0, dimensions])
735:       .normalize(2, -1);
736:   } else {
737:     // layer_norm → L2 normalize (no truncation)
738:     normalized = layerNorm(output, [fullDim]).normalize(2, -1);
739:   }
740: 
741:   // Extract per-text vectors from the batched tensor.
742:   const numTexts = texts.length;
743:   const vectors: Float32Array[] = [];
744:   const dim = truncate ? dimensions : fullDim;
745: 
746:   for (let i = 0; i < numTexts; i++) {
747:     const start = i * dim;
748:     const vec = new Float32Array(dim);
749:     vec.set(normalized.data.subarray(start, start + dim));
750:     vectors.push(vec);
751:   }
752: 
753:   return ...

Found 11 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/db.ts:
  Line 3013:   database.exec(`PRAGMA busy_timeout = ${BUSY_TIMEOUT_MS}`);

  Line 4689:     // busy_timeout=0 first so a reader mid-query can't make this busy-wait (~5s)

  Line 4692:       instance.exec("PRAGMA busy_timeout = 0");

  Line 4746:   conn.exec("PRAGMA busy_timeout = 0");

  Line 4753:     conn.exec(`PRAGMA busy_timeout = ${BUSY_TIMEOUT_MS}`);

  Line 4846:   if (opts?.noWait) conn.exec("PRAGMA busy_timeout = 0");

  Line 4850:     if (opts?.noWait) conn.exec(`PRAGMA busy_timeout = ${BUSY_TIMEOUT_MS}`);

  Line 4871:  * Runs on the writer connection; like checkpointWal it drops busy_timeout to 0 so a

  Line 4880:   conn.exec("PRAGMA busy_timeout = 0");

  Line 4884:     conn.exec(`PRAGMA busy_timeout = ${BUSY_TIMEOUT_MS}`);

  Line 5251:     reader.exec(`PRAGMA busy_timeout = ${BUSY_TIMEOUT_MS}`);

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/pool.ts</path>
<type>file</type>
<content>
520:     }
521:   }
522: 
523:   /** Consume one interrupted call's successful prefix. */
524:   takeTokenBatchCheckpoint(key: string): TokenBatchCheckpoint | undefined {
525:     this.pruneTokenBatchCheckpoints();
526:     const checkpoint = this.tokenBatchCheckpoints.get(key);
527:     if (!checkpoint) return undefined;
528:     this.tokenBatchCheckpoints.delete(key);
529:     return {
530:       ...checkpoint,
531:       vectors: cloneEmbeddingVectors(checkpoint.vectors),
532:     };
533:   }
534: 
535:   /** Retain bounded progress only after an interrupted durable drain. */
536:   storeTokenBatchCheckpoint(
537:     key: string,
538:     nextIndex: number,
539:     vectors: Float32Array[],
540:   ): void {
541:     if (this.closing || nextIndex <= 0 || vectors.length !== nextIndex) return;
542:     this.tokenBatchCheckpoints.set(key, {
543:       ...

Found 94 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding.test.ts:
  Line 22:   LocalProviderUnavailableError,

  Line 25:   _resetLocalProviderProbe,

  Line 26:   _markLocalProviderUnavailable,

  Line 366:     _resetLocalProviderProbe();

  Line 371:     // LocalProvider construction always succeeds; the ONNX model init

  Line 399:     // so these tests get a fresh LocalProvider with _markLocalProviderUnavailable.

  Line 407:     _resetLocalProviderProbe();

  Line 412:     _resetLocalProviderProbe();

  Line 415:     _markLocalProviderUnavailable();

  Line 419:   test("embed() throws LocalProviderUnavailableError when local provider is broken", async () => {

  Line 420:     _markLocalProviderUnavailable();

  Line 428:     expect(caught).toBeInstanceOf(LocalProviderUnavailableError);

  Line 432:     _markLocalProviderUnavailable();

  Line 435:       LocalProviderUnavailableError,

  Line 441:     _markLocalProviderUnavailable();

  Line ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
620: }
621: 
622: // ---------------------------------------------------------------------------
623: // Embed processing
624: // ---------------------------------------------------------------------------
625: 
626: /**
627:  * Truncate texts to a maximum number of tokens using the real tokenizer.
628:  * Only texts exceeding `maxTokens` chars are checked — at worst-case
629:  * ~1 char/token, shorter texts can never exceed the limit.
630:  */
631: function truncateTexts(texts: string[], maxTokens: number): string[] {
632:   const tk = tokenizer;
633:   if (!tk) return texts;
634:   return texts.map((text) => {
635:     if (text.length <= maxTokens) return text;
636:     // Exclude [CLS]/[SEP] special tokens so ids.length reflects pure content
637:     // token count — otherwise the 2 extra tokens skew the limit check.
638:     const ids = tk.encode(text, { ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
830: 
831: // ---------------------------------------------------------------------------
832: // Shutdown handling
833: // ---------------------------------------------------------------------------
834: 
835: let inflight = 0;
836: let shutdownRequested = false;
837: 
838: function maybeExit(): void {
839:   if (shutdownRequested && inflight === 0) {
840:     // Deferred process.exit(0) lets any pending setImmediate / microtask
841:     // callbacks (e.g. onnxruntime-node NAPI result conversion) complete
842:     // before we tear down the V8 isolate.  A plain port.close() is not
843:     // sufficient: native NAPI handles can keep the event loop alive
844:     // indefinitely.
845:     setTimeout(() => process.exit(0), 0);
846:   }
847: }
848: 
849: function post(msg: WorkerOutbound): void {
850:   port.postMessage(msg);
851: }
852: 
853: port.on("message", ...

Found 20 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-worker.ts:
  Line 7:  * and processes embed requests sequentially from a priority queue. Moving

  Line 14:  * The worker applies Nomic's recommended post-processing:

  Line 578: // Priority queue — high-priority (recall) jumps ahead of normal (backfill)

  Line 581: const queue: EmbedRequest[] = [];

  Line 582: let processing = false;

  Line 585:  * Enqueue an embed request. High-priority requests are inserted after any

  Line 589: function enqueue(msg: EmbedRequest): void {

  Line 593:     while (insertAt < queue.length && queue[insertAt].priority === "high") {

  Line 596:     queue.splice(insertAt, 0, msg);

  Line 598:     queue.push(msg);

  Line 604:  * Process queued requests one at a time. ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
570:     }
571:     return true;
572:   } catch {
573:     return false; // uncertain → safe fallback to purge
574:   }
575: }
576: 
577: // ---------------------------------------------------------------------------
578: // Priority queue — high-priority (recall) jumps ahead of normal (backfill)
579: // ---------------------------------------------------------------------------
580: 
581: const queue: EmbedRequest[] = [];
582: let processing = false;
583: 
584: /**
585:  * Enqueue an embed request. High-priority requests are inserted after any
586:  * existing high-priority items but before all normal-priority items (FIFO
587:  * within each priority level). Triggers drain if not already running.
588:  */
589: function enqueue(msg: EmbedRequest): void {
590:   if (msg.priority === "high") {
591:     // Insert after the last "high" item — keeps FIFO within high ...

Found 23 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/pool.ts:
  Line 226:       throw new LocalProviderUnavailableError("embedding pool is unavailable");

  Line 292:     const provider = new LocalProvider(

  Line 691:               : new LocalProviderUnavailableError();

  Line 720:         new LocalProviderUnavailableError("embedding pool is unavailable"),

  Line 764:     const shutdownError = new LocalProviderUnavailableError(


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/local.ts:
  Line 485:       throw new LocalProviderUnavailableError("embedding worker is closing");

  Line 488:       throw new LocalProviderUnavailableError(this.workerInitError);

  Line 494:         throw new LocalProviderUnavailableError();

  Line 499:         throw new LocalProviderUnavailableError("embedding worker is closing");

  Line 655:                 pending.reject(new LocalProviderUnavailableError(msg.error));

  Line 742:      ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-worker-types.ts</path>
<type>file</type>
<content>
1: /**
2:  * Shared message types for the embedding worker thread.
3:  *
4:  * The embedding worker (`embedding-worker.ts`) runs ONNX inference via
5:  * `@huggingface/transformers` in a separate `node:worker_threads` Worker
6:  * so the main thread's event loop stays free during inference. This file
7:  * defines the message protocol between the main thread (`LocalProvider`
8:  * in `embedding.ts`) and the worker.
9:  *
10:  * Imported by both sides — keep this file free of runtime dependencies.
11:  */
12: 
13: // ---------------------------------------------------------------------------
14: // Main thread → Worker
15: // ---------------------------------------------------------------------------
16: 
17: /** Request an embedding batch. */
18: export interface EmbedRequest {
19:   type: "embed";
20:   /** Monotonic request ID for correlating responses. ...

No files found

<path>/home/byk/Code/opencode-lore-embedding-runtime</path>
<type>directory</type>
<entries>
.craft.yml
.git
.github/
.gitignore
.jj/
.lore.md
.oxfmtrc.json
.oxlintrc.json
AGENTS.md
CHANGELOG.md
codecov.yml
LICENSE
node_modules/
package.json
packages/
patches/
pnpm-lock.yaml
pnpm-workspace.yaml
quality/
README.md
scripts/
stryker.config.mjs
supabase/
tsconfig.base.json
tsconfig.json
vitest.config.ts
vitest.evals.config.ts
vitest.mutation.config.ts

(28 entries)
</entries>

<path>/home/byk/Code/opencode-lore-embedding-runtime/.jj</path>
<type>directory</type>
<entries>
.gitignore
repo
working_copy/

(3 entries)
</entries>

tmzmtlsspstn 6a5b70c9ee77 fix(core): correct embedding runtime selection and admission
pxrvwsvvkmpo 2cbf40060c67 feat(cch): add seeds for Claude Code up to 2.1.272 (#1780)
packages/core/src/cgroup-memory.ts                 | 224 ++++++++++
packages/core/src/embedding/local.ts               | 146 +++----
packages/core/src/embedding/pool.ts                |  18 +-
packages/core/src/embedding-cap.ts                 |  77 ++-
packages/core/src/embedding-worker-types.ts        |  18 +-
packages/core/src/embedding-worker.ts              |  27 +-
packages/core/src/embedding.ts                     |   1 +
packages/core/src/ort-native.ts                    | 256 +++++++++++-
packages/core/test/cgroup-memory.test.ts           | 415 +++++++++++++++++++
packages/core/test/embedding-cap.test.ts           |  75 ++-
packages/core/test/embedding-pool-memory.test.ts   |  38 +-
packages/core/test/embedding-pool.test.ts          |  69 +++
packages/core/test/embedding-wasm-fallback.test.ts |  60 ...

Working copy changes:
A packages/core/src/cgroup-memory.ts
M packages/core/src/embedding/local.ts
M packages/core/src/embedding/pool.ts
M packages/core/src/embedding-cap.ts
M packages/core/src/embedding-worker-types.ts
M packages/core/src/embedding-worker.ts
M packages/core/src/embedding.ts
M packages/core/src/ort-native.ts
A packages/core/test/cgroup-memory.test.ts
M packages/core/test/embedding-cap.test.ts
M packages/core/test/embedding-pool-memory.test.ts
M packages/core/test/embedding-pool.test.ts
M packages/core/test/embedding-wasm-fallback.test.ts
M packages/core/test/embedding-worker-stdio.test.ts
M packages/core/test/embedding-worker-types.test.ts
M packages/core/test/ort-native.test.ts
M packages/gateway/script/bundle.ts
M packages/gateway/script/ort-platform-package.ts
M packages/gateway/test/bundle-exports.test.ts
M packages/gateway/test/ort-platform-package.test.ts
Working copy  (@) : tmzmtlss 6a5b70c9 fix/embedding-runtime-admission | fix(core): correct embedding runtime ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/db.ts</path>
<type>file</type>
<content>
130: 
131: function cachedProjectId(
132:   conn: Database,
133:   path: string,
134:   dataVersion: number,
135: ): string | undefined {
136:   const cache = projectIdCacheFor(conn);
137:   const cachedVersion = projectIdCacheDataVersion.get(conn);
138:   if (cachedVersion !== undefined && cachedVersion !== dataVersion) {
139:     cache.clear();
140:   }
141:   projectIdCacheDataVersion.set(conn, dataVersion);
142:   return cache.get(path);
143: }
144: 
145: /** True while either supported SQLite driver has an open transaction. */
146: export function databaseInTransaction(conn: Database): boolean {
147:   const transactionState = conn as Database & {
148:     /** node:sqlite transaction-state getter. */
149:     isTransaction?: boolean;
150:     /** bun:sqlite transaction-state getter. */
151:     inTransaction?: boolean;
152:   };
153:   if (typeof ...

Found 100 matches (more matches available)
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/semantic-lint.test.ts:
  Line 466:     expect(isIgnoredFile("docs/src/content/docs/getting-started.mdx")).toBe(

  Line 1491:     // caller maxTokens, the linter's tiny default budget (256 tokens) would be

  Line 1495:     const captured: Array<{ maxTokens?: number; reasoningEffort?: string }> =

  Line 1501:         opts?: { maxTokens?: number; reasoningEffort?: string },

  Line 1536:       expect(captured[0].maxTokens).toBeGreaterThanOrEqual(25_600);


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-pool.test.ts:
  Line 101:     this.emit("message", { type: "started", id });


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-pool-memory.test.ts:
  Line 35: type EmbedMsg = { type: string; id: number; maxTokens: number };

  Line 142:     expect(posted.maxTokens).toBe(dividedCap);

  Line 146:     ...

Found 13 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding.test.ts:
  Line 14:   checkConfigChange,

  Line 308:   test("tracks concurrent recall embeds additively", async () => {

  Line 985:   test("concurrent embed() calls are serialized correctly", async () => {

  Line 987:       embed([`concurrent test ${i}`], "document"),

  Line 1015: describe("checkConfigChange", () => {

  Line 1025:     const changed = checkConfigChange();

  Line 1039:     checkConfigChange();

  Line 1040:     const changed = checkConfigChange();

  Line 1049:     const changed = withTransaction(() => checkConfigChange());

  Line 1083:     const changed = checkConfigChange();


/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-pool.test.ts:
  Line 232:     // After bootstrap, genuine concurrent demand grows the pool as before. ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding.test.ts</path>
<type>file</type>
<content>
1000:       "document",
1001:     );
1002:     const queryPromise = embed(["urgent query"], "query");
1003: 
1004:     const [docs, [queryVec]] = await Promise.all([docPromise, queryPromise]);
1005:     expect(docs).toHaveLength(5);
1006:     for (const vec of docs) {
1007:       expect(vec).toBeInstanceOf(Float32Array);
1008:       expect(vec.length).toBe(768);
1009:     }
1010:     expect(queryVec).toBeInstanceOf(Float32Array);
1011:     expect(queryVec.length).toBe(768);
1012:   }, 60_000);
1013: });
1014: 
1015: describe("checkConfigChange", () => {
1016:   const PROJECT = "/test/embedding/configchange";
1017: 
1018:   beforeEach(() => {
1019:     const pid = ensureProject(PROJECT);
1020:     db().query("DELETE FROM knowledge WHERE project_id = ?").run(pid);
1021:     db().query("DELETE FROM kv_meta WHERE key LIKE 'lore:%'").run();
1022:   });
1023: 
1024:   ...

f3e63e7875bfcc56f89dc572ef3bcd6b60d4f5410ca325b9b8bddd80e1625902  -
ff99f80d82b48d0e915c4be9ea84f8c60365436825ffb4bf3aa80b6d42afb458  -

f3e63e7875bfcc56f89dc572ef3bcd6b60d4f5410ca325b9b8bddd80e1625902  -
cddde2d402cf5328b51e1c49e54db8bd957caee7859aa1ce3d2ffcbfb65b04f7  -
dd922bec9215c345d879a44885078044567deafa9574538aaa6ac414c6cc95df  -
5913c6bd181657e8276cd1e83a499e2e870c2e0f02900bc32c3ceb97d363deab  -

tmzmtlsspstn 6a5b70c9ee77 fix(core): correct embedding runtime selection and admission
pxrvwsvvkmpo 2cbf40060c67 feat(cch): add seeds for Claude Code up to 2.1.272 (#1780)
xpzmsyowwmzs e977b4b1da8f feat(semantic-lint): lint small PRs holistically (#1778)
owulpnqtvpkq 991d7553c728 refactor(installer): move installation into the standalone CLI (#1743)
uwoyklqrsuux 1e3657a08fd8 fix(gateway): recover principal Responses transport failures (#1776)
zystzmxonwzs 81514b3cb46c feat(semantic-lint): pass bounded PR context to judges (#1772)
zuqrxwsvzwrx 8ef91360cc9f feat(cch): add seeds for Claude Code up to 2.1.270 (#1769)
twstwootrpsp b764c13fcfed Fix OpenCode lifecycle regression and SIGTERM shutdown (#1768)
xuuwzpqyytkt 11cfe10bae21 Dispose embedded Lore gateway with OpenCode (#1767)
tkzvmrtrwnsr 425cdac58fba refactor(core): split embedding runtime, pools, and backfills (#1766)
uvyrknxypzxo 7092beefbb73 Handle gateway signals early and own worker teardown (#1764)
wkxwwrwmttzz 7fdbe279e8df ...

tmzmtlsspstn 6a5b70c9ee77 fix(core): correct embedding runtime selection and admission
twuzlmysntnr 44062927f956 perf(core): persist and bound temporal embedding work (#1691)
nzmnxwsxsquz b836620716a2 feat(core): throttle the temporal re-chunk backfill CPU on constrained hosts (#1184)
nzwtupxuypoq ef54b5b4451c fix(core): clamp embedding memory sizing to the cgroup limit (container OOM) (#1168)
ykunusosxmsp 91dc2f4504e9 refactor(core): rename WASM_SUSTAINABLE_MAX_TOKENS → EMBED_TOKEN_CEILING (#1145)
qosllkttmxxr 327db1b621ef fix(core): bound embedding cap by the fixed WASM heap ceiling (#999) (#1134)
runoymymwlzl f5556659e2d7 perf(core): parallelize embedding into a memory-gated worker pool (#999) (#1115)
xqrulyqorouk 63feee107a4a feat(core): embed part-selective text for temporal messages (#1061)
spuqrquqkvqx 0a2640f2560d fix(core): calibrate embedding cap constants from measured WASM footprint (#867)
kqwnnqwlwyrp f96047fcfaca fix(core): adaptive token cap for local embedding OOM ...

M packages/core/src/embedding/local.ts
M packages/core/src/embedding-cap.ts
M packages/core/test/embedding-cap.test.ts

Commit ID: 44062927f956ff73152acf6425881363e8b70ac6
Change ID: twuzlmysntnryypvxtkwuvprnlkktpqk
Author   : Burak Yigit Kaya <ben@byk.im> (2026-08-27 14:12:51)
Committer: GitHub <noreply@github.com> (2026-08-27 14:12:51)

    perf(core): persist and bound temporal embedding work (#1691)

    ## Summary

    Replaces unbounded per-message temporal embedding work with a durable,
    coalescing queue and a bounded single-flight scheduler. This removes a
    major CPU, memory, and event-loop pressure amplifier from tool-heavy
    sessions.

    ## Changes

    - Adds a local-only v85 queue that stores only message ID, content hash,
    policy fingerprint, and enqueue time; message content and vectors are
    never copied or synchronized.
    - Batches semantic units across up to eight messages within a 256 KiB
    projection budget, validates provider vectors, and commits through
    stale-content and storage-mode CAS checks. ...

Error: Revision `packages/core/src/embedding.ts` doesn't exist

diff --git a/packages/core/src/embedding-cap.ts b/packages/core/src/embedding-cap.ts
new file mode 100644
index 0000000000..ff2dcc3686
--- /dev/null
+++ b/packages/core/src/embedding-cap.ts
@@ -0,0 +1,143 @@
+/**
+ * Adaptive token-cap math for local ONNX embedding inference.
+ *
+ * Local inference OOMs on long inputs: the O(L²) attention tensor for a long
+ * sequence blows the WASM heap, and because WASM linear memory never shrinks,
+ * an in-process retry cannot recover. We instead cap the input sequence length
+ * up-front and adapt that cap to the host — start from a memory-aware estimate,
+ * and on each OOM lower the cap ×0.7 and respawn the worker on a fresh heap.
+ *
+ * This module holds the pure, dependency-free math so it can be unit-tested in
+ * isolation. The stateful pieces (persistence, worker lifecycle, telemetry)
+ * live in embedding.ts.
+ */
+
+/** Floor token cap. ...

Found 5 matches
/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/db.ts:
  Line 429:   CREATE TABLE IF NOT EXISTS kv_meta (

  Line 566:   -- Separate from kv_meta (plugin state) — this holds installation-scoped

  Line 2336:       `INSERT INTO kv_meta (key, value)

  Line 3646:     CREATE TABLE IF NOT EXISTS kv_meta (

  Line 6729: // migration v29. Distinct from kv_meta (plugin state) and metadata

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/db.ts</path>
<type>file</type>
<content>
405: 
406:   -- Sync triggers
407:   CREATE TRIGGER IF NOT EXISTS distillation_fts_insert AFTER INSERT ON distillations BEGIN
408:     INSERT INTO distillation_fts(rowid, observations) VALUES (new.rowid, new.observations);
409:   END;
410: 
411:   CREATE TRIGGER IF NOT EXISTS distillation_fts_delete AFTER DELETE ON distillations BEGIN
412:     INSERT INTO distillation_fts(distillation_fts, rowid, observations)
413:     VALUES('delete', old.rowid, old.observations);
414:   END;
415: 
416:   CREATE TRIGGER IF NOT EXISTS distillation_fts_update AFTER UPDATE ON distillations BEGIN
417:     INSERT INTO distillation_fts(distillation_fts, rowid, observations)
418:     VALUES('delete', old.rowid, old.observations);
419:     INSERT INTO distillation_fts(rowid, observations) VALUES (new.rowid, new.observations);
420:   END;
421:   `,
422:   `
423:   -- Version 8: Embedding BLOB column ...

Working copy changes:
A packages/core/src/cgroup-memory.ts
M packages/core/src/embedding/local.ts
M packages/core/src/embedding/pool.ts
M packages/core/src/embedding-cap.ts
M packages/core/src/embedding-worker-types.ts
M packages/core/src/embedding-worker.ts
M packages/core/src/embedding.ts
M packages/core/src/ort-native.ts
A packages/core/test/cgroup-memory.test.ts
M packages/core/test/embedding-cap-persistence.test.ts
M packages/core/test/embedding-cap.test.ts
M packages/core/test/embedding-pool-memory.test.ts
M packages/core/test/embedding-pool.test.ts
M packages/core/test/embedding-wasm-fallback.test.ts
M packages/core/test/embedding-worker-stdio.test.ts
M packages/core/test/embedding-worker-types.test.ts
M packages/core/test/ort-native.test.ts
M packages/gateway/script/bundle.ts
M packages/gateway/script/ort-platform-package.ts
M packages/gateway/test/bundle-exports.test.ts
M packages/gateway/test/ort-platform-package.test.ts
Working copy  (@) : tmzmtlss 450c8844 ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-cap-persistence.test.ts</path>
<type>file</type>
<content>
1: import { describe, it, expect, beforeEach } from "vitest";
2: import { db } from "../src/db";
3: import { _persistEmbedCap, _readPersistedEmbedCap } from "../src/embedding";
4: 
5: // Mirrors EMBED_CAP_KV_KEY in embedding.ts (private). Only used to seed corrupt
6: // rows for the guard tests below.
7: const KV_KEY = "lore:embedding_cap";
8: 
9: function writeRaw(value: string): void {
10:   db()
11:     .query(
12:       "INSERT INTO kv_meta (key, value) VALUES (?, ?) ON CONFLICT(key) DO UPDATE SET value = ?",
13:     )
14:     .run(KV_KEY, value, value);
15: }
16: 
17: describe("embedding cap persistence", () => {
18:   beforeEach(() => {
19:     db().query("DELETE FROM kv_meta WHERE key = ?").run(KV_KEY);
20:   });
21: 
22:   it("returns null when no cap is persisted", () => {
23:     expect(_readPersistedEmbedCap()).toBeNull();
24:   ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-cap.ts</path>
<type>file</type>
<content>
55:  *  O(L²) attention tensor, so footprint ≈ baseline + K·L². Measured at ~116
56:  *  bytes/token² (two runs: 116.8, 115.7) on the bundled WASM worker
57:  *  (eval/measure-embed-cap.mjs, #857; R²=0.999 across L=256..4096), rounded up
58:  *  for margin. The backoff corrects any residual. */
59: export const EMBED_ATTENTION_BYTES_PER_TOKEN_SQ = 120;
60: 
61: /** Free-memory ratio band within which a persisted learned cap is trusted
62:  *  as-is (i.e. memory is "close enough" to learn-time to skip re-converging). */
63: export const EMBED_CAP_TRUST_BAND = 0.25;
64: 
65: /** Persisted learned cap + the free memory at learn time (kv_meta JSON). ...

<path>/home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/local.ts</path>
<type>file</type>
<content>
35:   LocalProviderUnavailableError,
36:   type ShutdownableWorker,
37:   WORKER_SHUTDOWN_TIMEOUT_MS,
38:   awaitWorkerShutdown,
39:   isRecallEmbed,
40: } from "./contract";
41: 
42: const EMBED_CAP_KV_KEY = "lore:embedding_cap";
43: 
44: const EMBED_REPROBE_INTERVAL_MS = 5 * 60_000;
45: 
46: function readPersistedEmbedCap(): PersistedEmbedCap | null {
47:   try {
48:     const row = db()
49:       .query("SELECT value FROM kv_meta WHERE key = ?")
50:       .get(EMBED_CAP_KV_KEY) as { value: string } | null;
51:     if (!row) return null;
52:     const parsed = JSON.parse(row.value) as Partial<PersistedEmbedCap>;
53:     if (
54:       typeof parsed.cap !== "number" ||
55:       typeof parsed.freeMemBytes !== "number"
56:     ) {
57:       return null;
58:     }
59:     return {
60:       cap: parsed.cap,
61:       freeMemBytes: parsed.freeMemBytes,
62:       ...