Dashboard › Code › Session s35b8c634e09
s35b8c634e098391e 118 │ - ······setTimeout(reject,·CHECKIN_SEND_TIMEOUT_MS,·new·Error("Check-in·send·timed·out"))
118 │ + ······setTimeout(
119 │ + ········reject,
120 │ + ········CHECKIN_SEND_TIMEOUT_MS,
121 │ + ········new·Error("Check-in·send·timed·out")
122 │ + ······)
119 123 │ );
120 124 │ await Promise.race([send, timeout]);
Checked 820 files in 12s. No fixes applied.
Found 1 error.
check ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
× Some errors were emitted while running checks.
error: script "lint" exited with code 1
Just a formatting issue — the setTimeout call needs to be broken across lines.
[fix/security-vitest-tmp 6f8300f] fix: prettier formatting for type cast in spread 1 file changed, 1 insertion(+), 1 deletion(-) remote: remote: GitHub found 4 vulnerabilities on getsentry/craft's default branch (2 critical, 2 high). To find out more, visit: remote: https://github.com/getsentry/craft/security/dependabot remote: To https://github.com/getsentry/craft.git dc1f893..6f8300f fix/security-vitest-tmp -> fix/security-vitest-tmp
Now let me wait for CI to run again:
Did you also implement change D or not?
Yes, I did implement change D. It's in embedding-worker.ts ensurePipeline() — right before the pipeline() call, I added:
const wasmEnv = (env as Record<string, unknown>).backends as
| { onnx?: { wasm?: { numThreads?: number; proxy?: boolean } } }
| undefined;
if (wasmEnv?.onnx?.wasm) {
wasmEnv.onnx.wasm.numThreads = 1;
wasmEnv.onnx.wasm.proxy = false;
}
It's in the code, but I can't guarantee it solves the OOM because:
env.backends.onnx.wasm path is how transformers.js exposes the ORT config, but I couldn't verify at runtime that it actually takes effect before pipeline() — it might need to be set earlier, or the property path might differ slightly in the bundled version.numThreads=1 takes effect, the OOM might not be caused by the shared-memory path specifically — it could be a different Bun WASM bug (#18145 $bunfs Aborted) that numThreads doesn't address.The only way to know is to build and test on an affected machine. The storm fix (A1-A4, B, C) is the guaranteed win; D is our best-effort WASM tuning that needs validation.
$ biome check --write --no-errors-on-unmatched --max-diagnostics=none ./ Checked 820 files in 22s. Fixed 1 file.
Lint fixes fail 25s https://github.com/getsentry/craft/actions/runs/26978157916/job/79610335002 Analyze (actions) pass 45s https://github.com/getsentry/craft/actions/runs/26978155507/job/79610331365 Analyze (javascript) pass 57s https://github.com/getsentry/craft/actions/runs/26978155507/job/79610331519 Build pass 23s https://github.com/getsentry/craft/actions/runs/26978157964/job/79610435379 Build Docs pass 29s https://github.com/getsentry/craft/actions/runs/26978157964/job/79610435402 Changelog Preview / Semver Impact pass 0 https://github.com/getsentry/craft/actions/runs/26978157918 Patch CodeQL pass 2s https://github.com/getsentry/craft/runs/79610438561 Cursor Bugbot pass 39s https://cursor.com/docs/bugbot Node 22 pass 33s https://github.com/getsentry/craft/actions/runs/26978157964/job/79610335349 Node 24 pass 33s https://github.com/getsentry/craft/actions/runs/26978157964/job/79610335411 Secret ...
"Lint fixes" failed again. Let me check what it is this time:
[feat/monitor-checkins 146b8d54d] fix: address second-round review findings 1 file changed, 22 insertions(+), 12 deletions(-) remote: remote: GitHub found 2 vulnerabilities on getsentry/cli's default branch (1 moderate, 1 low). To find out more, visit: remote: https://github.com/getsentry/cli/security/dependabot remote: To https://github.com/getsentry/cli.git d3b2032f5..146b8d54d feat/monitor-checkins -> feat/monitor-checkins
Pushed. Let me wait for CI and check for any new review comments.
Lint fixes Type Check 2026-06-04T20:38:49.5376986Z ##[group]Run pnpm typecheck
Lint fixes Type Check 2026-06-04T20:38:49.5377284Z ^[[36;1mpnpm typecheck^[[0m
Lint fixes Type Check 2026-06-04T20:38:49.5394203Z shell: /usr/bin/bash -e {0}
Lint fixes Type Check 2026-06-04T20:38:49.5394472Z env:
Lint fixes Type Check 2026-06-04T20:38:49.5394731Z PNPM_HOME: /home/runner/setup-pnpm/node_modules/.bin
Lint fixes Type Check 2026-06-04T20:38:49.5395180Z ##[endgroup]
Lint fixes Type Check 2026-06-04T20:38:49.7765144Z
Lint fixes Type Check 2026-06-04T20:38:49.7766097Z > @sentry/craft@2.27.0-dev.0 typecheck /home/runner/work/craft/craft
Lint fixes Type Check 2026-06-04T20:38:49.7766781Z > tsc --noEmit
Lint fixes Type Check 2026-06-04T20:38:49.7767002Z
Lint fixes Type Check 2026-06-04T20:38:54.8759211Z ##[error]src/status_providers/__tests__/github.test.ts(34,10): error TS2348: Value of type 'Mock<Procedure | Constructable>' is not callable. Did you mean to include 'new'? ...
TypeScript error in a different test file. The Mock type changed in vitest 4. Let me look at it:
<path>/home/byk/.local/share/opencode/worktree/55fa901aca64a01fa4b5c2abce1c85c1c27475e7/swift-squid/src/status_providers/__tests__/github.test.ts</path>
<type>file</type>
<content>
1: import { describe, it, expect, vi, beforeEach } from 'vitest';
2:
3: vi.mock('../../logger', () => ({
4: logger: {
5: trace: vi.fn(),
6: debug: vi.fn(),
7: info: vi.fn(),
8: log: vi.fn(),
9: warn: vi.fn(),
10: error: vi.fn(),
11: success: vi.fn(),
12: withScope: vi.fn().mockReturnThis(),
13: },
14: }));
15:
16: vi.mock('../../utils/githubApi', () => ({
17: getGitHubClient: vi.fn().mockReturnValue({
18: repos: {
19: getCombinedStatusForRef: vi.fn(),
20: get: vi.fn(),
21: },
22: checks: {
23: listSuitesForRef: vi.fn(),
24: listForRef: vi.fn(),
25: },
26: }),
27: }));
28:
29: import { GitHubStatusProvider } from '../github';
30: import { CommitStatus } from '../base';
31: import { getGitHubClient } from ...
The issue is on line 34: (getGitHubClient as ReturnType<typeof vi.fn>)() — in vitest 4, Mock type changed to Mock<Procedure | Constructable> which isn't directly callable. The fix is to cast more specifically. Let me fix it:
Edit applied successfully.
Let me verify this fixes the type error locally:
> @sentry/craft@2.27.0-dev.0 typecheck /home/byk/.local/share/opencode/worktree/55fa901aca64a01fa4b5c2abce1c85c1c27475e7/swift-squid > tsc --noEmit src/status_providers/__tests__/github.test.ts(34,10): error TS2348: Value of type 'Mock<Procedure | Constructable>' is not callable. Did you mean to include 'new'? ELIFECYCLE Command failed with exit code 2.
The unknown cast didn't help because TypeScript still resolves the return type. Let me use a different approach — cast directly to a function type:
Edit applied successfully.
> @sentry/craft@2.27.0-dev.0 typecheck /home/byk/.local/share/opencode/worktree/55fa901aca64a01fa4b5c2abce1c85c1c27475e7/swift-squid
> tsc --noEmit
src/status_providers/__tests__/github.test.ts(34,10): error TS2352: Conversion of type 'Octokit & RestEndpointMethods & Api & { paginate: PaginateInterface; }' to type '{ repos: { getCombinedStatusForRef: Mock<Procedure | Constructable>; get: Mock<Procedure | Constructable>; }; checks: { ...; }; }' may be a mistake because neither type sufficiently overlaps with the other. If this was intentional, convert the expression to 'unknown' first.
The types of 'repos.getCombinedStatusForRef' are incompatible between these types.
Type '{ (params?: (RequestParameters & { per_page?: number | undefined; page?: number | undefined; owner: string; repo: string; ref: string; }) | undefined): Promise<OctokitResponse<{ state: string; ... 5 more ...; url: string; }, 200>>; defaults: <O extends RequestParameters = RequestParameters>(newDefaults: O) => ...
The issue is that vi.mocked() returns the actual type signature of getGitHubClient (which returns Octokit), and the as cast to the mock type fails. I need to go through unknown first:
Build npm Package (Node 24) pending 0 https://github.com/getsentry/cli/actions/runs/26978263298/job/79611295092 warden: find-bugs pending 0 https://warden.sentry.dev warden pending 0 https://warden.sentry.dev Build npm Package (Node 22) pending 0 https://github.com/getsentry/cli/actions/runs/26978263298/job/79611295115 warden pending 0 https://github.com/getsentry/cli/actions/runs/26978263285/job/79610704499 Build Binary (linux-x64) pending 0 https://github.com/getsentry/cli/actions/runs/26978263298/job/79611295080
Core checks (Unit Tests, Lint & Typecheck, Seer) passed. Builds and Warden still in progress. Let me wait for the remaining ones.
[fix/embedding-oom-storm 717ae79] fix(embedding): stop OOM event storm, single-thread WASM, remove auto-fallback 4 files changed, 100 insertions(+), 146 deletions(-)
Edit applied successfully.
> @sentry/craft@2.27.0-dev.0 typecheck /home/byk/.local/share/opencode/worktree/55fa901aca64a01fa4b5c2abce1c85c1c27475e7/swift-squid > tsc --noEmit src/__mocks__/logger.ts(3,14): error TS2742: The inferred type of 'logger' cannot be named without a reference to '.pnpm/@vitest+spy@4.1.8/node_modules/@vitest/spy'. This is likely not portable. A type annotation is necessary. src/__mocks__/logger.ts(29,14): error TS2742: The inferred type of 'setLevel' cannot be named without a reference to '.pnpm/@vitest+spy@4.1.8/node_modules/@vitest/spy'. This is likely not portable. A type annotation is necessary. ELIFECYCLE Command failed with exit code 2.
Another vitest 4 type issue — the mock types reference @vitest/spy which isn't directly importable. Let me fix the logger mock:
remote: remote: Create a pull request for 'fix/embedding-oom-storm' on GitHub by visiting: remote: https://github.com/BYK/loreai/pull/new/fix/embedding-oom-storm remote: To https://github.com/BYK/loreai.git * [new branch] fix/embedding-oom-storm -> fix/embedding-oom-storm
Now let me create the PR:
<path>/home/byk/.local/share/opencode/worktree/55fa901aca64a01fa4b5c2abce1c85c1c27475e7/swift-squid/src/__mocks__/logger.ts</path>
<type>file</type>
<content>
1: import { vi } from 'vitest';
2:
3: export const logger = {
4: trace: vi.fn(),
5: debug: vi.fn(),
6: info: vi.fn(),
7: log: vi.fn(),
8: warn: vi.fn(),
9: error: vi.fn(),
10: success: vi.fn(),
11: withScope: vi.fn().mockReturnThis(),
12: pause: vi.fn(),
13: resume: vi.fn(),
14: };
15:
16: export const LogLevel = {
17: Fatal: 0,
18: Error: 0,
19: Warn: 1,
20: Log: 2,
21: Info: 3,
22: Success: 3,
23: Debug: 4,
24: Trace: 5,
25: Silent: -Infinity,
26: Verbose: Infinity,
27: };
28:
29: export const setLevel = vi.fn();
(End of file - total 29 lines)
</content>
The issue is that TypeScript can't infer the type of logger and setLevel without referencing the internal @vitest/spy package. The fix is to add explicit type annotations. Let me check what the original module exports look like:
Warning: 1 uncommitted change https://github.com/BYK/loreai/pull/545
PR created: https://github.com/BYK/loreai/pull/545
The "1 uncommitted change" warning is just the docs/plans/ directory we intentionally didn't stage. Let me check CI status:
27:export function formatTable(
47:export { LogLevel as LogLevel };
57:export const logger = createLogger();
63:export function setLevel(logLevel: LogLevel): void {
The simplest fix is to add explicit type annotations using Mock from vitest:
test pending 0 https://github.com/BYK/loreai/actions/runs/26978538679/job/79611634661
CI is running. Let me wait and check again:
Edit applied successfully.
> @sentry/craft@2.27.0-dev.0 typecheck /home/byk/.local/share/opencode/worktree/55fa901aca64a01fa4b5c2abce1c85c1c27475e7/swift-squid > tsc --noEmit
Typecheck passes. Let me also make sure tests still pass:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/.github/workflows/docs-preview.yml</path>
<type>file</type>
<content>
1: name: Docs Preview
2:
3: on:
4: push:
5: branches: [main]
6: paths:
7: - 'docs/**'
8: - '.github/workflows/docs-preview.yml'
9: pull_request:
10: types: [opened, reopened, synchronize, closed]
11: paths:
12: - 'docs/**'
13: - '.github/workflows/docs-preview.yml'
14:
15: permissions:
16: contents: write
17: pull-requests: write
18:
19: concurrency:
20: group: docs-preview-${{ github.ref }}
21: cancel-in-progress: true
22:
23: jobs:
24: preview:
25: runs-on: ubuntu-latest
26: steps:
27: - uses: actions/checkout@v6
28:
29: - name: Ensure .nojekyll at gh-pages root
30: # Fork PRs can't push to the base repo (GITHUB_TOKEN is read-only on
31: # pull_request from forks). ...
Actually, thinking about this more carefully: the simplest approach that matches the reference implementations is to keep paths on the trigger but use dorny/paths-filter only for the deploy steps (not the cleanup). But that adds complexity.
A cleaner solution: the rossjrw/pr-preview-action with action: auto does nothing on closed events for PRs that never had a preview deployed. So even if closed fires for a non-docs PR, the action will simply find no preview to remove and exit gracefully. This means we can safely remove paths from the pull_request trigger and just keep it on push:
Total threads: 7, Unresolved: 2 - [sentry] **Bug:** An uncleared `setTimeout` in `sendCheckInSafely` causes the process to hang for up to 60 seconds after the main... - [cursor] ### Check-in timeout orphans fetch **Medium Severity** <!-- DESCRIPTION START --> In `sendCheckInSafely`, `Promise.rac...
Both findings point to the same issue: the setTimeout in the timeout race is never cleared, causing the process to hang after the main work completes (the timer keeps Node alive). Let me fix this properly by clearing the timer when the fetch completes.
<path>/home/byk/Code/getsentry/cli/src/commands/monitor/run.ts</path>
<type>file</type>
<content>
99: * the child process spawn or exit.
100: */
101: async function sendCheckInSafely(
102: dsn: string,
103: dsnComponents: ReturnType<typeof makeDsn>,
104: checkIn: ReturnType<typeof buildCheckIn>,
105: phase: "in-progress" | "final"
106: ): Promise<void> {
107: try {
108: const envelope = createCheckInEnvelope(
109: checkIn,
110: undefined,
111: undefined,
112: undefined,
113: dsnComponents
114: );
115: const body = serializeEnvelope(envelope);
116: const send = sendEnvelopeRequest(dsn, body);
117: const timeout = new Promise<never>((_, reject) =>
118: setTimeout(
119: reject,
120: CHECKIN_SEND_TIMEOUT_MS,
121: new Error("Check-in send timed out")
122: )
123: );
124: await Promise.race([send, timeout]);
125: } catch (err) {
126: log.error(
127: `Failed to send ${phase} ...
Edit applied successfully.
Binary smoke (darwin-arm64) pass 29s https://github.com/BYK/loreai/actions/runs/26978538679/job/79611907801 Binary smoke (windows-x64) pass 1m46s https://github.com/BYK/loreai/actions/runs/26978538679/job/79611907805 Build Nightly Binaries skipping 0 https://github.com/BYK/loreai/actions/runs/26978538679/job/79611908158 Generate Delta Patches skipping 0 https://github.com/BYK/loreai/actions/runs/26978538679/job/79611908797 Generate Release Patches skipping 0 https://github.com/BYK/loreai/actions/runs/26978538679/job/79611908122 Publish Nightly to GHCR skipping 0 https://github.com/BYK/loreai/actions/runs/26978538679/job/79611908932 test pass 1m32s https://github.com/BYK/loreai/actions/runs/26978538679/job/79611634661
All CI green:
skipping (nightly/release-only — expected for a PR)PR is ready: https://github.com/BYK/loreai/pull/545
0
=== lint ===
Checked 820 files in 22s. Fixed 1 file.
Checked 820 files in 12s. No fixes applied.
=== tests ===
Test Files 1 passed (1)
Tests 9 passed (9)
Start at 20:51:45
Duration 2.93s (transform 1.72s, setup 58ms, import 2.53s, tests 191ms, environment 0ms)
Clean. Let me commit and push.
.github/workflows/ci.yml | 8 +++++--- .github/workflows/docs-preview.yml | 32 +++++++++++++++++++++++--------- 2 files changed, 28 insertions(+), 12 deletions(-)
[feat/monitor-checkins d306c2623] fix: clear timeout timer in sendCheckInSafely to prevent process hang 1 file changed, 10 insertions(+), 5 deletions(-) remote: remote: GitHub found 2 vulnerabilities on getsentry/cli's default branch (1 moderate, 1 low). To find out more, visit: remote: https://github.com/getsentry/cli/security/dependabot remote: To https://github.com/getsentry/cli.git 146b8d54d..d306c2623 feat/monitor-checkins -> feat/monitor-checkins
Pushed. Let me wait for CI and check for any new review comments.
You are a senior engineer performing an adversarial pre-merge review of PR #545. Be thorough and critical — classify findings as CRITICAL, MEDIUM, or LOW. Give a final MERGE-READY or BLOCK verdict.
This PR fixes an ONNX embedding worker OOM event storm (11k Sentry events) and silent knowledge loss. The standalone binary redirects onnxruntime-node → onnxruntime-web (WASM), and Bun's shared/threaded WASM engine fails a ~271 MiB allocation on model init. The fix has several layers:
Read and review these files in full:
packages/core/src/embedding-worker.ts — the worker threadpackages/core/src/embedding.ts — the main-thread provider, focus on lines 187-198, 203-223, 355-430, 630-760packages/core/test/embedding.test.ts — the updated tests, focus on lines 183-300.lore.md — the updated/added knowledge entriesFor each file, check:
Give exact file:line references for all findings. Provide concrete fix suggestions for CRITICAL/MEDIUM items.
Return your verdict at the end: MERGE-READY or BLOCK, with reasoning.
I'll start by reading all the files mentioned in the review scope.
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding-worker.ts</path> <type>file</type> <content> 1: /** 2: * Embedding worker thread — runs ONNX inference via @huggingface/transformers 3: * off the main thread. 4: * 5: * This file is the entry point for a `node:worker_threads` Worker spawned by 6: * `LocalProvider` in `embedding.ts`. It owns the transformers.js pipeline 7: * and processes embed requests sequentially from a priority queue. Moving 8: * inference here keeps the main thread's event loop free — HTTP requests, 9: * SSE streams, and session APIs are no longer blocked during embedding. 10: * 11: * Communication uses `parentPort` message passing with structured clone. 12: * Float32Array vectors are sent back directly (Bun preserves identity). 13: * 14: * The worker applies Nomic's recommended post-processing: 15: * 1. Mean pooling (via pipeline option) 16: * 2. ...
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts</path>
<type>file</type>
<content>
1: /**
2: * Embedding integration for vector search.
3: *
4: * Supports multiple embedding providers behind a common interface:
5: * - "local" (default): @huggingface/transformers + nomic-embed-text-v1.5
6: * (768 dims, Matryoshka-capable). Runs ONNX inference in a worker thread.
7: * - "voyage": Voyage AI API (voyage-code-3, 1024 dims)
8: * - "openai": OpenAI API (text-embedding-3-small, 1536 dims)
9: *
10: * Provides embedding generation, pure-JS cosine similarity, and vector search
11: * over the knowledge and distillation tables. All operations are gated behind
12: * `search.embeddings.enabled` config + the provider's API key env var — falls
13: * back silently to FTS-only when unavailable.
14: */
15:
16: import { db } from "./db";
17: import { config } from "./config";
18: import * as log ...
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/test/embedding.test.ts</path>
<type>file</type>
<content>
1: import {
2: afterAll,
3: afterEach,
4: describe,
5: test,
6: expect,
7: beforeEach,
8: mock,
9: } from "bun:test";
10: import { db, ensureProject } from "../src/db";
11: import {
12: cosineSimilarity,
13: toBlob,
14: fromBlob,
15: isAvailable,
16: vectorSearch,
17: vectorSearchEntities,
18: checkConfigChange,
19: _shutdownAndDisable,
20: _saveAndClearProvider,
21: _restoreProvider,
22: embed,
23: LocalProviderUnavailableError,
24: pickRemoteFallback,
25: _resetLocalProviderProbe,
26: _markLocalProviderUnavailable,
27: } from "../src/embedding";
28:
29: describe("cosineSimilarity", () => {
30: test("identical vectors return 1.0", () => {
31: const a = new Float32Array([1, 2, 3]);
32: expect(cosineSimilarity(a, a)).toBeCloseTo(1.0, 5);
33: });
34:
35: ...
Now let me read the rest of embedding.ts and the .lore.md changes:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts</path>
<type>file</type>
<content>
1433: // log.error sends to Sentry via captureException
1434: log.error(
1435: `distillation embedding backfill batch failed (${batch.length} items):`,
1436: err,
1437: );
1438: // Provider is dead — no point retrying remaining batches.
1439: if (err instanceof LocalProviderUnavailableError) break;
1440: }
1441:
1442: if (embedded >= nextProgressAt) {
1443: log.info(`embedding distillations: ${embedded}/${rows.length}…`);
1444: nextProgressAt = embedded + PROGRESS_INTERVAL;
1445: }
1446: // No yieldToEventLoop() needed — embed() is truly async (worker thread).
1447: }
1448:
1449: if (embedded > 0) {
1450: log.info(`embedded ${embedded} distillations`);
1451: }
1452: return embedded;
1453: }
1454:
1455: // ...
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/.lore.md</path> <type>file</type> <content> 1: <!-- Managed by lore (https://github.com/BYK/loreai) — manual edits are imported on next session. --> 2: 3: ## Long-term Knowledge 4: 5: ### Architecture 6: 7: <!-- lore:019e550f-29b4-768d-ade6-4e46c014effe --> 8: * **DB schema current version: 26 migrations, key columns per migration**: DB schema: 33 migrations. v31=tool\_calls (call\_id, tool, status, error\_type, error\_message, duration\_ms, session\_id, project\_id, project\_path) + rebuild 6 FTS5 tables unicode61 remove\_diacritics 0; v32=rebuild 6 FTS5 tables; v33=rebuild 7 FTS5 tables. \`migrate()\` uses \`MIGRATIONS.length\`. \`SCHEMA\_VERSION=16\` is dead code. \`recoverMissingObjects()\` does NOT recreate FTS tables. ...
Now let me check the diff to understand exactly what changed in this PR, and look at the embedding-worker-types for the message protocol:
717ae79 fix(embedding): stop OOM event storm, single-thread WASM, remove auto-fallback ed07369 docs(site): add 'Why Lore' comparison page and extract shared theme (#542) ce1c0be fix(test): make Sentry-off-during-tests CWD-independent (#530) (#541) 88c503b fix(core): clean up knowledge_transfers on entry removal (#543) 66f57e8 test: block live models.dev fetch in test preload (#540)
.lore.md | 5 +- packages/core/src/embedding-worker.ts | 44 +++++++++++++- packages/core/src/embedding.ts | 88 +++++++++++++-------------- packages/core/test/embedding.test.ts | 109 ++++------------------------------ 4 files changed, 100 insertions(+), 146 deletions(-)
diff --git a/packages/core/src/embedding-worker.ts b/packages/core/src/embedding-worker.ts
index 30196f1..a990b68 100644
--- a/packages/core/src/embedding-worker.ts
+++ b/packages/core/src/embedding-worker.ts
@@ -127,6 +127,26 @@ async function ensurePipeline(): Promise<void> {
env.allowRemoteModels = false;
}
+ // Force single-threaded WASM execution to avoid Bun's buggy
+ // shared-memory/pthread WASM paths. The threaded build uses
+ // `new WebAssembly.Memory({shared:true})` which triggers open Bun bugs:
+ // - oven-sh/bun#25677: SharedArrayBuffer writes invisible to workers
+ // - oven-sh/bun#31158: SIGPWR storm with native threads + WASM
+ // - oven-sh/bun#18145: $bunfs + WASM Aborted() in --compile binaries
+ // Single-thread avoids all three. ...
diff --git a/packages/core/src/embedding.ts b/packages/core/src/embedding.ts
index 7598af9..df16d35 100644
--- a/packages/core/src/embedding.ts
+++ b/packages/core/src/embedding.ts
@@ -188,8 +188,9 @@ export class LocalProviderUnavailableError extends Error {
constructor(cause?: unknown) {
super(
"Local embedding provider unavailable: '@huggingface/transformers' failed to initialize. " +
- "Configure search.embeddings.provider to 'voyage' or 'openai', or " +
- "set VOYAGE_API_KEY/OPENAI_API_KEY for automatic remote fallback.",
+ "Recall will use FTS-only search. To use a remote provider instead, set " +
+ "search.embeddings.provider to 'voyage' or 'openai' in .lore.json " +
+ "and provide the corresponding API key (VOYAGE_API_KEY / OPENAI_API_KEY).",
);
this.name = "LocalProviderUnavailableError";
if (cause !== undefined)
@@ -360,9 +361,24 @@ class LocalProvider implements EmbeddingProvider {
if (pending) {
...
diff --git a/packages/core/test/embedding.test.ts b/packages/core/test/embedding.test.ts
index 1583eda..80ab4ee 100644
--- a/packages/core/test/embedding.test.ts
+++ b/packages/core/test/embedding.test.ts
@@ -180,122 +180,35 @@ describe("local provider unavailable fallback", () => {
});
});
-describe("auto-fallback to remote provider when local provider is unavailable", () => {
- let savedVoyage: string | undefined;
- let savedOpenAI: string | undefined;
- let savedFetch: typeof fetch;
+describe("local provider unavailable — no auto-fallback (remote is opt-in)", () => {
let savedProvider: unknown;
beforeEach(() => {
- savedVoyage = process.env.VOYAGE_API_KEY;
- savedOpenAI = process.env.OPENAI_API_KEY;
- delete process.env.VOYAGE_API_KEY;
- delete process.env.OPENAI_API_KEY;
- savedFetch = globalThis.fetch;
savedProvider = _saveAndClearProvider();
});
afterEach(() => {
- if (savedVoyage !== undefined) process.env.VOYAGE_API_KEY = ...
diff --git a/.lore.md b/.lore.md index e3f0300..8093f33 100644 --- a/.lore.md +++ b/.lore.md @@ -43,7 +43,7 @@ * **bootstrapDailySpend() overcounts multi-day sessions and has double-count risk**: cost-tracker.ts: v30 \`daily\_costs\` ledger (day, bucket, cost; PK=(day,bucket)). \`bootstrapDailySpend()\` reads ledger only (no session\_state fallback). \`addDailyCost()\` guard: \`!Number.isFinite(cost) || cost <= 0\`. \`computeThrottleDelay\`: \`if (spendFraction>=1.0) return MAX\_THROTTLE\_DELAY\`; pressure²×tanh(overshoot/3) formula. \`getDailyBudget()\` priority: LORE\_DAILY\_BUDGET env → KV \`'daily\_budget'\` → 0. \`aggregateDistillationsBySession\` SQL: \`SUM(CASE WHEN call\_type='batch' THEN token\_count ELSE 0 END) as batch\_tokens\` — proportional split by call count diverges >10% when token distributions differ. \`HISTORICAL\_CACHE\_TTL\_MS\`=5min. Design: prefer EMA-style velocity detection with small incremental delays over hard blocks or large fixed delays. <!-- ...
Now let me check the embedding-worker-types.ts to understand the message protocol, and look at any callers that might have depended on the old auto-fallback behavior:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding-worker-types.ts</path> <type>file</type> <content> 1: /** 2: * Shared message types for the embedding worker thread. 3: * 4: * The embedding worker (`embedding-worker.ts`) runs ONNX inference via 5: * `@huggingface/transformers` in a separate `node:worker_threads` Worker 6: * so the main thread's event loop stays free during inference. This file 7: * defines the message protocol between the main thread (`LocalProvider` 8: * in `embedding.ts`) and the worker. 9: * 10: * Imported by both sides — keep this file free of runtime dependencies. 11: */ 12: 13: // --------------------------------------------------------------------------- 14: // Main thread → Worker 15: // --------------------------------------------------------------------------- 16: 17: /** Request an embedding batch. ...
Found 5 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts:
Line 276: * auto-fallback to remote providers.
Line 395: `Set VOYAGE_API_KEY/OPENAI_API_KEY for automatic remote fallback.`,
Line 571: // auto-fallback to a remote provider or FTS-only search.
Line 674: export function pickRemoteFallback(): {
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/entities.ts:
Line 365: * 2. `git config user.name` / `user.email` (auto-detect fallback)
Let me check who calls pickRemoteFallback() and also look at the stale comments/messages referencing auto-fallback:
Found 8 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/test/embedding.test.ts:
Line 24: pickRemoteFallback,
Line 215: describe("pickRemoteFallback", () => {
Line 234: expect(pickRemoteFallback()).toBeNull();
Line 239: const result = pickRemoteFallback();
Line 245: const result = pickRemoteFallback();
Line 252: const result = pickRemoteFallback();
Line 258: expect(pickRemoteFallback()).toBeNull();
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts:
Line 674: export function pickRemoteFallback(): {
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts</path>
<type>file</type>
<content>
270: this.dimensions = dimensions;
271: }
272:
273: /**
274: * Ensure the worker thread is running. Worker startup failure is
275: * surfaced as `LocalProviderUnavailableError` to trigger the existing
276: * auto-fallback to remote providers.
277: */
278: private async ensureWorker(): Promise<void> {
279: if (this.workerReady) return;
(Showing lines 270-279 of 1530. Use offset=280 to continue.)
</content>
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts</path>
<type>file</type>
<content>
388: this.workerInitError = msg.error;
389: this.workerReady = false;
390: localProviderKnownBroken = true;
391: if (!localProviderErrorLogged) {
392: localProviderErrorLogged = true;
393: log.error(
394: `local embedding provider failed to init: ${msg.error}. ` +
395: `Set VOYAGE_API_KEY/OPENAI_API_KEY for automatic remote fallback.`,
396: new Error(`embedding worker init failed: ${msg.error}`),
397: );
(Showing lines 388-397 of 1530. Use offset=398 to continue.)
</content>
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts</path>
<type>file</type>
<content>
565:
566: switch (providerName) {
567: case "local": {
568: // Construct the provider optimistically — the ONNX model init
569: // happens lazily in the worker thread on first `embed()` call.
570: // If it fails, `LocalProviderUnavailableError` triggers the
571: // auto-fallback to a remote provider or FTS-only search.
572: cachedProvider = new LocalProvider(model, cfg.dimensions);
573: break;
574: }
(Showing lines 565-574 of 1530. Use offset=575 to continue.)
</content>
Now let me check whether pickRemoteFallback is called from any production code (not just tests), and look for the race condition around process.exit(1) in the worker and the pending requests:
Found 8 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/test/embedding.test.ts:
Line 24: pickRemoteFallback,
Line 215: describe("pickRemoteFallback", () => {
Line 234: expect(pickRemoteFallback()).toBeNull();
Line 239: const result = pickRemoteFallback();
Line 245: const result = pickRemoteFallback();
Line 252: const result = pickRemoteFallback();
Line 258: expect(pickRemoteFallback()).toBeNull();
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts:
Line 674: export function pickRemoteFallback(): {
Found 6 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding-worker.ts:
Line 258: function isOomError(msg: string): boolean {
Line 278: function isWasmFatalError(msg: string): boolean {
Line 286: if (isOomError(msg)) return true;
Line 361: if (!isOomError(raw) || !tokenizer) throw err;
Line 403: if (isWasmFatalError(raw)) {
Line 413: const msg = isOomError(raw)
Now let me examine the interaction between the worker's process.exit(1) and the error message handler — specifically, when the worker sends an error message and then exits, there's a timing question about which main-thread handler fires first:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts</path>
<type>file</type>
<content>
348: this.worker.on("message", (msg: WorkerOutbound) => {
349: switch (msg.type) {
350: case "result": {
351: const pending = this.pendingRequests.get(msg.id);
352: if (pending) {
353: this.pendingRequests.delete(msg.id);
354: this.updateWorkerRef();
355: pending.resolve(msg.vectors);
356: }
357: break;
358: }
359: case "error": {
360: const pending = this.pendingRequests.get(msg.id);
361: if (pending) {
362: this.pendingRequests.delete(msg.id);
363: this.updateWorkerRef();
364: // If the worker reports a WASM-fatal or OOM error, reject with
365: // LocalProviderUnavailableError so callers (embed() → ...
Now let me check for the isFatal regex vs isWasmFatalError consistency more carefully, and look at the worker's error message wrapping:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
393: } catch (err) {
394: // Don't re-post init-error — it was already sent in ensurePipeline().
395: if (!initFailed) {
396: const raw = err instanceof Error ? err.message : String(err);
397:
398: // Fatal WASM errors (e.g. "Aborted()") leave the ONNX runtime in an
399: // unrecoverable state — every subsequent request would also fail,
400: // generating unbounded Sentry events. Report the error for this
401: // request and exit the worker so the main thread marks the provider
402: // as broken and stops sending work.
403: if (isWasmFatalError(raw)) {
404: post({
405: type: "error",
406: id: req.id,
407: error: `WASM fatal error (worker exiting): ${raw}`,
408: });
409: process.exit(1);
410: return; ...
Now I have all the information needed. Let me also quickly check whether there are any other callers that relied on the auto-fallback behavior in embed():
Found 17 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts:
Line 187: export class LocalProviderUnavailableError extends Error {
Line 195: this.name = "LocalProviderUnavailableError";
Line 275: * surfaced as `LocalProviderUnavailableError` to trigger the existing
Line 281: throw new LocalProviderUnavailableError(this.workerInitError);
Line 286: if (localProviderKnownBroken) throw new LocalProviderUnavailableError();
Line 365: // LocalProviderUnavailableError so callers (embed() → isAvailable)
Line 376: pending.reject(new LocalProviderUnavailableError(msg.error));
Line 387: // LocalProviderUnavailableError on all pending + future requests.
Line 400: p.reject(new LocalProviderUnavailableError(msg.error));
Line 415: p.reject(new LocalProviderUnavailableError(err));
Line 435: new ...
Found 22 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding.ts:
Line 19: import { vendorModelInfo } from "./embedding-vendor";
Line 24: } from "./embedding-worker-types";
Line 826: const vec = fromBlob(row.embedding);
Line 850: const vec = fromBlob(row.embedding);
Line 880: const vec = fromBlob(row.embedding);
Line 931: const vec = fromBlob(row.embedding);
Line 1077: const vec = fromBlob(row.embedding);
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/embedding-worker.ts:
Line 29: } from "./embedding-worker-types";
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/ltm.ts:
Line 12: import * as embedding from "./embedding";
Line 1406: embeddingMap.set(row.id, embedding.fromBlob(row.embedding));
Line 1636: ...
Let me check how callers use embed() — specifically recall.ts and others — to see if they catch LocalProviderUnavailableError or rely on the old auto-fallback:
Found 17 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/ltm.ts:
Line 176: if (embedding.isAvailable()) {
Line 220: if (embedding.isAvailable() && input.content !== undefined) {
Line 614: if (sessionContext.trim().length > 20 && embedding.isAvailable()) {
Line 620: const [contextVec] = await embedding.embed([sessionContext], "query");
Line 1598: if (!embedding.isAvailable()) return { promoted: 0, clusters: [] };
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/curator.ts:
Line 671: if (cfg.crossProject && embedding.isAvailable()) {
Line 693: if (result.entitiesCreated > 0 && embedding.isAvailable()) {
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/temporal.ts:
Line 99: if (embedding.isAvailable()) {
Line 122: if ...
Good — all callers gate on isAvailable() before calling embed(), except pattern-echo.ts:111. Let me check that one:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/pattern-echo.ts</path>
<type>file</type>
<content>
100: observations: string;
101: projectPath: string;
102: sessionID: string;
103: llm: LLMClient;
104: model?: { providerID: string; modelID: string };
105: }): Promise<void> {
106: // Rate limit check
107: const lastTime = lastExtraction.get(input.sessionID) ?? 0;
108: if (Date.now() - lastTime < PATTERN_COOLDOWN_MS) return;
109:
110: // Step 1: Embed the new distillation (awaited, not fire-and-forget)
111: const [vec] = await embedding.embed([input.observations], "document");
112: db()
113: .query("UPDATE distillations SET embedding = ? WHERE id = ?")
114: .run(embedding.toBlob(vec), input.distillId);
115:
116: // Step 2: Search for similar distillations across the project (wide net)
117: const pid = ensureProject(input.projectPath);
118: const hits = ...
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/pattern-echo.ts</path>
<type>file</type>
<content>
80: export function detectPatternEchoes(input: {
81: distillId: string;
82: observations: string;
83: projectPath: string;
84: sessionID: string;
85: llm: LLMClient;
86: model?: { providerID: string; modelID: string };
87: }): Promise<void> {
88: const p = _detect(input).catch((err) => {
89: log.error("pattern echo detection failed:", err);
90: });
91: return p;
92: }
93:
94: // ---------------------------------------------------------------------------
95: // Internal implementation
96: // ---------------------------------------------------------------------------
97:
98: async function _detect(input: {
99: distillId: string;
(Showing lines 80-99 of 338. Use offset=100 to continue.)
</content>
Good — pattern-echo.ts calls embed() without gating on isAvailable(), but its caller detectPatternEchoes() wraps in .catch(). Let me check where detectPatternEchoes is called to see if it gates on isAvailable():
Found 3 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/distillation.ts:
Line 14: import { detectPatternEchoes } from "./pattern-echo";
Line 1078: const echoPromise = detectPatternEchoes({
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/pattern-echo.ts:
Line 80: export function detectPatternEchoes(input: {
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/lucky-cactus/packages/core/src/distillation.ts</path>
<type>file</type>
<content>
1070: }
1071:
1072: // Embed the distillation for vector search. When knowledge extraction
1073: // is enabled, also detect behavioral pattern echoes — similar segments
1074: // across sessions indicate implicit user preferences.
1075: // When urgent (e.g., /lore:curate), await so entries are created before
1076: // the curate response is sent. Otherwise fire-and-forget.
1077: if (embedding.isAvailable() && config().knowledge.enabled) {
1078: const echoPromise = detectPatternEchoes({
1079: distillId,
1080: observations: result.observations,
1081: projectPath: input.projectPath,
1082: sessionID: input.sessionID,
1083: llm: input.llm,
1084: model: input.model,
1085: });
1086: if (input.urgent) await echoPromise;
1087: } else if (embedding.isAvailable()) ...
Good — the detectPatternEchoes call at line 1077 is gated by embedding.isAvailable(), so if local is broken, it won't be called. The .catch() in detectPatternEchoes (line 88) is a safety net for any other failures.
Now I have everything I need for a thorough review. Let me compile my findings.
This PR addresses an ONNX embedding worker OOM event storm (11k Sentry events) through four layers: (A) classify numeric OOM as fatal + latch breaker, (B) remove auto-fallback from local→remote, (C) silent FTS-only degrade, (D) numThreads=1 for WASM. The changes are well-scoped and well-reasoned.
(none)
isFatal regex will never match the ^\d{6,}$ pattern — dead branchFile: packages/core/src/embedding.ts:372
The worker wraps OOM error messages before posting them:
// embedding-worker.ts:407
error: `WASM fatal error (worker exiting): ${raw}`
So msg.error arriving at the main-thread handler is always "WASM fatal error (worker exiting): 284792864", never the bare "284792864". The main-thread isFatal regex at line 372:
/^\d{6,}$/.test(msg.error)
will never match because msg.error is prefixed with "WASM fatal error (worker exiting): ". This is a dead branch — but it's not a correctness bug because the /WASM fatal error/i regex at line 371 catches the same message first. The isFatal check is defense-in-depth and correctly handles the prefixed form.
However, if a future code change removes the "WASM fatal error" prefix from the worker's error wrapping, the ^\d{6,}$ regex would need to work, and it currently can't because the other OOM patterns (/out.of.memory|alloc.*fail|oom/i) also won't match a bare numeric code. Actually wait — the isOomError check at line 373 does match ^\d{6,}$ on the raw msg.error, but only if the prefix weren't there. ...