Dashboard › sentry › Session 0Y0vwwPwgrRH
0Y0vwwPwgrRHgmGtMInvestigate Sentry issue LOREAI-GATEWAY-10: Error: Worker embedding failed: pipe is not a function
Context from Sentry:
The stack trace:
at Worker.? (dist/index.cjs:1409:4158) [in-app]
at Worker.emit (node:events:508:28)
at MessagePort.? (node:internal/worker:332:53)
The error message "Worker embedding failed: pipe is not a function" suggests:
.pipe() on somethingYour task:
Read packages/core/src/embedding.ts — find:
on("message") callback).pipe() on a responseRead packages/core/src/embedding-worker.ts — find:
Search the codebase for "pipe is not a function" to see if this error message is constructed somewhere
Search for .pipe( in embedding.ts and embedding-worker.ts
Check if this could be a Node.js version compatibility issue (v24.14.0) where a Worker API changed
The "Worker embedding failed:" prefix suggests the error is caught and re-wrapped — find where that wrapping happens
Return: (1) exact code path that produces the error with file:line references, (2) root cause analysis, (3) proposed fix
I'll investigate this systematically. Let me start by reading the key files and searching for relevant patterns.
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding.ts</path>
<type>file</type>
<content>
1: /**
2: * Embedding integration for vector search.
3: *
4: * Supports multiple embedding providers behind a common interface:
5: * - "local" (default): @huggingface/transformers + nomic-embed-text-v1.5
6: * (768 dims, Matryoshka-capable). Runs ONNX inference in a worker thread.
7: * - "voyage": Voyage AI API (voyage-code-3, 1024 dims)
8: * - "openai": OpenAI API (text-embedding-3-small, 1536 dims)
9: *
10: * Provides embedding generation, pure-JS cosine similarity, and vector search
11: * over the knowledge and distillation tables. All operations are gated behind
12: * `search.embeddings.enabled` config + the provider's API key env var — falls
13: * back silently to FTS-only when unavailable.
14: */
15:
16: import { db } from "./db";
17: import { config } from "./config";
18: import * as log ...
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path> <type>file</type> <content> 1: /** 2: * Embedding worker thread — runs ONNX inference via @huggingface/transformers 3: * off the main thread. 4: * 5: * This file is the entry point for a `node:worker_threads` Worker spawned by 6: * `LocalProvider` in `embedding.ts`. It owns the transformers.js pipeline 7: * and processes embed requests sequentially from a priority queue. Moving 8: * inference here keeps the main thread's event loop free — HTTP requests, 9: * SSE streams, and session APIs are no longer blocked during embedding. 10: * 11: * Communication uses `parentPort` message passing with structured clone. 12: * Float32Array vectors are sent back directly (Bun preserves identity). 13: * 14: * The worker applies Nomic's recommended post-processing: 15: * 1. Mean pooling (via pipeline option) 16: * 2. ...
No files found
Found 1 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding.ts:
Line 377: new Error(`Worker embedding failed: ${msg.error}`),
Now I've found the error wrapping site. Let me search for .pipe( patterns and examine the worker types to understand the full message flow.
No files found
No files found
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker-types.ts</path> <type>file</type> <content> 1: /** 2: * Shared message types for the embedding worker thread. 3: * 4: * The embedding worker (`embedding-worker.ts`) runs ONNX inference via 5: * `@huggingface/transformers` in a separate `node:worker_threads` Worker 6: * so the main thread's event loop stays free during inference. This file 7: * defines the message protocol between the main thread (`LocalProvider` 8: * in `embedding.ts`) and the worker. 9: * 10: * Imported by both sides — keep this file free of runtime dependencies. 11: */ 12: 13: // --------------------------------------------------------------------------- 14: // Main thread → Worker 15: // --------------------------------------------------------------------------- 16: 17: /** Request an embedding batch. ...
Good -- no .pipe( calls in either embedding file. The error message "pipe is not a function" is NOT constructed by lore code. It originates from within the ONNX/transformers.js pipeline. Let me trace this further.
No files found
Found 21 matches /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts: Line 6: * `LocalProvider` in `embedding.ts`. It owns the transformers.js pipeline Line 15: * 1. Mean pooling (via pipeline option) Line 51: * value. The pipeline's built-in `truncation: true` caps at model_max_length Line 65: /** The transformers.js pipeline instance, typed loosely since the exact Line 66: * return type depends on the pipeline task. */ Line 79: let pipe: FeatureExtractionPipeline | null = null; Line 108: * Ensure the transformers.js pipeline is loaded. Lazy — first call triggers Line 109: * the dynamic import + pipeline creation, subsequent calls return immediately. Line 113: if (pipe) return; Line 115: throw new Error(initError ?? ...
Found 1 matches /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding.ts: Line 238: * The worker owns the transformers.js pipeline and processes requests
Critical finding! Line 182 in embedding-worker.ts:
if (!pipe) throw new Error("pipeline init completed but pipe is null");
The variable is named pipe. Now let me check what happens when pipe is null and someone tries to call it as a function, or if there's a scenario where the error "pipe is not a function" could be generated by JavaScript itself.
Found 7 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts:
Line 79: let pipe: FeatureExtractionPipeline | null = null;
Line 113: if (pipe) return;
Line 160: pipe = (await pipeline("feature-extraction", modelId, {
Line 167: tokenizer = (pipe as unknown as { tokenizer: typeof tokenizer })
Line 182: if (!pipe) throw new Error("pipeline init completed but pipe is null");
Line 259: // `ensurePipeline()` (awaited by the caller) guarantees both `pipe` and
Line 261: const pipeline = pipe;
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
255: // (shared with the main thread to prevent classification drift).
256:
257: /** Run inference on `texts` and return per-text vectors. */
258: async function runInference(texts: string[]): Promise<Float32Array[]> {
259: // `ensurePipeline()` (awaited by the caller) guarantees both `pipe` and
260: // `layerNormFn` are set; capture into locals for narrowing.
261: const pipeline = pipe;
262: const layerNorm = layerNormFn;
263: if (!pipeline || !layerNorm) {
264: throw new Error("pipeline not initialized");
265: }
266:
267: // Run feature extraction with mean pooling.
268: // truncation: true caps each text at the model's max length (8192 tokens
269: // for Nomic v1.5) as a last-resort safety net. ...
Now I need to understand the error more precisely. The error message is pipe is not a function. In JavaScript, this is a TypeError that happens when you try to call something that isn't a function. Let me check how pipe is used in the bundled CJS output and look at the pipeline() call more carefully. The key question is: could pipe be assigned a non-function value?
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
155: //
156: // device: "cpu" — in npm mode, transformers.js uses onnxruntime-node
157: // (native CPU). In the compiled binary, onnxruntime-node is redirected
158: // to onnxruntime-web by the build plugin, which handles "cpu" via its
159: // WASM+SIMD backend (API-compatible, ~2x faster on batch workloads).
160: pipe = (await pipeline("feature-extraction", modelId, {
161: dtype: "q8",
162: device: "cpu",
163: })) as unknown as FeatureExtractionPipeline;
164:
165: // Stash a reference to the pipeline's tokenizer for token-level
166: // truncation during OOM retries.
167: tokenizer = (pipe as unknown as { tokenizer: typeof tokenizer })
168: .tokenizer;
169:
170: layerNormFn = layer_norm as typeof layerNormFn;
171: ...
Found 1 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts:
Line 160: pipe = (await pipeline("feature-extraction", modelId, {
Now let me look at the esbuild bundling to understand how the worker gets bundled into the CJS output. The error comes from dist/index.cjs:1409 — the bundled gateway. Let me check how the worker is built.
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/build.ts /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/build-binary-sea.ts
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/script/build.ts
Found 12 matches /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/build.ts: Line 55: "embedding-worker.js", Line 56: 'export * from "../../core/src/embedding-worker.ts";\n', /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/build-binary-sea.ts: Line 384: const workerSrc = join(repoRoot, "packages/core/src/embedding-worker.ts"); /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/bundle.ts: Line 138: entryPoints: [join(packageDir, "..", "core", "src", "embedding-worker.ts")], Line 145: outfile: join(distDir, "embedding-worker.cjs"), Line 156: // ./embedding-worker.js (see core/src/embedding.ts:300-303). Line 159: entryPoints: [join(packageDir, "..", "core", "src", "embedding-worker.ts")], Line 166: outfile: join(distDir, ...
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/build.ts</path> <type>file</type> <content> 1: /** 2: * Build @loreai/gateway. 3: * 4: * Two build modes: 5: * 6: * 1. `bun run script/build.ts` (default) 7: * Produces dist/index.js — publishable ESM bundle for npm. 8: * @loreai/core is external (workspace dep, installed alongside). 9: * 10: * 2. `bun run script/build.ts --binary` 11: * Delegates to `script/build-binary-sea.ts` which produces a 12: * standalone Node SEA binary via fossilize. The legacy Bun 13: * `--compile` pipeline was removed in #551 in favor of Node SEA 14: * because Bun's WASM engine has unfixed bugs that cause ONNX 15: * embedding OOM on all platforms (oven-sh/bun#18145, #25677, #31158). ...
Let me look at the bundle script which creates the actual production CJS output:
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/bundle.ts
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/bundle.ts</path> <type>file</type> <content> 1: /** 2: * Bundle @loreai/gateway into a self-contained CJS package for npm/npx. 3: * 4: * Produces: 5: * dist/index.cjs — single CJS bundle (gateway + core + all JS deps) 6: * dist/bin.cjs — thin CLI wrapper with Node.js version check 7: * 8: * Everything is bundled except: 9: * - node:* built-ins (resolved at runtime) 10: * 11: * Source code is pure Node.js — no `Bun.*` polyfill layer is needed. 12: * (xxHash64 lives in src/xxhash.ts as a standalone module since there is 13: * no Node.js equivalent of `Bun.hash.xxHash64`.) 14: * 15: * Debug IDs are injected into the JS + sourcemap after bundling for Sentry 16: * source map resolution. When SENTRY_AUTH_TOKEN is set, sourcemaps are 17: * uploaded to Sentry and then deleted (they shouldn't ship to users). ...
Now I see the critical detail. The worker is bundled as a separate file (dist/embedding-worker.cjs), not inlined into dist/index.cjs. The error at dist/index.cjs:1409 is the main thread's worker message handler. Let me now understand the actual error path — the "pipe is not a function" error originates from inside the worker and is sent back as a string via post({ type: "error", id: req.id, error: msg }).
Let me trace the exact path:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
256:
257: /** Run inference on `texts` and return per-text vectors. */
258: async function runInference(texts: string[]): Promise<Float32Array[]> {
259: // `ensurePipeline()` (awaited by the caller) guarantees both `pipe` and
260: // `layerNormFn` are set; capture into locals for narrowing.
261: const pipeline = pipe;
262: const layerNorm = layerNormFn;
263: if (!pipeline || !layerNorm) {
264: throw new Error("pipeline not initialized");
265: }
266:
267: // Run feature extraction with mean pooling.
268: // truncation: true caps each text at the model's max length (8192 tokens
269: // for Nomic v1.5) as a last-resort safety net.
270: const output = await pipeline(texts, { pooling: "mean", truncation: true });
271:
272: // Post-process following Nomic's recipe:
273: // 1. ...
Now let me look at line 270 — const output = await pipeline(texts, ...). Here pipeline is the local alias for pipe. The error "pipe is not a function" is a JavaScript TypeError. But wait — in the minified CJS bundle, variable names get mangled. The variable pipe in the source becomes something shorter. But the error message would say the mangled name. Unless...
Let me check if esbuild preserves the name pipe in minified output, or if the error comes from the transformers.js library itself:
No files found
Found 1 matches /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/tool-trace.ts: Line 62: regex: /type error|is not a function|undefined is not|cannot read propert/,
Now let me think about this more carefully. The error pipe is not a function is a JavaScript TypeError. It means somewhere code does pipe(...) and pipe is not a function. In the minified CJS bundle, pipe could be a minified variable name that happens to collide with the source name pipe.
But wait — the stack trace says the error is at dist/index.cjs:1409:4158 which is the main thread bundle, not the worker. The worker is dist/embedding-worker.cjs. So the error happens in the main thread's on("message") handler.
Let me re-read the stack trace:
at Worker.? (dist/index.cjs:1409:4158) [in-app]
at Worker.emit (node:events:508:28)
at MessagePort.? (node:internal/worker:332:53)
This is the worker.on("message") callback at line 350 of embedding.ts. The error is thrown INSIDE that callback. But looking at the callback code (lines 350-404), there's no .pipe() call at all — it only accesses msg.type, msg.id, msg.vectors, msg.error, this.pendingRequests.
Wait. Let me re-read the error message more carefully: "Worker embedding failed: pipe is not a function". This is the wrapped error from line 377:
new Error(`Worker embedding failed: ${msg.error}`)
So msg.error = "pipe is not a function". This means the WORKER sent back { type: "error", id: ..., error: "pipe is not a function" }. The raw error came from inside the worker!
Let me trace where in the worker this error originates:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
305: async function processEmbed(req: EmbedRequest): Promise<void> {
306: try {
307: await ensurePipeline();
308:
309: // Try inference at full length first. On ONNX OOM, retry with
310: // progressively halved token limits using the real tokenizer.
311: // This preserves maximum semantic content for normal texts while
312: // handling dense-token content (code, CJK, base64) adaptively.
313: //
314: // attempt 0 = original texts (no truncation)
315: // attempt 1 = truncated to 4096 tokens
316: // attempt 2 = truncated to 2048 tokens
317: // attempt 3 = truncated to 1024 tokens
318: let texts = req.texts;
319: let lastError: Error | undefined;
320:
321: for (let attempt = 0; attempt <= OOM_MAX_RETRIES; attempt++) {
322: try {
323: const vectors ...
Now I have the full picture. The error path is:
processEmbed() → runInference() → pipeline(texts, { pooling: "mean", truncation: true }) at line 270 throws TypeError: pipe is not a functionif (!isOomError(raw) || !tokenizer) throw err; — it throwspost({ type: "error", id: req.id, error: msg }) where msg = raw = "pipe is not a function"new Error("Worker embedding failed: pipe is not a function")Now the key question: WHY does pipeline(texts, ...) throw TypeError: pipe is not a function? This is happening INSIDE @huggingface/transformers. Let me check what version is being used and look for known issues:
Found 1 matches /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/package.json: Line 28: "@huggingface/transformers": "^3.7.1",
No files found
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/package.json</path>
<type>file</type>
<content>
1: {
2: "name": "@loreai/core",
3: "version": "0.24.1",
4: "type": "module",
5: "license": "FSL-1.1-Apache-2.0",
6: "description": "Shared memory engine for Lore — three-tier storage, distillation, gradient context management",
7: "main": "./dist/node/index.js",
8: "types": "./dist/node/index.d.ts",
9: "exports": {
10: ".": {
11: "types": "./dist/types/index.d.ts",
12: "bun": "./dist/bun/index.js",
13: "default": "./dist/node/index.js"
14: }
15: },
16: "imports": {
17: "#db/driver": {
18: "bun": "./src/db/driver.bun.ts",
19: "default": "./src/db/driver.node.ts"
20: }
21: },
22: "scripts": {
23: "typecheck": "tsc --noEmit",
24: "build": "tsx script/build.ts"
25: },
26: "dependencies": {
27: "@huggingface/hub": "2.11.0",
28: ...
(no output)
The node_modules isn't populated in this worktree. Let me look at the lockfile and also search the transformers.js source for "pipe" usage patterns to understand what could throw this error:
No files found
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/pnpm-lock.yaml
Found 3 matches /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/pnpm-lock.yaml: Line 38: '@huggingface/transformers': Line 920: '@huggingface/transformers@3.8.1': Line 4592: '@huggingface/transformers@3.8.1':
So @huggingface/transformers@3.8.1 is installed. Now let me understand the core issue. The pipeline() function from @huggingface/transformers returns a pipeline object. At line 160-163:
pipe = (await pipeline("feature-extraction", modelId, {
dtype: "q8",
device: "cpu",
})) as unknown as FeatureExtractionPipeline;
The pipeline is called as a function at line 270:
const output = await pipeline(texts, { pooling: "mean", truncation: true });
The error "pipe is not a function" means the pipeline variable (local alias for pipe) is not callable. But wait — in the minified CJS bundle, pipe might get renamed. The error message from a TypeError uses the variable name as seen in the source/runtime.
Let me check the bundled dist output to see if it exists:
No files found
No files found
No built dist in this worktree. Let me now focus on the root cause. Let me look at the transformers.js pipeline() API more carefully — specifically, can the returned pipeline object be non-callable under certain conditions?
/**
* @file Pipelines provide a high-level, easy to use, API for running machine learning models.
*
* **Example:** Instantiate pipeline using the `pipeline` function.
* ```javascript
* import { pipeline } from '@huggingface/transformers';
*
* const classifier = await pipeline('sentiment-analysis');
* const output = await classifier('I love transformers!');
* // [{'label': 'POSITIVE', 'score': 0.999817686}]
* ```
*
* @module pipelines
*/
import {
AutoTokenizer,
PreTrainedTokenizer,
} from './tokenizers.js';
import {
AutoModel,
AutoModelForSequenceClassification,
AutoModelForAudioClassification,
AutoModelForTokenClassification,
AutoModelForQuestionAnswering,
AutoModelForMaskedLM,
AutoModelForSeq2SeqLM,
AutoModelForSpeechSeq2Seq,
AutoModelForTextToWaveform,
AutoModelForTextToSpectrogram,
AutoModelForCTC,
AutoModelForCausalLM,
AutoModelForVision2Seq,
AutoModelForImageClassification,
...
Now I see the critical piece. The Pipeline class extends Callable. Let me find the Callable class:
No files found
/**
* A base class for creating callable objects.
* See [here](https://stackoverflow.com/q/76073890) for more information.
*
* @type {new () => {(...args: any[]): any, _call(...args: any[]): any}}
*/
export const Callable = /** @type {any} */ (class {
/**
* Creates a new instance of the Callable class.
*/
constructor() {
/**
* Creates a closure that delegates to a private method '_call' with the given arguments.
* @type {any}
* @param {...any} args Zero or more arguments to pass to the '_call' method.
* @returns {*} The result of calling the '_call' method.
*/
let closure = function (...args) {
return closure._call(...args)
}
return Object.setPrototypeOf(closure, new.target.prototype)
}
/**
* This method should be implemented in subclasses to provide the
* functionality of the callable object. ...
This is the smoking gun. Let me now look at the FeatureExtractionPipeline class to understand the full picture:
Found 1 matches
/home/byk/.local/share/opencode/tool-output/tool_ea41021980012iGCHYZ6MOBgZm:
Line 1295: export class FeatureExtractionPipeline extends (/** @type {new (options: TextPipelineConstructorArgs) => FeatureExtractionPipelineType} */ (Pipeline)) {
<path>/home/byk/.local/share/opencode/tool-output/tool_ea41021980012iGCHYZ6MOBgZm</path>
<type>file</type>
<content>
1290: * // data: Int8Array [49, 108, 24, ...],
1291: * // dims: [1, 48]
1292: * // }
1293: * ```
1294: */
1295: export class FeatureExtractionPipeline extends (/** @type {new (options: TextPipelineConstructorArgs) => FeatureExtractionPipelineType} */ (Pipeline)) {
1296: /**
1297: * Create a new FeatureExtractionPipeline.
1298: * @param {TextPipelineConstructorArgs} options An object used to instantiate the pipeline. ...
Now I have the complete picture. Let me now verify the Callable pattern and how it interacts with Object.setPrototypeOf in Node.js v24. The Callable class creates a closure function and uses Object.setPrototypeOf to set its prototype to the class's prototype. This is a well-known pattern that makes class instances callable as functions.
The key issue is: when pipeline() returns a FeatureExtractionPipeline instance, the instance IS a function (because Callable creates a closure). But the worker code at line 160-163 casts it:
pipe = (await pipeline("feature-extraction", modelId, {
dtype: "q8",
device: "cpu",
})) as unknown as FeatureExtractionPipeline;
Then at line 261, const pipeline = pipe; and at line 270, await pipeline(texts, ...).
The error "pipe is not a function" would occur if somehow the Callable pattern breaks — i.e., the returned object is not actually a function. Let me check if there's a Node.js v24 issue with Object.setPrototypeOf on closures, or if there's a CJS bundling issue:
Found 3 matches
/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts:
Line 67: type FeatureExtractionPipeline = {
Line 79: let pipe: FeatureExtractionPipeline | null = null;
Line 163: })) as unknown as FeatureExtractionPipeline;
No files found
Now let me look at the FeatureExtractionPipeline type definition in embedding-worker.ts more carefully:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
65: /** The transformers.js pipeline instance, typed loosely since the exact
66: * return type depends on the pipeline task. */
67: type FeatureExtractionPipeline = {
68: (
69: texts: string[],
70: options?: Record<string, unknown>,
71: ): Promise<{
72: dims: number[];
73: data: Float32Array;
74: tolist(): number[][];
75: }>;
76: dispose?(): Promise<void>;
77: };
78:
79: let pipe: FeatureExtractionPipeline | null = null;
80: let tokenizer: {
81: encode(text: string, options?: Record<string, unknown>): number[];
82: decode(ids: number[] | bigint[], options?: Record<string, unknown>): string;
83: } | null = null;
84: let layerNormFn:
(Showing lines 65-84 of 408. Use offset=85 to continue.)
</content>
Now I have all the pieces. Let me also check whether the Callable pattern using Object.setPrototypeOf on a closure is known to break under esbuild CJS minification:
No files found
Let me check one more critical detail — the error says "pipe is not a function" not "pipeline is not a function". In the source code, the variable is called pipe at module scope (line 79) and aliased to pipeline locally in runInference() (line 261). In the minified CJS bundle, these names get mangled. But JavaScript TypeError messages use the expression text, not just the variable name. A TypeError: x is not a function happens when you write x(...) and x isn't callable.
Wait — the error message is "pipe is not a function", and it's the string that the worker sends back to the main thread. Let me re-trace:
const output = await pipeline(texts, ...) — if pipeline (local alias for pipe) is not a function, JavaScript throws TypeError: pipeline is not a function (using the local variable name pipeline, not pipe)pipe = (await pipeline(...)) — here pipeline is the destructured import from @huggingface/transformers. If pipeline from transformers.js is not a function, it throws TypeError: pipeline is not a functionBut the error says "pipe is not a function", not "pipeline is not a function". In the minified bundle, variable names are mangled. Let me verify — the worker is bundled separately as embedding-worker.cjs with minify: true. In the minified output, the module-level variable pipe gets renamed to some short name, and the function parameter pipeline from the import also gets renamed.
Actually, wait. In a minified bundle, the error would say something like "a is not a function" or "e is not a function" — not "pipe". Unless pipe is a property access that survives minification. Let me look again...
Actually, in the context from the Sentry issue, the stack trace shows dist/index.cjs:1409:4158 — this is in the main bundle, not the worker bundle. ...
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
110: * On failure, marks the worker as permanently broken and posts `init-error`.
111: */
112: async function ensurePipeline(): Promise<void> {
113: if (pipe) return;
114: if (initFailed)
115: throw new Error(initError ?? "pipeline init previously failed");
116:
117: if (!initPromise) {
118: initPromise = (async () => {
119: const transformers = await import("@huggingface/transformers");
120: const { pipeline, env, layer_norm } = transformers;
121:
122: // Configure transformers.js environment
123: env.allowRemoteModels = !vendorModel;
124: env.allowLocalModels = true;
125:
126: if (vendorModel) {
127: // Binary mode: point at pre-extracted model files on disk.
128: env.localModelPath = vendorModel.localModelPath;
129: ...
Now I have a complete understanding. Let me verify one more thing — whether this is a CJS bundling issue with the Callable pattern. The Callable class uses Object.setPrototypeOf(closure, new.target.prototype) in its constructor, which is critical for making the pipeline instance callable. When esbuild bundles @huggingface/transformers into the CJS worker bundle, this pattern should be preserved. But let me check if there's a known issue:
Found 5 matches /home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/gateway/script/bundle.ts: Line 57: // onnxruntime-node: .node native binaries that esbuild can't handle. Line 59: const external = ["node:*", "onnxruntime-node", "sharp"]; Line 117: external: ["bun:*", "node:*", "onnxruntime-node", "sharp"], Line 144: external: ["onnxruntime-node", "sharp"], Line 165: external: ["bun:*", "node:*", "onnxruntime-node", "sharp"],
Note something critical at line 144: the CJS embedding worker build has external: ["onnxruntime-node", "sharp"] but does NOT have "node:*" in externals! This means Node.js built-in modules like node:worker_threads, node:path, etc. are bundled inline. However, @huggingface/transformers IS bundled into the worker CJS file.
Now let me think about the actual root cause. The Callable pattern in transformers.js:
constructor() {
let closure = function (...args) {
return closure._call(...args)
}
return Object.setPrototypeOf(closure, new.target.prototype)
}
This works because Object.setPrototypeOf(closure, new.target.prototype) is returned from the constructor, so new Pipeline(...) returns the closure function instead of the normal this. The closure IS a function, so calling pipe(texts, ...) should work.
BUT — here's the key insight: if onnxruntime-node fails to load or initialize (it's a native .node addon), the pipeline() function might return something unexpected or throw an error that gets partially caught. Let me check if there's a scenario where pipe gets assigned a non-function value.
Actually, let me reconsider. The error is "pipe is not a function" — in the minified worker bundle. Let me look at what esbuild does with the Callable pattern when it minifies. esbuild with target: "node22" and minify: true might transform class syntax in a way that breaks the Callable hack.
Let me search for Node.js 24 compatibility issues with the Callable pattern:
Issues · huggingface/transformers.js · GitHub
[Skip to content](#start-of-content)
## Navigation Menu
Toggle navigation
[](/)
[Sign in](/login?return_to=https%3A%2F%2Fgithub.com%2Fhuggingface%2Ftransformers.js%2Fissues%3Fq%3Dis%253Aissue%2B%2522is%2Bnot%2Ba%2Bfunction%2522%2Bcallable)
Appearance settings
- Platform
- AI CODE CREATION
- [
GitHub CopilotWrite better code with AI
](https://github.com/features/copilot)
- [
GitHub Copilot appDirect agents from issue to merge
](https://github.com/features/ai/github-app)
- [
MCP RegistryNewIntegrate external tools
](https://github.com/mcp)
- ...
Let me check issue #996 which has a similar pattern of minified "is not a function" error, and also check if there's a Node 24 specific issue:
Uncaught (in promise) TypeError: se\[e\] is not a function · Issue #996 · huggingface/transformers.js · GitHub
[Skip to content](#start-of-content)
## Navigation Menu
Toggle navigation
[](/)
[Sign in](/login?return_to=https%3A%2F%2Fgithub.com%2Fhuggingface%2Ftransformers.js%2Fissues%2F996)
Appearance settings
- Platform
- AI CODE CREATION
- [
GitHub CopilotWrite better code with AI
](https://github.com/features/copilot)
- [
GitHub Copilot appDirect agents from issue to merge
](https://github.com/features/ai/github-app)
- [
MCP RegistryNewIntegrate external tools
](https://github.com/mcp)
- DEVELOPER WORKFLOWS
- [
ActionsAutomate any workflow
...
Incorrect model selection by pipeline dtype parameter · Issue #1581 · huggingface/transformers.js · GitHub
[Skip to content](#start-of-content)
## Navigation Menu
Toggle navigation
[](/)
[Sign in](/login?return_to=https%3A%2F%2Fgithub.com%2Fhuggingface%2Ftransformers.js%2Fissues%2F1581)
Appearance settings
- Platform
- AI CODE CREATION
- [
GitHub CopilotWrite better code with AI
](https://github.com/features/copilot)
- [
GitHub Copilot appDirect agents from issue to merge
](https://github.com/features/ai/github-app)
- [
MCP RegistryNewIntegrate external tools
](https://github.com/mcp)
- DEVELOPER WORKFLOWS
- [
ActionsAutomate any workflow
...
Those issues are unrelated. Let me now focus on what I know and trace the exact root cause. The key insight is that the error occurs in the minified CJS worker bundle. Let me look at how esbuild handles the Callable class pattern and whether new.target works correctly in CJS mode:
No files found
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
103: let initPromise: Promise<void> | null = null;
104: let initFailed = false;
105: let initError: string | null = null;
106:
107: /**
108: * Ensure the transformers.js pipeline is loaded. Lazy — first call triggers
109: * the dynamic import + pipeline creation, subsequent calls return immediately.
110: * On failure, marks the worker as permanently broken and posts `init-error`.
111: */
112: async function ensurePipeline(): Promise<void> {
(Showing lines 103-112 of 408. Use offset=113 to continue.)
</content>
Now I have enough information to provide a complete analysis. Let me also quickly check one more thing — whether the initPromise guarding logic has a race condition that could result in pipe being non-null but not a function:
<path>/home/byk/.local/share/opencode/worktree/51d1015fd1f616acf9a5ffaad9331ec6b304586b/clever-circuit/packages/core/src/embedding-worker.ts</path>
<type>file</type>
<content>
112: async function ensurePipeline(): Promise<void> {
113: if (pipe) return;
114: if (initFailed)
115: throw new Error(initError ?? "pipeline init previously failed");
116:
117: if (!initPromise) {
118: initPromise = (async () => {
119: const transformers = await import("@huggingface/transformers");
120: const { pipeline, env, layer_norm } = transformers;
121:
122: // Configure transformers.js environment
123: env.allowRemoteModels = !vendorModel;
124: env.allowLocalModels = true;
125:
126: if (vendorModel) {
127: // Binary mode: point at pre-extracted model files on disk.
128: env.localModelPath = vendorModel.localModelPath;
129: env.allowRemoteModels = false;
130: }
131:
132: // Force single-threaded WASM execution to ...
Now I have complete understanding of the issue. Let me compile my findings.
The error traverses two threads:
Worker thread (packages/core/src/embedding-worker.ts, bundled as dist/embedding-worker.cjs):
processEmbed() calls await ensurePipeline()ensurePipeline() checks if (pipe) return; -- pipe is truthy (already initialized), so it returns immediatelyrunInference() captures const pipeline = pipe;const output = await pipeline(texts, { pooling: "mean", truncation: true }) -- throws TypeError: pipe is not a function (the variable name pipe is the minified name in the CJS bundle)const raw = err instanceof Error ? err.message : String(err) => raw = "pipe is not a function"isOomError("pipe is not a function") returns false, tokenizer is truthy, so the error is re-thrownraw = "pipe is not a function"isWasmFatalError("pipe is not a function") returns false (no "Aborted", "RuntimeError", or numeric pattern)isOomError(raw) returns falsepost({ type: "error", id: req.id, error: "pipe is not a function" }) -- sends the raw error string to the main threadMain thread (packages/core/src/embedding.ts, bundled into dist/index.cjs):
worker.on("message", (msg: WorkerOutbound) => { ... }) receives the messagecase "error": branch matchesisWasmFatalError("pipe is not a function") returns falsenew Error("Worker embedding failed: pipe is not a function") and calls pending.reject(...) -- this is the error that surfaces in Sentry at dist/index.cjs:1409:4158The ...