Distillation

Date: Sep 16, 2026

• 🔴 (12:44) User stated the source-workspace native ONNX Runtime fallback checks only the worker package’s own node_modules; it never selects an arbitrary ancestor’s potentially ABI-incompatible runtime.
• 🔴 (12:44) User stated resolveNativeOrtBindingPath() never throws: an unresolvable native package is an expected dist-only case and must return null, degrading to WASM rather than crashing embedding.
• 🔴 (12:44) User stated nativeIntraOpThreads() must cap native ONNX Runtime only when availableParallelism() < cpus().length; on an unconstrained host it returns undefined and never touches ORT’s default, avoiding an increase from ORT’s physical-core default to a hyper-threaded logical-core count.
• 🟡 (12:44) nativeIntraOpThreads() sanitizes parallelism and logicalCpus to finite integers of at least 1, returns the quota-sized avail when avail < total, and otherwise returns undefined; rationale: native ORT uses host physical-core concurrency, while os.availableParallelism() is cgroup-aware through libuv for cgroup v2 cpu.max, v1 CFS quota, and sched_getaffinity.
• 🟡 (12:45) Working copy tmzmtlss / commit e6d0f1edc850 on bookmark fix/embedding-runtime-admission is titled fix(core): correct embedding runtime selection and admission; parent is pxrvwsvv / 2cbf40060c67 on main, titled feat(cch): add seeds for Claude Code up to 2.1.272 (#1780). Both revisions reported conflict=false; bookmark fix/responses-private-references has conflicts.
• 🟡 (12:45) Working-copy diff spans 18 files with exactly 1,818 insertions and 90 deletions: added packages/core/src/cgroup-memory.ts and packages/core/test/cgroup-memory.test.ts; modified packages/core/src/embedding/local.ts, packages/core/src/embedding/pool.ts, packages/core/src/embedding-cap.ts, packages/core/src/embedding-worker-types.ts, packages/core/src/embedding-worker.ts, packages/core/src/embedding.ts, packages/core/src/ort-native.ts, packages/core/test/embedding-cap.test.ts, packages/core/test/embedding-pool-memory.test.ts, packages/core/test/embedding-pool.test.ts, packages/core/test/embedding-worker-types.test.ts, packages/core/test/ort-native.test.ts, packages/gateway/script/bundle.ts, packages/gateway/script/ort-platform-package.ts, packages/gateway/test/bundle-exports.test.ts, and packages/gateway/test/ort-platform-package.test.ts.
• 🔴 (12:45) User stated WASM linear memory never shrinks: after the first oversized tensor allocation grows the heap, smaller retries in the same worker also fail; recovery must respawn a fresh worker at a lower token cap through EMBED_OOM_EXIT_CODE backoff in embedding.ts, while isWasmFatalError() remains a defensive fallback for OOM posted as an error.
• 🟡 (12:45) isWasmFatalError() in packages/core/src/embedding-worker-types.ts classifies as fatal: 1. /WASM fatal error/i, 2. /\bAborted\b/i, 3. /\bRuntimeError\b/, 4. isOomError(msg), and 5. /is not a function/; the fifth is a safety net for an esbuild CJS + Node v24 callable-pipeline failure, whose primary guard is typeof pipe !== "function" in packages/core/src/embedding-worker.ts.
• 🔴 (12:45) User stated missing optional local-embedding dependencies must be distinguished from installed-but-broken initialization, and an unrelated resolution error such as a missing model file must never be misclassified.
• 🟡 (12:45) isMissingLocalStackError() requires both a module-resolution signal—ERR_MODULE_NOT_FOUND, Cannot find module, Cannot find package, or Could not locate the bindings file—and a reference to @huggingface/transformers, onnxruntime, or sharp; this supports the optional @loreai/core local stack, which can be omitted with --omit=optional to avoid approximately 480 MB of ML runtime.
• 🟡 (12:45) isNativeOrtLoadError() recognizes native runtime references including onnxruntime_binding.node, ONNX Runtime .so/.dylib/.dll libraries, or OrtGetApiBase, combined with load errors including ERR_DLOPEN_FAILED, invalid ELF/Mach-O/Win32, file too short, missing shared library/image/module, undefined or missing symbol, or dlopen.
• 🟡 (12:45) shouldRequestWasmRespawn() returns true only when all four conditions hold: usedNative is true, isVendored is false, fileLooksIntact is true, and isCorruptModelError(errorMessage) is true; canonical case is an npm bundle under Bun where native onnxruntime-node loads but InferenceSession.create() reports protobuf parsing failed for an intact approximately 137 MB model.
• 🟡 (12:45) shouldPostPerRequestError() exists to suppress duplicate or destructive reporting in two cases: initFailed, because ensurePipeline() already posted init-error; and wasmRespawnRequested, because posting a per-request error would reject and drop the pending request before respawnForWasm() can resubmit it to the fresh WASM worker.
• 🟡 (12:46) packages/core/src/embedding-worker.ts contains an inlined isNativeOrtLoadError() copy at line 136 and uses it at line 233; when !vendorModel && usedNativeBinding && isNativeOrtLoadError(msg), the worker logs a debug diagnostic, posts { type: "init-needs-wasm", error: msg }, sets wasmRespawnRequested = true, and returns without posting init-error.
• 🟡 (12:46) Corrupt-model recovery in packages/core/src/embedding-worker.ts is integrity-gated: for a structurally intact model, native parse failure requests a fresh WASM worker because the backend is already committed to the current module graph; an already-WASM parse failure retries loadPipeline() once without purging. For a non-intact model, purgeCachedModel() may purge and redownload once. Recoverable paths use console.debug, gated by stderrSilenced; only final failure posts { type: "init-error", error: initError }.
• 🟡 (12:46) After await initPromise, ensurePipeline() throws embedding worker awaiting WASM respawn when wasmRespawnRequested is true, ensuring the triggering embed settles while the main thread replaces the worker; otherwise a null pipeline throws pipeline init completed but pipe is null.
• 🟡 (12:46) Assistant identified three admission findings as the remaining merge blocker and began testing three hypotheses: 1. the tree may select the wrong cgroup hierarchy, 2. it may overprice unknown primary headroom, and 3. it may reuse one live-memory reserve across burst growth. Assistant committed to adding a regression that fails before changing any surviving defective guard.
• 🟡 (12:46) packages/core/src/cgroup-memory.ts parses /proc/self/cgroup and /proc/self/mountinfo, preferring an explicit cgroup v1 memory-controller membership over a unified cgroup v2 membership on hybrid hosts. It decodes procfs octal escapes, rejects paths lacking a leading /, containing NUL, or containing .., and normalizes with posix.normalize().
• 🟡 (12:46) resolveLinuxCgroupMemoryFiles() evaluates matching cgroup mounts per membership, prefers direct candidates sorted by longest mount root, accepts a namespaced interpretation only when exactly one candidate exists, and otherwise returns null; memoryFiles() rereads procfs on every call so admission does not use a stale cgroup directory or cached discovery miss.
• 🟡 (12:46) readLinuxCgroupMemoryHeadroomBytes() reads cgroup v2 memory.current against both memory.high and memory.max, or cgroup v1 memory.usage_in_bytes against memory.limit_in_bytes; it walks from the selected cgroup directory to the mount point and returns the minimum finite max(0, limit - usage) across boundaries. Invalid, unsafe, missing, or unreadable values return undefined, except a cgroup v2 root mount with root / may omit its own memory files.
• 🟡 (12:47) clampFreeToContainerLimit(hostFree, constrained, available?) treats a positive finite constrained value as a hard limit and a nonnegative finite available value as current headroom; when a finite limit exists but headroom is unavailable it returns 0, otherwise it returns Math.min(hostFree, limit, headroom).
• 🟡 (12:47) resolveMemoryHeadroom(constrained, linuxHeadroom?, runtimeAvailable?) chooses the tightest valid value: when Linux cgroup headroom exists, it returns the minimum of Linux headroom, optional runtime availability, and optional process limit; without Linux headroom, it returns undefined unless both a finite positive process limit and a finite nonnegative runtime value exist, in which case it returns their minimum. Bun’s host-wide availableMemory() is omitted on Linux.
• 🟡 (12:47) clampEmbedCap(n) returns MIN_EMBED_TOKENS for non-finite input and otherwise rounds and clamps to [MIN_EMBED_TOKENS, MODEL_MAX_TOKENS].
• 🟡 (12:47) EMBED_WASM_HEAP_MAX_BYTES is exactly 4 * 1024 * 1024 * 1024, derived from the bundled ort-wasm-simd-threaded memory declaration of 65,536 pages × 64 KiB; comments state a 0.85 usable fraction protects against WASM heap fragmentation and non-attention ONNX Runtime intermediates.