Distillation

Date: Sep 16, 2026

• 🔴 (12:36) User stated native ONNX resolution cleanup must never interrupt WASM fallback; failures, including closeSync() errors, must fail closed.
• 🔴 (12:36) User stated the source-workspace fallback must inspect only the gateway worker package’s own node_modules and never select an arbitrary ancestor’s potentially ABI-incompatible runtime.
• 🔴 (12:36) User stated resolveNativeOrtBindingPath() must never throw; an unresolvable native package is the expected dist-only case and must degrade to WASM rather than crash embedding.
• 🔴 (12:36) User stated native intra-op thread control must never touch ONNX Runtime’s default on an unconstrained host and must avoid raising the thread count above ORT’s physical-core default on hyper-threaded hosts.
• 🟡 (12:36) packages/core/src/ort-native.ts defines native targets in exact order: 1. linux-x64, 2. linux-arm64, 3. darwin-arm64, 4. win32-x64, 5. win32-arm64; platform packages use @loreai/onnxruntime-<os>-<arch>, and the binding filename is onnxruntime_binding.node.
• 🟡 (12:36) Native ONNX compatibility in packages/core/src/ort-native.ts: non-Linux platforms pass libc validation; Linux requires a parseable glibcVersionRuntime of at least 2.27, otherwise native selection fails closed to WASM.
• 🟡 (12:36) validNativeBinding() in packages/core/src/ort-native.ts:150-184 canonicalizes both package and binding paths, requires the binding to remain inside the package and be a regular file, then checks the first 4 bytes for ELF on Linux (7f 45 4c 46), either supported Mach-O magic on Darwin (cf fa ed fe or fe ed fa cf), or MZ on Windows; all errors return null.
• 🟡 (12:36) Published-package validation in resolveNativeOrtBindingPath() requires gateway name @loreai/gateway, platform package name matching @loreai/onnxruntime-<target>, gateway/package/declared versions all equal, loreOrtNodeVersion equal to the bundled expected version, and exact one-element os and cpu arrays matching the runtime.
• 🟡 (12:36) Workspace native resolution in resolveNativeOrtBindingPath() requires the gateway to declare the exact expected onnxruntime-node version in dependencies, devDependencies, or optionalDependencies; validates the resolved manifest name/version; requires bin to remain within the package; and accepts exactly 1 directory matching napi-v\d+.
• 🟡 (12:36) nativeIntraOpThreads() in packages/core/src/ort-native.ts:312-325 floors valid availableParallelism() and cpus().length values, defaults invalid parallelism to 1, and returns the available count only when it is lower than logical CPUs; otherwise it returns undefined. This limits cgroup CPU oversubscription without changing ORT defaults on unconstrained hosts.
• 🟡 (12:37) Working copy tmzmtlss / commit e6d0f1edc850, description fix(core): correct embedding runtime selection and admission, had 18 changed files totaling 1818 insertions and 90 deletions against parent pxrvwsvv / 2cbf40060c67 (main, feat(cch): add seeds for Claude Code up to 2.1.272 (#1780)); Jujutsu reported conflict=false for the working copy, while bookmark fix/responses-private-references had a conflict.
• 🟡 (12:37) Working-copy files were: added packages/core/src/cgroup-memory.ts; modified packages/core/src/embedding/local.ts, packages/core/src/embedding/pool.ts, packages/core/src/embedding-cap.ts, packages/core/src/embedding-worker-types.ts, packages/core/src/embedding-worker.ts, packages/core/src/embedding.ts, packages/core/src/ort-native.ts; added packages/core/test/cgroup-memory.test.ts; modified packages/core/test/embedding-cap.test.ts, packages/core/test/embedding-pool-memory.test.ts, packages/core/test/embedding-pool.test.ts, packages/core/test/embedding-worker-types.test.ts, packages/core/test/ort-native.test.ts, packages/gateway/script/bundle.ts, packages/gateway/script/ort-platform-package.ts, packages/gateway/test/bundle-exports.test.ts, and packages/gateway/test/ort-platform-package.test.ts.
• 🟡 (12:37) isNativeOrtLoadError() in packages/core/src/embedding-worker.ts:135-145 first requires a native-runtime identifier such as onnxruntime_binding.node, an ONNX Runtime shared library, or OrtGetApiBase, then recognizes loader failures including ERR_DLOPEN_FAILED, invalid ELF/Mach-O/Win32, short files, missing shared objects/images/modules, undefined or missing symbols, and dlopen.
• 🟡 (12:37) ensurePipeline() in packages/core/src/embedding-worker.ts requests a fresh worker with { type: "init-needs-wasm", error: msg } when a non-vendored native ONNX addon fails to load, sets wasmRespawnRequested = true, and avoids posting terminal init-error; the triggering caller receives embedding worker awaiting WASM respawn so it settles rather than hangs.
• 🟡 (12:38) Corrupt-model recovery in packages/core/src/embedding-worker.ts: if cached model files are structurally intact and native ONNX cannot parse them, the worker treats this as native-runtime incompatibility and requests a fresh WASM worker; an intact-model failure already on WASM retries loadPipeline() once without purging; a non-intact cache is purged and downloaded once; only a final failure posts init-error. Recoverable diagnostics use console.debug and respect stderrSilenced.
• 🟡 (12:38) npm bundle backend selection in loadPipeline() detects sibling ort-wasm-simd-threaded.mjs and ort-wasm-simd-threaded.wasm; it prefers a validated native @loreai/onnxruntime-<target> addon, but forceWasm skips native after a native parse incompatibility so the respawn genuinely changes backend. Dist-only and unsupported-platform installs use the shipped local WASM files.
• 🟡 (12:38) Assistant reported that direct regressions and guards existed for 3 admission defects: 1. hybrid/multi-mount cgroup selection, 2. fail-closed primary sizing, and 3. reservation of not-yet-resident workers. Planned sequence: validate those exact cases, close the native-loader fallback path, trim the oversized test patch, then run the full gates.
• 🔴 (12:38) User stated clampFreeToContainerLimit() must never raise the memory figure; the operation is monotonic and can only reduce memory use.
• 🟡 (12:38) packages/core/test/embedding-cap.test.ts verifies clampFreeToContainerLimit() leaves 7 GiB unchanged for unconstrained values 0, NaN, and -1; clamps 7 GiB to 512 MiB; clamps to current cgroup headroom including 23 MiB and 0; honors 23 MiB headroom with an unlimited hard limit; and returns 0 for a 512 MiB constrained limit with missing or NaN usage headroom.
• 🟡 (12:38) The monotonic memory-clamp regression iterates limits [0, 256 MiB, 4 GiB, 12 GiB, 128 GiB] and available values [0, 32 MiB, 2 GiB, 64 GiB], requiring every result from a 6 GiB host-free input to be at most 6 GiB.
• 🟡 (12:38) resolveMemoryHeadroom() regressions require the tighter of Linux and Node headroom, ignore Bun’s upstream host-wide figure, honor soft-only Linux headroom, use libuv headroom on constrained non-Linux Node, fail closed for constrained Bun without trustworthy usage, and avoid inventing headroom on an unconstrained host.
• 🟡 (12:38) Cgroup-aware sizing regressions fix PER_WORKER_MEM_BUDGET_BYTES equal to EMBED_WORKER_RESIDENT_RESERVE_BYTES and exactly 1536 MiB; desiredEmbedPoolSize(3 GiB - 1) must be 1 and desiredEmbedPoolSize(3 GiB) must be 2. A 512 MiB cgroup keeps the pool at 1 despite 7 GiB host free, and a 384 MiB cap lowers memoryModelEmbedCap() to MIN_EMBED_TOKENS.
• 🟡 (12:39) packages/core/test/embedding-pool.test.ts:815-840 verifies starting-worker reservation with pool size 4, host free 64 GiB, constrained memory 12 GiB, and available memory 1600 MiB: 3 requests may create exactly 2 workers, with one request on each worker, because the same observed worker budget cannot be spent again before initialization completes.