Distillation

Date: Sep 16, 2026

• 🔴 (13:17) User stated native ONNX embedding OOM is an uncatchable SIGKILL, so the ×0.7 OOM backoff never fires; the only defense is preventing over-allocation in the first place.
• 🔴 (13:17) User stated the embedding-pool OOM reported by Onur occurred because the pool grew to N workers and each independently sized its token cap to consume approximately half of free memory, causing their aggregate allocation to exhaust host RAM.
• 🔴 (13:17) User stated two required embedding-pool guards: (1) each worker sizes its token cap from free / ceiling; (2) the cap is re-clamped against current free memory on every request rather than only at construction.
• 🔴 (13:17) User stated the primary embedding worker is always present and has its own OOM backoff rather than being governed by pool sizing.
• 🔴 (13:17) User stated the single-worker regression case never grows the pool because its ceiling is 1.
• 🔴 (13:17) User stated invalid environment configuration must fall through to undefined, never NaN, and this behavior must be testable without spinning up workers.
• 🟡 (13:17) /home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-pool-memory.test.ts contains a 229-line regression suite, embedding pool memory sizing (OOM regression), using CapturingWorker to snapshot exact per-request EmbedMsg payloads (type, id, maxTokens) posted to worker threads.
• 🟡 (13:17) The first pool-memory regression test persists cap 8192, forces pool ceiling 2, supplies 6 * GB free memory, and requires the posted cap to equal memoryModelEmbedCap((6 * GB) / 2) and be strictly below memoryModelEmbedCap(6 * GB).
• 🟡 (13:17) The unknown-headroom regression test persists cap 8192, forces pool size 1, reports 64 * GB container-free memory with a 12 * GB constrained limit, injects Number.NaN available memory, and requires the primary worker request to use MIN_EMBED_TOKENS.
• 🟡 (13:17) The live-memory regression test forces ceiling 1, sends a first request at 6 * GB, then drops free memory to 2 * GB; the same worker must receive caps memoryModelEmbedCap(6 * GB) and memoryModelEmbedCap(2 * GB) respectively, with the second strictly lower.
• 🟡 (13:17) The OOM-respawn regression test starts at 16 * GB, drops free memory to 2 * GB, emits EMBED_OOM_EXIT_CODE, and requires a fresh worker to resubmit the same request ID with memoryModelEmbedCap(2 * GB), strictly below backoffEmbedCap(first.maxTokens). The retry cap is therefore min(backoff(first), current-memory cap), not merely the ×0.7 backoff.
• 🟡 (13:17) The working copy for revision tmzmtlss / commit e6d0f1edc850, description fix(core): correct embedding runtime selection and admission, bookmark fix/embedding-runtime-admission, contains added files packages/core/src/cgroup-memory.ts and packages/core/test/cgroup-memory.test.ts.
• 🟡 (13:17) The same working copy modifies packages/core/src/embedding/local.ts, packages/core/src/embedding/pool.ts, packages/core/src/embedding-cap.ts, packages/core/src/embedding-worker-types.ts, packages/core/src/embedding-worker.ts, packages/core/src/embedding.ts, packages/core/src/ort-native.ts, packages/core/test/embedding-cap.test.ts, packages/core/test/embedding-pool-memory.test.ts, packages/core/test/embedding-pool.test.ts, packages/core/test/embedding-worker-types.test.ts, packages/core/test/ort-native.test.ts, packages/gateway/script/bundle.ts, packages/gateway/script/ort-platform-package.ts, packages/gateway/test/bundle-exports.test.ts, and packages/gateway/test/ort-platform-package.test.ts.
• 🟡 (13:17) The working copy’s parent is revision pxrvwsvv / commit 2cbf40060c67 on main, description feat(cch): add seeds for Claude Code up to 2.1.272 (#1780).
• 🟡 (13:17) Jujutsu reported a bookmark conflict for fix/responses-private-references; suggested inspection with jj bookmark list and resolution with jj bookmark set <name> -r <rev>.