Distillation

Date: Sep 17, 2026

• 🔴 (09:49) User stated finite Node/libuv memory headroom must always be accepted without imposing a hard-limit requirement, including on constrained non-Linux Node runtimes.
• 🔴 (09:49) User stated a host logger or sink must never interrupt embedding-worker stream consumption.
• 🔴 (09:49) User stated embedding-pool sizing must never multiply memory pressure across workers; each worker must size its token cap from its share of free memory rather than independently claiming the full-memory budget.
• 🔴 (09:49) User stated memory-cap reconciliation must be monotonic: it may only lower the figure, never raise it, so it never increases memory use.
• 🟡 (09:49) packages/core/src/embedding/local.ts now imports readLinuxCgroupMemoryHeadroomBytes from ../cgroup-memory and resolveMemoryHeadroom from ../embedding-cap.
• 🟡 (09:49) packages/core/src/embedding/local.ts adds test override testAvailableMemoryBytes: number | null and exported helper _setAvailableMemoryForTest(bytes: number | null).
• 🟡 (09:49) Available-memory calculation in packages/core/src/embedding/local.ts calls resolveMemoryHeadroom(constrained, linuxHeadroom, fn()), combining constrained-memory state, Linux cgroup headroom, and Node/libuv available-memory reporting.
• 🟡 (09:49) Embedding-pool OOM root cause: the pool could grow to N workers while each worker independently sized its token cap against roughly half of total free memory; aggregate allocation exhausted host RAM. Native ONNX OOM causes an uncatchable SIGKILL, so the normal ×0.7 cap backoff cannot protect this path.
• 🟡 (09:49) Embedding-pool OOM fix has two guards: (1) size each worker from free memory / pool ceiling; (2) re-clamp maxTokens against current free memory on every request rather than relying only on the construction-time cap.
• 🟡 (09:49) In LocalEmbeddingProvider.embed() in packages/core/src/embedding/local.ts, each EmbedRequest.maxTokens is set through this.effectiveMaxTokens() so memory consumed by sibling workers and live sessions can lower the cap before dispatch.
• 🟡 (09:49) The OOM/WASM respawn path in packages/core/src/embedding/local.ts also recalculates every pending request’s p.payload.maxTokens through this.effectiveMaxTokens() immediately before resubmitting it to the fresh worker.
• 🟡 (09:49) packages/core/test/embedding-cap-persistence.test.ts covers lore:embedding_cap persistence: 1. absent row returns null; 2. _persistEmbedCap(1234) round-trips cap 1234 and records positive freeMemBytes; 3. writes 1000 then 500 use ON CONFLICT and retain 500; 4. corrupt value "not-json{" returns null without throwing; 5. JSON missing required fields returns null; 6. _persistEmbedCap(1000, 4 * 1024 * 1024 * 1024, 3000) round-trips knownBadCap=3000; 7. knownBadCap is omitted for both the default and explicit 0; 8. persisted non-positive 0 or nonnumeric "x" knownBadCap values are ignored.
• 🟡 (09:50) Added OOM regression suite packages/core/test/embedding-pool-memory.test.ts, using CapturingWorker to snapshot exact per-request EmbedMsg payloads because production payloads are mutated in place during OOM resubmission.
• 🟡 (09:50) packages/core/test/embedding-pool-memory.test.ts resets _setTestWorkerFactory, _setEmbedPoolSizeForTest, _setPoolFreememForTest, _setContainerFreeForTest, _setAvailableMemoryForTest, _setConstrainedMemoryForTest, and the local provider probe after each test; it also preserves and restores VOYAGE_API_KEY, OPENAI_API_KEY, and the prior provider.
• 🟡 (09:50) Pool-memory regression test 1 persists cap 8192 with freeMemBytes=0, sets pool ceiling 2 and free memory 6 * GB, then verifies the posted cap equals memoryModelEmbedCap((6 * GB) / 2) and is strictly below memoryModelEmbedCap(6 * GB).
• 🟡 (09:50) Pool-memory regression test 2 persists cap 8192, sets pool ceiling 1, container free memory 64 * GB, constrained memory 12 * GB, and available memory Number.NaN; it verifies unknown constrained primary-worker headroom forces maxTokens to MIN_EMBED_TOKENS.
• 🟡 (09:50) Pool-memory regression test 3 persists cap 8192, sets pool ceiling 1, container free memory 64 * GB, constrained memory 0, and available memory Number.NaN; it verifies failed cgroup accounting without a hard limit also forces maxTokens to MIN_EMBED_TOKENS.
• 🟡 (09:50) Pool-memory regression test 4 keeps one worker, lowers free memory from 6 * GB to 2 * GB between requests, and verifies the first and second caps equal memoryModelEmbedCap(6 * GB) and memoryModelEmbedCap(2 * GB) respectively, with the second strictly lower.
• 🟡 (09:50) Pool-memory regression test 5 starts at 16 * GB, drops free memory to 2 * GB, emits worker exit code EMBED_OOM_EXIT_CODE, and verifies the replacement worker receives the same request ID with maxTokens=memoryModelEmbedCap(2 * GB), strictly below backoffEmbedCap(first.maxTokens); this pins current-memory clamping on the native-SIGKILL retry path rather than relying only on ×0.7 backoff.