Distillation

Date: Sep 16, 2026

• 🔴 (13:16) User stated MODEL_MAX_TOKENS is the Nomic v1.5 maximum sequence length and the adaptive embedding cap never exceeds it.
• 🔴 (13:16) User stated each embedding OOM lowers the token cap by ×0.7 and respawns the worker with a fresh WASM heap.
• 🔴 (13:16) User stated MIN_EMBED_TOKENS = 256.
• 🔴 (13:16) User stated that if free memory never improves, the embedding runtime must never re-probe upward.
• 🔴 (13:16) User stated persisted-cap reconciliation must prevent a memory-rich reboot from re-probing back up to a cap that the WASM heap has already shown it cannot support.
• 🔴 (13:16) User stated embedding pool sizing must never fall below 1 because the primary worker always runs.
• 🔴 (13:16) User stated that inside a memory-capped container, os.freemem() reports host memory and can therefore be unsafe; the WASM path may self-limit before the ×0.7 OOM backoff ever fires.
• 🔴 (13:16) User stated container-memory clamping is monotonic: it may only reduce the memory figure and is otherwise a no-op on an unconstrained host.
• 🔴 (13:16) User stated upward cap re-probing never steps down; if the current memory-model ceiling is below the learned cap, the re-probe path cannot return a cap above that ceiling.
• 🟡 (13:16) Technical search located cgroup handling in /home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/cgroup-memory.ts, including resolveLinuxCgroupMemoryFiles() at line 75 and readLinuxCgroupMemoryHeadroomBytes() at line 152.
• 🟡 (13:16) Technical search located memory sizing in /home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding-cap.ts, including desiredEmbedPoolSize() at line 125.
• 🟡 (13:16) Technical search located runtime headroom logic in /home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/local.ts: availableMemoryHeadroom() at line 99, containerFreeBytes() at line 130, learned-cap free-memory capture at line 567, live memory-model cap calculation at line 571, and additional free-memory reads at lines 868 and 937.
• 🟡 (13:16) Technical search located pool headroom use in /home/byk/Code/opencode-lore-embedding-runtime/packages/core/src/embedding/pool.ts line 285: const headroom = availableMemoryHeadroom();.
• 🟡 (13:16) Cgroup coverage exists in /home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/cgroup-memory.test.ts, /home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-pool.test.ts, /home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-pool-memory.test.ts, /home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-oom-recovery.test.ts, and /home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-cap.test.ts.
• 🟡 (13:17) /home/byk/Code/opencode-lore-embedding-runtime/packages/core/test/embedding-cap.test.ts verifies clampFreeToContainerLimit() behavior: unconstrained 0, NaN, and negative limits leave host free memory unchanged; roomy containers preserve host free memory; constrained containers clamp to current cgroup headroom; discovered headroom is honored even with an unlimited hard limit; unknown usage in a constrained cgroup fails closed to 0; and the result never exceeds the original host-free figure.
• 🟡 (13:17) resolveMemoryHeadroom() tests require the tighter Linux or Node headroom, honor a soft-only Linux boundary, use libuv headroom on constrained non-Linux Node runtimes, fail closed for constrained Bun without trustworthy usage, and avoid inventing headroom on unconstrained hosts.
• 🟡 (13:17) Cgroup-aware sizing tests define PER_WORKER_MEM_BUDGET_BYTES = EMBED_WORKER_RESIDENT_RESERVE_BYTES = 1536 * MB; desiredEmbedPoolSize(3 * GB - 1) returns 1, while desiredEmbedPoolSize(3 * GB) returns 2.
• 🟡 (13:17) Tiny-container tests use a host report of 7 * GB with a 512 * MB cgroup cap: clamping keeps the pool at the primary worker (1), whereas the unclamped host figure would produce DEFAULT_MAX_EMBED_POOL.
• 🟡 (13:17) Token-cap sizing tests use a 384 * MB constrained headroom: memoryModelEmbedCap() floors at MIN_EMBED_TOKENS, while an unclamped 7 * GB host figure reaches EMBED_TOKEN_CEILING. The rationale is that after the approximately 680 MB baseline there is no room for the O(L²) attention allocation.
• 🟡 (13:17) Roomy-host regression coverage models Burak’s box with constrainedMemory() = 12 GiB and host-free values 1 * GB, 3 * GB, Math.round(6.72 * GB), and 7 * GB; clamping must be byte-identical and leave both desiredEmbedPoolSize() and memoryModelEmbedCap() unchanged.
• 🟡 (13:17) clampEmbedCap() tests require values below the floor to become MIN_EMBED_TOKENS, values above the ceiling to become MODEL_MAX_TOKENS, finite in-range values to be rounded, and NaN, positive infinity, and negative infinity to resolve to MIN_EMBED_TOKENS.
• 🟡 (13:17) backoffEmbedCap() tests require approximately ×0.7 reduction per step: backoffEmbedCap(MODEL_MAX_TOKENS) equals Math.round(8192 * 0.7) or 5734, and backoffEmbedCap(1000) equals 700.