Distillation

Date: Sep 16, 2026

• 🔴 (23:53) User stated the adaptive embedding cap must never exceed the Nomic v1.5 maximum sequence length.
• 🔴 (23:53) User stated each embedding OOM lowers the cap by a factor of 0.7 and respawns the worker with a fresh heap.
• 🔴 (23:53) User stated that when free memory never improves, upward embedding-cap re-probing must never occur.
• 🔴 (23:53) User stated a persisted failed-cap bound must ensure that even a memory-rich reboot never re-probes up to a cap at which the WASM heap has already failed.
• 🔴 (23:53) User stated embedding pool sizing must never fall below 1 worker because the primary worker always runs.
• 🔴 (23:53) User stated container-aware memory limiting must self-limit the WASM path before an uncatchable process OOM, in which case the ×0.7 OOM backoff never fires.
• 🟡 (23:53) Embedding sizing code exports MIN_EMBED_TOKENS = 256.
• 🟡 (23:53) Pool size configuration comes from search.embeddings.embedPoolSize or LORE_EMBED_POOL_SIZE; memory-based sizing uses Math.floor(free / PER_WORKER_MEM_BUDGET_BYTES) while retaining at least the primary worker.
• 🟡 (23:53) The container-memory adjustment is monotonic: it may only reduce reported free memory and is a no-op on an unconstrained host; this addresses runtimes where os.freemem() reports host memory from inside a memory-capped container.
• 🟡 (23:53) The embedding-cap model accounts for the ONNX WASM runtime linear-memory hard cap, WASM heap fragmentation, and ONNX Runtime non-attention intermediates.
• 🟡 (23:53) The sizing rationale states the model’s effective context plateaus around approximately 2048 tokens, so capping at approximately 4962 tokens trades little effective context while reducing memory risk.
• 🟡 (23:53) Upward cap probing never steps downward; if its computed ceiling is below the current cap, it retains the current cap, and the probe path can never produce a cap above the ceiling even when given an already-above-ceiling cap.