Distillation

Date: Sep 17, 2026

• 🔴 (09:49) User stated the adaptive embedding cap never exceeds the Nomic v1.5 maximum sequence length.
• 🔴 (09:49) User stated a fixed memory limit never improves, so the embedding-cap logic never re-probes under that condition.
• 🔴 (09:49) User stated persisted free-memory baseline handling can cause a memory-rich reboot to never re-probe back up to a cap that the WASM heap has.
• 🔴 (09:49) User reiterated embedding worker admission is never below 1: the primary worker always runs.
• 🔴 (09:49) User stated the ×0.7 OOM backoff never fires for an uncatchable WASM out-of-memory event because the WASM path self-limited beforehand.
• 🟡 (09:49) packages/core/src/embedding-cap.ts defines MIN_EMBED_TOKENS = 256.
• 🟡 (09:49) Embedding-cap source documentation says each OOM lowers the cap by multiplying it by 0.7, then respawns the worker on a fresh heap.
• 🟡 (09:49) Embedding memory sizing documentation records a measured baseline of approximately 665 MB, subtracting the model footprint from free memory before sizing transient allocation and relying on backoff to correct residual error.
• 🟡 (09:49) Pool admission computes how many workers fit in freeBytes, using Math.floor(free / PER_WORKER_MEM_BUDGET_BYTES), while preserving the primary-worker floor of 1.
• 🟡 (09:49) Source documentation warns that inside a memory-capped container, os.freemem() reports host memory rather than container memory; the associated adjustment is monotonic and can only reduce the relevant bound.
• 🟡 (09:49) Current-headroom selection uses Math.min(limit, runtime) after reconciling Linux/container and runtime-available memory signals.
• 🟡 (09:49) The embedding-cap model clamps an attention-memory estimate computed as Math.sqrt(budget / EMBED_ATTENTION_BYTES_PER_TOKEN_SQ).
• 🟡 (09:49) Source documentation states effective embedding context plateaus around ~2048 tokens, so capping around ~4962 tokens trades little useful context while reducing memory risk.
• 🟡 (09:49) Persisted-cap reconciliation uses ratio <= 1 + EMBED_CAP_TRUST_BAND as a trust-band condition and guards against stale caps learned before a newer ceiling existed, exemplified by an old cap of 7000.
• 🟡 (09:49) Upward re-probing is monotonic: it never steps down; its result cannot exceed the current ceiling even if passed an above-ceiling cap.