Distillation

Date: Sep 16, 2026

• 🔴 (13:37) User stated cgroup procfs discovery must be refreshed so a worker is never admitted from a stale cgroup directory or a cached discovery miss.
• 🔴 (13:37) User stated every descendant cgroup boundary must remain readable so sibling usage is never subtracted from the wrong scope.
• 🔴 (13:37) User stated container-memory clamping must be monotonic: it can only reduce memory use and must never raise the input figure.
• 🔴 (13:37) User stated embedding-cap handling for non-finite input must return the token floor, never NaN or Infinity.
• 🔴 (13:37) User stated embedding-cap backoff must never drop below the floor.
• 🟡 (13:37) packages/core/src/cgroup-memory.ts numeric parsing behavior: readBytes() accepts only decimal digits and safe integers; readLimit() maps "max" and decimal values above Number.MAX_SAFE_INTEGER to Number.POSITIVE_INFINITY, while malformed values return undefined.
• 🟡 (13:37) clampFreeToContainerLimit() in packages/core/src/embedding-cap.ts:160-174 treats a positive finite constrained value as the hard limit, accepts only finite nonnegative available headroom, returns 0 when a finite hard limit exists but headroom is unavailable, and otherwise returns Math.min(hostFree, limit, headroom).
• 🟡 (13:37) clampEmbedCap() in packages/core/src/embedding-cap.ts:209-212 maps non-finite input to MIN_EMBED_TOKENS; finite input is rounded and clamped to [MIN_EMBED_TOKENS, MODEL_MAX_TOKENS].
• 🟡 (13:37) WASM memory constants and rationale in packages/core/src/embedding-cap.ts: EMBED_WASM_HEAP_MAX_BYTES = 4 * 1024 * 1024 * 1024, derived from 65536 pages × 64 KiB; EMBED_WASM_HEAP_USABLE_FRACTION = 0.85 reserves headroom for fragmentation and ORT intermediates, yielding approximately 4950 tokens and matching observed safe convergence of ≤4962.
• 🟡 (13:38) packages/core/src/embedding/local.ts test hooks are _setConstrainedMemoryForTest(), _setContainerFreeForTest(), and _setAvailableMemoryForTest(). constrainedMemoryLimit() accepts only positive finite process.constrainedMemory() values and otherwise returns 0.
• 🟡 (13:38) availableMemoryHeadroom() in packages/core/src/embedding/local.ts:99-128 reads Linux cgroup headroom, excludes Bun’s host-wide availableMemory(), uses Node’s process.availableMemory() when callable, and falls back through resolveMemoryHeadroom() if the function is missing or throws.
• 🟡 (13:38) Current containerFreeBytes() in packages/core/src/embedding/local.ts:130-137 calls constrainedMemoryLimit() directly and then calls availableMemoryHeadroom(), which independently calls constrainedMemoryLimit() again; this confirms the constrained-memory value is sampled twice per sizing decision.
• 🟡 (13:38) packages/core/test/embedding-pool-memory.test.ts documents the native ONNX OOM regression: a pool grew to N workers, each sized its token cap from roughly half of total free memory, and aggregate allocation exhausted RAM; native OOM causes uncatchable SIGKILL, so the ×0.7 backoff cannot fire.
• 🟡 (13:38) packages/core/test/embedding-pool-memory.test.ts contains 4 ordered memory regressions: 1. each worker sizes from free / ceiling—fixture uses _persistEmbedCap(8192, 0), pool ceiling 2, and 6 * GB; 2. constrained primary-worker headroom of Number.NaN with a 12 * GB limit and 64 * GB host free yields MIN_EMBED_TOKENS; 3. live free memory dropping from 6 * GB to 2 * GB re-clamps the same worker’s next request; 4. after free memory drops from 16 * GB to 2 * GB, an EMBED_OOM_EXIT_CODE respawn resubmits the same request ID at memoryModelEmbedCap(2 * GB), below backoffEmbedCap(first.maxTokens).
• 🟡 (13:39) Existing cgroup resolver tests in packages/core/test/cgroup-memory.test.ts:7-103 cover: 1. cgroup v2 resolution through /run/my-cgroup rather than a fixed path; 2. namespaced cgroup v2 with escaped mount point /run/cgroup\040mount; 3. selecting only the cgroup v1 memory controller; 4. preferring explicit v1 memory membership on a hybrid host when both valid mounts exist; 5. preferring a mount root containing /tenant/job over unrelated root /other; 6. returning null for malformed data, CPU-only controller data, and traversal membership 0::/../../escape.