Distillation

Date: Sep 17, 2026

• 🔴 (09:27) User asserted resolveMemoryHeadroom always accepts finite Node headroom without a hard limit.
• 🔴 (09:27) User asserted a host logger/sink must never interrupt stream consumption.
• 🔴 (09:27) User asserted embedding-pool behavior must never multiply memory pressure on a constrained host; each added local worker carries a full model/runtime baseline that embedding-cap backoff cannot reclaim, whereas reducing worker count can.
• 🔴 (09:27) User asserted memory clamping never increases memory use.
• 🔴 (09:27) User asserted clampFreeToContainerLimit never raises the figure: it is monotonic and can only reduce memory use.
• 🔴 (09:27) User provided a patch to packages/core/src/embedding/local.ts that imports readLinuxCgroupMemoryHeadroomBytes from ../cgroup-memory and resolveMemoryHeadroom from ../embedding-cap.
• 🔴 (09:27) User added _setAvailableMemoryForTest(bytes: number | null) and availableMemoryHeadroom() in packages/core/src/embedding/local.ts; the latter combines constrainedMemoryLimit(), Linux cgroup headroom, and process.availableMemory() through resolveMemoryHeadroom.
• 🔴 (09:27) User’s availableMemoryHeadroom() design treats Bun specially because Bun’s implementation currently aliases host freemem() inside a constrained cgroup, while Node’s libuv implementation is process-limit-aware.
• 🔴 (09:27) User changed containerFreeBytes() in packages/core/src/embedding/local.ts to call clampFreeToContainerLimit(raw, constrainedMemoryLimit(), availableMemoryHeadroom()).
• 🔴 (09:27) User renamed the modeled local-worker estimate from exported PER_WORKER_MEM_BUDGET_BYTES to internal MODELED_WORKER_MEM_BUDGET_BYTES; it remains EMBED_MODEL_BASELINE_BYTES + EMBED_ATTENTION_BYTES_PER_TOKEN_SQ * EMBED_POOL_BUDGET_REF_TOKENS * EMBED_POOL_BUDGET_REF_TOKENS, described as the model plus a 2048-token attention estimate and used as a lower bound for pool admission.
• 🔴 (09:27) User asserted cgroup v2 may set memory.high while leaving memory.max unlimited, so an absent hard limit does not imply unconstrained available memory.
• 🔴 (09:27) User added tests that vary container limits over 0, 256 * MB, 4 * GB, 12 * GB, and 128 * GB, and available-memory values over 0, 32 * MB, 2 * GB, and 64 * GB, requiring every clamped result for 6 * GB host-free memory to remain at most 6 * GB.
• 🔴 (09:27) User added resolveMemoryHeadroom tests, including the expectation resolveMemoryHeadroom(12 * GB, undefined, 2 * GB) === 2 * GB and a case titled "always accepts finite Node headroom without a hard limit".
• 🔴 (09:27) User added local embedding tests using _setAvailableMemoryForTest, _persistEmbedCap(8192, 0), and fake-worker scenarios involving embed(["alpha"], "query"), embed(["second"], "query"), and fakes[0].completeNext().
• 🔴 (09:28) User showed packages/core/src/cgroup-memory.ts, which reads /proc and cgroup data using readFileSync, models cgroup membership as version 1 | 2, and safely decodes octal-escaped proc paths while rejecting non-absolute paths, NUL characters, and .. path components.
• 🔴 (09:28) User’s parseCgroupMemberships() logic makes a cgroup v1 memory-controller membership authoritative on hybrid v1/v2 hosts, even when malformed or unavailable, because falling through to v2 would account against the wrong hierarchy.
• 🔴 (09:28) User’s resolveLinuxCgroupMemoryFileCandidates() parses mountinfo, matches cgroup2 for v2 or a cgroup mount with the memory super-option for v1, handles cgroup namespaces, prefers direct candidates, and sorts direct candidates by descending root-path length.
• 🔴 (09:28) User’s cgroup headroom calculation walks ancestor boundaries, computes each finite-limit candidate as Math.max(0, limit - usage), and retains the minimum candidate across applicable boundaries.
• 🔴 (09:28) User’s cgroup reader allows the cgroup v2 root to omit memory.current, memory.high, and memory.max, but requires every descendant boundary to remain readable so sibling usage is never subtracted from the wrong scope.
• 🔴 (09:28) User’s cgroup reader re-reads /proc/self/cgroup and /proc/self/mountinfo after accounting; if either changed during the operation, it returns undefined rather than admitting work based on stale process membership or mounts.
• 🔴 (09:28) User’s cgroup-memory implementation fails closed to undefined on read, parsing, hierarchy-traversal, or consistency errors.