Distillation

Date: Sep 16, 2026

• 🟡 (13:08) packages/core/src/embedding/pool.ts token-batch checkpoint behavior: takeTokenBatchCheckpoint(key) prunes expired checkpoints, removes the selected checkpoint, and returns cloned vectors; storeTokenBatchCheckpoint(key, nextIndex, vectors) stores cloned vectors only when the pool is not closing, nextIndex > 0, and vectors.length === nextIndex, then prunes the checkpoint collection.
• 🟡 (13:08) packages/core/src/embedding/pool.ts recovery accounting: recordSlotSuccess(slot) marks the slot healthy; a success from the current initFailureGeneration clears recoveryGeneration, resets initFailures and initRetryAt, clears the provider latch when failureCause === "transient-init-exhausted", and resets errorLogged. Success from an already in-flight sibling instead calls preserveHealthyServiceAfterExhaustion() so it does not erase another slot’s retry debt or permit immediate respawns.
• 🟡 (13:08) runOperation() in packages/core/src/embedding/pool.ts uses an init watchdog followed by an execution watchdog, logs embedding worker watchdog expired: stage=${stage} timeout_ms=${timeoutMs}, and raises EmbeddingWorkerWatchdogError(stage).
• 🟡 (13:08) Pool operations deliberately omit every caller’s AbortSignal when invoking slot.provider.embed(...): the operation owns the worker slot until native inference settles, while callers own only their waiters.
• 🟡 (13:08) Successful pool operations clone and retain vectors only when operation.retainResult === true and no waiters remain; otherwise they settle all waiters and delete the operation. Failures settle every waiter with the owned error, and LocalProviderUnavailableError or EmbeddingWorkerWatchdogError retires the slot.
• 🟡 (13:08) Pool dispatch does not replace the last active slot or grow the pool while a retired worker still owns its native model/heap: when slots.length === 0 && retiredWorkers.size > 0, dispatch pauses until retirement completes.
• 🟡 (13:08) embed(texts, inputType, signal?) rejects an already-aborted signal with EmbeddingRequestAbortedError, rejects a closing pool with LocalProviderUnavailableError("embedding pool is unavailable"), copies the input via texts.slice(), deduplicates by embeddingOperationKey(ownedTexts, inputType), and attaches duplicate callers as waiters to the existing operation.
• 🔴 (13:08) User stated that in a memory-capped container, os.freemem() reports host-wide free memory and can cause excessive native ONNX workers and oversized token caps; a cgroup OOM-killer SIGKILL is uncatchable, so the ×0.7 OOM backoff “never fires,” whereas the WASM path self-limited against its fixed 4 GiB heap and native has no equivalent wall.
• 🔴 (13:08) User stated that container-memory clamping is monotonic: it can only lower the free-memory figure, never raise it, so it “never increases memory use.”
• 🟡 (13:08) clampFreeToContainerLimit(hostFree, constrained, available?) in packages/core/src/embedding-cap.ts clamps host free memory to both a finite positive process/container limit and finite nonnegative discovered headroom. If a finite limit exists but no headroom is available, it returns 0; otherwise it returns Math.min(hostFree, limit, headroom).
• 🟡 (13:08) resolveMemoryHeadroom(constrained, linuxHeadroom?, runtimeAvailable?) treats finite nonnegative Linux cgroup headroom as authoritative and returns the tightest of Linux headroom, runtime availability, and the process limit. Without Linux headroom, runtime availability is used only when a finite positive process limit is also known; otherwise it returns undefined.
• 🟡 (13:08) clampEmbedCap(n) clamps rounded finite values into [MIN_EMBED_TOKENS, MODEL_MAX_TOKENS]; non-finite input returns MIN_EMBED_TOKENS.
• 🟡 (13:08) WASM embedding memory constants in packages/core/src/embedding-cap.ts: EMBED_WASM_HEAP_MAX_BYTES = 4 * 1024 * 1024 * 1024, derived from 65536 pages × 64 KiB; EMBED_WASM_HEAP_USABLE_FRACTION = 0.85, yielding an approximately 4962-token hard ceiling and observed safe convergence of ≤4962 tokens.
• 🔴 (13:09) User stated that Jujutsu operations “never fail on conflict”; conflicts are recorded in the resulting commit and resolved afterward by manually editing files.
• 🔴 [enforced-workflow] (13:09) User’s Jujutsu workflow requires using jj rather than git for mutations whenever .jj/ exists, including colocated .jj/ plus .git/ repositories; Git is restricted to read-only operations or unsupported tasks such as git submodule.
• 🔴 [enforced-workflow] (13:09) User’s Jujutsu workflow forbids interactive flags and commands such as -i, --interactive, jj resolve, and jj diffedit; commands that describe or commit changes must include -m "msg" to prevent an editor from opening.
• 🔴 [enforced-workflow] (13:09) User requires verifying jj squash, jj abandon, jj rebase, jj restore, and jj commit mutations with jj st and jj log; stable letter-based change IDs are preferred over rewritten hexadecimal commit IDs.
• 🟡 (13:09) Jujutsu recovery and conflict guidance: use jj undo to reverse the latest operation, jj op log to inspect operations, and jj op restore <op-id> to rewind repository state; after rebase, new, or squash, inspect jj st, manually resolve conflict markers, and confirm the conflict clears automatically.
• 🟡 (13:09) Jujutsu workflow caveats: bookmarks do not auto-advance; after jj commit, content is in @- and @ is empty; jj new creates a new empty change rather than committing; :: is a DAG range while .. is set difference; immutable tracked commits such as main must not be directly rebased or described; stale working copies should use jj workspace update-stale.
• 🟡 (13:09) Current six-item work plan, in exact order: 1. “Close native ORT resolver and loader-fallback defects with deterministic regressions” — in_progress, high; 2. “Fix cgroup discovery, primary sizing, and pool reserve accounting defects” — pending, high; 3. “Prove each critical guard with fail-first and mutation checks” — pending, high; 4. “Run complete validation and freeze the exact candidate” — pending, high; 5. “Obtain independent correctness and security reviews of the frozen candidate” — pending, high; 6. “Finalize the Jujutsu change and report runtime guidance” — pending, medium.