Dashboardopencode-loreEmbedding pool memory admission and OOM…

Embedding pool memory admission and OOM recovery

Category: preference
Confidence: 0.80
ID: 01a0aeaf-53c5-7722-ac8c-9b41d3c039bf
Project ID: 6f4be9ff-ed84-4cca-a9e7-732a0b0b8677
Cross-project: No
Recalled in other projects: 0
Source session: 1JmtIdaVnn5PTPdxS
Created: 2026-09-17 08:25:00
Updated: 2026-09-17 09:25:20

Content

Treat native ONNX OOM as an uncatchable SIGKILL and prevent it through fail-closed admission. Always retain one primary worker. Admit extra workers only from fresh container-aware memory data, reserving memory for admitted-but-unhealthy workers and dividing shared budget by the pool ceiling. Clamp each worker's token cap against its share and current trustworthy free memory on every request; unknown constrained headroom uses the minimum cap, never host-memory guesses. Re-clamp direct post-respawn resubmissions while preserving request IDs. Never increase usable memory during clamping or re-probe upward without genuine improvement or beyond a known failed cap. Keep pool behavior tests separate from dedicated per-worker OOM/respawn tests.

Move to: