Dashboardopencode-loreembedInTokenBatches progress invariant

embedInTokenBatches progress invariant

Category: preference
Confidence: 1.00
ID: 01a0ad05-9b26-7ad1-81ff-607df672d21f
Project ID: 6f4be9ff-ed84-4cca-a9e7-732a0b0b8677
Cross-project: No
Recalled in other projects: 0
Source session: 0VohlSjRK0TAIkdIN
Created: 2026-09-17 01:40:20
Updated: 2026-09-17 01:40:20

Content

Every embedding path must use embedInTokenBatches, preserve input order, and call storeTemporalChunks once. The MAX_BATCH_TOKEN_AREA=4096 limit must always admit the first item even when it alone exceeds the area; rejecting it looks stricter but wedges the batching loop with no progress. Temporal input remains capped at 64 chunks by retaining 63 and joining the remainder into one tail.

Move to: