Distillation

Date: Sep 16, 2026

• 🟡 (13:12) Independent projection correctness review returned DO-NOT-MERGE for candidate unznwpyl / ab979df5cddfec3952680e423e39564664e881fe based directly on main@origin / 2cbf40060c67df229731cf1b9e34d5bf22ec2145; clean empty child was npkrwpky / c518e49f.
• 🟡 (13:12) Projection review integrity passed: candidate had no conflicts, the reported bookmark conflict was unrelated, exactly 2 paths changed, patch SHA-256 matched 53ea2616ed8429b1392937d82acdeff5323759cb5247fa5e1709f2544ec6461f, bundle SHA-256 matched ffc32be52e490a976caff4e324691940856a47c4f38bb0e66dc50a0c93a22aed, beginning/end identities and state matched, and no files or VCS state were changed.
• 🟡 (13:12) Projection review MUST-FIX finding: sparse principal frames forwarded before recall still use private source indices. packages/gateway/src/pipeline.ts:11511-11520 remaps transactional events after recall, but principal events emitted at packages/gateway/src/pipeline.ts:11584-11604 remain unchanged.
• 🟡 (13:12) Sparse-index root cause: the late-recall guard at packages/gateway/src/pipeline.ts:10149-10158 compares the source item’s map position before and after hiding recall rather than comparing the already-emitted coordinate with the public coordinate. With visible source index 7 followed by recall index 9, both maps assign source 7 to public position 0, so recall is accepted despite the client already receiving output_index: 7.
• 🟡 (13:12) Focused reproduction of the projection defect emitted every visible principal lifecycle frame with output_index: 7, the synthetic anchor at public index 1, and continuation at public index 2, while the terminal output placed IDs at dense array positions 0, 1, and 2; terminal compaction occurs at packages/gateway/src/pipeline.ts:9879-9950 and 9851-9859.
• 🟡 (13:12) Review found the intended sparse-projection regressions insufficient: terminal-discovered sparse recall at packages/gateway/test/openai-responses-recall-aware-stream.test.ts:2673-2753, late sparse recall at :5788-5826, and deferred sparse output at :5867-5928 do not call expectVisibleOutputIndicesToMatchTerminal() from lines 86-107; therefore their assertions pass with incorrect coordinates, do not fail on base, and do not prove the new guard. The supplied 40/40 property evidence also omitted this sequence.
• 🟡 (13:12) Required projection fix from review: establish and apply the public index projection before every principal output-bearing frame is enqueued; reuse the same stable mapping for held continuation events and the terminal array; fail closed before side effects if a later visible or private item would alter an emitted coordinate; add expectVisibleOutputIndicesToMatchTerminal() to all 3 sparse tests; and prove failure when either principal remapping or the reindex guard is removed.
• 🟡 (13:12) Projection review passed these behaviors: internal accumulation before suppression at pipeline.ts:10340-10350 and 10983-10989; continuation-index overflow checking before the first recall side effect at pipeline.ts:10608-10620; lower-index late recall failure before recall execution with fixed output: [] through pipeline.ts:11762-11767 and 11829-11853; generic remapping of transactional continuation frames before sequence numbering at pipeline.ts:9963-10003 and 11516-11520; semantic-field allowlisting and validated incomplete reasons in terminal rebuilding at pipeline.ts:9837-9865; enqueue bookkeeping only after successful delivery at pipeline.ts:10038-10063; and projection bounded by existing frame, retained-state, stream-byte, and transactional-byte limits.
• 🟡 (13:12) Focused candidate validation had 11 tests passed and 201 skipped; the exhaustive suite was not rerun. One attempted read-only base replay through stdin timed out without results and was excluded from evidence.
• 🟡 (13:12) Assistant accepted sparse principal lifecycle projection as the active blocker and planned to repair principal projection with fail-first identity-to-terminal assertions plus guard-removal mutation proof before revalidation.
• 🟡 (13:13) Active ordered task queue became: 1. Fix sparse principal Responses projection with fail-first and mutation proof — in progress, high; 2. Revalidate, freeze, and review corrected projection candidate — pending, high; 3. Finish provider failure exhaustive validation and independent reviews — pending, high; 4. Resolve accumulation correctness findings in additive child — pending, high; 5. Resolve recovery review findings and revalidate — pending, high; 6. Push reviewed candidates and update/create focused pull requests — pending, high.
• 🟡 (13:13) Assistant chose not to mutate the old projection workspace because its head remained under active security review; instead it planned a separate additive JJ workspace above immutable unznwpyl / ab979df5 to preserve review evidence and workspace isolation.
• 🟡 (13:14) Created isolated JJ workspace /home/byk/Code/opencode-lore-responses-projection-fix from frozen parent unznwpyl / ab979df5cddf (fix(responses): project hidden output consistently). Initial empty working copy was ozrmvotw / f3590a18; the workspace added 1072 files with 0 modified and 0 removed, and had no working-copy changes.
• 🟡 (13:14) Existing verification child npkrwpkyowmx / c518e49ffd2e was empty, conflict-free, and described as verify projection fix on current main; unrelated bookmark fix/responses-private-references remained conflicted.
• 🟡 (13:14) New correction change was described as fix(responses): stabilize sparse principal projection; empty conflict-free working copy ozrmvotwxzuy / a41d9ec5642e remained based on unznwpyl / ab979df5cddf.
• 🟡 (13:14) Assistant planned to trace terminal ordering, direct enqueue, deferred events, recall admission, and mapping state together before writing the fail-first test, to avoid fixing only the emitted number and creating another stream/terminal mismatch.
• 🔴 (13:15) User stated tests use a temporary SQLite database through the Vitest setup file packages/core/test/setup.ts, never the production DB.
• 🔴 (13:15) User stated releases use Sentry Craft via GitHub Actions and versions must never be manually bumped or CHANGELOG.md manually edited because Craft handles both automatically.
• 🔴 (13:15) User stated principal response events must always accumulate into internal state for postResponse; the corresponding invariant is documented at packages/gateway/src/pipeline.ts:10340.
• 🔴 (13:15) User supplied repository architecture: @loreai/core at packages/core/ is the shared memory engine for DB, distillation, knowledge/LTM, recall, gradient context management, and prompts; @loreai/gateway at packages/gateway/ is the transparent LLM proxy and lore CLI; @loreai/opencode at packages/opencode/ is the OpenCode plugin adapter using @opencode-ai/plugin; and @loreai/pi at packages/pi/ is the Pi coding-agent extension adapter.
• 🔴 (13:15) User supplied LTM structure for packages/core/src/ltm.ts: categories are decision, pattern, preference, architecture, and gotcha; fields include id (UUIDv7), project_id, category, title, content (maximum 1200 characters), confidence (0.0–1.0), and cross_project.
• 🔴 (13:15) User supplied key repository paths: database/schema packages/core/src/db.ts; knowledge CRUD packages/core/src/ltm.ts; distillation packages/core/src/distillation.ts; context management packages/core/src/gradient.ts (largest file, approximately 1950 lines); recall/search packages/core/src/recall.ts and packages/core/src/search.ts; prompts packages/core/src/prompt.ts; file sync packages/core/src/agents-file.ts; curator packages/core/src/curator.ts; configuration loader packages/core/src/config.ts for .lore.json; gateway server packages/gateway/src/server.ts; compaction intercept packages/gateway/src/compaction.ts; LLM proxy pipeline packages/gateway/src/pipeline.ts; CLI commands packages/gateway/src/cli/; and OpenCode hooks packages/opencode/src/index.ts.
• 🔴 (13:15) User stated gateway build script packages/gateway/script/build.ts produces a CJS bundle, script/bundle.ts creates a standalone binary, and core build script packages/core/script/build.ts produces Node.js-compatible CJS output.