Distillation

Date: Sep 16, 2026

• 🔴 (21:27) Repository /home/byk/Code/opencode-lore-responses-reference-v2 is a colocated Jujutsu repository containing both .jj/ and .git/; mutations must use jj, not git, so the operation log remains consistent.
• 🔴 (21:27) User stated that jj never fails on conflict: operations such as rebase, new, and squash record conflicts in the resulting commit rather than aborting. Afterward, run jj st, resolve files manually, and verify the conflict clears; do not use interactive jj resolve.
• 🔴 (21:27) Jujutsu workflow requirements: never use interactive flags (-i, --interactive); always pass -m "msg" when describing or committing; verify squash, abandon, rebase, restore, and commit mutations with jj st and jj log; prefer stable change IDs over hexadecimal commit IDs; never rebase or describe immutable commits.
• 🔴 (21:27) Jujutsu recovery commands are jj undo, jj op log, jj op restore <op-id>, and jj workspace update-stale; the working copy is commit @, its parent is @-, edits auto-amend @, and there is no staging area.
• 🔴 (21:27) Repository currently has 28 root entries, including .craft.yml, .git/, .github/, .jj/, .lore.md, AGENTS.md, CHANGELOG.md, codecov.yml, package.json, packages/, patches/, quality/, scripts/, supabase/, vitest.config.ts, vitest.evals.config.ts, and vitest.mutation.config.ts.
• 🔴 (21:28) Lore is a TypeScript/pnpm-workspaces monorepo implementing a three-tier memory architecture: temporal storage, LLM distillation, and gradient context management. Runtime requirement is Node.js >= 22.5; storage is SQLite in WAL mode with FTS5 at ~/.local/share/lore/lore.db.
• 🔴 (21:28) Monorepo packages are: @loreai/core at packages/core/ (shared memory engine), @loreai/gateway at packages/gateway/ (transparent LLM proxy and lore CLI), @loreai/opencode at packages/opencode/ (OpenCode lifecycle adapter), and @loreai/pi at packages/pi/ (Pi coding-agent extension).
• 🔴 (21:28) Lore’s temporal tier is implemented in packages/core/src/temporal.ts; store() writes converted message text to temporal_messages with FTS5 indexing, uses a distilled=0/1 flag, and separates chunks with \x1f.
• 🔴 (21:28) Lore’s distillation tier is implemented in packages/core/src/distillation.ts; run() performs resetOrphans() → fetch undistilled messages → detectSegments() → distillSegment(), with a maximum of 16K tokens per segment. More than 20 generation-0 distillations triggers metaDistill(); generation-0 segments remain recall-searchable but are excluded from the in-context prefix.
• 🔴 (21:28) Lore’s gradient context manager is implemented in packages/core/src/gradient.ts; transform() uses 4 layers: Layer 0 full passthrough, Layer 1 distilled prefix plus compressed raw content, Layer 2 aggressive tool-output stripping, and Layer 3 emergency compression.
• 🔴 (21:28) LTM entries in packages/core/src/ltm.ts use categories decision, pattern, preference, architecture, and gotcha; fields are id (UUIDv7), project_id, category, title, content (maximum 1200 characters), confidence (0.0–1.0), and cross_project.
• 🔴 (21:28) LTM creation paths are: 1. LLM curator in packages/core/src/curator.ts, 2. regex-based pattern extraction in packages/core/src/pattern-extract.ts, 3. .lore.md import in packages/core/src/agents-file.ts, and 4. curator.ts consolidate() when entry count exceeds maxEntries of 25.
• 🔴 (21:28) Tests use a temporary SQLite database configured through packages/core/test/setup.ts—never the production DB.
• 🔴 (21:28) Build and validation commands are pnpm install, pnpm test, pnpm run typecheck, pnpm run lint, pnpm run lint:fix, pnpm run format, pnpm run format:check, and pnpm run build.
• 🔴 (21:28) Every non-trivial PR requires adversarial correctness review before merge under quality/REVIEW.md; the review process separates correctness review from maintainability review and uses a two-reviewer rule.
• 🔴 [requested-tests] (21:28) User’s project mandates failing-first regression-test discipline: a bug fix test must fail on the base branch, pass with the fix, and fail if the relevant guard is deleted. Never trust the fix without seeing it fail.
• 🔴 (21:28) Property-test stability must be checked across 10 runs (×10), and review evidence must end with an explicit overall MERGE or DO-NOT-MERGE, not an unstructured “LGTM.”
• 🔴 (21:28) Releases use Sentry Craft through GitHub Actions. Never manually bump versions or edit CHANGELOG.md; Craft handles both automatically.
• 🔴 (21:28) Release commands are gh workflow run release.yml -f version=auto for conventional-commit-derived semver or gh workflow run release.yml -f version=0.23.0 for an explicit version.
• 🔴 (21:28) Release workflow order is: 1. scripts/bump-version.sh updates all package.json versions, 2. changelog is generated from conventional commits, 3. a release/X.Y.Z branch is created and pushed, 4. CI runs tests and builds npm tarballs, standalone binaries, and delta patches, 5. Craft opens a publish issue, 6. adding the accepted label triggers npm OIDC publishing and a GitHub Release with binaries and patches.
• 🔴 (21:28) Root package.json identifies lore-monorepo as private ESM software under FSL-1.1-Apache-2.0; it pins pnpm@10.28.0 and Volta versions Node 24.16.0 and pnpm 10.28.0.
• 🔴 (21:28) Root scripts include "test": "vitest run", "pretest": "pnpm --filter @loreai/gateway run bundle", "test:coverage": "pnpm --filter @loreai/gateway run bundle && vitest run --coverage", "lint": "oxlint --type-aware", "format": "oxfmt", "evals": "vitest run --config vitest.evals.config.ts", and "mutation": "stryker run".
• 🔴 (21:28) OpenAI Responses-related tests are located at packages/gateway/test/openai-responses-recall-aware-stream.test.ts, packages/gateway/test/openai-responses.test.ts, and packages/gateway/test/openai-responses-stream.test.ts.
• 🔴 (21:28) Current jj working-copy change is change ID zkttmwun, commit ID a51d94f1, description fix(responses): close reference lifecycle gaps; its parent is change ID wywumkpt, commit ID 8063355c, description fix(responses): keep item references private.
• 🔴 (21:28) Current modified files are packages/gateway/src/pipeline.ts and packages/gateway/test/openai-responses-recall-aware-stream.test.ts.
• 🔴 (21:28) Bookmark fix/responses-private-references-v2 points at the parent commit, while bookmark fix/responses-private-references is conflicted; jj advises inspecting with jj bookmark list and resolving via jj bookmark set <name> -r <rev>.