Dashboard › opencode › Distillation
854f5390-2f40-4742-87e6-fb067048edd2["lore_tm_v1_y9V4QMRswWcLEjfr9XznPeP2Z_xudZvmheGNSrpJWKg"]
/home/byk/Code/opencode-lore-v2/AGENTS.md points long-term knowledge managed by lore to project-root .lore.md; the pointer section is maintained between lore markers.quality/REVIEW.md, covering regression-test discipline, adversarial-order state setup, fan-out registry coverage, recurring bug-class batteries, and the two-reviewer rule; every non-trivial PR requires an adversarial correctness review before merge.pnpm workspaces.~/.local/share/lore/lore.db.@loreai/core at packages/core/ for the shared memory engine; @loreai/gateway at packages/gateway/ for the transparent LLM proxy, lore CLI, context management, and web dashboard; @loreai/opencode at packages/opencode/ for OpenCode lifecycle hooks via @opencode-ai/plugin; and @loreai/pi at packages/pi/ for the Pi coding-agent extension adapter.@loreai/core is consumed by gateway, opencode, and pi; the gateway bundles core into one CJS file via esbuild, while the opencode package ships raw TypeScript.lore / lore-gateway CLI enters through packages/gateway/src/cli/bin.ts and then packages/gateway/src/cli/main.ts function _cli().run (default) handled by cli/run.ts, starts the gateway and auto-detects/launches an AI agent; start via cli/start.ts, starts only the gateway server; data via cli/data.ts, supports list, show, clear, delete, merge, and recover; recall via cli/recall-cmd.ts, searches project memory; and upgrade via cli/upgrade.ts, self-updates the binary.cli/agents.ts scans $PATH for Claude Code, Codex, Pi, and OpenCode, then sets routing environment variables such as ANTHROPIC_BASE_URL.packages/core/src/temporal.ts; store() converts message parts to text, stores them in temporal_messages with FTS5 indexing, tags messages with distilled=0/1, and uses \x1f (ASCII Unit Separator) as the chunk separator.packages/core/src/distillation.ts; run() is invoked on idle or urgency and executes resetOrphans() → fetch undistilled messages → detectSegments() using time gaps/token limits with a maximum of 16K tokens per segment → distillSegment() using an LLM observer prompt → store a generation-0 distillation.metaDistill() consolidates entries into generation 1 or higher; generation-0 segments remain archived and recall-searchable but are excluded from the in-context prefix.packages/core/src/gradient.ts; transform() runs on every message transform using four layers in order: Layer 0 full passthrough, Layer 1 distilled prefix plus compressed raw content, Layer 2 aggressive tool-output stripping, and Layer 3 emergency compression. It dynamically calibrates from actual API token counts and applies cost-aware caps based on model pricing.packages/core/src/ltm.ts use categories decision, pattern, preference, architecture, and gotcha; fields are id (UUIDv7), project_id, category, title, content (maximum 1200 characters), confidence (0.0–1.0), and cross_project.packages/core/src/curator.ts, periodically sending conversation plus existing entries to an LLM for create/update/delete operations; 2. regex-based, non-LLM Pattern Extraction in packages/core/src/pattern-extract.ts after each distillation; 3. File Import in packages/core/src/agents-file.ts, parsing .lore.md markers of the form <!-- lore:UUID --> and upserting into the database; 4. curator.ts function consolidate(), using an LLM to merge/trim when entry count exceeds maxEntries of 25.packages/core/src/ltm.ts functions are create(), update(), and remove() for CRUD with a deduplication guard; forSession() for relevance-ranked, budget-capped system-prompt entries; and search() / searchScored() for FTS5 BM25 search with a LIKE fallback..lore.md begins with <!-- Managed by lore (https://github.com/BYK/loreai) — manual edits are imported on next session. -->, contains a ## Long-term Knowledge section, groups entries under category headings, and identifies entries using markers such as <!-- lore:019e18ec-e328-76c4-9c3c-09dbe8d51c6c -->..lore.md entries are sorted alphabetically by title within each category for deterministic, merge-friendly output.packages/core/src/agents-file.ts, exportLoreFile() builds .lore.md from the database and skips writing when the content hash is unchanged; importLoreFile() parses Markdown and updates a known UUID whose content changed or creates an entry for an unknown UUID; shouldImportLoreFile() first performs a fast mtime check and then compares content hashes.exportToFile() writes the lore pointer section into AGENTS.md between markers, while splitFile() preserves all content outside those markers, making manual content outside the markers safe.packages/core/src/db.ts; knowledge CRUD packages/core/src/ltm.ts; distillation packages/core/src/distillation.ts; context management packages/core/src/gradient.ts (largest file, approximately 1950 lines); recall/search packages/core/src/recall.ts and packages/core/src/search.ts; LLM prompts packages/core/src/prompt.ts; file synchronization packages/core/src/agents-file.ts; curator packages/core/src/curator.ts; configuration packages/core/src/config.ts loading .lore.json; gateway server packages/gateway/src/server.ts; compaction interception packages/gateway/src/compaction.ts; LLM proxy pipeline packages/gateway/src/pipeline.ts; CLI commands packages/gateway/src/cli/; and OpenCode hooks packages/opencode/src/index.ts.temporal.store() → SQLite/FTS5 temporal_messages; distillation.run() → observer LLM → generation-0 distillations; metaDistill() → reflector LLM → generation-1+ distillations, with generation-0 archived rather than deleted; curator.run() → curator LLM → knowledge-table create/update/delete; exportLoreFile() → project-root .lore.md; exportToFile() → AGENTS.md; next-session importLoreFile() from changed .lore.md; forSession() + formatKnowledge() → system-prompt injection; and recall tool searchRecall → agent tool response.pnpm install for workspace dependencies; pnpm test for all Vitest tests; pnpm run typecheck for all packages; pnpm run lint for CI-gated type-aware oxlint with pnpm run lint:fix for autofix; pnpm run format for oxfmt with pnpm run format:check for CI verification; and pnpm run build for all esbuild-based package builds.packages/core/test/setup.ts; tests must never use the production database at ~/.local/share/lore/lore.db.packages/gateway/script/build.ts to produce a CJS bundle and script/bundle.ts to produce a standalone binary; core builds use packages/core/script/build.ts to produce Node.js-compatible CJS output.CHANGELOG.md; Sentry Craft handles both automatically through GitHub Actions.gh workflow run release.yml -f version=auto, where conventional commits determine semver, or gh workflow run release.yml -f version=0.23.0 for an explicit version.scripts/bump-version.sh updates every package.json version; 2. generates a changelog from conventional commit messages; 3. creates and pushes a release/X.Y.Z branch; 4. CI tests and builds npm tarballs, standalone binaries, and delta patches on that branch; 5. Craft opens a publish issue, and adding the accepted label triggers the Publish workflow; 6. Publish performs npm publishing through OIDC trusted publishing and creates a GitHub Release containing binaries and patches..craft.yml defines release targets for 4 npm workspaces, the legacy opencode-lore alias, and a GitHub Release containing binaries.