Dashboard › institutional-transition-lab › Distillation
b3145412-0089-431d-9ca4-c8ece3c11b0a["lore_tm_v1_x4DP6ofSpu8sl2JY0ksXgPYYEcmPNS61_4vJNmoJDZE","lore_tm_v1_iHgGpkJtHajzwZx9UtqdWOw5E5_uARWN_ZFpoUSi7t4","lore_tm_v1_YE1ck7MKXDZoNK8uvWJ9xh8TlkZ9snA5WO5x5W40ERg","lore_tm_v1_dqWmB_TTMTQHdfQQRxD8-ManGCUCHTB2XLkO4LBDl3I","lore_tm_v1_uLW5PUgb6JGtbuFlNUYkslPPOrjv-Kz4KDX8uW3NuVc","lore_tm_v1_Q99DalSfEHKhjoPzD963qy4_auHrWKWLTG_-ypt-qe8","lore_tm_v1_D9JFl6Jcf38DDTePQoQBwLHk6npXHq9iczy36zltZ1s","lore_tm_v1_AqMLwhFcv0-UkkgaYzT40AfNPGnznnrWY1DiuxsWgB4","lore_tm_v1_uHseRYWSBQHGq3Z3fcQ3qgjwJ6Fuj-DF89oFPExz52Q","lore_tm_v1_chyROXbMCqYVVdzKDXaZemUo9FhNKelUCcef4gFln4A","lore_tm_v1_A_oBVKFBUpBG2g3_mRsfVD9TFxB5Xypn6Adx0EjKmSI","lore_tm_v1_I6nXIayPmtauJvMWfc0acy5NJSpu3Q4ssvcfmCLAV7I","lore_tm_v1_Jmib40_mIWj4ufNbFaM0fbe4WRXldSz_y_rbXR1kqL0","lore_tm_v1_xZAr73qPRMjxqe9s3JV2P-oY--gLlAoXRBuCEha1LQ8"]
/home/byk/Code/institutional-transition-lab/src/institution_lab/llm_event_probe.py defines _canonical_hash(payload) at lines 128–130 using canonical json.dumps(payload, sort_keys=True, separators=(",", ":")).encode() and SHA-256.validate_response(response, source_text) in /home/byk/Code/institutional-transition-lab/src/institution_lab/llm_event_probe.py begins at line 133 and deterministically validates exact RESPONSE_KEYS, allowed decision and EventKind values, nullable non-empty title/affected_scope, ISO or null announced_on/effective_on, finite numeric confidence in [0,1], nullable non-empty ambiguity, and arrays of non-empty verbatim evidence_quotes.validate_response validates each power_changes item as an object with exactly POWER_CHANGE_KEYS; requires non-empty actor, target, scope, and evidence_quote; restricts right_kind, direction, and change_status to their allowed sets; and requires each evidence_quote to occur verbatim in source_text.validate_response decision contracts are: institutional_event and control_event require event_kind, title, at least one of announced_on/effective_on, affected_scope, and at least one evidence quote; institutional_event additionally requires an institutional kind and at least one power change; control_event requires a product/external kind and no power changes; no_event requires null event fields plus empty power changes and evidence; abstain requires null event fields, no power changes, and a non-empty ambiguity explanation._safe_ratio(numerator, denominator) in /home/byk/Code/institutional-transition-lab/src/institution_lab/llm_event_probe.py returns numerator / denominator when the denominator is nonzero and 0.0 otherwise._indexed_records(records, name) in /home/byk/Code/institutional-transition-lab/src/institution_lab/llm_event_probe.py requires a non-empty list of objects, each with a non-empty unique document_id, and returns a dictionary indexed by document_id.validate_benchmark(documents_payload, labels_payload) in /home/byk/Code/institutional-transition-lab/src/institution_lab/llm_event_probe.py begins at line 286; requires a non-empty documents benchmark_id, matching labels benchmark_id, and labels_payload["outcome_blinded"] is True; requires one-to-one document/label ID alignment; enforces exact DOCUMENT_KEYS; requires non-empty document entity_id, publisher, source_url, source_type, and text; validates nullable published_on as an ISO date and nullable source_context as non-empty when present.evaluate_probe in /home/byk/Code/institutional-transition-lab/src/institution_lab/llm_event_probe.py builds threshold checks for sorted QUALITY_METRICS, with each check containing value, floating-point threshold, and passed = value >= threshold.contract_errors, every response is schema-valid, at least one evidence quote exists, and grounding_rate == 1.0; the quality gate passes only when every metric threshold check passes; overall passed requires both contract and quality-gate passage.2 records benchmark_id, label_status, outcome_blinded, canonical SHA-256 hashes of documents and labels, the first available response model and reasoning_effort, document/schema counts and rates, metrics, power_change_document_accuracy, institutional_f1, evidence grounding/count, accuracy by expected decision class, decision confusion, thresholds/checks, contract errors, and pass statuses.render_markdown(summary, results) in /home/byk/Code/institutional-transition-lab/src/institution_lab/llm_event_probe.py renders: heading ## Institutional-event extraction benchmark; contract PASS/FAIL; quality target MET/MISSED; model, reasoning effort, and outcome-blinded development-document count; metrics for schema validity, decision accuracy, event-kind accuracy, announcement-date accuracy, institutional precision/recall, required power-edge recall, and evidence grounding; per-expected-class accuracy; and per-document expected/predicted decision, kind match, power match, and validity.llm_event_probe.py CLI requires --documents, --labels, --raw-responses, --validated-output, and --summary-output; accepts optional --markdown-output, --fail-on-contract-error, and --fail-below-threshold; exits with status 2 on a requested contract failure and status 3 on a requested overall threshold failure./home/byk/Code/institutional-transition-lab/tests/test_model_policy.py tests that load_event_triage_policy() selects model gemini-3.7-flash, reasoning_effort="default", and selection_scope="event_triage_only"; does not authorize canonical_power_graph_serialization; and records selection by workflow run_id=33086454444 and artifact_id=9652609349.test_policy_rejects_scope_drift modifies cases/model-selection/event-triage-v1.json from "selection_scope": "event_triage_only" to "selection_scope": "all_llm_research" and expects load_event_triage_policy(path) to raise ValueError matching "wrong research role".test_retrieval_workflow_uses_policy_without_changing_blinded_coders verifies .github/workflows/retrieve-oss-governance.yml uses --model-policy cases/model-selection/event-triage-v1.json before - name: Prepare performance-blinded coding bundle, excludes --model gpt-5.6-luna and --reasoning-effort medium from that triage section, and retains --model gpt-5.6-luna --reasoning-effort medium plus --model gpt-5.6-terra in coding steps./home/byk/Code/institutional-transition-lab/src/institution_lab/contracts.py defines EntityKind values COMPANY="company" and OSS_PROJECT="oss_project"; MetricRole values OUTCOME="outcome", MEDIATOR="mediator", and CONTROL="control"; and EventKind values in order: leadership, board_or_steering, control_rights, reorganization, foundation_transfer, license, fork, reunification, capital_allocation, strategy, product, and external.Observation in /home/byk/Code/institutional-transition-lab/src/institution_lab/contracts.py is a frozen slotted dataclass with required fields entity_id, metric_id, observed_on, value, unit, role, and source_id, plus optional segment_id, lower_bound, and upper_bound.InstitutionalEvent in /home/byk/Code/institutional-transition-lab/src/institution_lab/contracts.py is a frozen slotted dataclass with event_id, entity_id, kind, nullable announced_on/effective_on, title, source_ids, optional affected_segments, power_edges_added, power_edges_removed, confidence=1.0, and tags=frozenset(); __post_init__ requires at least one event date and confidence in [0,1].Create canonical dated power-graph diffs from adjudicated records, is OPEN at https://github.com/BYK/institutional-transition-lab/issues/5; its objective is to transform only completed, source-grounded adjudications into dated institutional power-graph diffs for Terraform/OpenTofu and OSS comparison cases, and it is blocked on completion of the frozen 40-record adjudication issue.Add adoption and ecosystem-share outcome series, is OPEN at https://github.com/BYK/institutional-transition-lab/issues/6; its objective is outcome-independent Terraform/OpenTofu adoption and ecosystem-share time series rather than GitHub issue volume.schema/governance-coding-v1.schema.json → bfc735a7d8602d1f236a88c2d23b195f0600a8bae86fb8c8936afe7883a41b8e; cases/review/oss-governance-enriched-envelope-v1.json → 2ee9dda0306fe11df574e42adb9ed27643473f8e5854faa0526a2c88141bbb93; research/oss-governance-coding-protocol-v1.2.md → 06d4b3407677fe7fb23046f70164a366db06638e8a29af3daf7661f00d565933.opentofu-github-pr-2830 (https://github.com/opentofu/opentofu/pull/2830) has complete file listing, truncated patch selection, patch_unavailable_count=0, untruncated source text, 43 changed files, draft=False, merged=True, state="closed", base SHA 6f0d3d3a07a49309b30960d57225c9f7c701e9a9, head SHA da4ac00ca09d1d9e06e47efd77523a235d49f10c, merge SHA 59d24390b712b87954ee175c38912e56d8f5d974, and selected patches CHARTER.md (13,368), GOVERNANCE.md (5,643), and CONTRIBUTING.md (30,989). It merged at 2025-05-23T12:18:56Z. (meaning May 23, 2025)opentofu-github-pr-2953 (https://github.com/opentofu/opentofu/pull/2953) has complete file listing, truncated patch selection, patch_unavailable_count=0, untruncated source text, 43 changed files, draft=False, merged=True, state="closed", base SHA 3c170157023ac1290c89567ce432c35aa185099a, head SHA 5f5705b8947cd62834733e6c20635d8d13a616dd, and merge SHA 75bf1c2f65ad4baabd51a5e88873f805f5b2a1c7. It merged at 2025-06-25T17:50:10Z. (meaning June 25, 2025)opentofu-github-pr-2953 selected patches, in order, are: CHARTER.md (13,447), GOVERNANCE.md (5,728), CONTRIBUTING.md (1,398), contributing/FAQ.md (1,702), TSC/2023-11-02_NOTES.md (3,185), TSC/2023-12-05_NOTES.md (3,732), TSC/2023-12-11_NOTES.md (4,119), TSC/2024-01-07_NOTES.md (598), TSC/2024-01-22_NOTES.md (2,265), TSC/2024-02-21_NOTES.md (2,259), TSC/2024-02-27_NOTES.md (4,309), TSC/2024-04-03_NOTES.md (2,878), and TSC/2024-05-07_NOTES.md (4,380).terraform-github-pr-33661 (https://github.com/hashicorp/terraform/pull/33661) has complete file listing, truncated patch selection, patch_unavailable_count=0, untruncated source text, 1,294 changed files, draft=False, merged=True, state="closed", base SHA 8a085b427b74ce3829500a59508b77465f1bbef0, head SHA 53c34ff49cfbc1f70d7cdd3dca8040551c53737a, and merge SHA 1f011cbaada5bf9cbddadb58ba7c84b223e99467. It merged at 2023-08-10T22:58:09Z. (meaning August 10, 2023)terraform-github-pr-33661 selected patches, in order, are: LICENSE (19,255), .github/actions/equivalence-test/action.yml (241), .github/actions/go-version/action.yml (283), .github/ISSUE_TEMPLATE/bug_report.yml (229), .github/ISSUE_TEMPLATE/config.yml (168), .github/ISSUE_TEMPLATE/documentation_issue.yml (222), .github/ISSUE_TEMPLATE/feature_request.yml (204), .github/scripts/e2e_test_linux_darwin.sh (164), .github/scripts/equivalence-test.sh (164), .github/scripts/get_product_version.sh (164), .github/scripts/verify_docker (157), .release/ci.hcl (139), .release/release-metadata.hcl (276), .release/security-scan.hcl (160), checkpoint.go (142), codecov.yml (159), commands.go (142), copyright_headers.go (161), Dockerfile (240), and docs/plugin-protocol/tfplugin5.0.proto (176).terraform-github-pr-34847 (https://github.com/hashicorp/terraform/pull/34847) has complete file listing, truncated patch selection, patch_unavailable_count=0, untruncated source text, 86 changed files, draft=False, merged=True, state="closed", base SHA ca2be46b0e4db9ab64e649da771d527dcd08c505, head SHA 625b32501cd6f637b82cd6c537033b9c3c6cabf1, and merge SHA 84df5562f21ae587bbb7e9b2a1c5ada582d0ba11. It merged at 2024-03-14T16:58:30Z. (meaning March 14, 2024)terraform-github-pr-34847 selected patches, in order, are: internal/addrs/module_source.go (11,622), internal/configs/configtesting/synth_body.go (149), internal/configs/configtesting/synth_body_test.go (153), internal/getmodules/getter.go (2,766), internal/getmodules/moduleaddrs/detect_abs_filepath.go (2,400), detect_gcs.go (965), detect_gcs_test.go (632), detect_git.go (3,356), detect_git_test.go (2,616), detect_remote_shorthands.go (2,680), detect_remote_shorthands_test.go (711), detect_s3.go (2,119), detect_s3_test.go (1,973), package.go (3,581), source_parsing.go (9,104), and source_parsing_test.go (5,173).valkey-github-issue-4276 (https://github.com/valkey-io/valkey/issues/4276) has files_listing_complete=None, patch_selection_truncated=False, patch_unavailable_count=0, source_text_truncated=True, no metadata, and no patches.valkey-github-pr-1788 (https://github.com/valkey-io/valkey/pull/1788) has complete file listing, truncated patch selection, patch_unavailable_count=0, untruncated source text, 31 changed files, draft=False, merged=True, state="closed", base SHA 1ad0c4feafb149a6f03627de03f4f810801ab875, head SHA 461b170f84d76594cddef4674733f5f815945dee, and merge SHA 089b830479f4984e77e570560f85e37212879740. It merged at 2025-02-27T18:38:26Z. (meaning February 27, 2025)valkey-github-pr-1788 selected patches, in order, are: COPYING (144), deps/fast_float_c_interface/fast_float_strtod.cpp (247), src/allocator_defrag.c (318), src/asciilogo.h (638), src/blocked.c (502), src/cluster.c (364), src/cluster_legacy.c (400), src/cluster_slot_stats.c (270), src/commandlog.c (641), src/defrag.c (295), src/dict.c (295), src/eval.c (375), src/expire.c (273), src/functions.c (292), src/hyperloglog.c (296), src/io_threads.c (233), src/lua/engine_lua.c (164), src/memory_prefetch.c (391), src/module.c (371), and src/rdb.c (319).terraform-github-pr-22332 / luna_a parsed successfully but power_changes[0].evidence_refs[1] and [2] were not grounded; terraform-github-pr-22745 / luna_b failed parsing with invalid JSON: Extra data: line 1 column 680 (char 679); terraform-github-pr-33661 / luna_a had ungrounded power_changes[0].evidence_refs[2]; terraform-github-pr-34847 / luna_a had ungrounded power_changes[1].evidence_refs[1].terraform-github-pr-38385 / both luna_a and luna_b had ungrounded power_changes[0].evidence_refs[0]; valkey-github-issue-4509 / luna_a had ungrounded power_changes[0].evidence_refs[0]; and valkey-github-pr-2927 / terra_advisory had ungrounded power_changes[1].evidence_refs[0].9653837747, node ID MDg6QXJ0aWZhY3Q5NjUzODM3NzQ3, is named oss-governance-retrieval-v2-41edae23dbf9fe61c48b2235d13b2e15bb47e806, has size 1,396,121 bytes, digest sha256:65dba871145d26feab8eb5938a0e9ef44dc36c95215b207549d95cfe9691a6b9, is not expired, and belongs to workflow run 33086454759 on main at head SHA 41edae23dbf9fe61c48b2235d13b2e15bb47e806.9653837747 API URL is https://api.github.com/repos/BYK/institutional-transition-lab/actions/artifacts/9653837747 and archive URL is https://api.github.com/repos/BYK/institutional-transition-lab/actions/artifacts/9653837747/zip; it was created/updated at 2026-08-27T15:48:53Z and expires at 2026-09-26T15:48:51Z. (meaning September 26, 2026)README.md stating change-point detection operates only on performance series, governance coding occurs without consulting performance series, there were three raw statistical candidates and zero robust transitions, and the seed inventory was assembled after pilot outcomes were inspected and is not blinded evidence.research/session-handover-2026-08-27.md states the first detector produced three OpenTofu candidates—February 2024 issue latency, June 2024 closed issues, and July 2024 opened issues—and the result was 3 raw candidates → 0 robust transitions, attributed to startup and backlog effects.docs/terraform-opentofu.html states apparent February–July 2024 changes had only 37%–60% support; PROTOCOL.md states the phase detector sees only outcome and declared control series; and research/luna-event-extraction-development-benchmark.md keeps performance series and prior model results outside the prompt.derived/source-retrieval/oss-governance-development-v2/audit-envelope-enriched.json → SHA-256 f2f970bc3041594f7d989d5517156fb4466ffc1787f54f579fc8f3ad8e6b2985, 389,362 bytes; documents.json → 08ce70392d4a88618d3db7b2f8de08a42a4e6fc105df2cd28b562e67193004e2, 1,092,530 bytes; audit.json → 0e6ff64853e67c70e123832e7efae30fa2682ecda5ea0d0f739ecb971363d409, 336,307 bytes; coding/documents.json → 2a62ab05687982c112a29f6126227c26c3ff8defc89ca64662753aaf85c3c0ce, 445,940 bytes.github_pull_request and 14 are github_issue; source-kind counts are 110 github_pr_patch, 40 github_body, and 26 github_pr_metadata.opentofu-github-pr-2959, opentofu-github-pr-3559, terraform-github-pr-34096, and terraform-github-pr-34103.opentofu-github-pr-2830, opentofu-github-pr-2953, terraform-github-pr-33661, terraform-github-pr-34847, valkey-github-issue-4276, and valkey-github-pr-1788; no pull request has an incomplete file listing./home/byk/Code/institutional-transition-lab/src/institution_lab/github_client.py defines retryable GitHub status codes {429, 500, 502, 503, 504} and GitHubRestClient defaults: base_url="https://api.github.com", max_attempts=4, retry_backoff_seconds=1.0, request timeout 60, API version 2022-11-28, and token resolution from the explicit token or GITHUB_TOKEN.GitHubRestClient._write_once() writes immutable cache objects via a .tmp sibling followed by replace, but returns without writing when the target already exists; _request_key() sorts and URL-encodes parameters and hashes the canonical URL with SHA-256.min(retry_backoff_seconds * (2**attempt), 30.0) and, when a valid retry-after header exists, returns min(max(backoff, float(retry_after)), 60.0); transport failures raise GitHubApiError after all attempts, while retryable HTTP responses are retried until the final attempt.GitHubRestClient caches canonical compact JSON under cache_dir/objects/{sha256}.json and request metadata under cache_dir/requests/{request-key}.json; metadata includes URL, retrieval timestamp, SHA-256, ETag, rate-limit remainder, and for paginated responses the next URL.GitHubRestClient._request_from_link() rejects pagination links whose scheme, host, port, or base-path scope differs from the configured base_url; paginate() defaults to per_page=100, max_pages=1000, accepts only per_page from 1 through 100, rejects repeated pagination URLs, and raises if the page cap is exceeded.GitHubRestClient.search() defaults to max_results=100, accepts only 1 through 1,000 results, requests up to 100 results per page, rejects repeated pagination URLs, and preserves raw GitHubSearchPage objects; search responses must contain a list of object items, integer non-boolean total_count, and boolean incomplete_results.preliminary-summary.json for coding_id="oss-governance-enriched-coding-v1" and documents SHA-256 d70f7f795847968339a645b6dbd6d4428af6bd6044de44fa2602d4c4343b0ee9 has schema version 1, 40 records, 29 adjudications, 0 canonical-ledger records, luna_a_invalid_count=5, luna_b_invalid_count=2, Luna class agreement 0.675, Luna strict-edge agreement 0.6, 11 provisional exact agreements, terra_completed=False, and terra_invalid_count=None.preliminary-summary.json route-reason counts are: abstention 4, bounded evidence 6, class disagreement 13, edge disagreement 16, event-field disagreement 13, invalid response 6, and low confidence 5.summary.json retains 40 records, 29 adjudications, 0 canonical-ledger records, Luna class agreement 0.675, Luna strict-edge agreement 0.6, 11 provisional exact agreements, and the same route-reason counts as preliminary-summary.json, but records terra_completed=True and terra_invalid_count=3.v1.2-summary.json has schema version 1, 40 records, 35 adjudications, 0 canonical-ledger records, full_response_agreement_count=0, luna_a_invalid_count=5, luna_b_invalid_count=2, Luna class agreement 0.675, Luna strict-edge agreement 0.55, provisional_core_agreement_count=5, terra_completed=True, and terra_invalid_count=1.v1.2-summary.json route-reason counts are: abstention 4, body/patch relation disagreement 11, bounded evidence 6, class disagreement 13, edge disagreement 18, event-field disagreement 13, invalid response 6, and low confidence 5.