Dashboard › institutional-transition-lab › Session 0UbFLH0b7K0x
0UbFLH0b7K0xhCtiDREAD-ONLY adversarial audit of the uncommitted governance adjudication validator in /home/byk/Code/institutional-transition-lab. Inspect only: src/institution_lab/governance_adjudication.py, schema/governance-adjudication-v1.schema.json, tests/test_governance_adjudication.py, pyproject.toml, cases/review/oss-governance-adjudication-v1.inputs.json, and if needed src/institution_lab/governance_coding.py. Do not inspect governance records, model outputs, research outcome files, or performance data. Do not modify files.
Find concrete correctness/security/provenance defects that could: open canonicalization with malformed/unresolved records; accept self-consistent substituted inputs; admit unauthorized reviewer models; fail to ground institutional claims; misreport unresolved records or graph-edge revisions; crash on schema-valid or malformed input rather than return validation errors; or make the CLI's generated audit misleading. For each finding give severity, exact file:line, an adversarial input, and expected regression test. Explicitly check manual validator/schema parity and CLI composition. If no findings, state PASS and residual risks. Return a substantive result, never empty.
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
1: from __future__ import annotations
2:
3: import argparse
4: import hashlib
5: import json
6: import zipfile
7: from collections import Counter
8: from pathlib import Path
9: from typing import Any
10:
11: from jsonschema import Draft202012Validator, FormatChecker
12:
13: from institution_lab.governance_coding import validate_response
14:
15: DECISIONS = {"accept", "revise", "reject", "abstain"}
16: INSTITUTIONAL_CLASSES = {
17: "effective_institutional_change",
18: "announced_institutional_change",
19: "proposal_only",
20: }
21: REVIEWER_TYPES = {"human", "llm", "human_llm_assisted"}
22: REVIEWER_ROLES = {"source_adjudicator", "evidence_auditor", "synthesizer"}
23: RESPONSE_SOURCES = {"luna_a", "luna_b", "terra_advisory"}
24: REQUIRED_INPUT_PINS = {
25: "workflow_run_id",
26: "artifact_id",
27: "artifact_sha256",
28: "documents_sha256",
29: ...
<path>/home/byk/Code/institutional-transition-lab/schema/governance-adjudication-v1.schema.json</path>
<type>file</type>
<content>
1: {
2: "$schema": "https://json-schema.org/draft/2020-12/schema",
3: "$id": "https://institutional-transition-lab.local/schema/governance-adjudication-v1.schema.json",
4: "title": "Performance-blinded governance adjudication v1",
5: "type": "object",
6: "additionalProperties": false,
7: "required": [
8: "schema_version",
9: "adjudication_id",
10: "design_status",
11: "transition_dates_excluded",
12: "outcome_data_used",
13: "inputs",
14: "reviewers",
15: "records"
16: ],
17: "properties": {
18: "schema_version": {"const": 1},
19: "adjudication_id": {"type": "string", "minLength": 1},
20: "design_status": {
21: "const": "performance_blinded_llm_assisted_source_adjudication"
22: },
23: "transition_dates_excluded": {"const": true},
24: "outcome_data_used": {"const": false},
25: ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path> <type>file</type> <content> 1: from __future__ import annotations 2: 3: import copy 4: import json 5: import zipfile 6: from pathlib import Path 7: 8: import pytest 9: 10: from institution_lab.governance_adjudication import ( 11: DECISIONS, 12: FROZEN_INPUT_PINS, 13: REQUIRED_INPUT_PINS, 14: REVIEWER_ROLES, 15: REVIEWER_TYPES, 16: validate_adjudication, 17: validate_against_schema, 18: verify_input_manifest, 19: ) 20: from institution_lab.governance_coding import ( 21: BODY_PATCH_RELATIONS, 22: CHANGE_STATUSES, 23: DIRECTIONS, 24: EVENT_KINDS, 25: RECORD_CLASSES, 26: RIGHT_KINDS, 27: ) 28: 29: ROOT = Path(__file__).parents[1] 30: 31: 32: def test_adjudication_schema_matches_validator_ontology() -> None: 33: schema = json.loads((ROOT / "schema/governance-adjudication-v1.schema.json").read_text()) 34: coding = schema["$defs"]["coding"]["properties"] 35: power_change = ...
<path>/home/byk/Code/institutional-transition-lab/pyproject.toml</path> <type>file</type> <content> 1: [build-system] 2: requires = ["hatchling"] 3: build-backend = "hatchling.build" 4: 5: [project] 6: name = "institutional-transition-lab" 7: version = "0.1.0" 8: description = "Empirical analysis of institutional power transitions and performance regimes" 9: readme = "README.md" 10: requires-python = ">=3.12" 11: dependencies = [ 12: "jsonschema>=4.23,<5", 13: ] 14: 15: [project.scripts] 16: institution-lab-oss = "institution_lab.oss_pipeline:main" 17: institution-lab-llm-eval = "institution_lab.llm_event_probe:main" 18: institution-lab-llm-triage = "institution_lab.llm_candidate_triage:main" 19: institution-lab-retrieve-oss-sources = "institution_lab.source_retrieval:main" 20: institution-lab-enrich-oss-sources = "institution_lab.source_enrichment:main" 21: institution-lab-governance-coding = "institution_lab.governance_coding:main" 22: institution-lab-governance-adjudication = ...
<path>/home/byk/Code/institutional-transition-lab/cases/review/oss-governance-adjudication-v1.inputs.json</path>
<type>file</type>
<content>
1: {
2: "adjudication_schema_sha256": "6181f5cb60a0462f83be00be301763daa89f11581e68ee1c46a84e8bf4e5be31",
3: "artifact_id": 9642983578,
4: "artifact_sha256": "1f6be91fcd98e43e99d82d5f274536f6c23616e7e6c711e26324a2cc4b06ef06",
5: "coding_package_file_sha256": "14644759f53071adfe1173d4e25bb01931b8c1591fa8a48295dbd0556a754769",
6: "coding_schema_sha256": "bfc735a7d8602d1f236a88c2d23b195f0600a8bae86fb8c8936afe7883a41b8e",
7: "documents_file_sha256": "2a62ab05687982c112a29f6126227c26c3ff8defc89ca64662753aaf85c3c0ce",
8: "documents_sha256": "d70f7f795847968339a645b6dbd6d4428af6bd6044de44fa2602d4c4343b0ee9",
9: "protocol_sha256": "06d4b3407677fe7fb23046f70164a366db06638e8a29af3daf7661f00d565933",
10: "workflow_run_id": 33063470968
11: }
(End of file - total 11 lines)
</content>
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_coding.py</path>
<type>file</type>
<content>
1: """Prepare and validate performance-blinded coding of enriched governance records."""
2:
3: from __future__ import annotations
4:
5: import argparse
6: import hashlib
7: import json
8: import math
9: from collections import Counter
10: from datetime import date
11: from pathlib import Path
12: from typing import Any
13:
14: RECORD_CLASSES = {
15: "effective_institutional_change",
16: "announced_institutional_change",
17: "proposal_only",
18: "control_event",
19: "no_event",
20: "abstain",
21: }
22: EVENT_KINDS = {
23: "leadership",
24: "board_or_steering",
25: "control_rights",
26: "reorganization",
27: "foundation_transfer",
28: "license",
29: "fork",
30: "reunification",
31: "strategy",
32: "product",
33: "external",
34: }
35: RIGHT_KINDS = {
36: "appoint",
37: "remove",
38: ...
<path>/home/byk/Code/institutional-transition-lab</path> <type>directory</type> <entries> .git/ .github/ .gitignore .lore.md .pytest_cache/ .ruff_cache/ .venv/ AGENTS.md cases/ data/ docs/ node_modules/ package-lock.json package.json PROTOCOL.md pyproject.toml README.md research/ schema/ scripts/ src/ tests/ uv.lock (23 entries) </entries>
diff --git a/pyproject.toml b/pyproject.toml index e6ab3f2..e976154 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -8,7 +8,9 @@ version = "0.1.0" description = "Empirical analysis of institutional power transitions and performance regimes" readme = "README.md" requires-python = ">=3.12" -dependencies = [] +dependencies = [ + "jsonschema>=4.23,<5", +] [project.scripts] institution-lab-oss = "institution_lab.oss_pipeline:main" @@ -17,6 +19,7 @@ institution-lab-llm-triage = "institution_lab.llm_candidate_triage:main" institution-lab-retrieve-oss-sources = "institution_lab.source_retrieval:main" institution-lab-enrich-oss-sources = "institution_lab.source_enrichment:main" institution-lab-governance-coding = "institution_lab.governance_coding:main" +institution-lab-governance-adjudication = "institution_lab.governance_adjudication:main" [project.optional-dependencies] analysis = [
M pyproject.toml ?? cases/review/oss-governance-adjudication-v1.inputs.json ?? schema/governance-adjudication-v1.schema.json ?? src/institution_lab/governance_adjudication.py ?? tests/test_governance_adjudication.py
============================= test session starts ============================== platform linux -- Python 3.13.11, pytest-8.4.2, pluggy-1.6.0 rootdir: /home/byk/Code/institutional-transition-lab configfile: pyproject.toml plugins: anyio-4.14.2 collected 42 items tests/test_governance_adjudication.py .................................. [ 80%] ........ [100%] ============================== 42 passed in 0.31s ==============================
unauthorized_model [] open
whitespace_claims [] open
missing_basis [] open
duplicate_edges [] open
bounded_resolved [] open []
schema_manual True [] open
malformed_crash TypeError unhashable type: 'list'
invalid_revision ['record example-1: graph_edge_revision.added is not derived'] [{'record_id': 'example-1', 'basis': 'luna_a', 'added': [{'actor': 'The council', 'right_kind': 'approve', 'target': 'releases', 'direction': 'added', 'change_status': 'effective', 'scope': 'project releases', 'evidence_refs': [{'source_id': 'body', 'quote': 'The council may approve releases.'}]}], 'removed': []}]
- `example-1`: 1 added, 0 removed
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
1: from __future__ import annotations
2:
3: import argparse
4: import hashlib
5: import json
6: import zipfile
7: from collections import Counter
8: from pathlib import Path
9: from typing import Any
10:
11: from jsonschema import Draft202012Validator, FormatChecker
12:
13: from institution_lab.governance_coding import validate_response
14:
15: DECISIONS = {"accept", "revise", "reject", "abstain"}
16: INSTITUTIONAL_CLASSES = {
17: "effective_institutional_change",
18: "announced_institutional_change",
19: "proposal_only",
20: }
21: REVIEWER_TYPES = {"human", "llm", "human_llm_assisted"}
22: REVIEWER_ROLES = {"source_adjudicator", "evidence_auditor", "synthesizer"}
23: RESPONSE_SOURCES = {"luna_a", "luna_b", "terra_advisory"}
24: REQUIRED_INPUT_PINS = {
25: "workflow_run_id",
26: "artifact_id",
27: "artifact_sha256",
28: "documents_sha256",
29: ...
<path>/home/byk/Code/institutional-transition-lab/schema/governance-adjudication-v1.schema.json</path>
<type>file</type>
<content>
1: {
2: "$schema": "https://json-schema.org/draft/2020-12/schema",
3: "$id": "https://institutional-transition-lab.local/schema/governance-adjudication-v1.schema.json",
4: "title": "Performance-blinded governance adjudication v1",
5: "type": "object",
6: "additionalProperties": false,
7: "required": [
8: "schema_version",
9: "adjudication_id",
10: "design_status",
11: "transition_dates_excluded",
12: "outcome_data_used",
13: "inputs",
14: "reviewers",
15: "records"
16: ],
17: "properties": {
18: "schema_version": {"const": 1},
19: "adjudication_id": {"type": "string", "minLength": 1},
20: "design_status": {
21: "const": "performance_blinded_llm_assisted_source_adjudication"
22: },
23: "transition_dates_excluded": {"const": true},
24: "outcome_data_used": {"const": false},
25: ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path> <type>file</type> <content> 1: from __future__ import annotations 2: 3: import copy 4: import json 5: import zipfile 6: from pathlib import Path 7: 8: import pytest 9: 10: from institution_lab.governance_adjudication import ( 11: DECISIONS, 12: FROZEN_INPUT_PINS, 13: REQUIRED_INPUT_PINS, 14: REVIEWER_ROLES, 15: REVIEWER_TYPES, 16: validate_adjudication, 17: validate_against_schema, 18: verify_input_manifest, 19: ) 20: from institution_lab.governance_coding import ( 21: BODY_PATCH_RELATIONS, 22: CHANGE_STATUSES, 23: DIRECTIONS, 24: EVENT_KINDS, 25: RECORD_CLASSES, 26: RIGHT_KINDS, 27: ) 28: 29: ROOT = Path(__file__).parents[1] 30: 31: 32: def test_adjudication_schema_matches_validator_ontology() -> None: 33: schema = json.loads((ROOT / "schema/governance-adjudication-v1.schema.json").read_text()) 34: coding = schema["$defs"]["coding"]["properties"] 35: power_change = ...
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_coding.py</path>
<type>file</type>
<content>
228: def parse_response(text: str) -> dict[str, Any]:
229: candidate = text.strip()
230: if candidate.startswith("```"):
231: lines = candidate.splitlines()
232: if len(lines) < 3 or lines[-1].strip() != "```":
233: raise ValueError("unterminated JSON code fence")
234: candidate = "\n".join(lines[1:-1]).strip()
235: value = json.loads(candidate)
236: if not isinstance(value, dict):
237: raise ValueError("response must be an object")
238: return value
239:
240:
241: def validate_response(response: dict[str, Any], document: dict[str, Any]) -> list[str]:
242: errors: list[str] = []
243: if set(response) != RESPONSE_KEYS:
244: errors.append("response has missing or unexpected keys")
245: record_class = response.get("record_class")
246: if record_class not in ...
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_coding.py</path>
<type>file</type>
<content>
1: """Prepare and validate performance-blinded coding of enriched governance records."""
2:
3: from __future__ import annotations
4:
5: import argparse
6: import hashlib
7: import json
8: import math
9: from collections import Counter
10: from datetime import date
11: from pathlib import Path
12: from typing import Any
13:
14: RECORD_CLASSES = {
15: "effective_institutional_change",
16: "announced_institutional_change",
17: "proposal_only",
18: "control_event",
19: "no_event",
20: "abstain",
21: }
22: EVENT_KINDS = {
23: "leadership",
24: "board_or_steering",
25: "control_rights",
26: "reorganization",
27: "foundation_transfer",
28: "license",
29: "fork",
30: "reunification",
31: "strategy",
32: "product",
33: "external",
34: }
35: RIGHT_KINDS = {
36: "appoint",
37: "remove",
38: ...
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_coding.py</path>
<type>file</type>
<content>
377: def _edge_signature(response: dict[str, Any] | None) -> tuple[tuple[str, ...], ...]:
378: if not response:
379: return ()
380: edges = []
381: for edge in response.get("power_changes", []):
382: references = tuple(
383: sorted(
384: (
385: str(reference.get("source_id", "")).strip(),
386: str(reference.get("quote", "")).strip(),
387: )
388: for reference in edge.get("evidence_refs", [])
389: if isinstance(reference, dict)
390: )
391: )
392: edges.append(
393: (
394: *(str(edge.get(key, "")).strip() for key in (
395: "actor",
396: "right_kind",
397: "target",
398: "direction",
399: ...
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
1: from __future__ import annotations
2:
3: import argparse
4: import hashlib
5: import json
6: import zipfile
7: from collections import Counter
8: from pathlib import Path
9: from typing import Any
10:
11: from jsonschema import Draft202012Validator, FormatChecker
12:
13: from institution_lab.governance_coding import validate_response
14:
15: DECISIONS = {"accept", "revise", "reject", "abstain"}
16: INSTITUTIONAL_CLASSES = {
17: "effective_institutional_change",
18: "announced_institutional_change",
19: "proposal_only",
20: }
21: REVIEWER_TYPES = {"human", "llm", "human_llm_assisted"}
22: REVIEWER_ROLES = {"source_adjudicator", "evidence_auditor", "synthesizer"}
23: RESPONSE_SOURCES = {"luna_a", "luna_b", "terra_advisory"}
24: REQUIRED_INPUT_PINS = {
25: "workflow_run_id",
26: "artifact_id",
27: "artifact_sha256",
28: "documents_sha256",
29: ...
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
681: lines.extend(["", "## Frozen inputs", ""])
682: for field, value in adjudication.get("inputs", {}).items():
683: lines.append(f"- `{field}`: `{value}`")
684: lines.extend(["", "## Reviewer provenance", ""])
685: for reviewer in summary["reviewer_provenance"]:
686: model = reviewer.get("model") or "none"
687: lines.append(
688: f"- `{reviewer['reviewer_id']}`: {reviewer['reviewer_type']}, "
689: f"`{model}`, {reviewer['role']}; {reviewer['assistance']}"
690: )
691: lines.extend(["", "## Graph-edge revisions", ""])
692: for revision in summary["graph_edge_revisions"]:
693: added = len(revision.get("added", []))
694: removed = len(revision.get("removed", []))
695: if added or removed:
696: lines.append(f"- `{revision['record_id']}`: {added} added, {removed} removed")
697: if lines[-1] == "":
698: lines.append("- ...
Your previous validator-audit result was empty. Resume the exact same non-overlapping adversarial audit from your prior context. Do not change files. Audit the current issue #4 implementation, including validate_adjudication_fragment, for fail-open behavior, crashes on malformed JSON, provenance substitution, incorrect gate opening, evidence-grounding bypasses, invalid reviewer provenance, record-set/order bypasses, accepted-response mismatch, and graph-diff mismatch. Run deterministic probes/tests where useful. Return substantive findings first with exact file:line evidence and a deterministic regression for every defect. Classify each as PASS, CONCERN, or MUST-FIX. If no defect exists, state PASS explicitly and list residual risks/testing gaps. Never return empty. End exactly MERGE or DO-NOT-MERGE.
I’m resuming the audit against the current validator and fragment path, within the original file boundary. I’ll probe malformed structures and semantic bypasses directly, then cross-check schema/manual/CLI composition without modifying the worktree.
[
{
"content": "Inspect current full and fragment validators, schema constraints, and existing regression coverage",
"status": "in_progress",
"priority": "high"
},
{
"content": "Run deterministic adversarial probes for gate, provenance, grounding, ordering, acceptance, and graph diffs",
"status": "pending",
"priority": "high"
},
{
"content": "Run separate correctness and security audit passes within the frozen file scope",
"status": "pending",
"priority": "high"
},
{
"content": "Verify findings with targeted existing tests and produce exact line-cited verdict",
"status": "pending",
"priority": "high"
}
]
Subagent depth limit reached (1). Increase "subagent_depth" to allow nested subagents.
Found 64 matches
/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py:
Line 17: validate_adjudication_fragment,
Line 33: def test_adjudication_schema_matches_validator_ontology() -> None:
Line 39: assert set(schema["$defs"]["reviewer"]["properties"]["reviewer_type"]["enum"]) == (
Line 42: assert set(schema["$defs"]["reviewer"]["properties"]["role"]["enum"]) == REVIEWER_ROLES
Line 55: def test_schema_enforces_uri_and_date_formats(field: str, value: str) -> None:
Line 166: "reviewers": [
Line 168: "reviewer_id": "opencode-gpt-5.6-sol-primary",
Line 169: "reviewer_type": "llm",
Line 181: "reviewer_ids": ["opencode-gpt-5.6-sol-primary"],
Line 186: "claim_evidence_refs": [],
Line 188: "graph_edge_revision": {
Line 189: "basis": "luna_a",
Line 225: def test_one_abstention_keeps_canonicalization_gate_closed() -> None:
Line 233: "graph_edge_revision": None,
Line 244: ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path> <type>file</type> <content> 1: from __future__ import annotations 2: 3: import copy 4: import json 5: import zipfile 6: from pathlib import Path 7: 8: import pytest 9: 10: from institution_lab.governance_adjudication import ( 11: DECISIONS, 12: FROZEN_INPUT_PINS, 13: REQUIRED_INPUT_PINS, 14: REVIEWER_ROLES, 15: REVIEWER_TYPES, 16: validate_adjudication, 17: validate_adjudication_fragment, 18: validate_against_schema, 19: verify_input_manifest, 20: ) 21: from institution_lab.governance_coding import ( 22: BODY_PATCH_RELATIONS, 23: CHANGE_STATUSES, 24: DIRECTIONS, 25: EVENT_KINDS, 26: RECORD_CLASSES, 27: RIGHT_KINDS, 28: ) 29: 30: ROOT = Path(__file__).parents[1] 31: 32: 33: def test_adjudication_schema_matches_validator_ontology() -> None: 34: schema = json.loads((ROOT / "schema/governance-adjudication-v1.schema.json").read_text()) 35: coding = ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path> <type>file</type> <content> 690: input_directory = ROOT / "cases/review/oss-governance-adjudication-v1" 691: manifest = json.loads( 692: (ROOT / "cases/review/oss-governance-adjudication-v1.inputs.json").read_text() 693: ) 694: 695: verified = verify_input_manifest( 696: manifest, 697: artifact=input_directory / "artifact-9642983578.zip", 698: documents=input_directory / "documents.json", 699: coding_package=input_directory / "v1.2-coding-package.json", 700: protocol=ROOT / "research/oss-governance-coding-protocol-v1.2.md", 701: coding_schema=ROOT / "schema/governance-coding-v1.schema.json", 702: adjudication_schema=ROOT / "schema/governance-adjudication-v1.schema.json", 703: ) 704: 705: assert verified == FROZEN_INPUT_PINS 706: 707: 708: def test_committed_partial_fragment_validates_but_cannot_be_the_full_envelope() -> None: 709: input_directory = ROOT / ...
<path>/home/byk/Code/institutional-transition-lab/schema/governance-adjudication-v1.schema.json</path>
<type>file</type>
<content>
1: {
2: "$schema": "https://json-schema.org/draft/2020-12/schema",
3: "$id": "https://institutional-transition-lab.local/schema/governance-adjudication-v1.schema.json",
4: "title": "Performance-blinded governance adjudication v1",
5: "type": "object",
6: "additionalProperties": false,
7: "required": [
8: "schema_version",
9: "adjudication_id",
10: "design_status",
11: "transition_dates_excluded",
12: "outcome_data_used",
13: "inputs",
14: "reviewers",
15: "records"
16: ],
17: "properties": {
18: "schema_version": {"const": 1},
19: "adjudication_id": {"type": "string", "minLength": 1},
20: "design_status": {
21: "const": "performance_blinded_llm_assisted_source_adjudication"
22: },
23: "transition_dates_excluded": {"const": true},
24: "outcome_data_used": {"const": false},
25: ...
.......................................... [100%] 42 passed, 2 deselected in 0.14s
The scoped suite passes, but that does not exercise the CLI and only lightly covers fragment malformation. The current implementation still composes schema and manual validation separately, so I’m testing inputs that the schema rejects before manual code handles them; these are the likely crash and fail-open boundaries.
/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py
. [100%] 1 passed in 0.09s
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path>
<type>file</type>
<content>
86: def _documents() -> dict[str, object]:
87: return {
88: "schema_version": 1,
89: "coding_id": "oss-governance-enriched-coding-v1",
90: "coding_schema_sha256": "coding-schema-hash",
91: "design_status": "performance_blinded_model_informed_development",
92: "transition_dates_excluded": True,
93: "outcome_data_used": False,
94: "model_input_fields": [],
95: "documents": [
96: {
97: "record_id": "example-1",
98: "source_url": "https://github.com/acme/example/issues/1",
99: "sources": [
100: {
101: "source_id": "body",
102: "kind": "github_body",
103: "filename": None,
104: "text": "The council may approve releases.",
105: }
106: ],
107: "evidence_bounds": {
108: "source_text_truncated": False,
109: "files_listing_complete": None,
110: "patch_selection_truncated": False,
111: ...
Traceback (most recent call last):
File "<string>", line 1, in <module>
from institution_lab.governance_adjudication import _duplicates; print(_duplicates([{"invalid":"record ID"}]))
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py", line 123, in _duplicates
counts = Counter(values)
File "/home/byk/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/collections/__init__.py", line 611, in __init__
self.update(iterable, **kwds)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/home/byk/.local/share/uv/python/cpython-3.13.11-linux-x86_64-gnu/lib/python3.13/collections/__init__.py", line 703, in update
_count_elements(self, iterable)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'dict'
["schema records.0.record_id: {'invalid': 'record ID'} is not of type 'string'", 'fragment record IDs must be a known subset in frozen document order']
Traceback (most recent call last):
File "<string>", line 1, in <module>
import runpy,json; import institution_lab.governance_adjudication as g; n=runpy.run_path("tests/test_governance_adjudication.py"); a=n["_adjudication"](); d=n["_documents"](); p=n["_coding_package"](); p["summary"]["documents_sha256"]=g.canonical_sha256(d); f={"schema_version":a["schema_version"],"design_status":a["design_status"],"reviewers":a["reviewers"],"records":a["records"]}; f["records"][0]["record_id"]={"invalid":"record ID"}; print([r.get("record_id") for r in f["records"]]); print(g._duplicates([r.get("record_id") for r in f["records"]])); print(g.validate_adjudication_fragment(f,d,p,json.loads(open("schema/governance-adjudication-v1.schema.json").read())))
...
["schema records.0.record_id: {'bad': 1} is not of type 'string'", 'fragment record IDs must be a known subset in frozen document order']
def validate_adjudication_fragment(
fragment: dict[str, Any],
documents: dict[str, Any],
coding_package: dict[str, Any],
schema: dict[str, Any],
) -> list[str]:
fragment_schema = {
"$schema": schema.get("$schema"),
"type": "object",
"additionalProperties": False,
"required": ["schema_version", "design_status", "reviewers", "records"],
"properties": {
"schema_version": {"const": 1},
"design_status": {
"const": "performance_blinded_llm_assisted_source_adjudication"
},
"reviewers": {
"type": "array",
"minItems": 1,
"items": {"$ref": "#/$defs/reviewer"},
},
"records": {
"type": "array",
"minItems": 1,
"items": {"$ref": "#/$defs/record"},
},
},
"$defs": schema.get("$defs", {}),
}
errors = validate_against_schema(fragment, fragment_schema)
reviewers = fragment.get("reviewers", [])
if not isinstance(reviewers, list):
reviewers = []
for index, reviewer in ...
The worktree changed during the audit: the fragment validator now filters non-string IDs and checks row count, while the earlier read did not. I’m treating the latest on-disk version as authoritative and will re-read each cited section before the verdict; no audit edits have been made.
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
115: raise ValueError(f"artifact member mismatch: {member_name}")
116: documents_value = json.loads(documents.read_text())
117: if manifest.get("documents_sha256") != canonical_sha256(documents_value):
118: raise ValueError("input manifest mismatch: documents_sha256")
119: return dict(manifest)
120:
121:
122: def _duplicates(values: list[str]) -> set[str]:
123: counts = Counter(values)
124: return {value for value, count in counts.items() if count > 1}
125:
126:
127: def _source_index(document: dict[str, Any]) -> dict[str, str]:
128: sources = {
129: source["source_id"]: source["text"]
130: for source in document.get("sources", [])
131: if isinstance(source, dict)
132: and isinstance(source.get("source_id"), str)
133: and isinstance(source.get("text"), str)
134: }
135: metadata = {
136: field: document.get(field)
137: for ...
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
648:
649:
650: def validate_against_schema(adjudication: dict[str, Any], schema: dict[str, Any]) -> list[str]:
651: validator = Draft202012Validator(schema, format_checker=FormatChecker())
652: return [
653: f"schema {'.'.join(str(part) for part in error.absolute_path) or '<root>'}: "
654: f"{error.message}"
655: for error in sorted(validator.iter_errors(adjudication), key=lambda item: list(item.path))
656: ]
657:
658:
659: def render_markdown(adjudication: dict[str, Any], summary: dict[str, Any]) -> str:
660: lines = [
661: "# OSS governance adjudication v1",
662: "",
663: f"Canonicalization gate: **{summary['canonicalization_gate']}**",
664: "",
665: "## Decisions",
666: "",
667: ]
668: for decision, count in summary["decision_counts"].items():
669: lines.append(f"- `{decision}`: {count}")
670: lines.extend(["", "## Final ...
Found 8 matches /home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py: Line 17: validate_adjudication_fragment, Line 222: return validate_adjudication(adjudication, documents, package, verified_pins=pins) Line 352: errors, summary = validate_adjudication( Line 715: assert validate_adjudication_fragment(fragment, documents, coding_package, schema) == [] Line 727: errors = validate_adjudication_fragment(fragment, documents, coding_package, schema) /home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py: Line 392: def validate_adjudication( Line 539: def validate_adjudication_fragment( Line 736: errors, summary = validate_adjudication(
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_coding.py</path>
<type>file</type>
<content>
100:
101:
102: def _hash(payload: object) -> str:
103: data = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
104: return hashlib.sha256(data).hexdigest()
105:
106:
107: def _valid_date(value: object) -> bool:
108: if value is None:
109: return True
110: if not isinstance(value, str):
111: return False
112: try:
113: return date.fromisoformat(value).isoformat() == value
114: except ValueError:
115: return False
116:
117:
118: def _nonempty(value: object) -> bool:
119: return isinstance(value, str) and bool(value.strip())
120:
121:
122: def prepare_coding_bundle(
123: enriched: dict[str, Any],
124: documents_payload: dict[str, Any],
125: *,
126: schema_sha256: str = "unversioned-test-schema",
127: ) -> dict[str, Any]:
128: """Join source bodies to the ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path>
<type>file</type>
<content>
240: assert summary["unresolved_record_ids"] == ["example-1"]
241: assert summary["canonicalization_gate"] == "closed"
242:
243:
244: @pytest.mark.parametrize("mutation", ["duplicate", "missing"])
245: def test_record_ids_must_match_frozen_documents_exactly_once(mutation: str) -> None:
246: adjudication = _adjudication()
247: if mutation == "duplicate":
248: adjudication["records"].append(copy.deepcopy(adjudication["records"][0]))
249: else:
250: adjudication["records"].clear()
251:
252: errors, summary = _validate(adjudication)
253:
254: assert any("record IDs must match frozen documents exactly once" in error for error in errors)
255: assert summary["canonicalization_gate"] == "closed"
256:
257:
258: def test_accept_must_equal_the_named_frozen_response() -> None:
259: adjudication = _adjudication()
260: ...
Subagent depth limit reached (1). Increase "subagent_depth" to allow nested subagents.
The latest code fixes the fragment-ID crash, but several adversarial cases still return no errors and open the gate. I’m isolating each case against the current file so the final findings distinguish confirmed defects from policy assumptions.
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
1: from __future__ import annotations
2:
3: import argparse
4: import hashlib
5: import json
6: import zipfile
7: from collections import Counter
8: from pathlib import Path
9: from typing import Any
10:
11: from jsonschema import Draft202012Validator, FormatChecker
12:
13: from institution_lab.governance_coding import validate_response
14:
15: DECISIONS = {"accept", "revise", "reject", "abstain"}
16: INSTITUTIONAL_CLASSES = {
17: "effective_institutional_change",
18: "announced_institutional_change",
19: "proposal_only",
20: }
21: REVIEWER_TYPES = {"human", "llm", "human_llm_assisted"}
22: REVIEWER_ROLES = {"source_adjudicator", "evidence_auditor", "synthesizer"}
23: RESPONSE_SOURCES = {"luna_a", "luna_b", "terra_advisory"}
24: REQUIRED_INPUT_PINS = {
25: "workflow_run_id",
26: "artifact_id",
27: "artifact_sha256",
28: "documents_sha256",
29: ...
unauthorized_model (["schema records: [{'record_id': 'example-1', 'source_url': 'https://github.com/acme/example/issues/1', 'decision': 'accept', 'accepted_from': 'luna_a', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'source_audit': {'inspected_before_codings': True, 'source_ids': ['body'], 'evidence_bounds': {'source_text_truncated': False, 'files_listing_complete': None, 'patch_selection_truncated': False, 'patch_unavailable_count': 0}}, 'rationale': 'The source contains no institutional transition.', 'evidence_limitations': None, 'unresolved_reason': None, 'claim_evidence_refs': [], 'final_coding': {'record_class': 'no_event', 'event_kind': None, 'title': None, 'announced_on': None, 'effective_on': None, 'affected_scope': None, 'body_patch_relation': 'not_applicable', 'power_changes': [], 'confidence': 1.0, 'ambiguity': None}, 'graph_edge_revision': {'basis': 'luna_a', 'added': [], 'removed': []}}] is too short"], [], 'open')
whitespace_claim (["schema records: [{'record_id': ...
([], {'record_count': 1, 'decision_counts': {'revise': 1}, 'class_counts': {'effective_institutional_change': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'revise', 'final_class': 'effective_institutional_change', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': None, 'added': [{'actor': 'The council', 'right_kind': 'approve', 'target': 'releases', 'direction': 'added', 'change_status': 'effective', 'scope': 'project releases', 'evidence_refs': [{'source_id': 'body', 'quote': 'The council may approve releases.'}]}], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'open'})
A confirmed fail-open exists: a single-space quote is schema-valid, occurs in ordinary prose, satisfies every institutional claim field, and leaves the gate open. I’m now checking whether malformed envelopes can prevent the CLI from writing its required closed-gate audit, and whether unresolved states beyond explicit decision="abstain" are omitted.
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_coding.py</path>
<type>file</type>
<content>
228: def parse_response(text: str) -> dict[str, Any]:
229: candidate = text.strip()
230: if candidate.startswith("```"):
231: lines = candidate.splitlines()
232: if len(lines) < 3 or lines[-1].strip() != "```":
233: raise ValueError("unterminated JSON code fence")
234: candidate = "\n".join(lines[1:-1]).strip()
235: value = json.loads(candidate)
236: if not isinstance(value, dict):
237: raise ValueError("response must be an object")
238: return value
239:
240:
241: def validate_response(response: dict[str, Any], document: dict[str, Any]) -> list[str]:
242: errors: list[str] = []
243: if set(response) != RESPONSE_KEYS:
244: errors.append("response has missing or unexpected keys")
245: record_class = response.get("record_class")
246: if record_class not in ...
TypeError unhashable type: 'list'
TypeError unhashable type: 'list'
(['record example-1: source_audit source_ids do not match frozen document'], {'record_count': 1, 'decision_counts': {'accept': 1}, 'class_counts': {'no_event': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'accept', 'final_class': 'no_event', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': 'luna_a', 'added': [], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'closed'})
TypeError unhashable type: 'list'
TypeError unhashable type: 'list'
TypeError unhashable type: 'list'
(['record example-1: accepted coding does not equal luna_a', 'record example-1: power_changes[0] has missing ...
(['record example-1: resolved decision cannot use abstain final class'], {'record_count': 1, 'decision_counts': {'revise': 1}, 'class_counts': {'abstain': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'revise', 'final_class': 'abstain', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': None, 'added': [], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'closed'})
missing_basis ['record example-1: accept revision basis must equal accepted_from'] closed missing_added ['record example-1: graph_edge_revision.added is not derived'] closed missing_records ['adjudication record IDs must match frozen documents exactly once and in order'] closed missing_summary [] open Traceback (most recent call last): File "<stdin>", line 25, in <module> File "<stdin>", line 5, in check TypeError: list indices must be integers or slices, not str
The manual validator also trusts any non-Gemini model name. Since the frozen workflow names the authorized adjudicator in the existing fixture but defines no model allowlist, anthropic/claude and even invented model IDs pass and open canonicalization; this is a separate provenance defect from the whitespace citation bypass.
Found 100 matches (more matches available)
/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py:
Line 39: assert set(schema["$defs"]["reviewer"]["properties"]["reviewer_type"]["enum"]) == (
Line 42: assert set(schema["$defs"]["reviewer"]["properties"]["role"]["enum"]) == REVIEWER_ROLES
Line 91: "design_status": "performance_blinded_model_informed_development",
Line 94: "model_input_fields": [],
Line 166: "reviewers": [
Line 168: "reviewer_id": "opencode-gpt-5.6-sol-primary",
Line 169: "reviewer_type": "llm",
Line 170: "model": "openai/gpt-5.6-sol",
Line 181: "reviewer_ids": ["opencode-gpt-5.6-sol-primary"],
Line 508: adjudication["reviewers"][0]["model"] = "google/gemini-3.7-flash"
Line 512: assert any("Gemini is restricted to event triage" in error for error in errors)
Line 518: adjudication["reviewers"][0]["role"] = "synthesizer"
Line 526: def test_record_reviewer_ids_must_be_unique() -> ...
package summary missing ['frozen coding package documents_sha256 mismatch'] closed package summary list CRASH AttributeError 'list' object has no attribute 'get' document sources dict ['frozen coding package documents_sha256 mismatch', 'record example-1: source_audit source_ids do not match frozen document'] closed document bounds missing ['frozen coding package documents_sha256 mismatch', 'record example-1: source_audit evidence_bounds do not match frozen document'] closed revision added tuple ['record example-1: graph_edge_revision.added is not derived'] closed claim refs dict ['record example-1: claim_evidence_refs must be an array'] closed reviewer ids dict ['record example-1: requires a source_adjudicator', 'record example-1: reviewer_ids must be a non-empty array'] closed inputs list ['inputs must be an object', 'input pin mismatch: adjudication_schema_sha256', 'input pin mismatch: artifact_id', 'input pin mismatch: artifact_sha256', 'input pin mismatch: ...
duplicate final edges [] open bounded resolved [] open revise baseline absent [] open reject baseline absent [] open
None ([], {'record_count': 1, 'decision_counts': {'revise': 1}, 'class_counts': {'no_event': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'revise', 'final_class': 'no_event', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': None, 'added': [], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'open'})
terra_advisory ([], {'record_count': 1, 'decision_counts': {'revise': 1}, 'class_counts': {'no_event': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'revise', 'final_class': 'no_event', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': ...
([], {'record_count': 1, 'decision_counts': {'revise': 1}, 'class_counts': {'no_event': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'revise', 'final_class': 'no_event', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': 'terra_advisory', 'added': [], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'open'})
(['record example-1: terra_advisory is not a valid frozen response'], {'record_count': 1, 'decision_counts': {'accept': 1}, 'class_counts': {'no_event': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'accept', 'final_class': 'no_event', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': 'terra_advisory', 'added': [], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'closed'})
# OSS governance adjudication v1 Canonicalization gate: **closed** ## Decisions - `revise`: 1 ## Final classes - `abstain`: 1 ## Record decisions - `example-1`: `revise` as `abstain`; `opencode-gpt-5.6-sol-primary` ## Unresolved records - None ## Frozen inputs - `workflow_run_id`: `33063470968` - `artifact_id`: `9642983578` - `artifact_sha256`: `0000000000000000000000000000000000000000000000000000000000000000` - `documents_sha256`: `30db86645590e61b04eef35eeed0a6eb08385a0a7f35c74ca0a0e7bed149c06c` - `documents_file_sha256`: `1111111111111111111111111111111111111111111111111111111111111111` - `coding_package_file_sha256`: `2222222222222222222222222222222222222222222222222222222222222222` - `protocol_sha256`: `3333333333333333333333333333333333333333333333333333333333333333` - `coding_schema_sha256`: `4444444444444444444444444444444444444444444444444444444444444444` - `adjudication_schema_sha256`: `5555555555555555555555555555555555555555555555555555555555555555` ## Reviewer ...
([], {'record_count': 1, 'decision_counts': {'revise': 1}, 'class_counts': {'effective_institutional_change': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'revise', 'final_class': 'effective_institutional_change', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': None, 'added': [{'actor': 'a', 'right_kind': 'approve', 'target': 'b', 'direction': 'added', 'change_status': 'effective', 'scope': 'x', 'evidence_refs': [{'source_id': 'body', 'quote': 'council'}]}], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'open'})
proposal_only proposed None None [] effective_institutional_change effective None 2024-01-01 [] effective_institutional_change effective 2025-01-01 2024-01-01 []
bool workflow (['input pin mismatch: workflow_run_id'], 'closed') whitespace reviewer id ([], 'open') whitespace rationale ([], 'open') whitespace assistance ([], 'open') whitespace unresolved ([], 'closed')
document-id ('TypeError', "unhashable type: 'list'")
coding-id ('TypeError', "unhashable type: 'list'")
sources ([], {'record_count': 1, 'decision_counts': {'accept': 1}, 'class_counts': {'no_event': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'accept', 'final_class': 'no_event', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': 'luna_a', 'added': [], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'open'})
sources ('TypeError', "unhashable type: 'list'")
All checks passed! ............................................ [100%] 44 passed in 0.16s
([], {'record_count': 1, 'decision_counts': {'accept': 1}, 'class_counts': {'no_event': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'accept', 'final_class': 'no_event', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': 'terra_advisory', 'added': [], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'open'})
anthropic/claude [] invented/model [] openai/gpt-5.6-sol []
([], {'record_count': 1, 'decision_counts': {'revise': 1}, 'class_counts': {'no_event': 1}, 'unresolved_record_ids': [], 'record_decisions': [{'record_id': 'example-1', 'decision': 'revise', 'final_class': 'no_event', 'reviewer_ids': ['opencode-gpt-5.6-sol-primary'], 'evidence_limitations': None}], 'graph_edge_revisions': [{'record_id': 'example-1', 'basis': 'luna_a', 'added': [], 'removed': []}], 'reviewer_provenance': [{'reviewer_id': 'opencode-gpt-5.6-sol-primary', 'reviewer_type': 'llm', 'model': 'openai/gpt-5.6-sol', 'role': 'source_adjudicator', 'assistance': 'Primary source review and synthesis'}], 'canonicalization_gate': 'open'})
TypeError object of type 'NoneType' has no len()
The latest probes confirm both fail-open acceptance and malformed-input crashes. I’m pinning each defect to the current on-disk lines and running separate correctness and security passes before the final merge verdict.
Subagent depth limit reached (1). Increase "subagent_depth" to allow nested subagents.
[
{
"content": "Re-read current validator, schema, and tests within the frozen audit scope",
"status": "in_progress",
"priority": "high"
},
{
"content": "Verify each suspected defect with deterministic adversarial probes",
"status": "pending",
"priority": "high"
},
{
"content": "Run scoped lint/tests and issue exact line-cited merge verdict",
"status": "pending",
"priority": "high"
}
]
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
1: from __future__ import annotations
2:
3: import argparse
4: import hashlib
5: import json
6: import zipfile
7: from collections import Counter
8: from pathlib import Path
9: from typing import Any
10:
11: from jsonschema import Draft202012Validator, FormatChecker
12:
13: from institution_lab.governance_coding import validate_response
14:
15: DECISIONS = {"accept", "revise", "reject", "abstain"}
16: INSTITUTIONAL_CLASSES = {
17: "effective_institutional_change",
18: "announced_institutional_change",
19: "proposal_only",
20: }
21: REVIEWER_TYPES = {"human", "llm", "human_llm_assisted"}
22: REVIEWER_ROLES = {"source_adjudicator", "evidence_auditor", "synthesizer"}
23: RESPONSE_SOURCES = {"luna_a", "luna_b", "terra_advisory"}
24: REQUIRED_INPUT_PINS = {
25: "workflow_run_id",
26: "artifact_id",
27: "artifact_sha256",
28: "documents_sha256",
29: ...
<path>/home/byk/Code/institutional-transition-lab/schema/governance-adjudication-v1.schema.json</path>
<type>file</type>
<content>
1: {
2: "$schema": "https://json-schema.org/draft/2020-12/schema",
3: "$id": "https://institutional-transition-lab.local/schema/governance-adjudication-v1.schema.json",
4: "title": "Performance-blinded governance adjudication v1",
5: "type": "object",
6: "additionalProperties": false,
7: "required": [
8: "schema_version",
9: "adjudication_id",
10: "design_status",
11: "transition_dates_excluded",
12: "outcome_data_used",
13: "inputs",
14: "reviewers",
15: "records"
16: ],
17: "properties": {
18: "schema_version": {"const": 1},
19: "adjudication_id": {"type": "string", "minLength": 1},
20: "design_status": {
21: "const": "performance_blinded_llm_assisted_source_adjudication"
22: },
23: "transition_dates_excluded": {"const": true},
24: "outcome_data_used": {"const": false},
25: ...
Found 57 matches
/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py:
Line 17: validate_adjudication_fragment,
Line 33: def test_adjudication_schema_matches_validator_ontology() -> None:
Line 55: def test_schema_enforces_uri_and_date_formats(field: str, value: str) -> None:
Line 91: "design_status": "performance_blinded_model_informed_development",
Line 94: "model_input_fields": [],
Line 170: "model": "openai/gpt-5.6-sol",
Line 183: "rationale": "The source contains no institutional transition.",
Line 185: "unresolved_reason": None,
Line 186: "claim_evidence_refs": [],
Line 188: "graph_edge_revision": {
Line 225: def test_one_abstention_keeps_canonicalization_gate_closed() -> None:
Line 231: "unresolved_reason": "Required source evidence is incomplete.",
Line 233: "graph_edge_revision": None,
Line 240: assert summary["unresolved_record_ids"] == ["example-1"]
Line 244: ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path>
<type>file</type>
<content>
225: def test_one_abstention_keeps_canonicalization_gate_closed() -> None:
226: adjudication = _adjudication()
227: record = adjudication["records"][0]
228: record.update({
229: "decision": "abstain",
230: "accepted_from": None,
231: "unresolved_reason": "Required source evidence is incomplete.",
232: "final_coding": None,
233: "graph_edge_revision": None,
234: })
235:
236: errors, summary = _validate(adjudication)
237:
238: assert errors == []
239: assert summary["record_count"] == 1
240: assert summary["unresolved_record_ids"] == ["example-1"]
241: assert summary["canonicalization_gate"] == "closed"
242:
243:
244: @pytest.mark.parametrize("mutation", ["duplicate", "missing"])
245: def test_record_ids_must_match_frozen_documents_exactly_once(mutation: str) -> None:
246: adjudication = _adjudication()
247: if mutation == "duplicate":
248: ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path> <type>file</type> <content> 1: from __future__ import annotations 2: 3: import copy 4: import json 5: import zipfile 6: from pathlib import Path 7: 8: import pytest 9: 10: from institution_lab.governance_adjudication import ( 11: DECISIONS, 12: FROZEN_INPUT_PINS, 13: REQUIRED_INPUT_PINS, 14: REVIEWER_ROLES, 15: REVIEWER_TYPES, 16: validate_adjudication, 17: validate_adjudication_fragment, 18: validate_against_schema, 19: verify_input_manifest, 20: ) 21: from institution_lab.governance_coding import ( 22: BODY_PATCH_RELATIONS, 23: CHANGE_STATUSES, 24: DIRECTIONS, 25: EVENT_KINDS, 26: RECORD_CLASSES, 27: RIGHT_KINDS, 28: ) 29: 30: ROOT = Path(__file__).parents[1] 31: 32: 33: def test_adjudication_schema_matches_validator_ontology() -> None: 34: schema = json.loads((ROOT / "schema/governance-adjudication-v1.schema.json").read_text()) 35: coding = ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path>
<type>file</type>
<content>
585: @pytest.mark.parametrize("field", ["workflow_run_id", "artifact_id"])
586: def test_frozen_input_identifiers_are_reverified(field: str) -> None:
587: adjudication = _adjudication()
588: adjudication["inputs"][field] += 1
589:
590: errors, summary = _validate(adjudication)
591:
592: assert f"input pin mismatch: {field}" in errors
593: assert summary["canonicalization_gate"] == "closed"
594:
595:
596: def test_input_manifest_rejects_changed_file_bytes(
597: tmp_path: Path,
598: monkeypatch: pytest.MonkeyPatch,
599: ) -> None:
600: artifact = tmp_path / "artifact.zip"
601: documents = tmp_path / "documents.json"
602: coding_package = tmp_path / "coding-package.json"
603: protocol = tmp_path / "protocol.md"
604: coding_schema = tmp_path / "coding-schema.json"
605: adjudication_schema = tmp_path / "adjudication-schema.json"
606: paths = {
607: ...
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_coding.py</path>
<type>file</type>
<content>
228: def parse_response(text: str) -> dict[str, Any]:
229: candidate = text.strip()
230: if candidate.startswith("```"):
231: lines = candidate.splitlines()
232: if len(lines) < 3 or lines[-1].strip() != "```":
233: raise ValueError("unterminated JSON code fence")
234: candidate = "\n".join(lines[1:-1]).strip()
235: value = json.loads(candidate)
236: if not isinstance(value, dict):
237: raise ValueError("response must be an object")
238: return value
239:
240:
241: def validate_response(response: dict[str, Any], document: dict[str, Any]) -> list[str]:
242: errors: list[str] = []
243: if set(response) != RESPONSE_KEYS:
244: errors.append("response has missing or unexpected keys")
245: record_class = response.get("record_class")
246: if record_class not in ...
[
{
"content": "Re-read current validator, schema, and tests within the frozen audit scope",
"status": "completed",
"priority": "high"
},
{
"content": "Verify each suspected defect with deterministic adversarial probes",
"status": "in_progress",
"priority": "high"
},
{
"content": "Run scoped lint/tests and issue exact line-cited merge verdict",
"status": "pending",
"priority": "high"
}
]
schema_manual True [] open unauthorized_model [] open [] whitespace_claims [] open [] terra_accept [] open [] missing_named_baseline [] open [] duplicate_edges [] open [] bounded_resolved [] open [] malformed_record_id TypeError unhashable type: 'list' malformed_package_summary AttributeError 'list' object has no attribute 'get' malformed_markdown TypeError object of type 'NoneType' has no len() resolved_as_abstain_summary ['record example-1: resolved decision cannot use abstain final class'] closed [] fragment_bad_summary AttributeError 'list' object has no attribute 'get'
The probes confirm four crash paths and several cases where direct validation opens the gate for invalid provenance or evidence. I’m separating defects in the public validator contract from cases that the CLI’s schema pass already catches, then I’ll run the repository’s scoped checks and give the merge verdict.
No files found
The adversarial cases are now reproducible. I’m pinning each defect to the latest file contents and checking whether existing tests prove the failure; the final review will distinguish fail-open canonicalization defects from audit-output and malformed-input robustness defects.
[
{
"content": "Re-read current validator, schema, and tests within the frozen audit scope",
"status": "completed",
"priority": "high"
},
{
"content": "Verify each suspected defect with deterministic adversarial probes",
"status": "completed",
"priority": "high"
},
{
"content": "Run scoped lint/tests and issue exact line-cited merge verdict",
"status": "in_progress",
"priority": "high"
}
]
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
110: try:
111: member_bytes = archive.read(member_name)
112: except KeyError as error:
113: raise ValueError(f"artifact member missing: {member_name}") from error
114: if member_bytes != durable_path.read_bytes():
115: raise ValueError(f"artifact member mismatch: {member_name}")
116: documents_value = json.loads(documents.read_text())
117: if manifest.get("documents_sha256") != canonical_sha256(documents_value):
118: raise ValueError("input manifest mismatch: documents_sha256")
119: return dict(manifest)
120:
121:
122: def _duplicates(values: list[str]) -> set[str]:
123: counts = Counter(values)
124: return {value for value, count in counts.items() if count > 1}
125:
126:
127: def _source_index(document: dict[str, Any]) -> dict[str, str]:
128: sources = {
129: source["source_id"]: source["text"]
130: for source in ...
<path>/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_adjudication.py</path>
<type>file</type>
<content>
416: reviewers = adjudication.get("reviewers")
417: if not isinstance(reviewers, list) or not reviewers:
418: reviewers = []
419: errors.append("reviewers must be a non-empty array")
420: for index, reviewer in enumerate(reviewers):
421: errors.extend(_validate_reviewer(reviewer, index))
422: reviewer_id_values = [
423: reviewer.get("reviewer_id")
424: for reviewer in reviewers
425: if isinstance(reviewer, dict) and isinstance(reviewer.get("reviewer_id"), str)
426: ]
427: if duplicates := _duplicates(reviewer_id_values):
428: errors.append(f"duplicate reviewer IDs: {sorted(duplicates)}")
429: reviewer_ids = set(reviewer_id_values)
430: reviewer_roles = {
431: reviewer.get("reviewer_id"): reviewer.get("role")
432: for reviewer in reviewers
433: if isinstance(reviewer, dict)
434: }
435:
436: document_rows = ...
<path>/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py</path> <type>file</type> <content> 1: from __future__ import annotations 2: 3: import copy 4: import json 5: import zipfile 6: from pathlib import Path 7: 8: import pytest 9: 10: from institution_lab.governance_adjudication import ( 11: DECISIONS, 12: FROZEN_INPUT_PINS, 13: REQUIRED_INPUT_PINS, 14: REVIEWER_ROLES, 15: REVIEWER_TYPES, 16: validate_adjudication, 17: validate_adjudication_fragment, 18: validate_against_schema, 19: verify_input_manifest, 20: ) 21: from institution_lab.governance_coding import ( 22: BODY_PATCH_RELATIONS, 23: CHANGE_STATUSES, 24: DIRECTIONS, 25: EVENT_KINDS, 26: RECORD_CLASSES, 27: RIGHT_KINDS, 28: ) 29: 30: ROOT = Path(__file__).parents[1] 31: 32: 33: def test_adjudication_schema_matches_validator_ontology() -> None: 34: schema = json.loads((ROOT / "schema/governance-adjudication-v1.schema.json").read_text()) 35: coding = ...
Found 6 matches
/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py:
Line 184: "evidence_limitations": None,
Line 268: def test_resolved_decision_cannot_leave_final_coding_abstained() -> None:
Line 283: assert any("resolved decision cannot use abstain final class" in error for error in errors)
Line 359: assert any("bounded evidence requires evidence_limitations" in error for error in errors)
Line 719: def test_malformed_fragment_record_id_fails_closed() -> None:
Line 746: "evidence_limitations": None,
<path>/home/byk/Code/institutional-transition-lab/schema/governance-adjudication-v1.schema.json</path>
<type>file</type>
<content>
45: "artifact_sha256",
46: "documents_sha256",
47: "documents_file_sha256",
48: "coding_package_file_sha256",
49: "protocol_sha256",
50: "coding_schema_sha256",
51: "adjudication_schema_sha256"
52: ],
53: "properties": {
54: "workflow_run_id": {"type": "integer", "minimum": 1},
55: "artifact_id": {"type": "integer", "minimum": 1},
56: "artifact_sha256": {"$ref": "#/$defs/sha256"},
57: "documents_sha256": {"$ref": "#/$defs/sha256"},
58: "documents_file_sha256": {"$ref": "#/$defs/sha256"},
59: "coding_package_file_sha256": {"$ref": "#/$defs/sha256"},
60: "protocol_sha256": {"$ref": "#/$defs/sha256"},
61: "coding_schema_sha256": {"$ref": "#/$defs/sha256"},
62: "adjudication_schema_sha256": {"$ref": "#/$defs/sha256"}
63: ...
Subagent depth limit reached (1). Increase "subagent_depth" to allow nested subagents.
Independent subagent review remains unavailable because the session is already at the nesting limit, so I’m completing separate correctness and security passes directly. The confirmed failures do not depend on disputed governance semantics; they arise from validator composition, provenance authorization, grounding rules, and malformed input handling.
============================= test session starts ============================== platform linux -- Python 3.13.11, pytest-8.4.2, pluggy-1.6.0 rootdir: /home/byk/Code/institutional-transition-lab configfile: pyproject.toml plugins: anyio-4.14.2 collected 44 items tests/test_governance_adjudication.py .................................. [ 77%] .......... [100%] ============================== 44 passed in 0.22s ==============================
* 🟡 (23:39) `/home/byk/Code/institutional-transition-lab/schema/governance-adjudication-v1.schema.json` is a 270-line Draft 2020-12 JSON Schema with `additionalProperties: false`; its top-level envelope requires `schema_version`, `adjudication_id`, `design_status`, `transition_dates_excluded`, `outcome_data_used`, `inputs`, `reviewers`, and `records`, and requires exactly 40 records via `minItem…
* 🟡 (23:38) An adversarial probe terminated with `TypeError: object of type 'NoneType' has no len()`. * 🟡 (23:38) Assistant stated that the latest probes confirmed both fail-open acceptance and malformed-input crashes and that defects were being pinned to current on-disk lines before separate correctness and security passes and a final merge verdict. * 🟡 (23:38) A nested-subagent attempt faile…
* 🟡 (23:15) Assistant reported that the latest code fixed a malformed fragment-ID crash, but several adversarial inputs still produced no errors and opened the canonicalization gate; investigation focused on separating confirmed defects from policy assumptions. * 🟡 (23:17) Adversarial validation showed that an otherwise normal one-record adjudication produced the schema error `schema records: […
* 🔴 (23:15) `prepare_coding_bundle()` in `/home/byk/Code/institutional-transition-lab/src/institution_lab/governance_coding.py:122-225` joins enriched records to source documents, omits model-side triage metadata, and defaults `schema_sha256` to `"unversioned-test-schema"`. * 🔴 (23:15) `prepare_coding_bundle()` requires `enriched["records"]` and `documents_payload["documents"]` to be lists cont…
* 🔴 (23:04) Scoped test suite completed with `42 passed, 2 deselected in 0.14s`. * 🟡 (23:04) Assistant assessed that the scoped suite did not exercise the CLI and only lightly covered malformed fragments. Assistant identified a likely crash/fail-open boundary because schema validation and manual validation are composed separately, then began testing inputs rejected by the schema before manual h…
* 🔴 (23:03) User provided the complete 270-line `schema/governance-adjudication-v1.schema.json`, a JSON Schema Draft 2020-12 document with `$id` `https://institutional-transition-lab.local/schema/governance-adjudication-v1.schema.json`, title `Performance-blinded governance adjudication v1`, and top-level `additionalProperties: false`. * 🔴 (23:03) The top-level adjudication schema requires exac…
Date: Sep 8, 2026 * 🔴 (22:35) User provided `src/institution_lab/governance_coding.py:228-315`: `parse_response()` strips an optional fenced block, requires a terminating ``` fence, parses JSON with `json.loads()`, and raises `ValueError("response must be an object")` unless the result is a dictionary; `validate_response()` validates exact response keys, ontology values, nonempty nullable string…
Date: Sep 8, 2026 * 🔴 (22:31) User provided `/home/byk/Code/institutional-transition-lab/tests/test_governance_adjudication.py`, a 727-line pytest suite for `institution_lab.governance_adjudication`. * 🔴 (22:31) `tests/test_governance_adjudication.py` imports `DECISIONS`, `FROZEN_INPUT_PINS`, `REQUIRED_INPUT_PINS`, `REVIEWER_ROLES`, `REVIEWER_TYPES`, `validate_adjudication`, `validate_against_s…
Date: Sep 8, 2026 * 🔴 (22:30) User provided `/home/byk/Code/institutional-transition-lab/schema/governance-adjudication-v1.schema.json`, a JSON Schema Draft 2020-12 document with `$id` `https://institutional-transition-lab.local/schema/governance-adjudication-v1.schema.json` and title `Performance-blinded governance adjudication v1`. * 🔴 (22:30) The governance-adjudication root schema requires …
* 🔴 (22:30) User provided governance-adjudication result statuses: `unauthorized_model [] open`, `whitespace_claims [] open`, `missing_basis [] open`, `duplicate_edges [] open`, and `bounded_resolved [] open []`. * 🔴 (22:30) User reported `schema_manual True [] open`. * 🔴 (22:30) User reported malformed input crashes with `TypeError unhashable type: 'list'` under `malformed_crash`. * 🔴 (22:30…
* 🟡 (22:26) Test run in `/home/byk/Code/institutional-transition-lab` using Python 3.13.11, pytest 8.4.2, pluggy 1.6.0, and anyio 4.14.2 collected 42 tests from `tests/test_governance_adjudication.py`; all 42 passed in 0.31s.
* 🟡 (21:51) `cases/review/oss-governance-adjudication-v1.inputs.json` pins 9 provenance values: `adjudication_schema_sha256=6181f5cb60a0462f83be00be301763daa89f11581e68ee1c46a84e8bf4e5be31`, `artifact_id=9642983578`, `artifact_sha256=1f6be91fcd98e43e99d82d5f274536f6c23616e7e6c711e26324a2cc4b06ef06`, `coding_package_file_sha256=14644759f53071adfe1173d4e25bb01931b8c1591fa8a48295dbd0556a754769`, `c…
Date: Sep 8, 2026 * 🟡 (21:51) `tests/test_governance_adjudication.py` is a 727-line pytest suite importing `DECISIONS`, `FROZEN_INPUT_PINS`, `REQUIRED_INPUT_PINS`, `REVIEWER_ROLES`, `REVIEWER_TYPES`, `validate_adjudication()`, `validate_against_schema()`, and `verify_input_manifest()` from `institution_lab.governance_adjudication`, plus coding ontology constants from `institution_lab.governance_…
Date: Sep 8, 2026 * 🔴 [requested-review] (21:50) User requested a substantive READ-ONLY adversarial audit of the uncommitted governance adjudication validator in `/home/byk/Code/institutional-transition-lab`; files must not be modified. * 🔴 [enforced-scope] (21:50) User restricted inspection to `src/institution_lab/governance_adjudication.py`, `schema/governance-adjudication-v1.schema.json`, `t…