Distillation

Date: September 9, 2026

• 🔴 (05:38) User stated issue #4 in /home/byk/Code/institutional-transition-lab concerns actual decompressed-byte enforcement, is disjoint from declared-size arithmetic, and replaces a resource-accounting review that returned empty twice.
• 🔴 [requested-security-audit] (05:38) User requested a fresh, minimal, substantive, read-only security audit limited to actual ZIP decompressed-byte enforcement in /home/byk/Code/institutional-transition-lab.
• 🔴 [enforced-read-only-scope] (05:38) User required no file edits and restricted review to relevant ZIP stream code/constants in src/institution_lab/governance_adjudication.py, narrow tests, and small read-only probes.
• 🔴 (05:38) User defined the permitted audit scope as: member stream read amount and loop behavior; actual per-member and aggregate decompressed-byte accounting; short/partial reads; streams producing more or fewer bytes than declared; compression-bomb termination; and cleanup/error behavior.
• 🔴 (05:38) User explicitly excluded: central declared-size arithmetic; CRC; required-member identity; content equality; archive names, paths, modes, encryption, and member count; regular-file reads; JSON, pins, and hashes; CLI outputs; and all governance records, codings, outcomes, reports, dates, detectors, and source adjudication.
• 🔴 [requested-regression-tests] (05:38) User required deterministic regression guidance for every defect.
• 🔴 (05:38) User required findings first, exact file:line references, PASS/CONCERN/MUST-FIX/BLOCKED labels, substantive evidence rather than an empty response, and a final line of exactly MERGE or DO-NOT-MERGE.
• 🟡 (05:38) Assistant committed to tracing only ZIP member streaming and actual decompressed-byte counters, running focused read-only tests and synthetic stream probes, and avoiding excluded metadata, governance, provenance, and CLI paths.
• 🟡 (05:39) Relevant constants identified in src/institution_lab/governance_adjudication.py: MAX_INPUT_BYTES = 16 * 1024 * 1024 at line 65, MAX_ARTIFACT_BYTES = 32 * 1024 * 1024 at line 66, MAX_ARCHIVE_MEMBER_BYTES = 16 * 1024 * 1024 at line 68, and MAX_ARCHIVE_EXPANDED_BYTES = 64 * 1024 * 1024 at line 69.
• 🟡 (05:39) Relevant ZIP enforcement code identified in src/institution_lab/governance_adjudication.py: archive opened via zipfile.ZipFile(io.BytesIO(artifact)) at line 180; declared aggregate check sum(info.file_size for info in infos) > MAX_ARCHIVE_EXPANDED_BYTES at lines 188-189; required-member iteration at lines 190-194; declared per-member check at lines 199-200; member stream opened at line 201; one-shot source.read(MAX_ARCHIVE_MEMBER_BYTES + 1) at line 202; returned-buffer length check at lines 203-204; content comparison at lines 205-206; zipfile.BadZipFile mapped to ValueError("artifact is not a valid ZIP archive") at lines 207-208.
• 🟡 (05:39) Existing ZIP-related tests found in tests/test_governance_adjudication.py include ZIP construction at lines 65-67, 703-705, 743-745; duplicate-member handling around lines 883-892; excessive-member-count handling around lines 908-919; and test_committed_partial_fragment_validates_but_cannot_be_the_full_envelope() at line 979.
• 🟡 (05:42) Runtime inspected was Python 3.12.3. CPython ZipExtFile.read(n) was observed to loop through _read1(n) until the requested return amount or EOF, while _read1() can decompress input and then clip output with data = data[:self._left]; this means bytes produced internally by the decoder can exceed bytes returned to the caller.
• 🟡 (05:49) CPython ZipExtFile._read2(n) was observed to cap each compressed-source request by _compress_left, call self._fileobj.read(n), decrement _compress_left by the partial amount actually read, and raise EOFError on an empty read.
• 🟡 (05:51) CPython’s ZIP LZMA decompressor was observed to call self._decomp.decompress(data) without a maximum-output argument, then expose only its EOF state; this supports the concern that a supplied compressed chunk may expand fully before ZipExtFile clips returned bytes.
• 🟡 (05:52) An initial probe invocation failed with ModuleNotFoundError: No module named 'institution_lab'; later read-only probes ran successfully.
• 🟡 (05:53) Synthetic BZIP2 probe result for a stream expanding beyond the limit but declaring a one-byte member: more-than-declared encoded_expansion=16777217 declared=1 returned=1 next_read=0 peak_traced=47587880.
• 🟡 (05:53) Synthetic shorter-than-declared stream probe result: fewer-than-declared encoded_expansion=1 declared=8 returned=1 next_read=0.
• 🟡 (05:53) Partial compressed-source-read probe result: partial-compressed-reads returned=32768 underlying_partial_reads=46.
• 🟡 (05:53) Cleanup/error probe result: read-error OSError source_closed=True archive_closed=True.
• 🟡 (05:58) Assistant reported a MUST-FIX finding: returned length does not bound actual decompression. Although MAX_ARCHIVE_MEMBER_BYTES and MAX_ARCHIVE_EXPANDED_BYTES are defined at src/institution_lab/governance_adjudication.py:68-69, the stream code only calls source.read(MAX_ARCHIVE_MEMBER_BYTES + 1) at lines 201-202, checks the returned buffer at lines 203-204, and maintains no actual aggregate decompressed-byte counter across the member loop at lines 194 and 201-204.
• 🟡 (05:58) Assistant identified the root cause as CPython 3.12.3 ZipExtFile clipping returned data to the member’s declared size after its BZIP2 and LZMA paths may have decompressed an entire supplied compressed chunk. Consequently, len(member_bytes) measures returned bytes rather than actual decoder output.
• 🟡 (05:58) Assistant concluded from the BZIP2 probe that actual expansion reached 16777217 bytes—one byte beyond the 16 MiB member limit—while line 203 observed only one returned byte; peak traced memory was 47587880 bytes. The same behavior leaves aggregate actual decompression unbounded and permits compression-bomb CPU/memory exhaustion before either returned-length check runs.
• 🟡 (05:58) Assistant concluded from the shorter-than-declared probe that a stream declaring 8 bytes but producing 1 byte reached EOF without a stream error (returned=1 next_read=0), so current stream handling does not explicitly reject underproduction relative to the declaration.
• 🟡 (05:58) Assistant prescribed 4 deterministic regressions in this exact order: 1. Lower limits with monkeypatch; construct BZIP2 and LZMA streams whose expansion is member_limit + 1 but whose declared length matches a short valid prefix, and require rejection based on actual expansion. 2. Construct two streams individually below the member limit but collectively above a lowered aggregate limit, and require rejection while reading the second stream. 3. Construct a stream shorter than its declared length, and require an explicit stream-length error. 4. Keep fixtures small and compressible so the tests fail safely on the current implementation without substantial memory consumption.
• 🟡 (05:58) Assistant reported PASS for partial compressed reads: for the concrete ZipExtFile opened at src/institution_lab/governance_adjudication.py:201, the one-shot call at line 202 internally loops until the requested amount or EOF. A source limited to one byte per underlying read returned all 32768 bytes after 46 partial reads. Assistant noted this does not mitigate the decompressor-output bypass.
• 🟡 (05:58) Assistant reported PASS for ordinary read-error cleanup and fail-closed behavior: nested context managers at src/institution_lab/governance_adjudication.py:180 and line 201 closed both the archive and member source when reading raised OSError; the exception propagated rather than being swallowed. Compression-bomb process exhaustion remained part of the MUST-FIX finding.
• 🟡 (05:58) Assistant found no existing narrow test referencing decompression codecs, actual-byte counters, stream reads, or the relevant size-limit constants.
• 🟡 (05:58) Assistant stated unrelated tests were not run because they were outside the requested scope and confirmed no files were edited.
• 🟡 (05:58) Assistant’s final disposition was DO-NOT-MERGE.