fix(ledger): report malformed entries during verification

[sturlese]

Summary

This bughunt fix makes ledger verification report every malformed record as an integrity failure instead of crashing audit or report commands, and keeps every ledger reader on the same strict UTF-8 boundary.

Bug

flightdeck audit verify and reporting leaked uncaught decoding, parser-limit, type, key, or Unicode errors when a ledger record was corrupt. Byte input could also be auto-detected as UTF-16 and incorrectly verify or render as valid. These paths violated the documented UTF-8 JSONL integrity-failure contract.

Root cause

Ledger readers assumed records were UTF-8 JSON objects with safe scalar content and every required field before parsing or chain verification.

Fix

• Stream non-empty records as bytes and decode them strictly as UTF-8 before JSON parsing.
• Apply the same strict UTF-8 boundary to entries() so audit tail cannot accept transcoded records.
• Convert JSON syntax and parser-limit errors into controlled verification failures.
• Validate required fields and their types before chain verification.
• Contain invalid Unicode discovered during canonical hash calculation.
• Preserve first-break semantics, total non-empty record counts, and existing valid-chain behavior.

Testing

• Regression tests verified failing against each affected source revision before the fixes.
• .venv/bin/python -m pytest tests/test_ledger.py tests/test_cli_and_html.py — 47 passed.
• .venv/bin/python -m pytest --cov=flightdeck --cov-fail-under=85 — 245 passed, 94.26% coverage.
• .venv/bin/ruff check src tests — passed.
• Independent 16-case adversarial corruption matrix across library, CLI, and report formats — passed.
• Offline demo, ledger verification, and dashboard artifact check — passed.

[sturlese]

Bughunt adversarial gate: REQUEST CHANGES

The second and final automated gate round reproduced three HIGH-severity corruption paths on commit 8de4e67:

• json.loads(bytes) auto-detects UTF-16, so a correctly hashed record transcoded from UTF-8 to UTF-16 verifies as valid instead of failing encoding validation.
• A JSON record with a 5,000-digit integer raises Python's integer-limit ValueError, escaping VerifyResult and crashing both audit and report flows.
• A JSON-escaped lone surrogate reaches hash canonicalization and raises UnicodeEncodeError, again bypassing the controlled integrity-failure path.

All three were independently reproduced against the PR head. The streaming/counting logic, entry-shape validation, existing chain semantics, and regression coverage otherwise passed review.

Per the bughunt merge guardrail, this PR is intentionally left open and unmerged after two gate rounds. A human should review the findings before another implementation round.

[sturlese]

Third adversarial gate: PASS

The final review found one consistency gap: Ledger.entries() still allowed Python's JSON decoder to auto-detect UTF-16 even though Ledger.verify() rejected it. The final patch now decodes records explicitly as UTF-8 in both paths, with a regression test that failed before the fix.

Final evidence on commit 1f790203f39db50042d2e2fcd48aa68f75f30a8a:

• Focused ledger and CLI/report suite: 47 passed.
• Full suite: 245 passed with 94.26% coverage.
• Ruff and diff checks: passed.
• Independent 16-case corruption matrix across direct verification, audit CLI, tail, and terminal/JSON/HTML reports: passed without exceptions or false positives.
• Build Squad audit re-review: APPROVE, no P1/P2 findings.
• Strong-model final refutation: APPROVE, no HIGH/P1/P2 findings.

All findings from the three adversarial rounds are resolved.