Integrate PRs #73–#77 with cross-PR regression fixes

[JordanCoin]

Merges all five open PRs together and fixes the defects that only appear when they are combined. Each PR was green on its own, but all five touch records_search in cli.py and none had ever been tested against the others.

Closes #73, closes #74, closes #75, closes #76, closes #77.

Conflicts resolved

• fix: keep records raw output valid JSON #73 vs docs: list all records search sources #74 — both add tests/test_records_cli.py (add/add). Kept both tests: raw-JSON output from fix: keep records raw output valid JSON #73, help-text coverage from docs: list all records search sources #74.
• The other four merged cleanly.

Defects found by reviewing the integrated result

crossref.py (from #76)

• A failing source raised, which discarded every hit the checker had already collected. One unreadable ICIJ CSV erased real matches from the files that did open. _SourceCheckError now carries partial hits, _check_icij keeps reading the remaining files, and the loop merges those hits while still marking the source ERRORED. Failure stays visible; evidence stops disappearing.
• Every remote checker fails through result.error, which was wrapped in a bare RuntimeError — so a 503, a DNS failure and a schema change all reported ERRORED(RuntimeError), while opensanctions (a different code path) reported the real kind in the same run. The adapter's own failure kind now comes through. Only the leading exception name travels: the detail can echo the looked-up entity name via the URL, and these statuses land in a report the user may share.
• An ICIJ directory containing no CSVs reported status checked — a clean bill of health after reading zero bytes. A failed or nested extraction now reports NoICIJData.

cli.py (from #73 and #75)

• records search --raw on an adapter error printed [] and exited 0, so a script could not tell an API failure from "no results". The error goes to stderr with exit 1, leaving stdout parseable JSON.
• The adapter-crash handler still wrote its message to stdout via rprint, contradicting the pipe-safety comment two lines above it. It goes to stderr now.
• The "0 results is not proof that no filing exists" caveat only fired with --type, so the broader untyped SEC search — where EDGAR full-text coverage gaps bite hardest — kept the flat "No results found".

sec_edgar.py (from #77)

• The archive-shard walk fired every request back to back. For a long-lived filer that is 20+ unpaced hits on data.sec.gov, which is the burst SEC's fair-access policy cuts off. Requests are now paced, and shards SEC dates entirely before --since are skipped rather than fetched and discarded.
• _filing_entity was annotated RecordEntity | None but never returns None, leaving its caller dereferencing an Optional.

Verification

• ruff check and ruff format --check clean on openfoia/ and tests/.
• 318 passed, 20 skipped. Every fix above has a regression test.
• tests/test_security_metadata.py has 2 failures in the dev container (pyo3/_cffi_backend), reproduced on unmodified main — environmental, and green in GitHub CI on all five source PRs.
• mypy gains 2 errors, both the untyped-decorator/no-untyped-def pattern the other 48 CLI commands already have. Not a CI gate (continue-on-error).

Not addressed

records search sends the query to a third party with no warning and no confirmation — no --tor, no prompt. #77's new records filings does warn and confirm; #74 documents the gap in the README rather than closing it. That is a Principle 1 gap, but closing it changes CLI behavior for existing scripts, so it is left as a separate decision.

🤖 Generated with Claude Code

https://claude.ai/code/session_016LaGpHLuSRHzGARb38X7YS

---

Generated by Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.