Skip to content

Fix MemoryError during HTML report generation (port of iLEAPP #1746) - #56

Merged
abrignoni merged 1 commit into
mainfrom
fix/report-sidebar-memoryerror
Aug 7, 2026
Merged

Fix MemoryError during HTML report generation (port of iLEAPP #1746)#56
abrignoni merged 1 commit into
mainfrom
fix/report-sidebar-memoryerror

Conversation

@abrignoni

Copy link
Copy Markdown
Owner

What this does

Ports the iLEAPP MemoryError fix (abrignoni/iLEAPP#1746, fixed by JSup in iLEAPP commit 0a4aee10) to DLEAPP. Report generation used to read each artifact's entire .temphtml page into memory and rebuild it with string concatenation to inject the navigation sidebar. On large extractions an artifact page can reach several GB, so that peaked at roughly three times the page size and raised MemoryError.

The new stream_insert_sidebar_code() copies each page to its final location in fixed size chunks. The sidebar placeholder sits near the top of every page, so only a small head buffer is held while searching for it (including a placeholder split across a chunk boundary). The large table body that follows is streamed with shutil.copyfileobj. Output is byte identical to the old approach.

Verification

  • Ran the iLEAPP regression checks locally against this port: byte identical output across page shapes (placeholder at start, end, split across a chunk boundary, missing, empty file), and peak memory bounded under half the file size on a 16 MB+ synthetic page. Not committed as a test file because iLEAPP removed its copy from main after the fix merged; the port matches iLEAPP main as it stands.
  • pylint --disable=C,R on scripts/report.py: 10.00/10, zero warnings. admin/scripts/lint_changed.py against the merge base: PASS, no new warnings.
  • End to end: full 88 module run against the registered discord_macos corpus sample (zip input, hash verified against the registry before the run, extraction done on a temporary copy). Report generated clean: 17 HTML pages, sidebar navigation present in every one, zero leftover placeholders, zero .temphtml remnants, no parser errors.

Data proven on the corpus run above for the normal path; the missing placeholder fallback is exercised by the local regression checks only.

🤖 Generated with Claude Code

Report generation loaded each artifact's entire .temphtml page into memory
and rebuilt it via string concatenation to inject the navigation sidebar.
For large extractions an artifact page can reach several GB, so this peaked
at roughly 3x the page size and raised MemoryError.

Replace the whole-file read/concat with stream_insert_sidebar_code(), which
copies the page to its final location in fixed-size chunks. The sidebar
placeholder is near the top of every page, so only a small head buffer is
retained while searching for it (handling a placeholder split across a chunk
boundary); the large table body that follows is streamed with
shutil.copyfileobj. Output is byte-identical to the previous approach.

Ported from iLEAPP commit 0a4aee10 (issue #1746, authored by JSup).

Co-Authored-By: JSup <hanjisang0914@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@abrignoni abrignoni added the Needs Cross Core Leveling for a PR that needs to be applied across other cores. remove once leveled to all. label Aug 7, 2026
@abrignoni
abrignoni merged commit 82d76e5 into main Aug 7, 2026
4 checks passed
@abrignoni
abrignoni deleted the fix/report-sidebar-memoryerror branch August 7, 2026 05:37
@abrignoni abrignoni removed the Needs Cross Core Leveling for a PR that needs to be applied across other cores. remove once leveled to all. label Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant