Repository navigation
Parse a file again, alone, after it first crashes a worker - #142
Merged
Merged
Conversation
A parse worker that crashed on a file made the extract skip it at once. On master's CI, a Windows worker crashed once on vlib/v/types/checker_parallel.v, which parsed on every other run, and same-graph failed for the 306 symbols left out. Now the first crash requeues the file in a batch of its own; only a second crash skips it, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The extract parses files in worker processes (
graphify _parse-batch), in batches of 200, so that a crash in the parser costs one file, not the extract. When a worker died, the extract skipped the file it was on at once: the file was reported as unparseable, its last extract was carried forward if there was one, and the rest of the batch was requeued.A crash that doesn't happen again therefore cost the file too. In master's CI run 40875434161 (after #141), the Windows worker crashed on
vlib/v/types/checker_parallel.v:The Windows graph had 306 fewer symbols than the macOS and Linux graphs, and
same-graphfailed. The PR's two runs, with the same graphify and the same V commit (b5a2c25), parsed the file on Windows, and so did a re-run of the master run. The crash couldn't be reproduced on macOS, with CRLF line ends or with a 512 KB stack.Change
ParseRun.parsekeeps the files that have crashed a worker (ParseRun.crashed). The first time a file does, it is requeued in a batch of its own, ahead of the queue, and the rest of the batch is requeued as before.A file crashes a worker at most twice, so the extract still ends. One that crashes every time now costs one more worker start.
Tests
check_crash_onceintest_a_worker_crash_is_recovered_from_on_both_paths(graphify_test.v): the stand-in worker crashes oncrash.vonce. The file is neither failed nor stale, and its symbols and those of the file queued behind it are in the graph. Without the change it fails:crash.vis infailed.v test .andv run bin/test-crlf.vshpass. An extract of the V tree with a-prodbuild skips no files.🤖 Generated with Claude Code