Let the Demuxer read straight into the packet it hands out - #1695
Open
NicolasHug wants to merge 1 commit into
Open
Let the Demuxer read straight into the packet it hands out#1695NicolasHug wants to merge 1 commit into
NicolasHug wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1695
Note: Links to docs will display an error until the docs builds have been completed. ❌ 6 Cancelled Jobs, 1 Unclassified FailureAs of commit 2d404db with merge base 2312413 ( UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:
CANCELLED JOBS - The following jobs were cancelled. Please retry:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
NicolasHug
force-pushed
the
demuxer-owned-packets
branch
2 times, most recently
from
September 4, 2026 09:28
7ebd839 to
2b0ac1b
Compare
NicolasHug
force-pushed
the
demuxer-owned-packets
branch
from
September 4, 2026 10:02
2b0ac1b to
665ea66
Compare
next_packet() has to return a packet the caller owns indefinitely: the Python-level Packet can be held, moved to another thread, and decoded much later. That is a UniqueAVPacket, and the AutoAVPacket / ReferenceAVPacket pair is for the opposite thing - a loop that reuses one allocation and unrefs at the end of each iteration. So next_packet() was using the loop idiom and then undoing it: it read into the shared packet, allocated a second one anyway, and moved the reference across. Neither half of the idiom paid for itself - the allocation isn't saved, and av_packet_move_ref() leaves the source blank so the scope-exit unref has nothing to do. read_next_packet() has no opinion on how the packet is owned, it just needs somewhere to write, so it takes the AVPacket itself. Its callers keep the ownership type that suits them: a ReferenceAVPacket for the two read loops (SingleStreamDecoder and scan), a UniqueAVPacket for next_packet(). It also unrefs between iterations, which av_read_frame() documents as the caller's job rather than something it does for you.
NicolasHug
force-pushed
the
demuxer-owned-packets
branch
from
September 4, 2026 10:18
665ea66 to
2d404db
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
next_packet() has to return a packet the caller owns indefinitely: the
Python-level Packet can be held, moved to another thread, and decoded much
later. That is a UniqueAVPacket, and the AutoAVPacket / ReferenceAVPacket
pair is for the opposite thing - a loop that reuses one allocation and
unrefs at the end of each iteration.
So next_packet() was using the loop idiom and then undoing it: it read into
the shared packet, allocated a second one anyway, and moved the reference
across. Neither half of the idiom paid for itself - the allocation isn't
saved, and av_packet_move_ref() leaves the source blank so the scope-exit
unref has nothing to do.
read_next_packet() has no opinion on how the packet is owned, it just needs
somewhere to write, so it takes the AVPacket itself. Its callers keep the
ownership type that suits them: a ReferenceAVPacket for the two read loops
(SingleStreamDecoder and scan), a UniqueAVPacket for next_packet(). It also
unrefs between iterations, which av_read_frame() documents as the caller's
job rather than something it does for you.
Stack created with GitHub Stacks CLI • Give Feedback 💬