Skip to content

Let the Demuxer read straight into the packet it hands out - #1695

Open
NicolasHug wants to merge 1 commit into
blocks-planes-coveragefrom
demuxer-owned-packets
Open

Let the Demuxer read straight into the packet it hands out#1695
NicolasHug wants to merge 1 commit into
blocks-planes-coveragefrom
demuxer-owned-packets

Conversation

@NicolasHug

Copy link
Copy Markdown
Contributor

next_packet() has to return a packet the caller owns indefinitely: the
Python-level Packet can be held, moved to another thread, and decoded much
later. That is a UniqueAVPacket, and the AutoAVPacket / ReferenceAVPacket
pair is for the opposite thing - a loop that reuses one allocation and
unrefs at the end of each iteration.

So next_packet() was using the loop idiom and then undoing it: it read into
the shared packet, allocated a second one anyway, and moved the reference
across. Neither half of the idiom paid for itself - the allocation isn't
saved, and av_packet_move_ref() leaves the source blank so the scope-exit
unref has nothing to do.

read_next_packet() has no opinion on how the packet is owned, it just needs
somewhere to write, so it takes the AVPacket itself. Its callers keep the
ownership type that suits them: a ReferenceAVPacket for the two read loops
(SingleStreamDecoder and scan), a UniqueAVPacket for next_packet(). It also
unrefs between iterations, which av_read_frame() documents as the caller's
job rather than something it does for you.


Stack created with GitHub Stacks CLIGive Feedback 💬

@pytorch-bot

pytorch-bot Bot commented Sep 4, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1695

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 Cancelled Jobs, 1 Unclassified Failure

As of commit 2d404db with merge base 2312413 (image):

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

CANCELLED JOBS - The following jobs were cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 4, 2026
@NicolasHug
NicolasHug force-pushed the demuxer-owned-packets branch 2 times, most recently from 7ebd839 to 2b0ac1b Compare September 4, 2026 09:28
@NicolasHug
NicolasHug force-pushed the demuxer-owned-packets branch from 2b0ac1b to 665ea66 Compare September 4, 2026 10:02
next_packet() has to return a packet the caller owns indefinitely: the
Python-level Packet can be held, moved to another thread, and decoded much
later. That is a UniqueAVPacket, and the AutoAVPacket / ReferenceAVPacket
pair is for the opposite thing - a loop that reuses one allocation and
unrefs at the end of each iteration.

So next_packet() was using the loop idiom and then undoing it: it read into
the shared packet, allocated a second one anyway, and moved the reference
across. Neither half of the idiom paid for itself - the allocation isn't
saved, and av_packet_move_ref() leaves the source blank so the scope-exit
unref has nothing to do.

read_next_packet() has no opinion on how the packet is owned, it just needs
somewhere to write, so it takes the AVPacket itself. Its callers keep the
ownership type that suits them: a ReferenceAVPacket for the two read loops
(SingleStreamDecoder and scan), a UniqueAVPacket for next_packet(). It also
unrefs between iterations, which av_read_frame() documents as the caller's
job rather than something it does for you.
@NicolasHug
NicolasHug force-pushed the demuxer-owned-packets branch from 665ea66 to 2d404db Compare September 4, 2026 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant