stats.lua when watching 3D. Incorrectly formatted info stats with gpu.next vs gpu #12462

[netExtra]

mpv Information

Other Information

- Windows version:
- GPU model, driver and version:
- Source of mpv:
- Latest known working version:
- Issue started after the following happened:

Reproduction Steps

Thanks for trying to fix this. It works fine for Over/Under. But looking at your stereo-in command you only have options for half sbs, not full sbs. So I'm guessing there isn't any code in mpv for full sbs which results in the problem below.

Expected Behavior

See screenshots

Actual Behavior

See screenshots

Log File

Sample Files

No response

I carefully read all instruction and confirm that I did the following:

• I tested and confirmed that the issue exists with the latest release version or newer.
• I provided all required information including system and mpv version.
• I produced the log file with the exact same set of files, parameters, and conditions used in "Reproduction Steps", with the addition of --log-file=output.txt.
• I produced the log file while the behaviors described in "Actual Behavior" were actively observed.
• I attached the full, untruncated log file.
• I attached the backtrace in the case of a crash.

[kasper93]

Could you provide sample, full mpv command and screenshot of gpu vs gpu-next?

[netExtra]

Could you provide sample, full mpv command and screenshot of gpu vs gpu-next?

I ran both with the following mpv.conf and vo=gpu-next then vo=gpu
hwdec=d3d11va-copy
d3d11va-zero-copy

The results are identical between gpu and gpu-next. BUT what I found is that if I start mpv in windowed mode, play the movie then bring up the info page, everything is in scale. (See screenshot). When I watch in 3D mode, the stats are unified perfectly with no issue until I enter fullscreen 3840x2160.

So I think the issue is that FSBS videos are 3840x1080 but 4k PC resolution is 3840x2160

[kasper93]

OSD is rendered relative to the window size, not video size. So you need to make sure your window is not cropped, else osd will be cropped too.

[netExtra]

OSD is rendered relative to the window size.

First thanks for dealing with this issue quickly. Second,so what can I do? Just not run it fullscreen if I want to see full stats?

[netExtra]

OSD is rendered relative to the window size, not video size. So you need to make sure your window is not cropped, else osd will be cropped too.

Apologies but I don't understand. Window is running fullscreen at 3840x2160. Video resolution is 3840x1080. Nothing is cropped. If anything it's the opposite with the smaller resolution video trying to fit into the larger uncropped window.

If there is something I am doing wrong then let me know so we can close this down. But I am happy with this being mostly sorted so if it has to be this way then cool. I would just like to properly understand. Thanks again.

[kasper93]

OSD is rendered relative to window, while we expect the full window to be displayed . Now in 3D, depending on mode it seems to not be the case. Also I tried to figure out how add compensation for OSD position/size/aspect ratio. But I don't know how to detect if it's half sbs or full sbs. Matroska spec says that

The pixel count of the track
(PixelWidth/PixelHeight) is the raw number of pixels (for example,
3840x1080 for full HD side by side), and the DisplayWidth/
DisplayHeight in pixels is the number of pixels for one plane
(1920x1080 for that full HD stream).

But only 1 sample I have actually sets that correctly, others just set DisplayWidth/DisplayHeight to same values as pixel. And looking at your screenshots your samples also doesn't set it, because they would be stretched.

When I said about cropping, I meant OSD being cropped. Anyway, maybe you could help me understand how you set up the playback.

Do you select manually the 3d mode to be half sbs and full sbs? In full sbs mode, full screen resolution is always cropped per eye? See in full screen we send 3840x2160, but in your screenshots it's merged to 1920x1080 by merging eyes and cropping. If that's the case, I presume that running mpv in windowed mode, without black bars, works as expected?

However, in half sbs mode, which is stretched in coded frame, I would guess OSD is after displaying on 3d device, squashed, because we render it as square pixels.

It's not hard to fix, but I feel like there is information missing here for us to always be correct.

[netExtra]

OSD is rendered relative to window, while we expect the full window to be displayed . Now in 3D, depending on mode it seems to not be the case. Also I tried to figure out how add compensation for OSD position/size/aspect ratio. But I don't know how to detect if it's half sbs or full sbs. Matroska spec says that

The pixel count of the track
(PixelWidth/PixelHeight) is the raw number of pixels (for example,
3840x1080 for full HD side by side), and the DisplayWidth/
DisplayHeight in pixels is the number of pixels for one plane
(1920x1080 for that full HD stream).

But only 1 sample I have actually sets that correctly, others just set DisplayWidth/DisplayHeight to same values as pixel. And looking at your screenshots your samples also doesn't set it, because they would be stretched.

When I said about cropping, I meant OSD being cropped. Anyway, maybe you could help me understand how you set up the playback.

Do you select manually the 3d mode to be half sbs and full sbs? In full sbs mode, full screen resolution is always cropped per eye? See in full screen we send 3840x2160, but in your screenshots it's merged to 1920x1080 by merging eyes and cropping. If that's the case, I presume that running mpv in windowed mode, without black bars, works as expected?

However, in half sbs mode, which is stretched in coded frame, I would guess OSD is after displaying on 3d device, squashed, because we render it as square pixels.

It's not hard to fix, but I feel like there is information missing here for us to always be correct.

The files are encoded at 3840x1080 with an aspect ratio is 32:9 not standard 16:9. If I open an mpv window with no geometry or fullscreen and play the file, it stretches across the screen as shown in the two examples below.

Full SBS Scope

Full SBS IMAX

The window is streamed by the Virtual Desktop app to my VR headset which has software to play back full or half SBS. Does this help?

[kasper93]

The window is streamed by the Virtual Desktop app to my VR headset which has software to play back full or half SBS. Does this help?

How do the app knows if it is full or half sbs? Do you select it manually?

[netExtra]

The window is streamed by the Virtual Desktop app to my VR headset which has software to play back full or half SBS. Does this help?

How do the app knows if it is full or half sbs? Do you select it manually?

Manually. The app is running in the VR headset not the PC so it's up to me to choose full or half. On the PC is just a streaming agent that streams the desktop back to the VR headset.

[danielcamposramos]

@kasper93 wrote in March:

But I don't know how to detect if it's half sbs or full sbs.

It is detectable, and not from the stereo metadata, which is the part that surprised me. I encoded four clips, all with the H.264 frame packing SEI (-x264-params frame-packing=3 for SBS, =4 for TAB):

file	coded size	SAR	DAR	SEI present	per-eye DAR
full_sbs.mp4	3840x1080	1:1	32:9	yes	1.778	full
half_sbs.mp4	1920x1080	1:1	16:9	yes	0.889	half
full_tab.mp4	1920x2160	1:1	8:9	yes	1.778	full
half_tab.mp4	1920x1080	1:1	16:9	yes	3.556	half

All four carry identical side data. The SEI tells you the arrangement, it never tells you full or half. The frame geometry does:
SBS: per_eye_dar = (width / 2) * SAR / height
TAB: per_eye_dar = width * SAR / (height / 2)
0.889 for half SBS, 3.556 for half TAB. mpv already has every value in that formula, so the OSD compensation does not need to ask the user.

Build the four clips and check for yourself:

mk() { # $1=out $2=eyeW $3=eyeH $4=stack $5=fp
  ffmpeg -v error -y -f lavfi -i "testsrc2=size=$2x$3:rate=24:duration=2" \
         -f lavfi -i "smptebars=size=$2x$3:rate=24:duration=2" \
         -filter_complex "[0:v][1:v]$4=inputs=2[v]" -map "[v]" \
         -c:v libx264 -crf 24 -pix_fmt yuv420p -x264-params frame-packing=$5 "$1"
}
mk full_sbs.mp4 1920 1080 hstack 3
mk half_sbs.mp4  960 1080 hstack 3
mk full_tab.mp4 1920 1080 vstack 4
mk half_tab.mp4 1920  540 vstack 4

for f in full_sbs half_sbs full_tab half_tab; do
  printf "%-9s " "$f"
  ffprobe -v error -select_streams v:0 \
    -show_entries stream=width,height,sample_aspect_ratio,display_aspect_ratio \
    -of csv=p=0 "$f.mp4"
done

ffmpeg 9.0.1, libavcodec 63.1.101.

This assumes the container DAR describes how one eye should land on screen. 
Files that lie about their aspect exist, and no metadata rule survives wrong metadata.

[kasper93]

Your four clips are MP4 with an H.264 SEI. My comment in March was about Matroska, and so are the files in this issue. Those are two different situations, and neither gives you what you claim.

MP4 has no per-eye display size anywhere. pasp and the track header describe the whole packed frame, st3d is a mode byte, and the H.264 frame packing SEI carries no aspect information at all. So "frame geometry" there is only the aspect ratio of the packed frame, and turning that into full or half is the assumption that one eye is 16:9. Half SBS of a 2.39:1 film gives 1.19, full SBS of 4:3 material gives 1.33, and no threshold separates them, because the source aspect ratio is exactly what is not known. That is a guess, not detection. HEVC even added upsampled_aspect_ratio_flag to its frame packing SEI to say whether the SAR applies to the packed frame or to the upsampled constituent frames, so the people who wrote the SEI knew this is ambiguous. H.264 has no such flag.

Matroska is the case where the information actually exists. RFC 9559 section 18.10 says

The pixel count of the track (PixelWidth/PixelHeight) is the raw number of pixels (for example, 3840x1080 for full HD side by side), and the DisplayWidth/DisplayHeight in pixels is the number of pixels for one plane (1920x1080 for that full HD stream).

With that, full or half needs no assumption about the content. Each eye is stretched by DisplayWidth / (PixelWidth / 2), which is 1 for the 3840x1080 example and 2 for a 1920x1080 half SBS file. The problem is that files do not follow it. The mkvmerge tagged files I have, including the one in #18348, put the whole frame size into DisplayWidth. libavformat follows the RFC on both ends, matroskaenc writes PixelWidth / 2 and matroskadec multiplies it back in mkv_stereo_mode_display_mul(). Nothing in the file says which convention the muxer used, and that is the information that is missing. You can watch the two readings collide inside mpv. Same 3840x1080 side by side track with DisplayWidth 1920

--demuxer=mkv    par=0.5   whole frame reading
--demuxer=lavf   par=1     RFC reading

and with DisplayWidth 3840

--demuxer=mkv    par=1
--demuxer=lavf   par=2     7680x1080 display size

The SEI does not help, as you say, it has no full or half distinction in either codec. What the OSD compensation needs is how much the display stretches each eye, and that comes down to the same unknown. So this stays a manual choice for now, the same way it is selected in the VR player from this issue. If someone wants to work on it, the question is not a formula, it is what demux_mkv should do with files that follow the RFC, because right now it applies the whole frame reading to them and shows them squeezed.

Note, that this is not the only ambiguity with Matroska, things like cropping metadata is even more "fun".

[danielcamposramos]

Thanks for taking the time to write this out.
A follow-up with the norm references, because precision matters here.

On full vs half.
Agreed, and this is a convergence, not a countersuit: MP4 carries no per-eye geometry (pasp and the track header describe the whole packed frame, st3d is a mode byte, and the H.264 packing SEI has no aspect field; HEVC only gained upsampled_aspect_ratio_flag in Annex D.2.7), and nothing in #18490 derives full or half from container aspect.
It maps only the layout type and eye order, for OSD and subtitle placement and video-params.
The geometry notes on my clips post were there to document how the fixtures were generated, not a detection method.

On "Matroska is where the information actually exists".
In the schema, yes; in the files, no.
You wrote yourself that mkvmerge-tagged files put the whole frame into DisplayWidth, which is exactly the population this patch addresses: stock-muxed rips.
For those files RFC 9559 §18.10 is unusable, nothing says which convention the muxer followed, and the only machine-readable layout signal present is the SEI.
Reading the signal the files actually carry is the entire point of the patch.

One more Matroska fact worth registering, fresh from upstream MKVToolNix this week (mbunkus, reviewing !6311 and !6312): in Matroska semantics, element absent and element at default are the same statement.
StereoMode's default is 0 (mono), so "unknown at container level" is simply unexpressible — an explicit mono set at mux time does not survive a remux, in stock v101 and v102 alike.
His merged AVC change and my HEVC one (in review) therefore probe the bitstream SEI when the container says nothing, and his planned reader adjustment ("if the source StereoMode is semantically 0, still use the bitstream") covers both codecs.
The upstream direction is now: when the container is silent or default, trust the stream.
#18490 does the same on the display side.

On carriage frequency, since it bounds what the SEI can promise.
The frame packing arrangement SEI (H.264 Annex D, payloadType 45; same semantics in HEVC Annex D.2.7) persists until cancelled or replaced, and encoders for file content repeat it per IDR — my x264 fixtures at GOP 48 carry it on frames 1, 49, 97, 145, 193 and nowhere else.
Broadcast application standards went the other way: ETSI TS 101 547-2 (DVB SFC-3DTV) requires the FPA SEI with every coded frame, and defines transition assistance where frame_packing_arrangement_cancel_flag = 1 is sent for at least two seconds around a 2D/3D switch; MPEG-2's Amendment 4 (Annex L) recommends per-picture presence; 3GPP TS 26.346 §10.5.2, on the storage/mobile side, requires it only at random access points.
Mid-stream 3D/2D/3D switching exists in the spec for that broadcast service model.
In file content nobody ships switching streams: the arrangement is constant per asset, because displays resync visibly on a layout change.
I could not find a produced file that switches arrangement mid-title, and I would genuinely welcome a counter-example if one exists.

That is also why #18490 v1 never injected anything into any frame.
It read the SEI when lavc surfaced it (IDR cadence) and held the last seen value for display purposes in between.
On your review the hold leaves mpv entirely: v2 reads the side data exactly when it arrives and lets the container tag stand otherwise, and the persistence question, where it is genuinely needed, is an ff_h2645_sei_to_frame() change to be brought to FFmpeg separately, following the film-grain pattern.

Related open issues, surveyed 2026-09-21, to save anyone the tracker dig:
#18489 (the stream-signalled layout issue — closed by PR #18490), #18380 (MVC Blu-ray frame-packed playback), #18348 (full vs half SBS/TAB selection), #17632 (this one, stats and OSD on 3D), #15225 (format:stereo-in ignored when encoding), #9046 and #7155 (subtitle rendering in 3D), #1945 (HDMI frame packing output) and #2583 (frame-sequential output), the latter two from 2015.
The closed ancestors #12462, #13347, #17547, #4811 and #1513 show the topic has a decade of history in this tracker.
No open issue duplicates what this thread asks; the closest unsolved one is this thread itself.