Skip to content

HDR output for a profile that names its tone mapping pass (Breath of the Wild) - #970

Open
zxcheergo wants to merge 1 commit into
seedds:mainfrom
zxcheergo:botw-hdr
Open

zxcheergo wants to merge 1 commit into
seedds:mainfrom
zxcheergo:botw-hdr

Conversation

@zxcheergo

Copy link
Copy Markdown
Contributor

What and why

On a screen with HDR (an iPhone, a Mac with an XDR display), a Switch game is still shown as its SDR picture: the game tone maps its HDR scene into 0..1 before it presents. This adds opt-in HDR output for a profile that names its game's tone mapping pass, with Breath of the Wild as the first: its brightest parts (the sun, the sky round it, sunlit grass, fire, reflections) go above SDR white up to the screen's current headroom, while everything darker, the HUD, the menus, the map and the title stay the game's own picture.

  • Profile key hdr <pixel shader key> <MSL file> (dm/games/game.py, DM_GAME_HDR*): the host compiles the profile's MSL in place of that translated pixel shader and draws it with the translated shader's reflection (nvn_mtl.m hdr_lib). The replacement writes the game's own result unchanged and, into a host side texture (fragment texture 0 of the draw table), how much brighter each pixel gets tone mapped up to the peak the host passes at fragment buffer 30, plus the SDR luminance it wrote.
  • Host (nvn_mtl.m, HDR output): present snapshots carry the highlights. The copy into a ring slot becomes a draw (dm/gfx/msl/hdr_merge.metal) that keeps the game's 8-bit picture and writes, into the slot's RG11B10Float highlights texture, the side texture's excess wherever the presented picture still has the luminance the tone mapping wrote (so the HUD, fades and later effects drawn over the 3D picture take none). The present scales the 8-bit picture as before (bilinear, or MetalFX in its perceptual mode), decodes it to linear and adds the highlights (dm/gfx/msl/present.metal, snap_excess), into an extended-range layer. The peak follows the headroom the front end reports four times a second, eased over about half a second; frames without the tone mapping pass and a peak of 1 take the plain copy.
  • Front ends: an RGBA16Float, extended linear sRGB layer with wantsExtendedDynamicRangeContent while HDR output is on; the iOS app turns it off on a screen without EDR headroom (potentialEDRHeadroom 1), and reads currentEDRHeadroom (the Mac: maximumExtendedDynamicRangeColorComponentValue).
  • Breath of the Wild (hdr_tonemap.metal): its tone mapping pass (uking_pass_shader, f7ee6fe10b925543) written out, with the curve's top raised: the game's curve up to 0.6, then an exponential shoulder with the game's slope there that rises toward the peak instead of 1, so the picture below 0.6 and its colours stay the game's.
  • Switches: launcher setting HDR (DM_HDR, on where the profile offers it); DM_HDR_PEAK, DM_HDR_MAX (8), DM_HDR_KNEE (0.6), DM_HDR_GAIN; DM_HDR_SHOT_RAW writes a screenshot's extended-range picture as half floats; the Mac's DM_CTL takes hdrpeak <n> and hdrknee <v>. An explicit DM_PRESENT_SNAP=0 turns HDR output off, since the snapshots' copy carries the highlights.

The first version of this copied into half float slots and let MetalFX scale those: MetalFX then ran its HDR mode on half floats, 4.1 ms a present on an iPhone Duo (1915x1344 → 2853x2007) against about 2 ms in its perceptual mode on 8 bits, and heavier scenes dropped to 30 fps. Keeping the picture 8-bit through MetalFX and adding the highlights after it removed that.

The details and numbers are in the game note's new section (HDR output).

Checked

  • Tests: make -C dm/host/tests GAME=01007EF00011E000 run (all pass; t_hid gained stubs for the two DM_CTL setters); make -C dm/gfx/tests GAME=01007EF00011E000 run on an M1 (all pass; t_link_ticks sets DM_PRESENT_SNAP=0, which now turns HDR output off with it); dm/tools/tests test_docs, test_env_switches, test_launcher_settings, test_game_profile, test_settings, test_player_env; dm/tools/lint_python.py with .github/requirements/lint.txt (0 new). The C lint was not run here (no LLVM clang-tidy on this Mac): left to CI.
  • Games run, with title ID and version: The Legend of Zelda: Breath of the Wild 01007EF00011E000 1.9.0 with both DLC, built on this base (e5620b1a) with the NSP script (--platform mac), the modules rewritten with this branch's profile, headless, a save by an NPC in Kakariko Village at dusk, DM_HDR_PEAK=4, DM_PRESENT_UPSCALE=metalfx, screenshots of the extended-range picture through MetalFX at 2560x1440 (DM_HDR_SHOT_RAW, DM_NVN_SHOT_OUT): highlights on the campfire, torches and lanterns (0.28% of the field above SDR white, up to 3.6), none on the HUD, on the NPC's dialog box or on the inventory's panels (only the 3D Link behind them); at peak 1 nothing is added. No [RED] line from the change; the replacement compiled in 347 ms at load. Earlier, on an older base: a field at noon about 4% above SDR white up to 2.4 at peak 4, sunset 15%, the inventory 0.06%; peak 1 gives the game's picture exactly (largest value 1.0).
  • Devices and OS: Apple M1, macOS 27.0.1, Xcode 27.1 (this Mac's Metal compiler is 32023.921, the lock's 32023.922: the shaders were built with --allow-other-metal); iPhone Duo (A20 Pro), iOS 27.1, with the same change on an older base.
  • scripts/regress.py impact origin/main..HEAD: Breath of the Wild (its profile), Tears of the Kingdom and Pokémon Scarlet (the present and its host shaders). regress.py check was not run: this Mac has no harness root or the routes' saves, and building Tears of the Kingdom on this base would have taken another hour. For a profile without hdr, mtl_hdr_on is 0: the present shader takes two more constants that are 0 and a second texture it does not read, the snapshot copy is the blit as before, and the layer stays BGRA8.
  • Not checked: other games with an HDR pass (none has an hdr key); the profile's MSL is compiled at run time rather than with the host's own libraries (288 ms once on the iPhone Duo).

Docs

  • The finding is in its game note (dm/games/01007EF00011E000/README.md, HDR output); dm/games/README.md describes the hdr key; docs/settings.md the HDR setting; docs/env_switches.tsv the six switches; docs/agents/game-patches.md regenerated (hdr classified in docs_patches.py)
  • A chain tool changed (none)
  • No game files, keys or data extracted from a game in the change or the attachments

🤖 Generated with Claude Code

…the Wild)

On a screen with HDR, a Switch game is still its SDR picture: the game
tone maps its HDR scene into 0..1 before it presents. A profile can now
name its game's tone mapping pixel shader and an MSL file that replaces
it (game.tsv `hdr <key> <file>`, DM_GAME_HDR*). The replacement keeps
the translated shader's bindings and writes the game's result unchanged;
beside it, into a host side texture, it writes how much brighter each
pixel gets tone mapped up to the screen's headroom, and the SDR
luminance it wrote.

The present snapshots carry the highlights: the copy into a ring slot is
a draw (dm/gfx/msl/hdr_merge.metal) that keeps the game's 8-bit picture
and writes the excess, into the slot's RG11B10Float texture, wherever
the picture still has the luminance the tone mapping wrote, so the HUD,
fades and later effects drawn over the 3D picture take none. The present
scales the 8-bit picture as before (bilinear, or MetalFX in its
perceptual mode), decodes it and adds the highlights into an
extended-range layer (RGBA16Float, extended linear sRGB). The peak
follows the headroom the front end reports four times a second, eased;
frames without the tone mapping pass and a peak of 1 take the plain
copy. Copying into half float slots instead and letting MetalFX scale
those made it run its HDR mode: 4.1 ms a present on an iPhone Duo
(1915x1344 to 2853x2007) against about 2 ms on 8 bits, and heavier
scenes fell to 30 fps.

Breath of the Wild's pass (uking_pass_shader, f7ee6fe10b925543) is
written out in hdr_tonemap.metal with the curve's top raised: the game's
curve up to 0.6, then an exponential shoulder with the game's slope
there rising toward the peak instead of 1.

Switches: launcher setting HDR (DM_HDR, on where offered; the iOS app
turns it off on a screen without EDR headroom), DM_HDR_PEAK, DM_HDR_MAX,
DM_HDR_KNEE, DM_HDR_GAIN, DM_HDR_SHOT_RAW, and DM_CTL hdrpeak / hdrknee
on the Mac. An explicit DM_PRESENT_SNAP=0 turns HDR output off, since
the snapshots' copy carries the highlights (t_link_ticks sets it).

Checked on an M1 Mac (macOS 27.0.1), the game built on this base with
the NSP script (--allow-other-metal), headless, peak 4, screenshots of
the extended-range picture through MetalFX at 2560x1440: the highlights
on fire, torches and lanterns in Kakariko Village at dusk, none on the
HUD, an NPC's dialog or the inventory's panels; at peak 1 no highlights.
The host and Metal graphics tests pass. The same change on an older
base ran on an iPhone Duo (iOS 27.1): headroom 8 at 50% brightness.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant