HDR output for a profile that names its tone mapping pass (Breath of the Wild)

dm/games/01007EF00011E000/README.md

@@ -70,6 +70,14 @@ Used at `DM_FPS_TARGET=0` and for the interval the framework keeps while the VFR
 - **Shrines at 20–27 fps (GitHub issues #534 and #598; current, the fix not yet seen in a shrine).** The reporters' shrines (Oman Au on a MacBook Air M2; every shrine on an M3) run at 20–27 fps where the open world holds 60. In the #598 log the slow stretches have about 1,000 draws a frame, as the fast ones, but a GPU time of about 36 ms against 9–13 ms. They have about 240 `passes ended to land them for a read` a frame, all `refused … other GPU writes`, and 280–330 `load-all store-all passes` against about 100. The probe on the Shrine of Resurrection save found the one write behind that refusal there: the GPU particle dispatch `9f8116baffddaf81` binds its three outputs with size 0, which counted as a write anywhere ([A storage buffer bound with size 0 counted as a write anywhere](../../../docs/reference/rendering.md#a-storage-buffer-bound-with-size-0-counted-as-a-write-anywhere)). With the fix, that save's world-load burst (1,200 refusals) is gone. Opening Oman Au's final gate ends the slowdown even out of view (#534): an emitter that stops with the gate would fit, but no shrine save was here to show it.
 - **Present snapshots (Mac).** The game's two swapchain textures held its CPU work to display ticks and the GPU at 444 MHz on an M2 Max ([present snapshots](../../../docs/reference/frame-rate.md#present-snapshots-the-games-swapchain-texture-free-at-once)). Judged by cost, the frame step stepped to 30 at a GPU time of 0.75 vsyncs. Zora's Domain from a Cemu save (120 Hz), frames on screen longer than 1/60 s by a quarter: 64 of 8,684 with neither, 41 with the GPU filler alone, 4 of 10,800 with snapshots alone (no step to 30 in 3 minutes, at 444 MHz throughout).
 
+### HDR output (`DM_HDR`, current)
+
+The game renders its scene in RG11B10 float and tone maps it in one pass (`uking_pass_shader`, pixel shader `f7ee6fe10b925543`): x = min(scene, 10000) + glow, a mix of a luminance-preserving and a per-channel 1 − e^−x weighted by the luminance's own curve squared, then a saturation grade; the result goes into an RGB10A2 target, and the AA / upscale pass, the HUD and the copy into the RGBA8 swapchain's sRGB view all work on those linear values. With the profile's `hdr` key, `hdr_tonemap.metal` replaces that shader: it writes the game's result unchanged and, into the host's side texture, how much more the same pass gives with the curve's top raised to the screen's headroom. The raised curve is the game's up to 0.6 (`DM_HDR_KNEE`), then an exponential shoulder with the game's slope there that rises toward the peak instead of 1, so the picture below 0.6 and the colours stay the game's. The host adds the excess in the present where the picture still has the luminance the pass wrote ([`nvn_mtl.m`, HDR output](../../gfx/nvn/nvn_mtl.m)), so the HUD, menus, map and title, which have no tone mapping pass under them, stay the game's own.
+
+- **Measured on the Mac** (M1, macOS 27, headless, peak fixed at 4, screenshots as half floats with `DM_HDR_SHOT_RAW`): at peak 1 the picture's largest value is 1.0, the game's; at noon in a field about 4% of the pixels go above SDR white, the brightest sunlit grass and the clouds by the sun up to 2.4; at sunset 15%, the sky around the sun; the inventory 0.06% (the 3D Link behind it). A knee of 0.5 had lifted 13% of the field above white at noon. Through MetalFX at 2853x2007, the highlights stay on the sun, the sunlit flowers and Link's hair and none falls on the HUD.
+- **On an iPhone Duo** (A20 Pro, iOS 27.1, MetalFX on): the headroom rose from 1.2 at launch to 8 once the layer showed extended values (screen brightness 50%); the shader compiled in 288 ms at load. Copying the picture into half float slots and scaling those made MetalFX run its HDR mode on half floats: 4.1 ms a present for 1915x1344 → 2853x2007, against about 2 ms in its perceptual mode on 8 bits, and heavier scenes dropped to 30 fps; the picture now stays 8-bit through MetalFX and the highlights are added after it. After about 8 minutes the phone reached thermal state serious (the frame step's 30 fps cap) at full screen brightness, by when iOS had lowered the headroom to 1.
+- **Not done:** the profile's MSL is compiled at run time, not with the host's own libraries; a translucent HUD element over a highlight whose luminance stays within the tolerance can take some of its excess.
+
 ## Rendering
 
 - **Resolved: the 3D scene black, only particles and the HUD drawn.** The engine binds almost every texture with `nvnCommandBufferBindTexturesDeferred` (`main+0xec2e64`, one handle, the pointer being the handle field inside the texture-sampler object): the GPU reads the handle when the command executes, and the game records its command blocks once and writes the handles afterwards. The host copied them at recording, mostly zeros, so the deferred-lighting composite read the dummy texture 0. The bind record keeps the source address and replay refreshes the handles first (`bh_fresh`). Found with a data-flow table from `DM_NVN_PASSDUMP_ALL`: the composite's `B` lines were all `tid=0 dummy` (without `_ALL` only the first draw of each pass is listed).

dm/games/01007EF00011E000/game.tsv

@@ -18,6 +18,11 @@ app_category	public.app-category.adventure-games
 pad_layout	full
 
 # ---- Graphics ----
+# HDR output (nvn_mtl.m, HDR output; DM_HDR): the scene is RG11B10 float up to the tone mapping pass (uking_pass_shader, pixel shader f7ee6fe10b925543), which writes
+# 1 - exp(-x) into an RGB10A2 target; the AA / upscale pass, the HUD and the copy into the RGBA8 swapchain's sRGB view all work on those linear values. hdr_tonemap.metal is
+# that pass again: it writes the game's result unchanged and, beside it, how much brighter each pixel gets with the curve's top raised from 1 to the screen's headroom (the
+# game's curve up to 0.6), which the host adds in the present where the 3D picture still shows, so the HUD and the menus stay the game's own
+hdr	f7ee6fe10b925543	hdr_tonemap.metal
 # The frame-interval controller (60 fps cuts in patches.py) and dynamic resolution read the GPU time between the frame's two timestamp reports: each report takes its segment's GPU start / end
 gpu_timestamps	segment
 # 60 fps (30 on the Switch): the host decides each frame's step in whole vsyncs and holds the frame that long (rewriter stubs at setWaitVsyncInterval and VFR::updateInterval,

dm/games/01007EF00011E000/hdr_tonemap.metal

@@ -0,0 +1,78 @@
+// Breath of the Wild's tone mapping pass for HDR output (game.tsv hdr; nvn_mtl.m, HDR output): the host compiles this in place of the translated pixel shader
+// f7ee6fe10b925543 (uking_pass_shader) and draws it with that shader's reflection, so the argument buffer below keeps the translation's bindings in their order.
+// The game's pass, written out:
+//   x = min(scene, 10000) + glow                                     (both RG11B10 float)
+//   l = dot(x, (0.2989, 0.587, 0.1145)), v(x) = 1 - exp(-x)          (the luminance weights after the first come from the shader's constants, c1)
+//   t = saturate(mix(x * v(l) / l, v(x), v(l)^2))                    (dark colours tone mapped by luminance, keeping their hue; bright ones per channel, going white)
+//   saturation grade: s = (r + g + b) * 2/3, g = c3[0].y * (1 - (s - 1)^2) + c3[0].x, out = m + (t - m) * g with m the largest channel
+// written into an RGB10A2 target (which clamps it to [0, 1]); the AA pass, the HUD and the copy into the swapchain follow. This writes that result unchanged, and the same pass
+// tone mapped up to the screen's headroom into the host's side texture, as what it adds to the SDR result (rgb) and the SDR result's luminance (a): nvn_mtl.m keeps it wherever
+// the presented picture still has that luminance (hdr_merge) and adds it in the present. The HDR curve is the game's up to the knee (v = knee), then an exponential shoulder with the game's
+// slope there (1 - knee) that rises toward the peak instead of 1 (with a peak of 1 it is the game's curve: nothing is added); the blend weight and the saturation grade are the
+// game's own, so colours change only in how bright the highlights get.
+#include <metal_stdlib>
+using namespace metal;
+
+struct Cb { float4 v[4096]; };
+struct Border { uint mask; float4 k[32]; };   // the translation's border colours: texture i sampled with its border where mask bit i is set (the chain's dm_border_t)
+struct Args {
+    constant Cb *c1 [[id(0)]];   // the shader's constants: v[0] = (0.587, 0.1145, 2/3)
+    constant Cb *c3 [[id(1)]];   // uniform block 0: v[0].x / .y = the saturation grade's bias and scale (0.99, 0.18 in the field)
+    texture2d<float> scene [[id(2)]];
+    sampler scene_s [[id(3)]];
+    texture2d<float> glow [[id(4)]];
+    sampler glow_s [[id(5)]];
+    sampler scene_w [[id(6)]];   // the border-colour samplers and their mix, as the translation samples (its dm_bs)
+    sampler glow_w [[id(7)]];
+    constant Border *border [[id(8)]];
+};
+struct In { float4 uv [[user(locn0)]]; };
+struct Out { float4 c [[color(0)]]; };
+// The host's HDR constants (nvn_mtl.m hdr_consts, fragment buffer 30): peak = the brightest value to tone map to, in SDR whites (the screen's EDR headroom; 1 = SDR); gain = the
+// excess's scale; knee = the SDR value where the curve leaves the game's (DM_HDR_KNEE)
+struct Hdr { float peak, gain, knee, pad; };
+
+constant float kLog2e = 1.44269502f;
+
+static float3 game_curve(float3 x) { return 1.0f - exp2(x * -kLog2e); }
+static float3 hdr_curve(float3 x, float peak, float knee) {
+    float xk = -log2(1.0f - knee) / kLog2e, slope = 1.0f - knee, room = peak - knee;   // where the game's curve reaches the knee, and its slope there
+    float3 up = knee + room * (1.0f - exp2((x - xk) * (-slope * kLog2e / room)));
+    return select(game_curve(x), up, x > xk);
+}
+static float3 tex(texture2d<float> t, sampler s, sampler w, constant Border *b, uint i, float2 uv) {
+    float3 r = t.sample(s, uv).xyz;
+    if (b->mask >> i & 1u) r += (t.sample(w, uv).xyz - r) * b->k[i].xyz;
+    return r;
+}
+static float3 grade(float3 t, float3 ref, float bias, float scale, float third2) {
+    float s = (ref.r + ref.g + ref.b) * third2 - 1.0f;
+    float g = scale * (1.0f - s * s) + bias;
+    float m = max(t.r, max(t.g, t.b));
+    return m + (t - m) * g;
+}
+
+fragment Out main0(In in [[stage_in]], float4 pos [[position]], constant Args &a [[buffer(0)]], constant Hdr &h [[buffer(30)]],
+                   texture2d<half, access::write> side [[texture(0)]]) {
+    float2 uv = select(in.uv.xy, float2(0.0f), isnan(in.uv.xy));
+    float3 x = min(tex(a.scene, a.scene_s, a.scene_w, a.border, 0, uv), 10000.0f) + tex(a.glow, a.glow_s, a.glow_w, a.border, 1, uv);
+    float4 k = a.c1->v[0];
+    float3 lw = float3(0.2989f, k.x, k.y);
+    float l = dot(x, lw);
+    float il = l > 0.0f ? 1.0f / l : 0.0f;
+    float vl = game_curve(float3(l)).x, w = vl * vl;
+    float3 sdr = saturate(mix(x * (vl * il), game_curve(x), w));
+    float bias = a.c3->v[0].x, scale = a.c3->v[0].y;
+    float3 o = grade(sdr, sdr, bias, scale, k.z), shown = saturate(o);   // what the game writes; what its target keeps
+    float3 more = 0.0f;
+    if (h.peak > 1.0001f) {
+        float knee = clamp(h.knee, 0.05f, 0.95f);
+        float3 hdr = clamp(mix(x * (hdr_curve(float3(l), h.peak, knee).x * il), hdr_curve(x, h.peak, knee), w), 0.0f, h.peak);
+        more = max(clamp(grade(hdr, sdr, bias, scale, k.z), 0.0f, h.peak) - shown, 0.0f) * max(h.gain, 0.0f);
+    }
+    uint2 p = uint2(pos.xy);
+    if (p.x < side.get_width() && p.y < side.get_height()) side.write(half4(half3(more), half(dot(shown, lw))), p);
+    Out out;
+    out.c = float4(o, 0.0f);
+    return out;
+}

dm/games/README.md

@@ -217,6 +217,7 @@ One row per profile; the notes are each profile's `README.md`.
 | `tex_dyn_slots` | `on` / `off` (the default): a shader whose buffer-texture handle the translator could not trace to one slot gets Ryujinx's fallback arrays (`<stage>_<tb|s>_cb0_40000000`: 32 buffer textures and samplers indexed by the handle's texture / sampler ID; any other sampled texture indexes the whole pool without a key, [Texture handles from game memory index the pools](../../docs/reference/shaders.md#texture-handles-from-game-memory-index-the-pools)). With the key the replay puts slot indices in that shader's c2 texture and sampler tables (texture slot r → r, sampler slot r → r << 20) and binds slot k at element k, so the shader's own arithmetic picks the binding (`NVN_RB_TEXD` / `NVN_RB_SAMPD` in `nvn_mtl.m`). Right only where the handle comes from c2's tables: Octopath Traveler 0's 61 such vertex shaders read separate texture slots 0-10 and samplers 0-4 there and nothing else (FINAL FANTASY RESONANCE's 2,077 the texture table words 0x168-0x1c0 and sampler table words 0x568-0x590). Off, such shaders do not load. Not part of the shader chain version |
 | `gpu_timestamps` | What a timestamp report (`nvnCommandBufferReportCounter`) records: `submission` (the default) = the time the whole submission completed, the same for every report in it; `segment` = the GPU start (`TIMESTAMP_TOP`) or end (the other types) of the segment the report was encoded in (the host commits a submission in segments of 64 draws, `sub_cut` in `nvn_mtl.m`), so two reports of one submission bracket the GPU time between them, as on hardware. Breath of the Wild needs `segment`: its frame-interval controller and its dynamic resolution read the GPU time between the frame's first and last report, and with one time for both they read 0 (the game kept 60 fps in slow motion when the GPU could not keep up). `DM_NVN_GPU_TS=segment` / `submission` overrides it, also from `Documents/env.txt` |
 | `vconv_range` | `on` / `off` (the default): vertex attributes the replay converts on the CPU (integer shader inputs fed float-type formats and the like, `vconv_bind` in `nvn_mtl.m`) are converted only over the vertices each draw reads (first / count, or the smallest and largest index of a direct indexed draw, kept while the index pages stay unwritten) instead of the whole bound range. Age of Calamity binds whole mesh pools: in a battle that was millions of vertices a frame. Indirect draws, per-instance streams and indices with a GPU write queued on them keep the whole range. `DM_NVN_VCONV_CHECK=1` scans the indices again at every reuse and reports differences; `DM_NVN_VCONV_RANGE=0` / `1` overrides it |
+| `hdr` | `<pixel shader key> <file>` (none by default): HDR output. The file, an MSL source in the profile folder, replaces the game's tone mapping pixel shader of that key and keeps its bindings (the host draws it with the translated shader's reflection): it writes the game's own result as before and, into the host's side texture (fragment texture 0, `RGBA16Float`), how much brighter each pixel gets tone mapped up to the peak the host passes at fragment buffer 30 (`hdr_consts`), and the SDR luminance it wrote. The host adds that excess in the present where the picture still has that luminance, into an extended-range layer, and turns present snapshots on (`nvn_mtl.m`, HDR output; `DM_HDR`). The file goes into the host as a string (`DM_GAME_HDR_MSL`) and is compiled when the game loads that shader. Breath of the Wild: `f7ee6fe10b925543 hdr_tonemap.metal` |
 | `shader_key_skip` | `<offset>+<length>` (none by default): control-section bytes every shader key hashes as zeros, in `shader_split.py` (`key_ctl`) and the host (`hle_nvn.c` `keys_hash`), for an engine that shares one control section among shaders that differ only there. Batman: Arkham Knight: `0x770+0x78`, GLSLC's compile information (+0x778..+0x791 and +0x7d0..+0x7e7); without it the game asks for shaders the map does not have, each the control section of one record with the code area of another. It must stay clear of `+0x700..+0x75f`, which the tools read; it is a shader chain input, as every `shader_*` key |
 | `shader_groups`, `shader_note` | `shader_split.py` source groups and the map header note (Engage: `engine` for `data.unity3d` and `Resources/`, `assets` for the Addressables bundles). Both are shader inputs, the note too: changing either one, even by a word, builds every shader of the game again. Keep them as they are when a provisional profile moves into `dm/games` |
 

dm/games/game.py

@@ -148,6 +148,14 @@ def header(g):
     if dmo not in ('gl', 'zero'): sys.exit(f'{g.id}: depth_mode must be gl or zero: {dmo}')
     api = g.get('device_api', '55.15')   # the NVN API major.minor version nvnDeviceGetInteger reports (hle_nvn.c)
     if not re.fullmatch(r'\d+\.\d+', api): sys.exit(f'{g.id}: device_api must be major.minor: {api}')
+    # HDR output (nvn_mtl.m, HDR output): the key of the game's tone mapping pixel shader and the profile's MSL file that replaces it,
+    # with the same bindings, tone mapping up to the screen's headroom; the file goes into the host as a string (DM_GAME_HDR_MSL), compiled when the host loads that shader
+    hdr = g.words('hdr', '')
+    if hdr and (len(hdr) != 2 or not re.fullmatch(r'[0-9a-f]{16}', hdr[0]) or not (g.dir / hdr[1]).is_file()):
+        sys.exit(f'{g.id}: hdr is <pixel shader key> <MSL file in the profile folder>: {" ".join(hdr)}')
+    hdr_msl = (g.dir / hdr[1]).read_text() if hdr else ''
+    if not hdr_msl.isascii(): sys.exit(f'{g.id}: {hdr[1]} must be ASCII (it goes into a C string)')
+    cs = lambda s: '"' + s.replace('\\', '\\\\').replace('"', '\\"').replace('\n', '\\n"\\\n"') + '"'   # a C string over several lines
     src = f'dm/games/{g.id}' if g.dir.parent == HERE else str(g.dir)
     L = [f'// Generated by dm/games/game.py from {src}/game.tsv; do not edit',
          '#pragma once',
@@ -189,6 +197,9 @@ def header(g):
          f'#define DM_GAME_TEX_DYN_SLOTS {int(tds == "on")}',
          f'#define DM_GAME_GPU_TS_SEGMENT {int(gts == "segment")}',
          f'#define DM_GAME_VCONV_RANGE {int(vcr == "on")}',
+         f'#define DM_GAME_HDR {int(bool(hdr))}',
+         f'#define DM_GAME_HDR_KEY {q(hdr[0] if hdr else "")}',
+         f'#define DM_GAME_HDR_MSL {cs(hdr_msl)}',
          *([f'#define DM_GAME_NVN_API_MAJOR {api.split(".")[0]}', f'#define DM_GAME_NVN_API_MINOR {api.split(".")[1]}'] if g.get('device_api', '') else []),   # without the key: by the SDK version (hle_nvn.c nvn_api)
          '']
     return '\n'.join(L)

dm/games/settings.tsv

@@ -22,6 +22,7 @@ DM_PRESENT_UPSCALE	choice	off=Standard,metalfx=MetalFX	off	always	Game	Upscaling
 DM_LANG	choice	languages	build:DM_LANG	languages>1	Game	Game language		level=basic
 DM_FULLSCREEN	choice	0=Window,1=Fullscreen	0	platform:mac	Game	Display mode		level=basic
 DM_SHOW_FPS	toggle	-	build:DM_SHOW_FPS	platform:ios	Game	Frame-rate counter
+DM_HDR	toggle	-	1	DM_GAME_HDR!=0	Game	HDR	A little more GPU time and memory; bright scenes use more power. Needs a screen with HDR.
 DM_TOUCH_HAPTICS	toggle	-	1	platform:ios	Controls	Touch feedback
 DM_RUMBLE	toggle	-	1	always	Controls	Game rumble		level=basic
 DM_RUMBLE_GAIN	range	0:3:0.1	1.0	always	Controls	Rumble strength		needs=DM_RUMBLE=1

dm/gfx/msl/hdr_merge.metal

@@ -0,0 +1,17 @@
+// HDR output's present copy (nvn_mtl.m hdr_merge): the swapchain's bytes into the ring slot as they are (pic), and the side texture's excess into the slot's highlights texture (hx)
+// where the picture's luminance, decoded from sRGB, is still what the tone mapping wrote (within 1.2 % + 5 %, fading out over as much again). scale: the tone mapping target's
+// share of the side texture
+#include <metal_stdlib>
+using namespace metal;
+struct V { float4 p [[position]]; float2 uv; };
+vertex V vs(uint id [[vertex_id]]) { float2 uv = float2((id << 1) & 2, id & 2); V o; o.p = float4(uv * float2(2, -2) + float2(-1, 1), 0, 1); o.uv = uv; return o; }
+struct M { float2 scale; float2 pad; };
+struct O { float4 pic [[color(0)]]; float4 hx [[color(1)]]; };
+fragment O fs(V in [[stage_in]], texture2d<float> pic [[texture(0)]], texture2d<float> side [[texture(1)]], sampler s [[sampler(0)]], constant M &m [[buffer(0)]]) {
+    float4 v = pic.read(uint2(in.p.xy));
+    float3 c = select(pow((v.rgb + 0.055) / 1.055, float3(2.4)), v.rgb / 12.92, v.rgb <= 0.04045);
+    float4 g = side.sample(s, in.uv * m.scale);
+    float tol = 0.012 + 0.05 * g.a;
+    O o; o.pic = v; o.hx = float4(g.rgb * (1 - smoothstep(tol, 2 * tol, abs(dot(c, float3(0.2989, 0.587, 0.1145)) - g.a))), 0);
+    return o;
+}