Skip to content

Yakuza Kiwami plays on the Mac and the iPhone; ADR jump tables with a far bounds check; PARC/SLLZ shader split - #912

Open
joshhhhyyyy wants to merge 5 commits into
seedds:mainfrom
joshhhhyyyy:yakuza-kiwami
Open

joshhhhyyyy wants to merge 5 commits into
seedds:mainfrom
joshhhhyyyy:yakuza-kiwami

Conversation

@joshhhhyyyy

@joshhhhyyyy joshhhhyyyy commented Nov 2, 2026 •

Copy link
Copy Markdown

What and why

Yakuza Kiwami (0100C9801FEE6000, update 1.01 = v65536, SDK 18.3.2, Ryu Ga Gotoku Studio's engine with CRIWARE, NVN) plays on the Mac and on an iPhone: boot, the title, the story with its cutscenes and fights. Four commits, the first two standing on their own:

  • Rewriter (dm/tools/patch_nso.py, jump_adr_far_check): an ADR-relative jump table whose cmp is scheduled 31 instructions before its br (main+0x91b24c: 23 float and bitfield instructions between the cmp and its b.hi) stops the rewrite on main with AssertionError: jump table 91b24c. The new form finds the check by the flags the b.hi reads, not by a distance, and is tried only after every other form, that is only where the rewrite used to stop. Entry: docs/reference/rewriter.md, "ADR-relative jump table whose bounds check is scheduled far before its branch".

  • Shader split (dm/gfx/tools/shader_split.py): the game keeps its shaders as whole GLSLC outputs in SLLZ-compressed shader objects (.vso / .pso / .fxo) inside PARC archives, and the engine's own in SLLZ streams in main, so the split found none and the NSP script stopped. It now lists PARC archives by their tables, decompresses only the shader objects (SLLZ versions 1 and 2, after ParManager's ParLibrary), and reads main's streams: 0 → 2,462 shaders. Entry: docs/reference/shaders.md, "GLSLC outputs in Ryu Ga Gotoku's PARC archives and SLLZ streams".

  • Profile and note (dm/games/0100C9801FEE6000/, games table row): the provisional profile's keys, except the memory layout. The game reserves a 32 GB address region and carves 13 GB of reservations from it (a 4 GB heap and pools of 1, 4, 2 and 2 GB), so the profile gives it one 36.62 GB block with a 36 GB alias region (mem_reserve 0x928000000, mem_alias 0x900000000). An iPhone19,4 on iOS 27.2 reserves that with the Increased Memory Limit entitlement the builds carry by default: a standalone mmap probe found a 64 GB largest block with it and 6 GB without (docs/reference/memory.md). The shader tables and the shader_* keys are the provisional profile's, unchanged, as every shader_* key is in each shader's cache key (as in The Legend of Zelda: Echoes of Wisdom (01008CF01BAAC000, base game 1.0.0): game profile #152).

  • Gallery (docs/images/yakuza-kiwami.jpg, gallery.tsv, README regenerated): a frame of a street fight from a Mac run of this branch.

No game-specific rewriter cut, no host change, no shader-chain change.

Checked

  • Tests (a Linux VM, Python 3.10, DM_GAME=010015100B514000 as in CI): every file of dm/tools/tests, dm/ios/tests and dm/gfx/tests. test_docs.py, the new test_split_par.py (11 tests), test_jump_copies.py with the new AdrFarCheck cases and test_game_profile.py pass; 23 other files fail there with the same counts on main (they need macOS tools or a newer Python). ruff 0.17.2 and pyright 1.1.414, the pinned versions: no new finding in the changed files against main. Host and GPU tests not run: no host or graphics code changed.
  • Rewriter against the build the game was played on: main (all seven output files), subsdk0 and the game's NRO rewrite byte for byte the same; on main, the rewrite of this main stops with the assertion. Reach: the new form runs only where every older one failed, so a module that rewrote before rewrites the same: Tomodachi Life: Living the Dream 1.0.4's main and sdk, the other game on hand, do byte for byte.
  • Split: shader_split.py here is byte for byte the version that split the game (2,462 shaders: 1,259 vertex, 1,193 pixel, 9 geometry, 1 compute), and the profile's tables are that split's. Reach: the new paths run only on files that start with PARC or SLLZ and on SLLZ streams; Tomodachi Life's 66,343 RomFS files and its main have none.
  • profile_guess.py --check: the guesses match the profile except mem_reserve, the measured layout (mem_alias shows under "heap check / measured").
  • Games run, from a local main with other PRs merged (below):
    • MacBook Pro (M3 Pro), with a window: a logged run of 24,600 frames (about 14 minutes) plus longer play by hand, cutscenes and fights. No [RED] line, no unimplemented call, no shader key missing from the map. Most frames take 33.3 ms (30 fps), with stretches at 60, as the game paces itself under present interval 1. Peak 2.54 GB resident and 1.39 GB of GPU allocations; the regions' highest mapped page +33.75 GB of the 36 GB alias region, 2.25 GB mapped at most.
    • iPhone19,4 (A20 Pro, 12 GB, iOS 27.2), an unsigned IPA from the NSP script, sideloaded: 4,500 frames, the guest block at 0x7000000000. No [RED] line, no crash; peak 2.04 GB resident and 1.36 GB of GPU allocations, 6.15 GB below the memory limit at the lowest; thermal state 0.
  • This branch on the Mac (build_ipa_from_nsp.py --platform mac from this tree): extraction reused; main, subsdk0 and the NRO rewritten byte for byte as above, the sdk with main's 5 SDK-wide cuts; the split 2,462 shaders, its tables passing the check against this profile's; 2,462 Mac shaders and the host built. Then 8,400 frames with a window, from a save into street fights: no crash, no unimplemented call, no shader key missing from the map; the regions' highest mapped page +33.65 GB (+33.75 GB with the older cuts below); peak 2.40 GB resident, 1.38 GB of GPU allocations. One red line, once, at frame 2614 as a new pipeline was made: texture type mismatch: shader d303b2b1a0f401f7_b (stage 1) declares slot 134 as Metal type 5, texture ID 6493 is type 7 (NVN target 2, format 0x9a, Metal format 80); bound as is (a 3D texture in a cube slot; nothing wrong on screen; the note's Open). The gallery frame is from this run.
  • Regression harness: not run. regress.py needs macOS, and the reach of both tool changes is nil by construction (the two "Reach" checks above), so no game's run can change.

Tested with

The earlier Mac and iPhone runs were made on a local main (2026-11-02, before #855 and #689) with these PRs merged, none of them part of this PR or needed by it: #627 (Port Tomodachi Life: Living the Dream 1.0.4), #683 (iOS: adapt gameplay to Duo portrait and tabletop poses) and #812 (translate shaders the split missed at run time with maxmetal), plus a small local iOS build fix. #812 is on by default, and its host prints a [RED] shader … is not in the shader map: maxmetal translated it at run time line for each shader it translates; neither run printed one, so every shader came from this profile's map.

One difference follows from #627: its branch predates 17f670a, and merging it brought back the old per-build cut rows for this game's sdk (f118ae1c…, Absolum's SDK 18.3.2 build). So the earlier runs had regions of 1 GB and more aligned to 32 MB, where this branch has main's SDK-wide rule (2 MB alignment from 4 MB up, as SDK 20), which only takes away alignment slack in front of the game's regions: the run of this branch above reached +33.65 GB where the older cuts reached +33.75. The game note records it.

Not checked

  • The iPhone with this exact branch (the iPhone run had the older sdk cuts; the new rule only lowers the regions' reach).
  • The cube-slot texture mismatch above: which pass, and whether the slot is read.
  • Versions other than 1.01, and phones whose largest single reservation is about 6 GB (they would need the game's own sizes cut: the note's Open).

Docs

  • The findings are in docs/reference/rewriter.md, docs/reference/shaders.md, docs/reference/memory.md and the game note, and python3 dm/tools/docs_symptoms.py was run (the symptom index)
  • No shader chain tool changed (dm/gfx/nvn/dmsh_chain.h untouched)
  • No game files, keys or data extracted from a game in the change or the attachments

🤖 Generated with Claude Code

joshhhhyyyy and others added 4 commits November 2, 2026 00:08
…scheduled far before their branch

Yakuza Kiwami 1.01's main stopped the rewrite with `AssertionError: jump table 91b24c`. The dispatch at
main+0x91b24c (b.hi; mov w16, w21; adrp / add of the table; adr x18; ldrsw; add; br x18) checks its index with
`cmp w21, seedds#3` 31 instructions before the br: the compiler scheduled 23 floating-point and bitfield instructions
between the compare and its b.hi. jump_adr_offsets looks 16 instructions back for the compare and the other forms
expect it right before the b.hi; a wider window alone would not prove that the flags the b.hi reads come from it.

jump_adr_far_check, tried after every other form so the tables they recognize keep their targets, finds the check by
the flags: back from the load through straight-line code the first control flow must be the b.hi, and before it the
first instruction that writes the flags must be a cmp of the index with an immediate. Between that compare and the br
nothing may call, branch, trap or write the index, the entry or x18, no branch or known table of the function may
enter the run, and every target must be an instruction of the function outside it. The entry is in
docs/reference/rewriter.md, "ADR-relative jump table whose bounds check is scheduled far before its branch".

Checked: test_jump_copies.py's AdrFarCheck (the far and the near check, every kind of flag writer between the check
and its branch, writes of each register involved, control flow into and out of the run, undecodable words, the real
rewriter on a synthetic module). Yakuza Kiwami's main rewrites, the table resolving to four entries of which 1 and 3
share a case block; a census of its modules found no other unrecognized table. On main the rewrite of that main stops
with the assertion; with this change its seven output files are byte for byte those of the build the game was played
on. The new form runs only where every older one failed, which stopped the rewrite before, so a module that rewrote
before rewrites the same: Tomodachi Life: Living the Dream 1.0.4's main and sdk, the other game on hand, do byte for
byte. ruff and pyright find nothing new.

Co-Authored-By: Claude <claude@users.noreply.github.com>
Yakuza Kiwami's NSP build stopped at `the shader split found no shaders in main or the RomFS`, after
`[RED] main control section @... no code area` lines. Ryu Ga Gotoku Studio's engine keeps nearly all its files in PARC
archives, most of them SLLZ-compressed, and its shaders are shader objects (GSVS4 .vso, GSPS4 .pso, GSFX4 .fxo) that
hold whole GLSLC outputs, all SLLZ version 1, so a scan of the stored bytes finds no magic. The engine's own objects
are SLLZ streams in main's data, whose literals copy the GLSLC and control magics verbatim: those were the control
sections with no code area.

shader_split.py lists a file that starts with PARC by its tables (par_entries) and decompresses only the files whose
first 8 decoded bytes are a shader object, a GLSLC output or another PARC archive (par_shaders; sllz peeks), so the
archives of models, textures and motions are only listed. Each GLSLC output is taken whole, one stage each, and the
pixel-input cover pairs them. sllz_main_objects reads the objects in main, raw_shaders leaves out control magics
inside SLLZ streams, and a file that is SLLZ as a whole is decompressed first. One damaged file is left out with a red
line, not its archive. The layouts follow ParManager's ParLibrary (Kaplas80), whose SLLZ decoder credits Rick Gibbed.
The entry is in docs/reference/shaders.md, "GLSLC outputs in Ryu Ga Gotoku's PARC archives and SLLZ streams".

Yakuza Kiwami 1.01: 0 -> 2,462 shaders (1,259 vertex, 1,193 pixel, 9 geometry, 1 compute). 2,399 are in the four
archives under data/shader (3,172 objects with 3,752 outputs) and 63 more in 59 of main's 60 SLLZ streams, among them
the only geometry and compute shaders. Its other 2,969 archives (17 GB) hold no shader object; data/chara.par
(1.5 GB) is listed in 1.6 s.

The new paths run only on files that start with PARC or SLLZ and on SLLZ streams. Tomodachi Life: Living the Dream,
the other game on hand, has none: no file of its RomFS (66,343) starts with either, and its main (54 MB) holds no
SLLZ byte, so its split is unchanged.

Checked: test_split_par.py, 11 tests (SLLZ versions 1 and 2 in both byte orders, overlapping copies, peeks and damaged
streams; PARC listings in both byte orders with subfolders; stored and compressed nested archives; only shader objects
decompressed; one damaged file left out alone; .par and SLLZ-wrapped files through scan_file_in; main's streams and
raw_shaders' skip).

Co-Authored-By: Claude <claude@users.noreply.github.com>
Update 1.01 (v65536), Ryu Ga Gotoku Studio's engine with CRIWARE, SDK 18.3.2, on NVN: a profile with its note and
games table row. Played on the Mac (M3 Pro) and an iPhone19,4 (A20 Pro, iOS 27.2): boot, the title, the story with its
cutscenes and fights. No game-specific rewriter cut or host change: it needed the two general fixes before this commit.

The game maps all its memory itself. main reserves a 32 GB address region (main+0x379938) and its own allocators carve
13 GB of reservations from it (a 4 GB growable heap and pools of 1, 4, 2 and 2 GB), about 1.5 GB of them used at the
title. The profile holds them unchanged in a 36 GB alias region, one 36.62 GB block (mem_reserve 0x928000000,
mem_alias 0x900000000): the regions reached +33.72 GB in the NSP script's address region check, the highest mapped page
+33.75 GB in play, with at most 2.25 GB mapped. A standalone mmap probe on the iPhone19,4 found one 64 GB range with the
Increased Memory Limit entitlement the builds carry by default (6 GB with neither entitlement), so the block fits there;
docs/reference/memory.md records it. A phone limited to about 6 GB would need the game's sizes cut (the note's Open).

The other keys are the provisional profile's static guesses, kept because they boot and play; profile_guess.py --check
differs only in mem_reserve, the measured layout. The shader tables are the provisional profile's, byte for byte (the
split of the commit before: 2,462 shaders), and shader_groups / shader_note stay as it wrote them, as every shader_* key
is in each shader's cache key. The version is v65536, the update's, which the NSP script reads from the [v65536] tag in
an update's file name (a single update without one is taken for it).

The Mac run (24,600 frames) and the iPhone run (4,500) printed no [RED] line, no unimplemented call and no shader key
missing from the map, and neither crashed. Most frames take 33.3 ms (30 fps), with stretches at 60: the game paces
itself under present interval 1. Peak memory: Mac 2.54 GB resident and 1.39 GB of GPU allocations, iPhone 2.04 GB and
1.36 GB, with 6.15 GB still below its memory limit at the lowest. Those runs had an older form of the sdk's cuts than
this tree's SDK-wide rule (regions of 1 GB and more aligned to 32 MB, instead of 2 MB from 4 MB up); the note says so.

Co-Authored-By: Claude <claude@users.noreply.github.com>
The gallery picture is a frame of a street fight from a Mac run of this branch's own build (DM_NVN_SHOT, frame 6000,
1280x720 scaled to 960x540). That run took the sdk's SDK-wide cuts (2 MB alignment from 4 MB up) where the earlier
runs had the older per-build ones: 8,400 frames from a save into street fights, the regions' highest mapped page at
+33.65 GB of the 36 GB alias region (+33.75 GB before), peak 2.40 GB resident and 1.38 GB of GPU allocations, no
crash, no unimplemented call, no shader key missing from the map. Its one red line, a 3D texture bound in a shader's
cube slot once in a fight, is in the note's Open.

Co-Authored-By: Claude <claude@users.noreply.github.com>
@joshhhhyyyy
joshhhhyyyy force-pushed the yakuza-kiwami branch 3 times, most recently from 2bcfa14 to fd279cb Compare November 2, 2026 18:06
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant