Skip to content

NeXTSTEP CD-ROM builder, old-tar import fixes, RISC-V CI + GUI fixes - #84

Merged
danifunker merged 14 commits into
mainfrom
add-next-ISO-builder
Sep 25, 2026
Merged

danifunker merged 14 commits into
mainfrom
add-next-ISO-builder

Conversation

@danifunker

@danifunker danifunker commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds rb-cli optical new next-ufs (alias nextstep) and a TUI wizard target that build NeXTSTEP / OPENSTEP CD-ROM images. A NeXT "ISO" is not ISO 9660: every NeXTSTEP 3.x CISC distribution disc is a dlV3 NeXT disk label in 2048-byte sectors wrapping one 4.3BSD UFS partition, and that is the shape this writes.

Building real discs from a NeXT software archive exposed several import bugs, fixed here too.

NeXT CD builder

  • partition::next_cd_builder streams the label + UFS (8 KiB blocks / 2 KiB fragments, fs_fsbtodb 0, 32×64 geometry, 80-sector porch). For a disc the size of NeXT's own NeXTTIME CD, every derived superblock word and the label's slot 0 match the real disc byte for byte (pinned in unit tests).
  • ufs_format::Bsd43Geometry makes the 4.3BSD track geometry / rps / maxcontig / maxbpg configurable; hard-disk output is unchanged.
  • Bug fix: NeXT's DIRBLKSIZ is a compile-time 1024 even on 2048-byte media. The reader derived 2048 from fsize >> fsbtodb, so editing any NeXT CD wrote directory chunks a NeXT kernel would reject. Verified against all 3168 directories on the NeXTSTEP 3.3 CISC disc.
  • NextPartitionSpec::cpg, boot-block numbers scaled to d_secsize.
  • CLI flags mirror sgi-efs (--size/auto, --from-dir, --expand-archives, --flatten-folders, ...) plus --expand-gunzip and --bytes-per-inode. TUI: new CD-ROM target with an extra Gunzip toggle.

Import

  • --expand-gunzip (also on rb-cli import): strip one gzip layer only (x.tar.gz -> x.tar, f.gz -> f).
  • Pre-POSIX (v7) tar sniffing, and typeflag-less name/ entries read as directories.
  • Tar hard links land as copies of their target (spooled, not loaded into RAM). They used to be dropped, e.g. Opener.app's gunzip, gcc's cross-driver names.
  • GNU tar 1.11 @@MaNgLeD.N long names restored from the trailing N entry.
  • Old NeXT tar puts a hard link's target size in the link header with no data behind it; a small header normalizer zeroes it, as GNU tar ignores it.
  • A member repeated within one archive: the later copy wins, as with GNU tar.

Raw file-name bytes (NeXT only)

NeXTSTEP names are raw NeXTSTEP-encoded or EUC-JP bytes, not UTF-8. fs::raw_name carries undecodable bytes through the &str API as Private Use Area-B placeholders and writes them back byte for byte. Gated by EditableFilesystem::stores_raw_names, which only NeXT UFS volumes report; every other filesystem and UFS flavour keeps its previous behaviour (non-NeXT tar targets get %XX).

RISC-V CI and GUI

  • CI (Build Linux RISC-V 64 GUI): glib is Multi-Arch: same, so the riscv64 sysroot needs the exact glib version of the runner's amd64 install. The old pin to the release pocket broke whenever the runner image's glib moved on (intermittent on main since 2026-09-10). The install step now tries each glib version available for riscv64 dev packages and amd64, newest first, pins it on both arches, and keeps the first one apt-get install -s resolves (today 2.80.0-6ubuntu3.9). Green on 1c662201.
  • GUI segfault on RISC-V hardware: on an Orange Pi RV2 (PowerVR BXE-2-32, Ubuntu 24.04) eframe picked wgpu, wgpu picked the PowerVR Vulkan driver, and the driver segfaulted; a segfault bypasses the wgpu -> glow -> software fallback. Linux on aarch64 / arm / riscv64 now starts on OpenGL (glow) first (GL_FIRST), which also targets the slow wgpu path reported on the Raspberry Pi 5. Other platforms are unchanged.
  • Renderer override through elevation: pkexec strips the environment and the elevated relaunch did not pass RUSTY_BACKUP_RENDERER / WGPU_BACKEND, so "Unlock Physical Devices" dropped a working override. Both now pass through. The override is documented in the README install steps.

Also

  • fix(os): device_open_error no longer compiles on Windows, where it has had no caller since c85fa9b, so clippy -D warnings passes on a Windows host again.
  • Docs: README, cli-examples ("Build a NeXTSTEP CD-ROM"), OPEN-WORK (§3.3 follow-ups, §6.2 GUI path, §10 audit).

Scope cuts

  • Data discs only: no boot blocks. RISC (HP-PA/SPARC hybrid) discs are not produced. No GUI path yet (OPEN-WORK §6.2).
  • The UFS writer still gives each file tail a whole 8 KiB block (NeXT packs tails into fragments); --size auto budgets for it. OPEN-WORK §3.3.

Testing

  • scripts/preflight.sh green: cargo test --release, MiSTer feature set, Rust 1.73 vintage floor, doc parity, rb-regress.
  • New unit tests: geometry / superblock / label pinned to the NeXTTIME CD, reader + fsck round trip, v7 / hard-link / mangled-name / duplicate-member / non-UTF-8 tar cases, raw-name round trip including the on-disk byte. CLI integration: tests/cli_suite/cli_next_cdrom.rs.
  • Built nine 640 MiB discs from a NeXTSTEP software archive with --expand-archives; all pass our fsck and an independent Python UFS walk (no directory-chunk violations).
  • RISC-V GUI: on the Orange Pi RV2 the old default segfaults every time; WGPU_BACKEND=gl and RUSTY_BACKUP_RENDERER=glow run clean. This PR's build (v2026-09-25-14-54), launched with no overrides, logs Using glow, draws the full window, and runs until closed. Raspberry Pi 5 not tested here.
  • Real NeXT: under the Previous emulator (NeXTstation, 68040, v66 ROM, NeXTSTEP 3.3) the disc auto-mounts in Workspace on boot, as root and as a normal user. The Developer disc also reads on the MiSTer NeXT core. Launching apps from the disc under Previous is not yet confirmed (keyboard-only driving through WSLg).

🤖 Generated with Claude Code

danifunker and others added 14 commits September 24, 2026 15:08
Since c85fa9b (R-068) Windows opens raw devices through
`windows::open_source_for_reading`, so `device_open_error` has no Windows
caller and `cargo clippy -D warnings` fails on a Windows host with
dead-code. Narrow its cfg to the platforms that call it and drop the
unreachable Windows arm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…'s own CDs

A NeXT "ISO" is not ISO 9660: every NeXTSTEP 3.x CISC disc is a dlV3 disk
label in 2048-byte sectors wrapping one 4.3BSD UFS partition. The new
`partition::next_cd_builder` streams that shape from scratch, reusing the
existing label writer and 4.3BSD formatter.

For a disc the size of NeXTTIME.iso, every derived superblock word, the label
header and slot 0 match the reference disc byte for byte; cylinder-group
headers and fs_postbl / fs_rotbl differ only where the reference holds files.

- ufs_format: `Bsd43Geometry` fixes ntrak / nsect / rps / maxcontig / maxbpg
  (preset `NEXT_CDROM`: 32 x 64, 5 rps, 20000, 512); HD behaviour unchanged.
- ufs: `DIRBLKSIZ` on NeXT is the kernel's compile-time 1024, not the media's
  device block. All 3168 directories on the 3.3 CISC CD chunk at 1024, while
  `fsize >> fsbtodb` said 2048, so editing a NeXT CD was writing chunks a NeXT
  kernel would call corrupt. `dir_block_size` caps it for reader + formatter.
- next: `NextPartitionSpec::cpg` (16 on disks, 2 on CDs); boot-block numbers
  are scaled to d_secsize (32/96 at 1024 is 16/48 at 2048).
- Scope: data discs only (no boot blocks); RISC hybrids not produced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tars

`--expand-archives` unpacks a tarball fully. `--expand-gunzip` is the lighter
option: it strips only the gzip layer and keeps what is inside, so
`x.tar.gz` / `x.tgz` land as `x.tar` and `disk.iso.gz` as `disk.iso`. With
both, tarballs are unpacked and only non-tar gzip files are decompressed.
A file named `.gz` that is not gzip is copied through untouched.

The decompressed size is counted by streaming (MultiGzDecoder), not taken
from the trailer's ISIZE, which wraps at 4 GiB; `measure_dir_for` and the
preflight use the same count so `--size auto` sizes to the unpacked tree.

Tar detection also accepts pre-POSIX (v7) headers, which carry no `ustar`
magic and were silently copied in unexpanded: two of NeXT's own 3.3 patch
tarballs are this shape. The fallback requires a name, octal numeric fields
and a matching header checksum.

- `rb-cli import --expand-gunzip`; the CD/HD builders pass `false` for now.
- `ImportStats::gunzipped` counts it; `import` reports it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`rb-cli optical new next-ufs IMG.iso` (alias `nextstep`) writes the NeXT
label + 4.3BSD UFS disc from `partition::next_cd_builder`, beside the
existing `sgi-efs` / `mac-hfs` / `mac-hfsplus` targets and with the same
flags: `--size` (default 600M, or `auto`), `--name`, `--from-dir`,
`--expand-archives`, `--flatten-folders`, `--force`, `--no-permissions`,
`--include-appledouble`, plus `--expand-gunzip` and `--bytes-per-inode`.

- `--size auto` budgets a whole 8 KiB block per entry: our UFS writer gives
  each file's tail a full block, where NeXT packs tails into fragments.
- A size the builder refuses is caught before the output file is created.
- Discs past 700 MB are built, with an Info line that they will not burn.
- v7 tar: a typeflag-less entry ending in `/` is a directory (GNU tar reads
  it the same way). NeXT's 3.3_Int_68k_Patch1.tar failed to expand without it.

Tests: tests/cli_suite/cli_next_cdrom.rs covers both archive depths, the v7
directory, ls / fsck / get, and the refused-size path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CD-ROM class now offers "NeXTSTEP (NeXT label + 4.3BSD UFS)", which runs
`optical new next-ufs` through the same path as the SGI and Mac targets:
path, size (600M or auto), name, source folder, and the expand-archives
toggle. The NeXT target adds one more toggle row, "Gunzip", for
`--expand-gunzip`; the other targets keep their five rows, since their verbs
do not take the flag. The optical command palette lists the new verb too.

The two toggle rows share one renderer rather than duplicating it.

Tests: the gunzip row exists only for NeXT and holds no text; a wizard
build leaves an image with a NeXT label at block 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README: `optical new next-ufs` examples, the NeXT-label row explains the CD
  shape (2048-byte sectors, not ISO 9660), the UFS row notes the fixed 1024
  DIRBLKSIZ on NeXT CDs, and `--expand-gunzip` beside `--expand-archives`.
- cli-examples: a "Build a NeXTSTEP CD-ROM" section covering both archive
  depths and why they default off (.pkg / .tar.Z stay for NeXT's Installer).
- OPEN-WORK: §3.3 follow-ups (tail-fragment packing, bootable discs, RISC
  hybrids, a real-kernel size check under Previous), §6.2 the missing GUI
  path for creating any CD image, and a §10 audit entry.

No picker or MiSTer-matrix change: `.iso` is already in DISK_IMAGE_EXTS and
NeXT CDs were already readable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… names

Building NeXTSTEP CDs from the NeXT software mirror dropped ~60 members
and misnamed more:

- Hard links were skipped as unrepresentable, so Opener.app lost its
  gzip / gunzip / toast tools, gcc its i386- / m68k-next-nextstep3-gcc
  drivers, and several .rtfd help documents their images. A link now
  lands as a copy of its already-imported target (`Importer::lookup`
  resolves it); a link whose target never landed is still skipped.
- GNU tar 1.11 stores members with names past 100 bytes as
  `@@MaNgLeD.N` and names them in a trailing type-`N` entry
  ("Rename @@MaNgLeD.N to <path>", "Symlink <target> to <path>").
  Those members were written under their placeholders, leaving e.g.
  UsingCalendarPalette.nib empty. They are now held (up to 256 MiB) until
  the list arrives and written under their real names; any it never
  names keep the placeholder.

`ImportStats` gains `hardlinks_copied` / `mangled_renamed`, reported by
`rb-cli import`; the preflight counts a hard link as the file it becomes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three more archive shapes from the NeXT software mirror aborted an
--expand-archives import:

- Old NeXT tar records a hard link's target size in the link header with
  no data behind it. GNU tar ignores the size on link / device / dir
  headers; the tar crate skipped that many bytes and then failed on a
  "header" in the middle of file data (eval.3.3.s.tar.gz). A small reader
  (`NormalizedTar`) now zeroes the size on those headers before the tar
  crate sees them, for import, measure and preflight alike.
- Member names in EUC-JP or Latin-1 were refused ("only Unicode paths are
  supported on Windows"). Names now come from the raw bytes, and bytes
  that are not UTF-8 become `%XX`, so they stay distinct and ASCII-safe
  (Steroidgrundger%FCst.lookMol).
- A member stored twice in one archive stopped the import on a name
  conflict (edsnd.1.42). As with GNU tar, the later copy now replaces the
  earlier one; conflicts with anything outside the archive still follow
  the caller's policy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NeXTSTEP wrote file names in its own 8-bit NeXTSTEP encoding (0xF6 is
u-diaeresis, 0xBC an ellipsis; Unicode VENDORS/NEXT/NEXTSTEP.TXT) and
NeXTSTEP-J in EUC-JP. Neither is UTF-8 and a disk does not record which
it used, so the only faithful thing is to keep the bytes. Our filesystem
API passes names as `String`, which forced lossy decoding on read and a
%XX escape on tar import, so a NeXT saw "Steroidgrundger%F6st.lookMol"
instead of "Steroidgrundgerust" with its umlaut.

`fs::raw_name` maps each byte that is not UTF-8 to a placeholder in the
last 256 code points of Supplementary Private Use Area-B (U+10FF00 + b):
`decode` on the way in, `encode` back to the exact bytes on the way out.

- UFS: dirent names and symlink targets decode on read and encode on
  create / mkdir / symlink / rename / delete, so a name read off one NeXT
  disk and written to another lands byte for byte.
- tar import: member names go through `decode` instead of `%XX`.
- `rb-cli ls` prints the raw bytes as `\xNN` rather than placeholder glyphs.
- A host file extracted with placeholders in its name re-imports the same.

Tests pin the round trip for NeXTSTEP, EUC-JP and 0xFE/0xFF bytes, and that
a UFS dirent holds the raw 0xF6 with no placeholder bytes on disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pies

ab6c13f carried non-UTF-8 name bytes through every UFS and every tar
import, which reached further than intended: FreeBSD / Solaris / SunOS
UFS names changed from U+FFFD to placeholders, and a tar import into
FAT, HFS or ext stored the placeholder code points themselves, since
only UFS turns them back into bytes (ext4 got four bytes of junk each).

`EditableFilesystem::stores_raw_names` (default false) now gates it, in
the same single-predicate shape as `supports_symlinks`. Only a NeXT UFS
volume (4.3BSD cylinder groups) says yes:

- UFS: raw bytes on NeXT; every other UFS flavour keeps its old lossy
  UTF-8 reading and writes names as given.
- tar import: placeholders only when the target stores raw names; any
  other target gets `%XX` for a byte that is not UTF-8, as in a84c107.

Hard-link copies now stream through `tempfile::spooled_tempfile` at the
shared `fs::copy::SPOOL_THRESHOLD` instead of `read_file` into RAM, per
CONTRIBUTING's streaming rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CONTRIBUTING caps a comment at two lines. Four doc comments added with
`optical new next-ufs` (the subcommand, --size, --expand-gunzip,
--flatten-folders) ran three to five. The detail they carried is in
docs/cli-examples.md "Build a NeXTSTEP CD-ROM".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…f a pocket

The RISC-V GUI job failed with "dconf-gsettings-backend : Depends:
libglib2.0-0t64 (>= 2.79.0) but it is not installable". glib is
Multi-Arch: same, so libglib2.0-0t64:riscv64 must be the exact version of
the runner's installed libglib2.0-0t64:amd64. 8790528 pinned riscv64 glib
to the release pocket to dodge a ports-mirror lag (2.80.0-6ubuntu3.8's new
libglib2.0-dev-bin-linux had no riscv64 build), but the runner image's
amd64 glib follows noble-updates on its own schedule, so the release
version stops matching whenever the image moves: the intermittent
failures on main (2026-09-10) and on PR #84.

The install step now tries each glib version that has a riscv64
libglib2.0-dev and an amd64 libglib2.0-0t64, newest first, pins it on both
arches, and keeps the first one `apt-get install -s` resolves for the full
package list, downgrading amd64 glib if needed. That covers both the
ports lag and the runner skew without hard-coding a version, and fails
with a clear error if no version works.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ve one

c4b3a2c's candidate loop ran `apt-cache madison libglib2.0-dev`, which
lists only the native (amd64) architecture, so the riscv64 filter found
nothing, no version was tried, and the job stopped at "no glib version
resolves". Qualify both queries by architecture (`:riscv64`, `:amd64`).

The failed run's unpinned dry-run confirms the approach: apt resolves
2.80.0-6ubuntu3.9 on both arches today. The 74 packages it removes (the
runner's python3 / LLVM / cloud-init stack) match the last green run on
main, so they are the job's existing behaviour, not a regression.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… through elevation

On an Orange Pi RV2 (SpacemiT/Ky X1, PowerVR BXE-2-32, Ubuntu 24.04) the
GUI segfaults on launch: eframe prefers wgpu, wgpu picks the PowerVR
B-Series Vulkan driver (24.2@6603887), and the driver crashes. A segfault
kills the process, so the wgpu -> glow -> software fallback never runs.
Measured on the board: default (wgpu/Vulkan) segfaults every time;
WGPU_BACKEND=gl and RUSTY_BACKUP_RENDERER=glow both run clean. wgpu is
also reported very slow on the Raspberry Pi 5's V3D.

- Linux on aarch64 / arm / riscv64 now tries OpenGL (glow) first, then
  wgpu, then software (`GL_FIRST`). Every other platform keeps wgpu first.
  The chain is one ordered loop instead of nested matches.
- pkexec strips the environment, and the elevated relaunch passed through
  neither RUSTY_BACKUP_RENDERER nor WGPU_BACKEND, so "Unlock Physical
  Devices" dropped a working override and relaunched onto the crash. Both
  are now passed through.
- README: the renderer override is documented for users.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@danifunker danifunker changed the title NeXTSTEP CD-ROM builder (optical new next-ufs) + old-tar import fixes NeXTSTEP CD-ROM builder, old-tar import fixes, RISC-V CI + GUI fixes Sep 25, 2026
@danifunker
danifunker merged commit 363d725 into main Sep 25, 2026
29 checks passed
@danifunker
danifunker deleted the add-next-ISO-builder branch September 25, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant