diff --git a/README.md b/README.md index 5c1bf70c..cc306bc4 100644 --- a/README.md +++ b/README.md @@ -705,7 +705,7 @@ PC Engine CD, CD32, GameCube, Wii, CD-i, and 3DO. | SGI-DkLabel | Yes | Yes; also writes whole labels from scratch | The pre-IRIX SGI disk label, on IRIS 2000 / 3000 series disks. One `struct disk_label` at block 0: drive geometry, the alternate-block region, and 8 `{base, size}` slots with no type field — roles come from `d_bootfs` / `d_swapfs` / `d_rootfs`, and the whole-disk wrapper slots are excluded from the list. Big-endian, packed for the 68020's 2-byte alignment, and **auto-detects the byte-swapped images** period SGI disk controllers produce. The detected orientation is reported by `inspect` / `ls` and on the GUI's Inspect tab, which also offers a *Swap Word Order...* button; `rb-cli swab16` is the scriptable equivalent. Surfaces its slots to the SGI EFS v1 driver (browse / inspect / extract, and editing the volumes). The eight `{d_base, d_size}` slots are editable — resize / move / add / delete, plus moving `d_bootfs` — with edits written back in the image's own word order and partial slot overlap refused (whole-disk wrapper slots are allowed, since containment is how the label spells "the whole drive"). There is no per-slot type field, so `set-type` is refused with that reason. `rb-cli new hd sgi-dklabel` writes a fresh label with the slots you size and give a role (`root` / `swap` / `boot` / `slice`), cylinder-aligned from `--heads` / `--sectors`, plus the whole-disk wrapper slot every label of the era carries; on the reference geometry (987c/7h/17s) the slots land on the same blocks a real IRIS 3130 uses. Fill them with `rb-cli new volume efs-v1` and `--fill N=PATH`, then `rb-cli swab16` if the target machine wants the controller's reversed-word order. **Full-disk backup works through the single-file-CHD layout**: the label and boot area ahead of the first slot ride verbatim, each slot lands at its declared offset, and an as-is restore is a byte copy — the word order is preserved because those bytes are never interpreted. A resize on restore rewrites the eight slots in the label's own word order. A resize at backup or export time runs the same rewrite before staging; the per-partition layouts (zstd / gzip / raw / VHD) are refused, since restoring one would have to rewrite the label. | | AHDI | Yes | No (browse); writes whole tables from scratch | Atari ST / TT / Falcon hard disks. Four primary entries at 0x1C6 plus XGM extended chains, big-endian, no magic number — detection keys off the 0x1234 word-sum and plausible geometry. `rb-cli new hd atari` writes a fresh root sector with the tags you name (GEM / BGM / RAW); a GEM partition over 16 MiB is promoted to BGM, which is what TOS needs. Creating an XGM chain, and grafting in a bootable bootstrap, are future work.. **Full-disk backup works through the single-file-CHD layout**: the root sector rides verbatim ahead of the first partition, an as-is restore is a byte copy, and a resize on restore, at backup time or on export rewrites the entries and the 0x1234 word-sum in place. A per-partition backup previously *succeeded* and then failed at restore with "no MBR data available"; it is refused up front now. A disk with an **XGM extended chain** is imaged as one body instead of per-partition — MBR gets away with that shape only because `build_ebr_chain` rebuilds its EBRs on restore, and nothing rebuilds an XGM chain | | Sun | Yes | No (browse); writes whole labels from scratch | Sun disk label / SMI VTOC (SPARC Solaris / SunOS). 8 big-endian slices (magic `0xDABE`), geometry-derived offsets; the whole-disk "backup" slice is excluded from the list. **SunOS 4.x wrote no VTOC at all** — `sanity`, `version` and `nparts` are zero, so every tag reads 0 and the tag-5 test can't find that alias; there it is identified by geometry instead (a slice from cylinder 0 spanning the label's whole data area), and the slices are shown as `untagged` rather than claiming an `unassigned` tag we never read. Surfaces the UFS slices to the existing big-endian-SPARC UFS reader (browse / inspect / extract). `rb-cli new hd sun` writes a fresh label with the slice tags you name (`root`, `usr`, `swap`, … or a bare tag number), cylinder-aligned from `--heads` / `--sectors`, with slice 2 reserved for the whole-disk alias. Parser and writer both cross-validated against `fdisk` / `sfdisk`; editing an existing label is future work. **Full-disk backup works through the single-file-CHD layout**: the head region — here nothing, because a SunOS root slice starts at cylinder 0 and carries the label in its own first sector — plus every slice at its declared offset, restored as-is by a byte copy, or with the slices rewritten on cylinder boundaries and the checksum restamped when a size changes on restore. The per-partition layouts (zstd / gzip / raw / VHD) are refused with that reason, since restoring one would have to rewrite the label. The same rewrite runs before staging when a size changes at backup or export time. | -| NeXT | Yes | Yes | NeXT disk label (NeXTSTEP / OPENSTEP, black m68k hardware **and** NeXTSTEP/Intel). Up to 8 partitions, big-endian on both architectures, written as four checksummed copies at 512-byte blocks 0/15/30/45 — all four share one checksum, because it is computed with `dl_label_blkno` read as zero. Partition offsets are counted in the label's own `d_secsize` (1024 bytes on every disk we have) and measured from the end of a front porch, so `PartitionInfo` carries an explicit byte offset for them. Probed ahead of MBR because a NeXTSTEP/Intel disk also carries a valid `0xAA55` boot sector with an empty partition table. Surfaces its partitions to the existing big-endian UFS reader (browse / inspect / extract / edit). `rb-cli new hd next` writes a fresh `dlV3` label with the partitions you name (`4.3BSD`, `swap`, or any 8-byte type string), a 160-sector front porch, and the entry's optional NAME field as the partition's `p_mountpt`; `--heads` / `--sectors` set the recorded geometry and are counted in the label's own 1024-byte sectors. Fill it with `rb-cli new volume ufs-43bsd`, which writes the pre-4.4BSD UFS1 NeXTSTEP actually reads. **`rb-cli optical new next-ufs`** builds a NeXTSTEP CD-ROM image the same way — NeXT CDs are not ISO 9660 but this label in **2048-byte** sectors (32 x 64, an 80-sector porch) around one `4.3BSD` UFS with 2 KiB fragments, and for a disc the size of NeXT's own `NeXTTIME` CD every superblock word and the label's slot 0 come out identical to it. `--from-dir` fills it in one step; `--expand-archives` unpacks tarballs (pre-POSIX tars included) and `--expand-gunzip` only strips the gzip layer, leaving `.pkg` bundles and their `.tar.Z` for NeXT's Installer. Data discs only: no boot blocks. The 8 slots are editable — resize / move / add / delete, `set-type` on the `p_type` name, and `set-bootable` moving `d_rootpartition` — with every edit converted out of 512-byte LBAs into porch-relative label sectors, overlap refused, and **only the copies the disk actually has** rewritten, so a NeXTSTEP/Intel disk keeps the PC boot sector that stands where its block-0 copy would be. **Full-disk backup works through the single-file-CHD layout**: the front porch — label copies, boot blocks and all — is copied verbatim, each partition lands at its declared offset, and an as-is restore is a byte copy; a resize on restore rewrites all four label copies. The per-partition layouts (zstd / gzip / raw / VHD) are refused with that reason, since restoring one would have to rewrite the label. The same rewrite runs before staging when a size changes at backup or export time. | +| NeXT | Yes | Yes | NeXT disk label (NeXTSTEP / OPENSTEP, black m68k hardware **and** NeXTSTEP/Intel). Up to 8 partitions, big-endian on both architectures, written as four checksummed copies at 512-byte blocks 0/15/30/45 — all four share one checksum, because it is computed with `dl_label_blkno` read as zero. Partition offsets are counted in the label's own `d_secsize` (1024 bytes on every disk we have) and measured from the end of a front porch, so `PartitionInfo` carries an explicit byte offset for them. Probed ahead of MBR because a NeXTSTEP/Intel disk also carries a valid `0xAA55` boot sector with an empty partition table. It is probed ahead of a Sun label too: CDs that also boot on SPARC (the NeXTSTEP 3.3 RISC disc and the OPENSTEP 4.2 Install CD) keep one at block 0 and their NeXT copies at blocks 16 / 30 / 48 instead, which is where a rewrite updates them. Surfaces its partitions to the existing big-endian UFS reader (browse / inspect / extract / edit). `rb-cli new hd next` writes a fresh `dlV3` label with the partitions you name (`4.3BSD`, `swap`, or any 8-byte type string), a 160-sector front porch, and the entry's optional NAME field as the partition's `p_mountpt`; `--heads` / `--sectors` set the recorded geometry and are counted in the label's own 1024-byte sectors. Fill it with `rb-cli new volume ufs-43bsd`, which writes the pre-4.4BSD UFS1 NeXTSTEP actually reads. **`rb-cli optical new next-ufs`** builds a NeXTSTEP CD-ROM image the same way — NeXT CDs are not ISO 9660 but this label in **2048-byte** sectors (32 x 64, an 80-sector porch) around one `4.3BSD` UFS with 2 KiB fragments, and for a disc the size of NeXT's own `NeXTTIME` CD every superblock word and the label's slot 0 come out identical to it. `--from-dir` fills it in one step; `--expand-archives` unpacks tarballs (pre-POSIX tars included) and `--expand-gunzip` only strips the gzip layer, leaving `.pkg` bundles and their `.tar.Z` for NeXT's Installer. Data discs only: no boot blocks. The 8 slots are editable — resize / move / add / delete, `set-type` on the `p_type` name, and `set-bootable` moving `d_rootpartition` — with every edit converted out of 512-byte LBAs into porch-relative label sectors, overlap refused, and **only the copies the disk actually has** rewritten, so a NeXTSTEP/Intel disk keeps the PC boot sector that stands where its block-0 copy would be. **Full-disk backup works through the single-file-CHD layout**: the front porch — label copies, boot blocks and all — is copied verbatim, each partition lands at its declared offset, and an as-is restore is a byte copy; a resize on restore rewrites all four label copies. The per-partition layouts (zstd / gzip / raw / VHD) are refused with that reason, since restoring one would have to rewrite the label. The same rewrite runs before staging when a size changes at backup or export time. | | Solaris-x86 | Yes | Yes | Solaris x86 nests a 16-slice VTOC in **sector 1 of an MBR partition** (type `0x82` on Solaris 2.x-9, `0xBF` on 10+) rather than replacing the MBR the way SPARC's Sun label does. Little-endian, slice offsets relative to the Solaris partition. Type `0x82` is shared with Linux swap, so detection requires the `0x600DDEEE` sanity word, `v_version == 1`, and slices that fit inside the partition. Surfaces the UFS slices to the existing reader; the disk's other MBR primaries and EBR logicals list after them, since a Solaris disk is still an MBR disk. `rb-cli new hd solaris-x86` writes both halves at once: an MBR with one bootable `0x82` entry starting at cylinder 1, and a full `struct dk_label` in its second sector — VTOC, geometry tail, `0xDABE` magic and the XOR checksum Solaris validates. Slices are cylinder-aligned; slice 2 is the whole-partition backup alias, slice 8 the boot cylinder and slice 9 the two alternates cylinders, so user slices start at the disk's fourth cylinder. The 16 slices are editable — resize / move / add / delete, and `set-type` on the VTOC tag by name or number — with absolute LBAs translated into partition-relative sectors, slices bounded by the label's own `dkl_ncyl` data area, partial overlap refused (the backup alias containing everything is not), and the checksum re-stamped so Solaris still accepts the label. `set-bootable` is refused with the reason: the boot bit is on the MBR entry hosting the label, not on the slices inside it. Backup treats the disk as the MBR it is (the Solaris partition rides as one body) and records the VTOC in `solaris_x86.json`. | | X68k | Yes | No (browse); writes whole tables from scratch | Sharp X68000 SASI/SCSI hard disks — Human68k's native scheme. 16-byte header plus 8 entries at byte 2048, big-endian, no magic number. Both geometries are auto-detected: SCSI (`X68SCSI1`, table at 0x800, 1024-byte sectors) and SASI (table at 0x400, 256-byte sectors), including custom-IPL game disks. `rb-cli new hd x68k` synthesizes a bootable disk with the Sharp IPL signature and a Human68k FAT volume. | | DSD | Yes | — (fixed floppy geometry) | Double-sided Acorn DFS (`.dsd`). Not a table on the disk: the two sides are stored track-interleaved, so the reader de-interleaves them and this scheme presents them as **two** Acorn DFS partitions — side 0 at byte 0, side 1 at half the image. Edits to either side re-interleave on save. | diff --git a/docs/cli-html-help/index.html b/docs/cli-html-help/index.html index f0801350..5f76f6d1 100644 --- a/docs/cli-html-help/index.html +++ b/docs/cli-html-help/index.html @@ -78,7 +78,7 @@

Verbs

  • tarArchive a filesystem (or a subtree) to a single `.tar.gz` / `.tar.zst` / `.tar`. Preserves exact case-sensitive names and real symlinks, so extracting on a case-insensitive host won't clobber files that differ only in case
  • terminalOpen an interactive rb-cli shell (rustyline-based REPL)
  • tuiLaunch the full-screen terminal UI (preview): a menu-driven ratatui app that runs anywhere rusty-backup does, including serial consoles and vintage terminals. Needs an interactive terminal
    • -
  • untarImport a `.tar.gz` / `.tar.zst` / `.tar` archive's contents INTO a filesystem in an image (the inverse of `tar`). Recreates the tree, streams files in, and recreates symlinks where the target FS supports them
    • +
  • untarImport a `.tar.gz` / `.tar.bz2` / `.tar.zst` / `.tar` archive's contents INTO a filesystem in an image (the inverse of `tar`). Recreates the tree, streams files in, and recreates symlinks where the target FS supports them
  • updateCheck for a newer release and (when built with `--features tui-update`) self-update. Without that feature it reports that updates weren't compiled in and prints the releases URL, exiting non-zero. Pass `--apply` to download and replace this binary in place
  • writeStream an image file onto a block device
  • xattrRead or edit a file's extended attributes (`list` / `set` / `rm`) — the scriptable form of the GUI and TUI File Info panel. SquashFS today; other xattr-bearing filesystems as their write side lands
    • diff --git a/docs/cli-html-help/optical-new-next-ufs.html b/docs/cli-html-help/optical-new-next-ufs.html index b240abaf..7a5bb7e0 100644 --- a/docs/cli-html-help/optical-new-next-ufs.html +++ b/docs/cli-html-help/optical-new-next-ufs.html @@ -22,7 +22,7 @@

    Options

    --from-dir
    Populate the disc from this host directory after formatting it. The directory's *contents* land at the volume root
    --expand-archives
    -
    With `--from-dir`: unpack tarballs (`.tar`, `.tar.gz`, `.tgz`, pre-POSIX tars included) into a directory named after each, instead of copying them
    +
    With `--from-dir`: unpack tarballs (`.tar`, `.tar.gz`, `.tgz`, `.tar.bz2`, pre-POSIX tars included) into a directory named after each, instead of copying them
    --expand-gunzip
    With `--from-dir`: strip one gzip layer only (`x.tar.gz` -> `x.tar`, `f.gz` -> `f`). With `--expand-archives` too, tarballs unpack fully and this takes the other `.gz` files
    --flatten-folders
    diff --git a/docs/cli-html-help/untar.html b/docs/cli-html-help/untar.html index a8a66660..ba88b46e 100644 --- a/docs/cli-html-help/untar.html +++ b/docs/cli-html-help/untar.html @@ -7,7 +7,7 @@

    rb-cli untar

    -

    Import a `.tar.gz` / `.tar.zst` / `.tar` archive's contents INTO a filesystem in an image (the inverse of `tar`). Recreates the tree, streams files in, and recreates symlinks where the target FS supports them

    +

    Import a `.tar.gz` / `.tar.bz2` / `.tar.zst` / `.tar` archive's contents INTO a filesystem in an image (the inverse of `tar`). Recreates the tree, streams files in, and recreates symlinks where the target FS supports them

    Usage

    Usage: untar [OPTIONS] <IMAGE> <ARCHIVE> [DEST]

    Arguments

    @@ -15,7 +15,7 @@

    Arguments

    <IMAGE>
    Image reference (`path` or `path@N` for the 1-based partition index)
    <ARCHIVE>
    -
    Host archive to import (`.tar.gz` / `.tar.zst` / `.tar`; the compression is detected from the file's contents, not its name)
    +
    Host archive to import (`.tar.gz` / `.tar.bz2` / `.tar.zst` / `.tar`; the compression is detected from the file's contents, not its name)
    <DEST>
    Destination directory inside the filesystem. Defaults to the root
    diff --git a/docs/cli-reference.md b/docs/cli-reference.md index ce9976d3..1d1cdd96 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -1585,7 +1585,7 @@ Usage: next-ufs [OPTIONS] - `--size` — Disc size (`600M`, `650M`, ...) or `auto` to fit `--from-dir`; rounded up to a 2048-byte sector. Past ~700M suits an emulator, not a burner - `--from-dir` — Populate the disc from this host directory after formatting it. The directory's *contents* land at the volume root -- `--expand-archives` — With `--from-dir`: unpack tarballs (`.tar`, `.tar.gz`, `.tgz`, pre-POSIX tars included) into a directory named after each, instead of copying them +- `--expand-archives` — With `--from-dir`: unpack tarballs (`.tar`, `.tar.gz`, `.tgz`, `.tar.bz2`, pre-POSIX tars included) into a directory named after each, instead of copying them - `--expand-gunzip` — With `--from-dir`: strip one gzip layer only (`x.tar.gz` -> `x.tar`, `f.gz` -> `f`). With `--expand-archives` too, tarballs unpack fully and this takes the other `.gz` files - `--flatten-folders` — With `--expand-archives`: unpack every archive into the volume root, not one folder each. Entries that already exist are skipped unless `--force` is given - `--force` — With `--from-dir`: overwrite entries that already exist rather than skipping them. Only meaningful alongside `--flatten-folders` @@ -2266,7 +2266,7 @@ Usage: tui ### `untar` -Import a `.tar.gz` / `.tar.zst` / `.tar` archive's contents INTO a filesystem in an image (the inverse of `tar`). Recreates the tree, streams files in, and recreates symlinks where the target FS supports them +Import a `.tar.gz` / `.tar.bz2` / `.tar.zst` / `.tar` archive's contents INTO a filesystem in an image (the inverse of `tar`). Recreates the tree, streams files in, and recreates symlinks where the target FS supports them ``` Usage: untar [OPTIONS] [DEST] @@ -2275,7 +2275,7 @@ Usage: untar [OPTIONS] [DEST] **Arguments** - `` — Image reference (`path` or `path@N` for the 1-based partition index) -- `` — Host archive to import (`.tar.gz` / `.tar.zst` / `.tar`; the compression is detected from the file's contents, not its name) +- `` — Host archive to import (`.tar.gz` / `.tar.bz2` / `.tar.zst` / `.tar`; the compression is detected from the file's contents, not its name) - `` — Destination directory inside the filesystem. Defaults to the root **Options** diff --git a/src/cli/mod.rs b/src/cli/mod.rs index ff6eecc7..b5f60414 100644 --- a/src/cli/mod.rs +++ b/src/cli/mod.rs @@ -115,7 +115,7 @@ pub enum Command { /// files that differ only in case. Tar(verbs::tar::TarArgs), - /// Import a `.tar.gz` / `.tar.zst` / `.tar` archive's contents INTO a + /// Import a `.tar.gz` / `.tar.bz2` / `.tar.zst` / `.tar` archive's contents INTO a /// filesystem in an image (the inverse of `tar`). Recreates the tree, /// streams files in, and recreates symlinks where the target FS /// supports them. diff --git a/src/cli/verbs/new_next_cdrom.rs b/src/cli/verbs/new_next_cdrom.rs index e591f204..e2112f81 100644 --- a/src/cli/verbs/new_next_cdrom.rs +++ b/src/cli/verbs/new_next_cdrom.rs @@ -42,7 +42,7 @@ pub struct NewNextCdromArgs { #[arg(long = "from-dir")] pub from_dir: Option, - /// With `--from-dir`: unpack tarballs (`.tar`, `.tar.gz`, `.tgz`, pre-POSIX + /// With `--from-dir`: unpack tarballs (`.tar`, `.tar.gz`, `.tgz`, `.tar.bz2`, pre-POSIX /// tars included) into a directory named after each, instead of copying them. #[arg(long = "expand-archives", requires = "from_dir")] pub expand_archives: bool, diff --git a/src/cli/verbs/untar.rs b/src/cli/verbs/untar.rs index 4a3619d7..94e639fb 100644 --- a/src/cli/verbs/untar.rs +++ b/src/cli/verbs/untar.rs @@ -1,4 +1,4 @@ -//! `rb-cli untar IMG[@N] ARCHIVE [DEST]` — import a `.tar.gz` / `.tar.zst` +//! `rb-cli untar IMG[@N] ARCHIVE [DEST]` — import a `.tar.gz` / `.tar.bz2` / `.tar.zst` //! / `.tar` archive's contents INTO a filesystem in a disk image. The //! inverse of `tar`. //! @@ -29,7 +29,7 @@ pub struct UntarArgs { /// Image reference (`path` or `path@N` for the 1-based partition index). pub image: ImageRef, - /// Host archive to import (`.tar.gz` / `.tar.zst` / `.tar`; the + /// Host archive to import (`.tar.gz` / `.tar.bz2` / `.tar.zst` / `.tar`; the /// compression is detected from the file's contents, not its name). pub archive: PathBuf, diff --git a/src/fs/dir_import.rs b/src/fs/dir_import.rs index 6332b0d0..c13573c3 100644 --- a/src/fs/dir_import.rs +++ b/src/fs/dir_import.rs @@ -40,7 +40,7 @@ pub struct DirImportOptions { /// Unpack archives found in the tree into a directory named after each, /// instead of copying them in as opaque files. Two families: /// - /// - **tar** (`.tar`, `.tar.gz`, `.tgz`, `.tar.zst`, and anything else + /// - **tar** (`.tar`, `.tar.gz`, `.tgz`, `.tar.bz2`, `.tar.zst`, and anything else /// carrying the `ustar` magic — IRIX `.tardist` files included), /// - **classic Mac** (`.sit`, `.sea`, `.cpt`, `.hqx`, `.mar`), which land /// with both forks and Finder type/creator intact. @@ -166,6 +166,7 @@ fn expanded_dir_name(file_name: &str) -> String { ".tgz", ".tzst", ".tbz2", + ".tbz", ".txz", ".tardist", ".gnutar", @@ -871,6 +872,56 @@ mod tests { b.into_inner().unwrap().finish().unwrap(); } + /// pbzip2 writes one bzip2 stream per block, so the reader must not stop after the first. + #[test] + fn a_multi_stream_tar_bz2_expands_into_its_own_folder() { + use std::io::Write; + let tmp = tempfile::tempdir().unwrap(); + let mut b = tar::Builder::new(Vec::new()); + let big = vec![b'x'; 300_000]; + for (name, data) in [("src/a.c", &b"int a;"[..]), ("src/big.dat", &big[..])] { + let mut h = tar::Header::new_gnu(); + h.set_size(data.len() as u64); + h.set_mode(0o644); + h.set_cksum(); + b.append_data(&mut h, name, data).unwrap(); + } + let tar_bytes = b.into_inner().unwrap(); + let mut f = std::fs::File::create(tmp.path().join("egcs-1.1.2.tar.bz2")).unwrap(); + let (head, tail) = tar_bytes.split_at(1024); + for part in [head, tail] { + let mut enc = bzip2::write::BzEncoder::new(Vec::new(), bzip2::Compression::fast()); + enc.write_all(part).unwrap(); + f.write_all(&enc.finish().unwrap()).unwrap(); + } + drop(f); + + let (files, _, bytes) = measure_dir(tmp.path(), true, false).unwrap(); + assert_eq!((files, bytes), (2, 300_006)); + + let img = blank_fat(); + let mut fs = + crate::fs::fat::FatFilesystem::open(std::io::Cursor::new(img), 0).expect("open"); + let root = Filesystem::root(&mut fs).expect("root"); + let opts = DirImportOptions { + expand_archives: true, + ..Default::default() + }; + let stats = import_dir(&mut fs, &root, tmp.path(), &opts, &|_| {}).expect("import"); + assert_eq!((stats.archives_expanded, stats.files), (1, 2)); + let find = |fs: &mut crate::fs::fat::FatFilesystem<_>, dir: &FileEntry, name: &str| { + fs.list_directory(dir) + .unwrap() + .into_iter() + .find(|e| e.name.eq_ignore_ascii_case(name)) + .unwrap_or_else(|| panic!("{name} missing")) + }; + let top = find(&mut fs, &root, "egcs-1.1.2"); + let src = find(&mut fs, &top, "src"); + let dat = find(&mut fs, &src, "big.dat"); + assert_eq!(fs.read_file(&dat, usize::MAX).unwrap(), big); + } + /// Default is to copy an archive in verbatim; `--expand-archives` unpacks /// it into a directory named after it. Both must be reachable, because /// which one is right depends on what the disc is for. @@ -1311,6 +1362,10 @@ mod tests { assert_eq!(expanded_dir_name("foo-1.2.tar.gz"), "foo-1.2"); assert_eq!(expanded_dir_name("bar.tardist"), "bar"); assert_eq!(expanded_dir_name("baz.tgz"), "baz"); + assert_eq!( + expanded_dir_name("egcs-1.1.2-2.s.tar.bz2"), + "egcs-1.1.2-2.s" + ); assert_eq!(expanded_dir_name("qux.TAR.GZ"), "qux"); // GNU tar's own names, common on NeXT archives. assert_eq!( diff --git a/src/fs/tar_import.rs b/src/fs/tar_import.rs index faf7fad6..82ed44ee 100644 --- a/src/fs/tar_import.rs +++ b/src/fs/tar_import.rs @@ -1,4 +1,4 @@ -//! Import a `.tar.gz` / `.tar.zst` / `.tar` archive's contents INTO a disk +//! Import a `.tar.gz` / `.tar.bz2` / `.tar.zst` / `.tar` archive's contents INTO a disk //! image's filesystem — the inverse of [`crate::fs::tar_export`]. //! //! Reads the archive (compression auto-detected from magic), recreates the @@ -36,32 +36,18 @@ pub use crate::fs::import_sink::{ /// Knobs for [`import_tar`]. Alias of the shared [`ImportOptions`]. pub type TarImportOptions = ImportOptions; -/// Cheap content sniff: does `path` look like a tar archive — plain, gzip-, or -/// zstd-compressed? Only a small prefix is (de)compressed to check for the tar +/// Cheap content sniff: does `path` look like a tar archive — plain, gzip-, +/// bzip2- or zstd-compressed? Only a small prefix is (de)compressed to check for the tar /// `ustar` magic at offset 257, so a gzip *disk image* (`.adz` / `.hdz`) is /// **not** mistaken for a tarball just because it's gzip. Used by the GUI to /// auto-route a dropped/added tar archive into the import flow. pub fn looks_like_tar_archive(path: &Path) -> bool { - let Ok(mut f) = File::open(path) else { + let Ok(stream) = open_tar_stream(path) else { return false; }; - let mut magic = [0u8; 4]; - let n = f.read(&mut magic).unwrap_or(0); - if f.seek(SeekFrom::Start(0)).is_err() { - return false; - } // Need to see at least the first tar header (512 B) -> read ~600 of the // (decompressed) stream so offset 257..262 is covered. - let prefix = if n >= 2 && magic[0] == 0x1f && magic[1] == 0x8b { - read_prefix(flate2::read::GzDecoder::new(f), 600) - } else if n >= 4 && magic == [0x28, 0xb5, 0x2f, 0xfd] { - match crate::rbformats::zstd_compat::decoder(f) { - Ok(d) => read_prefix(d, 600), - Err(_) => return false, - } - } else { - read_prefix(f, 600) - }; + let prefix = read_prefix(stream, 600); // ustar magic: "ustar\0" (POSIX) or "ustar " (GNU) — both start "ustar". (prefix.len() >= 262 && &prefix[257..262] == b"ustar") || is_v7_tar_header(&prefix) } @@ -241,6 +227,25 @@ fn header_size(field: &[u8]) -> u64 { tar_octal(field).unwrap_or(0) } +/// Open `path` as a tar stream, peeling one gzip / bzip2 / zstd layer chosen by magic. +pub(crate) fn open_tar_stream(path: &Path) -> Result> { + let mut file = + File::open(path).with_context(|| format!("opening archive {}", path.display()))?; + let mut magic = [0u8; 4]; + let n = file.read(&mut magic).unwrap_or(0); + file.seek(SeekFrom::Start(0)).context("rewind archive")?; + Ok(if n >= 2 && magic[0] == 0x1f && magic[1] == 0x8b { + Box::new(flate2::read::GzDecoder::new(file)) + } else if n >= 4 && magic == [0x28, 0xb5, 0x2f, 0xfd] { + Box::new(crate::rbformats::zstd_compat::decoder(file).context("init zstd decoder")?) + } else if n >= 4 && &magic[..3] == b"BZh" && (b'1'..=b'9').contains(&magic[3]) { + // Multi-stream: pbzip2 writes one bzip2 stream per block. + Box::new(bzip2::read::MultiBzDecoder::new(file)) + } else { + Box::new(file) + }) +} + fn read_prefix(mut r: impl Read, n: usize) -> Vec { let mut buf = vec![0u8; n]; let mut filled = 0; @@ -265,25 +270,10 @@ pub fn import_tar_from_path( opts: &TarImportOptions, progress: &dyn Fn(&TarImportStats), ) -> Result { - let mut file = - File::open(path).with_context(|| format!("opening archive {}", path.display()))?; - let mut magic = [0u8; 4]; - let n = file.read(&mut magic).unwrap_or(0); - file.seek(SeekFrom::Start(0)).context("rewind archive")?; - + let stream = open_tar_stream(path)?; let times = archive_times(path); efs.begin_bulk(); - let result = if n >= 2 && magic[0] == 0x1f && magic[1] == 0x8b { - let dec = flate2::read::GzDecoder::new(file); - import_tar_inner(efs, dest, dec, opts, progress, times) - } else if n >= 4 && magic == [0x28, 0xb5, 0x2f, 0xfd] { - match crate::rbformats::zstd_compat::decoder(file).context("init zstd decoder") { - Ok(dec) => import_tar_inner(efs, dest, dec, opts, progress, times), - Err(e) => Err(e), - } - } else { - import_tar_inner(efs, dest, file, opts, progress, times) - }; + let result = import_tar_inner(efs, dest, stream, opts, progress, times); efs.end_bulk(); result } @@ -345,22 +335,8 @@ pub fn import_tar_from_path_into( opts: &TarImportOptions, progress: &dyn Fn(&TarImportStats), ) -> Result { - let mut file = - File::open(path).with_context(|| format!("opening archive {}", path.display()))?; - let mut magic = [0u8; 4]; - let n = file.read(&mut magic).unwrap_or(0); - file.seek(SeekFrom::Start(0)).context("rewind archive")?; - - let times = archive_times(path); - if n >= 2 && magic[0] == 0x1f && magic[1] == 0x8b { - let dec = flate2::read::GzDecoder::new(file); - import_tar_inner(efs, dest, dec, opts, progress, times) - } else if n >= 4 && magic == [0x28, 0xb5, 0x2f, 0xfd] { - let dec = crate::rbformats::zstd_compat::decoder(file).context("init zstd decoder")?; - import_tar_inner(efs, dest, dec, opts, progress, times) - } else { - import_tar_inner(efs, dest, file, opts, progress, times) - } + let stream = open_tar_stream(path)?; + import_tar_inner(efs, dest, stream, opts, progress, archive_times(path)) } /// Total (files, dirs, content bytes) an archive would expand to, read from @@ -385,18 +361,7 @@ pub fn measure_tar_expanded(path: &Path) -> Result<(u64, u64, u64)> { Ok((files, dirs, bytes)) } - let mut file = - File::open(path).with_context(|| format!("opening archive {}", path.display()))?; - let mut magic = [0u8; 4]; - let n = file.read(&mut magic).unwrap_or(0); - file.seek(SeekFrom::Start(0)).context("rewind archive")?; - if n >= 2 && magic[0] == 0x1f && magic[1] == 0x8b { - tally(flate2::read::GzDecoder::new(file)) - } else if n >= 4 && magic == [0x28, 0xb5, 0x2f, 0xfd] { - tally(crate::rbformats::zstd_compat::decoder(file).context("init zstd decoder")?) - } else { - tally(file) - } + tally(open_tar_stream(path)?) } fn import_tar_inner( @@ -698,23 +663,7 @@ pub fn preflight_tar_from_path( path: &Path, opts: &TarImportOptions, ) -> Result { - let mut file = - File::open(path).with_context(|| format!("opening archive {}", path.display()))?; - let mut magic = [0u8; 4]; - let n = file.read(&mut magic).unwrap_or(0); - file.seek(SeekFrom::Start(0)).context("rewind archive")?; - - if n >= 2 && magic[0] == 0x1f && magic[1] == 0x8b { - preflight_tar(efs, flate2::read::GzDecoder::new(file), opts) - } else if n >= 4 && magic == [0x28, 0xb5, 0x2f, 0xfd] { - preflight_tar( - efs, - crate::rbformats::zstd_compat::decoder(file).context("init zstd decoder")?, - opts, - ) - } else { - preflight_tar(efs, file, opts) - } + preflight_tar(efs, open_tar_stream(path)?, opts) } /// Read-only preflight scan of a tar stream. Mirrors [`import_tar`]'s diff --git a/src/partition/mod.rs b/src/partition/mod.rs index 2bec8023..772e1abd 100644 --- a/src/partition/mod.rs +++ b/src/partition/mod.rs @@ -1057,18 +1057,8 @@ impl PartitionTable { )?)); } - // Sun disk label (SPARC Solaris / SunOS): magic 0xDABE at byte 508 plus - // a valid XOR checksum. Checked before MBR — the magic sits at a - // different offset than the MBR 0xAA55 (byte 510) and the checksum - // makes a false positive on a real MBR effectively impossible. - if SunDiskLabel::detect(&mbr_data) { - return Ok(PartitionTable::Sun(SunDiskLabel::parse(&mbr_data)?)); - } - - // NeXT disk label (NeXTSTEP / OPENSTEP). Four checksummed copies at - // 512-byte blocks 0/15/30/45. This runs before MBR parsing because a - // NeXTSTEP/Intel disk also carries a valid 0xAA55 boot sector with an - // empty partition table, which would otherwise win. + // NeXT label ahead of MBR (NeXTSTEP/Intel also has an empty 0xAA55 sector) and of Sun + // (a multi-architecture OPENSTEP CD keeps a Sun label at block 0 for SPARC boot). reader .seek(SeekFrom::Start(0)) .map_err(RustyBackupError::Io)?; @@ -1079,6 +1069,14 @@ impl PartitionTable { .seek(SeekFrom::Start(0)) .map_err(RustyBackupError::Io)?; + // Sun disk label (SPARC Solaris / SunOS): magic 0xDABE at byte 508 plus + // a valid XOR checksum. Checked before MBR — the magic sits at a + // different offset than the MBR 0xAA55 (byte 510) and the checksum + // makes a false positive on a real MBR effectively impossible. + if SunDiskLabel::detect(&mbr_data) { + return Ok(PartitionTable::Sun(SunDiskLabel::parse(&mbr_data)?)); + } + // Check for APM (Driver Descriptor Record signature 0x4552 at offset 0) let ddr_sig = u16::from_be_bytes([mbr_data[0], mbr_data[1]]); if ddr_sig == 0x4552 { diff --git a/src/partition/next.rs b/src/partition/next.rs index f9129a96..21695943 100644 --- a/src/partition/next.rs +++ b/src/partition/next.rs @@ -40,6 +40,13 @@ //! `p_base` from the **disk** origin even for a nested label, so the label's own //! answer is already absolute; a whole-disk NeXTSTEP label counts from its own //! block 0 with a zero container offset, which is why one expression covers both. +//! +//! **Multi-architecture CDs.** The OPENSTEP 4.2 Install CD boots on m68k, Intel +//! and SPARC, so block 0 holds a Sun label (and an HP-PA LIF header) instead of +//! a NeXT copy. Its NeXT copies sit at 512-byte blocks 16, 30 and 48 (bytes +//! 8192 / 15360 / 24576), and the two outside the standard set record +//! `dl_label_blkno` in 2048-byte units (4 and 12). [`RISC_LABEL_BLOCKS`] covers +//! them, and [`write_copies`] keeps those stamps when it rewrites a label. use byteorder::{BigEndian, ByteOrder}; use serde::{Deserialize, Serialize}; @@ -57,6 +64,12 @@ pub const NEXT_LABEL_V3: u32 = 0x646c_5633; /// 512-byte block numbers NeXTSTEP writes the four label copies at. pub const LABEL_BLOCKS: [u64; 4] = [0, 15, 30, 45]; +/// Extra copies on multi-architecture CDs, whose block 0 holds a Sun label (see module header). +pub const RISC_LABEL_BLOCKS: [u64; 2] = [16, 48]; + +/// Every block a label copy is looked for at: the standard four first. +const PROBE_BLOCKS: [u64; 6] = [0, 15, 30, 45, 16, 48]; + /// Bytes one label copy owns (15 × 512) — the spacing between copies. pub const LABEL_SPAN: usize = 15 * 512; @@ -451,7 +464,7 @@ pub fn present_copies(reader: &mut R) -> Vec { let Ok(disk_size) = reader.seek(SeekFrom::End(0)) else { return out; }; - for block in LABEL_BLOCKS { + for block in PROBE_BLOCKS { let offset = block * 512; if offset + LABEL_SPAN as u64 > disk_size || reader.seek(SeekFrom::Start(offset)).is_err() { continue; @@ -465,8 +478,7 @@ pub fn present_copies(reader: &mut R) -> Vec { } /// Write `copy` to each of `blocks`, stamping every copy's own -/// `dl_label_blkno`. The checksum reads that field as zero, so one stamp on -/// `copy` covers all four. +/// `dl_label_blkno`. The checksum reads that field as zero, so one stamp covers all. pub fn write_copies( out: &mut W, copy: &[u8], @@ -474,14 +486,23 @@ pub fn write_copies( ) -> std::io::Result<()> { let mut buf = copy.to_vec(); for &block in blocks { - BigEndian::write_u32(&mut buf[4..8], block as u32); + BigEndian::write_u32(&mut buf[4..8], recorded_blkno(block) as u32); out.seek(SeekFrom::Start(block * 512))?; out.write_all(&buf)?; } out.flush() } -/// Probe the four label copies and return the first that validates. +/// The `dl_label_blkno` a copy at 512-byte `block` carries; the RISC copies count 2048-byte units. +fn recorded_blkno(block: u64) -> u64 { + if RISC_LABEL_BLOCKS.contains(&block) { + block * 512 / 2048 + } else { + block + } +} + +/// Probe the label copies and return the first that validates. pub fn detect(reader: &mut R) -> Option { detect_at(reader, 0) } @@ -490,7 +511,7 @@ pub fn detect(reader: &mut R) -> Option { /// than at block 0 — Mac OS X Server 1.x nests one inside an APM slice. pub fn detect_at(reader: &mut R, base: u64) -> Option { let disk_size = reader.seek(SeekFrom::End(0)).ok()?; - for block in LABEL_BLOCKS { + for block in PROBE_BLOCKS { let offset = base + block * 512; if offset + LABEL_SPAN as u64 > disk_size { continue; diff --git a/src/partition/next_cd_builder.rs b/src/partition/next_cd_builder.rs index 403344f2..c9a73cee 100644 --- a/src/partition/next_cd_builder.rs +++ b/src/partition/next_cd_builder.rs @@ -356,6 +356,62 @@ mod tests { ); } + /// The OPENSTEP 4.2 Install CD shape: Sun label at block 0, NeXT copies at blocks 16 / 30 / 48. + #[test] + fn a_sun_labelled_multi_arch_disc_opens_through_its_next_label() { + use crate::partition::next::{present_copies, LABEL_SPAN, RISC_LABEL_BLOCKS}; + let (mut img, layout) = build(&NextCdOptions::new(24 * 1024 * 1024, "OPENSTEP_4.2")); + let copy = img[..LABEL_SPAN].to_vec(); + for block in LABEL_BLOCKS { + let at = (block * 512) as usize; + img[at..at + 0x230].fill(0); + } + // Sun label: magic at 508, and the XOR-of-words checksum at 510 cancels it. + img[0] = 0x80; + img[508..512].copy_from_slice(&[0xDA, 0xBE, 0x80 ^ 0xDA, 0xBE]); + let mut cur = Cursor::new(img); + assert!(matches!( + PartitionTable::detect(&mut cur).expect("detect"), + PartitionTable::Sun(_) + )); + + let blocks = [RISC_LABEL_BLOCKS[0], 30, RISC_LABEL_BLOCKS[1]]; + write_copies(&mut cur, ©, &blocks).unwrap(); + let mut found = present_copies(&mut cur); + found.sort_unstable(); + assert_eq!(found, blocks.to_vec()); + let img = cur.get_ref(); + let stamp = |b: u64| BigEndian::read_u32(&img[(b * 512) as usize + 4..][..4]); + assert_eq!([stamp(16), stamp(30), stamp(48)], [4, 30, 12]); + + let table = PartitionTable::detect(&mut cur).expect("detect"); + assert!(matches!(table, PartitionTable::Next(_)), "{table:?}"); + let parts = table.partitions(); + assert_eq!(parts.len(), 1, "{parts:?}"); + assert_eq!(parts[0].byte_offset(), layout.fs_offset); + + let mut fs = UfsFilesystem::open(cur, layout.fs_offset).expect("open"); + let root = fs.root().unwrap(); + fs.create_file( + &root, + "README", + &mut Cursor::new(b"hello".to_vec()), + 5, + &CreateFileOptions::default(), + ) + .unwrap(); + fs.sync_metadata().unwrap(); + let report = fsck_ufs(&mut fs).expect("fsck"); + assert!(report.errors.is_empty(), "{:?}", report.errors); + let names: Vec = fs + .list_directory(&root) + .unwrap() + .into_iter() + .map(|e| e.name) + .collect(); + assert!(names.iter().any(|n| n == "README"), "{names:?}"); + } + #[test] fn a_disc_below_the_floor_is_refused() { assert!(plan_next_ufs_cd(&NextCdOptions::new(1024 * 1024, "tiny")).is_err());