From 18b0926e287ac05279ecde00cc93c33fc10bcc7b Mon Sep 17 00:00:00 2001 From: Richie Caputo Date: Wed, 26 Aug 2026 14:13:27 -0400 Subject: [PATCH] Pin Bun 1.4.0, compose the asset matrix, share the download cache Bun 1.4.0 shipped six days before #13 pinned 1.3.14 in eleven places, and the pin was hard to move: `bundledChecksum` matched the version with a literal, and `bunArchiveSha256` was rejected unless `bunArchiveUrl` was set too -- contradicting its own scaladoc. - Replace the single-version checksum map and the `version == "1.3.14"` guard with a version -> asset -> sha256 table holding 1.3.14 and 1.4.0. All 24 entries verified against the official SHASUMS256.txt. - Compose asset names from (os, arch, musl, baseline) instead of enumerating, and reject combinations Bun does not publish. This adds the musl (Alpine) and -baseline (pre-AVX2 x64) builds the plugin previously could not download at all: 12 assets per version, not 6. musl is detected from the dynamic loader; baseline is opt-in via `bunUseBaseline` because the JVM cannot see CPU feature flags. - `bunArchiveSha256` now works alone, deriving the URL from `bunVersion`, so running an unbundled Bun is a two-line override. A custom `bunArchiveUrl` still requires a checksum so a mirror stays verified. - Add a checksum-keyed shared download cache. `downloadedBunExecutable` is a task on the trait, so a build with N Bun modules previously downloaded the ~35MB archive N times into N task dests. - `BunToolchainModule.DefaultBunVersion` is the single pin; `bunVersion` and `bunTypesVersion` both derive from it. `@types/bun` is published in lockstep with Bun, so the stale 1.3.11 pin can no longer drift. - Point the integration suite at the managed toolchain by default. It ran with MILL_BUN_USE_SYSTEM=true, so it tested whatever Bun the developer had rather than the version the plugin pins -- and broke outright once the pin moved ahead of the local Bun. The shared cache makes this affordable: the whole suite now uses one download. Unit 57 passing; integration 37 passing against managed Bun 1.4.0 with a single cached archive. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 2 +- .github/workflows/release.yml | 2 +- CHANGELOG.md | 5 +- README.md | 15 +- build.mill | 8 +- docs/MIGRATING-0.3.md | 2 +- .../resources/typescript-env/build.mill | 3 + .../BunManagedToolchainIntegrationTests.scala | 2 +- .../bun/BunTypeScriptIntegrationTests.scala | 3 +- millbun/src/mill/bun/BunToolchainModule.scala | 210 +++++++++++++++--- .../bun/BunTypeScriptModule.scala | 8 +- .../test/src/mill/bun/BunToolchainTests.scala | 107 ++++++++- 12 files changed, 315 insertions(+), 52 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d68d02a..e36b0e8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -21,7 +21,7 @@ jobs: - uses: oven-sh/setup-bun@v2 with: - bun-version: 1.3.14 + bun-version: 1.4.0 - name: Compile run: ./mill --no-server millbun.compile diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7fe604a..f84f2aa 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -31,7 +31,7 @@ jobs: - name: Set up Bun uses: oven-sh/setup-bun@v2 with: - bun-version: 1.3.14 + bun-version: 1.4.0 - name: Cache Coursier uses: actions/cache@v4 diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ca8938..1418fd4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Tag-driven release workflow for Maven Central publishing and GitHub releases. - Release runbook covering secrets, version sweep, annotated tags, and verification. -- Checksum-verified managed Bun 1.3.14 for macOS, Linux, and Windows on x64 and arm64. +- Checksum-verified managed Bun 1.4.0 for macOS, Linux, and Windows on x64 and arm64, including + musl (auto-detected) and `-baseline` (via `bunUseBaseline`) builds. +- `bunArchiveSha256` may be set on its own to run a Bun version with no bundled checksum; the + download URL is derived from `bunVersion` and the detected platform. - Strict text-lockfile workflow with `bunLock`, frozen installs, and actionable missing-lock failures. - Canonical `bundle`, `bundleFast`, `compileExecutable`, and `compileExecutables` task names. - Paired `BunScalaJSWebModule` and `BunTypeScriptWebModule` HTML workflows. diff --git a/README.md b/README.md index 8c9f970..aefe80d 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ It keeps Mill's task graph, caching, module relationships, Scala.js linker, and - Mill 1.1.5+ - JDK 17+ -Bun does not need to be installed. The plugin downloads and verifies Bun 1.3.14 by default on supported macOS, Linux, and Windows x64/arm64 hosts. Set `MILL_BUN_USE_SYSTEM=true` to opt into the Bun on `PATH`. +Bun does not need to be installed. The plugin downloads and checksum-verifies Bun 1.4.0 by default on macOS, Linux, and Windows x64/arm64. musl-based Linux (Alpine) is detected automatically; on x64 CPUs without AVX2, set `bunUseBaseline`. Set `MILL_BUN_USE_SYSTEM=true` to opt into the Bun on `PATH` instead. ## Scala.js quick start @@ -129,9 +129,18 @@ Resolution order is: 1. `bunExecutableOverride` 2. system `PATH` when `bunUseSystem` or `MILL_BUN_USE_SYSTEM=true` -3. checksum-verified managed Bun 1.3.14 +3. checksum-verified managed Bun 1.4.0 -Use `./mill app.bunDoctor` to print and validate the resolved executable, version, revision, mode, and linker. Custom mirrors or unbundled Bun versions must configure both `bunArchiveUrl` and `bunArchiveSha256`. +Use `./mill app.bunDoctor` to print and validate the resolved executable, version, revision, mode, selected release asset, and linker. + +To run a Bun version with no bundled checksum, set `bunVersion` and `bunArchiveSha256` — the download URL is derived from the version and the detected platform: + +```scala +def bunVersion = Task { "1.4.1" } +def bunArchiveSha256 = Task { Some("") } +``` + +Set `bunArchiveUrl` as well to download from a mirror; a mirror always requires `bunArchiveSha256` so the archive stays verified. ## Mixed Scala.js and TypeScript workspaces diff --git a/build.mill b/build.mill index efe4ec0..0219c36 100644 --- a/build.mill +++ b/build.mill @@ -37,9 +37,11 @@ object millbun extends ScalaModule, PublishModule { Map( "MILL_EXECUTABLE_PATH" -> millScript.toString, "COURSIER_REPOSITORIES" -> repos.mkString("|"), - // Existing fixtures exercise installed Bun behavior. Dedicated tests cover - // managed resolution and strict lockfile failures. - "MILL_BUN_USE_SYSTEM" -> Task.env.getOrElse("MILL_BUN_USE_SYSTEM", "true"), + // Default to the managed toolchain, so the suite exercises the same Bun the plugin + // pins rather than whatever the developer happens to have on PATH. The shared + // download cache means all fixtures reuse one archive. Export MILL_BUN_USE_SYSTEM=true + // to run against a local Bun instead. + "MILL_BUN_USE_SYSTEM" -> Task.env.getOrElse("MILL_BUN_USE_SYSTEM", "false"), "MILL_BUN_REQUIRE_LOCKFILE" -> Task.env.getOrElse("MILL_BUN_REQUIRE_LOCKFILE", "false") ) } diff --git a/docs/MIGRATING-0.3.md b/docs/MIGRATING-0.3.md index 78ada32..e38fa53 100644 --- a/docs/MIGRATING-0.3.md +++ b/docs/MIGRATING-0.3.md @@ -44,7 +44,7 @@ The old names are planned for removal at 1.0. ## Toolchain behavior -The default is now checksum-verified managed Bun 1.3.14. To preserve the old PATH behavior: +The default is now checksum-verified managed Bun 1.4.0. To preserve the old PATH behavior: ```bash export MILL_BUN_USE_SYSTEM=true diff --git a/millbun/integration/resources/typescript-env/build.mill b/millbun/integration/resources/typescript-env/build.mill index b1d1228..22ed549 100644 --- a/millbun/integration/resources/typescript-env/build.mill +++ b/millbun/integration/resources/typescript-env/build.mill @@ -13,5 +13,8 @@ object app extends BunTypeScriptModule { override def npmDevDeps = Task { Seq("typescript@5.7.3") } def bunProxy = Task.Source(moduleDir / "bun-proxy") override def managedBunExecutable = Task { Some(PathRef(bunProxy().path)) } + // The proxy execs whatever Bun is on PATH, so its --version reports the developer's Bun rather + // than bunVersion. This fixture tests env propagation, not the version pin. + override def bunVerifyVersion = Task { false } override def bunEnv = Task { Map("BUN_PROXY_MARKER" -> "present") } } diff --git a/millbun/integration/src/mill/bun/BunManagedToolchainIntegrationTests.scala b/millbun/integration/src/mill/bun/BunManagedToolchainIntegrationTests.scala index 8b85402..3aae15f 100644 --- a/millbun/integration/src/mill/bun/BunManagedToolchainIntegrationTests.scala +++ b/millbun/integration/src/mill/bun/BunManagedToolchainIntegrationTests.scala @@ -19,4 +19,4 @@ object BunManagedToolchainIntegrationTests extends TestSuite: assert(result.isSuccess) val executable = tester.out("app.bunExecutable").value[String] val version = os.proc(executable, "--version").call(stdout = os.Pipe).out.text().trim - assert(version == "1.3.14") + assert(version == "1.4.0") diff --git a/millbun/integration/src/mill/bun/BunTypeScriptIntegrationTests.scala b/millbun/integration/src/mill/bun/BunTypeScriptIntegrationTests.scala index f6a42db..ae52d1a 100644 --- a/millbun/integration/src/mill/bun/BunTypeScriptIntegrationTests.scala +++ b/millbun/integration/src/mill/bun/BunTypeScriptIntegrationTests.scala @@ -107,7 +107,8 @@ object BunTypeScriptIntegrationTests extends TestSuite { val packageJson = ujson.read(os.read(tester.workspacePath / "out" / "app" / "npmInstall.dest" / "package.json")) val devDeps = packageJson("devDependencies").obj - assert(devDeps("@types/bun").str == "1.3.11") + // Pinned, and pinned to the Bun we ship: @types/bun is published in lockstep with Bun. + assert(devDeps("@types/bun").str == BunToolchainModule.DefaultBunVersion) assert(devDeps("@types/bun").str != "latest") assert(!devDeps.contains("@types/node")) } diff --git a/millbun/src/mill/bun/BunToolchainModule.scala b/millbun/src/mill/bun/BunToolchainModule.scala index 9965eaf..32b0155 100644 --- a/millbun/src/mill/bun/BunToolchainModule.scala +++ b/millbun/src/mill/bun/BunToolchainModule.scala @@ -10,6 +10,14 @@ import java.util.zip.ZipInputStream object BunToolchainModule { + /** + * The Bun release this plugin is tested against and downloads by default. + * + * Referenced by `bunVersion`, by `bunTypesVersion` (`@types/bun` is published in lockstep), and + * by the tests, so a bump is a one-line change here rather than a hunt for literals. + */ + val DefaultBunVersion = "1.4.0" + private val ModeledPackageJsonFields = Set( "dependencies", "devDependencies", @@ -26,39 +34,102 @@ object BunToolchainModule { executableName: String ) - private val Bun1314Checksums = Map( - "bun-darwin-aarch64.zip" -> "d8b96221828ad6f97ac7ac0ab7e95872341af763001e8803e8267652c2652620", - "bun-darwin-x64.zip" -> "4183df3374623e5bab315c547cfa0974533cd457d86b73b639f7a87974cd6633", - "bun-linux-aarch64.zip" -> "a27ffb63a8310375836e0d6f668ae17fa8d8d18b88c37c821c65331973a19a3b", - "bun-linux-x64.zip" -> "951ee2aee855f08595aeec6225226a298d3fea83a3dcd6465c09cbccdf7e848f", - "bun-windows-aarch64.zip" -> "89841f5a57f2348b67ec0839b718f4bf4ea7d07c371c9ba4b77b6c790f918953", - "bun-windows-x64.zip" -> "0a0620930b6675d7ba440e81f4e0e00d3cfbe096c4b140d3fff02205e9e18922" + /** + * SHA-256 for every Bun release asset this plugin can download, keyed by version then asset. + * + * Sourced from the `SHASUMS256.txt` published with each Bun release. To add a version, append a + * complete entry — [[bundledChecksum]] deliberately has no fallback, so a partial table fails + * loudly on the missing platform rather than silently skipping verification. + */ + private val BundledChecksums: Map[String, Map[String, String]] = Map( + "1.3.14" -> Map( + "bun-darwin-aarch64.zip" -> "d8b96221828ad6f97ac7ac0ab7e95872341af763001e8803e8267652c2652620", + "bun-darwin-x64.zip" -> "4183df3374623e5bab315c547cfa0974533cd457d86b73b639f7a87974cd6633", + "bun-darwin-x64-baseline.zip" -> "3e35ad6f53971a9834bf9e6786e2adf72b5f1921cc9a9c5fde073d2972944076", + "bun-linux-aarch64.zip" -> "a27ffb63a8310375836e0d6f668ae17fa8d8d18b88c37c821c65331973a19a3b", + "bun-linux-aarch64-musl.zip" -> "b98e0ad3625c5c00d1d5b5ff55605c7adddbfae151861e68ade57b2d3b8703bb", + "bun-linux-x64.zip" -> "951ee2aee855f08595aeec6225226a298d3fea83a3dcd6465c09cbccdf7e848f", + "bun-linux-x64-baseline.zip" -> "a063908ae08b7852ca10939bbdc6ceed3ddabce8fb9402dce83d65d73b36e6c7", + "bun-linux-x64-musl.zip" -> "14bd9aedeebf1dba67e8def9531c89bc989ecfdf1de42e5bfcaf1b8cd9294719", + "bun-linux-x64-musl-baseline.zip" -> "56a7d6806cf155536c0178f0ea5fbd098e684fa509ebdb4fc0a7e19fb65382dc", + "bun-windows-aarch64.zip" -> "89841f5a57f2348b67ec0839b718f4bf4ea7d07c371c9ba4b77b6c790f918953", + "bun-windows-x64.zip" -> "0a0620930b6675d7ba440e81f4e0e00d3cfbe096c4b140d3fff02205e9e18922", + "bun-windows-x64-baseline.zip" -> "538f9c846355d9e847b2671bc00c47da4229a0befb24df3282b739770f3b475f" + ), + "1.4.0" -> Map( + "bun-darwin-aarch64.zip" -> "c669e97f6164e1c96e0701748db98dfa77492908cbd8394c7557134a735de381", + "bun-darwin-x64.zip" -> "1d0211b8f1dc991182344687ad15e72ee86f154845a5f7fa477994cd341dd9b0", + "bun-darwin-x64-baseline.zip" -> "da9b9f1b4ba766c6f299711f38dfaa98623e1ed9c40896aa53db803c52ec1fa0", + "bun-linux-aarch64.zip" -> "4b1a332ee861983eb93bcfe6f770fff94e3e31b2c388bdaea3c8ed35e58eed0e", + "bun-linux-aarch64-musl.zip" -> "576300ce33ff16ffcd455bf178c2f095f9df845c6cc3d0284ba1c96ca0e80473", + "bun-linux-x64.zip" -> "2d03fb5fb83ac8b567aca0a281b2ce1a1a19d488f56c2968d88c3f25e92fe452", + "bun-linux-x64-baseline.zip" -> "184fb4595f0d401a217cf7c78c1bc430ba83314dab7a8b94805babbf7fa7097f", + "bun-linux-x64-musl.zip" -> "83b5f12fd258dd8d4fdcaea65ede954366aa717dab399e20093ecab280d54e7a", + "bun-linux-x64-musl-baseline.zip" -> "618c4bc1f94b02337ee210003c0b7c066f11548a8cdc5109df10db043dc47ca2", + "bun-windows-aarch64.zip" -> "f473bfe2df73ee770548c93dd5d380aea7120c218ec2aa1afdd0bbba7bf18c47", + "bun-windows-x64.zip" -> "e6f093d39da486b20262ca8cdd5ed6a9e8bc9c2f275b78e6d3a0c5b28cc95901", + "bun-windows-x64-baseline.zip" -> "b929c54a9badb104a16dedd23aab6152c86793ae653d4e6b13983ffd0c882a66" + ) ) - private[bun] def distribution(osName: String, architecture: String): Either[String, Distribution] = { + /** Versions with a complete bundled checksum table, for diagnostics. */ + private[bun] def bundledVersions: Seq[String] = BundledChecksums.keys.toSeq.sorted + + /** + * Compose a Bun release asset name from the platform axes. + * + * Bun names assets `bun-{os}-{arch}[-musl][-baseline].zip`. The two modifiers are constrained: + * `-musl` exists only for Linux, and `-baseline` (for x64 CPUs without AVX2) only for x64. This + * returns `Left` for combinations Bun does not publish rather than constructing a 404 URL. + */ + private[bun] def distribution( + osName: String, + architecture: String, + musl: Boolean = false, + baseline: Boolean = false + ): Either[String, Distribution] = { val osPart = osName.toLowerCase match { case name if name.contains("mac") || name.contains("darwin") => Right("darwin") - case name if name.contains("linux") => Right("linux") + case name if name.contains("linux") => Right("linux") case name if name.contains("windows") => Right("windows") case other => Left(s"Unsupported operating system '$other'") } val archPart = architecture.toLowerCase match { - case "aarch64" | "arm64" => Right("aarch64") + case "aarch64" | "arm64" => Right("aarch64") case "amd64" | "x86_64" | "x64" => Right("x64") - case other => Left(s"Unsupported architecture '$other'") + case other => Left(s"Unsupported architecture '$other'") } for { os <- osPart arch <- archPart + _ <- + if (musl && os != "linux") Left(s"Bun publishes no musl build for '$os'") + else Right(()) + _ <- + if (baseline && arch != "x64") Left(s"Bun publishes no baseline build for '$arch'") + else Right(()) } yield Distribution( - assetName = s"bun-$os-$arch.zip", + assetName = s"bun-$os-$arch${if (musl) "-musl" else ""}${if (baseline) "-baseline" else ""}.zip", executableName = if (os == "windows") "bun.exe" else "bun" ) } + /** + * Detect a musl-based Linux (Alpine and friends), where the glibc build cannot run. + * + * Probing for the musl dynamic loader is more reliable than parsing `ldd --version`, which musl + * writes to stderr and glibc to stdout. + */ + private[bun] def detectMusl(root: os.Path = os.root): Boolean = + try + val libDir = root / "lib" + (os.exists(libDir) && os.list(libDir).exists(p => p.last.startsWith("ld-musl-"))) || + os.exists(root / "etc" / "alpine-release") + catch case _: Exception => false + private[bun] def bundledChecksum(version: String, assetName: String): Option[String] = - if (version == "1.3.14") Bun1314Checksums.get(assetName) else None + BundledChecksums.get(version).flatMap(_.get(assetName)) private[bun] def sha256(path: os.Path): String = { val digest = MessageDigest.getInstance("SHA-256") @@ -118,6 +189,30 @@ object BunToolchainModule { } } + /** + * Move a verified executable into the shared cache, tolerating a concurrent writer. + * + * Mill evaluates modules in parallel, so two tasks can extract the same archive at once. Both + * move into the same destination; whoever loses the race simply uses the winner's file, which is + * byte-identical because the path is keyed by checksum. + */ + private[bun] def publishToCache(staged: os.Path, cached: os.Path): os.Path = { + os.makeDir.all(cached / os.up) + try + java.nio.file.Files.move( + staged.toNIO, + cached.toNIO, + java.nio.file.StandardCopyOption.ATOMIC_MOVE + ) + cached + catch + case _: java.nio.file.FileAlreadyExistsException => cached + case _: java.nio.file.AtomicMoveNotSupportedException => + // Cache on a different filesystem than the task dest: fall back to a plain copy. + os.copy.over(staged, cached, createFolders = true) + cached + } + /** Parse package.json-style `name@specifier` declarations without slicing scoped names incorrectly. */ private[bun] def parseDependency(input: String): Either[String, NpmDependency] = { val trimmed = input.trim @@ -241,7 +336,23 @@ object BunToolchainModule { trait BunToolchainModule extends Module { /** Tested Bun release downloaded by default when no override is configured. */ - def bunVersion: T[String] = Task { "1.3.14" } + def bunVersion: T[String] = Task { BunToolchainModule.DefaultBunVersion } + + /** + * Download the `-baseline` build, for x64 CPUs without AVX2. + * + * Not auto-detected: the JVM cannot see CPU feature flags, and guessing wrong surfaces as an + * illegal-instruction crash rather than a diagnosable error. Set this when Bun aborts on startup + * with `SIGILL` on older x64 hardware. + */ + def bunUseBaseline: T[Boolean] = Task { false } + + /** + * Download the musl build, required on Alpine and other musl-based Linux distributions. + * + * Auto-detected from the presence of the musl dynamic loader; override to force either build. + */ + def bunUseMusl: T[Boolean] = Task.Input { BunToolchainModule.detectMusl() } /** Resolve Bun from PATH instead of using the managed distribution. */ def bunUseSystem: T[Boolean] = Task { @@ -261,9 +372,29 @@ trait BunToolchainModule extends Module { /** Optional managed distribution mirror. Must be paired with [[bunArchiveSha256]]. */ def bunArchiveUrl: T[Option[String]] = Task { None } - /** SHA-256 for [[bunArchiveUrl]], or for an unbundled Bun version. */ + /** + * SHA-256 of the Bun archive to download. + * + * Set this alone to run a [[bunVersion]] with no bundled checksum — the URL is derived from the + * version and the detected platform. Set it together with [[bunArchiveUrl]] to use a mirror. + */ def bunArchiveSha256: T[Option[String]] = Task { None } + /** + * Directory holding managed Bun downloads, shared by every module in the build. + * + * `downloadedBunExecutable` is a task on this trait, so without a shared cache a build with N + * Bun modules downloads the ~35 MB archive N times into N separate `Task.dest` directories. + * Entries are keyed by the archive's verified SHA-256, so a partial or tampered download can + * never be reused and two modules on the same pin share one file. + * + * Lives outside the workspace, so Mill's filesystem checker does not restrict it. + */ + def bunDownloadCacheDir: T[os.Path] = Task.Input { + Task.env.get("MILL_BUN_CACHE_DIR").filter(_.nonEmpty).map(os.Path(_)) + .getOrElse(os.home / ".cache" / "mill-bun") + } + /** Verify that an override/system executable matches [[bunVersion]]. */ def bunVerifyVersion: T[Boolean] = Task { true } @@ -369,15 +500,17 @@ trait BunToolchainModule extends Module { val version = bunVersion() val dist = BunToolchainModule.distribution( System.getProperty("os.name", "unknown"), - System.getProperty("os.arch", "unknown") + System.getProperty("os.arch", "unknown"), + musl = bunUseMusl(), + baseline = bunUseBaseline() ).fold( message => Task.fail(s"$message. Set bunExecutableOverride or bunUseSystem."), identity ) val customUrl = bunArchiveUrl() val customChecksum = bunArchiveSha256() - if (customUrl.isDefined != customChecksum.isDefined) { - Task.fail("bunArchiveUrl and bunArchiveSha256 must be configured together.") + if (customUrl.isDefined && customChecksum.isEmpty) { + Task.fail("bunArchiveUrl requires bunArchiveSha256 so the mirrored archive stays verified.") } val url = customUrl.getOrElse( s"https://github.com/oven-sh/bun/releases/download/bun-v$version/${dist.assetName}" @@ -385,19 +518,30 @@ trait BunToolchainModule extends Module { val checksum = customChecksum .orElse(BunToolchainModule.bundledChecksum(version, dist.assetName)) .getOrElse(Task.fail( - s"No bundled checksum for Bun $version (${dist.assetName}). Set bunArchiveUrl and bunArchiveSha256." + s"No bundled checksum for Bun $version (${dist.assetName}). " + + s"Bundled versions are ${BunToolchainModule.bundledVersions.mkString(", ")}. " + + "Set bunArchiveSha256 to the archive's SHA-256 to use this version anyway." )) .toLowerCase - val archive = Task.dest / dist.assetName - val executable = Task.dest / dist.executableName - BunToolchainModule.download(url, archive) - val actual = BunToolchainModule.sha256(archive) - if (actual != checksum) { - Task.fail(s"Bun archive checksum mismatch for $url: expected $checksum, received $actual") + if (!checksum.matches("[0-9a-f]{64}")) { + Task.fail(s"bunArchiveSha256 must be 64 hexadecimal characters, received '$checksum'.") + } + + // Keyed by the verified checksum, so a cache hit is proof of the right bytes. + val cached = bunDownloadCacheDir() / checksum / dist.executableName + if (os.exists(cached)) PathRef(cached) + else { + val archive = Task.dest / dist.assetName + val staged = Task.dest / dist.executableName + BunToolchainModule.download(url, archive) + val actual = BunToolchainModule.sha256(archive) + if (actual != checksum) { + Task.fail(s"Bun archive checksum mismatch for $url: expected $checksum, received $actual") + } + BunToolchainModule.extractExecutable(archive, dist.executableName, staged) + PathRef(BunToolchainModule.publishToCache(staged, cached)) } - BunToolchainModule.extractExecutable(archive, dist.executableName, executable) - PathRef(executable) } private def verifyBunVersion(executable: String, expected: String, verify: Boolean): Unit = { @@ -451,6 +595,18 @@ trait BunToolchainModule extends Module { println(s"Bun version: ${bunVersion()}") if (revision.nonEmpty) println(s"Bun revision: $revision") println(s"Bun linker: ${bunLinker()}") + if (mode == "managed") { + val dist = BunToolchainModule.distribution( + System.getProperty("os.name", "unknown"), + System.getProperty("os.arch", "unknown"), + musl = bunUseMusl(), + baseline = bunUseBaseline() + ) + println(s"Bun asset: ${dist.fold(identity, _.assetName)}") + println(s"Bun libc: ${if (bunUseMusl()) "musl" else "glibc"}") + println(s"Bun baseline: ${bunUseBaseline()}") + } + println(s"Bun lockfile: ${bunLockfile().fold("none")(_.path.toString)}") } /** Run a Bun command. All task values must be resolved before calling this. */ diff --git a/millbun/src/mill/javascriptlib/bun/BunTypeScriptModule.scala b/millbun/src/mill/javascriptlib/bun/BunTypeScriptModule.scala index 33da9e9..170e922 100644 --- a/millbun/src/mill/javascriptlib/bun/BunTypeScriptModule.scala +++ b/millbun/src/mill/javascriptlib/bun/BunTypeScriptModule.scala @@ -61,8 +61,12 @@ trait BunTypeScriptModule extends TypeScriptModule with BunToolchainModule with /** Node ambient types used for node-targeted Bun builds. */ def nodeTypesVersion: T[String] = Task { "22.10.9" } - /** Bun ambient types used for bun-targeted Bun builds. */ - def bunTypesVersion: T[String] = Task { "1.3.11" } + /** Bun ambient types used for bun-targeted Bun builds. + * + * `@types/bun` is published in lockstep with Bun itself, so this tracks [[bunVersion]] by + * default and the two cannot drift. + */ + def bunTypesVersion: T[String] = Task { bunVersion() } /** Ambient runtime types aligned to the configured Bun target. */ protected def ambientTypeDeps: T[Seq[String]] = Task { diff --git a/millbun/test/src/mill/bun/BunToolchainTests.scala b/millbun/test/src/mill/bun/BunToolchainTests.scala index 9774c7e..d1ac89c 100644 --- a/millbun/test/src/mill/bun/BunToolchainTests.scala +++ b/millbun/test/src/mill/bun/BunToolchainTests.scala @@ -27,19 +27,104 @@ object BunToolchainTests extends TestSuite: assert(BunToolchainModule.distribution("Plan 9", "x64").isLeft) assert(BunToolchainModule.distribution("Linux", "riscv64").isLeft) - test("bundles checksums for every supported Bun 1.3.14 asset"): - val assets = Seq( - "bun-darwin-aarch64.zip", - "bun-darwin-x64.zip", - "bun-linux-aarch64.zip", - "bun-linux-x64.zip", - "bun-windows-aarch64.zip", - "bun-windows-x64.zip" + test("composes musl and baseline asset names"): + assert( + BunToolchainModule.distribution("Linux", "x64", musl = true) == Right( + BunToolchainModule.Distribution("bun-linux-x64-musl.zip", "bun") + ) ) - assets.foreach: asset => - val checksum = BunToolchainModule.bundledChecksum("1.3.14", asset) + assert( + BunToolchainModule.distribution("Linux", "x64", baseline = true) == Right( + BunToolchainModule.Distribution("bun-linux-x64-baseline.zip", "bun") + ) + ) + assert( + BunToolchainModule.distribution("Linux", "x64", musl = true, baseline = true) == Right( + BunToolchainModule.Distribution("bun-linux-x64-musl-baseline.zip", "bun") + ) + ) + assert( + BunToolchainModule.distribution("Windows 11", "x64", baseline = true) == Right( + BunToolchainModule.Distribution("bun-windows-x64-baseline.zip", "bun.exe") + ) + ) + + test("rejects modifier combinations Bun does not publish"): + // musl is Linux-only, baseline is x64-only. + assert(BunToolchainModule.distribution("Mac OS X", "x64", musl = true).isLeft) + assert(BunToolchainModule.distribution("Windows 11", "x64", musl = true).isLeft) + assert(BunToolchainModule.distribution("Linux", "aarch64", baseline = true).isLeft) + assert(BunToolchainModule.distribution("Mac OS X", "aarch64", baseline = true).isLeft) + + test("bundles a checksum for every asset of every pinned version"): + // Every combination distribution() can produce must be downloadable, or the managed + // toolchain fails on a platform we claim to support. + val platforms = Seq( + ("Mac OS X", "aarch64"), + ("Mac OS X", "x64"), + ("Linux", "aarch64"), + ("Linux", "x64"), + ("Windows 11", "aarch64"), + ("Windows 11", "x64") + ) + val assets = + for + (os, arch) <- platforms + musl <- Seq(false, true) + baseline <- Seq(false, true) + dist <- BunToolchainModule.distribution(os, arch, musl, baseline).toOption + yield dist.assetName + + assert(assets.distinct.size == 12) + for + version <- BunToolchainModule.bundledVersions + asset <- assets.distinct + do + val checksum = BunToolchainModule.bundledChecksum(version, asset) assert(checksum.exists(_.matches("[0-9a-f]{64}"))) - assert(BunToolchainModule.bundledChecksum("1.3.15", assets.head).isEmpty) + + test("pins the versions the docs and CI claim"): + assert(BunToolchainModule.bundledVersions == Seq("1.3.14", "1.4.0")) + // The default must be one we ship checksums for, or the managed path cannot work offline. + assert(BunToolchainModule.bundledVersions.contains(BunToolchainModule.DefaultBunVersion)) + + test("unknown versions have no bundled checksum"): + assert(BunToolchainModule.bundledChecksum("1.3.15", "bun-linux-x64.zip").isEmpty) + + test("musl detection probes the loader and the alpine marker"): + val glibc = os.temp.dir() + os.makeDir.all(glibc / "lib") + os.write(glibc / "lib" / "ld-linux-x86-64.so.2", "") + assert(!BunToolchainModule.detectMusl(glibc)) + + val musl = os.temp.dir() + os.makeDir.all(musl / "lib") + os.write(musl / "lib" / "ld-musl-x86_64.so.1", "") + assert(BunToolchainModule.detectMusl(musl)) + + val alpine = os.temp.dir() + os.makeDir.all(alpine / "etc") + os.write(alpine / "etc" / "alpine-release", "3.20.0") + assert(BunToolchainModule.detectMusl(alpine)) + + assert(!BunToolchainModule.detectMusl(os.temp.dir())) + + test("publishing to the download cache is idempotent under a race"): + val root = os.temp.dir() + val cached = root / "cache" / "abc123" / "bun" + + val first = root / "first" / "bun" + os.write(first, "bun-binary", createFolders = true) + assert(BunToolchainModule.publishToCache(first, cached) == cached) + assert(os.read(cached) == "bun-binary") + assert(!os.exists(first)) + + // A second module extracting the same checksum concurrently must not fail, and must not + // clobber the entry another task may already be executing. + val second = root / "second" / "bun" + os.write(second, "bun-binary", createFolders = true) + assert(BunToolchainModule.publishToCache(second, cached) == cached) + assert(os.read(cached) == "bun-binary") test("computes SHA-256"): val file = os.temp(contents = "hello")