bundler: run 66 ledger (#1989)

Author: claude

entries/bundler/20261102T150627Z.md

@@ -0,0 +1,31 @@
+[agent] 2026-11-02: Bundler (RubyGems) bug-hunt run
+
+**Tested:** main `cbbbdb9` (unchanged since run 60; no gem code changes), release v5.0.1. Linux only: Ruby 3.3.6 with Bundler 2.6.9 and 4.1.0.beta3 (still the newest; 4.1.0 final hasn't shipped). Harness: the repo's `e2e_vendor_gem_build.rs`, plus scratch tests in `e2e_redirect_gem_build.rs` (not committed): mock upstream + patch registry, real `gem build` fixtures, real `bundle` commands and frozen fresh installs.
+
+**Re-triage**
+- #1831: still open; PR #1841 still unmerged, so no comment.
+- #1109: still open (`agent:needs-human`); no gem code changes, so no comment.
+- No human replies on #316.
+
+**New in Bundler 4.1.0.beta3 (2026-10-28):** `override` takes `with:` / `in:` (was `to:` / `from:`), and the lock records overrides in a new `OVERRIDES` section between DEPENDENCIES and CHECKSUMS. With `override "other-dep", with: "vuln-gem", in: "parent"`, the lock keeps `parent`'s original edge (`other-dep`), and no spec sub-line names `vuln-gem`. 4.1 also adds a `CONTENT ADDRESSES` section, which needs RubyGems 4.1 (`Gem::ContentAddress`); it's now on the backlog.
+
+**Cells**
+
+| Cell | 2.6.9 | 4.1.0.beta3 |
+| --- | --- | --- |
+| Repo vendored suite `e2e_vendor_gem_build` (`--ignored`, `REQUIRED=1`) | pass (11) | pass (11) |
+| Hosted gem reached only through `override …, with: "vuln-gem", in: "parent"`: scan appends the block, converging install, frozen fresh install, OVERRIDES kept intact | n/a | pass |
+| Same, `rollback` / `remove` → byte-exact pair | n/a | **fail #1989** (×2 each): a new top-level `gem "vuln-gem", "1.0.0"` plus `vuln-gem (= 1.0.0)` in DEPENDENCIES; frozen install passes, and `bundle lock --update vuln-gem` can't move past 1.0.0 |
+| Control: `parent` depends on `vuln-gem` normally, `rollback` / `remove` | n/a | pass (byte-exact) |
+
+**Filed:** #1989 (https://github.com/SocketDev/socket-patch/issues/1989). **Commented / closed:** nothing.
+
+**Ruled out:** nothing new. #457 (closed) had the same symptom but a different cause (a missing blank line), so #1989 isn't a duplicate.
+
+**Probes:** none (leftover `bughunt/bundler/*` branches still block them).
+
+**Next**
+1. Re-check #1989 and #1831 (#1841) once fixed.
+2. Other `OVERRIDES` shapes on 4.1: `override "vuln-gem", version:` / `:ignore_upper` on the hosted gem, `with: nil` dropping `tiny-dep`, CHECKSUMS + rubygems.org spelling, and vendored mode over an override-reached gem.
+3. `CONTENT ADDRESSES` with RubyGems 4.1 (`gem update --system 4.1.0.beta3`) over a hosted / vendored lock.
+4. 4.1.0 final when it ships; 2.2 / 2.4 legs on the next main that touches gem code.

state/bundler.md

@@ -1,6 +1,6 @@
 [agent] Progress ledger for the scheduled Bundler (RubyGems) bug-hunt routine (label pm:bundler).
 
-Last updated: 2026-11-02 (run 65), main `cbbbdb9` (unchanged since run 60; no gem code changes since `b291caa`), latest release v5.0.1 (npm; no git tags). Newest Bundler: 4.0.23 stable, 4.1.0.beta3. Run 65 covered `bundle update --all`, `bundle lock --update vuln-gem`, `bundle lock --update tiny-dep` and `bundle lock --patch` over a converged hosted lock after newer upstream `tiny-dep 1.0.1` / `vuln-gem 1.0.1` were published, with and without CHECKSUMS, on the 127.0.0.1 spelling and the rubygems.org spelling + `BUNDLE_MIRROR__HTTPS://RUBYGEMS__ORG/`, then re-scan, frozen fresh install PATCHED, `vex`, `rollback` and a frozen install of the restored pair, on 2.6.9, 4.0.23 and 4.1.0.beta3 (all pass; see the run-65 entry). Run 64 covered `bundle outdated` / `bundle update --conservative` / `bundle add` over a converged hosted lock followed by re-scan, frozen install and `rollback` / `remove` on 2.6.9, 4.0.23 and 4.1.0.beta3 (all pass), and the stale-install guard with an excluded group plus a system-home copy (conservative warning, documented; see Known non-bugs). Run 63 covered `bundle lock --remove-platform` / `--normalize-platforms` over a converged CHECKSUMS redirect, `bundle remove` of the hosted gem followed by `rollback` / `remove`, and `BUNDLE_ONLY` / `BUNDLE_WITHOUT` on 2.6.9, 4.0.23 and 4.1.0.beta3 (all pass; see the run-63 table). Run 62 covered `bundle lock --add-platform` after a converged CHECKSUMS redirect, a superseding generation over a user-checksummed lock on 4.1.0.beta3 and 2.6.9 (backlog item 5), and `BUNDLE_WITHOUT` excluding the hosted gem (all pass; see the run-62 table). Run 61 covered `bundle lock --add-checksums` on an unconverged redirect, a superseding patch generation over a user-checksummed lock, and the transitive unwind with a trailing blank line (all pass; see the run-61 table). Run 60 covered the hosted lifecycle when a user adds CHECKSUMS after a converged redirect (`bundle lock --add-checksums`), on 2.6.9, 4.0.23 and 4.1.0.beta3, via a scratch test in `e2e_redirect_gem_build.rs` (upstream respelled `https://rubygems.org` in the pair after the scan, `BUNDLE_MIRROR__HTTPS://RUBYGEMS__ORG/=<mock>/upstream/` for Bundler, `SOCKET_RUBYGEMS_URL=<mock>/upstream` for socket-patch). Run 59 closed backlog item 4 (Bundler 4.0.23 phantom platform lock entry: re-scan, remedy, re-converge, `vex`, `rollback` / `remove`) and the 2.6.9 hosted-unwind direct-dep and `group` cells. Run 59 used a standalone harness instead of a scratch Rust test: a Python mock (upstream compact index with ruby + `x86_64-linux` variants, patch registry, patches API) plus real `gem build` fixtures. Pass `--patch-server-url <mock>` to `rollback` / `remove` / `vex`, or a 127.0.0.1 pin isn't treated as hosted and the commands report `manifest_not_found`. Ruby 3.1.6 is in the sandbox (`/opt/rbenv/versions/3.1.6/bin`); `gem install bundler -v 1.17.3` / `2.1.4` there and export `BUNDLER_VERSION` + `SOCKET_PATCH_BUNDLER_E2E_VERSION`. A fresh container's Ruby 3.3.6 only has 2.5.22 / 4.0.18, so `gem install bundler -v 2.6.9` / `4.0.23` / `4.1.0.beta3` first and export `BUNDLER_VERSION`. Bundler 2.6 has no `install -q` (use `--quiet`).
+Last updated: 2026-11-02 (run 66), main `cbbbdb9` (unchanged since run 60; no gem code changes since `b291caa`), latest release v5.0.1 (npm; no git tags). Newest Bundler: 4.0.23 stable, 4.1.0.beta3. Run 66 found #1989: on 4.1.0.beta3, a hosted gem reached only through an `override …, with:, in:` (the new `OVERRIDES` lock section) comes back from `rollback` / `remove` as a new top-level exact pin. The repo vendored suite passed on 2.6.9 and 4.1.0.beta3 (see the run-66 table). Run 65 covered `bundle update --all`, `bundle lock --update vuln-gem`, `bundle lock --update tiny-dep` and `bundle lock --patch` over a converged hosted lock after newer upstream `tiny-dep 1.0.1` / `vuln-gem 1.0.1` were published, with and without CHECKSUMS, on the 127.0.0.1 spelling and the rubygems.org spelling + `BUNDLE_MIRROR__HTTPS://RUBYGEMS__ORG/`, then re-scan, frozen fresh install PATCHED, `vex`, `rollback` and a frozen install of the restored pair, on 2.6.9, 4.0.23 and 4.1.0.beta3 (all pass; see the run-65 entry). Run 64 covered `bundle outdated` / `bundle update --conservative` / `bundle add` over a converged hosted lock followed by re-scan, frozen install and `rollback` / `remove` on 2.6.9, 4.0.23 and 4.1.0.beta3 (all pass), and the stale-install guard with an excluded group plus a system-home copy (conservative warning, documented; see Known non-bugs). Run 63 covered `bundle lock --remove-platform` / `--normalize-platforms` over a converged CHECKSUMS redirect, `bundle remove` of the hosted gem followed by `rollback` / `remove`, and `BUNDLE_ONLY` / `BUNDLE_WITHOUT` on 2.6.9, 4.0.23 and 4.1.0.beta3 (all pass; see the run-63 table). Run 62 covered `bundle lock --add-platform` after a converged CHECKSUMS redirect, a superseding generation over a user-checksummed lock on 4.1.0.beta3 and 2.6.9 (backlog item 5), and `BUNDLE_WITHOUT` excluding the hosted gem (all pass; see the run-62 table). Run 61 covered `bundle lock --add-checksums` on an unconverged redirect, a superseding patch generation over a user-checksummed lock, and the transitive unwind with a trailing blank line (all pass; see the run-61 table). Run 60 covered the hosted lifecycle when a user adds CHECKSUMS after a converged redirect (`bundle lock --add-checksums`), on 2.6.9, 4.0.23 and 4.1.0.beta3, via a scratch test in `e2e_redirect_gem_build.rs` (upstream respelled `https://rubygems.org` in the pair after the scan, `BUNDLE_MIRROR__HTTPS://RUBYGEMS__ORG/=<mock>/upstream/` for Bundler, `SOCKET_RUBYGEMS_URL=<mock>/upstream` for socket-patch). Run 59 closed backlog item 4 (Bundler 4.0.23 phantom platform lock entry: re-scan, remedy, re-converge, `vex`, `rollback` / `remove`) and the 2.6.9 hosted-unwind direct-dep and `group` cells. Run 59 used a standalone harness instead of a scratch Rust test: a Python mock (upstream compact index with ruby + `x86_64-linux` variants, patch registry, patches API) plus real `gem build` fixtures. Pass `--patch-server-url <mock>` to `rollback` / `remove` / `vex`, or a 127.0.0.1 pin isn't treated as hosted and the commands report `manifest_not_found`. Ruby 3.1.6 is in the sandbox (`/opt/rbenv/versions/3.1.6/bin`); `gem install bundler -v 1.17.3` / `2.1.4` there and export `BUNDLER_VERSION` + `SOCKET_PATCH_BUNDLER_E2E_VERSION`. A fresh container's Ruby 3.3.6 only has 2.5.22 / 4.0.18, so `gem install bundler -v 2.6.9` / `4.0.23` / `4.1.0.beta3` first and export `BUNDLER_VERSION`. Bundler 2.6 has no `install -q` (use `--quiet`).
 
 ## Coverage matrix
 
@@ -12,7 +12,7 @@ Cells are "pass", "fail #N" or "untested". Hosted and vendored cells use a local
 | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 | Linux | 3.1.6 | 1.17.3 | pass (suites, run 51) | untested | untested | untested | n/a | untested | untested | untested | untested | untested | untested | untested |
 | Linux | 3.3.6 | 2.2–2.4 | untested | fixed #340 (#637) | untested | untested | n/a | untested | fixed #390 (#431) | untested | fixed #341 (#431) | untested | untested | untested |
-| Linux | 3.3.6 | 2.6.9 | pass | untested | untested | untested | pass | untested | untested | untested | untested | untested | untested | untested |
+| Linux | 3.3.6 | 2.6.9 | pass | untested | untested | untested | pass | untested | untested | pass (vendored suite, run 66) | pass (suite, run 66) | pass (suite, run 66) | untested | untested |
 | Linux | 3.3.6 | 4.0.17 | pass | fixed #340 (#637) | pass | pass (switches to patched ruby gem) | pass | pass | fixed #390 (#431) | pass | fixed #341 (#431) | fixed #390 (#431) | refused (documented) | fixed #420 (#517) |
 | Linux (CI) | 2.7 / 3.3 / 3.4 | 2.4 / 2.5 / 2.6 | untested | untested (OS-independent) | untested | untested | untested | untested | untested (OS-independent) | untested | untested (OS-independent) | untested | untested | fixed #420 (#517) |
 | macOS (CI) | 2.7 / 3.3 / 3.4 | 2.4 / 2.5 / 2.6 | untested | untested (OS-independent) | untested | untested | untested | untested | untested (OS-independent) | untested | untested (OS-independent) | untested | untested | fixed #420 (#517, probe pass) |
@@ -604,6 +604,15 @@ Also on 4.0.17: BOM `Gemfile` passes; symlinked `Gemfile` / lock is refused (pas
 | windows-latest, deny-write ACE on the gem dir | 3.4.9 | pass | n/a | n/a | pass (exit 1 `partial_failure`, nothing written) | n/a | pass (refuses) | n/a | pass (fails closed; the ACE also blocks Ruby's own load) |
 | macos-14 system Ruby 2.6 / Bundler 1.17, `-g` inside a project | 2.6 | pass | pass (project `vendor/bundle` untouched) | untested | pass | pass | pass | n/a | n/a |
 
+### Run 66 (main `cbbbdb9`, v5.0.1; Linux Ruby 3.3.6; repo vendored suite + scratch tests in `e2e_redirect_gem_build.rs`, real `bundle install` / `bundle lock`)
+
+| Cell | 2.6.9 | 4.1.0.beta3 |
+| --- | --- | --- |
+| Repo `e2e_vendor_gem_build` (`--ignored`, `REQUIRED=1`) | pass (11) | pass (11) |
+| Hosted gem reached only via `override "other-dep", with: "vuln-gem", in: "parent"`: scan, converge, frozen fresh install, OVERRIDES intact | n/a | pass |
+| Same, `rollback` / `remove` byte-exact | n/a | **fail #1989** |
+| Control (`parent` depends on `vuln-gem`), `rollback` / `remove` | n/a | pass |
+
 ### User-added CHECKSUMS around a hosted redirect (Linux, Ruby 3.3.6, run 61)
 
 | Cell | 2.6.9 | 4.0.23 | 4.1.0.beta3 |
@@ -626,13 +635,14 @@ Also on 4.0.17: BOM `Gemfile` passes; symlinked `Gemfile` / lock is refused (pas
 
 ## Backlog
 
-1. Re-check #1831 once fixed (PR #1841, not merged as of run 65): hosted redirect on a CRLF Gemfile keeps CRLF in the in-place and the append (transitive) arms; `rollback` / `remove` still byte-exact; an autocrlf re-checkout of a redirected LF Gemfile re-scans as a no-op. Then the 2.6.9 / 4.0.23 CRLF Gemfile hosted unwind.
-2. Re-run #1109 (`agent:needs-human`) when it's fixed.
-3. The gem suites on 4.1.0 final when it ships; a 2.2 / 2.4 leg (Ruby 3.1.6) on the next main that touches gem code.
-4. `bundle add` of a second gem that resolves from the patch registry source (inside the Socket block). (`bundle update --all` is done, run 65.)
-5. A superseding patch generation for `vuln-gem 1.0.1` after `bundle update vuln-gem` moved past a relaxed (`~>`) user pin, if the redirect ever keeps a non-exact requirement; today it pins the exact version, so the update can't move it (run 65).
-6. Vendored stub-vs-lock runtime-dependency drift, only if the service `gem-stub-gemspec` can declare other deps than the lifted GEM spec block (a file patch can't).
-7. macOS / Windows probes stay blocked until a maintainer deletes the leftover `bughunt/bundler/*` branches and push-triggered runs work again (none has triggered since 2026-10-05).
+1. Re-check #1989 once fixed (`rollback` / `remove` of an `OVERRIDES`-reached hosted gem must come back byte-exact), and #1831 once PR #1841 merges (CRLF Gemfile in-place and append arms, `rollback` / `remove` byte-exact, autocrlf re-scan no-op), then the 2.6.9 / 4.0.23 CRLF Gemfile hosted unwind.
+2. More `OVERRIDES` shapes on 4.1: `override "vuln-gem", version: ">= 1.0"` / `:ignore_upper` on the hosted gem, `with: nil` dropping `tiny-dep`, a CHECKSUMS lock with the rubygems.org + mirror spelling, and vendored mode (eject / `vendor --revert`) over an override-reached gem.
+3. `CONTENT ADDRESSES` (Bundler 4.1 with RubyGems 4.1, `gem update --system 4.1.0.beta3`) over hosted and vendored locks.
+4. Re-run #1109 (`agent:needs-human`) when it's fixed.
+5. The gem suites on 4.1.0 final when it ships; a 2.2 / 2.4 leg (Ruby 3.1.6) on the next main that touches gem code.
+6. `bundle add` of a second gem that resolves from the patch-registry source (inside the Socket block).
+7. Vendored stub-vs-lock runtime-dependency drift, only if the service `gem-stub-gemspec` can declare other deps than the lifted GEM spec block.
+8. macOS / Windows probes stay blocked until a maintainer deletes the leftover `bughunt/bundler/*` branches and push-triggered runs work again (none has triggered since 2026-10-05).
 
 ## Known non-bugs