Skip to content

Keep ruby unit tests off the host Bundler env - #2066

Merged
Mikola Lysenko (mikolalysenko) merged 2 commits into
mainfrom
ci-janitor/bundle-env-scrub
Nov 13, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 2 commits into
mainfrom
ci-janitor/bundle-env-scrub

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

CI was healthy this run: the merge queue was empty, all merge_group runs since 11-12 18:56 UTC passed, main's latest push run (43324491043) and today's nightly (43373720347) are green, and nothing was queued. The two recent red nightlies (11-10, 11-12, Windows) were already fixed by #2049 and #2061. So this run takes a priority-4 robustness target from #1919, the same class as #2054–#2057 and #2063–#2065: core unit tests that pass only because CI's env is clean.

The hostile-env probe on #1919 found BUNDLE_GEMFILE and BUNDLE_APP_CONFIG still leak into the core lib tests. Measured on origin/main (de81f1b), cargo test -p socket-patch-core --lib:

host env main this PR
clean 7720/7720 7721/7721 (1 new test)
BUNDLE_GEMFILE=<other dir>/Gemfile 156 failed all pass
BUNDLE_APP_CONFIG=<dir with config> 5 failed all pass
HOME with .bundle/config setting BUNDLE_PATH__SYSTEM + BUNDLE_GEMFILE 154 failed (not in the earlier probe) all pass
all of the above + BUNDLE_PATH__SYSTEM=true BUNDLE_IGNORE_CONFIG=1 157 failed all pass

Failing modules on main: vendor::gem, vex::discover::gem, vendor::lock_inventory, vex::discover, upstream::gem.

Root cause

crawlers/ruby_crawler.rs reads every BUNDLE_* var (31 call sites) with std::env::var_os and the user's global Bundler config through ambient_home() → home_dir(), with no test seam. BundlerEnv/*_with_env seams exist, but the vendor and vex paths reach the ambient wrappers.

Fix

Under cfg(test) only, two small seams in ruby_crawler.rs, same pattern as utils::go_env::process_env (#2057):

  • bundle_env(key) replaces the 31 std::env::var_os("BUNDLE_*") reads; in tests it returns only values a live EnvGuard holds (test_env::guarded_var).
  • ambient_home() in tests resolves HOME/USERPROFILE through home_from_env over guarded values only, so a developer's ~/.bundle/config is never read.

Non-test builds are byte-for-byte the old code path (std::env::var_os, home_dir()), including the CLI integration tests, which link the non-test core. New test bundler_env_sees_only_guarded_variables (#[serial]) pins both halves: an unguarded export is not seen, guarded values are.

This is deliberately the minimal seam; it doesn't touch the Bundler-settings parsing that #1935 plans to move, so it shouldn't conflict with that refactor.

Proof

  • Table above (main measured with the same binary built from de81f1b, then from this branch).
  • New test looped 50x: 50/50.
  • cargo clippy --locked -p socket-patch-core --all-features -- -D warnings (CI invocation) is clean; rustfmt --check clean on the touched file. --all-targets only shows a pre-existing cloned_ref_to_slice_refs lint in test code this PR doesn't touch.
  • No test removed or moved; nothing changes in CI config.

Review (/code-review high + Bugbot)

Bugbot: no findings on 75b2596. Fixed in 01c8817 from /code-review high:

  • bundler_source_mirror still read BUNDLE_MIRROR__* values from the host. They now go through bundle_env. The core lib passes 7721/7721 with BUNDLE_MIRROR__ALL exported, alone and combined with every Bundler row in the table.
  • The seam test set BUNDLE_GEMFILE raw for a moment. It now sets a probe key that no code reads, so a gem child spawned by a concurrent test can't inherit anything that matters.

Not changed, and why:


Generated by Claude Code

The core unit tests read BUNDLE_* vars and the user's ~/.bundle/config
straight from the process env. A developer with BUNDLE_GEMFILE exported
saw 156 core lib tests fail on a clean checkout, BUNDLE_APP_CONFIG 5,
and a ~/.bundle/config with path.system/gemfile 154 (#1919).

Under cfg(test) only, ruby_crawler's BUNDLE_* reads and its home
lookup now see only values a live EnvGuard holds, the seam go_env
(#2057) uses. Non-test builds are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aeNvBQRDzpbqHAEjNAY
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) label Nov 13, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor

cursor Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Test-only behavior changes behind cfg(test); production Bundler discovery paths are unchanged.

Overview
Adds cfg(test) seams in the Ruby crawler so unit tests no longer inherit the host BUNDLE_* exports or HOME-backed Bundler config. All ambient BUNDLE_* reads go through bundle_env, and ambient_home() only resolves home from EnvGuard-held vars in tests; release builds still use std::env::var_os and home_dir() unchanged.

A serial unit test bundler_env_sees_only_guarded_variables asserts unguarded shell exports are invisible and guarded values are visible—fixing the ~156 failures when BUNDLE_GEMFILE (and related env) leak into socket-patch-core lib tests (#1919).

Reviewed by Cursor Bugbot for commit 75b2596. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 75b2596. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 13, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review (burn-down agent).

  • Head: 75b2596
  • CI: 50/50 checks settled green (31 success, 19 skipped); no main-wide failures.
  • Bugbot: reviewed 75b2596, no findings.
  • Mergeable, no conflicts; test-only change (no CHANGELOG edits).
  • Slack announcement not sent: the Slack send tool was unavailable in this run; the next run should retry.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does. Test-only change. All 31 std::env::var_os("BUNDLE_*") reads in ruby_crawler.rs go through a new bundle_env(key), and ambient_home() gets a cfg(test) fork; in unit tests both see only values a live EnvGuard holds. That stops a developer's BUNDLE_GEMFILE, BUNDLE_APP_CONFIG or ~/.bundle/config from breaking ~150 core tests (#1919).

Risk: low. The non-test bundle_env is exactly std::env::var_os(key) and the non-test ambient_home is unchanged, so production behavior is identical; the rest of the diff is a mechanical call-site swap plus rustfmt reflow.

Look here

  • [ambient_home / bundle_env test vs non-test forks](https://github.com/SocketDev/socket-patch/blob/75b2596ecd0cc70606759ff0e0c88fdadcadac5c/crates/socket-patch-core/src/crawlers/ruby_crawler.rs#L1711-L1745)
  • [bundler_ignores_config now via the seam](https://github.com/SocketDev/socket-patch/blob/75b2596ecd0cc70606759ff0e0c88fdadcadac5c/crates/socket-patch-core/src/crawlers/ruby_crawler.rs#L2199-L2206)
  • [BundlerPathSettings::from_ambient_env](https://github.com/SocketDev/socket-patch/blob/75b2596ecd0cc70606759ff0e0c88fdadcadac5c/crates/socket-patch-core/src/crawlers/ruby_crawler.rs#L1340-L1351) (one of the 31 swapped reads)
  • [New test bundler_env_sees_only_guarded_variables](https://github.com/SocketDev/socket-patch/blob/75b2596ecd0cc70606759ff0e0c88fdadcadac5c/crates/socket-patch-core/src/crawlers/ruby_crawler.rs#L2606-L2643)

Verified

  • Read the full diff (1 file, +99/-36) and the description; they match. I checked no other BUNDLE_* env read remains in core src/ (only the integration tests in tests/crawler_ruby_e2e.rs, which link the non-test core and keep using real env). No CHANGELOG change, no debug leftovers.
  • Ran locally at 75b2596: cargo test -p socket-patch-core --lib -- ruby_crawler 109/109, and with BUNDLE_GEMFILE=/elsewhere/Gemfile exported, -- vendor::gem vex::discover::gem 194/194 (the PR body shows main fails 156 under that env).
  • CI 50/50 green on head, ci-ok and clippy green. Bugbot: reviewed, no open threads.

Changes I made. None.

Open questions. None.

Auto-merge is armed, so approving sends it straight to the merge queue.


Generated by Claude Code

/code-review high found bundler_source_mirror still read
BUNDLE_MIRROR__* values from the host env. Route the values through
bundle_env. The seam test now sets a probe key no code reads, so a
gem child spawned by a concurrent test can't see it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aeNvBQRDzpbqHAEjNAY
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] Heads-up: the final-review brief above covers 75b2596. The head is now 01c8817, pushed at 10:58 UTC after /code-review high. It adds one more seamed read: bundler_source_mirror read BUNDLE_MIRROR__* values through std::env::vars_os(), a read the var_os("BUNDLE_*") grep missed. The seam test now uses a probe key that no code reads. Core lib passes 7721/7721 with BUNDLE_MIRROR__ALL exported and with every hostile Bundler var combined. The PR description lists the review triage. CI on 01c8817 is running.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief (head 01c8817, replaces the brief for 75b2596)

What it does. Test-only change. Every BUNDLE_* env read in ruby_crawler.rs goes through a new bundle_env(key), and ambient_home() gets a cfg(test) fork; in unit tests both see only values a live EnvGuard holds. That stops a developer's BUNDLE_GEMFILE, BUNDLE_APP_CONFIG, BUNDLE_MIRROR__* or ~/.bundle/config from breaking ~150 core tests (#1919).

Risk: low. The non-test bundle_env is exactly std::env::var_os(key) and the non-test ambient_home is unchanged. The new commit makes bundler_source_mirror take keys from vars_os() and the value from bundle_env(key), which in production is the same value it read before.

Look here

  • [ambient_home / bundle_env test vs non-test forks](https://github.com/SocketDev/socket-patch/blob/01c8817c158a4ae0a0f7008469de0f8854e8b953/crates/socket-patch-core/src/crawlers/ruby_crawler.rs#L1711-L1745)
  • [bundler_source_mirror: the BUNDLE_MIRROR__ scan, new in 01c8817](https://github.com/SocketDev/socket-patch/blob/01c8817c158a4ae0a0f7008469de0f8854e8b953/crates/socket-patch-core/src/crawlers/ruby_crawler.rs#L2160-L2176)
  • [New test bundler_env_sees_only_guarded_variables (now uses a probe key no code reads)](https://github.com/SocketDev/socket-patch/blob/01c8817c158a4ae0a0f7008469de0f8854e8b953/crates/socket-patch-core/src/crawlers/ruby_crawler.rs#L2614-L2650)

Verified

  • Read the full diff (1 file, +103/-38) and the 75b2596..01c8817 delta; description matches; no CHANGELOG change; no debug leftovers. EnvGuard sets the real env and registers the key, so a guarded BUNDLE_MIRROR__* is both enumerated and read; an unguarded host one is skipped.
  • Ran locally at 01c8817 with BUNDLE_MIRROR__ALL and BUNDLE_GEMFILE=/elsewhere/Gemfile exported: cargo test -p socket-patch-core --lib -- ruby_crawler vendor::gem vex::discover::gem 303/303.
  • CI green on head (ci-ok, clippy). Bugbot reviewed, no open threads.

Changes I made. None.

Open questions. None.

Auto-merge is armed, so approving sends it straight to the merge queue.


Generated by Claude Code

Merged via the queue into main with commit e1371d0 Nov 13, 2026
49 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the ci-janitor/bundle-env-scrub branch November 13, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants