WIP: [ruby] Resolve @rpath/@loader_path dylib references when relinking - #5160
okorohelijah wants to merge 6 commits into
Conversation
The `Mac_arm64 ruby` CIPD builder fails in the vendored ruby_ship packaging step (auto_relink_dylibs.rb) after Ruby has been compiled: fileutils.rb:1394:in 'initialize': No such file or directory @ rb_sysopen - @rpath/libbrotlicommon.1.dylib (Errno::ENOENT) from auto_relink_dylibs.rb:40 (FileUtils.copy) The script walks `otool -L` output and copies every dependency into bin/darwin_ruby/dylibs using the literal string reported by otool. Homebrew's brotli 1.2.0 bottle (pulled in by curl) references its sibling library as `@rpath/libbrotlicommon.1.dylib`, which is a dyld search token rather than a filesystem path, so the copy fails. Because the script recurses into the *copied* dylib, it also no longer knows where the original lived and could not find the sibling even if it tried. Fix: - Thread the original source directory through the recursion. - Before copying an `@rpath/`, `@loader_path/` or `@executable_path/` dependency, resolve it to a real file by trying, in order: the referencing library's original directory, each LC_RPATH entry from `otool -l` (with `@loader_path` expanded against the original directory), and finally the dylibs directory itself if the library was already copied. - If none of those exist, print which reference could not be resolved and exit 1 instead of crashing with an opaque ENOENT. - `install_name_tool -change` still receives the exact original string from the load command as its `old` argument, so the rewrite matches. Absolute paths behave exactly as before, and `/usr/lib/` and `/System/Library/` entries are still skipped. Note that this script is vendored from stephan-nordnes-eriksen/ruby_ship (third_party/ruby_ship); this is the first local change to the .rb file (ruby_ship_build.sh was already modified locally in flutter#3186). Failing build: https://ci.chromium.org/b/8669644619614462833 Related: flutter/flutter#164665
There was a problem hiding this comment.
Code Review
This pull request introduces support for resolving relative Mach-O dependencies (such as @rpath, @loader_path, and @executable_path) in auto_relink_dylibs.rb by parsing otool -l output and searching for candidates. The review feedback highlights three key areas for improvement: correctly resolving @executable_path relative to the main executable directory instead of the referring library's directory, making the otool -l output regex parser more robust against whitespace variations, and escaping file paths passed to shell commands to prevent shell injection or execution failures.
| def resolve_dylib_path(libfile, source_dir, rpaths) | ||
| return libfile unless libfile.start_with?("@") | ||
|
|
||
| basename = File.split(libfile)[-1] | ||
| candidates = [] | ||
| if libfile.start_with?("@loader_path/", "@executable_path/") | ||
| candidates << File.expand_path(libfile.sub(/\A@(loader_path|executable_path)/, source_dir)) | ||
| end | ||
| candidates << File.join(source_dir, basename) | ||
| rpaths.each do |rpath| | ||
| rpath = File.expand_path(rpath.sub(/\A@(loader_path|executable_path)/, source_dir)) | ||
| candidates << File.join(rpath, basename) | ||
| end | ||
| candidates << File.join(@new_dylib_path, basename) | ||
|
|
||
| candidates.find { |candidate| File.file?(candidate) } | ||
| end |
There was a problem hiding this comment.
Currently, @executable_path is resolved to source_dir (the directory of the referring library). However, @executable_path refers to the directory of the main executable (e.g., bin/darwin_ruby/bin/), whereas @loader_path refers to the directory of the referring library. If a library references a dependency via @executable_path, resolving it to source_dir will fail to find the dependency if the referring library is located in dylibs/ or lib/.
We can resolve @executable_path correctly by computing the executable directory relative to @new_dylib_path (which is File.expand_path(File.join(@new_dylib_path, "..", "bin"))).
Additionally, we can simplify the substitution logic and use the more idiomatic File.basename instead of File.split(libfile)[-1].
def resolve_dylib_path(libfile, source_dir, rpaths)
return libfile unless libfile.start_with?("@")
basename = File.basename(libfile)
executable_dir = File.expand_path(File.join(@new_dylib_path, "..", "bin"))
candidates = []
if libfile.start_with?("@loader_path/", "@executable_path/")
resolved_libfile = libfile.sub(/\A@loader_path/, source_dir).sub(/\A@executable_path/, executable_dir)
candidates << File.expand_path(resolved_libfile)
end
candidates << File.join(source_dir, basename)
rpaths.each do |rpath|
resolved_rpath = rpath.sub(/\A@loader_path/, source_dir).sub(/\A@executable_path/, executable_dir)
candidates << File.join(File.expand_path(resolved_rpath), basename)
end
candidates << File.join(@new_dylib_path, basename)
candidates.find { |candidate| File.file?(candidate) }
end| def parse_rpaths(otool_l_output) | ||
| otool_l_output.scan(/cmd LC_RPATH\n\s*cmdsize \d+\n\s*path (.+?) \(offset \d+\)/).flatten | ||
| end |
There was a problem hiding this comment.
The current regular expression for parsing otool -l output assumes strict newline (\n) and spacing formatting. To make the parsing more robust against variations in whitespace, tabs, or line endings (e.g., carriage returns) across different macOS/Xcode versions, it is safer to use \s+ instead of literal newlines and spaces.
def parse_rpaths(otool_l_output)
otool_l_output.scan(/cmd\s+LC_RPATH\s+cmdsize\s+\d+\s+path\s+(.+?)\s+\(offset\s+\d+\)/).flatten
end| def rpaths_for_file(file) | ||
| parse_rpaths(`otool -l "#{file}" 2> /dev/null`) | ||
| end |
There was a problem hiding this comment.
Using string interpolation directly inside backticks (otool -l "#{file}") can lead to shell injection or command failures if the file path contains spaces, double quotes, or other shell metacharacters. It is highly recommended to escape the file path using Shellwords.escape to ensure the command executes safely and robustly.
def rpaths_for_file(file)
require "shellwords" unless defined?(Shellwords)
parse_rpaths(`otool -l #{Shellwords.escape(file)} 2> /dev/null`)
endOn 2026-09-27 homebrew-core moved the `openssl` alias from openssl@3 to openssl@4 (Homebrew/homebrew-core 9a12b94a). build.sh installs `openssl@3`, but ruby_ship_build.sh resolved `brew --prefix openssl`, which now returns /opt/homebrew/opt/openssl@4 -- a directory that does not exist on the bots. The resulting `-L/opt/homebrew/opt/openssl@4/lib` makes ld emit "directory not found" warnings, Ruby 3.1's configure treats any warning under -Werror as a failed probe (e.g. "function name string predefined identifier... no"), and the build then dies with `use of undeclared identifier 'RUBY_FUNCTION_NAME_STRING'`. Seen on led build https://ci.chromium.org/b/8669384638160948033; the previous prod run (2 days earlier) still resolved the alias to openssl@3. Pin the prefix to openssl@3 in both places it is used, matching what build.sh installs and the openssl@3 cert.pem path it already copies. Related: flutter/flutter#164665
bundler 4.0.21 (released 2026-09-16) requires Ruby >= 3.2 and RubyGems >= 3.4.1. `gem install -f bundler` skips that check, so it was force- installed onto the Ruby 3.1.3 build, and the `bin/bundler --version` smoke test in tools/build.sh crashed inside the RubyGems resolver: rubygems/resolver/conflict.rb:47:in `conflicting_dependencies': undefined method `request' for nil:NilClass (NoMethodError) Seen on led build https://ci.chromium.org/b/8669383997159272673, right after the dylib relink and gem installs had all succeeded. Pin to `~> 2.5.0`, which resolves to bundler 2.5.23 (Ruby >= 3.0, RubyGems >= 3.2.3; Ruby 3.1.3 ships RubyGems 3.3.26). Note that a bare `~> 2.5` would resolve to 2.7.2, which already requires Ruby >= 3.2, so the patch-level constraint is deliberate. Related: flutter/flutter#164665
- Only expand `@loader_path/` references (and LC_RPATH entries) relative to the referencing library's original directory. `@executable_path` depends on whichever executable loads the library, which is not known when packaging a Homebrew dylib, so such references are matched by basename only (original directory, LC_RPATH dirs, already-copied dylibs) and `@executable_path` LC_RPATH entries are skipped. - Make the `otool -l` LC_RPATH regex tolerant of arbitrary whitespace. - Shell-escape the path passed to `otool -l`. - Use File.basename.
`gem install activesupport -v 7.0.8` pulls in the newest concurrent-ruby (1.3.8 today). concurrent-ruby 1.3.5 (2025-01-15) removed its implicit `require "logger"`, which activesupport 7.0 silently relied on, so the `bin/pod --version` smoke test fails with: activesupport-7.0.8/lib/active_support/logger_thread_safe_level.rb:12: uninitialized constant ActiveSupport::LoggerThreadSafeLevel::Logger (NameError) Seen on led build https://ci.chromium.org/b/8669375943591764321, after the relink, openssl@3 and bundler fixes had taken the build all the way to the last smoke test. The package is built into a fresh GEM_HOME, so installing concurrent-ruby 1.3.4 first means RubyGems keeps it when resolving activesupport (which requires `~> 1.0, >= 1.0.2`; cocoapods-core wants `~> 1.1`, i18n and tzinfo `~> 1.0`, all satisfied by 1.3.4), and `pod` activates 1.3.4 at runtime instead of 1.3.8. Related: flutter/flutter#162341
Installing concurrent-ruby 1.3.4 first was not enough on its own: the RubyGems bundled with Ruby 3.1.3 (3.3.26) still resolves activesupport's `concurrent-ruby ~> 1.0` to the newest release and installs 1.3.8 next to 1.3.4, and at runtime `pod` activates the newest installed version, so `bin/pod --version` kept failing with uninitialized constant ActiveSupport::LoggerThreadSafeLevel::Logger (led build https://ci.chromium.org/b/8669366981838817409, which shows both `Successfully installed concurrent-ruby-1.3.4` and, during the activesupport install, `Successfully installed concurrent-ruby-1.3.8`). Newer RubyGems (3.6) keeps the already-installed 1.3.4, which is why a local dry run did not show the problem. After the cocoapods install, uninstall every concurrent-ruby newer than 1.3.4 (`-a` all matching versions, `-x` executables, `-I` skip the dependency prompt). `gem uninstall` exits 0 with "is not installed" when nothing matches, so this is a no-op on RubyGems versions that already keep 1.3.4. Related: flutter/flutter#162341
What failed
Mac_arm64 rubyhas been failing since the Xcode bump in #5155 let it get past Homebrew and actually compile Ruby. It then died in the vendored packaging step,cipd_packages/ruby/third_party/ruby_ship/auto_relink_dylibs.rb:Failing prod build: https://ci.chromium.org/b/8669644619614462833
While verifying the fix on led, three more, unrelated breakages showed up behind it, one after another. This PR fixes all four.
1.
@rpathdependencies inauto_relink_dylibs.rbThe script walks
otool -Lfor every file inbin/darwin_ruby, copies each dependency intobin/darwin_ruby/dylibs/using the literal path otool printed, and then recurses into the copy. Homebrew's brotli 1.2.0 bottle (a curl dependency) references its sibling as@rpath/libbrotlicommon.1.dylib. That is a dyld search token, not a filesystem path, soFileUtils.copyblows up. And because the script recurses into the copiedlibbrotlidec.1.dylib, it has also lost track of where the original lived.Fix:
@rpath/,@loader_path/or@executable_path/, resolve it to a real file before copying, trying in order: the referencing library's original directory, eachLC_RPATHfromotool -l(with@loader_pathexpanded against the original directory), and finally thedylibs/directory in case it was already copied.@executable_pathdepends on whichever executable loads the library, which isn't known at packaging time, so those are matched by basename only.exit 1, rather than failing with an opaqueENOENT.install_name_tool -changestill gets the exact original string from the load command asold, so the rewrite matches.Absolute paths behave exactly as before;
/usr/lib/and/System/Library/are still skipped.2.
brew --prefix opensslnow points at openssl@4On 2026-09-27 homebrew-core moved the
opensslalias fromopenssl@3toopenssl@4(Homebrew/homebrew-core@9a12b94a).build.shinstallsopenssl@3, butruby_ship_build.shresolvedbrew --prefix openssl, which now returns/opt/homebrew/opt/openssl@4, a directory that does not exist on the bots. The bogus-L/opt/homebrew/opt/openssl@4/libmakesldwarn, Ruby 3.1's configure treats warnings under-Werroras failed probes (checking for function name string predefined identifier... no), andmakedies withuse of undeclared identifier 'RUBY_FUNCTION_NAME_STRING'. The last prod run (two days earlier) still resolved the alias toopenssl@3, so prod would hit this on its next run regardless of the relink fix. Pinned tobrew --prefix openssl@3in the two places it is used.3.
gem install -f bundlerpicks up bundler 4bundler 4.0.21 (released 2026-09-16) requires Ruby >= 3.2 and RubyGems >= 3.4.1.
-fbypasses that check, so it was force-installed onto this Ruby 3.1.3 build and thebin/bundler --versionsmoke test crashed inside the RubyGems resolver (rubygems/resolver/conflict.rb:47: undefined method 'request' for nil:NilClass). Pinned to-v '~> 2.5.0', which resolves to 2.5.23 (Ruby >= 3.0, RubyGems >= 3.2.3). A bare~> 2.5would resolve to 2.7.2, which already needs Ruby >= 3.2, hence the patch-level constraint.4.
pod --versionfails to load ActiveSupport (flutter/flutter#162341)With everything above fixed, the last smoke test failed with the error this builder used to produce back in mid-2025 whenever it got this far:
activesupport 7.0.8 uses
Loggerwithout requiring it and only worked because concurrent-ruby <= 1.3.4 happened torequire "logger"; concurrent-ruby 1.3.5 (2025-01-15) removed that, andgem install activesupport -v 7.0.8pulls the newest concurrent-ruby (1.3.8). Two commits deal with it:~> 1.0, >= 1.0.2, cocoapods-core~> 1.1, i18n and tzinfo~> 1.0).podactivates the newest one. Newer RubyGems keeps the installed 1.3.4, which is why a local dry run looked fine. So after the cocoapods install the script now runsgem uninstall concurrent-ruby -a -x -I -v '> 1.3.4', which removes anything newer and is a no-op (exit 0) when there is nothing to remove.Note:
third_party/ruby_shipis vendored from stephan-nordnes-eriksen/ruby_ship.ruby_ship_build.shwas already modified locally in #3186; this is the first local change toauto_relink_dylibs.rb.How it was verified
bash -non the shell script;ruby -c, plus a throwaway minitest (not committed) covering absolute paths,@rpathnext to the original,@rpaththrough absolute and@loader_pathLC_RPATHentries,@loader_path/...relative references,@executable_pathbasename fallback (and that its relative form is not expanded), the already-copied fallback, the unresolvable case, andotool -lparsing including CRLF/tab output. 12 tests, all green.otool/install_name_toolreproducing the brotli layout (libcurl -> /abs/libbrotlidec.1.dylib -> @rpath/libbrotlicommon.1.dylib): all three libraries land indylibs/,-changeuses@rpath/libbrotlicommon.1.dylibas the old name, and removinglibbrotlicommonmakes it exit 1 with a clear message.Mac_arm64 ruby, checking outrefs/pull/5160/head; CIPD upload is skipped on led because the recipe only registers from the prod bucket):auto_relink_dylibs.rbfinish withRelinking of dylib done, and the previously failing spot now reads--COPIED-- .../dylibs/libbrotlicommon.1.dylib/Linked: .../libbrotlicommon.1.dylib to .../libbrotlidec.1.dylib. bundler, activesupport 7.0.8 and cocoapods 1.16.2 install; thenbin/bundler --versioncrashed with the bundler 4 problem in (3).flutter-devicelab-mac-5, which has no Command Line Tools, so Homebrew stopped atbrew reinstall --build-from-source gdbmwithError: Xcode alone is not sufficient on Sequoia(Mac rubyfailing due to missing command line tools flutter#162344). Nothing to do with this change; the staging arm64 fleet is just inconsistent on CLT.flutter-devicelab-mac-55): https://ci.chromium.org/b/8669375943591764321 — configure/compile fine, all three relink passes finish, bundler 2.5.23 / activesupport 7.0.8 / cocoapods 1.16.2 install;bin/bundler --versionprintsBundler version 2.5.23,bin/gem --versionprints3.3.26; fails atbin/pod --versionwith the ActiveSupportLoggerNameError in (4).Successfully installed concurrent-ruby-1.3.4and then, during the activesupport install,Successfully installed concurrent-ruby-1.3.8;pod --versionfails the same way. That is what prompted the uninstall step.ruby_ship_build.shreaches its DONE banner; the log showsSuccessfully installed concurrent-ruby-1.3.4, laterSuccessfully installed concurrent-ruby-1.3.8(pulled in by activesupport, as in run 4), thenSuccessfully uninstalled concurrent-ruby-1.3.8from the new step. All four smoke tests intools/build.shpass:bin/bundle --version→ri 6.4.0(see note below),bin/bundler --version→Bundler version 2.5.23,bin/gem --version→3.3.26,bin/pod --version→1.16.2. The recipe'sbuild mac-arm64step then packagedbuild/into a CIPD instance (flutter/ruby/mac-arm64, instanceqTUDFbJFw8l9dWrWncVXCope1qBH-Rc1JUr8UGnQ6qMC). Nothing was uploaded:cipd.pyL72–73 only registers when the bucket isprodand the build is forrefs/heads/main; a led build runs instaging.shadowwith no gitiles commit, so the register step is skipped by design.Note:
bin/bundle --versionprintingri 6.4.0is a pre-existing bug in the vendored script —ruby_ship_build.shL126–129 generatesbin/bundlepointing at theriwrapper instead ofdarwin_bundle.sh. It exits 0 so it never failed the build; left for a separate one-line follow-up.Out of scope
Mac ruby(Intel) is separately blocked on missing Command Line Tools (flutter/flutter#162344) and is not touched here.Related: flutter/flutter#164665