Skip to content

Make the Windows ROCm load path work and fix has_imma on ROCm - #2102

Draft
IxMxAMAR wants to merge 3 commits into
bitsandbytes-foundation:mainfrom
IxMxAMAR:fix/windows-rocm-loading
Draft

IxMxAMAR wants to merge 3 commits into
bitsandbytes-foundation:mainfrom
IxMxAMAR:fix/windows-rocm-loading

Conversation

@IxMxAMAR

@IxMxAMAR IxMxAMAR commented Oct 18, 2026 •

Copy link
Copy Markdown

Two things on the Windows + ROCm path, both of which make a broken install look like a bitsandbytes bug.

Loading the library. On CPython 3.8+ the Windows loader uses LOAD_LIBRARY_SEARCH_DEFAULT_DIRS, which does not include PATH and does not include the directory of the DLL being loaded. When the ROCm runtime isn't already resident in the process, an extension sitting next to a perfectly good ROCm install still can't resolve amdhip64_*.dll, and ctypes reports it as:

FileNotFoundError: Could not find module '...\libbitsandbytes_rocm72.dll'
(or one of its dependencies). Try using the full path with constructor syntax.

That names the wrong file, so it reads as "bitsandbytes is broken" when the runtime just isn't resolvable. Two changes: register the usual runtime directories with os.add_dll_directory before LoadLibrary, and classify the error as a missing dependency so the advice points at the runtime instead. The handles are kept alive in a module-level list, because closing one removes the directory again.

Worth being precise about the scope, because I initially overstated it. If torch's ROCm build is imported before bitsandbytes, amdhip64.dll is already loaded in the process and a plain LoadLibrary resolves the dependency regardless of search path. I checked that directly: with ROCm absent from PATH and no add_dll_directory, the load still succeeds once torch is up. So the ordinary import order was never affected, and this is not "Windows ROCm cannot load at all". The failure needs a process where the runtime isn't resident yet, which is what the reproduction below does. The registration makes that case work and, more usefully, makes the error message stop blaming the wrong file.

Reading the capability. has_imma compared highest_compute_capability >= (7, 5). On ROCm that means gfx906 reports 9.0 and gets treated as having MFMA, which it doesn't, and the int8 kernels gate on this. It's decided from the gfx target now, with gfx900/906/90c excluded. get_rocm_gpu_arch reads gcnArchName off torch.cuda.get_device_properties instead of parsing rocminfo, and returns "unknown" rather than raising when there's no HIP. Also small: BNB_ROCM_VERSION only accepted the compact form (72), so 7.2 was rejected as garbage; both are accepted.

Verified on an RX 9070 XT (gfx1201, ROCm 7.2, clang 23, MSVC 18), Windows 11:

  • Reproduced the load failure for real rather than only asserting on a string: in a bare interpreter with torch not imported and ROCm stripped off PATH, ct.cdll.LoadLibrary on the built extension fails with FileNotFoundError and winerror=None, then loads once os.add_dll_directory is given the ROCm bin.
  • That reproduction is what caught a bug in my first version of this patch. I matched [WinError 126] and "The specified module could not be found", and the error ctypes actually raises contains neither. The matcher covers both forms now and there's a test carrying the verbatim message.
  • 19 tests in tests/test_cextension_errors.py, and test_linear8bitlt.py gives 39 passed / 16 skipped, which is the path has_imma gates.

Not verified: the CUDA wording of the hint, and installs that put ROCm somewhere outside ROCM_PATH / HIP_PATH / ROCM_HOME / CONDA_PREFIX / torch's lib / PATH. The registration scans those and skips directories without amdhip64*.dll, so an unusual layout falls back to the current behaviour instead of breaking. There's no GPU-arch-specific testing either: this was all on one gfx1201 card, and the gfx906/gfx900 has_imma expectations are from the target names, not from hardware.

Seven defects on the same code path, all reachable on Windows + ROCm:

* os.add_dll_directory was never called. CPython loads with
  LOAD_LIBRARY_SEARCH_DEFAULT_DIRS, which does not search PATH or the DLL's own
  directory, so an SDK whose amdhip64*.dll lives in bin/ resolved only by luck.
  Register ROCM_PATH/HIP_PATH/ROCM_HOME/CONDA_PREFIX, torch/lib and PATH
  entries that actually contain a runtime DLL.
* The 'no compatible binary' guidance keyed on '<backend> binary not found',
  but the loader raises '<backend> binary found at'. The branch was dead, so
  users were told to build a CPU-only library instead of being told their
  backend/version did not match.
* The CPU-only probe hardcoded '.so', so it never matched on Windows.
* Missing-dependency advice was Linux-only (LD_LIBRARY_PATH, .bashrc, lib64).
  It now branches on platform and backend.
* A missing dependency on Windows surfaces as [WinError 126], not as a
  dlerror() string, so it never reached the dependency advice at all.
* has_imma returned torch.version.hip -- a version string -- despite being
  annotated -> bool, so every AMD GPU was reported as IMMA-capable. Derive it
  from the gfx target instead.
* BNB_ROCM_VERSION=7.2, the natural spelling, was a hard error.

get_rocm_gpu_arch() no longer shells out to hipinfo.exe/rocminfo: the
subprocess is not on PATH for pip-installed SDKs, its output differs per
platform, and torch already exposes gcnArchName.
…sing dependency

ctypes reports a missing DLL dependency as FileNotFoundError with winerror unset and
the text 'Could not find module ... (or one of its dependencies)'. Matching only on
'[WinError 126]' / 'The specified module could not be found' misses it, which is the
exact failure this path exists to explain. Found by reproducing the load against a copy
of the built extension with ROCm off PATH.
@IxMxAMAR
IxMxAMAR marked this pull request as draft October 18, 2026 00:32

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant