Repository navigation
[AMD GPU installation] The Rocm-bitsandbytes installation issues #1608
Description
System Info
Issues: The installation of the latest multi-backend-refactor branch failed in the AMD GPU. While switching to the Rocm-bitsandbytes repo, by using the rocm_enabled_multi_backend branch, the installation was successfully. Could you please check if the right branch was selected, thanks so much!
Official Repo:
https://github.com/bitsandbytes-foundation/bitsandbytes/tree/multi-backend-refactor
Test Environment:
AMD MI300X GPU
Docker image:
Rocm6.4 is the latest version for the AMD Rocm release.
docker pull rocm/pytorch:rocm6.4_ubuntu24.04_py3.12_pytorch_release_2.4.1
Reproduction
How to reproduce:
Step by Step:
Rocm6.4 is the latest version for the AMD Rocm release.
docker pull rocm/pytorch:rocm6.4_ubuntu24.04_py3.12_pytorch_release_2.4.1
docker run -it --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --device=/dev/kfd --device=/dev/dri -v /:/workspace --group-add video --ipc=host --name bitsandbytes01 rocm/pytorch:rocm6.4_ubuntu24.04_py3.12_pytorch_release_2.4.1
Inside the docker:
git clone -b multi-backend-refactor https://github.com/bitsandbytes-foundation/bitsandbytes.git && cd bitsandbytes/
Compile & install
apt-get install -y build-essential cmake
cmake -DCOMPUTE_BACKEND=hip -S .
make
pip install -e .
After installation done:
Successfully built bitsandbytes
Installing collected packages: bitsandbytes
Attempting uninstall: bitsandbytes
Found existing installation: bitsandbytes 1.0.0
Uninstalling bitsandbytes-1.0.0:
Successfully uninstalled bitsandbytes-1.0.0
Successfully installed bitsandbytes-1.0.0
Verify the installation once done.
python -m bitsandbytes
root@93db47d5b637:/var/lib/jenkins/bitsandbytes# python -m bitsandbytes
Could not load bitsandbytes native library: /var/lib/jenkins/bitsandbytes/bitsandbytes/libbitsandbytes_rocm64.so: undefined symbol: _Z36__device_stub__kOptimizer32bit1StateI12hip_bfloat16Li2EEvPT_S2_PfS3_ffffffiffbi. If you use Intel CPU or XPU, please pip install intel_extension_for_pytorch
Traceback (most recent call last):
File "/var/lib/jenkins/bitsandbytes/bitsandbytes/cextension.py", line 115, in
lib = get_native_library()
^^^^^^^^^^^^^^^^^^^^
File "/var/lib/jenkins/bitsandbytes/bitsandbytes/cextension.py", line 86, in get_native_library
dll = ct.cdll.LoadLibrary(str(binary_path))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py_3.12/lib/python3.12/ctypes/init.py", line 460, in LoadLibrary
return self._dlltype(name)
^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py_3.12/lib/python3.12/ctypes/init.py", line 379, in init
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /var/lib/jenkins/bitsandbytes/bitsandbytes/libbitsandbytes_rocm64.so: undefined symbol: _Z36__device_stub__kOptimizer32bit1StateI12hip_bfloat16Li2EEvPT_S2_PfS3_ffffffiffbi
ROCm Setup failed despite ROCm being available. Please run the following command to get more information:
python -m bitsandbytes
Inspect the output of the command and see if you can locate ROCm libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/bitsandbytes-foundation/bitsandbytes/issues
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
ROCm specs: rocm_version_string='64', rocm_version_tuple=(6, 4)
PyTorch settings found: ROCM_VERSION=64
The directory listed in your path is found to be non-existent: /opt/ompi/lib
The directory listed in your path is found to be non-existent: /opt/ompi
The directory listed in your path is found to be non-existent: /opt/ucx
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Checking that the library is importable and ROCm is callable...
Couldn't load the bitsandbytes library, likely due to missing binaries.
Please ensure bitsandbytes is properly installed.
For source installations, compile the binaries with cmake -DCOMPUTE_BACKEND=hip -S ..
See the documentation for more details if needed.
Trying a simple check anyway, but this will likely fail...
Traceback (most recent call last):
File "/var/lib/jenkins/bitsandbytes/bitsandbytes/diagnostics/main.py", line 73, in main
sanity_check()
File "/var/lib/jenkins/bitsandbytes/bitsandbytes/diagnostics/main.py", line 42, in sanity_check
adam.step()
File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/optim/optimizer.py", line 484, in wrapper
out = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/jenkins/bitsandbytes/bitsandbytes/optim/optimizer.py", line 292, in step
self.update_step(group, p, gindex, pindex)
File "/opt/conda/envs/py_3.12/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/jenkins/bitsandbytes/bitsandbytes/optim/optimizer.py", line 522, in update_step
F.optimizer_update_32bit(
File "/var/lib/jenkins/bitsandbytes/bitsandbytes/functional.py", line 1266, in optimizer_update_32bit
return backends[g.device.type].optimizer_update_32bit(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/jenkins/bitsandbytes/bitsandbytes/backends/cuda.py", line 780, in optimizer_update_32bit
optim_func = str2optimizer32bit[optimizer_name][0]
^^^^^^^^^^^^^^^^^^
NameError: name 'str2optimizer32bit' is not defined
Above we output some debug information.
Please provide this info when creating an issue via https://github.com/TimDettmers/bitsandbytes/issues/new/choose
WARNING: Please be sure to sanitize sensitive info from the output before posting it.
Debugging details:
If we use the latest /ROCm/bitsandbytes to install, the installation was successful.
docker run -it --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --device=/dev/kfd --device=/dev/dri -v /:/workspace --group-add video --ipc=host --name bitsandbytes01 rocm/pytorch:rocm6.4_ubuntu24.04_py3.12_pytorch_release_2.4.1
Inside the docker:
git clone -b rocm_enabled_multi_backend https://github.com/ROCm/bitsandbytes.git
cd bitsandbytes
git checkout rocm_enabled_multi_backend
pip install -r requirements-dev.txt
cmake -DCOMPUTE_BACKEND=hip -S . #Use -DBNB_ROCM_ARCH="gfx90a;gfx942" to target specific gpu arch
make
pip install .
Verify the installation once done,
python -m bitsandbytes
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
ROCm specs: rocm_version_string='64', rocm_version_tuple=(6, 4)
PyTorch settings found: ROCM_VERSION=64
The directory listed in your path is found to be non-existent: /opt/ompi/lib
The directory listed in your path is found to be non-existent: /opt/ompi
The directory listed in your path is found to be non-existent: /opt/ucx
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Checking that the library is importable and ROCm is callable...
SUCCESS!
Installation was successful!
Expected behavior
Could you please check the branch difference, so that the multi-backend-refactor could be installed successfully in the latest Rocm6.4 environment, thanks so much !
Activity
deep1401 commentedon May 9, 2025
Facing the same error, wonder if you found a solution to this? I have gfx1100 arch of the GPU and cannot compile successfully on bare metal
smCloudInTheSky commentedon Jun 13, 2025
Got the same issue with AMD 790XTX within a devcontainer.
if you need any logs from my install please say so I'll provide it/test anything if needed !
TotallyTroll commentedon Jul 17, 2025
With latest release support matrix shows that AMD GPUs (gfx1100) are now supported in Linux.
Yet when installing from release wheel - it uninstalls pytorch-rocm.
So, when I run 'python -m bitsandbytes' it shows next output:
How to install and use it properly with ROCm?
matthewdouglas commentedon Jul 17, 2025
@TotallyTroll
You could try to install a ROCm PyTorch first, and then use
--no-depsif the installation of bitsandbytes causes torch to inadvertently reinstall with a CUDA build.TotallyTroll commentedon Jul 18, 2025
@matthewdouglas
It installs bitsandbytes but it doesn't work without deps:
icefairy64 commentedon Jul 23, 2025
@TotallyTroll - works fine for me using older official ROCm 6.3.4 Docker image:
$ podman run -it --rm --device /dev/kfd --device /dev/dri/renderD128 musubi-tuner:20250723-6.3.4 python3 -m bitsandbytes================ bitsandbytes v0.47.0.dev0 ================= Platform: Linux-6.14.11-300.fc42.x86_64-x86_64-with-glibc2.39 libc: glibc-2.39 Python: 3.12.9 PyTorch: 2.4.0a0+git7cecbf6 CUDA: N/A HIP: 6.3.42134-a9a80e791 XPU: N/A Related packages: accelerate: 1.6.0 diffusers: 0.32.1 numpy: 1.26.0 pip: 25.0 peft: not found safetensors: 0.4.5 transformers: 4.46.3 triton: 3.0.0 trl: not found ============================================================ PyTorch settings found: ROCM_VERSION=63 Checking that the library is importable and ROCm is callable... SUCCESS!Notably, using images with ROCm 6.4+ fails:
ai-nikolai commentedon Aug 26, 2025
@TotallyTroll @matthewdouglas @icefairy64 - is there any update on this and any recommended way of building
bitsandbytesfrom "source" on AMD?paulogonc commentedon Aug 29, 2025
Same issue on my machine using Docker image: rocm/pytorch:rocm6.4.3_ubuntu24.04_py3.12_pytorch_release_2.6.0 and manual installation of this latest main release bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_x86_64.whl
matthewdouglas commentedon Sep 15, 2025
Hi everyone,
We do not build for ROCm 6.4+ yet, so it would be best for the time being to use ROCm 6.1-6.3 if you're installing from our preview wheels.
If you're building from source, you can clone the repo and build/install with:
deep1401 commentedon Sep 16, 2025
Hi @matthewdouglas,
I tried the command you provided but it kept doing a core dump on
python -m bitsandbytes.I found this somewhere and this seems to be working for me:
NO-ob commentedon Sep 19, 2025
this didnt work i had to add -DBNB_ROCM_ARCH="gfx1100" to the cmake command to get the 8bit optimizers
mrs83 commentedon Nov 5, 2025
On the main branch:
This is probably due to the pre-compiled pytorch version. I will try again by compiling pytorch as well. Unfortunately bitsandbytes wheel is not available on PyTorch's https://download.pytorch.org/whl/nightly/rocm7.0
I also have difficulties to make this working in a project using
uvas package manager (instead of pip). the pre-compiled wheel also doesn't work in my fresh PopOS 24.04 installation with ROCm 7.1 (after following AMD installation guide https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/quick-start.html). Any suggestion?mrs83 commentedon Nov 6, 2025
I confirm this works with ROCm 7.0.
redo33 commentedon Dec 22, 2025
I wonder how to compile bitsandbytes with rocm7 on Windows 11.
Thanks.
rwfsmith commentedon Dec 26, 2025
I've been trying to get this working, but it's not been very successful yet. I've gotten successful builds, but it returns junk data or blank lines when trying load a model in 4bit and perform the inference.
rwfsmith commentedon Dec 26, 2025
I just tried it again with the latest TheRock ROCm build and it... worked? Maybe? I'll run some more tests.
rwfsmith commentedon Dec 26, 2025
I was able to load Qwen3 30B A3B Instruct 2507 and run some inference, although slowly. There's some odd versioning issues with the 7.10 preview and the TheRock builds, some things return the version as 7,2 and other things as 7.1, but seems to load fine after updating some of the libraries to work with windows and also to switch to using hipinfo to check the warpsize.
rwfsmith commentedon Dec 26, 2025
I don't think I have permission to create a branch, but here's the changes I (Claude Opus 4.5) made:
bitsandbytes-rocm-windows.patch
And the documentation
Building bitsandbytes for ROCm on Windows
This guide describes how to build bitsandbytes for AMD ROCm on Windows using the PyTorch ROCm nightly builds.
Prerequisites
Setup
1. Create and activate a virtual environment
2. Install PyTorch with ROCm support for gfx1151
pip install --index-url https://rocm.nightlies.amd.com/v2/gfx1151/ torch torchaudio torchvision pip install --index-url https://rocm.nightlies.amd.com/v2/gfx1151/ "rocm[libraries,devel]"3. Extract the devel package into the SDK core directory
The ROCm devel package installs as a separate tar archive that needs to be manually extracted:
This extracts the
_rocm_sdk_develsubdirectory from the tar file into_rocm_sdk_core, skipping any files that already exist. This provides the CMake config files (hip-lang, hipblas, etc.) needed for the build.4. Clone bitsandbytes and apply the Windows patch
git clone https://github.com/bitsandbytes-foundation/bitsandbytes.git cd bitsandbytes git apply ..\bitsandbytes-rocm-windows.patchBuild Commands
Run these commands from a Developer Command Prompt for VS 2022 or a regular Command Prompt with the environment variables set.
Set environment variables
Adjust the paths below to match your installation:
Note: Adjust the Windows SDK version (10.0.22621.0) and MSVC version (14.44.35207) to match your installation.
Configure with CMake
Update -DCMAKE_HIP_ARCHITECTURES=gfx1151 to match your GPU architecture.
Build
Build wheel
The wheel will be created in the
dist\folder.Install
Test
python -c "import torch; import bitsandbytes as bnb; layer = bnb.nn.Linear4bit(512, 256, bias=False, compute_dtype=torch.float16, quant_type='nf4'); layer = layer.cuda(); x = torch.randn(1, 512, dtype=torch.float16, device='cuda'); y = layer(x); print('Success! Output shape:', y.shape)"Patch Summary
The patch makes the following changes:
CMakeLists.txt
amdhip64androcblaslibraries for Windows - The upstream intentionally excludeship::amdhip64to avoid accuracy issues in tests, but this breaks linking on Windows since the HIP runtime symbols are required.RUNTIME_OUTPUT_DIRECTORYsettings for all Windows builds (not just MSVC), so the DLL is automatically placed in thebitsandbytes/folder.bitsandbytes/cuda_specs.py
hipinfo.exeinstead ofrocminfoon Windows to detect GPU architecture and wavefront size.csrc/ops_hip.cuh
<unistd.h>only on non-Windows platforms (it doesn't exist on Windows).Troubleshooting
"HIP error: invalid argument" at runtime
https://rocm.nightlies.amd.com/v2/gfx1151/CMake can't find hip-lang-config.cmake
pip install --index-url https://rocm.nightlies.amd.com/v2/gfx1151/ "rocm[devel]"Linker errors about undefined symbols (hipMalloc, rocblas_create_handle, etc.)
hip::amdhip64androc::rocblaslibraries on WindowsI'm not really an expert in this stuff, I'm just a lowly .NET developer, so I probably can't help debug any issues, so good luck!
matthewdouglas commentedon Feb 24, 2026
Hi all,
There's multiple issues being discussed here. Since we ship ROCm 6.2 - 7.2 binaries for Linux x86-64 starting in v0.49.0, I would recommend trying the latest release first.
For issues relating to the ROCm version you build with being mismatched with the PyTorch version, we've merged #1878 which will be included in our next release.
On Windows we've merged #1846, but do not yet distribute this build, so it needs to be built from source.
If there's further problems please open a new issue. Thanks!