quantize_blockwise (CPU) occurs Segmentation fault when all block elements are zero

System Info

Python: 3.10.18
bitsandbytes: 0.47.0
CPU: Intel(R) Xeon(R) CPU E5-2682 v4 @ 2.50GHz
System: centos 7

Reproduction

from bitsandbytes.functional import quantize_blockwise, create_dynamic_map
import torch

if __name__ == "__main__":
    block_size = 256

    # success
    tensor = torch.rand(10000)
    tensor[: block_size - 1] = 0
    state, quant_state = quantize_blockwise(A=tensor, blocksize=block_size)
    print(state)

    # success
    state, quant_state = quantize_blockwise(A=tensor.cuda(), blocksize=block_size)
    print(state)

    # success
    tensor = torch.rand(10000)
    tensor[:block_size] = 0
    state, quant_state = quantize_blockwise(A=tensor.cuda(), blocksize=block_size)
    print(state)

    # Segmentation fault (core dumped)
    state, quant_state = quantize_blockwise(A=tensor, blocksize=block_size)
    print(state)

Expected behavior

When all block elements are zero, the quantize-blockwise function can run successfully on the GPU, but segmentation errors will occur on the CPU.

Hi guys, I have the same issue in Intel i7-10750H.

t = torch.randn([64, 64], dtype=dtype, device=device, requires_grad=requires_grad)
# make_nf4 has the same steps like this post
q = make_nf4(t) # work
q2 = make_nf4(torch.empty_like(t)) # work
q3 = make_nf4(torch.zeros_like(t)) # SEG FAULT
q4 = make_nf4(torch.zeros_like(t), device="cuda") # work

This is the last call stack:

Fatal Python error: Segmentation fault
Thread 0x000072f184fbc140 (most recent call first):
/site-packages/bitsandbytes/backends/cpu/ops.py", line 41 in _
...

Line 40-47 in cpu/ops.py is :

lib.cquantize_blockwise_cpu_fp32(
            get_ptr(code),
            get_ptr(A),
            get_ptr(absmax),
            get_ptr(out),
            ct.c_longlong(blocksize),
            ct.c_longlong(n),
        )

So the problem maybe in the get_ptr or .
Also, the same problem, raise at the exactly above line, that when try to quantize the tensor that has numel()<256

t = torch.randn([16, 16], device=device, dtype=dtype, requires_grad=requires_grad)
nf4 = make_nf4(t) # work 

t = torch.randn([8, 8], device=device, dtype=dtype, requires_grad=requires_grad)
nf4 = make_nf4(t) # SEG FAULT, BUT at line 40, ``lib.cquantize_blockwise_cpu_fp32`...`

# And again, error occurs only in cpu, this code works.
t = torch.randn([8, 8], device="cuda", dtype=dtype, requires_grad=requires_grad)
nf4 = make_nf4(t)

System Info

Python: 3.12.3
bitsandbytes: 0.47.0
CPU: Intel i7-10750H (12) @ 5.000GHzSystem: centos 7
OS: Kubuntu 24.04.2 LTS x86_64

More info

This is the implementations of make_nf4 above:

        deq_state: tuple[Tensor, Tensor] = torch.ops.bitsandbytes.quantize_4bit.default(
            hp_tensor,
            config.block_size,
            config.quant_type,
            torch.uint8,
        )
        # shape (flatten_shape, 1), (ceil(flatten_shape/block_size), )
        quant_data, data_scales = deq_state
        assert quant_data.device == hp_tensor.device == data_scales.device

        # 2. Quantize data scales residual ->  quant_scale_res, residual_scales, data_scales_offset
        # bitsandbytes.backends.default.ops.quantize_blockwise
        # Quantize the scale residuals
        device = data_scales.device
        scales_dtype = config.scales_dtype
        data_scales_offset = data_scales.mean()

        deq_state2: tuple[Tensor, Tensor] = (
            torch.ops.bitsandbytes.quantize_blockwise.default(
                (data_scales - data_scales_offset).to(scales_dtype).contiguous(),  # Residual
                cls._get_double_quantize_code(device).to(scales_dtype).contiguous(),
                config.scale_block_size,
            )
        )
        quant_scale_res, residual_scales = deq_state2

#... Saving all states....

Actually i don't need the fix right now, I just hope to know all the case that can cause the problem, so i can do validation before quantization.
Thank you guys.

Hi, I found the problem, all above cases fail because of this:

So that, when all elements in one block are zeros, the absmax_block also be zeros, and div zeros return nan and cause segmentation fault in bin_searcher.
Also, the case that Tensor.numel()<blocksize leading to len(absmax.shape)==1 and the A (offset) ==absmax_block==0.

Hope the fix soon.

This is fixed by #1804.

Navigation Menu

Repository navigation

quantize_blockwise (CPU) occurs Segmentation fault when all block elements are zero #1744

System Info

Reproduction

Expected behavior

System Info

More info

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

quantize_blockwise (CPU) occurs Segmentation fault when all block elements are zero #1744

Description

System Info

Reproduction

Expected behavior

Activity

hieubnt235 commented on Sep 12, 2025

System Info

More info

hieubnt235 commented on Sep 12, 2025

matthewdouglas commented on Dec 11, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions