[C++][Python] Safe decimal narrowing to Decimal32 drops coefficient bits before checking #53288

Describe the bug, including details regarding any error messages, version, and platform.

pc.cast to Decimal32 can return a different value with default safety enabled. It can also reject an exact scale-changing conversion.

Default safe narrowing can silently change decimal amounts, so downstream calculations may use incorrect values. Downstream application effects were not tested.

Reproduced on CPython 3.12.3, Linux x86_64, glibc 2.39, with the official pyarrow==26.0.0 wheel on 2026-10-11. PyArrow and Arrow C++ both report 26.0.0. The wheel is pyarrow-26.0.0-cp312-cp312-manylinux_2_28_x86_64.whl, SHA256 6a628922ba20705fa964ca73e4ef959c2fb2f14b9bbec5589a6a1e68e6257c85.

Reviewed release source: apache-arrow-26.0.0 at 446167169a0dc547e00e1bc29f417a40c0ca267a. The wheel does not expose its build commit. Main 5ef3c5200359bc8e2375daf5f989369b0c8c089b was read, not built or tested. Its cited kernel file is byte-identical to the release source.

Reproduction

The inline listing is an unexecuted shortening of the retained original reproducer and independent probes.

• Original reproducer and run record at 50bf6b998dd2c12ee3c04c6de2d339cd9b84ca3c: findings/pyarrow/decimal32-safe-cast/repro.py and tasks/pyarrow-b-r01/reproduction-results.json
• Independent probes, transcript and environment at c686294656f1f25e795ba897e4bccbec685aad99: findings/pyarrow/verify_independent.py, findings/pyarrow/decimal32-safe-cast/VERIFY.md and findings/pyarrow/VERIFY-ENVIRONMENT.md

Run these commands on Linux with Python 3.12.3, save the inline code as repro.py, then run the last command:

python3.12 -m venv .venv
.venv/bin/python -m pip install --index-url https://pypi.org/simple --only-binary=:all: pyarrow==26.0.0
.venv/bin/python repro.py

from decimal import Decimal
import pyarrow as pa
import pyarrow.compute as pc

assert pa.__version__ == pa.cpp_version == "26.0.0"
cases = [
    ("5000000000", 0, 0),
    ("536870912000", 0, -3),
    ("30000000.00", 2, 0),
]
for text, source_scale, target_scale in cases:
    value = Decimal(text)
    data = pa.array([value], pa.decimal128(12, source_scale))
    data.validate(full=True)
    assert data.to_pylist() == [value]
    assert pc.cast(data, pa.decimal128(15, target_scale)).to_pylist() == [value]
    try:
        got = pc.cast(data, pa.decimal32(9, target_scale)).to_pylist()
    except pa.ArrowInvalid as error:
        got = "ArrowInvalid: " + str(error)
    print(text, got)

Retained execution excerpts from findings/pyarrow/decimal32-safe-cast/VERIFY.md:

Original reproducer (exit 1):

PyArrow 26.0.0; C++ 26.0.0
Failed conformance cases: 96
AssertionError: Safe decimal32 narrowing violates the independently computed result

Independent probe:

The independent probe's exit 0 records successful input/control checks while printing disagreements; it does not mean the casts conform.

input 5000000000 target coefficient 5000000000 expected overflow error observed [Decimal('705032704')]
input 536870912000 target coefficient 536870912.000 expected [Decimal('536870912000')] observed [Decimal('0E+3')]
input 30000000.00 target coefficient 30000000.00 expected [Decimal('30000000.00')] observed ArrowInvalid: Rescaling Decimal value would cause data loss
PASS: raw/builder agreement, wider rescale, precision-edge and fractional-loss controls

Expected and observed

The cast contract enables overflow checks by default. CastOptions also disables decimal precision truncation by default. The Decimal32 definition permits negative scales.

The second destination coefficient is 536870912. It fits nine digits exactly.

Source

DecimalConversions<Decimal32, InDecimal> constructs Decimal32 from the low bits. SafeRescaleDecimal then rescales and checks precision. Source inspection indicates that this order checks an already narrowed coefficient. Keeping the wider coefficient through rescaling and validation would address that ordering.

Scope

Verification covered Decimal64/128/256 sources, default and explicit safe casts, scalar, array, slice, chunked and null inputs. Wider rescaling, precision-edge and fractional-loss rejection controls passed. The inline example selects three default safe array cases. Other releases, platforms and source builds were not tested.

The inline example is provided under CC0.

Assistance covered the example and report text as well as discovery and verification.

Found by ChipLab's AI verification agents and independently reproduced.

Component(s)

C++, Python

ChipLab ref: CLAB-2026-257