Skip to content

gh-156939: Detect buffer overflow in bytearray - #157382

Open
vstinner wants to merge 10 commits into
python:mainfrom
vstinner:bytearray_consistency
Open

gh-156939: Detect buffer overflow in bytearray#157382
vstinner wants to merge 10 commits into
python:mainfrom
vstinner:bytearray_consistency

Conversation

@vstinner

@vstinner vstinner commented Sep 12, 2026

Copy link
Copy Markdown
Member

Add bytearray_check_consistency() and
bytearray_check_trailing_null_byte() functions in call them in most bytearray methods.

Add bytearray_check_consistency() and
bytearray_check_trailing_null_byte() functions in call them in most
bytearray methods.
@vstinner

Copy link
Copy Markdown
Member Author

Tests / Sanitizers / TSan (pull_request): Failing after 13m

It's a legit bug:

WARNING: ThreadSanitizer: data race (pid=15518)
  Write of size 8 at 0x55b013af2f80 by main thread:
    #0 __tsan_memcpy <null> (python+0xfc452) (BuildId: 77d4ccbb4bf7bb80d66adcdbb8a43d9cd1f555bc)
    #1 set_allocator_unlocked /home/runner/work/cpython/cpython/Objects/obmalloc.c (python+0x38417f) (BuildId: 77d4ccbb4bf7bb80d66adcdbb8a43d9cd1f555bc)
    #2 PyMem_SetAllocator /home/runner/work/cpython/cpython/Objects/obmalloc.c:1143:5 (python+0x38417f)
    (...)

I reported the failure as #157415 and I proposed a fix (skip the test if TSAN is used).

@vstinner
vstinner marked this pull request as ready for review September 13, 2026 13:37
@vstinner

Copy link
Copy Markdown
Member Author

The PR adds bytearray_check_consistency(). Python has similar functions for other types:

  • any Python object: _PyObject_CheckConsistency()
  • type: _PyType_CheckConsistency()
  • dict: _PyDict_CheckConsistency()
  • str: _PyUnicode_CheckConsistency()
  • PyWideStringList C API: _PyWideStringList_CheckConsistency()

@vstinner

Copy link
Copy Markdown
Member Author

@cmaloney: Would you mind to review this change?

I didn't measure the overhead on runtime benchmark of a Python debug build. If bytearray_check_consistency() overhead is too high, the PR can be limited to add bytearray_check_buffer_overflow() assertions.

@vstinner

Copy link
Copy Markdown
Member Author

I just made a similar change to detect buffer overflow in PyBytesWriter: PR gh-156943.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant