XMLParser can use a freed custom-entity value during dict mutation

[Nievesjyl]

Description

XMLParser.entity is a public mutable dictionary. When Expat reports an unresolved entity, expat_default_handler() obtains its value with PyDict_GetItemWithError() and retains only the borrowed result. It then passes that value to treebuilder_handle_data() or a Python callback without first taking a strong reference. Concurrent replacement of the same entry can release the value before the handler finishes using it.

Observed Behavior

The parser processed 500,000 custom-entity occurrences while another thread repeatedly replaced the corresponding dictionary value. The free-threaded ASan build reported a negative reference count on an already-freed object and aborted during cleanup. The GIL-enabled control completed normally.

Affected Version

CPython 3.14.7 at commit 823f0323ee6ec1402088b73bce1a38473cac36dc, built with --disable-gil --with-address-sanitizer.

Reproduction

ASAN_OPTIONS=detect_leaks=0:halt_on_error=1 PYTHON_GIL=0 python3.14 poc/reproduce.py --entities 500000

Setting PYTHON_GIL=1 provides the GIL-enabled control.

PoC Source Code

poc/reproduce.py

#!/usr/bin/env python3
"""Race XMLParser custom-entity resolution with entity-dict replacement."""

import argparse
import threading
import xml.etree.ElementTree as ET


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--entities", type=int, default=500_000)
    args = parser.parse_args()

    xml_parser = ET.XMLParser()
    xml_parser.entity["entity"] = "initial-" * 20
    document = (
        "<!DOCTYPE points [<!ENTITY % p SYSTEM 'x'>%p;]><document>"
        + "&entity;" * args.entities
        + "</document>"
    )
    stop = threading.Event()

    def mutate() -> None:
        generation = 0
        while not stop.is_set():
            xml_parser.entity["entity"] = (str(generation) + "-") * 50
            generation += 1

    worker = threading.Thread(target=mutate)
    worker.start()
    try:
        xml_parser.feed(document)
        xml_parser.close()
    finally:
        stop.set()
        worker.join()
    print("completed")


if __name__ == "__main__":
    main()

[ZeroIntensity]

Refer to #157124 (comment)