Repository navigation
Conversation
Build resdata's riscv64 manylinux wheels from the upstream tag with cibuildwheel, mirroring upstream's testing.yml. Conan builds the statically linked dependencies from source; their licence texts are staged into the wheel and the elfutils source and recipe are archived for the release.
|
Same mechanic as the other recent campaign PRs:
|
…ated topk-sdk (#3230), zope-index (#3228), zccache (#3229), kfnetlist (#3231), ligo-skymap (#3233), quantile-forest (#3232), fbuild (#3235), py-ballisticcalc-exts (#3238), resdata (#3237), oxapy (#3239), pyramids-gis (#3240), biscuit-python (#3223), ansys-mapdl-reader (#3227) all had open PRs that queue.yml never reflected (status still queued, pr still null). Verified each PR independently before updating.
…urce pip install "cmake<4" can never resolve a riscv64 wheel -- the cmake PyPI project has published one only from 4.0 on -- so it fell back to building CMake's own C++ source, which failed on a missing OpenSSL dev package this image doesn't carry for an unrelated build dependency. Patches pyproject.toml's build-system requirement to drop the <4 upper bound (CMakeLists.txt's own cmake_minimum_required is 3.16, well under 4.0's removed-compatibility floor) and applies it the same way every other patched port in this repo does (checkout python-wheels alongside the external repo, git apply before the build step); updates CIBW_BEFORE_BUILD's matching manual pip install to match.
The cp314-manylinux_riscv64 leg died during CIBW_BEFORE_BUILD while Conan built ninja from source: cmake's compiler check link-tests with -static-libstdc++ -static-libgcc, and the manylinux_2_39_riscv64 (Rocky 10) image ships only the shared libstdc++.so, not libstdc++.a, so the link fails with "/usr/bin/ld: cannot find -lstdc++". This happens before any Python-version-specific code runs, so it is not cp314-specific - it would equally hit cp312/cp313 once their legs reach the same step. Same fix already proven in this repo for the identical image/error in build-ddtrace.yml, build-libclang.yml and build-coacd.yml: install libstdc++-static via CIBW_BEFORE_ALL_LINUX (CRB is already enabled on this image, so no --enablerepo flag is needed).
|
Failure: Root cause: this is riscv64-image-specific, not a cp314/pybind11/ABI issue, and not an upstream resdata bug. Fix: added I couldn't run a full riscv64 rehearsal locally (this sandbox's Docker can't reach |
Fixes PR #3237's riscv64 build failure: scikit-build's check_for_bad_installs() trips on the stale CMake cache left behind by python -m build's own metadata-then-wheel double invocation of setup.py (scikit-build/scikit-build#521, closed not-planned).
The check_for_bad_installs() failure ("CMake-installed files must be
within the project root", with .bin/.libs/.include under cmake-build/)
is caused by CMake 4, not by a stale _skbuild cache: it reproduces on a
fresh x86_64 checkout with cmake 4.4.4 and goes away with cmake 3.31.
setup.py passes -DCMAKE_INSTALL_{BIN,LIB,INCLUDE}DIR=python/resdata/...
untyped, so they enter the cache as UNINITIALIZED. CMake 4's
GNUInstallDirs no longer converts them to PATH, and CMakeLists.txt's
if(SKBUILD) block then does set(... CACHE PATH ... FORCE) on them,
which turns the relative value into an absolute path under the build
directory. The generated cmake_install.cmake then installs to
cmake-build/python/resdata/... instead of ${CMAKE_INSTALL_PREFIX}/...,
which skbuild rejects. CMake 3.x types the entries in GNUInstallDirs
first, so the forced set keeps them relative.
Upstream pins cmake<4 for this. The earlier premise that no riscv64
cmake<4 wheel exists was wrong: pypi.riseproject.dev carries
cmake-3.31.6-py3-none-manylinux_2_39_riscv64, but pip preferred PyPI's
newer, sdist-only 3.31.10 and tried to compile it (gotcha 30).
PIP_ONLY_BINARY=cmake makes both the build-isolation install and
CIBW_BEFORE_BUILD resolve to our wheel, so restore upstream's cmake<4
and drop both patches along with the steps that applied them.
All three legs of run 39022584611 stopped printing right after
tests/rd_tests/test_rd_cmp.py and sat silent until the job's
timeout-minutes: 300 cancelled them. The next test is
test_rd_file.py::ResdataFileTest::test_broken_file, which opens a text
file named CASE.FINIT. The F prefix makes it a formatted file, so
rd_kw_fread_header() calls rd_kw_qskip() to find the opening quote.
That loop stores fgetc() in a plain char and stops on c == EOF. char is
unsigned on riscv64, so EOF reads back as 255, the test is always false,
and the loop spins forever at the end of a file with no quote in it.
Reproduced on x86_64 by building resdata 6.3.3 with -funsigned-char:
test_broken_file hangs in ResdataFile("CASE.FINIT") (pytest-timeout's
thread method shows the stack; the default signal method never fires,
since control never returns to Python). With the patch, the whole suite
passes on that build.
util_fskip_lines() and fscanf_grdecl_data() have the same bug, so the
patch changes all three variables to int. Upstream fixed them after the
6.3.x branch (ce341a3, d9eacc1, c052640); no 6.3.x release carries the
fixes yet.
…ethod Only the first 27% of the suite has run on riscv64 so far, and a hang there holds a runner for the rest of the job's 300 minutes before it shows up as a bare "cancelled". pytest-timeout is already in upstream's test group, so this adds no dependency. The thread method is the one that works here: test_broken_file's hang is a busy loop in libresdata, so control never returns to Python and the default SIGALRM method never fires (checked on an x86_64 -funsigned-char build). The thread method dumps every thread's stack and exits, which fails the job and names the test. The slowest test seen on the riscv64 runner, test_grid_large_case, takes about 108 s (18.8 s on x86_64), so 900 s leaves wide margin.
The cp313 job did not hang a second time: it stopped at the last test of test_view_summary.py, test_performance_with_many_keys, whose own @pytest.mark.timeout(10) takes precedence over --timeout=900. With the thread method that marker inherits, the timeout ends the whole pytest session instead of failing one test, which made it read as a hang. The work is bounded: a call slower than max_time (1 s) makes pytest-benchmark run exactly 7 calls (2 calibration + 5 rounds). The captured stdout holds 6 complete summary.x tables, so the 10 s ran out during the seventh and last call, at about 1.4 s a call against 0.1 s on x86_64 (1.2 s for the whole test). A cProfile of one call is linear in the 5000 keys, with no quadratic lookup. 60 s keeps the test a regression guard while leaving the riscv64 runners about 5x headroom (gotcha 38).
… poison it The Apply patches step's git apply leaves the setuptools_scm checkout dirty, so git describe reports 6.3.4.dev0+g<hash>.d<date> instead of the clean 6.3.3 tag (gotcha 31) - that's why the publish job failed: the wheel's version has no matching entry in docs/packages/resdata.yaml, and PyPI's 6.3.3 build has no dev suffix to compare against anyway.
…SDATA suffix The previous commit's SETUPTOOLS_SCM_PRETEND_VERSION_FOR_RESDATA had no effect: setup.py calls setuptools_scm.get_version() directly without a dist_name argument, so setuptools_scm never learns the package name and can't match the per-package env var. Only the generic SETUPTOOLS_SCM_PRETEND_VERSION, which applies regardless of dist_name, actually reaches get_version() here. Confirmed by the publish job: the wheel version was still the dirty 6.3.4.dev0+g6bed49710.d20261018 string after the first fix.
resdata6.3.3Compiles the libresdata C/C++ core and its pybind11 extensions, with backward-cpp, fmt and elfutils built from source by Conan. Upstream publishes no riscv64 wheel.
Mirrors upstream's
testing.yml.Differs from upstream
build-frontendisbuild-build[uv]needs a host uv the runner lacks.before-buildalso stages the Conan dependencies' licence texts.gpl_sourcesjob archives the elfutils tarball and Conan recipe for the release.CIBW_ENVIRONMENTpinsSETUPTOOLS_SCM_PRETEND_VERSION- the Apply patches step'sgit applydirties the checkout, sogit describereports a post-6.3.3 dev version instead of the clean tag (gotcha 31). The bare env var is needed, not the_FOR_RESDATAsuffix:setup.pycallssetuptools_scm.get_version()directly without adist_name, so the per-package variant is never checked.Matrix: cp312/cp313/cp314 - upstream ships no free-threaded wheel.
Testing
test_performance_with_many_keysgets 60 s instead of upstream's 10 s (patch 0002).License: Wheel statically links elfutils (GPL-2.0-or-later or LGPL-3.0-or-later), backward-cpp, fmt, pybind11, zlib, bzip2, xz and zstd; upstream ships no licence text for them, so the build adds it.
Patches
0001-Fix-infinite-loops-at-EOF-where-char-is-unsigned.patch- Backport [https://github.com/equinor/resdata/commit/ce341a3666855c57839466d9daa876252b54c8c0]. Without ittest_broken_fileloops forever, sincecharis unsigned on riscv64. Reproduces on x86_64 with-funsigned-char.0002-Raise-test_performance_with_many_keys-timeout-for-sl.patch- Inappropriate [the 10s ceiling is sized for upstream's x86 CI]. The benchmark always does 7summary.xcalls over 5000 keys, about 1.4 s each on riscv64 against 0.1 s on x86_64. The 10 s marker ran out on the seventh call, and since its timeout takes precedence over--timeout=900, it ended the whole session. The work is linear in the key count; nothing loops.All three build legs (cp312/cp313/cp314) pass on real riscv64 hardware.