fix(vtk-module): route remaining serializer imports through vtk_module - #124
Conversation
data.py, utils.py and serialize.py still imported from vtkmodules directly, so VTK_MODULE_NAME only applied to part of the vtk serializers. With a custom build and no stock vtk installed the serializers fail to import; with both installed, objects from two VTK builds mix and fail later with wrapped-type TypeErrors. Same change as 090b504. serialize.py only used vtk for vtkCommand.DeleteEvent, so it now imports vtkCommand from vtk_module.vtkCommonCore instead of the legacy monolithic vtk module. Default is unchanged: VTK_MODULE_NAME still falls back to vtkmodules.
trame's serializers hardcode two `vtkmodules` imports, so they build stock-VTK objects even when VTK_MODULE_NAME points elsewhere. Fixed upstream in Kitware/trame-vtk#124; until that is released the export cannot work on an alternative build, and pulling stock VTK into the process alongside cvista crashes the interpreter rather than failing cleanly. Three exclusions, all removable when trame-vtk#124 ships: - PYVISTA_BUILDING_GALLERY=false in the integration-cvista env. The gallery vtksz capture in show() is what dragged 198 otherwise-unrelated plotting tests (picking, widgets, renderer, charts) through the export. - skip_vtk_backend on the 9 tests that drive the export directly, or force BUILDING_GALLERY on themselves: the trame export tests, test_tinypages, and the two scraper tests. - The jupyter session warm-up fixture skips its export on a non-stock build. It runs before any marker can apply, so it took the whole session down. Verified against released trame-vtk 2.11.15: cvista 53 passed / 3 skipped (was an interpreter crash), stock VTK 56 passed, core suites unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
…error Four review points, plus the trame workaround replaced by a real validation. Probe the flat namespace instead of inferring it from the backend name. `_VTK_ROOT != 'vtkmodules'` assumed every custom build was flat, so pointing PYVISTA_VTK_BACKEND at a stock-layout build failed at `import pyvista` with "Cannot import name 'vtkMatrix3x3'" -- blaming the class for a missing module. It now imports the root and checks whether classes are actually on it, and a non-importable backend says so by name. Skip reasons are defined once in tests/vtk_backend_divergence.py and imported. Nine sites repeated the same vtkPOpenFOAMReader string, four the snake_case one. When cvista starts shipping a module, that is now one edit. The trame mismatch raises RuntimeError, not ImportError. show() captures scenes under a guard that ignores a missing trame, which swallowed the very error meant to replace a silent failure. A test asserts the raised error is not an ImportError. Dropped the test-count from a tox comment. It said 198; the next run said 627. CI installs trame-vtk from the commit proposed in Kitware/trame-vtk#124 (pinned by sha, not branch) rather than skipping around the bug. That validates the upstream fix on every run and lets the gate exercise scene export for real: the nine skip markers, the jupyter warm-up guard and PYVISTA_BUILDING_GALLERY=false are all gone. Locally with that trame: jupyter/trame 56 passed on cvista with no skips, 56 on stock, core 5235/5259. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
|
Hi @jourdain would appreciate a review on this as it's a blocker for pyvista/pyvista#8787 Thanks! |
|
@akaszynski thx for the PR. |
Thibault-Pelletier
left a comment
There was a problem hiding this comment.
Merge pending pre-commit fix
The preamble is module-level code, so every import after it is E402. data.py already marks all of them, including the relative ones; serialize.py only had the vtk_module line marked because that was the line I changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
|
Thanks @Thibault-Pelletier. Fixed in 5cef7f4. The preamble is module-level code, so every import below it is E402. Worth flagging one thing for whenever this is released: pinning the serializer imports to |
Kitware/trame-vtk#124 merged and shipped in 2.11.16, so the gate no longer needs a git sha. Now a normal requirement: trame-vtk>=2.11.16. Verified with the release rather than the branch: tests/plotting/jupyter is 56 passed under the cvista backend. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
Reverts the swap to `trame-vtk>=2.11.16`. trame-pyvista 0.1.6 upper-pins
`trame-vtk<2.11.16`, so requiring the release makes the environment
unsatisfiable:
trame-pyvista 0.1.6 has requirement trame-vtk<2.11.16,
but you have trame-vtk 2.11.16
Caught by @user27182 in review. Now pinned to the UPSTREAM merge commit of
Kitware/trame-vtk#124 rather than my fork's branch: same code, reports version
2.11.15, which the pin allows. `pip check` is clean.
pyvista/trame-pyvista#96 lifts the upper pin and adds the matching runtime check.
Once that ships this becomes `trame-vtk>=2.11.16`, one line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
trame-pyvista 0.1.7 lifted the `trame-vtk<2.11.16` upper pin (pyvista/trame-pyvista#96), so the gate no longer needs a git sha to get the serializer fix from Kitware/trame-vtk#124. Now `trame-vtk>=2.11.16` and `trame-pyvista>=0.1.7`, both released. `pip check` is clean on that pair, and tests/plotting/jupyter is 56 passed on the cvista backend with them. This was @user27182's objection: nothing in the gate is pinned to a git commit any more. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
VTK_MODULE_NAME(added in 448530a) only works for part of trame. Three imports in the vtk serializers still hardcodevtkmodules, so a custom VTK build gets used for some of the code and stock VTK for the rest.serializers/data.pyshows it best. It setsvtk_moduleon line 7, imports from it on line 9, then goes back tovtkmodulesfour lines later:The other two are
serializers/utils.py(from vtkmodules.util import numpy_support) andserializers/serialize.py(a bareimport vtk).What it looks like in practice. With a custom build and no stock
vtkinstalled alongside it, the serializers don't import at all:If stock
vtkis installed too, the import succeeds and you end up with objects from two VTK builds in one process. That fails later and further away, with errors likeTypeError: GetGlobalId argument 1:andTypeError: ExportLegacyFormat argument 1:.Fix. Same change as 090b504, applied to the three that were missed.
serialize.pyonly usedvtkforvtk.vtkCommand.DeleteEvent, so it becomesfrom vtk_module.vtkCommonCore import vtkCommand.Backwards compatibility. The default is still
os.environ.get("VTK_MODULE_NAME", "vtkmodules"), so nothing changes unless the variable is set. PyVista's jupyter/trame suite, three ways:VTK_MODULE_NAMEset, aftervtk.vtkCommand is vtkmodules.vtkCommonCore.vtkCommandis True, so theserialize.pychange is a no-op on stock VTK. It also takesimport vtkout of the serializer import path, which pulls in all of VTK: 384 ms against 48 ms forvtkmodules.vtkCommonCoreon this machine.Left out.
modules/paraview/protocols/mouse_handler.pyandpublish_image_delivery.pyhave the same hardcodedvtkWebCoreimport. I didn't touch them since I have no way to test against ParaView. Happy to include them if you'd rather do it in one go.Tested on Linux with Python 3.12 and 3.13, against stock VTK 9.6.2 and a custom 9.6.2 build. Not tested on Windows, macOS, or ParaView.
Attribution: this was investigated and written with Claude Opus 5.0. I reviewed the work and helped point it at the root cause, but the model did most of the digging and verification. I don't want to pass it off as solely my own.