Skip to content

fix(vtk-module): route remaining serializer imports through vtk_module - #124

Merged
Thibault-Pelletier merged 2 commits into
Kitware:masterfrom
akaszynski:fix/vtk-module-remaining-imports
Aug 14, 2026
Merged

fix(vtk-module): route remaining serializer imports through vtk_module#124
Thibault-Pelletier merged 2 commits into
Kitware:masterfrom
akaszynski:fix/vtk-module-remaining-imports

Conversation

@akaszynski

Copy link
Copy Markdown
Contributor

VTK_MODULE_NAME (added in 448530a) only works for part of trame. Three imports in the vtk serializers still hardcode vtkmodules, so a custom VTK build gets used for some of the code and stock VTK for the rest.

serializers/data.py shows it best. It sets vtk_module on line 7, imports from it on line 9, then goes back to vtkmodules four lines later:

vtk_module_name = os.environ.get("VTK_MODULE_NAME", "vtkmodules")
sys.modules["vtk_module"] = importlib.import_module(vtk_module_name)

from vtk_module.vtkFiltersGeometry import (...)
from vtkmodules.vtkCommonCore import vtkIdTypeArray   # missed

The other two are serializers/utils.py (from vtkmodules.util import numpy_support) and serializers/serialize.py (a bare import vtk).

What it looks like in practice. With a custom build and no stock vtk installed alongside it, the serializers don't import at all:

ModuleNotFoundError: No module named 'vtk'

If stock vtk is installed too, the import succeeds and you end up with objects from two VTK builds in one process. That fails later and further away, with errors like TypeError: GetGlobalId argument 1: and TypeError: ExportLegacyFormat argument 1:.

Fix. Same change as 090b504, applied to the three that were missed. serialize.py only used vtk for vtk.vtkCommand.DeleteEvent, so it becomes from vtk_module.vtkCommonCore import vtkCommand.

Backwards compatibility. The default is still os.environ.get("VTK_MODULE_NAME", "vtkmodules"), so nothing changes unless the variable is set. PyVista's jupyter/trame suite, three ways:

setup result
stock vtk, variable unset, before this PR 56 passed
stock vtk, variable unset, after 56 passed
custom build + VTK_MODULE_NAME set, after 56 passed (all failed before)

vtk.vtkCommand is vtkmodules.vtkCommonCore.vtkCommand is True, so the serialize.py change is a no-op on stock VTK. It also takes import vtk out of the serializer import path, which pulls in all of VTK: 384 ms against 48 ms for vtkmodules.vtkCommonCore on this machine.

Left out. modules/paraview/protocols/mouse_handler.py and publish_image_delivery.py have the same hardcoded vtkWebCore import. I didn't touch them since I have no way to test against ParaView. Happy to include them if you'd rather do it in one go.

Tested on Linux with Python 3.12 and 3.13, against stock VTK 9.6.2 and a custom 9.6.2 build. Not tested on Windows, macOS, or ParaView.


Attribution: this was investigated and written with Claude Opus 5.0. I reviewed the work and helped point it at the root cause, but the model did most of the digging and verification. I don't want to pass it off as solely my own.

data.py, utils.py and serialize.py still imported from vtkmodules directly,
so VTK_MODULE_NAME only applied to part of the vtk serializers. With a custom
build and no stock vtk installed the serializers fail to import; with both
installed, objects from two VTK builds mix and fail later with wrapped-type
TypeErrors.

Same change as 090b504. serialize.py only used vtk for vtkCommand.DeleteEvent,
so it now imports vtkCommand from vtk_module.vtkCommonCore instead of the
legacy monolithic vtk module.

Default is unchanged: VTK_MODULE_NAME still falls back to vtkmodules.
akaszynski added a commit to pyvista/pyvista that referenced this pull request Aug 13, 2026
trame's serializers hardcode two `vtkmodules` imports, so they build stock-VTK
objects even when VTK_MODULE_NAME points elsewhere. Fixed upstream in
Kitware/trame-vtk#124; until that is released the export cannot work on an
alternative build, and pulling stock VTK into the process alongside cvista
crashes the interpreter rather than failing cleanly.

Three exclusions, all removable when trame-vtk#124 ships:

- PYVISTA_BUILDING_GALLERY=false in the integration-cvista env. The gallery
  vtksz capture in show() is what dragged 198 otherwise-unrelated plotting
  tests (picking, widgets, renderer, charts) through the export.
- skip_vtk_backend on the 9 tests that drive the export directly, or force
  BUILDING_GALLERY on themselves: the trame export tests, test_tinypages, and
  the two scraper tests.
- The jupyter session warm-up fixture skips its export on a non-stock build.
  It runs before any marker can apply, so it took the whole session down.

Verified against released trame-vtk 2.11.15: cvista 53 passed / 3 skipped
(was an interpreter crash), stock VTK 56 passed, core suites unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
akaszynski added a commit to pyvista/pyvista that referenced this pull request Aug 13, 2026
…error

Four review points, plus the trame workaround replaced by a real validation.

Probe the flat namespace instead of inferring it from the backend name.
`_VTK_ROOT != 'vtkmodules'` assumed every custom build was flat, so pointing
PYVISTA_VTK_BACKEND at a stock-layout build failed at `import pyvista` with
"Cannot import name 'vtkMatrix3x3'" -- blaming the class for a missing module.
It now imports the root and checks whether classes are actually on it, and a
non-importable backend says so by name.

Skip reasons are defined once in tests/vtk_backend_divergence.py and imported.
Nine sites repeated the same vtkPOpenFOAMReader string, four the snake_case
one. When cvista starts shipping a module, that is now one edit.

The trame mismatch raises RuntimeError, not ImportError. show() captures scenes
under a guard that ignores a missing trame, which swallowed the very error meant
to replace a silent failure. A test asserts the raised error is not an
ImportError.

Dropped the test-count from a tox comment. It said 198; the next run said 627.

CI installs trame-vtk from the commit proposed in Kitware/trame-vtk#124 (pinned
by sha, not branch) rather than skipping around the bug. That validates the
upstream fix on every run and lets the gate exercise scene export for real: the
nine skip markers, the jupyter warm-up guard and PYVISTA_BUILDING_GALLERY=false
are all gone. Locally with that trame: jupyter/trame 56 passed on cvista with no
skips, 56 on stock, core 5235/5259.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
@akaszynski

Copy link
Copy Markdown
Contributor Author

Hi @jourdain would appreciate a review on this as it's a blocker for pyvista/pyvista#8787

Thanks!

@Thibault-Pelletier
Thibault-Pelletier self-requested a review August 14, 2026 08:41
@Thibault-Pelletier

Thibault-Pelletier commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

@akaszynski thx for the PR.
It looks good to me. I'll let you address the pre-commit issue before merging!

@Thibault-Pelletier Thibault-Pelletier left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge pending pre-commit fix

The preamble is module-level code, so every import after it is E402. data.py
already marks all of them, including the relative ones; serialize.py only had
the vtk_module line marked because that was the line I changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
@akaszynski

Copy link
Copy Markdown
Contributor Author

Thanks @Thibault-Pelletier. Fixed in 5cef7f4.

The preamble is module-level code, so every import below it is E402. data.py already marks all of them including the relative ones; I had only marked the line I changed in serialize.py. Verified with the pinned ruff 0.15.10 against the repo config: clean, and ruff format --check reports no changes.

Worth flagging one thing for whenever this is released: pinning the serializer imports to vtk_module is what lets a custom build work end to end. PyVista's cvista integration goes from 118 failures to 4 with this branch installed, which is how the remaining three imports were found in the first place.

@Thibault-Pelletier
Thibault-Pelletier merged commit 47e4af0 into Kitware:master Aug 14, 2026
3 checks passed
akaszynski added a commit to pyvista/pyvista that referenced this pull request Aug 14, 2026
Kitware/trame-vtk#124 merged and shipped in 2.11.16, so the gate no longer needs
a git sha. Now a normal requirement: trame-vtk>=2.11.16.

Verified with the release rather than the branch: tests/plotting/jupyter is
56 passed under the cvista backend.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
akaszynski added a commit to pyvista/pyvista that referenced this pull request Aug 14, 2026
Reverts the swap to `trame-vtk>=2.11.16`. trame-pyvista 0.1.6 upper-pins
`trame-vtk<2.11.16`, so requiring the release makes the environment
unsatisfiable:

    trame-pyvista 0.1.6 has requirement trame-vtk<2.11.16,
    but you have trame-vtk 2.11.16

Caught by @user27182 in review. Now pinned to the UPSTREAM merge commit of
Kitware/trame-vtk#124 rather than my fork's branch: same code, reports version
2.11.15, which the pin allows. `pip check` is clean.

pyvista/trame-pyvista#96 lifts the upper pin and adds the matching runtime check.
Once that ships this becomes `trame-vtk>=2.11.16`, one line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
akaszynski added a commit to pyvista/pyvista that referenced this pull request Aug 14, 2026
trame-pyvista 0.1.7 lifted the `trame-vtk<2.11.16` upper pin
(pyvista/trame-pyvista#96), so the gate no longer needs a git sha to get the
serializer fix from Kitware/trame-vtk#124.

Now `trame-vtk>=2.11.16` and `trame-pyvista>=0.1.7`, both released. `pip check`
is clean on that pair, and tests/plotting/jupyter is 56 passed on the cvista
backend with them.

This was @user27182's objection: nothing in the gate is pinned to a git commit
any more.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nc3mSGXDvAuGDG3X9NiQPR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants