Repository navigation
VTK algorithms' dataset cache: an attribute of the VTK dataset's wrapper, no observer - #340
Merged
Merged
Conversation
…ped at the next use The cache of Tack datasets per VTK dataset drops an entry from a DeleteEvent observer, a function only the VTK object referred to. When Python's collector freed such a dataset in a reference cycle made after it was read (a failing test's traceback holding its frames), it took the observer for garbage too and cleared its globals and closure first; the deletion then called it, and Python segfaulted (_PyEval_EvalFrameDefault from vtkPythonCommand::Execute, from vtkObject::ObjectFinalize, from the collector). VTK 9.7.1 and master both; a VTK-only reproduction is VTK bugs 19 in the vault. The observers are now kept in a module-level dict until they fire, and do no more than note the address; the cache drops the entry, and the observer, at its next use, outside whatever freed the dataset. The regression test makes the cycle after the dataset was read, as the crash needs; it segfaulted before.
… the input The summaries reduce everything they are given -- every dataset of a partitioned dataset or a tree, on every MPI rank (the run's world) -- to one answer, as VTK's vtkDescriptiveStatistics and vtkIntegrateAttributes reduce composite data; every rank gets it. - Statistics: a vtkTable, a row per field and a vector field's components one by one (ExtractComponent), columns field, where, component, td.statistics' values under Tack's names, and the unit. - Entropy: the same rows, with the entropy in bits over `bins` bins. - Integrate: vtkIntegrateAttributes' output, one vertex at the measure-weighted centroid (the integral of position over the measure), each point field's integral as point data and each cell field's as cell data, the measure as Length, Area or Volume by the cells' dimension. - Fields: `fields`, SetInputArrayToProcess, or every field. Held to td.statistics and td.entropy called directly, Integrate to vtkIntegrateAttributes (centroid, volume, a point and a vector cell field), pieces and trees to the whole (each shared point counted once where GhostCells or GenerateGlobalIds mark its owner), a surface's measure to its area, and 3 MPI ranks to the whole.
…per, no observer The cache of Tack datasets per VTK dataset kept entries by address and dropped them from a DeleteEvent observer, which needed a registry to keep the observers reachable (VTK bugs 19: the collector cleared one as garbage, then the deletion called it) and deferred drops. The entry is now an attribute of the dataset's Python wrapper: VTK keeps a wrapper's attributes while its C++ object lives, on every later wrapper of it (GetInputData's in the next filter), and frees them with the object, so an entry lives exactly as long as its dataset with no observer. Measured (yavin, the wheel and a VTK master build): on master a wrapper costs 0.6 us however many objects keep attributes (10,000), and Contour over 1,000 and 5,000 boxes runs as fast with the attribute as with the observer (951 against 943 ms, 4,803 against 4,817); VTK 9.7.1 sweeps kept attributes at every wrapper (50 us with 10,000), which it does for the attributes the pythonic API gives every wrapper anyway, and the two caches cost the same there too. An entry is stale when the dataset's modification time or time step differs from when it was cached, as before. The comment says what that sees: Modified() (an array's, the points'), arrays added or removed; not values written in place without it, nor InsertNextCell, which VTK's own pipeline would not run again for either. Tests: the entry outlives the wrapper and goes with the dataset, goes in the collector's cycle (the crash's case), and is stale after Modified() or a new time step but not after a write without it.
This was referenced Nov 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #301 (its commits show here until it merges; this merges after it).
The cache of Tack datasets per VTK dataset (one Tack dataset per VTK dataset, so locators, faces and point links are built once) kept entries by address and dropped them from a
DeleteEventobserver. #301 kept those observers reachable to work around VTK bugs 19 (the collector cleared an observer as garbage, then the deletion called it). This replaces the observers altogether: the entry is an attribute of the VTK dataset's Python wrapper. VTK keeps a wrapper's attributes while its C++ object lives, on every later wrapper of it (the oneGetInputDatagives the next filter), and frees them with the object, so an entry lives exactly as long as its dataset.Measured on yavin, VTK 9.7.1 (wheel) and VTK master (a local build):
Contourover 1,000 boxes, cached: observer / attributeContourover 5,000 boxes, cached: observer / attributeThe two caches cost the same on both; 9.7.1's growth with the box count is its sweep of the attributes the pythonic API gives every wrapper, with either cache.
Staleness is as before: an entry is converted again when the dataset's modification time or time step differs from when it was cached. The comment and the changelog say what that sees (
Modified(), arrays added or removed; not values written in place withoutModified(), norInsertNextCell, which VTK's own pipeline would not rerun for either) and ask forModified()after changing a dataset in place.Tested: the entry outlives the wrapper and goes with the dataset; it goes in the collector's cycle (VTK bugs 19's case, which crashed before #301); it is stale after
Modified()or a new time step, not after a write without it. VTK tests 201 passed on 9.7.1 and 217 on master; 3 MPI ranks; quick tier 3081 passed.🤖 Generated with Claude Code