refactor: centralize zarr object-blob writes in _write_object_array#4527
Open
galenlynch wants to merge 1 commit intoSpikeInterface:mainfrom
Open
refactor: centralize zarr object-blob writes in _write_object_array#4527galenlynch wants to merge 1 commit intoSpikeInterface:mainfrom
galenlynch wants to merge 1 commit intoSpikeInterface:mainfrom
Conversation
Introduces a single helper in zarrextractors for writing JSON/Pickle-coded object-dtype arrays, and routes the six existing call sites (recording/sorting provenance + extension dict/object data) through it. Byte-identical to the previous inline create_dataset(..., object_codec=...) pattern under zarr v2, verified by analyzer round-trip diff. This prepares the call sites for the upcoming zarr-python 3 migration, where object_codec= is gone and object codecs live in filters= instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Groups the six call sites that write JSON/Pickle-coded object arrays (recording provenance, sorting provenance, and extension dict/object data) behind a single helper in
zarrextractors.py. Behavior-preserving on zarr v2 — verified by analyzer round-trip diff againstmain(identical bytes except for the non-deterministicruntime_sfield).Why
The
create_dataset(..., object_codec=numcodecs.JSON() | Pickle())pattern is a zarr v2 API shape —object_codec=is gone in zarr-python 3, where object codecs live infilters=(wrapped vianumcodecs.zarr3.*). Centralizing the six scattered call sites now means the zarr v3 cut-over only has to flip one function instead of chasing the pattern across two files.This is prep work for the eventual zarr-python 3 migration (currently blocked on
hdmf-zarrupstream, which pinszarr<3). The refactor stands on its own and is safe to merge independently of that migration's timeline.Changes
_write_object_array(group, name, data, codec="json"|"pickle")tosrc/spikeinterface/core/zarrextractors.py.SortingAnalyzer.create_zarrand two extension-write sites in the zarr extension path with calls to the helper.import numcodecsfrom two functions insortinganalyzer.py.Test plan
pytest src/spikeinterface/core/tests/test_zarrextractors.pypytest src/spikeinterface/core/tests/test_sortinganalyzer.pypytest src/spikeinterface/core/tests/test_baserecording.py src/spikeinterface/core/tests/test_basesorting.py src/spikeinterface/core/tests/test_analyzer_extension_core.pymain— identical moduloruntime_s.