Repository navigation
Conversation
This was referenced Sep 18, 2026
Owner
Author
|
🤖 Human guided, AI assisted comment (using this skill). AI text below. 🤖 FindingsSame process, rat atlas (1,074 MB), both decode paths: Results are Partial downloads: the only "downloaded" signal is the Wrong level: Assisted-by: Claude-Code:Opus-5 |
tabedzki
force-pushed
the
fix/atlas-decode-peak
branch
from
September 18, 2026 15:47
8d358ae to
8912dde
Compare
…ht level brainglobe's `annotation` property materialises the volume with `dask.compute()`: every chunk is decompressed into its own buffer and only then concatenated, so the process briefly holds two copies. Measured on whs_sd_rat_39um (1.07 GB array): 2,305 MB peak RSS. On a 3 GB container that peak, on top of the Bokeh baseline and whatever atlas is already resident, is what OOM-kills the server when a user computes an overlay with a second large atlas. `load_annotation` reads the same chunks straight through zarr indexing into one preallocated array. Peak drops to 1,263 MB (the array plus baseline), the decode is ~2x faster, and the result is byte-identical. The pyramid level is chosen from the standard OME `multiscales` metadata by matching the atlas resolution. Nothing in this module touches `atlas.annotation` any more. Owning that step exposes two more problems in the same layer: * A killed download passed for a finished one. `fs.get` writes chunks one at a time, and the `c` directory it creates is the only "already downloaded" signal both this loader and brainglobe use; an OOM kill mid-transfer (which a large atlas is exactly the thing to cause) left a partial `c` that was trusted forever and read back with the missing chunks as zeros. Counting chunks can't detect this -- a complete download omits all-zero chunks (the rat atlas ships 217 of 512) -- so the fetch now lands in a staging directory and is renamed into place. * `is_downloaded` accepted *any* cached pyramid level. One annotation set serves a whole atlas family: allen_mouse_25um is level s1, allen_mouse_10um is s0. With 25 µm baked into the image, the 10 µm atlas (4.8 GB, nothing local) reported as downloaded, so the GUI took its "this is free" path and the download ran inline on the Bokeh event loop. The check now resolves the atlas's own level. `ensure_downloaded` now loads via `canonical_annotation` so the volume decoded on the worker thread is the one the overlay compute reads, not a throwaway copy. Assisted-by: Claude-Code:Opus-5
tabedzki
force-pushed
the
fix/atlas-decode-peak
branch
from
September 18, 2026 15:49
8912dde to
c902868
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 Human guided, AI assisted PR (using this skill). AI text below. 🤖
Problem. brainglobe decodes via
dask.compute(), holding every chunk plus the result: the 1.07 GB rat atlas peaks at 2,305 MB. Two related bugs: a download killed mid-way leaves acdir that is trusted forever (missing chunks read as 0; chunk counts can't detect it since all-zero chunks are omitted), andis_downloadedaccepted any pyramid level, so with 25 µm cached the 4.6 GB 10 µm atlas reported as downloaded and fetched inline on the event loop.Change.
load_annotationreads through zarr into one preallocated array (peak 1,263 MB, 2× faster, byte-identical). Downloads stage in<level>.partialand rename into place._annotation_is_cachedchecks the atlas's own level.ensure_downloadedloads viacanonical_annotation.Tests use a tiny real OME-Zarr pyramid with an
annotationproperty that raises. Independent of #2; second to land needs a small rebase.Assisted-by: Claude-Code:Opus-5