Repository navigation
Write the paint logic once: Python that both the server and the browser run - #152
Merged
Merged
Conversation
The binary, broadcast, affinity, distance and interval targets move into cellmap_flow/portable/targets.py as numpy functions; the server's target_transforms.py classes keep their API and wrap them, with the same results (checked against the old classes on random patches and the 24 parity cases before the fixture went). The browser's head trainer, whole-network trainer and ranking call them in the engine's Pyodide worker through portable/targets_calls.py, from web/src/finetune/ target-call.ts, which keeps only what the browser knows of a model: that it is an affinity model, and the offsets its channel names say (no manifest on Hugging Face carries offsets). The affinity target takes the offsets as given, so the two modes agree however they were found; an offset as long as the patch or longer pairs nothing, where the torch version raised between one and two patch lengths. Deleted: web/src/finetune/targets.ts, its parity test and fixture, and the targets section of make_fixtures.py. A node-Pyodide test checks the browser's call answers; pytest checks the maths and the call's marshalling. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
cellmap_flow/portable/label_it.py holds what Seed from Prediction makes of a model's output: the request's options, the channels a seed reads (an affinity model's offsets by its channel names), which served voxel lies under each annotation voxel, the probability, and the components method (threshold, connected components, smallest size, per slice). view_labels.py calls it around the model server's read; post/segment.py and serving/probe.py import connected_components and the activation classes from it. The browser calls it through label_it_calls.py in the engine's Pyodide worker, and web/src/label-it keeps only the region read and the mutex watershed (mwatershed has no Pyodide wheel), whose input the Python answers. The ranking's placement and probability come from the same Python. scipy is imported inside the functions, so it loads on the first seed, not when a dataset opens; the node Pyodide harness loads it as the worker does. The sigmoid is numpy's, within an ulp of scipy's expit. The browser follows the server's rules now: affinity channels counted, not read by suffix; a channel range past the model's is cut, not thrown; connectivity null is refused; with no dataset a seed is refused first. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
segment.ts is the Label-it branch's (the targets branch only re-pointed an import the new file no longer has); both changelog bullets kept. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
…ython, for both modes cellmap_flow/portable/volume.py plans a volume on numbers alone (the raw level the model reads, the model's voxel sizes and blocks) and holds the labels' dtype rule; portable/regions.py holds the good regions' rules (the region a mark makes and its size, which stored entries are regions, the block a region anchors). The server's plan_volume, annotation_dtype_for, good-region routes and rehearsal pool call them in-process; the browser calls them through portable/volume_calls.py in its Pyodide worker, and its TypeScript copies of the rules are gone with their hand-computed tests. The volume lies on the grid its model's prediction is served on in both modes: the level the model reads times the model's ratio, from that level's corner, with the far partial output voxel rounded up. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
…h modes The targets branch kept the browser's suffix rule (_aff_z/_y/_x) for training while the Label-it branch gave its seed the server's count rule (portable/label_it.affinity_offsets), so the browser had two. Now the server's rule is the one: targets_calls.targets takes the manifest's channel names and decides with it, target-call.ts sends the names instead of offsets, and autodetect_output_type on the server calls the same function instead of its inline copy. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
Both sides' changelog bullets and docstring sentences kept; the volume branch's narrower Inference types and its run-test fake, which reaches the real Python for both the targets and the anchors. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
…thon cellmap_flow/portable/ai_annotate.py holds what both modes decide about an AI annotation: the plane planned around the point (plan_plane, raw_box, the screen layout), the image sent (laid out, resampled, scaled to 8 bits), the mask in the model's reply (resized back to the size sent, the padding cut off, the colour thresholded), its objects on the write box, the review's overlay and the labels Accept writes. portable/fill.py holds the box decisions under it and under the Label it tools: box_voxels, fresh ids, seed_labels, relabel_objects, and which voxels a write changes and an undo puts back. The server's ai_annotate modules, the Vertex backend, the accept route and finetune/session/fill.py call them around their zarr reads, MinIO writes and PNGs, under the same names; the browser calls them in chunkmirage's Pyodide worker through portable/ai_annotate_calls.py. The resampling is Pillow's filters (BOX, BILINEAR in float and in 8-bit fixed point, NEAREST) followed in numpy, equal to Pillow's to the bit in the tests, and the server uses it too: the pinned chunkmirage loads only scipy and scikit-image on first import, so Pillow could load only when a dataset opens. scipy, which the mask's objects are labelled with, loads while the model works on the first run. The TypeScript twins go: geometry.ts keeps the viewer's layout names and formatG, plane.ts the raw read, mask.ts the calls (and percentile, which the ranking uses); the canvas resize, make_ai_fixtures.py, ai_annotate.json and the parity and unit tests of the twins are deleted, and one node-Pyodide test checks the call paths. In the browser a reply at another size is now resized by Pillow's bilinear filter before the threshold (the server's mask edges), the overlay rounds halves to even, and Accept's instance ids count up by the server's rule on any volume of more than 8-bit ids. labels.ts and fill.ts keep the Label it routes' maths (view-labels.ts calls them without the engine's Python) and their label-it parity cases. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
Both sides' changelog bullets and docstring paragraphs kept, and the Label-it branch's lazy loader in the node Pyodide harness (it purges a half-imported package before the retry). The label-it fixture generator gets back the `import zarr` the AI branch's copied helpers need, and a comment in ai-annotate-job.ts no longer names the deleted annotationDtypeFor. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
Seed from Prediction's write, All Background, Split Objects and Undo in the browser (web/src/backend/routes/view-labels.ts) now call cellmap_flow/portable/fill.py, the rules the server's routes run, through the new portable/fill_calls.py (box, seed, background, split, undo) in the engine's Pyodide worker: the box the view covers, the ids a seed or a split gives, which voxels each write changes and what an undo puts back. The route reads the box from the paint store, hands it over, writes back the box answered and pushes the undo entry, as the AI annotation's accept does; the answers, defaults and statuses are the server's as before. With no dataset open the tools are refused (409), as a seed already was, since the Python runs in that dataset's engine; Undo keeps its entry then. web/src/ai-annotate/labels.ts goes, and fill.ts keeps only the paint store's box reads and writes and the undo stack. The parity fixture loses its relabel_objects and rewrite_foreground sections and label-it.test.ts its fill cases; the mutex watershed's stay. tests/portable/test_fill.py gains the per-slice split, which only the parity fixture held. fill.py counts a box's (object, id) pairs by one int64 key per pair in place of np.unique over the pairs as rows: the same pairs in the same order (the old fixture's cases and random boxes give identical labels and counts), in about 1 ms instead of 75 for a 64^3 box natively, so a Split in the browser takes as long as the TypeScript took (about 30 ms) instead of 210 ms. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
…illow's bits CI installs Pillow 13.0, which rounds its 8-bit bilinear one level differently from 12.3 at a non-integer ratio (half the pixels of the 70×33 → 64×32 case) and its box filter by a float32 ulp. The copy follows 12.3 to the bit; against the installed Pillow the test now asks for a float32 ulp on the plane, one level on the model's reply and the same pixel for the objects, which is all the colour threshold downstream can tell. The changelog says which Pillow the copy follows. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
davidackerman
marked this pull request as ready for review
October 22, 2026 03:19
davidackerman
changed the title
#150 made the cellmap model type read the metadata's inference shapes, which fixed jrc_mus-livers_16nm_to_8nm_mito (traced at its inference 288³, it fails at its training 216³) and broke mito_aff_unet_setup_16 (traced at its training 178³ → 56³, it answers 56³ to the inference 378³, so every chunk failed shape validation, after a 378³ forward that on a CPU outlived the local launcher's startup timeout). cellmap_models traces at whichever shape the exporter passed and records both without saying which, so the type now probes: a forward at the training shape that answers the training output means the training shapes, anything else the inference shapes. The probe is the cheap one (the smaller input; a mismatch fails fast): 3.7 s for the liver model and 9.1 s for setup_16 on a CPU, model load included. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
…first import chunkmirage's PR #50 (a7eb44e6) has the browser engine's Pyodide workers load any package Pyodide ships the first time a page's Python imports it, found by its module in Pyodide's lockfile (PIL is Pillow's), where only scipy and scikit-image loaded that way. So portable/ai_annotate.py calls Pillow for the plane's resize (box when shrinking, bilinear otherwise, an "F" image), the model's reply (RGB, bilinear) and the objects' nearest, importing it inside those functions, and its numpy copy of Pillow's Resample.c and Geometry.c goes, with the test that compared the two. resize_rgb, which only wrapped the copy, is folded into model_mask; the server is back on its installed Pillow, and the browser uses Pyodide 0.28.3's Pillow 11.3.0 (a 1 MB wheel, the same pixels as 12.3 on the deleted test's cases). The server takes chunkmirage>=0.1.0a2,<0.2 from PyPI in place of a git commit, as PyPI refuses an upload whose dependencies name a URL; 0.1.0a2 was built from 584b343, an ancestor of a7eb44e6, whose src/ is the release's. web/chunkmirage.lock pins a7eb44e6, and test_pins.py now holds pyproject.toml to a version and the lock to one full commit. pixi.lock changes only chunkmirage's entries (the PyPI wheel 0.1.0a2). The node harness (web/test/unit/pyodide.ts) finds a package by the lockfile as the worker does, in place of its fixed table. Claude-Session: https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW
This was referenced Oct 22, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four pieces of logic that the browser app had as hand-written TypeScript ports of the server's Python now exist once, in
cellmap_flow/portable/, and both modes run that: the server directly, the browser in chunkmirage's Pyodide workers (the way #150 already runs the rank score and the paint carry). The ports, their fixture generators and their parity tests are deleted. Where the port and the Python had drifted apart, the server's rule is now the rule in both modes.Against main: 90 files, +4,928 −10,232 (
web/src−1,170,web/test−6,110,cellmap_flow+1,550 of Python with docstrings,tests+410).What moved
portable/targets.pyfinetune/targets.ts, its fixture and parity testportable/label_it.pylabel-it/segment.tsandprediction.ts, their fixture sectionsportable/volume.py,portable/regions.pyfinetune/volume.ts,good-regions.ts,infer/geometry.tsportable/ai_annotate.pyai-annotate/geometry.ts,plane.ts,mask.ts,make_ai_fixtures.pyand its JSONportable/fill.pyai-annotate/labels.ts, the maths offill.ts, the last fill fixturesThe server's classes and routes keep their names and JSON; they call the portable functions.
finetune/target_transforms.pywraps them in torch;view_labels.py,session/volume.py,good_regions.py,ai_annotate/*andsession/fill.pykeep the I/O.What a user sees
relabel_objectsis faster (77 ms → 0.8 ms for a 258-object box) with identical output; the sigmoid is numpy's instead of scipy'sexpit(one float32 ulp on some values).Rule changes to review
The server's rule won in every case but one; each is in the changelog.
_affchannels counted, as z, y, x). The browser used name suffixes (_aff_z…). Same answer for every model on Hugging Face today.connectivity: nullis refused (browser read 1);min_sizeandselect_channelareint()-coerced.size_nmof 0 or null means one output block in both modes (browser refused 0); strings and NaN are refused in both (server accepted"40").counts_up(instance_target, dtype).Still TypeScript, on purpose
infer/affinity.ts): mwatershed is Rust with no Pyodide wheel. The Python answers the affinities, the TS joins them, and its parity cases stay.volumeCornerNm(one small copy used byrank.ts) and request parsing.chunkmirage: 0.1.0a2 from PyPI for the server,
a7eb44e6for the browserThe AI-annotation resampling first shipped here as a numpy copy of Pillow's filters, because chunkmirage's workers loaded only scipy and scikit-image lazily; CI showed the copy bit-equal to Pillow 12.3 but not 13.0. chunkmirage #50 (merged as
a7eb44e6) makes the workers load any Pyodide package on first import from Pyodide's lockfile, so both modes now call Pillow itself (from PIL import Imageinside the function; Pyodide 0.28.3 ships Pillow 11.3, 1 MB loaded once in 0.15 s) and the copy is gone: a 1024² reply resizes in 8 ms where the copy took 70–80. The browser pin moves toa7eb44e6; the server's dependency becomeschunkmirage>=0.1.0a2,<0.2from PyPI (0.1.0a2 is built from584b343, whose Python is identical toa7eb44e6) instead of a git URL, which PyPI refuses in a release's metadata — so cellmap-flow can be published again. The server's ops needed no change for the new chunkmirage;pixi.lockchanged only in chunkmirage's entries.Also merged here, and as its own PR #153 to main: the
cellmapmodel type asks a TorchScript export which of its two recorded shapes it runs at (#150's "inference shapes always" brokemito_aff_unet_setup_16, traced at its training shape).Checks
Python suite: 1,198 passed, the one known import-hygiene failure. vitest: 58 files, 519 tests (762 on main; the deleted ports took their tests), run with Pyodide in node. Typecheck,
gen:check,export_data --checkand the fixture generators'--checkare current. Not run in a real browser; the timings above are node's Pyodide.Background: stage 2 of the architecture review, https://claude.ai/artifact/72paR8LRuo5BC1ahxmMYaM (section 11). Five branches merged with merge commits:
portable-targets,portable-label-it,portable-volume,portable-ai,portable-fill.https://claude.ai/code/session_01EzWMS4pG3tm1ZtvKULfSfW