ExVEM

Expansion Microscopy → Virtual Electron Microscopy

Self-supervised 3-D image translation for EM-style visualization of expansion microscopy volumes.

Quick start · Inference guide · Training guide · Model & demo data · Full dataset

---

ExVEM converts expansion microscopy (ExM) volumes into virtual electron microscopy (EM) volumes. It learns from unpaired ExM and EM data using an anisotropic 3-D U-Net, multi-scale discriminators, cycle consistency, and intensity-group consistency.

Native 3-D translation	Unpaired learning	Seamless large-volume inference
Preserves context across adjacent z-slices	Does not require matched ExM–EM fields of view	Blends overlapping patches to suppress tile boundaries

This repository focuses on ExM-to-virtual-EM training and inference. EM segmentation and post-processing packages are not required.

ExM → virtual EM

ExM input	ExVEM output

Downloads

Resource	Contents	Access
Pretrained generator	G_AB_Mouse_pretrained.pth · ExM → virtual EM	Download ↗
Demo volume	ExM_input.tif · uint8 · [64,1024,1024]	Download ↗
Full ExM/ExVEM dataset	Large-scale source and translated volumes	BioStudies dataset ↗

Note

The released G_AB checkpoint is all that is required for inference. Resuming adversarial training additionally requires G_BA, discriminators, optimizers, and scheduler states.

Quick start

System requirements

Component	Tested configuration	Notes
Operating system	Ubuntu 22.04 LTS	Linux is the tested platform
Python	3.11.15	Python 3.11 recommended
PyTorch	2.6.0	Installed from requirements.txt
CUDA / driver	CUDA 12.4 / NVIDIA 550.144.03	Required for training
GPU	NVIDIA RTX 3090, 24 GB	≥12 GB VRAM recommended; 24 GB preferred

Inference supports CPU execution. Training currently requires an NVIDIA CUDA GPU. The default patch size is 16 × 256 × 256 in Z × Y × X order.

Installation

git clone git@github.com:NICALab/ExVEM.git
cd ExVEM

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Typical installation time is 10–20 minutes on a desktop computer with a broadband connection. No custom CUDA extensions need to be compiled.

Verify the installation:

python -m src.inference --help
python -m src.train --help

Run the demo

1. Download ExM_input.tif and G_AB_Mouse_pretrained.pth from the ExVEM Google Drive folder.
2. Place them under demo_data/ and checkpoints/ as shown in the demo layout.
3. Run:

python -m src.inference \
  --input demo_data/ExM_input.tif \
  --checkpoint checkpoints/G_AB_Mouse_pretrained.pth \
  --output outputs/demo_virtual_em_uint8.tif \
  --roi 0 16 0 256 0 256 \
  --patch-size 16 256 256 \
  --overlap 8 128 128 \
  --batch-size 1 \
  --device auto

Expected output and runtime

The demo writes a uint8 TIFF with shape [16,256,256] and prints the output shape, selected device, and patch count. The one-patch ROI takes a few seconds on an RTX 3090 and approximately 1–5 minutes on a typical CPU-only desktop.

Remove the --roi option to translate the complete [64,1024,1024] demo volume. Runtime depends on hardware and storage throughput.

For all options, see the complete inference tutorial.

Use ExVEM on your data

Inference

ExVEM accepts 3-D TIFF stacks and Zarr arrays in [Z,Y,X] order. The released model expects uint8 ExM intensities by default.

python -m src.inference \
  --input /path/to/exm_volume.tif \
  --checkpoint /path/to/G_AB_Mouse_pretrained.pth \
  --output /path/to/virtual_em.tif \
  --patch-size 16 256 256 \
  --overlap 8 128 128 \
  --batch-size 1 \
  --device cuda:0

Use --output-dtype float32 to preserve raw generator values. The default uint8 writer applies round(clip(output, 0, 1) × 255).

Training

Training uses unpaired ExM and EM Zarr volumes:

python -m src.train \
  --ExM_data_path /path/to/exm.zarr \
  --EM_data_path /path/to/em.zarr \
  --exp_name ExVEM_training \
  --patch_size 16 256 256 \
  --patch_overlap 8 128 128 \
  --batch_size 1 \
  --n_epochs 500

See the complete training tutorial. The Google Drive TIFF is an inference demo. The large-scale ExM inputs and translated ExVEM outputs are available from the BioStudies dataset.

Data and model conventions

Item	Convention
Volume axes	[Z,Y,X]
Default input	uint8, scaled internally by 1/255
Unit-range input	Use --input-scale unit for floating point data in [0,1]
Output formats	TIFF or Zarr
Public checkpoint	G_AB.state_dict(); optional distributed module. prefix is accepted
Default patch / overlap	[16,256,256] / [8,128,128]

Architecture

ExM volume → overlapping 3-D patches → anisotropic 3-D U-Net → blended virtual EM volume

The generator begins with a 1 × 5 × 5 convolution, uses four in-plane downsampling stages and 3-D residual convolution blocks, and reconstructs the output through skip connections. Training adds a reverse generator and multi-scale discriminators; inference requires only G_AB.

Project structure

ExVEM/
├── assets/                  README visuals
├── checkpoints/             pretrained-weight instructions
├── demo_data/               demo-data instructions
├── docs/                    inference and training tutorials
├── model/                   generators and discriminators
├── src/                     training and inference code
├── tests/                   inference smoke test
├── requirements.txt         pinned pip dependencies
└── LICENSE                  MIT License

Citation

Publication and citation details will be added here when available.

License

ExVEM is available under the MIT License.