Perception-inspired self-supervision enables cell segmentation for large-scale calcium imaging

PROFILE is a self-supervised cell segmentation method that learns morphological features from active neurons and uses them to segment cells with weak or undetectable activity, reflecting how human annotators identify cells from both temporal activity and cellular morphology.

This repository provides the inference code and scripts for selected PROFILE examples. Pretrained checkpoints and most example data are distributed separately in the PROFILE demo data package. A small mouse 2D mesoscope example is included for a quick test.

You can download the accompanying data package in this Google Drive link.

Included content

profile_inference/
  PROFILE model definitions, checkpoint loading, vector-field inference, postprocessing, and IoU evaluation.

scripts/
  Command-line reproduction scripts for synthetic, mouse 2D, mouse 2D mesoscopic, and LBM examples.

notebooks/
  Notebook wrappers around the command-line scripts.

configs/
  JSON inference configs for each released checkpoint.

examples/
  Small demo datasets after copying the release data package into the repository root.

models/
  Pretrained PROFILE checkpoints after copying the release data package into the repository root.

outputs/
  Output masks and post-processing images generated by the demo scripts.

System requirements

Tested operating system:

Ubuntu 24.04.3 LTS

Tested software packages:

Python 3.9
PyTorch 2.6.0+cu124
NumPy 2.0.2
SciPy 1.13.1
tifffile 2024.8.30
Matplotlib 3.9.4
Pandas 2.3.3
scikit-image 0.24.0
ipykernel 6.30.1, optional for notebooks

Hardware:

CPU inference is supported for the small 2D demos.
An NVIDIA CUDA GPU is optional for faster inference and recommended for 3D/LBM examples.
For the released LBM 3D examples, GPU and RAM (>32GB) are recommended.

Installation guide

Create and activate a conda environment:

conda create -n profile-demo python=3.9 -y
conda activate profile-demo

Install PyTorch following the command appropriate for your machine from the PyTorch installation page. We tested with PyTorch 2.6.0+cu124.

Install the remaining Python dependencies:

pip install -r requirements.txt

Optional, only if you want to run notebooks interactively:

pip install notebook ipykernel

Typical installation time is approximately 5-15 minutes, dominated by downloading PyTorch. Installation can be faster if PyTorch is already present.

Add example data and checkpoints

Use the accompanying data package. After downloading, copy the data package into the repository root:

cp -a /path/to/PROFILE_demo_data/. /path/to/PROFILE/

After copying, the repository should contain the following example data and checkpoints.

examples/
  synthetic_2d/mesoscopic/input_stack.tif
  synthetic_2d/mesoscopic/gt_mask.tif
  synthetic_2d/microscopic/input_stack.tif
  synthetic_2d/microscopic/gt_mask.tif
  synthetic_3d/mesoscopic/input_volume.tif
  synthetic_3d/mesoscopic/gt_volume.tif
  synthetic_3d/microscopic/input_volume.tif
  synthetic_3d/microscopic/gt_volume.tif
  mouse_target_scale/test_fovs/test_fov_01_input.tif
  mouse_target_scale/test_fovs/test_fov_01_gt.tif
  mouse_2d_mesoscope/full_fov_mean_projection.tif
  lbm_0p6/input_volume.tif
  lbm_0p6/gt_mask.tif
  lbm_2mm/input_volume.tif
  lbm_2mm/profile_mask.tif

models/
  profile_2d_synthetic_mesoscopic.pt
  profile_2d_synthetic_microscopic.pt
  profile_3d_synthetic_mesoscopic.pt
  profile_3d_synthetic_microscopic.pt
  profile_mouse_target_scale_n10.pt
  profile_mouse_target_scale_n20.pt
  profile_mouse_target_scale_n40.pt
  profile_mouse_target_scale_n60.pt
  profile_mouse_target_scale_n80.pt
  profile_mouse_target_scale_n100.pt
  profile_mouse_target_scale_n115.pt
  profile_mouse_2d_mesoscope.pt
  profile_lbm_0p6.pt
  profile_lbm_2mm.pt

Demo

Minimal 2D synthetic demo

Run PROFILE on the released mesoscopic 2D synthetic stack:

python scripts/inference/run_profile_inference.py \
  --config configs/profile_2d_synthetic_mesoscopic.json \
  --input examples/synthetic_2d/mesoscopic/input_stack.tif \
  --gt examples/synthetic_2d/mesoscopic/gt_mask.tif \
  --output-dir outputs/demo_synthetic_2d \
  --device cpu

Expected output:

outputs/demo_synthetic_2d/input_stack_profile_mask.tif
outputs/demo_synthetic_2d/input_stack_profile_board.tif

Expected runtime is approximately 1-3 minutes on CPU, or less than 1 minute on a CUDA GPU.

Mouse 2D mesoscope demo

Run the mouse 2D mesoscope full-FOV example:

python scripts/mouse/reproduce_mouse_2d_mesoscope.py --device cpu

Expected output:

outputs/mouse_2d_mesoscope/mouse_2d_mesoscope_profile_mask.tif
outputs/mouse_2d_mesoscope/mouse_2d_mesoscope_profile_board.tif
outputs/mouse_2d_mesoscope/mouse_2d_mesoscope_paper_panel.svg
outputs/mouse_2d_mesoscope/mouse_2d_mesoscope_summary.json

Expected runtime is approximately 1-5 minutes on CPU, or less than 1 minute on a CUDA GPU.

Mouse 2D training-set size demo

Run the mouse checkpoints with different training-set sizes on five fixed held-out test FOVs:

python scripts/mouse/reproduce_mouse_target_scale.py --device auto

Expected output:

outputs/mouse_target_scale/profile_mouse_target_scale_n10/test_fov_01_mask.tif
outputs/mouse_target_scale/profile_mouse_target_scale_n10/test_fov_01_board.tif
...
outputs/mouse_target_scale/profile_mouse_target_scale_n115/test_fov_05_mask.tif
outputs/mouse_target_scale/profile_mouse_target_scale_n115/test_fov_05_board.tif

Expected runtime is approximately 5-15 minutes on CPU for all released mouse target-scale checkpoints, or 1-3 minutes on a CUDA GPU.

LBM inference demo

Run PROFILE on the released LBM example volumes.

python scripts/lbm/run_lbm_inference.py --root . --output-dir outputs/lbm_inference --device auto

Expected output:

outputs/lbm_inference/lbm_0p6/profile_mask.tif
outputs/lbm_inference/lbm_0p6/profile_board.tif
outputs/lbm_inference/lbm_2mm/profile_mask.tif
outputs/lbm_inference/lbm_2mm/profile_board.tif
outputs/lbm_inference/lbm_inference_summary.csv

Using PROFILE with new data

Inference requires an input TIFF, a trained checkpoint, and its matching JSON config. Choose a checkpoint and config that match the dimensionality, imaging scale, and preprocessing of your data. The released checkpoints were evaluated on their corresponding example domains and the performance on data from different imaging conditions has not been evaluated.

Run 2D inference on a new time-lapse TIFF stack:

python scripts/inference/run_profile_inference.py \
  --config /path/to/matching_2d_config.json \
  --input /path/to/your_movie.tif \
  --output-dir outputs/your_movie_profile \
  --device auto

Run 3D inference on a new average volume:

python scripts/inference/run_profile_inference.py \
  --config /path/to/matching_3d_config.json \
  --input /path/to/your_volume.tif \
  --output-dir outputs/your_volume_profile \
  --device cuda:0

If a ground-truth label image is available, pass it with --gt to compute F1-IoU AUC and related summary metrics:

python scripts/inference/run_profile_inference.py \
  --config configs/profile_2d_synthetic_mesoscopic.json \
  --input /path/to/your_movie.tif \
  --gt /path/to/your_gt_mask.tif \
  --output-dir outputs/your_movie_profile \
  --device auto

Use --device cpu to force CPU inference, --device cuda:0 to force a specific GPU, and --save-vector-field to also save the raw predicted vector field.

To adapt a config to a different checkpoint or postprocessing setting, copy one of the JSON files in configs/ and edit these fields:

checkpoint: path to the `.pt` model file
vf_clip: vector-field clipping value
threshold: postprocessing threshold
min_size: minimum output instance size in pixels or voxels
normalization: input normalization mode

Notebook use

Launch Jupyter from the environment where the dependencies were installed:

jupyter notebook notebooks/reproduce_synthetic_examples.ipynb
jupyter notebook notebooks/run_lbm_inference.ipynb
jupyter notebook notebooks/reproduce_mouse_target_scale.ipynb
jupyter notebook notebooks/reproduce_mouse_2d_mesoscope.ipynb

Notes

• The 2D synthetic inputs are video stacks; the inference code uses their mean projection.
• The 3D inputs are average volumes, due to the large file sizes.