Skip to content

aicell-lab/bioengine-worker

Repository files navigation

BioEngine Logo

BioEngine

Execution and adaptation layer for bioimage AI — run, screen, fine-tune, and deploy BioImage Model Zoo models through AI agents

GitHub Docker Image License Python


What is BioEngine?

Foundation models and curated repositories have transformed bioimage AI, yet most researchers cannot readily run, adapt, or extend them on available hardware. BioEngine fills this gap as the execution and adaptation layer between curated AI and scalable compute, deployable on a laptop, workstation, or cluster.

BioEngine exposes its capabilities through a SKILL.md contract — a plain-text file designed for general-purpose AI agents to acquire domain knowledge and invoke GPU services directly. A scientist describes their imaging goal in plain language to any AI agent. The agent parses the contract, selects the appropriate service, and dispatches the GPU workflow. Results return as segmented images, ranked comparison tables, or a live web application, with no command-line access, software installation, or IT ticket required.

AI agent? Load the BioEngine skill: https://bioimage.io/skills/bioengine/SKILL.md

Capabilities

Capability Description
Model screening Agent queries BioImage Model Zoo, filters by compatibility, runs inference, and ranks by mAP — 58 candidates screened to 4 ranked in a single session
Real-time inference Sub-second latency for live microscopy feedback loops; per-frame statistics (cell count, masks, morphology) returned to the controlling agent
Collaborative fine-tuning Browser-based annotation against Cellpose-SAM pre-segmentations; fine-tuning triggered with one click; F1 rose from 0.36 → 0.71 across 1,600 training epochs on PlantSeg data
Agent-built applications Agent generates deployment manifest, GPU workflow, and web UI from a single plain-language prompt; mean F1 = 0.920 ± 0.037 on Lucchi++ FIB-SEM benchmark

Quick Start

Public BioEngine (no setup)

Test AI models instantly via the community instance:

  1. Visit BioImage.IO Model Zoo
  2. Select any model and click "TEST RUN MODEL"
  3. Execution runs on the public BioEngine worker (bioimage-io/bioengine-worker)

CLI

pip install "bioengine[cli] @ git+https://github.com/aicell-lab/bioengine-worker.git"

# Call any service method
bioengine call bioimage-io/bioengine-worker get_status

# List running applications
bioengine apps list --worker bioimage-io/bioengine-worker

# Run a model
bioengine call bioimage-io/my-app predict --arg input=image.tif

Python SDK

from hypha_rpc import connect_to_server

server = await connect_to_server({"server_url": "https://hypha.aicell.io", "token": token})
worker = await server.get_service("bioimage-io/bioengine-worker")

# Screen models
status = await worker.get_status()

# Deploy an application
app_id = await worker.deploy_app(
    artifact_id="bioimage-io/cellpose-finetuning",
    application_id="cellpose-finetuning",
)

# Get application status
app_status = await worker.get_app_status(application_ids=[app_id])

Deploy Your Own Worker

# Docker (single machine)
git clone https://github.com/aicell-lab/bioengine-worker.git
cd bioengine-worker
mkdir -p .bioengine data
UID=$(id -u) GID=$(id -g) docker compose up

See Deployment Guide for SLURM/HPC and Kubernetes modes.

Architecture

┌────────────────────────────────────────┐
│            Hypha Server                │
│   (RPC, service discovery, artifacts)  │
└────────────┬───────────────────────────┘
             │ WebSocket / RPC
┌────────────▼───────────────────────────┐
│         BioEngineWorker                │
│                                        │
│  ┌─────────────────────────────────┐   │
│  │  SKILL.md contract              │   │
│  │  (agent-readable interface)     │   │
│  └─────────────────────────────────┘   │
│  ┌─────────────────────────────────┐   │
│  │  Ray Cluster                    │   │
│  │  (SLURM / single / Kubernetes)  │   │
│  └─────────────────────────────────┘   │
│  ┌─────────────────────────────────┐   │
│  │  Applications Manager           │   │
│  │  (Ray Serve lifecycle +         │   │
│  │   artifact management)          │   │
│  └─────────────────────────────────┘   │
│  ┌─────────────────────────────────┐   │
│  │  Datasets Manager               │   │
│  │  (Zarr HTTP streaming)          │   │
│  └─────────────────────────────────┘   │
└────────────────────────────────────────┘

Stack: Ray + Ray Serve for distributed GPU inference, Hypha for RPC service discovery and artifact management.

Deployment Modes

Mode Use Case Guide
single-machine Workstation, development, small-scale
external-cluster Kubernetes, pre-configured Ray clusters
slurm HPC clusters with SLURM scheduler

Worker Service API

The worker registers as a Hypha service. Key methods:

Method Description
get_status() Worker and cluster status
deploy_app(artifact_id, ...) Deploy an application
stop_app(application_id) Stop a running application
get_app_status(application_ids) Status of specific applications
list_apps() All deployed applications
upload_app(...) Create/update application artifact
run_code(code, ...) Run Python in a Ray task
list_datasets() Available datasets

Applications

BioEngine applications are self-contained deployable units: a manifest.yaml + Python deployment code + optional web frontend. They can compose multiple AI models, wrap models with custom pre/post-processing, and expose arbitrary web UIs.

An AI agent given a SKILL.md contract can generate and deploy a new application from a plain-language prompt — generating the manifest, GPU workflow, and web interface with no manual programming.

Reference apps:

See Applications Guide for full documentation.

Documentation

Development Setup

conda activate bioengine-worker
pip install -e ".[worker,cli,dev]"
source .env   # loads HYPHA_TOKEN

# Run locally
python -m bioengine.worker \
    --mode single-machine \
    --head-num-gpus 1 \
    --workspace-dir ~/.bioengine \
    --debug

# Run tests
pytest tests/end_to_end/ -v

Citation

BioEngine is described in the following preprint (bioRxiv, submitted):

Mechtel N, Dettner Källander H, Cheng S, Zhang H, AI4Life Consortium, Ouyang W. BioEngine: scalable execution and adaptation of bioimage AI through agent-readable interfaces. bioRxiv (2025).

License

MIT — see LICENSE.

Acknowledgements

BioEngine is built on Ray, Hypha, and Zarr. Supported by the SciLifeLab & Wallenberg Data Driven Life Science Program, AI4Life (EU Horizon Europe grant 101057970), and the Berzelius GPU resource.

Releases

No releases published

Packages

 
 
 

Contributors