HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space

HyperVLP is a hierarchical surgical video-language pretraining framework. It maps video and text representations into a shared Lorentz hyperbolic manifold, uses geometry-aware hyperbolic contrastive learning to reduce structural false negatives, and enforces parent-child consistency with hyperbolic entailment.

Installation

Use Python 3.10 and install a CUDA-compatible PyTorch build for your machine.

git clone https://github.com/huyjj/HyperVLP.git
cd HyperVLP

python -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt
pip install -e .

Data

Set HYPERVLP_DATA_ROOT to the dataset root directory. Individual paths can also be overridden in configs/paths.py.

export HYPERVLP_DATA_ROOT=/path/to/HyperVLP-data

Expected default layout:

SVL/frames
SVL/lmdb
SVL/svl_hierarchy.json
Cholec80/videos
AutoLaparo/T1-PhaseRecognition/videos
MultiBypass140/datasets/BernBypass70/frames
MultiBypass140/datasets/StrasBypass70/frames

SVL is used for hierarchical video-text pretraining. Cholec80, AutoLaparo, and MultiBypass140 (Stras70/Bern70) are used for downstream phase-recognition evaluation.

Training

Stage I trains the full visual encoder, text encoder, and hyperbolic projection layers with geometry-aware hyperbolic contrastive learning:

torchrun --nproc_per_node=8 scripts/train_torchrun.py \
  --config configs/train_hypervlp.py \
  --output-dir outputs/hypervlp \
  --per-gpu-batch-size 32

Stage II freezes the visual/textual encoders and optimizes the hyperbolic mapping layers with GeoHCL plus inter-modal and intra-modal entailment:

torchrun --nproc_per_node=1 scripts/train_torchrun.py \
  --config configs/train_hypervlp_stage2.py \
  --output-dir outputs/hypervlp_stage2 \
  --pretrain \
  --resume-checkpoint outputs/hypervlp/checkpoint_final.pth \
  --per-gpu-batch-size 128

Evaluation

Zero-shot phase recognition:

CUDA_VISIBLE_DEVICES=0 python scripts/evaluate.py \
  --config configs/eval_zero_shot_classification.py \
  --train-config outputs/hypervlp_stage2/config.yaml \
  --checkpoint-path outputs/hypervlp_stage2/checkpoint_final.pth

Few-shot linear-probe phase recognition:

CUDA_VISIBLE_DEVICES=0 python scripts/evaluate.py \
  --config configs/eval_linprobe_classification.py \
  --train-config outputs/hypervlp_stage2/config.yaml \
  --checkpoint-path outputs/hypervlp_stage2/checkpoint_final.pth

Citation

@inproceedings{hu2026hypervlp,
  title={HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space},
  author={Hu, Yaojun and Yuan, Kun and Navab, Nassir and Ying, Haochao and Wu, Jian and Padoy, Nicolas},
  booktitle={International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)},
  year={2026}
}