MedSAM2 introduces a robust foundation model for promptable segmentation in 3D medical images and temporal video data, built by fine-tuning SAM2.1 on a large-scale curated medical dataset.

Key Highlights:

• 3D & Video Segmentation Foundation Model – Tailors SAM2.1-Tiny for medical domains, supporting volumetric scans (CT, MRI, PET) and sequential video modalities (ultrasound, endoscopy) with a unified architecture.
• Memory-Aware Temporal Modeling – Employs a streaming memory attention module with cross-frame conditioning to maintain context across slices or frames, enhancing anatomical continuity and temporal coherence.
• Promptable via Bounding Boxes – Uses 2D bounding box prompts on the central slice or frame, propagating masks bidirectionally for full 3D or temporal coverage with minimal supervision.
• Hierarchical Vision Transformer Backbone – Integrates Hiera for efficient multiscale feature extraction, outperforming naive ViTs on medical data in speed and accuracy.
• Full Fine-Tuning Strategy – Trains all model components end-to-end (encoder, decoder, memory, prompt encoder) to maximize medical domain adaptation.
• SOTA Performance Across Modalities – Outperforms EfficientMedSAM and SAM2.1 variants in CT/MRI/PET organs and lesion segmentation, including difficult structures like the pancreas and epicardium.
• Human-in-the-Loop Annotation Boost – Enables >85% annotation time reduction on large-scale datasets (5,000 CT lesions, 3,984 liver MRIs, 251k echo frames) via iterative refinement pipeline.
• Deployment-Ready Ecosystem – Plug-and-play support for 3D Slicer, Gradio, Google Colab, JupyterLab, and terminal CLI for both local and cloud environments.
• Open Source – Code, models, and plugins available

Resources

• Project:  https://medsam2.github.io/
• Paper: https://arxiv.org/abs/2504.03600
• Github: https://github.com/bowang-lab/MedSAM2
• Colab: https://colab.research.google.com/drive/1MKna9Sg9c78LNcrVyG58cQQmaePZq2k2?usp=sharing

Related articles from LearnOpenCV:

• SAM2 Explained: https://learnopencv.com/sam-2/
• Fine-tuning SAM2: https://learnopencv.com/finetuning-sam2/
• Medical Image Segmentation: https://learnopencv.com/medical-image-segmentation/