Foundation Model Adaptation for Retinal Vessel Segmentation from Sparse Centerline Annotations in OCT

A Systematic Evaluation of DINOv3 and SegFormer Encoders Under Sparse Landmark Supervision

Retinal blood vessel segmentation in Optical Coherence Tomography (OCT) is essential for diagnosing vascular pathologies, yet existing approaches struggle with sparse centerline annotations that cover only a fraction of each image. We adapt pretrained foundation models (DINOv3 ViT, SegFormer MiT) using a lightweight SegDINO architecture and show that enriching the supervision signal through coarse label learning and circular blob annotations raises Dice from 0.243 (UNet++ baseline) to 0.455. A hybrid post-processing pipeline further reduces vessel fragmentation by 99.9%.

Project Details

• StudentsChristof Weickhardt, Yasha Luescher
• AdvisorProf. Dr. Susanne Suter
• Co-AdvisorSimone Bonato
• External PartnersPeter Maloca, Philippe Valmaggia (Eye Clinic, University of Basel)
• SemesterHS25 (Fall 2025), IP5
• TechnologiesPython, PyTorch, PyTorch Lightning, Hydra, W&B, SLURM

Keywords

OCT · Retinal Vessel Segmentation · Foundation Models · Vision Transformers · DINOv3 · SegFormer · SegDINO · Sparse Annotations · Instance Segmentation · LoRA · Coarse Label Learning · GradCAM · Post-Processing · Medical Image Analysis

Problem and Motivation

Optical Coherence Tomography (OCT) provides non-invasive, depth-resolved cross-sectional images of the retina. Blood vessels appear as shadow-casting structures whose pulsatile flow patterns enable artery-vein classification. Automated segmentation of these vessels supports clinical assessment of conditions such as glaucoma, diabetic retinopathy, and retinal vein occlusion.

A core challenge is the nature of available annotations: only sparse 7x7 pixel centerline patches mark vessel locations, leaving most of each image unlabelled. Unlabelled pixels are ambiguous, as they could be true background or simply unannotated vessels. This makes standard CNN-based approaches prone to fragmented, incomplete predictions. A predecessor project using nnU-Net achieved a Dice score of only 0.294 on this task.

We hypothesise that foundation models pretrained through self-supervised learning on large-scale image data encode geometric and structural priors that can compensate for the limited annotation signal. This project systematically investigates whether such models, combined with appropriate adaptation strategies, outperform conventional CNN baselines under these constrained conditions.

Approach: SegDINO Architecture

We implement a SegDINO architecture that pairs a high-capacity pretrained encoder with a lightweight MLP decoder. The design rationale is intentional asymmetry: the encoder provides rich visual representations from self-supervised pretraining, while the deliberately shallow decoder reduces overfitting under sparse supervision.

Pipeline

A single-channel OCT B-scan (384x384 pixels) passes through an input adapter that maps to 3 channels, then into the pretrained backbone. The MLP decoder extracts features from four intermediate layers, projects each to a common dimension via 1x1 convolutions, upsamples to a shared resolution, and concatenates them. Dual output heads produce a binary vessel segmentation mask and a centerline heatmap regression target.

Backbone Encoders

• DINOv3 ViT-S/16: Self-supervised Vision Transformer (22.3M parameters)
• SegFormer MiT-B0 through MiT-B5: Supervised hierarchical transformers (3.8M to 82.0M parameters)

Encoder Adaptation Strategies

• Frozen: Encoder weights fixed, only decoder trains
• LoRA: Low-rank adaptation (rank 16) on attention projections
• Trainable: Full fine-tuning of all parameters

Loss Functions

A composite loss combines focal loss (handling class imbalance), Dice loss (direct overlap optimisation), and heatmap regression loss. An overlap-component masked loss variant only penalises predicted components that overlap with ground truth annotations, preventing suppression of correct but unannotated vessel detections.

Research Questions

• F1 (Metric Adequacy): Which evaluation metrics are appropriate for sparse landmark supervision? Are standard overlap metrics (Dice, IoU) sufficient, or are instance-level and distance-based metrics necessary?
• F2 (Model Adequacy): Do foundation-model-based architectures outperform the CNN baseline (UNet++) on retinal vessel segmentation in OCT?
• F3 (Training Strategy): How do parameterisation choices (decoder design, backbone adaptation) affect performance under limited supervision?

Key Results

We conducted six experiment groups evaluating training strategies, backbone scale, decoder configuration, annotation types, and coarse label learning. The dataset comprises 113 OCT scans from 33 subjects with 222 annotated vessel centerlines (82% arteries, 18% veins).

Performance Comparison

The following table summarises the main results. Instance recall at threshold 0 (any overlap counts as detection) is the primary metric because every miss is unambiguous under sparse annotations.

Segmentation performance across configurations. Best Dice in bold.

Model	Dice	mIoU	Inst. F1 (t=0)	Inst. Recall (t=0)
UNet++ (baseline)	0.243	0.147	0.750	0.973
MiT-B0 frozen	0.139	0.084	0.696	0.718
MiT-B0 LoRA	0.234	0.142	0.769	0.915
MiT-B0 trainable	0.282	0.180	0.807	0.889
MiT-B1 + coarse labels	0.428	0.293	0.715	0.958
MiT-B3 + circular blobs	0.455	0.314	0.759	0.966

The best configurations enrich the supervision signal: MiT-B3 with circular blob annotations achieves the highest Dice (0.455), while MiT-B1 with coarse label learning reaches the highest recall (0.958). Both surpass the UNet++ baseline (Dice 0.243) by a wide margin. The UNet++ baseline achieves high instance recall (0.973) but at the cost of lower overlap, illustrating the divergence between metric families under sparse labels.

Encoder Plasticity Dominates

Frozen MiT-B0 achieves only Dice 0.139; full fine-tuning raises this to 0.282 (+0.143). This is the single largest factor affecting performance. LoRA recovers most of the recall gap (0.915 vs 0.889 trainable) at a fraction of the parameter cost, making it a practical alternative when compute is limited.

Annotation Quality Outweighs Model Size

Coarse label learning, which models annotation noise through per-pixel confusion matrices, enables a frozen MiT-B1 encoder (13.7M parameters) to reach Dice 0.428 --- surpassing even the fully trainable MiT-B0 (Dice 0.282). Circular blob annotations provide a similar boost (MiT-B3 Dice 0.278 to 0.455). Both findings confirm that under sparse supervision, how the model interprets its labels matters more than raw model capacity.

Backbone Scale: A Non-Monotonic Relationship

When frozen, the MiT encoder family shows a U-shaped performance curve: the smallest (B0) and largest (B5) variants perform best, while mid-range models (B1 through B4) collapse or diverge. With full training, this pattern disappears and all backbones converge to comparable performance (Dice 0.229--0.304), confirming that the frozen failures stem from the domain gap rather than architectural deficiency.

Post-Processing: Restoring Vessel Continuity

Raw model predictions exhibit vessel fragmentation: individual vessels appear as disconnected segments across the 512-frame OCT volume. Causes include speckle noise, patient micro-motion, and low contrast in slow-flow vessels. We developed a hybrid volumetric post-processing pipeline that combines morphological dilation with a VesselLinker trust-handover algorithm.

Conclusions

F1: Metrics Under Sparse Labels

Standard overlap metrics (Dice, mIoU) are systematically pessimistic under sparse annotations: they penalise correct predictions that fall outside the small labelled patches. Instance-level recall and centroid distance provide unambiguous evaluation and should accompany overlap metrics in all sparse-label settings.

F2: Foundation Models for OCT

SegDINO with augmented supervision surpasses the UNet++ baseline in overlap metrics: MiT-B3 with blob annotations reaches Dice 0.455 (vs 0.243 for UNet++) and MiT-B1 with coarse label learning reaches Dice 0.428 and recall 0.958. The UNet++ baseline achieves the highest instance recall (0.973) despite lower Dice, illustrating the divergence between overlap and instance metrics. Frozen encoders fall below the baseline (MiT-B0 frozen: Dice 0.139), confirming that pretrained features require domain adaptation.

F3: What Matters Most

Encoder adaptation strategy is the dominant performance factor. Supervision quality ranks second: coarse label learning and blob annotations both yield larger gains than scaling backbone capacity. Backbone scale and decoder design have secondary effects. The overarching conclusion is that annotation quality, not model size, is the binding constraint.

Data and Infrastructure

• Dataset113 OCT scans, 512 frames each, 496 x 1024 pixels, from 33 subjects (University of Basel Eye Clinic)
• Annotations222 vessel centerlines (183 arteries, 39 veins) as sparse 7x7 patches
• Training200 epochs (SegDINO), 100 epochs (UNet++), batch size 12, Adam optimiser
• InfrastructureFHNW SLURM GPU cluster with Singularity containers
• Experiment TrackingWeights & Biases (W&B)
• CodePython, PyTorch Lightning, Hydra configuration

Team

• Christof Weickhardt
• Yasha Luescher

Advisor Prof. Dr. Susanne Suter

Co-Advisor Simone Bonato