fidoptMSLesSeg
Raw FLAIR MR of the brain of one patient with multiple sclerosis at the first two visits of the MSLesSeg training set, 0.1 to 3.2 years apart, each on its own grid as acquired.
Read before comparing (10)
- Truth. SynthSeg makes the labels from each FLAIR at 1 mm, brought back by nearest neighbour onto the image's grid, whose slices are 1.7 to 6 mm thick: the dataset is ranked apart with the other network-made truths. SynthMorph was trained on SynthSeg label maps to overlap 21 of these structures, so it may be favoured.
- Lesions. The dataset's lesion masks, drawn by one junior rater and validated by a neuroradiologist and a neurologist, are shown (dice_lesion), never ranked. Lesions appear, grow and shrink between visits, the second holding 0.83 to 7.3 times the first's volume on the test pairs, so their overlap rewards a field that resizes a lesion. Labels are 0 inside each visit's lesion mask.
- Masks. prepare.py brings each lesion mask from MNI152 onto the raw FLAIR's grid through the provider's FLIRT matrix, by nearest neighbour; on thick slices a mask is its 1 mm drawing sampled mid-slice. On the first 26 visits built the lesions sit on hyperintensities of the raw FLAIR.
- Images. The raw FLAIR of each visit, not the dataset's MNI152 images, which arrive aligned by two affine registrations to a template. They come skull-stripped, zero outside the brain and a margin; no mask is given. The scans come from many clinics at 1.5 T and 3 T with no single protocol; on test pairs P28 and P51 the two visits differ in resolution.
- Alignment. As delivered the labels overlap with a Dice of 0.00 to 0.52. The moving FLAIR's, labels' and lesions' headers carry one rigid start from elastix mutual information inside each head, no label used. Rigid, since a translation left P19 visibly tilted: the starts turn the moving FLAIR by 1.5 to 10.5 degrees.
- Baseline. With no registration after that start the 10 test pairs score a label Dice of 0.80 (0.73 to 0.83) and an HD95 of 1.9 mm (1.3 to 4.4), and a lesion Dice of 0.52 (0.16 to 0.77) with an HD95 of 19 mm.
- Cut fields. The FLAIR does not always hold the whole brain: on the test pairs P20's first visit and P28's second stop in the cerebellum, and P12's and P19's first visits below the top of the brain. A structure one image cuts cannot overlap whole, whatever the field.
- Visits. The pairs lie a median of 1.0 year apart, the test pairs 0.5 to 3.2. The raw archive holds three visits without masks; prepare.py takes the visits the dataset annotates, for P7 the raw T2 and T3.
- Outside numbers. None: the challenge, MSLesSeg at ICPR 2024, ranks segmentation, not registration.
- No method here does better than no registration on Dice (0.800): as far as these annotations can tell, the pairs already match.
| Accuracy | Smoothness | Cost | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Change | HD95 mm | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 1 | DemonsCPUapp | 0.727 ±0.127 | −0.073 | 3.16 ±2.85 | 0.034 | 0.50 | 12 | 2.1 | ||
| No registration | 0.800 ±0.020 | 1.89 ±0.66 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on Dice, HD95 mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
1 result not ranked: an older run, pairs still missing, or a failure on every pair
| Accuracy | Smoothness | Cost | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Change | HD95 mm | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Machine | Images |
| ConvexAdam (MIND)GPUapp9/10 pairs | 0.778 ±0.049 | −0.021 | 2.60 ±1.36 | 0.026 | 0.10 | 8 | 2.3 | 1.6 | Laptop | ||
Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- 10 test pairs drawn with seed 2026 among the 15 patients outside development, and 10 development pairs. The pool is the 25 training patients with two annotated visits or more; a pair is their first two visits. The dataset's 22 test patients have one visit each.
- Truth
- contours by a program lesion by experts SynthSeg's segmentation run by us on each FLAIR (SynthSeg-robust 2.0 with the cortical parcellation, FreeSurfer 7.4.1): 30 structures (white matter, ventricles, deep grey nuclei, hippocampus, amygdala, cerebellum, brainstem, CSF) and 68 Desikan-Killiany cortical regions, outside the lesions the dataset segments at that visit
- Source
- doi.org/10.6084/m9.figshare.27919209
doi.org/10.1038/s41597-025-05250-y
github.com/alessiarondinella/MSLesSeg-2024 - Licence
- CC BY 4.0
10 pairs for tuning, never tested
P1, P2, P3, P14, P31, P22, P5, P49, P6, P11