fidoptMSLesSeg, multiple sclerosis lesions
FLAIR MR of a multiple sclerosis patient's brain, registered to MNI152 at 1 mm and skull-stripped by the provider, every hyperintense lesion outlined by a junior rater and validated by two specialists (MSLesSeg, ICPR 2024).
10 test casesbrainimage MR
Read before comparing (9)
- Pool. 75 patients aged 18 to 59, 48 women and 27 men, scanned on 1.5 T and 3 T scanners with no single protocol, each visit with a T1, a T2 and a FLAIR. The 22 test patients are 21 relapsing-remitting and one primary progressive, P58, whom the paper's Table 1 calls secondary progressive. No other dataset of the corpus names the collection.
- Images. The provider registered each series to MNI152 with FSL's flirt, a 12-degree-of-freedom affine, and skull-stripped it with bet, experts checking each by eye: the image is zero outside the brain, and no mask is given. The values are 32-bit floats without unit, on scales that differ between scanners. The T1 and T2 are not given, so a score here comes from the FLAIR alone.
- Truth. One junior rater, trained with two senior experts, outlined in JIM 9.0 every lesion hyperintense on the FLAIR, reading the T1 and T2 where the FLAIR was ambiguous. The experts reviewed the work as it went, and a neuroradiologist and a neurologist who specialise in multiple sclerosis validated every mask. No agreement between raters is given.
- Agreement. On the 22 test patients the masks reach a Dice of 0.69 against the STAPLE consensus of the truth and the 15 teams' masks, and 0.75 with the five best teams only; the paper puts the gap on an error the networks share, outlines wider than large lesions or lesions in unusual places.
- Thick slices. The test FLAIRs were acquired in slices 2 to 5 mm thick (median 3 mm). Brought to 1 mm, the thick ones stay blurred head to foot (P56, P60 and P68), and their masks follow that staircase.
- Cut brains. P60's FLAIR misses the top and the bottom of the brain, and the affine has most likely stretched what it holds. P58's is cut flat 77 mm above the origin, about 4 mm under the top of the other test brains. Nothing is drawn where the image holds nothing.
- Lesions. The test masks hold 0.92 to 25 ml of lesion (median 5.7 ml) and 4 to 193 lesions (P56: 4 in 1.8 ml; P65: 193 in 25 ml), the development masks 0.79 to 43 ml. The clinical table's volumes equal the masks' to the voxel on all 50 patients of the pool.
- Room. Small lesions weigh on the Dice: the test masks moved by one voxel along one axis keep a Dice of 0.67 to 0.84 (mean 0.76). The surface Dice, at 1 mm since the competition sets no tolerance, forgives such a move. The 15 teams, given all three contrasts, reached 0.50 to 0.71 on the 22 test patients.
- Outside numbers. None is scored as here, on ten of the 22 and from the FLAIR alone. The test masks are public: a network trained on the whole release may have seen these patients.
No method has a result on every case of this dataset yet.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
The first test caseClick to look at every case.
- Test cases
- 10 test patients drawn with seed 2026 among the competition's 22 test patients (P54 to P75), one visit each, and 10 development patients among its 28 training patients with one visit, the last of the same seeded order. The 25 training patients with two visits or more (P1 to P14, P19, P20, P22, P28, P31, P33, P49 to P53) are left out: they are the registration dataset MSLesSeg's.
- Truth
- contours by experts the dataset's mask of the lesions hyperintense on the FLAIR at the patient's visit, outlined by one junior rater on the FLAIR registered to MNI152, the T1 and the T2 read for the doubtful ones, reviewed with the senior experts as the work went, then validated by a neuroradiologist and a neurologist who specialise in multiple sclerosis
- Source
- doi.org/10.6084/m9.figshare.27919209
api.figshare.com/v2/articles/27919209
doi.org/10.1038/s41597-025-05250-y
github.com/alessiarondinella/MSLesSeg-2024 - Licence
- CC BY 4.0
10 cases for tuning, never tested
P48, P43, P21, P34, P46, P42, P37, P16, P30, P35