fidoptMRBrainS18
Brain MR of an adult over 50 (diabetes, dementia or Alzheimer's disease, or a control with raised cardiovascular risk), T1, T1 inversion recovery and FLAIR aligned as one three-channel image, eight brain structures segmented by hand (MRBrainS18, UMC Utrecht).
Read before comparing (10)
- Cohort. 30 subjects over 50 scanned on a 3 T scanner at UMC Utrecht: patients with diabetes, dementia or Alzheimer's disease and matched controls with increased cardiovascular risk, with varying atrophy and white matter lesions (the record's readme). MRBrainS18 revises and extends MRBrainS13, whose 20 subjects it contains.
- Images. The provider's bias-corrected (N4ITK) T1 and T1 inversion recovery registered to the multi-slice FLAIR, and the FLAIR, as one image of channels t1, t1ir, flair: 240 x 240 x 48 voxels of 0.958 x 0.958 x 3 mm, oblique, 32-bit floats without unit. The 3D T1 at 1 mm and the unregistered scans are not given, nor is a mask. Some T1-IR show artefacts at the bottom of the scan (the readme).
- Channels. No method of the corpus takes t1, t1ir and flair yet; a method that takes a single image does not run here.
- Truth. One reference a subject, made by experts; no agreement between raters is published for this set. Graded expert. The CSF label holds the venous sinuses and the falx; vessels are not segmented apart.
- Ignored. Label 9 (infarction) appears on 5 test subjects (subject_24, 25, 26, 28, 30; 392 to 7,897 voxels) and 2 development subjects; label 10 on none. As in the challenge's evaluation, those voxels are taken out of the truth and of the prediction (ignore). subject_13 has no white matter lesion.
- Checks. On every test subject the T1 reads white matter brighter than cortex and cortex brighter than the ventricles, and the lesions read 1.24 to 1.77 times the white matter on the FLAIR.
- Room. Each structure of the test subjects moved one voxel in plane keeps a Dice of 0.57 or more; with 3 mm slices the surface Dice at 1 mm is strict.
- Start. Labelling the whole head (the FLAIR's non-zero voxels) as each structure scores a mean Dice of 0.07 over the eight on the test subjects. TotalSegmentator has no brain-tissue task for MR.
- Overlap. No other dataset of the corpus names MRBrainS or UMC Utrecht's brain data, and no built image has the grid of any of the 30 subjects (12,304 stores compared by size and spacing, a check that finds AbdomenDIRQA's case20 in MSD_Pancreas).
- Outside numbers. The challenge ranked on these 23 subjects with labels 9 and 10 left out, Dice, HD95 and volume similarity per structure: comparable in definition, not in scoring code.
| Accuracy | Cost | |||||||
|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Images |
| 1 | SynthSeg 1.0CPUapp6 of 8 structures | 0.840 ±0.006 | 0.855 ±0.008 | 4.03 ±0.34 | 91 | 10.7 | 0.3 | |
| 2 | FastSurferVINNGPUapp7 of 8 structures | 0.767 ±0.012 | 0.806 ±0.010 | 6.18 ±0.49 | 52 | 3.9 | 3.9 | |
The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 23 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.
Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
- Test cases
- The challenge's test set taken whole: its 23 subjects, whose references are now public. Development: its 7 training subjects, all of them. One examination a subject.
- Truth
- contours by experts the challenge's manual reference on the FLAIR's grid, made by experts in brain segmentation: every structure drawn on the T1 except the white matter lesions (on the FLAIR) and the outer border of the CSF (on the T1 and the T1-IR); MRBrainS13's protocol, which this set revises and extends, has trained research assistants draw every axial slice with closed freehand splines, checked and corrected by three experts. Labels 9 (infarction) and 10 (other) are not scored, as in the challenge
- Source
- doi.org/10.34894/E0U32Q
dataverse.nl/api/datasets/:persistentId/?persistentId=doi:10.34894/E0U32Q
mrbrains18.isi.uu.nl
github.com/umcu-isi/mrbrains18
doi.org/10.34894/645ZIN - Licence
- UMC Utrecht Data Access License
7 cases for tuning, never tested
subject_1, subject_27, subject_7, subject_14, subject_5, subject_29, subject_4