fidoptT2-weighted Kidney MRI
Coronal T2-weighted MR of the kidneys of a healthy volunteer or a patient with chronic kidney disease, both kidneys traced by hand as one label (University of Nottingham, Daniel et al. 2021).
Read before comparing (7)
- Scans. A coronal 2D HASTE on a 3 T Philips Ingenia, tuned for the contrast around the kidneys, in one breath-hold of 17 to 23 s: 13 slices of 1.46 x 1.46 mm, 5 mm thick and 5.5 mm apart. The values are the files' own floats, without a unit. The headers' axes are right, checked on the anatomy.
- Subjects. 30 healthy volunteers and 30 patients with chronic kidney disease, stages 1 to 5, whose kidneys often hold cysts.
- Truth. Each subject's mask was drawn by hand in MRIcron by one of three trained observers, the renal hilum and cysts left out. The files hold one label for both kidneys, so a method's kidney_left and kidney_right are merged before scoring, and a swapped side goes unseen.
- Room. On four healthy test subjects traced by all three observers, two observers agree at a Dice of 0.93 to 0.94 and an HD95 of 3.6 to 5.8 mm, the left kidney a little less, its border with the spleen faint. A method at a Dice of 0.93 is as close to the truth as a second observer.
- Split. The provider trained its network, Renal_Segmentor, on the 50 subjects scanned once, and tested it on the ten scanned five times in one session, placed again between scans. Only the first scan of each test subject is in the corpus, so no subject is in both test and development.
- Surfaces. The slices are 5.5 mm apart: front to back a kidney's surface is known to a slice, so the HD95 and the surface Dice at 1 mm, the provider giving no tolerance, move by whole slices there.
- Outside numbers. The provider's network scored a Dice of 0.93 on all five scans of the ten test subjects: the same subjects, on five scans each where the corpus holds one. A model trained on the whole record has seen the test scans.
| Accuracy | Cost | |||||||
|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Surface Dice | HD95 mm | Time s | RAM GB | GPU GB | Images |
| 1 | KonfAI segmentation app · TotalSegmentator (total_mr), KonfAI appGPUapp | 0.876 ±0.017 | 0.587 ±0.055 | 8.88 ±2.88 | 10 | 2.3 | 3.3 | |
| 2 | KonfAI segmentation app · TotalSegmentator (total_mr), the authors' packageGPUapp | 0.867 ±0.019 | 0.572 ±0.063 | 9.04 ±3.09 | 19 | 9.8 | 2.7 | |
| 3 | KonfAI segmentation app · MRSegmentator, KonfAI appGPUapp | 0.266 ±0.252 | 0.168 ±0.166 | 91.88 ±65.38 | 2.4 | 5.0 | ||
The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.
Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.
Case by case
Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.
About the data
- Test cases
- The provider's test set: the ten subjects it held out to test its network, 5 healthy and 5 CKD, the first of their five scans each. 10 development cases, ten of the 50 others in the seeded order, 7 healthy and 3 CKD; with the test set fixed by the provider, the draw has no strata.
- Truth
- contours by experts both kidneys under one label, traced by hand by one of three trained observers, the renal hilum and cysts left out
- Licence
- CC BY 4.0
10 cases for tuning, never tested
HC_16, HC_02, CKD_20, HC_11, HC_04, CKD_07, HC_23, CKD_15, HC_09, HC_07