Abdomen CT (Learn2Reg 2020, Task 3)

Abdominal CT of one patient with 13 organs labelled (the Learn2Reg 2020 abdomen task, from the Multi-Atlas Labeling Beyond the Cranial Vault challenge).

10 test casesabdomenimage CTresearch only

Look at the images

Read before comparing (1)

Cases. The test cases are the ten images of L2R_AbdomenCTCT's validation pairs, each taken once and named after the first pair that holds it; the 45 pairs join them two by two. The development cases are both images of L2R_AbdomenCTCT's development pairs.

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Images
1	KonfAI segmentation app · MRSegmentator, KonfAI appGPUapp	0.365 ±0.034	0.210 ±0.026	62.90 ±13.76		2.7	4.9
22–3	KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUapp	0.264 ±0.023	0.186 ±0.021	85.28 ±5.43		14.8	4.3
32–3	KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp	0.259 ±0.017	0.182 ±0.017	85.40 ±4.46	23	2.8	4.9
4	CT-FM whole-body segmentationGPUapp	0.209 ±0.016	0.132 ±0.012	92.22 ±7.10	5	2.4	5.2
5	MONAI wholeBody CTGPUapp	0.183 ±0.013	0.116 ±0.007	93.90 ±7.12	17	3.6	9.3

The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.

1 result not ranked: an older run, cases still missing, or a failure on every case

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
	MONAI wholeBody CTGPUappolder run	0.183 ±0.013	0.116 ±0.007	93.90 ±7.13	11	2.3	9.4	Laptop

Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.

Case by case

Measure

Dice	1	2	3	4	5	6	7	8	9	10
KonfAI segmentation app · MRSegmentator, KonfAI app	0.351	0.316	0.293	0.338	0.412	0.446	0.342	0.375	0.355	0.418
KonfAI segmentation app · TotalSegmentator (total), the authors' package	0.263	0.204	0.274	0.281	0.270	0.284	0.310	0.241	0.226	0.290
KonfAI segmentation app · TotalSegmentator (total), KonfAI app	0.252	0.218	0.267	0.280	0.246	0.266	0.301	0.244	0.239	0.273
CT-FM whole-body segmentation	0.237	0.204	0.164	0.233	0.188	0.223	0.229	0.195	0.208	0.211
MONAI wholeBody CT	0.181	0.171	0.156	0.176	0.192	0.191	0.203	0.155	0.198	0.206

Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.

About the data

Truth: contours by experts the challenge's validation labels
Source: learn2reg.grand-challenge.org
github.com/MDL-UzL/L2R
Licence: unclear: the BTCV terms on Synapse are not public

20 cases for tuning, never tested

0030_0000_0026_0000/fixed, 0030_0000_0026_0000/moving, 0024_0000_0011_0000/fixed, 0024_0000_0011_0000/moving, 0017_0000_0006_0000/fixed, 0017_0000_0006_0000/moving, 0003_0000_0015_0000/fixed, 0003_0000_0015_0000/moving, 0023_0000_0021_0000/fixed, 0023_0000_0021_0000/moving, 0008_0000_0027_0000/fixed, 0008_0000_0027_0000/moving, 0002_0000_0005_0000/fixed, 0002_0000_0005_0000/moving, 0018_0000_0009_0000/fixed, 0018_0000_0009_0000/moving, 0014_0000_0029_0000/fixed, 0014_0000_0029_0000/moving, 0012_0000_0020_0000/fixed, 0012_0000_0020_0000/moving