AMOS CT, abdominal organs

Abdominal CT of one patient from the CT part of AMOS 2022, with the AMOS annotations of 15 organs.

10 test casesabdomenimage CT

Read before comparing (1)

Cases. The fixed images of AMOS_CTCT's pairs, each on its pair's side of the split. The moving images are left out: their headers carry the pair's start, so they are not the images as delivered.

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
1	KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUapp	0.776 ±0.091	0.699 ±0.119	7.57 ±3.36	142	24.4	7.2	Laptop
2	KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp	0.766 ±0.099	0.688 ±0.128	20.27 ±14.71	35	2.7	11.0	Laptop
3	CT-FM whole-body segmentationGPUapp	0.351 ±0.135	0.275 ±0.115	68.58 ±29.66	56	10.9	15.0	Laptop
44–5	MONAI wholeBody CTGPUapp14 of 15 structures9 failed	0.055 ±0.124	0.037 ±0.083	18.98	19	4.4	9.2	Workstation
54–5	KonfAI segmentation app · MRSegmentator, KonfAI appGPUapp14 of 15 structures	0.219 ±0.077	0.144 ±0.060	71.58 ±14.12	29	2.7	10.7	Laptop

The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it.

1 result not ranked: an older run, cases still missing, or a failure on every case

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
	MONAI wholeBody CTGPUapp14 of 15 structuresolder run	0.714 ±0.109	0.619 ±0.134	12.98 ±4.82	17	3.2	17.0	Laptop

Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.

Case by case

Measure

Dice	1	2	3	4	5	6	7	8	9	10
KonfAI segmentation app · TotalSegmentator (total), the authors' package	0.851	0.682	0.810	0.839	0.901	0.622	0.895	0.887	0.739	0.535
KonfAI segmentation app · TotalSegmentator (total), KonfAI app	0.844	0.665	0.804	0.838	0.897	0.581	0.891	0.891	0.736	0.511
CT-FM whole-body segmentation	0.122	0.189	0.432	0.389	0.591	0.108	0.509	0.454	0.560	0.159
MONAI wholeBody CT	failed	failed	failed	failed	failed	0.512	failed	failed	failed	failed
KonfAI segmentation app · MRSegmentator, KonfAI app	0.322	0.082	0.169	0.231	0.298	0.035	0.310	0.278	0.186	0.129

Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.

About the data

The first test caseClick to look at every case.

Truth: contours by experts 15 organs, the dataset's annotations
Source: zenodo.org/records/7155725
Licence: CC BY 4.0

10 cases for tuning, never tested

pair_0001_0004/fixed, pair_0005_0006/fixed, pair_0007_0009/fixed, pair_0197_0198/fixed, pair_0152_0153/fixed, pair_0054_0057/fixed, pair_0045_0047/fixed, pair_0142_0143/fixed, pair_0030_0033/fixed, pair_0103_0104/fixed