Pancreas-CT, 8 organs

Portal venous abdominal CT of one person with Gibson et al.'s reference segmentations of eight organs (NIH Pancreas-CT).

10 test casesabdomenimage CT

Look at the images

Read before comparing (1)

Cases. The fixed images of PancreasCT8's pairs: a test pair's image is a test case, a development pair's a development case. The moving images are left out.

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Images
11–2	KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp	0.841 ±0.027	0.665 ±0.048	11.36 ±3.41	21	3.6	6.6
21–2	KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUapp	0.840 ±0.025	0.663 ±0.049	11.49 ±3.43		20.5	4.2
33–5	CT-FM whole-body segmentationGPUapp	0.766 ±0.055	0.559 ±0.072	22.93 ±10.57	86	16.8	6.3
43–5	MONAI wholeBody CTGPUapp6 failed	0.317 ±0.295	0.232 ±0.219	16.13 ±13.63	21	4.5	9.1
53–5	KonfAI segmentation app · MRSegmentator, KonfAI appGPUapp4 failed	0.156 ±0.098	0.070 ±0.047	83.18 ±24.04	77	29.2	7.2

The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.

1 result not ranked: an older run, cases still missing, or a failure on every case

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
	MONAI wholeBody CTGPUappolder run	0.808 ±0.044	0.608 ±0.063	18.35 ±8.86	14	2.8	10.6	Laptop

Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.

Case by case

Measure

Dice	1	2	3	4	5	6	7	8	9	10
KonfAI segmentation app · TotalSegmentator (total), KonfAI app	0.752	0.834	0.864	0.829	0.868	0.826	0.843	0.857	0.897	0.839
KonfAI segmentation app · TotalSegmentator (total), the authors' package	0.768	0.838	0.863	0.815	0.864	0.823	0.831	0.855	0.900	0.839
CT-FM whole-body segmentation	0.660	0.739	0.805	0.742	0.850	0.800	0.684	0.811	0.890	0.678
MONAI wholeBody CT	0.677	failed	failed	failed	0.860	failed	0.786	0.846	failed	failed
KonfAI segmentation app · MRSegmentator, KonfAI app	failed	0.283	0.262	0.273	failed	failed	failed	0.236	0.304	0.201

Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.

About the data

The first test caseClick to look at every case.

Truth: contours by experts 8 organs, Gibson et al.'s reference segmentations: spleen, left kidney, gallbladder, oesophagus, liver, stomach, pancreas, duodenum
Source: www.cancerimagingarchive.net/collection/pancreas-ct
doi.org/10.7937/K9/TCIA.2016.tNB1kqBU
zenodo.org/records/1169361
doi.org/10.1109/TMI.2018.2806309
Licence: CC BY 3.0 (images), CC BY 4.0 (labels)

10 cases for tuning, never tested

0004_0047/fixed, 0005_0013/fixed, 0008_0022/fixed, 0048_0012/fixed, 0029_0043/fixed, 0017_0003/fixed, 0042_0040/fixed, 0021_0031/fixed, 0034_0045/fixed, 0018_0046/fixed