MSD Spleen

Portal venous abdominal CT of a patient treated by chemotherapy for liver metastases at Memorial Sloan Kettering Cancer Center, with the spleen segmented (Medical Segmentation Decathlon, Task09 Spleen).

10 test casesabdomenimage CT

Look at the images

Read before comparing (8)

Pool. The decathlon's Task09: 61 portal venous CTs of patients treated by chemotherapy for liver metastases at Memorial Sloan Kettering, 41 training scans with labels and 20 test scans. The cases here are training cases.
Truth. A spline drawn on the spleen was grown by a level set in the Scout application, then an expert abdominal radiologist adjusted the contours by hand; no second reading is reported. Every map holds 0 and 1 on its image's grid, the spleen 107 to 427 ml on the test cases.
Images. The decathlon's NIfTI files as delivered: Hounsfield units with values below -1024 set to -1024, the voxels transposed without resampling to the closest RAS orientation.
Left out. spleen_12, spleen_14, spleen_18 and spleen_19 are scans of TCIA's Colorectal-Liver-Metastases, from which CRLMCT draws its pairs: each shares a slice with CRLM-CT-1032, -1033, -1114 and -1003, and spleen_19 is, voxel for voxel once clipped, the fixed image of CRLMCT's development pair 1003_1091.
Overlap. The task shares six files byte for byte with the hepatic vessel task and three scans with the colon task; spleen_32, a development case, is colon_041, which MSD_Colon leaves out. Of the 100 cases built for the five MSD datasets none has another's grid; spleen_9 shares CRLM-CT-1108's slice count and pixel size but holds no slice of it.
Public labels. The test cases are public training cases with their labels: a network trained on the decathlon's spleen task, as the challenge's entries were, has seen them.
Room. Little: on the hidden test set the 19 entries' mean Dice ran from 0.82 to 0.97 (median 0.94; nnU-Net 0.96), and the live challenge's median rose to 0.97. TotalSegmentator scores 0.954 on spleen_10 (HD95 2.2 mm, surface Dice 0.976). The surface Dice is at 3 mm, the error the challenge's clinicians accepted.
Outside numbers. None compare: the challenge ranks its entries on its hidden test set.

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Images
11–2	KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp	0.943 ±0.017	0.969 ±0.030	2.99 ±1.90	22	3.2	7.1
21–2	KonfAI segmentation app · TotalSegmentator (total), the authors' packageGPUapp	0.942 ±0.019	0.966 ±0.033	3.44 ±2.55		16.7	5.0
22–5	MONAI wholeBody CTGPUapp7 failed	0.285 ±0.328	0.294 ±0.338	2.33 ±1.54	17	3.9	7.7
42–5	CT-FM whole-body segmentationGPUapp5 failed	0.245 ±0.277	0.225 ±0.251	67.17 ±48.55	19	6.9	6.9
52–5	KonfAI segmentation app · MRSegmentator, KonfAI appGPUapp	0.000 ±0.000	0.000 ±0.000	206.04 ±16.02		6.3	6.8

The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Workstation.

1 result not ranked: an older run, cases still missing, or a failure on every case

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
	MONAI wholeBody CTGPUappolder run	0.942 ±0.017	0.963 ±0.036	4.05 ±3.75	19	2.7	11.8	Laptop

Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.

Case by case

Measure

Dice	1	2	3	4	5	6	7	8	9	10
KonfAI segmentation app · TotalSegmentator (total), KonfAI app	0.952	0.939	0.961	0.937	0.942	0.955	0.947	0.880	0.964	0.956
KonfAI segmentation app · TotalSegmentator (total), the authors' package	0.954	0.938	0.960	0.932	0.939	0.957	0.948	0.871	0.962	0.956
MONAI wholeBody CT	failed	failed	failed	0.940	failed	failed	failed	failed	0.960	0.951
CT-FM whole-body segmentation	0.810	0.058	failed	0.017	failed	failed	failed	failed	0.608	0.961
KonfAI segmentation app · MRSegmentator, KonfAI app	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000

Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.

About the data

The first test caseClick to look at every case.

Test cases: 10 test cases drawn with seed 2026 among the 27 training cases outside development, and 10 development cases. Four scans of Colorectal-Liver-Metastases are left out. The labels of the decathlon's 20 test scans were never released.
Truth: contours by a program, verified by people the spleen (label 1), the decathlon's mask: a spline drawn on it grown by a level set in the Scout application, the contours then adjusted by hand by an expert abdominal radiologist
Source: medicaldecathlon.com
arxiv.org/abs/1902.09063
doi.org/10.1038/s41467-022-30695-9
msd-for-monai.s3-us-west-2.amazonaws.com/Task09_Spleen.tar
www.cancerimagingarchive.net/collection/colorectal-liver-metastases
Licence: CC BY-SA 4.0

10 cases for tuning, never tested

spleen_13, spleen_31, spleen_26, spleen_56, spleen_9, spleen_25, spleen_22, spleen_32, spleen_8, spleen_46