VerSe, vertebrae

Spine CT of one patient, from the cervical spine alone to the whole spine, with one mask a vertebra (VerSe 2019 and 2020).

22 test casesspineimage CT

Look at the images

Read before comparing (2)

Cases. Both images of each VerSe pair, as delivered: a test pair's images are test cases, a development pair's development cases.
Field of view. A scan holds only the vertebrae in its field of view, 5 to 20 of the 25 named.

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
1	KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp24 of 25 structures4 failed	0.574 ±0.140	0.598 ±0.146	5.96 ±3.82	34	3.6	5.8	Workstation
2	CT-FM whole-body segmentationGPUapp24 of 25 structures5 failed	0.272 ±0.105	0.257 ±0.098	41.65 ±23.06	232	5.1	16.0	Laptop
3	MONAI wholeBody CTGPUapp24 of 25 structures18 failed	0.113 ±0.109	0.112 ±0.108	6.48 ±6.82	16	3.7	4.5	Workstation

The order averages each method's ranks on Dice, Surface Dice, HD95 mm, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 22 cases. Two neighbours are tied when a paired Wilcoxon test on the shared cases does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it.

2 results not ranked: an older run, cases still missing, or a failure on every case

		Accuracy			Cost
#	Method	Dice	Surface Dice	HD95 mm	Time s	RAM GB	GPU GB	Machine	Images
	KonfAI segmentation app · TotalSegmentator (total), KonfAI appGPUapp24 of 25 structures4 failedolder run	0.574 ±0.140	0.598 ±0.146	5.96 ±3.82	15	2.8	10.0	Laptop
	MONAI wholeBody CTGPUapp24 of 25 structures3 failedolder run	0.510 ±0.110	0.500 ±0.108	10.43 ±4.67	23	3.3	15.1	Laptop

Dice: how much the organ contours overlap, 1 is perfect. Surface Dice: the share of the contours' surfaces within a tolerance of each other, 1 is perfect. HD95: distance between the contours' surfaces.

Case by case

Measure

Dice	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22
KonfAI segmentation app · TotalSegmentator (total), KonfAI app	0.663	failed	failed	0.791	0.672	0.705	0.828	0.584	0.770	0.725	failed	0.565	0.782	0.858	0.267	0.860	0.753	0.751	failed	0.306	0.866	0.785
CT-FM whole-body segmentation	failed	failed	failed	0.759	0.510	0.554	0.126	0.081	0.080	0.047	failed	0.456	0.404	0.443	0.221	0.458	0.297	0.469	failed	0.201	0.621	0.212
MONAI wholeBody CT	0.652	failed	failed	0.591	0.654	0.598	failed	failed	failed	failed	failed	failed	failed	failed	failed	failed	failed	failed	failed	failed	failed	failed

Click a score to look at that result: the image, the truth in green and the method's map in red. Underlined scores have their image; for the others you see the case without the result.

About the data

The first test caseClick to look at every case.

Truth: contours by experts the dataset's vertebra masks, one label a vertebra, corrected by hand and approved by radiologists
Source: github.com/anjany/verse
osf.io/nqjyw
osf.io/t98fz
doi.org/10.1016/j.media.2021.102166
doi.org/10.1148/ryai.2020190138
doi.org/10.1038/s41597-021-01060-0
Licence: CC BY-SA 4.0

20 cases for tuning, never tested

sub-verse145_sub-verse095/fixed, sub-verse145_sub-verse095/moving, sub-verse505_sub-verse642/fixed, sub-verse505_sub-verse642/moving, sub-verse716_sub-verse565/fixed, sub-verse716_sub-verse565/moving, sub-verse591_sub-verse529/fixed, sub-verse591_sub-verse529/moving, sub-verse600_sub-verse821/fixed, sub-verse600_sub-verse821/moving, sub-verse596_sub-verse558/fixed, sub-verse596_sub-verse558/moving, sub-gl047_sub-verse587/fixed, sub-gl047_sub-verse587/moving, sub-gl016_sub-verse549/fixed, sub-gl016_sub-verse549/moving, sub-verse762_sub-verse752/fixed, sub-verse762_sub-verse752/moving, sub-verse599_sub-verse833/fixed, sub-verse599_sub-verse833/moving