Learn2Reg 2020, Task 4, validation

Hippocampus MR crops of two people, with the head and the tail of the hippocampus labelled (Learn2Reg 2020 Task 4, from the Medical Segmentation Decathlon).

60 test pairsbraininter-patientfixed MR, moving MRdelivered aligned

Look at the images

Read before comparing (2)

Crops. The images are crops of the Medical Segmentation Decathlon's hippocampus task, a left and a right crop of each person: consecutive numbers 2k-1 and 2k look like one person, their intensity profiles about ten times closer than other pairs', and the numbers missing from the release drop out two by two.
Validation pairs. Two official pairs, 0230 to 0231 and back, would then join the two crops of one person rather than two people.

No method has a result on every pair of this dataset yet.

		Accuracy			Smoothness		Cost
#	Method	Dice	Change	HD95 mm	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Images
	No registration	0.568 ±0.030		3.68 ±0.32	0.000	0.00

Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Scored the way the challenge scores

The same results with the Learn2Reg 2020, Task 4 challenge's own measures, on its validation pairs: mean ± standard deviation.

Method	DSC	HD95	SDlogJ
No registration	0.568 ± 0.115	3.676 ± 1.222	0.000 ± 0.000

What the challenge published

On the challenge's test set: other pairs than the ones above, and entries trained or tuned for the task. It shows the level of the challenge; these are not results of this benchmark.

Method	DSC	HD95	SDlogJ
No registration	0.550	3.91
LapIRN	0.880	1.30	0.050
MEVIS	0.850	1.55	0.050
ConvexAdam	0.840	1.85	0.070
lWM	0.790	2.20	0.080
Estienne	0.850	1.51	0.090
PDD-Net	0.780	2.23	0.070
NiftyReg	0.760	2.72	0.090
corrField	0.720	2.89	0.050
Gunnarsson	0.740	2.82	0.160

Source: Hering et al., Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning, IEEE TMI 2023, Table IV.

Pair by pair

Measure

Dice	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45	46	47	48	49	50	51	52	53	54	55	56	57	58	59	60
No registration	0.506	0.768	0.623	0.761	0.721	0.589	0.416	0.543	0.571	0.509	0.602	0.465	0.660	0.591	0.588	0.753	0.683	0.538	0.372	0.418	0.434	0.406	0.483	0.528	0.633	0.464	0.420	0.682	0.617	0.708	0.708	0.617	0.682	0.420	0.464	0.633	0.528	0.483	0.406	0.434	0.418	0.372	0.538	0.683	0.753	0.588	0.591	0.660	0.465	0.602	0.509	0.571	0.543	0.416	0.589	0.721	0.761	0.623	0.768	0.506

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

The first test caseClick to look at every case.

Test pairs: The 60 official validation pairs are the test cases. The 10 development pairs join the task's training images 0001 to 0200, which no validation pair uses, put in the order of sha256("2026:L2R_HippocampusMR:<number>") and paired two by two.
Truth: contours by experts the challenge's validation labels
Source: learn2reg.grand-challenge.org
github.com/MDL-UzL/L2R
Licence: CC BY-SA 4.0 (Medical Segmentation Decathlon)

10 pairs for tuning, never tested

0034_0000_0200_0000, 0048_0000_0038_0000, 0063_0000_0122_0000, 0137_0000_0171_0000, 0188_0000_0016_0000, 0049_0000_0051_0000, 0001_0000_0021_0000, 0045_0000_0155_0000, 0110_0000_0132_0000, 0123_0000_0165_0000