Learn2Reg 2020, Task 3, validation

Abdominal CT scans of two different patients, a validation pair of Learn2Reg 2020 Task 3, with 13 labels on both.

45 test pairsabdomeninter-patientfixed CT, moving CTdelivered alignedresearch only

Read before comparing (1)

Board. The board's HD95 is in voxels of the 2 mm grid: half the value in mm shown here.

No method has a result on every pair of this dataset yet.

		Accuracy			Smoothness		Cost
#	Method	Dice	Change	HD95 mm	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Images
	No registration	0.259 ±0.022		40.21 ±3.44	0.000	0.00

Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Scored the way the challenge scores

The same results with the Learn2Reg 2020, Task 3 challenge's own measures, on its validation pairs: mean ± standard deviation.

Method	DSC	HD95	SDlogJ
No registration	0.259 ± 0.072	20.106 ± 5.718	0.000 ± 0.000

What the challenge published

On the challenge's test set: other pairs than the ones above, and entries trained or tuned for the task. It shows the level of the challenge; these are not results of this benchmark.

Method	DSC	HD95	SDlogJ
No registration	0.280	21.78
ConvexAdam	0.690	11.03	0.060
LapIRN	0.670	12.51	0.120
Estienne	0.690	11.77	0.180
MEVIS	0.510	18.21	0.140
corrField	0.490	17.22	0.280
PIMed	0.490	15.75	0.050
PDD-Net	0.490	17.75	0.410
Joutard	0.400	17.25	0.050
NiftyReg	0.450	20.70	0.360
Gunnarsson	0.430	18.55	0.130

Source: Hering et al., Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning, IEEE TMI 2023, Table V.

Pair by pair

Measure

Dice	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40	41	42	43	44	45
No registration	0.205	0.189	0.339	0.288	0.289	0.223	0.357	0.362	0.278	0.199	0.234	0.155	0.202	0.132	0.250	0.333	0.263	0.298	0.122	0.290	0.259	0.214	0.196	0.148	0.231	0.418	0.286	0.338	0.274	0.262	0.228	0.176	0.257	0.228	0.291	0.303	0.296	0.233	0.217	0.225	0.208	0.181	0.431	0.379	0.366

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

Test pairs: The 45 official validation pairs are the test cases. The 10 development pairs join the 20 of the task's 30 training subjects that are in no validation pair, two by two in the order of sha256("2026:L2R_AbdomenCTCT:<subject>").
Truth: contours by experts the challenge's validation labels
Source: learn2reg.grand-challenge.org
github.com/MDL-UzL/L2R
Licence: unclear: the BTCV terms on Synapse are not public

10 pairs for tuning, never tested

0030_0000_0026_0000, 0024_0000_0011_0000, 0017_0000_0006_0000, 0003_0000_0015_0000, 0023_0000_0021_0000, 0008_0000_0027_0000, 0002_0000_0005_0000, 0018_0000_0009_0000, 0014_0000_0029_0000, 0012_0000_0020_0000