ThoraxCBCT (Learn2Reg 2023), validation

One Learn2Reg 2023 ThoraxCBCT validation pair on the board's 1 mm grid, CBCT fixed and FBCT moving, with the Dec 2023 validation labels (13 of the 15 classes) and manual landmarks (fixed_keypoints.csv: Dec 2023 keypoints).

6 test pairslungsame patientfixed CBCT, moving CTdelivered aligned

Look at the images

Read before comparing (3)

Labels. The board scores 15 labels on its own grid; the released validation labels hold 13 of them, since no released map has the inferior vena cava or the carina. Each map renumbers its labels from 1: the original ids were recovered against the board's per-label Dice at zero displacement.
Scores. The labels were put back on the board's 1 mm grid: Dice and HD95 here are within 0.008 and 0.8 mm of the board's per-label values at zero displacement, and are not the board's numbers. The board's keypoints are not public, so the landmark error here is on the released manual landmarks, 8 to 19 a pair.
Development. The training pairs' label maps are TotalSegmentator's (task total). renumber_development.py puts them into the challenge's numbering (totalsegmentator_ids.json): the lobes as the lungs (3), the vertebrae and ribs as one label (4), the other scored structures under their own ids, the rest dropped. TotalSegmentator makes no pulmonary artery or tumour, so these pairs hold 9 to 11 of the 13 named labels.

		Accuracy					Smoothness		Cost
#	Method	Dice	Change	HD95 mm	TRE mm	rTRE	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Images
11–3	greedyCPUapp	0.588 ±0.053	+0.243	56.21 ±3.83	5.02 ±1.70	0.0089 ±0.0030	0.042	0.00	110	6.1
21–3	ConvexAdam (MIND)GPUapp	0.594 ±0.023	+0.250	58.34 ±3.36	4.62 ±1.00	0.0082 ±0.0018	0.117	0.95	15	3.9	8.4
31–3	uniGradICONGPUapp	0.642 ±0.020	+0.297	48.37 ±4.62	9.50 ±1.83	0.0168 ±0.0032	0.170	0.18	88	3.8	12.7
44–5	corrFieldGPUapp	0.589 ±0.033	+0.244	40.83 ±2.53	6.70 ±0.86	0.0118 ±0.0015	0.801	3.35	62	3.9	20.1
54–5	anatomix + ConvexAdamGPUapp	0.511 ±0.132	+0.167	56.88 ±3.93	6.46 ±4.67	0.0114 ±0.0082	0.096	0.22	117	4.6	20.7
	No registration	0.345 ±0.113		57.88 ±3.81	10.75 ±3.95	0.0190 ±0.0070	0.000	0.00

The order averages each method's ranks on Dice, HD95 mm, TRE mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 6 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.

Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Scored the way the challenge scores

The same results with the ThoraxCBCT (Learn2Reg 2023) challenge's own measures, on its validation pairs: mean ± standard deviation.

Method	DSC	HD95	TRE	SDlogJ
greedy	0.588 ± 0.046	56.204 ± 3.332	5.015 ± 1.476	0.042 ± 0.006
ConvexAdam (MIND)	0.594 ± 0.020	58.337 ± 2.921	4.623 ± 0.872	0.117 ± 0.011
uniGradICON	0.642 ± 0.017	48.367 ± 4.016	9.500 ± 1.590	0.170 ± 0.009
corrField	0.589 ± 0.029	40.829 ± 2.202	6.699 ± 0.744	0.801 ± 0.189
anatomix + ConvexAdam	0.511 ± 0.115	56.879 ± 3.418	6.462 ± 4.060	0.096 ± 0.008
No registration	0.345 ± 0.098	57.876 ± 3.317	10.753 ± 3.437	0.000 ± 0.000

What the challenge published

On the challenge's test set: other pairs than the ones above, and entries trained or tuned for the task. It shows the level of the challenge; these are not results of this benchmark. The challenge's keypoints and two of its 15 labels are not public: the lines of this board above are on the released landmarks and 13 labels.

Method	DSC	HD95	TRE	SDlogJ
No registration	0.290	48.13	11.91
ConvexAdam	0.684	38.00	4.93	0.100
SynDeeds	0.702	22.97	6.01	0.150
Niftyreg	0.621	43.41	4.90	0.050
DeedsBCV	0.690	22.38	8.76	0.160
Fourier-Net	0.619	47.80	4.03	0.070
SSKJLBW	0.695	17.60	5.80	0.260
Voxelmorph++	0.463	33.34	13.06	0.120

Source: Heyer et al., OncoReg: Medical Image Registration for Oncological Challenges, MELBA 2026, Table 3 (10 test pairs).

Pair by pair

Measure

Dice	1	2	3	4	5	6
greedy	0.577	0.578	0.655	0.503	0.608	0.606
ConvexAdam (MIND)	0.607	0.561	0.618	0.600	0.605	0.575
uniGradICON	0.663	0.613	0.661	0.633	0.642	0.637
corrField	0.611	0.528	0.609	0.608	0.594	0.583
anatomix + ConvexAdam	0.498	0.572	0.586	0.263	0.559	0.591
No registration	0.263	0.410	0.411	0.179	0.336	0.468

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

The first test caseClick to look at every case.

Test pairs: The 6 official validation pairs are the test cases. The 10 development pairs are the task's training patients 0000 to 0010, CBCT fixed and planning CT moving, with network labels and automatic keypoints.
Truth: contours as released landmarks by experts the challenge's validation labels the challenge's public points
Source: learn2reg.grand-challenge.org
github.com/MDL-UzL/L2R
Licence: CC BY 3.0 (TCIA 4D-Lung)

10 pairs for tuning, never tested

ThoraxCBCT_0000_0001, ThoraxCBCT_0005_0002, ThoraxCBCT_0009_0001, ThoraxCBCT_0005_0001, ThoraxCBCT_0000_0002, ThoraxCBCT_0001_0001, ThoraxCBCT_0004_0002, ThoraxCBCT_0002_0001, ThoraxCBCT_0003_0002, ThoraxCBCT_0004_0001