Abdominal-DIR-QA

Contrast-enhanced CT of the abdomen of one patient at two dates (Abdominal-DIR-QA).

10 test pairsabdomensame patientfixed CT, moving CTaligned once by the benchmark

Read before comparing (6)

Truth. Vessel bifurcations placed by hand on one image, carried to the other by a registration and refined automatically or by hand; two observers checked every pair. The authors measure 1.2 mm between manual placements.
Alignment. The scans keep their native voxels and grids, both with their origin at zero. The moving image's header carries one translation from elastix mutual information inside each body, the best of several starts, no annotation used. A rigid fit turns the scans 0.3 to 6.5 degrees on the test pairs, a change of posture and breathing, so only the translation is kept.
Baseline. With no registration the 10 test pairs start at a landmark error of 12.6 mm (5.3 to 21.8; 7.4 mm after the rigid landmark fit used before), a label Dice of 0.43 (0.27 to 0.59) and an HD95 of 15 mm.
Labels. TotalSegmentator (task total), run by us on each image in Hounsfield units, shown beside the landmarks, which stay the ranked truth. A method that registers on that network's features is scored against its own view of the anatomy.
Headers. The files' headers would mirror the patient left to right and the values carry an offset of 1024: prepare.py writes the true direction and Hounsfield units. On every image built the liver lies on the patient's right of the spleen.
Overlap. 25 of the 30 pairs are also in Liver-DIR-QA, with landmarks inside the liver only; LiverDIRQA holds none of them.

		Accuracy					Smoothness		Cost
#	Method	Dice	Change	HD95 mm	TRE mm	rTRE	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Machine	Images
11–3	DemonsCPUapp	0.712 ±0.052	+0.281	9.92 ±2.55	7.08 ±2.75	0.0101 ±0.0040	0.088	0.37	44	6.9		Laptop
11–3	greedyCPUapp	0.744 ±0.055	+0.313	10.28 ±2.96	7.34 ±3.40	0.0106 ±0.0051	0.055	0.00	111	9.7		Workstation
31–3	NiftyRegCPUapp	0.775 ±0.050	+0.344	14.24 ±8.36	5.28 ±2.57	0.0076 ±0.0038	0.479	0.59	174	4.6		Workstation
4	anatomix + ConvexAdamGPUapp5 failed	0.575 ±0.119	+0.143	12.99 ±3.07	10.16 ±3.52	0.0142 ±0.0048	0.042	0.24	724	11.2	21.7	Laptop
	No registration	0.431 ±0.076		15.06 ±3.20	12.60 ±3.78	0.0179 ±0.0055	0.000	0.00

The order averages each method's ranks on Dice, HD95 mm, TRE mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it.

1 result not ranked: an older run, pairs still missing, or a failure on every pair

		Accuracy					Smoothness		Cost
#	Method	Dice	Change	HD95 mm	TRE mm	rTRE	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Machine	Images
	ConvexAdam (MIND)GPUapp9/10 pairs	0.765 ±0.048	+0.333	8.94 ±2.90	4.67 ±2.01	0.0069 ±0.0030	0.097	0.48	31	4.7	11.4	Laptop

Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Pair by pair

Measure

Dice	1	2	3	4	5	6	7	8	9	10
Demons	0.705	0.647	0.758	0.615	0.747	0.722	0.720	0.762	0.842	0.606
greedy	0.722	0.717	0.779	0.722	0.736	0.747	0.757	0.834	0.855	0.574
NiftyReg	0.718	0.663	0.786	0.778	0.811	0.759	0.826	0.861	0.866	0.685
anatomix + ConvexAdam	failed	0.641	0.770	0.709	0.720	0.707	failed	failed	failed	failed
ConvexAdam (MIND)	0.721	0.712	0.782	0.763	0.778	0.769		0.833	0.865	0.658
No registration	0.540	0.274	0.568	0.445	0.397	0.430	0.378	0.389	0.590	0.303

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

The first test caseClick to look at every case.

Test pairs: 10 test pairs drawn with seed 2026 by source, public collections or Barnes Jewish Hospital, among the 20 outside development, and 10 development pairs. The few pairs the authors flag as not refined are left out.
Truth: contours by a program landmarks by a program, verified by people TotalSegmentator (task total) run by us on each image, in Hounsfield units the dataset's vessel bifurcation pairs over the abdomen, about 63 a pair
Source: zenodo.org/records/14362785
github.com/deshanyang/Abdominal-DIR-QA
arxiv.org/abs/2501.09162
Licence: CC BY 4.0 (upstream TCIA collections under CC BY 3.0)

10 pairs for tuning, never tested

case01, case02, case03, case20, case08, case11, case19, case27, case25, case16