AMOS MR/MR

Abdominal MR scans of two patients, paired by us from AMOS 2022, with 12 or 13 organs labelled on both.

10 test pairsabdomeninter-patientfixed MR, moving MRaligned once by the benchmark

Read before comparing (5)

Alignment. The scans keep their native voxels, with unrelated origins. The moving image's header carries one translation from elastix mutual information inside each body, the best of several starts, no annotation used. A rigid fit turns the bodies 1 to 16 degrees on the test pairs without superposing the organs better (pair_0538_0586), so only the translation is kept.
Baseline. With no registration the 10 test pairs start at a Dice of 0.27 (0.13 to 0.38), against 0.08 as delivered and 0.33 after a fit on the organ centres, and an HD95 of 34 mm (19 to 42).
Scanners. Most scans come from a Siemens Prisma; by the archive's metadata at least 7 of the 28 pairs join two scanners (Prisma, Philips Ingenia, GE Signa HDe) and 6 two hospitals. The intensities share no scale: the paper's table 9 gives median intensities from 845 to 57422 by scanner.
Truth. A network's pre-labels, corrected by five junior radiologists and checked by three senior ones, one annotator a scan. The bladder and the prostate or uterus are labelled on one MR scan of the 56 only: the fields of view stop above the pelvis.
Outside numbers. None: the pairing is ours.

		Accuracy			Smoothness		Cost
#	Method	Dice	Change	HD95 mm	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Images
1	ConvexAdam (MIND)GPUapp	0.426 ±0.089	+0.161	28.82 ±5.09	0.122	1.08	14	2.9	4.4
22–3	DemonsCPUapp	0.358 ±0.086	+0.093	30.72 ±5.34	0.112	0.68	14	2.3
32–3	multiGradICONGPUapp	0.410 ±0.063	+0.145	32.24 ±7.95	0.161	0.22	93	3.3	12.6
	No registration	0.265 ±0.065		33.72 ±5.14	0.000	0.00

The order averages each method's ranks on Dice, HD95 mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.

Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Pair by pair

Measure

Dice	1	2	3	4	5	6	7	8	9	10
ConvexAdam (MIND)	0.492	0.300	0.416	0.168	0.362	0.493	0.469	0.491	0.615	0.456
Demons	0.243	0.314	0.379	0.151	0.289	0.488	0.423	0.482	0.518	0.295
multiGradICON	0.507	0.268	0.278	0.466	0.420	0.423	0.348	0.531	0.421	0.437
No registration	0.179	0.194	0.244	0.131	0.219	0.369	0.335	0.384	0.369	0.228

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

The first test caseClick to look at every case.

Test pairs: 10 test pairs drawn with seed 2026 among the 18 pairs outside development, and 10 development pairs. The pool is the 60 MR scans with public labels, less four that the archive's metadata shows to be repeat scans of three patients; each patient enters one pair only.
Truth: contours by experts the dataset's annotations of 15 organs, of which the MR scans hold 13 (12 or 13 in both scans of a pair)
Source: zenodo.org/records/7262581
arxiv.org/abs/2206.08023
Licence: CC BY 4.0

10 pairs for tuning, never tested

pair_0581_0570, pair_0595_0548, pair_0592_0563, pair_0557_0553, pair_0589_0550, pair_0585_0559, pair_0551_0544, pair_0549_0554, pair_0573_0588, pair_0555_0517