QIN-PROSTATE-Repeatability

Prostate T2 MR of one patient at two exams within two weeks, on the same scanner with an endorectal coil (QIN-PROSTATE-Repeatability, test and retest).

10 test pairsprostatesame patientfixed MR, moving MRaligned once by the benchmark

Look at the images

Read before comparing (7)

Truth. One radiologist contoured each of the 30 exams on its own, shuffled so that the reader knew neither the patient nor which exam came first. The two contours of a structure differ by the reader's own variation as well as by the anatomy: no registration reaches a Dice of 1.
Labels. The contours overlap, so the map paints the peripheral zone over the gland: label 1 is the gland outside the peripheral zone, label 2 the peripheral zone. Lesion voxels keep the label the gland and zone contours give them. The normal region drawn on some exams is left out.
Lesion. A suspected lesion is drawn in both exams of 7 of the 10 test pairs, small (20 to 813 mm³) on 3.5 mm slices. In PCAMPMRI-00004 and -00012 its two contours lie 8 and 14 mm apart, the reader's variation or two foci. Its Dice is noisy: shown (dice_lesion), never ranked.
Alignment. Each exam is in its session's scanner frame, up to 20 cm apart, the slice normals 0.2 to 10.7 degrees apart on the test pairs. The moving header and its truth coordinates carry one elastix translation from the intensities, no contour used; neither side is resampled. Rotation and the endorectal balloon's deformation remain for the methods.
Baseline. With no registration the ten test pairs start at a mean label Dice of 0.64 (0.38 to 0.82); the lesion Dice is 0.30 (0.00 to 0.75) on the seven pairs where it is defined.
Images. Native oblique grids of 0.2734 to 0.3906 mm in plane and 3.0 to 3.5 mm between slices. Patients 1 to 7 on a GE Signa HDxt, 8 to 15 on a GE Discovery MR750w, at 3 T with an endorectal coil in an air-filled balloon, both exams of a patient on one scanner.
Label fix. On the first exam of PCAMPMRI-00005 the reader painted the gland once more seven empty slices above its top: prepare.py clears a slice of the label map that lies apart from the gland.

		Accuracy			Smoothness		Cost
#	Method	Dice	Change	HD95 mm	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Images
1	deedsCPUapp	0.772 ±0.030	+0.128	4.25 ±1.03	0.066	0.16	25	1.4
22–4	greedyCPUapp	0.703 ±0.066	+0.060	5.60 ±1.30	0.061	0.00	22	1.6
32–4	ConvexAdam (MIND)GPUapp	0.688 ±0.083	+0.045	5.21 ±1.35	0.082	0.26	7	2.5	2.3
42–4	NiftyRegCPUapp	0.626 ±0.114	−0.017	10.32 ±4.00	0.365	8.69	46	0.8
5	DemonsCPUapp	0.519 ±0.085	−0.125	9.81 ±2.30	1.180	10.76	7	1.2
	No registration	0.643 ±0.104		5.71 ±2.02	0.000	0.00

The order averages each method's ranks on Dice, HD95 mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.

Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Pair by pair

Measure

Dice	1	2	3	4	5	6	7	8	9	10
deeds	0.737	0.820	0.775	0.773	0.725	0.797	0.790	0.845	0.734	0.721
greedy	0.556	0.792	0.541	0.747	0.726	0.738	0.754	0.815	0.667	0.698
ConvexAdam (MIND)	0.534	0.807	0.457	0.751	0.700	0.770	0.720	0.811	0.646	0.690
NiftyReg	0.395	0.731	0.591	0.667	0.701	0.734	0.656	0.802	0.294	0.687
Demons	0.347	0.523	0.447	0.554	0.665	0.714	0.543	0.588	0.399	0.406
No registration	0.376	0.682	0.457	0.703	0.716	0.808	0.686	0.820	0.528	0.656

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

The first test case of QIN-PROSTATE-Repeatability

The first test caseClick to look at every case.

Test pairs: 10 test pairs drawn with seed 2026 and 5 development pairs: all 15 patients of the collection, one pair each.
Truth: contours by experts lesion by experts the gland outside its peripheral zone, and the peripheral zone, contoured on each exam's T2 by one radiologist
Source: www.cancerimagingarchive.net/collection/qin-prostate-repeatability
doi.org/10.7937/K9/TCIA.2018.MR1CKGND
doi.org/10.1038/sdata.2018.281
doi.org/10.1097/RLI.0000000000000382
Licence: CC BY 4.0

5 pairs for tuning, never tested

PCAMPMRI-00001, PCAMPMRI-00002, PCAMPMRI-00003, PCAMPMRI-00005, PCAMPMRI-00008