Pelvic-Reference-Data

Treatment CBCT (fixed) and planning CT (moving) of the pelvis of one radiotherapy patient, 4 to 20 days apart (Pelvic-Reference-Data).

10 test pairspelvissame patientfixed CBCT, moving CTaligned once by the benchmark

Look at the images

Read before comparing (7)

Truth. Points on bone that experts picked in pairs on the CBCT and the CT with the authors' MATLAB tool, to validate rigid registration. They say nothing of the bladder, rectum or prostate, which move between the two days: the board ranks how well a method keeps the bones in place.
Slices. In all but one of the 58 patients, the two points of every pair lie a fixed number of slices apart, 3 mm slices in both images: along the body the landmarks give one shift a case, to a slice, and nothing of a deformation.
Points. The files give the CT points on a grid centred on the isocentre; prepare.py puts them in the CT's frame through the dataset's isocentre spreadsheet. They then fall on bone in both images, and moving them by a millimetre or a slice fits the bone no better.
Labels. TotalSegmentator's labels on both images are shown, not ranked. The CBCT holds 26.4 cm of the body, the CT 37.5 to 48 cm, so the scored labels are the pelvis's: the bladder, colon and small bowel on every case, the prostate on five of the ten test pairs, and the bones, iliac vessels and muscles where the network finds them.
CBCT. TotalSegmentator, trained on CT, is weaker on the CBCT, whose values are not Hounsfield units. On the four CBCTs that read lowest (Pelvic-Ref-006, 011, 014 and 057) it finds the bladder, the bowel and little else, and on 029 and 037 the bones in part. Where it works, the hip bones and femurs overlap 0.78 to 0.92 before any registration: the labels cannot rank the last millimetres.
Alignment. The CBCT comes in the treatment machine's frame, centred on its isocentre, the CT in the scanner's. The moving headers carry one elastix translation on mutual information inside each body, no landmark used; both images keep their voxels.
Baseline. With no registration the 10 test pairs leave a landmark error of 2.7 mm (1.3 to 4.2), 5.1 mm at the 95th percentile, and a label Dice of 0.45 (0.06 to 0.68): the board measures the last millimetres.

		Accuracy					Smoothness		Cost
#	Method	TRE mm	Change	Dice *	HD95 mm *	rTRE	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Images
11–2	greedyCPUapp	2.21 ±0.51	−0.51	0.475 ±0.163	41.36 ±20.72	0.0029 ±0.0007	0.047	0.00	60	4.4
21–2	ConvexAdam (MIND)GPUapp	2.19 ±0.54	−0.53	0.466 ±0.163	42.21 ±20.86	0.0028 ±0.0007	0.140	0.67	23	3.3	6.1
33–4	NiftyRegCPUapp	4.85 ±3.46	+2.13	0.406 ±0.155	44.60 ±22.68	0.0063 ±0.0045	0.127	0.50	97	2.4
43–4	deedsCPUapp	2.40 ±0.59	−0.32	0.480 ±0.161	41.73 ±20.45	0.0031 ±0.0008	0.186	0.00	74	4.0
	No registration	2.72 ±0.67		0.445 ±0.151	42.10 ±20.62	0.0035 ±0.0009	0.000	0.00

The order averages each method's ranks on TRE mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on TRE mm (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.

* shown, not ranked: on this dataset it is checked against the output of a program.

Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Pair by pair

Measure

Dice	1	2	3	4	5	6	7	8	9	10
greedy	0.381	0.701	0.065	0.423	0.745	0.295	0.688	0.385	0.724	0.340
ConvexAdam (MIND)	0.353	0.705	0.059	0.417	0.731	0.290	0.675	0.389	0.713	0.330
NiftyReg	0.317	0.669	0.044	0.379	0.638	0.243	0.580	0.226	0.656	0.308
deeds	0.381	0.707	0.072	0.426	0.745	0.309	0.686	0.395	0.729	0.353
No registration	0.338	0.663	0.055	0.387	0.678	0.323	0.662	0.376	0.661	0.308

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

The first test case of Pelvic-Reference-Data

The first test caseClick to look at every case.

Test pairs: 10 test pairs drawn with seed 2026 among the 46 outside development, and 10 development pairs. Two patients are left out: on 028 the landmarks fall on the bone of neither image, on 002 the CT points sit one slice and 2 mm off.
Truth: contours by a program landmarks by experts TotalSegmentator (task total) run by us on each image the dataset's bony landmark pairs, picked by experts on both images, 96 to 211 a pair
Source: www.cancerimagingarchive.net/collection/pelvic-reference-data
doi.org/10.7937/TCIA.2019.WOSKQ5OO
doi.org/10.31579/2640-1053/076
doi.org/10.1002/acm2.12965
Licence: CC BY 3.0

10 pairs for tuning, never tested

Pelvic-Ref-001, Pelvic-Ref-003, Pelvic-Ref-004, Pelvic-Ref-005, Pelvic-Ref-024, Pelvic-Ref-049, Pelvic-Ref-035, Pelvic-Ref-033, Pelvic-Ref-054, Pelvic-Ref-013