fidoptPelvic landmarks CT/CT (LUMC)
Two planning CTs of one cervical cancer patient's pelvis, full bladder (fixed) and empty bladder (moving), with corresponding landmarks placed by people on both (LUMC interobserver study of landmark placement).
Read before comparing (11)
- Overlap. PelvisLandmarksMRCT comes from the same study, and each image is in one case only. P05's full-bladder CT is also the CT of its MRI-CT pair; it stays here, where 13 observers placed its points against 6 there.
- MRI-MRI. The study's MRI-MRI pairs are left out: their fixed image is the MRI of the MRI-CT pairs, and no registration leaves them 3.9 to 4.3 mm apart, only 2.4 to 2.7 times the truth's own error.
- Truth. Each observer saw the full-bladder CT with its 23 reference points and placed each one's match on the empty-bladder CT: 13 observers, 11 radiation therapy technologists and two PhD students. The moving point is the geometric median of their placements; the fixed point is the reference placement, one person's checked by a second.
- Truth error. Placements lie a median 1.4 mm from their geometric median on bone and 3.0 mm on soft tissue, some up to 49 mm away. Resampling the observers 1,000 times moves the geometric median a mean 1.2 mm (1.0 to 1.6 a case): the truth's own error, all on the moving side.
- Exclusions. The study's exclusions apply: three observers who misread the top of the uterus, a placement with left and right on one spot, and landmarks just outside one scan. On P02, P03 and P05 the empty-bladder CT stops below the third lumbar vertebra, the kidneys' lower tips and the aortic bifurcation, which leave the pair. 17 to 23 pairs a case, 5 to 7 on bone.
- Alignment. The CTs come in their own scanner frames. The moving image's, labels' and landmarks' headers carry one elastix translation on mutual information inside each body, no landmark or label used.
- Baseline. With no registration the landmarks lie 8.0 mm apart (2.9 to 14.7), mostly on soft tissue around the bladder, which held 178 to 707 mL full and 36 to 109 mL empty: 6.7 times the truth's own error. P01 has the least room, 2.9 mm against 1.0 mm.
- Labels. TotalSegmentator's, shown and not ranked, starting at a Dice of 0.62 (0.42 to 0.77), HD95 31.5 mm. The RTSTRUCTs also hold clinical contours by unnamed authors, left out.
- Scans. Philips Brilliance Big Bore CT, 120 kV, 3 mm slices, for radiotherapy planning at the LUMC in 2019 and 2020. Four patients' two CTs are of one day; P04's are dated 2020-06-16 (full, with iodine contrast) and 2020-07-15 (empty).
- Points. prepare.py reads them from each scan's RTSTRUCT, in the scan's own frame; they agree with the study's tables within 0.02 mm.
- Ranked apart. Five test pairs miss the main board's ten. People placed 17 to 23 points a pair on both images, and the start is 6.7 times the truth's own error.
| Accuracy | Smoothness | Cost | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Change | HD95 mm | TRE mm | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 11–5 | deedsCPUapp | 0.758 ±0.092 | +0.143 | 21.79 ±14.37 | 4.19 ±2.49 | 0.0049 ±0.0026 | 0.090 | 1.27 | 129 | 7.0 | ||
| 21–5 | ConvexAdam (MIND)GPUapp | 0.751 ±0.096 | +0.136 | 19.93 ±12.11 | 5.39 ±2.94 | 0.0062 ±0.0030 | 0.138 | 1.61 | 33 | 4.3 | 9.9 | |
| 31–5 | greedyCPUapp | 0.732 ±0.129 | +0.116 | 27.56 ±18.01 | 5.66 ±5.15 | 0.0065 ±0.0055 | 0.034 | 0.00 | 101 | 7.2 | ||
| 31–5 | NiftyRegCPUapp | 0.731 ±0.105 | +0.116 | 29.91 ±18.90 | 5.24 ±3.19 | 0.0060 ±0.0033 | 0.026 | 0.00 | 189 | 3.4 | ||
| 51–5 | DemonsCPUapp | 0.728 ±0.117 | +0.112 | 27.85 ±17.41 | 5.83 ±3.30 | 0.0068 ±0.0034 | 0.087 | 1.56 | 31 | 5.2 | ||
| No registration | 0.615 ±0.167 | 31.49 ±19.11 | 7.99 ±5.53 | 0.0092 ±0.0058 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on Dice, HD95 mm, TRE mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 5 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- All five pairs of the study's CT group are test pairs, one a patient; none is left for development.
- Truth
- landmarks by experts contours by a program TotalSegmentator (task total) run by us on each image (datasets/segment.py), its prostate taken out (prepare.py --labels): these women have none; a secondary truth, graded automatic: shown, not ranked, the landmarks rank 23 anatomical landmarks defined for pelvic CT, 7 of them on bone: on the full-bladder CT the reference point, placed by an experienced radiation therapy technologist and verified by a gynaecological radiation oncologist; on the empty-bladder CT the geometric median of the 10 to 13 observers' placements, the study's own consensus. 17 to 23 a pair
- Source
- doi.org/10.4121/b49aadc1-e435-4d67-a7d2-460d9ed02558.v1
data.4tu.nl/datasets/b49aadc1-e435-4d67-a7d2-460d9ed02558/1
doi.org/10.1016/j.radonc.2026.111430
ir.cwi.nl/pub/36285 - Licence
- CC BY-NC-ND 4.0