fidoptHNTS-MRG 2024
Two T2-weighted neck MRs of one patient with head and neck cancer, before radiotherapy (fixed) and two to four weeks into it (moving) (HNTS-MRG 2024 training set).
Read before comparing (9)
- Tumour. The tumour shrinks between the scans: on the first 13 cases built the mid-RT primary keeps 0 to 0.95 of its pre-RT volume (median 0.33), the nodes 0.11 to 2.2 (median 0.66). No registration can make the contours overlap, so their Dice is shown (dice_tumour), never ranked; the board ranks the distance between the contours' centres.
- Floor. Uneven regression moves a centre by an amount nobody measured. The provider's own elastix B-spline registration leaves the centres 0.7 to 11 mm apart on the first 13 cases built: part of a pair's error may be no registration error, and no method can be expected to reach zero.
- Points. One or two a case, the primary tumour and the nodes, all the nodes of a side as one. A structure on one image gives no pair: test cases 105, 177 and 34 are scored on the nodes alone, 80 on the primary tumour alone. The complete responses at mid-RT (21, 25, 29 and 42) give no pair and keep the labels only.
- Truth. The contours are the STAPLE consensus of 3 to 4 physicians, checked by a senior radiation oncologist. They agree less at mid-RT (median Dice between two of them 0.56 for the primary tumour, 0.81 for the nodes) than before treatment (0.75 and 0.85), and could use their registered pre-RT contours there: a mid-RT contour may carry the shape a registration gave it.
- Labels. TotalSegmentator's total_mr labels are shown, not ranked: a method that registers on that network's features is scored against its own view. They hold the cervical vertebrae as one label, the discs, the cord, the bottom of the brain, and fragments at the edges, often under 1 cm³, that pull the mean Dice down.
- Alignment. The two sessions, sometimes on two scanners and grids, keep their native voxels, their frames up to 25 cm apart. The moving image's header carries one translation from elastix mutual information inside each head, the best of several starts, no annotation used. A rigid fit, turning the sessions 0.3 to 3.6 degrees (12.8 on 69), leaves the centres about where the translation does: 4.0 mm against 4.2.
- Baseline. With no registration the 15 test pairs start at a centre error of 4.2 mm (1.0 to 8.7), a label Dice of 0.51 (0.27 to 0.68), an HD95 of 6.2 mm and a tumour Dice of 0.59 (0.19 to 0.80). The neck is not bent the same way at both sessions.
- Images. The fixed image is the pre-RT T2 as acquired, not the provider's copy registered to mid-RT. T2-weighted, with or without fat suppression, without contrast, at 1.5 T or 3 T, the patient in a thermoplastic mask. Each scan was cropped by hand from the clavicles to the nasal septum, so the two fields of view end at different levels.
- Outside numbers. None: the challenge ranks segmentation, not registration.
| Accuracy | Smoothness | Cost | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | TRE mm | Change | Dice * | HD95 mm * | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 11–2 | ConvexAdam (MIND)GPUapp | 3.52 ±1.36 | −0.70 | 0.619 ±0.047 | 4.87 ±1.12 | 0.0083 ±0.0036 | 0.067 | 0.34 | 19 | 3.2 | 5.7 | |
| 21–2 | DemonsCPUapp | 3.93 ±1.15 | −0.28 | 0.552 ±0.066 | 5.51 ±1.00 | 0.0093 ±0.0031 | 0.076 | 0.24 | 19 | 3.0 | ||
| No registration | 4.21 ±1.35 | 0.507 ±0.059 | 6.24 ±1.37 | 0.0097 ±0.0031 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on TRE mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 15 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on TRE mm (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
* shown, not ranked: on this dataset it is checked against the output of a program.
Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- 15 test pairs drawn with seed 2026 among the 137 outside development, and 13 development pairs. The pool is the challenge's 150 public training patients of MD Anderson Cancer Center, mostly oropharyngeal cancer or cancer of unknown primary; its 50 test patients are not public.
- Truth
- landmarks by experts contours by a program tumour by experts TotalSegmentator (task total_mr) run by us on each image the centre of the primary tumour (GTVp) and the centre of the involved nodes (GTVn) on each image, from the dataset's consensus contours, one pair for each structure both images hold: 1 or 2 a pair
- Source
- zenodo.org/records/11199559
hntsmrg24.grand-challenge.org
arxiv.org/abs/2411.18585
doi.org/10.1007/978-3-031-83274-1_1 - Licence
- CC BY-NC 4.0
13 pairs for tuning, never tested
2, 3, 4, 115, 169, 157, 53, 104, 175, 159, 163, 196, 42