fidoptLung-DIR-QA
Two chest CTs of one patient, with vessel bifurcations matched between them (Lung-DIR-QA).
Read before comparing (5)
- Pairs. The dataset's 30 pairs are 17 EMPIRE10 scan pairs (cases 1 to 17), 8 TCGA-LUAD and TCGA-LUSC inspiration and expiration pairs (18 to 25) and 5 clinical pairs (26 to 30).
- Truth. The landmarks are vessel bifurcations found and matched by a semi-automatic procedure, then checked by the dataset's authors: 592 to 1658 a test pair, 1276 on average. The board ranks on them.
- Labels. TotalSegmentator's on each image, in Hounsfield units; the masks are each image's lungs, its five lobes together. A method that registers on that network's features is scored against its own view of the anatomy, so the label overlap is shown, not ranked.
- Alignment. The two scans come on different grids with their origin at zero. The moving scan's, labels' and landmarks' headers carry one elastix translation on mutual information inside each body, no landmark or label used; both scans keep their voxels and grids.
- Baseline. With no registration the 10 test pairs score a landmark error of 13.0 mm (3.4 to 24.2), 25.0 mm at the 95th percentile (28 mm as delivered), and a label Dice of 0.26 (0.12 to 0.54).
| Accuracy | Smoothness | Cost | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | TRE mm | Change | Dice * | HD95 mm * | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 1 | corrFieldGPUapp | 0.84 ±0.27 | −12.14 | 0.339 ±0.069 | 63.41 ±15.77 | 0.0016 ±0.0006 | 0.043 | 0.03 | 54 | 3.4 | 13.6 | |
| 2 | DemonsCPUapp | 9.79 ±4.73 | −3.20 | 0.355 ±0.085 | 65.02 ±15.33 | 0.0190 ±0.0111 | 0.061 | 0.01 | 64 | 10.5 | ||
| No registration | 12.99 ±3.92 | 0.256 ±0.096 | 66.68 ±15.60 | 0.0245 ±0.0095 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on TRE mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on TRE mm (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
* shown, not ranked: on this dataset it is checked against the output of a program.
1 result not ranked: an older run, pairs still missing, or a failure on every pair
| Accuracy | Smoothness | Cost | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | TRE mm | Change | Dice * | HD95 mm * | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Machine | Images |
| ConvexAdam (MIND)GPUapp5/10 pairs | 4.46 ±8.37 | −8.53 | 0.369 ±0.128 | 71.70 ±23.33 | 0.0100 ±0.0195 | 0.089 | 0.42 | 34 | 5.4 | 15.2 | Laptop | ||
Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- 10 test pairs drawn with seed 2026 among the 20 outside development, in proportion to their source, and 10 development pairs.
- Truth
- contours by a program landmarks by a program, verified by people TotalSegmentator (task total) run by us on each image, in Hounsfield units 592 to 1658 vessel-bifurcation landmark pairs a test case, found by a semi-automatic pipeline and checked by the dataset's authors
- Licence
- CC BY 4.0, but 17 of its 30 pairs are EMPIRE10 scans
10 pairs for tuning, never tested
case01, case02, case03, case10, case08, case13, case24, case23, case27, case12