fidoptSpine-generic multi-subject
Sagittal 3D T2w MR of the cervical spine of two adults scanned on one scanner with the spine-generic protocol, paired by us (spine-generic multi-subject, release r20250310).
Read before comparing (9)
- Pairing. Ours: two subjects of one site, so one scanner and one protocol, paired in a seeded hash order; 127 pairs from the 267 subjects, one left out at each of the 13 sites with an odd count. The beijingVerio site and three subjects that exclude.yml leaves out of the provider's measurements stay in the pool; none is drawn.
- Subjects. Adults: 203 healthy, 62 with an asymptomatic cord compression and 2 with degenerative cervical myelopathy. One test pair holds a compressed cord; development pair sub-tokyoSkyra02_sub-tokyoSkyra01 joins a myelopathy and a compression.
- Landmarks. The posterior tip of each intervertebral disc on one sagittal slice, placed by one rater for the whole dataset, with no repeat, so their error is unknown. A pair is scored on the levels both subjects hold, 9 to 12 points from the top of C1 down to T1/T2 to T4/T5. They say which disc meets which, little of the rest of the neck.
- Cord. A network's mask (sct_deepseg_sc) corrected by hand, each image by one of six annotators; how much was redrawn is not recorded. The cord is a tube of 58 to 85 mm² in section, so a millimetre off costs much Dice. Later commits replace the healthy subjects' T2w masks with a newer network's uncorrected output; they are not used.
- Alignment. The scans come in unrelated scanner frames. The moving image's and truth headers carry one elastix translation from the intensities, no disc point or cord mask used; no voxel is resampled.
- Baseline. With no registration the twelve test pairs start at a mean disc error of 12.2487 mm (6.2266 to 22.5471), a cord Dice of 0.3298 (0.0152 to 0.6286) and an HD95 of 13.8127 mm (4.0792 to 22.7121). Hardest are sub-cmrra06_sub-cmrra05 (TRE 22.5471 mm) and sub-perform04_sub-perform06 (Dice 0.0152). Neck length, curve, posture and cord coverage are left to the methods.
- Coverage. The cord masks stop at the bottom at levels 1 to 40 mm apart between a test pair's images (40 mm on sub-perform04_sub-perform06). The longer mask's end overlaps nothing whatever the field: it caps the Dice below 1 and weighs on HD95.
- Images. Sagittal 3D at 0.8 mm from the posterior fossa to the upper thoracic spine, defaced by the provider. Values are as delivered and differ by site, up to 1.8 million on Sherbrooke's Philips scanner. No mask is given.
- Outside numbers. None: the pairing is ours.
| Accuracy | Smoothness | Cost | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | Dice | Change | HD95 mm | TRE mm | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 1 | ConvexAdam (MIND)GPUapp | 0.853 ±0.047 | +0.523 | 7.11 ±5.48 | 5.86 ±2.34 | 0.0160 ±0.0064 | 0.107 | 0.61 | 5 | 2.4 | 1.9 | |
| 1 | NiftyRegCPUapp | 0.853 ±0.048 | +0.524 | 6.73 ±4.96 | 8.99 ±2.71 | 0.0246 ±0.0074 | 0.183 | 1.27 | 30 | 0.8 | ||
| 33–4 | DemonsCPUapp | 0.632 ±0.112 | +0.302 | 9.74 ±4.70 | 9.20 ±2.88 | 0.0252 ±0.0079 | 0.067 | 0.00 | 6 | 1.0 | ||
| 43–4 | greedyCPUapp | 0.790 ±0.102 | +0.460 | 9.83 ±6.15 | 9.42 ±2.73 | 0.0258 ±0.0075 | 0.082 | 0.00 | 20 | 1.4 | ||
| No registration | 0.330 ±0.122 | 13.81 ±4.21 | 12.25 ±2.82 | 0.0335 ±0.0077 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on Dice, HD95 mm, TRE mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 12 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- Twelve test pairs drawn with seed 2026 from the 127 subject-disjoint within-site pairs, stratified by vendor (eight Siemens, two Philips, two GE), and eleven development pairs of other subjects, the last of the same seeded order, those the test draw rests on kept, each vendor in proportion (seven Siemens, two Philips, two GE). A scan appears in one pair only; SpineGeneric_Seg takes its images and split from this dataset.
- Truth
- contours by a program, verified by people landmarks by experts the spinal cord mask of each image (label 1): a network's output (sct_deepseg_sc) corrected by hand the posterior tip of each intervertebral disc, placed by hand, matched by level: 9 to 12 a pair
- Source
- github.com/spine-generic/data-multi-subject/tree/r20250310
doi.org/10.5281/zenodo.4299140
doi.org/10.1038/s41597-021-00941-8
doi.org/10.1038/s41596-021-00588-0
spine-generic.readthedocs.io/en/latest/analysis-pipeline.html
github.com/spine-generic/data-multi-subject/pull/178
spinalcordtoolbox.com/stable/overview/concepts/labeling-conventions.html - Licence
- CC BY 4.0
11 pairs for tuning, never tested
sub-tokyoSkyra02_sub-tokyoSkyra01, sub-sherbrooke01_sub-sherbrooke02, sub-tokyo750w06_sub-tokyo750w04, sub-tokyoSkyra06_sub-tokyoSkyra04, sub-hamburg06_sub-hamburg04, sub-ucdavis04_sub-ucdavis01, sub-cardiff01_sub-cardiff06, sub-cmrrb01_sub-cmrrb02, sub-geneva01_sub-geneva04, sub-nottwil05_sub-nottwil04, sub-beijingGE04_sub-beijingGE03