fidopt Interactive view
Brain MR, inter-subject

UltraCortex

Brain T1 MR at 9.4 T of two healthy adults, skull-stripped, 0.6 to 0.8 mm, with a manual segmentation of the cortex and white matter of each hemisphere (UltraCortex).

3 test pairsbraininter-patientfixed MR, moving MRaligned once by the benchmark
Look at the images
Read before comparing (7)
  • Labels. FreeSurfer's cortical ribbon: the cortex and white matter of each hemisphere. The deep grey nuclei count as white matter; the hippocampi, amygdalae, cerebellum and brainstem are background. The labels are large, so the Dice is high from the start and moves little with the folds; HD95 says more of the cortical surface.
  • Truth. FreeSurfer 7.1.1's ribbon (recon-all -hires), corrected voxel by voxel in ITK-SNAP by trained segmenters under neuroradiologists' review, then checked by two neuroradiologists. No agreement between raters is given. Near the ear canals the signal drops out in every subject and the cortex was drawn closed: there the labels are a guess.
  • Alignment. Each brain comes in its own scanner space. The start is an elastix translation on mutual information inside each brain, no label used, written in the moving image's and labels' headers. It beats the images as delivered (Dice 0.17 to 0.30) and with their centres together (0.36 to 0.46); elastix-rigid did no better. The difference in brain size is left to the methods.
  • Baseline. With no registration the three test pairs score a Dice of 0.50 (0.42 to 0.55) and an HD95 of 10.9 mm (5.3 to 20.2). sub-45_sub-73 keeps a pitch of the head.
  • Images. The dataset's skull-stripped T1, from SynthStrip, which was not made for 9.4 T, so the brain outline is approximate; it holds 99.9 % of the labelled voxels. No mask is given, and the gradient non-linearity is not corrected. The field is asked on the fixed grid, 352 x 352 x 256 or 288 voxels at 0.6 mm.
  • Sequences. 9 subjects had MP2RAGE and 3 MP-RAGE (sub-37, sub-45, sub-57), at 0.6 mm except sub-29 (0.8 mm). Two of the three test pairs join the two sequences, whose contrast and intensity scale differ: MP2RAGE up to about 10,000, MP-RAGE under 1,000.
  • Power. Three test pairs separate only large differences between methods.
AccuracySmoothnessCost
#MethodDiceChangeHD95 mmSDlogJFolds %Time sRAM GBGPU GBImages
11–4greedyCPUapp0.715 ±0.249+0.2188.04 ±23.050.0660.00756.2
21–4ConvexAdam (MIND)GPUapp0.662 ±0.156+0.1658.11 ±24.080.0600.07173.88.5
31–4NiftyRegCPUapp0.636 ±0.516+0.1385.72 ±10.440.2052.521503.0
41–4DemonsCPUapp0.555 ±0.211+0.0589.46 ±18.120.0510.00364.4
No registration0.498 ±0.16110.86 ±20.250.0000.00

The order averages each method's ranks on Dice, HD95 mm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 3 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on Dice (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.

Dice: how much the organ contours overlap, 1 is perfect. HD95: distance between the contours' surfaces. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Pair by pair

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

The first test case of UltraCortex
The first test caseClick to look at every case.
Test pairs
Only 12 of the dataset's 86 images, one a subject on its first session, have a manual segmentation. A subject enters one pair only, so they make six pairs: 3 for development and the 3 others as test pairs. The other 74 images, repeat sessions included, stay out.
Truth
contours by experts the dataset's manual segmentation of the cerebral cortex and the cerebral white matter of each hemisphere (4 labels), corrected voxel by voxel from FreeSurfer's and validated by two neuroradiologists
Licence
CC0
3 pairs for tuning, never tested

sub-29_sub-44, sub-20_sub-70, sub-3_sub-57