Serial sections, mouse prostate and liver (Kartasalo 2018)
Two consecutive sections of one paraffin block, a mouse prostate or liver, stained with haematoxylin and eosin and scanned at 0.46 µm a pixel, with four point pairs placed by people (Kartasalo et al. 2018).
Read before comparing (11)
- Blocks. Two blocks, each one organ of a male FVB/N mouse: a prostate of 260 sections and a liver of 47, through which a laser drilled four holes as landmarks. Consecutive prostate images lie 5 µm apart, or 10 µm across a section kept aside, which the archive does not mark.
- Independence. Pairs of one block share the animal, fixation, stain and scanner, so the ten test pairs are not ten independent samples. Development pairs come from the same blocks, on other sections at least 20 µm (liver) and 125 µm (prostate) from a test pair's; the liver's show the same four holes.
- Damage. Cutting and mounting compressed, stretched, folded or tore the sections, worst on the liver: liver_001_002's moving section is torn in two, and liver_026_027's fixed section is cracked and folded. A fold runs through prostate_027_028's fixed section.
- Images. 8-bit colour as delivered, at the paper's pixel size, so distances are in µm. The provider set everything outside the tissue to white, 255; no mask is given.
- Truth. Observer 1's points, the set the paper reports: on a prostate pair four nuclei cut by the section plane, of the observer's choice; on a liver pair the centres of the four holes, 0.2 to 0.35 mm across. prepare.py takes MATLAB's convention, the first pixel's centre at 1; were it 0, every point would move by 0.46 µm.
- Truth error. On the liver the two observers' hole displacements differ by 9.3 µm (median), the truth's own error; where a hole meets a tear they part by up to 193 µm, reaching test pair liver_031_032 (70 and 182 µm). On the prostate repeated placements agree within a few micrometres; nothing measures a nucleus matched to the wrong one.
- Alignment. As delivered a test pair's points lie 160 to 1653 µm apart, turned by up to 27 degrees. The moving section's header and points carry an elastix rigid start on mutual information over the mean of the colours inside each section, no point used. With it the 10 test pairs start at a TRE of 67.8 µm (36.6 to 138.2).
- Earlier start. The least-squares rigid fit on observer 1's points used before left 66.3 µm (32.0 to 141.3) but handed the truth to the start. Against it ANTs' SyN, at the level it is given, ends at 49.5 µm (1.6 to 178.4) without folds, closer on nine pairs; on prostate_027_028 a fold needs a jump no smooth field makes.
- Level. Sections, up to 19457 x 21249 pixels, are stored as multiscale OME-Zarr; a method that reads none gets level 3, 3.68 µm a pixel, within the default 2^24 pixels. Points are scored at full resolution.
- Ranked apart. Four point pairs a pair miss the main board's median of ten, and the ten test pairs come from two blocks.
- Outside numbers. The paper ranks whole-stack reconstructions, with methods tuned on the prostate: no published number is scored as here.
| Accuracy | Smoothness | Cost | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | TRE µm | Change | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 11–6 | vismatch matchers · ALIKED + LightGlueGPU | 38.11 ±31.26 | −29.70 | 0.0030 ±0.0025 | 0.008 | 0.00 | 5 | 2.9 | 1.9 | |
| 11–6 | vismatch matchers · SIFT + LightGlueGPU | 35.51 ±20.20 | −32.29 | 0.0028 ±0.0015 | 0.013 | 0.00 | 4 | 2.7 | 0.5 | |
| 31–6 | vismatch matchers · LoFTRGPU | 42.40 ±31.57 | −25.40 | 0.0033 ±0.0025 | 0.006 | 0.00 | 5 | 2.6 | 19.7 | |
| 31–6 | vismatch matchers · RoMaGPU | 40.67 ±30.16 | −27.13 | 0.0032 ±0.0024 | 0.008 | 0.00 | 9 | 3.8 | 6.9 | |
| 31–6 | vismatch matchers · XFeatGPU | 33.53 ±20.46 | −34.27 | 0.0026 ±0.0016 | 0.020 | 0.00 | 4 | 2.8 | 0.5 | |
| 61–6 | vismatch matchers · DISK + LightGlueGPU | 39.34 ±31.43 | −28.46 | 0.0031 ±0.0025 | 0.014 | 0.00 | 5 | 2.8 | 3.2 | |
| No registration | 67.80 ±22.89 | 0.0053 ±0.0018 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on TRE µm, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on TRE µm (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- The earlier section of a pair is fixed. 10 test pairs drawn with seed 2026 from a pool of 20, five a block; the ten others are the development pairs. The pool is ten pairs a block, spread evenly through it and sharing no section: every 26th pair of the prostate, every 5th of the liver.
- Truth
- landmarks by experts observer 1's four point pairs a pair, placed by hand on both sections: four nuclei cut through by the section plane on a prostate pair, chosen by the observer; the centres of the four laser holes, marked with an ellipse tool, on a liver pair
- Source
- urn.fi/urn:nbn:fi:csc-kata20170705131652639702
etsin.fairdata.fi/dataset/c76335fa-cdcf-4ddc-ab1c-1882bad82861
metax.fairdata.fi/rest/v2/datasets?preferred_identifier=urn:nbn:fi:csc-kata20170705131652639702
doi.org/10.1093/bioinformatics/bty210
academic.oup.com/bioinformatics/article/34/17/3013/4978049#supplementary-data
github.com/BioimageInformaticsTampere/RegBenchmark - Licence
- CC BY 4.0
10 pairs for tuning, never tested
prostate_209_210, liver_036_037, prostate_105_106, liver_006_007, prostate_235_236, liver_016_017, prostate_001_002, liver_041_042, prostate_079_080, liver_011_012