fidoptFIRE
Two colour photographs of one eye's retina, from two viewpoints in one examination or at two examinations with changes between: the reference (fixed) and test (moving) image of a Fundus Image Registration pair.
Read before comparing (9)
- Images. Photographs of 39 patients of the Hypertension Unit of the Papageorgiou Hospital, Thessaloniki, taken with a Nidek AFC-210 camera: 2912 x 2912 pixels, a 45 degree field. They stay as the camera's JPEGs, in colour, one vector of red, green and blue a pixel. The dataset's mask of the circular field of view is each image's mask.
- Units. The camera gives no pixel size, so every distance is in pixels and prepare.py sets every spacing to 1. The rTRE divides by the image's diagonal, 4118 pixels.
- Categories. S: 71 pairs that overlap by more than 75% with no visible change. P: 49 pairs that overlap less, as a mosaic needs. A: 14 pairs that overlap widely, taken at two examinations where retinopathy progressed or receded. Any pair may show an eye deformed by myopia or hypermetropia.
- Eyes. The 134 pairs use 124 distinct images, one image in up to eight pairs, so a case is the first pair, in the seeded order, of a group linked by shared images: 42 candidates, 24 P, 13 S and 5 A. Two groups may still be one eye, or one patient's two eyes: the dataset names no patient.
- Truth. Ten point pairs a pair, chosen by hand on both images by one unnamed annotator, mostly on vessels and their crossings, about eight towards the edges of the overlap and two near its centre, then each moved to the best match of the patches around it. Nobody says the moved points were checked; ten points sample the overlap sparsely.
- Truth quality. On all 134 pairs a quadratic fitted to a pair's ten points leaves a median of 0.7 px on the S pairs, 1.5 px on the P pairs and 3.4 px on the A pairs, where the retina changed: the points are good to about a pixel. One point is wrong, the sixth of P37_1_2, 167 px off; P37 is not a case.
- Alignment. The pairs come unaligned, a test pair's landmarks 20 to 1232 px apart. The moving header carries one translation from elastix mutual information inside the fields of view, the best of several starts, no annotation used. A rigid fit turns the images by 4 degrees at most and leaves as much, so only the translation is kept.
- Baseline. With no registration the 10 test pairs score a TRE of 448 px (5.7 to 1075), rTRE 0.11, against 9.9 px after a rigid fit on the points. The S pairs keep 5.7 to 33 px and A09 43 px; the six P pairs keep 515 to 1075 px, a mosaic's overlap for the methods to find, since the start lines up the round fields of view.
- Outside numbers. None: the dataset's page ranks methods by the area under their success rate over the 134 pairs as delivered, other pairs and another start.
| Accuracy | Smoothness | Cost | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | TRE px | Change | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 1 | vismatch matchers · RoMaGPU | 8.24 ±5.12 | −440.02 | 0.0020 ±0.0012 | 0.011 | 0.00 | 11 | 4.0 | 6.9 | |
| 22–3 | vismatch matchers · XFeatGPU | 12.96 ±7.73 | −435.30 | 0.0031 ±0.0019 | 0.013 | 0.00 | 5 | 3.0 | 0.5 | |
| 32–3 | vismatch matchers · DISK + LightGlueGPU | 13.20 ±7.77 | −435.07 | 0.0032 ±0.0019 | 0.017 | 0.00 | 5 | 3.1 | 3.8 | |
| 32–3 | vismatch matchers · SIFT + LightGlueGPU | 13.82 ±7.83 | −434.44 | 0.0034 ±0.0019 | 0.015 | 0.00 | 5 | 3.0 | 0.6 | |
| 5 | vismatch matchers · ALIKED + LightGlueGPU | 26.59 ±16.60 | −421.67 | 0.0065 ±0.0040 | 0.032 | 0.00 | 5 | 3.2 | 2.2 | |
| No registration | 448.27 ±289.25 | 0.1089 ±0.0702 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on TRE px, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on TRE px (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
1 result not ranked: an older run, pairs still missing, or a failure on every pair
| Accuracy | Smoothness | Cost | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | TRE px | Change | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Machine | Images |
| vismatch matchers · LoFTRGPUfailed on every pair | 448.27 ±289.25 | = | 0.1089 ±0.0702 | 0.000 | Laptop | ||||||
TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- 10 test pairs drawn with seed 2026 among the 32 candidates outside development, from the three categories in proportion (six P, three S, one A), and 10 development pairs. The 42 candidates are one pair from each group of pairs linked by a shared image, so each case is its own eye and no two cases share an image.
- Truth
- landmarks by a program, verified by people the dataset's ten point pairs a pair, chosen by hand on both images by one annotator, mostly on vessels and their crossings, about eight towards the edges of the overlap and two near its centre, then each moved to the best match of the image patches around it (Hernandez-Matas et al. 2017, section 3.2)
- Source
- projects.ics.forth.gr/cvrl/fire
doi.org/10.35119/maio.v1i4.42
carlos.hernandez.im/papers/2017_07_JMO.pdf
arxiv.org/abs/2401.05901
github.com/ruc-aimc-lab/SuperRetina/blob/main/test_on_FIRE.py - Licence
- none stated
10 pairs for tuning, never tested
P23, P36, P20, S53, P39, S61, P25, S45, S46, P19