FIMD

Colour fundus photographs of one child's eye as myopia developed (FIMD): the first visit at 6 or 7 years old (fixed) and the last, four or five years later, on another camera (moving).

10 test pairseyesame patientfixed FUNDUS, moving FUNDUS2D, distances in pxaligned once by the benchmark

Look at the images

Read before comparing (8)

Pool. Ten eyes of five children, three girls and two boys, photographed at five or six visits from 6 or 7 years old to 11 or 12; on the cases the uncorrected visual acuity fell from 0.8 to 1.2 to 0.1 to 0.4. The dataset's 70 pairs of 57 photographs link into the ten eyes, and capture times and the screening table into five children.
Images. The first visit's photographs are 3888 x 2592 pixels from a Canon EOS 40D; the last visit's, from another camera, 2592 x 1728 (1924 x 1556 on case 31), with other brightness, contrast and field of view. Each camera leaves a reflex of its own in the field. The JPEG colour is kept; no field-of-view mask is given.
Units. The cameras give no pixel size, so prepare.py sets every spacing to 1 and distances are in pixels of the fixed photograph. The rTRE divides by its diagonal, 4673 pixels.
Truth. Twelve point pairs a pair, mainly at vessel bifurcations and crossings, placed with the help of experienced ophthalmologists by FIRE's method: chosen by hand on both images, then moved to the best patch correlation nearby. A reference photograph's points are the same in every pair that shares it, so only the later photograph's can have moved. Twelve points sample the retina sparsely.
Truth error. Points placed twice on one photograph lie a median of 1.0 px apart (245 points on 23 photographs, 8.1 px at most): about 2 px a pair on the fixed grid. A quadratic map fitted to a case's twelve pairs leaves 4.4 to 25 px (median 6.8): unlike FIRE's, these pairs change more than such a map follows.
Alignment. The photographs come unaligned and at other scales, the points 760 to 1080 px apart on average. The moving header carries only the image centres put together, nothing optimised. A fit on the points enlarges the last photograph 1.66 to 2.10 times onto the first: that scale, which cannot be told from the eye's growth, is left to the methods.
Baseline. With no registration the 10 test pairs leave a TRE of 506 px (433 to 554), rTRE 0.11, mostly the scale between the two cameras: an affine fit on the points leaves 14.6 px.
Outside numbers. None: the paper evaluates registration on FIRE and FIMD, but it is not open and was not read.

		Accuracy			Smoothness		Cost
#	Method	TRE px	Change	rTRE	SDlogJ	Folds %	Time s	RAM GB	GPU GB	Images
1	vismatch matchers · DISK + LightGlueGPU	8.12 ±2.28	−498.16	0.0017 ±0.0005	0.003	0.00	5	3.2	2.7
1	vismatch matchers · RoMaGPU	6.61 ±2.55	−499.67	0.0014 ±0.0005	0.003	0.00	10	4.1	6.9
33–5	vismatch matchers · LoFTRGPU	9.68 ±2.37	−496.60	0.0021 ±0.0005	0.003	0.00	6	3.0	16.8
33–5	vismatch matchers · SIFT + LightGlueGPU	7.67 ±2.20	−498.61	0.0016 ±0.0005	0.003	0.00	5	3.1	0.6
53–5	vismatch matchers · ALIKED + LightGlueGPU	9.53 ±3.32	−496.75	0.0020 ±0.0007	0.003	0.00	5	3.3	1.7
6	vismatch matchers · XFeatGPU	225.23 ±369.28	−281.05	0.0482 ±0.0790	0.722	4.88	5	3.2	0.5
	No registration	506.28 ±27.75		0.1083 ±0.0059	0.000	0.00

The order averages each method's ranks on TRE px, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on TRE px (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.

TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.

Pair by pair

Measure

TRE px	1	2	3	4	5	6	7	8	9	10
vismatch matchers · DISK + LightGlue	4.80	7.74	5.02	6.14	5.36	9.20	13.20	13.65	7.06	9.03
vismatch matchers · RoMa	2.70	4.22	4.53	4.59	4.57	5.74	12.21	13.54	6.91	7.04
vismatch matchers · LoFTR	7.77	8.14	8.69	6.13	7.27	7.52	16.55	13.91	11.59	9.23
vismatch matchers · SIFT + LightGlue	6.52	4.91	5.18	6.76	4.90	6.77	12.64	13.63	8.17	7.20
vismatch matchers · ALIKED + LightGlue	3.44	9.65	6.53	6.08	4.94	7.04	15.72	14.82	11.28	15.80
vismatch matchers · XFeat	4.14	226.01	94.22	11.01	6.96	7.09	169.93	1677.62	9.64	45.71
No registration	483.09	499.38	533.15	499.06	432.86	548.80	496.31	471.94	553.47	544.75

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

The first test caseClick to look at every case.

Test pairs: All ten cases are test pairs, one an eye: its first and last visits. The ten eyes are those of five children, so none is left for development.
Truth: landmarks by a program, verified by people the dataset's twelve point pairs a pair, mainly at the bifurcations and crossings of vessels, placed by hand on both images with the help of experienced ophthalmologists by FIRE's annotation method, whose correction by patch matching can at most have moved the later photograph's points
Source: data.mendeley.com/datasets/jkzsh6pcv4
doi.org/10.17632/jkzsh6pcv4.2
doi.org/10.1016/j.media.2024.103242
pubmed.ncbi.nlm.nih.gov/38901099
doi.org/10.21227/zwxk-kd91
doi.org/10.35119/maio.v1i4.42
Licence: CC BY 4.0