fidoptHPatches
Two colour photographs of one planar scene, a poster, print, mural or wall, from two viewpoints: a viewpoint sequence's reference image (fixed) and its last target image (moving), from HPatches.
Read before comparing (10)
- Pool. The 57 illumination sequences are left out: the camera stays. Of the 59 viewpoint sequences, the 12 whose images references.txt credits to earlier datasets are left out, and so is v_sunseason, whose folder holds the readme of Cordes et al.'s data and whose images have the size of four of the five Hannover sequences.
- Pairs. Image 1 against image 6, a rule set before the images were looked at, so that no image is in two cases. Image 6 is not always the largest change: after an affine fit it leaves the most on 10 of the 46 sequences, image 3 on 18.
- Images. The photographs as delivered, in colour, red, green and blue a pixel; a method that needs one value a pixel makes it in its adapter. Image 6 differs in size from image 1 on 9 of the 46 sequences. No mask. Distances are in pixels; rTRE divides them by the fixed image's diagonal, 945 to 1869 px on the test pairs.
- Truth. The dataset's homography from image 1 to image 6, from correspondences picked by hand, refined on interest points matched automatically. HPatches reports no error for it and nobody says it was checked: a program's truth from a person's start, graded provider and ranked apart.
- Points. The corpus's: a 10 x 10 grid on image 1, kept where the homography carries it inside image 6, 13 to 100 a test pair (median 85). They carry only the homography's eight numbers, so their errors are alike from one point to the next.
- Off the plane. On 7 of the 20 pairs built, part of image 1 lies off the plane, where no homography holds: a floor, a road, a neighbouring panel, a fence. There the points are kept inside a rectangle the corpus drew by eye: v_charing keeps 13 of 51, v_yuri 35 of 60, v_coffeehouse 55 of 93. Relief and reflections that move with the camera stay.
- Accuracy. On the 20 pairs built, patches of 41 px matched through the homography land a median of 0.4 to 2.1 px from where it puts them on the test pairs, against 0.1 px or less on an image made through the homography itself: the truth is good to one or two pixels on the plane. Positions count from 0 at the first pixel's centre, as in ITK.
- Alignment. Each image has its corner at the origin and gets no starting transform: finding the change of viewpoint, with its scale, shear and perspective, is the task. Neither image is resampled. With no registration the 10 test pairs score a TRE of 223 px (100 to 332), rTRE 0.15, more than 50 times the truth's own error.
- Grade. The truth misses a main board's first check, a person making or checking it on both images: two people placing about ten points each on the 20 pairs built would make it semi-automatic.
- Outside numbers. None: HPatches ranks descriptors on patches, not registrations of whole images.
| Accuracy | Smoothness | Cost | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| # | Method | TRE px | Change | rTRE | SDlogJ | Folds % | Time s | RAM GB | GPU GB | Images |
| 11–6 | vismatch matchers · ALIKED + LightGlueGPU | 0.86 ±0.48 | −222.10 | 0.0006 ±0.0003 | 0.282 | 0.00 | 3 | 2.5 | 1.8 | |
| 11–6 | vismatch matchers · RoMaGPU | 0.97 ±0.53 | −221.98 | 0.0007 ±0.0004 | 0.282 | 0.00 | 8 | 3.2 | 6.9 | |
| 11–6 | vismatch matchers · SIFT + LightGlueGPU | 0.76 ±0.32 | −222.20 | 0.0006 ±0.0002 | 0.282 | 0.00 | 3 | 2.2 | 0.5 | |
| 41–6 | vismatch matchers · DISK + LightGlueGPU | 1.01 ±0.49 | −221.95 | 0.0007 ±0.0004 | 0.282 | 0.00 | 4 | 2.3 | 3.0 | |
| 51–6 | vismatch matchers · XFeatGPU | 1.43 ±0.71 | −221.52 | 0.0011 ±0.0005 | 0.287 | 0.00 | 3 | 2.3 | 0.5 | |
| 61–6 | vismatch matchers · LoFTRGPU | 1.54 ±0.63 | −221.42 | 0.0012 ±0.0006 | 0.288 | 0.00 | 3 | 2.1 | 8.0 | |
| No registration | 222.96 ±59.74 | 0.1545 ±0.0344 | 0.000 | 0.00 | ||||||
The order averages each method's ranks on TRE px, SDlogJ, so the first one is not always the best in every column. ± is the 95% interval of the mean over the 10 pairs. Two neighbours are tied when a paired Wilcoxon test on the shared pairs does not find the upper one better on TRE px (5%): the places a method could hold are then shown under its own. The best value of each column is in colour. Click a row to see its images, a column title to sort by it. Time and memory measured on Laptop.
TRE: distance between matching landmarks. rTRE: the TRE over the length of the image's diagonal, to compare datasets of other sizes and units. SDlogJ: how irregular the deformation is, 0 is perfectly smooth. Folds: share of the image where the deformation folds over itself.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- 10 test sequences drawn with seed 2026 among the 36 outside development, and 10 development sequences, from 46 viewpoint sequences. One pair a sequence, image 1 against image 6. The illumination sequences, the 12 taken from earlier datasets and v_sunseason are left out.
- Truth
- landmarks as released a regular grid of 10 x 10 points on the reference image, those that the dataset's homography carries inside the target image and, on 7 of the 20 pairs built, that lie inside a rectangle drawn by eye around the plane, each with its image under the homography: 13 to 100 a pair
- Source
- github.com/hpatches/hpatches-dataset
github.com/hpatches/hpatches-dataset/blob/master/references.txt
huggingface.co/datasets/vbalnt/hpatches
arxiv.org/abs/1704.05939
www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/mikolajczyk_pami2004.pdf
www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/vibes_ijcv2004.pdf
www.tnt.uni-hannover.de/project/feature_evaluation - Licence
- MIT
10 pairs for tuning, never tested
v_wormhole, v_coffeehouse, v_tabletop, v_machines, v_tempera, v_beyus, v_london, v_yuri, v_strand, v_birdwoman