fidopt Interactive view
Photographs of scenes (2D), two viewpoints, points from depth and camera poses

Image Matching Challenge PhotoTourism

Two tourists' photographs of one landmark, taken years apart with different cameras: the two images of one of the Image Matching Challenge 2021's PhotoTourism stereo pairs.

9 test pairsscenesame patientfixed PHOTO, moving PHOTO2D, distances in pxdelivered alignedresearch only
Look at the images
Read before comparing (9)
  • Scenes. Famous landmarks photographed by many tourists: the British Museum, the side of Florence's cathedral, the Lincoln Memorial statue, London Bridge, Milan's cathedral, Mount Rushmore, Piazza San Marco, the Sagrada Familia and St Paul's Cathedral for the test cases; the Reichstag, Sacré-Cœur and St Peter's Square for development. The setting reads "same patient" for one place seen twice.
  • Pairs. One pair a scene, since the photographs of one landmark are not independent: among the challenge's stereo pairs (the pairs of the scene's 100 images that share at least 10% of what they see), the first in the order of sha256("2026:IMCPhotoTourism:<scene>:<pair>"), a rule set before any pair was looked at. The two differ in viewpoint, scale (by up to 2.5 times, on Piazza San Marco), season, light and camera.
  • Truth. The organisers reconstructed each landmark from its full collection with COLMAP, then cleaned its multi-view stereo depth. The corpus carries each grid point with the fixed image's depth and both poses into the moving image, kept where that image's depth agrees within 5%: 26 to 199 points a test pair (median 56), on the landmark, never on people, sky or trees. Made by programs, no person checking: graded automatic and ranked apart.
  • Looked at. On the pairs warped through the points the façades and statues fall on each other to a few pixels; the challenge itself ranks methods by the angular error of the pose they recover, at 5 to 10 degrees.
  • Images. The JPEG photographs as delivered, red, green and blue a pixel, 488 to 1082 px a side, the two of a pair of different sizes, one with a caption and a black frame. No mask. Distances are in pixels from 0 at the first pixel's centre; the rTRE divides them by the fixed image's diagonal.
  • Duplicates. WxBS also holds photographs of landmarks from Flickr. Their scenes differ (Kyiv, Prague, Moscow, Berlin's cathedral, Warsaw, Charlottenburg), and 32 x 32 thumbnails of the 69 WxBS images against the 1168 IMC images find no shared photograph: its two known pairs score 1.0 (WxBS's shared Kyiv photograph) and 0.985 (an IMC photograph rescaled, cropped and re-encoded), the closest WxBS against IMC 0.92, eight pairs at 0.90 or more, each two different places by eye.
  • Alignment. The photographs come unaligned, each corner at the origin, and get no starting transform: scale, viewpoint and depth differ, which no translation or rigid fit from the intensities takes out.
  • Baseline. With no registration the 9 test pairs score a TRE of 238 px (65 to 420), rTRE 0.21, milan_cathedral closest and mount_rushmore farthest.
  • Outside numbers. None compare: the challenge scores the camera pose recovered from matches, not points.

No method has a result on every pair of this dataset yet.

Pair by pair

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

Test pairs
The challenge's 9 PhotoTourism test scenes are the test cases, whole, and its 3 validation scenes the development cases, one pair a scene. Its other training scenes come only as archives of 3 to 13 GB each and are not used.
Truth
landmarks by a program pixels of a 16 x 16 grid on the fixed image carried into the moving one by the provider's reconstruction, its camera poses and multi-view stereo depth, kept where the moving image's depth shows the point is seen, 26 to 199 a pair
Licence
none stated; photographs from YFCC100M, each under its own Creative Commons licence
3 pairs for tuning, never tested

reichstag, sacre_coeur, st_peters_square