fidoptSUN3D
Two colour frames of a hand-held RGB-D video of one indoor place, half a second apart: images m (fixed) and m + 15 (moving) at the middle of a SUN3D sequence.
20 test pairsscenesame patientfixed PHOTO, moving PHOTO2D, distances in pxdelivered alignedresearch only
Read before comparing (8)
- Places. Hand-held RGB-D videos of 210 indoor places, rooms, corridors, flats and hotel rooms, most at MIT, Harvard and Brown, filmed by walking through them. One pair a place, from the first of its sequences in SUN3Dv1.txt that has camera poses and intrinsics: the sequences of one place film one room. Nine places have no such sequence. The setting reads "same patient" for one place seen twice.
- Pairs. Images m and m + 15 of the sequence, m its middle, half a second apart, a rule set before the draw: the camera turns and walks, 24 to 139 px of motion on average a test pair, with parallax between near and far surfaces.
- Truth. The provider's camera poses come from its structure from motion with generalised bundle adjustment, corrected by object labels on 11 sequences; the depth is the sensor's, registered to the colour image and matched by time stamp. The corpus carries each grid point of image m with its depth and the two poses into image m + 15, kept where that image's depth agrees within 5%, so occluded points and points without depth (windows, mirrors, far walls) drop out: 11 to 203 points a pair. Made by programs, no person checking: graded automatic and ranked apart.
- Precision. Patches of 31 px around the points, searched in image m + 15 by correlation, land a median of 3.2 px (0.9 to 17.0 a test pair) from where the truth puts them; a truth shifted 8 px on purpose reads 8.3 px. Five test pairs read 6 to 17 px, where the poses or the depth's registration drift or the patches fail on blur and blank walls: mit_26_100 (11 points) and mit_46_6bathroom are visibly off by about ten pixels.
- Images. The JPEG frames as delivered, red, green and blue a pixel, 640 x 480. No mask. Distances are in pixels from 0 at the first pixel's centre; the rTRE divides them by the fixed image's diagonal, 800 px.
- Alignment. The frames come together, half a second apart, and get no starting transform: the camera's motion and the parallax are the task.
- Baseline. With no registration the 20 test pairs score a TRE of 62.6 px (24.1 to 139.4), rTRE 0.078, mit_dorm_sh_basement closest and harvard_tea_1 farthest: about 20 times the points' median error.
- Outside numbers. None compare: two-view benchmarks built on SUN3D score relative camera poses, not points.
No method has a result on every pair of this dataset yet.
Pair by pair
Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.
About the data
- Test pairs
- 20 test places drawn with seed 2026 among the 201 places that have a sequence with camera poses, and 18 development places. One pair a place, from its first sequence with poses.
- Truth
- landmarks by a program pixels of a 16 x 16 grid on image m carried into image m + 15 by the sensor's depth and the provider's camera poses, kept where image m + 15's depth shows the point is seen, 11 to 203 a pair (median 162 on the test pairs)
- Source
- sun3d.cs.princeton.edu
sun3d.cs.princeton.edu/data
sun3d.cs.princeton.edu/SUN3Dv1.txt
sun3d.cs.princeton.edu/toolbox/SUN3Dreader.m
github.com/PrincetonVision/SUN3Dsfm
vision.princeton.edu/projects/2013/SUN3D/paper.pdf - Licence
- none stated
18 pairs for tuning, never tested
hotel_ucla_jx, home_ac, mit_dorm_baker_cj, mit_w85_2floor, brown_cogsci_2, mit_32_d466, mit_gym_z_squash, mit_w16, mit_32_g825, mit_w85_basement_bathroom, brown_cs_5, mit_46_5bathroom, mit_w85k1, mit_32_g442, home_ph_cz, mit_w85_11, mit_76_417, harvard_corridor_lounge