fidopt Interactive view
Frames of a scene (2D), optical flow, ranked apart

Middlebury optical flow

Two colour frames of one scene a moment apart, real, rendered or a stereo pair: frame 10 (fixed) and frame 11 (moving) of a Middlebury optical flow sequence with public ground truth.

8 test pairsscenesame patientfixed PHOTO, moving PHOTO2D, distances in pxdelivered alignedresearch only
Look at the images
Read before comparing (7)
  • Scenes. Three real laboratory scenes filmed under visible light with a hidden fluorescent texture (Dimetrodon, Hydrangea, RubberWhale), four rendered outdoor scenes of procedurally generated rocks and trees or buildings, each its own random scene (Grove2, Grove3, Urban2, Urban3), and Venus, a piecewise planar stereo pair from Middlebury's 2001 stereo set. The setting reads "same patient" for one scene seen twice.
  • Truth. Real scenes: windows of the high-resolution UV images tracked over 40 captured frames, cross-checked forwards and backwards, refined by Lucas-Kanade and downsampled six times, good to about 1/60 px by the authors' account; occluded or failed pixels carry no flow (1.6 to 6.6% of a frame). Rendered scenes: the 3D motion projected by a shader, the dominant of nine sub-pixel vectors, exact. Venus: the stereo disparity in steps of 1/8 px, vertical flow 0. A measuring process and a renderer, no person checking: graded automatic and ranked apart.
  • Points. A 16 x 16 grid of pixels on frame 10, kept where the flow is known and lands inside frame 11, 245 to 256 a pair, each with its image p + flow(p). On the rendered scenes the flow is defined behind an occlusion too, so a few points sit on surfaces frame 11 hides: the dataset scores those as well.
  • Images. The PNG frames as delivered, red, green and blue a pixel, 584 x 388 (real), 640 x 480 (rendered) or 420 x 380 (Venus). No mask. Distances are in pixels from 0 at the first pixel's centre; the rTRE divides them by the fixed image's diagonal.
  • Alignment. The frames come together, a moment apart, and get no starting transform: the motion, 1 to 22 px, is the task.
  • Baseline. With no registration the eight test pairs score a TRE of 4.1 px (1.3 to 8.3), rTRE 0.0056, RubberWhale closest and Urban2 farthest: small motions, yet more than ten times the truth's own error.
  • Outside numbers. None compare: the dataset ranks flow methods on its eight evaluation scenes, whose flow it keeps.

No method has a result on every pair of this dataset yet.

Pair by pair

Click a score to look at that result: the fixed image, and over it the moving image after registration, with the annotations. Underlined scores have their image; for the others you see the pair without the result. A green cell gained against no registration on that pair, a red one lost.

About the data

Test pairs
All eight scenes with public ground-truth flow are test cases, one pair a scene: the dataset's evaluation scenes keep their flow hidden. None is left for development.
Truth
landmarks by a program the dataset's flow from frame 10 to frame 11 at a 16 x 16 grid of pixels of frame 10, kept where it is known and lands inside frame 11, 245 to 256 a pair: measured by tracking a hidden fluorescent texture (3 real scenes), rendered (4 synthetic scenes) or a stereo disparity (Venus)
Licence
none stated