![[Screen Shot 2023-09-15 at 21.17.01.png|center|500]] Link to paper
Probably one of my favorite papers from so far (it also has like 6,000 citations). A review article comparing many early optical flow techniques (including pretty much every paper I’ve read that predates this one) to find the most robust and accurate techniques. I’m glad that someone finally decided to go around and benchmark these, as I swear every paper used different data or did something weird to make their method look good or just essentially had zero comparisons to other papers.
There are a few major takeaways, which I transcribe from the last section:
- (☆) [[1981.L&K—An Iterative Image Registration Technique with an Application to Stereo Vision (Lucas & Kanade, 1981)||L&K]] and [[1990.F&J—Computation of Component Image Velocity from Local Phase Information (Fleet & Jepson, 1990)||F&J]] work the best. F&J works spectacularly well on artificial image sequences, but the benefits are less obvious on real images.
- The method in which derivatives are taken for derivative estimation are very important. is very different from, say, (a central derivative).
- Temporal smoothing improves performance.
- (☆) Explicit, local smoothness constraints (such as that of L&K) work better than global smoothness constraints. I think this supports my idea that you can do everything with warping. Just warp really well—our end goal is generating good images anyway. I might try an L&K-inspired smoothing constraint and see it works better than an energy functional.![[Screen Shot 2023-09-15 at 21.28.09.png|center|500]]
- Second-order methods work, but are sensitive to first-order deformations (rotations, shear, etc.)
- Matching techniques work poorly on subpixel displacements. (I think this is because the studied techniques didn’t implement bilinear inteporlation warping or something like that. I still think matching objectives are the way to go.) Additionally, with matching objectives that incorporate context (i.e. all of them) they start breaking a lot when there are dilations.
- Phase-based things work quite well, although they’re sensitive to temporal aliasing, frequencies in the image, etc. (I’m really curious about trying to implement one.)
Tags
Book chapters, review articles Affine, parameterized optical flow Bandpass filters Energy or loss minimization Expectation-maximization Frequency analysis Gradient constancy Hand-designed filters Local methods NOT deep learning Old school Optical flow Phase constancy Probabilistic flow Second-order methods Smoothness constraints Unsupervised Video, multi-frame
Cited
- 1981.H&S—Determining Optical Flow (Horn & Schunck, 1981)
- 1981.L&K—An Iterative Image Registration Technique with an Application to Stereo Vision (Lucas & Kanade, 1981)
- 1985.A&B—Spatiotemporal energy models for the perception of motion (Adelson & Bergen, 1985)
- 1986.N&E—An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences (Nagel & Enkelmann, 1986)
- 1989.Fwork—A Computational Framework and an Algorithm for the Measurement of Visual Motion
- 1990.F&J—Computation of Component Image Velocity from Local Phase Information (Fleet & Jepson, 1990)
- 1991.Prob—Probability Distributions of Optical Flow (Simoncelli… 1991)
Cited By
Return: index