I really want to try blurring only now can we make blurring and coarse warping gradients equiavlent? (I think it sholud be possible with some sort of linear blurring)
weird coarse to fine idea:
sequential pyramid warp: generates warps
each with half the flow_max of the previous one
(i.e. stop gradient on ) loss calculated at lower layer & warping used is (or maybe discretize/round to the closest flow_max/2 multiple)
basically allows refinement? so maybe RAFT precursors have done this already
ehhh maybe this is just ‘done’ by diffusion (?)
bicubic interpolation
probabilistic formulation (for basics, see part 8 in ! 2005.F&W—Optical Flow Estimation (Fleet & Weiss, 2005))
depth pixel
- how to deal with shrinking?
adding weights back as a channel to opimize; weighing color differences by the weights
use a 3D unet (Adelson & Bergen)
penalize hue changes more than brightness changes?
match fourier transforms?? or match frequencies at a point (ideally I guess the frequencies/convolutions would be discovered by the NN?)
- what are nice properties of these filters that we can set as secondary objectives for the NN? or just do what F&J do and match phases(?)
need to speed up current training
++regularization
target baseline
what if we just predicted the Fourier transform of the flows?? Is warping nice in Fourier space lol
logging: measure similarity between gradient and true - predict (more similar = better)
translate the optical flow and use it to warp the image (L&K like) as smoothness constraint
warp inverse (255 - x) so it doesn’t do weird stuff in dark areas of the image
- actually I think black is -1 already so idk if this works. But i don’t like that 0 values can still be added arbitrarily this seems kinda bad
OF COURSE use vincent’s substitution to make the loss less weird
use a robust penalty (e.g. truncated quadratic or other stuff from 1996.B&A)