Claims:1. A system for performing optical flow for input images comprising:
a memory to store at least a portion of an image pair comprising a first image and a second image; and
one or more processors coupled to the memory, the one or more processors to:
partition the first image into a first set of one or more partitions and the second image into a corresponding second set of one or more partitions;
apply first deep learning based optical flow processing to a first input volume comprising a first partition of the first set of one or more partitions and a corresponding second partition of the second set of one or more partitions to generate first optical flow results;
apply, separate from the first deep learning based optical flow processing, second deep learning based optical flow processing to a second input volume comprising one of downsampled first and second images corresponding to the first and second images, respectively, or a third partition of the first set of one or more partitions and a corresponding fourth partition of the second set of one or more partitions to generate second optical flow results; and
merge the first and second optical flow results to generate a final optical flow map for the image pair.
, Description:BACKGROUND
In various contexts, such as stereoscopic image matching, estimating the optical flow between pairs of images (e.g., a stereo pair of images) is an important operation. For example, it may be desirable to perform such optical flow estimation for a stereo pair of images suitable for view interpolation applications. Such optical flow estimation and its effect on view interpolation results is an area of ongoing concern. Currently, deep learning for the purposes of optical flow estimation is being explored and has been found capable of producing improved results relative to traditional approaches. However, such deep learning approaches suffer from memory constraints and can only operate on low resolution images. To generate results for higher resolution images while satisfying memory constraints, the stereo pair of images are down sampled at the input and the estimated flows are up sampled at the output of the network to generate view interpolation results at full input resolution. Such techniques introduce undesirable artifacts in the view synthesis results.
Creating optical flow results between image pairs is critical in many imaging, artificial intelligence, virtual reality, artificial reality, and other contexts. It is desirable to have high quality optical flow results for high resolution images that do not elicit artifacts and other problems. It is with respect to these and other considerations that the present improvements have been needed. Such improvements may become critical as the desire to provide optical flow results in a variety of contexts becomes more widespread.