Real-Time Optical Flow Estimation Method Based on Cross-Stage Network
Source abstract
Abstract
In this paper, a real-time optical flow estimation method based on a cross-stage network is proposed. The proposed model is designed with a network structure with encoders and decoders. The proposed method combines cross-stage network technology with the network structure of FlowNet2 and RAFT to achieve improved parameter number and estimation performance. For real-time optical flow estimation, it is important to maintain performance while reducing the number of parameters in the network. In the proposed method, structural convergence is performed to increase performance while reducing the number of parameters by applying the cross-stage network structure. The proposed model is designed to solve the bottlenecks in model accuracy and complexity by separating feature extraction and flow estimation processes. Flying Chairs, Flying Things 3D, and KITTI datasets were used to evaluate the performance of the proposed model, and the experimental results show superior performance compared to previous traditional methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Advanced Image Processing Techniques · Image Enhancement Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of ICT Robotics and Mechanical Engineering, Hankyong National University, Anseong 456-749, Republic of Korea
Smart Convergence Technology Research Center, Hankyong National University, Anseong 456-749, Republic of Korea
Funding
Korea Institute of Planning and Evaluation for Technology in Food, Agriculture, and Forestry (IPET)
References (28)
Show source-provided references
1. Optical flow based robot obstacle avoidance10.5772/5715 ↗
2. Agarwal, A., Gupta, S., and Singh, D.K. (2016, January 14–17). Review of optical flow technique for moving object detection. Proceedings of the 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I), Greater Noida, India.10.1109/IC3I.2016.7917999 ↗
3. Han, X., Gao, Y., Lu, Z., Zhang, Z., and Niu, D. (2015, January 8–20). Research on moving object detection algorithm based on improved three frame difference method and optical flow. Proceedings of the 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC), Qinhuangdao, China.10.1109/IMCCC.2015.420 ↗
4. Roxas, M., and Oishi, T. (2018, January 12–15). Real-time simultaneous 3D reconstruction and optical flow estimation. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00102 ↗
5. Optical flow and scene flow estimation: A survey10.1016/j.patcog.2021.107861 ↗
6. Determining Optical Flow. Artificial Intelligence 1710.1016/0004-3702(81)90024-2 ↗
7. Lucas, B.D. (1981, January 24–28). An iterative technique of image registration and its application to stereo. Proceedings of the 7th IJCAI, Vancouver, BC, Canada.
8. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Van Der Smagt, P., Cremers, D., and Brox, T. (2015, January 7–13). Flownet: Learning optical flow with convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.316 ↗
9. Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., and Brox, T. (2017, January 21–26). Flownet 2.0: Evolution of optical flow estimation with deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.179 ↗
10. Ranjan, A., and Black, M.J. (2017, January 21–26). Optical flow estimation using a spatial pyramid network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.291 ↗
11. Sun, D., Yang, X., Liu, M.-Y., and Kautz, J. (2018, January 18–22). Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00931 ↗
12. Models matter, so does training: An empirical study of cnns for optical flow estimation10.1109/TPAMI.2019.2894353 ↗
13. Hui, T.-W., Tang, X., and Loy, C.C. (2018, January 18–22). Liteflownet: A lightweight convolutional neural network for optical flow estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00936 ↗
14. A lightweight optical flow cnn—Revisiting data fidelity and regularization10.1109/TPAMI.2020.2976928 ↗
15. Yang, G., and Ramanan, D. (2019). Volumetric correspondence networks for optical flow. Adv. Neural Inf. Process. Syst., 32, Available online: https://proceedings.neurips.cc/paper/2019/hash/bbf94b34eb32268ada57a3be5062fe7d-Abstract.html.
16. Teed, Z., and Deng, J. (2020, January 23–28). Raft: Recurrent all-pairs field transforms for optical flow. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK.10.1007/978-3-030-58536-5_24 ↗
17. Xu, H., Zhang, J., Cai, J., Rezatofighi, H., and Tao, D. (2022, January 18–24). Gmflow: Learning optical flow via global matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00795 ↗
18. Bai, S., Geng, Z., Savani, Y., and Kolter, J.Z. (2022, January 18–24). Deep equilibrium optical flow estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00070 ↗
19. Unifying flow, stereo and depth estimation10.1109/TPAMI.2023.3298645 ↗
20. Wang, C.-Y., Liao, H.-Y.M., Wu, Y.-H., Chen, P.-Y., Hsieh, J.-W., and Yeh, I.-H. (2020, January 14–19). CSPNet: A new backbone that can enhance learning capability of CNN. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00203 ↗
21. Park, M.-H., Cho, J.-H., and Kim, Y.-T. (2023). CNN Model with Multilayer ASPP and Two-Step Cross-Stage for Semantic Segmentation. Machines, 11.10.3390/machines11020126 ↗
22. Butler, D.J., Wulff, J., Stanley, G.B., and Black, M.J. (2012, January 7–13). A naturalistic open source movie for optical flow evaluation. Proceedings of the Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy.10.1007/978-3-642-33783-3_44 ↗
23. Vision meets robotics: The kitti dataset10.1177/0278364913491297 ↗
24. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2017, January 7–9). Automatic Differentiation in Pytorch. Proceedings of the Thirty-first Conference on Neural Information Processing Systems, Long Beach, CA, USA.
25. Loshchilov, I., and Hutter, F. (2017). Decoupled weight decay regularization. arXiv.
26. Yin, Z., Darrell, T., and Yu, F. (2019, January 16–20). Hierarchical discrete distribution decomposition for match density estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00620 ↗
27. Zhao, S., Sheng, Y., Dong, Y., Chang, E.I., and Xu, Y. (2020, January 13–19). Maskflownet: Asymmetric feature matching with learnable occlusion mask. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00631 ↗
28. Hur, J., and Roth, S. (2019, January 16–20). Iterative residual refinement for joint optical flow and occlusion estimation. Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00590 ↗