Hardware-Aware Mobile Building Block Evaluation for Computer Vision
Source abstract
Abstract
In this paper, we propose a methodology to accurately evaluate and compare the performance of efficient neural network building blocks for computer vision in a hardware-aware manner. Our comparison uses pareto fronts based on randomly sampled networks from a design space to capture the underlying accuracy/complexity trade-offs. We show that our approach enables matching of information obtained by previous comparison paradigms, but provides more insights into the relationship between hardware cost and accuracy. We use our methodology to analyze different building blocks and evaluate their performance on a range of embedded hardware platforms. This highlights the importance of benchmarking building blocks as a preselection step in the design process of a neural network. We show that choosing the right building block can speed up inference by up to a factor of two on specific hardware ML accelerators.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: CCD and CMOS Imaging Sensors · Advanced Neural Network Applications · Advanced Memory and Neural Computing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
IDLab-AIRO, Ghent University, imec, 9052 Ghent, Belgium
MICAS-ESAT, KU Leuven, 3001 Leuven, Belgium
Funding
Research Foundation Flanders (FWO-Vlaanderen)
References (33)
Show source-provided references
1. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
2. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–23). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
3. Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 27). Searching for mobilenetv3. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
4. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
5. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–23). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
6. Ma, N., Zhang, X., Zheng, H.T., and Sun, J. (2018, January 8–14). Shufflenet v2: Practical guidelines for efficient cnn architecture design. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_8 ↗
7. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
8. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
9. Baker, B., Gupta, O., Naik, N., and Raskar, R. (2016). Designing neural network architectures using reinforcement learning. arXiv.
10. Liu, H., Simonyan, K., and Yang, Y. (2018). Darts: Differentiable architecture search. arXiv.
11. Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L.J., Fei-Fei, L., Yuille, A., Huang, J., and Murphy, K. (2018, January 8–14). Progressive neural architecture search. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01246-5_2 ↗
12. Real, E., Aggarwal, A., Huang, Y., and Le, Q.V. (February, January 27). Regularized evolution for image classifier architecture search. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
13. Pham, H., Guan, M., Zoph, B., Le, Q., and Dean, J. (2018, January 10–15). Efficient neural architecture search via parameters sharing. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
14. Stamoulis, D., Ding, R., Wang, D., Lymberopoulos, D., Priyantha, B., Liu, J., and Marculescu, D. (2019, January 16–20). Single-path nas: Designing hardware-efficient convnets in less than 4 hours. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Würzburg, Germany.10.1007/978-3-030-46147-8_29 ↗
15. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., and Le, Q.V. (2019, January 15–20). Mnasnet: Platform-aware neural architecture search for mobile. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00293 ↗
16. Wu, B., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., Jia, Y., and Keutzer, K. (2019, January 15–20). Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01099 ↗
17. Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., and Sun, J. (2020, January 23–28). Single path one-shot neural architecture search with uniform sampling. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58517-4_32 ↗
18. Cai, H., Gan, C., Wang, T., Zhang, Z., and Han, S. (2019). Once-for-all: Train one network and specialize it for efficient deployment. arXiv.
19. Radosavovic, I., Johnson, J., Xie, S., Lo, W.Y., and Dollár, P. (November, January 27). On network design spaces for visual recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
20. Radosavovic, I., Kosaraju, R.P., Girshick, R., He, K., and Dollár, P. (2020, January 13–19). Designing network design spaces. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01044 ↗
21. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18–22). Learning transferable architectures for scalable image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00907 ↗
22. Satopaa, V., Albrecht, J., Irwin, D., and Raghavan, B. (2011, January 20–24). Finding a “kneedle” in a haystack: Detecting knee points in system behavior. Proceedings of the 2011 31st International Conference on Distributed Computing Systems Workshops, Minneapolis, MN, USA.10.1109/ICDCSW.2011.20 ↗
23. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017, January 21–26). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.634 ↗
24. Gupta, S., and Akin, B. (2020). Accelerator-aware neural network design using automl. arXiv.
25. Tan, M., and Le, Q. (2019, January 9–15). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
26. Boroumand, A., Ghose, S., Akin, B., Narayanaswami, R., Oliveira, G.F., Ma, X., Shiu, E., and Mutlu, O. (2021, January 26–29). Google neural network models for edge devices: Analyzing and mitigating machine learning inference bottlenecks. Proceedings of the 2021 30th International Conference on Parallel Architectures and Compilation Techniques (PACT), Atlanta, GA, USA.10.1109/PACT52795.2021.00019 ↗
27. Chowdhery, A., Warden, P., Shlens, J., Howard, A., and Rhodes, R. (2019). Visual wake words dataset. arXiv.
28. Zhang, L.L., Han, S., Wei, J., Zheng, N., Cao, T., Yang, Y., and Liu, Y. (July, January 24). nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices. Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, Virtual.
29. Xiong, Y., Liu, H., Gupta, S., Akin, B., Bender, G., Wang, Y., Kindermans, P.J., Tan, M., Singh, V., and Chen, B. (2021, January 20–25). Mobiledets: Searching for object detection architectures for mobile accelerators. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00382 ↗
30. Tan, M., and Le, Q. (2021, January 13–14). Efficientnetv2: Smaller models and faster training. Proceedings of the International Conference on Machine Learning, Virtual.
31. Gibson, P., Cano, J., Turner, J., Crowley, E.J., O’Boyle, M., and Storkey, A. (2020, January 6–8). Optimizing grouped convolutions on edge devices. Proceedings of the 2020 IEEE 31st International Conference on Application-Specific Systems, Architectures and Processors (ASAP), Manchester, UK.10.1109/ASAP49362.2020.00039 ↗
32. (2022, August 15). Google. EfficientNet-Lite. Available online: https://github.com/tensorflow/tpu/blob/master/models/official/efficientnet/lite/README.md.
33. Akin, B., Gupta, S., Long, Y., Spiridonov, A., Wang, Z., White, M., Xu, H., Zhou, P., and Zhou, Y. (2022, January 19–20). Searching for Efficient Neural Architectures for On-Device ML on Edge TPUs. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPRW56347.2022.00300 ↗