CFNet: Achieving Practical Speedup in Lightweight CNNs via Channel-Focused Design and Cross-Channel Mixing
Source abstract
Abstract
Convolutional Neural Networks (CNNs) have achieved remarkable performance in computer vision tasks, but their deployment on resource-constrained devices remains challenging. While existing lightweight CNNs reduce FLOPs significantly, their practical inference speed is limited by memory access bottlenecks. We hence propose CFNet, an efficient architecture that bridges the gap between theoretical efficiency and practical speed through synergistic design of channel-focused convolution (CFConv) and channel mixed unit (CMU). CFConv dynamically selects informative channels via learnable GroupNorm scaling factors and reparameterization, reducing both FLOPs and memory access, while CMU enables cross-channel communication through a split-transform-and-mix strategy to mitigate information loss. Experiments on CIFAR/ImageNet classification and MS COCO object detection demonstrate CFNet’s superior performance. On ImageNet-1K, CFNet-A achieves 35.5% and 189.4% GPU throughput improvements over MobileNetV2 and MobileViTv1-XXS respectively, while delivering 1.76% and 4.09% accuracy gains. CFNet-E attains 83.5% top-1 accuracy, outperforming Swin-S by 0.47% with 44.6% higher GPU throughput and 43.6% lower CPU inference latency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Adversarial Robustness in Machine Learning · Explainable Artificial Intelligence (XAI)
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China
Network Information Center, Xi’an Jiaotong University, Xi’an 710049, China
Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, Xi’an Jiaotong University, Xi’an 710049, China
MIGU Video Co., Ltd., Shanghai 201206, China
School of Continuing Education, Xi’an Jiaotong University, Xi’an 710049, China
Funding
National Science and Technology Major Project · NSFC · State Key Laboratory of Communication Content Cognition · China Knowledge Centre for Engineering Science and Technology
References (30)
Show source-provided references
1. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
2. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2017, January 22–29). Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.
3. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs10.1109/TPAMI.2017.2699184 ↗
4. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017, January 21–26). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.
5. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–23). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
6. Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., and Pang, R. (November, January 27). Searching for MobileNetV3. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
7. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–23). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
8. Ma, N., Zhang, X., Zheng, H.T., and Sun, J. (2018, January 8–14). ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01264-9_8 ↗
9. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., and Xu, C. (2020, January 5). GhostNet: More Features From Cheap Operations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00165 ↗
10. Chen, J., Kao, S.h., He, H., Zhuo, W., Wen, S., Lee, C.H., and Chan, S.H.G. (2023, January 17–24). Run, Don’t Walk: Chasing Higher FLOPS for Faster Neural Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01157 ↗
11. Iandola, F.N., Moskewicz, M.W., Ashraf, K., Han, S., Dally, W.J., and Keutzer, K. (2016, January 2–4). SqueezeNet: AlexNet-level Accuracy with 50x Fewer Parameters and <1 MB Model Size. Proceedings of the International Conference on Learning Representations, San Juan, Puerto Rico.
12. Tan, M., and Le, Q. (2019, January 9–15). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA.
13. Tan, M., and Le, Q. (2021, January 18–24). EfficientNetV2: Smaller Models and Faster Training. Proceedings of the 38th International Conference on Machine Learning, Virtual.
14. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-Excitation Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
15. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020, January 13–19). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01155 ↗
16. Li, X., Wang, W., Hu, X., and Yang, J. (2019, January 15–20). Selective Kernel Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00060 ↗
17. Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., and Sun, J. (2021, January 20–25). RepVGG: Making VGG-style ConvNets Great Again. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01352 ↗
18. Ding, X., Xia, C., Zhang, X., Chu, X., Han, J., and Ding, G. (2021). RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition. arXiv.
19. Ding, X., Guo, Y., Ding, G., and Han, J. (November, January 27). ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
20. Ding, X., Zhang, X., Han, J., and Ding, G. (2021, January 20–25). Diverse Branch Block: Building a Convolution as an Inception-like Unit. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01074 ↗
21. Vasu, P.K.A., Gabriel, J., Zhu, J., Tuzel, O., and Ranjan, A. (2023, January 17–24). MobileOne: An Improved One millisecond Mobile Backbone. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00764 ↗
22. Wu, Y., and He, K. (2018, January 8–14). Group Normalization. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01261-8_1 ↗
23. Krizhevsky, A. (2009). Learning Multiple Layers of Features from Tiny Images, University of Toronto. Technical Report.
24. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (June, January 2009;). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
25. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
26. Mehta, S., and Rastegari, M. (2022, January 25–29). MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer. Proceedings of the International Conference on Learning Representations, Online.
27. CycleMLP: A MLP-Like Architecture for Dense Visual Predictions10.1109/TPAMI.2023.3303397 ↗
28. Maaz, M., Shaker, A., Cholakkal, H., Khan, S., Zamir, S.W., Anwer, R.M., and Shahbaz Khan, F. (2022, January 23–27). EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-25082-8_1 ↗
29. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
30. Lv, X., Liang, J., Du, H., Xin, Y., Zhang, Y., and Zhang, J. (2025, January 15–18). CFNet: An Efficient Convolutional Neural Network with Channel-Focused Convolution and Cross-Channel Mixing. Proceedings of the 8th Chinese Conference on Pattern Recognition and Computer Vision, Shanghai, China.10.3390/app152312620 ↗