Layer-Wise Query Selection to Eliminate Redundant Queries in DETR
Source abstract
Abstract
Recent advancements in the detection Transformer model have demonstrated remarkable accuracy in real-time object detection using an end-to-end approach. DETR leverages the concept of object queries, which act as “questions” to determine the presence and location of objects. However, the excessive number of object queries significantly increases computational complexity, leading to higher training and inference times, greater memory consumption, and increased costs. To address this issue, this paper introduces the selected query detection Transformer, a novel approach that optimizes the selection of object queries in the decoder. By progressively filtering and reducing unnecessary queries, the selected query detection Transformer maintains model performance while significantly reducing computational overhead, resulting in faster training and inference times.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning and Data Classification · Neural Networks and Applications · Algorithms and Data Compression
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Pattern Recognition and Machine Learning Laboratory, School of Computing, Gachon University, Seungnam 13306, Republic of Korea
Funding
Defense Acquisition Program Administration (DAPA) · Korea government (MSIT)
References (31)
Show source-provided references
1. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23–28). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58452-8_13 ↗
2. Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L.M., and Shum, H.Y. (2022). DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection. arXiv.
3. Zong, Z., Song, G., and Liu, Y. (2023, January 1–6). Detrs with collaborative hybrid assignments training. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00621 ↗
4. Yao, Z., Ai, J., Li, B., and Zhang, C. (2021). Efficient detr: Improving end-to-end object detector with dense prior. arXiv.
5. Zhao, Y., Lv, W., Xu, S., Wei, J., Wang, G., Dang, Q., Liu, Y., and Chen, J. (2024, January 16–22). Detrs beat yolos on real-time object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01605 ↗
6. Rank-DETR for high quality object detection
7. Zhang, S., Wang, X., Wang, J., Pang, J., Lyu, C., Zhang, W., Luo, P., and Chen, K. (2023, January 17–24). Dense Distinct Query for End-to-End Object Detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00708 ↗
8. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2021, January 3–7). Deformable Detr: Deformable Transformers for End-to-End Object Detection. Proceedings of the ICLR 2021—9th International Conference on Learning Representations, Virtual Event.
9. Liu, S., Li, F., Zhang, H., Yang, X., Qi, X., Su, H., Zhu, J., and Zhang, L. (2022, January 25–29). Dab-Detr: Dynamic Anchor Boxes Are Better Queries for Detr. Proceedings of the ICLR 2022—10th International Conference on Learning Representations, Virtual Event.
10. Li, F., Zhang, H., Liu, S., Guo, J., Ni, L.M., and Zhang, L. (2022, January 18–24). DN-DETR: Accelerate DETR Training by Introducing Query DeNoising. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01325 ↗
11. Jia, D., Yuan, Y., He, H., Wu, X., Yu, H., Lin, W., Sun, L., Zhang, C., and Hu, H. (2023, January 17–24). Detrs with hybrid matching. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01887 ↗
12. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
13. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
14. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7–12). Faster R-CNN: Towards real-time object detection with region proposal networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
15. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
16. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Proceedings, Part I 14.10.1007/978-3-319-46448-0_2 ↗
17. Zhang, S., Chi, C., Yao, Y., Lei, Z., and Li, S.Z. (2020, January 13–19). Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00978 ↗
18. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (November, January 27). Centernet: Keypoint triplets for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
19. FCOS: A simple and strong anchor-free object detector
20. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Łukasz, K., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
21. Meng, D., Chen, X., Fan, Z., Zeng, G., Li, H., Yuan, Y., Sun, L., and Wang, J. (2021, January 11–17). Conditional detr for fast training convergence. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00363 ↗
22. Wang, Y., Zhang, X., Yang, T., and Sun, J. (March, January 22). Anchor detr: Query design for transformer-based detector. Proceedings of the AAAI Conference on Artificial Intelligence, Online.
23. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22–29). Deformable convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.89 ↗
24. Zhu, X., Hu, H., Lin, S., and Dai, J. (2019, January 15–20). Deformable convnets v2: More deformable, better results. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00953 ↗
25. Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., and Sun, J. (2021, January 20–25). Repvgg: Making vgg-style convnets great again. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01352 ↗
26. Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18–22). Path Aggregation Network for Instance Segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00913 ↗
27. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
28. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014). Microsoft COCO: Common objects in context. Proceedings of the Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer.10.1007/978-3-319-10602-1_48 ↗
29. Gao, P., Zheng, M., Wang, X., Dai, J., and Li, H. (2021, January 11–17). Fast Convergence of DETR with Spatially Modulated Co-Attention. Proceedings of the IEEE International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00360 ↗
30. Dai, X., Chen, Y., Yang, J., Zhang, P., Yuan, L., and Zhang, L. (2021, January 11–17). Dynamic DETR: End-to-End Object Detection with Dynamic Attention. Proceedings of the IEEE International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00298 ↗
31. Lv, W., Xu, S., Zhao, Y., Wang, G., Wei, J., Cui, C., Du, Y., Dang, Q., and Liu, Y. (2023). DETRs beat YOLOs on real-time object detection (2023). arXiv.