Adaptive Noise-Powered Diffusion Model for Efficient and Accurate Object Detection
Source abstract
Abstract
Recent advancements in object detection, particularly with DiffusionDet, have demonstrated impressive performance. However, its reliance on numerous random noise-based object candidates limits its efficiency. To overcome this limitation, we propose DifAda, a novel object detection model that incorporates adaptive noise into the diffusion framework. DifAda employs an adaptive noise mechanism that blends random noise with latent codes, enhancing object candidate utilization and improving object localization. Our model features a simplified decoder structure by employing a single decoder layer and utilizes an adaptive interaction mechanism to further refine feature representations, leading to improved performance with fewer parameters. In addition, DifAda supports flexible speed–accuracy trade-offs through adjustable sampling and iteration steps, requiring no retraining. Experimental results across multiple benchmarks, including COCO and RUOD, demonstrate that DifAda achieves competitive performance with significantly fewer object candidates and parameters. Our findings suggest that DifAda represents a step forward in efficient and scalable object detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Domain Adaptation and Few-Shot Learning · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer, Electronics and Information, Guangxi University, Nanning 530004, China
Guangxi Key Laboratory of Ocean Engineering Equipment and Technology, Beibu Gulf University, Qinzhou 535011, China
College of Naval Architecture and Ocean Engineering, Beibu Gulf University, Qinzhou 535011, China
Funding
Guangxi Key R&D Program
References (50)
Show source-provided references
1. Wei, Y., Zhao, L., Zheng, W., Zhu, Z., Zhou, J., and Lu, J. (2023, January 2–6). SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01986 ↗
2. Jiang, B., Chen, S., Xu, Q., Liao, B., Chen, J., Zhou, H., Zhang, Q., Liu, W., Huang, C., and Wang, X. (2023, January 2–6). VAD: Vectorized Scene Representation for Efficient Autonomous Driving. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00766 ↗
3. Pittner, M., Janai, J., and Condurache, A.P. (2024, January 17–21). LaneCPP: Continuous 3D Lane Detection using Physical Priors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01012 ↗
4. Lin, Z., Liu, Z., Xia, Z., Wang, X., Wang, Y., Qi, S., Dong, Y., Dong, N., Zhang, L., and Zhu, C. (2024, January 17–21). RCBEVDet: Radar-camera Fusion in Bird’s Eye View for 3D Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01414 ↗
5. Hamamci, I.E., Er, S., Simsar, E., Sekuboyina, A., Gundogar, M., Stadlinger, B., Mehl, A., and Menze, B. (2023, January 8–12). Diffusion-Based Hierarchical Multi-Label Object Detection to Analyze Panoramic Dental X-rays. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Vancouver, Canada.10.1007/978-3-031-43987-2_38 ↗
6. Orouskhani, M., Firoozeh, N., Xia, S., Mossa-Basha, M., and Zhu, C. (2023). nnDetection for Intracranial Aneurysms Detection and Localization. arXiv.
7. Xu, Z., Li, T., Liu, Y., Zhan, Y., Chen, J., and Lukasiewicz, T. (2023). PAC-Net: Multi-Pathway FPN with Position Attention Guided Connections and Vertex Distance IoU for 3D Medical Image Detection. Front. Bioeng. Biotechnol., 11.10.3389/fbioe.2023.1049555 ↗
8. Li, Y., Hou, Q., Zheng, Z., Cheng, M.M., Yang, J., and Li, X. (2023, January 2–6). Large Selective Kernel Network for Remote Sensing Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01540 ↗
9. Cof-Net: A Progressive Coarse-to-Fine Framework for Object Detection in Remote-Sensing Imagery
10. Deep Learning for Generic Object Detection: A Survey10.1007/s11263-019-01247-4 ↗
11. Object Detection with Deep Learning: A Review10.1109/TNNLS.2018.2876865 ↗
12. Object Detection in 20 Years: A Survey10.1109/JPROC.2023.3238524 ↗
13. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
14. Yang, Z., Liu, S., Hu, H., Wang, L., and Lin, S. (November, January 27). RepPoints: Point Set Representation for Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
15. Tian, Z., Shen, C., Chen, H., and He, T. (November, January 27). FCOS: Fully Convolutional One-Stage Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
16. Zhang, X., Wan, F., Liu, C., Ji, R., and Ye, Q. (2019, January 8–12). FreeAnchor: Learning to Match Anchors for Visual Object Detection. Proceedings of the 33rd International Conference on Neural Information Processing Systems 32 (NeurIPS 2019), Vancouver, Canada.
17. Bochkovskiy, A. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
18. Zhao, Y., Lv, W., Xu, S., Wei, J., Wang, G., Dang, Q., Liu, Y., and Chen, J. (2024, January 17–21). DETRs Beat YOLOs on Real-Time Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01605 ↗
19. Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L., and Shum, H.Y. (2023, January 1–5). DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection. Proceedings of the Eleventh International Conference on Learning Representations, Kigali, Rwanda.
20. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2021, January 3–7). Deformable DETR: Deformable Transformers for End-to-End Object Detection. Proceedings of the International Conference on Learning Representations, Virtual Event.
21. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23–28). End-to-End Object Detection with Transformers. Proceedings of the 16th European Conference on Computer Vision (ECCV 2020), Glasgow, UK.10.1007/978-3-030-58452-8_13 ↗
22. Sharshar, A., and Matsun, A. (2023). Innovative Horizons in Aerial Imagery: LSKNet Meets DiffusionDet for Advanced Object Detection. arXiv.
23. Sun, P., Zhang, R., Jiang, Y., Kong, T., Xu, C., Zhan, W., Tomizuka, M., Li, L., Yuan, Z., and Wang, C. (2021, January 10–25). Sparse R-CNN: End-to-End Object Detection with Learnable Proposals. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01422 ↗
24. Chen, S., Sun, P., Song, Y., and Luo, P. (2023, January 2–6). DiffusionDet: Diffusion Model for Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.01816 ↗
25. Dai, J., Li, Y., He, K., and Sun, J. (2016, January 5–10). R-FCN: Object Detection via Region-Based Fully Convolutional Networks. Proceedings of the 30th International Conference on Neural Information Processing Systems 29 (NeurIPS 2016), Barcelona, Spain.
26. Denoising Diffusion Probabilistic Models
27. Xia, B., Zhang, Y., Wang, S., Wang, Y., Wu, X., Tian, Y., Yang, W., and Van Gool, L. (2023, January 2–6). DiffIR: Efficient Diffusion Model for Image Restoration. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01204 ↗
28. Nag, S., Zhu, X., Deng, J., Song, Y.-Z., and Xiang, T. (2023, January 2–6). DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.00951 ↗
29. Flaborea, A., Collorone, L., di Melendugno, G.M.D.A., D’Arrigo, S., Prenkaj, B., and Galasso, F. (2023, January 2–6). Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.00947 ↗
30. Brempong, E.A., Kornblith, S., Chen, T., Parmar, N., Minderer, M., and Norouzi, M. (2022, January 18–24). Denoising Pretraining for Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPRW56347.2022.00462 ↗
31. Zhu, R., Pan, Y., Li, Y., Yao, T., Sun, Z., Mei, T., and Chen, C.W. (2024, January 17–21). SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00806 ↗
32. Dong, R., Yuan, S., Luo, B., Chen, M., Zhang, J., Zhang, L., Li, W., Zheng, J., and Fu, H. (2024, January 17–21). Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.02614 ↗
33. Baranchuk, D., Voynov, A., Rubachev, I., Khrulkov, V., and Babenko, A. (2022, January 25–29). Label-Efficient Semantic Segmentation with Diffusion Models. Proceedings of the International Conference on Learning Representations, Virtual Event.
34. Gu, Z., Chen, H., and Xu, Z. (2024, January 14–19). Diffusioninst: Diffusion Model for Instance Segmentation. Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10447191 ↗
35. Kolbeinsson, B., and Mikolajczyk, K. (2024, January 4–8). Multi-Class Segmentation from Aerial Views Using Recursive Noise Diffusion. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV57701.2024.00825 ↗
36. Yan, H., Zhang, H., Mu, X., Fan, J., and Zhang, Z. (November, January 29). FashionDiff: A Controllable Diffusion Model Using Pairwise Fashion Elements for Intelligent Design. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada.
37. Amit, T., Shaharbany, T., Nachmani, E., and Wolf, L. (2021). Segdiff: Image segmentation with diffusion probabilistic models. arXiv.
38. Zhang, J., Li, X., Sun, L., and Bai, C. (February, January 29). DPM-Det: Diffusion Model Object Detection Based on DPM-Solver++ Guided Sampling. Proceedings of the MultiMedia Modeling, Amsterdam, The Netherlands.
39. Ranasinghe, Y., Hegde, D., and Patel, V.M. (2024, January 17–21). MonoDiff: Monocular 3D Object Detection and Pose Estimation with Diffusion Models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01014 ↗
40. Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
41. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Part V 13.10.1007/978-3-319-10602-1_48 ↗
42. Rethinking General Underwater Object Detection: Datasets, Challenges, and Solutions10.1016/j.neucom.2022.10.039 ↗
43. Cai, Z., and Vasconcelos, N. (2018, January 18–22). Cascade R-CNN: Delving into High Quality Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00644 ↗
44. Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., and Xu, J. (2019). MMDetection: Open mmlab detection toolbox and benchmark. arXiv.
45. Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., and Girshick, R. (2023, November 11). Detectron2. Available online: https://github.com/facebookresearch/detectron2.
46. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
47. Glorot, X., and Bengio, Y. (2010, January 13–15). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy.
48. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18–24). High-Resolution Image Synthesis with Latent Diffusion Models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01042 ↗
49. Song, J., Meng, C., and Ermon, S. (2021, January 3–7). Denoising Diffusion Implicit Models. Proceedings of the International Conference on Learning Representations, Virtual Event.
50. Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., and Lin, D. (2019, January 15–20). Libra R-CNN: Towards Balanced Learning for Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00091 ↗