An Improved Small Target Segmentation Model Based on Mask Dino
Source abstract
Abstract
To address the issue of low segmentation accuracy for small objects in the Mask Dino segmentation method, we propose an improved small object segmentation model called FFMask Dino. Initially, we introduce scaled cosine attention and the log-cpb method into the Swin Transformer backbone network. Subsequently, by adjusting the network structure, we enhance the feature extraction process, which helps the model maintain generalization across different datasets and reduces the risk of overfitting. Lastly, we propose the FFPN module to optimize the pathways for feature fusion and transmission. The improved FPN reduces unnecessary computations, accelerates model inference speed, and integrates multi-scale feature details and high-level semantic information to complement object features, thereby enhancing model segmentation accuracy. Experimental results demonstrate that the improved segmentation model achieves a mean Intersection over Union (mIoU) of 42.15% on the ADE20K dataset for semantic segmentation tasks, representing a 0.96% increase compared to the Mask Dino method. On the CoCo dataset for instance segmentation tasks, with the Swin Transformer backbone, the Mask AP and Box AP are 47.10 and 52.60, respectively, showing improvements of 1% and 1.3% over the Mask Dino method. With the ResNet-50 backbone, the Mask AP and Box AP are 40.00 and 44.10, respectively, with improvements of 0.5% and 0.9% over the Mask Dino method. For the CoCo dataset’s panoptic segmentation tasks, with the Swin Transformer backbone, the PQ is 54.95, showing a 0.4% increase over the Mask Dino method. With the ResNet-50 backbone, the PQ is 46.93, showing a 0.9% increase over the Mask Dino method. These results effectively demonstrate the improved accuracy and precision of Mask Dino in segmenting small objects across various segmentation tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Industrial Vision Systems and Defect Detection · Advanced Measurement and Detection Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer and Information Engineering, Jiangxi Agricultural University, Nanchang 330045, China
School of Software, Jiangxi Agricultural University, Nanchang 330045, China
Funding
National Natural Science Foundation of China · Natural Science Foundation of Jiangxi Province
References (39)
Show source-provided references
1. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully Convolutional Networks for Semantic Segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
2. Per-Pixel Classification is Not All You Need for Semantic Segmentation
3. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). End-to-End Object Detection with Transformers. Computer Vision—ECCV 2020, Proceedings of the 16th European Conference, Glasgow, UK, 23–28 August 2020, Springer International Publishing.10.1007/978-3-030-58452-8_13 ↗
4. Vaswani, A. (2017, January 4–9). Attention is all you need. Advances in Neural Information Processing Systems. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
5. Cheng, B., Misra, I., Schwing, A., Kirillov, A., and Girdhar, R. (2022, January 18–24). Masked-Attention Mask Transformer for Universal Image Segmentation. Proceedings of the IEEE/CVF 2022: Conference on Computer Vision and Pattern Recognition, New Oreleans, LA, USA.10.1109/CVPR52688.2022.00135 ↗
6. Li, F., Zhang, H., Xu, H., Liu, S., Zhang, L., Ni, L.M., and Shum, H.Y. (2023, January 17–24). Mask dino: Towards a Unified Transformer-Based FRAMEWORK for object Detection and Segmentation. Proceedings of the IEEE/CVF 2023: Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00297 ↗
7. Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L.M., and Shum, H.-Y. (2022). DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection. arXiv.
8. Multi mode feature fusion network for benign and malignant classification of pulmonary nodules10.54254/2755-2721/8/20230147 ↗
9. Negative obstacle detection of open-pit mining area roads based on multi feature fusion
10. Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H.S. (2021, January 20–25). Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00681 ↗
11. A monocular 3D object detection method based on the fusion of depth and instance segmentation
12. He, K., Gkioxari, G., Dollár, P., and Girschick, R. (2017, January 21–26). Mask R-Cnn. Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/ICCV.2017.322 ↗
13. Remote Sensing Image Semantic Segmentation Method Based on U-Net Feature Fusion Optimization Strategy
14. A review of Transformer based U-shaped medical image segmentation networks
15. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10–17). Swintransformer: Hierarchical Vision Transformer Using Shifted Windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, Canada.10.1109/ICCV48922.2021.00986 ↗
16. Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
17. A semantic segmentation method for remote sensing images based on multi-scale feature fusion
18. Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18–23). Path Aggregation Network for Instance Segmentation. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00913 ↗
19. Ghiasi, G., Lin, T.Y., and Le, Q.V. (2019, January 15–20). Nas-fpn: Learning Scalable Feature Pyramid Architecture for Object Detection. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00720 ↗
20. Tan, M., Pang, R., and Le, Q.V. (2017, January 13–19). Efficientdet: Scalable and Efficient Object Detection. Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
21. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
22. Single soldier camouflage camouflage small target detection based on edge filling
23. Exploring thelimits of transfer learning witha unified text-to-text transformer
24. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv.
25. Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., and Dong, L. (2022, January 19–20). Swintransformer v2: Scaling Up Capacity Andresolution. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01170 ↗
26. Research on Infrared Small Target Segmentation Algorithm Based on Transformer
27. Ke, G., He, D., and Liu, T.Y. (2020). Rethinking Positional Encoding in Language Pre-training. arXiv.
28. Tinytarget detection based on improved Varifocal Net
29. Refined identification and classification of crop planting structure at plotscale based on Segformer network
30. Mixed-integer linear programming formulations andcolumn generation algorithms for the Minimum Normalized Cuts problem on networks10.1016/j.ejor.2024.02.033 ↗
31. Remote sensing image semantic segmentation algorithm based on selfattention multi-scale feature fusion
32. Feature-fused attention for remote sensing image target segmentation
33. High Speed Molten Pool Image Detection Based on Weighted Fusion and Parameter Extraction
34. Typical small target detection on water surfaces fusing attention and multi-scale features
35. Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., and Torralba, A. (2017, January 21–26). Scene Parsing Through ADE20k Dataset. Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.544 ↗
36. Lint, Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
37. Mashayekhi, M., Majd, S.A., Amiramjadi, A., and Mashayekhi, B. (2023). Radious: Unveiling the Enigma of Dental Radiology with BEIT Adaptor and Mask2Former in Semantic Segmentation. arXiv.
38. Zheng, J. (2024, January 19–21). ConvNeXt-Mask2Former: A Semantic Segmentation Model for Land Classification in Remote Sensing Images. Proceedings of the 2024 5th International Conference on Computer Vision, Image and Deep Learning (CVIDL), Zhuhai, China.10.1109/CVIDL62147.2024.10603728 ↗
39. Small target detection algorithm for aerial photography based on location awareness and cross-layer feature fusion