Generation and Annotation of Simulation-Real Ship Images for Convolutional Neural Networks Training and Testing
Source abstract
Abstract
Large amounts of high-quality image data are the basis and premise of the high accuracy detection of objects in the field of convolutional neural networks (CNN). It is challenging to collect various high-quality ship image data based on the marine environment. A novel method based on CNN is proposed to generate a large number of high-quality ship images to address this. We obtained ship images with different perspectives and different sizes by adjusting the ships’ postures and sizes in three-dimensional (3D) simulation software, then 3D ship data were transformed into 2D ship image according to the principle of pinhole imaging. We selected specific experimental scenes as background images, and the target ships of the 2D ship images were superimposed onto the background images to generate “Simulation–Real” ship images (named SRS images hereafter). Additionally, an image annotation method based on SRS images was designed. Finally, the target detection algorithm based on CNN was used to train and test the generated SRS images. The proposed method is suitable for generating a large number of high-quality ship image samples and annotation data of corresponding ship images quickly to significantly improve the accuracy of ship detection. The annotation method proposed is superior to the annotation methods that label images with the image annotation software of Label-me and Label-img in terms of labeling the SRS images.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Remote Sensing and LiDAR Applications · Maritime Navigation and Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
National Engineering Research Center for Water Transport Safety, Wuhan University of Technology, Wuhan 430063, China
College of Technology, Hubei Engineering University, Xiaogan 432000, China
Funding
National Natural Science Foundation of China · Teaching Research Project of College of Technology, Hubei Engineering University · Scientific Research Project of College of Technology, Hubei Engineering University
References (41)
Show source-provided references
1. Westward ho—The China dream and ‘one belt, one road’: Chinese foreign policy under Xi Jinping10.1111/1468-2346.12660 ↗
2. The Geopolitics of China’s Maritime Silk Road Initiative10.1080/14650045.2017.1291503 ↗
3. Deep learning for autonomous ship-oriented small ship detection10.1016/j.ssci.2020.104812 ↗
4. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 20–23). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
5. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition10.1109/TPAMI.2015.2389824 ↗
6. Girshick, R.B. (2015, January 13–16). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
7. Dai, J., Li, Y., and He, K. (2016). R-fcn: Object detection via region-based fully convolutional networks. arXiv, Available online: https://arxiv.org/pdf/1605.06409.pdf.
8. Ren, S.Q., He, K.M., and Girshick, R. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv, Available online: https://arxiv.org/pdf/1506.01497.pdf.
9. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
10. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, Faster, Stronger. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
11. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
12. Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
13. Liu, W., Anguelov, D., and Erhan, D. (2016, January 8–16). Ssd: Single shot multibox detector. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
14. Long, J., Shelhamer, E., and Darrell, T. (2015, January 8–10). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
15. He, K.M., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask r-cnn. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
16. Ye, F., and Yang, J. (2021). A Deep Neural Network Model for Speaker Identification. Appl. Sci., 11.10.3390/app11083603 ↗
17. Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the 13th European Conference on Computer Vision, Zurich, Switzerland.
18. Kuznetsova, A., Rom, H., and Alldrin, N. (2018). The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. arXiv, Available online: https://doi.org/10.1007/s11263-020-01316-z.10.1007/s11263-020-01316-z ↗
19. ImageNet classification with deep convolutional neural networks10.1145/3065386 ↗
20. Deng, J., Dong, W., and Socher, R. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 22th IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL, USA.10.1109/CVPR.2009.5206848 ↗
21. The pascal visual object classes (voc) challenge10.1007/s11263-009-0275-4 ↗
22. Prasad, D.K., Prasath, C.K., and Rajan, D. (2016). Challenges in video based object detection in maritime scenario using computer vision. arXiv.
23. Cubuk, E.D., Zoph, B., and Mane, D. (2018). Autoaugment: Learning augmentation policies from data. arXiv.10.1109/CVPR.2019.00020 ↗
24. Cubuk, E.D., Zoph, B., and Shlens, J. (2020, January 14–19). Randaugment: Practical automated data augmentation with a reduced search space. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00359 ↗
25. Buslaev, A., Iglovikov, V.I., and Khvedchenya, E. (2020). Albumentations: Fast and flexible image augmentations. Information, 11.10.3390/info11020125 ↗
26. SMOTE: Synthetic minority over-sampling technique10.1613/jair.953 ↗
27. Goodfellow, I.J., Pouget-Abadie, J., and Mirza, M. (2014). Generative adversarial networks. arXiv.
28. Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv.
29. Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv.
30. Zhu, J.Y., Park, T., and Isola, P. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
31. Liu, M.Y., and Tuzel, O. (2016). Coupled generative adversarial networks. arXiv.
32. Ledig, C., Theis, L., and Huszár, F. (2017, January 21–26). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
33. Karras, T., Aila, T., and Laine, S. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv.
34. Arjovsky, M., Chintala, S., and Bottou, L.E.O. (2017). Wasserstein gan. arXiv.
35. Zhang, H., Goodfellow, I., and Metaxas, D. (2019, January 3–6). Self-attention generative adversarial networks. Proceedings of the 2019 International Conference on Machine Learning(PMLR), Ghent, Belgium.
36. Brock, A., Donahue, J., and Simonyan, K. (2018). Large scale GAN training for high fidelity natural image synthesis. arXiv.
37. Karras, T., Laine, S., and Aila, T. (2019, January 16–19). A style-based generator architecture for generative adversarial networks. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Los Angeles, CA, USA.10.1109/CVPR.2019.00453 ↗
38. Dewi, C., Chen, R.C., and Liu, Y.T. (2021). Various Generative Adversarial Networks Model for Synthetic Prohibitory Sign Image Gen-eration. Appl. Sci., 11.10.3390/app11072913 ↗
39. A robust clustering algorithm using spatial fuzzy C-means for brain MR images10.1016/j.eij.2019.10.005 ↗
40. Image edge detection: A new approach based on fuzzy entropy and fuzzy divergence10.1007/s40815-020-01030-5 ↗
41. Imgaug: Image augmentation for machine learning experiments