Augmented Reality Maintenance Assistant Using YOLOv5
Source abstract
Abstract
Maintenance professionals and other technical staff regularly need to learn to identify new parts in car engines and other equipment. The present work proposes a model of a task assistant based on a deep learning neural network. A YOLOv5 network is used for recognizing some of the constituent parts of an automobile. A dataset of car engine images was created and eight car parts were marked in the images. Then, the neural network was trained to detect each part. The results show that YOLOv5s is able to successfully detect the parts in real time video streams, with high accuracy, thus being useful as an aid to train professionals learning to deal with new equipment using augmented reality. The architecture of an object recognition system using augmented reality glasses is also designed.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Augmented Reality Applications · Advanced Neural Network Applications · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Polytechnic of Coimbra, ISEC, 3045-093 Coimbra, Portugal
ISR, University of Coimbra, 3004-531 Coimbra, Portugal
Centre for Mechanical Engineering, Materials and Processes—CEMMPRE, 3030-788 Coimbra, Portugal
References (28)
Show source-provided references
1. Object detection with deep learning: A review10.1109/TNNLS.2018.2876865 ↗
2. Imagenet classification with deep convolutional neural networks
3. Sharma, V. (2020). Face Mask Detection Using YOLOv5 for COVID-19. [Mater’s Thesis, Master of Science in Computer Science, California State University San Marcos].
4. Tolbert, S.W. (2021, March 15). Artificial Intelligence for Real Time Threat Detection and Monitoring. Available online: https://stevenwtolbert.com/pdf/threat_detection.pdf.
5. Liu, L., Li, H., and Gruteser, M. (2019, January 21–25). Edge assisted real-time object detection for mobile augmented reality. Proceedings of the 25th Annual International Conference on Mobile Computing and Networking, Los Cabos, Mexico.10.1145/3300061.3300116 ↗
6. Deep learning-based smart task assistance in wearable augmented reality10.1016/j.rcim.2019.101887 ↗
7. Virtual and augmented reality applications in manufacturing10.3182/20130619-3-RU-3018.00637 ↗
8. Charoenseang, S., and Tonggoed, T. (2011). Human–robot collaboration with augmented reality. International Conference on Human-Computer Interaction, Springer.10.1007/978-3-642-22095-1_19 ↗
9. Kim, M., Choi, S.H., Park, K.B., and Lee, J.Y. (2021). A Hybrid Approach to Industrial Augmented Reality Using Deep Learning-Based Facility Segmentation and Depth Prediction. Sensors, 21.10.3390/s21010307 ↗
10. Kim, M., Choi, S.H., Park, K.B., and Lee, J.Y. (2019). User Interactions for Augmented Reality Smart Glasses: A comparative evaluation of visual contexts and interaction gestures. Appl. Sci., 9.10.3390/app9153171 ↗
11. Situation-dependent remote AR collaborations: Image-based collaboration using a 3D perspective map and live video-based collaboration with a synchronized VR mode10.1016/j.compind.2018.06.006 ↗
12. Bosh (2021, April 15). Bosch Connected Devices and Solutions GmbH—Interplay of Augmented Reality and IoT. Available online: https://www.bosch-connectivity.com/newsroom/blog/interplay-of-augmented-reality-and-iot/.
13. Microsoft (2021, April 15). Mixed Reality in der Automobilindustrie oder wie Bosch Mobilität Digitaler Gestaltet. Available online: https://news.microsoft.com/de-de/mixed-reality-in-der-automobilindustrie-oder-wie-bosch-mobilitaet-digitaler-gestaltet/.
14. IBM (2021, April 23). What Is a CMMS?. Available online: https://www.ibm.com/topics/what-is-a-cmms.
15. Farinha, J.M.T. (2018). Asset Maintenance Engineering Methodologies, CRC Press. [1st ed.].10.1201/9781315232867 ↗
16. Johns, R. (2021, February 10). Real Python—PyTorch vs. Tensorflow For Your Python Deep Learning Project. Available online: https://realpython.com/pytorch-vs-tensorflow.
17. Microsoft (2021, April 21). Micrisoft-HoloLens2. Available online: https://www.microsoft.com/en-us/hololens/hardware.
18. Vuzix (2021, April 21). Vuzix—Manufacturing Solutions. Available online: https://www.vuzix.com/solutions/manufacturing.
19. (2021, April 10). Andreas Geiger. Welcome to the KITTI Vision Benchmark Suite!. Available online: http://www.cvlibs.net/datasets/kitti/.
20. (2021, April 10). The Comprehensive Cars (CompCars) Dataset. Available online: http://mmlab.ie.cuhk.edu.hk/datasets/comp_cars/index.html.
21. Analytics India Magazine (2021, February 10). Analytics India Magazine—Guide to Yolov5 for Real-Time—Object Detection. Available online: https://analyticsindiamag.com/yolov5/.
22. (2021, April 21). Glenn Jocher. Available online: https://github.com/ultralytics/yolov5.
23. Buric, M., Pobar, M., and Ivasic-Kos, M. (2018, January 12–14). Ball detection using YOLO and Mask R-CNN. Proceedings of the 2018 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA.10.1109/CSCI46756.2018.00068 ↗
24. Simon, M., Amende, K., Kraus, A., Honer, J., Samann, T., Kaulbersch, H., Milz, S., and Michael Gross, H. (2019, January 16–17). Complexer-yolo: Real-time 3d object detection and tracking on semantic point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.10.1109/CVPRW.2019.00158 ↗
25. Solawetz, J., and Nelson, J. (2021, April 21). How to Train YOLOv5 on a Custom Dataset. Available online: https://blog.roboflow.com/how-to-train-yolov5-on-a-custom-dataset/.
26. Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
27. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
28. Redmon, J., and Farhadi, A. (2018, January 12–14). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2017.690 ↗