Automated Car Damage Assessment Using Computer Vision: Insurance Company Use Case
Source abstract
Abstract
Automated car damage detection using computer vision techniques has been studied using several datasets, but real cases for insurance companies are usually dependent on private methods and datasets. Furthermore, there are no metrics or standardized processes that describe the situation in which the company analyzes the customer’s images, the models used for the inference, and the results. We perform extensive experiments to show that our proposal, an ensemble of 10 deep learning detectors based on YOLOv5, improves the state-of-the-art not only in terms of typical metrics but also in terms of inference speed, allowing scalability to thousands of instances per minute. A comparison with YOLOv8 is carried out, showing the differences between both ensembles. Furthermore, a dataset called TartesiaDS, labeled under the supervision of professional appraisers from insurance companies, is available to the community for evaluation of future proposals.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Vehicle License Plate Recognition · Advanced Neural Network Applications · Autonomous Vehicle Technology and Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Valora Peritaciones, 41011 Sevilla, Spain
Department of Computer Science and Artificial Intelligence, Universidad de Sevilla, 41012 Sevilla, Spain
Department of Languages and Computer Systems, Universidad de Sevilla, 41012 Sevilla, Spain
SCORE Lab, I3US, Universidad de Sevilla, 41012 Sevilla, Spain
Funding
Valora Group · MCIN/AEI/10.13039/5011000 11033 · European Union NextGenerationEU/PRTR · NVIDIA
References (58)
Show source-provided references
1. Convolutional Neural Networks for vehicle damage detection
2. Kyu, P.M., and Woraratpanya, K. (2020, January 1–3). Car damage detection and classification. Proceedings of the 11th International Conference on Advances in Information Technology, Bangkok, Thailand.10.1145/3406601.3406651 ↗
3. Recognizing the Damaged Surface Parts of Cars in the Real Scene Using a Deep Learning Framework10.1155/2022/5004129 ↗
4. CarDD: A New Dataset for Vision-Based Car Damage Detection10.1109/TITS.2023.3258480 ↗
5. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
6. Ren, S., He, K., Girshick, R., and Sun, J. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.10.1109/TPAMI.2016.2577031 ↗
7. Cai, Z., and Vasconcelos, N. (2017). Cascade R-CNN: Delving into High Quality Object Detection. arXiv.10.1109/CVPR.2018.00644 ↗
8. Wang, W., Dai, J., Chen, Z., Huang, Z., Li, Z., Zhu, X., Hu, X., Lu, T., Lu, L., and Li, H. (2023, January 17–24). Internimage: Exploring large-scale vision foundation models with deformable convolutions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01385 ↗
9. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22–29). Deformable convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.89 ↗
10. Su, W., Zhu, X., Tao, C., Lu, L., Li, B., Huang, G., Qiao, Y., Wang, X., Zhou, J., and Dai, J. (2023, January 17–24). Towards all-in-one pre-training via maximizing multi-modal mutual information. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01525 ↗
11. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). SSD: Single shot multibox detector. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Proceedings, Part I 14.10.1007/978-3-319-46448-0_2 ↗
12. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
13. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
14. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
15. Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., and Nie, W. (2022). YOLOv6: A single-stage object detection framework for industrial applications. arXiv.
16. Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv.10.1109/CVPR52729.2023.00721 ↗
17. Yajing, L., and Zhongjian, D. (2021, January 22–24). Abnormal Behavior Detection in Crowd Scene Using YOLO and Conv-AE. Proceedings of the 2021 33rd Chinese Control and Decision Conference (CCDC), Kunming, China.10.1109/CCDC52312.2021.9602095 ↗
18. Liu, Y., and Hong, W. (2021, January 22–24). Target detection based on DB-YOLO in road environment. Proceedings of the 2021 33rd Chinese Control and Decision Conference (CCDC), Kunming, China.10.1109/CCDC52312.2021.9602650 ↗
19. A real-time vehicle detection and a novel vehicle tracking systems for estimating and monitoring traffic flow on highways10.1016/j.aei.2021.101393 ↗
20. HPMC: A multi-target tracking algorithm for the IoT10.32604/iasc.2021.016450 ↗
21. Rivero-Palacio, M., Alfonso-Morales, W., and Caicedo-Bravo, E. (2021, January 26–28). Mobile application for anemia detection through ocular conjunctiva images. Proceedings of the 2021 IEEE Colombian Conference on Applications of Computational Intelligence (ColCACI), Virtual.10.1109/ColCACI52978.2021.9469593 ↗
22. DenseSPH-YOLOv5: An automated damage detection model based on DenseNet and Swin-Transformer prediction head-enabled YOLOv5 with attention mechanism10.1016/j.aei.2023.102007 ↗
23. Multi-model ensemble with rich spatial information for object detection10.1016/j.patcog.2019.107098 ↗
24. YOLOv5 vs. YOLOv8: Performance Benchmarking in Wildfire and Smoke Detection Scenarios10.18178/joig.12.2.127-136 ↗
25. Gašparović, B., Mauša, G., Rukavina, J., and Lerga, J. (2023, January 20–23). Evaluating Yolov5, Yolov6, Yolov7, and Yolov8 in underwater environment: Is there real improvement?. Proceedings of the 2023 8th International Conference on Smart and Sustainable Technologies (SpliTech), Split and Bol, Croatia.10.23919/SpliTech58164.2023.10193505 ↗
26. Patil, K., Kulkarni, M., Sriraman, A., and Karande, S. (2017, January 18–21). Deep learning based car damage classification. Proceedings of the 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico.10.1109/ICMLA.2017.0-179 ↗
27. Shirode, A., Rathod, T., Wanjari, P., and Halbe, A. (2022, January 11–13). Car damage detection and assessment using CNN. Proceedings of the 2022 IEEE Delhi Section Conference (DELCON), New Delhi, India.10.1109/DELCON54057.2022.9752971 ↗
28. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
29. Mallikarjuna, B., and Arun Kumar, K.L. (2022). Vehicle Damage Detection and Classification Using Image Processing. Int. J. Adv. Res. Sci. Commun. Technol., 568–574.10.48175/IJARSCT-5414 ↗
30. You only look at one sequence: Rethinking transformer in vision through object detection
31. Hoang, V.D., Huynh, N.T., Tran, N., Le, K., Le, T.M.C., Selamat, A., and Nguyen, H.D. (2024). Powering AI-driven car damage identification based on VeHIDE dataset. J. Inf. Telecommun., 1–19.10.1080/24751839.2024.2367387 ↗
32. Reddy, D.A., Shambharkar, S., Jyothsna, K., Kumar, V.M., Bhoyar, C.N., and Somkunwar, R.K. (2022, January 8). Automatic Vehicle Damage Detection Classification framework using Fast and Mask Deep learning. Proceedings of the 2022 Second International Conference on Computer Science, Engineering and Applications (ICCSEA), Gunupur, India.10.1109/ICCSEA54677.2022.9936208 ↗
33. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
34. CES: Cost Estimation System for Enhancing the Processing of Car Insurance Claims10.21608/jocc.2024.339922 ↗
35. Kalshetty, J.N., Hrithik Devaiah, B., Rakshith, K., Koshy, K., and Advait, N. (2021). Analysis of Car Damage for Personal Auto Claim Using CNN. Sentimental Analysis and Deep Learning: Proceedings of ICSADL 2021, Springer.10.1007/978-981-16-5157-1_25 ↗
36. Bahhar, C., Ksibi, A., Ayadi, M., Jamjoom, M.M., Ullah, Z., Soufiene, B.O., and Sakli, H. (2023). Wildfire and Smoke Detection Using Staged YOLO Model and Ensemble CNN. Electronics, 12.10.3390/electronics12010228 ↗
37. Doshi, K., and Yilmaz, Y. (2020, January 10–13). Road damage detection using deep ensemble learning. Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA.10.1109/BigData50022.2020.9377774 ↗
38. Casado-García, Á., and Heras, J. (September, January 29). Ensemble methods for object detection. Proceedings of the ECAI 2020, Santiago de Compostela, Spain.
39. Lee, J., Lee, S.K., and Yang, S.I. (2018, January 17–19). An ensemble method of CN models for object detection. Proceedings of the 2018 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea.10.1109/ICTC.2018.8539396 ↗
40. Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L.M., and Shum, H.Y. (2022). DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection. arXiv.
41. Liu, S., Li, F., Zhang, H., Yang, X., Qi, X., Su, H., Zhu, J., and Zhang, L. (2022). DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR. arXiv.
42. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common objects in context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part V 13.10.1007/978-3-319-10602-1_48 ↗
43. Kennedy, J., and Eberhart, R. (December, January 29). Particle swarm optimization. Proceedings of the ICNN’95—International Conference on Neural Networks, Perth, WA, Australia.
44. Pedersen, M.E.H. (2010). Good parameters for particle swarm optimization. Hvass Lab. Cph. Den. Tech. Rep. HL1001, 1551–3203.
45. A deep-learning–based antifraud system for car-insurance claims10.1016/j.eswa.2023.120644 ↗
46. Singh, R., Ayyar, M.P., Pavan, T.V.S., Gosain, S., and Shah, R.R. (2019, January 11–13). Automating car insurance claims using deep learning techniques. Proceedings of the 2019 IEEE fifth international conference on multimedia big data (BigMM), Singapore.10.1109/BigMM.2019.00-25 ↗
47. Neo, T. (2023, February 13). Car Damage Detective. Available online: https://github.com/neokt/car-damage-detective.
48. Patel, N., Shinde, S., and Poly, F. (2020, January 28–29). Automated damage detection in operational vehicles using mask R-CNN. Proceedings of the Advanced Computing Technologies and Applications: Proceedings of 2nd International Conference on Advanced Computing Technologies and Applications—ICACTA 2020, Mumbai, Maharashtra.10.1007/978-981-15-3242-9_54 ↗
49. Balci, B., Artan, Y., Alkan, B., and Elihos, A. (2019, January 3–5). Front-View Vehicle Damage Detection using Roadway Surveillance Camera Images. Proceedings of the VEHITS, Crete, Greece.10.5220/0007724600002179 ↗
50. Waqas, U., Akram, N., Kim, S., Lee, D., and Jeon, J. (2020, January 26–29). Vehicle damage classification and fraudulent image detection including moiré effect using deep learning. Proceedings of the 2020 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), London, ON, Canada.10.1109/CCECE47787.2020.9255806 ↗
51. De Deijn, J. (2018, January 29). Automatic car damage recognition using convolutional neural networks. Proceedings of the 2018 Internship Report MSc Business Analytics, Amsterdam, The Netherlands.
52. Dwivedi, M., Malik, H.S., Omkar, S., Monis, E.B., Khanna, B., Samal, S.R., Tiwari, A., and Rathi, A. (2021). Deep learning-based car damage classification and detection. Advances in Artificial Intelligence and Data Engineering: Select Proceedings of AIDE 2019, Springer.10.1007/978-981-15-3514-7_18 ↗
53. Widjojo, D., Setyati, E., and Kristian, Y. (2022, January 19–21). Integrated Deep Learning System for Car Damage Detection and Classification Using Deep Transfer Learning. Proceedings of the 2022 IEEE 8th Information Technology International Seminar (ITIS), Surabaya, Indonesia.10.1109/ITIS57155.2022.10010292 ↗
54. Yan, B., Jiang, Y., Sun, P., Wang, D., Yuan, Z., Luo, P., and Lu, H. (2022, January 23–24). Towards grand unification of object tracking. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19803-8_43 ↗
55. Cheng, B., Choudhuri, A., Misra, I., Kirillov, A., Girdhar, R., and Schwing, A.G. (2021). Mask2former for video instance segmentation. arXiv.
56. Meinhardt, T., Kirillov, A., Leal-Taixe, L., and Feichtenhofer, C. (2022, January 18–24). Trackformer: Multi-object tracking with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00864 ↗
57. Zeng, F., Dong, B., Zhang, Y., Wang, T., Zhang, X., and Wei, Y. (2022). MOTR: End-to-End Multiple-Object Tracking with Transformer. arXiv.10.1007/978-3-031-19812-0_38 ↗
58. Yan, F., Luo, W., Zhong, Y., Gan, Y., and Ma, L. (2023). Bridging the Gap Between End-to-end and Non-End-to-end Multi-Object Tracking. arXiv.