Using Synthetic Data to Improve and Evaluate the Tracking Performance of Construction Workers on Site
Source abstract
Abstract
Vision-based tracking systems enable the optimization of the productivity and safety management on construction sites by monitoring the workers’ movements. However, training and evaluation of such a system requires a vast amount of data. Sufficient datasets rarely exist for this purpose. We investigate the use of synthetic data to overcome this issue. Using 3D computer graphics software, we model virtual construction site scenarios. These are rendered for the use as a synthetic dataset which augments a self-recorded real world dataset. Our approach is verified by means of a tracking system. For this, we train a YOLOv3 detector identifying pedestrian workers. Kalman filtering is applied to the detections to track them over consecutive video frames. First, the detector’s performance is examined when using synthetic data of various environmental conditions for training. Second, we compare the evaluation results of our tracking system on real world and synthetic scenarios. With an increase of about 7.5 percentage points in mean average precision, our findings show that a synthetic extension is beneficial for otherwise small datasets. The similarity of synthetic and real world results allow for the conclusion that 3D scenes are an alternative to evaluate vision-based tracking systems on hazardous scenes without exposing workers to risks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Surveying and Cultural Heritage · Infrastructure Maintenance and Monitoring · BIM and Construction Integration
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Chair of Computing in Engineering, Ruhr-University Bochum, 44801 Bochum, NRW, Germany
References (32)
Show source-provided references
1. Cheng, T., Yang, J., Teizer, J., and Vela, P.A. (2010, January 14–16). Automated Construction Resource Location Tracking to Support the Analysis of Lean Principles. Proceedings of the Annual Conference of the International Group for Lean Construction, Haifa, Israel.
2. Autonomous Pro-Active Real-Time Construction Worker and Equipment Operator Proximity Safety Alert System10.1016/j.autcon.2010.02.009 ↗
3. Nasr, E., Shehab, T., and Vlad, A. (2013, January 10–13). Tracking Systems in Construction: Applications and Comparisons. Proceedings of the Annual Conference of Associated Schools of Construction, San Luis Obispo, CA, USA.
4. SightSafety: A Hybrid Information and Communication Technology System for Reducing Vehicle/Pedestrian Collisions10.1016/j.autcon.2005.09.004 ↗
5. High-Precision Identification of Contextual Information in Location-Aware Engineering Applications10.1016/j.aei.2009.04.002 ↗
6. Measuring GNSS Multipath Distributions in Urban Canyon Environments10.1109/TIM.2014.2342452 ↗
7. Application of RFID Technology to Prevention of Collision Accident with Heavy Equipment10.1016/j.autcon.2009.12.008 ↗
8. A Proactive System for Real-Time Safety Management in Construction Sites10.1016/j.autcon.2011.04.019 ↗
9. Technological Assessment of Radio Frequency Identification Technology for Indoor Localization10.1061/(ASCE)0887-3801(2009)23:4(230) ↗
10. RFID Security and Privacy: A Research Survey10.1109/JSAC.2005.861395 ↗
11. Vision-Based Detection of Unsafe Actions of a Construction Worker: Case Study of Ladder Climbing10.1061/(ASCE)CP.1943-5487.0000279 ↗
12. Vision-Based Action Recognition of Construction Workers Using Dense Trajectories10.1016/j.aei.2016.04.009 ↗
13. Construction Worker Detection in Video Frames for Initializing Vision Trackers10.1016/j.autcon.2012.06.001 ↗
14. Continuous Localization of Construction Workers via Integration of Detection and Tracking10.1016/j.autcon.2016.08.039 ↗
15. Vision-Based Detection and Visualization of Dynamic Workspaces10.1016/j.autcon.2019.04.001 ↗
16. Vierling, A., Sutjaritvorakul, T., and Berns, K. (2018, January 20–25). Crane Safety System with Monocular and Controlled Zoom Cameras. Proceedings of the International Symposium on Automation and Robotics in Construction, Berlin, Germany.10.22260/ISARC2018/0123 ↗
17. Parker, S.P. (2003). McGraw-Hill Dictionary of Scientific and Technical Terms, McGraw-Hill Education.
18. Reading Text in the Wild with Convolutional Neural Networks10.1007/s11263-015-0823-z ↗
19. Gupta, A., Vedaldi, A., and Zisserman, A. (2016, January 27–30). Synthetic Data for Text Localisation in Natural Images. Proceedings of the Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.254 ↗
20. Tremblay, J., Prakash, A., Acuna, D., Brophy, M., Jampani, V., Anil, C., To, T., Cameracci, E., Boochoon, S., and Birchfield, S. (2018, January 18–22). Training Deep Networks With Synthetic Data: Bridging the Reality Gap by Domain Randomization. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.10.1109/CVPRW.2018.00143 ↗
21. Ros, G., Sellart, L., Materzynska, J., Vazquez, D., and Lopez, A.M. (2016, January 27–30). The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes. Proceedings of the Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.352 ↗
22. Atapour-Abarghouei, A., and Breckon, T.P. (2018, January 18–22). Real-Time Monocular Depth Estimation Using Synthetic Data With Domain Adaptation via Image Style Transfer. Proceedings of the Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00296 ↗
23. Richter, S.R., Vineet, V., Roth, S., and Koltun, V. (2016). Playing for Data: Ground Truth from Computer Games. Computer Vision—ECCV 2016, Springer International Publishing.10.1007/978-3-319-46475-6_7 ↗
24. Tschentscher, M., Prus, B., and Horn, D. (2017, January 11–14). A Simulated Car-Park Environment for the Evaluation of Video-Based on-Site Parking Guidance Systems. Proceedings of the IEEE Intelligent Vehicles Symposium, Redondo Beach, CA, USA.10.1109/IVS.2017.7995933 ↗
25. Horn, D., and Houben, S. (2018, January 26–30). Evaluation of Synthetic Video Data in Machine Learning Approaches for Parking Space Classification. Proceedings of the IEEE Intelligent Vehicles Symposium, Changshu, China.10.1109/IVS.2018.8500453 ↗
26. Zhu, J., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.244 ↗
27. Wang, Q., Gao, J., Lin, W., and Yuan, Y. (2019, January 16–21). Learning From Synthetic Data for Crowd Counting in the Wild. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00839 ↗
28. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
29. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the Conference on Computer Vision and Pattern Recognition, Miami, Florida, USA.10.1109/CVPR.2009.5206848 ↗
30. Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Dollár, P. (2020, March 10). COCO—Common Objects in Context. Available online: http://cocodataset.org/#detection-eval.
31. Vision-Based Object-Centric Safety Assessment Using Fuzzy Inference: Monitoring Struck-By Accidents with Moving Objects10.1061/(ASCE)CP.1943-5487.0000562 ↗
32. Two-Dimensional Visual Tracking in Construction Scenarios: A Comparative Study10.1061/(ASCE)CP.1943-5487.0000738 ↗