Research Status and Prospect of the Key Technologies for Environment Perception of Intelligent Excavators
Source abstract
Abstract
With the urgent need of the industry and the continuous development of artificial intelligence, research into intelligent excavators has achieved certain progress. However, intelligent excavators often face strong vibrations, dense dust, and complex objectives. These have brought severe challenges to environmental perception, and are important research difficulties that must be overcome in realizing the practical engineering applications of intelligent excavators. Many researchers have studied these problems in reducing vibration and dust noise for light detection and ranging (LiDAR) scanners, multi-sensor information fusion, and the segmentation and recognition of 3D scenes. This paper reviews the research status of these key technologies and discusses their development trends.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Infrastructure Maintenance and Monitoring · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechatronics Engineering, Henan University of Science and Technology, Luoyang 471023, China
School of Control Science and Engineering, Dalian University of Technology, Dalian 116024, China
Funding
National Natural Science Foundation of China · Henan Province Science and Technology Research and Development Plan Joint Fund Project · Natural Science Foundation Program of Liaoning Province · Science and Technology Major Project of Shanxi Province · Science and Technology Major Project of Xinjiang Uygur Autonomous Region
References (100+)
Show source-provided references
1. Intelligent Identification for Working-Cycle Stages of Excavator Based on Main Pump Pressure10.1016/j.autcon.2019.102991 ↗
2. Review of Hybrid Electric Systems for Construction Machinery10.1016/j.autcon.2018.04.005 ↗
3. Yusof, A.A., Saadun, M.N.A., Sulaiman, H., and Sabaruddin, S.A. (2016, January 25–27). The Development of Tele-Operated Electro-Hydraulic Actuator (T-EHA) for Mini Excavator Tele-Operation. Proceedings of the 2016 2nd IEEE International Symposium on Robotics and Manufacturing Automation (ROMA), Ipoh, Malaysia.10.1109/ROMA.2016.7847800 ↗
4. On Modeling, Identification, and Control of a Heavy-Duty Electrohydraulic Harvester Manipulator10.1109/TMECH.2003.812820 ↗
5. Control of a Heavy-Duty Robotic Excavator Using Time Delay Control with Integral Sliding Surface10.1016/S0967-0661(02)00027-8 ↗
6. Bradley, D.A., and Seward, D.W. (1995, January 13–15). Developing Real-Time Autonomous Excavation-the LUCIE Story. Proceedings of the 1995 34th IEEE Conference on Decision and Control, New Orleans, LA, USA.
7. Seward, D., Margrave, F., Sommerville, I., and Morrey, R. (1996, January 22–28). LUCIE the Robot Excavator-Design for System Safety. Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA.
8. Safety Analysis of Autonomous Excavator Functionality10.1016/S0951-8320(00)00045-4 ↗
9. A Robotic Excavator for Autonomous Truck Loading10.1023/A:1008914201877 ↗
10. Patrick, R. (1999). Adaptive Motion Planning for Autonomous Mass Excavation. [Ph.D. Thesis, Carnegie Mellon University].
11. Impedance Control of a Hydraulically Actuated Robotic Excavator10.1016/S0926-5805(00)00056-X ↗
12. A Control Architecture for Robotic Excavation in Construction10.1111/j.1467-8667.2004.00335.x ↗
13. Autonomous Excavation Using a Rope Shovel10.1002/rob.20132 ↗
14. Gu, Y.M. (2009). Research on Image Processing Technology in Vision Systems of Intelligent Excavators. [Ph.D. Thesis, Northeastern University].
15. Development of the Autonomous Hydraulic Excavator Prototype Using 3-D Information for Motion Planning and Control10.9746/sicetr.48.488 ↗
16. Research on Automatic Trajectory Control of a New Intelligent Excavator
17. Development of Optimized Point Cloud Merging Algorithms for Accurate Processing to Create Earthwork Site Models10.1016/j.autcon.2013.01.004 ↗
18. Energy-Minimum Optimization of the Intelligent Excavating Process for Large Cable Shovel through Trajectory Planning10.1007/s00158-018-2011-6 ↗
19. Zhang, L.Z. (2019). Trajectory Planning and Task Decision Making of Intelligent Excavator Robots. [Master’s Thesis, Zhejiang University].
20. Sun, H.Y. (2021). Identification of Typical Operational Stages and Trajectory Control of Work Devices for Intelligent Excavators. [Master’s Thesis, Harbin Institute of Technology].
21. Liu, W.S. (2022). 3D Point Cloud-Based Perception of the Working Environment for Intelligent Mining Excavator. [Master’s Thesis, Dalian University of Technology].
22. Data-Driven Excavation Trajectory Planning for Unmanned Mining Excavator10.1016/j.autcon.2024.105395 ↗
23. Development of a Vibration Monitoring Recorder for Mining Excavators
24. Denoising of Vibration Signals in the Hydraulic Excavator Backhoe Digging Process
25. Analysis and Optimization Design of Noise and Vibration Characteristics in Excavator Cabins
26. Vibration Study of Lifting Mechanism in Large Mining Excavators
27. Ma, H., and Wu, J. (2012, January 11–13). Analysis of Positioning Errors Caused by Platform Vibration of Airborne LiDAR System. Proceedings of the 2012 8th IEEE International Symposium on Instrumentation and Control Technology, London, UK.10.1109/ISICT.2012.6291650 ↗
28. Image Processing of Airborne Synthetic Aperture Laser Radar Under Vibration Conditions10.3724/SP.J.1300.2014.13132 ↗
29. Hu, X., Li, D., Du, J., Ma, M., and Zhou, J. (2016, January 10–15). Vibration Estimation of Synthetic Aperture Lidar Based on Division of Inner View Field by Two Detectors along Track. Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium, Beijing, China.10.1109/IGARSS.2016.7730190 ↗
30. Research on Impact Vibration Noise Reduction Method for Vehicle-Borne Laser Radar Based on Wavelet Analysis
31. Study on Laser Radar Imaging Blur Induced by Residual Vibration
32. Analysis of the Effect of Linear Vibration on Synthetic Aperture Laser Radar Imaging10.3788/AOS201232.0428001 ↗
33. E Influence of Angular Vibration on Synthetic Aperture Laser Radar Imaging10.3724/SP.J.1010.2011.00571 ↗
34. Design and Implementation of Vibration Isolation System for Mobile Doppler Wind LIDAR10.3807/JOSK.2013.17.1.103 ↗
35. Vibration Protection of Sensitive Components of Infrared Equipment in Harsh Environments10.1155/2001/501572 ↗
36. Vibration Protection of Critical Components of Electronic Equipment in Harsh Environmental Conditions10.1006/jsvi.2002.5164 ↗
37. Design of Composite Vibration Isolation for Airborne Electro-Optical Surveillance Platform10.3788/OPE.20111901.0083 ↗
38. Vibration Isolation Design of Vehicle-Borne Doppler Wind Lidar System
39. Li, Y. (2014). Studies on Environmental Adaptability of Airborne Electro-Optical Reconnaissance Platform. [Ph.D. Thesis, University of Chinese Academy of Sciences].
40. Try, P., and Gebhard, M. (2023). A Vibration Sensing Device Using a Six-Axis IMU and an Optimized Beam Structure for Activity Monitoring. Sensors, 23.10.3390/s23198045 ↗
41. Sun, Z., Wu, Z., Ren, X., and Zhao, Y. (2021, January 27–31). IMU Sensor-Based Vibration Measuring of Ship Sailing. Proceedings of the 2021 IEEE 11th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems, Jiaxing, China.10.1109/CYBER53097.2021.9588268 ↗
42. Kaswekar, P., and Wagner, J.F. (2015, January 22–23). Sensor Fusion Based Vibration Estimation Using Inertial Sensors for a Complex Lightweight Structure. Proceedings of the 2015 DGON Inertial Sensors and Systems Symposium, Karlsruhe, Germany.10.1109/InertialSensors.2015.7314265 ↗
43. Jin, L. (2024). Studies on Passive Vibration Isolation Method Using Second-Order Spring-Damping System for Lidar SLAM Mapping in Orchards. [Master’s Thesis, Hangzhou University of Electronic Science and Technology].
44. Wang, H.N. (2020). Design, Modeling, and Experimental Validation of Rubber Vibration Isolator for UAV LiDAR. [Master’s Thesis, Northeastern University].
45. Studies on Multi-Modal Vibration Suppression Algorithm for Hydraulic Excavator Track Chassis
46. Study on Noise and Vibration Performance of Excavator Based on Substructure Power Flow
47. When the Dust Settles: The Four Behaviors of LiDAR in the Presence of Fine Airborne Particulates10.1002/rob.21701 ↗
48. Laux, T.E., and Chen, C. (2014, January 7–8). 3D Flash LIDAR Vision Systems for Imaging in Degraded Visual Environments. Proceedings of the Degraded Visual Environments: Enhanced, Synthetic, and External Vision Solutions, Baltimore, MD, USA.10.1117/12.2068339 ↗
49. A Method of Background Noise Reduction in Lidar Data10.1007/s00340-013-5447-9 ↗
50. Cheng, Y., Cao, J., Hao, Q., Xiao, Y., Zhang, F., Xia, W., Zhang, K., and Yu, H. (2017). A Novel De-Noising Method for Improving the Performance of Full-Waveform LiDAR Using Differential Optical Path. Remote Sens., 9.10.3390/rs9111109 ↗
51. Goodin, C., Durst, P.J., Prevost, Z.T., and Compton, P.J. (2013, January 1–2). A Probabilistic Model for Simulating the Effect of Airborne Dust on Ground-Based LIDAR. Proceedings of the Active and Passive Signatures IV, Baltimore, MD, USA.10.1117/12.2016434 ↗
52. Performance of Laser and Radar Ranging Devices in Adverse Environmental Conditions10.1002/rob.20310 ↗
53. Bilateral Filtering Denoising Algorithm for Point Cloud Models
54. Bilateral Filtering Point Cloud Denoising Algorithm Based on Feature Selection
55. Feature-Based Classification for 3D Point Data Denoising10.3788/OPE.20162406.1465 ↗
56. An EEMD-SVD Method Based on Gray Wolf Optimization Algorithm for Lidar Signal Noise Reduction10.1080/01431161.2023.2249597 ↗
57. Noise Reduction of LiDAR Signal via Local Mean Decomposition Combined with Improved Thresholding Method10.1109/ACCESS.2020.3003597 ↗
58. Wind Lidar Signal Denoising Method Based on Singular Value Decomposition and Variational Mode Decomposition10.1364/AO.442716 ↗
59. De-Noising and Retrieving Algorithm of Mie Lidar Data Based on the Particle Filter and the Fernald Method10.1364/OE.23.026509 ↗
60. Development of a 3D Local Terrain Modeling System of Intelligent Excavation Robot10.1007/s12205-016-0309-8 ↗
61. Graeter, J., Wilczynski, A., and Lauer, M. (2018, January 1–5). LIMO: Lidar-Monocular Visual Odometry. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain.10.1109/IROS.2018.8594394 ↗
62. Shin, Y.S., Park, Y.S., and Kim, A. (2018, January 21–25). Direct Visual SLAM Using Sparse Depth for Camera-LiDAR System. Proceedings of the 2018 IEEE International Conference on Robotics and Automation, Brisbane, QLD, Australia.10.1109/ICRA.2018.8461102 ↗
63. De Silva, V., Roche, J., and Kondoz, A. (2018). Fusion of LiDAR and Camera Sensor Data for Environment Sensing in Driverless Vehicles, Preprint.
64. Velodyne LiDAR and Monocular Camera Data Fusion for Depth Map and 3D Reconstruction
65. Lee, H., Song, S., and Jo, S. (2016, January 1–4). 3D Reconstruction Using a Sparse Laser Scanner and a Single Camera for Outdoor Autonomous Vehicle. Proceedings of the 2016 IEEE 19th International Conference on Intelligent Transportation Systems, Rio de Janeiro, Brazil.10.1109/ITSC.2016.7795619 ↗
66. Building an Omnidirectional 3-D Color Laser Ranging System Through a Novel Calibration Method10.1109/TIE.2018.2890506 ↗
67. 3D Reconstruction of Building Facade with Fused Data of Terrestrial LiDAR Data and Optical Image10.1016/j.ijleo.2015.11.147 ↗
68. An Improved Building Boundary Extraction Algorithm Based on Fusion of Optical Imagery and LIDAR Data10.1016/j.ijleo.2013.03.045 ↗
69. Localization in Unstructured Environments Based on Camera and Swing LiDAR Fusion
70. SLAM for Mobile Robots Based on LiDAR and Stereo Vision
71. Pedestrian Detection Method Based on Fusion of Radar and Vision Information
72. Road Surface Vehicle Detection Based on Multi-Sensor Fusion
73. Shao, P. (2024). Research on Multi-UAV SLAM Technology Based on Multi-Sensor Information Fusion. [Master’s Thesis, University of Electronic Science and Technology].
74. Zhao, Q., and Chen, H. (2024). Robot Motion Trajectory Tracking Based on Sensor Information Fusion. Inf. Technol., 182–186.
75. Quan, M. (2021). Research on Monocular Vision SLAM Algorithm Based on Multi-Sensor Information Fusion. [Ph.D. Thesis, Harbin Institute of Technology].
76. Zhao, S. (2020). Research on Multi-Sensor Information Fusion Method for Intelligent Vehicles. [Master’s Thesis, Jilin University].
77. Yang, L. (2004). Research on Target Recognition and Localization Technology for Autonomous Operation of Intelligent Excavators. [Master’s Thesis, Jilin University].
78. Target Recognition of Intelligent Excavators Based on LiDAR Point Clouds
79. An Evidence-Based Approach to Object Pose Estimation from LiDAR Measurements in Challenging Environments10.1002/rob.21788 ↗
80. Is It What I Think It Is? Is It Where I Think It Is? Using Point-Clouds for Diagnostic Testing of a Digging Assembly’s Form and Pose for an Autonomous Mining Shovel10.1002/rob.21643 ↗
81. Agrawal, A., Nakazawa, A., and Takemura, H. (2009, January 12–17). MMM-Classification of 3D Range Data. Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan.10.1109/ROBOT.2009.5152539 ↗
82. Applying a Weighted Random Forests Method to Extract Karst Sinkholes from LiDAR Data10.1016/j.jhydrol.2015.12.012 ↗
83. Lai, X., Yuan, Y., Li, Y., and Wang, M. (2019). Full-Waveform LiDAR Point Clouds Classification Based on Wavelet Support Vector Machine and Ensemble Learning. Sensors, 19.10.3390/s19143191 ↗
84. Automatic Building Extraction from Very High-Resolution Image and LiDAR Data with SVM Algorithm10.1007/s12517-016-2664-7 ↗
85. Contextual Classification of Lidar Data and Building Object Detection in Urban Areas10.1016/j.isprsjprs.2013.11.001 ↗
86. Golovinskiy, A., Kim, V.G., and Funkhouser, T. (October, January 29). Shape-Based Recognition of 3D Point Clouds in Urban Environments. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.10.1109/ICCV.2009.5459471 ↗
87. Detection of 3-D Individual Trees in Urban Areas by Combining Airborne LiDAR Data and Imagery10.1109/LGRS.2013.2241390 ↗
88. Zhao, H., Liu, Y., Zhu, X., Zhao, Y., and Zha, H. (2010, January 3–7). Scene Understanding in a Large Dynamic Environment through a Laser-Based Sensing. Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA.
89. Object Detection in Terrestrial Laser Scanning Point Clouds Based on Hough Forest10.1109/LGRS.2014.2309965 ↗
90. A Multiscale and Hierarchical Feature Extraction Method for Terrestrial Laser Scanning Point Cloud Classification10.1109/TGRS.2014.2359951 ↗
91. Zeng, A., Yu, K.-T., Song, S., Suo, D., Walker, E., Rodriguez, A., and Xiao, J. (June, January 29). Multi-View Self-Supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge. Proceedings of the 2017 IEEE International Conference on Robotics and Automation, Singapore.10.1109/ICRA.2017.7989165 ↗
92. Li, Z., Gan, Y., Liang, X., Yu, Y., Cheng, H., and Lin, L. (2016, January 11–14). LSTM-CF: Unifying Context Modeling and Fusion with LSTMs for RGB-D Scene Labeling. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46475-6_34 ↗
93. Lawin, F.J., Danelljan, M., Tosteberg, P., Bhat, G., Khan, F.S., and Felsberg, M. (2017, January 22–24). Deep Projective 3D Semantic Segmentation. Proceedings of the Computer Analysis of Images and Patterns, Ystad, Sweden.10.1007/978-3-319-64689-3_8 ↗
94. Zhang, Z., Cui, Z., Xu, C., Jie, Z., Li, X., and Yang, J. (2018, January 8–14). Joint Task-Recursive Learning for Semantic Segmentation and Depth Estimation. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01249-6_15 ↗
95. Chen, Y., Yang, M., Wang, C., and Wang, B. (2019, January 9–12). 3D Semantic Modelling with Label Correction For Extensive Outdoor Scene. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium, Paris, France.10.1109/IVS.2019.8814075 ↗
96. Maturana, D., and Scherer, S. (October, January 28). VoxNet: A 3D Convolutional Neural Network for Real-Time Object Recognition. Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, Hamburg, Germany.10.1109/IROS.2015.7353481 ↗
97. Chang, A.X., Funkhouser, T., and Guibas, L. (2015). ShapeNet: An Information-Rich 3D Model Repository. arXiv.
98. Wang, L., Huang, Y., Shan, J., and He, L. (2018). MSNet: Multi-Scale Convolutional Network for Point Cloud Classification. Remote Sens., 10.10.3390/rs10040612 ↗
99. Roynard, X., Deschaud, J.-E., and Goulette, F. (2018). Classification of Point Cloud Scenes with Multiscale Voxel Deep Network. arXiv.
100. Qi, C.R. (2018). Deep Learning on Point Clouds for 3D Scene Understanding. [Ph.D. Thesis, Standford University].