Borehole Trajectory Optimization Design Based on the SAC Algorithm with a Self-Attention Mechanism
Source abstract
Abstract
Borehole trajectory planning under complex geological conditions poses significant challenges for intelligent drilling systems. To tackle this issue, a novel optimization framework is developed, leveraging the Soft Actor-Critic (SAC) algorithm enhanced by a self-attention mechanism. A three-dimensional heterogeneous geological model is constructed via generative adversarial networks (GANs), incorporating key formation features such as lithology, pressure, and fault zones. A tailored multi-objective reward function is introduced, balancing directional convergence, trajectory smoothness, obstacle avoidance, and formation adaptability. The self-attention mechanism is embedded into both the actor and critic networks to strengthen the agent’s capacity for spatial perception and decision stability. The proposed approach enables the agent to adaptively generate control sequences for efficient trajectory planning in highly variable formations. Experimental results demonstrate that the model exhibits superior convergence stability, improved curvature control, and enhanced obstacle avoidance, highlighting its potential for intelligent trajectory planning in challenging drilling environments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Multimodal Machine Learning Applications · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mathematics and Statistics, Northeast Petroleum University, Daqing 163318, China
School of Electrical and Information Engineering, Northeast Petroleum University, Daqing 163318, China
NEPU Sanya Offshore Oil & Gas Research Institute, Northeast Petroleum University, Sanya 572000, China
Funding
National Natural Science Foundation of China · Guiding Innovation Fund of Northeast Petroleum University
References (29)
Show source-provided references
1. Almarzooqi, L.Z., Fonseca, R.M., Buhindi, H.A., Binabadat, E.K., Al Hamlawi, I., Dolle, N., and Meshcheriakova, O. (November, January 31). Transforming well planning through an AI-assisted well trajectory optimization approach applied to an offshore field in the middle east. Proceedings of the Abu Dhabi International Petroleum Exhibition and Conference, Abu Dhabi, United Arab Emirates.
2. Gu, H., Yan, T., and Wu, Y. (2025). Research on intelligent optimization of wellbore trajectory in complex formation. Appl. Sci., 15.10.3390/app15031364 ↗
3. Directional drilling positioning calculations10.1016/j.jngse.2019.103081 ↗
4. A deep reinforcement learning algorithm suitable for autonomous vehicles: Double bootstrapped soft-actor-critic-discrete10.1109/TCDS.2021.3092715 ↗
5. Tan, H. (2021, January 28–30). Reinforcement learning with deep deterministic policy gradient. Proceedings of the 2021 International Conference on Artificial Intelligence, Big Data and Algorithms (CAIBDA), Xi’an, China.10.1109/CAIBDA53561.2021.00025 ↗
6. Reinforcement learning: An introduction10.1109/TNN.1998.712192 ↗
7. Liu, H. (2020). Research on a Full Closed-Loop Intelligent Drilling Method Based on Reinforcement Learning. [Master’s Thesis, China University of Petroleum].
8. Fan, C. (2022). Study on Intelligent Obstacle Avoidance Guided Drilling Algorithm with Spatial Attention Mechanism. [Master’s Thesis, China University of Petroleum].
9. Wang, F. (2022). An Adaptive Trajectory Tracking Control Method Based on Reinforcement Learning. [Master’s Thesis, China University of Petroleum].
10. Jian, Z. (2023). Optimization of Horizontal Well Trajectory Based on Drilling, Measurement, and Seismic Recording Results. [Master’s Thesis, Northeast Petroleum University].
11. Peshkov, G., Pavlov, M., Katterbauer, K., and Al Shehri, A. (2023, January 21–23). Real-time AI geosteering for horizontal well trajectory optimization. Proceedings of the SPE Annual Caspian Technical Conference, Baku, Azerbaijan.10.2118/217527-MS ↗
12. Yu, Y., Chen, W., Liu, Q., Chau, M., Vesselinov, V., and Meehan, R. (2021, January 8–12). Training an automated directional drilling agent with deep reinforcement learning in a simulated environment. Proceedings of the SPE/IADC Drilling Conference and Exhibition, Stavanger, Norway.10.2118/204105-MS ↗
13. Vishnumolakala, N., Kesireddy, V., Dey, S., Gildin, E., and Losoya, E.Z. (2023, January 16–18). Optimizing well trajectory navigation and advanced geo-steering using deep-reinforcement learning. Proceedings of the SPE Annual Technical Conference and Exhibition, San Antonio, TX, USA.10.2118/215011-MS ↗
14. A target-aware well path control method based on transfer reinforcement learning10.2118/218409-PA ↗
15. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10–15). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden.
16. Attention is all you need
17. Zhou, J., Li, X., Wang, Y., Liu, Z., Chen, H., Yang, J., Zhao, Q., Sun, M., Zhou, L., and Wu, X. (2024). Trajectory optimization using transformer-reinforced PPO. IEEE Trans. Ind. Inform., 1–10.
18. Real-time geosteering using reinforcement learning
19. Bit trajectory prediction with hybrid SAC-DDPG model
20. Adaptive wellbore control with multi-agent reinforcement learning
21. Hierarchical RL for trajectory planning in fractured formations
22. Deep policy optimization for automated directional drilling
23. A soft actor-critic reinforcement learning algorithm for network intrusion detection10.1016/j.cose.2023.103502 ↗
24. Chen, B., Zhang, L., Cheng, G., Liu, Y., and Chen, J. (2023). A novel approach for train tracking in virtual coupling based on soft actor-critic. Actuators, 12.10.3390/act12120447 ↗
25. Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., and Kaiser, L. (2021, January 3–7). Rethinking attention with performers. Proceedings of the International Conference on Learning Representations (ICLR), Vienna, Austria.
26. Luong, M.T., Pham, H., and Manning, C.D. (2015, January 17–21). Effective approaches to attention-based neural machine translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.10.18653/v1/D15-1166 ↗
27. Reward shaping with expert demonstrations for deep reinforcement learning in control tasks
28. Liu, X. (2023). Workflow Scheduling Research Based on Multi-Agent Reinforcement Learning with Soft Actor-Critic. [Master’s Thesis, Dongbei University of Finance and Economics].
29. Zhang, D. (2023). Energy Management Strategy for Fuel Cell Vehicles Based on Enhanced Soft Actor-Critic Algorithm. [Master’s Thesis, East China Normal University].