Thrust and Pressure Control in a Solid Propulsion System via Reinforcement Learning
Source abstract
Abstract
A reinforcement learning control method for a solid attitude and divert propulsion system is proposed. The system in this study includes four divert thrust nozzles, six attitude thrust nozzles, and a common combustion chamber. To achieve the required thrust, the pressure in the combustion chamber is first adjusted by controlling the total opening of the nozzles to generate the gas source. Next, by controlling the opening of nozzles at different positions, the required thrust is produced in the five-axis direction. Finally, the motor speed is regulated to drive the valve core to the specified position, completing the closed-loop control of the nozzle opening. The control algorithm used is the Proximal Policy Optimization (PPO) reinforcement learning algorithm. Through system identification and numerical modeling, the training environment for the intelligent agent is created. To accommodate different training objectives, multiple reward functions are implemented. Ultimately, through training, a multi-layer intelligent agent architecture for pressure, thrust, and nozzle opening is established, achieving effective system pressure and thrust control.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Rocket and propulsion systems research · Energetic Materials and Combustion · Advanced Control Systems Optimization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
Shanghai Space Propulsion Technology Research Institute, Shanghai 201109, China
Funding
Foundation Strengthening Plan Project
References (30)
Show source-provided references
1. Optimization Selection of Regulated Pressurization System Schemes for Liquid Attitude and Divert Propulsion Systems10.1016/j.proeng.2014.12.655 ↗
2. Recent Progress in R&D and Prospect of Divert and Attitude Control System(DACS)10.6108/KSPE.2012.16.6.062 ↗
3. Napior, J., and Garmy, V. (2006, January 2–6). Controllable Solid Propulsion for Launch Vehicle and Spacecraft Application. Proceedings of the 57th International Astronautical Congress, Valencia, Spain.10.2514/6.IAC-06-C4.2.04 ↗
4. Transient simulation of a differential piston warm gas self-pressurization system for liquid attitude and divert propulsion system10.1016/j.cja.2018.02.001 ↗
5. Dynamic characteristic modeling and simulation of an aerospike-shaped pintle nozzle for variable thrust of a solid rocket motor10.1016/j.actaastro.2022.09.031 ↗
6. Research on Divert and Attitude Control System Technology of Ballistic Missile Midcourse Maneuver Penetration Warhead10.54097/fcis.v3i1.6342 ↗
7. Davis, C., and Gerards, A. (2003, January 20–23). Variable Thrust Solid Propulsion Control Using Labview. Proceedings of the 39th AIAA/ASME/SAE/ASEE Joint Propulsion Conference and Exhibit, Huntsville, Alabama.10.2514/6.2003-5241 ↗
8. A survey of automatic control methods for liquid-propellant rocket engines10.1016/j.paerosci.2019.03.002 ↗
9. Design of nonlinear control method for pressure of pintle solid rocket motor10.1016/j.actaastro.2023.12.008 ↗
10. Bergmans, J., and Di Salvo, R. (2003, January 20–23). Solid Rocket Closed-Loop Pressure Control. Proceedings of the 39th AIAA/ASME/SAE/ASEE Joint Propulsion Conference and Exhibit, Huntsville, Alabama.10.2514/6.2003-4968 ↗
11. Joner, S., and Quinquis, I. (2006, January 21–24). Control of an Exoatmospheric Kill Vehicle with a Solid Propulsion Attitude Control System. Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit, Keystone, CO, USA.10.2514/6.2006-6572 ↗
12. Early detection of thermoacoustic instabilities in a cryogenic rocket thrust chamber using combustion noise features and machine learning10.1063/5.0038817 ↗
13. Wang, M., and Xie, Y. (2009, January 15–18). Design of the optimal thruster combinations table for the real time control allocation of spacecraft thrusters. Proceedings of the 48th IEEE Conference on Decision and Control (CDC) Held Jointly with 2009 28th Chinese Control Conference, Shanghai, China.10.1109/CDC.2009.5400664 ↗
14. Martel, F. (2004, January 11–15). Optimal 6 axis command of a space vehicle with a precomputed thruster selection catalogue table. Proceedings of the 18th International Symposium on Space Flight Dynamics, Munich, Germany.
15. Optimization of Spacecraft Thruster Management Function10.2514/1.11531 ↗
16. Efficient Thrust Distribution with Adaptive Pressure Control for Multinozzle Solid Propulsion System10.2514/1.B34645 ↗
17. Efficient Thrust Management Algorithm for Variable Thrust Solid Propulsion System with Multi-Nozzles10.2514/1.A34550 ↗
18. Thrust distribution for attitude control in a variable thrust propulsion system with four ACS nozzles10.1016/j.asr.2017.01.002 ↗
19. Artificial Intelligence Applications for Industry 4.0: A Literature-Based Study10.1142/S2424862221300040 ↗
20. A review On reinforcement learning: Introduction and applications in industrial process control10.1016/j.compchemeng.2020.106886 ↗
21. Xu, W., Yang, X., and Shi, G. (2024). The Maintenance of Orbital States in a Floating Partial Space Elevator Using the Reinforcement Learning Method. Aerospace, 11.10.3390/aerospace11100855 ↗
22. Skarka, W., and Ashfaq, R. (2024). Hybrid Machine Learning and Reinforcement Learning Framework for Adaptive UAV Obstacle Avoidance. Aerospace, 11.10.3390/aerospace11110870 ↗
23. Pham, T.-H., Aikins, G., Truong, T., and Nguyen, K.-D. (2024). Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning. Algorithms, 17.10.3390/a17100436 ↗
24. Wang, X., Yi, H., Xu, J., Xu, C., and Song, L. (2024). PID Controller Based on Improved DDPG for Trajectory Tracking Control of USV. J. Mar. Sci. Eng., 12.10.3390/jmse12101771 ↗
25. Transient flow characteristics and performance of a solid rocket motor with a pintle valve10.1016/j.cja.2020.04.023 ↗
26. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv.
27. Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. (2015). High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv.
28. Engstrom, L., Ilyas, A., Santurkar, S., Tsipras, D., Janoos, F., Rudolph, L., and Madry, A. (2020). Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO. arXiv.
29. Multi-dimensional two-phase flow modeling applied to interior ballistics10.1115/1.4004293 ↗
30. Wang, H., Fu, Z., Yan, J., Hua, Z., and Guan, Y. (2023, January 24–26). Modeling and Simulation of High Frequency and High Precision Multi Axis Cooperative Control Based on Reinforcement Learning. Proceedings of the 2023 3rd International Conference on Computer, Control and Robotics (ICCCR), Shanghai, China.10.1109/ICCCR56747.2023.10193913 ↗