BCAMP: A Behavior-Controllable Motion Control Method Based on Adversarial Motion Priors for Quadruped Robot
Source abstract
Abstract
In unpredictable scenarios, quadruped robots with behavior-controllable capabilities can often improve their adaptability through interaction with users. In this paper, we propose a behavior-controllable motion control method, integrating user commands with adversarial motion priors, enabling the quadruped robot to achieve behavior-controllable capabilities. Firstly, a motion trajectory library is constructed to provide motion prior knowledge. To obtain stable trajectory data for various motions, optimal control methods are used to generate dynamic trajectories with whole-body dynamic constraints. These trajectory data are then standardized and assigned different weights, resulting in the construction of a motion trajectory library for the quadruped robot. Secondly, an adversarial motion prior network structure combined with user commands is proposed. Reward functions tailored to different motion behaviors are designed to achieve behavior control. This network structure acts as a single-motion prior discriminator, which, compared to a multi-motion prior discriminator, avoids complex architectures. Furthermore, the incorporation of user commands effectively addresses the issue where the single-motion prior discriminator struggles to clearly select actions as the dataset expands. Finally, simulations and comparative experiments are conducted to evaluate the effectiveness of the proposed method.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotic Locomotion and Control · Robotic Path Planning Algorithms · Robot Manipulation and Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Intelligent Science and Technology, Beijing University of Civil Engineering and Architecture, Beijing 102616, China
Funding
Beijing Advanced Innovation Center for Intelligent Robots and Systems
References (30)
Show source-provided references
1. Learning quadrupedal locomotion over challenging terrain10.1126/scirobotics.abc5986 ↗
2. Park, J., and Park, J.H. (2012, January 7–12). Impedance control of quadruped robot and its impedance characteristic modulation for trotting on irregular terrain. Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.10.1109/IROS.2012.6385710 ↗
3. DeepGait: Planning and Control of Quadrupedal Gaits Using Deep Reinforcement Learning10.1109/LRA.2020.2979660 ↗
4. Di Carlo, J., Wensing, P.M., Katz, B., Bledt, G., and Kim, S. (2018, January 1–5). Dynamic Locomotion in the MIT Cheetah 3 Through Convex Model-Predictive Control. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8594448 ↗
5. Dario Bellicoso, C., Gehring, C., Hwangbo, J., Fankhauser, P., and Hutter, M. (2016, January 15–17). Perception-less terrain adaptation through whole body control and hierarchical optimization. Proceedings of the 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), Cancun, Mexico.10.1109/HUMANOIDS.2016.7803330 ↗
6. Kim, D., Carlo, J.D., Katz, B., Bledt, G., and Kim, S. (2019). Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control. arXiv.
7. Dynamic Locomotion Through Online Nonlinear Motion Optimization for Quadrupedal Robots10.1109/LRA.2018.2794620 ↗
8. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing Atari with Deep Reinforcement Learning. arXiv.
9. Li, T., Lambert, N., Calandra, R., Meier, F., and Rai, A. (August, January 31). Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.10.1109/ICRA40945.2020.9196642 ↗
10. Smith, L.M., Kostrikov, I., and Levine, S. (2022). A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning. arXiv.10.15607/RSS.2023.XIX.056 ↗
11. Margolis, G.B., and Agrawal, P. (2022). Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior. arXiv.
12. Learning robust perceptive locomotion for quadrupedal robots in the wild10.1126/scirobotics.abk2822 ↗
13. Cheng, X., Kumar, A., and Pathak, D. (June, January 29). Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.10.1109/ICRA48891.2023.10161470 ↗
14. Radac, M.B. (2022). Trajectory tracking within a hierarchical primitive-based learning approach. Entropy, 24.10.3390/e24070889 ↗
15. Merel, J., Hasenclever, L., Galashov, A., Ahuja, A., Pham, V., Wayne, G., Teh, Y.W., and Heess, N. (2018). Neural probabilistic motor primitives for humanoid control. arXiv.
16. Utilizing reinforcement learning to continuously improve a primitive-based motion planner
17. Three-level hierarchical model-free learning approach to trajectory tracking control10.1016/j.engappai.2016.06.009 ↗
18. SFV: Reinforcement learning of physical skills from videos10.1145/3272127.3275014 ↗
19. AMP: Adversarial motion priors for stylized physics-based character control10.1145/3450626.3459670 ↗
20. L’Erario, G., Hanover, D., Romero, A., Song, Y., Nava, G., Viceconte, P.M., Pucci, D., and Scaramuzza, D. (2024, January 14–18). Learning to Walk and Fly with Adversarial Motion Priors. Proceedings of the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Abu Dhabi, United Arab Emirates.10.1109/IROS58592.2024.10802391 ↗
21. Learning Robust and Agile Legged Locomotion Using Adversarial Motion Priors10.1109/LRA.2023.3290509 ↗
22. Peng, T., Bao, L., Humphreys, J., Delfaki, A.M., Kanoulas, D., and Zhou, C. (2024). Learning Bipedal Walking on a Quadruped Robot via Adversarial Motion Priors. arXiv.10.1007/978-3-031-72062-8_11 ↗
23. Vollenweider, E., Bjelonic, M., Klemm, V., Rudin, N., Lee, J., and Hutter, M. (June, January 29). Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.10.1109/ICRA48891.2023.10160751 ↗
24. Escontrela, A., Peng, X.B., Yu, W., Zhang, T., Iscen, A., Goldberg, K., and Abbeel, P. (2022, January 23–27). Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions. Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.10.1109/IROS47612.2022.9981973 ↗
25. A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems10.1016/S1474-6670(17)61205-9 ↗
26. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv.
27. Sutton, R.S., McAllester, D., Singh, S., and Mansour, Y. (December, January 29). Policy gradient methods for reinforcement learning with function approximation. Proceedings of the 12th International Conference on Neural Information Processing Systems, Cambridge, MA, USA. NIPS’99.
28. CasADi: A software framework for nonlinear optimization and optimal control10.1007/s12532-018-0139-4 ↗
29. Carpentier, J., Saurel, G., Buondonno, G., Mirabel, J., Lamiraux, F., Stasse, O., and Mansard, N. (2019, January 14–16). The Pinocchio C++ library: A fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives. Proceedings of the 2019 IEEE/SICE International Symposium on System Integration (SII), Paris, France.10.1109/SII.2019.8700380 ↗
30. Makoviychuk, V., Wawrzyniak, L., Guo, Y., Lu, M., Storey, K., Macklin, M., Hoeller, D., Rudin, N., Allshire, A., and Handa, A. (2021). Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning. arXiv.