Applying DDPG Algorithm to Swing-Up and Balance Control for a Double Inverted Pendulum on a Cart

Trong-Nguyen Ho^1,*, Thanh-Sang Tat², Hoang-Anh Ngo², Truong-Son Nguyen², Duc-Anh Bui², Thanh-Trung Le², Vu-Loc Le², Lac-Thien Huynh²

¹Onsemi Viet Nam
17A, No.10, Bien Hoa City, Dong Nai Province, 810000, Viet Nam
²Ho Chi Minh City University of Technology and Education
Vo Van Ngan St., No. 01, Thu Duc City, Ho Chi Minh City, 700000, Vietnam
* Corresponding author. E-mail: nguyen.ho@email.com

Robotica & Management, Vol. 28, No. 2, pp. 14-20
DOI: https://doi.org/10.24193/rm.2023.2.3

Abstract: In this study, we apply the Deep Deterministic Policy Gradient (DDPG) algorithm in reinforcement learning to control a double inverted pendulum on a cart (DIPC)- a high order single input-multi output (SIMO) system . The simulation results demonstrate DDPG’s stability and effectiveness in achieving swing-up and balance, showing its potential for tackling challenging control tasks in robotics.

Keywords: Reinforcement learning, DDPG, double inverted pendulum on a cart, swing-up, balance.

Full text

References

[1] Sutton R.S., and Andrew G. Barto.: “Reinforcement Learning: An Introduction, Second edition”, Adaptive Computation and Machine Learning, Cambridge, Mass: The MIT Press, 2018.

[2] Gustin J., Houyon J.: “Inverted Double Pendulum: Searching High-Quality Policies to Control an Unstable Physical System”, INFO8003: Optimal decision making for complex problems, 2022.

[3] Tran V.D, Ho T.N, Nguyen M.T, Nguyen V.D.H.: “Balancing Control For Double-Linked Inverted Pendulum On Cart: Simulation And Experiment”, Journal of Technical Education Science, No. 44A, 2017.

[4] Bogdanov A.: “Optimal Control of a Double Inverted Pendulum on a Cart”, Technical Report CSE-04-006, 2004.

[5] Lillicrap T.P., Hunt J.J., Pritzel A., Heess N., Erez T., Tassa Y., Silver D, Wierstra D.: “Continuous Control with Deep Reinforcement Learning”, ICLR, 2016.

[6] Gustafsson F.: “Control of Inverted Double Pendulum using Reinforcement Learning”, 2016.

[7] OpenAI.: “Deep Deterministic Policy Gradient”, 2018.

[8] Matlab.: “Single Pendulum in Simulink and Simscape Multibody”, 2023.

[9] Panjapornpon C., Chinchalongporn P., Bardeeniz S., Makkayatorn R., Wongpunnawat W.: “Reinforcement Learning Control with Deep Deterministic Policy Gradient Algorithm for Multivariable pH Process”, MDPI, 2022.