강화학습 기반 해상에서 UAV의 함상 자율착륙을 위한 경로계획

박한석¹^{, *} ; 방효충²

1해군 대위/KAIST 항공우주공학과 석사과정
2KAIST 항공우주공학과 교수

Reinforcement Learning Based Path Planning for Autonomous Shipboard Landing of UAV in Maritime

Hanseok Park¹^{, *} ; Hyochoong Bang²

1Lieutenant, ROK Navy/M.S. course student, Dept. of Aerospace Engineering, Korea Advanced Institute of Science and Technology
2Professor, Dept. of Aerospace Engineering, Korea Advanced Institute of Science and Technology

Correspondence to: ^*Han-Seok Park Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology, 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea E-mail: intoto6@kaist.ac.kr

초록

본 논문에서는 해상에서 UAV의 함상착륙을 위한 접근 및 착륙 과정에서 강화학습에 기반한 경로계획 기법을 제안한다. 제안된 모델은 연속적인 행동영역에 적용이 가능한 대표적인 강화학습 알고리즘인 심층 결정론적 정책 기울기(DDPG) 방식을 이용하여, 이동 장애물에 대한 회피와 함정의 3축 운동을 고려한 경로추종문제를 다루었다. 본 연구의 결과는 시뮬레이션을 통해 보여진다.

Abstract

This paper proposes a path planning model based on reinforcement learning in the process of approaching and landing for shipboard landing of UAV in maritime. The proposed model dealt with the path tracking problem considering avoidance of moving obstacles and triaxial motion of ships, using the deep deterministic policy gradient (DDPG) method, a representative reinforcement learning algorithm applicable to continuous behavioral areas. The results of this study are shown through simulation.

Keywords:

Unmanned Aerial Vehicles, Shipboard Landing, Reinforcement Learning, Path Planning

키워드:

무인 항공기, 함상착륙, 강화학습, 경로계획

References

G. J. Duan and P. F. Zhang, “Research on Application of UAV for Maritime Supervision,” Journal of Shipping and Ocean Engineering, No. 4, pp. 322–326, 2014.
Dan Gettinger, “Summary of Drone spending in the FY 2019 Budget Request,” Center for the Study of the Drone at Bard College, 2018.
Kim, J. G. and Lee, S. H., “Study on Possible Use of Navy's Future Military Drone,” Proceedings of the Korean Society of Computer Information Conference, Vol. 28, No. 1, pp. 83-86, 2020.
T. Zhang et al., “Current trends in the development of intelligent unmanned autonomous systems,” Frontiers Inf. Technol. Electron. Eng., Vol. 18, No. 1, pp. 68-85, 2017. [https://doi.org/10.1631/FITEE.1601650]
Z Gan et al., “UAV Maneuvering Target Tracking based on Deep Reinforcement Learning,” Journal of Physics: Conference Series. 1958 012015, 2021. [https://doi.org/10.1088/1742-6596/1958/1/012015]
Alejandro Rodriguez-Ramos, Carlos Sampedro, Hriday Bavle, Ignacio Gil Moreno and Pascual Campoy, “A Deep Reinforcement Learning Technique for Vision- Based Autonomous Multirotor Landing on a Moving Platform,” Proceeding of the 2018 IEEE/RSJ International Conference, pp. 1010-1017, 2018. [https://doi.org/10.1109/IROS.2018.8594472]
Bae, Y. K., “Strategic Response against Drones at Sea,” Journal of the KNST, Vol. 4, No. 1, pp. 36-41, 2021. [https://doi.org/10.31818/JKNST.2021.03.4.1.36]
Li, B., Yang, Z.P., Chen, D.Q., Liang, S.Y. and Ma, H., “Maneuvering target tracking of UAV based on MN-DDPG and transfer learning”, Def. Technol, Vol. 17, pp. 457–466, 2020. [https://doi.org/10.1016/j.dt.2020.11.014]
T. P. Lillicrap et al., "Continuous control with deep reinforcement learning", arXiv preprint arXiv:1509.02971, 2015.
Cho, Y. W., Lee, J. S. and Lee, K. Y., “CNN based Reinforcement Learning for Driving Behavior of Simulated Self-Driving Car”, The transactions of The Korean Institute of Electrical Engineers, Vol. 69, No.11, pp. 1740-1749, 2020. [https://doi.org/10.5370/KIEE.2020.69.11.1740]
Hanjie Hu., Yu Wu., Jinfa Xu. and Qingyun Sun.,“Path Planning for Autonomous Landing of Helicopter on the Aircraft Carrier,” Mathematics, 6 (10), 178, 2018. [https://doi.org/10.3390/math6100178]
Rho, S. G., "RL from basics," Youngjin.com Inc., Seoul, 2020.