수동 소나를 탑재한 두 무인 수상정의 잠수함 추적을 위한 다중 에이전트 강화학습 기법

김두영^*

해군 중령/해군사관학교 인공지능학과 부교수

Multi-Agent Reinforcement Learning for Submarine Tracking Using Two USVs with Bearing-Only Passive Sonar Sensors

Dooyoung Kim^*

CDR, ROK Navy/Associate professor, Dep. of Artificial Intelligence, Republic of Korea Naval Academy

Correspondence to: *Dooyoung Kim Tel: +82-55-549-5624 E-mail: dykim07@navy.ac.kr

초록

본 연구는 수동 소나의 방위 정보만으로 잠수함을 추적하는 두 무인 수상정의 협력 기동 명령 생성을 위한 다중 에이전트 강화학습 기법을 제안한다. 입자 필터로 표적 상태를 추정하고, 강화학습으로 목표 거리와 유리한 관측 기하를 유지하는 제어 명령을 생성하였다. 시뮬레이션을 통해 제안한 기법의 성능을 검증하였다.

Abstract

This study proposes a multi-agent reinforcement learning method for generating cooperative maneuver commands for two unmanned surface vehicles tracking a submarine using bearing-only passive sonar measurements. A particle filter is used to estimate the target state, while reinforcement learning generates control commands to maintain the desired standoff distance and favorable observation geometry. The performance of the proposed method is validated using simulations.

Keywords:

Unmanned Surface Vehicle, Target Tracking, Passive Sonar, Anti-Submarine Warfare, Multi-Agent Reinforcement Learning

키워드:

무인 수상정, 표적 추적, 수동 소나, 대잠전, 다중 에이전트 강화학습

Acknowledgments

이 논문은 2026년 해군사관학교 해양연구소 학술연구과제(강화학습을 이용한 무인수상정의 대잠 추적경로 생성) 연구비 지원으로 수행된 연구임.

References

Dooyoung Kim, ‘Online Sonobuoy Deployment Method with Bayesian Optimization for Estimating Location of Submarines,’ Journal of the Korea Institute of Military Science and Technology, VOL. 25, NO. 1, 2022, pp. 72-81. [https://doi.org/10.9766/KIMST.2022.25.1.072]
Kyung-Min Park & Dooyoung Kim, ‘Preprocessing Performance of Convolutional Neural Networks according to Characteristic of Underwater Targets,’ The Journal of the Acoustical Society of Korea, VOL. 41, NO. 6, 2022, pp. 629-636.
Dooyoung Kim, ‘Approach Path Planning for Maritime Moving Target Tracking by Unmanned Surface Vehicles,’ Journal of the KNST, VOL. 8, NO. 3, 2025, pp. 331-336. [https://doi.org/10.31818/JKNST.2025.9.8.3.331]
Dooyoung Kim, ‘Path Planning for Autonomous Unmanned Surface Vehicle Recovery: A Potential Field Approach,’ Journal of the KNST, VOL. 7, NO. 4, 2024, pp. 428-432. [https://doi.org/10.31818/JKNST.2024.12.7.4.428]
S. Nardone, A. Lindgren, & Kai Gong, ‘Fundamental Properties and Performance of Conventional Bearings-Only Target Motion Analysis,’ IEEE Transactions on Automatic Control, VOL. 29, NO. 9, 1984, pp. 775-787. [https://doi.org/10.1109/TAC.1984.1103664]
N. J. Gordon, D. J. Salmond, & A. F. M. Smith, ‘Novel Approach to Nonlinear/Non-Gaussian Bayesian State Estimation,’ IEE Proceedings F (Radar Signal Processing), VOL. 140, NO. 2, 1993, pp. 107-113. [https://doi.org/10.1049/ip-f-2.1993.0015]
Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, & Yi Wu, ‘The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games,’ in Advances in Neural Information Processing Systems, VOL. 35, 2022, pp. 24611-24624. [https://doi.org/10.52202/068431-1787]
Christopher Amato, ‘An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learning,’ 2024, arXiv:2409.03052, .