Computer Vision-based Fishing Net Detection for Autonomous Navigation of USVs

Kyeongmo Kang^*

해군 대위/해군사관학교 기계시스템공학과 조교수

무인수상정의 자율주행을 위한 컴퓨터 비전 기반 어망 탐지

강경모^*

LT, ROK Navy/Assistant Professor, Dept. of Mechanical System Engineering, Republic of Korea Naval Academy

Correspondence to: ^*Kyeongmo Kang Dept. of Mechanical System Engineering, Republic of Korea Naval Academy 1 Jungwon-ro, Jinhae-gu, Changwon-si, Gyungsangnam-do, 51704, Republic of Korea Tel: +82-55-907-5316 E-mail: kmkang@navy.ac.kr

Abstract

Unmanned Surface Vehicles (USVs) are increasingly deployed in maritime operations, but entanglement with fishing nets remains a significant challenge, causing mechanical failures and operational disruptions. This study presents an indirect detection approach that identifies fishing net buoys as reliable markers instead of detecting the nets directly. A computer vision-based system was developed to detect fishing net buoys and estimate their positions, utilizing a stereo camera and transfer learning on YOLO models. Experimental validation demonstrated high detection accuracy with fishing net buoy, achieving an mAP50 of 0.98, and the system estimated target positions within a distance error of ±12.08 % and an angular error of ±3.48 %. Based on these results, the feasibility of this approach for real-time USV fishing net avoidance is demonstrated.

초록

무인 수상정은 해양 작전에서 점점 더 많이 활용되고 있지만, 어망에 얽히는 문제는 여전히 기계적 고장과 작전 차질을 초래하는 중요한 과제로 남아 있다. 본 연구는 어망을 직접 탐지하는 대신, 어망 부표를 탐지하는 간접 탐지 방식을 제안한다. 어망 부표를 탐지하고, 그 위치를 추정하기 위한 컴퓨터 비전 기반 시스템이 스테레오 카메라와 YOLO 모델의 전이 학습을 활용하여 개발하였다. 실험 결과, mAP50 기준 0.984의 높은 탐지 성능을 보였으며, 거리 오차 ±12.08 %, 각도 오차 ±3.48 % 이내에서 표적의 위치를 추정하였다. 이러한 결과를 바탕으로, 본 접근 방식이 실시간 USV 어망 회피에 적용 가능함을 입증하였다.

Keywords:

Autonomous Navigation, Computer Vision, Fishing Net Detection, Stereo Camera, Unmanned Surface Vehicle

키워드:

자율운항, 컴퓨터 비전, 어망 탐지, 스테레오 카메라, 무인수상정

Acknowledgments

본 논문은 해군사관학교 해양연구소 학술연구과제 연구비의 지원으로 수행된 연구임.

References

P. Kimball, J. Bailey, S. Das, R. Geyer, T. Harrison, and C. Kunz, “The WHOI Jetyak: An Autonomous Surface Vehicle for Oceanographic Research in Shallow or Dangerous Waters,” in Proceedings of the 2014 IEEE/OES Autonomous Underwater Vehicles(AUV), Oct. 2014, pp. 1–7. [https://doi.org/10.1109/auv.2014.7054430]
G. Hitz, F. Pomerleau, M.-E. Garneau, C. Pradalier, T. Posch, and J. Pernthaler, “Autonomous Inland Water Monitoring: Design and Application of a Surface Vessel,” IEEE Robotics & Automation Magazine, Vol. 19, No. 1, pp. 62–72, Mar. 2012. [https://doi.org/10.1109/mra.2011.2181771]
A. T. Demetillo and E. B. Taboada, “Real-time Water Quality Monitoring for Small Aquatic Area Using Unmanned Surface Vehicle,” Engineering, Technology & Applied Science Research, Vol. 9, No. 2, pp. 3959–3964, Apr. 2019. [https://doi.org/10.48084/etasr.2661]
W. A. Maawali, A. A. Naabi, M. A. Yaruubi, A. Saleem, and A. A. Maashri, “Design and Implementation of an Unmanned Surface Vehicle for Oil Spill Handling,” in Proceedings of the 2019 1st International Conference on Unmanned Vehicle Systems-Oman(UVS), Mar. 2019, pp. 1–6. [https://doi.org/10.1109/uvs.2019.8658267]
D. Madeo, A. Pozzebon, C. Mocenni, and D. Bertoni, “A Low-cost Unmanned Surface Vehicle for Pervasive Water Quality Monitoring,” IEEE Transactions on Instrumentation and Measurement, Vol. 69, No. 4, pp. 1433–1444, Apr. 2020. [https://doi.org/10.1109/tim.2019.2963515]
O. Lewicka, M. Specht, A. Stateczny, G. Dardanelli, D. Brčić, B. Szostak, A. Halicki, M. Stateczny, and S. Widźgowski, “Integration Data Model of the Bathymetric Monitoring System for Shallow Waterbodies Using UAV and USV Platforms,” Remote Sensing, Vol. 14, No. 16, 4075, Aug. 2022. [https://doi.org/10.3390/rs14164075]
L.-F. Constantinoiu, M. Bernardino, and E. Rusu, “Autonomous Shallow Water Hydrographic Survey Using a Proto-type USV,” Journal of Marine Science and Engineering, Vol. 11, No. 4, 799, Apr. 2023. [https://doi.org/10.3390/jmse11040799]
M. Specht, C. Specht, M. Szafran, A. Makar, P. Dąbrowski, H. Lasota, and P. Cywiński, “The Use of USV to Develop Navigational and Bathymetric Charts of Yacht Ports on the Example of National Sailing Centre in Gdańsk,” Remote Sensing, Vol. 12, No. 16, 2585, Aug. 2020. [https://doi.org/10.3390/rs12162585]
C. Specht, E. Świtalski, and M. Specht, “Application of an Autonomous/unmanned Survey Vessel (ASV/USV) in Bathymetric Measurements,” Polish Maritime Research, Vol. 24.3, No. 95, pp. 36–44, Apr. 2023. [https://doi.org/10.1515/pomr-2017-0088]
A. Makar, “Coastal Bathymetric Sounding in Very Shallow Water Using USV: Study of public beach in Gdynia, Poland,” Sensors, Vol. 23, No. 9, 4215, Apr. 2023. [https://doi.org/10.3390/s23094215]
W. Zhang, X.-Z. Gao, C.-F. Yang, and Z.-Y. Chen, “A Object Detection and Tracking Method for Security in Intelligence of Unmanned Surface Vehicles,” Journal of Ambient Intelligence and Humanized Computing, Vol. 13, pp. 1279–1291, Oct. 2022. [https://doi.org/10.1007/s12652-020-02573-z]
G. Casalino, A. Turetta, and E. Simetti, “A Three-layered Architecture for Real Time Path Planning and Obstacle Avoidance for Surveillance USVs Operating in Harbour Fields,” in Proceedings of the Oceans 2009-Europe, May 2009, pp. 1–8. [https://doi.org/10.1109/oceanse.2009.5278104]
J. Wang, W. Gu, J. Zhu, and J. Zhang, “An Unmanned Surface Vehicle for Multi-mission Applications,” in Proceedings of the 2009 International Conference on Electronic Computer Technology, Feb. 2009, pp. 358–361. [https://doi.org/10.1109/icect.2009.136]
Y. Wang, W. Liu, J. Liu, and C. Sun, “Cooperative USV–UAV Marine Search and Rescue with Visual Navigation and Reinforcement Learning-based Control,” ISA Transactions, Vol. 137, pp. 222–235, Jun. 2023. [https://doi.org/10.1016/j.isatra.2023.01.007]
T. Yang, Z. Jiang, R. Sun, N. Cheng and H. Feng, “Maritime Search and Rescue Based on Group Mobile Computing for Unmanned Aerial Vehicles and Unmanned Surface Vehicles,” IEEE Transactions on Industrial Informatics, Vol. 16, No. 12, pp. 7700–7708, Feb. 2020. [https://doi.org/10.1109/tii.2020.2974047]
X. Xiao, J. Dufek, T. Woodbury, and R. Murphy, “UAV Assisted USV Visual Navigation for Marine Mass Casualty Incident Response,” in Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep. 2017, pp. 6105–6110. [https://doi.org/10.1109/iros.2017.8206510]
A. Matos, E. Silva, N. Cruz, J. C. Alves, D. Almeida, M. Pionto, A. Martins, J. Almeida, and D. Machado, “Development of an Unmanned Capsule for Large-scale Maritime Search and Rescue,” in Proceedings of the 2013 OCEANS, Sep. 2013, pp. 1–8.
R. Qin, X. Zhao, W. Zhu, Q. Yang, B. He, G. Li, and T. Yan, “Multiple Receptive Field Network (MRF-Net) for Autonomous Underwater Vehicle Fishing Net Detection Using Forward-looking Sonar Images,” Sensors, Vol. 21, No. 6, pp. 1933, Mar. 2021. [https://doi.org/10.3390/s21061933]
X. Ye and X. Wang, “Deep Generative Network and Regression Network for Fishing Nets Detection in Real-time,” in Proceedings of the 2018 37th Chinese Control Conference, Jul. 2018, pp. 9466–9471. [https://doi.org/10.23919/chicc.2018.8483142]
X. Liu, X. Wang, P. Ren, Y. Cao, Y. Zhou, and Y. Liu, “Automatic Fishing Net Detection and Recognition Based on Optical Gated Viewing for Underwater Obstacle Avoidance,” Optical Engineering, Vol. 56, No. 8, 083101, Aug. 2017.
J. Kim, “Camera-based Net Avoidance Controls of Underwater Robots,” Sensors, Vol. 24, No. 2, pp. 674, Jan. 2024. [https://doi.org/10.3390/s24020674]
G. Jocher, J. Qiu, and A. Chaurasia, “Ultralytics YOLO,” Jan. 2023, https://github.com/ultralytics/ultralytics
B. Bingham, C. Aguero, M. McCarrin, J. Klamo, J. Malia, K. Allen, T. Lum, M. Rawson, and R. Waqar, “Toward Maritime Robotic Simulation in Gazebo,” in Proceedings of the OCEANS 2019 MTS/IEEE Conference, Oct. 2019, pp. 1–10. [https://doi.org/10.23919/oceans40490.2019.8962724]