LLM, RAG 기반 잠수함 전투체계 교범 시스템 구축 방안 연구

나현호¹^{, *} ; 임진규² ; 이재근¹ ; 서창원³

1한화시스템 해양연구소 전문연구원
2한화시스템 해양연구소 연구원
3한화시스템 해양연구소 선임연구원

Development of LLM- and RAG-Based Question Answering for Submarine Combat System Manuals

Hyun-Ho Na¹^{, *} ; Jin-Kyu Lim² ; Jae-Geun Lee¹ ; Chang-Won Seo³

1Principal engineer, Naval R&D Center, Hanwha Systems
2Engineer, Naval R&D Center, Hanwha Systems
3Senior engineer, Naval R&D Center, Hanwha Systems

Correspondence to: *Hyun-Ho Na 264-60, Sanho-daero, Gumi-si, Gyeongsangbuk-do, 39370, Republic of Korea Tel: +82-54-460-5230 Fax: +82-54-460-8519 E-mail: na.hh1@hanwha.com

초록

본 논문에서는 LLM과 RAG를 활용한 잠수함 전투체계 교범 질의응답 시스템 구축 방안을 제시한다. 문서 임베딩, 벡터 데이터베이스, fine-tuning을 포함한 파이프라인을 구성하고 객관식, 주관식 각 100문항의 벤치마크 데이터로 성능을 비교 평가하였다. 실험 결과 rag와 fine-tuning 적용 시 기본 모델 대비 응답 정확도와 문맥 적합성이 향상되었으며, 잠수함 전투체계 적용 가능성을 확인하였다.

Abstract

This paper proposes a method for large language model- and retrieval-augmented generation (RAG)-based question answering for submarine combat system manuals. A pipeline consisting of document embedding, a vector database, and fine-tuning was implemented, and the performance was comparatively evaluated using a benchmark dataset composed of 100 multiple-choice and 100 open-ended questions. The experimental results indicate that applying RAG and fine-tuning improves response accuracy and contextual relevance compared with the baseline model and confirm the applicability of the proposed approach to submarine combat systems.

Keywords:

Large Language Models, Retrieval-Augmented Generation, Fine-Tuning, Military AI, Submarine Combat Systems

키워드:

대형언어모델, 검색증강생성, 미세조정, 국방 인공지능, 잠수함 전투체계

References

Department of Defense Editorial Board, ‘Announcement of Defence Innovation 4.0 Framework Plan: Redesigning Defence at the Second Army Level, Fostering AI Science and Technology Force,’ Defense & Technology, VOL. 530, 2023, pp. 10-11.
Department of Defense Editorial Board, ‘Report on the Implementation Plan for the Detailed Implementation of the National Defense Policy Direction: Establishment of a Three-Axis System, Strengthening of ROK-US Joint Exercises, and Creation of a Defense AI Center,’ Defense & Technology, VOL. 522, 2022, pp. 11-13.
Sang-Min Kwon & Seung-Mo Jung, ‘Virtualization Based High Efficiency Naval Combat Management System Design and Performance Analysis,’ Journal of the Korea Society of Computer and Information, VOL. 23, NO. 11, 2018, pp. 9-15.
Yeo Chan Yoon & Soo Kyun Kim, ‘Trends and Prospects of Retrieval-Augmented Generation (RAG) for Generative AI,’ The Journal of Korean Association of Computer Education, VOL. 28, NO. 2, 2025, pp. 69-80. [https://doi.org/10.32431/kace.2025.28.2.007]
Jae-Hak Jeong, ‘A Study on the Evaluation of RAG Using LLMs: Focusing on Inter-Firm ESG Reports,’ Master’s Thesis, Sogang University, 2025. http://www.dcollection.net/handler/sogang/000000079781
LangChain. https://www.langchain.com/, (accessed 2026.03.10.)
Min-Jun Son & Sung-Jin Lee, ‘Research on Prompt Engineering Techniques in Large Language Models,’ The Journal of Korean Institute of Communications and Information Sciences, VOL. 50, NO. 1, 2025, pp. 9-21. [https://doi.org/10.7840/kics.2025.50.1.9]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, & Jeffrey Dean, ‘Distributed Representations of Words and Phrases and Their Compositionality,’ arXiv:1310.4546, , 2013. [https://doi.org/10.48550/arXiv.1310.4546]
Hugging Face, KoE5. https://huggingface.co/nlpai-lab/KoE5, (accessed 2026.03.10.)
Hugging Face, SFTTrainer. https://huggingface.co/docs/trl/sft_trainer, (accessed 2026.03.10.)
unsloth. https://unsloth.ai/, (accessed 2026.03.10.)
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing et al., ‘Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,’ arXiv:2306.05685, , 2023. [https://doi.org/10.48550/arXiv.2306.05685]