Donghyeok Koh  Jeonghun Yuk  Byeongho Lee  Kyungtae Lim  Kyongha Lee  Taehoon Kim  Cheoneum Park

10.5626/JOK.2025.52.10.841

Korean benchmark large language model benchmark science and technology domain

We present a new Korean benchmark dataset for evaluating large language models (LLMs) in the science and technology information domain. We define eight specialized categories within scientific and technical fields, establishing levels and evaluation metrics for each question type to differentiate the science and technology information domain from the general domain. Our two-step synthetic data generation method enhances benchmark quality and domain specificity while reducing the time required for dataset construction. Experimental results with LLMs show accuracy scores ranging from 0.17 to 0.32 across multiple-choice question types, validating the benchmark’s effectiveness for the science and technology domains. This work provides a critical foundation for evaluating the domain-specific capabilities of language models trained for artificial general intelligence (AGI), marking Korea’s first specialized benchmark in this field.