Jae-kyun Kim  Jae-Hoon Kim

10.5626/JOK.2023.50.1.11

synonyms corpus augmentation NER corpus probabilistic sampling

A corpus is an essential resource for machine learning and deep learning in the field of natural language processing. In Korean, there are insufficient well-refined named entity corpus compared to advanced research countries such as the United States, Japan, and China. Most projects for building a named entity corpus proceed manually and/or semi-automatically and thus require a lot of cost and effort. In this paper, we propose a novel method for automatically augmenting a small-sized named entity corpus. The proposed method augments the corpus by automatically editing, for example, substituting, inserting, and deleting. We use probabilistic sampling rather than simple editing to make the augmented corpus natural and diverse. Through experiments, we have shown that the performance of Korean named entity recognition can be improved using the augmented corpus and the proposed method should be used in practice.