Comparison of BERT-based Model Performance in CBCA Criteria Classification

Junho Shin, Jungsoo Shin, Eunkyung Jo, Yeohoon Yoon, Jaehee Jung

http://doi.org/10.5626/JOK.2022.49.9.727

In the case of child sex crimes, the victim"s statement plays a critical role in determining the existence or innocence of the case, so the Supreme Prosecutors" Office classifies the statement into a total of 19 criteria according to Criteria-Based Content Analysis (CBCA), a victim"s statement analysis technique. However, this may differ in criteria classification according to the subjective opinion of the statement analyst. Thus, in this paper, two major classification methods were applied and analyzed to present an criteria classification model using BERT and RoBERTa. The two methods comprise of a method of classifying the entire criterion at the same time, as well as method of dividing it into four groups, and then classifying the criteria within the group secondarily. The experiment classified statements into 16 criteria of CBCA and performed comparative analysis using several pre-trained models. As a result of the classification, the former classification method performed better than the latter classification method in 13 of the total 16 criteria, and the latter method was effective in three criteria with a relatively insufficient number of training data. Additionally, the RoBERTa-based model performed better than the BERT-based model in 15 of the 16 criteria, and the BERT model, which was pre-trained using only Korean conversational colloquial language, classified the remaining one criterion uniquely. This paper shows that the proposed model, which was pre-trained using interactive colloquial data is effective in classifying children"s statement sentences.