Kisu Yang  Taesun Whang  Dongsuk Oh  Chanjun Park  Heuiseok Lim

10.5626/JOK.2021.48.2.154

Ensemble Deep Learning Natural Language Processing natural language inference

An ensemble method is a machine learning technique that combines several models to make the final prediction, which guarantees improved performance for deep learning models. However, most techniques require additional models or operations only for an ensemble. To address this problem, we propose a cross-validated ensemble method for reducing the costs of ensemble operations with cross-validation and for improving the generalization effects with the ensemble. To demonstrate the effectiveness of the proposed method, we show the improved performances of the proposed ensemble over the previous ensemble methods using the BiLSTM, CNN, ELMo and BERT models on the MRPC and RTE datasets. We also discuss the generalization mechanism involved in cross-validation along with the performance changes caused by the hyper-parameter of cross-validation.