Cross-Validated Ensemble Methods in Natural Language Inference

Kisu Yang, Taesun Whang, Dongsuk Oh, Chanjun Park, Heuiseok Lim

http://doi.org/10.5626/JOK.2021.48.2.154

An ensemble method is a machine learning technique that combines several models to make the final prediction, which guarantees improved performance for deep learning models. However, most techniques require additional models or operations only for an ensemble. To address this problem, we propose a cross-validated ensemble method for reducing the costs of ensemble operations with cross-validation and for improving the generalization effects with the ensemble. To demonstrate the effectiveness of the proposed method, we show the improved performances of the proposed ensemble over the previous ensemble methods using the BiLSTM, CNN, ELMo and BERT models on the MRPC and RTE datasets. We also discuss the generalization mechanism involved in cross-validation along with the performance changes caused by the hyper-parameter of cross-validation.