Confident Multiple Choice Learning-based Ensemble Model for Video Question-Answering

Gyu-Min Park, A-Yeong Kim, Seong-Bae Park

http://doi.org/10.5626/JOK.2022.49.4.284

The task of Video Question Answering(VQA) focuses on finding an answer to a question about the given video. VQA models should be able to process the multi-modal information and time-series information in the video in order to answer the questions appropriately. However, designing a model that answers all types of questions robustly is a challenging problem and takes a lot of time. Since the method of combining existing proposed models has different viewpoints of representing video by each model, ensemble models and ensemble learning methods that can reflect each model"s viewpoints are essential to improve the performance. This paper proposes an ensemble model for VQA with Confident Multiple Choice Learning(CMCL) to improve the performance on accuracy. Our experiment shows that the proposed model outperforms other VQA models and ensemble learning methods on the DramaQA dataset. We analyze the impact of the ensemble learning methods on each model.