Myungji Lee  Hongseok Kwon  WonKee Lee  Jong-Hyeok Lee

10.5626/JOK.2022.49.9.735

summarization movie summarization Multimodal emotion dynamics

Movie summarization is the task of summarizing a full-length movie by creating a short video summary containing its most informative scenes. This paper proposes an automatic movie summarization model that comprehensively considers the three main elements of the movie: characters, plot, and video information for movie summary. To accurately identify major events on the movie plot, we propose a Transformer-based architecture that uses the movie script"s dialogue information and the main characters" emotion dynamics information as model training features, and then combines the script and video information. Through experiments, the proposed method is shown to be helpful in increasing the accuracy of identifying major events in movies and consequently improves the quality of movie summaries.