Jangseong Bae  Hyunsun Hwang  Changki Lee

10.5626/JOK.2019.46.7.690

machine translation Multimodal Deep Learning image information decoding gate

Machine translation automatically converts a text in one language into another language. Conventional machine translations use only texts for translation which is a disadvantage in that various information related to input text cannot be utilized. In recent years, multimodal machine translation models have emerged that use images related to input text as additional inputs, unlike conventional machine translations which use only textual data. In this paper, image information was added at decoding time of machine translation according to recent research trends and used for English-to-Korean automated translation. In addition, we propose a model with a decoding gate to adjust the textual and image information at the decoding time. Our experimental results show that the proposed method resulted in better performance than the non-gated model.