Quality Estimation of Machine Translation using Dual-Encoder Architecture

Dam Heo, Wonkee Lee, Jong-Hyeok Lee

http://doi.org/10.5626/JOK.2022.49.7.521

Quality estimation (QE) is the task of estimating the quality of given machine translations (MTs) without their reference translations. A recent research trend is to apply transfer learning to a pre-training model based on Transformer encoder with a parallel corpus in QE. In this paper, we proposed a dual-encoder architecture that learns a monolingual representation of each respective language in encoders. Thereafter, it learns a cross-lingual representation of each language in cross-attention networks. Thus, it overcomes the limitations of a single-encoder architecture in cross-lingual tasks, such as QE. We proved that the dual-encoder architecture is structurally more advantageous over the single-encoder architecture and furthermore, improved the performance and stability of the dual-encoder model in QE by applying the pre-trained language model to the dual-encoder model. Experiments were conducted on WMT20 QE data for En-De pair. As pre-trained models, our model employs English BERT (Bidirectional Encoder Representations from Transformers) and German BERT to each encoder and achieves the best performance.