TY  - JOUR
T1  - Automatic Text Summarization Based on Selective OOV Copy Mechanism with BERT Embedding
AU  - Lee, Tae-Seok 
AU  - Kang, Seung-Shik 
JO  - Journal of KIISE, JOK
PY  - 2020
DA  - 2020/1/14
DO  - 10.5626/JOK.2020.47.1.36
KW  - random masked OOV
KW  - morpheme-to-sentence converter
KW  - text summarization
KW  - recognition of unknown word
KW  - deep-learning
KW  - generative summarization
AB  - Automatic text summarization is a process of shortening a text document via extraction or abstraction. Abstractive text summarization involves using pre-generated word embedding information. Low-frequency but salient words such as terminologies are seldom included in dictionaries, that are so called, out-of-vocabulary (OOV) problems. OOV deteriorates the performance of the encoder-decoder model in the neural network. To address OOV words in abstractive text summarization, we propose a copy mechanism to facilitate copying new words in the target document and generating summary sentences. Different from previous studies, the proposed approach combines accurately pointing information, selective copy mechanism, embedded by BERT, randomly masking OOV, and converting sentences from morpheme. Additionally, the neural network gate model to estimate the generation probability and the loss function to optimize the entire abstraction model was applied. Experimental results demonstrate that ROUGE-1 (based on word recall) and ROUGE-L (longest used common subsequence) of the proposed encoding-decoding model have been improved at 54.97 and 39.23, respectively.