Digital Library[ Search Result ]
Response-Considered Query Token Importance Weight Calculator with Potential Response for Generating Query-Relevant Responses
So-Eon Kim, Choong Seon Hong, Seong-Bae Park
http://doi.org/10.5626/JOK.2022.49.8.601
The conversational response generator(CRG) has made great progress through the sequence-to-sequence model, but it often generates an over-general response which can be a response to all queries or an inappropriate response. Some efforts have been made to modify the traditional loss function to solve this problem and reduce the generation of irrelevant responses to the query by solving the problem of the lack of background knowledge of the CRG, but they did not solve both problems. This paper propose the use of a query token importance calculator because the cause of generating unrelated and overly general responses is that the CRG does not capture the core of the query. Also, based on the theory that the questioner induces a specific response from the listener and designs the speech, this paper proposes to use the golden response to understand the core meaning of the query. The qualitative evaluation confirmed that the response generator using the proposed model was able to generate responses related to the query compared to the model that did not use the proposed model.
A Product Review Summarization Considering Additional Information
Jaeyeun Yoon, Ig-hoon Lee, Sang-goo Lee
http://doi.org/10.5626/JOK.2020.47.2.180
Automatic document summarization is a task that generates the document in a suitable form from an existing document for a certain user or occasion. As use of the Internet increases, the various data including texts are exploding and the value of document summarization technology is growing. While the latest deep learning-based models show reliable performance in document summarization, the problem is that performance depends on the quantity and quality of the training data. For example, it is difficult to generate reliable summarization with existing models from the product review text of online shopping malls because of typing errors and grammatically wrong sentences. Online malls and portal web services are struggling to solve this problem. Thus, to generate an appropriate document summary in poor condition relative to quality and quantity of the product review learning data, this study proposes a model that generates product review summaries with additional information. We found through experiments that this model showed improved performances in terms of relevance and readability than the existing model for product review summaries.
Solving for Redundant Repetition Problem of Generating Summarization using Decoding History
Jaehyun Ryu, Yunseok Noh, Su Jeong Choi, Seyoung Park, Seong-Bae Park
http://doi.org/10.5626/JOK.2019.46.6.535
Neural attentional sequence-to-sequence models have achieved great success in abstractive summarization. However, the model is limited by several challenges including repetitive generation of words, phrase and sentences in the decoding step. Many studies have attempted to address the problem by modifying the model structure. Although the consideration of actual history of word generation is crucial to reduce word repetition, these methods, however, do not consider the decoding history of generated sequence. In this paper, we propose a new loss function, called ‘Repeat Loss’ to avoid repetitions. The Repeat Loss directly prevents the model from repetitive generation of words by giving a loss penalty to the generation probability of words already generated in the decoding history. Since the propose Repeat Loss does not need a special network structure, the loss function is applicable to any existing sequence-to-sequence models. In experiments, we applied the Repeat Loss to a number of sequence-to-sequence model based summarization systems and trained them on both Korean and CNN/Daily Mail summarization datasets. The results demonstrate that the proposed method reduced repetitions and produced high-quality summarization.
Resolution of Answer-Repetition Problems in a Generative Question-Answering Chat System
http://doi.org/10.5626/JOK.2018.45.9.925
A question-answering (QA) chat system is a chatbot that responds to simple factoid questions by retrieving information from knowledge bases. Recently, many chat systems based on sequence-to-sequence neural networks have been implemented and have shown new possibilities for generative models. However, the generative chat systems have word repetition problems, in that the same words in a response are repeatedly generated. A QA chat system also has similar problems, in that the same answer expressions frequently appear for a given question and are repeatedly generated. To resolve this answer-repetition problem, we propose a new sequence-to-sequence model reflecting a coverage mechanism and an adaptive control of attention (ACA) mechanism in a decoder. In addition, we propose a repetition loss function reflecting the number of unique words in a response. In the experiments, the proposed model performed better than various baseline models on all metrics, such as accuracy, BLEU, ROUGE-1, ROUGE-2, ROUGE-L, and Distinct-1.
Regularizing Korean Conversational Model by Applying Denoising Mechanism
Tae-Hyeong Kim, Yunseok Noh, Seong-Bae Park, Se-Yeong Park
http://doi.org/10.5626/JOK.2018.45.6.572
A conversation system is a system that responds appropriately to input utterances. Recently, the sequence-to-sequence framework has been widely used as a conversation-learning model. However, the conversation model learned in such a way often generates a safe and dull response that does not provide appropriate information or sophisticated meaning. In addition, this model is also useless for input utterances appearing in various forms, such as with changed ending words or changed word order. To solve these problems, we propose a denoising response generation model applying a denoising mechanism. By injecting noise into original input, the proposed method creates a model that will stochastically experience new input made up of items that were not included in the original data during the training process. This data augmentation effect regularizes the model and allows the realization of a robust model. We evaluate our model using 90k input utterances-responses from Korean conversation pair data. The proposed model achieves better results compared to a baseline model on both ROUGE F1 score and qualitative evaluations by human annotators.
Knowledge Embedding Method for Implementing a Generative Question-Answering Chat System
Sihyung Kim, Hyeon-gu Lee, Harksoo Kim
http://doi.org/10.5626/JOK.2018.45.2.134
A chat system is a computer program that understands user"s miscellaneous utterances and generates appropriate responses. Sometimes a chat system needs to answer users’ simple information-seeking questions. However, previous generative chat systems do not consider how to embed knowledge entities (i.e., subjects and objects in triple knowledge), essential elements for question-answering. The previous chat models have a disadvantage that they generate same responses although knowledge entities in users’ utterances are changed. To alleviate this problem, we propose a knowledge entity embedding method for improving question-answering accuracies of a generative chat system. The proposed method uses a Siamese recurrent neural network for embedding knowledge entities and their synonyms. For experiments, we implemented a sequence-to-sequence model in which subjects and predicates are encoded and objects are decoded. The proposed embedding method showed 12.48% higher accuracies than the conventional embedding method based on a convolutional neural network.
Title Generation Model for which Sequence-to-Sequence RNNs with Attention and Copying Mechanisms are used
http://doi.org/10.5626/JOK.2017.44.7.674
In big-data environments wherein large amounts of text documents are produced daily, titles are very important clues that enable a prompt catching of the key ideas in documents; however, titles are absent for numerous document types such as blog articles and social-media messages. In this paper, a title-generation model for which sequence-to-sequence RNNs with attention and copying mechanisms are employed is proposed. For the proposed model, input sentences are encoded based on bi-directional GRU (gated recurrent unit) networks, and the title words are generated through a decoding of the encoded sentences with keywords that are automatically selected from the input sentences. Regarding the experiments with 93631 training-data documents and 500 test-data documents, the attention-mechanism performances are more effective (ROUGE-1: 0.1935, ROUGE-2: 0.0364, ROUGE-L: 0.1555) than those of the copying mechanism; in addition, the qualitative-evaluation radiative performance of the former is higher.
Sequence-to-sequence based Morphological Analysis and Part-Of-Speech Tagging for Korean Language with Convolutional Features
Jianri Li, EuiHyeon Lee, Jong-Hyeok Lee
Traditional Korean morphological analysis and POS tagging methods usually consist of two steps: 1 Generat hypotheses of all possible combinations of morphemes for given input, 2 Perform POS tagging search optimal result. require additional resource dictionaries and step could error to the step. In this paper, we tried to solve this problem end-to-end fashion using sequence-to-sequence model convolutional features. Experiment results Sejong corpus sour approach achieved 97.15% F1-score on morpheme level, 95.33% and 60.62% precision on word and sentence level, respectively; s96.91% F1-score on morpheme level, 95.40% and 60.62% precision on word and sentence level, respectively.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr