Digital Library[ Search Result ]
Multi-task Learning Approach Based on Pre-trained Language Models Using Temporal Relations
Chae-Gyun Lim, Kyo-Joong Oh, Ho-Jin Choi
http://doi.org/10.5626/JOK.2023.50.1.25
In the research on natural language understanding that can perform multiple tasks and produce a model that provides general performance, various studies of multi-task learning techniques are being attempted. In addition, documents written in natural language typically contain time-related information, and accurate recognize such information is essential to understand the overall content and context of the document. In this paper, we propose a multi-task learning technique that incorporates a temporal relation extraction task into the learning process of NLU tasks to use the temporal contextual information of Korean input sentences. In order to reflect the characteristics of multi-task learning, a new task for extracting temporal relations is designed, and the model is configured to learn in conjunction with existing NLU tasks. In the experiment, the difference in performance was analyzed by learning the effect of various task combinations and the temporal relationships compared to the case where only the existing NLU task is used. Through the experimental results, we discuss that the overall performance of the multi-task combination is higher than that of individual tasks, especially when temporal relationship with the name entity recognition shows greatly improved performance.
Automatic Extraction of Sentence Embedding Features for Question Similarity Analysis in Dialogues
Kyo-Joong Oh, Dongkun Lee, Chae-Gyun Lim, Ho-Jin Choi
http://doi.org/10.5626/JOK.2019.46.9.909
This paper describes a method for the automatic extraction of feature vectors that can be used to analyze the similarity among natural language sentences. Similarity analysis among sentences is a necessary aspect of measuring semantic or structural similarity in natural language understanding. The analysis results can be used to find answers in Question and Answer (Q&A) systems and dialogue systems. The similarity analysis uses sentence vectors extracted by two deep learning models: the Recurrent Neural Network (RNN) to reflect sequential information of expressions such as syllables and semantic morphemes, and the Convolutional Neural Network (CNN) for characterizing the appearance patterns of similar expressions such as words or phrases. In this paper, we examine the accuracy and quality of the method using sentence vectors that are automatically extracted by the models from dialogues related to banking service. This method can find more similar questions and answers in FAQs than existing methods. The automatic feature extraction method can be used to analyze the similarity of Korean sentences across various application domains and systems.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr