Digital Library[ Search Result ]
Improving Retrieval Models through Reinforcement Learning with Feedback
Min-Taek Seo, Joon-Ho Lim, Tae-Hyeong Kim, Hwi-Jung Ryu, Du-Seong Chang, Seung-Hoon Na
http://doi.org/10.5626/JOK.2024.51.10.900
Open-domain question answering involves the process of retrieving clues through search to solve problems. In such tasks, it is crucial that the search model provides appropriate clues, as this directly impacts the final performance. Moreover, information retrieval is an important function frequently used in everyday life. This paper recognizes the significance of these challenges and aims to improve performances of search models. Just as the recent trend involves adjusting outputs in decoder models using Reinforcement Learning from Human Feedback (RLHF), this study seeks to enhance search models through the use of reinforcement learning. Specifically, we defined two rewards: the loss of the answer model and the similarity between the retrieved documents and the correct document. Based on these, we applied reinforcement learning to adjust the probability score of the top-ranked document in the search model's document probability distribution. Through this approach, we confirmed the generality of the reinforcement learning method and its potential for further performance improvements.
SCA: Improving Document Grounded Response Generation based on Supervised Cross-Attention
Hyeongjun Choi, Seung-Hoon Na, Beomseok Hong, Youngsub Han, Byoung-Ki Jeon
http://doi.org/10.5626/JOK.2024.51.4.326
Document-grounded response generation is the task of aiming at generating conversational responses by “grounding” the factual evidence on task-specific domain, such as consumer consultation or insurance planning, where the evidence is obtained from the retrieved relevant documents in response to a user’s question under the current dialogue context. In this study, we propose supervised cross-attention (SCA) to enhance the ability of the response generation model to find and incorporate “response-salient snippets” (i.e., spans or contents), which are parts of the retrieved document that should be included and maintained in the actual answer generation. SCA utilizes the additional supervised loss that focuses cross-attention weights on the response-salient snippets, and this attention supervision likely enables a decoder to effectively generate a response in a “saliency-grounding” manner, by strongly attending to the important parts in the retrieved document. Experiment results on MultiDoc2Dial show that the use of SCA and additional performance improvement methods leads to the increase of 1.13 in F1 metric over the existing SOTA, and reveals that SCA leads to the increase of 0.25 in F1.
R²FID: Joint Reranker in Fusion-In-Decoder for Open Domain Question Answering over Tables
Sung-Min Lee, Eunhwan Park, Daeryong Seo, Donghyeon Jeon, Inho Kang, Seung-Hoon Na
http://doi.org/10.5626/JOK.2023.50.10.874
Open Domain Question Answering is a challenging problem that aims to generate an answer where reference documents relevant to a question are not provided. Considering that the importance of the QA system in structured data such as tables has recently gradually increased, this paper presents a method for table open domain question answering of Korean, focusing on tabular contents appearing in Wikipedia. In addition, we extensively apply the Joint Reranker based Fusion-In-Decoder to address limitations entailed in table retrieval, Resulting methods based on Joint Reranker led to improvements of an EM of 3.36 and a F1-Score of 3.25 over open domain question answering tasks.
BERT-based Two-Stage Classification Models and Co-Attention Mechanism for Diagnosing Dementia and Schizophrenia-related Disease
Min-Kyo Jung, Seung-Hoon Na, Ko Woon Kim, Byoung-Soo Shin, Young-Chul Chung
http://doi.org/10.5626/JOK.2022.49.12.1071
Noting the recently increasing number of patients, we present deep learning methods for automatically diagnosing dementia and schizophrenia by exploring the use of the novel two-stage classification and the co-attention mechanism. First, the two-stage classification consists of two steps-the perplexity-based classification and the standard BERT-based classification. 1) the perplexity-based classification first prepares two types of BERTs, i.e., control-specific and patients-specific BERTs, pretrained from transcripts for controls and patients as the additional pretraining datasets, respectively, and then performs a simple threshold-based classification based on the difference between perplexity values of two BERTs for an input test transcript; then, for ambiguous cases where the perplexity difference only does not provide sufficient evidence for the classification, the standard BERT-based classification is performed based on a fine-tuned BERT. Second, the co-attention mechanism enriches the BERT-based representations from a doctor’s transcript and a client’s one by applying the cross-attention over them using the shared affinity matrix, and performs the classification based on the enriched co-attentive representations. Experiment results on a large-scale dataset of Korean transcripts show that the proposed two-stage classification outperforms the baseline BERT model on 4 out of 7 subtasks and the use of the co-attention mechanism achieves the best F1 score for 4 out of 8 subtasks.
Korean Dependency Parsing using Subtree Linking based on Machine Reading Comprehension
Jinwoo Min, Seung-Hoon Na, Jong-Hoon Shin, Young-Kil Kim, Kangil Kim
http://doi.org/10.5626/JOK.2022.49.8.617
In Korean dependency parsing, biaffine attention models have shown state-of-the-art performances; they first obtain head-level and modifier-level representations by applying two multi-layer perceptrons (MLP) on the encoded contextualized word representation, perform the attention by regarding modifier-level representation as a query and head-level one as a key, and take the resulting attention score as a probability of forming a dependency arc between the corresponding two words. However, given two target words (i.e., candidate head and modifier), biaffine attention methods are basically limited to their word-level representations, not being aware of the explicit boundaries of their phrases or subtrees. Thus, without relying on semantically and syntactically enriched phrase-level and subtree-level representations, biaffine attention methods might be not effective in the case that determining a dependency arc is not simple but complicated such as identifying a dependency between “far-distant” words, where these cases may often require subtree or phrase-level information surrounding target words. To address this drawback, this paper presents the use of dependency paring framework based on machine reading comprehension (MRC) that explicitly utilizes the subtree-level information by mapping a given child subtree and its parent subtree to a question and an answer, respectively. The experiment results on standard datasets of Korean dependency parsing shows that the MRC-based dependency paring outperforms the biaffine attention model. In particular, the results further given observations that improvements in performances are likely strong in long sentences, comparing to short ones.
PrefixLM for Korean Text Summarization
Kun-Hui Lee, Seung-Hoon Na, Joon-Ho Lim, Tae-Hyeong Kim, Du-Seong Chang
http://doi.org/10.5626/JOK.2022.49.6.475
In this paper, we examine the effectiveness of PrefixLM that consists of half of the parameters of the T5"s encoder-decoder architecture for Korean text generation tasks. Different from T5 where input and output sequences are separately provided, the transformer block of PrefixLM takes a single sequence that concatenates both input and output sequences. By designing the attention mask, PrefixLM performs uni- and bi-directional attentions on input and output sequences, respectively, thereby enabling to perform two roles of encoder and decoder with a single transformer block. Experiment results on Korean abstractive document summarization task show that PrefixLM leads to performance increases of 2.17 and 2.78 more than 2 in Rouge-F1 score over BART and T5, respectively, implying that the PrefixLM is promising in Korean text generation tasks.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr