Digital Library[ Search Result ]
Enhancing Passage Selection and Answer Generation in FiD Systems Using Relevance Gating
Seung-ho Choi, Shihyun Park, Minsang Kim, Chansol Park, Junho Wang, Ji-Yoon Kim, Bong-Su Kim
http://doi.org/10.5626/JOK.2025.52.5.385
In this paper, we proposed a novel approach to enhance the performance of the Fusion-in-Decoder (FiD) model in open-domain question answering systems. The FiD model operates by independently encoding multiple passages and then combining them during the decoding stage to generate answers. However, this method has the drawback of not filtering out passages containing unnecessary information, thereby placing an excessive burden on the decoder. To address this issue, we introduced a Relevance Gate inspired by the forget gate of Long Short-Term Memory (LSTM). This gate can evaluate the relevance of each passage in parallel, selectively transmitting information to the decoder, thereby significantly improving the accuracy and efficiency of answer generation. Additionally, we applied a new activation function suitable for open-domain question answering systems instead of the sigmoid function to ensure the model's stability.
Enhancing Retrieval-Augmented Generation Through Zero-Shot Sentence-Level Passage Refinement with LLMs
Taeho Hwang, Soyeong Jeong, Sukmin Cho, Jong C. Park
http://doi.org/10.5626/JOK.2025.52.4.304
This study presents a novel methodology designed to enhance the performance and effectiveness of Retrieval-Augmented Generation (RAG) by utilizing Large Language Models (LLMs) to eliminate irrelevant content at the sentence level from retrieved documents. This approach refines the content of passages exclusively through LLMs, avoiding the need for additional training or data, with the goal of improving the performance in knowledge-intensive tasks. The proposed method was tested in an open-domain question answering (QA) environment, where it demonstrated its ability to effectively remove unnecessary content and outperform over traditional RAG methods. Overall, our approach has proven effective in enhancing performance compared to conventional RAG techniques and has shown the capability to improve RAG's accuracy in a zero-shot setting without requiring additional training data.
Multi-Level Attention-Based Generation Model for Long-Term Conversation
Hongjin Kim, Bitna Keum, Jinxia Huang, Ohwoog Kwon, Harksoo Kim
http://doi.org/10.5626/JOK.2025.52.2.117
Research into developing more human-like conversational models is actively underway, utilizing persona memory to generate responses. Many existing studies employ a separate retrieval model to identify relevant personas from memory, which can slow down the overall system and make it cumbersome. Additionally, these studies primarily focused on ability to respond by reflecting a persona well. However, the ability to determine the necessity of referencing a persona should precede this. Therefore, in this paper, we propose a model that does not use a retriever. Instead, the need to reference memory was determined through multi-level attention operations within the generation model itself. If a reference is deemed necessary, the response reflects the relevant persona; Otherwise, the response focuses on the conversational context. Experimental results confirm that our proposed model operates effectively in long-term conversations.
Explainable Supporting Facts Generation via Query-Focused Multi-Document Summarization for Open Domain Question Answering Model
http://doi.org/10.5626/JOK.2024.51.11.1020
"Open domain question answering system requires external knowledge not satisfied by knowledge inherent in the language model to answer a given query. It is a technology that is being studied importantly for solving the hallucination problem that occurs in recent large language models. In this paper, we propose a model that utilizes structural information of Query-attentive Semantic Graph (QSG) to summarize information between distant documents based on a query and utilize it as supporting factors for a multi-document-based question answering system. Query-based supporting factors generated by summarizing can improve answer generation performance and show better explainability than extracted supporting factors."
SBERT-PRO: Predicate Oriented Sentence Embedding Model for Intent and Event Detection
Dongryul Ko, Jeayun Lee, Dahee Lee, Yuri Son, Sangmin Kim, Jaeeun Jang, Munhyeong Kim, Sanghyun Park, Jaieun Kim
http://doi.org/10.5626/JOK.2024.51.2.165
Intent detection is a crucial task in conversational systems for understanding user intentions. Additionally, event detection is vital for identifying important events within various texts, including news articles, social media posts, and reports. Among diverse approaches, the sentence embedding similarity-based method has been widely adopted to solve open-domain classification tasks. However, conventional embedding models tend to focus on specific keywords within a sentence and are not suitable for tasks that require a high-level semantic understanding of a sentence as opposed to a narrow focus on specific details within a sentence. This limitation becomes particularly evident in tasks such as intent detection, which requires a broader understanding of the intention of a sentence, and event detection, which requires an emphasis on actual events within a sentence. In this paper, we construct a training dataset suitable for intent and event detection using entity attribute information and entity relation information. Our approach is inspired by the significance of emphasizing the embedding of predicates, which unfold the content of a sentence, as opposed to focusing on entity attributes within a sentence. Furthermore, we suggest an adaptive learning strategy for the existing sentence embedding model and demonstrate that our proposed model, SBERT-PRO (PRedicate Oriented), outperforms conventional models
R²FID: Joint Reranker in Fusion-In-Decoder for Open Domain Question Answering over Tables
Sung-Min Lee, Eunhwan Park, Daeryong Seo, Donghyeon Jeon, Inho Kang, Seung-Hoon Na
http://doi.org/10.5626/JOK.2023.50.10.874
Open Domain Question Answering is a challenging problem that aims to generate an answer where reference documents relevant to a question are not provided. Considering that the importance of the QA system in structured data such as tables has recently gradually increased, this paper presents a method for table open domain question answering of Korean, focusing on tabular contents appearing in Wikipedia. In addition, we extensively apply the Joint Reranker based Fusion-In-Decoder to address limitations entailed in table retrieval, Resulting methods based on Joint Reranker led to improvements of an EM of 3.36 and a F1-Score of 3.25 over open domain question answering tasks.
Improved Open-Domain Conversation Generative Model via Denoising Training of Guide Responses
Bitna Keum, Hongjin Kim, Jinxia Huang, Ohwoog Kwon, Harksoo Kim
http://doi.org/10.5626/JOK.2023.50.10.851
In recent open-domain conversation research, research is actively conducted to combine the strengths of retrieval models and generative models while overcoming their respective weaknesses. However, there is a problem where the generative model either disregards the retrieved response or copies the retrieved response as it is to generate a response. In this paper, we propose a method of mitigating the aforementioned problems. To alleviate the former problem, we filter the retrieved responses and use the gold response together. To address the latter problem, we perform noising on the gold response and the retrieved responses. The generative model enhances the ability to generate responses via denoising training. The effectiveness of our proposed method is verified through human and automatic evaluation.
CommonAI: Quantitative and Qualitative Analysis for Automatic-generation of Commonsense Reasoning Conversations Suitable for AI
Hyeon Gyu Shin, Hyun Jo You, Young Sook Song
http://doi.org/10.5626/JOK.2023.50.5.407
Human-like common sense reasoning is now considered an essential component for improving the quality of natural language generation for chatbots and conversational agents, However, there is no clear consensus at present about to what extent AI systems require common sense. We discussed common sense requirements for AI chatbots based on quantitative and qualitative analysis of results from two experimental surveys to show differences between gender and age groups and variations according to conversation topics. The contribution of this paper is to refine preferences for chatbot conversations that provide useful information and show an appropriate level of empathy.
Performance Improvement of a Korean Open Domain Q&A System by Applying the Trainable Re-ranking and Response Filtering Model
Hyeonho Shin, Myunghoon Lee, Hong-Woo Chun, Jae-Min Lee, Sung-Pil Choi
http://doi.org/10.5626/JOK.2023.50.3.273
Research on Open Domain Q&A, which can identify answers to user inquiries without preparing the target paragraph in advance, is currently being undertaken as deep learning technology is used for natural language processing. However, existing studies have limitations in semantic matching using keyword-based information retrieval. To supplement this, deep learning-based information retrieval research is in progress. But there are not many domestic studies that have been empirically applied to real systems. In this paper, a two-step performance enhancement method was proposed to improve the performance of the Korean open domain Q&A system. The proposed method is a method of sequentially applying a machine learning-based re-ranking model and a response filtering model to a baseline system in which a search engine and an MRC model was combined. In the case of the baseline system, the initial performance was an F1 score of 74.43 and an EM score of 60.79, and it was confirmed that the performance improved to an F1 score of 82.5 and an EM score of 68.82 when the proposed method was used.
Response-Considered Query Token Importance Weight Calculator with Potential Response for Generating Query-Relevant Responses
So-Eon Kim, Choong Seon Hong, Seong-Bae Park
http://doi.org/10.5626/JOK.2022.49.8.601
The conversational response generator(CRG) has made great progress through the sequence-to-sequence model, but it often generates an over-general response which can be a response to all queries or an inappropriate response. Some efforts have been made to modify the traditional loss function to solve this problem and reduce the generation of irrelevant responses to the query by solving the problem of the lack of background knowledge of the CRG, but they did not solve both problems. This paper propose the use of a query token importance calculator because the cause of generating unrelated and overly general responses is that the CRG does not capture the core of the query. Also, based on the theory that the questioner induces a specific response from the listener and designs the speech, this paper proposes to use the golden response to understand the core meaning of the query. The qualitative evaluation confirmed that the response generator using the proposed model was able to generate responses related to the query compared to the model that did not use the proposed model.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr