Digital Library[ Search Result ]
Enhancing Passage Selection and Answer Generation in FiD Systems Using Relevance Gating
Seung-ho Choi, Shihyun Park, Minsang Kim, Chansol Park, Junho Wang, Ji-Yoon Kim, Bong-Su Kim
http://doi.org/10.5626/JOK.2025.52.5.385
In this paper, we proposed a novel approach to enhance the performance of the Fusion-in-Decoder (FiD) model in open-domain question answering systems. The FiD model operates by independently encoding multiple passages and then combining them during the decoding stage to generate answers. However, this method has the drawback of not filtering out passages containing unnecessary information, thereby placing an excessive burden on the decoder. To address this issue, we introduced a Relevance Gate inspired by the forget gate of Long Short-Term Memory (LSTM). This gate can evaluate the relevance of each passage in parallel, selectively transmitting information to the decoder, thereby significantly improving the accuracy and efficiency of answer generation. Additionally, we applied a new activation function suitable for open-domain question answering systems instead of the sigmoid function to ensure the model's stability.
Enhancing Retrieval-Augmented Generation Through Zero-Shot Sentence-Level Passage Refinement with LLMs
Taeho Hwang, Soyeong Jeong, Sukmin Cho, Jong C. Park
http://doi.org/10.5626/JOK.2025.52.4.304
This study presents a novel methodology designed to enhance the performance and effectiveness of Retrieval-Augmented Generation (RAG) by utilizing Large Language Models (LLMs) to eliminate irrelevant content at the sentence level from retrieved documents. This approach refines the content of passages exclusively through LLMs, avoiding the need for additional training or data, with the goal of improving the performance in knowledge-intensive tasks. The proposed method was tested in an open-domain question answering (QA) environment, where it demonstrated its ability to effectively remove unnecessary content and outperform over traditional RAG methods. Overall, our approach has proven effective in enhancing performance compared to conventional RAG techniques and has shown the capability to improve RAG's accuracy in a zero-shot setting without requiring additional training data.
Explainable Supporting Facts Generation via Query-Focused Multi-Document Summarization for Open Domain Question Answering Model
http://doi.org/10.5626/JOK.2024.51.11.1020
"Open domain question answering system requires external knowledge not satisfied by knowledge inherent in the language model to answer a given query. It is a technology that is being studied importantly for solving the hallucination problem that occurs in recent large language models. In this paper, we propose a model that utilizes structural information of Query-attentive Semantic Graph (QSG) to summarize information between distant documents based on a query and utilize it as supporting factors for a multi-document-based question answering system. Query-based supporting factors generated by summarizing can improve answer generation performance and show better explainability than extracted supporting factors."
Evaluating Table QA with Generative Language Models
Kyungkoo Min, Jooyoung Choi, Myoseop Sim, Minjun Park, Stanley Jungkyu Choi
http://doi.org/10.5626/JOK.2024.51.10.892
Tables in documents can be described as collections of information that condense and aggregate important data. Research on table question answering techniques focusing on querying such tables is underway, with studies utilizing language models showing promising results. This study applied an emerging generative language model technology to table question answering, examining outcomes based on changes in the language model and prompts. It analyzed results using methods suitable for short-answer and generative outcomes. Applying our custom-developed EXAONE 1.7B model to the KorWiki dataset yielded an EM of 92.49 and an F1 score of 94.81. This demonstrates that fine-tuning smaller models can achieve better performance than larger models such as GPT-4. Additionally, the EXAONE 25B model exhibited the best performance among tested models on the KorQuAD2Table dataset.
Robust Korean Table Machine Reading Comprehension across Various Domains
Sanghyun Cho, Hye-Lynn Kim, Hyuk-chul Kwon
http://doi.org/10.5626/JOK.2023.50.12.1102
Unlike regular text data, tabular data has structural features that allow it to represent compressed information. This has led to their use in a variety of domains, and machine reading comprehension of tables has become an increasingly important aspect of Machine Reading Comprehension(MRC). However, the structure of tables and the knowledge required for each domain are different, and when a language model is trained for a single domain, the evaluation performance of the model in other domains is likely to be reduced, resulting in poor generalization performance. To overcome this, it is important to build datasets of various domains and apply various techniques rather than simply pre-trained models. In this study, we design a language model that learns cross-domain invariant linguistic features to improve domain generalization performance. We applied adversarial training to improve performance on evaluation datasets in each domain and modify the structure of the model by adding an embedding layer and a transformer layer specialized for tabular data. When applying adversarial learning, we found that the model with a structure that does not add table-specific embeddings improves performance. On the other hand, while adding a table-specific transformer layer and having the added layer receive additional table-specific embeddings as input, shows the best performance on data from all domains.
Knowledge-based Supporting Facts Generation Model for Question and Answer
http://doi.org/10.5626/JOK.2023.50.11.940
In this study, we intend to create supporting facts from the knowledge base to add information to the question and answer process, and provide a form that is easy for humans to read. Data from two knowledge bases, DBpedia and Wikidata, related to supporting documents in HotpotQA were collected through crawling, and the supporting facts generators were trained using collected triples. The answer generator was trained with generated supporting facts and questions as inputs. Regardless of both DBpedia and Wikidata, supporting facts generated based on the knowledge base improved answer generation performance by providing positive additional information about questions, and generated human-understandable sentences.
R²FID: Joint Reranker in Fusion-In-Decoder for Open Domain Question Answering over Tables
Sung-Min Lee, Eunhwan Park, Daeryong Seo, Donghyeon Jeon, Inho Kang, Seung-Hoon Na
http://doi.org/10.5626/JOK.2023.50.10.874
Open Domain Question Answering is a challenging problem that aims to generate an answer where reference documents relevant to a question are not provided. Considering that the importance of the QA system in structured data such as tables has recently gradually increased, this paper presents a method for table open domain question answering of Korean, focusing on tabular contents appearing in Wikipedia. In addition, we extensively apply the Joint Reranker based Fusion-In-Decoder to address limitations entailed in table retrieval, Resulting methods based on Joint Reranker led to improvements of an EM of 3.36 and a F1-Score of 3.25 over open domain question answering tasks.
Performance Improvement of a Korean Open Domain Q&A System by Applying the Trainable Re-ranking and Response Filtering Model
Hyeonho Shin, Myunghoon Lee, Hong-Woo Chun, Jae-Min Lee, Sung-Pil Choi
http://doi.org/10.5626/JOK.2023.50.3.273
Research on Open Domain Q&A, which can identify answers to user inquiries without preparing the target paragraph in advance, is currently being undertaken as deep learning technology is used for natural language processing. However, existing studies have limitations in semantic matching using keyword-based information retrieval. To supplement this, deep learning-based information retrieval research is in progress. But there are not many domestic studies that have been empirically applied to real systems. In this paper, a two-step performance enhancement method was proposed to improve the performance of the Korean open domain Q&A system. The proposed method is a method of sequentially applying a machine learning-based re-ranking model and a response filtering model to a baseline system in which a search engine and an MRC model was combined. In the case of the baseline system, the initial performance was an F1 score of 74.43 and an EM score of 60.79, and it was confirmed that the performance improved to an F1 score of 82.5 and an EM score of 68.82 when the proposed method was used.
KorSciQA 2.0: Question Answering Dataset for Machine Reading Comprehension of Korean Papers in Science & Technology Domain
Hyesoo Kong, Hwamook Yoon, Mihwan Hyun, Hyejin Lee, Jaewook Seol
http://doi.org/10.5626/JOK.2022.49.9.686
Recently, the performance of the Machine Reading Comprehension(MRC) system has been increased through various open-ended Question Answering(QA) task, and challenging QA task which has to comprehensively understand multiple text paragraphs and make discrete inferences is being released to train more intelligent MRC systems. However, due to the absence of a QA dataset for complex reasoning to understand academic information in Korean, MRC research on academic papers has been limited. In this paper, we constructed a QA dataset, KorSciQA 2.0, for the full text including abstracts of Korean academic papers and divided the difficulty level into general, easy, and hard for discriminative MRC systems. A methodology, process, and system for constructing KorSciQA 2.0 were proposed. We conducted MRC performance evaluation experiments and when fine-tuning based on the KorSciBERT model, which is a Korean-based BERT model for science and technology domains, the F1 score was 80.76%, showing the highest performance.
Confident Multiple Choice Learning-based Ensemble Model for Video Question-Answering
Gyu-Min Park, A-Yeong Kim, Seong-Bae Park
http://doi.org/10.5626/JOK.2022.49.4.284
The task of Video Question Answering(VQA) focuses on finding an answer to a question about the given video. VQA models should be able to process the multi-modal information and time-series information in the video in order to answer the questions appropriately. However, designing a model that answers all types of questions robustly is a challenging problem and takes a lot of time. Since the method of combining existing proposed models has different viewpoints of representing video by each model, ensemble models and ensemble learning methods that can reflect each model"s viewpoints are essential to improve the performance. This paper proposes an ensemble model for VQA with Confident Multiple Choice Learning(CMCL) to improve the performance on accuracy. Our experiment shows that the proposed model outperforms other VQA models and ensemble learning methods on the DramaQA dataset. We analyze the impact of the ensemble learning methods on each model.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr