Digital Library[ Search Result ]
Enhancing Passage Selection and Answer Generation in FiD Systems Using Relevance Gating
Seung-ho Choi, Shihyun Park, Minsang Kim, Chansol Park, Junho Wang, Ji-Yoon Kim, Bong-Su Kim
http://doi.org/10.5626/JOK.2025.52.5.385
In this paper, we proposed a novel approach to enhance the performance of the Fusion-in-Decoder (FiD) model in open-domain question answering systems. The FiD model operates by independently encoding multiple passages and then combining them during the decoding stage to generate answers. However, this method has the drawback of not filtering out passages containing unnecessary information, thereby placing an excessive burden on the decoder. To address this issue, we introduced a Relevance Gate inspired by the forget gate of Long Short-Term Memory (LSTM). This gate can evaluate the relevance of each passage in parallel, selectively transmitting information to the decoder, thereby significantly improving the accuracy and efficiency of answer generation. Additionally, we applied a new activation function suitable for open-domain question answering systems instead of the sigmoid function to ensure the model's stability.
Enhancing Retrieval-Augmented Generation Through Zero-Shot Sentence-Level Passage Refinement with LLMs
Taeho Hwang, Soyeong Jeong, Sukmin Cho, Jong C. Park
http://doi.org/10.5626/JOK.2025.52.4.304
This study presents a novel methodology designed to enhance the performance and effectiveness of Retrieval-Augmented Generation (RAG) by utilizing Large Language Models (LLMs) to eliminate irrelevant content at the sentence level from retrieved documents. This approach refines the content of passages exclusively through LLMs, avoiding the need for additional training or data, with the goal of improving the performance in knowledge-intensive tasks. The proposed method was tested in an open-domain question answering (QA) environment, where it demonstrated its ability to effectively remove unnecessary content and outperform over traditional RAG methods. Overall, our approach has proven effective in enhancing performance compared to conventional RAG techniques and has shown the capability to improve RAG's accuracy in a zero-shot setting without requiring additional training data.
Evaluating Table QA with Generative Language Models
Kyungkoo Min, Jooyoung Choi, Myoseop Sim, Minjun Park, Stanley Jungkyu Choi
http://doi.org/10.5626/JOK.2024.51.10.892
Tables in documents can be described as collections of information that condense and aggregate important data. Research on table question answering techniques focusing on querying such tables is underway, with studies utilizing language models showing promising results. This study applied an emerging generative language model technology to table question answering, examining outcomes based on changes in the language model and prompts. It analyzed results using methods suitable for short-answer and generative outcomes. Applying our custom-developed EXAONE 1.7B model to the KorWiki dataset yielded an EM of 92.49 and an F1 score of 94.81. This demonstrates that fine-tuning smaller models can achieve better performance than larger models such as GPT-4. Additionally, the EXAONE 25B model exhibited the best performance among tested models on the KorQuAD2Table dataset.
SCA: Improving Document Grounded Response Generation based on Supervised Cross-Attention
Hyeongjun Choi, Seung-Hoon Na, Beomseok Hong, Youngsub Han, Byoung-Ki Jeon
http://doi.org/10.5626/JOK.2024.51.4.326
Document-grounded response generation is the task of aiming at generating conversational responses by “grounding” the factual evidence on task-specific domain, such as consumer consultation or insurance planning, where the evidence is obtained from the retrieved relevant documents in response to a user’s question under the current dialogue context. In this study, we propose supervised cross-attention (SCA) to enhance the ability of the response generation model to find and incorporate “response-salient snippets” (i.e., spans or contents), which are parts of the retrieved document that should be included and maintained in the actual answer generation. SCA utilizes the additional supervised loss that focuses cross-attention weights on the response-salient snippets, and this attention supervision likely enables a decoder to effectively generate a response in a “saliency-grounding” manner, by strongly attending to the important parts in the retrieved document. Experiment results on MultiDoc2Dial show that the use of SCA and additional performance improvement methods leads to the increase of 1.13 in F1 metric over the existing SOTA, and reveals that SCA leads to the increase of 0.25 in F1.
KorQuAD 2.0: Korean QA Dataset for Web Document Machine Comprehension
Youngmin Kim, Seungyoung Lim, Hyunjeong Lee, Soyoon Park, Myungji Kim
http://doi.org/10.5626/JOK.2020.47.6.577
KorQuAD 2.0 is a Korean question and answering dataset consisting of a total of 100,000+ pairs. There are three major differences from KorQuAD 1.0, which is the standard Korean Q & A data. The first is that a given document is a whole Wikipedia page, not just one or two paragraphs. Second, because the document also contains tables and lists, it is necessary to understand the document structured with HTML tags. Finally, the answer can be a long text covering not only word or phrase units, but paragraphs, tables, and lists. As a baseline model, BERT Multilingual is used, released by Google as an open source. It shows 46.0% F1 score, a very low score compared to 85.7% of the human F1 score. It indicates that this data is a challenging task. Additionally, we increased the performance by no-answer data augmentation. Through the distribution of this data, we intend to extend the limit of MRC that was limited to plain text to real world tasks of various lengths and formats.
Korean Coreference Resolution using the Multi-pass Sieve
Cheon-Eum Park, Kyoung-Ho Choi, Changki Lee
Coreference resolution finds all expressions that refer to the same entity in a document. Coreference resolution is important for information extraction, document classification, document summary, and question answering system. In this paper, we adapt Stanford`s Multi-pass sieve system, the one of the best model of rule based coreference resolution to Korean. In this paper, all noun phrases are considered to mentions. Also, unlike Stanford`s Multi-pass sieve system, the dependency parse tree is used for mention extraction, a Korean acronym list is built ‘dynamically’. In addition, we propose a method that calculates weights by applying transitive properties of centers of the centering theory when refer Korean pronoun. The experiments show that our system obtains MUC 59.0%, B3 59.5%, Ceafe 63.5%, and CoNLL(Mean) 60.7%.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr