Digital Library[ Search Result ]
Korean Dependency Parsing Using Sequence Labeling
http://doi.org/10.5626/JOK.2024.51.12.1053
Dependency parsing is a crucial step in language analysis. It identifies relationships between words within a sentence. Recently, many models based on a pre-trained transformer have shown impressive performances in various natural language processing research. hey have been also applied to dependency parsing. Generally, traditional approaches to dependency parsing using pre-trained models consist of two main stages: 1) merging token-level embeddings generated by the pre-trained model into word-level embeddings; and 2) analyzing dependency relations by comparing or classifying the merged embeddings. However, due to a large number of parameters and additional layers required for embedding construction, comparison, and classification, these models can be inefficient in terms of time and memory usage. This paper proposes a dependency parsing technique based on sequential labeling to improve the efficiency of training and inference by defining dependency parsing units and simplifying model layers. The proposed model eliminates the necessity of the word-level embedding merging step by utilizing special tokens to define parsing units. It also effectively reduces the number of parameters by simplifying model layers. As a result, the training and inference time is significantly shortened. With these optimizations, the proposed model maintains meaningful performance in dependency parsing.
Explainable Video Search System using Token Space-based Representation
Jiyeol Park, Dooyoung Kim, Youngjoong Ko
http://doi.org/10.5626/JOK.2024.51.12.1068
Query-video retrieval is a field that finds the most relevant video to the query input by the user. For this, existing studies have presented the query and video in an appropriate latent vector space. However, the method of calculating the relevance between the query and the video simply uses the dot product of the two vectors without implying the meaning or explainability. In this paper, we propose a model that converts the query and video into embeddings located in a token-based space, searches the video like a document, and calculates semantic similarity. Experimental results show that the performance of the final model proposed in this paper is improved in Recall@1, Recall@5, and Recall@10 compared to baseline on MSVD dataset. Furthermore, the proposed model is approximately 3.33 times faster than CLIP4Clip. When applying BM25 with minimal modifications, it achieves a speedup of about 208.11 times. Additionally, qualitative evaluations demonstrate that tokens extracted from videos exhibit relevance comparable to subtitles, proving an explainability-based structure.
LLMEE: Enhancing Explainability and Evaluation of Large Language Models through Visual Token Attribution
Yunsu Kim, Minchan Kim, Jinwoo Choi, Youngseok Hwang, Hyunwoo Park
http://doi.org/10.5626/JOK.2024.51.12.1104
Large Language Models (LLMs) have made significant advancements in Natural Language Processing (NLP) and generative AI. However, their complex structure poses challenges in terms of interpretability and reliability. To address this issue, this study proposed LLMEE, a tool designed to visually explain and evaluate the prediction process of LLMs. LLMEE visually represents the impact of each input token on the output, enhancing model transparency and providing insights into various NLP tasks such as Summarization, Question Answering, Text Generation. Additionally, it integrates evaluation metrics such as ROUGE, BLEU, and BLEURTScore, offering both quantitative and qualitative assessments of LLM outputs. LLMEE is expected to contribute to more reliable evaluation and improvement of LLMs in both academic and industrial contexts by facilitating a better understanding of their complex workings and by providing enhanced output quality assessments.
Enhancing Container Security Using Machine Learning Based on Kernel Tracing Logs
Hyeonseok Shin, Minjung Jo, Hosang Yoo, Yongwon Lee, Jiyeon Lee, Byungchul Tak
http://doi.org/10.5626/JOK.2024.51.11.947
The use of container technology has been rapidly increasing as it gains attention in cloud environments. Containers are lighter and more advantageous for deployment than virtual machines because they do not require a separate operating system. However, containers can have security vulnerabilities due to their characteristic of sharing the same host kernel. In this paper, we designed and implemented a security system to address these vulnerabilities by using eBPF technology, kernel tracing logs, and an ensemble machine learning model. Our system can effectively detect attacks leveraging race conditions and the heap spray technique used in kernel memory vulnerabilities. Unlike traditional security policy-based approaches, it allows for rapid and dynamic responses without needing profile creation. For detecting attacks leveraging race conditions, the system achieved over 99% accuracy in Precision, Recall, and F1-Score, while it recorded over 97% accuracy across all metrics for heap spray detection.
Explainable Supporting Facts Generation via Query-Focused Multi-Document Summarization for Open Domain Question Answering Model
http://doi.org/10.5626/JOK.2024.51.11.1020
"Open domain question answering system requires external knowledge not satisfied by knowledge inherent in the language model to answer a given query. It is a technology that is being studied importantly for solving the hallucination problem that occurs in recent large language models. In this paper, we propose a model that utilizes structural information of Query-attentive Semantic Graph (QSG) to summarize information between distant documents based on a query and utilize it as supporting factors for a multi-document-based question answering system. Query-based supporting factors generated by summarizing can improve answer generation performance and show better explainability than extracted supporting factors."
Evaluating Table QA with Generative Language Models
Kyungkoo Min, Jooyoung Choi, Myoseop Sim, Minjun Park, Stanley Jungkyu Choi
http://doi.org/10.5626/JOK.2024.51.10.892
Tables in documents can be described as collections of information that condense and aggregate important data. Research on table question answering techniques focusing on querying such tables is underway, with studies utilizing language models showing promising results. This study applied an emerging generative language model technology to table question answering, examining outcomes based on changes in the language model and prompts. It analyzed results using methods suitable for short-answer and generative outcomes. Applying our custom-developed EXAONE 1.7B model to the KorWiki dataset yielded an EM of 92.49 and an F1 score of 94.81. This demonstrates that fine-tuning smaller models can achieve better performance than larger models such as GPT-4. Additionally, the EXAONE 25B model exhibited the best performance among tested models on the KorQuAD2Table dataset.
Generating Relation Descriptions with Large Language Model for Link Prediction
http://doi.org/10.5626/JOK.2024.51.10.908
The Knowledge Graph is a network consisting of entities and the relations between them. It is used for various natural language processing tasks. One specific task related to the Knowledge Graph is Knowledge Graph Completion, which involves reasoning with known facts in the graph and automatically inferring missing links. In order to tackle this task, studies have been conducted on both link prediction and relation prediction. Recently, there has been significant interest in a dual-encoder architecture that utilizes textual information. However, the dataset for link prediction only provides descriptions for entities, not for relations. As a result, the model heavily relies on descriptions for entities. To address this issue, we utilized a large language model called GPT-3.5-turbo to generate relation descriptions. This allows the baseline model to be trained with more comprehensive relation information. Moreover, the relation descriptions generated by our proposed method are expected to improve the performance of other language model-based link prediction models. The evaluation results for link prediction demonstrate that our proposed method outperforms the baseline model on various datasets, including Korean ConceptNet, WN18RR, FB15k-237, and YAGO3-10. Specifically, we observed improvements of 0.34%p, 0.11%p, 0.12%p, and 0.41%p in terms of Mean Reciprocal Rank (MRR), respecitvely.
A Comparative Study on Server Allocation Optimization Algorithms for Accelerating Parallel Training of Large Language Models
Jinkyu Yim, Yerim Choi, Jinho Lee
http://doi.org/10.5626/JOK.2024.51.9.783
As large-scale language models (LLMs) come to be increasingly utilized in various fields, there is an increasing demand to develop models with higher performance. Significant computational power and memory capacity will be needed to train such models. Therefore, researchers have used 3D parallelization methodology for large-scale language model learning on numerous servers equipped with GPUs. However, 3D parallelization requires frequent large-scale data transfers between servers, which bottlenecks the overall training time. To address this, prior studies have proposed a methodology that identifies non-uniform cluster network conditions in advance and arranges servers and GPUs in an optimized parallel configuration. The existing methods of this type use the classical optimization algorithm SA (Simulated Annealing) for mapping. In this paper, we apply genetic algorithms as well as SAT(satisfiability) algorithms to the problem, and compare and analyze the performance of each algorithm under various experimental environments.
Photovoltaic Power Forecasting Scheme Based on Graph Neural Networks through Long- and Short-Term Time Pattern Learning
Jaeseung Lee, Sungwoo Park, Jaeuk Moon, Eenjun Hwang
http://doi.org/10.5626/JOK.2024.51.8.690
As the use of solar energy has become increasingly common in recent years, there has been active research in predicting the amount of photovoltaic power generation to improve the efficiency of solar energy. In this context, photovoltaic power forecasting models based on graph neural networks have been presented, going beyond existing deep learning models. These models enhance prediction accuracy by learning the interactions between regions. Specifically, they consider how the amount of photovoltaic power in a specific region is affected by the climate conditions of adjacent regions and the time pattern of photovoltaic power generation. However, existing models mainly rely on a fixed graph structure, making it difficult to capture temporal and spatial interactions. In this paper, we propose a graph neural networks-based photovoltaic power forecasting scheme that takes into account both long-term and short-term time patterns of regional photovoltaic power generation data. We then incorporate these patterns into the learning process to establish correlations between regions. Compared to other graph neural networks-based prediction models, our proposed scheme achieved a performance improvement of up to 7.49% based on the RRSE, demonstrating its superiority.
Adaptation of A Hierarchical Cumulative Prompting with Generative Large-scale Language Models in the Legal Domain
Yeenheui Yeen, HaeIn Jung, MinJu Kim, Jeong Yang, Minhye Kim, Hyunji Jang, Myoung-Wan Koo
http://doi.org/10.5626/JOK.2024.51.7.592
This study introduces a stepwise hierarchical prompting method suitable for large-scale generative language models in complex legal reasoning tasks. Complex logical problems are decomposed into multiple steps, accumulating results from each step to set prompts for subsequent ones. It was confirmed that when this method was applied to the evaluation process of the Korean bar exam's essay-type questions, it achieved better results than fine-tuning with original data. Notably, in the final evaluation by legal experts, both tasks showed a human precision of over 0.70, indicating its capability to produce interpretations based on accurate evidence. This prompting technique suggests a solution to the hallucination issue in large language models and demonstrates its effective application. Future research will consider the introduction of a specialized retriever to reflect more accurate legal knowledge in the large language model, aiming to incorporate more precise evidence into prompts. While the current research applied the prompting method only to the legal field, it is expected to be applicable to other complex logical reasoning tasks that rely on specialized knowledge.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr