Digital Library[ Search Result ]
LLMEE: Enhancing Explainability and Evaluation of Large Language Models through Visual Token Attribution
Yunsu Kim, Minchan Kim, Jinwoo Choi, Youngseok Hwang, Hyunwoo Park
http://doi.org/10.5626/JOK.2024.51.12.1104
Large Language Models (LLMs) have made significant advancements in Natural Language Processing (NLP) and generative AI. However, their complex structure poses challenges in terms of interpretability and reliability. To address this issue, this study proposed LLMEE, a tool designed to visually explain and evaluate the prediction process of LLMs. LLMEE visually represents the impact of each input token on the output, enhancing model transparency and providing insights into various NLP tasks such as Summarization, Question Answering, Text Generation. Additionally, it integrates evaluation metrics such as ROUGE, BLEU, and BLEURTScore, offering both quantitative and qualitative assessments of LLM outputs. LLMEE is expected to contribute to more reliable evaluation and improvement of LLMs in both academic and industrial contexts by facilitating a better understanding of their complex workings and by providing enhanced output quality assessments.
Biometrics Performance Improvement of Face Recognition Smart Door Using Binary Classifier
Taeseong Kim, Changsoo Eun, Jongwon Park
http://doi.org/10.5626/JOK.2023.50.7.598
Face recognition based smart door is a biometric system that collects images using a camera and decides whether a visitor is registered by recognizing the face. Recently, with increasing number of single-person households, demand for access convenience has increased. Accordingly, research on smart doors using face recognition method is active. Face recognition based smart doors use deep learning method to recognize visitor"s faces. Difference between the visitor"s face and the registrant"s face is converted into a distance through encoding. If the distance between the two faces is less than the threshold value, the door is opened as it is determined to be the same person. Facial similarity thresholds differ according to region, race, and clothing cultures. Also, biometrics performance varies according to threshold settings. In previous studies, a constant of 0.4 was used as the facial similarity threshold, which was the criterion for determining registration. In this paper, facial similarity thresholds were calculated using five binary classifiers and biometric performance was compared. As a result of the experiment using the LFW dataset, the average EER was improved by 16.59% compared to that when the constant was used.
Epoch Score: Dataset Verification using Quantitative Data Quality Assessment
Sungryeol Kim, Taewook Hwang, Sangkeun Jung, Yoonhyung Roh
http://doi.org/10.5626/JOK.2023.50.3.250
It is tough to determine whether a dataset is suitable for a model or specified field or whether there is an error. In this paper, we propose an Epoch Score that indicates the degree of difficulty of the data as a score using incorrect answer data obtained through learning several times under the same conditions but different seeds. Through this, we verified KLUE"s Topic Classification dataset, and about 0.8% performance improvement derived by correcting high-scoring data, which we judge to have errors. Epoch Score can be used for all supervised learning regardless of the data type, such as natural language or images, and the performance of the model can be inferred by the area the of the Epoch Score.
A Malicious Traffic Detection Method Using X-means Clustering
Myoungji Han, Jihyuk Lim, Junyong Choi, Hyunjoon Kim, Jungjoo Seo, Cheol Yu, Sung-Ryul Kim, Kunsoo Park
Malicious traffic, such as DDoS attack and botnet communications, refers to traffic that is generated for the purpose of disturbing internet networks or harming certain networks, servers, or hosts. As malicious traffic has been constantly evolving in terms of both quality and quantity, there have been many researches fighting against it. In this paper, we propose an effective malicious traffic detection method that exploits the X-means clustering algorithm. We also suggest how to analyze statistical characteristics of malicious traffic and to define metrics that are used when clustering. Finally, we verify effectiveness of our method by experiments with two released traffic data.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr