Yunsu Kim  Minchan Kim  Jinwoo Choi  Youngseok Hwang  Hyunwoo Park

10.5626/JOK.2024.51.12.1104

Large Language Model (LLM) Visual Analytics Explainable AI (XAI) TokenAttribution model evaluation metrics

Large Language Models (LLMs) have made significant advancements in Natural Language Processing (NLP) and generative AI. However, their complex structure poses challenges in terms of interpretability and reliability. To address this issue, this study proposed LLMEE, a tool designed to visually explain and evaluate the prediction process of LLMs. LLMEE visually represents the impact of each input token on the output, enhancing model transparency and providing insights into various NLP tasks such as Summarization, Question Answering, Text Generation. Additionally, it integrates evaluation metrics such as ROUGE, BLEU, and BLEURTScore, offering both quantitative and qualitative assessments of LLM outputs. LLMEE is expected to contribute to more reliable evaluation and improvement of LLMs in both academic and industrial contexts by facilitating a better understanding of their complex workings and by providing enhanced output quality assessments.