Digital Library[ Search Result ]
In-Depth Evaluations of the Primality Testing Capabilities of Large Language Models: with a Focus on ChatGPT and PaLM 2
http://doi.org/10.5626/JOK.2024.51.8.699
This study aims to thoroughly evaluate the primality testing capabilities of two large language models, ChatGPT and PaLM 2. We pose two different yes/no questions for a given number, assessing whether it is prime or composite. To deem a model successful, it must correctly answer both questions while also avoiding any division errors in the generated prompt. Analyzing the inference results using a dataset consisting of 664 prime and 1458 composite numbers, we discovered a decrease in testing accuracy as the difficulty of the target numbers increased. Considering the calculation errors, both models experienced a decrease in testing accuracy, with PaLM 2 failing to conduct primality testing for all composite numbers with four high-difficulty digits. These findings highlight the potential for misleading evaluations of language models' reasoning abilities based on simple questions, emphasizing the need for comprehensive assessments.
Creating a of Noisy Environment Speech Mixture Dataset for Korean Speech Separation
Jaehoo Jang, Kun Park, Jeongpil Lee, Myoung-Wan Koo
http://doi.org/10.5626/JOK.2024.51.6.513
In the field of speech separation, models are typically trained using datasets that contain mixtures of speech and overlapping noise. Although there are established international datasets for advancing speech separation techniques, Korea currently lacks a similar precedent for constructing datasets with overlapping speech and noise. Therefore, this paper presents a dataset generator specifically designed for single-channel speech separation models tailored to the Korean language. The Korean Speech mixture with Noise dataset is introduced, which has been constructed using this generator. In our experiments, we train and evaluate a Conv-TasNet speech separation model using the newly created dataset. Additionally, we verify the dataset's efficacy by comparing the Character Error Rate (CER) between the separated speech and the original speech using a pre-trained speech recognition model.
A Contrastive Learning Method for Automated Fact-Checking
Seonyeong Song, Jejun An, Kunwoo Park
http://doi.org/10.5626/JOK.2023.50.8.680
As proliferation of online misinformation increases, the importance of automated fact-checking, which enables real-time evaluation, has been emphasized. In this study, we propose a contrastive learning method for automated fact-checking in Korean. The proposed method deems a sentence similar to evidence as a positive sample to determine the authenticity of a given claim. In evaluation experiments, we found that the proposed method was more effective in the sentence selection step of finding evidence sentences for a given claim than previous methods. such as a finetuned pretrained language model and SimCSE. This study shows a potential of contrastive learning for automated fact-checking.
An Embedding Method of Emotes for the Detection of Popular Clips on Twitch.tv
Hyeonho Song, Kunwoo Park, Meeyoung Cha
http://doi.org/10.5626/JOK.2020.47.12.1153
This study presents an embedding method that effectively learns emote’s meaning in Twitch.tv to understand the audience reaction in live streaming. The proposed method first trains an embedding matrix for text and emotes, respectively, and merges the two matrices into one. Using 2,220,761 clips shared on Twitch.tv, this study conducted two experiments: clustering and clip popularity prediction. Results showed that the approach identifies emote clusters that express a similar emotion and detects popular clips. Future studies could utilize the proposed emote embedding method for the highlight prediction of a live stream.
Automatic Test Case Generation through Concolic Testing to Improve SW Quality of Defense Weapon System
Kunwoo Park, Joohyun Lee, Hyunggon Song, Kyu Tae Cho, Yunho Kim, Moonzoo Kim
http://doi.org/10.5626/JOK.2019.46.9.926
To improve SW quality of defense weapon system, automatic and systematic generation of test cases is necessary; however, that is not the case in the traditional practice of labor-intensive and manual SW testing. The paper applies concolic testing to the defense weapon system SW, effectively generates test cases that achieve high coverage, and discovers defects which contributes to the improvement in SW quality. Also, two methods are proposed using 4 search strategies in concolic testing and using LIA logic, to increase the efficiency of concolic testing for a program with high complexity. In addition, a symbolic modeling method is proposed as an example to extend concolic testing for practitioners.
Churn Analysis of Maximum Level Users in Online Games
In MMORPG (Massively Multiplayer Online Role-Playing Game), users advance their own characters to get to the maximum (max) level by performing given tasks in the game scenario. Although it is crucial to retain users with high levels for running online games successfully, little efforts have been paid to investigate them. In this study, by analyzing approximately 60 million in-game logs of over 50,000 users, we aimed to investigate the process through which users achieve the max level and churn of such users since the moment of achieving the max level, and determine possible indicators related to churn after the max level. Based on the result, we can predict churn of the max level users by employing behavioral patterns before the max level. Moreover, we found users who are socially active and communicate with many people before the max level are less likely to leave the service (p<0.05). This study supports that communication patterns are important factors for persistent usage of the users who achieve the max level, which has practical implications to guide elite users on enjoying online games in the long run.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr