Digital Library[ Search Result ]
Categorization of Korean News Articles Based on Convolutional Neural Network Using Doc2Vec and Word2Vec
http://doi.org/10.5626/JOK.2017.44.7.742
In this paper, we propose a novel approach to improve the performance of the Convolutional Neural Network(CNN) word embedding model on top of word2vec with the result of performing like doc2vec in conducting a document classification task. The Word Piece Model(WPM) is empirically proven to outperform other tokenization methods such as the phrase unit, a part-of-speech tagger with substantial experimental evidence (classification rate: 79.5%). Further, we conducted an experiment to classify ten categories of news articles written in Korean by feeding words and document vectors generated by an application of WPM to the baseline and the proposed model. From the results of the experiment, we report the model we proposed showed a higher classification rate (89.88%) than its counterpart model (86.89%), achieving a 22.80% improvement. Throughout this research, it is demonstrated that applying doc2vec in the document classification task yields more effective results because doc2vec generates similar document vector representation for documents belonging to the same category.
Title Generation Model for which Sequence-to-Sequence RNNs with Attention and Copying Mechanisms are used
http://doi.org/10.5626/JOK.2017.44.7.674
In big-data environments wherein large amounts of text documents are produced daily, titles are very important clues that enable a prompt catching of the key ideas in documents; however, titles are absent for numerous document types such as blog articles and social-media messages. In this paper, a title-generation model for which sequence-to-sequence RNNs with attention and copying mechanisms are employed is proposed. For the proposed model, input sentences are encoded based on bi-directional GRU (gated recurrent unit) networks, and the title words are generated through a decoding of the encoded sentences with keywords that are automatically selected from the input sentences. Regarding the experiments with 93631 training-data documents and 500 test-data documents, the attention-mechanism performances are more effective (ROUGE-1: 0.1935, ROUGE-2: 0.0364, ROUGE-L: 0.1555) than those of the copying mechanism; in addition, the qualitative-evaluation radiative performance of the former is higher.
Water Level Forecasting based on Deep Learning : A Use Case of Trinity River-Texas-The United States
Quang-Khai Tran, Sa-kwang Song
http://doi.org/10.5626/JOK.2017.44.6.607
This paper presents an attempt to apply Deep Learning technology to solve the problem of forecasting floods in urban areas. We employ Recurrent Neural Networks (RNNs), which are suitable for analyzing time series data, to learn observed data of river water and to predict the water level. To test the model, we use water observation data of a station in the Trinity River, Texas, the U.S., with data from 2013 to 2015 for training and data in 2016 for testing. Input of the neural networks is a 16-record-length sequence of 15-minute-interval time-series data, and output is the predicted value of the water level at the next 30 minutes and 60 minutes. In the experiment, we compare three Deep Learning models including standard RNN, RNN trained with Back Propagation Through Time(RNN-BPTT), and Long Short-Term Memory (LSTM). The prediction quality of LSTM can obtain Nash Efficiency exceeding 0.98, while the standard RNN and RNN-BPTT also provide very high accuracy.
Document Summarization Considering Entailment Relation between Sentences
Youngdae Kwon, Noo-ri Kim, Jee-Hyong Lee
Document summarization aims to generate a summary that is consistent and contains the highly related sentences in a document. In this study, we implemented for document summarization that extracts highly related sentences from a whole document by considering both similarities and entailment relations between sentences. Accordingly, we proposed a new algorithm, TextRank-NLI, which combines a Recurrent Neural Network based Natural Language Inference model and a Graphbased ranking algorithm used in single document extraction-based summarization task. In order to evaluate the performance of the new algorithm, we conducted experiments using the same datasets as used in TextRank algorithm. The results indicated that TextRank-NLI showed 2.3% improvement in performance, as compared to TextRank.
Image Caption Generation using Recurrent Neural Network
Automatic generation of captions for an image is a very difficult task, due to the necessity of computer vision and natural language processing technologies. However, this task has many important applications, such as early childhood education, image retrieval, and navigation for blind. In this paper, we describe a Recurrent Neural Network (RNN) model for generating image captions, which takes image features extracted from a Convolutional Neural Network (CNN). We demonstrate that our models produce state of the art results in image caption generation experiments on the Flickr 8K, Flickr 30K, and MS COCO datasets.
Drug Toxicity Prediction Using Integrated Graph Neural Networks and Attention-Based Random Walk Algorithm
Jong-Hoon Park, Jae-Woo Chu, Young-Rae Cho
http://doi.org/10.5626/JOK.2025.52.3.234
The traditional drug development process is often burdened by high costs and lengthy timelines, leading to increasing interest in AI-based drug development. In particular, the importance of AI models for preemptively evaluating drug toxicity is being emphasized. In this study, we propose a novel drug toxicity prediction model, named Integrated GNNs and Attention Randon Walk (IG-ARW). The proposed method integrates various Graph Neural Network (GNN) models and uses attention mechanisms to compute random walk transition probabilities, extracting graph features precisely. The model then conducts random walks to extract node features and graph features, ultimately predicting drug toxicity. IG-ARW was evaluated on three different datasets, demonstrating strong performances with AUC scores of 0.8315, 0.8894, and 0.7476, respectively. Notably, the model was proven to be highly effective not only in toxicity prediction, but also in predicting other drug characteristics.
Copy-Paste Based Image Data Augmentation Method Using
http://doi.org/10.5626/JOK.2022.49.12.1056
In the field of computer vision, massive well-annotated image data are essential to achieve good performance of a convolutional neural network (CNN) model. However, in real world applications, gathering massive well-annotated data is a difficult and time-consuming job. Thus, image data augmentation has been continually studied. In this paper, we proposed an image data augmentation method that could generate more diverse image data by combining generative adversarial network (GAN) and copy-paste based augmentation. The proposed method generated not pixel-level or image-level augmentation, but object-level augmentation by cutting off segmentation boundaries(mask) instead of bounding boxes. It then applyied GAN to transform objects.
Re-Generation of Models via Generative Adversarial Networks and Bayesian Neural Networks for Task-Incremental Learning
http://doi.org/10.5626/JOK.2022.49.12.1115
In contrast to the human ability of continual learning, deep learning models have considerable difficulty maintaining their original performance when the model learns a series of incrementally arriving tasks. In this paper, we propose ParameterGAN, a novel task-incremental learning approach based on model synthesis. The proposed method leverages adversarial generative learning to regenerate neural networks themselves which have a parameter distribution similar to that of a pre-trained Bayesian network. Also, using pseudo-rehearsal methods, ParameterGAN enables continual learning by regenerating the networks of all previous tasks without catastrophic forgetting. Our experiment showed that the accuracy of the synthetic model composed of regenerated parameters was comparable to that of the pre-trained model, and the proposed method outperformed the other SOTA methods in the comparative experiments using the popular task-incremental learning benchmarks Split-MNIST and Permuted-MNIST.
Layered Abstraction Technique for Effective Formal Verification of Deep Neural Networks
Jueun Yeon, Seunghyun Chae, Kyungmin Bae
http://doi.org/10.5626/JOK.2022.49.11.958
Deep learning has performed well in many areas. However, deep learning is vulnerable to errors such as adversarial examples. Therefore, much research exists on ensuring the safety and robustness of deep neural networks. Since deep neural networks are large in scale and the activation functions are non-linear, linear approximation methods for such activation functions are proposed and widely used for verification. In this paper, we propose a new technique, called layered abstraction, for non-linear activation functions, such as ReLU and Tanh, and the verification algorithm based on that. We have implemented our method by extending the existing SMT-based methods. The experimental evaluation showed that our tool performs better than an existing tool.
Solving Korean Math Word Problems Using the Graph and Tree Structure
Kwang Ho Bae, Sang Yeop Yeo, Yu Chul Jung
http://doi.org/10.5626/JOK.2022.49.11.972
In previous studies, there have been various efforts to solve math word problems in the English sentence system. In many studies, improved performance was achieved by introducing structures such as trees and graphs, beyond the Sequence-to-Sequence approaches. However, in the study of solving math problems in Korean sentence systems, there are no model cases, using structures such as trees or graphs. Thus, in this paper, we examine the possibility of solving math problems in Korean sentence systems for models using the tree structure, graph structure, and Korean pre-training language models together. Our experimental results showed that accuracy improved by approximately 20%, compared to the model of the Seq2seq structure, by introducing the graph and tree structure. Additionally, the use of the Korean pre-training language model showed an accuracy improvement of 4.66%-5.96%.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr