Digital Library[ Search Result ]
Continual Learning using Memory-Efficient Parameter Generation
Hyung-Wook Lim, Han-Eol Kang, Dong-Wan Choi
http://doi.org/10.5626/JOK.2024.51.8.747
Continual Learning with Parameter Generation shows remarkable stability in retaining knowledge from previous tasks. However, it suffers from a gradual decline in parameter generation performance due to its lack of adaptability to new tasks. Furthermore, the difficulty in predetermining the optimal size of the parameter generation model (meta-model) can lead to memory efficiency issues. To address these limitations, this paper proposed two novel techniques. Firstly, the Chunk Save & Replay (CSR) technique selectively stored and replayed vulnerable parts of the generative neural network, maintaining diversity in the parameter generation model while efficiently utilizing memory. Secondly, the Automatically Growing GAN (AG-GAN) technique automatically expanded the memory of the parameter generation model based on learning tasks, enabling effective memory utilization in resource-constrained environments. Experimental results demonstrated that these proposed techniques significantly reduced memory usage while minimizing performance degradation. Moreover, their ability to recover from deteriorated network performance was observed. This research presents new approaches to overcoming limitations of parameter generation-based continual learning, facilitating the implementation of more effective and efficient continual learning systems.
An Image Harmonization Method with Improved Visual Uniformity of Composite Images in Various Lighting Colors
Doyeon Kim, Jonghwa Shim, Hyeonwoo Kim, Changsu Kim, Eenjun Hwang
http://doi.org/10.5626/JOK.2024.51.4.345
Image composition is a technique that creates a composite image by arranging foreground objects extracted from other images onto a background image. To improve the visual uniformity of the composite images, deep learning-based image harmonization techniques that adjust the lighting and color of foreground objects to match the background image have been actively proposed recently. However, existing techniques have limited performance in visual uniformity because they adjust colors only for the lighting color distribution of the dataset used for training. To address this problem, we propose a novel image harmonization scheme that has robust performance for various lighting colors. First, iHColor, a new dataset composed of various lighting color distributions, is built through data preprocessing. Then, a pre-trained GAN-based Harmonization model is fine-tuned using the iHColor dataset. Through experiments, we demonstrate that the proposed scheme can generate harmonized images with better visual uniformity than existing models for various lighting colors.
Polyphonic Music Generation with Sequence Generative Adversarial Networks
Sang-gil Lee, Uiwon Hwang, Seonwoo Min, Sungroh Yoon
http://doi.org/10.5626/JOK.2024.51.1.78
In this paper, we propose an application of sequence generative adversarial networks (SeqGAN) for generating polyphonic musical sequences. We introduce a representation of polyphonic MIDI files that could encapsulate both chords and melodies with dynamic timings. This method condensed the duration, octaves, and keys of both melodies and chords into a single word vector representation. Our generator composed of recurrent neural networks was trained to predict distributions of musical word sequences. Additionally, we employed the least square loss function for the discriminator to stabilize training of the model. Our model could create sequences that are musically coherent. It exhibited improved quantitative and qualitative measures.
Rehearsal with Stored Latent Vectors for Incremental Learning Over GANs
http://doi.org/10.5626/JOK.2023.50.4.351
Unlike humans, sequential learning of multiple tasks is a difficult problem in a deep learning model. This problem is not only for discriminative models, but also for generative models, such as GAN. The Generative Replay method, which is frequently used in GAN continual learning, uses images generated by GAN provided in the previous task together for learning new tasks, but does not generate good images for CIFAR10, which is a relatively challenging task. Therefore, we can consider a rehearsal-based method that stores a portion of the real data, which cannot store a huge amount of images in limited memory because of large dimension of the real image. In this paper, we propose LactoGAN and LactoGAN+, continual learning methods that store latent vectors that are the inputs of GANs rather than storing real images, as the existing rehearsal-based approaches. As a result, more image knowledge can be stored in the same memory; thus, showing better results than the existing GAN continual learning methods.
Re-Generation of Models via Generative Adversarial Networks and Bayesian Neural Networks for Task-Incremental Learning
http://doi.org/10.5626/JOK.2022.49.12.1115
In contrast to the human ability of continual learning, deep learning models have considerable difficulty maintaining their original performance when the model learns a series of incrementally arriving tasks. In this paper, we propose ParameterGAN, a novel task-incremental learning approach based on model synthesis. The proposed method leverages adversarial generative learning to regenerate neural networks themselves which have a parameter distribution similar to that of a pre-trained Bayesian network. Also, using pseudo-rehearsal methods, ParameterGAN enables continual learning by regenerating the networks of all previous tasks without catastrophic forgetting. Our experiment showed that the accuracy of the synthetic model composed of regenerated parameters was comparable to that of the pre-trained model, and the proposed method outperformed the other SOTA methods in the comparative experiments using the popular task-incremental learning benchmarks Split-MNIST and Permuted-MNIST.
Copy-Paste Based Image Data Augmentation Method Using
http://doi.org/10.5626/JOK.2022.49.12.1056
In the field of computer vision, massive well-annotated image data are essential to achieve good performance of a convolutional neural network (CNN) model. However, in real world applications, gathering massive well-annotated data is a difficult and time-consuming job. Thus, image data augmentation has been continually studied. In this paper, we proposed an image data augmentation method that could generate more diverse image data by combining generative adversarial network (GAN) and copy-paste based augmentation. The proposed method generated not pixel-level or image-level augmentation, but object-level augmentation by cutting off segmentation boundaries(mask) instead of bounding boxes. It then applyied GAN to transform objects.
VACS: Virtual Try-on Artifact Correction System using the Fashion Object Segmentation Method
Wonjung Park, Youjin Chung, Soonchan Park, Jinah Park
http://doi.org/10.5626/JOK.2022.49.10.802
Virtual try-on (VITON) technology is receiving a lot of attention with the development of Generative Adversarial Networks (GANs) [1]. Previous approaches to VITON synthesized 2D model images and in-shop clothing images using a generative model. However, when synthesizing the top, VITON erroneously changes pixels in unintended areas, such as the background and pants. In this study, we propose the VITON Artifact Correction System (VACS), which divides and protects targeted clothes synthesized in VITON by fashion object segmentation, and replaces the pixels corresponding to the remaining areas with the original model image to increase the realism of the final composition.
Super Resolution-based Robust Image Inpainting for Large-scale Missing Regions
Jieun Lee, SeungWon Jung, Jonghwa Shim, Eenjun Hwang
http://doi.org/10.5626/JOK.2022.49.9.708
Image inpainting is a method of filling missing regions of an image with plausible imagery. Even though the performance of recent inpainting methods has been significantly improved owing to the introduction of deep learning, unnatural results can be obtained when an input image has a large-scale missing region, contains a complex scene, or is a high-resolution image. In this study, we propose a super resolution-based two-stage image inpainting method, motivated by the point that inpainting performance in low-resolution images is better than in high-resolution images. In the first step, we convert a high-resolution image into a low-resolution image and then perform image inpainting, which results in the initial output image. In the next step, the initial output image becomes the final output image, with the same resolution as the original input image using the super resolution model. To verify the effectiveness of the proposed method, we conducted quantitative and qualitative evaluations using the high-resolution Urban100 dataset. Furthermore, we analyzed the inpainting performance depending on the size of the missing region and demonstrated that the proposed method could generate satisfactory results in a free-form mask.
Identification of Generative Adversarial Network Models Suitable for Software Defect Prediction
Jiwon Choi, Jaewook Lee, Duksan Ryu, Suntae Kim
http://doi.org/10.5626/JOK.2022.49.1.52
Software Defect Prediction(SDP) helps effectively allocate quality assurance resources which are limited by identifying modules that are likely to cause defects. Software defect data suffer from class imbalance problems in which there are more non-defective instances than defective instances. In most machine learning methods, the defect prediction performance is degraded when there is a disproportionate number of instances belonging to a particular class. Therefore, this research aimed to solve the class imbalance problem and improve defect prediction performance by using a Generative Adversarial Network(GAN) model. To this end, we compared different kinds of GAN models for their suitability for SDP and checked the applicability of GAN models that were not applied in the related work. In our study, Vanilla-GAN(GAN), Conditional GAN (cGAN), and Wasserstein GAN (WGAN) models which were initially proposed for image generation were adapted for software defect prediction. Then those modified models were compared with Tabular GAN(TGAN) and Modeling Tabular data using Conditional GAN(CTGAN). Our experimental results showed that the CTGAN model is suitable for SDP data. We also conducted a sensitivity analysis examining which hyper-parameter values of CTGAN increase the recall rate and lower the probability of false alarm (PF). Our experimental results indicated that the hyper-parameters should be adjusted according to the dataset. We expect that our proposed approach can help effectively allocate limited resources by improving the performance of SDP.
Automatic Data Augmentation for Named Entity Recognition using a Text Infilling technique and Generative Adversarial Network
Cheon-Young Park, Kong Joo Lee
http://doi.org/10.5626/JOK.2021.48.4.462
Deep neural networks have been widely used in many NLP applications, However, successful construction of deep networks requires a large training corpus. Collecting a large training corpus that contains label information such as named entities is difficult and leads to a lack of data. Automatic data augmentation represents a solution to data scarcity problem. In this paper, we propose an automatic data augmentation technique for named entity recognition(NER) based on a text infilling model and generative adversarial networks. A text infilling model is used to fill missing components of a template to generate complete sentences. Using the text infilling model, we can fill in the blank of the template to generate complete and semantically coherence text with accurately named entity labels. Sentences generated by our model show lower perplexity and higher diversity than those generated in the previous approaches. Also text augmentation based on our model can improve the performance of a conventional NER system.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr