Digital Library[ Search Result ]
A Data Imbalance Minimization Strategy for Scalable Deep Learning Training
Sanha Maeng, Euhyun Moon, Sungyong Park
http://doi.org/10.5626/JOK.2023.50.10.836
As deep neural network training is compute-intensive and takes a very long time, distributed training using clusters with multiple graphics processing units (GPUs) has been widely adopted. The distributed training of deep neural networks is severely slowed due to straggler, i.e., the slowest worker. Hence, previous studies have proposed solutions to the straggler problem. The existing approaches assume that all data samples, such as images, have a constant size, and they do not recognize data imbalance issues, caused by data samples with different sizes, such as videos and audios, while solving the straggler problem. In this paper, we propose a data imbalance minimization (DIM) strategy that considers data imbalance problems to solve the straggler problem caused by imbalanced data samples. Our evaluation on eight NVIDIA Tesla T4 GPUs shows that DIM outperforms the state-of-the-art systems by up to 1.77x speedup with comparable scalability.
Improving the Quality of Generating Imbalance Data in GANs through an Exhaustive Contrastive Learning Method
Hyeonjun Shin, Sangbaek Lee, Kyuchul Lee
http://doi.org/10.5626/JOK.2023.50.4.295
As the performance of deep learning algorithms has improved, they are being used as a way to solve various problems in the real world. In the case of data that reflect the real world, imbalance data may occur depending on the frequency of occurrence of events or the difficulty of collection. Data with an inconsistent number of classes that make up the data are called imbalance data, and in particular, it is difficult to learn the minority classes with relatively little data through Deep Learning algorithms. Recently, Generative Adversarial Nets (GANs) have been applied as a method for data augmentation, and self-supervised learning-based pre-learning has been proposed for minority class learning. However, because class information of imbalance data is utilized in the process of learning the Generative Model, the quality of generated data is poor due to poor learning of minority classes. To solve this problem, this paper proposes a similarity-based exhaustive contrast learning method. The proposed method is quantitatively evaluated through the Frechet Inception Distance (FID) and Inception Score (IS). The method proposed in this paper confirmed the performance improvement of the Frechet Inception Distance of 16.32 and the Inception Score of 0.38, as compared to the existing method.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr