Jinseo Choi  Donghyun Kang

10.5626/JOK.2022.49.6.407

Multi-threads Deep Learning Tensorflow GPU and CPU utilization

Most deep learning systems spend a lot of time on model training and validation. However, they sometimes tend to waste GPU and CPU resources because the pre-processing and batch processes based on a single thread result in a wait time. In this paper, we propose a new scheme that efficiently handles training and validation processes based on multi-threads. The proposed scheme can overlap the training and validation processes as much as possible by using a model copy operation that extends the processes with multi-threads. As a result, it improves the overall utilization of CPU and GPU. For evaluation, we implemented a convolutional neural network (CNN) using the TensorFlow framework. As a result, we clearly confirm that the proposed scheme saves the total training and validation time by up to 22.4% compared with the traditional schemes.