Namcheol Lee  Geonha Park  Woobean Seo  Seongsoo Hong

10.5626/JOK.2025.52.9.727

GPU preemption GPU kernel OpenCL user-space

As AI becomes increasingly prevalent in modern embedded systems, ensuring real-time performance for deep neural network (DNN) inference using GPUs has become a critical challenge. However, since the GPU is a shared resource with high preemption costs, process-level preemption is not effectively supported. As a result, priority inversion occurs during resource contention among processes, complicating the implementation of real-time multitasking in embedded systems utilizing GPUs. Previous studies have explored hardware-dependent approaches to GPU preemption, but these methods often lack portability and scalability. To overcome these limitations, this study proposes a software-level GPU preemption technique that enables preemption without relying on hardware-specific mechanisms. The proposed method intercepts GPU kernel execution requests from processes and forwards them to a user-space OpenCL kernel scheduler, which controls the execution order of GPU kernels based on process priorities. This approach reduces delays for high-priority processes caused by lower-priority ones. Experimental results confirm that the proposed method achieves high execution determinism.