Junhyuk Jo  Jihyun Lim  Sunwoo Lee

10.5626/JOK.2025.52.10.851

sharpness-aware minimization asynchronous model perturbation system-aware batch size Optimization

Sharpness-Aware Minimization (SAM) is an optimization technique designed to enhance the generalization performance of machine learning models. However, its high computational cost associated with model perturbation has restricted its use in real-world applications. In this paper, we propose a novel asynchronous-parallel SAM, which decouples the data dependency between model perturbation and update steps, enabling efficient gradient norm regularization. By adjusting the perturbation batch size in a system-aware manner, our method fully hides the perturbation overhead and effectively utilizes heterogeneous resources such as CPUs and GPUs without sacrificing accuracy. Our experiments on the CIFAR and Oxford Flowers 102 benchmarks show that asynchronous SAM achieves 1-4% higher accuracy than SGD and slightly outperforms the original SAM. Additionally, it maintains accuracy comparable to recent SAM variants, while on CIFAR-10 with ResNet-20, it demonstrates a training speed comparable to that of SGD (about 1.02×).