Accuracy Evaluation Method for Image Classification Deep Learning Model considering Potentially Misclassified Data

Young-Woo Lee, Min-Ju Song, Heung-Seok Chae

http://doi.org/10.5626/JOK.2021.48.6.657

Image classification deep learning models have a problem of misclassifying the type of image when the image is modified. As the existing accuracy evaluation methods do not take into account images that can potentially be misclassified due to modification of the image, it becomes to trust the evaluation result even if the type of image is accurately classified. In this study, we have proposed a method for evaluating the accuracy of image classification deep learning models considering the potentially misclassified data. We have measured the boundary cost to identify potentially misclassified data for each model and data set and identified potentially misclassified data based on the boundary cost. Also, we have measured the accuracy considering the potentially misclassified data. As a result of identifying potentially misclassified data, 0.1～4.2% of the data were identified as potentially misclassified data, of which approximately 18～60% were actually misclassified. As a result of the accuracy evaluation, it was estimated that the higher-accuracy model was more robust to image modification, and the lower-accuracy model was less robust to image modification.