Super Resolution-based Robust Image Inpainting for Large-scale Missing Regions

Jieun Lee, SeungWon Jung, Jonghwa Shim, Eenjun Hwang

http://doi.org/10.5626/JOK.2022.49.9.708

Image inpainting is a method of filling missing regions of an image with plausible imagery. Even though the performance of recent inpainting methods has been significantly improved owing to the introduction of deep learning, unnatural results can be obtained when an input image has a large-scale missing region, contains a complex scene, or is a high-resolution image. In this study, we propose a super resolution-based two-stage image inpainting method, motivated by the point that inpainting performance in low-resolution images is better than in high-resolution images. In the first step, we convert a high-resolution image into a low-resolution image and then perform image inpainting, which results in the initial output image. In the next step, the initial output image becomes the final output image, with the same resolution as the original input image using the super resolution model. To verify the effectiveness of the proposed method, we conducted quantitative and qualitative evaluations using the high-resolution Urban100 dataset. Furthermore, we analyzed the inpainting performance depending on the size of the missing region and demonstrated that the proposed method could generate satisfactory results in a free-form mask.