Controllable High-resolution Cloth Image Generation based on the Diffusion Model

Jaeha Choi, Jangho Lee

http://doi.org/10.5626/JOK.2025.52.8.644

This paper presents a novel approach for generating high-resolution clothing images using ControlNet, which enables image manipulation guided by both images and prompts. The objective is to create clothing images that reflect the preferences of designers or users, customized with prompts and flat sketches. Given the challenges of obtaining flat sketches from designers, this study employs a pseudo flat sketch—referred to as the pseud flat sketch—derived from an edge extraction algorithm as input for ControlNet. Extensive experiments were conducted under various learning conditions based on pseudo flat sketches generated by DiffusionEdge, which was pre-trained on the BIPED and BSDS datasets, alongside prompt generation for clothing data using BLIP. The results indicate that the model, newly prompted by BLIP and DiffusionEdge pre-trained on BIPED, exhibits superior performance across four main evaluation metrics, demonstrating the feasibility of creating customized high-resolution clothing images that align with user prompts. Additionally, the experimental results on controllability reveal that the fine-tuned ControlNet can adjust colors and patterns in accordance with user prompts.