Layout Code Generation using Large Multimodal Models

Yangsoo Choi, Jeongwoo Na, Dongcheol Lee, Jongwuk Lee

http://doi.org/10.5626/JOK.2025.52.8.677

GUI layout generation entails the analysis and organization of user interface components into structured formats. This paper introduces a novel method that leverages Large Multimodal Models (LMMs) to transform GUI layout images into structured code. The proposed framework enables LMMs to effectively comprehend both the visual and structural attributes of GUI images and produce the corresponding layout code without requiring additional training. The method begins by extracting feature vectors from an input image, followed by retrieving similar examples and applying visual and spatial augmentation techniques to create few-shot prompts. Importantly, it selects augmented examples that are least similar to the input image, encouraging the model to generalize and better capture the semantic relationship between the image and its associated code. Experimental results indicate that our approach outperforms existing text-based prompting methods in both quantitative and qualitative evaluations. This work offers a practical and effective strategy for GUI code generation using LMMs and underscores the potential of multimodal prompting in layout generation tasks.