Cumincad : CUMINCAD Papers : Paper cdrf2023_24:Text Semantics to Image Generation: A Method of Building Facades Design Base on Stable Diffusion Model

id	cdrf2023_24
authors	Haoran Ma, Hao Zheng
year	2023
title	Text Semantics to Image Generation: A Method of Building Facades Design Base on Stable Diffusion Model
doi	https://doi.org/https://doi.org/10.1007/978-981-99-8405-3_3
source	Proceedings of the 2023 DigitalFUTURES The 5st International Conference on Computational Design and Robotic Fabrication (CDRF 2023)
summary	Stable Diffusion model has been extensively employed in the study of architectural image generation, but there is still an opportunity to enhance in terms of the controllability of the generated image content. A multi-network combined text-to-building facade image generating method is proposed in this work. We first fine-tuned the Stable Diffusion model on the CMP Facades dataset using the LoRA (Low-Rank Adaptation) approach, then we apply the ControlNet model to further control the output. Finally, we contrasted the facade generating outcomes under various architectural style text contents and control strategies. The results demonstrate that the LoRA training approach significantly decreases the possibility of fine-tuning the Stable Diffusion large model, and the addition of the ControlNet model increases the controllability of the creation of text to building facade images. This provides a foundation for subsequent studies on the generation of architectural images.
series	cdrf
email	hazheng@cityu.edu.hk
full text	file.pdf (9,876,068 bytes)
references	Content-type: text/plain
last changed	2024/05/29 14:04