| id |
caadria2023_106 |
| authors |
Li, Yuqian and Xu, Weiguo |
| year |
2023 |
| title |
Research on Architectural Sketch to Scheme Image Based on Context Encoder |
| source |
Immanuel Koh, Dagmar Reinhardt, Mohammed Makki, Mona Khakhar, Nic Bao (eds.), HUMAN-CENTRIC - Proceedings of the 28th CAADRIA Conference, Ahmedabad, 18-24 March 2023, pp. 69–78 |
| doi |
https://doi.org/10.52842/conf.caadria.2023.1.069
|
| summary |
Architects are used to hand drawing sketches to express the architectural creation intention. To present these abstract sketches, architects and teams need to convert sketches into architectural scheme images, which requires a lot of time and labour. Deep learning may have the potential to improve the efficiency of this work. The common sketch-to-image generation is based on Generative Adversarial Network (GAN), and the research of edge-to-image has made a big progress. But these methods require strict alignment of data pairs, which is difficult to achieve. Zhu et al. proposed the loss of Cycle-Consistent, which solved the problem that pairs of data sets are difficult to collect. However, most of the image translation methods require strict alignment between image data pairs, which can be achieved only for the edge mapping extracted from the image; but the sketch is very different from the edge. Due to the abstractness and fuzziness of the sketch, any simple distortion cannot complete the task of providing pixel-level alignment between the sketch and image; And image translation is the transfer of image features such as colour and texture. The original image has a strong constraint on the generated image, which makes the original structure of the image impossible to be changed. By image inpainting, we address this topic using a joint image completion approach with Context-Encoder, where the architectural sketch provides the image context for generating the scheme images. This setting has two advantages: first, the joint images can avoid the complexity of cross modal problems and the strict alignment of the data pairs as image-to-image translation; second, because of the weak constraint, the outputs have greater freedom, which perhaps can generate more imaginative results. The Context-Encoder generates scheme images on the data sets of general architectural sketches. The results present that the applicability of the completion method is better than that of the method of image translation. And scheme images that is different from the original architectural sketch contours have been generated. |
| keywords |
Sketch, Building Scheme Image, Image Completion, Context-Encoder |
| series |
CAADRIA |
| email |
|
| full text |
 file.pdf (1,300,561 bytes) |
| references |
Content-type: text/plain
|
Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, C. (2000)
 Image Inpainting
, Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH '.Available at: https://doi.org/1.1145/344779.344972
|
|
|
|
|
Chan, Y. H., & Spaeth, B. (2020)
Architectural Visualisation with Conditional Generative Adversarial Networks (cgan): What Machines Read in Architectural Sketches.
, Proceedings of the 38th International Conference on Education and Research in Computer Aided Architectural Design in Europe (ECAADe),Volume 2.Available at: https://doi.org/1.52842/conf.ecaade.22.2.299
|
|
|
|
|
Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., & Sen, P. (2012)
Image Melding: Combining Inconsistent Images Using Patch-based Synthesis
, ACM Transactions on Graphics, 31(4), 1-1. Available at: https://doi.org/1.1145/218552.2185578
|
|
|
|
|
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Bengio, Y. (2014)
Generative Adversarial Nets
, Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, 2672-268. MIT Press.
|
|
|
|
|
Hadhoud, M.M., Moustafa, K. A.,&Amp; Shenoda, S. Z. (2008)
Digital Images Inpainting Using Modified Convolution Based Method
, Proceedings of SPIE :The International Society for Optical Engineering, 1(1). Available at: https://doi.org/1.1117/12.81674
|
|
|
|
|
Li, Y., & Xu, W. (2021)
Using Cyclegan to Achieve the Sketch Recognition Process of Sketch-based Modeling
, Proceedings of the 221 DigitalFUTURES, 26-34.Available at: https://doi.org/1.17/978-981-16-5983-63
|
|
|
|
|
P. Isola, J. -Y. Zhu, T. Zhou And A. A. Efros (2017)
Image-to-image Translation with Conditional Adversarial Networks
, 217 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Available at: https://doi.org/1.119/CVPR.217.632.
|
|
|
|
|
Pathak, D., Krahenbuhl, P., Donahue, J. , Darrell, T., & Efros, A.A. (2016)
Context Encoders: Feature Learning By Inpainting
, 216 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)Available at: https://doi.org/1.119/CVPR.216.278.
|
|
|
|
|
Rojas, D. J., Fernandes, B. J., & Fernandes, S. M. (2020)
A Review on Image Inpainting Techniques and Datasets
, 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 22.Available at: https://doi.org/1.119/sibgrapi51738.22.4
|
|
|
|
|
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2017)
High-resolution Image Synthesis and Semantic Manipulation with Conditional Gans
, 218 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Available at: https://doi.org/1.119/CVPR.218.917
|
|
|
|
|
Werner, L And Koering, D. (2020)
Anthropologic: Architecture and Fabrication in the Cognitive Age
, Proceedings of the 38th eCAADe Conference - Volume 2, TU Berlin, Berlin, Germany, 16-18 September 2020, pp. 299-308. https://doi.org/10.14279/depositonce-10568
|
|
|
|
|
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017)
Unpaired Image-to-image Translation Using Cycle-consistent Adversarial Networks
, 217 IEEE International Conference on Computer Vision (ICCV), 2242-2251. Available at: https://doi.org/1.119/ICCV.217.244
|
|
|
|
| last changed |
2023/06/15 23:14 |
|
