| id |
ecaade2024_153 |
| authors |
Tsurunaga, Shinya; Fukuda, Tomohiro; Yabuki, Nobuyoshi |
| year |
2024 |
| title |
Enhanced Landscape Visualization of Post-Structure Removal: Integrating 3D reconstruction techniques and diffusion models through machine learning |
| doi |
https://doi.org/10.52842/conf.ecaade.2024.1.549
|
| source |
Kontovourkis, O, Phocas, MC and Wurzer, G (eds.), Data-Driven Intelligence - Proceedings of the 42nd Conference on Education and Research in Computer Aided Architectural Design in Europe (eCAADe 2024), Nicosia, 11-13 September 2024, Volume 1, pp. 549–558 |
| summary |
In urban redevelopment, demolition of existing buildings often occur and landscape assessment plays an important role in avoiding various environmental issues. Both residents and professionals should be involved to create a virtual three-dimensional (3D) space after demolition, which would enable even non-experts to understand the future landscape. Research efforts aimed at creating virtual 3D spaces by removing unnecessary objects utilize techniques such as neural radiance fields (NeRF). These techniques reconstruct spaces into virtual 3D spaces from RGB images by removing redundant objects. However, a challenge arises from the low-quality images generated from the resultant space. Additionally, methods for reconstructing 3D images face limitations in acquiring images of portions previously obscured by structures slated for demolition. This often leads to numerous artifacts in 3D reconstruction after structure removal, which hinders accurate space construction. This study proposes a system that integrates 3D Gaussian splatting, capable of high-quality 3D reconstruction through machine learning, and image completion processing using a diffusion model. This integration aims to reduce the impact of artifacts in 3D reconstruction after building removal in complex and large-scale urban areas. This will contribute to the intuitive understanding and decision-making of non-experts, such as residents, in future landscape assessments after building removal. |
| keywords |
3D reconstruction, diffusion model, landscape visualization, view synthesis, real-time rendering, 3D Gaussian splatting |
| series |
eCAADe |
| email |
|
| full text |
 file.pdf (875,712 bytes) |
| references |
Content-type: text/plain
|
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021)
 Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
, Proceedings of the IEEE International Conference on Computer Vision. https://doi.org/10.1109/ICCV48922.2021.00580
|
|
|
|
|
Dosovitskiy, A., & Brox, T. (2016)
Inverting visual representations with convolutional networks
, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December. https://doi.org/10.1109/CVPR.2016.522
|
|
|
|
|
Goesele, M., Snavely, N., Curless, B., Hoppe, H., & Seitz, S. M. (2007)
Multi-view stereo for community photo collections
, Proceedings of the IEEE International Conference on Computer Vision. https://doi.org/10.1109/ICCV.2007.4408933
|
|
|
|
|
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., . . . Bengio, Y. (2014)
Generative adversarial nets
, Paper presented at the Advances in Neural Information Processing Systems, , 3(January) 2672-2680
|
|
|
|
|
Ho, J., Jain, A., & Abbeel, P. (2020)
Denoising diffusion probabilistic models
, Paper presented at the Advances in Neural Information Processing Systems, 2020-December
|
|
|
|
|
Iizuka, S., Edgar Simo-Serra, Ishikawa, H.: (2017)
Globally and Locally Consistent Image Completion
, ACM Transaction on Graphics (Proc. of SIGGRAPH)
|
|
|
|
|
Ishii, H., Joseph, E.B., Underkoffler, J., Yeung, Y., Chak, D., Kanji, Z., and Piper, B.: (2002)
Augmented Urban Planning Workbench: Overlaying Drawings
, Physical Models and Digital Simulation, Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR 2002), 1-9
|
|
|
|
|
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023)
3D Gaussian Splatting for Real-Time Radiance Field Rendering
, ACM Transactions on Graphics, 42(4). https://doi.org/10.1145/3592433
|
|
|
|
|
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020)
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
, Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12346 LNCS. https://doi.org/10.1007/978-3-030-58452-8_24
|
|
|
|
|
Mugita, Y., Fukuda, T., & Yabuki, N. (2023)
Future Landscape Visualization by Generating Images Using a Diffusion Model and Instance Segmentation
, Proceedings of the International Conference on Education and Research in Computer Aided Architectural Design in Europe, 2. https://doi.org/10.52842/conf.ecaade.2023.2.549
|
|
|
|
|
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: (2016)
Context Encoders: Feature Learning by Inpainting
, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2016-December,7780647, pp. 2536-2544
|
|
|
|
|
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022)
High-resolution image synthesis with latent diffusion models
, Paper presented at the Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, , 2022-June 10674-10685. doi:10.1109/CVPR52688.2022.01042
|
|
|
|
|
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006)
A comparison and evaluation of multi-view stereo reconstruction algorithms
, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1. https://doi.org/10.1109/CVPR.2006.19
|
|
|
|
|
Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., & Lempitsky, V. (2022)
Resolution-robust Large Mask Inpainting with Fourier Convolutions
, Proceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022. https://doi.org/10.1109/WACV51458.2022.00323
|
|
|
|
|
Tomasi, C., & Kanade, T. (1992)
Shape and motion from image streams under orthography: a factorization method
, International Journal of Computer Vision, 9(2). https://doi.org/10.1007/BF00129684
|
|
|
|
|
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004)
Image quality assessment: From error visibility to structural similarity
, IEEE Transactions on Image Processing, 13(4). https://doi.org/10.1109/TIP.2003.819861
|
|
|
|
|
Wiberg, A. H., Lovhaug, S., Mathisen, M., Tschoerner, B., Resch, E., Erdt, M., & Prasolova-Forland, E. (2019)
Visualisation of KPIs in zero emission neighbourhoods for improved stakeholder participation using virtual reality
, IOP Conference Series: Earth and Environmental Science, 323, 012074
|
|
|
|
| last changed |
2024/11/17 22:05 |
|
