| id |
caadria2024_248 |
| authors |
Hu, Anqi, Yabuki, Nobuyoshi and Fukuda, Tomohiro |
| year |
2024 |
| title |
Generating 4D Plant Models for Virtual Reality Environments Using the Instant Neural Graphics Primitives and Stable Diffusion Model |
| source |
Nicole Gardner, Christiane M. Herr, Likai Wang, Hirano Toshiki, Sumbul Ahmad Khan (eds.), ACCELERATED DESIGN - Proceedings of the 29th CAADRIA Conference, Singapore, 20-26 April 2024, Volume 3, pp. 421–430 |
| doi |
https://doi.org/10.52842/conf.caadria.2024.3.421
|
| summary |
This paper addresses the challenge of enhancing realism in virtual reality (VR) environmental design, particularly by overcoming the limitations of traditional 3D plant modeling methods that fail to capture dynamic and temporal nuances across annual seasons. The approach integrates realistic, time-sensitive modifications into VR plant modeling. It employs a methodology where source images for instant neural graphics primitives (Instant-ngp) are preprocessed using a Stable Diffusion model optimized by a Low-Rank Adaptation (LoRA) focusing on tree structures. This preprocessing step enriches Instant-ngp's input data, enabling the creation of 4D plant models that exhibit both spatial detail and temporal dynamics, mirroring natural seasonal variations. Stable Diffusion and LoRA are applied beforehand to improve the realism of the generated models. Virtual source trees are utilized for testing and refining the approach, aiming to enhance the representation of plant models in VR environments. This research contributes to making VR simulations more immersive and realistic, with potential applications in virtual landscaping, urban planning, and therapeutic environments. The study acknowledges the initial nature of this research and the ongoing need for exploration to fully realize these applications' potential. |
| keywords |
Neural Radiance Fields (NeRF), Diffusion Models, 4D Plant Modeling, Virtual Reality (VR) Environments, Environmental Design |
| series |
CAADRIA |
| email |
|
| full text |
 file.pdf (796,235 bytes) |
| references |
Content-type: text/plain
|
Bournez, E., Landes, T., Saudreau, M., Kastendeuch, P., & Najjar, G. (2017)
 Kohyas GUI [Python]
, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W3, 113-120. https://doi.org/10.5194/isprs-archives-XLII-2-W3-113-2017
|
|
|
|
|
Hore, A., & Ziou, D. (2010)
A complete, cross-platform solution to record, convert and stream audio and video.
, 2010 20th International Conference on Pattern Recognition, 2366-2369. https://doi.org/10.1109/ICPR.2010.579
|
|
|
|
|
Müller, T., Evans, A., Schied, C., & Keller, A. (2022)
LoRA: Low-Rank Adaptation of Large Language Models
, ACM Transactions on Graphics, 41(4), 1-15. https://doi.org/10.1145/3528223.3530127
|
|
|
|
|
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022)
Learning Generative Models of Textured 3D Meshes from Real-World Images
, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10674-10685. https://doi.org/10.1109/CVPR52688.2022.01042
|
|
|
|
|
Schonberger, J. L., & Frahm, J.-M. (2016)
Structure-from-Motion Revisited
, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4104-4113. https://doi.org/10.1109/CVPR.2016.445
|
|
|
|
|
Sun, R., Jia, J., & Jaeger, M. (2009)
Intelligent tree modeling based on L-system
, 2009 IEEE 10th International Conference on Computer-Aided Industrial Design & Conceptual Design, 1096-1100. https://doi.org/10.1109/CAIDCD.2009.5375256
|
|
|
|
|
Talton, J. O., Lou, Y., Lesser, S., Duke, J., Mìch, R., & Koltun, V. (2011)
Metropolis procedural modeling
, ACM Transactions on Graphics, 30(2), 1-14. https://doi.org/10.1145/1944846.1944851
|
|
|
|
|
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018)
Adding Conditional Control to Text-to-Image Diffusion Models
, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 586-595. https://doi.org/10.1109/CVPR.2018.00068
|
|
|
|
| last changed |
2024/11/17 22:05 |
|
