| id |
ecaade2021_037 |
| authors |
Kikuchi, Takuya, Fukuda, Tomohiro and Yabuki, Nobuyoshi |
| year |
2021 |
| title |
Automatic Diminished Reality-Based Virtual Demolition Method using Semantic Segmentation and Generative Adversarial Network for Landscape Assessment |
| source |
Stojakovic, V and Tepavcevic, B (eds.), Towards a new, configurable architecture - Proceedings of the 39th eCAADe Conference - Volume 2, University of Novi Sad, Novi Sad, Serbia, 8-10 September 2021, pp. 529-538 |
| doi |
https://doi.org/10.52842/conf.ecaade.2021.2.529
|
| summary |
In redevelopment projects in mature cities, it is important to visualize the future landscape. Diminished reality (DR) based methods have been proposed to represent the future landscape after the structures are removed. However, two issues remain to be addressed in previous studies. (1) the user needs to prepare 3D models of the structure to be removed and the background structure to be rendered after removal as preprocessing, and (2) the user needs to specify the structure to be removed in advance. In this study, we propose a DR method that detects the objects to be removed using semantic segmentation and completes the removal area using generative adversarial networks. With this method, virtual removal can be performed without preparing 3D models in advance and without specifying the removal target in advance. A prototype system was used for verification, and it was confirmed that the method can represent the future landscape after removal and can run at an average speed of about 8.75 fps. |
| keywords |
landscape visualization; virtual demolition; diminished reality (DR); deep learning; generative adversarial network (GAN); semantic segmentation |
| series |
eCAADe |
| email |
|
| full text |
 file.pdf (6,051,991 bytes) |
| references |
Content-type: text/plain
|
Bishop, I.D. and Lange, E. (2005)
 Visualization in landscape and environmental planning
, Technology and applications
|
|
|
|
|
Caudell, T.P. and Mizell, D.W. (1992)
Augmented reality: an application of heads-up display technology to manual manufacturing processes
, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences, pp. 659-669
|
|
|
|
|
Chao, P., Kao, C.-Y., Ruan, Y., Huang, C.-H. and Lin, Y.-L. (2019)
HarDNet: A low memory traffic network
, Proceedings of the IEEE International Conference on Computer Vision, pp. 3551-3560
|
|
|
|
|
Flores, A. and Belongie, S. (2010)
Removing pedestrians from Google Street View images
, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010, pp. 53-58
|
|
|
|
|
Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014)
Generative adversarial nets
, Advances in Neural Information Processing Systems, pp. 2672-2680
|
|
|
|
|
Huang, W. and Zheng, H. (2018)
Architectural drawings recognition and generation through machine learning
, Recalibration on Imprecision and Infidelity - Proceedings of the 38th Annual Conference of the Association for Computer Aided Design in Architecture, ACADIA 2018, pp. 156-165
|
|
|
|
|
Iizuka, S., Simo-Serra, E. and Ishikawa, H. (2017)
Globally and locally consistent image completion
, ACM Transactions on Graphics
|
|
|
|
|
Inoue, K., Fukuda, T. and Yabuki, N. (2018)
Tracking Robustness and Green View Index Estimation of Augmented and Diminished Reality for Environmental Design: PhotoAR+DR 2017 project
, Proceedings of the 23rd International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA 2018), pp. 339-348
|
|
|
|
|
Ki, D. and Lee, S. (2021)
Analyzing the effects of Green View Index of neighborhood streets on walking time using Google Street View and deep learning
, Landscape and Urban Planning, 205, p. -
|
|
|
|
|
Kido, D., Fukuda, T. and Yabuki, N. (2020)
Diminished reality system with real-time object detection using deep learning for onsite landscape simulation during redevelopment
, Environmental Modelling and Software, 131, p. -
|
|
|
|
|
Li, Q., Shi, Y., Auer, S., Roschlaub, R., Möst, K., Schmitt, M., Glock, C. and Zhu, X.X. (2020)
Detection of undocumented building constructions from official geodata using a convolutional neural network
, Remote Sensing, 12, pp. 1-21
|
|
|
|
|
Long, J., Shelhamer, E. and Darrell, T. (2015)
Fully convolutional networks for semantic segmentation
, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 431-440
|
|
|
|
|
Mann, S. (eds) (1994)
Mediated Reality
, M.I.T. Media Lab Perceptual Computing Section Cambridge, Ma Technical Report TR 260
|
|
|
|
|
Mori, S., Ikeda, S. and Saito, H. (2017)
A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects
, IPSJ Transactions on Computer Vision and Applications, 9, pp. 17-
|
|
|
|
|
Navab, N., Bani-Kashemi, A. and Mitschke, M, (1999)
Merging visible and invisible: Two Camera-Augmented Mobile C-arm (CAMC) applications
, Proceedings - 2nd IEEE and ACM International Workshop on Augmented Reality, IWAR 1999, pp. 134-141
|
|
|
|
|
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T. and Efros, A.A. (2016)
Context Encoders: Feature Learning by Inpainting
, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2536-2544
|
|
|
|
|
Ullman, S. (1979)
The interpretation of structure from motion
, Proceedings of the Royal Society of London. Series B. Biological Sciences, 203, pp. 405-426
|
|
|
|
|
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X. and Huang, T.S. (2018)
Generative Image Inpainting with Contextual Attention
, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 5505-5514
|
|
|
|
|
Zokai, S., Esteve, J., Genc, Y. and Navab, N. (2003)
Multiview paraperspective projection model for diminished reality
, Proceedings - 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR 2003, pp. 217-226
|
|
|
|
| last changed |
2022/06/07 07:52 |
|
