| id |
caadria2023_43 |
| authors |
Onishi, Ryo, Fukuda, Tomohiro and Yabuki, Nobuyoshi |
| year |
2023 |
| title |
Remote Sharing System for 3D Real Objects with Point Cloud Reconstruction Using Deep Learning Point Cloud Completion |
| source |
Immanuel Koh, Dagmar Reinhardt, Mohammed Makki, Mona Khakhar, Nic Bao (eds.), HUMAN-CENTRIC - Proceedings of the 28th CAADRIA Conference, Ahmedabad, 18-24 March 2023, pp. 381–390 |
| doi |
https://doi.org/10.52842/conf.caadria.2023.2.381
|
| summary |
Currently, teleconferencing via the internet is widely used in society. However, physical models such as design study models, which are often used in face-to-face meetings in the fields of architecture and urban design, cannot be shared in teleconferences where information is shared on a display. Telepresence is a technology for sharing 3D real objects at a distance that gives the sensation of sharing and experiencing the environment and objects at a remote location. As one such technology, a system has been developed in which the point cloud of a real object acquired by a camera is divided into objects by instance segmentation, and the divided point cloud is transmitted to the remote user, who can manipulate it on mixed reality. There is a problem of missing point clouds in areas not seen by the RGB-D camera, such as occlusion and the back of the camera. This research aims to develop a system that can remotely manipulate point clouds with more accurate geometry by using a point cloud completion technique based on deep learning to complement missing point clouds. This system is expected to contribute to smoother teleconferencing of remote participants. |
| keywords |
Remote meeting, Real-time sharing, Three-dimensional remote sharing, Mixed Reality, Point cloud completion |
| series |
CAADRIA |
| email |
|
| full text |
 file.pdf (1,939,371 bytes) |
| references |
Content-type: text/plain
|
Dai, A., Qi, C. R., & Nie§ner, M. (2017)
 Shape Completion Using 3D-encoder-predictor Cnns and Shape Synthesis
, Paper presented at the Proceedings - 3th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 217, 217-January 6545-6554. Available at: https://doi.org/1.119/CVPR.217.693
|
|
|
|
|
Kurillo, G., Bajcsy, R., Nahrsted, K., & Kreylos, O. (2008)
Immersive 3D Environment for Remote Collaboration and Training of Physical Activities
, Paper presented at the Proceedings - IEEE Virtual Reality, 269-27. Available at: https://doi.org/1.119/VR.28.448795
|
|
|
|
|
Onishi, R., Fukuda, T. & Yabuki, N. (2022)
A Remote Sharing Method of 3D Physical Objects with Instance-segmented 3D Point Cloud Acquired in Real Time
, Proceedings of the 27th International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA 222), pp. 395-44
|
|
|
|
|
Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017)
Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation
, Paper presented at the Proceedings - 3th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 217, 217-January 77-85. Available at: https://doi.org/1.119/CVPR.217.16
|
|
|
|
|
Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017)
Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
, Paper presented at the Advances in Neural Information Processing Systems, 17-December 51-519
|
|
|
|
|
Qian, P., Huynh, V. S. H., Wang, N., Anmulwar, S., Mi, D., & Tafazolli, R. R. (2022)
Remote Production for Live Holographic Teleportation Applications in 5g Networks
, IEEE Transactions on Broadcasting, 68(2), 451-463. Available at: https://doi.org/1.119/TBC.222.3161745
|
|
|
|
|
Stotko, P., Krumpen, S., Hullin, M. B., Weinmann, M., & Klein, R. (2019)
Slamcast: Large-scale, Real-time 3D Reconstruction and Streaming for Immersive Multi-client Live Telepresence
, IEEE Transactions on Visualization and Computer Graphics, 25(5), 212-2112. Available at: https://doi.org/1.119/TVCG.219.2899231
|
|
|
|
|
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017)
Attention is All You Need
, Paper presented at the Advances in Neural Information Processing Systems, 217-December 5999-69
|
|
|
|
|
Wang, P., Bai, X., Billinghurst, M., Zhang, S., Wei, S., Xu, G., He, W., Zhang, X., & Zhang, J. (2021)
3Dgam: Using 3D Gesture and Cad Models for Training on Mixed Reality Remote Collaboration
, Multimedia Tools and Applications, 8(2), 3159-3184. Available at: https://doi.org/1.17/s1142-2-9731-7
|
|
|
|
|
Yang, Y., Feng, C., Shen, Y., & Tian, D. (2018)
Foldingnet: Point Cloud Auto-encoder Via Deep Grid Deformation
, Paper presented at the Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 26-215. Available at: https://doi.org/1.119/CVPR.218.29
|
|
|
|
|
Yu, X., Rao, Y., Wang, Z., Liu, Z., Lu, J., & Zhou, J. (2021)
Pointr: Diverse Point Cloud Completion with Geometry-aware Transformers
, Paper presented at the Proceedings of the IEEE International Conference on Computer Vision, 12478-12487. Available at: https://doi.org/1.119/ICCV48922.221.1227
|
|
|
|
|
Yuan, W., Khot, T., Held, D., Mertz, C., & Hebert, M. (2018)
Pcn: Point Completion Network
, Paper presented at the Proceedings - 218 International Conference on 3D Vision, 3DV 218, 728-737. Available at: https://doi.org/1.119/3DV.218.88
|
|
|
|
| last changed |
2023/06/15 23:14 |
|
