Moving museums into the Metaverse

Authors

  • Ruier Zhang

DOI:

https://doi.org/10.61173/ttjnt195

Keywords:

Neural Radiance Fields (NeRF), Virtual Reality, Metaverse, Cultural Heritage, Museum, Digital Preservation

Abstract

In recent years, the fusion of advanced technologies with virtual reality has opened new cultural preservation and engagement avenues. This dissertation explores the innovative application of Neural Radiance Fields (NeRF) technology in transcending the boundaries of physical museums and transporting their treasures into the metaverse. While classical computer vision has seen substantial progress, a developing intersection exists between NeRF and cultural heritage preservation. This study bridges this gap by introducing an approach that amalgamates NeRF techniques with the rich cultural wealth of museums.The conventional museum experience is extended into the metaverse through a novel methodology that leverages NeRF’s capabilities. The core objective is to enable individuals to explore digitized museum artifacts with unparalleled realism. NeRF technology captures intricate visual details and enables immersive interactions by rendering scenes with volumetric precision, transforming how cultural artifacts are experienced and understood.This dissertation delves into the technical intricacies of integrating NeRF technology into the metaverse. The implementation involves the reconstruction of 3D artifact models. The results underscore the potential of NeRF to reshape the cultural heritage landscape by bridging the gap between traditional museums and the boundless possibilities of the metaverse.

References

[1] O’Brian, Matt; Chan, Kelvin (October 28, 2021). “EXPLAINER: What is the metaverse and how will it work?”. ABC News. Associated Press. Archived from the original on December 4, 2021. Retrieved December 4, 2021.

[2] Chen, J. V., Farnham, S. D., & Hughes, J. J. (2011). The Metaverse as a Sociotechnical System of Systems.

[3] Morgan, J. (2014). Defining the Metaverse.

[4] Kakadiaris, I. A., Papadias, D., & Tzovaras, D. (2017). The Metaverse: A Collective Virtual Shared Space with Paradigms of Interaction.

[5] “Computer Vision – ECCV 2020”, Springer Science and Business Media LLC, 2020

[6] Richard Szeliski. “Computer Vision,” Springer Science and Business Media LLC, 2022

[7] Schönberger, J. L., & Frahm, J. M. (2017). 3D Match: Leveraging Local Geometry for Accurate 3D Reconstruction and Matching.

[8] Schönberger, J. L., Zheng, E., Pollefeys, M., & Frahm, J. M. (2018). A Benchmark for RGB-D Visual Odometry, 3D Reconstruction, and SLAM.

[9] Hong, Zhonghua & Yang, Yahui & Liu, Jun & Jiang, Shenlu & Haiyan, Pan & Zhou, Ruyan & Zhang, Yun & Han, Yanling & Wang, Jing & Yang, Shuhu & Zhong, Changyue. (2022). Enhancing 3D Reconstruction Model by Deep Learning and Its Application in Building Damage Assessment after Earthquake. Applied Sciences. 12. 9790. 10.3390/app12199790.

[10] Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In European Conference on Computer Vision (ECCV).

[11] Sitzmann, V., Zollhofer, M., & Wetzstein, G. (2021). NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. In International Conference on Computer Vision (ICCV).

[12] Srinivasan, P. P., Mildenhall, B., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2021). PlenOctrees for Real-time Rendering of Neural Radiance Fields. In ACM Transactions on Graphics (TOG), SIGGRAPH.

[13] Frank Dellaert. “Neural radiance fields explode on the scene,” Communications of the ACM, 2022

[14] Zhou, X., Li, Z., Zhu, R., Shi, C., & Tong, X. (2020). Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance. In SIGGRAPH Asia.

[15] Chen, X., Shen, Y., Zhang, Y., Shen, C., & Yan, Y.

[2021] . SPLATNet: Sparse Lattice Networks for Point Cloud Processing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Downloads

Published

2024-02-19