A Comprehensive Survey on Generative AI for Metaverse: Enabling Immersive Experience

Cao Y, Li S, Liu Y, Yan Z, Dai Y, Yu PS, Sun L. A comprehensive survey of AI-generated content (AIGC): a history of generative AI from GAN to ChatGPT. 2023. arXiv preprint arXiv:2303.04226

Buchanan BG. A (very) brief history of artificial intelligence. Ai Magazine. 2005;26(4):53–53.

Google Scholar 

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial networks. Communications of the ACM. 2020;63(11):139–44.

MathSciNet  Google Scholar 

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Process Syst. 2017;30.

OpenAI R. GPT-4 technical report. arXiv; 2023. pp. 2303–08 774.

Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, Chen M, Sutskever I. Zero-shot text-to-image generation. In: International Conference on Machine Learning. PMLR; 2021. pp. 8821–31.

Pichai S. An important next step on our AI journey. The keyword: Google; 2023.

Google Scholar 

Hanna DM. The use of artificial intelligence art generator “midjourney’’ in artistic and advertising creativity. J Design Sci Appl Arts. 2023;4(2):42–58.

Google Scholar 

Rephrase.ai: convert text into engaging AI videos in minutes. https://www.rephrase.ai/. Accessed 02 Dec 2023

Nichol A, Jun H, Dhariwal P, Mishkin P, Chen M. Point-E: a system for generating 3D point clouds from complex prompts. 2022. arXiv preprint arXiv:2212.08751

Podell D, English Z, Lacey K, Blattmann A, Dockhorn T, Müller J, Penna J, Rombach R. SDXL: improving latent diffusion models for high-resolution image synthesis. 2023. arXiv preprint arXiv:2307.01952

Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, Bashlykov N, Batra S, Bhargava P, Bhosale S et al. Llama 2: open foundation and fine-tuned chat models. 2023. arXiv preprint arXiv:2307.09288

Gozalo-Brizuela R, Garrido-Merchán EC. A survey of generative AI applications. 2023. arXiv preprint arXiv:2306.02781

Park S-M, Kim Y-G. A metaverse: taxonomy, components, applications, and open challenges. IEEE access. 2022;10:4209–51.

Google Scholar 

Azuma RT. A survey of augmented reality. Presence: teleoperators & virtual environments. 1997;6(4):355–85.

Rokhsaritalemi S, Sadeghi-Niaraki A, Choi S-M. A review on mixed reality: current trends, challenges and prospects. Appl Sci. 2020;10(2):636.

Google Scholar 

Zheng J, Chan K, Gibson I. Virtual reality. Ieee Potentials. 1998;17(2):20–3.

Google Scholar 

Davis A, Murphy J, Owens D, Khazanchi D, Zigurs I. Avatars, people, and virtual worlds: foundations for research in metaverses. J Assoc Inf Syst. 2009;10(2):1.

Google Scholar 

Abbate S, Centobelli P, Cerchione R, Oropallo E, Riccio EA. first bibliometric literature review on metaverse. In: IEEE Technology and Engineering Management Conference (TEMSCON EUROPE). IEEE. 2022;2022:254–60.

The Facebook company is now meta | meta. https://about.fb.com/news/2021/10/facebook-company-is-now-meta/. Accessed 02 Dec 2023.

Official site | second life - virtual worlds, virtual reality, VR, avatars, and free 3D chat. https://secondlife.com/. Accessed 02 Dec 2023.

History of second life - second life wiki. https://wiki.secondlife.com/wiki/History_of_Second_Life. Accessed 02 Dec 2023.

Greenwold S. Spatial computing. Master: Massachusetts Institute of Technology; 2003.

Google Scholar 

Qian L, Luo Z, Du Y, Guo L. Cloud computing: an overview. In: Cloud Computing: First International Conference, CloudCom 2009, Beijing, China, December 1–4, 2009. Proceedings 1. Springer; 2009. pp. 626–31.

Madakam S, Lake V, Lake V, Lake V, et al. Internet of things (IoT): a literature review. J Comput Commun. 2015;3(05):164.

Google Scholar 

Shi W, Cao J, Zhang Q, Li Y, Xu L. Edge computing: vision and challenges. IEEE Internet Things J. 2016;3(5):637–46.

Google Scholar 

Evans A, Romeo M, Bahrehmand A, Agenjo J, Blat J. 3D graphics on the web: a survey. Comput Graph. 2014;41:43–61.

Google Scholar 

Xu M, Ng WC, Lim WYB, Kang J, Xiong Z, Niyato D, Yang Q, Shen XS, Miao C. A full dive into realizing the edge-enabled metaverse: visions, enabling technologies, and challenges. IEEE Commun Surv Tutorials. 2022.

Bale AS, Ghorpade N, Hashim MF, Vaishnav J, Almaspoor Z. A comprehensive study on metaverse and its impacts on humans. Adv Hum Comput Interact. 2022;2022.

Pallavicini F, Pepe A, Minissi ME. Gaming in virtual reality: what changes in terms of usability, emotional response and sense of presence compared to non-immersive video games? Simulation & Gaming. 2019;50(2):136–59.

Google Scholar 

Bourlakis M, Papagiannidis S, Li F. Retail spatial evolution: paving the way from traditional to metaverse retailing. Electron Commer Res. 2009;9:135–48.

Google Scholar 

Wang G, Badal A, Jia X, Maltz JS, Mueller K, Myers KJ, Niu C, Vannier M, Yan P, Yu Z, et al. Development of metaverse for intelligent healthcare. Nat Mach Intell. 2022;4(11):922–9.

Google Scholar 

Tasa UB, Görgülü T. Meta-art: art of the 3-d user-created virtual worlds. Digital creativity. 2010;21(2):100–11.

Google Scholar 

Asara C. Real estate in the metaverse. 2022.

Moneta A. Architecture, heritage, and the metaverse. Tradit Dwellings Settlements Rev. 2020;32(1):37–49.

Google Scholar 

Gursoy D, Malodia S, Dhir A. The metaverse in the hospitality and tourism industry: an overview of current trends and future research directions. J Hosp Mark Manag. 2022;31(5):527–34.

Google Scholar 

Bibri SE, Allam Z. The metaverse as a virtual form of data-driven smart urbanism: on post-pandemic governance through the prism of the logic of surveillance capitalism. Smart Cities. 2022;5(2).

Hwang G-J, Chien S-Y. Definition, roles, and potential research issues of the metaverse in education: an artificial intelligence perspective. Comput Educ Artif Intell. 2022;3:100082.

Google Scholar 

Popescu GH, Ciurlău CF, Stan CI, Băcănoiu C, Tănase A. Virtual workplaces in the metaverse: immersive remote collaboration tools, behavioral predictive analytics, and extended reality technologies. Psychosociological Issues Hum Resour Manag. 2022;10(1):21–34.

Google Scholar 

Ning H, Wang H, Lin Y, Wang W, Dhelim S, Farha F, Ding J, Daneshmand M. A survey on the metaverse: the state-of-the-art, technologies, applications, and challenges. IEEE Internet Things J. 2023.

Chamola V, Bansal G, Das TK, Hassija V, Reddy NSS, Wang J, Zeadally S, Hussain A, Yu FR, Guizani M et al. Beyond reality: the pivotal role of generative AI in the metaverse. 2023. arXiv preprint arXiv:2308.06272

Qin HX, Hui P. Empowering the metaverse with generative AI: survey and future directions. In: 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW). IEEE; 2023. pp. 85–90.

Huynh-The T, Pham Q-V, Pham X-Q, Nguyen TT, Han Z, Kim D-S. Artificial intelligence for the metaverse: a survey. Eng Appl Artif Intell. 2023;117:105581.

Google Scholar 

Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z et al. A survey of large language models. 2023. arXiv preprint arXiv:2303.18223

Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, Lee P, Lee YT, Li Y, Lundberg S et al. Sparks of artificial general intelligence: early experiments with GPT-4. 2023. arXiv preprint arXiv:2303.12712

Hoffmann J, Borgeaud S, Mensch A, Buchatskaya E, Cai T, Rutherford E, de Las Casas D, Hendricks LA, Welbl J, Clark A, et al. An empirical analysis of compute-optimal large language model training. Adv Neural Inf Process Syst. 2022;35(30):016–30.

Google Scholar 

Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, Chen M, Sutskever I. Zero-shot text-to-image generation. In: International Conference on Machine Learning. PMLR; 2021. pp. 8821–31.

Midjourney. https://www.midjourney.com/home. Accessed 08 Feb 2024.

Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. pp. 10 684–695.

Craiyon - your free AI image generator tool: create AI art!. https://www.craiyon.com/. Accessed 09 Nov 2023.

Liu M, Shi J, Cao K, Zhu J, Liu S. Analyzing the training processes of deep generative models. IEEE transactions on visualization and computer graphics. 2017;24(1):77–87.

Google Scholar 

Make-a-video. https://makeavideo.studio/. Accessed 12 Dec 2023.

Imagen video. https://imagen.research.google/video/. Accessed 12 Dec 2023.

Synthesia - #1 ai video generator. https://www.synthesia.io/. Accessed 12 Dec 2023.

AI animation maker. https://www.krikey.ai/. Accessed 09 Nov 2023.

Research. https://openai.com/research/overview. Accessed 09 Nov 2023 .

AI voice generator: versatile text to speech software | murf ai. https://murf.ai/. Accessed 09 Nov 2023.

AI music generator - royalty free music for creators | soundful. https://soundful.com/. Accessed 26 Nov 2023.

Borsos Z, Marinier R, Vincent D, Kharitonov E, Pietquin O, Sharifi M, Roblek D, Teboul O, Grangier D, Tagliasacchi M, et al. Audiolm: a language modeling approach to audio generation. IEEE/ACM Transactions on Audio: Speech, and Language Processing; 2023.

Google Scholar 

Wang T, Zhang B, Zhang T, Gu S, Bao J, Baltrusaitis T, Shen J, Chen D, Wen F, Chen Q et al. Rodin: a generative model for sculpting 3D digital avatars using diffusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. pp. 4563–73.

Poole B, Jain A, Barron JT, Mildenhall B. Dreamfusion: text-to-3D using 2D diffusion. 2022. arXiv preprint arXiv:2209.14988

Li C, Zhang C, Waghwase A, Lee L-H, Rameau F, Yang Y, Bae S-H, Hong CS. Generative AI meets 3D: a survey on text-to-3D in AIGC era. 2023. arXiv preprint arXiv:2305.06131

Build more engaging games with ML agents | unity. https://unity.com/products/machine-learning-agents. Accessed 13 Dec 2023.

Documentation | sidefx. https://www.sidefx.com/docs/. Accessed 09 Nov 2023.

Tsugi studio | software for creatives. http://tsugi-studio.com/web/en/index.html. Accessed 09 Nov 2023.

Nvidia gameworks documentation - nvidia gameworks documentation. https://docs.nvidia.com/gameworks/index.html. Accessed 13 Dec 2023.

Nash C, Ganin Y, Eslami SA, Battaglia P. Polygen: an autoregressive generative model of 3D meshes. In: International conference on machine learning. PMLR; 2020, pp. 7220–7229.

Scenario - AI-generated game assets. https://www.scenario.com/. Accessed 30 May 2024.

AI dungeon. https://aidungeon.com/. Accessed 30 May 2024.

Plut C, Pasquier P. Generative music in video games: state of the art, challenges, and prospects. Entertainment Computing. 2020;33:100337.

Google Scholar 

Salge C, Green MC, Canaan R, Togelius J. Generative design in minecraft (GDMC) settlement generation competition. In: Proceedings of the 13th International Conference on the Foundations of Digital Games. 2018. pp. 1–10.

Jones D, Snider C, Nassehi A, Yon J, Hicks B. Characterising the digital twin: a systematic literature review. CIRP journal of manufacturing science and technology. 2020;29:36–52.

Google Scholar 

Hoffmann J, Borgeaud S, Mensch A, Buchatskaya E, Cai T, Rutherford E, Casas DdL, Hendricks LA, Welbl J, Clark A et al. Training compute-optimal large language models. 2022. arXiv preprint arXiv:2203.15556

Thoppilan R, De Freitas D, Hall J, Shazeer N, Kulshreshtha A, Cheng H-T, Jin A, Bos T, Baker L, Du Y et al. LAMDA: language models for dialog applications. 2022. arXiv preprint arXiv:2201.08239

Beattie C, Leibo JZ, Teplyashin D, Ward T, Wainwright M, Küttler H, Lefrancq A, Green S, Valdés V, Sadik A et al. Deepmind lab. 2016. arXiv preprint arXiv:1612.03801

Gong J, Foo LG, He Y, Rahmani H, Liu J. LLMS are good sign language translators. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. pp. 18 362–72.

Brown PF, Cocke J, Della Pietra SA, Della Pietra VJ, Jelinek F, Mercer RL, Roossin P. A statistical approach to language translation. In: Coling Budapest 1988 Volume 1: International Conference on Computational Linguistics. 1988.

Razavi AH, Inkpen D, Uritsky S, Matwin S. Offensive language detection using multi-level classification. In: Advances in Artificial Intelligence: 23rd Canadian Conference on Artificial Intelligence, Canadian AI 2010, Ottawa, Canada, May 31–June 2, 2010. Proceedings 23. Springer; 2010. pp. 16–27.

López-Gil J-M, Pereira J. Turning manual web accessibility success criteria into automatic: an LLM-based approach. Universal Access in the Information Society. 2024. pp. 1–16.

Metaverse-retail service quality: a future framework for retail service quality in the 3D internet. J Mark Manag. 29(13-14). https://www.tandfonline.com/doi/abs/10.1080/0267257X.2013.835742. Accessed 28 Jan 2024.

Sitaram S, Choudhury M, Patra B, Chaudhary V, Ahuja K, Bali K. Everything you need to know about multilingual LLMS: towards fair, performant and reliable models for languages of the world. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 6: Tutorial Abstracts). 2023. pp. 21–6.

Musical metaverse: vision, opportunities, and challenges | personal and ubiquitous computing. https://link.springer.com/article/10.1007/s00779-023-01708-1. Accessed 28 Jan 2024.

Sun Y, Xu Y, Cheng C, Li Y, Lee CH, Asadipour A. Travel with wander in the metaverse: An ai chatbot to visit the future earth. In: IEEE 24th International Workshop on Multimedia Signal Processing (MMSP). IEEE. 2022;2022:1–6.

Vu MD, Wang H, Li Z, Chen J, Zhao S, Xing Z, Chen C. GPTVoicetasker: LLM-powered virtual assistant for smartphone. 2024. arXiv preprint arXiv:2401.14268

Interactive tools for education in automatic control | ieee journals & magazine | ieee xplore. https://ieeexplore.ieee.org/abstract/document/687617. Accessed 28 Jan 2024.

Krauss C, Bassbouss L, Upravitelev M, An T-S, Altun D, Reray L, Balitzki E, El Tamimi T, Karagülle M. Opportunities and challenges in developing educational AI-assistants for the metaverse,” in International Conference on Human-Computer Interaction. Springer; 2024. pp. 219–238.

International journal on artificial intelligence tools. https://www.worldscientific.com/doi/abs/10.1142/S0218213011000188. Accessed 28 Jan 2024.

Alhawiti KM. Natural language processing and its use in education. Int J Adv Comput Sci Appl. 2014;5(12).

Virtual reality therapy in mental health | annual review of clinical psychology. https://www.annualreviews.org/doi/abs/10.1146/annurev-clinpsy-081219-115923. Accessed 02 Feb 2024.

King DR, Nanda G, Stoddard J, Dempsey A, Hergert S, Shore JH, Torous J. An introduction to generative artificial intelligence in mental health care: considerations and guidance. Current psychiatry reports. 2023;25(12):839–46.

Google Scholar 

Kholmogorova A, Tarhanova P, Shalygina O. Standards of physical beauty and mental health in children and young people in the era of the information revolution. Int J Cult Ment Health. 2018;11(1):87–98.

Google Scholar 

Therapy in virtual environments-clinical and ethical issues | telemedicine and e-health. https://www.liebertpub.com/doi/abs/10.1089/tmj.2011.0195. Accessed 02 Feb 2024.

Soviero B, Kuhn D, Salle A, Moreira VP. ChatGPT goes shopping: LLMS can predict relevance in ecommerce search. In: European Conference on Information Retrieval. Springer; 2024. pp. 3–11.

Hudson J. Virtual immersive shopping experiences in metaverse environments: predictive customer analytics, data visualization algorithms, and smart retailing technologies. Linguistic and Philosophical Investigations. 2022;21:236–51.

Google Scholar 

Liu Y, Shi D, Skaar SB, Tan J. Development and experiment of CSM-based industrial robot servoing control system. In: 2013 IEEE International Conference on Cyber Technology in Automation, Control and Intelligent Systems. IEEE; 2013. pp. 108–113.

Mirage. https://mirageml.com/. Accessed 09 Nov 2023.

Wu J, Zhang C, Xue T, Freeman B, Tenenbaum J. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. Adv Neural Inf Process Syst. 2016;29.

Research. https://openai.com/research/overview. Accessed 09 Nov 2023.

Lensa - prisma labs. https://prisma-ai.com/lensa. Accessed 10 Nov 2023.

Research room - AI generated artwork - nightcafe creator. https://creator.nightcafe.studio/creation/0hVaw7Kw6AD3qhL460Av. Accessed 10 Nov 2023.

AI art generator | create AI images and photos online free | openart. https://openart.ai/. Accessed 10 Nov 2023.

Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov RR. Le QV. Xlnet: Generalized autoregressive pretraining for language understanding. Adv Neural Inf Process Syst; 2019. p. 32.

Google Scholar 

Chowdhery A, Narang S, Dev

Comments (0)

No login
gif