Pengyuan Zhou, Lik-Hang Lee, Zhi Liu, Hang Qiu, Tristan Braud, Aaron Yi Ding, Sasu Tarkoma, Pan Hui
The metaverse aims to blur the boundary between the physical world and digital content. To achieve this goal, the metaverse relies heavily on extended reality (XR), the Internet of Things, and communication technologies. Concurrently, connected vehicles and intelligent transportation systems (ITSs) are envisioned as the future paradigm of driving and becoming reality thanks to increasingly powerful onboard vehicular processing capacity and advanced vehicle-to-everything networking technologies.
Observing a large number of overlapping enabling technologies, we expect a convergence between the metaverse and connected vehicles that would eventually benefit both fields. Connected and automated vehicles are mobile platforms equipped with significant sensing and computing capabilities that can broaden metaverse use cases. On the other hand, immersive metaverse applications can improve the driver’s experience as well as passengers’ in-vehicle entertainment and passenger.
Richer information collected and created from the metaverse has created new challenges, such as information filtering, object positioning, vision transformation, and so on. These challenges are often computation intensive and bring considerable additional delay to connected and automated vehicles, which demand near-real-time reactions. Researchers have thus proposed edge and cloud computing, machine learning, and computer vision solutions to tackle such challenges.
This special issue of IEEE Vehicular Technology Magazine aims to present works that apply different techniques to improve the driving experience of automatic/connected vehicle services in terms of facing the coming metaverse era. We received 23 high-quality submissions, of which five were eventually accepted after a rigorous review process. These articles cover several different aspects of the metaverse and connected vehicles.
The first article [A1] highlights the potential of connected automated vehicle (CAV)-assisted mobile crowdsensing in shaping the metaverse. By leveraging edge intelligence, the proposed framework offers enhanced data quality and transmission efficiency. The authors point out that the transition to 6G demands content-aware communication that aligning with the metaverse’s nuances. However, the acceleration of mobile crowdsensing in the metaverse raises concerns about data privacy and security, emphasizing the importance of robust anomaly detection and security measures.
The next article [A2] delves into the integration of pretrained foundation models with edge intelligence for the metaverse. The authors present a model caching and inference framework optimized for mobile edge networks. A novel least context algorithm is introduced, leveraging a proposed metric, namely, age of context, to enhance mobile artificial intelligence (AI) service accuracy. This research offers a glimpse into the future interplay between edge AI and the metaverse.
The third article [A3] highlights a multifaceted approach to enhance vehicular automation by harnessing the power of digital twin networks (DTNs), with a keen focus on perception, planning, and control. By integrating diverse learning methodologies, the authors set a new benchmark for traffic management and vehicular safety. This research not only underscores the future of sustainable transportation but also underscores the synergy between CAVs and DTNs for optimized driving experiences.
In the fourth article [A4], the authors introduced a DT- and AI-empowered panoramic video streaming scheme for XR-assisted connected AVs that reduces transmission latency and intelligently responds to user requirements. Specifically, the authors proposed a DT-enabled distributed XR service management framework that provides low latency and smooth XR services across different domains in the vehicular metaverse. In addition, they presented a case study on XR streaming-based virtualized resource allocation and a novel deep reinforcement learning-based method that minimizes transmission latency. Quantitative experimental results demonstrated that the positive role of AI in connected AV networks can be enhanced by DTs. Finally, open issues and potential research directions for the XR-assisted vehicular metaverse were discussed.
In the final article [A5], the authors discuss the challenges and requirements of edge caching for metaverse CAVs (meta-CAVs) and ITSs (meta-ITSs). The specific requirements of vehicular applications translate to specific needs for dynamic, secure, and intelligent caching. To address these challenges, the authors propose using AI-assisted content caching for meta-CAVs and meta-ITSs. They survey the techniques commonly found in the literature that can improve caching performance. These techniques, such as long short-term memory, reinforcement learning, and federated learning, allow for predicting the popular content, deciding caching location based on spatial and temporal data, and devising dynamic and secure caching policies. A case study is presented, showcasing the advantages of using multiagent federated reinforcement learning in edge caching for meta-CAVs and meta-ITSs.
In conclusion, the articles featured in this special issue shed light on the intricate interplay and potential synergies among the metvaverse, connected vehicles, and intelligent transportation systems, presenting both the inherent challenges and exciting opportunities at this crossroad. We are deeply grateful to the authors for their trailblazing contributions and to the reviewers for their meticulous and insightful evaluations that have significantly enhanced the quality of each article. Our sincere appreciation also goes out to Prof. Javier Gozalvez, the editor-in-chief, and the committed team at IEEE Vehicular Technology Magazine, whose support and guidance were instrumental in bringing this special issue to fruition. It is our earnest hope that readers will find the insights presented in these articles both intriguing and practically valuable.
Pengyuan Zhou (pyzhou@ustc.edu.cn) is a research associate professor at the University of Science and Technology of China, Hefei 230000, China. He received his Ph.D. degree from the University of Helsinki. He was a Europe Union Marie Curie Innovative Training Network Early-Stage Researcher from 2015 to 2018. His research focuses on distributed networking artificial intelligence systems, mixed-reality development, and vehicular networks.
Lik-Hang Lee (lik-hang.lee@polyu.edu.hk) is an assistant professor at Hong Kong Polytechnic University (PolyU), Hong Kong. He received his Ph.D. degree from the Hong Kong University of Science and Technology. He was an assistant professor with Korea Advanced Institute of Science and Technology before joining PolyU. His research interests are building and designing various human-centric computing systems, specializing in augmented and virtual realities.
Zhi Liu (liuzhi@uec.ac.jp) is an associate professor at the University of Electro-Communications, Chofu 182-8585, Japan. His research interests include video network transmission and mobile edge computing. He is an editorial board member of IEEE Transactions on Multimedia, IEEE Open Journal of the Computer Society, and Wireless Networks. He is a Senior Member of IEEE.
Hang Qiu (hangq@ucr.edu) is an assistant professor at the University of California, Riverside (UCR), Riverside, CA 92521 USA. He received his Ph.D. degree from the Networked Systems Lab at the University of Southern California. Prior to joining UCR, he was a software engineer at Waymo. His research focuses on networked systems’ problems at the intersection of machine learning systems, robotics, cyber-physical systems, edge computing, and networking.
Tristan Braud (braudt@ust.hk) is an assistant professor at the Hong Kong University of Science and Technology, Hong Kong. He received his Ph.D. degree from Université Grenoble Alpes in 2016. His primary research interests are the metaverse and augmented and virtual reality, with interests in pervasive and cloud computing as well as human-centered system designs.
Aaron Yi Ding (Aaron.Ding@tudelft.nl) is an associate professor at the Delft University of Technology, 2628 Delft, The Netherlands. He received his Ph.D. degree from the University of Helsinki in 2015. His research focuses on edge artificial intelligence solutions for cyber-physical systems in smart health, mobility, and energy domains.
Sasu Tarkoma (sasu.tarkoma@helsinki.fi) is a professor and dean of the Faculty of Science at the University of Helsinki, 00100 Helsinki, Finland. His research interests include Internet technology, distributed systems, data analytics, and mobile and ubiquitous computing. He is a fellow of the Institution of Engineering and Technology and the European Alliance for Innovation.
Pan Hui (panhui@ust.hk) is a chair professor at the Hong Kong University of Science and Technology, Hong Kong. He is also the Nokia Chair in Data Science and a professor at the University of Helsinki. His works have covered a wide spectrum of topics, including augmented reality, mobile computing, and networking. He is a Fellow of IEEE, an international fellow of the Royal Academy of Engineering, and a distinguished scientist of the Association for Computing Machinery.
[A1] X. Yu et al., “When connected and automated vehicles meet mobile crowdsensing: A perception and transmission framework in the metaverse,” IEEE Veh. Technol. Mag., vol. 18, no. 4, pp. 22–34, Dec. 2023, doi: 10.1109/MVT.2023.3320865.
[A2] M. Xu et al., “Sparks of generative pretrained transformers in edge intelligence for the metaverse: Caching and inference for mobile artificial intelligence-generated content services,” IEEE Veh. Technol. Mag., vol. 18, no. 4, pp. 35–44, Dec. 2023, doi: 10.1109/MVT.2023.3323757.
[A3] Y. Kang, Q. Song, J. Song, F. Pan, L. Guo, and A. Jamalipour, “How does a digital twin network work well for connected and automated vehicles: Joint perception, planning, and control,” IEEE Veh. Technol. Mag., vol. 18, no. 4, pp. 45–55, Dec. 2023, doi: 10.1109/MVT.2023.3328107.
[A4] S. Li, X. Lin, J. Wu, W. Zhang, and J. Li, “Digital twin and artificial intelligence-empowered panoramic video streaming: Reducing transmission latency in the extended reality-assisted vehicular metaverse,” IEEE Veh. Technol. Mag., vol. 18, no. 4, pp. 56–65, Dec. 2023, doi: 10.1109/MVT.2023.3321172.
[A5] B. Mao, Y. Liu, J. Liu, and N. Kato, “AI-assisted edge caching for metaverse of connected and automated vehicles: Proposal, challenges, and future perspectives,” IEEE Veh. Technol. Mag., vol. 18, no. 4, pp. 66–74, Dec. 2023, doi: 10.1109/MVT.2023.3327514.
Digital Object Identifier 10.1109/MVT.2023.3333444