Bomin Mao, Yangbo Liu, Jiajia Liu, Nei Kato
©SHUTTERSTOCK.COM/BLUE PLANET STUDIO
The upcoming metaverse will significantly promote the safety and efficiency of connected and automated vehicles (CAVs) as well as intelligent transportation systems (ITSs) with immersive information exchange between the parallel digital and physical worlds. To enable the virtual world to better reflect the physical world, a great deal of sensed information in types of text, pictures, voice, and videos should be fetched by metaverse applications. Edge caching has been considered to improve transmission quality and data protection by storing the needed contents near users rather than in the cloud. However, qualified edge caching for the metaverse of CAVs (meta-CAVs) and metaverse of ITSs (meta-ITSs) is challenged by ubiquitous mobilities, diversified requirements, dynamic content popularity, and heterogeneous infrastructure. In this article, we elaborate on the requirements and challenges of edge caching for meta-CAVs and meta-ITSs. We then discuss how artificial intelligence (AI) can be used in edge caching to improve the performance and security of meta-CAVs and meta-ITSs. To evaluate our idea, a case study with the Multi-Agent Federated Reinforcement Learning (MAFRL)-based intelligent edge caching is provided. Some perspective research directions are given to illuminate more ideas.
With the development of big data, 5G, the Internet of Things (IoT), and AI, researchers from academia and industry have been studying the metaverse through extended reality devices, including virtual reality (VR), augmented reality, and mixed reality equipment, which creates an avatar for every physical user to simulate the interactions between human beings and the actual world [1]. As the virtual world is created and operated with sensed multidimensional real-time information of the actual world, how the digital world changes in response to the different behaviors of avatars can be recorded to guide the practical policy design in the physical world, which can significantly improve reliability. Moreover, since an avatar is created for every user, the metaverse can provide users with immersive experiences just like their avatars feel in the digital world. Due to these advantages, the meta-ITS has been proposed to guide the construction and management of ITS infrastructure by road governors, while CAVs can make more efficient and reasonable driving decisions. Additionally, passengers free from driving can enjoy the benefits of the metaverse in in-automobile work and entertainment without worrying about safety issues.
Compared to conventional game-based or social-based metaverse services, meta-CAVs and meta-ITSs are expected to have the features of anticipating the future traffic situation in a few seconds in the digital world through local-edge-cloud collaborative perception and computation, which can be adopted to guide the traffic participants and avoid accidents in the real world. The structure of meta-CAVs and meta-ITSs consists of three parts, as shown in Figure 1, including the actual world, the digital world, and the infrastructure, to realize and connect the two worlds. In the actual world, CAVs cooperatively perceive the environment through the sensors deployed on board and in the ITS infrastructure [2], such as cameras, vehicle-equipped radars, and pedestrian-equipped devices to support different metaverse applications. Besides real-time information, historical data are also available since more details of the actual world can increase the similarity of the digital world. The required information is transmitted to servers of different metaverse applications, while different communication technologies, including cellular vehicle-to-everything (C-V2X) networks, space-air-ground integrated networks (SAGINs), and Wi-Fi, as shown in Figure 1, provide convenience for meta-ITSs and meta-CAVs. Then, the digital twin technology enables the virtual world to be created based on the information.
Figure 1 The structure of the meta-CAVs and ITSs.
The avatars in the digital world need to not only imitate the behaviors of the corresponding physical entities but also make their own decisions according to the traffic conditions. With the bidirectional data transmissions between two worlds, the digital world can better simulate the actual scenario. However, the application of meta-CAV and meta-ITS services will have to propose strict requirements for transmission quality. For instance, the CAV motion control information is required to be fetched within 0.5 ms, and the ITS management data need to be fetched within 10 ms [3]. Moreover, some side effects, including an excessive traffic burden, will be caused, which is beyond the transmission capability of the current cloud storage architecture. Through caching, the content in edge servers proximate to the users can significantly alleviate the communication overhead and reduce the latency [4]. Furthermore, emerging transmission technologies, like millimeter-wave (mm-wave) and terahertz (THz), which work around 60 GHz and 300 GHz, respectively, can enlarge the available spectrum resources greatly and enhance the transmission quality, relieving the side effects of meta-CAV and meta-ITS services.
On the other hand, unlike traditional network services, meta-CAVs and meta-ITSs have proposed specialized requirements for edge caching. First, metaverse services generate various kinds of content, including texts, photos, audio, and videos, which have diversified quality of service (QoS) requirements. Second, meta-CAVs and meta-ITSs are highly dynamic, meaning that the caching solutions should converge fast to update the content. Third, the heterogeneous networks, including SAGIN, C-V2X, and Wi-Fi, require edge caching solutions to be flexible and suitable for different networks and hardware. We can find that how to cache various kinds of content is concerned with many factors, including network overhead, QoS requirements, and content popularity, and that AI has been adopted to solve such issues [5] as conventional caching strategies, such as first-in first-out (FIFO) and last-in first-out (LIFO), merely concentrate on recent request frequency. Specifically, researchers have evaluated the improved performance of AI in popularity prediction, caching placement, and content delivery, which demonstrates its promising perspectives in content caching for meta-CAVs and meta-ITSs.
Additionally, edge caching can cause concern from passengers and pedestrians for security and privacy due to limited protection in edge servers. To address this issue, federated learning (FL) and blockchain have been recognized as two key paradigms for protecting content security for metaverse applications [6].
This article focuses on AI-assisted content caching for meta-CAVs and meta-ITSs to improve the performance of provided services. Specifically, we discuss the new challenges and requirements of the metaverse for content caching. Then, how AI should be adopted on the network edge to improve the performance and security protection of content caching is explained. To explain our proposal clearly, we give a case study where FL and reinforcement learning are utilized to cache the content of meta-CAVs and meta-ITSs on the network edge. We further provide some future directions to illuminate more meaningful ideas before summarizing our article.
Due to the service requirements of autonomous driving and the characteristics of vehicular networks, edge caching for meta-CAVs and meta-ITSs has confronted distinctive challenges. In this section, we introduce the challenges and requirements of edge caching.
The meta-CAVs connecting with roadside infrastructure and passengers can provide multiple metaverse services. Aiming at stabilizing and enhancing the QoS, edge servers can gather, preprocess, and cache the corresponding data for meta-CAVs to fetch. However, due to the diversified QoS requirements and content popularity, the caching strategy should not regard every service as the same [7]. In some cases, the content caching decisions may contradict when considering the content’s urgency and popularity. A tradeoff should be made to decide the caching priority according to those service properties.
For instance, the autonomous driving service of meta-CAVs requires the adjacent environment data, including the blind areas, traffic jams, and road conditions, to be completely transmitted within a few seconds to avoid accidents. There is no doubt that traffic safety-related contents should be granted the top priority to be cached regardless of its popularity. Instead, the content affecting the driving path selection, such as traffic load variation, charging station deployments, and road scenarios, has milder delay thresholds, for which it has lower priorities in caching policy design. Besides content urgency requirements, popularity is another key factor to be considered in caching policy design, especially for the multimedia contents required by passengers for work and entertainment. It should be noted that the QoS requirement and contents popularity vary with time and location on account of city functions, population composition, and mobility patterns.
As we know, the dynamics of traditional vehicular networks come from moving automobiles and pedestrians. For meta-CAVs and meta-ITSs, the dynamics can be caused by many factors since an increasing number of factors are considered to create the digital world. For example, changes in weather conditions and road infrastructure should be considered since both of them affect caching policy design. Moreover, different entities in the physical world have various dynamics, some of which are regular and predictable, while others are random and intangible. To enable the digital world to better reflect the changes of the physical world and return the feedback to guide the CAVs and ITSs in time, the regularity of the dynamic factors, such as the distribution of automobiles and pedestrians in different periods, should be studied. Once the regularity is available, the required contents can be predicted and cached at the right positions in advance, which can significantly improve the cache hit ratio (CHR). For the irregularly changing factors, their importance to metaverse services can be analyzed, and some less important factors should be neglected to simplify the complexity. Furthermore, the dynamics of CAVs degrade the quality of content transmissions due to the Doppler frequency shift and signal jitter [8], which should be considered in the caching policy design.
The increasing dynamics in the physical world result in a growing demand for frequent updates of cached contents, which requires the caching policy to respond to the changes in time. On the other hand, due to the limited storage and computing resources, the local servers usually cache the contents in a distributed and cooperative manner, which significantly increases the potential solution space. Thus, the convergence of the algorithms to select and cache the content is a great challenge to the latency of metaverse services. Many traditional strategies, including game theory and greedy search, may fail to meet the convergence requirements.
Since various kinds of sensors and cameras are adopted in the physical world, massive personal information, such as the daily route, home address, and vehicle number plate, is inevitably recorded. There is no doubt that such information can increase the similarity of the digital world to the physical world. However, users’ concern for privacy leakage should be considered when the content is cached in third-party edge servers or utilized to provide metaverse services for others. This is because private information may be collected and utilized for illegal or ungranted purposes. For instance, the daily routes of CAVs can be used by merchants to send out advertisements, totally violating the users’ original intentions to share their data for improved metaverse services. Users’ interests, which are usually utilized to infer content popularity, have been reported to be collected to intervene in a national election. Content security is another concern for edge caching in meta-CAVs and meta-ITSs because most edge servers have limited protection due to resource constraints. Adversaries can tamper with the content cached in the edge server to degrade the QoS of provided metaverse services. Some illegal users can also upload fake contents or even poisoned files to the edge servers to disrupt the operation or stop the service provision.
Generally speaking, the edge caching techniques for meta-CAVs and meta-ITSs have much higher requirements for content security and privacy due to the stringent QoS requirements, safety, and cooperative manner. On the other hand, when designing the protection policy, the tradeoff between service performance and security/privacy is considered for multifold reasons [9]. The solutions to provide strong security and privacy protection generate a great deal of computation and storage costs, which can result in the degraded performance of metaverse services. Another reason is that more details of the cached content can improve the performance of caching solutions while increasingly exposing privacy.
With the rapid refresh and update of popular contents, a caching strategy based on historical request data may not exactly capture the dynamic user preference. Thus, caching devices should not only focus on the past but also gather the new coming requests for accurate caching decision making, enabling the flexible cached content replacement. Moreover, with the enlargement of content size—for example, a VR video needs to be viewed from different angles [10]—a complete caching strategy may no longer be applicable for edge servers. The edge devices can choose to cache part of the content according to the available space and communication quality, while such a caching strategy increases the complexity and computational load. Besides, due to the mobility of CAVs and pedestrians, the caching strategy should allow multiple edge devices in the moving direction to cooperatively cache content to improve efficiency.
Furthermore, considering caching space waste and dynamic topology, the cached content can also be migrated to the servers at a higher level. For instance, contents cached in nearby CAVs can be moved to servers near the roadside unit (RSU) or even the macro base station (MBS). Fetching contents from the higher level servers undoubtedly causes extra transmission delay but can reserve caching space for contents with more stringent latency requirements. Fortunately, as shown in Figure 1, the networking technologies, including C-V2X and SAGIN with heterogeneous edge servers deployed hierarchically, enable content to be cached with more flexibility.
Another important caching approach to relieve the defect of changing network topology is to mine the trajectory regulation of meta-CAVs to predict where they may appear and precache the required content in advance. The increased prediction accuracy can significantly alleviate the latency for meta-CAVs to fetch the content. Moreover, the technique of big data enables the adoption of AI to analyze how the popularity of content changes. With the predicted popularity, the content providers can improve the CHR to satisfy more users. Furthermore, each metaverse application is usually concerned with different types of content, including audio, video, photos, and text. The caching policy design should intelligently meet the diversified requirements of different content for latency, throughput, packet loss rate, and transmission jitter, meaning that we need to consider resource allocation, remaining available time, transmission quality, and many other factors to enhance the caching rationality. Similar to content caching with dynamic sizes, the AI-based approaches to improve the caching policy design also need massive computation resources.
As shown in Figure 2, the meta-CAVs and meta-ITSs are expected to anticipate the future situation and provide driving assistance to avoid the occurrence of accidents. Such a service requires the CAVs and roadside infrastructure to fetch and process the data collected from heterogeneous devices, while the provided services should meet the requirements mentioned in the previous section. However, the edge caching policy design is challenged by heterogeneous hardware, diversified QoS requirements, various communication conditions, and ubiquitous dynamics. The main problems in edge caching can be categorized into what, where, when, and how to cache. Different kinds of AI techniques, including evolution AI, symbolic AI, and connectionist AI, have been studied to address these four issues. In this section, we discuss the related research on how AI can be adopted in edge caching, as shown in Table 1.
Figure 2 Using AI to optimize the caching policy. GCN: graph convolutional neural network; DDPG: deep deterministic policy gradient; DBSCAN: density-based spatial clustering of applications with noise; DQN: deep Q-network; RL: reinforcement learning.
Table 1 Recent works on AI-based edge caching.
An adaptable caching decision should cover the needs of the majority of vehicles along with some requests in an emergency. Therefore, the urgency and popularity of content are two main factors to decide the cached content, as shown in Figure 2. In the meta-CAVs and meta-ITSs, the transportation safety-related contents should have the top priority to be cached, and the latency threshold of such metaverse services should be given more attention. To measure and predict the request possibility of each piece of content, the authors of [4] utilize long short-term memory (LSTM) to predict the content-requesting frequency based on historical data. Then, they normalize the predicted frequency using Zipf distribution on the basis of the sorted index of each piece of content, with which the content popularity can be calculated. Moreover, the technique of big data enables us to collect more information to predict users’ interests for different content. To optimize QoS, the authors of [7] concentrate on seeking the optimal caching decision, making a tradeoff between transmission delay and CHR.
It has reached a consensus that contents can be cached not only in the cloud but also in the edge servers deployed on RSUs, vehicles, and even unmanned aerial vehicles (UAVs) in SAGIN, as shown in Figure 2. Each application of meta-CAVs and meta-ITSs consists of various types of content with diversified QoS requirements, as we mentioned previously. This content has to be cached in heterogeneous edge servers due to the constrained storage. Selecting an appropriate location for the optional content should consider the service requirements, content size, communication quality, user distribution, and mobility patterns of meta-CAVs. The authors of [11] propose a caching scheme that simultaneously makes decisions for CAVs and RSUs to optimize the total system latency considering different QoS requirements. Furthermore, the authors of [12] propose a cooperative caching strategy with UAVs and MBSs to relieve the transmission pressure caused by surging content requests.
Precaching certain content at a suitable time interval can elevate the transmission efficiency and QoS considering CAV mobility and resource occupation. However, the time interval to cache the content should be carefully analyzed due to the dynamics of metaverse services, CAVs, users’ preferences, and environment issues. Caching the content too early means a waste of storage. Since AI is efficient in analyzing the temporal changes of vehicle requests and user preference, utilizing AI to decide when to cache has aroused researchers’ attention. The authors of [13] propose a coverage-aware LSTM model to predict not only the future CAVs’ routes but also the changing overhead of each edge device, which can be used to guide the caching policy design. The network composed of different RSUs and their serving vehicles can be organized as a graph; thus, the graph neural network (GNN) and graph convolutional neural network (GCN) are suitable for learning and extracting the relationship among different nodes. The authors of [14] propose a traffic flow prediction scheme based on GNN to efficiently extract the spatiotemporal features of traffic. With the increased awareness of network traffic and content popularity, the time to cache the content for UAVs, as well as the navigated trajectory, can be optimized [8].
Partitioning the content before caching can increase flexibility and address the issue that contents of a large size cannot be completely transferred before the connections between the moving vehicles and roadside infrastructure break. How to partition the content affects the following caching policy design since the content size is a key factor. The authors of [15] partition the content into segments and use the Deep Deterministic Policy Gradient (DDPG) for flexible and capacity-saving caching decision making. Moreover, the communication quality should be analyzed when designing the caching policy. For instance, the authors of [7] optimize the bandwidth allocation during caching policy design, which can further improve the QoS.
From another point of view, the public now attaches more attention to the protection of their privacy. Privacy leakage caused by request forwarding and redirecting may lead to unexpected advertisements being pushed. As shown in Figure 2, FL and blockchain have been widely recognized as two important techniques to protect security and privacy in distributed systems. Specifically, FL enables models deployed in different edge servers to share the parameters of local AI models instead of their privacy-related datasets. To solve the failure in fast convergence, light and pretrained models are deployed in edge servers to extract the domestic spatiotemporal features and accelerate the training process, while the assembled model is deployed in the cloud to deal with complicated tasks and unexpected inputs. The security of shared parameters of distributed models can be further protected by the blockchain technique. Specifically, the update and exchange of parameters will be recorded in each distributed ledger using consensus protocols, enabling a tamper-proof and transparent FL training process with guaranteed reliability and security [6].
Utilizing MAFRL, we have conducted a case study on edge caching, aimed at ensuring that each transmission task can be accomplished within its delay threshold. Figure 3 shows that meta-CAVs have the capability to foresee the traffic load and potential dangers through collaborative perception to enhance the experience of meta-CAVs. Obviously, the content generated by the aforementioned metaverse service is diverse in size and urgency level. For instance, the information on potential dangers should be cached and transmitted to meta-CAVs within a stringent delay threshold regardless of its low popularity, while the metaverse traffic load information and multimedia contents of large size have milder delay requirements, for which the caching decision can concentrate more on their popularity instead of urgency.
Figure 3 The considered vehicular network with metaverse services.
The considered scenario consists of several RSUs and a centralized MBS distributed in the urban area. Each vehicle can connect only one MBS at every time slot. We consider the covered area of each RSU as an individual function district with independent content popularity following the Zipf distribution. Each piece of considered content has its urgency level to represent the delay requirement. Also, the size of each piece of content is assumed large and not valuable enough to be completely cached. Thus, we divide the content into blocks with the same popularity and urgency level. The bandwidth and transmission power of RSUs and MBS are 15 MHz, 5 MHz, 38 dBm, and 30 dBm, respectively. The noise power is of 10−11 mw. The sizes of three kinds of content are 150, 250, and 350 MB with delay requirements of 15, 30, and 40 s, respectively. Also, the parameters of the Poisson process and Zipf distribution are both one.
In view of the spatiotemporal preference difference in each area and the concerns for privacy leakage, we first adopt the FL, which locally trains the LSTM models to predict content popularity using local data. We further adopt the MAFRL to relieve the computing complexity of the centralized reinforcement learning strategy. The states of each agent consist of four factors: the number of serving vehicles, the requested blocks of the vehicles and their popularity, the rest time, and the number of received blocks. The action space is the binary vector indicating the caching decision. The reward is defined according to the rest time until the delay threshold. Once a transmission task fails, a punishment value is added to the reward function. Thus, our model can balance the delay requirement and content popularity according to the spectrum occupation, content popularity, and delay requirement.
We consider two conventional caching strategies, FIFO and LIFO, as the benchmarks. The simulation results in terms of CHR and successful transmission ratio are given in Figures 4 and 5, respectively. Specifically, Figure 4 shows that the CHR of our proposed MAFRL can reach 0.60 on average, which is much higher than traditional caching strategies. The explanation of why the CHR cannot reach a higher level is that agents in MAFRL prefer to cache the content with stringent urgency or limited rest time. The successful transmission ratio shown in Figure 5 also provides potent proof for the explanation mentioned previously. In Figure 5, when the vehicle number is 14, the successful transmission ratio of MAFRL is more than 0.85, while those of the other two strategies are nearly 0.65 and 0.60, respectively. The results show that AI-based content caching can significantly improve performance. Moreover, the performance degrades with the increase in vehicle numbers due to the limited storage of edge servers.
Figure 4 A comparison of the CHR.
Figure 5 A comparison of a successful transmission ratio.
As the latency that affects the performance of metaverse services results from content transmissions and processing, sound caching solutions need to consider diversified communication conditions with increasing computation and communication resources from heterogeneous devices. Thus, a more intelligent resource allocation strategy to jointly optimize caching, delivery, and computing needs to be proposed to meet the QoS requirements of meta-CAV services.
Unlike existing research, which mainly considers one type of content, each metaverse application usually consists of diversified contents with different QoS requirements. Using the heterogeneous infrastructure to collaboratively cache different contents can improve flexibility and is beneficial to contents integration during the creation of the digital world. The collaborative caching strategy should be designed by considering the content selection, placement, update, and even the partition scheme.
Security and privacy will be critical concerns in metaverse applications. Since safety is a public issue for all the participants in ITSs, how to protect security and privacy without affecting safety is a challenging issue for meta-CAVs and meta-ITSs. Blockchain has been recognized as a key technology for the metaverse. However, how to improve the efficiency and alleviate the computation burden of blockchain has been a common issue. In meta-CAVs and meta-ITSs, the dynamics can increase the oscillation of blockchain participants. Thus, the research on blockchain-enhanced intelligent caching needs more endeavors.
Distributed AI techniques will be increasingly utilized in edge caching for meta-CAVs and meta-ITSs due to the network scale and privacy concerns. Similar to the blockchain technique, the convergence of distributed AI-based edge caching algorithms is also challenged by network dynamics. However, the metaverse applications of CAVs and ITSs pose stringent latency requirements for content caching solutions. Thus, how to accelerate the distributed AI remains to be studied.
In this article, we comprehensively introduce edge caching for meta-CAVs and meta-ITSs. We discuss the emerging requirements and challenges after analyzing the characteristics of meta-CAVs and meta-ITSs. We can find that caching performance optimization is extremely complex and that AI techniques have been considered to address this problem. Then, the related research on AI-based edge caching has been explained. Moreover, a case study is provided to evaluate the advantages of AI in improving the caching performance of meta-CAVs and meta-ITSs. Finally, some perspective directions are discussed.
Jiajia Liu is the corresponding author.
Bomin Mao (maobomin@nwpu.edu.cn) is currently a full professor at the School of Cybersecurity, Northwestern Polytechnical University, 710072 Xi’an, China, and is with the Yangtze River Delta Research Institute, Northwestern Polytechnical University, Xi’an 710072, China. His research interests include satellite networks, satellite Internet of Things, vehicular networks, and edge computing. He received several Best Paper Awards from IEEE conferences, such as Globecom, ICC, and IC-NIDC. He is a Member of IEEE.
Yangbo Liu (liuyangbo@nwpu.edu.cn) is currently pursuing the M.S. degree at the School of Cybersecurity, Northwestern Polytechnical University, Xi’an 710072, China. His research interests include the Internet of Vehicles, space-air-ground integrated networks, and cybersecurity. He received the B.S. degree in information management and information systems from Shandong University of Science and Technology, Qingdao, China, in 2022.
Jiajia Liu (liujiajia@nwpu.edu.cn) is a full professor (vice dean) at the School of Cybersecurity, Northwestern Polytechnical University, Xi’an 710072, China. His research interests include intelligent and connected vehicles, mobile/edge/cloud computing and storage, Internet of Things security, wireless and mobile ad hoc networks, and space-air-ground integrated networks. He is the vice chair of the IEEE IOT-AHSN TC and is a Distinguished Lecturer of the IEEE ComSoc and VTS. He is a Senior Member of IEEE.
Nei Kato (kato@it.is.tohoku.ac.jp) is a full professor at and the dean of the Graduate School of Information Sciences, Tohoku University, Sendai 9808579, Japan. His research interests include computer networking, wireless mobile communications, satellite communications, ad hoc and sensor and mesh networks, unmanned aerial vehicle networks, smart grid, artificial intelligence, the Internet of Things, big data, and pattern recognition. He is a Fellow of IEEE, the Engineering Academy of Japan, and the IEICE.
[1] F. Tang, X. Chen, M. Zhao, and N. Kato, “The roadmap of communication and networking in 6G for the metaverse,” IEEE Wireless Commun., vol. 30, no. 4, pp. 72–81, Aug. 2023, doi: 10.1109/MWC.019.2100721.
[2] G. Thandavarayan, M. Sepulcre, and J. Gozalvez, “Generation of cooperative perception messages for connected and automated vehicles,” IEEE Trans. Veh. Technol., vol. 69, no. 12, pp. 16,336–16,341, Dec. 2020, doi: 10.1109/TVT.2020.3036165.
[3] T. Q. Duong, D. Van Huynh, S. R. Khosravirad, V. Sharma, O. A. Dobre, and H. Shin, “From digital twin to metaverse: The role of 6G ultra-reliable and low-latency communications with multi-tier computing,” IEEE Wireless Commun., vol. 30, no. 3, pp. 140–146, Jun. 2023, doi: 10.1109/MWC.014.2200371.
[4] R. Wang, Z. Kan, Y. Cui, D. Wu, and Y. Zhen, “Cooperative caching strategy with content request prediction in Internet of Vehicles,” IEEE Internet Things J., vol. 8, no. 11, pp. 8964–8975, Jun. 2021, doi: 10.1109/JIOT.2021.3056084.
[5] H. Guo and J. Liu, “Collaborative computation offloading for multiaccess edge computing over fiber–wireless networks,” IEEE Trans. Veh. Technol., vol. 67, no. 5, pp. 4514–4526, May 2018, doi: 10.1109/TVT.2018.2790421.
[6] J. Kang et al., “Blockchain-based federated learning for industrial metaverses: Incentive scheme with optimal AoI,” in Proc. IEEE Int. Conf. Blockchain (Blockchain), Espoo, Finland, 2022, pp. 71–78, doi: 10.1109/Blockchain55522.2022.00020.
[7] G. Qiao, S. Leng, S. Maharjan, Y. Zhang, and N. Ansari, “Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks,” IEEE Internet Things J., vol. 7, no. 1, pp. 247–257, Jan. 2020, doi: 10.1109/JIOT.2019.2945640.
[8] Z. Liu, C. Zhan, Y. Cui, C. Wu, and H. Hu, “Robust edge computing in UAV systems via scalable computing and cooperative computing,” IEEE Wireless Commun., vol. 28, no. 5, pp. 36–42, Oct. 2021, doi: 10.1109/MWC.121.2100041.
[9] Z. M. Fadlullah, B. Mao, and N. Kato, “Balancing QoS and security in the edge: Existing practices, challenges, and 6G opportunities with machine learning,” IEEE Commun. Surveys Tuts., vol. 24, no. 4, pp. 2419–2448, 4th Quart. 2022, doi: 10.1109/COMST.2022.3191697.
[10] Z. Liu et al., “Point cloud video streaming: Challenges and solutions,” IEEE Netw., vol. 35, no. 5, pp. 202–209, Sep./Oct. 2021, doi: 10.1109/MNET.101.2000364.
[11] T. Ma, X. Chen, Z. Ma, and Y. Chen, “Deep reinforcement learning for pre-caching and task allocation in Internet of Vehicles,” in Proc. IEEE Int. Conf. Smart Internet Things (SmartIoT), Beijing, China, 2020, pp. 79–85, doi: 10.1109/SmartIoT49966.2020.00021.
[12] S. Araf, A. S. Saha, S. H. Kazi, N. H. Tran, and M. G. R. Alam, “UAV assisted cooperative caching on network edge using multi-agent actor-critic reinforcement learning,” IEEE Trans. Veh. Technol., vol. 72, no. 2, pp. 2322–2337, Feb. 2023, doi: 10.1109/TVT.2022.3209079.
[13] J. Chen et al., “Multi-agent deep reinforcement learning for dynamic avatar migration in AIoT-enabled vehicular metaverses with trajectory prediction,” IEEE Internet Things J., early access, 2023, doi: 10.1109/JIOT.2023.3296075.
[14] Z. M. Fadlullah, B. Mao, F. Tang, and N. Kato, “Value iteration architecture based deep learning for intelligent routing exploiting heterogeneous computing platforms,” IEEE Trans. Comput., vol. 68, no. 6, pp. 939–950, Jun. 2019, doi: 10.1109/TC.2018.2874483.
[15] Z. Zhang and M. Tao, “Deep learning for wireless coded caching with unknown and time-variant content popularity,” IEEE Trans. Wireless Commun., vol. 20, no. 2, pp. 1152–1163, Feb. 2021, doi: 10.1109/TWC.2020.3030973.
Digital Object Identifier 10.1109/MVT.2023.3327514