©SHUTTERSTOCK.COM/RAULALMU
Xiang-Rui Huang, Liang-Bi Chen
©SHUTTERSTOCK.COM/220 SELFMADE STUDIO
With the advancement of medical care and technology, human life expectancy is increasing, many advanced countries have aging societies, and the elderly have increasing needs for society to address; these have become some of the main problems to be solved in modern society (Chang et al., 2021). To solve these problems, long-term care institutions have become one of the solutions. In addition, dementia is the most important social problem among elderly individuals (Tsang et al., 2020). In most Asian countries, the solution to the problem of dementia in the elderly is to establish long-term care institutions, such as villages for elderly individuals and nursing homes. However, there is still the problem of companionship in an aging society. The needs of the elderly lie in companionship. Through family companionship, the elderly can share the joys of daily life with their families.
In Asian countries, the cost of manual care for most elderly individuals is caused by dementia, so they are cared for in nursing homes, resulting in a long-term lonely life. How can we effectively solve this problem to improve the institutions, systems, and methods of long-term care institutions for elderly individuals? At present, the solution to the problem of companionship in an aging society is to rely on technological products (such as computers and mobile phones) as tools to facilitate contact with distant family members, but the elderly may have difficulty using technological products due to their declining cognitive skills. In addition, the familiar display technology of such products on the market today requires facility with the hardware for visualization through a screen. Although technological products can solve the problem of companionship in an aging society, their effectiveness is limited.
The issue of aging has significantly impacted family structures and relationships, especially in Asian societies. In the past, the family was the main source of care and support for elderly individuals. However, with urbanization and modernization, the family structure has become more nuclear, and traditional families are facing tremendous pressure to care for elderly individuals. Therefore, today’s society needs to provide more social support and care services for the elderly to reduce the burden on the family; pay attention to caregivers’ needs and pressures; and provide comprehensive care services for older people, including daycare centers and community care. Moreover, integrating medical resources can effectively support families and ensure that older people receive appropriate care. In addition, aging also involves issues of social participation and mental health. Many older adults feel lonely and socially isolated, negatively impacting their mental health. How can we encourage older adults to be socially active and socially interactive to promote their spiritual and emotional well-being as well as provide cultural, recreational, and educational activities to enhance their self-esteem and life satisfaction?
Immersive technologies offer new avenues to address aging issues and can provide older adults with rich virtual experiences and opportunities for interaction to compensate for their real-life limitations. In the home life of elderly individuals, immersive technology can provide various supports, such as virtual social platforms, that allow the elderly to make video calls with others; participate in virtual communities and games to reduce their loneliness and social isolation; and offer virtual tour experiences, cultural exhibits, and educational events that allow them to enjoy a variety of enriching experiences at home, expanding their social engagement and entertainment choices.
Furthermore, immersive technology can also provide cognitive training and memory improvement applications, which can actively promote the brain activity and thinking flexibility of elderly individuals. Through VR scenarios, elderly individuals can participate in various cognitive challenges and intellectual games, such as memory games, puzzle solving, and problem solving, to stimulate the cognitive functions of the brain, improving memory, concentration, and reflexes.
Therefore, immersive technology has great potential in the physical health and rehabilitation of elderly individuals. Older people can perform sports and rehabilitation training through VR technology, enhancing the body’s function and balance. Immersive technology can also provide a variety of cognitive training and memory improvement applications to help elderly individuals maintain brain activity and thinking flexibility as well as promote their physical health and mental ability.
As shown in Figure 1, when we introduce immersive technologies, such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), for the social problems of aging, it can increase the sense of interactive experience and make the service of long-term care for the elderly more comprehensive and complete. Immersive technology uses computer vision (Cwierz et al., 2021), somatosensory touch (Ahmed et al., 2023), auditory perception (Moraes et al., 2021), and olfactory perception (Sexton et al.,) to create a virtual–real environment and improve human beings’ sensory experiences, communication, and interactions in the online virtual world. Today’s immersive technology is the main technology development direction. Immersive technology can lead users to visit various virtual worlds. AR, VR, and MR are immersive technologies based on a virtual 3D space created by virtual technology in real space. Immersive technology has accomplished many complex computer vision tasks (Melo et al., 2022).
Fig 1 Immersive technology introduces long-term care services for elderly individuals. AR: augmented reality; MR: mixed reality; VR: virtual reality.
In immersive technology, the potential of AR, VR, and MR needs to overcome different key technologies. For example, the main challenges of AR are the digitization and 3D-ization of displayed information as well as the matching of the back-end display and 3D space. VR challenges include content design and creation, VR 360 video processing, and display technology. MR mainly needs to overcome high-precision body recognition technology, low-latency interactive processing and transmission technology, and virtual somatosensory display technology.
Today, AR and VR immersive technologies have solved many problems in practical applications in aging societies. We list some common practical applications of AR and VR immersive technology for elderly individuals:
VR consists of three elements: 3D space, real-time interaction, and self-projection. VR is a technology that uses VR wearable devices to build a virtual world. Through computer vision simulation technology, users are immersed in a virtual environment created by a computer. In terms of the hardware requirements of the overall system, VR glasses or other controllers are needed. VR glasses are a kind of head-mounted display device. There are two dimensions of the spectacle lenses: the interpupillary distance (IPD) is the distance between the centers of the pupils of both eyes, and the lens spacing is the distance between the clearest points in the two lenses in the head-mounted device. The IPD and lens spacing are consistent to achieve the highest image clarity. In terms of user space requirements, more than 6 m2 are generally required for a sitting interaction; more than 13 m2 are mainly required for a walking interaction. In addition, motion-to-photon (MTP) latency is required in VR technology to optimize the user experience. MTP latency refers to the delay in time from when a user performs an action to when it is displayed on the screen. The shorter the delay time, the better the user experience. If the delay exceeds 20 ms, there will be an obvious sense of delay.
VR requires the use of VR wearable glasses equipment, and the physiological functions of the user must be considered—for example, VR may not be appropriate for patients with high blood pressure, heart disease, epilepsy, eye diseases, or limb disabilities. A user may have symptoms of simulator sickness when using VR; that is, when the user himself is not moving but the virtual world character is moving, the state of balance is disrupted by the imbalance of the movement seen by the human eye and the vestibular system feel. Symptoms such as fainting, nausea, and loss of appetite appear when physical activity is not in line with visual perception (Lee et al., 2022). In addition, young children should not use VR because the VR device screen is close to the eyes, and, although it will not cause direct damage to the eyes, young children cannot control the use time and screen brightness, causing too much light (blue light) to enter the eyes.
Generally, AR uses an electronic wearable device (such as glasses) as a medium. The AR system tracks and locates the position and direction of the user’s head and digitally supplements the physical environment with virtualized objects and information (Adams et al., 2022). By adding the technology of real space to the virtual situation, users can escape the limitations of space and time through the virtual world (images and texts are added to the pictures generated by users observing the surrounding environment) and experience the atmosphere of different time and space backgrounds.
Compared with VR, the hardware conditions required for AR are simpler, and most of them only need one device, such as a smartphone, a tablet computer, AR glasses, or a camera. The operation process is also simpler than that of VR. First, start the AR software in the device and turn on the lens of the AR device. The device will automatically display AR effects or information, and users can perform virtual situations in specific scenes in the real world to interact. AR technology creates a combination of virtuality and reality through displays, such as head-mounted displays, handheld displays, and the most common fixed-handheld displays, which are worn in front of the eyes and project images on a small screen. The principle of AR displays can be divided into an optical perspective and a video perspective.
The optical perspective uses the beam splitter method to position the mirror position of the user and project the computer monitor image light and the surrounding ambient light into the user’s line of sight. In the past, the beam splitter method was most commonly used on fighter jet pilots’ heads-up displays, where instrument data were projected onto the heads-up display for the pilot to monitor real-time flight data. However, the image in the current optical see-through display is transparent and cannot completely shield the object behind it. Therefore, under a complex background, the ambient light and the image light in the display are likely to cause visual confusion. In addition, according to the user’s visual experience, the user cannot focus on two repetitive objects in front of him at the same time, and visual confusion cannot achieve the best VR experience. Therefore, using optical perspective methods to project objects must be viewed as simple and with a single background.
The video perspective is a video mixing technology that transmits images from head-mounted cameras and computer special effects, allowing users to directly see real images. There are two main methods of video mixing technology from a video perspective: the background offset method and the channel control pixel method. The background offset method is used to perform background offset between the image returned by the camera and the mixed image. The mixed image will cover the real objects and background behind it, confusing the user’s visual senses. The channel control pixel method uses the channel control pixel ratio, and different channels store the information of each pixel to synthesize a translucent image. The video perspective can completely cover the physical objects, the images of the virtual and physical objects are displayed on the screen, and there is no problem of user focus difference; however, it lacks realism, and the visual quality is poor.
MR is a hybrid of VR and AR (Gu et al., 2023) that blurs the boundary between the real world and the virtual world by generating an illusory simulation space through VR and AR. MR is also paired with a head-mounted device, but the user sees the real environment, and additional virtual objects are stacked. MR technology mainly focuses on the following technologies: high-precision body recognition technology, low-latency transmission technology, and virtual somatosensory display technology.
At present, body recognition technology can be divided into large-scale recognition (the head, fingers, and feet) and small-scale recognition (the head, fingers, and eyes) technology. Among them, there are challenges in small areas. How can recognition accuracy be improved through relevant algorithms and accurate body recognition technology with virtual information technology? In addition to accuracy, the immediacy required for virtual–real interaction is also relatively required to achieve virtual–real interaction without an inductive delay without causing the human body to feel abrupt. In addition, virtual somatosensory display technology is constantly breaking through. In addition to vision and hearing, which are the most mature technologies, touch, taste, and smell are the main challenges of virtual somatosensory display technology. Based on the current technological development, visual virtual somatosensory display technology is widely used in various simulation experiences.
Because broadband Wi-Fi technology has been widely implemented in most terminal devices, it can transmit data with a large number of users. The evolution goals of immersive technology in terms of transmission technology are divided into broadband wireless technology and multicast and broadcast technology. In terms of broadband wireless technology, the current development of broadband Wi-Fi technology, IEEE 802.11ac, uses 5 GHz as the frequency band for transmission signals, which can effectively improve multipath attenuation and other problems and improve channel utilization. The maximum transmission rate of IEEE 802.11ac can reach approximately 867 Mb/s. For immersive technology applications, IEEE 802.11ac is more likely to be used in public spaces, while IEEE 802.11ad is more suitable for local short-range wireless connections. In addition, considering the needs of a large number of users, multicast and broadcast technologies need to be used. Compared with unicast techniques, multicast and broadcast technology saves bandwidth waste and meets immersive technology applications in public venues. Today, IEEE 802.11ac already supports multicasting and broadcasting technology, along with whether to maintain the highest transmission rate without causing interference and congestion during multicasting and broadcasting.
The immersive technology of over-the-top (OTT) services brings more convenient, personalized, and rich entertainment and information experiences to older people. Older people can watch movies, TV shows, documentaries, etc. through the OTT platform and choose according to their interests and preferences. They can also provide customized content recommendations based on personal needs and preferences for a better user experience. Older people can watch various types and styles of films and television programs through the OTT platform to broaden their horizons. At the same time, they can also learn and improve their knowledge and skills by watching educational resources.
OTT media service is an online streaming video service provided directly to users through the Internet. The applications of OTT services include medical care, engineering, military, education, video games, and live events. Immersive technology brings many new technical challenges to OTT services, including the network bandwidth, codec technology, and user interface. In terms of the network bandwidth, if the VR image needs to achieve full high-definition image quality, it needs to transmit larger resolution and larger network bandwidth to transmit image information. At present, 4K VR panoramic video images have appeared. How to effectively expand the bandwidth and utilize the limited bandwidth to provide better picture quality is one of the challenges. In terms of encoding and decoding technology, since VR panoramic video requires better picture quality, how to properly apply computing resources to obtain a certain picture quality and resolution with encoding and decoding technology will be the main difficult challenge. In terms of the user interface, at present, most terminal devices for VR panoramic video are presented in head-mounted form. While VR panoramic video users constantly change their posture, the terminal device will continuously provide images corresponding to the posture. In addition, the user is immersed in the virtual world during the experience, and the traditional human–machine interface must be modified.
OTT services rely on high-bandwidth and low-latency Internet, especially for AR, VR, and MR, which emphasize high-resolution image transmission as well as real-time identification processing and require extremely high bandwidth and latency. Relying on the Internet, the network transmission delay from endpoint to endpoint is generally greater than 500 ms. For the network delay problem, the edge computing method can solve the delay problem, and the service and computing are built on the edge side to serve (Liu et al., 2020). Instead of putting computing services in the cloud or a centralized data center, edge computing only needs near-end network transmission, the delay is fewer than 5 ms, and even near-real-time transmission at the microsecond level can be achieved. The database of the near-end service has regional characteristics, so the database is small, which can shorten the query and calculation time to allocate the calculation of the terminal device, save the power consumption of the mobile device, and reduce the cost of the terminal device.
Recent years have seen the emerging development of the concept of metaverses (Truong et al., 2023), which are visual-image-rich virtual spaces where people can work, play, shop, socialize, etc. Users can imagine that they truly exist in a virtual world, can connect with other people, and can also have a trading system with which to conduct digital asset transactions. The rise of metaverse technology has brought brand-new learning, entertainment, and social platforms for older people. Older people can participate in virtual meetings, courses, and community activities through the metaverse, interacting with, communicating with, and demonstrating their talents and creativity to people worldwide. The platform not only expands the social circles of older people but also enhances their self-esteem and self-confidence and promotes their spiritual and emotional well-being. Metaverses are an extension of AR, VR, and MR in which the concept of virtual world interconnection is combined with decentralization. Decentralized interconnection can achieve instant end-to-end interactivity and is more secure in terms of privacy. We introduce the metaverse properties as follows:
Immersive technologies come with privacy and security concerns, and data privacy is an important one. Immersive technologies often need to collect users’ data to provide a more personalized and realistic experience. However, these data may contain sensitive information, such as personally identifiable information, behavior habits, and geographic location. Protecting users’ data privacy has become a crucial task, and improving regulatory agencies and legal systems is also an important part of ensuring users’ data privacy and security. Technology developers and related organizations should formulate strict privacy policies and clearly inform users of the purpose and scope of data collection; data processing and storage should comply with relevant regulations, such as compliance with personal data protection laws and privacy regulations. Users should expressly agree to the collection and use of data and have the right to choose whether to participate.
Immersive technology is an immersive experience that simulates the real world or creates a virtual environment in which to immerse users. It can provide rich sensory stimulation and change the way people interact with information and the environment. It has a wide range of applications in several fields, including entertainment, education, and medical care. Immersive technology has the potential to have a significant impact on aging societies and can play a key role in improving the psychological and emotional well-being of older adults. In virtual worlds, older adults can be brought into different environments, allowing them to relive past memories, engage in nostalgic activities, or visit places they cannot experience in person, helping combat loneliness, depression, and isolation as well as promoting social connection and emotional satisfaction.
Furthermore, immersive technology plays an important role in education and lifelong learning. Immersion technology can create immersive learning experiences that allow seniors to explore new subjects, visit historic sites, and interact with educational content in a visually engaging way that promotes intellectual stimulation, reduces cognitive decline, and fosters curiosity and awareness of personal growth.
In future work, we must pay attention to the challenges and problems that may be faced when adopting immersive technologies in an aging society. The availability of these technologies should be prioritized to ensure that older adults can easily operate and use them. In addition, cost, privacy issues, and customized digital content for different physical abilities must be considered important factors.
This study is supported in part by the National Science and Technology Council (NSTC), Taiwan, under the following grant: NSTC 111–2622-E-346-001. This study is also supported in part by the Higher Education Sprout Project, Ministry of Education, Taiwan, under the following grant: MOE 112G0004-1.
• W.-J. Chang, C.-H. Hsu, and L.-B. Chen, “A pose estimation-based fall detection methodology using artificial intelligence edge computing,†IEEE Access, vol. 9, pp. 129,965–129,976, Sep. 2021, doi: 10.1109/ACCESS.2021.3113824.
• G. Tsang, X. Xie, and S.-M. Zhou, “Harnessing the power of machine learning in dementia informatics research: Issues, opportunities, and challenges,†IEEE Rev. Biomed. Eng., vol. 13, pp. 113–129, 2020, doi: 10.1109/RBME.2019.2904488.
• H. Cwierz, F. DÃaz-Barrancas, J. G. Llinás, and P. J. Pardo, “On the validity of virtual reality applications for professional use: A case study on color vision research and diagnosis,†IEEE Access, vol. 9, pp. 138,215–138,224, Oct. 2021, doi: 10.1109/ACCESS.2021.3118438.
• I. Ahmed, V. J. Harjunen, G. Jacucci, N. Ravaja, T. Ruotsalo, and M. M. Spapé, “Touching virtual humans: Haptic responses reveal the emotional impact of affective agents,†IEEE Trans. Affective Comput., vol. 14, no. 1, pp. 331–342, Jan./Mar. 2023, doi: 10.1109/TAFFC.2020.3038137.
• N. Moraes, R. Flynn, A. Hines, and N. Murray, “The role of physiological responses in a VR-based sound localization task,†IEEE Access, vol. 9, pp. 122,082–122,091, Aug. 2021, doi: 10.1109/ACCESS.2021.3108446.
• J. P. Sexton, A. A. Simiscuka, K. Mcguinness, and G.-M. Muntean, “Automatic CNN-based enhancement of 360° video experience with multisensorial effects,†IEEE Access, vol. 9, pp. 133,156–133,169, Sep. 2021, doi: 10.1109/ACCESS.2021.3115701.
• M. Melo, G. Gonçalves, P. Monteiro, H. Coelho, J. Vasconcelos-Raposo, and M. Bessa, “Do Multisensory stimuli benefit the virtual reality experience? A systematic review,†IEEE Trans. Vis. Comput. Graphics, vol. 28, no. 2, pp. 1428–1442, Feb. 2022, doi: 10.1109/TVCG.2020.3010088.
• S. Lee, S. Kim, H. G. Kim, and Y. M. Ro, “Assessing individual VR sickness through deep feature fusion of VR video and physiological response,†IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 5, pp. 2895–2907, May 2022, doi: 10.1109/TCSVT.2021.3103544.
• H. Adams, J. Stefanucci, S. Creem-Regehr, G. Pointon, W. Thompson, and B. Bodenheimer, “Shedding light on cast shadows: An investigation of perceived ground contact in AR and VR,†IEEE Trans. Vis. Comput. Graph., vol. 28, no. 12, pp. 4624–4639, Dec. 2022, doi: 10.1109/TVCG.2021.3097978.
• P. Gu, X. Xu, X. Qian, and T.-H. Weng, “Leveraging extended reality for autistic individuals: A scoping review of technical features and technology affordances,†IEEE Trans. Learn. Technol., vol. 16, no. 1, pp. 133–149, Feb. 2023, doi: 10.1109/TLT.2022.3197430.
• Y. Liu, M. Peng, G. Shou, Y. Chen, and S. Chen, “Toward edge intelligence: Multiaccess edge computing for 5G and Internet of things,†IEEE Internet Things J., vol. 7, no. 8, pp. 6722–6747, Aug. 2020, doi: 10.1109/JIOT.2020.3004500.
• V. T. Truong, L. Le, and D. Niyato, “Blockchain meets metaverse and digital asset management: A comprehensive survey,†IEEE Access, vol. 11, pp. 26,258–26,288, Mar. 2023, doi: 10.1109/ACCESS.2023.3257029.
Xiang-Rui Huang (cren30475@gmail.com) is currently pursuing his B.S. degree in computer science and information engineering at National Penghu University of Science and Technology, 880011 Penghu, Taiwan. His research interests include deep learning, smart aquaculture, fish feeding systems design, robots, artificial intelligence of things (AIoT) systems design, and computer vision for the IoT.
Liang-Bi Chen (liangbi.chen@gmail.com) earned his Ph.D. degree in electronic engineering from Southern Taiwan University of Science and Technology, Tainan, Taiwan. He is an assistant professor in the Department of Computer Science and Information Engineering at National Penghu University of Science and Technology, 880011 Penghu, Taiwan. He serves as an associate editor for IEEE Access and as a chair of the Internet of Things Technical Committee of the IEEE Consumer Technology Society. He is a Senior Member of IEEE.
Digital Object Identifier 10.1109/MPOT.2023.3298141
