AI-Driven Companion in XR

Artificial Intelligence (AI) avatars can simulate human-like interactions, providing personalized experiences in areas such as mental health support and social companionship. The development of AI avatars for companionship has rapidly advanced in recent years, combining natural language processing (NLP), computer vision, and affective computing. These avatars are becoming more capable of engaging in lively, empathetic conversations by recognizing and responding to emotional cues through text and voice. Nowadays, AI avatars primarily facilitate user interaction through screen-based devices such as monitors and smartphones. The visualization of AI avatars depends on the photorealism presented on the display to persuade users of the existence of the companion. Users will not perceive the co-presence, which is the user’s sense of being physically co-located with the companion, due to the lack of the perceptual realism explored specifically in research on extended reality (XR).

Accordingly, leveraging visual experience and spatial distance to influence users’ sense of co-presence with an AI companion can facilitate the development of companionship that aims to enhance sensory experience. XR is thus considered technologies enabling the AI avatar to appear in users’ physical environment and interact with physical objects. This session explores the potential of integrating augmented reality (AR) technologies to visualize AI avatars as interactive companions. Using augmented imaging, users can see virtual companions as if they are part of the real environment when viewing through AR glasses. This approach transcends the constraints of screen-based interactions, bringing AI engagement into real-world settings.