In this work, I introduce a new type of virtual Augmented Reality companion referred to as "sidekick." This name is chosen to emphasize the unique close relationship between the user and a customized character that is neither a direct representation of its creator nor an autonomous agent. Its primary role is to provide emotional support and induce a social buffering effect that mitigates stress responses in the presence of a social partner. I propose a framework to design such emotion-aware AR companion.
Emotional well-being is essential for mental and physical health, and can be improved through strategies like emotion recognition, stress management, and better communication.¹ The “sidekick” trope — often a pet-like companion supporting the main character — has inspired studies on virtual pets in VR and AR, which show benefits for emotional support and social interaction. With AR technologies like XREAL and Apple Vision becoming mainstream, interest is growing in empathetic Mixed Reality agents. This trend highlights the need for positive computing and emotionally aware HCI frameworks that support psychological well-being.
The “sidekick” design process began with an ideation workshop, where 11 participants created companion concepts — most sharing traits like cuteness or pet-like features. Two AR prototypes followed. The first, a 3D plush dog named "Cheesecake," was tested at Keio and Stanford, and while users appreciated its charm, they wanted more personalization and interactivity.
The second prototype, presented at Miraikan in Tokyo, let families customize characters on an iPad and choose emotions via a mobile app. Though users enjoyed the experience, limited expressions and no sound made emotional cues harder to read. They expressed interest in characters that could better reflect and support their emotions.
Using this feedback, a final prototype was created in Blender — a chubby, customizable character animated with seven emotions and a greeting. Users added sound effects for a personal touch and viewed the animations through Unity on XREAL AR glasses.
In follow-up experiments, participants used their custom “sidekicks” during stressful tasks. Biosignal data (skin conductance and heart rate), surveys, and interviews showed reduced stress and improved emotional regulation. Users felt more understood and comforted by their sidekick, with negative emotions feeling less intense and positive ones more uplifting.
Designing AR companions involves spatial computing, character design, and HCI principles. Three key areas include:
Custom Design – Let users create 3D characters to evoke comfort and emotional connection through choices like shape, color, voice, and name.
Technical Implementation – AR enhances presence by placing virtual companions in real-world spaces. Future AI integration can personalize behavior, streamline creation, and enable emotion recognition.
Engaging Interaction – Add adaptive features so companions learn from users over time, either mirroring emotions or offering support based on individual needs.
This study provides a framework for designing an empathetic AR companion for emotional support and lowering stress levels. It was shown that “sidekick” is perceived as something in-between self and external being, and users form a close relationship with their creation.
Participants reported lower task load while using “sidekick” during a stressful situation. Based on the self-report emotion scale, participants reported negative feelings as less upsetting and less intense, but positive feelings were more uplifting and more intense, which suggests improved emotion regulation and awareness.
There were a number of limitations, but a "sidekick" concept has a great capability for future works. This is one step toward a future where thoughtful technology serves humans instead of having a negative impact on their mental health.
