Character presentation for the mixed-reality AI companion
Project showcase
An AI personality built for spatial computing
Meta Quest 3
Unity 6
OpenXR
Local LLM
Voice + Generative Tools
Luna is a local AI companion developed for the Expanded Realities study programme. Instead of a traditional chat interface, the project combines a character presence, voice conversation and spatial tools for image generation, 3D generation and mind-mapping inside mixed reality.
Spatial interaction & orb system
I developed the orb menu, mode switching, microphone interaction, visual feedback and user-facing Unity systems.
Unity ↔ AI backend
I worked on real-time WebSocket communication and the application flow between Quest, voice input and AI responses.
LLM, TTS & server setup
I also worked with the local Linux/Docker server environment, LLM deployment and the speech pipeline used by Luna.
Trailer
See the experience in action
Spatial tools
Four modes, one interaction language
The orb interface gives access to conversation, image generation, 3D model generation and a spatial mind map. Using the same interaction pattern across all modes keeps the experience readable and consistent in-headset.
UX & onboarding
Designed for clarity inside XR
The menu follows the user and provides immediate visual state feedback. The onboarding flow introduces interaction, controls and Luna without relying on a conventional desktop-style tutorial.
The design was iterated to reduce menu clutter and make recording, mode selection and first-time use easier to understand.
Local AI architecture
Speech, memory and generation on a local server
The backend combines speech-to-text, short- and long-term memory, RAG, LLM inference and text-to-speech. Separate generation workflows extend the system with image and 3D content while keeping the core architecture local and expandable.
Character feedback
A character that reacts, not just responds
Luna uses facial expressions and color states to make feedback more immediate and to give the AI system a recognizable personality inside the experience.
Calm / welcomingA friendly visual state for conversation and neutral interaction.
Expressive / alertA stronger color and facial state used to communicate mood and reaction.
The Lady of AVRD was a six-person team project. This case study focuses on the parts of the system and experience that I personally contributed to, while the project visuals represent the complete team result.