Extended Reality
Spatial computing, mixed reality, immersive simulations, and XR applications bridging physical and digital space.
ETC CAVERN
A 270-degree circular projection system with a 20-foot-wide play area, designed for local multiplayer interactions within virtual spaces.
Immersive Multilingual Storytelling
Multi-wall projected touchscreen exhibit featuring 3D virtual human avatars trained with multilingual chatbot personalities from cultures around the world.
Participatory Multicultural Storytelling
Enhanced interactive installation with directional ultrasonic haptics and physical embodied humanoid robot avatars, expanding the Immersive Multilingual Storytelling exhibit.
Son of Sara — LLM-Based Embodied Conversational Agent
An LLM-powered embodied conversational agent and successor to the SARA virtual agent platform, developed at CMU ArticuLab and INRIA. Features a modular real-time pipeline with ASR, LLM, TTS, predictive turn-taking (VAP), and a Unity-based virtual body with JSON-driven gesture, facial expression, and lip-sync control.
ARENAXR Digital Twins & 3D Scanning
Collection of 3D LiDAR photogrammetry scans, digital twin environments, and interactive ARENA scenes built at WiSELab @ CMU ECE.
Dynamic Projector Mapper
Real-time 3D dynamic projection mapping system that tracks multiple rigid objects via Optitrack motion capture and maps media onto them from a fixed 4K projector.
IoT Digital Twins in XR
Hybrid online AR/VR/XR experiences and interfaces for IoT Digital Twins, developed at WiSELab @ Carnegie Mellon ECE.
Multilingual LLM AI VR/AR/MR Chatbot Avatar
A multilingual LLM AI chatbot avatar NPC for Meta Quest 3 with expressive behaviours, speech recognition, and text-to-speech powered by Wit.ai and ConvAI/ChatGPT.
Projection-Mapped Pinball Game V2
AR pinball combining projection-mapped interactive digital media with a real, super-size physical pinball machine. Version 2 with bigger and better hardware.
SMALLab HTC Vive Network Streamer
Customized HTC Vive tracker network streamer for SMALLab K-12 interactive digital media systems by Edmotion Learning.
InteLLA — AI Tutor Avatar Pipeline
A photorealistic humanoid avatar generation pipeline for AI-powered tutors, built for Equmenopolis Inc.
Peerbots — Social Robot Face & Controller
Cross-platform social soft robot face and configurable controller app for Android, iOS, Mac, Windows, and Linux. Also works as a Misty II robot controller.
BLENDi — AR Museum Guide
An Augmented Reality museum guide and navigator running on HoloLens 2, built for Equmenopolis Inc.'s SXSW Edu showcase.
Digital Twin of Festivity
WebGL interactive virtual festival celebrating Chichibu Yomatsuri and Hadaka Matsuri festivals in Japan, built for Equmenopolis.
ETC ARCADE
Augmented Reality Creative Annotation Drawing Environment — a multiplayer AR iOS app for drawing and creating objects in a physics-enabled virtual sandbox.
Algorithmic Hiring Center
HoloLens AR experience exploring a futuristic algorithmic workplace, created with CMU ETC, KnowledgeWorks, and Remake Learning. Exhibited at Thrival 2019 and ISTE 2019.
HARP — Holographic Archive of Research Projects
A digital library of research projects displayed on an open-source Pepper's Ghost hologram pyramid projector. Includes 6 free laser-cut templates to build your own low-cost holographic display from a smartphone, tablet, or monitor. Supported by the Frank-Ratchye STUDIO for Creative Inquiry and CMU ProSeed.
Articulab RAPT — Rapport-Aware Peer Tutor
A virtual peer tutor that builds rapport with students and fosters socio-emotional awareness to improve communication, collaboration, and Algebra learning in 6th–8th graders. John Choi overhauled the Unity environment with modern real-time rendering techniques — bringing the virtual classroom from PS2-era to PS4-quality graphics.
Articulab SARA — Socially-Aware Robot Assistant
A 3D virtual humanoid AI agent showcased at the World Economic Forum in Davos (2017) and Tianjin (2016). Personalizes interaction and improves task performance by building rapport through social intelligence. John Choi built the new Unity 2017 framework with JSON-driven body animation, blendshape facial expressions, lip-sync, and TTS.
Articulab SCIPR — Sensing Curiosity in Play and Responding
A mixed-reality educational system using virtual human agents, augmented reality toys, and multimodal sensing to foster curiosity and exploration in elementary school students. John Choi built the virtual peer's gesture/expression system and an automatic fiducial-marker card tracking system that recognizes 27 marker positions at 5 FPS.
Project Codetta
Experimental Google Cardboard VR RPG — enter the digital world inside your smartphone. Built by John Choi and Daniel Hua. Scored 3rd Place at the Global Archiact Jam.
SMART: Some Mobile Augmented Reality Thing
An AR tower defense game built with Unity and Qualcomm Vuforia for Android. Place 4 physical marker cards on any flat surface and defend headquarters from robotic spider monsters in full 3D augmented reality.