Immersive AI Art Installation
Using embodied interaction to build a gamified storytelling experience.
Generative AI
Embodied Interaction
Overview
Combined embodied interaction and AI to build an interactive gamified storytelling experience for installation. The project objective was to use Embodied Interaction to build a new kind of storytelling game experience. An Immersive AI Art installation that provides storytelling and gaming experience at the same time. Providing autonomy to the player to choose and listen to their perspective and engage with the characters.
Tools
Touchdesigner
Midjourney
ChatGPT
D-ID Studio
Timeline
Jan 2024 (3 Weeks)
Project
Embodied Interaction Course Project
Work
Group Work
My Role
Initiated the core idea for an interactive, movement-based narrative art installation in collaboration with one teammate.
Key Experience Design
The experience was designed to be immersive, intuitive, and emotionally engaging, with the participant’s body acting as the primary interface.
The Problem
Bring awareness among the people related to the topics of present-day needs.
What sparked the idea?
I wanted to explore how movement could drive a story and make people feel more involved in the experience.
Tools & Tech
Touchdesigner acted as the primary software to run the whole installation smoothly. Everything was integrated into it, from detection, tracking, to audio visual display.
Used Midjourney for character and background images, and D-ID Studio for fully AI-driven character videos after testing several platforms.
ChatGPT generated the characters’ responses, enabling dynamic interaction.
Installation Building Process
Choose the type of Interaction
Choose jumping as a simple action to get started, and stepping on the character projected on the ground to interact.
Narrative Building
The characters related to the topic were designed with Midjourney, and videos were made with D-ID studio.
Installation of tools & tech
Additional projectors, speakers, microphone were arranged, and the whole installation setup was done.
Testing with the complete setup
Testing was done before the demo day to fine-tune the position markers for a smooth run.
Testing
The technical part required extensive testing and fine-tuning of parameters, including display, audio, and motion tracking. In-room testing was crucial to ensure precise detection of movement and accurate projection mapping on the floor. We iterated frequently to align visuals and responses seamlessly, ensuring everything functioned smoothly for the final demonstration.
Reflections & Learnings
This project was my first step into designing an immersive art installation, and it reshaped how I think about experience design.
Previous
Conversational UI Design
Game UX