Moontower AR music video

A pre-show AR music installation produced in collaboration with the band Moontower (@thisismoontower). We demoed the experience to around 300 attendees over the course of several shows. Attendees stepped inside a lit pod to experience the 3-minute interactive music video.

A student collective I co-founded called Merciv created the experience around Moontower’s music debut and artistic vision. The collective was made up of a dozen artists, developers, and business students.

Engineering challenges

I led the engineering team that integrated the full tech stack that allowed us to run guests through multiplayer, wireless, 6-DOF AR experience — in 2017, before this was available off the shelf! Our final experience used Mira Prism AR headsets, Unity, Vive trackers, and a Leap Motion controller.

While building this, we learned how difficult it was to build both quality and cost-effective AR experiences using the current tech.

High-end headsets were too expensive and fragile to be used in a lively concert setting with attendees switching in and out every few minutes. Low-cost headsets lacked features and visual quality for a compelling AR experience. It’s a fine tradeoff but I think we struck the right balance for this experience.

I also personally learned a lot about how to manage a team of people with varying skillsets and who have limited bandwidth (because of school). Lots of planning, high-level architecture, and realistic scoping of tasks!

 A flexible music playback engine running on each headset allows us to incorporate rich metadata from songs and stems into music visualizations using minimal CPU during playback
A VIVE base station installed inside the pod. The position and rotation data for each headset is received through OpenVR, transformed to the experience coordinate system, and sent over the network to each client.
A server application running on a laptop manages each show and headset