My summer research uses motion capture and green screen to explore the choreographies of rock climbing. I am interested in how route design influences a climber’s interpretation and experience of a rock wall. This summer research is an opportunity for me to gain the technical skills needed to create a live performance installation that explores rock climbing and dance through motion capture and projection design. My goal is to apply this research to a fall 2022 performance installation with a greenscreen bouldering wall.

So far this summer, I have recorded motion capture data of climbers ascending the 50-foot rock wall on campus. I have also experimented with a greenscreen body suit at a local bouldering crag. I have learned how to send data from the Perception Neuron 3 body suit into the Unreal Engine. From there, I have experimented with sending the Unreal video into Isadora via Spout and NDI. I entered the fellowship with a basic understanding of this workflow through Unity. However, through this fellowship, I am learning that Unreal Engine may be more compatible with my future research goals. I solidified this Unreal workflow with both recorded and live data. I also created an digital manual of this Unreal workflow for future students to use. Additionally, I have explored using two PN3 suits simultaneously to capture data of multiple climbers.

Next, I plan to learn the OptiTrack Motion Capture System. Learning this system will provide insight to even more possibilities of motion capture design. I am setting up a greenscreen climbing wall with this system to capture and record data of bouldering sequences. My goal is to then compile this footage for projections and video designs through After Effects.

While most of this research will be applied to projects in the 2022-2023 academic year, I plan to map footage of these experiments onto small rocks at the end of this summer. Since my project uses many different programs and forms of motion capture, this installation will help me realize the ways in which I can apply my research to future performances and art installations.