I have spent the majority of the second half of the summer troubleshooting the Spout and NDI plugins for Unreal Engine. Recently, I have been using Unreal Engine to send video from Axis Studio (the Perception Neuron 3 software) into a projection software called Isadora. With help from my point of contact Matthew Butler, I learned that the Spout and NDI plugin was not compatible with the version of Unreal Engine that I was using. I then had to familiarize myself with a previous version of Unreal Engine, reinstall the Perception Neuron plugin, relearn how to send Axis Studio data into Unreal Engine 4.27, and then send video out of UE via NDI. I experimented with NDI by using both live and recorded avatars simultaneously. I learned that the program crashes unless I add a second avatar later in the connection process. I have also spent the past week trying to connect two live avatars into Axis Studio using two separate Perception Neuron 3 motion capture bodysuits. I am happy to report that today I successfully connected two avatars simultaneously into Axis Studio by using different channels on the receivers!

I also learned how to connect a camera to my avatar’s head in the Unity software. This enabled me to output a first-person perspective of the avatar into Isadora. In doing so, I was able to experiment with virtual climbing through a first-person perspective. So far, I have keyed out the Unity background and replaced it with images of indoor climbing walls.

I have also spent a lot of time moving and constructing a tread wall and tension wall. My goal is to paint these green so that I can easily gather green screen video footage that isolates a climbing body from any rock background. With this footage, I plan to study the climbing movement and sequences. My goal is to use the motion capture body suits on these green screen walls, and to create virtual interactive performances with the wall. This fellowship has given me the time to develop the digital skills needed to pursue digital projects on these walls.

The remainder of my time in this fellowship will be spent on learning the OptiTrack motion capture software. Once I am familiar with the camera system of OptiTrack, I can then repeat the data flow and share it into Unreal/Unity and Isadora.