The Microsoft Kinect is one of the most popular, if not the best, skeleton tracking system available commercially. However, even with its power, it’s API and access to data can be limiting in cases of hyper-realistic requirements. 

To create an experience that allowed ultra-realistic puppet movement I had to re-imagine the interaction between user and puppet in the digital world. Using Kinect directly there was to much lag, jittering and unrealistic control over elements such as the hands, arms and legs.

Using Touchdesigner’s unique bone system constructor with their detailed constraining mechanics, I embedded a bespoke inverse IK solver system that took the defined bone structures to limit, control and direct the movement of arms and legs in hyper-human style for a number of different application sources.

This method allowed for limited use of real-world data, while maintaining a clear user presence and control over any environment without compromising on quality.