This morning in the Hilton, John Boiles of Yelp and Dan Fernandez of Microsoft held a panel on Kinect hacking. Their special guests were the popular boxing Kinect robots last seen on Saturday night at the official opening party.
As you might already know, the key components of a Kinect are the IR Emitter, Color Sensor, IR Depth Sensor, Tilt Motor and a microphone array. With these tools, a Kinect can track up to 20 joints on up to 2 people simultaneously at 30fps. The new version (Kinect for Windows) adds such features as speech recognition, directional microphone capabilities and the ability to track objects as close as 18″. Interestingly, one can hook up multiple Kinects to one PC and have the same program control them all. The caveat is that only one of the Kinects in an array can track skeletal data. But color, depth and audio sensing can be assigned to any of them.
In one demo, Mr. Boiles showed how they used a programmatic interface for Half-Life 2 to insert a Kinect version of themselves into a sandbox environment powered by the real game physics engine and interact with game objects. He was able to kick objects and swing a barrel around.
Mr. Fernandez demonstrated a nerf gun that is attached to a servo motor and then through to a computer and a Kinect. When the Kinect sensor picks up a person it locks on and tracks them. Kinect measures proximity down to 10cm of accuracy, and when the person gets too close, it fires the nerf ordinance directly at their crotch.
Another demo that is in the works is the ability for a Windows Phone client to pick up Kinect data over a network. This data can then be integrated into apps on this platform. This opens up a whole new field of interaction possibilities for out of home marketing.
Mr. Boiles demonstrated a Kinect hack that allowed him to control a toy Helicopter by waving his arms. This was the biggest hit of the session, until the boxing robots slugged it out, which is always a big crowd pleaser.
