Processing, the Kinect and OpenNI

The Kinect was a easy decision for our project because it is not sensitive to the light conditions in the room at the time it is captured. Hence, if we use this in a dark room it will not be an issue. The Kinect camera works by creating a depth image. It uses infrared light to create an image that captures where the objects are in space. The Kinect camera resolution is 640x480. You can bump the camera up to 1280x1024 but the data will arrive at 10 frames per second rather than 20 frames per second.

The next decision we had to make was determining which processing library would work the best for what we are trying to accomplish with the Kinect. It boiled down to OpenKinect and OpenNI.

Dan Shiffman built on the work of the OpenKinect project to create a library for working with the Kinect in processing. OpenKinect drivers provide access to the Kinect's servo motors and has a very simple software license. The contributors to the OpenKinect project released their drivers under a fully open source license. In short, this means that you can use OpenKinect code in your own commercial and open source projects without having to pay a license fee to anyone. In response to OpenKinect PrimeSense released their own software for working with the Kinect. PrimeSense included more sophisticated software that would process the raw depth image to detect users and locate the position of their joints in three dimensions. They called their software OpenNI, NI standing for "Natural Interaction". OpenNI provides two key pieces of software that is useful to our goals. First is the OpenNI framework. this includes the drivers for accessing the basic depth data from the Kinect. This piece of software has a similar licensing situation as OpenKinect. However, the other feature that OpenNI provides does not have such a simple license. This feature is the user tracking. This license is provided by an external module, called NITE. NITE is not available under an open source license. It is a commercial product that belongs to PrimeSense. PrimeSense does provide a royalty-free license that you can use to make projects that use NITE with OpenNI. 

We chose to use OpenNI because it provides us with the option to use the user tracking and there was a good amount of reading material that explained and used the OpenNI library in Processing. OpenNI also has the advantage because it is designed to work with not just the kinect but also other depth cameras. This means that code we write using OpenNI to work with the Kinect will continue to work with newer depth cameras as they are released, saving us from needing to rewrite our applications depending on what camera we want to use. 

OpenNI recognizes heads, shoulders, elbows, wrists, chests, hips, knees, ankles, and feet. This is going to be an important part in tracking the crowd in our installation. One of the techniques we are considering using OpenNI's 3D features and Point Cloud systems. Below is are screenshots of a test sketch of a point cloud system that we created that creates the illusion that it is rotating completely around what the kinect is seeing.

Rotating Point Cloud Sketch Test Image (1)

Rotating Point Cloud Sketch Test Image (2)

Research and tutorials referenced from Making Things See by Greg Borenstein