Pd and Generative Music

Here's a taste of what I (Nate) have been up to for the last week.  I have created a patch in Pure Data that randomly generates music.  It utilizes Markov Chains and Random Walk Generators.  I made this first one in Rondo form (ABACABA) using a i - bIII - bVI progression.  I also have Pure Data reading information from my Arduino and am trying to incorporate that and Drum Beats in this week,

Wearable Tech Research

We have been researching wearable technology within the scope of our project.  While RFID seems promising, the big limitation with it is its range.  It seems as though Bluetooth will be the way to go as there are affordable smart bracelets with sensors already embedded within them.  I like the graphic above as it breaks down all the things that are going on within the world of wearable tech these days.

Wearable Tech at Music Festivals

Events have begun using RFID wristbands to enhance the consumer's experience.  Whether it be to trigger lasers, get a cold beverage delivered or even to gauge the activity level of the participant.  Another application is the ability to trade your information with someone via the press of a button.  Mobile devices and their prominence are another factor that we are interested in using to our advantage.

Generative Inspiration

Playing with some generative art sketches in Processing!!  Definite inspiration for future work!

Processing, the Kinect and OpenNI

The Kinect was a easy decision for our project because it is not sensitive to the light conditions in the room at the time it is captured. Hence, if we use this in a dark room it will not be an issue. The Kinect camera works by creating a depth image. It uses infrared light to create an image that captures where the objects are in space. The Kinect camera resolution is 640x480. You can bump the camera up to 1280x1024 but the data will arrive at 10 frames per second rather than 20 frames per second.

The next decision we had to make was determining which processing library would work the best for what we are trying to accomplish with the Kinect. It boiled down to OpenKinect and OpenNI.

Dan Shiffman built on the work of the OpenKinect project to create a library for working with the Kinect in processing. OpenKinect drivers provide access to the Kinect's servo motors and has a very simple software license. The contributors to the OpenKinect project released their drivers under a fully open source license. In short, this means that you can use OpenKinect code in your own commercial and open source projects without having to pay a license fee to anyone. In response to OpenKinect PrimeSense released their own software for working with the Kinect. PrimeSense included more sophisticated software that would process the raw depth image to detect users and locate the position of their joints in three dimensions. They called their software OpenNI, NI standing for "Natural Interaction". OpenNI provides two key pieces of software that is useful to our goals. First is the OpenNI framework. this includes the drivers for accessing the basic depth data from the Kinect. This piece of software has a similar licensing situation as OpenKinect. However, the other feature that OpenNI provides does not have such a simple license. This feature is the user tracking. This license is provided by an external module, called NITE. NITE is not available under an open source license. It is a commercial product that belongs to PrimeSense. PrimeSense does provide a royalty-free license that you can use to make projects that use NITE with OpenNI. 

We chose to use OpenNI because it provides us with the option to use the user tracking and there was a good amount of reading material that explained and used the OpenNI library in Processing. OpenNI also has the advantage because it is designed to work with not just the kinect but also other depth cameras. This means that code we write using OpenNI to work with the Kinect will continue to work with newer depth cameras as they are released, saving us from needing to rewrite our applications depending on what camera we want to use. 

OpenNI recognizes heads, shoulders, elbows, wrists, chests, hips, knees, ankles, and feet. This is going to be an important part in tracking the crowd in our installation. One of the techniques we are considering using OpenNI's 3D features and Point Cloud systems. Below is are screenshots of a test sketch of a point cloud system that we created that creates the illusion that it is rotating completely around what the kinect is seeing.

Rotating Point Cloud Sketch Test Image (1)

Rotating Point Cloud Sketch Test Image (2)

Research and tutorials referenced from Making Things See by Greg Borenstein

Installation size research

How much distance could we effectively use our tech in? 

Finding a balance between comfort for a group of users to experience our interactive floor with others, realistic size of fabricating the project and number of participants is a tough task. We want to let as many people as possible experience and play together, but we have to be mindful of how much room people need to stand around and be able to move their bodies. On the other hand, we also want people to feel like they are having fun together and don't want to give too much cushion between bodies. Finally, we have to realize that what we are building has to conform with the size of our lab and our budget so we can not build a studio with a stage.

Imagination can do so much, let's really see it!

As researched in our previous blog post Floor Size and Negative Space Research we have a good point of reference thanks to Dance Deck of Signature Systems Group, LLC. They are the world's largest center for renting dance floors and recommend a minimum of 4.5 square feet per person to have enough room on a dance floor. Considering this figure per person and combining it with the maximum effective distance of a X-Box Kinect which we will be using in our project we can make a good decision on the size of our floor plan.

These constraints have led us to choose a 100x100 foot square for the time being. Ideally this could populate 20 people. Unlike a traditional dance floor, our project will include display walls and a place for our performer (at this point, a DJ) to set up their materials. We have reserved a space for this which we have deemed 10 square feet, leaving 90 square feet for the participants which still equates to 20. For testing purposes we are choosing not to work in the ideal scope as we tread new waters. We are setting our maximum at 10 users until further notice and aiming for a minimum of 5 users for ideal conditions.

We want to be able to let users really go wild in our environment. Furthermore, we understand that there are limitations to the technologies we will be using during this project due to our budget restraints. All hope isn't lost friends so don't fret. Hopefully after user testing encountering the many unforeseen variables of this project we can involve more participants within this floor space, or expand our floor. Stay tuned fans! There are no instruction manuals here, just discoveries being made! #FBA!

Seeing is believing. It was crucial for us to see our floor plan in person before moving forward.

Thermistor and Installation Size

The first picture above is the thermistor hooked into the Arduino UNO.  A thermistor gives readings about temperature change because temperature effects the thermistor's resistance (Temperature Sensor + Resistor = Thermistor.

The second picture is Nate and Miguel's first attempt at installation size.  Each box marks a corner of the installation size.  This is a very important aspect of our project as it will affect the user's experience.