Category Archives: kinect

Playing Angry Birds with a Kinect

Posted on by
5

Recently I had to use OpenCV in a project inside the Igalia Interactivity and I took the chance to code a little demo I had in my mind for a while: play Angry Birds with a Kinect and using only Free Software.

Here’s the result:


(direct link to video in Vimeo)

How it’s done

The demo uses Skeltrack (the only Free Software skeleton tracking library) to get user’s hands’ positions. The picked hand’s position will be used to move the mouse pointer. This part is the same that is used in the Skeltrack Desktop Control demo.
Once the hand’s position is known, I calculate an area around its point in the original depth image given by GFreenect and then use OpenCV to get the hand’s contours and their convexity defects. An open hand palm will produce very distinguished convexity defects and by counting them I assume the user’s hand palm is open. After this, all that is missing is to tie a detected closed palm to a mouse press and an open palm to a mouse release.

As the video shows, the demo is not polished yet but it shows one of the many possibilities that allying Skeltrack with other computer vision software gives us.

Going to GUADEC without leaving town

Posted on by
Reply

That’s right, this year GUADEC is taking place in the city I moved to more than 3 years ago in order to become an Igalian: A Coruña. It’s fun to see this event happening just 20 minutes walking from my place when in the previous editions I had to catch several planes in order to attend it :)

Going to GUADEC

In this year’s GUADEC, I am presenting two projects I have created:

  • OCRFeeder, the most complete OCR Free Software solution;
  • Skeltrack, the first Free Software library to perform human skeleton tracking from depth buffers such as the ones given by the Kinect.

If this sounds interesting, be sure to attend the talks or have a chat about the projects when you see me.

Since I feel pretty much like a local, I can tell you that you must not leave the town without trying “pulpo á feira” (octopus + olive oil + paprika) or, in case you’re not into cephalopods, just go to some traditional Galician bar, have a beer and enjoy the folk music of Celtic origins.

Skeltrack 0.1.4 “smoothie skeleton”

Posted on by
Reply

Here is the new version of Skeltrack, the Free Software library for human skeleton tracking from depth buffers.

Since the last release, I have presented Skeltrack at two events and built a cool demo of what can be done with this library.

What’s new

Apart from some bug fixes, this 0.1.4 version also introduces unit tests for making our development easier.
Yet, the big feature introduced in this version is the joints’ smoothing. If you have tried Skeltrack or watched the videos, you might have noticed that the skeleton is all jittery as if the joints were doing some caribbean dance. This is due to noise and other small changes in the depth buffer that happen in devices like the Kinect (and if you are wondering, it happens to the proprietary alternatives as well), so I implemented a way to smooth the joints’ jitters.

Smooth and quiet

This improvement was implemented using Holt’s Double Exponential Smoothing (which, for example, is what Microsoft’s Kinect SDK uses).
There are two properties that control this feature. The self-explanatory “enable-smoothing” property will turn the smoothing on or off and the “smoothing-factor” determines how much it should be smoothed. This value ranges from 0 to 1 and works as described in the mentioned Holt’s algorithm: values closer to 0 will give more weight to previous values of the joints as opposed to values closer to 1 that will consider the latest and current joints’ values more important. This means that values closer to 0 will offer a better smoothing but a bit of lag might be noticed so users should really choose this value themselves, as there’s no “good for all” factor. The smoothing is enabled by default with a factor of 0.5.

To show how well this might improve your Skeltrack-powered application, here is a video showing Skeltrack with the smoothing disabled and then enabled with a factor of 0.25.


(direct link to video in Vimeo)

The application shown in the video is the Kinect example we ship with Skeltrack which was also updated to allow controlling the smoothing feature.

Contribute

There are still a good number of tasks to be done that will improve Skeltrack. You can contribute too by forking the project at GitHub and sending us patches or you can of course hire Igalia to boost its development and make it rock even harder.

Let me know of any applications you’re developing with Skeltrack and how we can make it better. Check out the documentation and file some bugs if you run into trouble.

Stay smooth.