After being able to control the basic features of a Kinect device (or any time-of-flight camera), the next thing many users look for is skeleton tracking. Skeleton tracking means to easily retrieve different joints of the human skeleton from depth images.
The most famous solutions (or the only ones) to do so are the Microsoft Kinect SDK or the OpenNI framework. If you are looking for a Free software solution though, you are out of luck. Microsoft’s SDK, apart from obviously being close, does not even allow a commercial use of it which is something the OpenNI framework does but this is as far as the meaning of the word “open” in OpenNI goes… You cannot adapt/improve their code nor learn from it. To solve this problem, Igalia has just created Skeltrack.
Free Software Skeleton Tracking
Skeltrack is a Free and Open Source Software library whose goal is to provide easy to use human skeleton tracking.
Skeltrack’s implementation was based on a paper by Andreas Baak but, apart from other internal differences, it doesn’t use a pose database. This means that the skeleton joints extraction is based on mathematics and heuristics, no calibration pose nor pose database is needed.
It provides an asynchronous API written in GLib, supports single user tracking (one skeleton only) and tracks up to 7 joints currently: head, shoulders, elbows and hands.
Take a look at the video below to get an idea of what it can do:
(watch it in Vimeo)
How to use it
We took a more modular approach than the two projects mentioned above. This means that Skeltrack expects to be given a depth buffer (i.e. depth image from the Kinect) with nothing but the user on it as opposed to connecting directly to a Kinect device.
Still, three easy steps should be all that is needed for Skeltrack to be ready to use:
1) Use the GFreenect library we released a couple of months ago, connect to the depth stream signal and get the depth buffer;
2) If there are other objects (chairs, tables, walls, etc.) apart from the user, then these should be removed by performing background subtraction or by excluding everything that is out of a threshold;
2) Skeltrack performs some calculations that might be heavy depending on the machine and the buffer size so users should reduce the size of the buffer that is given to Skeltrack by using its dimension-reduction property as the reduction factor.
Take a look at the test-kinect example (shown in the video) shipped with Skeltrack where the steps above are implemented.
The documentation for the project is available here.
Skeltrack is in its early beginnings and we want to detect more skeleton joints and work on stabilizing the results so, its features and API might change in the future.
Feel free to get the code, file bugs and send patches at its GitHub repository.
Hopefully more people will contribute to it until we finally have a rock solid, easy to use, Free Software skeleton tracking library.
I need to thank Søren Hauberg who was kind enough to point me in the right direction when I needed.