Salutations Interactive Installation: Making of

Since the post about the Salutations Interactive Installation in the Museum für Kommunikation in Berlin, many people asked me for videos of the installation so we, the Igalia Interactivity team, came up with a better idea — a making of.

The video shows the an early test application that was created from the one shipped with Skeltrack, then some testing with colleagues at Igalia’s office and finally the installation in Berlin and the final result:


(link to original video in YouTube)

You can get the source code for the application from Igalia’s GitHub.

Skeltrack 0.1.8 released

Skeltrack, the Open Source library for skeleton tracking, keeps being improved here in Igalia and today we are releasing version 0.1.8.
Since July we have had the valuable extra help of Iago López who is doing an internship in Igalia’s Interactivity Team.

What’s new

Several bug fixes (including the introspection), both in the library and the supplied example were fixed.
The threading model was simplified and the skeleton tracking implementation was divided in several files for a better organized source code.

While the above is nice, the coolest thing about this release (and kudos to Iago for this) is that it makes Skeltrack work better with scenes where the user is not completely alone. The issue was that if there was another person or object (think chairs, tables, etc. for a real life example) was in the scene they would confuse the skeleton tracking. After this version, while not being perfect (objects/people cannot be touching the user), the algorithm will try to discard objects that are not the user.
But what about having two people in a scene, which one will it choose? To control this, we have introduced a new function:

skeltrack_skeleton_set_focus_point (SkeltrackSkeleton *skeleton,
gint x,
gint y,
gint z)

This function will tell Skeltrack to focus on the user closer to this point, thus allowing to focus on a user in real time by constantly changing this point to, for example, the user’s head position.
So, even if there is no multi-user support, the current API makes it easy to just run other instances of Skeltrack and try to pick users from other points in the scene.
It should also be easier to use Skeltrack for a typical installation where there is a user controlling something in a public space while other people are passing or standing by.

Contribute

We will keep betting on this great library.
If you wanna help us, read the docs, check out Skeltrack’s GitHub and send us patches or open issues.

Skeltrack 0.1.4 “smoothie skeleton”

Here is the new version of Skeltrack, the Free Software library for human skeleton tracking from depth buffers.

Since the last release, I have presented Skeltrack at two events and built a cool demo of what can be done with this library.

What’s new

Apart from some bug fixes, this 0.1.4 version also introduces unit tests for making our development easier.
Yet, the big feature introduced in this version is the joints’ smoothing. If you have tried Skeltrack or watched the videos, you might have noticed that the skeleton is all jittery as if the joints were doing some caribbean dance. This is due to noise and other small changes in the depth buffer that happen in devices like the Kinect (and if you are wondering, it happens to the proprietary alternatives as well), so I implemented a way to smooth the joints’ jitters.

Smooth and quiet

This improvement was implemented using Holt’s Double Exponential Smoothing (which, for example, is what Microsoft’s Kinect SDK uses).
There are two properties that control this feature. The self-explanatory “enable-smoothing” property will turn the smoothing on or off and the “smoothing-factor” determines how much it should be smoothed. This value ranges from 0 to 1 and works as described in the mentioned Holt’s algorithm: values closer to 0 will give more weight to previous values of the joints as opposed to values closer to 1 that will consider the latest and current joints’ values more important. This means that values closer to 0 will offer a better smoothing but a bit of lag might be noticed so users should really choose this value themselves, as there’s no “good for all” factor. The smoothing is enabled by default with a factor of 0.5.

To show how well this might improve your Skeltrack-powered application, here is a video showing Skeltrack with the smoothing disabled and then enabled with a factor of 0.25.


(direct link to video in Vimeo)

The application shown in the video is the Kinect example we ship with Skeltrack which was also updated to allow controlling the smoothing feature.

Contribute

There are still a good number of tasks to be done that will improve Skeltrack. You can contribute too by forking the project at GitHub and sending us patches or you can of course hire Igalia to boost its development and make it rock even harder.

Let me know of any applications you’re developing with Skeltrack and how we can make it better. Check out the documentation and file some bugs if you run into trouble.

Stay smooth.

Grilo Vimeo plugin

Vimeo is one of the main video sharing places in the web and I thought it would be useful to develop a Grilo plugin to search videos on it.

Yesterday Juan committed the code which means you should now be able to easily search videos from Vimeo and watch them in your desktop using Grilo’s test UI. Here are a couple of screenshots:

Grilo test UI
Vimeo video on Totem

I really like the way Grilo is going. Together with GStreamer, the effort needed to create and media player with sources such as your hard drive, YoutTube, Vimeo, Flickr, etc. is just minimum.

Let’s hope more plugins will arrive at Grilo!

Text Prediction on GNOME

I was disappointed with the text completion provided by the N900 (eZiText) that, on top of that, is closed and I wondered if it was possible to have an Open Source solution to provide text prediction and completion.

I searched a bit and besides my original intentions of developing a library to search Free and Open Source dictionaries’ words from a prefix, I found Presage.
Presage is better than most text prediction systems I have seen out there because it really is text prediction, not text completion. This C++ library, retrieves words taking into account the surrounding text, not only the prefix or frequency of words. It uses a database representing N-grams that can be trained with more text; the more you train it, the more accurate it can be.

This means that is you type something like:
“I m”
instead of suggesting nonsense things like:
“I mouse” “I mother” “I market” or “I more
it suggests something more like:
“I must” “I met” “I mean” or “I might
The difference is obvious!

So I developed a little wrapper around Presage in C that provides a yet very basic API to get text completion. Then I created a GTK+ Input Method context to control the user’s input in regular GTK+ text widgets and used the wrapper to process the inputted text. I called it: Predictor Input Method (not very original I know…).
The result is that Predictor suggests you words, even if you type a prefix or not, and lets you accept the candidate word or scroll through a list of suggestions as you can see in the video below:

Text prediction in GNOME from Joaquim Rocha on Vimeo.

How to use it

The current key bindings are:

Ctrl+Enter -> Selects the current candidate
Ctrl+Up/Down -> Scrolls through the list of candidates
Backspace -> Deletes the character previous to the cursor and suggests again
Directional arrows -> Move cursor and discard suggestions

Who should use it

This kind of assistance technology can have many applications but the main ones are: the usage in small/mobile devices and the assistance of users with disabilities. Both have the same reasons behind: speeding the input and reducing failed characters, because the input required gets minimized;
Of course, you can as well use it in your GNOME desktop regularly for faster typing your emails, etc.

In the case of users with disabilities, a popup menu could be added to show a complete list of candidates and the bound fast-access keys.

Why is Free Software important in this

This is the kind of technology that everybody should have an interest in using a FOSS solution because of the obvious advantage that is developers from all over the world being able to modify it.
Suppose you’re creating a mobile phone and you choose a closed solution to provide text prediction for your phone. And then you find out you’re disappointing all your users from country X because that library you’re paying for does not support their language and the library owner is not interested that much in adding it. Now if you’re using an open solution, local communities from many places in the world can add support for their languages and your phone can have a better acceptance in places you hadn’t even imagined.

Software that reaches an international audience with different languages is software you want to have open.

How to get Predictor Input Method

You can find the Predictor Input Method’s source its Gitorious page: http://gitorious.org/text-predictor-input-method
Of course, you should also install Presage for it to work.

If you are not using GTK+ Input Methods then you can use the wrapper text-predictor.cpp which is not tight to the Input Method code itself. And of course, you can copy the little tricks used on the Input Method code and apply it to your source (like delaying the retrieval of the candidates some fractions of a second to not block the input, etc.).

Hope you like it.