Controlling GNOME with Leap

When I explained how the Leap Motion device could be used on Fedora 19, I mentioned how I had one of those early prototypes. Well, Leap Motion was extremely kind and sent me an actual device as a thank you for starting the thread asking for Linux support. Now that GUADEC is over and I am spending my vacation in Portugal, I had a little time to play with my fancy new device and wrote a relatively small script to control GNOME with it. I call it the Ć¼ber original name of: Leap GNOME Controller!

For those who don’t care about technical details, here’s the video showing what can be done with Leap, GNOME and this script. Technical details follow below the video:

The two videos that compose the one above were recorded with an HD camera and GNOME Shell’s screencast recorder. I tried to sync them the best I could but a certain delay can be noticed, especially at the end of the video.

The code

Leap Motion provides a “close source” shared library and a high-level API with the respective documentation for the many bindings it has. To code it quickly, I used the Python bindings and Xlib to fake the input events.

Leap Motion’s APIs make it really easy for one to simulate a touch-screen. It even offers a “screen tap” gesture that should be the obvious choice when mapping a finger touch/tap to a mouse click. However, this didn’t work very well. The problem is that if we are tracking one finger to control the mouse movement, when performing the “screen tap” gesture, the finger (and mouse) will of course move. Making it as frustrating as seen on ArsTechnica hands-on video.

I came up with a solution for this by dropping the “screen tap” gesture and using the “key tap” instead. The “key tap” is a simple, quick down-and-up finger movement, like pressing a key. This is much more precise and easier for a user to do than the “screen tap”. Of course that when the finger moves for performing the gesture, the mouse pointer would move as well, so I came up with a little trick to work around this: when the mouse pointer doesn’t move more than a couple of pixels for half a second, it will stop and only move again if the user makes it move for more than 150 pixels. This allows for the user to stop the pointer with precision where it needs to be and perform the gesture without making the pointer move.

Future

The Leap device offers a lot of possibilities for adding many gestures. Ideally they should be implemented per application but being able to control the shell is already pretty useful, so it would be wonderful to fine-tune the current gestures and add new ones. I also wish the library’s source code were open because I ran into small issues and I wish I could take a look at the source code, instead of trying to fix it based on the theories of what might be wrong.

I haven’t explored the AirSpace appstore yet so I don’t know if it is worth adding (or possible to add) this script there but I will check it out.

Have fun with Leap and GNOME!

OCRFeeder 0.7.11 released

Here is 2013’s first version of OCRFeeder, version 0.7.11.

For this version, a number of bugs were fixed, especially some that were affecting saving and loading projects.
Some small improvements were also made such as being able to load multiple images at once and being able to choose the OCR engine from the command line interface version of OCRFeeder (using the -e option).

Now for the main feature, I developed something that had been requested by a good number of users: being able to easily choose the language for the OCR engine.
When I developed OCRFeeder, I wanted to make it easy for users to use system-wide OCR engines from the layout analysis that OCRFeeder performs but I also wanted it to remain powerful and that’s why the engines are configured in a general, abstract way, as if from the command line.
Some OCR engines support setting the language in order to get a better recognition and while, users could already set the language of an engine manually using the OCR editor dialog, they wanted to have a nice drop-down list with the languages instead.
This represented a real challenge: to keep the old and flexible configuration and, at the same time, offer a high-level way of choosing the language.

OCRFeeder's new configuration
So here is how it works. There is a new special argument keyword $LANG that will be replaced by the new field “language argument” and the currently set language. Since engines support different languages (or none) and call them different names (e.g. Tesseract expects “por” for the Portuguese, others may expect “pt”) there is another new field called “languages” which should be a map between the language code in the ISO 639-1 and the name of the language of the engine expects, as shown in the screenshot.

Languages combo
To show the languages, there is a new tab in the areas’ editor called Misc (in lack of a better name for a tab that’s holding more stuff in the future) with the languages combo. This combo shows a check on the languages that the currently selected engine recognizes as seen in the screenshot.

There is also a new setting in the preferences dialog with the default language and the first time the application runs, it will assign it to the user’s locale.
One thing must be taken into account: even though Tesseract supports an extensive list of languages, the users must have those packages installed in their distros, otherwise, recognition will of course fail.

To finish, related to my recent job search, I have spent this week in San Francisco getting to know some people from an exciting start-up and despite the jet lag, I managed to finish this release so I can now say that least part of OCRfeeder was designed and developed in California šŸ˜›

Source tarball
Git
Bugzilla

Winds of Change

In my previous post I mentioned that 2013 would be a year of change. Well, here is the moment to say why that will be so: I have quit Igalia.

Igalia is a very special company to me, I joined it in December 2008. These were 4 intense years where I saw how the company evolved, how it moved to a cool new office, how it grew and I learned a lot in there. I had the chance to participate in several important projects like Maemo or Meego and also to create others. I could even tell the world about them in the many conferences I spoke at and I am also proud to have accomplished things such as putting the company’s name for the first time in the highlights of online media like ArsTechnica.

So the question people always ask is: why did I leave!?
As some of you may know, Igalia is organized in a flat structure where we take more responsibilities than just coding and the ultimate part of a career in the company is to become a partner. I knew this when I joined and I think this is a wonderful thing. Being at the end of my 4th year, the next stage would be to become a partner, however, for a while now I have been feeling the need of a change, of trying something different. I take my responsibilities seriously so joining as a partner would 1) only perpetuate these feelings and 2) not be fair to my colleagues. This and other factors led me to make the very difficult decision of leaving.

The future

My wife and I moved to A CoruƱa (Galicia, Spain) shortly after I joined Igalia. We like the city and its people but moving is part of that change I was talking about and the truth is that we were only here for Igalia in the first place. (I will probably write a few more words about this beautiful city when we actually leave)
The most difficult part of it is definitely leaving our friends. We met very nice people during these 4 years in CoruƱa and we consider some of them good friends rather than simply coworkers. But life is like this and I am sure we’ll stay in touch.
On the other hand, the good thing of working in a Free Software company is that you can keep contributing to the projects you worked on in there if you want, so I hope I will keep doing that.

Since I have only started looking for a new job after I notified Igalia of my decision, I still do not know where we will move to but we are open to many places.

If you are interested in what I can do for your project or company, be sure to contact me through email or LinkedIn so I can send you my CV.

That is all. I am already in touch with some companies so wish me luck!