Thoughts 'n Stuff

How we enable others to write 3rd party plugins with Maliit

We finally published a video about Maliit - an input method framework including a virtual keyboard - and 3rd party plugins. Kudos goes to Jon for making time for that.

This video highlights one of Maliit's key features: pluggable input methods which come with their very own user interfaces. The Chinese input methods show how Maliit offers support for composed characters. The video is proof that 3rd party development for Maliit (open-source and proprietary) is not only possible but also happening.

maliit.org states that "it should be easy to customize existing input methods or develop powerful new input methods, whether for profit, research or fun", we actually mean it.

The harder question is of course how to motivate others to actually get started on input method development with Maliit. For that, we have a multipronged strategy:

1. Provide sufficiently polished reference plugins that can show off Maliit capabilities but also serve as inspiration for new plugins (hence the BSD license for reference plugins). Our reference plugins are currently using Qt/C++ (Maliit Keyboard) and QML (Nemo Keyboard). We also have PySide support, but no one contributed a reference plugin yet. This gives choice to interested input method developers, and we think that's important. The reference plugins serve another role when it comes to designing new API: They become our testbed, allowing us to verify our API proposals.

2. Ship Maliit with a bunch of example plugins and example applications. None of them try to be complete. They are all self-contained though and usually show one feature at a time. This can be tedious to maintain, but we believe that examples need to stay small and focused, otherwise developers won't look at them.

3. Documentation that is easy to consume. Our documentation is not as concise and clear as we'd like it to be, but it's slowly improving. We also experiment with videos that can serve as an introduction to more in-depth (text) documentation.

4. Packages for most common Linux distributions. This one seems obvious, but sadly, it's quite a lot of work for us to keep up with it (and we already use automated services such as Launchpad and OpenSuse Build Service). In the hope to attract dedicated packagers we wrote down some packaging guidelines

5. An architecture that had 3rd party plugins and multiple toolkit support in mind from the start. The plugin developer facing API needs to be easy to use and clearly documented. This will be the focus of the upcoming 0.9x series.

We will demo Maliit @ FOSDEM 2012, hope to see you there!

Using C++ enums in QML

When mapping Qt/C++ API's to QML, or, to put it more precisely, making a Qt/C++ API available to QML, road bumps are to be expected. One such bump is the mapping of C++ enums.

If you happen to create enums inside a QObject, then it will be exported to QML via the Q_ENUMS helper:

SomeEnumsWrapper
    : public QObject
{
    Q_OBJECT
    Q_ENUMS(SomeState)
public:
    enum SomeState {
        BeginState,        // Remember that in QML, enum values must start
        IntermediateState, // with a capital letter!
        EndState
    };
};

You will still need to declare this class as an abstract type for QML to be able to use enums from it (put in your main function for example):

qmlRegisterUncreatableType<SomeEnumsWrapper>("com.mydomain.myproject", 1, 0,
                                             "SomeEnums", "This exports SomeState enums to QML");

Now in QML, the enums can be accessed as '''SomeEnums.BeginState'''. Note how the enum is accessed through the exported type name, not an instance.

But what if you've put your enums into a dedicated C++ namespace? Then the same mechanism can be used. Let's start with the namespace:

namespace SomeEnums {
    enum SomeState {
        BeginState,
        IntermediateState,
        EndState
    };
}

We can re-use the idea of wrapping enums in a QObject type, with one tiny change:

SomeEnumsWrapper
    : public QObject
{
    Q_OBJECT
    Q_ENUMS(SomeState)
public:
    enum SomeState {
        BeginState = SomeEnums::BeginState, // Keeps enum values in sync!
        IntermediateState = SomeEnums::IntermediateState,
        EndState = SomeEnums::EndState
    };
};

The process of forwarding all your enums through this mechanism can be tedious, but being able to use enums properly in QML properly will improve the readability and maintainability of your QML code.

For a fully working example check Maliit's Qt Quick support

Writing QML-based input methods for Maliit

This week I pushed a simple QML-based virtual keyboard to our MeeGo Keyboard repository. It's functional, but don't expect too much. This functionality will most likely never arrive in MeeGo 1.2, sorry folks.

Getting Started

Let's continue with the good parts instead: The required C++ code was kept to an absolute minimum and in fact it's only needed to wrap the QML resources into a Qt/C++ plugin that can be loaded by our framework. If you want to write your own QML-based input method plugin for Maliit, then this is the required interface:

class MyQmlBasedInputMethodPlugin
    : public QObject, public MInputMethodQuickPlugin
{
    Q_OBJECT
    Q_INTERFACES(MInputMethodQuickPlugin MInputMethodPlugin)

public:
    MyQmlBasedInputMethodPlugin();

    //! \reimpl
    virtual QString name() const;
    virtual QString qmlFileName() const;
    //! \reimpl_end
};

The plugin's name is used for a specific GConf key. Once set to that name, the framework will load this plugin as the currently active one:

$ gconftool-2 -s /meegotouch/inputmethods/plugins/onscreen -t string "WhateverNameYouCameUpWith"

qmlFileName returns the (absolute) file name (or URL) to your main QML file. The framework will set up the required QML environment and try to load this file.

QML components can interface with the application through a set of properties and slots, tied to a root context named MInputMethodQuick. It exposes the following read-only Qt properties:

• int screenWidth,
• int screenHeight,
• int appOrientation (0, 90, 180, 270 with 0 being the typical landscape orientation and with 270 being the typical portrait orientation).

Additionally, QML components can invoke the following Qt slots:

• setInputMethodArea(const QRect &area)
• sendPreedit(const QString &text)
• sendCommit(const QString &text)
• pluginSwitchRequired(int switchDirection)

Please refer to the header file of MInputMethodQuick to access the full documentation (sorry, not available in MeeGo API docs).

Also, big thanks to Kimmo Surakka and Miika Silfverberg for all their help with this side project.

Self Compositing

Any QML plugin using MInputMethodQuickPlugin will support self compositing out of the box. That is, instead of letting the system compositor blend the application window and virtual keyboard window together into one, the virtual keyboard bypasses the system compositor and redirects the application's window to itself. This usually improves raw FPS but also cuts down on latency. Try running the MeeGo Input Methods service like so:

$ meego-im-uiserver -use-self-composition

This should result in a noticable speedup for your QML plugin. Self compositing is tricky, as it requires a well-behaving window manager (and believe it or not, mcompositor is the only one available getting it nearly 100% right). Usually an additional parameter combination of -manual-redirection, -bypass-wm-hint and -software will do the trick. For MeeGo 1.3, we want to activate self compositing by default, so it's good to know that your QML-based plugin will support this useful feature for free.

A QML-based virtual keyboard for Fremantle

As part of the exercise I also packaged MeeGo Keyboard Quick for Fremantle - it's available from Maemo's extras-devel repository. Be warned though, this is a rough tech demo and as such, fairly useless. A nice advantage of it is that it allowed me to package a MeeGo Input Methods version for Fremantle that is completely free of any MTF/libmeegotouch components.

How to testdrive:

• Install meego-keyboard-quick (after you convinced yourself that you really know what you do - activating extras-devel can brick your N900!),
• Install meego-im-demos

• Open a terminal, start the MeeGo Input Methods service:

  $ meego-im-uiserver -bypass-wm-hint -software -graphicssystem raster

   

• Open a second terminal, to run the meego-im-demos-plainqt application:

  $ export QT_IM_MODULE=MInputContext$ meego-im-demo-plainqt -fullscreen -software -graphicssystem raster

   

Instead of the test application, one should be able to use it in any other Qt application, as long as the MeeGo Input Methods service is running and the QT_IM_MODULE environment variable is set correctly.

To fight wrong expectations: I won't have the time to polish or to maintain this port, and from my first tests it would need undefined amounts of integration work to get it to run properly in Fremantle, starting with proper GTK+/Hildon support. If you are, however, interested in furthering this port, then please don't hesitate to contact us.

QGraphicsItem: When you're doing it wrong

When you read through the Qt GraphicsView documentation, you might miss the detail that QGraphicsItem::pos() refers to the center of an item: "Their coordinates are usually centered around its center point (0, 0), [...] At item's position is the coordinate of the item's center point in its parent's coordinate system;".

Oh great, so the word "usually" indicates it is not enforced, and everyone can choose his own semantics when implementing QGraphicsItems ... which is precisely what I did, accidentally. My QGraphicsItems' pos() refers to their top left corner. One chooses the semantics when overriding boundingRect() (which apparently uses an item's postion to map the bounding rect into the parent's space). So let's check the boundingRect documentation, whether it contains this hint. Hm, no. But it contains an example:

QRectF CircleItem::boundingRect() const
{
    qreal penWidth = 1;
    return QRectF(-radius - penWidth / 2, -radius - penWidth / 2,
                   diameter + penWidth, diameter + penWidth);
}

Damn, so the bounding rect does not start at (0, 0), even though it's in item's coordinate space ... I ended up introducing a policy for differentiating between graphics items who do it right (by following the Qt conventions), and for those that I created, for whenever I have to deal with item positions. The other possibility - to fix every item to follow the Qt convention - would have been too much work, sadly.

I wish I had discovered this earlier. (Extra rant: That's why too much documentation that is more about story telling than about being to the point is just as wrong as no documention.)

If generic programming beats OOP …

I've read a bit about the history of the C++ STL over the weekend (go do it, it's quite interesting), mostly because I think I am slowly arriving at a stage where I can form my own opinion about generic programming. The thing that stuck with me was "STL, at least for me, represents the only way programming is possible." and the almost famous OOP rant: "I think that object orientedness is almost as much of a hoax as Artificial Intelligence. I have yet to see an interesting piece of code that comes from these OO people." (credit for this goes to Alexander Stepanov, the mastermind behind the STL).

Well, if that's true, then the visitor pattern (I have no clue why the wikipedia version of this pattern is so wrong, and why it claims that you'd need callbacks and such; I refer to the original version of Gamma et al) must be easier to implement with templates than with interfaces. Please forgive that I use an easy example, but if Stepanov keeps using max and swap, and states those are "algorithms", then the visitor pattern is an algorithm, too.

With templates:

template<class T> void visitSomePlaces( const T &visitor)
{
    for (std::vector<CollectionElement>::iterator iter = collection.begin();
        iter != collection.end();
        ++iter)
    {
        // We can omit the "element must accept visitor" stuff - the check
        // can be performed by ::process just as well:
        visitor.process(*iter); 
     }
}

With interfaces:

class Visitor
{
public:
    virtual void process(CollectionElement &element) const = 0;
};

void visitSomePlaces(const Visitor &visitor)
{
    for (std::vector<CollectionElement>::iterator iter = collection.begin();
        iter != collection.end();
        ++iter)
    {
        visitor.process(*iter); 
     }
}

• The templates solution is shorter, +1.
• The templates solution needs to keep the implementation in the header and slows down compile time, +1 for interface solution.
• The templates solution works for any T, but will fail at compile time if ::process is not implemented. Same is true for the interface solution, but the interface solution clearly states this fact (imagine the template code was longer, and more complex: you'd first have to go through all of that to understand what T needs to implement (the compiler could help with that, if templates-related compile errors weren't so complex themselves). So slight advantage for the interface solution here, +0.5.
• Templates can't be virtual, which I find is quite restrictive. There's template specialization though, which can be used to work around this limitation. -1.

So it's 0:1.5, and the winner is: the interface solution. So why is it then that someone as knowledgeable as Stepanov puts down OOP as "technically, philosophically and methodologically unsound"?

Well, on the OOP side, those are the things that I really miss in C++ (from lowest to highest):

• loops etc. that are objects and as such can be easily reused as closures (can put lots of expressiveness in a few lines of readable, algorithm-like code!),
• traits (similar to mixins but without the need to be inherited),
• final classes (so important to get interface design right - people will otherwise try to inherit everything),
• virtual constructors,
• and yes, I'll say it: garbage collection (can save so much debugging time in bigger teams/projects).

Perhaps Stepanov made the analytical mistake to compare the (certainly interesting) generic programming features of C++ vs. its (not so modern) take on OOP?

Disclaimer: nope, not a C++ rant - I feel quite OK with it (or the bastardization of it =p) at the moment.

How to customize your view with delegates

The usual way in Qt, when an item view wants to render itself, is to ask the item model for everything, layout, style and the actual data. Storing view-specific layout information in the model itself means that views and models can always only exist in a 1:1 relationship. The logical conclusion: If you need to share a model between different views, you will probably have to add proxy models for each view. I wasn't satisfied with that idea, so I continued to research QStyledItemDelegates. They have the following nice properties:

1. They are owned by the view.
2. They have complete control over the cell they are assigned to (although that might not be too obvious).
3. They only get called if there's work to do, that is, nothing is wasted unless a cell becomes visible in a view.

So instead of having one proxy model per view, I can now keep the single model/multiple views approach by writing custom delegates for each view. For a tabular view, that means I want to install custom delegates per column or row (see Qlom's main window implementation).

How to change colors of a cell

Use the delegate's paint method, apply your changes to the QStyleOptionViewItem's palette and forward the paint request to the parent class. I had problems getting the correct background role, that is why I simply used the supplied painter to draw the background myself.

How to format data from the model in the view

The delegates' displayText method comes in handy. The model's data is wrapped in a QVariant already, so your custom delegate can apply all kinds of string formatting here. Just be aware that this method will never be called if the QVariant returned from the model (for the queried model index) is null.

How to replace a cell with a custom widget

For this, we abuse the fact that delegates are owned by their view. We only need to find a method that has a model index parameter, e.g., the paint method, and we are good to go! Inside that method, we query the parent() and use the item view's setIndexWidget method (perhaps check if there already is a widget at the given index, so that we don't end up re-creating widgets for every paint request).

In Qlom, the embedded widget is a simple button, so I was interested in its pressed signal. For that, I used a QSignalMapper to bind custom data (here: the model index) to the delegate's buttonPressed signal. Now the view can connect to the delegates' buttonPressed signal, unwrap the custom QObject to find the model index and display a nice message box with the exact model index of the clicked button.

Feedback and especially corrections welcome!

Fragile API and invalid iterators

For the Miniature project I sometimes need to iterate over all graphics items in a QGraphicsScene, or more precisely, all items of a specific parent item, the chess board itself. For that, I use the QGraphicsItem::childItems() API. However, when used with STL-style iterators you have to be careful, since value types of that form can easily break the iterator idiom:

for (QList<QGraphicsItem *>::iterator iter = parent_item->childItems().begin();
     iter != parent_item->childItems().end();
     ++iter)
{
    (*iter)->doSth(); // Ooops! Invalid iterator deref'ing! 
}

The value-type list that is returned by parent_item->childItems() has not been bound to a variable explicitly, so it is bound to an anonymous variable instead. The compiler will not complain here although the code is borken, from a C++ point of view. Worse yet, "(*iter)->doSth();" might work often enough for you to not see the crashes right away, depending on the compiled code and how the architecture handles memory.

Eventually though it will crash, and according to Murphy this will happen right after the big release. Valgrind of course would have complained about the illegal memory access right away (when run through that part of your code), because the list that we queried in the loop header is invalidated as soon as we enter the loop body, leaving us with an iterator that points into the void. This is C++-specific, in other languages even anonymous variables are only ever cleaned up after leaving the current block context, but in C++ their life time is only guaranteed for the scope of the current expression.

There is one fix and one workaround to that problem. First the fix: let QGraphicsItem::childItems() return a reference to a list member instead, or simply expose the begin/end iterators for the internal data structure directly. Assuming this is not possible, we are left with the workaround - bind the returned list to a variable explicitly, before the iterator is used:

QList<QGraphicsItem *> children = parent_item->childItems();
for (QList<QGraphicsItem *>::iterator iter = children.begin();
     iter != children.end();
     ++iter)
{
    (*iter)->doSth(); // Fine!
}

This of course prolongs the life time of the returned list unnecessarily. We only needed it inside the loop, but now it won't be cleaned up until the flow of control leaves the surrounding block context. To finally get the desired behaviour we could limit the surrounding block context:

{
    QList<QGraphicsItem *> children = parent_item->childItems();
    for (QList<QGraphicsItem *>::iterator iter = children.begin();
         iter != children.end();
         ++iter)
    {
        (*iter)->doSth(); // Fine!
    }
}

Perhaps a cleaner way is to move the loop code in a function of its own instead, although that might not always be feasible, depending on the code inside the loop body.

Another issue might be the need to dynamically cast QGraphicsItems (QGI) to the desired type, once you implemented your custom QGraphicsItem type. This wouldn't be too bad if there wasn't this horrible mix of QGraphicsObjects and QGraphicsItems in Qt 4.6 - some items inherit from QObject (QGraphicsSvgItem), others don't (QGraphicsPixmapItem). So now you also have to decide between dynamic_casts and qobject_casts, how nice ... not! Honestly, it would be easier here to simply forget about qobject_casts, but that might break in subtle ways.

These two issues were enough for me to avoid the QGI::childItems() API whenever possible. One possible replacement can be provided by signals and slots [1]: First, add a slot representing the loop body to your custom QGI type. Second, connect a signal to the instances of your QGI type that would have been iterated in the loop, that is, have a custom signal representing the loop iteration. Then, instead of iterating over the children of the parent item you simply emit the custom signal which triggers the "loop body" execution for each connected custom QGI instance.

[1] requires QObject inheritance for your custom QGraphicsItem type

About delegates and cell renderers - data formatting in Qt

Warning: the following blog post has a rant-to-usefulness ratio of 3:1.

Last week I needed to perform some data formatting on Qt list view. From reading the documentation alone I could not find a satisfying answer. When asking on IRC the answer was to use proxy models. This would have worked, but having two models for one view can create all kind of correspondence problems (think of sorting etc.). And there is this interesting bugreport, complaining that there is no easy way to format data. Status: rejected! So formatting should be a responsibility of the view? Wow, who would have thought ... however, if proxy models are a no-go we are left with ... custom delegates.

(cut some nonsense about MVC)

In the Qt world, a delegate is reponsible for 3 tasks:

• editing contents displayed in a view (possibly by creating new editor widgets on the fly),
• updating the model with the modified contents,
• rendering the contents in a view.

Somehow, that's two tasks too many, for something that is not a controller. That's where I prefersLet's take a look at GTK+ cell renderers: they only perform one task, and that's exactly what their names suggests(correction: actually, they are responsible for editing and updating as well, should have read more). A text cell renderer renders text, a pixbuf cell renderer renders pixbufs, and so on. Simple but tremendously flexible.

So how can I inject this flexibility into delegates? The QItemDelegate won't be very useful unless you want to use a QPainter for everything. But there is this styled item delegate, added with version 4.4 of Qt. And if we look at the roles & accepted types table we can see how this - together with displayText - could translate nicely into Qt "cell renderers". So we create custom styled delegates for each data type we need to display in our view: text delegates, pixmap^Wdecoration delegates ... Once we have defined a set of custom delegate classes we then request the view to use them on a per-row or on a per-column basis. A proof of concept can be found here.

EDIT: Had to correct some nonsense, this post was too much of a rant and I wasn't thinking clearly. What I originally wanted to express was that with QStyledItemDelegate we can have something very similar to GtkCellRenderer and use them both in a very similar way, too. I think the Qt documentation could have been easier to follow with straight and simple formatting data example, hence my rant. Sorry =/

Wt - a Qt-ish web toolkit written in C++

First of all, C++ for web apps doesn't necessarily make it easier to write them. Compared to other solutions (let's say Django) you'll end up writing a lot more code. And you have to be very very careful with memory leaks.

But that isn't the point. The main advantage of this framework is how close it is to desktop applications. Porting your Qt desktop application to the web is certainly easier with Wt than with any other solution, as you keep most of the widget API and also the signal and slots paradigm (which should allow to port the app reusing the same business logic as on the desktop).

Also, this is the first time I could attach a powerful debugger, namely gdb, to a web app and debug it is as if it were a normal desktop app. Together with the compiler-guaranteed type safety this is a huge improvement for code robustness.

A toolkit that actually uses C++

There are also some distinct advantages of this toolkit over Qt itself:

• no MOC preprocessing (unless you integrate it with your Qt libraries, of course),
• boost::signals & boost::bind wrapped in an easy-to-use WSignal, keeping most of the boost API accessible for the user. Which means: the compiler can check whether your signal connections will work!
• boost::any instead of QVariant: another big advantage. Boost::any is type-inferred using template voodoo so again, the compiler can check its correct usage for you. Getting values out sadly requires a (static) cast it seems.

Installation

The fastest way is to simply get the source from the git repo and to follow the instructions:

• (enter your jhbuild shell if your project happens to use any GNOME stuff)
• enter the cloned git repo
• "$ mkdir build"
• "$ cd build #dont ignore this, it helps later on!"
• "$ cmake -i ../ #interactive mode asks lots of stupid things but also various path settings!"
• "$ make"
• "$ make install"

For your own projects make sure to link against all the needed Wt libs: libwt (core), libwtext (if you want to use anything from Wt::Ext, see below), libwthttp (if you want to build apps with inbuilt web server, quite helpful). If you happen to use autotools then you might want to include AC_CHECK_LIB macros into your configure.ac. I had troubles finding non-name-mangled function names, but "$ readelf --dynamic --symbols /path/to/lib/lib.so | grep -v _Z | tail" should help.

ExtJS

Wt wrappes a highly advanced (in terms of bringing desktop UX to the web) Javascript framework - ExtJs. Sadly, it is also big (roughly 80kb have to travel to the client first) and very buggy. It would seem that you could use ExtJS and make it use jQuery instead, perhaps that's worth a try. For now I disabled ExtJS in my little project, since I couldn't debug some of its issues. The Wt-native widgets seem to be quite solid in comparison (even if they look a bit boring).

The installation instructions that come with Wt don't tell you how to install ExtJS, so here is what I found out:

• download ExtJS framework (you probably want version 3),
• copy it into your chosen wt webroot (let's say "/var/www/wt/"), consult your CMakeCache.txt,
• extract the zip archive,
• THEN copy /var/www/wt/ext/adapter/ext/ext-base.js to /var/www/wt/ext/ (the error message when starting the web server gave that hint away),
• make sure your project is linked against the libwtext library.

First steps with Qt

I started to look into Qt with the help of "Foundations of Qt Development" by Johan Thelin. As with its counter-part it shares the decent text and the sometimes not-so-convincing examples. Perhaps code examples from a book should generally be maintained like a real project (including bugtracker, mailing list/forum and public repository), since that allows the examples to improve over time.

Now, a personal highlight while reading the first chapters of the book was the screenshot of a paper'n'pencil design at the beginning of the second chapter. From there it continued with the derived use cases and then went on to explain the needed events and event handlers (signals and slots in QtSpeak) - a very convincing approach. However, the opportunity of introducing the reader to TDD was missed, and Qt unit testing is only mentioned late in the book (chapter 16). This actually implies that writing unit tests after writing the application code is OK. Nothing could be further away from the truth since testability is a design choice.

My first impressions of Qt are two-folded. There are already quite a few things I strongly dislike in Qt. Nevertheless I have to admit that it enables you to rapidly develop solid desktop applications, without boilerplating your code.