Category Archives: Computing

ListenerSet using variadic templates

Important: variadic templates are only available from C++11, make sure your compiler supports it.

Who has never used the Observer pattern? As long as you have been involved in any medium sized project, chances are you have come across it at some point.

The problem

It is extremely common to have an event generating system other components would like to subscribe to. However, oftentimes I see code to manage a collection of listeners being unnecessarily duplicated on a per system basis. That is registration, un-registration and notification. A lot of nonsensical boilerplate, which makes for code that is harder to read and easier to get wrong.

Let’s take a look at a typical input event dispatching system, simplified for the purposes of this article. We could have an IInputListener interface that handles a couple of events, keyDown() and keyUp().

struct IInputListener
{
    virtual ~IInputListener() {}
    virtual bool keyDown(KeyCode code) = 0;
    virtual bool keyUp(KeyCode code) = 0;
};

Our InputSystem class could hold an std::set of IInputListener pointers. Registration and un-registration is made possible through addListener() and removeListener() respectively. Bad news is that, every single time we want to send an event to the listeners, we are forced to iterate over the collection. Also, God kills a kitten.

class InputSystem
{
public:
    void addListener(IInputListener* listener)
    {
        m_listeners.insert(listener);
    }
    
    void removeListener(IInputListener* listener)
    {
        m_listeners.erase(listener);
    }

    void update()
    {
        // Touch down detected
        for (auto listener : m_listeners)
        {
            listener->keyDown(code);
        }
        
        ...
        
        // Touch up detected
        for (auto listener : m_listeners)
        {
            listener->keyUp(code);
        }
    }

private:
    std::set<IInputListener*> m_listeners;
};

As if this wasn’t enough, there is another important gotcha here. What happens when a listener un-registers as a result of an event? The m_listeners collection is modified in the middle of the for loop, thus current iterators are no longer valid. The second we try to increment the internal iterator to fetch the next listener…

BAM!

Not good…

The solution is simple but annoying. We can just add new listener registration and un-registration requests to a pending list while in the middle of a dispatch. Those pending lists would be processed once it’s safe to modify the collection of listeners.

Moreover, some people use std::vector instead of std::set for performance reasons, which is completely legit. However, that involves adding code to ensure listener uniqueness in the collection.

Every time.

Honestly? I’m lazy so I don’t want to be the guy who implements this over and over.

Okay, so what do you suggest?

The way towards the solution

Following our intuition, we realise that a way to generalise this behaviour is in order. Ideally, it would meet the following criteria.

  • Avoids all code duplication.
  • Ensures listener uniqueness.
  • It is safe, can register, un-register while notifying.
  • Avoids manually going through the listener collection to send an event.
  • Compatible with any kind of event listener.

Intuitively, we could have a ListenerSet<Type> template class that handles listener duplication as well as registration and un-registration safety.

Note: we’ll be using std::set rather than std::vector for simplicity. Both can be used as long as the appropriate precautions are taken.

template <class Type> class ListenerSet
{
public:
    ListenerSet() {}
    virtual ~ListenerSet() {}
 
    inline void addListener(Type listener)
    {
        if (m_notifying)
        {
            m_pendingAddition.insert(listener);
            m_pendingRemoval.erase(listener);
        }
        else
        {
            m_listeners.insert(listener);
        }
    }
 
    inline void removeListener(Type listener)
    {
        if (m_notifying)
        {
            m_pendingRemoval.insert(listener);
            m_pendingAddition.erase(listener);
        }
        else
        {
            m_listeners.erase(listener);
        }
    }
 
private:
    bool m_notifying;
    std::set<Type> m_pendingRemoval;
    std::set<Type> m_pendingAddition;
    std::set<Type> m_listeners;
};

We’re missing the notification functionality, that’s the tricky part. We would like to have an interface such as to be able to do something like this.

m_listenerSet.notify(&IInputListener::keyDown, keyCode);

ListenerSet::notify() would iterate over all the registered listeners calling IInputListener::keyDown and passing keyCode as a parameter.

Maybe easier said than done?

True, ListenerSet::notify() needs to support an arbitrary number of arguments, as different event handlers won’t necessarily have the same signature. Moreover, IInputListener::keyDown(and every other potential handler) needs to receive a this pointer along the remaining arguments.

This is where variadic templates come into play. Basically, they’re templates that can take an arbitrary number of parameters of any type.

Interestingly enough, std::bind() also takes an arbitrary number of parameters. That means we can create an std::function> object that has everything we need: the event handler function pointer, this and the remaining arguments.

Take a look at the code.

template <class Function, class... Arguments>
inline void notify(Function&& f, Arguments&&... args)
{
    m_notifying = true;
    for (Type listener : m_listeners)
    {
        auto callback = std::bind(f, listener, args...);
        callback(listener);
    }
    m_notifying = false;

    for (Type listener : m_pendingRemoval)
    {
        m_listeners.erase(listener);
    }

    m_pendingRemoval.clear();

    for (Type listener : m_pendingAddition)
    {
        m_listeners.insert(listener);
    }

    m_pendingAddition.clear();
}

Note how we process the pending requests to add or remove listeners after we finish sending the events, that’s when it’s safe.

Now we can do exactly what we wanted!

ListenerSet<IInputListener*> listeners;

listeners.addListener(new PlayerInputListener());
listeners.addListener(new UserInterfaceInputListener());

m_listenerSet.notify(&IInputListener::keyDown, keyCode);
m_listenerSet.notify(&IInputListener::keyUp, keyCode);

Templates are instantiated at compile time, which means the compiler will complain if we’re doing something dodgy such as trying to bind the wrong thing. Compile time checks are good.

Managing a set of listeners and sending events is now a lot easier, safer and readable. Last but not least, it’s significantly less error prone.

Hooray!!!

Full source

For those interested, I’ve made the code available as a GitHub Gist.

ListenerSet implementation

Contribute back to open source projects

Developers love Libgdx, it’s an extremely efficient, easy to use, open source, feature rich, cross platform framework. It has a huge community, and a very active repository. What’s not to love? The fact that 1.40% apps on AppBrain use Libgdx backs that up.

1.40 of all AppBrain apps are made with Libgdx

Have you thought about contributing back?

Who, me?

Yes and here’s why.

So much win

The beauty of a healthy community driven project such as Libgdx is that decisions are crowd-sourced and code is always peer reviewed. Although this may feel intimidating, don’t let it put you off, take it as the fantastic opportunity to learn that it actually is. The most active developers over there are talented, very talented. That’s the kind of people you want to hang out with, just so eventually, you end up absorbing some of their skills.

Personally, that’s what I do.

Some pull requests contain interesting discussions about software design and programming.

Low barrier of entry

Regardless of your level of expertise, there surely is something you can do. From maintaining the documentation to fixing that annoying old bug or introducing a small feature. All contributions are welcome. Actually, even the tiniest thing will bring you closer to the codebase and lead you to a better of understanding of it.

Libgdx pull request

Everyone makes mistakes, and when that happens, people will politely point it out, giving the contributor a chance to fix it and resubmit. The community is understanding and I haven’t seen any attempts to bring shame to anyone.

This is not a League of Legends game.

Unfortunately, this is not the case for every project. We’re lucky Libgdx is full of love.

This is all lovely but what’s in it for me?

Portfolio, exposure and recognition.

At least, do it for yourself

Even from a purely selfish standpoint, there are pretty strong reasons to get involved with open source projects. Even more so if you’re a student or will be looking for a job in the foreseeable future.

Yes! I want to get into the industry!

Sorry to break the bad news for you, once you finish university you’ll be out in the wild with thousands of other employment hungry graduates. They tend to complain no one will hire them because of their lack of industry experience. See the vicious circle? Luckily enough, that’s nothing more than a bad excuse in the software industry, where the requirements to ‘make it yourself’ are negligible. All it takes is a computer with an Internet connection, which I’m pretty sure you already have.

Working on personal projects that you might eventually open source is fantastic and will make you stand out. However, collaborating on a big project with other people reflects so much better on you. Companies need people to jump into projects and get the hang of it ASAP. That involves being comfortable with gigantic codebases written by people you don’t know or just met and being able to identify as well as fix the problems within it.

A big open source project gives you that education, for free. Companies will appreciate that enormously.

I’m experienced and looking for a job.

No doubt other similarly experienced people are seeking similar positions right now. You will compete against them in the hiring process. What’s going to set you apart? Exactly, going the extra mile, possibly through open source projects, being active in a community, blogging or public speaking… You get the hang of it.

How to

I guess if you’ve read this far, we agree to some degree and you might be interested in getting involved.

Good!

In order to start contributing back to Libgdx, it’s advisable to read the Contributing wiki page. Later on you should do the following.

  1. Clone the repository
  2. Check the issues and feature requests section
  3. Work on it
  4. Send a pull request
  5. Rinse and repeat

When in doubt, you can always ask in the forums or discuss in the #libgdx IRC channel at irc.freenode.net.

Happy coding!

Using Ant behind a proxy

Many Java projects use Ant to automate their build process. Libgdx, for instance, is among them. One could possibly say Ant is the Java version of Make. Targets, dependencies and other settings are defined in XML scripts.

Downloading dependencies while your machine is behind a proxy might prove problematic. Luckily enough, you can specify your proxy settings inside the Ant script.

<setproxy proxyhost="host" proxyport="port" proxypassword="password"/>

Users behind a proxy would now be able to download dependencies with Ant. More documentation about the setproxy task is available inside the official user manual.

This comes from me trying to get the Libgdx development environment in my office’s desktop. We obviously have a proxy, which put this little bump on the road.

Smart enums in C++ or: “what is this madness?”

Needless to say, C++ is a glorious language. Sadly, the attention it gets on the blog is far from representative of my appreciation towards it. Despite it being my main language at work, I only tend to talk about Libgdx side projects around here. Well, enough is enough!

Truth be told, over the years, standard after standard, C++ has become a behemoth of a language. As Scott Meyers likes to say, it’s actually a set of languages.

  • C with classes.
  • C with classes and the STL.
  • All of the above and templates.

One of the beauties of C++ is that I can always learn something new about it, which almost makes up for its ever growing syntax complexity.

Let’s talk macros

Let’s talk about macros. Everyone knows about the preprocessor and the basic usage of macros, right?

#define PI 3.14
#define SQR(x) x * x

Actually, let’s talk about macros that take other macros as arguments.

Sorry what?

The problem

Let us imagine we define the following perfectly standard enum to represent kinds of animals in a farm.

enum Animal
{
	eAnimal_Dog,
	eAnimal_Cat,
	eAnimal_Cow,
	eAnimal_Count,
};

Our application can request a server a list of animals a given farm has. The server sends the animal names as strings, so we need to convert them to enum values. Alright, let’s define a function to do that.

Animal getAnimalFromString(const char* str)
{
	if (!strcmp(str, "Dog")) return eAnimal_Dog;
	if (!strcmp(str, "Cat")) return eAnimal_Cat;
	if (!strcmp(str, "Cow")) return eAnimal_Cow;
	return eAnimal_Count;
}

Err… That is not the most elegant piece of code you’ve ever seen, is it? We also might have to convert from an enum value to a string literal in case we want to send a report back to the server.

const char* getStringFromAnimal(Animal animal)
{
	switch (animal)
	{
	case eAnimal_Dog: return "Dog";
	case eAnimal_Cat: return "Cat";
	case eAnimal_Cow: return "Cow";
	default: return "None";
	}
}

That was horrendous enough, but just the thought of adding new species to the catalogue makes me shiver badly. Why not simply use strings to represent animal species? Enums are nice because they heavily reduce the domain of values a variable can have and prevent us from making silly typos everywhere.

There must be a better solution.

Higher order macros

To make things nicer, we’d need some sort of compile time mechanism that traverses the list of enum values generating the time consuming boilerplate code for us.

The preprocessor!

Take a look at this way of defining our list of animals.

#define ANIMAL_LIST(m)	\
	m(Animal, Dog)	\
	m(Animal, Cat)	\
	m(Animal, Cow)

ANIMAL_LIST is a higher order macro that takes a macro m as a parameter. Then it passes each one of our animal species to m.

The previous macro is not really useful by itself. Let’s make things slightly more interesting

#define SMARTENUM_VALUE(typeName, value) e##typeName##_##value,
#define SMARTENUM_DEFINE_ENUM(typeName, values) enum typeName { values(SMARTENUM_VALUE) e##typeName##_Count, };

SMARTENUM_DEFINE_ENUM takes the enum name and the list of values as parameters and defines an enum for us. It also appends the total count.

Note: ## is used by the preprocessor to concatenate parameters with other pieces of text.

When we write

SMARTENUM_DEFINE_ENUM(Animal, ANIMAL_LIST)

The preprocessor expands it to

enum Animal { eAnimal_Dog, eAnimal_Cat, eAnimal_Cow, eAnimal_Count };

Okay, that was a lot of gibber jabber for very little gain.

Hang on a minute, we can go further. Let’s write a small mechanism to convert an enum value to a string literal.

#define SMARTENUM_STRING(typeName, value) #value, 
#define SMARTENUM_DEFINE_NAMES(typeName, values) const char* typeName##Array [] = { values(SMARTENUM_STRING) };
#define getStringFromEnumValue(typeName, value) typeName##Array[##value]

SMARTENUM_DEFINE_NAMES takes an enum type and the list of values and generates an array with the string representations of our list values. Interestingly enough, the values in the array map directly to those of the enum. This allows us to write the third macro, which accesses the array at the right position to return the corresponding string literal for a given enum value.

Note: # is used by the preprocessor to convert a piece of text in a string literal.

When we write

SMARTENUM_DEFINE_NAMES(Animal, ANIMAL_LIST)

The preprocessor expands it to

const char* AnimalArray [] = { "Dog", "Cat", "Cow"}

So

const char* animalName = getStringFromEnumValue(Animal, eAnimal_Cow);

Will result in

const char* animalName = "Cow";

Aha!

Getting an enum value from a string literal is slightly trickier but still achievable. We’ve come this far, so it’d be a shame not to finish the job.

#define SMARTENUM_DEFINE_GET_VALUE_FROM_STRING(typeName, name)		\
	typeName get##typeName##FromString(const char* str)		\
	{								\
		for (int i = 0; i &lt; e##typeName##_Count; ++i)	\
			if (!strcmp(##typeName##Array[i], str))		\
				return (##typeName##)i;			\
		return e##typeName##_Count;				\
	}

#define getEnumValueFromString(typeName, name)	get##typeName##FromString(##name)

The first abomination generates a function that iterates over the previously defined array of names doing string comparisons until it finds the desired enum value. The second one is nothing more than a convenience macro to spare the user from remembering the preprocessor generated function name.

Now if we do

SMARTENUM_DEFINE_GET_VALUE_FROM_STRING(Animal, ANIMAL_LIST)

We can simply go

Animal animal = getEnumValueFromString(Animal, animalname);

Neat, isn’t it?

For future references

Now, every time we want to define an enum type that we need to convert to and from strings we’ll be lucky to be able to do the following.

#define CAR_LIST(m)	\
	m(Car, Fiat)	\
	m(Car, Ford)	\
	m(Car, Audi)

SMARTENUM_DEFINE_ENUM(Car, CAR_LIST)
SMARTENUM_DEFINE_NAMES(Car, CAR_LIST)
SMARTENUM_DEFINE_GET_VALUE_FROM_STRING(Car, CAR_LIST)

Voilà!

Admittedly, using macros like this doesn’t make for readable code and sometimes results in debugging complications. However, I’ve been able to debug my code normally in Visual Studio 2012 when using similar macros.

String conversion of enums is just the tip of the iceberg, this approach can be used to generate all sorts of repetitive boilerplate code.

Download it

In case you are interested, SmartEnums.h is available as a GitHub gist.

Note: this post was inspired by the article Higher Order Macros in C++

Don’t be a dick with the community

Open source is sometimes much more than just coding and committing to the most trendy repository at that point in time. It’s also about the community. When a piece of software grows, stops being a pet project and makes a certain impact, it is to be expected to see people flocking around it. Who knows, even other developers could start contributing via pull requests, documentation or support in your forums.

Awesome!

Indeed, but…

Well, sometimes projects can grow too much, its main developer may be trapped under a massive pile of work or simply doesn’t feel like investing so much time in it anymore, which is fair enough. However, some authors adopt quite an aggressive approach when it comes to dealing with their community and that is not so okay.

Hostile communities

Earlier this week a fellow programmer and countryman sent a pull request to Tiny Tiny RSS. Interestingly enough, that’s the Google Reader alternative I talked about less than a month ago. Sadly, he was received with rude comments and a complete lack of appreciation for his efforts. Take a couple of minutes to read through the brief conversation and judge by yourself.

I don’t want to get into the details of the pull request itself, but lets be over generous and assume my friend didn’t approach the initial problem the best way possible. Regardless of the quality of the patch, there’s no reason to drop all common courtesy except for, perhaps, enhancing, an already large ego.

Typical environment in a hostile community

Typical environment in a hostile community

Making this all about this particular case would be a bit ugly and, since there is a much obvious and better example, let’s bring it up. Linus Torvalds and the Linux Kernel! There are countless mailing list entries with serious insults coming from the controversial programmer aimed at contributors, who, in most cases, are renowned professionals. Surely he is an incredibly busy man and his project is titanic. Nevertheless, excuse me for saying that not even such situation legitimises being a dick.

It’s really not that hard to be polite

It goes without saying that a much better response for these situations be something like what follows.

Thanks for the pull request! However, I believe X is not an optimal solution cause of Y. If you could rework it and update the commit later, that’d be ace. Otherwise, we’ll try to address the underlying issue as soon as possible.

Even this is fine.

Sorry it took so long to review this. I’m sorry, I can’t accept this patch because of Y. This topic has already been discussed in the forums, please query the search tool before tackling a problem in the future. I’m too busy to fix this myself, feel free to try again.

See? That wasn’t too hard, was it?

Obviously, this guy, or anyone for that matter, can do whatever he pleases. The matter of whether his community would prosper under this environment or not, is a whole different story. I would imagine, a minority of them just don’t give a monkey’s. Ultimately, I guess some people are simply difficult to deal with. However, if you want to work in a community, I think it’s good to be aware of good practises, generally common sense and politeness work.

Healthy communities

Fortunately, there are plenty of cheerful and healthy communities. Some more than others, but my point is that they exist. I can’t say my experience with crowd powered open source development is great but I’ve been involved in a couple projects. As some of you may know, one of them is libgdx.

Libgdx’s forums, IRC channel and Github repository are full of friendly folks. As opposed to the previous examples, I’ve never seen a poisonous comment nor an insult. Actually it’s quite the opposite, for instance, all pull requests are nicely handled, even if they’re incorrect. People are constantly posting extensions, showcasing their work and the acid humor that floods the IRC is just brilliant.

Typical atmosphere in a healthy community

Typical atmosphere in a healthy community

That kind of atmosphere is truly splendid, as it encourages people to be more active. Ironically, the hazardous environment of hostile communities scare newcomers off because they think they’d be attacked as soon as they participate. Sadly, their negativity gives the open source world its current reputation of being swarmed by nerds with gigantic egos.

So what’s the moral of this story?

Don’t be a grumpy person, don’t be a dick, be nice to people. Also, on a side note, don’t let an unhealthy community knock down your morale.