Sunday, 3 September 2017

The requirements of good matchmaking

Matchmaking is a highly important part of most modern multiplayer games. Build it badly and the user might be stomped by players way too good for them, get a really laggy connection or wait forever to get into a match. It's also a difficult topic to analyse and test before launch because there are so many unpredictable factors. For example, how are your players spread out over the globe? How much do match durations vary? How long are players willing to wait? How good are their internet connections? All of these and many other factors influence whether any given matchmaking implementation actually works well. Surprisingly few developers have written about the matchmaking in their games in detail, so I figured it would be useful to share our experiences. Over the years we've had several different approaches to matchmaking in Awesomenauts, and have iterated live on each of those, so let's have a look at what we've learned so far!



Matchmaking is such a big and complex topic that I'm going to write a series of posts about it. Today I'm going to kick off this topic by looking at the requirements of "ideal" matchmaking. In future posts I'm going to explore the actual matchmaking implementations we've used and their pros and cons. This will start with the standard matchmaking systems that the big platform manufacturers provide (like Valve, Sony, Microsoft and Sony), followed by an analysis of why we switched away from that and how we built our own. I'm also going to dive into some more controversial topics, like the psychological side of matchmaking and how we managed to massively reduce the percentage of people who leave a match before it finishes.

What defines good matchmaking? This depends on the game of course, but usually the matchmaker will try to achieve most of the following things:
  • Low waiting times to get into a match.
  • Low ping.
  • Match players of similar skill.
  • Match premades with premades.
  • All players in a match start at the same time.

Many additional requirements are possible, such as:
  • Match players who speak the same language.
  • Punish bad apples by matching them with each other (for example leavers and griefers).
  • Don't get matched with the same people too often.
  • Gameplay variation: maybe you don't want to be in the same kind of matchup every time.
  • Match players who use voice chat with others who also do so.
  • Match based on team composition (if one team has a rare team composition then maybe match them with others who also don't have the standard set of tank/healer/harasser/etc).

It's rarely possible to achieve all of these requirements simultaneously. Even if your game is a big success and 100 people or more search for a match every minute, then some requirements might still not be doable with acceptable waiting times. For example, if you want to match Australians only with other Australians, then waiting times deep in the Australian night will be extremely long for all but the biggest games. What works for a game with a playerbase as large as League Of Legends or Call Of Duty doesn't work for most other games.



I've previously written a blogpost about how many players you need for quality matchmaking and the short summary is that 1000 concurrent players isn't much from a matchmaking perspective. However, it is for most games: especially indie games are considered a hit if they reach 1000 simultaneous players every day. Even if you're convinced your game will be an indie hit you still need to think long and hard about how to make the matchmaking work with that relatively small playerbase. And this isn't limited to indies either: even a big triple-A multiplayer production like Lawbreakers currently has only a few hundred simultaneous players.

Because it's usually impossible to achieve all matchmaking goals simultaneously, you need to think about how important each of them is. Especially waiting times suffer when you make other requirements more important. The more requirements you can ignore, the better you can do on the other ones. For example, in Awesomenauts we've chosen to ignore ping with teammates: the matchmaker only looks at ping with enemies. This is because you can't hit your teammates anyway, so if there's more lag there then that will be much less of a problem. In an ideal world we would also want a low ping with teammates, but in practice we can achieve better ping with enemies if we ignore ping with teammates, and thus in our case this is a good choice to make.

One important thing to keep in mind when designing a multiplayer game is that adding more gamemodes often means splitting the playerbase and thus increasing waiting times or reducing matchmaking quality. Think long and hard before adding an additional matchmade game mode!

Reducing the matchmaking requirements isn't the only way to reduce waiting times. For example, if you can design your game in a way that allows for drop-in-drop-out, then players will usually be able to get into a match right away. Or if that's not possible, then maybe if rounds are short enough players can spectate a round while waiting for it to finish and then play in the next one. This doesn't reduce the waiting time, but it does make waiting a lot more enjoyable. I've previously written a blogpost that explores a bunch of different approaches to designing a game so that multiplayer works with a smaller playerbase.



Since this series of posts is for a large part based on our own experiences, I will mostly talk about matchmaking for MOBAs. In other words: real-time team-based games where skill is extremely important, where matches take relatively long and where the same players should play a match together from start to finish. In a sense MOBAs are amongst the most demanding games in terms of matchmaking requirements.

It's important to realise that other types of games have different requirements. For example, in shooters skill is often mostly ignored. Instead, teams are reshuffled after every round to balance things out. Shooter players usually also expect to get into a match really quickly, while MOBA players are mostly okay with waiting a couple of minutes for a better match-up. Of course this varies from game to game: not all games in the same genre have the same matchmaking requirements.

In the end the most important thing is to take matchmaking seriously. Think about the requirements of your game and don't assume they are the same as for another game. In some cases you might even have to redesign your multiplayer experience to make matchmaking work, maybe by removing some game modes or adding something for players to do while waiting for matchmaking. Matchmaking can't be an afterthought if your multiplayer experience is to thrive!

Saturday, 15 July 2017

What we learned improvising a live game soundtrack

Recently I've done a couple of music performances that were quite different from any live game music performance I've ever seen. We let someone play the games Journey and Ori And The Blind Forest live on stage and we improvised a completely new soundtrack on the spot, reacting to whatever the player was doing. This was a unique experiment for us and we learned a lot about what works and what doesn't, so today I'd like to share our approach and experiences.



Usually when game music is performed live the original soundtrack is replicated by musicians on stage. We however completely ignored whatever the original soundtrack had sounded like and improvised based on what we saw. The resulting music is different with each performance and sounds nothing like the original soundtrack. The fun of improvisation is that it's entirely in the moment. You don't know what's going to happen and it will never happen in the exact same way again. A truly live experience! Sometimes we might make mistakes, but sometimes we might also improvise the most intensely awesome music. Doing this in response to a game that's being played live is really exciting! Here's a video compilation of our performance at Animecon in the Hague last month:


Some people might know my game Cello Fortress. While there are similarities between Cello Fortress and our live game soundtracks, they really are very different. In Cello Fortress I control the game by playing cello. The cello is a game controller and the game is designed around this unique input device. On the other hand, the live game soundtracks in this blogpost are for existing games: the music doesn't control the game at all. We just respond musically to whatever happens on the screen. This is a bit less extreme than Cello Fortress, but as far as I know it also hadn't been done before and it's a different, interesting approach.

Before I continue, a short word about who 'we' are. The original idea for this performance came from Bram Michielsen and Zuraida Buter, who asked us to improvise live music to Journey at the Screenshake festival in Antwerp in 2015. After that we did a couple more performances and added Ori And The Blind Forest to the mix. I did these performances together with Rene Derks, who played djembe, keyboards and pads. Rene is a lecturer in game design and production at the NHTV in Breda and previously worked on Loco Roco 2 and Gatling Gears. I myself played cello and in my daily life I'm lead programmer at Ronimo Games, mostly known for our games Awesomenauts, Swords & Soldiers and the original version of the student game De Blob. We're not professional musicians but music is a rather serious hobby for both of us.

Improvising interesting music is difficult, and doing so together with other musicians even more so: whatever you're playing needs to not just sound good on its own, but also sound good with what the other musicians are playing. This means you can't just suddenly change the rhythm or mode unless you have a lot of experience improvising together. The specific combination of djembe and cello helps a lot here. It's only two instruments and they both carry a very clear responsibility: the djembe is in charge of rhythm and the cello is in charge of melody.

That gives me total freedom to change the mode or chord without needing to worry that this will sound bad with the other instrument. This kind of freedom is extra important here because we need to respond not just to each other and the audience, but also to the game. It also leaves room for great accidentals: a fast djembe rhythm can work great with slow cello notes and vice versa, so as long as we change intensity at the same moment it's okay if we don't always agree on whether we're increasing or decreasing the intensity.

When preparing for our first gig we started by choosing specific parts of each game, looking for areas with enough of an arc to make them interesting musically. The part of Journey we chose starts relaxed in the desert, then goes down into a darker valley with machinery, then has a very fast descent and finally goes through a dark, scary area that ends in a climax as the player flees from flying monsters. This gives us plenty of room for variation and for having an emotional arc in the music.



The next step was to figure out the general mood we wanted in each area and to come up with some ideas for the kinds of things we could do there musically. We looked up some longplay videos of Journey and improvised over those. We didn't actually write any music but we did end up with a good idea of what kind of things would work were. We also decided where Rene would play djembe and where he would use atmospheric synthesizer pads. In some cases we defined rather specific musical ideas, but in most it was just general directions. For example, in the haunted caves in Journey we didn't have more than this idea: "the caves get low dark dissonant cello and the djembe keeps silent in the beginning and joins after a while."



We wanted to make it very clear to the audience that we were truly responding to the game and not just playing pre-composed music, so we looked for some player actions we could respond to. Journey turns out to be great for this: jumps can take as long as 10 seconds so they are long enough that we can respond to them musically. During a big jump I might for example play an upward flurry on cello, hold a long sliding note during the fall and then get back into the rhythm as the player lands on the ground. Musically it feels slightly childish to play an upwards line as the player jumps upward, but audiences responded very well to this. We could clearly hear in their responses that these were the moments where they truly 'got' it.

To tie it all together musically and to have some kind of theme that the audience can recognise and remember we choose a couple of events for which I wrote small melodies. This is very much like in Zelda, where a short melody plays whenever you open a chest. In Journey we chose the finishing of each level as the key moments and in Ori And The Blind Forest the opening of doors. I quite liked the melody I came up with for Journey so I've also reused it as an attack melody in Cello Fortress.


This kind of performance isn't just about the music, but also about the person who sits on stage playing the game. Figuring out what's best here was a challenge. On the one hand the most pure expression of the concept would be to have a random player from the audience play the game without any preparation. However, this is risky since the player might not know the game and get stuck or need to retry an area a dozen times. That's going to be really boring to watch. On the other hand a prepped player feels like faking it. So we ended up making sure we had a player who knew the basics of the game, but who hadn't played with us before so that we both didn't know what was going to happen exactly.

This worked well in general but during one performance it backfired. The player got stuck in a dark cave in Journey and walked around aimlessly for minutes. It's difficult to improvise anything good for that and I saw some people from the audience leave during that section. I think next time it might be better to look for a player who knows the game better and accept that the player might rush through some sections. That's probably more fun than watching someone be stuck, trying to figure out what the jump button is or looking in every nook and cranny to find every last item...



While Journey is a slow game in which we could respond to individual actions by the player, Ori And The Blind Forest is way too fast for this kind of approach. For example, combat often takes only 2 seconds. We tried switching to combat music every time an enemy was encountered, but the music alternating so much just didn't sound good. Music needs to flow. So instead we decided to respond to individual events very little and mostly just have different types of music per area. The player can quickly go from place to place but generally stays in an area for at least 30 seconds so it's okay if the music changes whenever you go in or out of an area.



In Ori We also took a more melodic and less improvisational approach: Rene played keyboards here and we defined most of the chords and arpeggios beforehand. This works, but it felt a bit too forced to our liking. Our goal with this project was to truly improvise and after doing a couple of performances especially Ori felt a bit stale.

Doing these performances was really exciting and we've learned a lot about what works and what doesn't. Our next experiment with this idea is going to have a different structure. At the Abunai convention we want to try a smaller, more cosy setting where we don't decide beforehand which games are going to be played. Instead we're just going to bring a list of suitable games and let the audience decide whatever they want to play and we'll improvise to that. If someone happens to be carrying their own Switch they could even hook that up and play their own games! This means we can't prepare beforehand, which will probably result in less polished music, but at the same time it will be much more spontaneous so I expect more fun, weird things will happen. I also hope that not being on a stage will make the interaction between the audience, the players and us more direct. I'm really looking forward to trying out this new experience!

Monday, 10 July 2017

The Ronimo coding style guide

Last week I showed the coding methodology that we use at Ronimo, which describes our workflow. This week we'll have a look at what our actual code looks like, which is defined in our coding style guide. The idea behind our style guide is that if all code is formatted in a similar manner then it's easier to read and edit each other's code. For example, reading a different bracing style or naming convention usually takes some getting used to, and we avoid that altogether by having a strict style guide that all programmers at Ronimo must follow.



I haven't seen a whole lot of style guides from other companies, but from what I've heard our style guide is quite a lot stricter than what's common elsewhere. I'm not sure whether that's true, but I can totally imagine it is since I'm known for being precise (sometimes maybe overly much). Our style guide isn't set in stone though: there's an exception to every rule. If our style guide really doesn't make sense in a particular situation, then it's okay if a coder ignores it somewhere. Just as long as there's a good reason for it.

Some of the choices in this document are quite arbitrary. Sometimes alternatives would have been equally good, but without clear choices you can't have similar formatting for all programmers. This is especially true for bracing style. I know that this is a heated subject and while I have a clear preference, good arguments can be made for alternative bracing styles. (Still, it would have been nice if advocates of the other major style wouldn't have called it the One True Bracing Style... ;) )

A key element in our style guide is that I want code to read like English wherever possible. Variable and function names should be descriptive and only the most commonly known abbreviations are allowed. Brevity isn't a concern of mine, readability is.

Not all points in our style guide are about formatting though. Others are about actual language constructions. C++ is a rich language with an enormous amount of possibilities, but quite a few are too confusing or have too much risk of bugs to actually use. For example, nesting ternary operators is totally possible in C++, but the result is rarely readable so we disallow it altogether.



Our style guide also contains some rules that are intended to make cross platform development easy. On consoles you usually can't choose your compiler, so you have to work with whatever Nintendo, Sony or Microsoft have chosen, including the limitations of their compilers. We've researched what features of C++ each supports and have forbidden some of the newer C++ constructions that we think might not work on one of the consoles. Since we're not currently actively developing on some of the consoles we went by documentation only though, but I'd rather be too strict here than too lenient.

Another thing you can see in our style guide is my dislike for complex language constructions. C++ allows for some highly impressive stuff, especially using templates and macros. While I appreciate that these tricks can sometimes be really useful, I generally dislike them whenever they become too difficult to read. In the rare cases where these tricks are truly needed they are allowed, but usually I prefer if complex language constructions are avoided.



One particularly hotly debated point in coding styles is whether to mark class member variables. If the Car class has a float speed, do we call that speed, mSpeed, _speed or something else still? I've chosen to simply call this speed. Here too the reason is that I want code to be as similar to English as possible. The more prefixes and underscores there are, the further it moves away from natural language and the more difficult it becomes to just read code and understand it like you would text.

However, there's a good reason many programmers mark their member variables: it's very important in code to know whether a variable is a class member, a function parameter or a local variable. This argument is true, but I think we have that covered elsewhere: our style guide contains limitations on how long a class or function can be. If a function is short and fits on one screen, then it's easy to immediately see where variables are coming from. I think if classes and functions are short enough, then markers for member variables aren't really needed.

Note by the way that the rule for how long functions and classes can be is the one broken most internally. Sometimes it's just really difficult to split a class or function in a neat way. In the end the goal of our style guide is to produce clear code, not to hinder that by forcing clumsy splits. Still, there's real skill in figuring out how to neatly split classes and functions into smaller, more maintainable units, so if you're not super experienced yet then more often than not a neat split is possible and you just don't see it. In my opinion the ideal size of a class is anywhere between 200 and 400 lines, but a rule that strict isn't feasible so what's listed in the style guide is more lenient.

Now that I've discussed the reasoning behind our coding style guide, let's finally have a look at what it's actually like!

The Ronimo Coding Style Guide


There is an exception to every rule. However, keep to these rules as much as possible to maintain a constant layout and style for all the code. A lot of this is taste and a constant code layout requires setting aside one's own taste and keeping to these rules. Once used to it, it's easier to read such code.

When working in another language than C++, try to keep as close as possible to the C++ coding standard, but of course within reason. There are some specific notes on C# at the bottom as well.

C++


  • All code and comments are in Great Britain English. Not American English. So these are right: colour, centre, initialiser. These are wrong: color, center, initializer.
  • Every comma is followed by a space, for example doStuff(5, 7, 8)
  • Spaces around operators, for example: 5 + 7 instead of 5+7
  • Tabs have the same size as four spaces. Tabs are stored as tabs, not as spaces.
  • Functions and static variables are in the same order in the .h and .cpp files.
  • Use #pragma once instead of include guards (we recently switched to this so you will still see a lot of old include guards in our code).
  • Try to make short functions, preferably no more than 50 lines.
  • Avoid making very large classes. Split a class whenever you anticipate the class will grow a lot and you can cleanly split it. In general, try to keep class sizes below 750 lines. However, don't split a class if the split would result in messy code. Some examples of messy divorces are: too tightly coupled classes, friend classes and complex class hierarchies.
  • Don't write lines that are longer than what fits on a normal 1920*1080 screen (keep in mind that the Solution Explorer is on that screen as well).
  • When splitting a long line over several lines, make sure indentation is correct with the relevant parenthesis. For example like this:
    myReallyLongFunctionName(Vector2(bananaXPos + xOffset,
                                     bananaYPos * multiplier),
                             explodingKiwi);
  • Use forward declaration where possible: put as few includes in header-files as possible. For example, use class Bert; instead and move the #include "Bert.h" to the .cpp-file.
  • Includes and forward declarations are ordered as follows:
    • Start with all forward declarations from our own code (in alphabetical order)
    • Then includes from our own code (in alphabetical order)
    • One empty line
    • Then per library:
      • Forward declarations from that library (in alphabetical order)
      • Includes from that library (in alphabetical order)
  • The include of the .h-file belonging to a .cpp file is always at the top.
  • Don't define static variables in a function. Use class member variables instead (possibly static ones).
  • Everything must be const correct.
  • Names of variables and functions should be descriptive. Long names are no problem, undescriptive names are. Only use abbreviations if they are very clear and commonly known.
  • The first letter of classes, structs and enum type names is a capital. Variables and functions start without a capital. Each further word in a name starts with a captical. Don't use underscores ( _ ) in variable names. So like this:
    class MyClass
    {
        void someFunction();
        int someVariable;
    };
  • Member variables aren't marked with something like an m in front of them or an _ behind. Functions should be short enough to keep track of what is declared in the function and what in the class. Don't mark member-variables with this-> either.
  • Implementations of functions are never in .h-files.
  • Templatised functions that can't be implemented in the .cpp file are implemented in an extra header file that is included from the main header file. Such a class can have 3 files: MyClass.h, MyClassImplementation.h and MyClass.cpp. So like this:
    class MyClass
    {
        template <typename T>
        void doStuff(T thingy);
    };
    
    
    #include "MyClassImplementation.h"
  • Start template typenames with T. If more info is relevant, you can add words after that, like TString.
  • Several classes can never be defined in the same header file, except if a class is part of another class (i.e. defined inside the other class).
  • Between functions there are two empty lines (in the .cpp file).
  • Use empty lines to structure and group code for readability.
  • Add loads of comments to code.
  • Write a short explanation of what a class does above each class. Especially make sure to explain relations there (e.g. “This class helps class X by doing Y for him”).
  • Curly braces { and } always get their own line, so they aren't put on the same line as the if or the for. They are also never left out. The only exception is if you have lots of similar single line if-statements right beneath each other. In that case it's okay to put them on a single line. Like in this example:
    if      ( banana &&  kiwi && length > 5) return cow;
    else if ( banana && !kiwi && length > 9) return pig;
    else if (!banana && !kiwi && length < 7) return duck;
    else                                     return dragon;
  • When writing a do-while function, put the while on the same line as the closing brace:
    do
    {
        blabla;
    } while (bleble);
  • Indent switch statements like this:
    switch (giraffeCount)
    {
        case 1: text = "one giraffe";  break;
        case 2: text = "two giraffes"; break;
        case 3:
            // If it's more than one line of code
            doStuffOnSeveralLines;
            text = "three giraffes";
            break;
        case 4:
        {
            // Can add curly braces for readability
            int x = getComplexThing();
            text = "quadruple giraffe";
            break;
        }
    }
  • Function parameters have the exact same names in the .h and the .cpp files.
  • If a function parameter and a class member variable have the same name, then either think of another name, or add an _ to the end of the function parameter. For example like this:
    void setHealth(float health_)
    {
        health = health_;
    }
  • Precompiler instructions (everything starting with #) should be kept to a minimum, except of course for #include and #pragma once.
  • Don't write macros.
  • Variables inside functions are declared at the point where they are needed, not all of them at the start of the function.
  • In constructors, prefer using initialiser lists over setting variables in the body of the constructor. Each initialisation in the initialiser list gets its own line. Make sure variables in the initialiser list are in the same order as in the class definition in the .h-file.
  • Don't use exceptions (unless you're using a library that requires it).
  • Don't use RTTI (so don't use dynamic_cast). RTTI costs a little bit of performance, but more important is that RTTI is almost always an indication of bad object oriented design.
  • Use reinterpret_cast and const_cast only when absolutely necessary.
  • Don't commit code that doesn't compile without errors or warnings (and don't disable warnings/errors either).
  • Don't commit code that breaks existing functionality.
  • No global variables. Use static member variables instead.
  • Use our own MathTools::abs instead of std::abs. This is because std::abs is inconsistently implemented between platforms, causing hard-to-find bugs.
  • Always use namespaces explicitely. Don't put things like using namespace std in code.
  • Never ever even think of using go-to statements. We have a thought detector and will electrocute you if you do.
  • Don't use the comma-operator, like in if (i += 7, i < 10)
  • Don't use unions.
  • Don't use function pointers, except where other libraries (like STL sort) require it.
  • Only use the ternary operator in extremely simple cases. Never nest the ternary operator. Example of a simple case where it can be used:
    print(i > 5 ? "big" : "small");
  • When exposing a counter for an artist or designer it starts at 0, just like it does for normal arrays in code. Some old tools might still start at 1 but anything newly developed for artists starts at 0.
  • When checking whether a pointer exists, make this explicit. So use if (myPointer != nullptr) instead of if (myPointer)
  • Use RAII (Resource Acquisition Is Initialization) where possible. So instead of creating a separate function initialise(), fully initialise the class in its constructor so that it's never in an incomplete state.
  • Write the constructor and destructor together: write the corresponding delete for every new immediately, so you don't forget to do it later on.
  • If you add temporary debugging code, then add a comment with QQQ to it. Never commit code with QQQ: remove the debugging stuff before committing.
  • If you want to leave a marker for something that needs to be done later on, use QQToDo. If it's something you need to do yourself then add your letter to it, for example QQToDoJ. Only commit QQToDo if it's really impractical to finish it right away.
  • Class definitions begin with all the functions and then all the variables. The order is public/protected/private. This might sometimes mean you have to use the keywords public/protected/private several times in a single header file (first for functions, then again for variables).
  • A class starts with its constructors, followed by its destructor. If these are private or protected, then they should still be at the top.
  • When something has two options, but isn't clearly true/false, then consider using an enum class instead of a boolean. For example, for direction don't use bool isRight. Instead, use enum Direction with the values Left and Right.
  • Instead of std::string and std::stringstream, use our own classes RString, RStringstream, WString and WStringstream (these tie into our own memory management).
  • The variable float time always stands for the time since the last frame, in seconds. If time means something else, make this explicit, for example by naming it timeExisting instead.
  • Make sure all code is framerate-independent, so use the variable float time often to achieve this.
  • Make the intended structure explicit and clear. For example, avoid doing things like this:
    if (yellow)
    {
        return banana;
    }
    return kiwi;
    What was really intended here, was that depending on yellow, either banana or kiwi is returned. It's more readable to make this explicit like this:
    if (yellow)
    {
        return banana;
    }
    else
    {
        return kiwi;
    }
  • Use nullptr instead of NULL.
  • We only use auto for things like complex iterator types. For everything else we explicitly type the type.
  • Use range-based for where applicable, for example:
    for (const Banana& ultimateFruit : myList)
    (Note that it's important to type that reference when not working with pointers, since otherwise the Banana would be copied in this case.)
  • Use override and final wherever applicable.
  • If a function is virtual, the virtual keyword should always be added to it, so not only in the parent class, but also in each version of the function in the child classes.
  • Use strongly typed enums, so with the class word. Their casing is the same as for classes. For example:
    enum class Fruit
    {
        Banana,
        Kiwi,
        ApplePie
    };
  • Don't use rvalue references (&&) unless you really really really need them.
  • Use unique_ptr wherever possible. If an object isn't owned, then it's stored as a normal pointer.
  • Avoid creating instances of complex types in the initialiser list. Simple copying and setting goes into the initialiser list, while more complex code like calling new goes into the body of the constructor. Here's an example of how that works:
    FruitManager::FruitManager(Kiwi* notMyFruit):
        notMyFruit(notMyFruit)
    {
        bestFruitOwnedHere.reset(new Banana());
    }
  • Only use shared_ptr in cases where shared ownership is truly needed. In general, try to avoid shared ownership and use unique_ptr.
  • Lambda's that are too big for a single line are indented like this:
    auto it = std::find_if(myVec.begin(),
                           myVec.end(),
                           [id = 42] (const Element& e)
                           { 
                               return e.id() == id;
                           });
  • Don't use MyConstructor = delete (this doesn't seem supported on some compilers we might need).
  • Don't use the C++11 feature for list initialization (this doesn't seem supported on some compilers we might need). So we don't use this:
    std::vector<int> thingy = {1, 2, 3};
  • Don't use any features that were added in C++14.

C#

  • Variables at the top of the file instead of at the bottom. We do a strict split between functions and variables (like we do in C++), so here it's first all variables, then all functions.
  • async functions must always end their name with Async, for example eatBananaAsync.

That's it, our coding style guide! ^_^ While I imagine you might disagree with a bunch of the specific rules, I think it's useful for any company that does programming to have some form of a style guide. Our style guide can be a good starting point for creating your own. I'm quite curious: what's the style guide like at your own company and how do you like it? Do you have one at all?

Sunday, 2 July 2017

The Ronimo coding methodology

I'm a strong believer in working in a structured way. As a game programmer you need to build complex systems and just going with the flow isn't good enough. That's why I've written two documents to describe how we code at Ronimo, which every programmer and intern gets to read on their first day. Our Methodology document explains the workflow while the Style Guide explains the layout of our code. Today I'd like to show our Methodology and describe the reasoning behind the rules in this document.



Note that the contents of this document aren't very original: most if it is a combination of common agile practices that I like.

Let's start by having a look at the actual methodology document:

The Ronimo coding methodology


General method of implementing a new feature:

  1. Analyse what should be made. Discuss approach with end users (usually artists and/or designers) and lead programmer.
  2. Split into small tasks of a day at most.
  3. Make a basic planning for the small tasks. Put the core of the functionality and the most difficult parts to implement at the front of the planning.
  4. Implement each small task.
  5. Evaluate result with end users and lead programmer.
  6. Explain to end user what the new functionality does, so that it's actually used.

Implementing a small task:

  1. Analyse what should be made.
  2. Verify that related existing code is bug-free (perform tests!).
  3. Research and experiment with any new technology required for this feature.
  4. Code-design for new functionality. Don't focus too much on theoretical future use, but do keep it in mind.
  5. Refactor existing code to make room for your new feature.
  6. Test whether older functionality is still intact.
  7. Implement new functionality.
  8. Test whether new functionality works.
  9. Finish code: add comments, clean up code and check the destructors.
  10. Test whether new functionality still works.
  11. Test whether older functionality still works.
  12. Evaluate result. If showable, also show this intermediate result to the end user.

Various other rules:

  • Keep a personal list of all the small things that you should do. If you come across some issue that you can't immediately take care of, or if someone asks for a feature that you promise to build later, then write that down in your list. Don't think you can always remember everything.
  • Never continue to work on something if you have encountered a crash or major bug. Always fix the crash first.
  • "Premature optimization is the root of all evil" (Donald Knuth). First implement a new feature in the simplest / clearest way possible, then analyse whether it works, then analyse the performance and only if necessary implement optimisations.
  • Don't worry too much about making your code in such a way that unknown future extensions might be easy to add. When new functionality is needed, the code can still be refactored. Of course, if it's little work to do, then do make things as generic as possible.

Most of these are quite clear, but it's interesting to discuss some of the reasons behind these rules. Often enough I've seen coding interns have loose ends everywhere in their code because they were working on five things at the same time and forgot to test, clean up and finish some of them. That's why our coding methodology requires that you finish what you were doing before you get to the next thing. This is also why I require that big tasks are split into smaller ones: a person can only remember so much and the more things you're working on simultaneously, the bigger the chance that you'll overlook something important.

At the same time I prefer the agile way: only make what you actually need and expand the codebase as you go. During early development you don't know all the features you'll need, nor do you know for all problems how you'll solve them. However, adding things one at a time will often make code bloated and without focus, and will muddy class responsibilities. That's why code needs to be refactored often. Refactoring requires discipline. Often you can hack in a new feature in just a few hours, or first spend half a day refactoring to make room for that feature in a good way. It's easy to skip or postpone refactoring, but doing so often produces unworkable code in the long run. That's why refactoring is an explicit step in our methodology.

Focusing only on one small new thing at a time shouldn't be taken too literally though. While I think it's important to not work on other things until what you're doing is finished and clean, that doesn't mean you shouldn't look ahead. When building something complex it's important to think about how you're going to make the whole system work. I once had an intern who had to rebuild a large part of a tool because he had taken our coding methodology too literally and hadn't thought ahead at all. The most complex features of the tool were not possible at all with what he had build. The key here is to find the right balance between thinking ahead and focusing on one thing at a time.



Another staple of mine is that programmers need to communicate directly with designers and artists. We believe extensive design documents are rarely a good idea, so the only way to know what's needed exactly is to talk to the designer or artist who needs a new feature. Often that person hasn't defined the exact details of the feature, so the coder needs to discuss it with them and think about the caveats, also from a design and art perspective. To smoothen this communication it helps a lot if the programmer has a little bit of experience in design and art, but even if that's not the case I think it's the programmer's job to talk the language of the designer or artist, not the other way around. It's really difficult for a designer to speak code, but a programmer should be able to talk about his work in comprehensible English (or Dutch in our case).

One thing that's surprisingly missing in our Coding Methodology is unit testing. We have a strong focus on testing our own code extensively, but the document doesn't say you need to write unit tests. This is because I think gameplay is often too chaotic and unpredictable to test well with unit tests. Certain things are testable with unit tests, but the bugs we encounter are often not things where I can imagine how a unit test would have found them. Often it's things that function fine but result in undesired gameplay.

I do realise that not making units tests makes us more vulnerable to bugs than a team that always writes unit tests, so we emphasise that if a crash or major bug is found, it needs to be fixed right away. We might have more bugs than software developers who write extensive unit tests, but at least we fix them quickly. I do think we ought to use unit tests more for things like server architecture. Unit testing isn't in our blood at all and it probably should be at least a little bit. I'm curious though: do you use unit tests for your gameplay or engine code?

Regardless of whether you agree with the particular rules in our methodology document, I think it's important that all programmers think about their workflow. Just doing whatever you feel like doing isn't good enough. Discipline and structure are important for anyone who works on larger, more complex systems. What's you're coding methodology like? If you happen to work at a company, is there an official document like the one I've shown today?

So that's it, the Ronimo Coding Methodology! Next week I'll show our Coding Style Guide, which is quite a bit more strict than most coders are used to.

Sunday, 29 January 2017

The many meanings of the confusing word 'lag'

'Lag' is one of the most confusing words used in gaming: it's used to describe wildly different problems. This makes communication difficult: when a player is complaining about 'lag', what's his problem exactly? Is he talking about his internet connection, framerate or input delay? So today I'd like to list the different meanings of the word 'lag', both to reduce confusion and to explain why all of them are important. I've grouped them into three main types.

It can be difficult to know exactly what's going on, but it would already help a lot to always at least specify which of the rough groups you mean, instead of just saying 'lag'.

Note that this post is not specifically about our own games. These problems can happen in most games, although the exact impact depends on how the game was been programmed.

Internet and connectivity issues


  • High ping: ping is the time it takes for an internet packet to travel from your computer to another. In most games the gameplay still feels smooth with high ping, but you'll often start seeing oddities like getting shot by someone who isn't in view anymore, or your shots missing despite perfect aim. The faster the game, the more problematic high ping is.
  • High jitter: jitter is an often overlooked aspect of ping. Jitter is when some packets take longer than others. Jitter often causes stuttery movement in games. You might have low average ping but still experience stutters because of high jitter. (Packet loss often looks the same as an extreme type of jitter so I'm not listing it separately here.)
  • Pingspikes: this is when occasionally the connection becomes extremely slow or even drops altogether for a few seconds. In most games ping spikes result in serious problems, like your controls not working anymore for a while, getting a lot of damage at once at the end of the spike, or even being disconnected.

All of these problems can have many causes, like a slow internet connection or your roommate downloading a torrent over the same connection. There's one common cause that's relatively easily solved though: WiFi. WiFi increases ping, causes jitter and packet loss and, worst of all, pingspikes. If you can, never play online games over WiFi. Connect to your router with an ethernet cable instead. Check this blogpost from the League Of Legends devs to see how bad WiFi really is.

Delay between input and seeing the result


  • Input delay: the time between you pressing a button and the game actually processing it. Games usually process input only once per frame, so the higher the framerate, the lower the input delay. Drivers and the operating system will also cause a little bit of input delay, or a lot if it's a complex controller like the Kinect camera.
  • Input jitter: this is when the input delay varies. This can happen if you press the button just before the frame update and then just after. If the game runs at 30 frames per second, this can create a 33 milliseconds difference in input delay. The higher the framerate, the lower this kind of jitter wil be.
  • Rendering delay: once the gameplay has processed your input, it needs to be rendered. The GPU is usually at least one frame behind, and in some cases the game itself can be as well. For example, if you set the Multithreading Mode in Awesomenauts to Higher Framerate, then the graphics are always one additional frame behind on the gameplay, causing additional input delay. Again, the higher the framerate, the smaller this effect.
  • Monitor delay: this is usually the most noticeable cause of delay between pressing a button and seeing the result. Many televisions have all kinds of clever processing to make the screen sharper and smoother, but this can add over 100ms of lag. That's a lot and can be very noticeable. When using a television you should always switch it to Game Mode to solve this. Computer monitors generally don't have this problem.

Framerate problems


Low framerate is also often called "lag". Players sometimes don't realise that there are different types of low framerate. Being precise when describing your problem helps a lot if you ever ask anyone for help with how to solve your framerate problems.

  • Constant low framerate: this usually means the videocard or processor is not fast enough to run the game.
  • Occasional long stutters: the game sometimes freezes for half a second or more. This might be caused by problems reading from the harddisk, so switching to an SSD might help. In general though, a game should be programmed in such a way as to do disk reads asynchronously and thus not freeze like this, but sometimes this is really difficult to achieve.
  • Regular short stutters: every second or so the game skips a few frames. Such stutters are really short but can be very noticeable, especially during smooth camera movements. This might be caused by running something else on your computer (like a virus scan or a lot of tabs in your internet browser). It can also be caused by the game not balancing the work well enough between frames, causing some frames to take longer.

Framerate problems and input delay are both often decreased by turning off VSync. This does come at the cost of getting screen tearing though. Which you prefer is a matter of personal taste.

Sunday, 15 January 2017

Adding more depth to 2D levels using scissor rectangles

While creating levels for Awesomenauts and Swords & Soldiers 2 our artists often struggled with how to create more depth. All those background elements from one area would slide into view in other parts of the level, limiting how deep the backgrounds could be. For the Starstorm level in Awesomenauts I've added a trick to our editors that solves this problem: scissor rectangles. It's a neat little trick that not only makes Starstorm have backgrounds with much more depth, but also makes rendering them a lot more efficient. So let's have a look: what's the problem exactly, and how do scissor rectangles solve it?

In a purely 2D game like Awesomenauts there's no real perspective: backgrounds are just a lot of 2D textures. As the camera moves all those layers move at different speeds to suggest depth. Traditionally in games that's called parallaxing. From a technical perspective that's basically all there is to 'perspective' and 'distance' in a game like Awesomenauts.

However, this simple idea of objects following the camera creates a big problem. An object that is really far away hardly moves on the screen when the camera moves. The mountain in front of you in the far distance? Walk 10 meters to the left and it's still in front of you. Walk 100 meters to the left and it's still in front of you. It's so far away that it's basically always in front of you. That's realistic and as intended, but what if two rooms in your level should have different objects in them? If the rooms are really deep (like for example a factory hall) then the objects in the back of one room will be visible in the other.



In a 3D game the solution to this is simple: just put a wall in between these two rooms. But this is not a 3D engine. This is a purely 2D engine and walls that go into the distance are not supported. Heck, we don't even have a z-buffer to figure out what parts would be in front of or behind that wall! Also, what if we don't want a wall? What if we want to suggest really deep spaces but don't want to put endless walls into the distance in between them?

The solution our artists used in older Awesomenauts maps is that such deep rooms can't be close to each other. For example, in Ribbit IV the top part of the level is a lush jungle that continues far into the background. Originally my colleague Ralph also added deep caves in the bottom part of the map, but parts from the jungle became visible in the caves, and vice versa. So Ralph made the caves less deep. The jungle parts are still rendered behind the caves, but because the caves have a solid wall close to the camera the jungle isn't visible there anymore.



Not only is this very limiting for our artists, it's also a huge waste of performance: it causes a lot of overdraw. 'Overdraw' is when several objects are rendered on top of each other. The more objects are rendered on top of each other, the higher the overdraw is and the worse the framerate gets. Sometimes that's needed to get a certain effect, like when having lots of overlapping smoke particles. But here it's a waste: the jungle isn't even visible, so why render it? High overdraw is the number one performance drain in Awesomenauts, so not doing this would be quite wonderful. (For more on overdraw have a look at this post: Optimising Special Effects In Awesomenauts.)



I had been thinking about this problem for a while, mostly coming up with solutions that were quite a lot of work to build, when one day I realised that videocards have a very simple feature that solves exactly this problem: scissor rectangles! The basic idea of a scissor rectangle is that when rendering an object, you tell the videocard to only render the parts of it that are within a certain area on the screen: the scissor rectangle. Everything else is cut off. It's like a mask in Photoshop, but in-game.



Scissor rectangles are a very simple and very limited feature. They only do rectangles that are aligned with the screen. So you can't have a rotated rectangle, or a shape that's more complex than a rectangle. If you want any of that you'll need to resort to more involved tech, like the stencil buffer, which isn't efficient to recreate differently for each object rendered. I was looking for something simpler, something that could be changed for each object rendered without causing a performance hit. Scissor rectangles provide exactly that.



The next question is how to integrate scissor rectangles with our tools: how can an artist easily define which objects are cut off by which scissor rectangle? I chose to link scissor rectangles to our animation system. An 'animation' in the RoniTech (our internal engine) is simply just a bunch of objects that might be animated, so a part of a level can also be an 'animation'. I made it so that each animation can have a custom scissor rectangle, placed by the artist. The resulting workflow is that a level artist creates a room or area in the animation editor, adds a scissor rectangle and then puts the entire 'animation' in the level. It's an easy and fast workflow.

Our levels are mostly decorated by my colleague Ralph, using art drawn by Gijs. On the Starstorm map Ralph used scissor rectangles extensively for the first time and I love the result: he put a lot of depth and detail in the backgrounds and the framerate is higher than on other maps because the scissor rectangles reduce overdraw. Here's a short demo video:


For any programmers that read this, here are some implementation details I've encountered. Feel free to skip this list if you're not a programmer. ;)
  • The videocard wants to know about scissor rectangles in screen coordinates, while the way we use them means our tools place them in world space. To use scissor rectangles this way you'll need to convert the coordinates yourself before sending them to the GPU.
  • If you accidentaly swap the top and bottom of the scissor rectangle then everything will still work fine on AMD and NVidia GPUs, but on some Intel GPUs the objects will disappear altogether.
  • It's not allowed to define scissor rectangles that are partially or entirely outside the screen, so be sure the cap them to the edges of the screen.
  • The relevant functions in OpenGL are glEnable(GL_SCISSOR_TEST) and glScissor()
  • DirectX 9: SetRenderState(D3DRS_SCISSORTESTENABLE, TRUE) and SetScissorRect()
  • There are equivalent functions that we use on Xbox One and Playstation 4. I haven't checked on mobile but I expect scissor rectangles are a standard feature on practically all videocards.



So there you have it: scissor rectangles! Not only did they enable our level artists to make deeper, richer levels, but they also improved performance by reducing overdraw and were quick and easy to add to our engine.