Adding a Resource Manager, aka “I’m really sick of passing all my shaders through all the functions in my program”

At the end of my last blog post I noted that I had implemented some changes I’d not yet blogged about. I ended up deciding they were too small and disjointed to bother with a blog post. In short, those changes were mostly me just removing all the old code from my pre component approach. If you’re super interested you can have a look at the commit here.

After that I decided it was finally time to reintroduce text into the game. In my old implementation a Text class was a child of the Sprite class. I took some time evaluating whether that was the best approach – an alternative I thought about (and even started to code for a while) was having a Text class as a child of the GameObject instead, and just contain a SpriteComponent. However in the end I decided on a “FontComponent” that is a child of the SpriteComponent.

The FontComponent is rather similar to my old Text class, so I won’t bother to post the code. The only difference is I gave the user the option to change the colour of the text in the Draw call. What I don’t like about this implementation is that I have to create an empty GameObject in each Screen that needs text, just so I can add a FontComponent to it. And then I have to load the same font texture and xml at the beginning of each Screen. It would be nice to only have to load these once at the beginning of the game…

Anyway, I tested this and discovered that the font texture doesn’t actually use an alpha channel, just black space to represent what isn’t part of a character. Which looks ugly if the background behind your text is anything but black. No problem, right? I can just write a fragment shader that turns any black pixels into transparent pixels. So I did:

in vec4 colour;
in vec2 TexCoord;

out vec4 FragColor;

uniform sampler2D textureSampler;

void main()
{
    vec4 fontTexColour = texture(textureSampler, TexCoord).bgra;
    FragColor = fontTexColour * colour;
    FragColor.a = fontTexColour.b;
}

Great! Basically in that last line, if a pixel is black then any of b, g or r will have a value of 0. Which is exactly what I want the alpha to be.

OK, so all that was left was to create a new GLSLProgram using this fragment shader instead of my default one, and then… pass it through all my scenes and components like I was doing with my original shader program?

I could see this was not the way to go, and started doing some research. It quickly became clear (and seriously, how had I gotten this far without realising this) that I needed some sort of Resource Manager. I read through quite a few implementations, but I ended up creating my own one for now (which I think was probably influenced by a similar setup we had at my last job).

I decided that each type of resource (shader program, texture, audio clip, etc) would have it’s own ResourceStore which was basically a map wrapped in more descriptive function names. I’m still not very good at working out when it’s better to have contiguous memory – but since I never loop through these stores I think in this case it doesn’t matter (so std::map is fine in this case).

I decided to use enums for the keys in the ResourceStore. We used strings at my previous workplace, but string comparisons are slow, right? So an enum seemed like a much better idea. I store these enums in a header file which currently looks like this:

enum class ShaderResources
{
    DEFAULT_SPRITE,
    ARIAL_FONT,
    NUM_SHADER_RESOURCES
};

Not super exciting, but once I migrate my other assets into the ResourceManager it will have more in there.

Then I created a ResourceManager to take care of all these ResourceStores. This is a Singleton that contains a number of ResourceStores. I decided that the ResourceManager should be responsible for actually loading each Resource from disk (with the idea that at some point some resources might be separated into loading stages – so some resources are only loaded for a particular scene, while some might be global). As soon as I started writing the code to load the resources I discovered it would be a good idea to store the file path locations of each resource in a corresponding map.

I wrapped this in a class called ResourceLocationMap. Then in the Load function the ResourceManager loops through the ResourceLocationMap for each Resource and passes the path to a more specialised load function:

void ResourceManager::LoadResources()
{
    for (unsigned int i = 0; ShaderResources(i) < ShaderResources::NUM_SHADER_RESOURCES; i++)
    {
        m_shaders.AddResource(ShaderResources(i), LoadShader(m_shaderLocations.GetFilePath(ShaderResources(i))));
    }
}

GLSLProgram&& ResourceManager::LoadShader(std::string a_path)
{
    std::ifstream readMarker(a_path);

    GLSLProgram shader;
    char path[255];

    while (readMarker.getline(path, 255)) {
        switch (path[0])
        {
        case 'v':
            shader.compileShaderFromFile(path + 2, GLSLShaderType::VERTEX);
            break;
        case 'f':
            shader.compileShaderFromFile(path + 2, GLSLShaderType::FRAGMENT);
            break;
        default:
            break;
        }
    }
    shader.link();
    return std::move(shader);
}

I somewhat modified my shader loading pipeline too. I created a file that basically just stores the filepaths of all the shader program pieces. This is read in by my LoadShader function which then hands over the actual reading of the individual shader pieces to the ShaderLoader class.

So then I created an InitResources function (for now) that will probably end up a large list of all the things I want to load in. So far it looks like this:

void InitResources()
{
    ResourceManager::getInstance().m_shaderLocations.AddResource(ShaderResources::DEFAULT_SPRITE, "triangle.shader");
    ResourceManager::getInstance().m_shaderLocations.AddResource(ShaderResources::ARIAL_FONT, "arial.shader");
    ResourceManager::getInstance().LoadResources();
}

I’ll probably end up wrapping the ability to add a path to a ResourceLocationMap so it’s a bit neater, but this works for now.

I then collect a pointer to my resources like so:

m_pSpriteShader = &ResourceManager::getInstance().m_shaders.GetResource(ShaderResources::DEFAULT_SPRITE);
m_pFontShader = &ResourceManager::getInstance().m_shaders.GetResource(ShaderResources::ARIAL_FONT);

And hooray, it all works well! I was pretty happy that my plan worked so well – I mostly only encountered small silly typos. One bug that did trip me up for a while was that I forgot to set the projection matrix uniform before trying to draw my text with this new shader (mental note to future me – make sure to remember to set all your shader uniforms again after switching shader program).

And finally I could test my new shader! Look – now I can see the colour behind the text properly!

 

transparentText
(OK, my boyfriend wasn’t super impressed either)

So now I need to migrate all my other resources into this system. And once that’s done I can start reworking my Audio system so it works within the Component system.

Anyway, please let me know what you think of my resource management system so far. And feel free to browse through the code on GitHub.

Communicating with pooled objects 2

So, my last post on this topic talked about adapting my component pool into a more generic packed-array object pool. It turns out that my implementation of this was absolutely littered with bugs. So many bugs that I’m surprised the game functioned at all, let alone in a remotely similar way to how it worked before.

The main reason for all the bugs was that I was still letting the object pool return a pointer to the actual new object that was just created. And I had objects holding on to this pointer, all the while the object pool was rearranging memory every time anything was deleted. This led, of course, to objects pointing to and manipulating incorrect data.

So the first thing to fix was making sure the Create function would return a handle rather than a pointer. But what to return when the object pool is full? I had been returning a nullptr. My initial fix for this had been to return the capacity of the pool, but of course that number is still a valid handle, and this caused a couple of tricky bugs that took me a while to track down. So I am now returning 0, since that is never a valid handle (unless the unsigned int variable overflows, but then I’ve got other problems..)

However I decided that Components still do need to store an actual pointer to their owning GameObjects. So if that GameObject moves as part of an object pool rearrangement, I need to fix the pointers in each of the Components. In order to do this I created a “post move” function for anything that can be pooled (hmm, should I make things inherit from a Poolable abstract class to make sure people don’t leave out the post move function? Or would the multiple inheritance become too much, since many of these things will also be Subjects for the Observer pattern too?) Anyway, in this post move function I simply loop through the component list for a game object and call the “SetGameObject” function from Component.

Once I’d pooled my bullets, I realised that the Init function wasn’t clearing the Component and Observer lists from the GameObject class. This resulted in a Bullet thinking it had more Observers than it really did, as well as too many Components. I ended up clearing these lists in the Bullet Init, but reading over this now, this should really be done in the GameObject Init and called by Bullet.

It also became obvious that I would need to pass the Object pools around to multiple different places. I ended up creating a “helper” class for this, since that meant I could just pass a pointer to this helper class, rather than needing to pass each individual object pool (my function argument lists had been getting ridiculously long). I tried my best to generalise this helper class, but once again my lack of practice with templates came back to bite me, and I started down a few idiotic paths without getting anywhere. This post on Stack Overflow seems similar to what I wanted, but for now I just hard coded in the 4 different components I’m using. If I start to add more components to my engine I will really need to fix this up properly.

So, once I’d fixed up all these bugs things returned a bit more to normal. I’ve actually done another series of changes since these (I’m behind on blogging, but I have been coding!) but I’ll leave those for another post. As always you can see my code on GitHub.

Location specifiers in vertex shaders

Once again it’s been quite a while between posts. I have actually been working though – it turns out that my switch to a packed array object pool was far from over. I’m surprised that I had anything at all by my last blog post that behaved in a similar way to the game pre-packed array. So I wanted to untangle this mess before making a new blog post.

While I was trying to fix this, I discovered a meetup near me where a bunch of female coders get together to work in a pub/cafe. A bit of company while I worked on my frustrating spaghetti code sounded nice, so I set up my laptop with all the tools I needed, downloaded the source from github and off I went.

Up until that point I’d been developing this only on my home desktop PC. Surprise, surprise, when I tried to run my project on my laptop all I got was a black screen. Great. Thus commenced a week-long (I was still working at my day job at this point) journey into working out why nothing was drawing.

So I started by having a look to see if OpenGL was giving me any errors. Which meant I ended up calling glGetError after every OpenGL call. I should really have written this blog post closer to the time of these errors (got distracted by GDC), but I can see I tweeted that I was receiving a GL_INVALID_OPERATION error after my call to glDrawElements. Which at least seemed to narrow things down, but wasn’t super helpful.

So, this helpful tweet led me down the path of trying to get some more verbose info out of my error. This apparently gave me some more useful error messages, but since I didn’t write them down I can’t post them here. Still no dice with getting anything to draw though.

I then looked through this awesome presentation by Elizabeth Baumel. I think I’d already done quite a few steps on her list, and I found myself up to the “Frame Capture and Analysis” slide. Alongside that, the creator of RenderDoc, Baldur Karlsson had appeared in my Twitter mentions with offers of help (Twitter is sometimes really amazing).

So I downloaded RenderDoc and started to work out how to use it. RenderDoc does not work with compatibility mode OpenGL (basically pre OpenGL 3.2). I had thought I was using modern OpenGL, but it turns out that wasn’t the case. You really need to explicitly specify that you only want to create a core context for that to be the case. After quite a lot of faffing about I worked out the correct code to do this was:

glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 4);
glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 3);
glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE);
glfwWindowHint(GLFW_OPENGL_DEBUG_CONTEXT, GLFW_TRUE);
window = glfwCreateWindow(800, 600, "Practice", NULL, NULL);

Once that was done I discovered I quickly got errors about vertex array objects (VAO). The compatibility profile had just been creating a default VAO for me, thus I’d never even really realised that I even needed one. Right, so created my own VAO that I also needed to pass around to my drawable objects and Sprite components, hooray. Now I no longer received that error, but I was still not seeing anything on screen.

OK, so it was time to actually learn how to use RenderDoc to get some more info about what was going on. Baumel’s presentation had a pretty decent walkthrough, and after an evening of staring at each of the different windows in RenderDoc, I started to realise that the vertex shader input looked a bit off. I spent quite a bit of time calculating by hand what it should look like (so glad I took those linear algebra classes at university), and then decided the values were definitely wrong.

Eventually I cottoned on to the fact that the inputs were just out of order. The correct numbers were being sent from my C++ code, but when we get into the GLSL they were being sent to the incorrect variables. This was because I had not used any location specifiers in my vertex shader.

layout (location = 0) in vec4 vertexPosition;
layout (location = 1) in vec4 vertexColour;
layout (location = 2) in vec2 vertexTexCoord;

So, why did this work on my desktop PC and not my laptop? Shader compilers don’t have any specification for assigning locations to parameters if not specified by the programmer. So the shader compiler on my desktop was assigning locations seemingly in the order I’d declared them (no idea if this was by design or chance), while the shader compiler on my laptop was assigning locations in a different order.

Once I had added these location specifiers… yes! It worked!

While frustrating, I am unlikely to make this particular mistake again. Also I learned a lot about the differences between core and compatibility OpenGL, and also the necessity of VAOs. AND how to use RenderDoc to debug things. So, not the worst bug I’d ever encountered.

I have also mostly untangled the original mess I created with my packed array object pools, but I’ll leave that for another blog post. You can also take a sneak peek by having a look at the current source on github.

Communicating with pooled objects

Well, it’s been a rather long time since my last blog post. In my defense I’ve had an eventful few months. Firstly, the company where I was working let go of all the staff, and so I’m losing my job when my contract ends in March (anyone in Amsterdam/Utrecht looking for a programmer??). 3 days after finding out about that I went to Australia for 3 weeks to see friends + family. On my return to Amsterdam I put all my efforts into helping run Global Game Jam 2017 which was a great success – over 7000 games made in over 90 countries last weekend! Then the very next day I had a cycling accident, which I’m still recovering from. So, these excuses are at least a little legitimate, right?

I’ve actually done two lots of updates since my last commit. Firstly I completely overhauled the ComponentPool class (and renamed it ObjectPool, since I also wanted to pool things other than Components), and turned it into a “packed array” implementation that I stumbled across on this Bitsquid blog post. It took me a while to get my head around it, especially how it was possible to wrap around the index array to find a free spot. Turns out there was a bug, which one commenter pointed out. I added the suggested change, as well as an Update method.

Right, now that the ObjectPool class is done I can get on with my previous dilemma – finding a way for a BulletObject to let my BulletManager know when it should delete the BulletObject from the pool. I started off by trying to implement the Event class I linked to in that post. But after creating a basic implementation and refreshing my knowledge on function pointers I learned that non static class member functions can’t actually be pointed to by a function pointer. OK, that’s not strictly true – you can, but only if the function pointer is set up with the class type as well. Which is not really what I wanted, ideally I’d want any class to be able to use the Event system.

After tossing the Event implementation aside I crawled back to my trusty old friend: Game Programming Patterns (which I did actually purchase at some point). I decided that I should probably just go with the Observer pattern for now. So I created my Subject class and decided that all GameObjects should probably be Subjects. Then my BulletManager class inherits from Observer and I override the OnNotify function with the following:

void BulletManager::OnNotify(Subject *subject)
{
    BulletObject* bulletToRemove = reinterpret_cast<BulletObject*>(subject);
    if (bulletToRemove != nullptr) {
        bulletToRemove->RemoveObserver(this);
        m_bulletPool->Destroy(bulletToRemove);
    }
}

I also made sure that, when creating a bullet in the Shoot function, I add the BulletManager as an Observer.

Once that was done I headed back to my GameScreen class to see what else I still needed to do in my conversion to object pooling. The Draw function had been looping through the array of SpriteComponents and calling their Draw functions. Since I no longer had direct access to that array I had to change this. I decided (though I’m now rethinking that), that I would add a function to the ObjectPool that returns a pointer to an object if given an index:

template<typename T>
T* ObjectPool::GetObjectByIndex(const unsigned int a_uiIndex)
{
    if (m_pIndices[a_uiIndex].index < m_uiCount) {
        return &m_pComponent[m_pIndices[a_uiIndex].index];
    }
    return nullptr;
}

And then GameScreen::Draw loops through like so:

void GameScreen::Draw()
{
    Screen::Draw();
    for (unsigned int i = 0; i < m_pSpriteComponentPool->GetCurrentSize(); ++i) {
        SpriteComponent* pSpriteComponent = m_pSpriteComponentPool->GetObjectByIndex(i);
        if (pSpriteComponent != nullptr) {
            pSpriteComponent->Draw();
        }
    }
}

In hindsight, perhaps it would just be easier and more efficient to just hand the whole packed array back? My OOP training tells me no (bad idea to give other objects control over that packed array, right?), but my sliver of data oriented design learnings tell me handing over the packed array would be the better idea.
(And, note to self, sprite drawing should probably be handled in the base screen class so I don’t have to repeat this code in other screen classes).

OK, I think that’s enough for now. Next steps might be trying to make sure all my other Screen classes work in the same way as GameScreen. As well as taking a long hard look at how I’m adding components to gameObjects, because it seems a bit clunky and inefficient right now.

As always, please let me know if you have any thoughts/answers to my questions. And you can see my code on GitHub. Oh, also if you know of any C++ games programming jobs in or near Amsterdam/Utrecht then I would love to hear from you.

Using my new object pooling system

So I sat down today to start to actually use the new object pooling system I created last time. I realised pretty quickly that my ComponentPool was poorly named since I also wanted to pool my Bullets (which are GameObjects).

Also, my BulletObjects are currently responsible (during their Update function) for checking to see if they need to mark themselves as inactive. The BulletManager then polls each BulletObject to see if it’s been listed as inactive. However with my new object pooling the BulletManager doesn’t actually have access to the array of BulletObjects – just the BulletObject pool that I added.

The object pool class also stores an array of bools to keep track of which elements are active. So, what is the best way for my BulletObject pool to work out that a BulletObject is no longer active?

Some quick thoughts – I could set up some sort of Visitor pattern for the object pool so I can send custom functions in? But the Visitor pattern has always felt rather messy and a bit like it’s cheating OOP. Then perhaps an Event class that acts similar to Events in C#? (Hey, I work in C# all day). This example looks promising. The the BulletManager could handle the event and call the object pool’s Destroy method for the BulletObject that triggered the event.

I think I’m leaning towards the Event implementation, but I’d be curious to hear other ideas!

Object pools, variadic templates and reference forwarding

Right, so after my last post it seemed pretty clear that I need to overhaul how I manage the creation and deletion of my components, and that object pooling looked like the best way to do that. So I referred back to Cameron’s comment from ages ago and started to write some object pooling classes for each of my components. However I very quickly worked out that I’d be writing the same thing over and over again.

So, clearly I needed to write a templated version. Once again Cameron had an amazing example that I could have a look at. I decided it was quite a bit more complex than what I needed for now, so I started making a simplified version. I can’t believe how much I’d forgotten about templates – it’s actually a bit embarrassing. It’s really driving home how much more I should be practising on my own C++ projects if I don’t want to forget everything I’ve ever learned while I am stuck using C# at work every day.

I converted the original PhysicsComponentPool I’d started to write into a ComponentPool pretty quickly. I had written a “create” function that accepted the parameters needed to initialise a PhysicsComponent so that I could do just that in the function and return the newly activated and initialised PhysicsComponent. Converting this into a more generic function stumped me for a while.

My options were:

  • write a number of overloaded “create” functions that would suit every component type I had made.
  • leave the responsibility of initialisation up to the class that called the “create” function. The create function would then just be responsible for activating a component and maintaining the state of the object pool.
  • work out how to send variable parameters into my “create” function and how to set up the “init” functions of my components to work with this.

The latter idea was based on Cameron’s “new_object” function in the FixedObjectPool class, but there was a lot going on in his implementation that I didn’t understand and it didn’t quite seem to fit what I wanted. So I did some research to both understand Cameron’s implementation and to find a solution to my problem.

I first worked out that template is an implementation of variadic templates which were introduced in C++11. Variadic templates allow me to send any number of parameters of any type(s) into a function, like so:

template<typename T>
template<typename... Args>
T* ComponentPool::Create(Args... args)
{

}

So, that’s great! Now I can just send these arguments along to the Init function for the component like this:

template
template
T* ComponentPool::Create(Args... args)
{
    for (unsigned int i = 0; i < m_iSize; ++i)
    {
        if (m_pActive[i] != true)
        {
            m_pComponent[i].Init(args);
            m_pActive[i] = true;
            return &m_pComponent[i];
        }
    }
    return nullptr;
}

Uh, no. That does not compile: argserror.

So I then looked at how to expand variadic parameters. It was in a recursive-like fashion where each recursive call to the function unpacked one more argument. Not exactly what I wanted. I just wanted to send ALL the arguments along to my init function. I took another look at Cameron’s example and discovered that he was using something called std::forward. Let’s try that:

template
template
T* ComponentPool::Create(Args... args)
{
    for (unsigned int i = 0; i < m_iSize; ++i)
    {
        if (m_pActive[i] != true)
        {
            m_pComponent[i].Init(std::forward(args)...);
            m_pActive[i] = true;
            return &m_pComponent[i];
        }
    }
    return nullptr;
}

That worked perfectly! I still didn’t really know what std::forward was really doing though. The documentation online was not very easy to understand, but I found a great blog post that made things a bit clearer. std::forward returns an r-value reference to its parameter and placing the ellipses (…) outside of the call to std::forward means that std::forward will be called for each argument in args, which effectively unpacks args while also preserving the lvalue or rvalue status of the arg.

Preserving an argument’s lvalue or rvalue nature is known as perfect forwarding. It’s explained quite well in the post I linked above, as well as in this stack overflow post, so I’m not going to try and re-explain it.

After reading those two posts it was clear that my Create function was not forwarding references properly, so I needed to change it a bit:

template<typename... Args>
T* Create(Args&&... args);

Much better 🙂

Midway through that research I did realise that std::forward wasn’t actually responsible for unpacking the argument list. It was the ellipses. So just having:

m_pComponent[i].Init(args...);

also worked the way I wanted. However it was probably a good idea to preserving the references (though I did try the non forwarded version with both arguments that I had thought were lvalues and arguments I thought were rvalues and everything worked correctly. Perhaps perfect forwarding is just to prevent lvalue copy inefficiencies?)

As always you can have a look at my code on github (still in the component-pattern-setup branch for the moment).

Thanks for all the responses to my last post – they gave me a lot to think about.

Pitfalls of my current component implementation

So I sat down today with the intention of adding some sort of component to handle sprite animations. After deciding that I should probably make an AnimatedSprite component that inherited from Sprite I started to think about how to add it to my GameScreen class with the rest of my components.

For those who haven’t been keeping up with my blogs, I keep all my components together in memory in arrays – one array for each type of component. So now I needed to make a new array for my new AnimatedSprite component type.

I have GameObjects which then store references to the relevant component instances in an array. I have Player, Enemy and Bullet classes which all inherit from the GameObject class, and it was looking likely that I’ll need an Explosion class that does the same.

At the moment all my GameObjects are created in my GameScreen class. However this new Explosion needs to be created due to logic in the Player and Enemy classes. Should I then create some sort of ExplosionManager like I have the BulletManager so that I can have a finite number of Explosions that can be active at any one time?

The idea doesn’t appeal to me. I’d really like to be able to create GameObjects during the Update loop instead of having to make sure they’re all created during the GameScreen setup.

Surely there must be a better way? Perhaps I should try to emulate Unity even more and aim for some kind of “Prefab” GameObject that can be instantiated from anywhere within my code? How do I do that and maintain cache coherence? Do I reserve space for X number of each possible Component type and grab one when I need it? How do Unity do this? They have so many component types, as well as the “custom” script components.

I’m also not happy with how each of my GameObjects must be set up – there seem to be too many steps to adding a Component to a GameObject. This could easily lead to errors if you forget a step (which will happen). Perhaps some implementation of the Factory Pattern could help here?

Hmm, I have a feeling I’m heading towards implementing some actual Object Pooling system, with a pool for each component type as suggested by Cameron earlier this year. Then perhaps a Factory for each type of GameObject that encapsulates the setup a bit more? (I guess a Prefab is kind of a Factory) Someone, please stop me if I’m heading down the completely wrong path here.

If you’d like to have a look at the code for my current setup for a better idea of my dilemma then please have a look here. It is a bit messy because some of the old code from my pre-component version is still hanging around as reference for myself, but it hopefully should be readable.

(PS – I am reading more and more that OOP is hell and design patterns are the devil, but I cannot understand how to solve some of the problems I’m having without them. Do I need to go through some phase of better understanding how to solve these problems with OOP before I can properly understand how to solve them better with DOD or some other method?)