top of page
STAY AND SLAY
DESCRIPTION
Game project #7 at The Game Assembly
MY CONTRIBUTIONS
-
Screen space ambient occlusion
-
Behavior trees and AI
-
Visual scripting nodes
-
Physically-based sound effects
TIMEFRAME
16 weeks (20h/week)
GENRE
First-person shooter
TEAM
5 programmers, 5 artists, 2 level designers
ENGINE
Our custom engine
”Retired from your former life as a hitman, you check into a hotel only to find yourself interrupting a heist. Vacation can wait: It's time to teach these amateurs a lesson!"
For our 7th game and also our first FPS, we wanted to take the opportunity to play into the genre's ability for immersion, which meant better graphics, smarter enemies, and more interactable and responsive environments.
With all gameplay taking place indoors inside a hotel filled with props and thus lots of straight angles and crevices, I added screen space ambient occlusion (SSAO) to our rendering pipeline as an effective way to increase the realism of the environment. Building on my work with navmeshes in our previous project, I also implemented all the enemy AI using my own behavior tree system, while adding features like waypoints to put more tools in the hands of our level designers.
To help them make more interactive levels, I also provided our level designers with custom-made visual scripting nodes, which they could assemble into scripts and place on game objects. Finally, I worked to connect our PhysX backend with our FMOD backend so that the player could receive auditory feedback from impacts and other physical events and be even further immersed.
With all gameplay taking place indoors inside a hotel filled with props and thus lots of straight angles and crevices, I added screen space ambient occlusion (SSAO) to our rendering pipeline as an effective way to increase the realism of the environment. Building on my work with navmeshes in our previous project, I also implemented all the enemy AI using my own behavior tree system, while adding features like waypoints to put more tools in the hands of our level designers.
To help them make more interactive levels, I also provided our level designers with custom-made visual scripting nodes, which they could assemble into scripts and place on game objects. Finally, I worked to connect our PhysX backend with our FMOD backend so that the player could receive auditory feedback from impacts and other physical events and be even further immersed.
SSAO
IMPLEMENTATION
Screen space ambient occlusion (SSAO) is a post-processing technique which approximates the ambient occlusion of a scene while using only the normals and depths as seen from the camera, hence the moniker "screen space". Since these were already computed and stored as part of our deferred rendering pipeline, I figured SSAO would be the perfect add-on to achieve higher realism.
My implementation was largely based on the one given in "Introduction to 3D Game Programming with DirectX 11" by Frank Luna. It proceeds as follows: For every pixel on screen, the world normal and position is sampled and we consider the hemisphere of a given radius there. Next, we iterate over a fixed number of pseudorandom points inside this hemisphere and find their corresponding pixel. We then sample the world position and depth at this pixel, compare it to our starting pixel's position and depth, and compute an occlusion term to be added to a running total. More specifically, a sample point contributes more occlusion the closer it is both to the hemisphere's center and apex.
The exact weighing is determined by a user-defined function, which in my implementation had four tweakable parameters.
Since the resulting ambient occlusion texture is very noisy owing to the small random sample, the final step is to apply an edge-preserving blur. Here I again followed Luna's book and used a pixel shader rather than the more standard compute shader to avoid the overhead of switching to compute mode, and also because it was more straightforward to implement. The blur is applied separately in the horizontal and vertical directions and use the normals and depths to estimate if a discontinuity/edge is present, and if so, discards the contribution on the other side of the edge. For my implementation, I chose to apply the horizontal+vertical blur pass four times simply because I thought it looked the nicest.
My implementation was largely based on the one given in "Introduction to 3D Game Programming with DirectX 11" by Frank Luna. It proceeds as follows: For every pixel on screen, the world normal and position is sampled and we consider the hemisphere of a given radius there. Next, we iterate over a fixed number of pseudorandom points inside this hemisphere and find their corresponding pixel. We then sample the world position and depth at this pixel, compare it to our starting pixel's position and depth, and compute an occlusion term to be added to a running total. More specifically, a sample point contributes more occlusion the closer it is both to the hemisphere's center and apex.
The exact weighing is determined by a user-defined function, which in my implementation had four tweakable parameters.
Since the resulting ambient occlusion texture is very noisy owing to the small random sample, the final step is to apply an edge-preserving blur. Here I again followed Luna's book and used a pixel shader rather than the more standard compute shader to avoid the overhead of switching to compute mode, and also because it was more straightforward to implement. The blur is applied separately in the horizontal and vertical directions and use the normals and depths to estimate if a discontinuity/edge is present, and if so, discards the contribution on the other side of the edge. For my implementation, I chose to apply the horizontal+vertical blur pass four times simply because I thought it looked the nicest.
RESULT
Here you can see the resulting occlusion texture after blurring and multiplying with the ambient occlusion term of the material (which is specified by our artists and has nothing to do with the occlusion produced by SSAO). Note that unobstructed flat surfaces have little to no occlusion, while crevices and corners as expected have high occlusion.
Being an approximate method, SSAO naturally comes with a number of undesirable artifacts. In my implementation and for certain choices of occlusion parameters, one apparent artifact was the presence of a static pattern which would follow the camera as the player moved. This can be attributed to the use of a random function which depend on pixel position. Another artifact was that occlusion near edges of the screen would seem to fade in/out of existence as it entered/exited the screen, which can be attributed to the algorithm not having any knowledge of the scene beyond what is present on the screen and is therefore an intrinsic limitation of SSAO itself.
Both of these artifacts could be made to be largely unnoticeable by tweaking the parameters of the algorithm, so I decided to leave it at that. I am aware however some implementations use a deterministic dither texture, which can help to mitigate the first issue, and this could have been a possible improvement.
In summary, although this version of SSAO had its drawbacks, it was easy to implement and add to our existing pipeline and with some tweaking could be made to produce perfectly usable ambient occlusion. Indeed, one of the graphics teachers was very happy to see it in our game!
Being an approximate method, SSAO naturally comes with a number of undesirable artifacts. In my implementation and for certain choices of occlusion parameters, one apparent artifact was the presence of a static pattern which would follow the camera as the player moved. This can be attributed to the use of a random function which depend on pixel position. Another artifact was that occlusion near edges of the screen would seem to fade in/out of existence as it entered/exited the screen, which can be attributed to the algorithm not having any knowledge of the scene beyond what is present on the screen and is therefore an intrinsic limitation of SSAO itself.
Both of these artifacts could be made to be largely unnoticeable by tweaking the parameters of the algorithm, so I decided to leave it at that. I am aware however some implementations use a deterministic dither texture, which can help to mitigate the first issue, and this could have been a possible improvement.
In summary, although this version of SSAO had its drawbacks, it was easy to implement and add to our existing pipeline and with some tweaking could be made to produce perfectly usable ambient occlusion. Indeed, one of the graphics teachers was very happy to see it in our game!
BEHAVIOR TREES AND AI
IMPLEMENTATION
I begun by making an abstract base class Node which all concrete nodes must inherit from. I already knew that one of the most useful features of behavior trees is that they allow for the reuse of subtrees (for the sake of both modularity and optimization) and I made sure to enable this by (1) using shared pointers so that nodes can be shared between trees, and (2) by passing the entity as an argument to Node::Tick() instead of keeping it as a member, thereby decoupling the tree from its owning entity.
Next, I implemented the usual flow nodes - the composites (Selector, Sequencer, Parallel) and the decorators (Succeeder, Failer, Inverter, DoOnce, etc.), which was straightforward. One of the first non-flow nodes I made was PrintString to assist me in debugging.
For movement, the first iterations involved simply turning and moving straight towards the player. As development progressed I fleshed it out by adding checks for player proximity and/or visibility (both by raycast and cone of vision), and once the hotel started getting full of props I integrated my navmesh system, so that enemies could follow the player while avoiding obstacles. I also added support for waypoints so that enemies could run for cover or take unexpected paths to reach the player. Attacking the player was also handled by dedicated nodes.
Once the game reached sufficient complexity, I added a blackboard to give enemies internal state. For example, once an enemy saw the player it would be "alert" and attempt to seek him even when not immediately visible. I could also use it to store the current target for pathfinding.
Finally, I made nodes for controlling the enemies' appearance, such as setting animation state and speed, and even to play 3D sounds.
Next, I implemented the usual flow nodes - the composites (Selector, Sequencer, Parallel) and the decorators (Succeeder, Failer, Inverter, DoOnce, etc.), which was straightforward. One of the first non-flow nodes I made was PrintString to assist me in debugging.
For movement, the first iterations involved simply turning and moving straight towards the player. As development progressed I fleshed it out by adding checks for player proximity and/or visibility (both by raycast and cone of vision), and once the hotel started getting full of props I integrated my navmesh system, so that enemies could follow the player while avoiding obstacles. I also added support for waypoints so that enemies could run for cover or take unexpected paths to reach the player. Attacking the player was also handled by dedicated nodes.
Once the game reached sufficient complexity, I added a blackboard to give enemies internal state. For example, once an enemy saw the player it would be "alert" and attempt to seek him even when not immediately visible. I could also use it to store the current target for pathfinding.
Finally, I made nodes for controlling the enemies' appearance, such as setting animation state and speed, and even to play 3D sounds.
IMPLEMENTATION, CONT.
Although one could manually use std::make_shared() and Composite/Decorator::AddChild() to build trees, I quickly found this to be cumbersome and hard to read. So, I added some helper methods to my Tree component to assist me when building them.
During construction a stack of nodes is held which represents the path from the root to the last added node. To add a leaf node, one could call a templated method Leaf and pass in the type and constructor arguments; this would create the leaf and pass it to the top node's AddChild method, thus connecting the leaf to the tree.
If instead a flow node (Composite or Decorator) was to be added, then one could call a templated method Branch, which would as before create and connect the node to the tree but also push it onto the stack. Thus adding any additional nodes would connect them onto the new flow node. Once the subtree with root in this node was completed, one called End() to pop it off the internal stack.
Also, existing trees could be added as a subtree using a method Subtree(). By making the tree stateless and static, one could therefore use the same subtree across all enemies. Finally, since each method returned a reference to the same tree, one could write code that very closely resembled the structure of the tree, which helped me tremendously when creating and debugging them.
I thought about making a visual scripting tool for inspecting and building trees, but decided not to as the only user would have been myself.
During construction a stack of nodes is held which represents the path from the root to the last added node. To add a leaf node, one could call a templated method Leaf and pass in the type and constructor arguments; this would create the leaf and pass it to the top node's AddChild method, thus connecting the leaf to the tree.
If instead a flow node (Composite or Decorator) was to be added, then one could call a templated method Branch, which would as before create and connect the node to the tree but also push it onto the stack. Thus adding any additional nodes would connect them onto the new flow node. Once the subtree with root in this node was completed, one called End() to pop it off the internal stack.
Also, existing trees could be added as a subtree using a method Subtree(). By making the tree stateless and static, one could therefore use the same subtree across all enemies. Finally, since each method returned a reference to the same tree, one could write code that very closely resembled the structure of the tree, which helped me tremendously when creating and debugging them.
I thought about making a visual scripting tool for inspecting and building trees, but decided not to as the only user would have been myself.
FEATURE: PATHFINDING AND WAYPOINTS
I reused my navmesh system from our previous project and made a node for accessing its pathfinding functionality; this way, enemies could use the navmesh to navigate around obstacles and from room to room in search of the player. If either the player or enemy left the navmesh, I would use a Selector node to fall back on seeking the player in a straight line, since this way you didn't need to extend the navmesh to literally every floor surface of the hotel.
As our level designers wanted some enemies to be able to move back and forth between covers or take unexpected routes to the player, I also implemented a waypoint system. By placing an empty game object in Unity and linking it to an enemy, the enemy would begin by seeking it out. Furthermore, by linking the waypoint to another, one could create a chain for the enemy to follow, or even create a loop. I also made it possible to add values to the waypoint, so one could for example specify how long the enemy should pause upon reaching the waypoint before proceeding to the next.
As our level designers wanted some enemies to be able to move back and forth between covers or take unexpected routes to the player, I also implemented a waypoint system. By placing an empty game object in Unity and linking it to an enemy, the enemy would begin by seeking it out. Furthermore, by linking the waypoint to another, one could create a chain for the enemy to follow, or even create a loop. I also made it possible to add values to the waypoint, so one could for example specify how long the enemy should pause upon reaching the waypoint before proceeding to the next.
FEATURE: BLACKBOARD
Since we were using an entity component system and since I passed in the entity into each node's Tick() method, I realized I could simply create a new component type to store the blackboard and use our ECS to access it within Tick() for reading or writing purposes.
We were already using a kind of blackboard from our previous project, where you could add a special component in Unity (our front-end editor) which could hold various key-value pairs, and which would be exported in JSON format. For simplicity I therefore copied this JSON object and made it a component alongside the behavior tree. For writing, I created a node SetValue which was templated on the type of the value to write and accepted the key-value pair in its constructor. For reading I made a couple of similar nodes, such as IsValueEqual and IsValueGreater. These too were templated on the value type.
To extend my blackboard with support for arithmetic, I created a templated node OperateOnValues, which in its constructor took three keys; two for the input and one for the output. Moreover, it had two template arguments, the first being the value type and the second being the function object to execute.
So for example, one could write OperateOnValues<float, std::minus>("x", "y", "z") to create a node capable of computing "z=x-y" inside the blackboard. By replacing std::minus with std::plus, one would instead get a node that computes "z=x+y", and so on.
As an application, these nodes were used to give certain enemies the ability to pause upon reaching a waypoint and shoot at the player for before seeking the next waypoint. To do this, I would store a wait time in the blackboard and use OperateOnValues to decrement it, only letting the enemy proceed once the value reached zero, which I checked using IsValueGreater.
We were already using a kind of blackboard from our previous project, where you could add a special component in Unity (our front-end editor) which could hold various key-value pairs, and which would be exported in JSON format. For simplicity I therefore copied this JSON object and made it a component alongside the behavior tree. For writing, I created a node SetValue which was templated on the type of the value to write and accepted the key-value pair in its constructor. For reading I made a couple of similar nodes, such as IsValueEqual and IsValueGreater. These too were templated on the value type.
To extend my blackboard with support for arithmetic, I created a templated node OperateOnValues, which in its constructor took three keys; two for the input and one for the output. Moreover, it had two template arguments, the first being the value type and the second being the function object to execute.
So for example, one could write OperateOnValues<float, std::minus>("x", "y", "z") to create a node capable of computing "z=x-y" inside the blackboard. By replacing std::minus with std::plus, one would instead get a node that computes "z=x+y", and so on.
As an application, these nodes were used to give certain enemies the ability to pause upon reaching a waypoint and shoot at the player for before seeking the next waypoint. To do this, I would store a wait time in the blackboard and use OperateOnValues to decrement it, only letting the enemy proceed once the value reached zero, which I checked using IsValueGreater.
VISUAL SCRIPTING NODES
VISUAL SCRIPTING + ENTT...
Our visual scripting system initially had no nodes other than Start and Update. My first contribution was to create nodes that enabled scripts to interplay with EnTT, our ECS of choice. After creating a new connection type for entities, I made nodes that could fetch the entity of the object holding the scripts, that could find the player, that could find an entity by tag, and so on. I also made a node for destroying an entity.
Some nodes I made which were unrelated to EnTT was a node for getting user keyboard input, and nodes for manipulating a blackboard, whose implementation was rather similar to the enemies' blackboard.
Some nodes I made which were unrelated to EnTT was a node for getting user keyboard input, and nodes for manipulating a blackboard, whose implementation was rather similar to the enemies' blackboard.
... + PHYSX + FMOD
I also added nodes to access functionality in PhysX, our physics backend. For example, I made a node to check if two colliders overlap and and to enable/disable simulation. I also made a node that triggered on its owning object being shot; this was used to make paintings fall off the wall when shot.
Moreover, I made nodes for playing FMOD Studio events, which were used e.g. to play a clip of a person begging to be let out when the player walked past a barricaded hotel room door.
Moreover, I made nodes for playing FMOD Studio events, which were used e.g. to play a clip of a person begging to be let out when the player walked past a barricaded hotel room door.