A Tutorial

When initially planning the project, I did not set in scope to create a tutorial, however, this changed when users came into contact with the game and did not understand how to interact with the project. The initial players did not understand what they needed to do or how to interact with the project. 

I struggled with how I would implement a tutorial in a way that I was happy with and fit the project, I thought that in game voice overs would be very odd to hear the player character refer to objects like the controller while in a realistic setting. 

The tutorial I decided would take the idea of a 'dream sequence' style, allowing me to be more abstract with the way I built it and to get more experience with the level blueprint system. I was keen to make sure that this level impressed as it would be a users first experience of the project.

The key aim is to ensure that the player knows what they have to do and how to interact with the world.

Opening section of the tutorial

The level only lasts for a minute, but should guide the player on how to play the game and interact with it.
The Oculus controller meshes are ones that where included within the engine, with a text render component added to guide the player. 

I also used the opportunity to add more atmosphere to the project with music and particle effects as well as some dialogue from the player character- I felt that the level just being a tutorial would be very straight forward and potentially turn the player off the project before they'd really experienced it. 

On reflection of this level I would potentially extend it, add more phycological elements like additional characters as if the player where reliving moments- this would serve to catch the player up on the characters history and backstory and add some intrigue to the introduction. 

Implementing the level has stalled my progress on the main level, so I may have to look at cutting elements  from the main level to compensate but the addition of the tutorial is one that I am happy with. 

Animation for the player's hands

 There where several key challenges when I set out to animating the project. 

1) I was using a full body avatar, most VR projects start as one hand mesh by default in unreal that is then duplicated for the left hand meaning only one animation can serve both hands

2) I needed to have an idea of what item the player picked up and in what hand. 

3) I needed the animations to play properly on each hand for each item.

One way of implementing this cheaply would be to hide the player mesh when picking up an item like Layers of Fear VR (Bloober Team, 2019), so the player has 'hands' showing until they interact with or pick up an object- however in my opinion this detracts from the player immersion when playing the game. With this in mind, I felt I had to look at alternate options. 

I tried several methods - such as starting to re-build the Unreal Engine 4.26 pick up system in Unreal 4.27

Old Pick up system

The idea of re-making the system using the newer unreal components however would have meant also rebuilding the player blueprint that I'd already been working on and potentially losing work- so I considered trying to implement the animation call into the picked up item itself which would tell the animation blueprint which hand it was grabbed in and what item it was; however this would have been a very costly was of implementing the system as the 'Cast' node is very expensive in terms of performance.

This kept me trying to work out how I could get the hand, the item and animate it. 

The solution was eventually fairly simple. I created an enum table with all the different items that i'd like to have in my game. 

Enum table showing the items you can pick up

The Enum table was then duplicated to each hand and then input into the animation blueprint, this meant that if each hand could get the information about what it was holding then it could update the animation accordingly based on an animation blendspace. 

Enum being fed into the blendspace

I had one blendspace for each hand, however I still did not have a way of knowing which hand held the object, until I realised that this information was already inside the player blueprint and already implemented, so I tried several methods out that did not work, this is where I found out I had to duplicate one enum for each hand otherwise you would pick up an item and, if it used one enum, then you would be stuck with both hands playing the same information. 

Inside the grab component I added a variable for "itemtype Right" and "Itemtype left" for each hand 
those variables are then read by the player blueprint and update according to which hand picked up the object and animating correctly.

The finished player blueprint reading the item type on the right hand

The amount of time that this took to implement has had an impact on my project schedule, meaning that I will have less time to work on the core mechanics but I believe that it has been worth it and while my implementation of the animation could be more refined- I'm happy with where the system is and I feel that to dedicate more time could jeopardise the remaining content that I have left to build

Implementing a VR Avatar for the player

 A key aim for me has been trying to implement a full body avatar for the player to inhabit in VR.

To achieve this I knew I couldn't model something in time so I settled on a Noir Character pack from the unreal engine marketplace (I'll be going over a more in depth review on use of assets- the pro's and con's that I've found so far towards the end of production.). 

The key here was I needed the body to be responsive to the players inputs but convincing enough not to take the player too far out of the experience. The solution seemed to lay in using IK to simulate the players arms inside of VR to re-create the movement in game. On top of this I wanted to implement some type of physics check to make sure that the player cannot easily 'clip' through the world i.e a hand that goes through walls but equally didn't collide with itself either which would cause a host of other issues and above all only take the player outside of the experience. 

This presented several obstacles to overcome-
1) I personally am not an expert in IK or animation. I have used both in the past but only at a surface level and this will be my first full exploration into the topic. 

2) It had to look good enough and work well enough- a VR avatar is getting more and more complicated as developers further explore what's possible. For example Boneworks had a full body IK solution- this is beyond my skill and scope for this project. 

3) Time. The implementation of this would probably have to be very basic if I couldn't do it in a reasonable time then I'd essentially only have a very good VR avatar at the end of this process and not much of a game. This isn't too much of a concern as there is value and merit in a great VR avatar but in terms of a project management point of view- it really needed to be completed as soon as possible so that I could start work on other key areas. 

The first stage I implemented physical animation and FABRIK, having tried two bone IK and not being satisfied with the results. This caused some interesting physic's issues, where the player arms would twist and contort in very unrealistic ways.

The next stage was refining and fixing FABRIK wasn't working as well as it could and when implementing animations for the players lower body I found issues with updating the skeletal mesh location vs. the players actual camera position- completely breaking the immersion and the collision versus my key aims. I took the decision to go back to research when I found the UBIK pug-in on GIT.

UBIK implementation of Animation Blueprint

The upper body IK (UBIK plugin) handle s the mesh bones from the spine upwards, everything lower is handled by a simple blendspace that gets movement and speed to use walking animations that I got from the third person starter animation pack. I'm happy with this for the moment but if I wanted to build on this I would add leg IK so the characters legs matched the in game floors better- however when I tried to implement this I encountered too many issues that would take time away from the rest of the project so the animation only approach was kept for now. Once again this is something I'd like to develop further but the time constraints mean that I have to keep features of the project in scope.

This solved a large amount of issues once I'd removed my old blueprints and done some tweaking. The physical animation's at present have broken the grab function in UE 4.27 to enable the 'pick' up of actors but I suspect the issue is related to the collision trace so I'll need to re-build that. The next stage is trying to set up a physical grab- I suspect that this will be more of a challenge and instead I'll fall back to collision trace then on hit insert item into a socket plus animation pose. 

Getting physics to work on the player body itself was another large challenge and one that went through many iterations as this would be key to selling the player the idea that they physically inhabited this virtual world - as they'd be able to respond to it almost as well as they could in real-life. 

Player blueprint showing several revisions.

I explored constraints and physics handles as options and I suspect if this where to expand as a project I would look into these further but based on my current implementation they did not perform as expected, they would cause the player body to glitch in unexpected ways such as jump around the environment and judder or cause unnecessary strain on CPU performance which led to hitches. I have left the old blueprints in the player Blueprint to show the various iterations.

updating the player location final version

The next task was to update the player skeletal mesh whenever they moved as for this project I elected to use thumbstick movement over teleporting, this is because I wanted to make sure that the player always inhabited the avatar body, and this led to its own interesting challenges as by default Unreal engine utilises teleportation as the primary method of movement for VR projects rather than smooth movement. I used the pawn movement system that's used by characters by default and the character movement system that is also there. The player body was still the problem however, because as the player moved in game it was not updating the collision of the virtual position of the character as they moved allowing the player to 'step out' of their character.

The first iteration of this blueprint was overly complicated, and I used to call it 'ontick' which meant every frame and as Vr usually needs to be 90- 120 FPS, it was very accurate but very performance heavy. 

Older more performance heavy implementation of player location

 

Instead I took the 'onEventplay' function and added a call by time on the created function which is updated every 0.2 seconds and this meant that the performance was much better and gives more overhead for adding other mechanics later on, because I now have that gain in performance over the old code. It also allows me to either speed up or slow down the check depending on how often the player moves- as I want the player to be fairly accurately mapped in the project 0.2 seemed a reasonable compromise between checking too often or not enough which would lead the player movements to feel as though they where juddering. 

I have uploaded a short video that demonstrates the first working iteration. I'm certain that I will make further tweaks and changes as the project progresses

 

Misc:

I have a theory that I will not have time to explore but I suspect having a 2nd hidden skeletal mesh with each body set to constrain the visible mesh (which would have collision disabled) would work very well for physics interactions but again, I have spent too long getting to this point and now need to focus on the gameplay mechanics and level design- although I'm using assets I still need to make sure they are all suitable and work correctly. 

Note: 23/11/2021: The Grab not working is an engine problem and only happens when the player blueprint is opened if the project is opened but not the player blueprint the 'grab' issue does not occur. There is also an engine bug that seems to be triggered by use of physical animation in 4.27 that has been in the engine since 4.26 and carries over to the 5.0 early access. This does mean that I've had to cut back on some of the physical animation aspects such as fingers bending when they get too close to a surface- this could also be done in animation but given how much time I've spent on finishing the player avatar to the point where I'm happy with it- it's something that I will come back to towards the end of the project if I have time. 

Asset creation and store bought assets

 

For this game I'm doing a mix of self-made assets and store bought assets to help fill out the gaps and put the game on the best possible foot for outside showing. 

For store bought assets I'm looking at the Unreal engine Marketplace and will compile a list of store assets that have been used in a future blog post. My aim with the store bought is to go back in over Christmas and refine and unify them so they get some tweaks based on my project needs and create a cohesive art style. The reason for the store bought assets is simply put I don't have the time to make everything and I want a complete and immersive VR world and that means detail, high resolution textures and I want to focus my time on creating the user experience and not just the art work. 

I've already had to manually re-size the textures on several bought assets due to having modular pieces that have all been in 4k which ground the performance of the project to a halt.

For asset creation I'm using a mix of Maya 2019 and Substance painter, both of these are powerful and adaptable tools that I've worked with in the past to achieve some fantastic results, as seen above with the elevator. I wanted to capture a dark and seedy atmosphere that immediately jumps you into the game and also allows me to do some behind the scenes loading in the engine so that the game doesn't seize up and lag which VR is incredibly sensitive to due to the increased rendering requirements. 

I created the room the 2nd level, takes place in, this was done using both Maya and substance, but the props inside the level for example are store assets. The gun was also a store bought asset that I needed to manually animate inside of Maya 2020 which was fairly simple to then bring back to the engine. This was a good time saver as I did not need to create and then rig the model I only needed to export it from Unreal Engine, animate the weapon and then import the animation using the existing mesh skeleton inside of Unreal. 

The completed room populated with props

I have also re-textured several assets to make them more unique to the game for example the desk seen below.

The original asset desk from the unreal engine marketplace

The desk looked too dirty to fit the overall style of the room so I decided to bring it into substance painter to give it a new texture, and keep the texture size to a reasonable limit- allowing me a greater deal of control over performance that will add up the more assets I work on. 

I decided to make the desk more ornate in this example, adding a pattern, a new wood texture and a little message for the player to find, it is a detail that in most games would be obscured, but taking advantage of VR I can add little details like this to enhance the level of detail and make it worthwhile for the player to explore the environment.

The use of the assets allow me to focus on the areas of the game that I feel are important, such as the player avatar code.

However there is a balance, with using assets from a wide range of sources there's a risk that they won't all fit together in a way that benefits the project and instead will leave the project looking odd- or as mentioned above- need additional work before they are suitable for the purpose I'm putting them to. I feel that even if I need to fix some assets and make changes to the texture or sometimes the model itself- it is still faster than trying to make everything myself. 

VR Performance

When looking at VR games, it is important to consider the performance costs involved, as 

the performance needs to be a minimum of 72 Frames per second for most headsets to avoid the user feeling sick.

IrisVR states "Studies have shown that in practice, any VR setup that generates frame rates below 90 frames per second (FPS) is likely to induce disorientation, nausea, and other negative user effects. The lower the frame rate, the worse the effects." (1)

When I used Unreal engines in built profiler for texture budget I found the scene was consuming far larger amounts of VRAM than needed- this was not good- it meant that although the general frame-rate was stable the project would 'hitch' and stutter. This led to a very uneven experience for the player. 

I diagnosed the issue as an asset from the marketplace that was using 4k textures on a modular building for each individual component. As seen below once 

I manually resized the textures I ending up with more VRAM overhead and as the props where outside the playable area as a backdrop I could lower the quality to around 512 pixels.

 

As seen in image 2, manually re-sizing is not difficult to accomplish in engine but the process was tedious as there where over 300 individual assets that I had to re-size, including the textures for the normal maps and ambient occlusion, but the performance gain was worth it. 

1)  https://help.irisvr.com/hc/en-us/articles/215884547-The-Importance-of-Frame-Rates (16/11/2021)

 Image 1

Image 2