Reflection: thoughts.

How has the project gone Versus initial expectations? 

I'm generally pleased with how the project has progressed, though it would have been nice to really add some more gameplay elements and mechanics. The implementation of the player body took much longer than expected in my initial production outline and that transformed the project from a game to more of a narrative experience. I would have liked to have spent more time on the crime scene apartment level- as it is I don't feel as though the player can do any real investigation but I simply ran out of time to implement more objects. The Narrative/ investigation objects would certainly have to be all modelled and custom versus the asset packs as generic objects wouldn't have stood out to the player enough as significant or tell the story I needed them to. 

There where compromises versus what I had initially pitched and that was unfortunate. I would have liked to flesh out the detective side of the game and give everything a less confined or cramped pacing as an experience.

I would also have wanted to create more models and individual assets for the project but again, the blueprint side of the project just took so much longer than I anticipated, meaning that I customised several and created several models but not enough I feel to give the game a definitively unique visual style. 

The change in style to more of a steampunk/ less grounded experience was done about 7 weeks into production and while there are elements of steampunk, in the final project I don't feel that they make much of an impact. 

I would suggest that the use of assets here was more of a hinderance as by their nature Asset packs from the Unreal Marketplace have to be fairly generic so they have a good audience for customers but because I relied on them more than I would have liked, they leave the project looking bland. 

How does the project match up to the individual Pitch?

The initial pitch was fairly accurate, but the investigation aspects where severely reduced to match the updated production schedule. I made the addition of a small tutorial level to try and orient the player and that was something I didn't plan to create initially but I'm happy with the way that it turned out. There are some small narrative beats and  the level as a whole sets the player up for the rest of the experience and hopefully grabs the players attention and keeps them engaged with the project. 

The project is closer to 10 minutes, versus the half an hour game that I initially pitched. 
The overall narrative structure is a fairly close match, but I had to cut some of the more obscure elements such as the man with the pigs head as I didn't have time to create or modify the existing models. It is a shame to lose some of those elements as I feel they would have made the project feel much more unique and darker in tone. 

What changed?

The scope of the project and the style of the project changed dramatically at points in the production process. The narrative aspects did get a slight simplification but the overall structure remained fairly consistent, just the scale of the levels changed. I think in some ways it makes the project a more pick up and play experience but in other ways I feel it lacks the depth I really would have wanted. The style went from a fairly straight 1940's murder investigation into a more surreal experience which is fine and I feel lends itself well to VR. In VR you can  re-create mundane experiences but to put a player in an immersive surreal experience can lead to better player engagement. 

Lessons Learned? Wisdom gained?

Overall The biggest thing I've learned is to be flexible with deliverables, if some things take longer than they should it's fine to tweak and change other areas of production. I've also learned that even the most prepared production will hit some road blocks and stumbles and ideally you would want to take those into account in the initial production schedule. I'm pleased with the project as a whole but taking these lessons on board for next time/ the next project is something that I'll look to do.

The blueprints aren't quite as robust as I would have liked, ideally a more systemic approach would be taken and indeed make additional content creation more robust, I didn't use inheritance to my advantage. I'll definately have to consider this in future projects. 

It was also very important to get feedback from the supervisor and other users. The supervisor was always willing to consider a change in scope as long as the rationale was there and offered feedback throughout the process that really enabled me to keep progressing without spending too long on areas that wouldn't have an impact on the project. User feedback also helped shape the project, though this is far more limited than planned but although feedback from the supervisor and the users wasn't always what I wanted to hear it was incredibly valuable in shaping the final project. 

Another note, the whole project had me considering the impact of accessability and VR- how can this be improved? Are there ways for the differently abled to enjoy these experiences- I'm sure work could be done to improve but ultimatly and unfortunatly this was outside the scope of this project. 

Project References

 Rain Sound https://soundbible.com/1908-2-Minute-Storm.html

Elevator Sound https://freesound.org/people/Tokun/sounds/391452/

White petal https://www.pngaaa.com/detail/1407511

Wind sound https://freesound.org/people/florianreichelt/sounds/459977/

Pistol gunshot sound https://freesound.org/people/morganpurkis/sounds/371440/

End of tutorial gun shot: https://freesound.org/people/morganpurkis/sounds/370304/

Light sound: https://freesound.org/people/mmaruska/sounds/232447/

Door Shutting sound: https://freesound.org/people/wjtaylor/sounds/266682/

Floorboard creak sound: https://freesound.org/people/bennstir/sounds/81080/

Footsteps on wood sound: https://freesound.org/people/tmkappelt/sounds/85687/

Font https://www.1001fonts.com/viellevarsovienf-font.html

Unreal Marketplace Assets

Abandoned props 01: https://www.unrealengine.com/marketplace/en-US/product/abandoned-props-01

Advanced Glass material pack: https://www.unrealengine.com/marketplace/en-US/product/advanced-glass-material-pack

Amplify LUT Pack: https://www.unrealengine.com/marketplace/en-US/product/amplify-lut-pack

Animation Starter Pack: https://www.unrealengine.com/marketplace/en-US/product/animation-starter-pack

Blood Pack: https://www.unrealengine.com/marketplace/en-US/product/blood-customizable-4k

Cigarette pack: https://www.unrealengine.com/marketplace/en-US/product/high-quality-cigarette-ashtray-mini-set

Death Animations:https://www.unrealengine.com/marketplace/en-US/product/death-animations-mocap-pack

Horror Sound Effects: https://www.unrealengine.com/marketplace/en-US/product/horror-sound-effects-01

Immersive dark ambient music pack: https://www.unrealengine.com/marketplace/en-US/product/immersive-dark-ambient-music-pack

Movement animset pro: https://www.unrealengine.com/marketplace/en-US/product/movement-animset-pro

Megascans Blood : https://www.unrealengine.com/marketplace/en-US/product/e2368ad39c084e77a626dd424b1efcbe

Megascans- Damages:https://www.unrealengine.com/marketplace/en-US/product/ad2dd8069a2145ff90d6a39a0c54919b

Megascans- Scary Restroom: https://www.unrealengine.com/marketplace/en-US/product/c8300f47fe2744ca9304ecf25badba01

Noir/ Mafia Characters: https://www.unrealengine.com/marketplace/en-US/product/noir-mafia-characters

Old office (modular): https://www.unrealengine.com/marketplace/en-US/product/classic-office

Scanned 3D People: https://www.unrealengine.com/marketplace/en-US/product/9c3fab270dfe468a9a920da0c10fa2ad

Thunder and Rain SFX Pack: https://www.unrealengine.com/marketplace/en-US/product/thunder-rain-sfx-pack

Victorian Interior Environment: https://www.unrealengine.com/marketplace/en-US/product/victorian-interior-environment

Apartment House 01: https://www.unrealengine.com/marketplace/en-US/product/apartment-house

Steampunk Megapack: https://www.unrealengine.com/marketplace/en-US/product/steampunk-victorian-megapack

200+ Grunge Decals: https://www.unrealengine.com/marketplace/en-US/product/200-grunge-decals

Upper body IK plugin: https://github.com/JonasMolgaard/UBIKSolver

Replica Studio AI voice acting: https://replicastudios.com/

Voice and sound

For this game I knew I wanted some form of voice acting- reading text boxes in VR and implementing such a system would have taken a fair amount of time to implement. I didn't have any scope for voice actors. I didn't have the budget nor the recording equipment.

I did some research, and Replica studios had a plugin directly to unreal, I didn't end up using this but I did end up using replica to create all the dialogue lines in the game. 

One benefit offered was the flexibility offered by AI voice acting, I could edit the dialogue after recording, I could tweak the lines as I went along the project, this is in stark contrast to traditional voice acting where I would need to have the content locked down or have to reschedule the recording session for altered dialogue. 

The only downside is the voice acting can feel flat, and there are certain words or phrases that do clearly sound like they're being generated by the AI- they sound flat or stilted. 

Overall though, generally speaking I'm happy with the quality of the project sound, it could be better- but it could also be much worse or even silent. 

Further Changes

The project has been coming along well up to this stage but given the time remaining until deadline I've decided to cut some of the 'gameplay' mechanics and pivot to more of an immersive experience style of project. This means that I will not have any mechanics for the journal as I planned, I potentially might still be able to have a puzzle aspect, but it won't be tracked by the game/ explicitly outlined for the player.

There will still be things for the player to find and explore but there will be no specific mechanics tied to them at this stage. This is disappointing from my perspective as I would have liked more time to explore the gameplay elements. I do have a working gun inside the project, but it does not have any reload mechanics or ammo tied to it at present- and I doubt that I will be able to develop it further at this stage.

While it is unfortunate to have to cut aspects of the project, it is always a factor when working towards a deadline. It’s often hard to tell what the final project will look like and how much of the initial vision carries forward as design decisions are made but I think the cuts are the right decision at this point in time as the player avatar work has taken considerably longer to implement.

A Tutorial

When initially planning the project, I did not set in scope to create a tutorial, however, this changed when users came into contact with the game and did not understand how to interact with the project. The initial players did not understand what they needed to do or how to interact with the project. 

I struggled with how I would implement a tutorial in a way that I was happy with and fit the project, I thought that in game voice overs would be very odd to hear the player character refer to objects like the controller while in a realistic setting. 

The tutorial I decided would take the idea of a 'dream sequence' style, allowing me to be more abstract with the way I built it and to get more experience with the level blueprint system. I was keen to make sure that this level impressed as it would be a users first experience of the project.

The key aim is to ensure that the player knows what they have to do and how to interact with the world.

Opening section of the tutorial

The level only lasts for a minute, but should guide the player on how to play the game and interact with it.
The Oculus controller meshes are ones that where included within the engine, with a text render component added to guide the player. 

I also used the opportunity to add more atmosphere to the project with music and particle effects as well as some dialogue from the player character- I felt that the level just being a tutorial would be very straight forward and potentially turn the player off the project before they'd really experienced it. 

On reflection of this level I would potentially extend it, add more phycological elements like additional characters as if the player where reliving moments- this would serve to catch the player up on the characters history and backstory and add some intrigue to the introduction. 

Implementing the level has stalled my progress on the main level, so I may have to look at cutting elements  from the main level to compensate but the addition of the tutorial is one that I am happy with. 

Animation for the player's hands

 There where several key challenges when I set out to animating the project. 

1) I was using a full body avatar, most VR projects start as one hand mesh by default in unreal that is then duplicated for the left hand meaning only one animation can serve both hands

2) I needed to have an idea of what item the player picked up and in what hand. 

3) I needed the animations to play properly on each hand for each item.

One way of implementing this cheaply would be to hide the player mesh when picking up an item like Layers of Fear VR (Bloober Team, 2019), so the player has 'hands' showing until they interact with or pick up an object- however in my opinion this detracts from the player immersion when playing the game. With this in mind, I felt I had to look at alternate options. 

I tried several methods - such as starting to re-build the Unreal Engine 4.26 pick up system in Unreal 4.27

Old Pick up system

The idea of re-making the system using the newer unreal components however would have meant also rebuilding the player blueprint that I'd already been working on and potentially losing work- so I considered trying to implement the animation call into the picked up item itself which would tell the animation blueprint which hand it was grabbed in and what item it was; however this would have been a very costly was of implementing the system as the 'Cast' node is very expensive in terms of performance.

This kept me trying to work out how I could get the hand, the item and animate it. 

The solution was eventually fairly simple. I created an enum table with all the different items that i'd like to have in my game. 

Enum table showing the items you can pick up

The Enum table was then duplicated to each hand and then input into the animation blueprint, this meant that if each hand could get the information about what it was holding then it could update the animation accordingly based on an animation blendspace. 

Enum being fed into the blendspace

I had one blendspace for each hand, however I still did not have a way of knowing which hand held the object, until I realised that this information was already inside the player blueprint and already implemented, so I tried several methods out that did not work, this is where I found out I had to duplicate one enum for each hand otherwise you would pick up an item and, if it used one enum, then you would be stuck with both hands playing the same information. 

Inside the grab component I added a variable for "itemtype Right" and "Itemtype left" for each hand 
those variables are then read by the player blueprint and update according to which hand picked up the object and animating correctly.

The finished player blueprint reading the item type on the right hand

The amount of time that this took to implement has had an impact on my project schedule, meaning that I will have less time to work on the core mechanics but I believe that it has been worth it and while my implementation of the animation could be more refined- I'm happy with where the system is and I feel that to dedicate more time could jeopardise the remaining content that I have left to build

Implementing a VR Avatar for the player

 A key aim for me has been trying to implement a full body avatar for the player to inhabit in VR.

To achieve this I knew I couldn't model something in time so I settled on a Noir Character pack from the unreal engine marketplace (I'll be going over a more in depth review on use of assets- the pro's and con's that I've found so far towards the end of production.). 

The key here was I needed the body to be responsive to the players inputs but convincing enough not to take the player too far out of the experience. The solution seemed to lay in using IK to simulate the players arms inside of VR to re-create the movement in game. On top of this I wanted to implement some type of physics check to make sure that the player cannot easily 'clip' through the world i.e a hand that goes through walls but equally didn't collide with itself either which would cause a host of other issues and above all only take the player outside of the experience. 

This presented several obstacles to overcome-
1) I personally am not an expert in IK or animation. I have used both in the past but only at a surface level and this will be my first full exploration into the topic. 

2) It had to look good enough and work well enough- a VR avatar is getting more and more complicated as developers further explore what's possible. For example Boneworks had a full body IK solution- this is beyond my skill and scope for this project. 

3) Time. The implementation of this would probably have to be very basic if I couldn't do it in a reasonable time then I'd essentially only have a very good VR avatar at the end of this process and not much of a game. This isn't too much of a concern as there is value and merit in a great VR avatar but in terms of a project management point of view- it really needed to be completed as soon as possible so that I could start work on other key areas. 

The first stage I implemented physical animation and FABRIK, having tried two bone IK and not being satisfied with the results. This caused some interesting physic's issues, where the player arms would twist and contort in very unrealistic ways.

The next stage was refining and fixing FABRIK wasn't working as well as it could and when implementing animations for the players lower body I found issues with updating the skeletal mesh location vs. the players actual camera position- completely breaking the immersion and the collision versus my key aims. I took the decision to go back to research when I found the UBIK pug-in on GIT.

UBIK implementation of Animation Blueprint

The upper body IK (UBIK plugin) handle s the mesh bones from the spine upwards, everything lower is handled by a simple blendspace that gets movement and speed to use walking animations that I got from the third person starter animation pack. I'm happy with this for the moment but if I wanted to build on this I would add leg IK so the characters legs matched the in game floors better- however when I tried to implement this I encountered too many issues that would take time away from the rest of the project so the animation only approach was kept for now. Once again this is something I'd like to develop further but the time constraints mean that I have to keep features of the project in scope.

This solved a large amount of issues once I'd removed my old blueprints and done some tweaking. The physical animation's at present have broken the grab function in UE 4.27 to enable the 'pick' up of actors but I suspect the issue is related to the collision trace so I'll need to re-build that. The next stage is trying to set up a physical grab- I suspect that this will be more of a challenge and instead I'll fall back to collision trace then on hit insert item into a socket plus animation pose. 

Getting physics to work on the player body itself was another large challenge and one that went through many iterations as this would be key to selling the player the idea that they physically inhabited this virtual world - as they'd be able to respond to it almost as well as they could in real-life. 

Player blueprint showing several revisions.

I explored constraints and physics handles as options and I suspect if this where to expand as a project I would look into these further but based on my current implementation they did not perform as expected, they would cause the player body to glitch in unexpected ways such as jump around the environment and judder or cause unnecessary strain on CPU performance which led to hitches. I have left the old blueprints in the player Blueprint to show the various iterations.

updating the player location final version

The next task was to update the player skeletal mesh whenever they moved as for this project I elected to use thumbstick movement over teleporting, this is because I wanted to make sure that the player always inhabited the avatar body, and this led to its own interesting challenges as by default Unreal engine utilises teleportation as the primary method of movement for VR projects rather than smooth movement. I used the pawn movement system that's used by characters by default and the character movement system that is also there. The player body was still the problem however, because as the player moved in game it was not updating the collision of the virtual position of the character as they moved allowing the player to 'step out' of their character.

The first iteration of this blueprint was overly complicated, and I used to call it 'ontick' which meant every frame and as Vr usually needs to be 90- 120 FPS, it was very accurate but very performance heavy. 

Older more performance heavy implementation of player location

 

Instead I took the 'onEventplay' function and added a call by time on the created function which is updated every 0.2 seconds and this meant that the performance was much better and gives more overhead for adding other mechanics later on, because I now have that gain in performance over the old code. It also allows me to either speed up or slow down the check depending on how often the player moves- as I want the player to be fairly accurately mapped in the project 0.2 seemed a reasonable compromise between checking too often or not enough which would lead the player movements to feel as though they where juddering. 

I have uploaded a short video that demonstrates the first working iteration. I'm certain that I will make further tweaks and changes as the project progresses

 

Misc:

I have a theory that I will not have time to explore but I suspect having a 2nd hidden skeletal mesh with each body set to constrain the visible mesh (which would have collision disabled) would work very well for physics interactions but again, I have spent too long getting to this point and now need to focus on the gameplay mechanics and level design- although I'm using assets I still need to make sure they are all suitable and work correctly. 

Note: 23/11/2021: The Grab not working is an engine problem and only happens when the player blueprint is opened if the project is opened but not the player blueprint the 'grab' issue does not occur. There is also an engine bug that seems to be triggered by use of physical animation in 4.27 that has been in the engine since 4.26 and carries over to the 5.0 early access. This does mean that I've had to cut back on some of the physical animation aspects such as fingers bending when they get too close to a surface- this could also be done in animation but given how much time I've spent on finishing the player avatar to the point where I'm happy with it- it's something that I will come back to towards the end of the project if I have time. 

Asset creation and store bought assets

 

For this game I'm doing a mix of self-made assets and store bought assets to help fill out the gaps and put the game on the best possible foot for outside showing. 

For store bought assets I'm looking at the Unreal engine Marketplace and will compile a list of store assets that have been used in a future blog post. My aim with the store bought is to go back in over Christmas and refine and unify them so they get some tweaks based on my project needs and create a cohesive art style. The reason for the store bought assets is simply put I don't have the time to make everything and I want a complete and immersive VR world and that means detail, high resolution textures and I want to focus my time on creating the user experience and not just the art work. 

I've already had to manually re-size the textures on several bought assets due to having modular pieces that have all been in 4k which ground the performance of the project to a halt.

For asset creation I'm using a mix of Maya 2019 and Substance painter, both of these are powerful and adaptable tools that I've worked with in the past to achieve some fantastic results, as seen above with the elevator. I wanted to capture a dark and seedy atmosphere that immediately jumps you into the game and also allows me to do some behind the scenes loading in the engine so that the game doesn't seize up and lag which VR is incredibly sensitive to due to the increased rendering requirements. 

I created the room the 2nd level, takes place in, this was done using both Maya and substance, but the props inside the level for example are store assets. The gun was also a store bought asset that I needed to manually animate inside of Maya 2020 which was fairly simple to then bring back to the engine. This was a good time saver as I did not need to create and then rig the model I only needed to export it from Unreal Engine, animate the weapon and then import the animation using the existing mesh skeleton inside of Unreal. 

The completed room populated with props

I have also re-textured several assets to make them more unique to the game for example the desk seen below.

The original asset desk from the unreal engine marketplace

The desk looked too dirty to fit the overall style of the room so I decided to bring it into substance painter to give it a new texture, and keep the texture size to a reasonable limit- allowing me a greater deal of control over performance that will add up the more assets I work on. 

I decided to make the desk more ornate in this example, adding a pattern, a new wood texture and a little message for the player to find, it is a detail that in most games would be obscured, but taking advantage of VR I can add little details like this to enhance the level of detail and make it worthwhile for the player to explore the environment.

The use of the assets allow me to focus on the areas of the game that I feel are important, such as the player avatar code.

However there is a balance, with using assets from a wide range of sources there's a risk that they won't all fit together in a way that benefits the project and instead will leave the project looking odd- or as mentioned above- need additional work before they are suitable for the purpose I'm putting them to. I feel that even if I need to fix some assets and make changes to the texture or sometimes the model itself- it is still faster than trying to make everything myself. 

VR Performance

When looking at VR games, it is important to consider the performance costs involved, as 

the performance needs to be a minimum of 72 Frames per second for most headsets to avoid the user feeling sick.

IrisVR states "Studies have shown that in practice, any VR setup that generates frame rates below 90 frames per second (FPS) is likely to induce disorientation, nausea, and other negative user effects. The lower the frame rate, the worse the effects." (1)

When I used Unreal engines in built profiler for texture budget I found the scene was consuming far larger amounts of VRAM than needed- this was not good- it meant that although the general frame-rate was stable the project would 'hitch' and stutter. This led to a very uneven experience for the player. 

I diagnosed the issue as an asset from the marketplace that was using 4k textures on a modular building for each individual component. As seen below once 

I manually resized the textures I ending up with more VRAM overhead and as the props where outside the playable area as a backdrop I could lower the quality to around 512 pixels.

 

As seen in image 2, manually re-sizing is not difficult to accomplish in engine but the process was tedious as there where over 300 individual assets that I had to re-size, including the textures for the normal maps and ambient occlusion, but the performance gain was worth it. 

1)  https://help.irisvr.com/hc/en-us/articles/215884547-The-Importance-of-Frame-Rates (16/11/2021)

 Image 1

Image 2

Tweaking the idea....

 When the project was initially pitched, it was to be set in an accurate 1940's setting and time. I have changed how I'm going to approach this as I found that in researching and trying to remain committed to the time period was limiting what I felt that I would be able to achieve with the game.

I decided to take the direction of tweaking the world setting to a fictional steam-punk-esq setting while retaining some of the more realistic aspects such as the player body. The style that I would potentially looking at would be one similar to Bioshock infinite (2k Games, 2013)

Bioshock Infinite (2K Games, 2013) 

 I found that trying to accurately re-create a 1940's environment from scratch plus a VR game would be too difficult. It would be a choice between the environment or making the game and within the time frame of the individual project I quickly found that the workload would not be reasonable. This will also allow me more freedom to use my library of assets as they would not all be strictly 1940's/ era- appropriate.
My focus for the project I feel is going to be a realistic player avatar makes the player feel immersed in the game world and the fiction of the setting. 

I'm still looking at sticking to the original guidelines of the project with respect of events and gameplay however I am also to reducing the scope to focus on quality over quantity. 

Building a poster

 At this early stage of the project I wanted to try and capture the feel and style of the project and create a personal reference for myself for the project. 

While developing the project I thought it would make for a productive break and offer any potential players a look into the project while it was still being built. 

I took a screenshot inside Unreal Engine for the assets so far, these are a mix of store assets and created assets inside a run down environment. I also used this as an opportunity to finally decide on a name for the project "Dead Flowers" .

I took the screenshot into photoshop to tweak the colours and add some noise to the image- I was aiming for a David Fincher style look so I added strong tints of colour with the noise and then overlaid the image with a font that I thought suited the game and the film noir outline of the project. 

Finished Poster