The Role of Audio in Video Game Immersion: December 2014

Monday, 29 December 2014

Location Recording Session 01

Today some recordings were carried out within a disused farm building containing large steel grain bins. The plan was to capture some metallic sounds to be used in the creation of assets for the dungeon level. A Zoom H2n along with a guitar pickup was used to capture the majority of the metallic recordings.

The guitar pickup was most successful when held a few millimetres away from the metal of the grain bin, whilst using a piece of angle steel to to create impact sounds and scraping sounds. By using the guitar pick up I maximised the resonant sound emitted from the bin whilst eliminating all room space noise. Examples are below:

Some additional recordings were also captured from various mechanical devices throughout the building as well as some footstep sounds, some additional editing of these will be required however due to the nature of the location. Background noise levels were an issue along with the large reverberant space of the building but that aside I feel the sounds can be cleaned up for application to the project.

Monday, 22 December 2014

Dungeon Escape level - Evaluation

The level is set in what appears to be an underground dungeon consisting of stone rooms and corridors. The dungeon is lit by flame torches for the most part, aside from one area which opens up into a cave which is lit through the roof (an opening to the outside world and moodlit sky).

The game objective is to try and escape the dungeon by finding 3 relics which are to be inserted into an alter which in turn opens the escape door. There are also 3 treasure chests in the level that can be opened by keys which are hidden within the level. In order to find both relics and keys the player must navigate through the dungeon by solving how to open gateways using various switches.

Some design concerns which need addressed:

These is no backstory or direction, the player has no idea why they are there and what they are supposed to do.

The initial idea here is to create a short backstory which test players can read before playing the level. Another possible route is to use expositional dialogue spoken in game by the character as they navigate through the level.

These are no enemies or any traps which end the characters life, this could lead to player boredom.

Sound could be used to imply the threat of danger such as the sound of a beast stalking the player, however this would only work on one completion of the level. Creating an actual creature which is present and visible is an option but is also well outside of my skillset. Perhaps an invisible enemy could be created and tracked by the player through the use of sound. A timer could also be enforced to push the player to focus more on completion.

There is no reward by collecting treasure, when the player escapes the end screen either reads that you escaped with or without treasure.

Hearing a gratifying sound when treasure is found will be essential but also more recognition should be awarded at the end sequence.

Saturday, 20 December 2014

Basic Recording Map

A map was created in reference to the Basic Sound Map and details which sounds are required to be recorded. Session plans will follow on from this map and detail suitable locations, props, recording techniques and so on.

Saturday, 13 December 2014

Game Level Concept - Game review (The Last of Us)

This is a review of some key audio features included in "The Last of Us".

One of the main factors that personally drew me to 'The Last of Us' was the dynamic use of audio to focus player attention on aspects of gameplay.

Listening focus

For much of the game the player must sneak around undetected if they wish to live. In order to be effective at moving around undetected the player can use a focus function where they are essentially listening for threats. When entered, the focus mode sends the bulk of in game audio through a low pass filter, with threat sounds such as enemy noises left largely unaffected. This works in combination with a visual effect of everything being slightly blurred and out of focus. This feature works to great effect especially as the main threat throughout the game comes from zombie like creatures that make a distinctive clicking sound. The 'clickers' have no vision and use this sound as an echo location mechanism, this also means that the player can also be detected if they make any sudden movements near a clicker. The focus feature at its core is a device which draws on more of the players attention, as well as make the player empathise with the characters situation. The fact that the player must actively listen (or pay the price) throughout much of the game is arguably a strong contributor to the games immersive nature.

The use of space is also a strong point in the game. Room spaces and tonal changes from one room to the next are highly audible. This is especially evident when in focus mode where the player can hear enemy sound from behind a wall. The spatialised presence of enemies along with realistic room acoustics solidifies the players 'presence' within the game. This is arguably one of the key immersive gameplay factor in the game as the player is essentially forced to tune into the character hearing system.

From an audio perspective a lot can be gained from the core mechanics of 'The Last Of Us ', the main point being to incorporate the players ears as much, if not more at times than the visual or mental elements of gameplay. It is the developers utilisation and effective balance of all the players senses that personally make this game so immersive.

In this case as in most AAA rated games the storyline is also a major hook to get the players attention. Here there has been an apocalyptic event where there is an outbreak of a fungal virus which takes over the human body, leading to a zombie like state. As impressive as the opening of the game was in 'The Last Of Us' it would have stood for nothing if the gameplay experience did not match expectations.

A story element for this project would be a consideration although not fitting with the objective of identifying the connection between audio and immersion.

Tuesday, 9 December 2014

Basic Sound Map for Dungeon Level

Here a basic map was created which identifies all sound within the level. This will be amended once a final level design has been established. The level design document will be presented as a walk through guide of the level, this will highlight any sound missed on this map.

A basic recording map will also be drawn up from this which will detail sounds that are to be recorded for the level. This will help guide recording sessions as well as maximise each session by ensuring that all sounds required are captured as needed.

Monday, 8 December 2014

Executive Summary - Up to December 2014

Development of Ideas

At the start of the honours year there were a number of different subject areas which were of interest, of these the most appealing was game audio and specifically the topic of audio implementation. This was a broad area however and further investigation was carried out as to what angle would be taken to direct an honours project. The core idea of the practical element was to create and implement audio assets into a game level with the use of a game engine, some initial research paths were covered to envision an approach to this and then further expanded upon with some clearer ideas.

An idea that was pursued involved an investigation into game audio fidelity where it was hoped that an argument could be made around the need for high fidelity audio in games. The idea consisted of creating multiple versions of the same game but each with a different audio fidelity. The questions centered around whether a player could tell the difference between each version and how audio fidelity affected their enjoyment of the level. In the first project presentation meeting this idea was discussed with staff and although the idea to design and implement audio using a game engine was seen as good starting point, the audio fidelity element was flawed, as discussed in the meeting.

Change of Direction

A new direction for the project was formulated based around the subject of player enjoyment. Some questions in mind that surrounded the topic were, why are games addictive by nature and what is it about games that draws the player in to the point of blocking out awareness of surroundings. More importantly what role does audio have in this element of games.
After some initial searching on this subject area terms describing the process of game engagement were found such as 'immersion', 'flow' and 'presence' to name a few. From here a basic statement of intent was drawn up for the honours project which tries to capture the basic direction of research. This was followed by a list which maps out the specifics of the project by highlighting possible objectives and tasks.A project timeline has also been drawn up in relation to the objectives, tasks and deliverables of the honours project for the remainder of the academic year. The latest update on the project direction takes the shape of an updated statement of intent however more work is to be done to finalise a specific research question which encapsulated the idea and aim as a whole.

Research

Initial research was centered around the idea of game audio implementation and primarily how this could been done given a lack of knowledge in this area. A few books were purchased which would be of use to fill in skill gaps, such as The Game Audio Tutorial (an interactive guide to audio implementation using UDK), The Essential Guide to Game Audio and The Master Handbook of Acoustics. The aim of these books was to try and understand some of the processes involved with game audio implementation and how sound behaves spatially as to better direct a more focused line of research.

After much consideration the project directed focus on player immersion in games and specifically what role audio played in the process. One of the main tasks involved with this route was to firstly try and identify what immersion is within the context of games and secondly could it be measured? Initial online searches provided some good peer reviewed papers on game immersion that also included a wealth of other peer reviewed references, some of which were added to the reading list.
Once a working definition had been established further reading provided some insight into immersion as a process. An analysis of a study by Brown & Cairns (2004) introduced the idea of structuring immersion into a framework as well as the idea of stages of immersion, happening over a period of game play time.

Another key aspect of research was to determine whether immersion can be tested and measured. Two study papers were analysed which appear to have successfully tested different aspects of immersion. The SCI-model developed by Ermi & Mäyrä (2005) provided a useful insight into game design aspects whilst showing their impact on immersion. The main similarity between the two studies was their use of questionnaires to assess test participants, although the study by Jannett et al (2008) included additional objective testing methods such as eye tracking. Neither specifically set out to test what role audio is playing but a lot can still be taken from the questions listed by Jannnett et al (2008) and perhaps they could be adapted to suite the honours project.
The most significant find throughout the reading list was a thesis written by Sander Huiberts (2010) which introduces the IZEA-model. The IZEA model is proposed as a framework to analyse the functioning of game audio during gameplay, which is then used to illustrate audio in relation to the immersion process. Huiberts uses the IZEA model in conjunction with existing theories on game immersion such as those from Ermi & Mäyrä (2005) and Brown & Cairns (2004) to highlight the role of audio (from a design perspective) in the process of player immersion. It is hoped that the use of the IZEA-model as a conceptual design tool will help direct key aspects of game audio design to create an immersive experience. Further critical analysis of the IZEA model and extensive thesis by Huiberts will be carried out to justify the reasoning for specific design aspects of the honours project.

Skills Development - Pre-Production Work

The bulk of practical work carried out thus far has been specifically targeting skills gaps, primarily aimed at the practical implementation of audio assets using a game engine. The UDK (Unreal Development Kit) game engine was chosen for the purposes of the honours project due to its visual based coding system (kismet) and advanced on-board (in comparrison to other engines such as Unity) audio system. With a lack of coding knowledge the visual format of the UDK kismet presents an easier learning curve which should allow for more time to focus on the project as a whole. The Game Audio Tutorial book has been utilised to help fill in the skills gap relating to implementation and using a game engine. The book contains a chapter by chapter practical walk through guide of audio implementation using UDK, as well as being packed with useful tips and tricks relating to game audio in general.
An in depth reflection on the exercises completed thus far which includes screenshots and video evidence as part of the documentation can be analysed HERE.

Level Concept

Originally the level was going to be fully designed and constructed for the purposes of the honours project, however this idea was dropped given the additional time required to gain enough skill in this area of development. Instead the idea to source an existing level which could be adapted for the project was viewed as the most logical step forward. Although the UDK engine ships with pre designed levels the search was broadened for independently created content which would be more suited to the project. From a personal point of view, games which have appeared to be more immersive than others, from a sound design position, have been in the genre of survival horror. Although some more research is required to back up this theory it was general starting point in the conception of the level design. A main characteristic of survival horror games is a darkened environment, so for the intentions of directing the focus of the player on audio this design aspect was a logical step. This observation has led to drawing up a list of games in this category for review and critical analysis, one of which has been completed thus far (Amnesia - The Dark Descent). The aim of these reviews is to provide some insight into what makes these games immersive from a design perspective, also importantly what practices can gained for the purpose of the honours project.

A pre-made level was sourced from an independent developer which ties in with the darkened style mentioned above. This level will be used for the implementation of audio within UDK. The level was created by Chris Holden who was contacted and has since given approval to use his level for this project. The level is set in a dungeon where the player has to navigate through various rooms and chambers to find keys and unlock chests of treasure with an aim to escape the dungeon. The gameplay encoded into the level is a good starting point, however the way the game plays out may be developed further once a full analysis of the level and coding behind it is complete.

Onwards

The next phase will involve developing the level in a manner which audio plays an important part of the gameplay experience. This will be influenced by further research as suggested in the reading list regarding specific design elements relating to immersion. The implementation will be realised and mapped out using skills gained from The Game Audio Tutorial book. Testing method will also be developed by using existing methods as a starting point.

Wednesday, 3 December 2014

Bibliography & Further Reading List

Here is a compilation of all sources that have been used in semester 1 as well as a list for future reading.

Bibliography

Brown, E. and Cairns, P. (2004). A Grounded Investigation of Game Immersion. Extended Abstracts of the 2004 Conference on Human Factors and Computing Systems, Vienna April 24-29 2004. New York: ACM. pp. 1297-1300. [online]. Available from: http://www-users.cs.york.ac.uk/~pcairns/papers/Immersion.pdf [Accessed 1st November 2014]

Calleja, G. (2007). Revising Immersion: A Conceptual Model for the Analysis of Digital Game Involvement. Digital Games Research Association (DiGRA). Situated Play, Proceedings of DiGRA 2007 Conference. [online]. Available from: http://www.digra.org/wp-content/uploads/digital-library/07312.10496.pdf [Accessed 3rd November 2014]

Collins, K. (2008). Game Sound An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Cambridge, Massachusetts: Massachusetts Institute of Technology.

Dovey, J. And Kennedy, H.W. (2006). Game Cultures: Computer games as new Media. Berkshire: Open University Press.

Ermi, L. and Mäyrä, F. (2005) Fundamantal Components of the Gameplay Experience: Analysing Immersion [online]. In Proceedings of DiGRA 2005 Conference: Changing Views ‐ Worlds in Play. Available from: http://www.digra.org/wp-content/uploads/digital-library/06276.41516.pdf [Accessed October 26 2014]

Huiberts, S. 2010. Captivating Sound, The Role of Audio for Immersion in Computer Games. [online]. Available from: http://download.captivatingsound.com/Sander_Huiberts_CaptivatingSound.pdf [Accessed 20 October 2014].

Jennett et al. (2008). Measuring and defining the experience of immersion in games. Int. J. Human-Computer Studies. 66(May): pp. 641-661.

McMahan, A. (2003). The Video Game Theory Reader. Chapter 3 - Immersion, Engagement, and Presence. New York: Toylor & Francis Books, Inc.

Murray, J. (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace. New York: Simon & Schuster.

Stevens and Raybould. 2011. The Game Audio Tutorial: A Practical Guide to Sound and Music for Interactive Games. Burlington, MA: Focal Press.

Stockburger, A. (2003). THE GAME ENVIRONMENT FROM AN AUDITIVE PERSPECTIVE. Level Up, Digital Games Research Conference 2003, Utrecht. [online]. Available from: http://www.stockburger.at/files/2010/04/gameinvironment_stockburger1.pdf [Accessed 23 October 2014]

Strauss, A. and Corbin, J. 1998. Basics of Qualitative Research. 2nd ed. Sage Publications, Inc. [online]. Available from: http://stiba-malang.ac.id/uploadbank/pustaka/RM/BASIC%20OF%20QUALITATIVE%20RESEARCH.pdf [Accessed 1st November 2014]

Taylor, L. 2002. Video Games: Perspective, Point-of-View, and Immersion. [online]. Available from: http://etd.fcla.edu/UF/UFE1000166/taylor_l.pdf [Accessed 25 October 2014]

The Role of Audio in Video Game Immersion

Project Areas