Location Recording Session 01

Today some recordings were carried out within a disused farm building containing large steel grain bins. The plan was to capture some metallic sounds to be used in the creation of assets for the dungeon level. A Zoom H2n along with a guitar pickup was used to capture the majority of the metallic recordings.

The guitar pickup was most successful when held a few millimetres away from the metal of the grain bin, whilst using a piece of  angle steel to to create impact sounds and scraping sounds. By using the guitar pick up I maximised the resonant sound emitted from the bin whilst eliminating all room space noise. Examples are below:


Some additional recordings were also captured from various mechanical devices throughout the building as well as some footstep sounds, some additional editing of these will be required however due to the nature of the location. Background noise levels were an issue along with the large reverberant space of the building but that aside I feel the sounds can be cleaned up for application to the project. 

Dungeon Escape level - Evaluation

The level is set in what appears to be an underground dungeon consisting of stone rooms and corridors. The dungeon is lit by flame torches for the most part, aside from one area which opens up into a cave which is lit through the roof (an opening to the outside world and moodlit sky).

The game objective is to try and escape the dungeon by finding 3 relics which are to be inserted into an alter which in turn opens the escape door. There are also 3 treasure chests in the level that can be opened by keys which are hidden within the level. In order to find both relics and keys the player must navigate through the dungeon by solving how to open gateways using various switches.

Some design concerns which need addressed:

• These is no backstory or direction, the player has no idea why they are there and what they are supposed to do.

The initial idea here is to create a short backstory which test players can read before playing the level. Another possible route is to use expositional dialogue spoken in game by the character as they navigate through the level.

• These are no enemies or any traps which end the characters life, this could lead to player boredom. 

Sound could be used to imply the threat of danger such as the sound of a beast stalking the player, however this would only work on one completion of the level. Creating an actual creature which is present and visible is an option but is also well outside of my skillset. Perhaps an invisible enemy could be created and tracked by the player through the use of sound. A timer could also be enforced to push the player to focus more on completion.

• There is no reward by collecting treasure, when the player escapes the end screen either reads that you escaped with or without treasure.

Hearing a gratifying sound when treasure is found will be essential but also more recognition should be awarded at the end sequence.

Basic Recording Map

A map was created in reference to the Basic Sound Map and details which sounds are required to be recorded. Session plans will follow on from this map and detail suitable locations, props, recording techniques and so on.

Game Level Concept - Game review (The Last of Us)

This is a review of some key audio features included in "The Last of Us".

One of the main factors that personally drew me to 'The Last of Us' was the dynamic use of audio to focus player attention on aspects of gameplay. 

Listening focus

For much of the game the player must sneak around undetected if they wish to live. In order to be effective at moving around undetected the player can use a focus function where they are essentially listening for threats. When entered, the focus mode sends the bulk of in game audio through a low pass filter, with threat sounds such as enemy noises left largely unaffected. This works in combination with a visual effect of everything being slightly blurred and out of focus. This feature works to great effect especially as the main threat throughout the game comes from zombie like creatures that make a distinctive clicking sound. The 'clickers' have no vision and use this sound as an echo location mechanism, this also means that the player can also be detected if they make any sudden movements near a clicker. The focus feature at its core is a device which draws on more of the players attention, as well as make the player empathise with the characters situation. The fact that the player must actively listen (or pay the price) throughout much of the game is arguably a strong contributor to the games immersive nature.

The use of space is also a strong point in the game. Room spaces and tonal changes from one room to the next are highly audible. This is especially evident when in focus mode where the player can hear enemy sound from behind a wall. The spatialised presence of enemies along with realistic room acoustics solidifies the players 'presence' within the game. This is arguably one of the key immersive gameplay factor in the game as the player is essentially forced to tune into the character hearing system.

From an audio perspective a lot can be gained from the core mechanics of 'The Last Of Us ', the main point being to incorporate the players ears as much, if not more at times than the visual or mental elements of gameplay. It is the developers utilisation and effective balance of all the players senses that personally make this game so immersive.    

In this case as in most AAA rated games the storyline is also a major hook to get the players attention. Here there has been an apocalyptic event where there is an outbreak of a fungal virus which takes over the human body, leading to a zombie like state. As impressive as the opening of the game was in 'The Last Of Us' it would have stood for nothing if the gameplay experience did not match expectations.

A story element for this project would be a consideration although not fitting with the objective of identifying the connection between audio and immersion.

Basic Sound Map for Dungeon Level

Here a basic map was created which identifies all sound within the level. This will be amended once a final level design has been established. The level design document will be presented as a walk through guide of the level, this will highlight any sound missed on this map.

A basic recording map will also be drawn up from this which will detail sounds that are to be recorded for the level. This will help guide recording sessions as well as maximise each session by ensuring that all sounds required are captured as needed.

Executive Summary - Up to December 2014

Development of Ideas

At the start of the honours year there were a number of different subject areas which were of interest, of these the most appealing was game audio and specifically the topic of audio implementation. This was a broad area however and further investigation was carried out as to what angle would be taken to direct an honours project. The core idea of the practical element was to create and implement audio assets into a game level with the use of a game engine, some initial research paths were covered to envision an approach to this and then further expanded upon with some clearer ideas.

An idea that was pursued involved an investigation into game audio fidelity where it was hoped that an argument could be made around the need for high fidelity audio in games. The idea consisted of creating multiple versions of the same game but each with a different audio fidelity. The questions centered around whether a player could tell the difference between each version and how audio fidelity affected their enjoyment of the level. In the first project presentation meeting this idea was discussed with staff and although the idea to design and implement audio using a game engine was seen as good starting point, the audio fidelity element was flawed, as discussed in the meeting.

Change of Direction

A new direction for the project was formulated based around the subject of player enjoyment. Some questions in mind that surrounded the topic were, why are games addictive by nature and what is it about games that draws the player in to the point of blocking out awareness of surroundings. More importantly what role does audio have in this element of games.
After some initial searching on this subject area terms describing the process of game engagement were found such as 'immersion', 'flow' and 'presence' to name a few. From here a basic statement of intent was drawn up for the honours project which tries to capture the basic direction of research. This was followed by a list which maps out the specifics of the project by highlighting possible objectives and tasks.A project timeline has also been drawn up in relation to the objectives, tasks and deliverables of the honours project for the remainder of the academic year. The latest update on the project direction takes the shape of an updated statement of intent however more work is to be done to finalise a specific research question which encapsulated the idea and aim as a whole. 

Research

Initial research was centered around the idea of game audio implementation and primarily how this could been done given a lack of knowledge in this area. A few books were purchased which would be of use to fill in skill gaps, such as The Game Audio Tutorial (an interactive guide to audio implementation using UDK), The Essential Guide to Game Audio and The Master Handbook of Acoustics. The aim of these books was to try and understand some of the processes involved with game audio implementation and how sound behaves spatially as to better direct a more focused line of research.

After much consideration the project directed focus on player immersion in games and specifically what role audio played in the process. One of the main tasks involved with this route was to firstly try and identify what immersion is within the context of games and secondly could it be measured? Initial online searches provided some good peer reviewed papers on game immersion that also included a wealth of other peer reviewed references, some of which were added to the reading list. 
Once a working definition had been established further reading provided some insight into immersion as a process. An analysis of a study by Brown & Cairns (2004) introduced the idea of structuring immersion into a framework as well as the idea of stages of immersion, happening over a period of game play time.

Another key aspect of research was to determine whether immersion can be tested and measured. Two study papers were analysed which appear to have successfully tested different aspects of immersion. The SCI-model developed by Ermi & Mäyrä (2005) provided a useful insight into game design aspects whilst showing their impact on immersion. The main similarity between the two studies was their use of questionnaires to assess test participants, although the study by Jannett et al (2008) included additional objective testing methods such as eye tracking. Neither specifically set out to test what role audio is playing but a lot can still be taken from the questions listed by Jannnett et al (2008) and perhaps they could be adapted to suite the honours project.  
The most significant find throughout the reading list was a thesis written by Sander Huiberts (2010) which introduces the IZEA-model. The IZEA model is proposed as a framework to analyse the functioning of game audio during gameplay, which is then used to illustrate audio in relation to the immersion process. Huiberts uses the IZEA model in conjunction with existing theories on game immersion such as those from Ermi & Mäyrä (2005) and Brown & Cairns (2004) to highlight the role of audio (from a design perspective) in the process of player immersion. It is hoped that the use of the IZEA-model as a conceptual design tool will help direct key aspects of game audio design to create an immersive experience. Further critical analysis of the IZEA model and extensive thesis by Huiberts will be carried out to justify the reasoning for specific design aspects of the honours project. 

Skills Development - Pre-Production Work

The bulk of practical work carried out thus far has been specifically targeting skills gaps, primarily aimed at the practical implementation of audio assets using a game engine. The UDK (Unreal Development Kit) game engine was chosen for the purposes of the honours project due to its visual based coding system (kismet) and advanced on-board (in comparrison to other engines such as Unity) audio system. With a lack of coding knowledge the visual format of the UDK kismet presents an easier learning curve which should allow for more time to focus on the project as a whole. The Game Audio Tutorial book has been utilised to help fill in the skills gap relating to implementation and using a game engine. The book contains a chapter by chapter practical walk through guide of audio implementation using UDK, as well as being packed with useful tips and tricks relating to game audio in general.
An in depth reflection on the exercises completed thus far which includes screenshots and video evidence as part of the documentation can be analysed HERE.

Level Concept

Originally the level was going to be fully designed and constructed for the purposes of the honours project, however this idea was dropped given the additional time required to gain enough skill in this area of development. Instead the idea to source an existing level which could be adapted for the project was viewed as the most logical step forward. Although the UDK engine ships with pre designed levels the search was broadened for independently created content which would be more suited to the project. From a personal point of view, games which have appeared to be more immersive than others, from a sound design position, have been in the genre of survival horror. Although some more research is required to back up this theory it was general starting point in the conception of the level design. A main characteristic of survival horror games is a darkened environment, so for the intentions of directing the focus of the player on audio this design aspect was a logical step. This observation has led to drawing up a list of games in this category for review and critical analysis, one of which has been completed thus far (Amnesia - The Dark Descent). The aim of these reviews is to provide some insight into what makes these games immersive from a design perspective, also importantly what practices can gained for the purpose of the honours project.

A pre-made level was sourced from an independent developer which ties in with the darkened style mentioned above. This level will be used for the implementation of audio within UDK. The level was created by Chris Holden who was contacted and has since given approval to use his level for this project. The level is set in a dungeon where the player has to navigate through various rooms and chambers to find keys and unlock chests of treasure with an aim to escape the dungeon. The gameplay encoded into the level is a good starting point, however the way the game plays out may be developed further once a full analysis of the level and coding behind it is complete. 

Onwards

The next phase will involve developing the level in a manner which audio plays an important part of the gameplay experience. This will be influenced by further research as suggested in the reading list regarding specific design elements relating to immersion. The implementation will be realised and mapped out using skills gained from The Game Audio Tutorial book. Testing method will also be developed by using existing methods as a starting point.

Bibliography & Further Reading List

Here is a compilation of all sources that have been used in semester 1 as well as a list for future reading.

Bibliography

Brown, E. and Cairns, P. (2004). A Grounded Investigation of Game Immersion. Extended Abstracts of the 2004 Conference on Human Factors and Computing Systems, Vienna April 24-29 2004. New York: ACM. pp. 1297-1300. [online]. Available from: http://www-users.cs.york.ac.uk/~pcairns/papers/Immersion.pdf [Accessed 1st November 2014]

Calleja, G. (2007). Revising Immersion: A Conceptual Model for the Analysis of Digital Game Involvement. Digital Games Research Association (DiGRA). Situated Play, Proceedings of DiGRA 2007 Conference. [online]. Available from: http://www.digra.org/wp-content/uploads/digital-library/07312.10496.pdf [Accessed 3rd November 2014]

Collins, K. (2008). Game Sound An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Cambridge, Massachusetts: Massachusetts Institute of Technology.

Dovey, J. And Kennedy, H.W. (2006). Game Cultures: Computer games as new Media. Berkshire: Open University Press.

Ermi, L. and Mäyrä, F. (2005) Fundamantal Components of the Gameplay Experience: Analysing Immersion [online]. In Proceedings of DiGRA 2005 Conference: Changing Views ‐ Worlds in Play. Available from: http://www.digra.org/wp-content/uploads/digital-library/06276.41516.pdf [Accessed October 26 2014]

Huiberts, S. 2010. Captivating Sound, The Role of Audio for Immersion in Computer Games. [online]. Available from: http://download.captivatingsound.com/Sander_Huiberts_CaptivatingSound.pdf [Accessed 20 October 2014].

Jennett et al. (2008). Measuring and defining the experience of immersion in games. Int. J. Human-Computer Studies. 66(May): pp. 641-661.

McMahan, A. (2003). The Video Game Theory Reader. Chapter 3 - Immersion, Engagement, and Presence. New York: Toylor & Francis Books, Inc.

Murray, J. (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace. New York: Simon & Schuster.

Stevens and Raybould. 2011. The Game Audio Tutorial: A Practical Guide to Sound and Music for Interactive Games. Burlington, MA: Focal Press.

Stockburger, A. (2003). THE GAME ENVIRONMENT FROM AN AUDITIVE PERSPECTIVE. Level Up, Digital Games Research Conference 2003, Utrecht. [online]. Available from: http://www.stockburger.at/files/2010/04/gameinvironment_stockburger1.pdf [Accessed 23 October 2014]

Strauss, A. and Corbin, J. 1998. Basics of Qualitative Research. 2nd ed. Sage Publications, Inc. [online]. Available from: http://stiba-malang.ac.id/uploadbank/pustaka/RM/BASIC%20OF%20QUALITATIVE%20RESEARCH.pdf [Accessed 1st November 2014]

Taylor, L. 2002. Video Games: Perspective, Point-of-View, and Immersion. [online]. Available from: http://etd.fcla.edu/UF/UFE1000166/taylor_l.pdf [Accessed 25 October 2014]

Further Reading List

Collins, K. (2008). Game Sound An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Cambridge, Massachusetts: Massachusetts Institute of Technology.

Droumeva, M. 2005. Understanding immersive audio: A historical and socio-cultural exploration of auditory displays. Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display. Limerick, Ireland, July 6-9 2005. Georgia Institute of Technology. [online] Available from: https://smartech.gatech.edu/bitstream/handle/1853/50196/Droumeva2005.pdf?sequence=1 Accessed [11 November 2014]

Ekman, I. 2013. On the Desire to Not Kill Your Players: Rethinking Sound in Pervasive and Mixed Reality Games. Conference: Foundations of Digital Games 2013. [online] Available from: http://www.fdg2013.org/program/papers/paper19_ekman.pdf Accessed on [20 November 2014]

Grimshaw, 2011. Game Sound Technology and Player Interaction: Concepts and Developments. Hershey PA. Information Science Reference (an imprint of IGI Global).

Grimshaw, Charlton and Jagger, 2011. First-Person Shooters: Immersion and Attention. Eludamos. Journal for Computer Game Culture. 2011; 5 (1), pp. 29-44 [online] Available from: http://www.eludamos.org/index.php/eludamos/article/viewArticle/vol5no1-3/html3 Accessed [19 November 2014]

Grimshaw, Lindley and Nacke. 2008. Sound and immersion in the first-person shooter: mixed measurement of the player's sonic experience. Audio Mostly - a conference on interaction with sound. Piteå, Sweden October 22-23 2008. www.audiomostly.com. pp. 9-15. [online] Available from: http://ubir.bolton.ac.uk/244/1/gcct_conferencepr-7.pdf Accessed [15 November 2014]

Horowitz and Looney. 2014 The essential guide to game audio: The theory and practice of sound for games. Burlington, MA: Focal Press.

Huiberts, S. 2010. Captivating Sound, The Role of Audio for Immersion in Computer Games. [online]. Available from: http://download.captivatingsound.com/Sander_Huiberts_CaptivatingSound.pdf [Accessed 20 October 2014].

Shilling, Zyda and Wardynski. 2002. Introducing Emotion into Military Simulation and Videogame Design: America’s Army: Operations and VIRTE. [online]. Available from: http://calhoun.nps.edu/bitstream/handle/10945/41580/ShillingGameon2002.pdf?sequence=1 [Accessed 10 November 2014]

Stevens and Raybould. 2011. The Game Audio Tutorial: A Practical Guide to Sound and Music for Interactive Games. Burlington, MA: Focal Press.

Research Proposal

Research Proposal

Can game audio be specifically designed and implemented for the purpose of leading the player into an immersive experience?

Abstract

This research proposal sets out a framework with clear objectives in which to investigate the impact audio has on the immersive nature of computer games, whilst also determining whether its effectiveness can be pre-designed and measured. Current definitions of the term ‘immersion’ will be discussed in detail through an analysis of literature and research carried out by practitioners in the field of game study. This overview will bring into focus the aural aspect of immersion and the research which is directed at this specific side of game design.

The proposal will set out a working plan of action to utilise current research on the subject of game immersion for a practical application which demonstrates how audio can be designed and implemented to enhance the immersive nature of games. This will be proposed as the implementation and design of audio assets for a short ‘First Person Shooter’ style gaming level where two versions of the level (one enhanced for immersion) will be tested and contrasted. Testing methods will be discussed based on recent studies carried out whilst suggestions put forth to tailor these methods to the project requirements. The proposal will also detail a timeline in which all aspects of project deliverables are mapped out over the remainder of the honours year. Finally a summary of the projects expected outcomes of the will provided.

Introduction

Can game audio be specifically designed and implemented for the purpose of leading the player into an immersive experience?

The term immersion or immersive is often used by gamers when describing games but what exactly does it mean to be immersed in a game and what defines an immersive experience? This project proposes to investigate the term immersion and define its meaning within the context of playing video games. With a clearer view on the process of becoming immersed in computer games, the project will focus in on the aural element of immersion. From here the relationship will be highlighted between immersion and game audio design whilst demonstrating its importance to the overall experience.

The project aims to clearly identify the bond between audio and immersion in games and will include research and critical analysis of current theories and studies on the subject. The research and investigation output will be utilised to create a 'First Person Shooter' gaming level which will be focused on using audio as an interactive design element to produce an immersive experience. This level will then be tested a variety of ways to firstly try and quantify the impact audio has on the immersive experience. Secondly the results should shed some light on whether audio can indeed be designed and implemented with immersion as a goal.

Main Objectives:

 Research – defining game immersion, case studies, similar projects
 Audio Asset Production – planning, recording, editing, testing
 Game Level Build – Skills based research, level editing, audio implementation, testing
 Project Evaluation/Testing - establish testing methods, source test group, carry out tests, reflect upon results

Literature Review

With the projects’ objective specifically aimed at creating an immersive experience the initial goal was to identify what exactly the term immersion means within the context of games and secondly, what is the importance, and role of audio within this context?
Janet Murray describes immersion as;

“Immersion is a metaphorical term derived from the physical experience of being submerged in water. We seek the same feeling from a psychologically immersive experience that we do from a plunge in the ocean or swimming pool: the sensation of being surrounded by a completely other reality, as different as water is from air, that takes over all of our attention, our whole perceptual apparatus” (Murray, 1997, p. 98)

Dovey and Kennedy describe immersion in a similar fashion by adding that it is “the experience of losing a sense of embodiment in the present whilst connecting on a meditated environment” (Dovey & Kennedy, 2006, p. 146). It is worth noting that immersion can also be related to other forms of entertainment such as books, or film but the difference here is that these are static or linear as opposed to interactive experiences. Brown & Cairns (2004) highlight that the term immersion or immersive is used frequently to describe games throughout the gaming community but add that the term is also used without a coherent explanation (Brown & Cairns. 2004, p.1). Other terms have also been used to describe the immersive state such as ‘Incorporation’ (Calleja, 2007) and ‘Presence’ (McMahan, 2003). Jennett et al (2008) also mention ‘Presence’ as well as other concepts such as ‘Flow’ and ‘Cognitive absorption’. However they argue that “immersion is clearly distinct from these established concepts and a better understanding of immersion would be crucial in understanding the relationship between people and videogames” (Jennett et al, 2008, p. 642).

A closer look at the theories behind game immersion provide a more unique analysis, whilst narrowing down the broad range of terms and descriptions that have been presented thus far. Laurie Taylor (2002) expands the definition in a gaming context by splitting it into two separate stages of engagement “diegetic immersion, where the player is immersed in the act of playing the video game, and as intra-diegetic or situated immersion, where the player is immersed in playing the game and in the experience of the game space as a spatial and narrated space” (Taylor. L. 2002, p. 12). The idea of different stages of engagement is shared by Brown & Cairns (2004). Their study tries to define immersion within the context of games using a ‘grounded theory’ approach (a method of qualitative research developed by Strauss & Corbin (1998)) by analysing interviews with seven gamers about their experiences when playing games. Much like Taylor (2002) their results define immersion as a process of stages of engagement with the game which “moves along the path of time and is controlled by barriers” (Brown & Cairns. 2004, p. 2), only here three stages are suggested.
The three stages of immersion described by Brown & Cairns are ‘Engagement’, ‘Engrossment’ and ‘Total Immersion’ with the latter being described as ‘Presence’. The barriers describe gateways to each of the stages and by opening these the player can move forward through these stages of immersion. For example the first stage is ‘Engagement’, this is where the player must first engage with the game, so a potential barrier might be that the player does not like the genre or style of the game, thus creating a barrier for further immersion to take place (Brown & Cairns. 2004, p.2). As stated by Brown & Cairns “To lower the barriers to enter this level, the gamer needs to invest time, effort, and attention” (2004, p. 2). They also describe three elements of player attention which are linked to the immersion process: visual, auditory and mental. They go on to suggest that the more of these attention aspects are pulling on the players’ senses the greater involved with the
game the player will become. “If gamers need to attend to sound, as well as sight more effort is needed to be placed into the game. The more attention and effort invested, the more immersed a gamer can feel.” (Brown & Cairns. 2004, p. 3).

Ermi and Mäyrä (2005) also pick up on the concept of stages of immersion with the development of the SCI-model which links aspects of game design to immersion and highlights their influence on the process. This model identifies three aspects of game design which have been drawn from an analysis of interviews with gamers regarding their playing experiences, it is then concluded how much these aspects impact the immersive quality of games. The design aspects are labelled ‘audio-visual quality & style’, ‘level of challenge’ and ‘imaginary world and fantasy’ which are linked to the core values of the SCI-model; Sensory immersion, Challenge based immersion and Imaginative immersion respectively (Ermi and Mäyrä 2005, pp. 7-9). The results from Ermi and Mäyrä (2005) as well as Brown & Cairns (2004) both acknowledge that audio play a role in the immersion processes they describe but neither study examines the audio factor in great detail.

Collins (2008) picks up on the theory of Ermi and Mäyrä (2005) with a more direct stance on game audio by suggesting that ‘imaginary world and fantasy’ element of the SCI-model “is strongly enhanced by audio” (Collins 2008, p.134). Collins also talks about the link between immersion and audio in more general terms by adding “The illusion of being immersed in a three-dimensional atmosphere is greatly enhanced by the audio” (Collins 2008, p. 132)

Huiberts (2010) makes a more explicit connection between immersion and audio in a thesis which examines the relationship from a conceptual design standing. The IZEA model (Huiberts. S 2010, pp. 20-35) is proposed as a framework to analyse the functioning of game audio during gameplay, which is then used to illustrate audio in relation to the immersion process. Huiberts uses the IZEA model in conjunction with existing theories on game immersion such as those from Ermi and Mäyrä (2005) and Brown & Cairns (2004) to highlight the role of audio (from a design perspective) in the process of player immersion. Huiberts study, which was largely based around an analysis of user surveys (Creative Heroes 2007) concludes “Audio is capable of enhancing the three dimensions of immersion by enhancing the sensory connection, the feeling of flow and the feeling of empathy of the player” (Huiberts. S 2010, p. 101).

Huiberts (2010) sets out a unique framework relating to the immersive properties of game audio which also encapsulates the ideas and theories of past research on game immersion. However there is no practical application of this demonstrated as part of the research carried out by Huiberts. This honours project proposes to act as a working demonstration of how audio can be implemented into a video game with the purpose of creating an immersive experience. The research alongside the practical application and results of this honours project will serve as a valuable base upon which to expand current knowledge and understanding regarding the design and implementation of sound for immersion.

Project Methodology

The IZEA model proposed by Huiberts (2010) will be utilised to help conceptualise the game audio design for the project. Further analysis of the IZEA model will be carried out to establish the connections Huiberts (2010) makes with the stages of immersion described by Ermi and Mäyrä (2005) and Brown & Cairns (2004). This section of research will justify the reasoning for specific design aspects of the test level.

It is the intention to try and design the test level with audio as the main focus rather than designing audio to fit the visual style. However this will be limited as the project will be working with existing visual assets which are not intended to be altered significantly. Gaining the skills and knowledge to build a visually convincing gaming environment from the ground up is beyond the scope and time constraints of this project. However design elements involving the way the test level functions from an interactive point of view will be a focus point. This will enable the audio to function more decisively as an immersive component of gameplay.

Two versions of the level will be created for an A/B comparison. One will contain an audio orientated design with a clear focus on the player having to listen in order to complete the level. The other will include the same sounds but they will not be used in the same interactive manner, instead the audio implementation and mixing style will be simplified and the player will be provided visual cues to complete the level.

A quantitative method will be used to collect data regarding the test level by the way of a user survey. This will be based on previous work carried out by Jennett et al (2008, pp. 658-650) who developed a survey (constructed on a Likert scale) to measure the level of immersion felt by each of their test subjects. The questions may be developed further to gain a better insight into the role audio is playing in the process. Further research into the theory behind these questions will also be carried out to gain a better understanding as to their meaning and purpose. As well as the survey a short interview should take place with participants with more direct questions relating to the audio side of the experience to provide some qualitative data for analysis.

A case study will also be carried out on games which have been reviewed as having an immersive audio experience. This may guide some of the sound design choices made for this project whilst highlighting if there is a specific genre or style of audio design that is more immersive than others.

Schedule

FOUND HERE

Expected Outcomes

The expectation is that the test results will show that audio offers a significant opportunity to enhance player immersion in games. The project should also demonstrate that game audio can be designed with immersion in mind whilst highlighting its potential to evoke emotional responses from the player.
The hope is that this project will set a foundation on which further research can be carried out into the area of video game immersion and specifically how the practical application of audio can greatly enhance the experience.

References

Brown, E. and Cairns, P. (2004). A Grounded Investigation of Game Immersion. Extended Abstracts of the 2004 Conference on Human Factors and Computing Systems, Vienna April 24-29 2004. New York: ACM. pp. 1297-1300. [online]. Available from: http://www-users.cs.york.ac.uk/~pcairns/papers/Immersion.pdf [Accessed 1st November 2014]

Calleja, G. (2007). Revising Immersion: A Conceptual Model for the Analysis of Digital Game Involvement. Digital Games Research Association (DiGRA). Situated Play, Proceedings of DiGRA 2007 Conference. [online]. Available from: http://www.digra.org/wp-content/uploads/digital-library/07312.10496.pdf [Accessed 7th November 2014]

Collins, K. (2008). Game Sound An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Cambridge, Massachusetts: Massachusetts Institute of Technology.

Dovey, J. And Kennedy, H.W. (2006). Game Cultures: Computer games as new Media. Berkshire: Open University Press.

Ermi, L. and Mäyrä, F. (2005) Fundamantal Components of the Gameplay Experience: Analysing Immersion [online]. In Proceedings of DiGRA 2005 Conference: Changing Views ‐ Worlds in Play. Available from: http://www.digra.org/wp-content/uploads/digital-library/06276.41516.pdf [Accessed October 26 2014]

Huiberts, S. 2010. Captivating Sound, The Role of Audio for Immersion in Computer Games. [online]. Available from: http://download.captivatingsound.com/Sander_Huiberts_CaptivatingSound.pdf [Accessed 20 October 2014].

Jennett et al. (2008). Measuring and defining the experience of immersion in games. Int. J. Human-Computer Studies. 66(May): pp. 641-661.

McMahan, A. (2003). The Video Game Theory Reader. Chapter 3 - Immersion, Engagement, and Presence. New York: Toylor & Francis Books, Inc.

Murray, J. (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace. New York: Simon & Schuster.

Strauss, A. and Corbin, J. 1998. Basics of Qualitative Research. 2nd ed. Sage Publications, Inc. [online]. Available from: http://stiba-malang.ac.id/uploadbank/pustaka/RM/BASIC%20OF%20QUALITATIVE%20RESEARCH.pdf [Accessed 1st November 2014]

Taylor, L. 2002. Video Games: Perspective, Point-of-View, and Immersion. [online]. Available from: http://etd.fcla.edu/UF/UFE1000166/taylor_l.pdf [Accessed 25 October 2014]

Project aim update - Statement of intent

Can game audio be specifically designed and implemented for the purpose of leading theplayer into an immersive experience?

The term immersion or immersive is often used by gamers when describing games but what
exactly does it mean to be immersed in a game and what defines an immersive experience? This
project proposes to investigate the term immersion and define its meaning within the context of
playing video games. With a clearer view on the process of becoming immersed in computer games,
the project will focus in on the aural element of immersion. From here the relationship will be
highlighted between immersion and game audio design whilst demonstrating its importance to the
overall experience.

The project aims to clearly identify the bond between audio and immersion in games and
will include research and critical analysis of current theories and studies on the subject. The research
and investigation output will be utilised to create a 'First Person Shooter' gaming level which will be
focused on using audio as an interactive design element to produce an immersive experience. This
level will then be tested a variety of ways to firstly try and quantify the impact audio has on the
immersive experience. Secondly the results should shed some light on whether audio can indeed be
designed and implemented with immersion as a goal.

• Main Objectives: Research – defining game immersion, case studies, similar projects
• Audio Asset Production – planning, recording, editing, testing
• Game Level Build – Skills based research, level editing, audio implementation, testing
• Project Evaluation/Testing - establish testing methods, source test group, carry out tests, reflect upon results

Project Timeline

A timeline has been drawn up which highlights the objectives and tasks associated with the honours project during semester two.

Research - Immersion Testing Methods

Testing The Level of Immersion in Games

Testing the level of immersion in games is not a well researched area and only a handful of papers have been sourced thus far. The following is a compilation of testing methods that have been used by researchers in this area.

Ermi & Mäyrä (2005)  Fundamantal Components of the Gameplay Experience: Analysing Immersion

In this study Ermi & Mäyrä (2005) examine immersion from a sensory perspective. They devised the SCI-Model which defines three aspects of sensory immersion .These are ‘audio-visual quality & style’, ‘level of challenge’ and ‘imaginary world and fantasy’ which are linked to the core values of the SCI-model, Sensory immersion, Challenge based immersion and Imaginative immersion respectively.
Their analysis is based on a questionnaire completed by gamers which involves  thirty statements addressing all three aspects of the model, with each statement answered on a 5-point Likert scale.  From this they draw conclusions about each of the games studied, which shows the levels of immersion (in relation to the three aspects of the SCI-Model) found in each game.

The test results for the SCI-model (Ermi & Mäyrä 2005, pp. 9-11) show that from the games that were used, the ‘challenge-based immersion’ aspect played a significant role in the overall immersion of the player. With this factor identified, an objective of this honours project is to attempt enrol the audio content as part of the game challenge. A situation of objective listening should be key to the game design where the player must be fully engaged with audio content in order to complete tasks and progress. Also worth noting is that the highest scoring game in their study from an immersive point of view was a FPS.

Jennett et al (2008) Measuring and defining the experience of immersion in games

The study by Jannett et al (2008) also includes a similar questionnaire to identify each players subjective experience of immersion. The study also includes three testing methods (each accompanied by a questionnaire to identify each players subjective experience of immersion) in order to objectively assess player immersion. These took shape as firstly testing player ability to switch between tasks (from playing a game to a task outside of the game). They showed in this test that the longer the player took to switch from the game to completing the external task, the more immersed in the game they were. The results do not point to any specific aspect of immersion however so this method may not be very suited to the needs of the honours project.

The second test used eye tracking software/hardware to measure players eye movement during game play. these results showed, along with the  supporting questionnaire, that the more the subjects eyes moved during game play the less immersed they were. They highlighted that a player who was immersed would be more fixated on the screen and with the game actions.

The last test measured whether game speed or pace influenced game immersion. In this test four versions of a simple clicking game (each with a different pace) were given to participants to play. These were rated as 'Slow', 'Self Paced', 'Fast' and 'Increasing'. Interestingly the results showed that the game with the 'Increasing' pace was rated as the most immersive, second was the 'Fast' game, third was 'Slow' and lastly 'Self Paced'. This is a good find in terms of creating an immersive level, it would appear that forcing a game pace upon the player has an impact on immersion.


Conclusions

Developing a questionnaire in relation to the effect of audio on immersion should make up part of the testing methods for the honours project. A closer look at the questions presented by both Jannett et al (2008) and Ermi & Mäyrä (2005) will follow next to identify if the formatting can be applied to the honours project.

References:

Ermi, L. and Mäyrä, F. (2005) Fundamantal Components of the Gameplay Experience: Analysing Immersion [online]. In Proceedings of DiGRA 2005 Conference: Changing Views ‐ Worlds in Play. Available from: http://www.digra.org/wp-content/uploads/digital-library/06276.41516.pdf [Accessed October 26 2014]

Jennett et al. (2008). Measuring and defining the experience of immersion in games. Int. J. Human-Computer Studies. 66(May): pp. 641-661.

Research - The IZEA Model

The IEZA Model

“The model is intended as a conceptual tool for conceptually designing and reflecting upon designing game audio and in this thesis its main function is to provide a coherent vocabulary for the definition and typology of game audio and to provide a conceptual model for understanding the functioning of game audio in relation to immersion”. (Huiberts. S, 2010, p. 12)

Game audio typologies

The classification and typology of audio in games are expressed in a number of ways by practitioners in the study of game audio, each structuring game audio content into meaningful partitions with regards to functionality. Sander Huiberts (2010, pp.15-20) describes some of the different frameworks that have been developed, which can be applied to help structure game audio content. Huiberts also points out some inconsistencies relating to each framework and makes it clear that that the subject of game audio needs a more coherent model. Some of the frameworks discussed by Huiberts are as follows:

·        Production based approach (basic framework consisting of sound types, for example vocal recordings, sound effect files and music files)

·        Game specific approach (sounds are organised into different groups depending on how they are implemented into the game, for example avatar sounds, object sounds, (nonplayer) character sounds, ornamental sounds and instructions.)

·        Approach based on origin of sound within the game environment (This includes the score, effect, interface, zone and speech which makes up the game environment)

·        Approach based on the meaning of sound to the player (categorised into four sections: diegetic sounds, symbolic sounds, masking sounds and non-diegetic sounds)

These approaches are helpful in defining certain aspects of game audio and would be of use at various stages of production but they do not take into consideration the many ways in which game audio can be can be communicated to the player.

Huiberts proposes the IEZA model (Huiberts. S, 2010, pp. 20-35) which looks at and categorizes game audio in way which reflects the communication of audio throughout interactive gameplay. The model is not applicable to non-interactive situations such as cut scenes for example. As it is the interactive part of gameplay that is important for immersion to occur it is beneficial to have a framework which deals with audio in this context.

The IZEA model first of all divides audio into two categories, ‘Diegetic’ and ‘Non-diegetic’, then further divides these up into four domains, ‘Zone’, ‘Effect’, ‘Affect’ and ‘Interface’. ‘Diegetic’ sounds are those which originate from within the game world, sounds like character footsteps, gunfire, and weather and so on. ‘Non-diegetic’ sounds are those which originate from outside the game world such as music or menu navigation sounds. Huiberts also points out that these domains are interdependent on each other which adds another dimension to the model, these are ‘Activity’ which connects ‘Interface’ and ‘Effect’, as well as Setting which connects ‘Zone’ and ‘Effect’. Huiberts identifies these as

“The Activity communicates events occurring in the game environment, while the Setting provides a background or context for the Activity.” (Huiberts. S, 2010, p. 24)

f:1

f:2

Illustration of IZEA Model f:1 and IZEA Model with general design properties; can be found on p25 (f:1) and p32 (f:2) in the following: (Huiberts. S, 2010)

Effect (Part of the diegetic division)

·        Sounds that are perceived as originating from within the game world for example sounds that would be heard by the character.

·        Sounds responsive to the players’ activity within the diegetic world, either by triggering directly or indirectly.

Zone (Part of the diegetic division)

·        Includes sound which is part of the diegetic environment, for example background ambience such as bird calls and wind in a forest setting.

·        Sound which do not interact with the player but add a sense of realism to the game world by providing a backdrop of enveloping sound.

·        “Communicating an ambient, background layer, which forms an auditory setting for the game world” (Huiberts. S, 2010, p. 27).

Interface (Part of the non-diegetic division)

·        Informing sounds which are outside of the game world such as health bar or score feedback, menu interactions

Affect (Part of the non-diegetic division)

·        Sounds which influence mood such as music at key game points to build tension

·        Sounds which are not part of the diegesis but are used to affect the players’ behaviour during gameplay.

Huiberts also highlights two primary roles for audio in games which are then linked into the IZEA model. These include sounds which are used to optimise gameplay and sounds which are used to dynamise gameplay (Huiberts. S, 2010, p. 29). Optimisation utilises audio to inform the player about gameplay (enhancing usability), whereas audio used to dynamise the gameplay would consist of sound to make parts of the game more exciting (enhancing the experience).

f:3

[f:3]These illustrations show how both Optimisation and Dynamisation are represented and prioritised within the IZEA model; can be found on p31 in the following: (Huiberts. S, 2010)

With the research topic of immersion in mind it is key to note that Huiberts makes a connection between optimisation and Dynamisation, and the potential positive impact these factors have on the immersive quality of a games.

Notes:

• Valid model for my research as is linked to immersion
• Good planning tool for conceptual design of game level used alongside the grounded theory of Brown & Cairns (2004)
• Designing specifically with immersion in mind (Can game audio be designed and implemented with the specific purpose of creating an immersive aural experience? Although the visual look of a game is recognised as a key aspect of game immersion, could this be overlooked if audio were to play a more prominent role?)

      References:

Brown, E. and Cairns, P. 2004. A Grounded Investigation of Game Immersion. Extended Abstracts of the 2004 Conference on Human Factors and Computing Systems, Vienna April 24-29 2004. New York: ACM. pp. 1297-1300. [online]. Available from: http://www-users.cs.york.ac.uk/~pcairns/papers/Immersion.pdf [Accessed 1st November 2014]

Huiberts, S. 2010. Captivating Sound, The Role of Audio for Immersion in Computer Games. [online]. Available from: http://download.captivatingsound.com/Sander_Huiberts_CaptivatingSound.pdf [Accessed 20 October 2014].

Game Level Concept - Game review (Amnesia - The Dark Descent)

This is a short review of the game Amnesia - The Dark Descent in relation to the honours project.
Of interest due to its reliance on audio to to enhance user experience.

Even before the game begins the developers have tried to set the scene and put the player in a specific mind set. Before the game can be played the player is instructed to adjust brightness levels, play in a dark room and to wear headphones to optimise playing experience. Finally it is added that "Amnesia should not be played to win. Instead, focus on immersing your self in the game world and story" (Frictional Games 2010).

Of these instructions, wearing headphones stands out as a significant factor to become immersed in the auditory world of the game. By excluding external sound sources the player can focus on game sounds which should be of importance for the honours project.

The game itself plays out in a dark environment with the player instructed to hide or run rather than fight. This also plays into the hands of audio as the player has to sneak around whilst trying to listen for aural clues. Just being a darkened environment alone encourages the player to rely more on auditory information. This should also be a key design aspect to include as part of the honours project.

The music plays a crucial role in this game mainly to set the scene and put the player in a frame of mind. the music conjures up preconceptions relating to the horror film genre and by making these connections the player then has an understanding of the kind of setting they should expect. Tension, fear and the unknown are all phrases that come to mind when listening to the music of Amnesia. Similarly the honours project may benefit from the use of genre specific music as a scene setting element which will help the narrative stand out. 

A nice audio feature which helps with player attachment to the game character is breathing. The character can be heard breathing which works dynamically by being linked to player actions and game events such as being under threat. Something so simple as hearing the character breath (even though at times can repetitive) is very effective at conveying the mood of the character and results in the player behaving in a way which was envisioned by the developers. A strong player to character attachment was a personal highlight of this game where the breath effect helped bridge the gap. With the honours project having a similar first person approach then this idea or another mechanism which makes the player to character bond stronger will need to be devised.

One aspect of the games' audio which was distracting was due to extreme panning. Here certain sound sources pan from one ear to the other (on headphones) very abruptly depending on which direction the player faces. At points the sound could jump from one side to the other with a full 100% of the signal being perceived one only one side. This unnatural movement of sound is the games' biggest flaw and is an issue present on many other games. A solution to this problem and any others which result in acoustic inaccuracies will be devised in order for the honours project to be most effective.

It has to be said that the story and visual aspect of the game has a lot to do with the immersion factor. The big question is, can audio lead this kind of experience or will it always just be one part of the immersion process?  

KEY NOTES:

• Game directs you on how to play for immersion
• Use of headphones to block external distractions
• Dark environment to heighten aural engagement
• Genre specific music used to support narrative
• Important player to character bond mechanism
• Extreme panning leading to a break in immersion

Research - Book added to reading list (Game Sound, Karen Collins 2008)

After reading through some of the initial papers I found the following book referenced on several occasions. I have now added it to my reading list to better inform the papers where it is referenced.

Research - Analysis of "A Grounded Investigation of Game Immersion" (Brown & Cairns, 2004)

Summary and conclusions about "A Grounded Investigation of Game Immersion" (Brown & Cairns, 2004)

Brown & Cairns (2004) highlight that the term ‘immersion’ is used frequently to describe games throughout the gaming community but with little coherent explanation as to the general meaning of the term. Their study tries to define immersion within the context of games using ‘grounded theory’ (method of qualitative research developed by Strauss & Corbin (1998)) by analysing interviews with seven gamers about their experiences when playing. Much like Taylor (2002) their results define immersion as a process of stages of involvement with the game which “moves along the path of time and is controlled by barriers.” (Brown & Cairns. 2004, p.2), only this time three stages are suggested.

The three stages of immersion described by Brown & Cairns are ‘Engagement’, ‘Engrossment’ and ‘Total Immersion’. The barriers describe gateways to each of the stages and by opening these the player can move forward through the stages of immersion. For example the first stage is ‘Engagement’, this is where the player must first engage with the game, so a potential barrier might be that the player does not like the genre or style of the game, thus creating a barrier for further immersion to take place. As stated by Brown & Cairns “To lower the barriers to enter this level, the gamer needs to invest time, effort, and attention” (2004, p.2). This will be an important point to follow when selecting a pool of gamers for testing immersive audio qualities within this project. If the selected gamers’ had no interest in the genre or style of the game then trying to guide them through these layers of immersion would be difficult.

The next stage of immersion ‘Engrossment’ is described by one interviewee as

“A Zen-like state where your hands just seem to know what to do, and your mind just carries on with the story.” (Brown & Cairns. 2004, p.3).

Brown & Cairns seem to link this stage to the games’ construction and the players’ emotional connection, as well as respect for the design work put into it. Again there is a barrier to this stage of investing time and effort to become emotionally involved with the game but also design flaws such as poor visual/aural feedback or even unrealistic gameplay. These factors could all contribute to the feeling of disengagement and bring up the barriers against progressing to the final stage of immersion described by Brown & Cairns.

Brown & Cairns describe the last stage of immersion ‘Total Immersion’ as ‘Presence’, this is where the gamers’ felt completely detached from the real world around them and were fully focused on the game. The barriers to ‘Presence’ that are suggested lay with empathy and atmosphere. In this sense empathy is defined not as the players’ empathetic feelings towards game characters or situations but as the “growth of attachment” to the game itself (Brown & Cairns. 2004, p.3). Interestingly the majority of games described by the gamers’ as ‘Totally Immersive’ were of a First person shooter model. Perhaps due to a direct visual illusion of seeing through the game characters eyes gives them a clearer feeling of ‘Presence’. The visual and audio elements combined with plot make up the game atmosphere and striking the right balance between these sensory interactions seems to be the key in reaching ‘Total Immersion’. Brown & Cairns give an example of how important the gamers’ attention factors into the ‘Total Immersion’ stage by stating

“If gamers need to attend to sound, as well as sight more effort is needed to be placed into the game. The more attention and effort invested, the more immersed a gamer can feel.” (Brown & Cairns. 2004, p.3).

They also describe three elements of player attention which are visual, auditory and mental. The above quotation would suggest that the more of these attention aspects are pulling on the players’ senses the greater involved with the game the player will become.

Conclusion

Although Brown & Cairns mention little about the role audio plays in the immersive process specifically in their paper at lot can still be taken from it. Identifying immersion as a process which happens over time is one point worth further investigation which could help define a timescale for each stage. This would be useful for testing as well as design of the projects gaming level with questions in mind such as, ‘how long should the experience take to complete’ or ‘how long should the test last’. A testing time too short or similarly a level that doesn’t take long to complete may not give enough time to see a balanced result.

The most important point to take is the idea that immersion is linked to attention and can be manufactured into the design of a game. By using game design to force players’ into relying on their senses in order to fulfil the games’ objectives, the resulting affect is drawing more of the gamers’ attention.

References

Brown, E. and Cairns, P. (2004). A Grounded Investigation of Game Immersion. University College London Interaction Centre (UCLIC). Conference on Human Factors in Computing Systems. Vienna, Austria. [online]. Available from: http://www-users.cs.york.ac.uk/~pcairns/papers/Immersion.pdf [Accessed 1st November 2014]

Strauss, A. and Corbin, J. 1998. Basics of Qualitative Research. 2^nd ed. Sage Publications, Inc. [online]. Available from: http://stiba-malang.ac.id/uploadbank/pustaka/RM/BASIC%20OF%20QUALITATIVE%20RESEARCH.pdf [Accessed 1^st November 2014]

Taylor, L. 2002. VIDEO GAMES: PERSPECTIVE, POINT-OF-VIEW, AND IMMERSION. [online]. Available from: http://etd.fcla.edu/UF/UFE1000166/taylor_l.pdf [Accessed 25 October 2014]