I. INTRODUCTION

Navigating and orienting within an unknown environment presents a significant challenge for individuals with visual impairments [11]. Relying on alternative sensory modalities such as hearing and touch becomes essential to compensate for lack of sight, with hearing being particularly powerful due to its large capacity for perceptual input [4]. However, the incorporation of audio in traditional video games has often been limited to ’decorative effects’[17], rendering these experiences largely inaccessible to visually impaired players. As 3D graphics continue to advance, while audio development massively falls behind, this player exclusion not only denies a significant portion of the population the opportunity to engage with video games but also blocks their ability to build mental representations of virtual environments.

As of 2020, more than 2.2 billion individuals worldwide are affected by some form of visual impairment [24]. Among them, over 43 million people live with complete blindness, while 295 million experience moderate-to-severe visual impairment [7]. This indicates that nearly a third of the global population struggle with visual impairment in varying degrees, a number that continues to rise.

In contrast to conventional views of video games primarily being visual mediums [33], leveraging audio cues can revolutionise accessibility and engagement for a wider audience, particularly for individuals with visual impairments. By harnessing spatial audio, games can provide an immersive and interactive platform for visually impaired players to construct spatial cognitive maps of their surroundings. Moreover, the ability to create these spatial cognitive maps can be extended beyond digital boundaries, allowing people to develop alternate sensory capabilities to improve navigation in unfamiliar surroundings within the real world [35].

This essay will explore the current state of auditory accessibility within the video game industry for visually impaired individuals, whilst looking into the effectiveness of spatial language as an approach to navigating 3D game-spaces. Additionally, it will examine the reasons behind the under-representation of accessibility challenges in the gaming industry, despite their significance [21].

II. SPATIAL AUDIO AND NAVIGATION

Spatial audio is a domain that focuses on spatial descriptions of the environment and any surrounding objects, often relative to the players position within a three-dimensional space [8]. This aspect allows player to discern the location of various in-game elements, significantly enhancing immersion. This is in addition to its immense potential to aid navigation and orientation within virtual environments, especially for visually impaired players who heavily rely on auditory cues for game-play assistance. For these individuals, having a clear mental map of their surroundings is essential for effective navigation and orientation [4].

As a result, Visually impaired individuals often adopt a parametric approach, moving along the edges of rooms or near walls to construct a comprehensive mental representation of the space they are traversing[20]. This method grants individuals greater mobility and provides reassurance of a path leading to their destination. However, this exploration technique is not without its drawbacks, primarily being due to its inefficiency in spatial mapping, stemming from the time-consuming nature of the process and the uncertainty regarding the environment size.

A study by Lokki and Gr¨ohn [18] demonstrated that audio cues are as accurate as graphical information for navigating unknown environments in a 3D game space. Remarkably, participants showed significantly better object localization using audio cues, particularly in the initial stages of the study, compared to localization based solely on visual cues. This study demonstrates the potential effectiveness of audio-based navigational aids, especially where visual cues may be limited or unavailable. In addition to this, they also found that audio cues within the 3D game space have to be informative and effective in their role to aid spatial navigation.



Fig. 1. (A) The architectural floor plan of a two story building, including 23 rooms, 2 stairwells and 3 exits. (B) The virtual rendering of the same building in which the players were exploring to build the spatial cognitive maps . [11].

A similar study conducted by Connors et al. [11] investigated the effectiveness of ’Audio- based Environment Simulators’ (AbES) in enabling participants to construct a spatial map of an unknown digital environment and apply it to real-world navigation within the same building, as detailed above. Participants were immersed in a virtual environment where spatial audio cues provided directional guidance, distance estimation, and obstacle detection to aid navigation. Moreover, spatial cues with pitch variations were employed to convey changes in elevation, such as ascending stairs, representing the height difference between floors, an area which has proven challenging to replicate previously. Despite the inherent challenges in comprehending and successfully implementing such auditory cues, participants demonstrated a remarkable ability to understand and utilize them effectively in a virtual audio-based environment.

The study’s findings indicate that immersive and self-exploratory environments facilitate the development of participants’ spatial cognitive skills, which could be transferred to real-world navigation. Engaging with the virtual environment allowed users to construct a robust mental representation of the spatial layout, leading to more flexible manipulation of spatial information.

Participants who extensively interacted with the digital environment exhibited greater efficiency in navigating the real-world building, suggesting that the gaming strategy facilitated the attainment and application of spatial knowledge that can be transferred between virtual and physical spaces.

III. SPATIALISATION CUES AND HOW THEY ARE USED IN 3D GAME-SPACE

The entire perceived auditory environment is defined by characteristics such as the distance or directional cues indicating the location of the audio source. Perception, an automatic and unconscious process, translates auditory signals and their characteristics, referred to as ”auditory dimensions”, into significant subjective information at the brain level [5]. These auditory signals play a crucial role in conveying relevant information concerning the perception of the environment, with perception being closely intertwined with the concept of sonification, a way of prioritising data within the game space, as evident from its definition:

”…sonification is the use of non-speech audio to convey information or perceptualize data.” [6]

This highlights the vital role of perception in interpreting auditory information. To provide a comprehensive description of a virtual setting, the association of various auditory dimensions is essential for representing multidimensional data.

Examples of these auditory dimensions include pitch and amplitude. Pitch, defined as the change in frequency of a sound wave on a musical scale, is easily detected by the human hearing system within a range of frequencies from 20 Hz to 20 kHz. Pitch is an effective tool for mapping changes in values of encoded data, and can influence if a sound is high or low in tone, demonstrating the height of an element in the 3D space. Amplitude, being the perceived intensity of a sound wave, influences the volume or loudness of a sound, such as if a sound is loud or quiet, demonstrating a 3 players distance from the element in the 3D space. This can be seen demonstrated in the AbES study from the previous section[11].

For a comprehensive portrayal of a virtual 3D environment, auditory dimensions, like pitch and amplitude, must be considered in conjunction. These dimensions interact harmoniously, influencing audio cues concurrently, and their integration can create substantial advantages in depicting changes in data.

Moreover, spatial location, as another crucial auditory dimension, referes to the perceived direction and/or position from which a sound cue originates. This aspect is essential for preserving orientation within an audio-based representation of the surroundings. Key components defining spatial location include the distance from the player, the direction, and the scale of the environment, as detailed below.

1) Interaural Time Difference (ITD): The time difference it takes for arrival of a sound stream at two sensors, often the players ears. An approximation of the ITD for an object infinitely distant can be computed by the following formula [14], where the player’s head is approximated to a sphere with the radius ’a’, θ is the azimuth angle2 and ’c’ is the speed of sound [22]:

IT D = a/c (Θ + SinΘ)

2) Interaural Level Difference (ILD): The difference of amplitude perception between the two sensors, once again often being the players ears.

3) Head Related Transfer Function (HRTF): How the shape of a person’s head and ears affects the way sound waves are filtered before reaching the inner ear, which is crucial for accurate sound localisation in spatial audio systems and virtual reality (VR) environments.

The resulting representation of the environment based on the dynamic perception of various sound cues such as ITD, ILD, and HRTF is what makes up the spatial cognitive map that serves as a guide for navigation and orientation in an unknown environment.

IV. AUDIO GAMES

Over the past decade, several mainstream computer games have emerged where sound serves as a central medium, falling into categories such as rhythm or music games. Examples include Guitar Hero [31] and Dance Dance Revolution [12]. However, these games often fall short for players with visual impairments due to their reliance on graphical cues and controller types. Nevertheless, they have paved the way for greater spatial flexibility in gaming, leading to the development of more audio and music-based games for a broader commercial gaming audience [16]. Despite this progress, audio games catering towards the visuall impaired community remain somewhat less advanced, drastically lagging behind graphical based games in terms of complexity and engagement, often resembling children’s games from 10-15 years ago [16].

On the other hand, several noteworthy case studies serve as prime examples of how spatial audio effectively assists navigation in 3D game environments for visually impaired players. One such example being AudioQuake [3], a first person shooter adaptation of the game Quake [28]. In AudioQuake, visual feedback is replaced by ”earcons”, which are types of structured sounds that follow musical conventions. They are designed to alert the player to specific objects or events, although they do not sound like their referents in the real world [34]. Utilizing stereo sound, AudioQuake strategically employs audio cues to the left, right, or center of the player, effectively indicating the presence of objects and room boundaries, thus facilitating navigation within the game environment.



Fig. 2. Game-play screenshot of AudioQuake, showing a text based editor which responds in combination with auditory feedback based on player input from the keyboard[27].

Similarly, Terraformers [32], a shooter adventure game designed for both sighted and visually impaired players, offers game-play options that accommodate different sensory preferences. Players can choose between utilizing the visual display or relying on auditory feedback to engage with the game. Terraformers uses a 3D sound response system such as a radar which players can ’ping’ to gauge the distance of objects from the player. Additionally, Terraformers incorporates a voice playback system, enabling players to identify what type of object is in front of them by pressing a key on the keyboard.



Fig. 3. Game-play screenshot of Terraformers. Whilst the game can be played with audio or graphics, the graphical display can be set to a high contrast mode for low-vision players [27].

Both of these games demonstrate the potential of spatial audio in fostering inclusive gaming experiences. However, its worth noting that both of these audio games have been created as research projects by official institutions, indicating there is little interest in making these types of games commercially. Furthermore, this does not necessarily solve major issues with mainstream games not being accessible to a large portion of the gaming population.

V. CURRENT MAINSTREAM GAME ACCESSIBILITY

Many mainstream video games lack comprehensive accessibility options, once again relying heavily on graphical interfaces and visual cues. This exclusionary approach not only alienates visually impaired players but also overlooks the potential for innovative design solutions.

One such game that has gained praise for a more inclusive design approach is The Last Of Us [13], which included features such as customisable visual and auditory cues, screen readers, and triggerable environmental sensors for scanning the 3D space. These features not only allow access to mainstream games for individuals with visual impairments, but can also enhance the gaming experience of all players by providing diverse customisation options. However, games such as The Last Of Us remain the rare exception rather than the norm, highlighting the need for broader industry-wide adoption of accessibility standards. And whilst these features alone do not allow players with visual impairments to finish the game [29], they provide a lot more than most commercial games on the marke today.



Fig. 4. Screenshot of the options menu for The Last Of Us outlining a screen reader that can be toggled on, as well as various auditory cues like cinematic descriptions and traversal auditory cues[13].

The TiM project (Tactile Interactive Multimedia) [2] pioneers sound-based games that push the boundaries of game audio design within the commercial realm of audio gaming. Through games like TiM’s Journey, they showcase innovative approaches to game design centered around auditory experiences. TiM’s Journey, their most complex project, features a sophisticated sound system that leverages ambient music to evoke mood and convey emotional depth to players. By placing player-generated sounds at the forefront of the stereo space and surrounding them with other environmental noises, the game creates an immersive audio landscape. Unlike traditional audio games that minimize auditory cues to avoid distraction [1], TiM’s Journey employs a flexible sound design that allows for layered audio experiences.

Based on Pierre Schaeffer’s analyses of sound objects [25], TiM’s sound design integrates three modes of human listening: casual listening, semantic listening, and reduced listening. Casual listening involves identifying the source of a sound and understanding its cause, while semantic listening focuses on interpreting auditory codes like speech. Reduced listening, less common but equally significant, entails appreciating the qualities of a 5 sound independently of its source, such as appreciating music, harmonies, and rhythms. In TiM’s Journey, all three listening modes converge to create a rich audio environment, where reduced listening captures the background ambience, semantic listening deciphers sonic codes, and casual listening recognizes the presence of objects within the game world

[16]. VI. USER INTERFACE ACCESSIBILITY

User interfaces in gaming present significant challenges for individuals with visual impairments due to their almost exclusive reliance on visual elements and cues. Traditional menu interfaces often incorporate text labels and graphical icons, and occasionally duplicated audio cues when hovering buttons, which can be completely inaccessible to severely visually impaired players [16]. This reliance on visual cues inherently excludes individuals with visual impairments from fully participating in the gaming experience, or to even access the game in the first place.

”Games are meant to be difficult but not difficult to access” [9]

To address this issue, there is a growing need for alternative navigation methods that prioritize auditory cues and instructions.

Spatial audio could revolutionize menu screens in audio games by providing an immersive and intuitive navigation experience for players. Instead of relying solely on textual or basic auditory prompts, spatial audio can simulate a three-dimensional environment where menu options are positioned spatially within the scene. For example, different menu items could be placed at distinct spatial locations around the player, with their positions corresponding to specific actions or categories [23]. As the player navigates through the menu, they would hear the spatialised audio cues shifting around them, providing clear auditory feedback about their options.

Beyond this, Voice-guided navigation systems represent one of many current solutions to enhance accessibility for visually impaired players within gaming interfaces. By providing auditory prompts and instructions, voice-guided navigation can assist such players in navigating complex menu structures. For example, games like Helldivers 2 [30] have implemented screen readers by default, which narrate on-screen text and interface elements until turned off, making the game more accessible to players with visual impairments. This integration of screen readers exemplifies a step towards inclusive design in gaming.

However, despite advancements in voice-guided navigation and screen reader integration, auditory user interfaces still lack any official standardized conventions. Unlike sighted users who can visually process entire menus at once, visually impaired users often encounter challenges in navigating menus sequentially, being forced to ’focus’ on one item at a time. Although accessible menus may utilize audio feedback and arrow keys for navigation which can help hard of sight users ’visualise’ the menus, there remains room for improvement in designing interfaces that cater specifically to the needs of visually impaired players. Enhancements such as clearer auditory cues, more intuitive navigation systems, and customisable interface options can contribute to a more inclusive gaming experience for all players, regardless of visual ability.

VII. SPATIAL AUDIO GAME GUIDELINES

In the realm of video games, there exists a pressing demand for inclusively and representation, aiming to ensure that all individuals, regardless of ability, have equal access to digital content. This can be seen outlined by several governments, putting laws in place to account for this, such as the European Union [15], who published a directive about website and software accessibility in the digital age. Accessibility in gaming transcends convenience; it embodies the fundamental right of every individual to access and engage with digital experiences on an equitable basis [26]. Unfortunately, many video games currently prioritize visual elements over audio or spatial cues, creating significant barriers for visually impaired players. These games often rely on text labels, graphical icons, and visual cues, rendering them inaccessible to those with visual impairments and hindering their participation in the gaming community.

Emerging technologies such as virtual reality (VR) and augmented reality (AR) offer promising opportunities to enhance accessibility through the implementation of spatial audio [10]. By leveraging spatial audio, developers can create immersive gaming experiences that transcend the limitations of traditional interfaces. Spatial audio allows for the simulation of three-dimensional environments, where sound cues are positioned spatially around the player, providing clear auditory feedback about their surroundings.



Fig. 5. The APX triangle as an example of a design method on making games more accessable [9]: first players having access to the game which then allows them to be enabled to meet the challenges of the game and where that is successful, leads to them having an accessible player experience [19].

This innovation has the potential to revolutionize gaming accessibility, enabling visually impaired players to navigate virtual worlds with greater ease and efficiency.

Despite the advancements in spatial audio technology, there remains a noticeable scarcity of high-quality audio games publicly available. [9] The limited availability of audio games presents a significant challenge for visually impaired gamers seeking immersive gaming experiences. While some audio games are developed for research purposes, they often lack interest from the broader gaming community. Bridging this gap and expanding the availability of accessible gaming experiences is crucial for fostering inclusivity and ensuring that all players can fully enjoy the diverse world of video games [16].

VIII. CONCLUSION

In conclusion, this essay highlights the transformative potential of spatial audio in revolutionizing accessibility and navigation within 3D game environments, particularly for severely visually impaired players. With over 2.2 billion individuals worldwide affected by visual impairment, addressing accessibility challenges in gaming is crucial. Spatial audio, with its ability to simulate three-dimensional environments and provide immersive auditory feedback, offers a promising solution to enhance navigation and orientation for players with visual impairments. By leveraging spatial audio cues, developers can create more inclusive gaming experiences that transcend the limitations of traditional graphical interfaces.

Moreover, this essay emphasizes the importance of user interface accessibility in accommodating the needs of visually impaired players within 3D game environments. Alternative navigation methods, such as voice-guided navigation systems and spatially positione audio cues, play a crucial role in facilitating seamless navigation for visually impaired players. However, there is still a great need for further research and development to optimize spatial audio cues and interface designs to better serve the needs of visually impaired and blind players.

Ultimately, fostering inclusivity and representation in gaming requires collaborative efforts from developers and the gaming community. By prioritizing the integration of spatial audio technologies and designing intuitive navigation systems, the gaming industry can create more accessible and engaging experiences for all players, regardless of their visual ability. Spatial audio not only enhances accessibility but also promotes diversity and inclusion within the gaming landscape, empowering visually impaired players to navigate virtual worlds with greater ease and efficiency.

REFERENCES

[1] Gavin Andresen. Playing by ear: Creating blind- accessible games. Gamasutra Article, 2002.
[2] Dominique Archambault, Dominique Burger, and S´ebastien Sabl´e. The tim project: Tactile interactive multimedia computer games for blind and visually impaired children. Assistive Technology–Added Value to the Quality of Life, Proceedings of the AAATE, 1:359–363, 2001.
[3] Matthew T Atkinson, Sabahattin Gucukoglu, Colin HC Machin, and Adrian E Lawrence. Making the mainstream accessible: redefining the game. In Proceedings of the 2006 ACM SIGGRAPH Sympo- sium on Videogames, pages 21–28, 2006.
[4] Oana Balan, Alin Moldoveanu, and Florica Moldoveanu. Navigational audio games: an effec- tive approach toward improving spatial contextual learning for blind people. International Journal on Disability and Human Development, 14(2):109– 118, 2015.
[5] Oana B˘alan, Alin Moldoveanu, Florica Moldoveanu, and Maria-Iuliana Dasc˘alu. 7 Navigational 3d audio-based game-training towards rich auditory spatial representation of the environment. In 2014 18th International Conference on System Theory, Control and Computing (ICSTCC), pages 682–687. IEEE, 2014.
[6] Constanta-Nicoleta Bodea, Maria-Iuliana Dascalu, and Miltiadis D Lytras. Advanced personalized feedback in e-assessment systems with recom- mender engine. In Information Systems, E-learning, and Knowledge Management Research: 4th World Summit on the Knowledge Society, WSKS 2011, Mykonos, Greece, September 21-23, 2011. Revised Selected Papers 4, pages 179–184. Springer, 2013.
[7] Flaxman S et al Bourne R, Steinmetz J. Trends in prevalence of blindness and distance and near vision impairment over 30 years: an analysis for the global burden of disease study. December 2020. Accessed via the IAPB Vision Atlas.
[8] James Broderick, Jim Duggan, and Sam Redfern. The importance of spatial audio in modern games and virtual environments. In 2018 IEEE Games, Entertainment, Media Conference (GEM), pages 1– 9, 2018.
[9] Paul Cairns, Christopher Power, Mark Barlet, and Greg Haynes. Future design of accessibility in games: A design vocabulary. International Journal of Human-Computer Studies, 131:64–71, 2019.
[10] Thomas Chatzidimitris, Damianos Gavalas, and Despina Michael. Soundpacman: Audio aug- mented reality in location-based games. In 2016 18th Mediterranean Electrotechnical Conference (MELECON), pages 1–6. IEEE, 2016.
[11] Erin Connors, Elizabeth Chrastil, Jaimie Sanchez, and Lotfi Merabet. Action video game play and transfer of navigation and spatial cognition skills in adolescents who are blind. Frontiers in Human Neuroscience, 8, 2014.
[12] Konami Game Music Division. Dance dance revo- lution. Konami, 1998.
[13] Naughty Dog. The last of us remastered. Sony Interactive Entertainment, 2014.
[14] Richard O Duda, Carlos Avendano, and V Ralph Algazi. An adaptable ellipsoidal head model for the interaural time difference. In 1999 IEEE In- ternational Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No. 99CH36258), volume 2, pages 965–968. IEEE, 1999.
[15] European Union. Directive (EU). accessibility of the websites and mobile applications of pub- lic sector bodies, available online at: https://eur- lex.europa.eu/eli/dir/2016/2102/oj, 2016.
[16] Johnny Friberg and Dan G¨ardenfors. Audio games: new perspectives on game audio. In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology, pages 148–154, 2004.
[17] Dan G¨ardenfors. Designing sound-based computer games. Digital Creativity, 14(2):111–114, 2003.
[18] Matti Gr¨ohn, Tapio Lokki, and Tapio Takala. Com- parison of auditory, visual, and audiovisual naviga- tion in a 3d space. ACM Transactions on Applied Perception (TAP), 2(4):564–570, 2005.
[19] Ciara Heavin and Daniel J Power. Challenges for digital transformation–towards a conceptual de- cision support guide for managers. Journal of Decision Systems, 27(sup1):38–45, 2018.
[20] Orly Lahav and David Mioduser. Blind persons’ acquisition of spatial cognitive mapping and ori- entation skills supported by virtual environment. International Journal on Disability and Human Development, 4(3):231–238, 2005.
[21] Carme Mangiron and Xiaochun Zhang. Game accessibility for the blind: Current overview and the potential application of audio description as the way forward. Researching audio description: New approaches, pages 75–95, 2016.
[22] Martin Nordlinder. Lair of beowulf: A study of 3d positional sound in an audio mostly game, 2007.
[23] Atlas Olsson. Virtual reality menu structures: How 2d and 3d menus affect the user experience in a virtual game environment., 2022.
[24] World Health Organization. Blindness and vision impairment, August 2023. https://www.who.int/news-room/fact- sheets/detail/blindness-and-visual-impairment.
[25] Carlos Palombini. Pierre schaeffer, 1953: to- wards an experimental music. Music & Letters, 74(4):542–557, 1993.
[26] George Powers, Vinh Nguyen, and Lex Frieden. Video game accessibility: A legal approach. Dis- ability Studies Quarterly, 35(1), 2015.
[27] Manjari Sapre. A framework for making virtual worlds accessible to the visually impaired. Univer- sity of Nevada, Reno, 2009.
[28] ID Software. Quake. GT Interactive, Quake engine, 1996.
[29] Markus Sp¨ohrer. Playing with auditory environ- ments in audio games: Snake 3d. In Research Anthology on Game Design, Development, Usage, and Social Impact, pages 644–661. IGI Global, 2023.
[30] Arrowhead Game Studios. Helldivers 2. Sony 8 Interactive Entertainment, 2024.
[31] Harmonix Music Systems. Guitar hero. Activision Blizzard, 2005.
[32] Thomas Westin. Game accessibility case study: Terraformers–a real-time 3d graphic game. In Proceedings of the 5th International Conference on Disability, Virtual Reality and Associated Technolo- gies, ICDVRAT, 2004.
[33] M.P Wolf. On the future of video games in digital media: Transformations in human communication. pages 187–195, 2006.
[34] Bei Yuan and Eelke Folmer. Blind hero: enabling guitar hero for the visually impaired. In Proceed- ings of the 10th international ACM SIGACCESS conference on Computers and accessibility, pages 169–176, 2008.
[35] Bei Yuan, Eelke Folmer, and Frederick C Harris. Game accessibility: a survey. Universal Access in the information Society, 10:81–100, 2011