On deaf ears: how smart visual design can make games more hearing-impaired accessible

XIIIPS2Video game designers have taught us, through intricately designed HUDs throwing all manner of information at the player every second, to pay attention to our surroundings.  Visual cues are more often than not integral to success, and much research and effort has gone into cramming as much vital information on screen in the most efficient way possible, to ensure that players have all the visual tools they need to stay alive.

But what about sound?   If you’ve ever tried to play a game with the sound muted you’ll know just how strange of an experience it is.  And subtitles can only go so far, because while they capture the spoken scripted tracks, it often entirely misses the ambient sound of the world around the player.  If you try it for yourself you’ll realise it’s not a trivial thing and that every genre –  from the sound of car engines roaring up behind in Forza or the ruffling of enemies in Bloodborne in a room head – immerses the player in the world and conveys amazing amounts of information through sound.

In a lot of ways XIII was a thoroughly average Playstation 2 game.  It was, like many other games at the time, piggybacking on both the enormous popularity of shooters and the rather infectious fascination with cell-shaded graphics.  But I liked its take on the European graphic novels I grew up with and at the time it was quite the looker.  It may not have been the best shooter, but it was certainly unique, and in an era where the carbon copy was king that was enough to hold my interest.

But what stood out to me – even at the time – was how its reliance on onomatopoeia could have potentially made it incredibly friendly to people with hearing impairments.  At times it just captured what was quite obviously happening on screen, with a BAM! here or a KABOOM! there, accompanying the sound effects pouring out of the speakers but also the action on-screen.  And this is a great way to convey the visceral nature of a game’s sound effects.  Games are a sensory experience – both visual and aural – and anything that can be done to capture or enhance the atmosphere of a game should be done.

But it was what happened in the quieter moments that really caught my attention.  Amongst the bazooka and assault rifle fuelled action that comprised most of the game, XIII was punctuated with some simple stealth sequences, that had you sneaking rather than shooting.  Gameplay wise it was nothing special, serving more to break up the action, while playing to the game’s espionage themes more than anything else.  But where it did revolutionise things was in how it used visual cues to convey the sound of nearby enemies, which if you’ve played a stealth game, you’ll know is key to experience.  SEEING the footsteps of an enemy is a nearby room get bigger as he draws nearer, as the words Tap Tap Tap move with him, is quite the experience even for someone who can hear.  While it may have been stylistic more than functional, it was an excellent way to visually convey sound, but an even more brilliant way to make the game’s stealth mechanics accessible to everyone.  And there is no reason this model couldn’t be adopted and adapted for every genre and every aesthetic – but perhaps more importantly no reason it shouldn’t be.

And while we always looks for new and innovative ways to give equal opportunity to people playing games, the hundreds year old answer may be staring the  interface and accessibility designers right in the face, without them knowing it.  Comic books and graphic novels may be just visual experiences, but because of this, they’ve had to find ways to visually represent .  Subtitles are the natural extension of this, but if games adopted the onomatopoeia used to convey sound where there is none, we’d improve the experience of video games for those who can’t hear them by ten fold.  It’s not just a style choice, it’s a matter of accessibility and equality of experience, and I’m all for that.

I would love to be in a world where everyone can enjoy the atmosphere created by video games regardless of any impairments.  And the great thing is it’s not a pipe dream.  But we just need designers to find more creative ways to improve the accessibility of their games.  I’d personally love to see VROOOOM! pop up at the bottom of the screen in Forza 6 as an opponent screams toward me, so I can only imagine the difference it would make to someone with a hearing disability.

I’d love to hear the game playing experiences of people with hearing impairments or other disabilities, so if this is you, or you know someone else please share your experiences in the comments below.

Screen Shot 2015-05-08 at 9.27.07 pm