Why binaural audio will make VR the star

Following a series of talks by Lead Designer‚ Luke Medlock and our User Research Team‚ Dale Jones explains why hedgehog lab place so much emphasis on audio when designing and building immersive experiences. 

At Generator’s recent UX Day‚ our BIMA 100 Designer‚ Luke Medlock‚ discussed the rise of virtual reality. Just a minute in‚ the audience was left gasping in disgust.

“VR is f*cking overrated.”

Some fainted‚ others spat their drinks out in revulsion‚ many even walked out in a fit of rage as a state of shock swept the room.

Okay‚ perhaps we’ve employed artistic license. Besides‚ one look at Luke and you’ll find yourself warmed inside‚ smiling involuntarily and ready to obey his every command.

Nevertheless‚ attendees of his immersive design presentation probably didn’t expect him to criticise an emerging technology that our team at hedgehog lab have promoted and invested so heavily.

But as they’d come to find out‚ it was a conclusion he’d arrived at on worthy grounds. As a designer who is always looking to stay ahead of his craft‚ Luke was quick to throw himself at the new opportunities afforded by hedgehog lab’s investment in VR and the subsequent projects that have followed.

Over the past 18 months‚ Luke has worked arduously to develop his understanding of UX and UI needs in extended reality (XR)‚ rapidly establishing himself as a respected figure in immersive design via presentations much like that mentioned earlier.

But it is the multiple VR products he has helped to design that brought him to make such a controversial judgment so confidently. It’s not VR that he was taking aim at‚ but rather its current claim to total immersion and the immersive silver bullet that developers routinely overlook: audio. In this blog‚ we share Luke’s rationale and explain why audio will be the key that unlocks entirely immersive experiences.

Virtual reality? Sound mate.

No respected emerging tech authority overlooks Gartner’s Hype Cycle. Of course then‚ Mr Medlock spread the word that though Virtual Reality isn’t quite the gamechanger some make it out to be‚ it is climbing the slope of enlightenment and soon will be.

As the technology begins to detach itself from gimmicks‚ companies are starting to conceptualise and build productive uses of the technology – perhaps none more so in the area of training‚ which is being altogether transformed by XR.

But although innovative applications will help grow VR’s reputation‚ it is not inventive uses of VR that Luke believes will thrust the technology into the famed plateau of productivity.

Luke believes that instead‚ it will be a change in the way we deliver such solutions.

With headsets capable of removing you from your surroundings and taking you into a virtual environment‚ it’s fair to say that VR is the most immersive of the technologies encompassed within the blanket term XR. But there remains room for improvement. Relying solely on visuals to make a VR experience immersive is a recipe for failure.

What is neglected by developers in formulating immersion is the importance of audio.

Just as our eyes allow us to see a three-dimensional world‚ to determine depth perception‚ placement‚ light and shadow‚ our ears enable us to hear a 3D world‚ providing stability‚ spatial awareness and 360° sound.

It is a fundamental need of virtual reality that we marry the two together if we’re to generate true immersion. To do this‚ we need to replace stereophonic with binaural audio.

What is binaural audio?

Binaural audio is nothing new. Indeed‚ since the 1800s it has been used in orchestras to capture the movement in sound from around the arc in which musicians perform. 

When explained‚ it doesn’t sound particularly groundbreaking‚ either.

Recorded with two microphones and often placed inside the ear canals of a Kunstkopf (a dummy head to me and you)‚ binaural recording mirrors how we hear sound in daily life.

The results for users include a lifelike sense of space as well as an ability to sense the movement of noise around them‚ ultimately delivering a three-dimensional audio experience. 

Naturally‚ this works far better in individual listening experiences than those designed for a larger space. But it’s this very characteristic that makes it perfect for the intimacy provided by virtual reality headsets.

Using binaural audio‚ designers can begin to consider how they can use sound to enhance experiences. Three considerations of particular importance are arrival‚ loudness and timbre.

Arrival allows users to work out direction. In virtual reality‚ this can be used to prompt users to look in a direction that is outside of their current field of view. Loudness can be used to enable users to judge distance‚ while timbre (voice distortion) should allow users to ‘pinpoint’ where a noise originates.

Want to check it out for yourself? Luke resurrected this famed classic that perfectly demonstrates how much more immersive binaural audio can be. Plug your headphones in and give it a listen.

Though we believe Luke’s observation that developers are undervaluing audio to be correct‚ it’s only fair we note some are embracing 3D sound.

Since 2012‚ the BBC has been experimenting with binaural broadcasting‚ explaining their desire to provide a richer sense of space in programme sound.

The need to sound out better audio experiences has been recognised by giants like Google too. In late 2017 the company‚ who are purveyors of VR headsets themselves‚ unveiled Resonance Audio‚ an audio software development kit designed to bring scalable 3D sound in immersive experiences.

When designing experiences for virtual reality headsets‚ our team work from graphics like the two that surround this paragraph. As can be seen‚ the user only ever has 120° of vision‚ while 240° remains the ‘unseen’‚ dubbed the curiosity zone.

The benefit of binaural audio is that‚ in an immersive world where we can only ever see 120°‚ we can hear across all 360°. It’s a sound format of particular use to areas such as entertainment. As an example‚ game developers and movie producers would be keen to ensure users don’t miss out on key content.

Loud and clear.

Whether or not virtual reality is overrated will remain much debated‚ but what is sure is that without binaural audio‚ applications of VR will not fulfil their potential.

This is a view backed up by the work of our User Research Team‚ whose work has only validated Luke’s thoughts that users see improved audio as a gamechanger. Indeed‚ the team even provided him with backup‚ presenting alongside him at a recent follow-up talk.

In our recent blog exploring the rise of augmented reality in enterprises‚ we covered best practices and mistakes to avoid in adoption. If reading this blog teaches you one thing‚ let it be putting audio high on your agenda.

---

Do you believe wider adoption of binaural audio can accelerate the adoption of virtual reality? Let us know your thoughts over on Twitter.

---