TUNING THE SOUND OF TECHNOLOGY: 

RESPONSIBLE SENSORY DESIGN

What are some of the biggest sound-emitting digital platforms you can think of? YouTube? Spotify? Well how about your email..? Consider the systems themselves, computer operating systems, that run all day, pinging us with notifications as they go, and you’ll find yourself amongst the work of today’s guest.

Formerly Microsoft’s Director of Sound + Sensory for 12 years, before founding Acoustic Ecology Design Partners, Matthew Bennett is the architect of all the system sounds you’ll hear on a Windows 10 machine, and he joins us from his home studio in Seattle, Washington, USA, close to Microsoft’s headquarters. 

Matthew’s written a paper ‘Tuning the Sound of Technology’ which details how the carefully thought out sounds for Windows 10 are pumping out the equivalent of 253 years of sound—8 billion seconds—every day, thanks to the one billion devices running the OS.

“Each of those sounds are intentionally interrupting somebody who’s trying to do something else…it’s our job. A lot of us are hearing messaging sounds of platforms all day long—but there’s no way you can focus or concentrate when you’re constantly being interrupted.”

There’s a lot of cognitive science showing that even small interruptions can take a while to recover from, so Matthew’s main aim is to learn from, and mimic, the human approach to how we interrupt people. When delivering a message, few people would loudly shout at their recipient without warning…

"You don’t just surprise them or touch them. You let them know you’re there, you might take a breath—there’s a preparation—so that we know they’re going to speak to us. And we build those little breaths into sound design.”

Matthew’s paper also mentions ‘a sensory model’. Our senses aren’t separate channels. As Richard Grove in The Quiet Mark Podcast Episode 1 mentioned “we don’t have ‘earlids’” meaning we can’t shut out sound. But as Matthew explains, our skin also receives sound as much as our ears. As he labels it “sound is touch from a distance—it has a haptic quality… It’s less about sound and hearing, more about sound and feeling.”

So thinking back to that example, that ‘breath’ prepares us to start to hear the message. In sound design, how does that translate, is it like a story’s beginning, middle and end? 

“There’s breath, bloom and tail… a little quiet upbeat (breath) before the main part of the sound (called the bloom) which lasts about 300-400ms, and (a tail)… a transparent extension of the sound so it dissipates back into the environment.”

Because ultimately all sounds we hear are being constantly monitored by our brains, and in an evolutionary response, sudden sounds trigger a ‘fight or flight’ response. A snapping twig in the forest (for our ancestors) could be a predator, so with sound and sensory design, the goal is to avoid ever triggering that.

Matthew subscribes to his own hippocratic oath—to do no harm. “Get rid of all the annoying sounds, have as much silence as you can, and the ones that remain… make them as beautiful and seamless and functional and inclusive as you can”. That’s a very solid brief, we’d say.

Demand on our time, and the colossal amount of notifications (including the ones Matthew’s sadly had nothing to do with) has only grown in recent years. It’s also changed dramatically as we bring our working lives home mid-pandemic. In turn, this has increased the noise pollution experienced by others living with us (for whom these notifications aren’t relevant).

“There’s a tension between this cultural crisis of anxiety which surrounds our relationship to technology right now. When you design at scale, in some cases for billions of people, you’re becoming part of their lives. There’s definitely an ethical responsibility that comes with that and how toxic bad sound design can be. It really, is, like pollution.”

Before 2008 (before smartphones with apps) the only audible ‘notifications’ we tended to hear were in fact alarms. While Matthew’s advanced creations have steered well clear of being alarming, not everyone’s designs are quite as forward thinking. He wishes not to ‘capture attention’ but to “sit on the periphery… rather than the centre”.

Sounds have to be just “noticeable… the JND… the just-noticeable difference. A threshold where, if missing, the experience is less alive or beautiful but they’re not impinging actively on consciousness and attention and focus.”

Nature, as ever, has a beautiful example of a richly textured, densely layered audio environment—it’s the rainforest. “It supports attention, and focus. You can shift your attention back and forth between the different layers of audio. The canopy, the winds, the vegetation, [can be heard] all very clearly, even though there are multiple audio-streams. They create a harmonious texture [and] most people find it very pleasing, soothing and calm.”

So, for Matthew, the kind of soundscape he’s trying to create would have that level of harmonious detail, where the sounds we hear would work perfectly together. In workshops he’s run, he’s observed ‘deep listening’, where once a sound is pointed out, our brain develops a kind of hypersensitivity to it. “You can’t un-hear that. It flips a switch in your brain. You become sensitised.”

One burgeoning trend he points to, and would like to champion, is the increase in people curating the sound of their environments (homes, especially) just as they might curate the interior design. “They do it much less with sound” he says. As you might expect, we’d agree that a quiet, thought-out home or office is one that’s going to have a huge impact on your ability to be happy and productive.

Matthew is an award-winning composer, performer, and producer with a background in ethnomusicology, where his graduate research focused on the anthropology of sound and sensory experience, before taking a sabbatical from PhD coursework to accept a 'temporary' position working at Microsoft. 

How have Matthew’s human-based learnings distilled into a product sound he’s designed? Well, in creating Microsoft’s calendar alert, he discovered there was a natural “pitch and prosody and rhythm” found in a particular phrase—‘ready to go?’

It turns out that in pretty much all languages spoken it follows the same upwards inflection. In fact it follows a specific inflection that’s “going up and down and up a little more at the end.”

“We wouldn’t normally have had a 4-note sound for the calendar invite, especially with those contours, but using the music of language allows us to create sounds that people recognise intuitively and emotionally. At this very primitive level, that can blend into experience more seamlessly… 

“Everyone’s an expert in their native language. If you can tie into those musical counters and tie them to the appropriate emotional moment of the experience, then you can really connect with people in some more deeper and intuitive ways.”

While we may be experts in our own native language, very few are expert sound and sensory designers. So Matthew is eager to connect with anyone for whom this makes sense. “If you’re interested, it’s a young field with a lot of potential. Come and help us define it!”. Rather than purely musicians, though, it was Matthew’s ethnomusicology (the study of musical cultures, especially non-Western ones) that set him up to succeed. 

It’s “combining art and science and engineering and psychology, but the art (the music) isn’t the only thing. The more kinds of humanities and liberal arts you can bring into it the better sound designer you can be.”

Now, we dare you, go back to your computer, and listen out for how many sounds you hear. When Windows reminds you of your next calendar invite, will you be ‘ready to go?’. As Matthew says—you can’t unhear that.