voice analysis

The person who coined the phrase:

“Eyes are the window to the soul”

was wrong.

If anything can offer us a unique insight into someone’s soul, it is the human voice. The voice tells us something about someone’s mood, someone’s mind, and someone’s history.

Sometimes it’s subtle. Sometimes it cries out to you.

The voice is an example of how mind and body are clearly connected. Our tone and texture changes when we’re in love, when we’re angry, when we’re feeling insecure, and when we’re sick.

The way someone speaks can tell us where he or she is from, how (and where) someone was educated, and it reveals something about someone’s (desired) social status.

By listening to someone’s voice, experts can diagnose certain health problems. A croaky voice may indicate acid reflux. A head cold voice can point to chronic sinusitis. A hoarse voice could be a sign of laryngeal cancer.

But there’s more.

We can change the meaning of words, simply by changing our tonality.  When our body language, the words we speak, and our tone of voice don’t match, we won’t be taken seriously. 

People can hear we’re not sincere. In fact, sincerity is so hard to hard fake that only pros can pull it off.

EMOTIONAL IMPACT

You and I have been touched by certain voices. For better, or for worse. Can you think of a few?

As kids, we’ve all experienced that when our mom or dad called us with that special tone of voice, we knew we were in trouble.

Certain teachers had the uncanny ability to terrify us, because of what they said, and how they said it. So much so, that years later, we can still recall their voices, and get an instantaneous physical reaction.

Someone’s voice can also induce a very positive mood.

Perhaps it’s just me, but I could never fall in love with someone who has a horrible voice. When our beloved whispers our name in that very special way, our heart melts, and we’re almost hypnotized. 

When a charismatic public speaker rallies the troops, we feel energized and inspired.

That first word from a child we brought into the world, is something we’ll always remember.

Our sensitivity to tonality comes from the time we were infants, when we learned to attribute feelings to certain words through the way they were spoken. 

UNIQUE OR UNIVERSAL

Now, there’s one thing I’ve always wondered. With so many cultures, languages, and dialects in the world, are certain vocal inflections universal, or limited to one geographic area? More importantly, do they mean the same thing?

Take the tonality of love, for instance. Is that something we have in common with every person on this planet? Does “angry” sound the same, wherever we go?

Yuval Mor and Yoram Levanon spent eighteen years researching more than sixty-thousand test subjects speaking twenty-six different languages. What they found was surprising: language and culture make little difference in what they call “emotion analysis.”

Emotional Analytics is a new scientific field that focuses on identifying and analyzing the full spectrum of human emotions and personality. Yuval and Yoram’s company Beyond Verbal, has developed a way to decode vocal intonations into their underlying emotions in real-time.

It’s all based on the notion that what we say is not as important as how we say it.

THERE’S AN APP FOR THAT

In 2013, Beyond Verbal launched a free app called Moodies to extract, decode, and interpret human emotions from voice samples that are as short as twenty seconds. The app claims to give information on the speaker’s mood, his or her attitude, and on someone’s personality.

Here’s how it works.

The software examines how we speak, and listens for specific patterns. It analyzes things like pitch, tempo, pauses, and the volume of the voice. It then compares these patterns to a database of research. The ongoing analysis on the screen, is presented in clusters as the subject speaks. 

To see this in action, here’s a short clip from an interview with whistleblower Edward Snowdon. Be sure to select HD in the YouTube settings before you start watching.

Beyond Verbal has an interesting YouTube channel with voice analysis of people like Steve Jobs, Jeb Bush, and Winston Churchill. 

It’s important to note that analyzing emotions is very different from detecting lies. That is something the software cannot do.

Currently, the program can recognize about four hundred different emotions. The makers say it’s about eighty percent accurate.

PRACTICAL APPLICATIONS

If Beyond Verbal’s method is correct, we now have a way to find out what people really feel, in spite of what they’re saying. That information could be useful in at least three areas:

1. Person to person interaction.

Beyond Verbal software is already used in call centers. It helps market researchers to find out how people genuinely feel about products, promotions, and… politicians. Researchers can get past the socially acceptable answers, and go with the emotional response.

Voice analysis is also used in job interviews and sales meetings. It can answer questions like: “Is the client truly receptive to our offer, or merely being polite? Is this applicant really confident, or is he putting on a show?”

It turns out that it’s easier to fool people than to mislead computers. 

2. Allowing machines to understand us better, and improve interaction.

At the moment, virtual assistants such as Siri and S Voice base their response on what we say, and not on how we say it. If they could read our mood, this could influence their answers. Beyond Verbal has already made their platform available to other developers to make the devices of the future more intuitive.

Let’s say we’d use voice control for a service like Netflix. Based on our intonation, Netflix could recommend movies that would fit the mood we’re in. iTunes could work the same way. Some video game controllers already respond to subtle pressure and body heat. What if they could hear our fear, and change the progress of the game accordingly?

What if voice analysis software in a car could pick up if a driver was under the influence of alcohol, or suffering from road rage? Based on that, it could start making adjustments, and e.g. slow the vehicle down.

3. Self-improvement; getting a better understanding of ourselves.

This is particularly interesting to me as a professional communicator. Quite often, there’s a disconnect between how we think we come across, and how our communication is perceived. Let’s say you have a piece of copy that needs to be read in a friendly, but convincing way. How do you know you hit the nail on the head? Do you call your coach, a friend or a colleague?

I took my iPhone, opened up the Moodies app, pressed the mic button, and started reading the script. After about fifteen seconds, I got my feedback in three layers (see picture on the left). The app keeps refreshing, so you can see if your adjustments have the desired effect.

When you’re done, and you concur with the analysis, you can click “Agree,” helping the software to be more accurate in the future.

I have to admit, before I tried Moodies I was very sceptical. I mean, can a machine really detect emotions? It’s hard enough for us, humans. But when I started using it, I was surprised by the results. Whether I was speaking Dutch (my first language), or English, it was quite accurate.

Moodies didn’t tell me what I wanted to hear, like horoscopes do. It told me what I needed to hear. Based on that, I changed my tonality to match the specs of the script. Getting this type of instantaneous feedback was refreshing!

LOOKING AHEAD

Beyond Verbal was launched in May 2013, with a 3.8 million dollar investment, and has about twenty employees. 

Examiner.com named Moodies the best iPhone app of 2014, and Forbes listed it as one of the five innovative marketing solutions that can help a business grow.

This Tel Aviv-based company is definitively onto something, and it seems they’ve only scratched the surface. 

Even though I believe a computer can never penetrate the depths of the human soul, it can certainly open a window to our emotions.

Today, it seems that one of the best ways to unlock that window, actually speaks for itself.

Our voice.

Paul Strikwerda

Be sweet. Please retweet.

photo credit: Mother via photopin (license)