Anyone who’s been on a first date knows that trying to read a total stranger’s emotions is completely bewildering. Your brain panics — Did I say something wrong? Should I not have joked about dead monkeys? — while you flash a nervous smile and sip your wine. These anguished questions have long gone unanswered, but a new emotion-reading smartwatch app prototype may soon provide hope for the communication- challenged.

The new system, designed by engineers at MIT’s CSAIL, uses artificial intelligence to tell whether an ongoing conversation is happy, sad, or neutral. It uses a person’s speech patterns and vital signs to measure the tone of the conversation as it goes on.

“Imagine if, at the end of a conversation, you could rewind it and see the moments when the people around you felt the most anxious,” graduate student Tuka Alhanai, who worked on a similar project, said in a statement. She referred to the new work as the first step to having a portable “social coach.”

Even rapidly changing emotions can be detected.
Even rapidly changing emotions can be detected.

In the unpublished study, which they will present at the Association for the Advancement of Artificial Intelligence conference next week, they show that their algorithm was 83 percent accurate in classifying conversations by emotion. They programmed their algorithms by first having participants put on smartwatches programmed with the algorithm (they were partial to the Samsung Simband, but theoretically any vitals-tracking device could be used), then asking them to tell a happy or sad story of their choosing. Those stories — the language they contained, together with the bodily responses and movements of the people telling them — gave the algorithms a basis for what “happy” and “sad” looked like; they were then trained to identify the overall tone of a conversation, together with that of five-second snippets.

story continues below
Any vitals-tracking smart wearable could do the job.
Any vitals-tracking smart wearable could do the job.

In a sense, what the algorithm really did is break down, systematically, what the body does when it’s having a happy or sad conversation. Happy stories are told through energetic speech patterns; sad ones are told with long, dolorous pauses and monotone voices, together with certain actions like fidgeting or raising hands to the face. Smart watches have the capacity to measure all of these factors through their accelerometers, microphones, and temperature sensors, making them the perfect tool for measuring this very telling data.

One major obstacle to turning this technology into a dating coach, however, is the fact that it’s only legally allowed to measure one side of the conversation (even though it could very well assess both.) It’s one thing to measure your own vital signs while you’re telling a story; it’s another, more legally hairy thing to measure someone else’s — or even just their words they’re saying (there are very strict laws about recording someone else’s speech without their knowledge).

Still, the emotionally dense among us can take heart: You won’t be in the dark forever. Further training, aimed toward increasing the algorithm’s emotional sensitivity to include “boring,” “tense,” and “excited,” should make the reality of future first dates objectively (and maybe painfully) clear.

Photos via MIT CSAIL, Jason Dorfman, MIT CSAIL