What is sentiment analysis and how can machine learning help customers?

When you think of artificial intelligence (AI), the word “emotion” doesn’t typically come to mind. But there’s an entire field of research using AI to understand emotional responses to news, product experiences, movies, restaurants, and more. It’s known as sentiment analysis, or emotion AI, and it involves analyzing views – positive, negative or neutral – from written text to understand and gauge reactions.

Sentiment analysis can be used for survey research, social media analyses, and tracking psychological trends. Picture software that scans articles, reviews, ratings and social media posts to determine sentiment changes for hotel guests. Hoteliers will, for example, aggregate and assess ratings and reviews in effort to improve guest satisfaction.

The tech behind sentiment analysis involves natural language processing or linguistic algorithms that assign values to positive, negative or neutral text (converting opinions into datasets), while machine learning processes the datasets to reveal relevant trends over time. There’s significant planning required: How do you ensure the algorithms capture useful information? Are you identifying the right phrases to analyze? How can you convert findings into better products, services, and experiences?

Analysis Helps Uncover Customer Needs

At Concur, understanding our users and their needs is important. This allows us to see what we’re doing well and where we can improve, and sentiment analysis can provide invaluable insights. Recently, Concur Labs and Concur UX Analytics developed a sentiment analysis tool for user product reviews. Our tool automatically extracts themes to determine how customers feel about our service, and helps identify which features people like most and which ones they find frustrating.

Unlike other apps, we measure success not in how much time you spend in our app, but how little. The faster you can expense a trip, for instance, the better. Our analysis found that people wished for even faster capabilities. It also revealed that people really like some of the lesser known features, like mileage tracking.

Emotion Gauging is Complicated

If we could categorize responses with just one emoji that would easy. But humans are far more complicated and fascinating. This complexity applies to sentiment analysis. For example, comments like “the film was very good” are easy to analyze. But it gets a little harder when you add negation: "The film wasn't bad." It gets much harder when you add terms that would normally come across as positive but are actually negative, based on context. For instance, "I wish this film was good. There were great many things it could have done right but didn't.”

As a relatively new field, approaches are varied and maturing. Analysis has been traditionally conducted by taking what's called a "bag of words" approach. Basically creating a list of all the words used along with how many times they were used. With this method, word order is thrown out the window. So "not bad" would come out as negative. Modern methods use recurrent neural networks called LSTMs (long short-term memory) to compress the entire sentence into a vector (a list of numbers) that encapsulates the meaning of the sentence, taking word order into account. This tends to have higher accuracy.

For businesses invested in customers, analyzing each piece of feedback by hand can be overwhelming. Sentiment analysis, developed within context, can help catch issues early and provide guidance on how to improve services. The related machine learning algorithms can take vast amounts of data; learn and perform specific tasks quickly; and sift through data based on your priorities. As the technology advances, businesses can benefit from these in-depth insights and customer satisfaction will surely follow suit.