Businesses thrive from making informed decisions so they can grow, develop and progress, but just how they get the data and information to make those decisions can be varied. One of the most beneficial and proven methods of valuable information and insights is sentiment analysis, with sentiment analysis tools helping businesses to collect and comprehend this cohesively. But how does sentiment analysis work and how can it help? Let us explain all…
Sentiment analysis is research to determine whether data is positive, negative or neutral in sentiment. It is usually carried out on textual information and processed through a sentiment analysis tool. In a business context, sentiment analysis is used to monitor perceptions, feelings and opinions on brands, products and services to help identify customer needs and wants.
At the end of the day, customers — potential, existing and past — are human, and understanding how they feel about a business can help to shape their activity in a way that meets their needs, reaches (if not exceeds) their expectations, and is beneficial to all parties.
How does sentiment analysis work?
Sentiment analysis is a natural language processing (NLP) technique. In a modern sense, sentiment analysis tools usually utilise artificial intelligence (AI) and run through a series of computer programs to analyse the content of the data supplied. The algorithms internal to the program analyse the data and present results.
Exactly how the sentiment analysis tool programming works depends on the method used, but there are several types of algorithms commonly found in such programs for businesses. The three most common are:
Rule-based system algorithm
A rule-based system uses a set of rules that are manually programmed into the sentiment analysis tool at the beginning of the analysis. These rules can be set by either the programmer or the business owner, and usually come in the form of computational linguistics rules using NLP techniques. For example, a rule-based system may be programmed with two lists of polarised words (positive words such as ‘the best’, ‘good’, ‘fast’, ‘beautiful’ and their negative opposites such as the ‘worst’, ‘bad’, ‘slow’, ‘ugly’). The sentiment analysis tool will then scan the text for the number of times each appear, judging the power of the sentiment based on the ratio of positive words to negative ones.
Rule-based algorithms in sentiment analysis tools are popular as they allow those using them to feel as though they have a level of control over the system. However, rule-based systems don’t take into accounts how words are used informally in speech or written text, or how they fit together in a sequence, which can easily skew results unnecessarily. Technology is moving fast in this area and these algorithms are developing with it; however, one flaw is when new rules are input on top of existing rules, often causing superfluous complexity. Rule-based systems also need regular maintenance and intervention to update and change rules lists to ensure data remains up-to-date and relevant.
Automatic system algorithm
A sentiment analysis tool using an automatic system algorithm relies on machine learning techniques to learn from data as it is processed. The program will tackle the text with a problem-solving method to best understand it. It returns the analysis with an overall category result — usually positive, negative or neutral.
The algorithm goes through a training process where it learns to associate a particular output with a corresponding output through test samples, then tags the output accordingly. For output it doesn’t yet know, it generates predictions (again, categorising as positive, negative or neutral). The programming allows the algorithm to understand the sentiment behind words used and group it with other instances that are similar, to help improve performance. The way the algorithm classifies and works through data is usually based on an input model, which may be either:
- Naïve Bayes — an algorithm that works through probabilities using the mathematical Bayes’ Theorem to predict a text’s category.
- Linear Regression — an algorithm used frequently by statisticians to predict values based on the features of the text (i.e. to take x and y to calculate z).
- Deep Learning — a set of algorithms working in conjunction with each other that attempt to mimic the human brain. Deep learning works by combining algorithms to predict the most accurate possible output to produce artificial neural networks, devising the most likely outcome in the manner that a human would.
- Support Vector Machines — a mapping system that categorises text into distinct regions. Different text with similarities are assigned a space close to one another and so the regions of the map each take on a distinctive character (or in this case, sentiment).
Hybrid system algorithm
Hybrid system algorithms work by combining rule-based systems with an automatic approach to deduce the most likely output (sentiment) whilst still abiding by the rules set by the user. This allows for the business using the sentiment analysis tool to still feel as though they have some control over the process and keeping it fit-for-purpose for their requirements, but without letting the limitations of a rule-based algorithm restrict the potential for output.
Who uses sentiment analysis?
Sentiment analysis tools can be used by any business, of any shape or size, in any industry — but in truth, it is only used by those who actually care about the perception, development and growth of their business. Truly customer-centric organisations use sentiment analysis tools in order to best understand those consuming their products and/or services and then use the insights to best tailor their business offerings for further success.
It is often misperceived that sentiment analysis tools, along with other similar customer insight facilities, are only the domain of big corporations with huge budgets. As it stands now, with technology having moved on so far and there being so many options available for those looking for how best to work with and for their customers, this need not be the case — and even the smallest of brands can benefit from using them.
What are the benefits of sentiment analysis tools?
Sentiment analysis tools are hugely beneficial to businesses because they allow for quick and easy navigable understanding of consumer perceptions, feelings and opinions. The results produced by sentiment analysis tools allow for the movement, development and growth of a business in order to properly align their (potential and existing) customers’ needs and wants, and best fulfil their requirements to ensure they return to purchase again and again.
The insights and business intelligence presented by sentiment analysis tools allow for those in the business to identify areas for celebration, areas for concern and areas for improvement — and focus effort and resource appropriately on each to resolve and improve these areas. For example, if a sentiment analysis tool repeatedly highlights negative feeling around one product, the business can investigate the language used and the issues raised to intervene and amend the product to fit customer expectations. If a sentiment analysis tool highlights positive feeling around another product, the business can work to market the product to more people and push it out to a wider market, knowing it is likely to be well-received.
Of course, there is nothing stopping those working within a business from collecting information themselves manually, printing it all out and highlighting in colour-coded markers the positive, negative and neutral text to gather an overall sentiment analysis output. However, this is a resource-intensive job and one that needs repeating on a sometimes-daily basis…but it can be done. Alternatively, the efficiencies that sentiment analysis tools present really make them worth investing in, as they’re able to sort the data at scale in seconds, present real-time analysis at any time of day or night and deliver consistent criteria. There are definitely some things that humans aren’t better at, and one of them is ‘agreement’ — it is estimated that people only agree on the exact sentiment of a piece of text 60-65% of the time, so words can end up being extremely subjective.
Now more than ever businesses are realising the power of consumers and are, quite rightly, moulding their practices and activities around them. Sentiment analysis tools are one such apparatus they can be used to truly work toward customer centricity; and the success that will surely follow it.
What are the limitations of sentiment analysis tools?
Sentiment analysis tools should form a part of a business’s overall customer insight work, but it must be noted that for the most part, they are computer-led and so should not completely replace ‘the human touch’. There will always be an element of consumer sentiment that can only be detected by humans, although it is important to note that those within a business may also have assumed biases which influence their input and output of a sentiment analysis exercise.
There are limitations to sentiment analysis tools, as with any computer, so the following should be kept in mind when using tool output to inform business decisions:
It is always important to ensure that the text analysed is understood in context with the question asked. For example, if a survey asks ‘what did you like about product x?’ and the customer answers ‘everything about it!’, this is a positive sentiment. But if the survey asks ‘what did you dislike about product x?’ and the same answer is given, it would be considered a negative sentiment. This may seem obvious and common sense, but to a computer the responses are identical, therefore context is crucial.
Tone and sarcasm
Machines can’t read verbal tone and when humans speak and write sarcastically, they use positive language to intone a negative message. Without a specific textual clue that the information being portrayed is false — which is something likely only conveyed in text by an experienced, professional, skilful writer — machine learning is unable to judge it as anything other than positive. For example, ‘yeah, it’s so great’ in the context of an overall negative review is unlikely to be a positive — but would rank so through machine learning.
When writing a comparison, it can be difficult for machine learning to understand which is the relevant product the computer should be analysing the text on. In these cases, algorithms may process the data incorrectly or as neutral, despite a sentiment being clear for a human to pick up one. For example, ‘this is better than nothing at all’ is a neutral sentiment but ‘this is better than the last version of this product’ is subjective, dependent on how the last version of the product was perceived and received.
Although emojis can be accounted for in machine learning algorithms as each has a set definition, humans rarely always use them for their intended purpose, which, when read by a computer, can skew the context and meaning of text. For example, the painted nails emoji is literally defined as ‘coloured nail polish being applied to fingernails’ — however, it is often used to show sass, flair, or nonchalance. Similarly, the emoji of the person tipping their hand out to their shoulder is actually an ‘information desk person’ but is usually used to convey sarcasm or indifference.
Defining exact neutrality in language can be difficult, particularly in a customer feedback, as true neutrality is likely to be complete indifference, which would probably result in the brand and its products or services not being mentioned at all.
Objective pieces of text — which don’t include any opinion or specific sentiments — can be programmed into a sentiment analysis tool to be read as neutral, but human writing rarely displays exactly this. Irrelevant information is often conveyed in human text and this may need to be discounted as neutral by the machine learning in order not to skew the output unnecessarily; however, doing this too often may still affect the output by rating it up or down toward a mid-point of sentiment in neutrality. Text may also include desires from the customer, such as ‘I would love if product x did this’ or ‘I wish you offered service y too, it would be so much easier!’. These both include positive language but aren’t explicitly positive sentiments given the context, so can be difficult to categorise.
Of course, what a sentiment analysis tool can never uncover is the accuracy given in any text it reads. A customer’s perception of what a product or service should or may do could not accurately reflect its actual performance, throwing off the end result because of miscommunication or misunderstanding. What’s more, particularly in the case of overtly negative text, it may be that some fabrication or embellishment is made within the information supplied that isn’t factual yet still slants the output in one way or the another. This is something frequently being seen with the likes of ‘bought’ reviews on Amazon and other e-commerce websites, but in a plainly positive manner.
If humans can’t always tell when other humans are telling the truth, how can machine learning be expected to? Uncovering the truth, and the intentions of its source, will always be a limitation — but one that also affects people, too.
What uses are there for sentiment analysis tools?
Sentiment analysis tools can gather information from a variety of sources, some of which we will cover in this article. Not all businesses have access to reams of customer feedback, but this need not stop them using sentiment analysis.
Tool #1: Feefo Sentiment Analysis
Online reviews are something that many consumers engage with — either in the reading or writing of such content — and are considered a trusted method of information because of their perceived authenticity and impartiality. When compiled and managed through a third-party independent system like Feefo, consumers are able to consume the review content with full trust in its legitimacy, as reviews are verified before they are submitted. However, independent review management isn’t just of benefit to the consumer. Businesses too can use the user generated content (UGC) created from submitted reviews to uncover a variety of customer insights and intelligence by using tools such as Feefo Smart Themes, which conducts sentiment analysis.
Feefo Smart Themes allow for easy identification of not just the sentiments echoed through review content, but also themes. The AI behind the algorithm groups popular themes from feedback and then highlights them to businesses and customers alike. For e-commerce businesses, these themes can even be displayed on the product purchase page to draw attention to positive sentiments (such as quality, feel, fit, value) raised by those who already own the product.
Furthermore, by working via the Feefo dashboard of services, sentiment can be gathered and grouped by product, service, demographic or date to focus in on areas for improvement, concern and celebration, and ultimately learn from them.
E-commerce businesses also have access to complete customisation over the reviews widget on their site, allowing them to brand all themes, topics and snippets to create the optimum user experience (UX) and generate the most relevant call to action (CTA). What’s more, of course, the positive reviews are available publicly for access by anyone, also acting as a superb marketing tool!
Tool #2: Multi-lingual Sentiment Analysis
Reading this article in English, it’s likely that you’re considering the application of sentiment analysis tools across text written in English, but businesses operating internationally may need their tools to be able to read and analyse input in a variety of different languages.
Multi-lingual sentiment analysis tools are available, although they do often require a degree of manual input. Companies such as Rosette and MeaningCloud are able to comprehend and process text in over 30 languages, so sentiment can be determined no matter the input’s linguistics. What’s more, this can help to identify specific issues or successes of any region or language, allowing the business to focus activity as and where required to serve different audiences for the best possible result.
Tool #3: Real Time Social Media Monitoring
Social media moves fast. About 350,000 tweets are sent every minute of the day, almost 9 million photos and videos are shared on Instagram every day and 4.75 billion posts are shared on Facebook daily. With manual input for sentiment analysis tools, there is simply no way that anyone would be able to keep up and ensure up-to-date accuracy. This is where real-time social media monitoring tools come in to play.
Sentiment analysis tools such as Social Mention and MonkeyLearn scan designated social media channels for publicly available text mentioning or tagging a specific brand name, business name, product or service. The AI then categorises the sentiment of keywords in the text and amalgamates them to judge an overall positive, negative or neutral categorisation of output.
Real-time social media monitoring can be hugely valuable to stem the flow of any issues, misinformation or areas of concern before they escalate, as well as to take advantage of positive ‘buzz’ when it occurs. As fast as social media moves, so too must businesses — and only being able to access information as it happens across these channels will allow then to do this.
Where to start with sentiment analysis tools
Sentiment analysis tools require input, so the key to gaining appropriate information from which to glean sentiment and consumer data is to ask customers! Customers are unlikely to respond to a ‘please tell us what you think’ request out of the blue with no context — or indeed understand why they are being asked — but will usually be receptive to providing review content, so Feefo is a great place to start!
Once customers are encouraged and empowered to engage and share their thoughts, feelings and opinions with the brand, work can begin to best understand what they’re saying, what they’re looking for and what they mean. This can be achieved through a whole host of sentiment analysis tools combined with a bit of businessperson ‘knowhow’, but also by using the complementary suite of services that you’ll find on the Feefo dashboard. Dive in to calculate a net promoter score (NPS), benchmark performance against other industry players, integrate positive content into website UX and advertising, and even use customer insights to influence business strategy. The key is the customer… you’ve just got to start listening!