Understanding Sentiment Analysis

Sentiment Analysis can be considered a subfield of information extraction. It is used in a wide range of areas and is sometimes referred to as Opinion Mining. Sentiment Analysis attempts to determine the overall attitude (positive or negative) expressed within a text. Its purpose is to represent emotional or affective meaning.

The technique’s success lies in an imperative need to standardize the measurement of human emotion in social media in order to efficiently monetize it. Signalling a users’ preference is a vital component of the Like economy that underpins the business models of all major social media companies. Peer recommendations are highly trusted by other peers. This makes Sentiment Analysis a valuable source of information.

Scoring 🎯

The simplest scale for sentiment scoring is a binary (positive–negative) or modal (positive–neutral–negative) categorization. Alternatively a score between 1 and –1 provides a more detailed range from very positive to very negative.

The level on which the sentiment is scored, depends on the needs. Establishing a sentiment classification on document-level is often seen. Although applying sentiment analysis on the level of sentences, phrases or named entities is also common.

The simplest approach for scoring a sentiment is a calculation based on a lexicon. This is a precompiled wordlist of terms that indicate positive or negative expressions of sentiment. Summing the positive and negative hits within a document will give the score. In some cases, a simple majority decides the final labeling of the post. This can be enhanced by recognizing negation, applying rulebased predicate structures or by preprocessing steps like lemmatization.

A more elaborate strategy would be a classification algorithm that interpretes on the level of sentences or phrases. The selection of linguistic features for clasification also requires choices. For example, term frequency has been found to be a poorer predictor than term presence. The usage of highly emotionally charged terms is more significant than their exact frequency. Word class, multi-word phrases, syntax, and negation have all been used as features, as have been the use of text length, exclamation marks, all caps, and character repetition.

Challenges ⛰️

• Often it is unclear what is measured, the polarity of a described entity or the emotional state of the writer.
• Issues arise when genre or domain-specific sentiment algorithmes and dictionaries are suddenly applied to another field and context-dependent word meanings no longer fit with the original context.
• The absence of detection of fake news and fraudulent reviews. Fake opinions try to deliberately mislead readers and algorithms by giving undeserving positive opinions to some target objects in order to promote the objects.
• People who are skeptical whether computers are able to understand the complexities of natural language assume scoring and interpretation are the same step. While the system only delivers a score, the result should still be interpreted.
• Language is easy to misinterpret. For example, misclassification can occur because sarcasm and humor are difficult to recognize.
• Spoken opinions complicate sentiment analysis. Because proper language structure is often ignored and signs of body language and tone of voice are not recorded.
• Sentiment analysis cannot determine why someone is unhappy. On the other hand, it is not easily solved by humans either.

Applications 💶

• Market Research: Understanding the voice of Customers when they express their desires, thoughts, preferences and frustrations
• Public Relations: Identify opinions towards public persons or organisations
• Customer Service and Support: Identify the needs to solve a customer request
• Human Resources: Understand the voice of Employees
• Healthcare: Understanding the feelings of Patients and measuring the effect of a chosen therapy
• Stock Market: Predicting Stock prices, as they are often driven by positive or negative information
• Recommender Systems: Help users to decide on products
• Business Analytics: Using Sentiment Analysis for decision support systems and business process improvement.

Sentiment vs. Affective Meaning 🗣

The expansion of the area of Sentiment Analysis has resulted in a new interest in the quantification of opinion, sentiment, affect, feeling, emotion, personality, mood and attitude. These terms are often used interchangeably.

We differentiate sentiment from affective meaning based on their duration. Sentiment lives longer, while an affective state has a short term duration. An effective sentiment analysis system is one which captures the sentiment of the opinion about an entity. The recognition of Affective Meaning is for example the automatic discovery of an emotional reaction, often of a single person. Unlike opinions, emotions are short-term.

Also read our post on understanding Affective Meaning