Improving Effectiveness and Efficiency of Sentiment Analysis

Modha Jalaj S.

Chapter – 1

1. Introduction:

Big Data has been created lot of buzz in Information Technology word. Big Data contain large amount of data from various sources like Social Media, News Articles, Blogs, Web, Sensor Data and Medical Records etc.

Big Data includes Structured, Semi-Structured and Unstructured data. All these data are very useful to extract the important information for analytics.

1.1 Introduction of Big Data: ^[26]

Big Data is differs for other data in 5 Dimensions such as volume, velocity, variety, and value. ^[26]

• Volume: Machine generated data will be large volume of data.

• Velocity: Social media websites generates large data but not massive. Rate at which data acquired from the social web sites are increasing rapidly.

• Variety: Different types of data will be generated when a new sensor and new services.

• Value: Even the unstructured data has some valuable information. So extracting such information from large volume of data is more considerable.

• Complexity: Connection and correlation of data which describes more about relationship among the data.

Big Data include social media, Product reviews, movie reviews, News Article, Blogs etc.. So, to analyze this kind of unstructured data is challenging task.

This thing makes Big Data a trending research area in computer Science and sentiment analysis is one of the most important part of this research area.

As we have lot of amount of data which is certainly express opinion about the Social issues, events, organization, movies and News which we are considering for sentiment analysis and predict the future trends and effect of certain event on society.

We can also modify or make the improve strategy for CRM after analysing the comments or reviews of the customer. This kind analysis is the application of Big Data.

1.2 Introduction of Sentiment Analysis:

Big Data is trending research area in computer Science and sentiment analysis is one of the most important part of this research area. Big data is considered as very large amount of data which can be found easily on web, Social media, remote sensing data and medical records etc. in form of structured, semi-structured or unstructured data and we can use these data for sentiment analysis.

Get Help With Your Essay

If you need assistance with writing your essay, our professional essay writing service is here to help!
Find out more about our Essay Writing Service

Sentimental Analysis is all about to get the real voice of people towards specific product, services, organization, movies, news, events, issues and their attributes^[1]. Sentiment Analysis includes branches of computer science like Natural Language Processing, Machine Learning, Text Mining and Information Theory and Coding. By using approaches, methods, techniques and models of defined branches, we can categorized our data which is unstructured data may be in form of news articles, blogs, tweets, movie reviews, product reviews etc. into positive, negative or neutral sentiment according to the sentiment is expressed in them.

Figure 1.2.1: Sentiment Analysis

Sentiment analysis is done on three levels ^[1]

• Document Level
• Sentence Level
• Entity or Aspect Level.

Document Level Sentiment analysis is performed for the whole document and then decide whether the document express positive or negative sentiment. ^[1]

Entity or Aspect Level sentiment analysis performs finer-grained analysis. The goal of entity or aspect level sentiment analysis is to find sentiment on entities and/or aspect of those entities.

For example consider a statement “My HTC Wildfire S phone has good picture quality but it has low phone memory storage.” so sentiment on HTC‟s camera and display quality is positive but the sentiment on its phone memory storage is negative. We can generate summery of opinions about entities. Comparative statements are also part of the entity or aspect level sentiment analysis but deal with techniques of comparative sentiment analysis.

Sentence level sentiment analysis is related to find sentiment form sentences whether each sentence expressed a positive, negative or neutral sentiment. Sentence level sentiment analysis is closely related to subjectivity classification. Many of the statements about entities are factual in nature and yet they still carry sentiment. Current sentiment analysis approaches express the sentiment of subjective statements and neglect such objective statements that carry sentiment ^[1].

For Example, “I bought a Motorola phone two weeks ago. Everything was good initially. The voice was clear and the battery life was long, although it is a bit bulky. Then, it stopped working yesterday. ^[1]” The first sentence expresses no opinion as it simply states a fact. All other sentences express either explicit or implicit sentiments. The last sentence “Then, it stopped working yesterday” is objective sentences but current techniques can’t express sentiment for the above specified sentence even though it carry negative sentiment or undesirable sentiment. So I try to solve out the above problematic situation using our approach. ^[1]

The Proposed classification approach handles the subjective as well as objective sentences and generate sentiment form them.

1.3 Objectives:

The objective of this research work is to improve the effectiveness and efficiency of classification as well as sentiment analysis because this analysis plays a very important role in analytics application.

Till now Sentiment analysis focus on Subjectivity or Subjective sentiment i.e. explicit opinion and get idea about the people sentiment view on particular event, issue and products. Sentiment analysis does not consider objective statements although objective statements carry sentiment i.e. implicit opinion.

So here the main objective is to handle subjective sentences as well as objective sentences and give better result of sentiment analysis.

Classification of unstructured data and analysis of classified unstructured data are major objectives of me.

Practical implementation will be also done by me in the next phase.

1.4 Scope:

Scope of this dissertation is described as below.

• We are considering implicit and explicit opinion so sentiment analysis expected to be improved
• Analysis of unstructured data gives us important information about people choice and view
• We are proposed an approach which can be applied for close domain like “Indian Political news article”, “Movie Reviews”, “Stock Market News” and Product Review” so, with the consideration of implicit and explicit opinions we can generate precise view of people so industries can define their strategies.
• Business and Social Intelligence applications use this sentiment analysis so with this approach it’ll be efficient.

5. Applications:

There are so many application of Sentiment Analysis which is used now-a-day to generate predictive analysis for unstructured data.

Areas of applications are Social and Business intelligence applications, Product reviews help us to define marketing or production strategies, Movie reviews analysis, News Analysis, Consider political news and comments of people and generate the analysis of election, Predict the effect of specific events or issues on people, Emotional identification of person can be also generated, Find trends in the world Comparative view can also be described for products, movies and events, Improve predictive analysis of return of investment strategies.

1.6 Challenges:

There are following challenges which are exists in sentiment analysis are

• Deal with noisy text in sentiment analysis is difficult.
• Create SentiWordNet for open domain is challenging task i.e. make a universal SentiWordNet is the Challenging task.
• When a document discusses several entities, it is crucial to identify the text relevant to each entity. Current accuracy in identifying the relevant text is far from satisfactory.^[5]
• There is a need for better modelling of compositional sentiment. At the sentence level, this means more accurate calculation of the overall sentence sentiment of the sentiment-bearing words, the sentiment shifters, and the sentence structure. ^[5]
• There are some approaches that use to identify sarcasm, they are not yet integrated within autonomous sentiment analysis systems.^[5]