Evaluation Of Ratings In Social Media Sites Information Technology Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

The task of identifying high-quality content in sites based on user contributions from social media sites becomes increasingly important nowadays. Social media provide a range of applications and services available to support communications, collaboration, and information sharing within a community. This paper proposes a framework for evaluation of ratings in social media sites in a distributed environment. The framework takes into account the rating in terms of the reliability of content and additional information about the expert who contributed the content. A framework for the computation of rating is presented based on the votes of users to a particular content, comment and answers. Ratings of their submitted contents, referred links that were found very useful, comments on other contents and reliability of their answers to questions are being computed to get the expert's overall rating. The author illustrated the efficiency of the rating scheme when using Bayesian average rather than Arithmetic average in the calculation of the rating. Also, the user's rating as an expert is reflected in the system via automated mechanism for user's expertise calculation.

1. Introduction

A lot of resources from the Internet increase everyday that makes us exposed to a very large amount of data. As the availability of user-generated content increases, the task of identifying high-quality content in sites based on user contributions from social media sites becomes increasingly important.

Social media [1] is best understood as a group of new kinds of online media. It provides a range of applications and services available to support communications, collaboration, and information sharing within a community. In general, Social media exhibit a rich variety of information sources. In addition to the content itself, a wide array of non-content information is also available, such as number of contents generated or referred by an expert and ratings of content from members of the community.

Rating systems [2] center around rating content, often user-contributed content, and they frequently help apply community values and acclaim to that content. They are very useful for ordering resources in terms of importance and usefulness. Resources are being rated by users and the results are presented in the form of ranking.

Statistical methods will become more important in a wide variety of fields particularly in interpreting and organizing the vast amount of information. [3] A wide variety of approaches to resource rating are being used on the Internet. Some of these use simple formulas to calculate the rating of a resource.

The majority of rating systems found on the Internet are based on mathematical calculations of rates given by users. Users' reliability to give information is not taken into account. There is a strong need to construct an efficient framework for evaluation of user ratings in a social media considering more information about users. Their ability to answer questions, give useful comments and level of expertise can be priceless for every rating system.

Associating a user with its ratings on generated contents is a capability that is essential in overcoming the limitations of rating systems. [4] This paper proposes a framework for evaluation of ratings in social media sites; it is taken into account the resources and the experts who are producing these electronic resources. The proposal for a framework based on the ability of experts to contribute reliable information, submit useful comments and their ability to answer questions based from their expertise.

The framework aggregates the information about the ratings of their submitted contents, referred links that were found very useful, giving helpful comments to other resources and reliability of their answers to questions. This process of rating provides efficiency because the user's rating as an expert is reflected which are based on reliability of his contributions, usefulness of his answers to questions and the amount of content a user created. An automated mechanism for user's expertise calculation is also provided by the system.

2. Related Work

Social media content is being used as an effective resource to a lot of Internet users. The particular context of social media communities we focus on this paper has been the object of some study in recent years. According to Hitlin, et. al. [5] 33 million American Internet users have reviewed or rated a product, service, or person using an online rating system. As more people use the Internet for entertainment, for building personal relationships, and as a tool for conducting business, online rating systems have become a significant element of Internet use. These systems, also referred to as "reputation systems," are online applications that allow users to express their opinions and read opinions posted by other participants.

Lerman [6] concluded that social media sites underscore the Web's transformation to a participatory medium in which users collaboratively create, evaluate, and distribute information. Innovations in social media have led to social information processing for the purpose of document recommendation and rating.

A research made by Turnbull [7], where he presents the idea of OpenChoice system. The project is created to build a community of rating users. The system asks every user to rate resources that are not rated sufficiently yet. The author also quotes some popular communities, well-known on the Internet. Almost every Web 2.0 webpage uses at least one type of rating. For example, Youtube [8] allows for interactive rating (1-5 stars), views or comments. Another popular community site Digg.come [9] focuses on simple "thumbs-up" or "thumbs-down" rating for sites. It also shows the quantity of comments and proposed category for each website.

Yahoo! Answers [10] is an example of social media site where anyone can post a question including queries that can't be answered by a traditional search engine. It has a system of points and levels to encourage participation and reward great answers. The points allow everyone to recognize how active and helpful a user have been. Levels are another way to keep track of the users. The higher the points a user accumulate, the higher the level he may attain. The problem with the Yahoo! Answers' points and levels system is that as long as a user keep answering even though his answer is not true, points will still be rewarded and eventually gain a certain level.

Ching, et. al. [11] concluded that the emergence of easily disseminated information technology gives unprecedented freedom for individuals to distribute information widely and cheaply either via informal "blogs" or via more formal and structure electronic information repositories. People must respond to this new technology in creative and constructive ways by seeking to utilize this technology for learning purposes.

Agichtein [12] introduced a general classification framework for combining the evidence from different sources of information, that can be tuned automatically for a given social media type and quality definition. The focus of the study is in community question/answering domain, particularly, Yahoo! Answers which is a large community question/answering portal. Their system was able to separate high-quality items from the rest with accuracy close to that of humans.

Marmolowski, et. al. [13] concluded that the most common problem in lots of rating systems is inadequate rating for new resource. The most popular approach is the arithmetic mean; it is much easier for new resources to get a high overall rating. An efficient solution for that problem is the use of Bayesian rating that decreases the influence on the overall average until the number of resources reaches a specified amount. Bayesian rating is using the Bayesian average. This is a mathematical term that calculates a rating of an item based on the "believability" of the votes. When there are very few rates, the Bayesian rating of an item will be closer to the average rating of all items.

Dom, et. al. [14] studied the performance of several link-based algorithms to rank people by expertise on a network of e-mail exchanges, testing on both real and synthetic data, and showing that in real data.

3. Theoretical Background

3.1. Average Rating Calculation

The main goal of this step is to calculate an average rate of an item; this may be an answer to a question, a post or a resource link given by an expert based on his knowledge and his interest or a comment given to a resource.

The framework is using the Bayesian average in getting the mean of a resource. First, get the weighted average [15] for the votes of all items.


Vi = number of votes for the item

Wi = weight for the item i

n = total number of items in the database

Second, calculate the average rating of all items.


Ri = rating for the item

n = number of votes

3.2. Bayesian Weighted Rate Calculation

The best way to make the ratings statistically sound is with volume. The fewer ratings it has, the more likely it is that the ratings are inaccurate in relationship to the database of ratings as a whole.

Ideally what is needed to do is give items with fewer ratings among the collection, less weight, and those with more ratings, higher weight. To do this, Bayesian average will be applied. The idea behind a Bayesian average is that the ratings be normalized by pushing them toward the average rating for the site, and it is done more for items with fewer ratings than those with more ratings.

For the computation of the Bayesian average, the following formula is used:


Av = the weighted average of all votes

Ar = the average rating of all items

Vi = the number of votes for the item

Ri = the rating of the item

Note that Av is the weight in this formula, the higher is its value, the more votes it takes to influence the Bayesian rating.

4. Proposed Evaluation Scheme

4.1. Expertise Value

The expertise value of a user can be calculated with the following formula:


BRp = Bayesian rating for expert's posts

Wp = weights of posts in percentage

BRc = Bayesian rating for expert's comments

Wc = weights of comments in terms of percentage

BRa = Bayesian rating for expert's answers

Wa = weights of answers in percentage

In this paper, Wp = 50%, Wc =20%, and Wa =30%, was used to come up with the overall rating of a user.

4.2. Expertise Classification

Table 1 User Level Classification


Number of Posts

Overall Rating

1 - Novice



2 - Contributor

1 - 50


3 - Expert



In Table 1, users in different levels of expertise were classified; the user's attained level is based on his number of posts and overall rating. The overall rating was computed based on the ratings given by other users from his contributed contents, comments to other posted contents and answers to questions.

4.3. Expertise Calculation

The formula for Expertise calculation contains three parts. A user is a complete novice who has no initial value, a contributor who has already published at least one resource with a user rating that is less than or equal to 3, an expert if he has already published more than one hundred resource with an overall rating of more than 3. The proper formula looks as following:

Ur=0, for P=0

EXP(Ur, P) Ur≤3, for P≤50

Ur>3, for P>50

The environment for experts and users is just like real life situations which enable an active participation.

Also, the major cause of its difference is the distinction of every user based on his expertise level. Users having high level of expertise are more reliable in rating. The reliability is also higher for users having actively participated in giving comments and answers.

5. Results of the Experiments

The results of the experiments conducted from different groups of randomly generated datasets of ratings prove the efficiency of our framework. We note that items 1 to 5 has only 10 votes and items 6 to 10 has 100 votes. The average rating for our site is 2.86 based from the random ratings with the scale of 1 to 5.

Figure 1. Bayesian average vs. arithmetic average on items with low ratings

Figure 1 illustrates that Bayesian rating has a better result compared to that of average rating when it comes to items with low ratings. Users in this case gave low rating to items which is either 1 or 2.

Figure 2. Bayesian average vs. arithmetic Average on items with high ratings

Figure 2 illustrates the result that Bayesian rating is more efficient than average rating. . Users in this case gave high rating to items which is either 4 or 5. The idea of our scheme is that the higher the number of votes of an item, the more reliable it is compared to items with only a few votes.

Figure 3. Bayesian average vs. arithmetic average on items with random ratings

Figure 3 demonstrates that Bayesian average has a more stable graph compared to arithmetic average on items with random ratings. . Users in this case gave random rating to items which is from 1 to 5.

The idea here is that our framework uses the Bayesian average which normalizes ratings by pushing them toward the average rating for our site which makes it a competent evaluation of social media. If we compare our framework to other frameworks which are using ratings based on arithmetical mean, we can observe that our framework is more efficient.

It is observed in our results that items with large votes highly influence the rating of an item than items with only a few votes. This applies to real life situation where the reliability depends on the rate of a lot of users than the rate of only a few users.

6. Design of the Evaluation Scheme

6.1. Framework Architecture

It is shown in Figure 4 that the framework offers a reliable and efficient solution for expertise and resource collection. People publishing and referring high quality resources can be thought as experts in that particular field.

Figure 4. Model Diagram for a Framework for Evaluation in Social Media Sites

The measure of quality can be achieved by other users' rates. A person having well-rated resources can be thought as an expert in that field. Also, acquiring the number of user-generated or referred contents, content rating, comment rating and Q&A rating is demonstrated. The number of posts and the overall rating of a user is a very important factor for the system to classify a user's expertise level.

The goal is to provide additional information about a user's level of expertise through the ratings of his resource and ratings of his answers on various questions. With these kinds of data, it is possible to make rating much more reliable. The use of reliable expertise data will mean that the resources being shared by an expert reflects to his overall rating and level of expertise. A rating mechanism was introduced for gathering the expert's rating and level through its number of shared resources and its overall ratings on various aspects.

Once a content resource is displayed in the webpage, the rating of the content is presented together with the level of expertise of a user who contributed that content. Additional metadata and tags were also presented to the user so that he will have a simple idea on what the document is all about before opening the whole document.

6.2. Social Media Evaluation Flow

In Figure 5, the sequence diagram illustrates the scenario in publishing contents, submission of comments, and providing answers to questions.

Figure 5. Sequence Diagram for the Evaluation of Social Media

The browser plays a major part of the framework; it serves as the interface between the users, other domains and the local database. An expert or a contributor has the privilege to submit contents while a novice can only ask questions. The database stores all the generated and referred contents. It also stores all the questions and answers for easy retrieval. Once a user (novice, expert or contributor) searches for a particular content, the results are displayed with their corresponding ratings.

7. Implementation

The proof of the framework is demonstrated in a webpage which contains a collection of digital resources. The general use of the collection is for agricultural experts, agriculture extension workers and farmers in their fast and reliable search of web documents related to Agriculture. The webpage connects to other websites for other data; it empowers agricultural experts and farmer groups by creating access to knowledge banks and an open environment that links various stakeholders. Researches to improve the lot of millions of small farmers and local communities and to advance and further develop agricultural practices and production are undoubtedly much needed. [16] Contents from research institutes, universities, government agencies and non-government agencies that are involved in agriculture are being aggregated to our website. There is also an authorization mechanism for security reasons.

WordPress [17] is an open source software which was used in the system as a Content Management System (CMS) [18] to create our website. It has a great feature of flexibility and has the largest community of free themes, plug-ins and support. Users can upload and update their own content easily and the site can be seen by anyone on the web.

The website has the ability to aggregate digital documents from different sources and displays it in the webpage. This helps in increasing the value of the website. Users don't need to go to a lot of links just to browse or search for a particular content as long as the content being search is linked to our webpage.

The ratings of the contents and experts plays a vital role in the website. Contents, comments and answers are rated by users in the website. In return, the ratings are calculated in real time, so that when a particular content is displayed to a user the overall rating of a particular content is already displayed in the webpage. Also, the rating of the expert who posts that content appears together with the content he contributed to our site. This makes the user to efficiently discover reliable contents in our site. Basically, contents with higher ratings are more reliable than contents with lower ratings, and this also applies to the ratings of our experts.

8. Conclusion & Recommendation

A great application of social media will be an environment where interaction between experts and users of information resources is present. Users can rate posted resources, give their comments on a resource, and ask questions to experts. User's rating as well as level of expertise will be reflected based on his posts, comments and answers to questions.

Improved communication and information access are directly related to social and economic development. The framework provides a helpful contribution to the community for efficient discovery of reliable agricultural documents in the Internet. Users of the site will no longer go through a lot of link in searching for reliable resources they need. It also offers users with new and efficient ways to create, share and organize content on the web.

Among other frameworks which are using ratings based on arithmetical mean, it is concluded that the framework for evaluation of social media is more efficient. The rating of contents and experts can be very useful in locating high quality content. It is also believe that the framework will be very important in the context of emerging technologies in the web.

The framework may be explored to other applications to provide a web community where users can interact with one another, exchanging knowledge and opinions, identifying the authoritative sources of reliable information.