Critical Evaluation On Heuristic Evaluation And Usability Testing Computer Science Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

The selected website is The carphone warehouse it is the worlds largest independent telecommunications retailer.100 millions of customers have already chose the Phone house, nnow they have 2,300 stores in 10 countries , which comprises 50% of best buy Europe group and 47.5 % in virgin mobile France. This company is evolving the existing retial proposition to provide broader range of products and services with "Connected World".This is an advanced stage for carphone for preparing launch of new "Best -Buy"-branded "Big Box" and online proposition, which offers consumers elctronics products and services .The carphone warehouse holds minority investments in three digital radio multiplexes and two wireless technology business.

The Carphone Warehouse a company with investors, customers which comprises a 50% interest in Best-buy Europe group and 47.5% percent interest in virgin mobile France.The phone house involved in the management of the business through regular participation in operational and strategic meeting as well as board representation. It operates 2,453 stores in nine Europeans countries, principally under carphone warehouse business is evolving its existing retail preposition to provide a broad range of products and serviced. Carphone property currently comprises the property with aggregate values of £66 million. The revenue is upto 3.7% in the first quarter deriving largely sales of smart phones. Gross margins were robust and target of 100 Wireless World Stores by March 2011.

2. Understanding the Context of Use:

2.1. Context of Use Table:

The below the context of use table is presented with site name, URL name, Users, Business Goals and Users tasks

This page contains context of use table

3. Usability Evaluation:

3.1. Critical Evaluation on Heuristic evaluation and Usability testing:

A core activity in Human Computer Interaction(HCI) studies for past 15 years to develop effective usability inspection techniques, where this between aim to uncover potential usability problem by evaluating inpection with user interface with a set of guidelines (Neilsen and Mack, 1994; Cockton, 2003).

The Heuristic evaluation and usability testing are evaluated with reference to Wixon's (2003) threefold criterion, where the usability evaluation methods can consider these three goals termed by Wixon i, e

Finding problems, understanding the problems and fix the problems.

The main aim before evaluation is to understand the context of use, where the company Goals, Rationale and Users Task.

Usability Evaluation came to focus on quality in use where interaction with systems would be observed and described (Gilbert Cockton, 2004).

If the person or group of person could not come to the consensus what was the problem was, how to explain it, or what the fix should be, then that particular problem in the system for the next test, where it understand the problen and rapidly we acted on immediately( Dennis Wixon, 2003).

Usability evaluation methods are presented to two methods they are

- Heuristic Evaluation

- Usability Testing

Heuristic Evaluation method is an analytical method developed by Nielsen (1994), where in this method experts guided a set of usability principles called Heuristics and a set of 10 heuristics was derived from analysis of 249 problems. This evaluation is used to focus user testing. This method is also said to be Discount Usability method. Usability testing is based on empirical data in which representative performing realistic tasks and data is collected by experiments, laboratory usability tests and field usabality studies.

The main goal of heuristic evaluation is to find major usability problems in system with using large resources. This technique is discount approach to usability as it uses simple and approximate methods. The nielson has oberved that this technique may produce discount results, as usability inspects it is not necessary to produce data of same precision or reliable as usability testing, therefore nielson and other partitions has observed and noted that using heuristic evaluation as supplement to usability testing( Michael.J and Muller, 2008). Other has described usability methods in terms of Object model, Process model, participation model and positive on product life cycle.

The Nielsen suggested 10 heuristics, which are applied by expert and non expert evaluators to a given interface in order to help improve usability. Where some of heuristics are for experiments or added additional heuristics to reflect the characterstics of the evaluated system to improve the performance of heuristic evaluation, therefore the different set of heuristic may results in difference performance of heuristic evaluation.

Cognitive walkthrough method developed by Wharton et al(1994), it is for evaluating designs for ease of learning and its analysis, where here we required to the main concept of usability to evaluate and improve the usability problems. If we move towards the next evaluation method is Usability testing, which is the Empirical method of testing

The next method of evaluation is Usability Testing, which is based on collection of real user data by three methods. Experiments, Laboratory Usability Tests, Field Usability studies here we are considering the wixon's theory with reference to this method to which extent these method are used for usability evaluation.

There are few basic principles to be considered based on criteria they

First and foremost, make your application easy to use.

Make your application fast.

Listen to your users

The best way to get usability feedback is to test the application with a QA group that knows nothing about it (Vikram V.Ingleshwar, 2007).

In Heuristic evaluation and Usability Evaluation, in these evaluation methods they have there strength and weakness with reference to there evaluation criteria. Now we compare the evaluation methods between heuristic evaluation and usability testing. As in heuristic evaluation experts guided a set of usability principles called Heuristic Evaluation. As it is names Discount method, it is very important to know the evaluation criteria in which aspect the evaluation is considered as in Nielsen heuristics. Visiblity of system status, User control freedom, Consistency and standards, Error prevention, Recognition rather than recall, flexibility and efficiency of use, Help users recognise, diagnose and recover from errors and help and documentation. These are some important heuristics from Nielsen 10 heuristics, which are in reference to wixon's threefold criterion, the visiblity of system status is important as in usability testing to define the goals that inform the test with identifying the users as participants.

During evaluation user control freedom should be within the bounds of that particular evaluator , we have to recruit the participants for evaluation where every aspects for freedom and user control is managed in advanced before the participants allow for testing.Consistency and standards are important in which the evaluator able to know the title, site, the links to it in usability test a particular task is set for the participants, in heuristic evaluation the focus is on user , in usability testing the focus is on areas of concern from inspections and user is recruited to inspect the usability problems where there is loss of time and money. In usability testing recruiting users is good advantage, where the error is noted at same time the solution for the particular error is decided.

The other heuristics were introduced by Shneiderman's Heuristics and Norman's heuristics for usability evaluation were the strive of consistency is common in all the three types of heuristics , where it is used for sequence of actions in similar circumstances, identical terminology should be used for promts, menus, help screens are considered. According to Nielson's advice

Evaluator should evaluate the interface several times

Firstly for to get systems scope and interaction.

Evaluator to focus on specific interface elements

Evaluator should compare the dialogue elements and features of 10 heuristics

Evaluator should also consider general usability principle

According to Nielsen's some heuristics are too general and not relevant to new products and the web, but these heuristics are used to evaluate the site ,so in some manner in any aspects in some way these heuristics are useful in every manner, but i conclude with experts there is a problem as new products are coming there are more methods to be considered to evaluate the new product.The heuristic evaluation with single user achives poor results , only 35% of usability problems are overcome, 5 evaluators find approximately 75% of usability problems.In usability testing the evaluator is itself is suppose to evaluate it with simplifies experiment said to be Quasi experiment and which focus on particulat areas to evaluate, but the evaluation is out of context.

In Usability testing the task is represented in realtime and undertake in laboratory and finding some reliable issues, conducted in Lab, Field, Remote testing and by Web Logging.The different evaluators have there own choice of testing on differnt areas like to evaluate product goals, Context of use and other applications. In usability evaluation as per wixon to find problems, understand problems and fix the problem. It is depend on the area of concern for inspection and to identify the problem.Once the user has found the problem, the next step is to understand the problem and fix it.

The strength and weakness of heuristic evaluation and usability testing are framed as per there methods of evaluation. As per wixon theory mentioned before to find, understand and fix the problem, these evaluation methods are depends on there evaluating. In finding the problem the heuristic evaluation using a single user only 30% of usability problem are observed, where the evaluation is in group the evalautor find the problem and rate it in the frequency, impact and persistence. In these specifications it is easy to evaluator to find the problems, it impacts and persistence and the evaluator can plan the method to fix the problem by allocating the resources. where it is easy to use, and it operation is fast because the evaluation can be undertaken in one day for analysis where the usability test take weeks to analysis. The heuristic evaluation is easy to use and it is cheap as no experiment or laboratory equipments are needed. As mentioned before the heuristic evaluation method is an discount usability method. In this method the evaluation is in context of use where the goals, tasks are understand.If we apply the wixon threefold criteria this method is good preferred for evaluation.

In usability testing, every task is undertaken in controlled environment and this method is evaluated using the simplified experiment a quasi experiment, it focus is on area of concern, if we consider the principles of usability evaluation , the most important point is the context of use, if the context of use is understand by user it is easy to evaluate. In wixon theory it is important to find , understand and fix the problem, without knowing the context of use it problem to find further usability problems, this method is out of context and the focus and motivation of use rmay be unnatural and there are relaibility issues in this method.

If we compare past findings for the good method, there is heuristic evaluation differentiated with Metaphors of Human Thinking (MOT), for inspection and design. Where the heuristic evaluation method compared with MOT, where MOT problems are more serious, more complex to repair, more likely to persist for expert and MOT have more difficult for Novice evaluators than heuristic evaluation. The MOT method also compared with cognitive walkthrough method, where there are more problems in MOT than in Cognitive Walkthrough method. The problems consolidated across similarity of types like user action framework, Problem persistence (Erik, F and Kasper .H, 2008).

According to Wixon the good outcome of any usability method should not only by knowing the number of problems found, but also by the people that are fixed. Which are motivated through RITE (Rapid Iterative Testing Evaluation) method, where the problems are fixed quickly because the entire team involves in analysis and interpretation? Using this method team gets the chance to verify, whether the fix worked and to iterate on it if it did not fix it. It improves user's performance very quickly and ironically that allows for the discovery of more problems, since user go further into the systems. During the test practically the evaluator can stop at any time with a high level of certainity, where the system has improved or we can continue till all the golas has met (Wixon, 2003), Which is for answering three questions in evaluating, what problem we see, can we explain them and can we fix it (Sharon Mc Donald, K. Monahana and Gilbert Cockton).


In usability evaluation methods, the heuristic and usability testing both are for the evaluation but before evaluation the main aim of evaluatortor is to describe and know the context of use, where the goals, users tasks are known.These are the principles of usability evaluation methods.In both evaluation methods critically the heuristic evaluation is considered to be a good method for evaluating, knowing the wixon theory to find, understand and fix the problem. In heuristic evaluation there is no restriction to work in controlled environment, where the usability test to be conducted in controlled environment but again the usability principles should be considered for both methods before evaluation. In usability testing the evaluation should not be used to address global concerns with a particular system, where in heuristic evaluation we can evalaute the site using the users we need mininum five and maximum user are more to find 85% usability problems. We can consider the usability testing a good evaluation to some extent, because it is conducted in laboratory, but again it cost more and user's motivation may be unnatural. According to Wixon's threefold criteria Heuristic evaluation and usability testing can find, understand and fix the problems, but using heuristic evaluation is convenient than usability testing method.

Word Count: 1,998 words

3.2. Usability Test Plan Detailing:

In usability test plan detail includes the below details for test plan

A) Identifying test user groups and appropriate numbers

B) Details tasks

C) Detail test procedure

D) Justifying Usability measures

Before detailing the test plan it's very important to know the Test Goals.

Test Goals:

The goal of the test is to know the websites usability, its navigation, it speed, the quality of website, its quick response to users.

A) Identifying test user groups and appropriate numbers:

The user groups are decided on the test goals of the site, and to decide who the participant should be

For this test iam using my 5 people group to test planning who visited the site Carphone warehouse,

2 sub groups

3 users in one sub group

2 users in one sub group

Here I have used 2 in 1 and 3 in 1 subgroup to know how the users usability problems are find by 3 users group and 2 users group,and the usability plans can me lined on based on user, but again this is depends upon users characterstics.

User's characterstics:

In user characterstics which includes

Work experience

General and specific computer experience

Product experience

Age and Gender

The above characterstics are set for the 5 people group for evaluating the site is presented

Work experience:

Users are working in an IT related company, have 7 months of experience in computer field

Product experience:

They are working in IT Company; they are experience to some extent, to identify the product it details.

Age and Gender:

The user group's age is between 24-27 male

B) Detailed Tasks:

Tasks for users:

i) The user to know whether the site is in context of use or out of context of use.

ii) To create an account on site.

iii) The users to visit the site and to find the usability problems.

iv) The user to interaction with site and its speed of browsing.

v) The user is supposed to check the navigation process on the site.

vi) The user wants to buy the product on the site (the response from the system is good or creating any problems for user to buy the product)

C) Detail test procedure:

The users are asked to know the above tasks on site


My friend is thinking to buy a Blackberry mobile phone, which is free from £20 per month. Here I explained my friend to browse the site and know more details at time of browsing; I will know the good and bad feature on site with below.I explained my friend the task to be done


The user are asked to know the site is in context of use

To create an account on site

The user is asked to know the product as designed and specified on site.

The user asked to know the navigation process on site

The speed of the site appearing and other pages on site while browsing the site with different icons

To collect data where there is usability problem by screen capturing, web logging and by observation.

It is the worked based task where user are suppose to know the site with the above tasks and other alternative tasks, which can be mentioned during the test.

Conducting the test:

The test will be conducted with user at particular time period for each subgroup on using the site to test.

Performance measures:

In performance measures the users are suppose to observe the below measures

Every action on site visited by user on product, on navigation and the speed of the web page appearing by clicking on any options on site or product.

Subjective Measures:

It perception to understand the data on site.

The user's opinion on the site.

Justifying good and bad features on site.

Likert scales:

Likert scale is good to measure the negative and positive usability on site it reflects the interaction, where user agree or disagree.

Verbal data:

The verbal data is to collect the information on site, making the decision on the site.There good and bad features and usability problems. Some time the data may be unnatural, depends on user characterstics.

D) Justifying Usability measures:

The usability measures depend on the performance and subjective measures, where we can know the speed, errors and every action or moment of site during the evaluation.The usability measures are always to be considered in any of the evaluation methods.

In usability measures with concern there are some organising problems like frequency, Impact and its persistence.

In accordance usability measures are must to find,understand and fix the problem as mentioned by wixon, there must be the finding of usability problem and certains measures must be undertaken to overcome these problems to be fix.

3.3 Pilot test:

Pilot test is runs to reveal the problems with the system, problems with user incompleting tasks, problems with the product.

The pilot test is run wit the representative participant.


The goal of the test is to know the websites usability, its navigation, it speed, the quality of website, its quick response to users

Pilot test begins:


List of problems:

1) Context of Use:

The context of use is good at every level but some options like my account there is problem in creating ir, home page icon is not designed on site, it use is good and icon there veiws are good.

2) My account:

When the user was creating the account on the site, when all the details filled but it has not accepted the details and was asking for account number (this account number will be on phone bill),

Here the customer is buying the product first time, here it to be noted that customer is buying the prodcut for fist time, how it is possible that the user have a phone bill. This option is completely problem for user to create the account.

Date: 27/08/10 time 17:30pm

Fixing the problem:

The customer should buy the product by going to store once to buy a product, where he get the receipt for the phone and the account number .Which is mentioned on the reciept , the user should enter the account number then the account is created on the site.

3) Home Page:

when the user is registering the account and want to back to home page, there is only option of Home in my account page, in other icons like Accessories, Laptops, Broadbands, Gaming, News & sport and help and support, when the user click on this option and wants to go back to the home page there is no option of home page,

Fixing the problem:

It means the site doesn't has the home page for the site, every site must have the home page which helps the users to know the site, what is the site useful, it goals

3) Buying of product:

When users was buying the product there are 2 options pay as you go and pay monthly, where for pay as you go the user can buy the product online but for pay monthly the user should visit the nearest carphone warehouse store to buy the product.

Fixing the problem:

The site should have the feature where the user can buy the pay as you go and pay monthly products, where the pay monthly amoount can be dedcuted from his bank account with reference to direct debit.

4) Naviagtion of Pages:

List of good feature:

4. Conclusion: