The Testing Web Application From Usability Perspective Information Technology Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Increasing number of personal computers makes everyone an internet user. The interaction of customers with internet through concepts like e-commerce and e-banking, forced the web application developers to improve the quality of the application to achieve business goals. Due to the versatile, dynamic and complex nature of web applications, testing of web applications becomes a very challenging task [5]. Due to a time limitation in web application development, the developers will have to prioritize the testing process by keeping in mind the factors like complexity of the problem and the objectives to achieve from the application. All errors during testing come under certain categories like Functional, Usability, Interface, Compatibility, performance and security. Available testing techniques for other traditional software applications cannot be used fully with the web applications due to some special and unique features of web applications. These features include diverse users with different proficiency levels and backgrounds, Execution flow of the application, Multi facet environment, compatibility and interoperability issues, content diversity and faster maintenance [1]. Easy to navigate and fully functional web site grabs attention of a uses even if it is not attractive in terms of design and organization. Issues like dead-ends, missing links, grammatical and spelling mistakes, poor navigation, complex interface, browser incompatibilities, graphics loading time and lacking of search capabilities are the areas to be focused for web testing, its success and acceptance to users. Moreover the additional challenges that include Immediacy of web applications, enforcement and implementation of vital quality attributes like usability and reliability [3], lack of documentation, missing testing objectives and cost make this task more cumbersome and gigantic.

Software testing is normally an ad hoc activity usually done at the end of the SDLC in most of the organizations rather it should be a parallel process with each of the development phase to cut the cost of the overall software at the end. Another ambiguous and opaque concept is Quality of software applications which according to [12] six fundamental categories of software quality: functionality, reliability, usability, efficiency, maintainability and portability are defined.

Immense user involvement is making Usability one of the most important critical success factors of any web application. The term Usability was originally derived from the term "user friendly" [5]. It is used in many different contexts such as execution time, performance, navigational ease, user satisfaction and ease of learning, taken together [12]. Web usability defines by ISO 9241, the satisfied completion of work with effectiveness and efficiency by any user.

Different methods can be used for inspecting the usability of an application (Nielsen & Mack, 1994) but no consensus has been build in software development arenas about which usability evaluation method is most efficient and cost effective. Due to increasing cost and time constraints the extensive usability evaluation is not feasible so the improved usability achievement up to a sufficient level is the core area under consideration. In this review, the main focus would be on transformation and gradual evolution of web Usability, Usability testing and its Methods used in an industry. In next section we compare the different usability inspection and testing methods and their effectiveness. Finally we present some conclusion followed by future work.

Literature Review

The large number of research scholars especially from software engineering (SE) community and human computer interaction (HCI) community has taken usability in to focus from the last so many years [2]. Simply usability is the art of learning and using any product or software with great ease without finding any difficulty [4]. Usability makes the user proficient to perform their tasks [4].

Jacob Nielsen, a Usability guru defines the usability according to the five components: Learnability, Efficiency, Memerobility, Few Errors, and Users' Satisfaction [5].

A number of benefits are associated with the successful implementation of Usability the organization who intended to do so. According to a survey conducted by a company Xerox [4], usability provides important benefits in terms of quality, product, and cost and customer satisfaction. Some of the major benefits of usability are following [4].

1) Improve productivity through fewer code revisions and more efficient design.

2) Earlier Identification of design problems in the development process, saving both time and money.

3) Training requirements reduction and user productivity improvement.

4) More satisfied customer indicates the success of the organization.

Creating a usable website is not an easy task. It involves detailed processes as described by T.Brinck, D. Gergle, and S. D. Wood [6]. They focused that the usability must be the part of every stage of web site design process. According to [6], usable website is a one that allows a user to perform the tasks quickly and efficiently. According to them characteristics of a usable website include Functionally correct, Efficient, Ease of learning, Memerobility, Error free, Subjectively pleasing [6].

Another approach supported by H. Shahizan and Li Feng [8] for the evaluation of the web usability is benchmarking. It's a technique just compare the web sites with their competitors. This comparison may be done on the basis of seven narrated factors: screen appearance, consistency, accessibility, and navigation, media use, interactivity and content. The organization is at liberty to choose all seven or less factors for comparison. Their [8] purpose of research was only to test applicability of framework. Their [8] benchmarking approach consists of eight phases which is more comprehensive.

Banati, Bedi and Grover [7] talk about web usability in terms of the user's perspective. They have elaborated a concept through pyramid. In the usability pyramid, moving from one stag to another is a sequential activity. Means one cannot move to the next stag while achieving previous one. According to them [7], it's impossible to achieve the usability in a single step. A repeated and continuous iteration is requires to evolve a usable web site. According to them [7], human aspect or user perspectives has the paramount importance while achieving the usability in true letter a spirit as user is the main actor in usability and his view point, behavior and attitude is normally a reflection of a web site. So User participation and the perception of his thoughts must be the part of effective implementation of usability. They describe an approach named Usability pyramid comprises of four pillars which depicts the user's attitude towards the web site. In [7] they, propose the list of criteria like appearance of the site, work satisfaction, Emotional satisfaction, state of features and Trustworthiness of site to measure usability besides Efficiency, Effectiveness, Learnability and Memerobility.

K.Guenther [9] suggested that early involvement of user will definitely affect the results of usability. Besides the knowledge of significance of usability aspects, there are very few organizations which take usability seriously. He also writes that, sophistication and to be more expensive is not the criteria for the successful implementation of usability.

Three famous approaches mentioned by Marsico and Levialdi [9] used for evaluating web usability. They are questionnaire, behavior assessment techniques and automatic tools examine.

The HCI community researchers [10] have done initial investigation into the website needs of Usability and they have presented useful guidelines for designers and developers [10].

A. Types of web sites and Usability [11]

Web sites can be broadly categorized into three categories, on the basis of functionality and they are:

1) Information oriented websites

2) Service Oriented websites

3) Business oriented websites

Due to the different categories, their design rules and design specifications are also different. So the generic evolutionary methods for usability for different type of web site are still missing. A future research may be done on making the generic usability approach for all web sites irrespective of categories mentioned above.

B. Usability Evaluation Methods

Usability evaluation methods (UEMs) are not standardized by classification. On the basis of different usability expert's opinions, the following section will briefly narrate the Usability evaluation methods and their classification.

1) Nielsen and Molich Usability Interpretation: Nielsen and Molich [12] divided usability evaluation into four categories: Formal, Automatic, Empirical, and inspections. Formal methods are tedious to apply so they are not very much in practice by the developers [12]. Automatic evaluations, on the other hand they work well only to very primitive checks [12]. Empirical testing and usability inspection fare widely used for usability evaluation in product development.

2) Andy Whitefield's Usability Interpretation: Andy Whitefield [5] presented a model which divides usability evaluation methods into four classes. They are Analytical Methods, User Report, Specialist Report and Observational Methods

3) Adelman and Riedel Usability Interpretation: Three types of Usability Evaluation methods identified and presented by Adelman and Riedel [33].

a) Heuristic (based on expert opinion)

b) Subjective (bases on User opinion)

c) Empirical (bases on user actions)

4) Wixon and Wilson Usability Interpretation: The Five classifications on the basis of methods by Wixon and Wilson studied usability methods in general. They are summarized below [12]:

a) Users are involved vs. users are not involved: Usability methods behaved differently when users are involved in evaluation, analysis and design.

b) Formalized methods vs. informal method: Many methods have been described formally, but the evaluators adapt the informal methods to their requirements.

c) Formative vs. summative methods: Formative methods are described ideal for generation of new ideas while summative methods are normally used to evaluate existing systems.

d) Complete methods vs. component methods: Some methods adopt all full tour needed to complete the usability design effort for example usability engineering as a whole is a complete method. Most methods adopt the component based approach and used to represent only a portion of full usability process

e) Discovery methods vs. decision methods: Discovery methods are sometimes also called qualitative methods. They are user-centric methods used to discover work habits, behavior or thinking patterns of a user. Decision methods are most suitable when take decisions amongst the different available options or alternatives. These methods are sometimes called quantitative methods.

C. Product development Perspective

Three phases of product development are:

1) Product before development

2) Product during development

3) Product after development

Keeping in view the product development, usability evaluation methods can be categorized into two main types [2]. They are

1) Summative Evaluation Methods

2) Formative Evaluation Methods

Summative usability evaluation methods (SEMs) are used before and after product development. A quality of the finished interface is judged by these methods. The main focus of the SEMs is Comparison of alternative designs and testing of definite performance requirements. Formative evaluation methods (FEMs) are used during product development. These methods help in improving interface design. Qualitative observations of what happened and error detection is the main focus of FEMs.

D. Usability Inspection Methods

Usability inspection and usability testing methods, discussed by many researchers. In this section we will briefly discussed the literature reviewed about these two techniques.

By definition, the Usability Inspection methods are those in which the evaluation is done and by the expert evaluators to examine the user interfaces to find out the different anomalies and problems [2]. These methods, in which experts are involved, are a cost effective and cheaper against those methods where the usability is done with the help of uses due to resources and cost involvement.

T.Hollingsed and D.Novick [2] throws light on the experience and practices of four important usability inspection methods. According to them [2], Heuristic evaluation and the cognitive walkthrough appear to be the most actively used and researched techniques. The pluralistic walkthrough remains a recognized technique but it is not the subject of significant further study. Formal usability inspections appear to have been incorporated into other techniques or largely abandoned in practice.

According to J. McKirdy [15], one of the main problems in software development practice is that both the development and evaluation of user interfaces (UI) are most often done by developers, who are in general not dedicated usability experts. M. Schmettow [14] thinks that Pattern Based Usability Inspection Method is appropriate for developers.

Z. Zhang, V. Basili, and B.Shneiderman [16] considers current usability inspection techniques rather ineffective. They challenged Heuristic evaluation method (HE) which is considered by many researchers and practitioners' most effective usability evaluation method. They compared Perspective-based Usability Inspection method (PUIM) with HE and concluded PUIM better one.

Karat [12] has done a general comparison of usability testing and usability inspection method. He mentions trade offs regarding inspection methods. According to him usability inspection methods may be compared according to the following set of possible differences [12].

1) Method employs individuals or teams

2) Evaluator expertise

3) Prescribed tasks versus self guided exploration

4) Utility of guidelines

5) Data collection and analysis

6) Generation of recommendations

7) Role of debriefing session

A group of researchers [17] have developed MiLE (Milano-Lugano Evaluation method) for web usability evaluation. It is the blend of Heuristic Evaluation (HE) and task-driven techniques.

Another group of researchers [18] challenged the Heuristic Evaluation method. They valued SUE (Systematic Usability Evaluation) a novel usability inspection technique in comparison with HE [18].

T Hollingsed and David G. Novick [2] discussed Heuristic Evaluation (HE), Cognitive Walkthrough (CW), Pluralistic Usability Walkthrough and Formal Usability Inspections Methods. They are the methods which are practically used in the industry for usability inspection methods:

1) Heuristic Evaluation: A new method for evaluating the user interfaces developed in 1990 by Nelsen and Molich. In this technique some usability experts are hired for evaluating the user interfaces against the standard available checklist and highlight the problematic areas where they exist. Initially this technique was famous because of its cost effectiveness and ease and it was taken as the substitute of empirical user testing. After having lot of research materials of this approach, the effectiveness of this approach can be measured only through the levels of expertise as expressed by Nielsen. Also he has mentioned some disadvantages of this research in which first and the foremost thing is evaluators must be expert and secondly large number of experts are needed for evaluation (between 3 to 6) and thirdly it's costly. Besides all the disadvantages, still this heuristic evaluation is a quick and cheaper approach as compare to others and its continuous effectiveness existence is still an active research area for the researchers [2].

2) Cognitive Walkthrough: The cognitive walkthrough is one of the usability inspection methods which normally demand several expert evaluators [12]. A set of appropriate and cognitive tasks to be completed is compiled. The evaluators then walk through each task and listed down the noted problems in each task. This method normally applies on very early phases of developments so the evaluators often work with mockups of interfaces such as paper prototypes and role-playing. This process includes preparatory phase and an analysis phase. In first phase the evaluators determine interfaces, users, tasks and actions to be taken during the task. In second phase the experimenters set the goals, determine the available actions, choose most appropriate closer to the goal and then perform the action and monitor the feedback. The common problems in this method highlighted by the researchers [12] are involvement of lot of documentation and limited problem found because of the non availability of expert evaluators and non availability of guidelines to make actions. This method is in use for quite some time with its refinements and extensions, and it can be applied to by novice experimenters along with expert users and cognitive scientists but the only problem they defined is the choice of scenario where we can apply cognitive walkthrough.

3) Pluralistic Usability Walkthrough: Pluralistic usability walkthrough is a usability evaluation method that brings users and system designers together to evaluate new design ideas [10]. The benefits of this method are feedback from the user due to the direct involvement and rapid iteration of the design cycle without any delays due to the presence of user. The drawbacks mentioned by the developers of this method, the scope of this approach is narrow and limited to user representative's paths not the generic user path through the interface. The ratio of finding the problem through this approach is 30 % higher than heuristic approach. It is very effective method and use of frequency frequently can be judged that in the industry it is heavily used but not even mentioned in their reports as it became such a mandatory practice that it need not to be mentioned.

4) Feature Inspections: This method basically emphasizes on the importance of functionality to achieve usability. Tasks and application features are normally identified by this approach which will be further used to run the application [10]. It may be done under the supervision of the design experts. It involves very strictly defined steps to with elements of both heuristic evolution and simplified of cognitive walkthrough. This review is quicker, thorough and technical because of the group of experts are involved and goal of this method is to find problems in the interfaces and features effectively and with great efficiency. As this approach is now group into other methods since its inception and because of the non availability of literature on this subject is lacking so its effectiveness is still a question mark for the designers.

E. Usability Testing Methods

The five usability testing methods: Remote Usability Testing, Coaching Method, Co-discovery Learning, Performance Measurement and Think Aloud Protocol are used practically by the web industry:

1) Remote Usability Testing: The idea of conducting remote usability tests emerged ten years ago [19]. It is a relatively distinct method with in other usability testing methods because user is not physically present during testing. This method becomes an ultimate choice when users and usability experts are sitting far away from each other. Recently a group of researchers [19] have compared remote usability with conventional usability testing. They [19] concluded that remote usability testing has the potential to cross organizational and geographical boundaries and support new approaches to software development such as outsourcing, global and open source software development.

2) Coaching Method: In Coaching Method usability expert works as a coach. Unlike other usability methods which disallow questioning, users are encouraged to ask questions from usability expert in coaching method [10]. The usability expert responds with appropriate instruction. By hearing typical user questions, problems are identified and help documentation can be designed [10].

3) Co discovery method: This method involves two participants that working together and verbalize their thoughts while exploring a product's interface [5]. This method also discovers how particular tasks are done [5]. Pair of users helps each other through difficulties

4) Performance Measurement: In this method the quantitative data are obtained about the test participant's performance while performing task Quantitative data is very useful in doing comparative testing, or testing against predefined benchmarks [10].

5) Think Aloud Protocol: In Think Aloud Protocol method, users are asked to speak their thoughts as they perform a task [10]. By thinking aloud while attempting to complete the task, users can explain their method of attempting to complete the task. This will clarify any difficulties they encounter in the process.

F. Web Usability Testing - Care Methodology [13]

Time, cost and resources are the main considerations for any software development process and web site development has no exception. As Web usability is a basic ingredient of web quality and with the absence of these fundamental usability features, the web quality will always be a building without a base. Due to the increasing significance of this area, a new methodology named CARE (Cheap, Accurate, Reliable, and Efficient) was proposed by a group of researchers [13].

The authors discarded the Inspection method for testing because they think that it involved the experts other than users and hiring of experts is always a costly solution. By keeping in view the pros and cons of other methods like User Testing, Thinking aloud protocol, Question asking protocol, Co-discovery method and Questionnaire, they proposed the CARE method.

The CARE method is the combination of other techniques which helps the users for trying to eliminate the prevailing disadvantages like time taking factor and expensiveness of other techniques. As technique consists of very few requirement steps which are as follows:

1) Problem definition: Properly well defined objective of the test.

2) Preparation of Task list: Predefined relevant tasks description for the users to conduct during test

3) Selecting sample users: Identification of Target

audience and then selection of sample users

4) Schedule and conduct the test: scheduling of test must be flexible for users and they must be encouraged by the observers

5) Data collection: Remarks, observations and suggestions are useful for making the application more effective.

6) Result analysis: Proper result analysis based on the data collection always enable developers for taking right decisions.

The Authors suggested that the involvement of users thorough the developments life cycle will prevent the developer from future changes of usability testing but it will affect the cheapness and efficiency. Table1 and Table2 show relative efficiency and relative cheapness against above mentioned techniques.

Table 1: Usability Testing Methods Comparison [13]


Web Usability Testing Methods

Relative Efficiency

Relative Cheapness


Remote Usability Testing




Coaching Method




Co-discovery Learning




Performance Measurement




Think Aloud Protocol



Table 2: Usability Testing Inspection Methods Comparison [13]


Web Usability Inspection Methods

Relative Efficiency

Relative Cheapness


Heuristic Evaluation (HE)




Cognitive Walkthrough (CW)




Pluralistic Usability Walkthrough




Feature Inspection


YCritical Review

The importance of usability evaluation has dramatically increased due to extremely fast growth in Internet technology. The website design is directly related to the purpose of the website. Website with poor usability can easily destroy the purpose of website. Various usability evaluation methods have been proposed in literature to identify usability problems. There is no standard on classification of usability evaluation methods.

The framework or mechanism is still lacking as for as the selection of usability evaluation methods are concerned when we take in focus the software development phases. Unlike traditional software applications the web development is critical in terms of time and money. So it is very difficult for the web developers to choose the appropriate method as per their requirements which best suited them. So keeping in view the facts mentioned regarding Usability techniques, Table 3 and Table 4 depicts the advantages and disadvantages for Usability Inspection as well as Usability testing methods. This comparison will help the users to adopt the appropriate technique or combination of techniques to fulfill the ever changing needs of testing the web developments.




Heuristic Evaluation


No requirement for advanced planning

Can be used early in the development process

Several Evaluator experts are needed

Evaluator must be experts

Most of issues identified by HE are minor

Difficult to summarize the findings from

multiple evaluators as different evaluators

report problems differently and at different


Cognitive walkthrough

Does not require functioning model of the product

Rests on an acceptable cognitive model of user activity during the phase of exploratory learning

Does not provide guidelines about what makes

an action clearly available to a user

Does not tell what types of actions are

considered by a broad range of users

Pluralistic Usability Walkthrough

Faster resolving of the usability issues

Greater number of usability problems are identified at one time

More significant in identifying problem areas of websites

Gives information when no prototype or previous versions of interface exist

Scheduling of group can be a problem

The group can move only as quick as its

slowest member

Feature inspection

Performs product inspection and usability inspection at a time

More significant in identifying problem areas of websites

Gives information when no prototype or previous versions of interface exist

Can not measure usability directly

Can not provide rich data about user's

experience of data

Judgment and interpretation about the feature'

ease of use is an extra burden for the evaluator

Provide only broad overview of product

Table 3: Major Advantages and Disadvantages for Usability Testing Methods [2][7][10][12][13]




Remote Usability Testing

Comparatively cheaper

Comparatively wider reach

Can bring security and performance issues

Has limited visual feedback

Makes more difficult to build relation and trust

Can be difficult to use for the participant

Coaching Methods

Builds the user's involvement in the evaluation

Users learn more quickly

Relatively time consuming

The coach has to do dual job i.e. coaching and


Performance Measurement

Provides quantitative data

Due to quantitative data the prediction is comparatively easy as compare to other UEM's

Users are needed to act naturally in unnatural


Requires rigorous test designs and extensive

resource to conduct usability evaluation.

Interaction between user and tester is


Think Aloud Protocol

Leads to direct design solutions

Verbalization gives the answer of both what and why problem arises with interfaces

Participant has to perform two tasks. i.e. doing

test and verbalizing what they are doing

Verbalization can interrupt the task that is

being performed by user

Co Discovery Method

More structured and explorative technique

Good for learning aspect of usability

Careful candidate screening is required

Verbalization can distract exploration

Table 4: Major Advantages and Disadvantages for Usability Inspection Methods [2][7][10][12][13]

Certainly the mix and match strategy of various methods may be a beneficial choice as for the diversity and dynamics of the web applications are concerned instead of choosing the numerous techniques or methods for different phases of development.

Despite of recognizing the value of usability evaluation methods for web application development, the usability evaluation methods are not consider as an integral part of web development projects.

conclusion and future work

Web application testing in the perspective of Usability which is the most critical quality attributes is the primary concern in the software industry. Due to the increase of business through Internet, its importance has also been increased drastically. So by neglecting the usability no one can get success in the business. Several methods and techniques are reviewed in this paper which are accepted and widely used by the industry.

Furthermore, we are of the opinion that it is a less professional approach to find appropriate usability evaluation method for specific type of website without finding appropriate usability evaluating method for all general websites. The main aim of this research review is to find usability evaluation methods which are suitable for web applications keeping in view the considerations of project development phases. Future work will be done to find appropriate web usability evaluation methods which are not web applications specific rather they must be generic in nature for dynamic web applications characteristics.


I would like to acknowledge the SZABIST Islamabad campus and my supervisor for providing adequate resources and guidance to complete this Herculean task in-time. It would have been impossible to complete this effort without their continuous support. I would also like to thanks my family for sacrificing their time and allow me to finish this task.