Heuristic Evaluation Of The Usability Of Egovernment Information Technology Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.


Usability implies designing interactive systems where a user can achieve his/her goals (1) effectively, (2) efficiently and (3) satisfyingly [6,7]. In order to measure the usability success of any website, the three factors (effectively, efficiently and satisfyingly) need to be evaluated.

The importance of usability evaluation for government websites, in particular, has increased over the past few years. In fact, usability reflects one aspect of a website quality. It has been acknowledged that a major problem of the underuse of government websites can be attributed to its bad usability.

Recently, some studies were carried out to evaluate the usability of government websites. These evaluations varied between using heuristics/experts evaluations and conducting usability tests. Yet, heuristic/expert evaluation was the most preferable starting method for evaluation due to its low-cost compared to other usability evaluation techniques. Besides, heuristic evaluation might be better for evaluating websites within a short period of time and utilizing fewer resources to uncover key usability problems. The heuristics results can then be used to form the bases for further usability testing.

In this paper we will exploit the heuristic evaluation technique as a first step to explore the usability of Saudi Arabia governmental websites.

Background on e-Government in Saudi Arabia

Saudi Arabia is the largest Information and Communication Technology (ICT) market in the Middle East. According to SAGIA [1], the Saudi telecommunications and information technology industries represent over 55% and 51% of the total Middle East markets. As a result, the government has paid special attention to ICT, both in its periodically updated five-year development plans and long-term national development plans.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

ICEGOV2010, October 25-28, 2010, Beijing, China

Copyright 2010 ACM 978-1-4503-0058-2/10/10… $5.00

The Internet service became officially available in the Kingdom in 1997. Since then the number of Internet users in the Kingdom has risen exponentially, according to Internet World Stats [9]. In March, 2008, there were approximately 6.2 million users and a 22.0% penetration rate, and in September 2007, there were 218,200 broadband connections [2]. It is foreseen that Internet usage will keep growing rapidly as the infrastructure improves, access costs reduce, the population increases, and the 60% of the population comprising teenagers and young adults adapt to the new technologies [3].

Saudi Arabia's state of 'e-readiness' is evidenced in developments in e-government. A number of e-government projects have been implemented or are under development. Examples include the e-government portal, e-government network, public key infrastructure (PKI), national smart ID cards, e-Payment gateway (Sadad), Social Insurance System, and electronic information exchange. Also a set of government institutions and agencies have provided some of their services electronically this include: the Ministry of Interior (Fig. 1), Ministry of Foreign Affairs, Ministry of Hajj, Ministry of Labor, Finance and General Investments Commission.

In 2005, the Ministry of Communications and Information Technology embarked on the e-government program in conjunction with the Ministry of Finance and Communications and Information Technology Commission (CITC). One of the main goals was to develop a governmental portal for services and to work on the presence of the government infrastructure. This program was called 'Yesser'. The key objectives of this program were to raise the public sector's productivity and efficiency; to provide better and more easy-to-use services for individual and business customers; to increase the return on investment (ROI); and to provide the required information in a timely and highly accurate fashion [4].

Figure 1. The Arabic version of the Ministry of Interior website with eServices listed on the right and left columns (date 20/4/2010).

By 2009, there was 238 governmental websites registered with (.gov.sa) domain extension according to the Saudi Network Information Center (http://www.nic.net.sa). Only 137 government websites were listed on the official e-government portal (http://www.saudi.gov.sa). And around 400 e-services were provided by various governmental agencies, categorized as follows:

Economy and Business (110)

Training and Education (87)

Communication and IT (11(

Personal Documents (25)

Health and Environment (14)

Transport (6(

Tourism and Travel (37)

Utilities (39)

Insurance (4(

Labor and Employment (35)

Complaints (61)

Municipal Services (76)

Hajj and Umrah (10(

Source: saudi.gov.sa

Most Saudi Arabia government websites provide bilingual homepages (Arabic and English). Also it is worth mentioning that these services are in their initial stages and are still not working together collaboratively.


Heuristics Checklist

As mentioned previously there are various methods for evaluating website usability, among these methods is the use of heuristic evaluation. Heuristic evaluation is defined as "an approach to evaluate the design of a given website to define any usability problems that may exist" [11]. In order to evaluate the usability of a given website, one or more experts need to check the website using a predefined set of heuristic guidelines.

Heuristics guidelines are based either on research, theory, standards or practitioner experience [5]. One of the most widely used practitioner heuristics is the Neilson and Molich heuristic [6]. Their heuristics consist of ten items derived from problems found in usability studies of user interfaces. Another well-known standard heuristics is the ISO 9241-151 [7]. This standard provides guidance on the human-centered design of software web user interfaces with the aim of increasing usability.

In terms of heuristics designed for evaluating government websites per se, little has been found in the literature. In fact, most of the existing heuristics were built using the Neilson and Molich heuristics with some additions to reflect the needs of e-government websites. Among these developed heuristics are: Garcia et al. [9], Soufi and Maguire [14] and EIDAROOS et al. [11].

Our heuristic checklist is based on the ISO 9241-151 and Travis checklist [13] and on those tailored to evaluate e-government websites such as Garcia et al. [9]. We also considered the general design heuristics provided by Yesser [10].

The resultant heuristic checklist consists of fifty-seven questions covering six components as in Table 1.

Table 1. Listing of the number of examined heuristics in each principle



Number of heuristics

design and consistency

Visually appealing design with consistent look and feel



Proper navigation system that aids the user when browsing the website.


data entry forms

Clear and easy to fill forms.


search functionality

Effective search feature which enables users to locate the required information easily.


Content precision and information privacy

Content must be precise and recent with clear information privacy policy.


help and feedback

The website has an effective mechanism to help its users and provide them with the proper feedback.


Sampling and Website Collection

The evaluated government websites were collected from the Saudi national e-government portal (http://www.saudi.gov.sa) based on their high-profile, importance, functionality and delivery of key services to the Kingdom citizens. The websites were first visited to check which are functional and which are still under construction.

Overall, 14 government websites from different sectors were chosen for evaluation. The evaluation was conducted in May 2010 by two experts with experience in web design, and with knowledge of web usability and accessibility. The experts inspected each government websites independently then used the compiled heuristic checklist for evaluation.


In order to have a prefect usable website, all of the heuristics should be met for each proposed component. Figure 2 shows the percentage of heuristics that were met for each component.

Figure 2. The results of heuristic evaluation for 14 government websites in Saudi Arabia

As we can see from the results, the Saudi government websites did not fully met any of the six proposed components. However all components scored above 50%, which might indicate a potential tendency towards implementing best usability practices.

Further analyzing the results we can find that the design and consistency component achieved the highest score with 80%. This can be attributed to the fact that most evaluated websites are fulfilling the heuristics in the design and consistency component, namely: the appealing visual design and layout of the website, consistency of the website pages, the lack of horizontal scrolling, the URLs friendliness and the adequate color contrast.

On the other hand, the search functionality scored the lowest value with 57%. The reason behind this low score can be attributed to the lack of advanced search functionality in most inspected websites. Moreover, the search scope was limited to certain content of the website (there is no full-text search) and in some cases the search results were not sorted or ranked relevantly and no proper feedback on the volume and the relevance of the search result was provided.

As for the content precision and information privacy component, the score was also high with a value of 75%. What helped this to happen is that most government websites provided "About us", "Contact us" pages, an up-to-date content with suitable language for its audience and a clear privacy/policy statement. However, most websites failed short in providing further information about the content's author(s) or/and the content sources.

The navigation is another main component of any usable website. Given the fact that the navigation system in the evaluated websites scored 74% with an easily identified and consistent navigation system as well as having a sitemap, nevertheless, some Saudi government websites failed to provide distinguishable hyperlinks with external ones that open in a new window.

Furthermore, the majority of government agency websites provided forms to get data or input from the citizens. So, examining forms usability is very important to improve the form completion rates. While evaluating the forms used in the Saudi government websites we found that less care was given to create accessible and easy to use web forms. Among the problems we encountered, are the unclear distinctions between required and optional fields and the lack of hints on how to fill the form. Additionally, some forms did not make it easy for the user to correct errors as well as not giving the user an appropriate feedback.

As for the help and feedback component, most government websites did not provide a FAQ page or an online help.

Finally, it is worth mentioning two other pressing factors that affect the usability of any website, which are the number of deadlinks and the page loading time. Using software tools for measuring these two factors, we found that Saudi government websites have on average 5.5 deadlinks per homepage and an average loading time of 1.87 seconds. These results indicate that the Saudi government websites need to take proper care of their homepage loading time and check their websites frequently for deadlinks.


E-government usability issues have largely been overlooked in the literature. Moreover, there is an absence of published research regarding the use of different usability evaluation techniques in the e-government domain.

However, there are few studies that entail heuristic evaluation to assess the usability of e-government websites. Among these studies is the work of Garcia et al. [9]. Garcia et al. proposed an extension of Nielsen's heuristic evaluation method to evalute 127 Brazilian e-govrnment websites. The heuristics consisted of sixteen component, which are: Visibility of system status, Match between system and the real world, User control and freedom, Consistency and standards, Error prevention, Recognition rather than recall, Flexibility and efficiency of use, Aesthetics and minimalist design, Help users recognize, diagnose, and recover from errors, Help and documentation, Accessibility, Interoperability, Security and privacy, Information truth and precision, Service Agility and Transparency.

Similarly, Soufi and Maguire [14] used a three level model as a heuristic tool to evaluate a UK e-government website. The three levels were: Level 1 - Required functions to meet user goals. Level 2 - Suitable information architecture supporting navigation. Level 3 - Effective page layout, design and accessibility.

On the other hand, Rodríguez et al. [8] presented a measurement framework for evaluating the design, content and services offered to citizens by municipal websites of 31 countries spread over different continents. The proposed framework was comprised of 152 metrics that tackles design, operations and information credibility.

Finally, two studies have focused on the usability of Saudi government websites. The first one was conducted by EIDAROOS et al. [11] where they carried out a heuristic evaluation on two e-Government websites in Saudi Arabia. The other study was by BURAGGA [12] where he also evaluated two Saudi e-Government websites. In the first study they compiled a heuristic checklist based on what have been used in the literature. However in the second study, the author has actually evaluated the accessibility of the websites, in a claim that this will uncover usability problems. Yet, both studies lack the evaluation of large number of websites.


The government of Saudi Arabia is working very hard to provide outstanding electronic services infrastructure to its citizens. In this paper we evaluated the usability of Saudi government websites using heuristic evaluation technique. The evaluation results identified many usability problems and highlighted areas for improvements. Therefore, in order to improve the current usability of the Saudi government websites, further evaluation assessments need to be carried out using usability testing with real users to uncover key usability problems regarding the interaction and the user stratification of the government websites.