Evaluation Of Educational Computer Games Education Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Abstract-Computer games have been used for educational purposes nowadays. The implementations of educational computer games bring some challenges and issues that need to be highlighted. There are three challenges in evaluating educational computer games which are the evaluation criteria, the evaluators and the evaluation process. Evaluation criteria have been identified and it consists of five issues which are Interface, Educational, Content, Playability and Multimedia. The evaluators are real and surrogate users. The third challenge is regarding evaluation process and one of the suitable processes is by implementing formative evaluation in order to develop a comprehensive and practical educational computer games.

Keywords-component; educational computer games; challenge; issues; evaluation crtiteria

Introduction And Motivation For Research

Computer games have been popular nowadays and it has been proposed as a potential learning tool by both educational researchers [1, 2] and game developers [3, 4]. There are many arguments and debate regarding the implementation of computer games in classroom and among them are; a) computer games able to raise active engagement in learners [1] b) computer games able to encourage active participation of learners, learning by doing and more motivation [5, 6, 7, 8, 9] and c) accommodates multiple learning styles and skills [10, 11, 12, 13, 14, 15, 16].

Implementation of computer game in classroom highlighted the importance of design and integration of pedagogical elements in order to develop effective educational computer games and brought along the importance of evaluation of the developed educational games [3, 4, 7, 8, 9, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]. This is important to make sure that fun and serious elements can be integrates very well to achieve predefined goal.

Effective usability evaluation for computer games that being used in teaching and learning process or sometime known as Educational Computer Games (ECG) is necessary in order to ensure that games adequately meet the requirements of fun and educational elements. A range of approaches have been used in the evaluation of computer games that relate to educational purposes. Most of the evaluation focuses on summative evaluation process with the objective of accessing how well completed games meet the pre-defined goals regarding the implementation of the developed games in teaching and learning purposes [6, 9, 29, 30, 31, 32, 33, 34, 35, 36].

In spite of the variety evaluation of ECG, most of the evaluation focus on effectiveness of the ECG and self generated questionnaire were used in their evaluation process. There were very little study has being reported regarding the usability evaluation of ECG especially that focused on interface of ECG [37]. Failure of focus to design usable game interfaces can interfere the goal of creating a convincing experience for users and may have a negative effects on the overall quality and success of developed games [37]. Specific methods are needed to identify usability problems in both early design and in more complete prototype [38, 39, 40, 41].

In evaluating ECG, there are few aspects need to be focused in order to achieve a good ECG that combine elements of fun and serious. This is important to attract users in using it and able to enhance their knowledge in specific subject. Details evaluation of ECG that integrates all of the mentioned elements needs to be done at the early stage of development and should involve various users either real or surrogate. Combination of real and surrogate users able to detect as many as possible usability problem of the evaluate computer system [42, 43].

This paper is organized as followed: Section 1 provides the background of this study; Section 2 illustrates some frameworks and methods being used to evaluate educational computer game; Section 3 presents some ideas and elements of interface evaluation that an educational game should contain; Last section of this paper concludes and provides future directions of this project.

Background And Related Work

Educational Computer Game (ECG)

People play computer games for many reasons, because of the enjoyment and entertaining attributes [44], for leisure and educational purposes [45]. Adolences prefer playing computer games as a leisure time activity, to avoid stress, to challenge their skills and to enjoy the attractive characteristics of the games [46]. Computer games can be effective educational tool [47, 48] and they can provide enjoyment during learning process to learners [46], allow users to engage in education while they enjoying themselves [6, 23, 49].

The aim of educational games is to facilitate the player's experience [50, 51], desired objective [6], allow uses to engage in education while they are enjoying themselves [52, 53]. Educational games seem to put leraners in the role of decision maker [2, 44], received immediate feedback of their action and decisions, inviting exploration and experimentation [32].

Evaluation of Educational Computer Game

Educational computer games has been used in many areas such as in engineering [54, 55], medical [56, 57], literacy [58] and mathematics [59]. Once the ECG being developed, the evaluation process normally take part for a group of students who use the game in their learning process and the effectiveness of the game being evaluated [9, 10, 55, 58, 59, 60, 61, 62, 63, 64, 65].

Most of the past evaluation of educational computer games use questionnaire [9, 26, 54, 66, 67], interview, log-files and observation [50]. Other than questionnaire, there were few attempt in developing framework to evaluate educational games [68]. Figure 1 shows the developed framework by [68].

A Framework for Evaluating games-and simulation-based education (Source:[68])

The Challenge

The challenge of evaluating educational games is divided into three categories, the evaluation criteria, evaluators and the suitable evaluation process. Evaluation criteria discuss about the necessary elements that need to be evaluated and why it is important. Second category discuss about the suitable evaluators and why they are needed as an evaluators, how they can contribute towards the enhancement of develop ECG. The last category is about the suitable evaluation process, either summative or formative and why it is important.

Evaluation Criteria

There are many issues involves in evaluating ECG. In any system, interface plays an importance role in order to provide appealing and attractive interface for the users. It implies the same concept for ECG, interface is the main issue that needs to be taken into consideration in evaluating ECG. Issues of consistency, interactivity, navigation, screen design and pleasant to use are the important elements that need to be evaluated. Since ECG integrates fun and education, pedagogical issue is another important element that needs to be evaluated. Elements of pedagogical such as clear goal and objective, motivation, learners control, learning values, immediate feedback and challenge need to be taken seriously in evaluating ECG.

The third issue is multimedia such as the usage of all multimedia elements should be suitable according to it functions, combination of multimedia elements in each screen and suitability of multimedia elements are among the focus of evaluation. Other then interface, pedagogical and multimedia, content is another issue that need to be highlighted in evaluating ECG. The reliability of the content, clear and understandable structure and interesting and engaging materials are among the importance elements of content issue.

The fifth element is playability. Since it is a game, the playability plays important elements in order to provide game that is fun and enjoyable and at the same time able to attract users to play and enhance their knowledge. Elements in playability that need to be evaluated are challenge, strategy, pace in balance, players able to control and able to know the progress of the game. Figure 2 shows the five evaluation issues and criteria that need to be evaluated in order to develop a comprehensive and practical ECG.

Educational Computer Games Evaluation Issue


Evaluation of educational computer games is not limited to evaluation criteria only. Issues of who are the suitable evaluators and why they are importance need to be taken into consideration. In usability evaluations, there are two types of users normally involve in evaluation process which are real users and surrogate users. The importance of evaluators for ECG need to be chosen based on their expertise. After thorough review and discussion, the involvement of the two types of evaluators is importance in the study because of their knowledge. Normally, surrogate users can be either experts in HCI or other related domain based on definition by the researchers. In ECG, after consideration of the highlighted issues that involve in evaluating ECG, the need to various evaluators is necessary, surrogate and real users.

Surrogate users that are needed in evaluation divided into four categories - HCI, educational, content and game developers' experts. They are needed because it is difficult to find an expert evaluator that has knowledge in all the five issues highlighted previously. HCI experts need to evaluate interface and multimedia issues based on developed heuristics as a guideline. The second surrogate evaluators are educational expert that need to evaluate issues of pedagogical and multimedia. The third surrogate evaluators are content expert need to evaluate content and multimedia issues. The last surrogate users are games developers that need to evaluate issues of playability and multimedia. Figure 3 shows the surrogate and real evaluators.

Multimedia issue is consider as universal issues that can be evaluated by all of the surrogate users because it is believed that with the knowledge of computer and familiarization of the such applications, evaluators able to evaluate multimedia elements. Other than that, since the entire surrogate evaluators need to evaluate multimedia issue, it is believe that usability problem of multimedia issue can be detected very well.

Real Evaluators - Potentail users

Surrogate Evaluators - Game Developers

Surrogate Evaluators - Content expert

Surrogate Evaluators - Educational expert

Surrogate Evaluators - HCI expert

ECG Evaluators


Real evaluators are the real users that consider as the target group of the developed ECG. Their involvement in evaluation is important in order to detect usability problem from users' perspective. In order to guide real users in evaluation process, the heuristics that being developed for experts evaluators will be designed in form of questionnaire so that it is much easier for real users during the evaluation process.

Evaluation Process

Once the evaluation criteria are highlighted and the evaluators have been identified, the next step is to plan for the evaluation process. In term of evaluation process, the debate either to use formative or summative evaluation is depend on the technique of evaluation. Since the focus in on the development of comprehensive and practical ECG, formative evaluation process is suitable to be used in this project. In order to cater for the formative evaluation and after comprehensive review of issues in ECG, Playability Heuristics Evaluation for Educational Games (PHEG) technique has been developed derived from Heuristics Evaluation [69, 70] and Heuristics Evaluation Playability [38, 39].

PHEG will be used by surrogate users as a guideline during evaluation process. PHEG has been divided into five sub heuristics which are Interface, Pedagogical, Content, Multimedia and Playability. Sub heuristics is used in order to cater for different surrogate users based on their expertise. Table II shows sub heuristics for Interface of ECG that will be used by HCI expert. Table III shows sub heuristics for pedagogical that will be used by education expert. Education expert need to be involve because not all of the HCI expert or content expert have knowledge in educational theories.

Table IV shows sub heuristics for content. This sub heuristics will be used by content expert. Content expert will be the lecturers who teach the particular subject. Content expert is needed to identify the developed ECG is comprehensive enough for the specific subject and follow necessary syllabus.

PHEG - Interface


Educational Game Usability Heuristics - Interface


Uses aesthetic and minimalist design


Maximizes consistency and matches standards


The uses of space, color and text are according to the principles of screen design


The uses of text, color and font follow the principles of readability


Ensure appropriate use of standard and propriety control


The interactivity of the game is suitable to learners level


The integration of presentation means is well-coordinated


Quality of user interface is acceptable


Provide specific and self- identified key for specific task (exit, glossary, main, objective)


Overall interface of the game is appealing

PHEG - Pedagogical / Educational


Educational Game Usability Heuristics - Educational/Pedagogical


Clear goal and learning objectives


The activities are interesting and engaging


The design and the contents are reliable and proven.


Can be used as self- directed learning tools.


Support for self- learning skills.


Medium for learning by doing.


Considers the individual differences.


Performance should be an outcome-based.


Offers the ability to select the level of difficulty in games


Ability to work in their own pace

PHEG - Content


Educational Game Usability Heuristics - Content


Reliable content with correct flow.


Clear and understandable structure of contents.


Navigation of content is easy and accurate.


Supporting materials are sufficient and relevant (quiz).


Materials are interesting and engaging.


Players able to understand the learning goal.


The content is chunk based on topic and subtopic


Major and minor topic is differentiate clearly

Playability is another sub heuristics in PHEG and it will be used for game developers as a guideline in evaluation process. Table V shows the sub heuristics of playability.

PHEG - Playability


Educational Game Usability Heuristics - Playability


Challenge provided are up to the users standard/level


Users able to strategies


The pace of the game are in balance


Players able to control the game


Progress of the game can be seen at anytime


Players able to perform to their best ability


Challenge is adequate - not too easy and not too difficult

The last sub heuristics in PHEG is multimedia and it will be used for all of the surrogate evaluators in evaluation process. Table VI shows the sub heuristics of multimedia. Sub heuristics Multimedia is believe can be evaluated by all of the evaluators because it is not too technical and evaluators experience in dealing with other application will help to detect usability problem for ECG.

PHEG - Multimedia


Educational Game Usability Heuristics - Multimedia


Usage of multimedia elements are acceptable


Combination of multimedia elements are adequate


The presentation of multimedia elements are well manage


Suitability of multimedia elements for specific use


Not too many multimedia element in one screen


The use of multimedia elements support meaningfully the text provided.


The quality of multimedia elements (text, image, animation, video and sound) used is acceptable.


The uses of multimedia elements enhance the presentation of information.

Real users will act as real evaluators that will be using questionnaire that has been modified based on PHEG and known as PAEG with additional elements of usability which are effectiveness, efficiency and subjective satisfaction. Details of PAEG will be discuses in another paper. Real and surrogate evaluators will be using same ECG (prototype) for the evaluation process and it is hoping that combination of both evaluators able to detect as many as usability problem possible.

Result And Disucssion

Evaluation of Educational Computer Games is important in order to develop ECG that is comprehensive and practical to be used in teaching and learning process. This paper highlighted the challenges in evaluating ECG that comprise of three which are evaluation criteria, the evaluators and evaluation process. These three challenges have been highlighted and potential solutions were discussed. Evaluation criteria can resolve using five evaluation issues which were Interface, Educational, Content, Multimedia and Playability.

Evaluators' issues can be established by combining real and surrogate evaluators that able to cater all of the highlighted evaluation criteria. The third issue, evaluation process will be using formative evaluation with the help of PHEG technique for surrogate users and list of questionnaire (PAEG) for real users that will be discussed details in another paper.


Three challenges in evaluating ECG have been highlighted. Evaluation criteria, the evaluators and evaluation process are the three challenges that need to be overcome in the evaluation process of ECG. It is believe that combination of the three highlighted challenges able to detect usability problem of ECG at early stage of development process. Improvement of ECG based on the usability problem highlighted able to produce comprehensive and practical ECG that can be used to help learners in their learning process.


This reseacrh is conducted using e-Science Funs Grant (01-01-02-SF0089), Universiti Kebangsaan Malaysia.