The History Of The Curriculum Evaluation Education Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Training program is an educational activity that has a specific duration. It has the main goal to improve the knowledge and skills of the participants by giving a series of subject courses through a specific curriculum. Monahan (2012) states that training program may cover a wide range of subjects about the main subjects. He also states that a training program has a goal to improve knowledge and skills in the participants' expertise in order to keep them motivated and bring the improvement to their institution by implementing the new knowledge in their job (2012). Therefore, conducting a training program is important for an institution or department to improve its quality by updating its employees' knowledge and skills. Training program is an essential effort to increase the productivity, motivate and inspire the employees by giving them important knowledge and skills they need to perform their jobs in the most effective way (Anonymous, 1998 in McClelland, 2002). It is an also important way to cope with the rapid change of technology and the diversity of the work (Hamdani, 2012).

However, not all training programs can be considered as effective ones. A training program is considered effective if it can improve the participants' knowledge and skills after they follow the program. According to Black & Mendenhall (1990), a training program is effective when it is able to develop participants' important specific skills and enhance their job performance. Similar with Black and Mendenhall statement, Noe (1986) also states that an effective training enables the participants to improve their knowledge and have potential outcome (promotion, raises, etc.) after they follow the program. The effectiveness of a training program itself is influenced by many aspects which relate to each other. Its effectiveness depends on the work of the organization as a whole that contributes the supporting atmosphere for the program (Kontoghiorghes, 2001). The aspects in the organizations can be derived from curriculum, implemented training methods, skills or task characteristic trained, and the choice of training evaluation, which are related to the effectiveness of training program (Jr., Jr., Edens, & Bell, 2003).

To measure the effectiveness of a training program, conducting a curriculum evaluation can be one of the alternatives. Through a curriculum evaluation, the administrator of a program can decide whether they want to continue the program or even reject a program (Lewy, 1973). In other words, the administrator of the training program can use the curriculum evaluation results to decide whether a training program is effective or not. It is because the curriculum evaluation helps the administrator of the training program to identify the strength of the training program that should be maintained or even improved, and the weakness of the program that should be minimized (Curriculum Evaluation Manual, 2011). Thus, conducting a systematic curriculum evaluation is considered as a crucial step that can give a substantial influence in the overall effectiveness of the training program (Goldstein & Ford, 2002; McGehee & Thayer, 1961; Sleezer, 1993; Zemke, 1994 in Jr., Jr., Edens, & Bell, 2003).

Participating in an effective training program is important, especially for researchers who have to improve their knowledge and skills regularly. In Indonesia, this needs becomes an urgent need in order to develop the research and increase the number of researchers. According to Subejo (2010), the research development in Indonesia is left behind compared to other countries, which is reflected from the number of the researchers and the research journal or research patent they produced. In Indonesia, training programs for researchers are conducted exclusively by Indonesian Institute of Sciences (Lembaga Ilmu Pengetahuan Indonesia - LIPI) based on Presidential Decree Number 128 year 1967. To fulfill the goal of conducting effective training programs for researchers, LIPI classifies these training programs into: training program for first-degree researchers (Diklat Fungsional Peneliti Tingkat Pertama - DFP Tk. Pertama) and technical training programs (Diklat Teknis dan Kedinasan) based on the Chairman of LIPI Decree No. 3212/M/2004. DFP Tk. Pertama is the main training program that is conducted by LIPI, with the consideration that this training program is very important to provide a proper basic knowledge and skills for the researchers to develop the research in Indonesia.

1.2. Target subject of the study

The target of this study is the first-degree researchers (Peneliti Tk. Pertama) who follow the DFP Tk. Pertama. It is because they will directly receive the effect of the training program. Therefore, their opinion toward the training program about the effectiveness of the curriculum, is important to improve the effectiveness of the DFP Tk. Pertama.

1.3. Objective of the study

This study aims to investigate the effectiveness of the DFP Tk. Pertama. This study will use the curriculum evaluation to evaluate the effectiveness of the training program. Therefore, the research questions of this study are defined as follows:

How far does the implementation of the existing curriculum in DFP Tk. Pertama influence the effectiveness of the training program?

Does the training program effectively contribute in improving the first degree researchers' knowledge and skills?

1.4. Context of the study

Indonesian Institute of Sciences (Lembaga Ilmu Pengetahuan Indonesia - LIPI) was established for the first time on August 1967 based on the Presidential Decree Number 128 year 1967, and then based on the MPRS (Majelis Permusyawaratan Rakyat Sementara - Temporary People's Consultative Assembly) Decree Number 18/B/1967. Based on those decrees, LIPI has a main duty to carry out governmental duties on scientific research in accordance with the provisions of the legislation in force. Including in this duty is the responsibility to act as the foster for local researchers and to collaborate with the foreign researchers to conduct research, inside and outside Indonesia ( (January 3, 2012)).

Based on the Chairman of LIPI Decree No. 2798/A/2001, LIPI has 47 work units which are divided into five main deputies: Deputy of Earth Sciences (4 Research Centers (RC) and 9 Technical Implementation Units (TIU)), Deputy of Life Sciences (2 RCs, 4 TIUs, and 1 center), Deputy of Engineering Sciences (5 RCs and 4 TIUs), Deputy of Social Sciences and Humanities (5 RCs), Deputy of Scientific Services (2 RCs, 3 TIUs, and 2 centers); and Principal Secretariat (4 bureaus, 1 center, and 1 RC).

Under the Principal Secretariat, LIPI established the National Training, Education Center for Researchers Development (Pusbindiklat Peneliti LIPI) based on the Chairman of LIPI Decree No. 3212/M/2004, on October 28, 2004. Pusbindiklat Peneliti LIPI has the main goal to improve the researchers' competence and the capacity of research and development at the Center and Region, in accordance with the policy setting by the Chairman of LIPI. Pusbindiklat Peneliti LIPI has the main duties to implement the education, training, and functional development of researchers both for central and regional researchers inside and outside LIPI; and to provide the education and training program for employees in LIPI ( (December 11, 2011)). Pusbindiklat Peneliti LIPI conducts several training programs each year to provide training programs for researchers all over Indonesia and also for LIPI's employees. These training programs are divided into two types: training program for first-degree researchers (DFP Tk. Pertama) and technical training program (Diklat Teknis dan Kedinasan). This study attempts to investigate the effectiveness of DFP Tk. Pertama, since it is the main training program that is conducted by LIPI and also this training program is aligned with LIPI's main function as the founder of research in Indonesia.

1.4.1. Training program for first-degree researchers (DFP Tk. Pertama)

DFP Tk. Pertama is known as the main training program conducted by Pusbindiklat Peneliti LIPI. This training program is conducted only for the researcher candidates from the Indonesian government institutes and/or ministers from all over Indonesia regardless their research background (social/technical/natural sciences). This training program has two main funding sources. The first funding source comes from Indonesian Institute of Sciences (LIPI) for the researchers who come from LIPI. In other word, LIPI's researchers can follow the training program for free regardless their origin departments. The second funding source is the government institutes/ministers outside LIPI, in which they send their researchers to follow the training program by paying around 12.625.000, - rupiahs (around 1075.82 euro) for the tuition fee.

DFP Tk. Pertama has the main goal to improve the knowledge and research skills based on the researchers' expertise and develop a professional character as researchers, in order to support the development of the research in Indonesia ( (December 11, 2011)). To maintain the effectiveness of DFP Tk. Pertama, Pusbindiklat Peneliti LIPI conducts some evaluations with the focus on: curriculum, the duration of the learning process, and the instructional strategies. Beside the evaluations, the administrator also conducts pre-test and post-test to find out how much the participants of the DFP Tk. Pertama improve their knowledge. The tests are conducted during the training program and administered under the supervision of the Head of the training program administrator.

DFP Tk. Pertama is conducted for three weeks, from Monday to Saturday. The program consists of five main subject domains: Introduction and Formulation of Research Proposal, Research Design, Data Collection, Data Analysis, and Technique for Scientific Writing. These course studies are placed in a specific sequence so that the participants will be able to understand the subject materials in the most effective and efficient way.

Each class in the DFP Tk. Pertama consists of 30 participants, which is separated based on their research expertise (social and technical/natural sciences). Every year, Pusbindiklat Peneliti LIPI conducts the DFP Tk. Pertama approximately for 12 times, which means that there are around 360 new researchers who are graduated from this training program every year.

1.4.2. Technical training programs (Diklat Teknis dan Kedinasan)

To support the DFP Tk. Pertama, Pusbindiklat Peneliti LIPI also conducts some technical training programs, in which the researchers can follow optionally. These training programs are:

English for academic purpose training program

English for academic purpose training program has the main goal to improve the ability in understanding scientific English for the researchers and other employees from Indonesian Institute of Sciences.

Training program for proposal writing

Training program for proposal writing has the main goal to enhance the researchers' ability in writing a research proposal by giving the knowledge about proposal writing formulation. This training program has two main subjects: scientific proposal writing and scientific presentation technique.

Training program for scientific writing

Training program for scientific writing has the main goal to enhance the researchers' capability in writing proposal or research report in scientific way. This training program can also be considered as the continuation of one of the subject in the DFP Tk. Pertama (Technique for Scientific Writing). Therefore, the main requirement to follow this training program is the participants must have already followed DFP Tk. Pertama, which is proven by the certificate of the training program.

Training program for research method and data analysis

This training program has the main goal to improve the data analysis competency for the researchers and other employees. After follow the program, the participants are expected to be able to do the data analysis by using an efficient and accurate method, and to have a conceptual, comprehensive and broad mind in doing the data analysis.

Training program for research design

The training program for research design has the main goal to improve the researchers' knowledge and skills in research and develop their dynamic and comprehensive mindset. After follow this training program, the participants are expected to understand the general concept of research design, and be able to make a research design based on the correct scientific methodology.

1.5. Overview of the remaining chapters

This study will be comprised of five chapters. Following the introductory chapter, which outlines the nature of the study and research questions, Chapter 2 will present reviews on related literature about the curriculum effectiveness in a training program. Chapter 3 will discuss about the development of the instrument and research design to measure how far the existing curriculum contributes to the effectiveness of DFP Tk. Pertama. This chapter will be followed by chapter 4, which will discuss about the result of the investigation toward the research questions. Finally, Chapter 5 will conclude the study through an articulation of the research findings, a discussion of the implications of these findings and the presentation of a set of recommendations.

II. Literature Review

2.1. Introduction

This chapter will give a review about some literatures which is relevant for this study. This review will discuss curriculum definition and curriculum evaluation that can lead into an effective training program.

2.2. Curriculum Definition

Curriculum has some definitions according to its usage in the training program. Some authors define curriculum as an implemented system to describe the content of a course in the learning process in the classroom, which is also providing the measurement of the participants' knowledge (Sconce and Howard, 1994; Curriculum Evaluation Manual, 2011). It is often used to indicate a program, whether for a subject, grade, the entire subjects, or even the whole range of a program cycle (Lewy, 1977). According to Leathwood and Phillips (2000), curriculum aims on the outcome oriented approach that emphasizes the skill and capabilities in which students lead to become capable graduates and fully prepared for their future career. Therefore, the curriculum quality will not merely depend on the teachers' skills, but also on the teachers' commitment and competence in curriculum improvement (Shobokshi and Sukkars, 1988).

Curriculum is an implemented system which consists of four different elements. The first element is textbooks. Hoover (1999) states that curriculum consists of plan books, courses of study, and text books. Further, he also states that curriculum intends to have an outcome that matched between what should be learned and what the students really learn. A good curriculum should support and facilitate the participants' effort to achieve similar result as the intended goals of the training program. Altrichter (2005) agrees that curriculum should provide text books, computer software, teaching strategies recommendation and working material for students. Posner (2004) states that textbooks are important for the curriculum effectiveness. Textbooks are important parts in curriculum because it functions as a day-to-day guide during the learning process to ensure curriculum effectiveness. Lewy (1977) emphasizes the importance of the textbooks by stating that textbooks are parts of the curriculum that can be evaluated to maintain the effectiveness of a training program.

The second element is course of the study. Course of study is important because it functions as a guide to view a curriculum as a series of courses that the participants must go through, which can ensure that they are not overlapping each other (Posner, 2004). Bokonjic, Steiner & Sonntag (2009) define course of the study as a subset of a program of study that should be designed through a very specific institutional procedure to ensure its effectiveness. Course of the study is also known as the essence of the curriculum, which should encourage and develop the critical thinking skills and problem-solving abilities of the training participants (Ashraff, 2011). Therefore, designing an efficient, effective and un-overlapping course of the study in a training program is crucial to create an effective curriculum.

The third element is classroom practices. Providing suitable classroom practices during the learning process could help to develop a better exploration and conceptualization about the course of studies (Langrange, 2005). Therefore, classroom practices indeed influence the improvement of students' achievement (Welingsky, 2001). Together with appropriate textbooks and effective course of study, suitable classroom practices is proven to be able to improve the participants' knowledge and skills in a training program.

The last element in the curriculum is the facilities in a training program. In a wider range, the curriculum involves infrastructure and learning resources, student activities, and supporting system around it (Verghese and Ponmudiraj, 2008; Cornbleth, 2012) that support the learning process in a program to achieve a better student outcome. In fact, training facilities have a direct effect on the learning process. Schneider (2003) states that poor training facilities cause difficulties for teacher to deliver an adequate education to the participants, which will influence the participants' knowledge and skills improvement level. A conducive classroom condition will help the participants to have a better performance and a better learning outcome(Hale, 2002; Hunter, 2006; Olson & Kellum, 2003; Bullock, 2007). Thus, good training facilities will support the effective curriculum implementation in a training program. To reach an effective training program, the administrator of a training program should implement all of the curriculum elements, since they influence one to another.

In general, curriculum is divided into three types that can bring a major influence on the students achievement. Those types are: intended curriculum, implemented curriculum, and attained curriculum. The first type, the intended curriculum, is the reflection of the institution's preference for teaching and learning. The intended curriculum is also known as the goal of learning, and it is usually written on a document and has an official status (Voogt and Pelgrum 2005; Martin & Kelly, 1996; Valverde, 2012; Johansson, 2005). The intended curriculum describes what students are expected to know and able to do, and under what condition these things will occur (Cuban 1992; Kissane, 2000; Plaza et al., 2007). Based on its function, the intended curriculum is placed at the educational level system, in which it "reflects societal visions, educational planning, and official or political sanctioning for educational objectives"(Johansson, 2005, p. 120) and intended to "directly influence teacher training and certification, school course offerings, instructional resources, and systems of accountability" (Valverde, 2012, p. 2).

The second type is the implemented curriculum. This type of curriculum is also recognized as "curriculum in action" (Remmen et al., 1999, p. 600), since it is what is taught in the classroom that consists of teachers' intention and objectives, and classroom activities (Stevenson and Baker, 1991; Martin and Kelly, 1996; Plaza, et al., 1997; Johansson, 2005). This curriculum is the implementation of the intended curriculum in the classroom that is strongly affected by textbook and classroom practices (Cuban, 1992; Kissane, 2000; Voogt and Pelgrum, 2005). According to Kissane (2000), classroom practices involve the emphasis of practical view of the subject and the use of the helping tools during the learning process (e.g. calculator in mathematics). Since the emphasis on curriculum implementation in the classroom will be on the context, needs, perceptions, and reactions of users, especially teachers (van de Akker and Verloop, 1994), there will be some differences between what is stated in the intended curriculum and the real implementation in the field (school/classroom).

The last type is the attained curriculum. This curriculum type is what the students actually learn as the result of the learning process in the classroom (Cuban, 1992; Martin and Kelly, 1996; Kissane, 2000; Plaza, 2007). The result of this curriculum is the skills, knowledge, and dispositions that students gain after they follow the program (Valverde, 2012) and the learning outcome for the teachers (Voogt and Pelgrum, 2005). Therefore, assessment becomes a key to understand the attained curriculum (Kissane, 2000), since it is strongly related to the test and the participants' perspective about the curriculum. The attained curriculum is influenced by the implemented curriculum in the sense of the curriculum materials those are taught by the teachers during the learning process in the classroom (Kissane, 2000; Voogt and Pelgrum, 2005).

This study will investigate the attained curriculum from the DFP Tk. Pertama to answer the proposed research question. The investigation will be conducted through pre-test, post-test and reaction sheet (questionnaire). The questionnaire will measure the participants' perspective about the existing curriculum, and the tests will measure the participants' knowledge improvement which will reflect the effectiveness of the DFP Tk. Pertama.

2.3. Curriculum Evaluation

In order to investigate the effectiveness of the DFP Tk. Pertama, this study attempts to conduct an evaluation toward the existing curriculum of the program. A curriculum evaluation can be used to maintain or even improve the quality of the training program, and to detect the lack in the training program that is needed to be improved. According to Lewy (1973), curriculum evaluation helps the program administrator to maintain or reject a program, make some specific modifications needed, and helps to explain the best condition to implement the program. Further, Jacob and Koehn (2004) agree that a curriculum evaluation is essential to maintain a program to prepare competent and high-quality practitioners. Therefore, curriculum evaluation can be used to control the quality of the training program by maintaining its effectiveness. In a broader scope, curriculum evaluation is conducted to improve the quality of education through the information collected during the evaluation implementation. In other word, the art of curriculum evaluation is to collect and spread the information than can be used to improve the quality of educational practice (Norris, 1998).

To evaluate the effectiveness of DFP Tk. Pertama, this study uses Kirkpatrick model. Kirkpatrick model was introduced in 1959 and represents some levels to evaluate a training program, in which each level influences the next level (Indira, 2008). The Kirkpatrick model has four level, those are: Reaction, Learning, Behavior, and Result (Kirkpatrick, 1967; 1975; 1994). Level 1 is Reaction, which measure the participants' satisfaction toward the program through a reaction sheet (Kirkpatrick 1975; 1994; 1967; Kirkpatrick & Kirkpatrick, 2009). The reaction evaluation result can be used to determine the effectiveness of a training program and how to improve it (Kirkpatrick, 1994) based on the reaction of the participants toward the program. The training program can be considered effective when the participants present their satisfaction toward the program. The feedback from the participants can also be used to improve the training program by improving the points in which the participants give the low mark. Therefore, the evaluation should be given during the program, so the evaluation result can be use to improve the last section of the program (Kirkpatrick, 1967).

Level 2 is Learning, which can be evaluated from how far the participants improve their knowledge and skills level after following the training program (Kirkpatrick, 1967). Unlike Level 1 evaluation that can be measured through reaction sheets (e.g. questionnaires), Level 2 evaluation is usually measured from the different score between pre-test and post-test that shows the learning outcomes (Chyung, 2009). Learning level is meant to measure the participants' knowledge, skill, and/or behavior before and after the training by using pre-test, post-test, and performance test for skills (Kirkpatrick, 1994; Kirkpatrick & Kirkpatrick, 2009). Further, Dick (2002) supports this statement by stating that pre-test and post-test was suggested in conducting the Learning evaluation to demonstrate the learning outcomes as a result of the learning process. To get the best picture about the participants' knowledge and skills improvement, the administrator should pay attention to the test materials. The test materials should cover the subject matters that have been taught in order to get the most reliable and valid result of how far the learning process has taken place (Kirkpatrick, 1994; Dick, 2002). Since the evaluation in the learning level is more complex rather than in the reaction level, it also requires more work to do. The Level 2 evaluation needs a lot of work to do from the planning of the evaluation procedure, conducting the evaluation, analyzing the obtained data result, and interpreting the result (Kirkpatrick, 1967). Alike with the Level 1 - Reaction, the Level 2 - Learning evaluation also takes place when the training program is still on progress.

Level 3 is Behavior. The main goal of the Level 3 evaluation is to find out whether the knowledge and skills learned in the training program is transferred to the participants' job. Level 3 evaluation is usually conducted after the participants finish their training program. Kirkpatrick (1967) states that Behavior evaluation is conducted to measure to what extend the participants change their behavior after following the training program. It means that the participants' behavior should change after they follow the training program so that the training program can be considered as a successful program. This evaluation is recognized as the most difficult and probably the most important level. It is because based on this evaluation result, the evaluator will be able to see whether the participants' behavior has changed, or to determine the reasons why changes has not occurred (Kirkpatrick, 1994).

Level 4 is Result. Result evaluation can be defined as the final results that happens after the participants follow the training program, which are recognized by the production increasing, cost reducing, profit increasing, etc. (Kirkpatrick, 1994; 1967; Kirkpatrick & Kirkpatrick, 2009). In the end, these changes will influence the organization's existence. Therefore, this evaluation is conducted some times after the participants left the training program in order to see the real results of the implementation participants' knowledge and skills which are attained from the training program.

Kirkpatrick model can be presented in the model below:

Training Evaluation

Trainee Reactions


Behavior Change


The ways which trainee behavior is different back on the job

What was learned in the training (i.e. the skills and knowledge acquired)

The trainees' views of the relevance, utility and value of the training

Improvement in result attributable to changed behaviors

(Fig. 1. The Evaluation View of the Kirkpatrick Model - Nickols, 2011)

The figure above gives a clear framework about the flow of the evaluation from reaction level to result level based on the Kirkpatrick model. In the reaction level, the participants give their opinion about the training program. Learning level evaluates how far the participants are able to gain knowledge and skills from the training program through the score difference in pre-test and post-test. The behavior level evaluates to what extend the participants' behavior change after they follow the program, and whether they can implement their new behavior in the work. The last level, the result level, shows the final result in the organization scale that happen after the participants follow the training program (Kirkpatrick, 1967).

Due to the limited time in doing the research, this study will only employ level 1 and level 2 evaluation to answer the research questions. The investigation of Level 1 will be conducted through the questionnaire, which will measure the participants' reaction toward the effectiveness of DFP Tk. Pertama. The Level 2 will be investigated through the score difference between pre-test and post-test. This score difference is used to find out how far the effectiveness of DFP Tk. Pertama is able to improve the participants' knowledge and skills after they follow the training program. Level 3 and 4 will not be investigated in this study because these level could takes months or even years before the result can be observed.

2.7. Summary

This study will focus on the curriculum effectiveness as one of the effectiveness indicators of DFP Tk. Pertama. It is because the curriculum consists of some crucial element in the learning process, such as textbooks, course of the study, classroom practices, and training facilities that support the program. The curriculum effectiveness in a training program influences the participants' ability to improve their knowledge and skills. In order to maintain the effectiveness of the training program, a curriculum evaluation is needed. The reason is, from the curriculum evaluation result, the administrator will be able to distinguish the strength and the weakness of the curriculum that will affect the effectiveness of the training program. Thus, the administrator will be able to maintain or even improve a program that improve the participants' knowledge, or reject a program that will bring no improvement to participants' knowledge.

Based on the explanation above, this study proposed a hypothesis that the curriculum elements (text books, classroom practices, course of the study, supporting facilities) have a positive correlation with the training program effectiveness.

In order to have a better understanding to the variables of the study, the following keywords are defined:

Training program is a program that is conducted in different specification based on the purpose to improve the knowledge and skills of the participants

Curriculum is an element in a program that is used to describe and explain the content of the course of the learning process that can measure the participants' knowledge.

The conceptual framework model of this study is built as follow:

Curriculum :

text book

classroom practices

course of the study

training facilities

The effectiveness of the training program

III. Methodology

3.1. Introduction

In the previous chapter we have discussed the literatures about curriculum and curriculum evaluation that may influence the effectiveness of the functional training program for first-degree researcher (DFP. Tk. Pertama). This chapter will discuss research design, research methods, samples, research instrumentation, and techniques to analyze the data result.

3.2. Research Design

This study will try to answer the main research question about the effectiveness of DFP Tk. Pertama. Due to the time limitation, this study will only employ a field survey with a limited sample size to investigate the research questions.

There are three research designs in conducting a research study: descriptive, explanatory and exploratory research design. The descriptive research is used to answer the "what" research question about the variables that will provide an objective, reliable, and scientifically valid description about the variables in their original state (Vaus, 2005; Mitchell & Jolley, 2010). This research merely describes and measures things as they are, without any effort to change the variable condition. In other words, the descriptive research observes the variables without intervening them (Hopkins, 2000). Therefore, this research is sponsored a lot by government especially in the social fields, such as the population research, household expenditure patterns, time use studies, employment, and the like (Vaus, 2005). The descriptive research takes several different form in order to be able to measure the variables based on the specific research questions. According to Hopkins (2000), the common forms of descriptive research are observation, case, case-series, cross-sectional studies, prospective or cohort studies, and case-control studies.

The second research design is the explanatory research design. This research design attempts to answer the "why" research question by developing a causal explanation (Vaus, 2005; Fraenkel & Wallen, 1993; van Akken, 2004). The main purpose of explanatory research design is to describe, explain, and predict in order to understand the setting of construction or improvement problems and to know the properties of the materials to be used (van Akken, 2004). Therefore, this research design has a deeper and more complex investigation toward the variables comparing to the descriptive research design. In the end of the study, we will find out whether the relationship between variables might or might not mean causation between them (Fraenkel & Wallen, 1993). By explaining the causation between variables, Vaus (2005) states that the explanatory research design is important to be used to collect information that enables the readers to see which empiric way will work best in order to avoid invalid inferences.

The third research design is exploratory research design. Exploratory research design is an empirical pilot study that explores the development of several conceptual measures and models and the implications of varying assumptions and hypotheses (Bontis, 1998; Bankes, 1993). The purpose of exploratory research design is to identify the underlying dimensional structure of a set of measure (Stewart, 2001). Therefore, the exploratory research design is the most complicated of the three research designs, since it requires a deeper research and skills to answer the research questions. The result of this research design is the researchers are able to improve their ability to collect and analyze the data to provide an effective means for evaluating model purpose (Leinhardt & Wasserman, 1979). The data that is collected during the exploratory research provides a rich description, from a small scope until broad range of population (Neuman, 1994 in Cuthill, 2002) which will enable the researchers to provide a thorough data analysis that can represent the trend in the population.

Based on the research questions of this study, we will employ the explanatory research design, since we want to investigate how far the curriculum effectiveness influence the effectiveness of the DFP Tk. Pertama.

3.3. Research Method

In general, research methods can be divided into three categories: qualitative, quantitative, and a mixed model between qualitative and quantitative. In qualitative research, a researcher observes the sample/population to collect and analyze data. Therefore, qualitative research is much more subjective rather than quantitative research, since it has no exact measurements or statistic but uses words, descriptions and quotes to explore the answer of the research questions (Anderson, 2006; Shields, 2003). On the other hand, quantitative research relies on the numbers, proportion and statistic usage to answer the research questions (Shields, 2003). The using of numbers and statistical methods in analyzing the data in a quantitative method can minimize the bias from the researchers' personal opinion that eventually will affect the final interpretation of the result. Anderson (2006, p. 1) states that a quantitative measurement will be "objective, quantitative and statistically valid". The last research method is the mixed model between quantitative and qualitative research. Basically, a mixed model combines both quantitative and qualitative research to answer the research questions. It represents research that involves collecting, analyzing, and interpreting both qualitative and quantitative data in a single study or series of studies (Leech & Onwegbuzie, 2009; Creswell & Plano Clark, 2007).

To answer the research questions, this study will employ the quantitative research method to analyze the evaluation results which are collected from the questionnaire survey, pre-test, and post-test. The choosing of quantitative method is based on the reality that the quantitative approach puts a considerable trust in numbers that represents opinion or concepts in a definite way (Amaratunga, 2002). Therefore, the result of the quantitative approach will be more definite and precise, and also has minimum subjectivity comparing to the qualitative approach, since the quantitative approach calculates relationships between variables by using statistical measurement (Hopkins, 2000; Anderson, 2006).

The research questions in this study requires a brief description about the implemented curriculum of DFP Tk. Pertama. Therefore, the usage of quantitative approach is appropriate, since according to Leech & Onwuegbuzie (2006) in Lowenthal & Leech (2009, p.8) most quantitative approaches have "quantitative research questions which are either descriptive, comparative, or relationship based". With the implementation of quantitative approach in analyzing the data result, this study will avoid the bias from our personal subjectivity and able to provide a more reliable and valid outcome.

3.4. Sample

In a study, sample size can be determined through several ways: by using a census, which can eliminate error and provide data for all the individuals in the population that usually can be resulted from questionnaires; imitating a sample size of similar studies, that can provide a guidance about typical sample sizes those are used; and using published tables, which provide sample size for the given set of criteria (Israel, 2009). This study will use a questionnaire to minimize the error and to collect information about the population.

The target population of this study is the participants of the current DFP Tk. Pertama, which consists of 30 participants. The information of the samples is collected from the questionnaire sheets, which will provide respondents' information in the sense of participants' prior education, gender, age, and their research experiences before they follow the training program. The gender of the respondents is 14 male (46.67%) and 16 female (53.33%). The age of the participants falls within 24 years old until 40 years old, with the average age is 29.57 (=30) years old. The educational background of the participants range from bachelor 90% and master degree 10% from different majors, the natural sciences 22 participants (73.33%), technical sciences 6 participants (20%), and social sciences 2 participants (6.67%). Before the respondents follow the training program, they already have research experiences for 0-5 years (... %), and 6-10 years (... %).

The respondents come from various Indonesian government departments and ministries, with the composition as follows: Ministry of Health (17 participants), The Ministry of Forestry (2 participants), National Institute of Aeronautics and Space (7 participants), and National Plantation Research, Inc. (4 participants). Total amount of the respondents is 30 participants.

3.5. Instrumentation

Data in this study is collected by using a questionnaire, pre-test and post-test. The type of the questionnaire is a closed questionnaire, in which the respondents will answer the questions based on 1-5 Likert scale. The questionnaire is used to investigate the existing curriculum in the DFP Tk. Pertama, and the pre-test and post-test is used to measure the development of the participants' knowledge and skills after they follow the training program.

3.5.1. Questionnaire

This questionnaire uses the adaptation of Level 1 from Kirkpatrick model which is adjusted to the current condition in the DFP Tk. Pertama. The questionnaire is developed to assess the participants' perspective about the existing curriculum.

The questionnaire contains some questions about respondents' profile, their view about the existing curriculum, and supporting facilities of the training program. The first part of the questionnaire will explore the personal information from the respondent, their gender, age, their original departments/institutions, and their research experiences before they follow the training program. The second part contains 30 statements on a 5 Likert scale in the range of 1 (very bad) to 5 (very good). This Likert scale will enable the respondents to express their opinion about the existing curriculum of the DFP Tk. Pertama.

3.5.2. Pre-test and Post-test

The pre-test and post-test in this study use the questions from the administrator of the DFP Tk. Pertama, since these questions are prepared by some teachers who are experts in their fields. The pre-test will be given in the beginning of the training program, before the participants follow any courses. This test has the main goal to measure the participants' knowledge from their prior education and experience(s) in research. The post-test will be conducted at the end of the program. The goal of the post-test is to measure the participants' knowledge and skills after they follow the whole program in the DFP Tk. Pertama. The improvement of the participants' knowledge and skills will be measured from the score differences between the pre-test and post-test to investigate the Level 2 of Kirkpatrick model.

This study will use quantitative methods to analyze the data result from the questionnaire survey, pre-test and post-test. The collected data will be analyzed using the SPSS.