This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.
Essay writing is a must for Malaysian secondary schools' students. There are few types of essays that the students need to master. Guided writing, summary writing and continuous writing are the three main types of essay that are tested in examination. These three different types of writing test students' ability to understand and use correct grammar, to apply language skills for interpersonal purposes, to apply language skills for informational purposes, and to apply language skills for aesthetic purposes (Malaysian Examinations Syndicate, 2004).
Students have to excel in these three types of writing to be able to apply it in real life situation. Students are assessed on their writing skills in school tests as well as in the national standardized test. Different methods of assessment are used in evaluating students' essays. At different levels and tests, there will be at least a minor difference in the method of assessing. In Malaysia secondary school, teachers are not given any specific guides to assess students' written works. They assess according to their own methods and experiences.
There are actually many methods that they can refer to for assessing students' essays. Every method has their own specific guidance and teachers can use them based on their needs and students performance. For example, holistic scoring method is used to assess students' essays in a generalized way that is to look at their works as a whole piece without analyzing it in details. Any method is analytic scoring method where teachers look at students' performance in details. Both methods can be applied by teachers in classroom- based assessment.
In this study, only two methods of essay scoring will be discussed which are mentioned above. Holistic and analytic scoring methods are very different and unique in their own way. Therefore, it is important that teachers know how to use both of them in assessing students' essays. The subjects that will be chosen for this study are English teachers in secondary schools in Malaysia. They will be investigated on the ways they use to assess their students' essay. A few students from their class will also be chosen to give feedbacks about their teachers' assessment.
1.1 Statement of Problem
Students' essays are marked without a standard scale in Malaysian secondary schools. Teachers score their students based on what they think is good for them. They use mostly holistic scoring method which might not be fair to students. The ministry of education might be responsible for what had happen. Teachers mark students' essays according to what the ministry suggests.
Holistic scoring method has long been implemented in schools. Is it the best method for teachers and students? There had been some problem with this method because it is not specific enough. It might also create biasness in the case where students are close to their teachers. This method views the essay as a whole without further analysis on each component. While analytic scoring method views the essay in a more specific way. It provides details for students to be able to know what is lacking in their essays. Analytic scoring method uses a rubric as a standard scale to mark essays.
Analytic scoring method is regarded as more work and expensive while holistic scoring method is considered cheaper and more efficient. Holistic scoring method provides less information to students than analytic soring method. Each of the methods has their own advantages but there should be one method that is more suitable for essay scoring.
1.2 Purpose of the study
This study aims to investigate the use of holistic scoring method and analytic scoring method in essay scoring for Malaysian secondary schools. This study is also aims to discover the best method in essay scoring between the two methods. Another goal of this study is to find out whether analytic scoring method is applicable in Malaysian secondary schools.
1.3 Significance of the study
Methods of essay scoring have been discussed by many experts all around the globe. It is important in a sense that it reflects students' ability in writing essay. The wrong method used to mark their essays will cause their performance to drop. How students write essays, sometimes depend on how the teachers mark their essay. There is a vital connection between the two.
Basically, Malaysian secondary schools teachers mark students' essay according to what they believe is correct and suitable for their students and the method they use can be considered as holistic scoring method. It is very important that they know a method in specific so that they can mark students' essay in details.
With this study, it is hope that teachers will realized that analytic scoring method is a more suitable method to be used in essay scoring as it will enhance students writing skills precisely and increase the quality of their essays.
1.4 Research questions
The research questions for this study are as follows:
1. What essay scoring method is used by teachers of Malaysian secondary schools?
2. Is analytic scoring method better than holistic scoring method?
3. Does analytic scoring method helps to refine current essay scoring method used in Malaysian secondary schools?
2.1 Literature review
Essay scoring methods are an interesting topic. Many researches had been carried out and the results they found were amazing. The two methods that are always being focus on are analytic scoring method and holistic scoring method.
A study of comparison of holistic and analytic scoring methods in the assessment of writing is done by Yuji Nakamura (2004). He suggested that for practical and economical reasons, holistic (one item evaluation) assessment can be used, but to avoid risky idiosyncratic ratings, analytic assessment (with several evaluation items) is strongly recommended. This shows that the two methods are applicable for essay scoring.
According to Liz Hamp-Lyons (1995), a holistic scoring system is a closed system, offering no windows through which teachers can look in and no access points through which researchers can enter. Scores generated holistically cannot be explained to other readers in the same assessment community; diagnostic feedback is out of the question. This research is about the problems with holistic scoring method.
An earlier study on validity and reliability of holistic scoring is done by Brian Huot (1990). He pointed out that the emphasis on reliability was a necessary phase in the growth of holistic scoring; he believes it is time for the profession to begin to ask questions about the validity of holistic scoring procedures. He also mentioned that the unquestioned assumptions, confusion, and neglect of validity in holistic scoring have gone on long enough.
A study was carried out by Moskal, Barbara M. (2000) on scoring rubrics. She stated that if an analytic scoring rubric is created, then each criterion is considered separately as the descriptions of the different score levels are developed. This process results in separate descriptive scoring schemes for each evaluation factor. For holistic scoring rubrics, the collection of criteria is considered throughout the construction of each level of the scoring rubric and the result is a single descriptive scoring scheme. As stated by Barbara, it is possible to set up a rubric for both the scoring methods.
Another study on the two methods was done by Nahla Bacha (2001). He indicated that although holistic scoring may blend together many of the traits assessed separately in analytic scoring, making it relatively easier and reliable, it is not as informative for the learning situation as analytic scoring. He also suggested that an analytic evaluation is better to assess students' essay.
All the researches had shown that the method used by teachers in assessing students' essay will definitely have an impact on students writing skills and ability. It is very vital to use the correct method so that students will improve their performance.
2.2 Definition of terms
Holistic scoring method is a method which teachers use to evaluate a piece of students' writing for its overall quality. It is a method used in essay scoring whereby a single mark is awarded to an essay from a range of marks categorized in different performance bands in which each band is characterized by descriptors based on the total impression of the essay as a whole. It is widely used in Malaysian schools to mark essays in school-based and public examinations.
Analytic scoring method is a method that teachers use to assess students' written works more specifically and in details. It is usually based on a well structure rubric where all the components that need to be assessed is stated out clearly and in detail. This method views an essay as technical as possible. It is not a common method used in Malaysian secondary schools.
The first purpose of this study is to investigate the use of holistic scoring method and analytic scoring method in essay scoring for Malaysian secondary schools. This study is also aims to discover the best method in essay scoring between the two methods. Another goal of this study is to refine the essay scoring method used by teachers in Malaysia secondary schools.
As for the research questions in this study, the first one is what method of essay scoring is used by teachers of Malaysian secondary schools. The second question is which method of scoring is better. The last question is will analytic scoring method helps to refine current essay scoring method.
To fulfill the purpose and research questions for this study, a few methods will be used. Interview with teachers, students will be carried out. Survey or questionnaire will also be distributed. Teachers scoring scheme and students' essays will be collected for research purpose.
3.2 Population and sampling
There will be two populations chosen for this study. The first population of this study will be Form 4 or Form 5 teachers from Malaysian secondary schools. The second population will be students from the class that the teachers are selected for this study.
The first sample for this study will be two to three teachers from Malaysian secondary schools who are teaching English. The second samples will be two to three students from the class where their teacher is selected for this study. The samples will be selected randomly. There will be no distinction between gender and races. These subjects are for interview purpose. As for answering questionnaire, about 15 teachers and 30 students will be given the questionnaire from one school.
The teachers that will be chosen for interview must have at least three years' experience in English essay scoring. They should also be cooperative and are willing to share their experience and the way the use to assess students' essays as detail as possible. The students that will be chosen as subject for interview must be proficient in English so that they can provide good feedback about their teacher's marking of their essays. Subjects for the questionnaire do not need to possess any specific qualifications. The main point is they must be teachers of English and students that write essays.
There will be a few instruments involve in this study. The first one is a holistic scoring scale. The second one is an analytic scoring rubric. The next one is the questionnaire, one set for teachers and another set for students. The following is interview questions, also two sets for teachers and students. The last one is one piece of students' essay.
3.4 Procedure and time frame
This research will take about 8 weeks to complete. It will start at the 6th week and end at the 14th week. During this period of time, the researcher will go to one or two Malaysian secondary school/s to give out the questionnaire and conducts interview sessions with chosen subjects.
First of all, two or three teachers and students will be selected as subjects for interview based on the criteria mentioned in sampling section. They will be presented with informed consent letters that will protect their real identities. After that, they will be interviewed based on the interview questions. The main focus is to find out how they mark students' essay.
Secondly, the teachers selected will be asked to pick one of their students essay. The essay will be photocopied into three copies. One copy is for the researcher. Another two will be marked by the teacher according to holistic scoring scale and analytic scoring rubric that are given to them. The marked essays will be compared for their validity and reliability. After that, the essay will be given back to the students.
Thirdly, students chosen as subjects will be interviewed to get their feedback about their score. The opinion they gave will be recorded. They will be asked on the two types of scoring methods, which one they prefer.
The next step is to distribute the questionnaires to students and teachers randomly. Questionnaires will be distributed separately. One set for teachers and another set for students. Lastly, all the questionnaires will be collected and all the data will be analyzed.
3.5 Analysis plan
All the data collected from the interview and survey regarding the method teachers use to mark students will be analyzed. They will be analyzed according to the criteria of holistic and analytic scoring method. This analysis will be able to answer the first research questions.
The two essays that the teachers mark according to the given rubrics will be compared. They will be compared based on the reliability, validity and practicality and impact. These four aspects will be determined by experienced and trained teachers. The one that gain more compliments in those aspects will be considered a better method.
The questionnaires on analytic scoring method that is distributed to students and teachers will be collected and analyzed based on the feedbacks they give. If there are more positive feedbacks than negative feedbacks, it suggests that analytic scoring method might be more suitable to be used in Malaysian secondary schools.
3.6 Validity and reliability
The questionnaire chosen is from a Master thesis (2006). It has been already
The first assumption is that the subjects chosen will represent the whole population. The second assumption is all the subjects answer the questionnaire honestly based on their experience and knowledge. The last assumption is all the instruments that will be used are suitable for this study and it is valid.
3.8 Scope and limitation
The first limitation is the small number of subjects that will be selected might not represent the whole population as expected. It might only valid for a small population.
Another limitation is time. There is not enough time allocated for this study. More time will ensure a better result.
The last limitation is the scoring rubrics. The scoring rubrics chosen might not be the best to use. It might not be suitable for Malaysian context.
Table 2.Â Features of the holistic rating scale used in this study.
Raters: Three trained native speakers of English.Â Items: One evaluation item (Overall)
Rating scale:Â A four-point scale (1, 2, 3, 4) was used with the following criteria:
the main idea was clearly stated
the essay was well organized
the choice of words was good
very few minor grammatical errors
the main idea was fairly clear
the essay was moderately well organized
the vocabulary was good
some minor grammatical errors
the main idea was indicated, but not clearly
the essay was not so well organized
the vocabulary choice was fair
some major grammatical errors
the main idea was hard to identify
the essay was poorly organized
the vocabulary was weak
many grammatical errors
Table 3.Â Features of the analytic rating scale used in this study.
Raters: Three trained native speakers of English.
Items:5 criteria were rated: (1) Originality of Content, (2) Organization, (3) Vocabulary, (4) Grammar, (5) Cohesion & Logical Consistency
Rating scale:Â A four-point scale (1, 2, 3, 4) was used with the following criteria:
Originality of Content
4 points:Â interesting ideas were stated clearly
3 points:Â interesting ideas were stated fairly clearly
2 points:Â ideas somewhat unclear
1 point:Â ideas not clear
4 points:Â well organized
3 points:Â fairly well organized
2 points:Â loosely organized
1 point:Â ideas disconnected
4 points:Â very effective choice of words
3 points:Â effective choice of words
2 points:Â fairly good vocabulary
1 point:Â limited vocabulary range of vocabulary
4 points:Â almost no errors
3 points:Â few minor errors
2 points:Â some errors
1 point:Â many errors
Cohesion & Logical Consistency
4 points:Â sentences logically combined
3 points:Â sentences fairly logically combined
2 points:Â sentences poorly combined
1 point:Â many unfinished sentences