This paper presents a study of the need to equitably assess students with diverse linguistic and cultural backgrounds in the learning environment. In order to provide better educational environments for students with diverse linguistic and cultural backgrounds, this study looks into the importance of developing assessments that take diversity of student populations into account.
This paper examines the following:
Possible factors that contribute to fair assessment or equity in assessing a ELL.
The role of classroom based, performance- based assessment and self assessment in promoting learning for ELLs in schools that are increasingly under pressure to prepare these students to pass high stakes standardized tests.
Factors that help build an assessment system that eliminates educational inequities.
In this study, English language learner is defined as a student who speaks a language other than English at home, including both limited English proficient and former limited English proficient.
Education can be defined, then, as the "transfer of learning that is the application of what is learned in one domain or context to that of another domain or context" (Amrein & Berliner, 2002, p.10). But what types of assessments are able to capture or promote progress toward education?
Second-Language Proficiency and Academic Achievement
Cummins (1980,1981) has provided a much-needed framework in the field of bilingual and English as a second Language (ESL) education. His critical work reveals why ELL students' academic achievement cannot be assessed in the same manner as that of their FEP (Fluent English Proficient) counterparts. He asserts that oral fluency cannot be regarded as academic competence in academic settings.
Cummins theorizes that there are two distinctively different proficiencies. Basic conversational language ability is acquired rapidly. ELL students take only a year or 2 to become proficient in conversational English (see also Hakuta, Butler, & Witt, 1999). In contrast, attaining grade level of academic English can take far longer, as long as 5 to 7 years. Academic English is necessary for tasks that are context reduced, such as reading chapters in a textbook that describes different math functions.
Foertsch (1998) describes second-language learning as an important factor that influences how children learn to read: "The ways in which children communicate in their home cultures are critical to the development of written language models of reading and writing. The home language of students provides the foundation for the emergence of reading and writing behaviours. If there is a mismatch between the structures, values, and expectations of the home language and school language, children may be at a disadvantage for success in early reading tasks, and thus spend their entire school careers attempting to catch up (Gay, 1988; Snow, 1992).
Research shows that language-minority students face many challenges in school. For example, they are 1.5 times more likely to drop out of school than native speakers (Cardenas, Robledo, & Waggoner, 1988). English-language learners also receive lower grades, are judged by their teachers to have lower academic abilities, and score below their classmates on standardized tests of reading and math (Moss & Puma, 1995).
It has long been recognized that a substantial achievement gap exists between language-minority students and native speakers of English (August & Hakuta, 1997; Silver, Smith, & Nelson, 1995). A significant gap in math scores, in particular, has caused widespread concern among educators (Khisty, 1997; secada, Fennema, & Adajian, 1995). Moreover, language-minority students are less likely to be represented in math-related majors in higher education, which affects their career opportunities and lifetime earnings (Bernardo, 2002; Cuevas, 1984; Torres & Zeidler, 2001). Apparently, math achievement plays a significant role in the academic and social stratification of minorities (Khisty, 1995; secada, 1992). Thus, English language learner (ELL) students' math achievement-or lack thereof-should be explored in light of new ways ELL students are being assessed.
Factors Influencing the Assessment of English Language Learners
Varying levels of proficiency in English
ELLs vary widely in their level of English language proficiency, and furthermore, ELLs may have varying levels of oral and written English proficiency. Do not assume that students who can converse easily in English will have the literacy skills necessary to understand the written directions for a standardized test.
Some ELLs may be proficient in the English used for interpersonal communications but not in the academic English needed to fully access content-area assessments. Studies show that the level of language proficiency has an influence on processing speed. In other words, compared with native speakers, ELLs generally take longer on tasks presented in English. This is important to keep in mind when designing and scoring the assessment, as well as when making decisions about testing accommodations.
Varying levels of proficiency in native language
ELLs also vary in their levels of proficiency and literacy in their native languages. Therefore, do not assume that speakers of other languages will be able to understand written test directions in their native languages. In fact, a large proportion of ELLs born in an English speaking country may not have had any formal schooling in their native language. This is important to keep in mind when considering the use of native language accommodations.
Educational Background Factors
Varying degrees of formal schooling in native language
As mentioned previously, ELLs vary widely in the level of formal schooling they have had in their native languages. The degree of native-language formal schooling affects not only native language proficiency-specifically, literacy in the native language-but also the level of content area skills and knowledge. For example, students from refugee populations may enter an educational system with little or no formal schooling in any language. These students must learn English and content-area knowledge simultaneously, while also being socialized into a school context that may be extremely unfamiliar. Other ELLs may come to the country with more formal schooling and may have received instruction in the content areas in their native languages. The primary challenge for these students is simply to transfer their existing content knowledge into English. Again, these factors come into play when making decisions about appropriate accommodations.
Varying degrees of formal schooling in English
ELLs also vary in the number of years they have spent in schools where English is the language of instruction. A distinction may also be made between students who have studied English as a foreign language while in their home countries and students who have studied English as a second language only in the United States. Furthermore, ELLs differ in the type of instruction they have received while in English-speaking schools. Bilingual, full English immersion, and English as a second language are but three of the many existing instructional programs for non-native English speakers, and there are great variations in how these programs are implemented.
Varying degrees of exposure to standardized testing
It should not be assumed that all ELLs have had the same exposure to the standardized testing. Students in some countries may have had no exposure to multiple-choice questions, while those from other countries may never have seen a constructed-response question. Even ELLs from educationally advantaged backgrounds and with high levels of English language proficiency may not be accustomed to standardized, large-scale assessments and may be at a disadvantage in these testing situations.
Cultural factors can also be potential sources of construct-irrelevant variance that add to the complexity of appropriately assessing ELLs.
Varying degrees of acculturation to mainstream
ELLs come from a wide range of cultural backgrounds, and cultural differences may place ELLs at a disadvantage in a standardized testing situation. Lack of familiarity with mainstream culture can potentially have an impact on test scores for ELLs. Students who are unfamiliar with culture may be at a disadvantage relative to their peers because they may hold different assumptions about the testing situation or the educational environment in general, have different background knowledge and experience, or possess different sets of cultural values and beliefs, and therefore respond to questions differently. Students from cultures where cooperation is valued over competition, for example, may be at a disadvantage in those testing situations where the goal is for each individual student to perform at his or her best on his or her own. Students from economically disadvantaged backgrounds may also respond to questions differently and may have background knowledge and experiences that are different from those presumed by a test developer.
The Role Of Classroom-based, Performance-based Assessment And Self- assessment In Promoting Learning For ELLs.
Research has shown that improved assessment practices at the classroom level can have powerful, beneficial effects on transfer of learning and measures of achievement, including standardized test scores (Black & Wiliam, 1998; Stiggins, 2002). In fact, Black & Wiliam, in a review of over 250 articles, found that improved formative or classroom assessment practices helped low achievers more than other students. This revealing finding has direct implications for NCLB and for school systems that want to close the achievement gap. To make improvements, however, teachers must be provided with the assessment tools they need for increasing the achievement of ELLs. New understandings of the learning process indicate that assessment and learning are intimately linked. These new understandings of learning need to be applied to classroom-based assessment practices (Marzano, Pickering, & McTighe, 1993). Among these practices, performance-based assessment appears to hold promise for improving the educational attainment of ELLs.
Using Performance-Based Assessment to Promote Learning
Performance-based assessment links assessment to instruction through the use of meaningful and engaging tasks. Performance tasks may also call for integration of language and content-area skills. Authentic assessment, a type of PBA, promotes application of knowledge and skills in situations that closely resemble those of the real world (Frisby, 2001; McTighe & Ferrara, 1998; Wiggins, 1998). Authentic assessments are potentially more motivating than other types because they engage students in realistic uses of language and content area concepts. Authentic assessment and other types of PBA can be used in the service of education to promote transfer of learning from facts and procedures to applications in meaningful contexts. A large range and number of tasks are needed over time, however, to ensure the generalisation of PBAs. A number of factors make PBAs more appropriate for ELLs than traditional testing formats (Frisby, 2001; Hamayan & Damico, 1991; O'Malley & Pierce, 1996). Well-constructed performance tasks are more likely than traditional types of assessment to do the following:
â€¢ provide comprehensible input to students
â€¢use meaningful, context embedded tasks through hands-on or collaborative activities
â€¢ show what students know and can do through a variety of assessment tasks
â€¢ support the language and cognitive needs of ELLs
â€¢ allow for flexibility in meeting individual needs
â€¢ use criterion-referenced assessment for judging student work
â€¢ provide feedback to students on strengths and weaknesses
â€¢ generate descriptive information that can guide instruction
â€¢ provide information for teaching and learning that results in improved student performance
Further, PBAs have the potential to provide in-depth information about a student's ability to integrate knowledge for specific curriculum objectives or standards.
The current trends in learner-centered language teaching approaches, and a growing interest in "authenticity and interactiveness" (Bachman & Palmer, 1996) have led to a greater interest in expanding the use of second language self-assessment (Bachman, 2000; Calfee & Hiebert, 1991; Hamayan, 1995). Many language testers have been inspired to investigate whether students are able to make a meaningful contribution to their own evaluation. This approach not only promotes autonomy in student learning, it also helps the teachers measure the students' progress in the course. Development-oriented self-assessment may best serve as a complementary instrument to traditional assessment presently. Because self-assessment is performed through complex cognitive processes which are affected by many uncontrollable factors, there still remains much disagreement in the discussion regarding the effective use of self-assessment. Despite a number of difficulties in appropriately implementing self-assessment, the ways in which we resolve these issues will certainly provide valuable insights into the nature of language teaching, learning, and assessment. When these challenges are met, it is hoped that language institutions and classroom teachers will consider the potential of self-assessment as both a valid and reliable supplement to traditional assessment.
Factors that help build an assessment system that eliminates educational inequities.
In order to protect students from unfair and damaging interpretations and to provide parents and communities with an accurate overall picture of student achievement, educators need to be aware of the promise and the challenges inherent in using alternative assessment practices for "high stake decisions" (such as student retention, promotion, graduation, and assignment to particular instructional groups), which have profound consequences for the students affected. Only then will educators be able to build and use an assessment system that is a vehicle for eliminating educational inequities. Although alternative assessments can help ensure ethnic, racial, economic, and gender fairness, equity cannot be achieved by reforms to assessment alone. Change will result only from a trio of reform initiatives aimed at ongoing professional development in curriculum and instruction, improved pedagogy, and quality assessment.
Multiple assessment indicators are especially important for assessing the performance of ethnic-minority and language-minority students (Koelsch, Estrin, and Farr (1995)). The real challenge comes in selecting or developing a combination of assessments that work together as part of a comprehensive assessment system to assess all students equitably within the school community.
The first and most critical step in assessing with equity is determining the purposes for assessing and clarifying whether those purposes are low stakes or high stakes (Winking & Bond, 1995). In many cases, schools, districts, and states have not a single purpose, but multiple purposes--some low stakes and some high stakes--for assessing student performance.
The need of an 'assessment briefing' to communicate requirements -
The following could help in assessment briefing:
Outlining broad assessment-related expectations of students and communicating the criteria on which students will be marked.
Students can be presented with a written guideline which contain explicit, unambiguous instructions and exemplars that model the appropriate discipline-based thinking, writing and/or performance to guide student efforts in completing assignments and studying for exams.
Clearly outlining the department policy and practice on extensions and special consideration along with defining the relevant resources and support.
Effective debriefings shed light on the underlying drivers of performance, and with these drivers illuminated, instructors can target discussion and teaching to what is meaningful to learners. This approach can be used in short ad hoc learning conversations to encourage reflection and promote deeper learning.
It is helpful to avoid assuming any difficulties these students may have with understanding assessment requirements are necessarily related to language. Many Second language learners have a high level of language proficiency but a low level of cultural knowledge. The use of a local jargon and idioms in instructions can affect the students' understanding of the task.
Students can become disheartened if they do not do as well as they thought they might have in assignments or exams. Often it is helpful to gently alert students that it may take time to adjust to the requirements of assessment and that many students do not get perfect or very high marks for assignments and exams, especially for those who are at University level, irrespective of their prior high scores.
Defining the Construct
A second criterion for validity is a precise and explicit definition of the construct the test is intended to measure. For K-12 assessments, state standards underlie the test specifications. Sometimes other state documents, such as curriculum frameworks, may clarify knowledge and skills stated in the standards. When defining a construct for an assessment to be given to ELLs, consider in particular how English language skills interact with the construct. For example, when defining the construct for a mathematics test, consider whether it is intended to be a test of mathematics, in which case the test should require no or absolutely minimal English proficiency, or a test of the ability to do mathematics within an English-language educational environment, in which case the ability to comprehend word problems in English may be part of the construct. Similarly, those who define the construct should pay attention to how much of the vocabulary of the discipline in English is to be viewed as part of the assessment. Defining English proficiency as part of a target construct for an assessment in mathematics or science is neither right nor wrong. It is essential, however, that these definitions be explicit. Furthermore, even if English proficiency is part of the construct, take care to define what level of English proficiency should be expected of students. When defining the linguistic demands to be included in the construct, make an effort to include professionals with backgrounds in educating ELLs.
Domain of Knowledge and Skills
School boards are likely to have documented content standards for the subject area to be assessed. They may also provide performance standards and other documents that define the domain and their expectations for student achievement. Test developers should review these documents carefully and note the degree to which each standard calls for the ability to read, write, speak, or listen in English. Some disciplines use everyday language to refer to certain disciplinary concepts (e.g., the terms energy and transfer in physics), while specific language terms are used for other concepts (e.g., the terms mitosis and metamorphosis in biology).
Relative Weights of Tasks and Skills
The weight of a task or content category is generally decided by the importance of the assessed task relative to the other tasks on the test and the degree to which the tasks tap content described in the state's standards. Often tasks that require more time to complete (and usually longer responses written in English) receive more weight in an assessment. Such weightings may disadvantage ELLs; therefore, develop a careful rationale for weighting to apply to all students' responses, taking both content knowledge and language skills into account.
Assessment and Response Forms
Just like students in the general population, ELLs vary greatly as individuals. Therefore, no one type of presentation or response is optimal for all ELLs. It has to be, however, kept in mind that while developing assessment specifications that depends on the content area being assessed, large amounts of text make it less likely that ELLs will understand what is being asked of them. Some testing programs also rely on tasks that require extended written responses to assess students' depth of knowledge in the content areas. Where possible tasks that allow examinees to respond in ways that do not require long responses written in English, such as by drawing a diagram or other visual representation, as appropriate, should be included.
Feedback is critical to the learning process. When most ELL students receive their assignments, tests or exams back, they carefully check for marks, comments or other feedback. The provision of as much helpful feedback as possible in writing and redirection to support resources and services as appropriate is likely to greatly assist learning. Consistency between markers and the use of marking guides can help achieve this. A brief assessment feedback session, where common strengths and weaknesses in student efforts are shared, can be useful.
Creation of Rubrics
The rubrics should make clear the role that English language skills should play in determining a score. (It may be helpful to have educators who are familiar with the performance of ELLs involved in the creation and review of rubrics). Generally, writing rubrics for content area tests should focus on content rather than on language use. Careful evaluation of the construct can help determine if, for example, writing an essay in English to provide evidence about a historical event would in fact require a certain degree of language skills. For assessments of English writing skills, the rubric should consider command of language (vocabulary, grammar, mechanics, etc.) but also make clear the role of critical thinking as distinct from fluency in English-language writing conventions.
Cummins has devised a model whereby the different tasks we expect our students to engage in can be categorized. In the diagram below tasks range in difficulty along one continuum from cognitively undemanding to cognitively demanding; and along the other continuum from context-embedded to context-reduced. A context-embedded task is one in which the student has access to a range of additional visual and oral cues; for example he can look at illustrations of what is being talked about or ask questions to confirm understanding. A context-reduced task is one such as listening to a lecture or reading dense text, where there are no other sources of help than the language itself. Clearly, a D quadrant task, which is both cognitively demanding and context- reduced, is likely to be the most difficult for students, particularly for non-native speakers in their first years of learning English. However, it is essential that ESL students develop the ability to accomplish such tasks, since academic success is impossible without it.
Quadrant to assess task difficulty