You are here: UK Essays » Essays » Teaching » Student Performance

Free Essays - Teaching Essays

Evaluating Student Performance in a Distance Learning Environment: A Survey of the Literature

Over the past two decades, the emergence of enterprising technology has made it possible for the world to have learning opportunities at its fingertips. Specifically, the World Wide Web has unleashed a powerful access medium, tapping into a educable market some probably thought unreachable. Educational institutions, particularly higher educational institutions, quickly gravitated toward this booming, technological phenomenon as a means to provide instruction to atypical students and, thereby, educate a generation who may not have had such an opportunity to engage in meaningful learning activities through normal classroom settings. While higher institutions of learning had become comfortable in offering onsite, night-time and evening instructional environments, technology offered more and could reach the masses. Increasingly, atypical students became mainstream students, and colleges, universities, and two-year educational systems began to cater to a more diverse student population (Jones and Jo, 1999 - 3).

Consequently, distance learning is the most proliferate form of national and global education. With instruction being delivered via the Internet, television and other media, it is expected to become an even more dominate force in educating students of all ages (McIsaac and Gunawardena, 1996 - 1). While that news may be encouraging, distance learning educators are still perplexed as to how to assess student performance accurately and fairly, given that students are spread out geographically and class sizes are often large. Additionally, instructors are challenged to maintain and ensure a balanced learning environment just as they have in synchronous learning environments. This paper is not an attempt to provide definitive solutions to those two issues, but it is an attempt to present what the literature reveals about assessing student performance in an asynchronous learning environment.

For students in an online learning environment, evidence of cognitive and affective learning is valuable, but even more valuable for policy makers of adult students is evidence of outcomes such as improved job performance. (Johnston, 2002 -67) Although, cognitive growth is an “enabling” outcome for the real goal of improved performance, some adult education programs that assist those who desire to improve life conditions do so with the intent to provide skills educational development for the benefit of improved job performance once a job is secured. Johnson describes several technology-based education projects as having three areas of assessment for adult learner outcomes: (a) workplace skill development and job attainment, (b) academic skill development, and (c) orientation to literacy.

First, in the workplace skill development area, the education projects seek to help students become gainfully employed, improve student performance in an existing job, or assist students in finding a better job. Adult students are taught workplace behavior skills such as team building, interpersonal communication, and employee/employer expectations. In addition to incorporating the literacy components of reading and math into its curriculum, the education projects also include job application and selection processes, interview techniques, and resume writing as well. (Johnston, 2002 -9) While Johnson champions the idea that a primary measure of success of this type skill development would be for students to attain a job and maintain employment for a minimum of 12 months after completing the program, he concedes that assessing mastery of these type skills is difficult. Second, the academic skills discussed in Johnson’s research that are connected to a major area of measurement in adult learning are those associated with the General Education Development (GED) test. The GED assesses writing skills, social studies, science, literature and arts, and mathematics. Measuring the performance of students enrolled in GED programs is more tangible; however, Johnson implies that confidence and anxiety levels of adults who question their own learning skills could impact outcomes and participation. Finally, what starts with adults’ lack of confidence in academic skills follows equally with their uncertainty in literacy and schooling (Johnston, 2002). What is most unfortunate about the shyness of adults in this regard is that literacy orientation plays a critical role in workplace and academic skills; it is the one skill adults need in order to be successful in the others. Johnson contributes this uncertainty of adult learners to previous school failures, embarrassment perhaps, and unawareness of their deficiencies.

Although a student population in the online environment consisting primarily of adults requires, perhaps, different attention to course delivery and participation, cognitive growth among all students in a distance learning environment is of chief importance to educators, mainly because how to measure cognitive growth, particularly higher-order cognitive processes such as problem solving, critical thinking, creativity, and knowledge transfer presents a challenge. (Johnson, 2002) It presents an even greater challenge in the asynchronous environment. Customary methods such as standardized tests have been relied upon heavily to reveal levels of student performance in basic skills but the tests are not true measures of higher-order cognitive processes (Cassady, 2001). Even in measuring basic skills, variability in administration of standardized tests and the scales upon which they report the results are not clear indicators of student performance (Cassady, 2001). Cassady advocates that tailored tests and authentic assessments are better assessments than traditional standardized tests, but they, too, fall short. Tailored tests evaluate definitive outcomes of particular intervention methods. These type tests are developed to specifically match what may be a unique learning environment. The tests can provide a clearer assessment of student performance if they are devised with adherence to basic test development criteria—that is, ensuring accurate validity, reliability, and scale of reporting scores. Authentic assessments are directly related to systematic observation (Cassady, 2001). This approach requires more in-depth analyses of individual learners because it involves direct examination of learners’ work quality, observing learner performance, or evaluating learner work products. (Cassady, 2001) In the asynchronous environment, two of these aspects is quite doable, but one, observation of performance, may not be as simple. Nevertheless, for some educators, performance observation may not necessarily be what one literally sees in a typical classroom setting; it can be what one views as participatory observation when evaluating student performance in distance learning.

Affective domain measures do offer quality information about the impact of learning on one’s emotional and motivational state, but affective variables are too broad and diverse to consider them singularly when assessing learner outcomes (Cassady 2001). On the other hand, since it is more recognizable that affective domain processes be included in evaluation strategies to assess learner emotional and motivational state, and the efficacy of the program under which learning is delivered, Cassady asserts that evaluators can consider a conceptual framework under which affective constructs can be used. The evaluation design proposed by Cassady employs those concepts which focus on attitude, motivation, and self-perceptions of ability and skill. Each conceptual category reveals characteristics of learners that determine interests and desires, achievement motivation, and patterns of performance, respectively. Just as with cognitive domain assessment processes, the development of affective domain assessments should adhere to acceptable design standards of evaluation. Additionally, according to Cassady, evaluators should consider seven principles of measurement design: (a) domain sampling, (b) social desirability, (c) response bias, (d) developmental appropriateness, (e) clarity of language (f) reliability and validity, and (g) triangulation. Again, Cassady emphasizes that the broad and diverse nature of affective domain measurement make it difficult to find a measure that meets the specific needs of an evaluation; in such cases, following established principles as those does allow the avoidance of problems that could limit convincing data.

Since distance, time, age, and gender no longer hinder one’s opportunity to pursue an education, a unique global community has been emerging, and the culture of education has changed significantly (Jones and Jo, 1998). As a result, the research suggests that flexibility in design of online instruction is a primary factor to consider to effect successful learner outcomes, while lack of face-to-face contact—or interactivity—may or may not have as big an impact. Stansfield, McLellan, and Connolly suggest that flexibility allows learners to manage their own learning by accessing course materials in whatever sequence they choose and determining whether in-depth or superficial studying is needed based on prior knowledge. Conversely, although Stansfield, McLellan, and Connolly, recognize the value of interactivity in the online environment, they imply that the quality of online instruction is far superior to its face-to-face counterpart. DeVries & Wheeler (1996-21) dispute this implication, citing the lack of face-to-face contact as a major barrier in distance education. Interactivity promotes understanding of concepts and lack of it in the online environment could cause problems of learner attrition, failure or dissatisfaction (Dennen, Darabi, and Smith, 2007 -12). Martin (1999 -21) [also mentions the lack of face-to-face interactivity as an obstruction to distance learning that could have an impact on learner outcomes. Despite the facts of research that support both sides of the interactivity argument, student performance is influenced if instructional strategies for engaging learners of different characteristics is not considered in the curriculum development process, and strong, consistent interactivity is especially needed between facilitator and students throughout the length of the class, program or seminar (Jones and Jo 1999 - 3 pg 2).

Because of the increasing demands of distance learning, online education programs are designed with all students in mind, developing various assessment methods and techniques as student demographics dictate. And, although the literature does reveal variables designers need to consider when dealing with various student population groups, the literature is also in agreement that standard instructional design principles are necessary to positively impact learner outcomes of all students. Thus, the Naval Air Warfare Center offers a tool or rubric for evaluating the quality of web-based instruction that could be used to evaluate a number of audiences. Debatably, the tool may perhaps be one of high interest to distance learning instructors because assessments in the online learning environment should be able to measure that students are learning what they are suppose to learn. Inevitably, then, assessments may sometimes be more crucial in web-based instruction because instructors are unable to adjust curriculum or testing methods as frequently or as freely perhaps as classroom instructors. Consequently, it is critical for designers of distance learning to have a quality and structured evaluation method or tool for not only effective performance assessment but also for effective feedback to learners. (Hayes, Stout, and Ryan-Jones, 2005 #20 pg 5). The following summarizes key points from the Naval Air Warfare Center’s discussion on performance assessment:

• Assessments should be relevant to what the learner cares about.
• Should be linked to course objectives.
• Should assess what the objective says it will assess.
• Should measure skills and knowledge required on the learner’s job.
• Should measure what students can do with what they know.

Relevancy appears to be a recurring theme throughout the literature. Researchers are careful to emphasize the connection between what is stated to be taught, what is actually taught, and the impact of both on learner knowledge and behavior. Research from Virginia Tech University and Middle East Technical University in Turkey states that the content has to be related to the field of study and be relevant to principles and standards (Yilmax and Cetinkaya, 2002 - 10, pg 81)

• Assessments should be learning activities themselves.
• The type of assessment determines how the student studies (and how much).
• Students are more likely to use “surface learning approaches” (recall) to study for multiple choice exams, but apply more meaningful learning methods when preparing for essay assignments.

Rote memory is purported to be lesser of an effort than elaborative rehearsal when facilitating meaningful learning (J.E. Ormrod, 2008). As with the authors of this report, the writer supports the opinion that higher-order thinking requires assessment measures beyond simple recall of facts.

• Assessments should have clear purpose and vary in design
• Design assessments to reflect collective information of course content.
• Assessments monitor student progress better when they are a part of a continuing process and not done sporadically
• Students need help in setting realistic goals that meaningful, attainable, and based on their experience
• Students respond to better when assessments are more engaging, interesting, and applicable

Regardless of what type assessment is being designed, much of the literature supports a design strategy that employs definitive methods of design principles. Some educators suggest creating an even more structured rubric such as individual electronic portfolios for students to assess their own work. The portfolios contain students’ coursework or independent studies brought together in the electronic environment (Yilmax and Cetinkaya, 2002 -10 pg 80). It is noted also that the design principles adhere to theoretical principles of curriculum development.

In the opinion of this writer, any review of the literature relative to assessing student performance in the distance learning environment would be remiss if it did not include what some educators would probably say is the benchmark in educational evaluation—Kirpatrick’s Four Levels of Evaluation. Aleckson and Martinelli apply Kirkpatrick’s Four Levels of Evaluation to an asynchronous distance learning environment.

Level One (learner’s reactions) measures the reactions of students to the program in which they are enrolled (Aleckson and Martinelli, 2005 -7 pg 1) . It attempts to answer questions related to relevancy and likability of the program. In an online environment, automated internet survey tools can collect data for level one evaluation. Positive reactions may not necessarily guarantee the transfer of knowledge through meaningful learning, but negative reactions more or less reduces the likelihood of either.

Level two evaluation (learning) now attempts to assess the extent students have advanced in skills, knowledge, or attitude. It can be computer-generated and should be done before, during, and after instruction. (Aleckson and Martinelli, 2005 -7 pg 1) In a web-based delivery environment, instructional designers could provide computer-generated pretests (to identify areas in which students already meet the standard or areas in which students need extra instruction) and in-course evaluation instruments.

Level Three (transfer) evaluation measures job performance and application of information learned (Aleckson and Martinelli, 2005 -7 pg 1). In the web-based environment, Internet surveys can collect feedback from managers about whether the student can apply course knowledge to job tasks. Formal survey instruments, linking specific learning objectives to job tasks, could survey supervisors about the students’ on-the-job performance. Some educational institutions may even consider conducting face-to-face evaluations, VTCs, or telephone interviews and focus groups. In those instances, this writer has found that the probability of collecting richer, more revealing feedback rises.

Although more appropriate for evaluating the “program” and not student performance, level four measurement is a direct result of having improved student performance through the first three levels. Level four measures the impact of training on organizational objectives. According to Aleckson and Martinelli, “Because of the expense and time, evaluating on this level is rarely done at the single eCourse level.” Evaluators are looking for such tangible results as increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment. From a business and organizational perspective, this kind of high-level evaluation occurs as part of accreditation reviews, organizational effectiveness studies, and the like.

Concerned, too, about student performance, the American Association for Higher Education (AAHE), a national organization dedicated to improving the quality of higher education in the United States, puts forth nine principles that it believes to be best practices for assessing student learning (AAHE, 2001). Although these principles are not specific to distance learning, they do have specific implications for evaluating students in distance learning environments:

• The assessment of student learning begins with educational values.
• Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.
• Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
• Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
• Assessment works best when it is ongoing not episodic.
• Assessment fosters wider improvement when representatives from across the educational community are involved.
• Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
• Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
• Through assessment, educators meet responsibilities to students and to the public (AAHE, 2002).

So popular are these best practices, in her research, this writer discovered them used by a number of universities and academic assessment offices. Principles one and seven are especially worthy of further discussion. AAHE advocates that assessment should be driven by the educational mission and values. This principle suggests that assessment should be more closely connected to on-the-job performance, a mostly agreed upon factor when assessing adult learner outcomes. AAHE further declares that assessment should address issues that people really care about. It “starts with the questions of decision-makers…[and] involves them in the gathering and interpreting of data.” This principle suggests that the assessment process should answer questions stakeholders have about students. Assessment processes should be particularly sensitive to the concerns of teachers, administrators, and school and trustee board officials. They should be involved in determining how to assess students and interpreting assessment data.

Regardless of how well designers of curriculum devise evaluation assessments and what structured methods, tools are best practices they use, students still play a crucial role in their own performance. While instructors can design the best assessments, learning is still a task online students have chosen to partake in and therefore must hold some accountability in their own outcome. Therefore, there is a need to consider the process of learning as it relates to types of students (Stansfield, McLellan, & Connelly, 2003). Educators cannot ignore this crucial factor. Stansfield, McLellan, and Connelly identify three approaches to learning by which students approach their own learning process: (a) deep, (b) surface, and (c) strategic. In the deep approach, students take on learning as a personal development and desire(Stansfield, McLellan, & Connelly, 2003). These students are interested in the learning, and their behavior changes are evident. Surface approach students are there to fill in a square (Stansfield, McLellan, & Connelly, 2003); they simply engage in rote memory to fulfill examination requirements. Finally, strategic learners establish a plan that leads to obtaining the best grades possible; their goal is master the assessments (Stansfield, McLellan, & Connelly, 2003).(2,pg 174) Each of these types of students has established his or her own process of learning and each process can influence learner outcomes. In the distance learning environment, it appears that learners determine their performance; teachers only assess the participation.

Principles of Good Practice for Assessing Student Learning. American Association for Higher Education: 1996. Retrieved January 18, 2008, from http://www.fctel.uncc.edu/pedagogy/assessment/9Principles.html

Kirkpatrick’s New Online Approach to Evaluating Asynchronous eCourses.” Proceedings from the 20th Annual Conference on Distance Teaching and Learning. Jon Aleckson and Julie Martinelli. University of Wisconsin (2005). http://www.uwex.edu/disted/conference/Resource_library/proceedings/04_1366.pdf

barrier meven argue that it is not just flexibility in learner access that is important but also interactivity—interactivity between facilitators and students, and between students and other students. [In discussing the importance of interactivity, Additionally, facilitator-learner rapport is particularly important in the online learning environment because there is a perceived positive impact on student performance and satisfaction (12, pg 1) Other research, however, (Stansfield #2) disputes this argument, saying that the quality of instruction is far superior to synchronous instruction that involves person to person interaction (2, pg 175)

Additionally, facilitators who teach several online classes or teach one or two from semester to semester must not confuse any of the classes with mixed, outdated, ambiguous or unrelated class instructions. Traditional-aged students may not take note, but adults will. Each of these principles is important for reaching the affective domain via the e-Information Highway, which, in turn, could impact motivation and subsequently motivation.

Technology is here to stay. The continued and extensive and powerful growth of the World Wide Web has affected the educational world forever. Instructional designers and facilitators of learning are called upon to be more creative, versatile, and adaptable than ever. In doing so, they must not

All of the essays in this section were written by students and then submitted to us to publish and help others. Thanks to all of the students who have submitted their essays to us. You should not hand in our essays as your own. We do not condone plagiarism! If you need custom essays on your exact essay questions, then have a look at our essay writing service.

Sign up and be the first to receive our latest offers: