In spite of the burgeoning literature in respect of online learning, there appears to be uncertainty about the effectiveness of this teaching and learning approach compared to traditional classroom methods. This case study of the 2007 cohort of Durban University of Technology students enrolled in its introductory microeconomics course examines their use of the online economics classroom (one element of the blended instructional approach employed), both in terms of duration and pattern, with a view to establishing whether online activity is associated with different levels of academic performance. The study, using linear regression analysis, finds that students' academic performance is significantly correlated with both their academic ability and their patterns of usage of the online classroom.
Key words: economics performance, blended learning, online learning, synchronous learning, asynchronous learning
The historically high failure rate of South African first-year economics students, as well as the consequences thereof, is well documented (Van der Merwe 2006 and 2007, Horn and Jansen 2009). Smith (2009) observes that various academic interventions such as parallel courses, bridging courses, extra tutorials and other special courses have been widely implemented in South African higher education institutions over the past 25 years or so with the objective of improving students' general academic performance. He notes, however, that relatively little research- both locally and internationally - has focused on the efficacy of these interventions.
An alternative, if not supplementary, approach to boosting students' academic performance is to adapt the mode of instruction. The advent of the internet, together with the increasing adoption of ever improving instructional technologies, has prompted a new emphasis on online education and training (Bartley and Golek, 2004). This development creates the opportunity for educators to move beyond the traditional face-to-face classroom and chalk and talk approaches to experiment with various mixes or "blends" of teaching and learning styles. Again, however, it seems that - at least as far as the online learning dimension of these innovative instructional styles is concerned - there is uncertainty about the depth of learning achieved (Bali, El-Lozy and Thompson, 2007). Bartley and Golek (2004) lament the lack of conclusive research in respect of the effectiveness of online education. As far back as 1999 Vachris (1999), and more recently Van der Merwe (2007), among others, encouraged closer study of performance issues related to online instruction.
This paper is a response to scholars' various invitations to examine the strength of association between academic performance and online instruction more thoroughly. Specifically, the study examines whether students' employment of the online economics classroom (one element of the blended instructional approach employed), both in terms of frequency and pattern is significantly associated with different levels of academic performance. Section 2 offers a brief review of the literature, Section 3 describes the online classroom, Section 4 discusses the study's research design, presentation of the data and analysis follow in Section 5 and Section 6 concludes.
2.0 Overview of the literature
2.1 Factors implicated in economics performance
Various factors have been identified as potential determinants of academic performance in economics. These include - in no particular order - age, gender, mathematical ability, English language proficiency, class attendance and pedagogic interventions (supplementary courses and materials). Table 1 reports correlations observed between these variables and academic achievement in economics for a sample of South African studies.
Table 1: Factors influencing economics academic performance.
Relationship between economics achievement and...(variable)
Identified correlation with economics achievement
Older students generally perform better than younger students (Parker, 2006).
No significant relationship detected between student age and economics achievement (Van der Merwe, 2006).
Males generally perform better than females in multiple choice assessments (Van Walbeek 2004, Parker 2006 and Horn and Jansen 2009).
No significant relationship identified between gender and economics achievement (Van der Merwe, 2006).
Robust and positive relationship between economic performance and mathematics scores (Edwards 2000, Smith 2004, Van Walbeek 2004, Horn and Jansen 2009).
English language proficiency
High school English language performance is not associated with university economics performance (Van Walbeek, 2004).
English as a home language is significantly associated with economics performance (Edwards 2000, Smith 2004).
English language verbal proficiency is a significant predictor of success in introductory microeconomics (Parker, 2006)
Lecture and tutorial attendance contribute positively to academic performance (Horn and Jansen, 2009).
Special supplementary modules and tutorials impact positively on students' performance (Smith, 2009)
Pedagogic devices that, notably, seem to have enjoyed comparatively little attention in the field of economics instruction are those of online and blended learning (Arbaugh, Godfrey, Johnson, Pollack, Niendorf and Wresch, 2009).
2.2 The concept of blended learning
Blended learning has been variously defined as the mixing of instructional modalities and methods (Graham, 2004). Carman (2005) identifies five elements of a blended learning process. These include live events, self-paced learning, collaboration, assessment and the availability of performance support materials. Graham (2004) considers the combination of online and face-to-face instruction, in particular, to best reflect the historical emergence of blended learning systems.
2.3 Weighing the evidence in support of online learning
It is by no means a settled matter that online learning (even as an element of a multimedia/blended learning technique) automatically translates into improved performance (Astleitner and Wiesner, 2004). Several studies have found, on balance, that there are no significant differences in academic performance when comparing traditional face-to-face classroom instruction with an online mode of delivery (Vachris 1999, Anon. 2008). In fact, the literature records findings indicating that learners who employed only online learning fared worse in terms of academic achievement than their peers who received instruction only in a traditional classroom setting ((Arbaugh et al, 2009, Molae 2007, Vachris 1999, Karr, Weck, Sunal and Cook 2003). An intriguing nuance of some of these studies, however, is their common conclusion that blending online and traditional modes of instruction is associated with superior academic performance compared to that produced by exclusively online or traditional classroom instruction (Molae 2007, Karr, Weck, Sunal and Cook 2003).
Various studies report favourably on the expected direct link between online learning and academic performance. St Clair (2009), for instance, states that online economics grades were generally higher than what he would have expected using a traditional classroom approach. Similarly, Bali et al (2007) find that an innovative instructional style with an online component produced better results than the traditional equivalent of the course. Snipes (2005) concludes that the US Navy's adoption in 2004 of a blended learning training approach resulted in, among other benefits, a 44 percent improvement in knowledge retention. Oellerman (2009) reports improved pass rates for various management courses following her employment of certain online assessment tools as part of her instructional technique.
A common deficiency of studies of the effects of pedagogic interventions on academic performance is that they may not account for the potentially large number of intervening variables and so their results may not be interpreted accurately or even correctly (Bali et al., 2007). Several scholars have highlighted the need to study the dynamics associated with online learning more robustly (Bali et al. 2007, Alstete and Beutell 2004, Van der Merwe 2006). Thus what effect might the following variables, for instance, have on academic performance in a blended learning environment (incorporating an online component): gender, ethnic/cultural background, ability, language proficiency, previous online experience, learning styles, instructor bias, learner motivation and so forth?
While it is unlikely that each potentially confounding variable can be considered in every analysis of the link between instructional approach and performance, at least some effort to this end should be expended for the sake of greater analytical rigour. Some studies have moved in this direction. Thompson (2000), having controlled for prior ability in economics and mathematical ability, finds that repeating students who accessed the relevant computer aided learning modules improved their learning outcomes as measured by their performance on examination questions. However, for non-repeating students this relationship was found to be statistically insignificant.
A meta-analysis of the literature published between 1996 and 2008 prepared for the United States Department of Education (2009) suggests that - on average - students in online learning conditions performed better than those receiving face-to-face instruction. They note that the performance dividend extracted by online learning was enhanced when some of its elements were blended with components of traditional classroom teaching. They caution, however, that these blended instructional approaches invariably included supplementary learning time and instructional elements not received by learners in traditional face-to-face settings. It follows that one cannot confidently attribute the positive effects of blended learning solely to the media employed. Alstete and Beutell (2004), for instance, find that online course grades, controlling for gender and age, are significantly positively correlated with online course activity but not with previous academic performance achieved in synchronous teaching and learning environments. They speculate that this result points to the possibility that online course performance is a unique type of academic aptitude that is not well understood. Alstete and Beutell (2004) conclude that, in the case of their undergraduate sample, neither age nor gender is related to online course performance. In the case of their postgraduate sample, though, they find that women enjoy superior online grades over men.
Studies such as those reviewed here which set out to explore the association between academic performance and pedagogic interventions often, somewhat gratuitously, rely on the assumption (whether explicit or implicit) that such interventions motivate learners. The appealing intuition that such endeavors depend on is that improved academic performances are attributable to improved learner motivation. In fact ,very few scholarly works have attempted to explore rigorously the link between some metric of learner motivation and consequent academic performance.
Thus it is that studies making specific claims that systematically designed technology-mediated instructional strategies can boost motivation and performance (Gabrielle 2003, Rienties and Woltjers 2004) have mostly not been tested. Van der Merwe (2006) finds no evidence to support this proposition. However, his results may have been tainted by his reliance on self-reported data in respect of the motivation levels of respondents. Doubts have been cast on the validity and reliability of such data (Alstete and Beutell, 2004). Song and Keller (2001) report somewhat stronger empirical evidence to support the hypothesis that improved learner motivation is significantly associated with increased achievement. In particular, they find that learners using motivationally adaptive (geared strictly to the indicated needs of learners) computer aided instruction (CAI) performed significantly better than those using motivationally saturated (indiscriminate provision of motivational tactics) or motivationally minimized (stripped of motivational tactics) CAI.
This case study, in examining the impact of online learning on academic achievement in a first year general economics course taught both synchronously in a traditional classroom setting and also asynchronously, indirectly probes the link between learner motivation and performance. Its method in setting out to draw some inference about the nature of this relationship is to analyse the online learning and performance nexus which is the primary focus of this study. Based on the work of Song and Keller (2001) it would not be unreasonable to speculate that flux in grade achievements associated with variable use of the economics online classroom most likely does reflect different levels of learner motivation.
Following a process of approximation, the blended learning style employed in this case study falls between Song and Keller's motivationally saturated CAI and motivationally minimized CAI. Thus, while it does not claim to offer an optimal, nor adaptive, array of motivational tactics, it does create a learning dimension that is simply not possible in a purely traditional chalk-and-talk approach. In this particular instructional blend, online learning functions such as threaded discussions, self-assessments with immediate feedback and interactive tutorials and lessons punctuated with reporting on current economic affairs, among others, are likely to attract learners' attention, demonstrate the relevance of course material more obviously and so promote learner confidence and satisfaction. A learning environment of this nature can reasonably be expected to be motivating both for learners who commence the course already motivated and also for those whose motivation must be coaxed into life. It is in this sense that learner motivation may be implicated with some confidence in variability in economics grade achievement associated with usage frequency and pattern of the economics online classroom.
3. A description of the blend
Economics 1 at DUT is offered as a compulsory minor course for candidates majoring in either three-year accounting or management degrees. Most students take Economics 1 in their first year of tertiary study with the exception of students enrolled in the Management Studies programme who take it in their second year. Economics 1 at DUT, in common with other tertiary institutions, has a significant failure rate so that a large proportion of students tend to repeat the course. The Economics 1 course comprises two modules: microeconomics offered in the first semester and macroeconomics in the second semester. Students may not register for macroeconomics without first having attempted microeconomics. Both modules are is taught in a traditional face-to-face lecture situation as well as online using the Blackboard Learning Management System interface. Learners are expected to attend scheduled traditional classroom lectures and, outside of formal classes, to follow these up by asynchronously consolidating their learning, at their own pace, in the online economics classroom.
Available online resources comprise electronic textbooks, comprehensive course notes, interactive lessons and tutorials, marking guides for tutorials, discussion topics and assessment/self assessment tools which include questions from past test and examination papers. Traditional classroom lectures are used to discuss sufficient theory so as to synchronously tackle exemplar exercises, problems and tasks that are designed to prepare candidates to achieve competence with respect to the prescribed learning outcomes and assessment criteria. Students are expected to compile their own detailed notes which can be checked against the notes supplied online. In addition, students are encouraged to attempt the online tutorials, exercises, tasks, past tests/examinations and quizzes with a view to consolidating content and concepts introduced in the physical classroom. The online quizzes are formative assessments and may be attempted multiple times. They are automatically marked and also offer limited feedback. Marking guides are provided online for all other assessments with the expectation that students will mark their own work. Unfortunately, however, many students regard the online economics classroom as an "optional extra" and so either do not use it as recommended or do not use it at all. As with many first-year economics courses, the DUT introductory microeconomics assessments consist of primarily multiple choice items (70-85%) while the balance are of the short question variety.
On commencement of classes all candidates enrolled in the microeconomics module are required to attend an orientation session in one of the institution's computer laboratories at which they are introduced to the Blackboard Learning Management System interface. Students use this once-off introductory session to learn how to log into the online classroom and to gain familiarity with its layout and functions such as the chat, discussion, calendar and announcement tools and other facilities. The expectation is that candidates enrolled for the microeconomics course will, following their introduction to the online classroom, begin to make regular use of it to supplement their traditional classroom lectures. Given that DUT offers adequate access to computer laboratories for all of its students, this is not an unreasonable requirement.
This analysis is based on a case study of the 2007 cohort of DUT Riverside campus students enrolled for the microeconomics module of its introductory economics course. As such, its sampling frame comprises the total population of 250 students registered for the microeconomics module. The sample, following data cleaning, totals 174 students or cases (69.6% of the population). The data cleaning process focused on purging cases (outliers) where the number of hits on the online classroom deviated significantly from the mean. Thus if the number of hits per case was too few (<15) it is likely that the students in question had forgotten their passwords relatively early on and had started using friends' passes to access the online economics classroom which would then account for abnormally high (>250) numbers of hits per case.
By virtue of its relatively large size, the sample can reasonably be assumed to be sufficiently representative of the population. Thus, with respect to age, for example, the sample mean age of females is 23.3 yrs (population = 23.4 years) while that of males is 23.8 yrs (population = 23.8yrs). The sample proportions of first, second and third year-and-older students are also similar to that of the population (36.8%, 33.6% and 29.9% respectively) as are the sample gender proportions (sample males = 41.4%, population males = 42%).
The analysis was conducted using the Statistical Package for Social Sciences (SPSS) programme. Cases with missing relevant variables were excluded listwise as per its default setting. A brief descriptive analysis of the data is followed by an Ordinary Least Squares linear regression exercise to test for expected relationships between economics academic achievement and various possible factors.
The study employs the student's introductory microeconomics course mark (MicDP), as opposed to his/her final mark, as the metric of performance in terms of gauging academic achievement. The reason for this is to preserve the largest possible sample size and thus also to minimize the risk of selection bias. Ordinarily a student's final mark determines whether he/she passes the module/course. The final mark is a weighted average of the student's course mark and final examination mark. However, an institutional academic exclusion rule prevents the student from sitting the final examination if he or she has not scored a course mark of at least 40%. A significant number of students, in any given academic year, are thus prevented from scoring both a final examination mark and final mark. The strategy of using the course mark as the primary measure of students' academic performance ensures that the impact of online learning on achievement can be tested for more candidates and not only for those who might be predisposed to good performance for reasons unrelated to their use of the online facility.
A danger that selection bias thus poses is that if the sampled individuals using the online economics classroom possess superior ability or more motivation than other members of the population who also use the online facility then some, if not all, of any improvement in academic performance of the sampled individuals would have been achieved anyway. In such a case, one cannot correctly conclude that online activity alone is significantly associated with improved performance. This study employs various measures to reduce the risk of sample selection bias. These include, as indicated, a large sample relative to the population and the use of the microeconomics course mark as the metric of academic performance which measure has the effect of including in the sample students from a wider range of academic ability than would be the case if the examination of final marks were used. In addition, various control variables are employed in the linear regression analysis in order to identify robust correlations.
5. Findings and discussion
5.1 Descriptive analysis
DUT student records were used to secure data pertaining to gender, age, academic record in the final year of high school, whether students were repeating the course and academic performance in introductory microeconomics at DUT in 2007. Additional student specific data were harvested from the Blackboard system which automatically logs online classroom activity. The two sets of data were subsequently reconciled and incorporated into an SPSS database.
Individuals included in the sample are predominantly second language English speakers (66.9%) and most (89%) took mathematics at some level in their final year of high school. A majority also took economics as a high school major (68.9%) at an advanced level/higher grade (57%). That economics is something of a problem subject for students is evident from the consideration that most students as represented by the sample were repeating the course (54%). This is also true of the population (52.4%).
Table 2 displays the descriptive statistics of students' online activity, the pedagogic intervention. Rounding to the nearest whole number, the mean number of hits (halfhits) per student over a roughly 4-5 month period leading up to the final examination in late May or early June was 91. Over the same period the mean number of specially posted online articles read (Read) was 3 with a negligible mean number of responses in terms of posting (Posted) responses to these items using the online discussion tool. The mean number of online multiple choice quizzes (around 80% of DUT economics assessments comprise multiple choice items) attempted (tASS) was roughly 8 and the mean total number of marks accumulated from these formative assessments was 20 (OnlineMicperfT). The average duration (durationhalf) of online activity was about 5 months.
Table 2: Descriptive statistics
Valid N (listwise)
4.2 Regression analysis
Using multiple regression analysis it is possible to test whether a set of independent variables explains the expected variance in the dependent variable, in this case, academic performance in introductory microeconomics. Drawing on the literature, likely predictors of economics performance were selected for inclusion in a significant linear regression model (Fâ‚‡, â‚†â‚ = 6.051, p < 0.0005 and Adjusted R square = 0.342). This model specified performance ("MicDP") as a function of gender (dummy variable "genderscale", male = 0/female = 1), high school mathematics marks ("mathmarksct2"), high school English marks ("engmarksct2"), high school economics marks ("econmarksct2"), duration of online activity in months ("durationhalf"), total number of online quizzes attempted ("tASS") and total marks accumulated from completed online quizzes ("OnlineMicperfT"). The last three variables represent various dimensions of the online intervention. Student age was excluded from the model on the basis that it is indicated as an insignificant factor and its inclusion reduces the variance in economics performance that it can potentially accounts for.
Table 3 sets out the regression model's coefficients and Table 4 its descriptive statistics.
Table 3: Coefficients
Table 4: Descriptive statistics
Mathematical proficiency (mathmarksct2) and economics (econmarksct2) and English performance (engmarksct2) in the final year of high school are significantly (p< 5%) and directly associated with microeconomics performance (MicDP) at first year university level. This finding largely accords with those reported in the literature. Similarly the duration of use of the online classroom facility (Duration), number of online assessments attempted (tASS) and performance in these assessments (OnlineMicperfT) are significantly associated with microeconomics achievement (p<0.05). As one might expect, the longer students use the online facility and the better their performance in online assessments (which are voluntary) the better their performance in microeconomics. The unexpected indicated negative association between microeconomics performance and total number of assessments attempted may be a consequence of weak but motivated students attempting each assessment multiple times thus increasing the strength of the inverse relationship.
Gender and age are not significantly associated with microeconomics performance, a result that finds some resonance in the literature. Whether students are first or second language English speakers (englishscale) is not indicated as being significantly associated with economics performance but this effect may have been captured by students' English marks. Similarly, the indicated insignificant relationship between higher/standard grade economics at school level may have been expressed in the direct relationship between school economics marks and microeconomics performance. That total number of hits per student on the online economics classroom was also not identified as a significant predictor of change in economics performance is not surprising given that students may have accessed the facility for various purposes such as checking on notes or announcements, reading articles, engaging in chat or discussions, marking tutorial work and doing online quizzes. One would anticipate that certain activities, such as completed assessments or tutorials, might carry a greater performance dividend than others such as reading or taking notes.
The key finding of this study, thus, is that students' use of the online component of the blended teaching and learning approach employed to deliver the microeconomics module, both in terms of duration and pattern, is significantly and directly associated with student achievement. This finding remains robust while controlling for gender, age, English fluency, mathematical ability and high school economics competence/experience.
While this study thus offers evidence of the potential performance yield of investment in online learning it is worth considering the nature of the mechanism that delivers effective learning through online delivery of subject content. A revelation might be approached by a process of deduction and by drawing on the literature. Thus a probable explanation for different grades achieved by two learners of roughly the same ability is that they may be differently motivated to engage with the subject or topic. Hence students who do not make sufficient use of the online economics classroom are more likely than those who do to lack the necessary motivation to immerse themselves in the subject matter of economics. On the other hand those who, in addition to traditional classroom lectures, avail themselves of the online economics facility gain an added learning dimension and greater scope for contextualizing and consolidating subject content. It follows that students who regularly use both the traditional classroom and the online economics facility must, all things being equal, gain a performance advantage over those who do not.
5. Limitations of the study
The case research design employed in this study implies that its results may be generalized with a measure of confidence only to the 2007 population of DUT students enrolled in introductory microeconomics. Future studies could profitably extend this line of inquiry by studying larger student populations at both universities of technology and traditional universities not only enrolled in economics studies but also in other disciplines. A further limitation is its narrow definition of "performance" as gauged by a mean summative assessment mark. This device potentially fails to score achievement in other areas of learning. Finally, while the study attempted to limit the risk of selection bias by controlling for ability it failed to account, empirically, for the impact of learner motivation on performance. Future studies could examine the learner motivation-performance nexus more closely.
Against the background of allegations of a deficiency of conclusive research in respect of the effectiveness of online education this case study set out to test the strength of association between academic performance and online instruction (as a significant aspect of a blended learning approach). The threat of selection bias was managed by creating a relatively large sample and also by controlling for gender, age and academic ability as proxied by high school English competence, mathematical proficiency and economics experience/knowledge/ability.
The regression model indicated that student characteristics such as mathematical, English language and economics ability are significant predictors of microeconomics performance at university level. Controlling for gender, age, mathematical ability, English language fluency and high school economics achievement students' performance in introductory microeconomics is directly and significantly associated with the total value of marks accumulated for completed online assessments and duration of online activity. This finding has important implications. Firstly, creating space online for assessments additional to those conducted in traditional classroom environments is likely to benefit students. Secondly, that the duration of online engagement is significantly related to performance, and not frequency (number of hits), suggests that more thought should be given to the design of online learning environments to attract longer visits. This may, for example, entail drawing students into activities, interactive tasks and online discussions/forums as opposed to merely employing the online medium as a repository for notes and assignment/test solutions. Thus the way in which the online medium is used (pattern) is likely to impact on learning achievement.
While this case study provides reasonably strong evidence that the online dimension of a learning blend can potentially deliver improved academic performance it was more difficult to explain the mechanism by which this occurs. It was argued that the online aspect of a bended learning approach offers an additional dimension in which learner motivation may be sparked. Given the ubiquitous reach of the internet and ever expanding access to it, it seems a logical forum in which to demonstrate the relevance and currency of economic theory. Indeed, this is likely to become students' expectation and the delivery of economics content needs to adapt accordingly if the discipline is to escape its bleak reputation and grow in stature. As in life, the key to academic success is motivation. The online component of a blended teaching style, if reasonably well designed, offers that much more scope to ignite learner motivation.