The consistency of school effects: An analysis of the 2006 PISA data

Published:

Introduction

School effectiveness is an important and prominent topic in educational research. Many large scale studies (eg. PISA, TIMMS, DELPHI,) have been conducted to investigate and compare the effectiveness of different educational systems. In general results on mathematics or language tests are used as a measure of school effectiveness. (Scheerens, 2000). Previous research from Van Damme, Van Landeghem, De Fraine, Opdenakker and Onghena (2004) stated that the ranking of schools can differ based on the chosen effectiveness criteria. Schools that perform well for one effectiveness criteria do not necessarily perform well for other effectiveness criteria (De Maeyer et al., 2008, Brookover, Beady, Flood, Scheitzer, & Wisenbacker, 1979; Knuver & Brandsma, 1993; Mortimore, Sammons, Stoll, Lewis & Ecob, 1988).

Many studies addressed the consistency of school effects, but some domains need to be further examined due to a lack of empirical evidence. At first the results concerning cognitive and non-cognitive criteria are inconclusive and conflicting. There is no clear indication that schools that perform well for mathematics or language necessarily perform well for non-cognitive measures (Brandsma, 1993; Mandeville & Anderson, 1987; Mortimore et al., 1988; Reynolds, 1976; Rutter, Maughan, Mortimore, Ouston & Smith, 1979, Gray, Jesson & Sime, 1983). Secondly in previous research much attention was given to classical effectiveness measures (e.g. mathematics and language) but little research has focused on other criteria. Thirdly, nothing is known about the situation in Flanders.

The consistency issue

Lady using a tablet
Lady using a tablet

Professional

Essay Writers

Lady Using Tablet

Get your grade
or your money back

using our Essay Writing Service!

Essay Writing Service

Most previous research regarding the consistency of school effects was carried out in the United States, United Kingdom and the Netherlands. Studies in these countries have shown that there exists an average consistency between cognitive measures. Brandsma and Knuver (1989) study results showed a correlation .72 using data from Dutch primary schools. In 1994 Yelton, Miller and Ruscoe (1994) reported a mediate level of consistency between mathematics and reading. In a review study Bosker and Scheerens (1989) a correlation of .72 was found. These similar results across the United States (Mandeville & Anderson, 1987), the UK (Mortimore et al., 1988) and the Netherlands (Scheerens & Bosker, 1989) make it possible to conclude that the correlations between different cognitive effectiveness in primary education are mediate positive.

However in secondary education the consistency seems somewhat lower than in primary education and the correlations between subjects are between .40 and .50 (Cuttance, 1987; Fitz-Gibbon, 1991; Nuttal et al., 1992; Smith & Tomilson, 1989; Thomas & Nuttal , 1993; Thomas et al., 1993; Wilms & Raudenbush, 1989; Sammons, Mortimore & Thomas, 1993). Scheerens (1999) states that this is due to the fact that in secondary education there is not one teacher for all courses, but specific teachers for certain subjects.

The consistency within subjects seems somewhat larger than the consistency across courses. (Mandeville & Anderson, 1987; Crone et al., 1994). School effects within mathematics are somewhat more consistent than within language. This is probably due to the fact that pupils' mathematical knowledge is mostly taught at school. When pupils learn a language this is strongly influenced by external factors (Mandeville & Anderson, 1987).

The consistency between the school effects on cognitive and non cognitive criteria is inconclusive. Some studies indicate that the two are weakly positively related. Others think they are completely independent and others even found weak negative associations (Brookover, et al, 1979; Knuver & Brandsma, 1993; Rutter et al., 1979; Smyth, 1999). Reynolds (1976) and Rutter et al. (1979) found rather strong correlations between schools that are effective for academic skills and their results on non-academical effectiveness criteria. Rutter et al. (1979) concluded: "On the whole, schools which have high levels of attendance and good behaviour also tend to have high levels of exam success". Research by Gray, Jesson en Sime (1983) indicates that scores on "appreciation of the school" and "attendance" are partly independent from the academic results. In the Netherlands Knuver en Brandsma (1993) investigated the relationship between school effects on a variety of affective variables (attitudes towards language and mathematics, academically self concept, well being at school and motivation to perform) on language and mathematics. The correlations found, were very small.

It is possible to conclude that previous research mostly focused on cognitive effectiveness measures (Cuttance, 1987; Fitz-Gibbon, 1991; Nuttal et al., 1992; Smith & Tomilson, 1989; Thomas & Nuttal , 1993; Thomas et al., 1993; Wilms & Raudenbush, 1989; Sammons, Mortimore & Thomas, 1993. Only few implemented both cognitive and non cognitive effectiveness measures. (Knuver & Brandsma, 1993; Mandeville & Anderson, 1986; Mortimore et al., 1988; Reynolds, 1976; Rutter et al., 1979). Because schools also need to focus on the social development of pupils it is important that this is part of future research (Sammons, 1999).

Research questions

Lady using a tablet
Lady using a tablet

Comprehensive

Writing Services

Lady Using Tablet

Plagiarism-free
Always on Time

Marked to Standard

Order Now

In this article we will focus on three questions. First of all we will focus on the consistency of school effects between cognitive output criteria. Do schools that perform well for one cognitive effectiveness criteria also perform well for other cognitive output criteria? Secondly the focus is within science. Do schools that perform well for one scientific ability also perform well for other scientific abilities? As previous research concerning cognitive and non-cognitive output criteria is conclusive and conflicting. We will also examine if a school that performs well for a cognitive output criteria also performs well for non-cognitive output criteria. As schools can't be held accountable for the students' intake characteristics we will estimate both a gross and a net model. In the net model we will control for students' intake characteristics.

Methodology

Data

This study reanalyzes data from the PISA 2006 study. PISA is a three yearly survey organized by the Organization for Economic Co-operation and Development (OECD). In 2006, the survey focused on science. In this study only the Flemish data were derived from the complete dataset. The dataset consisted out of 4889 pupils in 154 schools..

Variables

Cognitive effectiveness criteria

To measure the consistency of the school effects between cognitive effectiveness criteria we will check the consistency between: mathematical literacy, reading literacy and scientific literacy. In previous studies mathematics and language were used. There was little focus on the consistency between science and other output measures.

Mathematical literacy

Mathematical literacy is defined broader than only solving classical mathematics exercises: it is defined as follows::Mathematical literacy is an individual's capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgements and to use and engage with mathematics in ways that meet the needs of that individual's life as a constructive, concerned and reflective citizen." (De Meyer & Pauly, 2007; OECD, 2007).

Reading literacy

Reading literacy emphasizes the skills that pupil need to use written information in real contexts. It is defined: "Reading literacy is understanding, using and reflecting on written texts, in order to achieve one's goals, to develop one's knowledge and potential and to participate in society." (De Meyer & Pauly, 2007; OECD, 2007).

Scientific literacy

For the purposes of PISA 2006, scientific literacy refers to an individual's:

• Scientific knowledge and use of that knowledge to identify questions, acquire new

knowledge, explain scientific phenomena and draw evidence-based conclusions about

science-related issues

• Understanding of the characteristic features of science as a form of human knowledge

and enquiry

• Awareness of how science and technology shape our material, intellectual, and cultural

environments

• Willingness to engage in science-related issues and with the ideas of science, as a reflective

citizen

Scientific abilities

The PISA survey in 2006 focused on science. Therefore several scientific abilities will be included to see if schools that perform well across these abbilities.

Indicating scientific issues

It is important to be able to distinguish scientific issues and content from other forms of issues. The competence 'identifying scientific issues' for example includes recognizing key features of a scientific investigation: what things should be compared, what variables should be changed or controlled, what additional information is needed, or what action should be taken so that relevant data can be collected. (De Meyer & Pauly, 2007; OECD, 2007).

Explaining scientific phenomena

Students demonstrate the competency "explaining phenomena scientifically" by applying appropriate knowledge of science in a given situation. The competency includes describing or interpreting phenomena and predicting changes, and may involve recognizing or identifying appropriate descriptions, explanations, and predictions. (De Meyer & Pauly, 2007; OECD, 2007).

Using scientific evidence

Using scientific evidence includes accessing scientific information and producing arguments and conclusions based on scientific evidence (Kuhn, 1992; Osborne, erduran, Simon and monk, 2001). the competency may also involve: selecting from alternative conclusions in relation to evidence; giving reasons for or against a given conclusion in terms of the process by which the conclusion was derived from the data provided; and identifying the assumptions made in reaching a conclusion. reflecting on the societal implications of scientific or technological developments is another aspect of this competency. Students may be required to express their evidence and decisions, through their own words, diagrams or other representations as appropriate, to a specified audience. In short, students should be able to present clear and logical connections between evidence and conclusions or decisions. (De Meyer & Pauly, 2007; OECD, 2007).

Non-cognitive output criteria

Lady using a tablet
Lady using a tablet

This Essay is

a Student's Work

Lady Using Tablet

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Examples of our work

Instrumental motivation for science

Due to the observed deficit of students in the Flemish higher education it is important to see if this trend is going to continue. The instrumental motivation is an important predictor of course selection, career choices and performance (Eccles, 1994; Eccles & Wigfield, 1995; Wigfield et al., 1998). In the survey instrumental motivation was measured by 5 questions. Students had to indicate if their idea was in line with the following question: "What I learn during physics is important for me because I need it for what I want to do later". Pupils could answer four categories: "Stongly agree", "agree", "don't agree" en " strongly don't agree" (De Meyer & Pauly, 2007; OECD, 2007).

Interest in science.

The interest in science was selected because research has indicated that a early interest in science can be a predictor for choosing a scientific course and/ or a career in the technological or scientific area The survey collected data about the involvement of students in social questions relating to science, their readiness to acquire scientific knowledge and skills; their recital for a career in a scientific area. The scale shows an alpha of .85

Control variables

Type of secondary education

From the original PISA dataset of Flanders three types of secondary education were selected, general secondary education, technical secondary education and vocational education. In the other categories only very few respondents were included. The amount of pupils in general secondary education equals 2296, in technical secondary education 1585 and in vocational secondary education 1008. Two dummy variables were in which general secondary education is used as the reference category

Language at home

The language at home of a pupil is implemented dummy coded. There are 3485 pupils in the dataset that have the same language as the test, 935 (21,7%) had another language.

Orign

The ethnical status is also a predictor of a student's score (Bosker & Hofman, 1994; Driessen, 1995; De Jong, 1989). Two dummy variables are implemented. There were 4572 native, 132 immigrants of the First generation and 143 immigrants of the second generation. Thus 94,3% of the respondents are native, 2,7% immigrant of the first generation and 2,7% of the second generation.

Gender

Gender is implemented as a dummy category; the dataset consisted out 2297 girls and 2592 boys. In other words 47% girls and 53% boys.

Year of secondary education

The previously acquired knowledge seems to have a considerable influence on the output criteria (Béguin, de Jong, Rekers-Mombarg & Bosker, 2000; Rekers-Mombarg et al., 2000; Kuyper & van der Werf, 2005). Within the PISA dataset there are no indicators of the amount of knowledge that was previously acquired by the pupils. However it is possible to include one variable which can be seen as an indicator of the student's previously acquired knowledge. Probably students whom already proceeded to a year which is higher than others, will probably perform better on the implemented output criteria. The year will be implemented so that schools with a lot of students in the 3rd year of secondary education have no disadvantage anymore. The 4th year of secondary education is the reference category, while other categories are 3rd and 5th year of secondary education.

ESCS

The social and economical status of a pupil is a strong predictor of pupils' scores (Bosker, 1990; Sammons, 1995; Van de Gaer, Van Damme, & De Munter, 2001; Engels, Aelterman, Schepens, & Deconinck, 2001). In the PISA survey also a cultural aspect was added.

Table 1: Descriptive statistics

n=

%

n=

%

Type of secondary education

Gender

General secondary education

2296

47

Boys

2592

53

Technical secondary education

1585

32,4

Girls

2297

47

Vocational secondary education

1008

20,6

Year

Origin

3th year

1076

22

Native

4572

94,3

4th year (ref.cat.)

3775

77,2

First gen. immigrants

132

2,7???

5th year

35

0,8

Second gen. immigrants

143

2,7???

Language at home

Same language

3485

28,3

Different language

935

21,7

Analysis

Multilevel analysis is the appropriate analysis technique. It is possible to opt for several univariate analyses or to use multivariate analyses. The use of a multivariate model has the advantage that it reduces type 1 errors and also has more statistical power, so that the chance for type 2 errors decreases (De Maeyer, Rymenans, Van Petegem & van den Bergh, 2004). When one wants to analyze the consistency between different output measures it is necessary to model a multivariate model. Another issue that needs to be addressed is that we both model net and brut effects. A brut school effect is uncorrected, thus it may strongly reflect the income population (De Fraine, Van Damme & Onghena, 2002). The results of pupils aren't only depending on the quality of the education they receive, but are also influenced by factors that are outside of the control of the school. Examples are the capacities of the pupil, the home situation,… Schools can't be hold responsible for the aspects that they can't influence (De Fraine et al., 2002). Therefore the net models correct for the pupils characteristics (De Fraine et al., 2002). In this way it is possible to see how it's going on the reality (brut effects) and on the other hand on a statistical honest basis (net effect)

Building the model

At first a basic model will be estimated to examine the intra class correlation between the two levels of variance. Secondly the control variables will be added to the gross model.

Yijk= (β1X1jk + ν1k + µ1jk) + (β2X2jk + ν 2k + µ2jk) + (β3X3jk. + ν3k + µ3jk)

Than the pupil characteristics will be added to the model:

Yijk= (β1X1jk + ν1k + µ1jk) + (β2X2jk + ν2k + µ2jk) + (β3X3jk + ν3k + µ3jk) + (β4X4i) + … + (β10X10i)

Results

Gross model

There is significant variance at the school level, for all the effectiveness criteria. In other words depending on the school a pupil visits, his or hers score on the used effectiveness criteria will be higher or lower than the mean score. The variance for cognitive effectiveness measures is much higher than the non-cognitive output criteria. This means that school scores differ much more on cognitive measures than for measures that capture non cognitive criteria. Accept for the variance we also estimated the covariance. Except for the covariance between 'interest in sciences' and 'mathematical literacy', and on the other hand 'interest in science and 'scientific literacy', all the other estimates are significant. The covariance is also transferred into a correlation coefficient which makes it easier to compare the results.

Between the cognitive effectiveness criteria very strong were correlations found, .96 for scientific literacy and mathematical literacy, .91 for mathematical literacy and reading and .913 between scientific literacy and reading. A Flemish school that performs well for one of these characteristics thus in general also performs for the other output measures. The same image applies within sciences. Schools that perform well for one of the scientific abilities will also score well on the other scientific abilities (.93 - .95). Between cognitive and non-cognitive output the correlation coefficients are much lower. The correlation coefficient between interest in science and science score is even not significant. Thus we find no empirical evidence that a school that does well for interest in science will also perform well for scientific literacy. However in these models we do not control for students intake. It's thus possible that the results represent the student population of the school.

Tabel 1: Parameter estimates of the variances and covariances at the school and student level and ICC. (Correlationcoefficients shown in bold, not significant variances or covariances underlined.)

Between cognitive measures

math.

sci.

read

School level

math..

.390 (.046)

.958

.912

sci.

.380 (.046)

.404 (.048)

.913

read

.349 (.043)

.356 ( .044)

.376 (.045

Pupil level

math.

.512 (.011)

.867

.712

sci.

.439 (.010)

.501 (.048)

.856

read

.354 (.043)

.421 (.044)

.483 (.045)

ICC

.432

.446

.437

Scientific abilities

explain

indicate

use

School level

explain

.369 (.044)

.945

.948

indicate

.345 (.043)

.372 (.044)

.930

use.

.369 (.045)

.370 (.045)

.411 (.049)

Pupil level

explain

.565 (.012)

.861

.906

indicate

.471 (.010)

.529 (.011)

.895

use

.473 (.010)

.452 (.010)

.482 (.010)

ICC

.395

.412

.460

Cognitive and non-cognitive

instrum.

interest.

math.

sci.

School level

instrum.

.069 (.012)

.414

.685

n.s.

interest.

.027 (.008)

.061 (.010)

.685

n.s

Wisk.

.112 (.019)

.026 (.016)

.390 (.046)

.958

Wet.

.115 (.019)

.031 (.016)

.390 (.046)

.376 (.045)

Pupil level

instrum

.938 (.021)

.323

.201

.213

IIW

.300 (.015)

.920 (.019)

n.s.

.06

math.

.139 (.011)

.002 (.010)

.512 (.011)

.867

sci.

.146 (.011)

.041 (.010)

.439 (.010)

.501 (.010)

ICC

.069

.062

.432

.446

math. = mathematical literacy instrum. = instrumental motivation for science

sci. = scientific literacy explain = explaining scientific phenomena

use. = using scientific evidence. indicate: indicating scientific phenomena interest. = interest in science ICC = Intra-class correlation coefficient

n.s or underlined = not significant.

Net models

Table 2 shows that the variation at the school level is still significant, after controlling for the students background characteristics. The previously determined differences between schools are thus not completely attributable to student characteristics (that were implemented in the model). As the variance has decreased it is possible to state that the student characteristics matter when examining the consistency of school effects.

The cognitive output criteria will be described first. The ICC decreased opposed to the previous model. In the net model it is .15 for mathematical literacy, .13 for scientific literacy and .16 for reading. This means that from the total variation in mathematical literacy scores 15% can be attributed to the school and 13% for scientific literacy en 16% for reading. This is somewhat higher than in previous research was found (Bosker & Witziers, 1996; Hill et al., 1995). Possibly this can be explained by the fact that we couldn't control for IQ an important student characteristic. As the correlations in the net model are lower, it is possible to say that the consistency of school effects can be partly attributed to the pupil population. The correlation between scientific literacy and mathematical literacy is the highest (.71), then between mathematical literacy and reading (.60) followed by scientific literacy and reading (.53). Probably this is because science and mathematics are taught at school, while languages are influence stronger by other contexts. (Mandeville & Anderson, 1987). Another possibility is the fact that the same teacher might teach science and mathematics, while for language subjects there is a different teacher. In previous research it was indicated that the teacher also had a significant influence on the consistency of the school effect (Scheerens & Bosker, 1997; Scheerens, 1999).

The results for scientific abilities are for a big part due to schools intake characteristics. The ICC show that 13% for explaining scientific phenomena, 15% for indicating scientific subjects and 16% for using scientific proof is situated at the school level. The correlation coefficients have dropped, but they stay high indicating that Flemish schools that perform well for one scientific ability also perform well for others.

The variance for the different non-cognitive output criteria equals .04 and .05 which is very low; this could also be found in the brut model. The ICC is .04 for instrumental motivation and .05 for interest in science. Thus 4% of the variation in instrumental motivation for science and 5% for interest in science are situated at the school level. The school factor thus has only a small influence on these output criteria, in every case smaller than the cognitive criteria. The correlation coefficient is only significant between interest in science and mathematical literacy This correlation coefficient is with .05 very small and therefore the chance that a school performs well for mathematical literacy also performs well for interest in science is small. In figure 2 the covariance between the residuals at the school level are pictured, in this visual overview it is possible to see that the coherence between cognitive criteria (mathematical and scientific literacy) is higher than between non cognitive and cognitive measures.

Tabel 2: Parameter schattingen voor het fixed en random gedeelte na toevoeging van de leerlingenkenmerken. (OV1)

Between cognitive measures

math.

sci.

read

School level

math..

.057 (.008)

.704

.597

sci.

.037 (.006)

.049 (.007)

.527

read

.035 (.007)

.029 (.006)

.061 (.008)

Pupil level

math.

.326 (.007)

.803

.605

sci.

.260 (.006)

.323 (.007)

.818

read

.197 (.006)

.256 (.006)

.326 (.007)

ICC

.152

.132

.158

Scientific abilities

explain

indicate

use

School level

explain

.052 (.007)

.618

.693

indicate

.037 (.007)

.067 (.009)

.673

use.

.039 (.007)

.043 (.008)

.061 (.008)

Pupil level

explain

.378 (.008)

.835

.875

indicate

.322 (.008)

.392 (.008)

.871

use

.301 (.007)

.305 (.007)

.313 (.007)

ICC

.121

.146

.163

Cognitive and non-cognitive

instrum.

interest.

math.

sci.

School level

instrum.

.035 (.008)

n.s

n.s

n.s

interest.

.012(.006)

.049 (.009)

.054

n.s

Wisk.

.011 (.006)

.043 (.006)

.057 (.008)

.705

Wet.

.007 (.005)

.004 (.006)

.037 (.006)

.049 (0.007)

Pupil level

instrum

.889 (.021)

.340

.166

.172

IIW

.303 (.015)

.893 (.019)

.043

.109

math.

.089 (.009)

.023 (.008)

.326 (.007)

.803

sci.

.092 (.009)

.059 (008)

.260 (.006)

.323 (.007)

ICC

.037

.052

.152

.132

math. = mathematical literacy instrum. = instrumental motivation for science

sci. = scientific literacy explain = explaining scientific phenomena

use. = using scientific evidence. indicate: indicating scientific phenomena interest. = interest in science ICC = Intra-class correlation coefficient

n.s or underlined = not significant.

interest.

Math.

Math.

Sci.

Sci.

Sci.

Sci.

math.

interest.

interest.

instrum.

instrum.

instrum.Figure: 1 Visualization of the covariance between the residuals at the school level for the non-cognitive and cognitive output criteria (OV3)

Conclusion and discussion

One of the questions that are being posed within the school effectiveness research is the question about the consistency of school effects. (De Maeyer et al., 2008). Does a school that performs well for one effectiveness criteria also performs well for other criteria? Within the current research literature there is lack of Flemish empirical evidence. Results about the consistency between cognitive and non-cognitive criteria are inconclusive (Brandsma, 1993; Mandeville & Anderson, 1986; Mortimore et al., 1988; Reynolds, 1976; Rutter et al., 1979, Gray et al., 1983). Previous research mostly focused on language and mathematics as effectiveness measures (Scheerens, 2000). This would indicate that schools only have to focus on science and language, while there is more and more a tendency that schools also have to focus on non-cognitive output criteria. Based on basic multivariate multilevel models it was possible to conclude that schools that perform well for one cognitive effectiveness criteria (e.g language, science or mathematics) also perform well for the other effectiveness criteria. The same is true for scientific abilities, if a school performs well for one scientific ability it will also perform well for the other abilities. If we also compare cognitive and non-cognitive measures it's possible to say that schools that perform well for one cognitive effectiveness criterion do not necessarily perform well for non cognitive output criteria.

As schools can't be held responsible for the pupil population that the school inhabits it is necessary to control for pupil characteristics. When taking into account the students intake characteristics although the correlations are somewhat lower it is possible to say that schools that perform well for one cognitive output criteria (e.g. science, mathematics, language) also perform well for the other cognitive output criteria. The same is true within science, if a school performs well for one scientific ability the school also performs well for the other scientific abilities. It is hard to say that if a school performs well for one cognitive output criteria (mathematics, science) also performs well for non cognitive output criteria (interest and instrumental motivation to learn science) If we compare the correlation coefficients they are somewhact higher than .40 - .50 (Cuttance, 1987; Fitz-Gibbon, 1991; Nuttal, et al., 1992; Smith & Tomilson, 1989; Thomas & Nuttal , 1993; Thomas, et al., 1993; Wilms & Raudenbush, 1989).

Uit deze cijfers valt op te maken dat Vlaamse secundaire scholen die het goed doen voor één cognitief outputcriterium het waarschijnlijk ook goed doen voor de andere opgenomen cognitieve outputmaten ongeacht hun leerlingeninstroom.

In further research it would be possible to further investigate the consistency between cognitive and non cognitive output criteria. If we can also include certain schoolcharacteritics it would be possible to check of schools with certain characteristics perform more consistent than others. The present study shows that school that perform well for cognitive effectiveness criteria don't necessarily perform well for non-cognitive effectiveness criteria. This might mean that characteristics of a school that increase the cognitive effectiveness criteria do not necessarily increase non-cognitive effectiveness criteria. Off course we also have to aks what do schools need to achieve and which invulling men moet geven aan het effectiviteitsconcept of zoals Reid, Hopkins en Holly (1987) concludeerden: "while all reviews assume that effective schools can be differentiated from ineffective ones there is no consensus yet on just what constitutes an effective school."

The correlation coefficients have dropped, but they stay high indicating that Flemish schools that perform well for one scientific ability also perform well for others. Mandeville en Anderson (1987) en Crone (1994) stelden dat de correlatie binnen vakken hoger is dan deze tussen vakken. Hanteren we in dit onderzoek de correlatiecoëfficiënt tussen wiskundige en wetenschappelijke geletterdheid dan blijkt dit niet het geval te zijn.

The found samenhang between the cognitive criteria underlines the importance of the use of multivariate mulitlevel models in future effectiveness research. By using several univariate analysis, we ignore the relationship between the dependent variables; and in this way the chance for type "1' errors. This means that we could find an effect were in reality there isn't any. By using multivariate multilevel models we reduce the chance for type '1' errors. A multivariate model models the correlation between the different dependent variables, by which the chance for a type 1 error reduces.

This study is also important for the increasing importance of accountability in current education. In France, the Netherlands and the UK, rankings are used to compare schools. (Van Petegem & Van Hoof, 2006). Based don these findings we can advise to include several outputcritiria, because including only cognitive criteria doesn't necessarily means that these schools also perform well for non-cognitive output criteria. Using rankings means also that we have to think about using net or gross effects. Because like other research papers the current research underlines the importance of the pupil population for the effectiveness of a school. Uit het theoretisch kader kon worden besloten dat de richting en de omvang van schooleffecten op cognitieve en niet cognitieve criteria tot op heden onduidelijk blijkt te zijn (Brandsma, 1993; Mandeville & Anderson, 1986; Mortimore et al., 1988; Reynolds, 1976; Rutter et al.., 1979, Gray et al., 1983). Dit kan deels bevestigd worden door de correlatiecoëfficiënten die in dit onderzoek gevonden worden. Deze zijn zo uitermate klein of niet significant, dat niet besloten kan worden dat Vlaamse secundaire scholen die het goed doen voor cognitieve outputmaten het ook goed doen voor niet-cognitieve outputmaten.