Validation Of The Dreem Instrument Education Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

The Dundee Ready Education Environment Measure (DREEM), a universal diagnostic inventory, was developed and validated by Roff et al. in 1997 to assess "the whole or parts of the educational environment and climate of health professions/medical schools" (Roff et al., 1997: 295). It gives valid information through students' perceptions of their institution's strengths and weaknesses on learning environment.

The DREEM was developed by a combination of qualitative and quantitative techniques, and validated by over 1000 students from around the world, so as to be a non-culturally specific instrument (Roff, 2005). It is combined of 50 items, 41 of them are positively worded statements and nine are negative ones, relating to Students' Perceptions of Learning (12 items); Students Perceptions of Teachers (11 items); Students' Academic Self-Perceptions (eight items); Students' Perceptions of Atmosphere (12 items); Students' Social Self-perceptions (seven items) (Roff, 2005: 322). Each item is scored in a five-point Likert-type scale (from 0-strongly disagree to 5-strongly agree), except for the nine statements which are scored in reverse so as the high scores indicate a good environment. The maximum score of 200 (50 answers x 4) indicates the ideal educational environment as perceived by the students.

Since 1997, this instrument has been translated in many languages and used in several health sciences educational settings (Medicine, Dentistry, Chiropractic) around the world (Roff, 2005; Kossioni et al., 2011), by it has never been applied in other disciplines.

The instrument has been previously administered to greek medical (Dimoliatis) and dental students (Kossioni) and discipline related differences have been recorded.

The aim of the present study was to investigate whether DREEM can measure the educational environment in a different educational setting, in an educational discipline, and compare the findings with those of the previous DREEM studies in GReece

Materials and methods

Τhe DREEM questionnaire was delivered at the Department of Educational Sciences and Early Childhood Education (DESECE) at the University of Patras, Greece. The DESECE is one of the oldest departments of educational studies in early childhood education in Greece. Throughout its history, the DESECE has always been concerned with conducting research to promote the sciences of education and providing its graduates with a high-level scientific knowledge and the practical skills for succeeding in their profession. The studies endure four years and during the last two years (3rd and 4th) the students attend their "teaching practice". The attendance in the teaching practice is compulsory. After the acquisition of the degree they may pursue a professional career as Kindergarten teachers.

The DREEM questionnaire was translated in Greek and validated in Medical Schools in 2010 by Dimoliatis and colleagues (Dimoliatis et al., 2010). Based on the slightly modified instrument in 2011 for the Greek Dental School (Kossioni et al., 2011) we proceeded to minor further modifications to better address our curriculum.

A pilot study was conducted in 20 volunteer students in order to obtain conceptual equivalence and identify any problem in understanding the meaning of the items, since the initial translation was performed for Greek medical students. Minor modifications in the wording were performed in a few questions to better address the DESECE curriculum, as undergraduate pedagogical education includes the performance of school work by the students instead of ward watching. Τhe modified questionnaire is presented in Table 1. The statement number 6 "The teachers are patient with patients" was changed to "The persons responsible of your teaching practice are patient with children", the statement number 11 "The atmosphere is relaxed in the clinics" was changed to "The atmosphere is relaxed during the teaching practices", the statement number 18 "The teachers have good communication skills with patients" was changed to "The persons responsible of your teaching practices have good communication skills with children", and the statement number 45 "Much of what I have to learn seems relevant to a career in healthcare" was changed to "Much of what I have to learn seems relevant to a career as a preschool teacher".

The statements were scored from 0 to 5 as follows: 0: strongly disagree, 1: disagree, 2: slightly disagree, 3: slightly agree, 4: agree, 5: strongly agree, except for the nine negative statements (4, 8, 9, 17, 25, 35, 39, 48, 50) which were scored in reverse. To ease comparisons, all scores were then transformed in a 1-100 scale.

The modified DREEM questionnaire was distributed to the 2nd and 3rd-year students of the DESECE in the middle of the year, in January. A total of 160 students with a mean age of 20.6 ± 1.2 years (range: 18-28 years) participated in the validation study. The questionnaires were administered by a teacher during a course after explaining the purpose of the study. They were anonymously completed and collected at the same time. The 1st-year students were excluded as they had very limited experience on their studies (4 months) and could not understand and answer all the questions.

Since the questionnaire was administered in a totally different group of students (non-health sciences education students), a first selection of items was made from the descriptive response distribution for each question. The criteria used to guide item selection/deletion werehigh rates of nonresponse and "not applicable" response¨s (>20%), ceiling and floor effects (>50%). In the case that an item had more than half the answers in the "extremely" or "not at all" categories, the expert committee had to agree if the item would remain or not in the questionnaire.

Cronbach's alpha analysis was used to test the reliability of the resulting scales and to decide which low-contributing items should be removed from the scale. Cronbach's α "if item deleted" were also calculated. For reliability, the observed scale α should be .0.70 to be acceptable, .0.80 to be good, and .0.90 to be excellent (Kline)

An explorative factor analysis (EFA) using principal components analysis and varimax rotation was performed to identify the underlying clusters of items (subscales) in the questionnaire. The retained number of factors were identified according to criteria previously defined by J. Schönrock-Adema et al. (2009):

Descriptive statistics were used to calculate means, minimum and maximum values and standard deviations.

T test were performed to identify any semester related variation and for assessing differences among the five subscales related to the year of the studies. The level of statistical significance was set at P < 0.05. The data were statistically analysed using SPSS 19.0 (SPSS Inc., Chicago, IL, USA).


Of the 160 students that returned the questionnaire, 133 were women and 3 men, while 24 did not report their gender. Of the respondents, 69 attended the 2nd year (response rate: 43.1%) and 91 the 3rd year (response rate: 66.9%).

Items 8, 46, 48 and 49 are missing the value 0, and the statement number 37 is missing the values 0 and 20 but it was decided from the experts committee to remain in the final analysis (Table 2).

Cronbach a, for the total questionnaire (50 items) was 0.925, based on standardised items and increased to 0.934 after the omission of 3 items. This value indicates that the total scale has excellent internal consistency. Three items that had a low item total correlation , q17 (r = 0.081), q25 (r = 0.020) and q26 (r = -0.120) ,and were removed from the final EFA. The average inters item correlations (coefficient α-values) for the total scale was 0.205, ranging from -0.238 to 0.770 for each item (Table 3).

After exploring four possible EFA models from 4 to 7 factors, the final analysis identified five new factors accounting for 44.2% of the variance (Table 2).

The first factor consisted of items 41, 43, 44, 29, 21, 42, 27, 12, 13, 22, 32, 36, 3, 5, 14, 30, 20 and 16, which referred to teaching, and learning. Items 22 and 36 loaded into two factors. The second factor consisted of items 2, 1, 40, 37, 45, 23, 24, 7, 38, 11 and 35, which referred to teaching, while the third consisted of items 18, 34, 19, 6, 15, 46, 47, and 49 and referred to teaching, atmosphere and social relationships. The fourth factor consisted of items 31, 28, 10, 4 and 33, explaining student's emotions. Finally the fifth factor consisted of items 9, 39, 48, 8 and 50 referring to negative feelings, behaviors and attitudes of teachers and students.

Many items with similar meaning were allocated to different factors.

Factor analysis revealed a five factor model but not all items were able to load higher than 0,400. As can be seen in Table 3, items 38, 11 and 35 (in the second factor) and item 49 (in the third factor) had loadings lower than 0,400 (adema) but higher than 0,300 that is often used in factor analysis loadings, so the experts committee had to decide whether or not the items should be omitted from the questionnaire. Τhe experts decided to keep those questions.

Cronbach a, for the 5 subscales was…..

Regression analysis showed that all items were significant (P < 0.000) determinants of the overall questionnaire, with loadings (beta) for each question from 0.28 to 0.75, meaning that every question contributed to the overall score about 2.8% to 7.5% (Table 2).

The overall mean score of the educational students' perception of the educational environment was 62% (Table 2). The results for each one of the 50 statements, the five new factors and the five original subscales are presented in Table 2.

T test recorded significant differences between 2nd and 3rd year students in items 6, 11, 25, 39, 32 and 41 (Table 4). When the five subscales identified in the present analysis were analyzed, statistically significant differences were recorded only for the first factor, identified between the 2nd (Mean=50.2±14.9) and the 3rd (Mean=55.2±12.9) year of studies (t=-2.124 p<0.05) (Τable 4).


Κυρια σημεια συζητησης

1. This validation study has shown that DREEM instrument had an excellent internal consistency ) factors????

2.EFA in this sample revealed a different model from the one proposed in the original DREEM study(Roff et al 1997) . Other DREEM studies in a Health Sciences environment also could not find the same factor solutions as the original scale , as items were allocated to different factors or the meaning of the factors was different (Jacobsson et al., 2011, Kossioni et al, Dimoliatis et al 2010, Wand et al 2009, Oliveira) This has been attributed to the consensus generation of the original factors (Dimoliatis et al 2010), to different perceptions of the learning environment in residents compared to undergraduate students (de Oliveira Filho et al., 2005), or to cultural differences between countries (Wang).

The fact that the original factors were not found in all three studies in Greece, even in different schools (Kosioni, dimoliatis) , shows that both methodological factors and cultural factors may be implicated

3. According to the statistical analysis the statements number 17, 25, 26 had a low item total correlation and were removed from the scale. The same low correlation for the item 17 has been found in the studies of Wang et al., 2009 and Jacobsson et al., 2011). Perhaps the Greek students have also misinterpreted the item about cheating as it wasn't clear for them if we implied cheating only at exams or in general. The content of the negative statement number 25 probably was not understood by the students. Finally, the statement number 26 had a very low item total correlation probably because the inventory was distributed to students at the end of the semester so that is wasn't clear for them to which academic year indicated. Another interpretation is that the content of the statement was not clear to students. Therefore we propose to reformulate the statement emphasising the practical training of students, where they can see in practice the results of last year's work: "Last year's work has been a good preparation for your teaching practice of this year".

4. According to the scores' interpretation offered by Domiliatis (..) and Kossioni (..) a score of 61% is considered as good. The lowest scores were recorded for subscale 1 because…….. (εδώ θελει αναλυση των φακτορων)

5. sxolio για διαφορα σε φακτορα 1 μεταξύ ετων και συγκριση με τα αλλα ελληνικα dreem

Oysiastika leei ότι teaching and learning obtains the lowest value from all factors, although a little improved in the 3rd year students. The opposite was recorded in dental DREEM, as the clinical students gave lower scores to the learning environment than the non-clinical students (kossion) . This reveals cultural differences in different schools, but also that teaching and learning need more attention. However items were not clearly allocated in the new factorial model and no accurate conclusions can be drawn.

The differences, related to the year of studies, in statements number 6, 11 and 39 lead us to conclude that the DREEM inventory should be administrated only by older students who already have experience in teaching practice or otherwise should removed from the inventory the questions relating to the teaching practice. It seems logical that the 2nd-year students who have not any idea about the teaching practice to respond to a hypothetical statement with various answers.

Τhere are some limitations in the present study

The questionnaire was administered to a relatively small number of students, all attending the same school, thus the results cannot be generalised in the field of sciences of education

The 3rd year students don't have an actual practical experience and some items were difficult to respond

We were not able to investigate gender differences due to the small amount of male students that attend the school

Comparison with other relevant scales was not performed as there is no other similar instrument in Greek measuring the environment in educational sciences

Under the limitations of the present investigation the use of DREEM instrument in an educational environment other than a health sciences school offered interesting and useful observations. Although having an excellent internal consistency, the original factors were not maintained while some items seemed irrelevant.

It identified cultural differences between different educational domains, in students originated from the same country. Further research is needed to add more relevant items in the questionnaire and remove those that seem irrelevant.