Evaluation of studies at the Faculty

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

In its short career the Faculty of Commercial and Business Sciences (Celje, Slovenia) has created an important place in the Slovenian higher education space. Every year, more students enrol in the Faculty, which offers more courses. Today we have about 500 full time and 2500 part time students enrolled in three courses at all levels. Who are the Faculty's students? How successful are they in their studies and employment? What are their study experiences in the educational process at the Faculty? Does the study meet their expectations? These are questions which we try to analyze and answer in our paper.

The faculty held separate databases on the performance of students, successfulness of graduates and the satisfaction of students with their studies. Analysts preparing evaluation reports and using these resources have neither a single method nor the technology for extracting the relevant information. Reports are therefore non-transparent and do not say much about the quality of school work.

In the paper we will explain the applicability of the instrument (the expert model), as well as the methodology of its use. The instrument and the methodology may be useful for internal or external evaluation of studies at the Faculty.


The Faculty collects numerous data and information about the study performance of students and graduates as well as about the satisfaction of students with their studies. These are separate databases. Analysts have problems compiling evaluation reports and analyses. They do not have the instruments or computer tools to prepare quality reports.

Reports and analyses are not related, as they contain data from separate databases. It cannot be deduced from them, for example, how study performance of students is related to successful and timely graduation or the satisfaction of students at the faculty. In order to be able to effectively monitor the quality of the organizational and educational work, it is necessary to establish how the the following three parameters influence the quality of work: performance of students, successfulness of graduates and the satisfaction of students with their studies.

We will therefore present an expert model designed in the computer programme Dexi (http://kt.ijs.si/MarkoBohanec/dexi.html), which enables to link data from different databases and present uniform information about the quality of studies at the faculty. It needs to be emphasized that the data reflects the opinion of students.


The sample for the analysis comprised students who enrolled in the Faculty of Commercial and Business Sciences for the first time in the years 2007/2008, 2008/2009 and 2009/2010. The analysis included 206 full time and 1271 part time students. A profile of students at the Faculty was based on these data. The data were collected from the student database (Pedagoška dokumentacija 2010).

The sample for the analysis of graduates comprised students who graduated from the Factuly in the period between October 2007 and September 2009, i.e. in the study years 2007/08 and 2008/09. The analysis included 787 graduates. The data were collected from the graduate database.

The satisfaction with the Faculty's organization and management, the implementation of the education process, with teachers and other staff and with other activities of the Faculty is evaluated by students with the help of survey questionnaires. The satisfaction with the Faculty's organization and management is evaluated in respect of the adequacy of the Faculty's facilities, the work of the student affairs office, information on the Faculty's website, the organization of exams and the computer equipment in lecture rooms. The satisfaction with educational work is measured in respect of the quality of the student timetable, study contents and the quality of the implementation of lectures and practical classes (lecturers and other teachers). Some other factors also influence the satisfaction: integration of the faculty into the environment, motivation of student, coordination of work etc. The analysis took into account the survey results from 2007/2008 and 2008/2009 (Šuster Erjavec 2008 and 2009). For the purpose of evaluating the Faculty's organization and management we analysed 220 questionnaires in the study year 2007/2008 and more than 3000 in the next study year.

2.1 Profile of students

We wanted to establish a profile of the students enrolled in our Faculty, which would take into account the following: gender, status (full time or part time), study year, the period between the completion of previous education and the enrolment in our Faculty, the school completed prior to enrolling in our Faculty and the results achieved in previous education.

In the analysed years the majority of full time students are female and their share is increasing by almost 10% each year. The ratio between female and male student is much more balanced among part-time students. A more balanced ratio in both study groups is desirable.

From year to year the enrolment in full time study is decreasing, which is the result of the decrease of enrolment in secondary schools (dwindling population). The enrolment in part time studies, however, is increasing (the result of life-long learning). The number of students enrolled in the first year is also on the increase, which indicates that less students transfer from other institutions of higher education. In the light of financial effects, this is desirable.

From the data for enrolled full time students we concluded that at least one year passes on average between the completion of previous education and the enrolment in our studies. The average grade of final examinations at previous education is improving, which means that each year better performing student enrol in our Faculty. On average, students enrol in part time studies two years after completing their previous education, but the actual data are much more dispersed than the data for full time students (some decide to study even after a 10-year break). The data on the grade of final examination at previous education are kept very inconsistently (21% of the sample includes this information), and can thus not be taken into account as a relevant indicator of the part time students profile.

Full time students enrol from 26 secondary and higher schools in the region, most of them from secondary schools of economics. Part time student enrol from a wide range of secondary schools in Slovenia (the records include 151 different secondary and higher schools).

2.2 Profile of graduates

The majority of graduates are female (71.4%), which is understandable, as 2/3 of the student population are female. On average, the study at our faculty lasts 2.5 years, which shows that many students enrol directly to the second year (graduates of higher schools). The average final thesis grade is 9.31, while the grade point average is 8.18. In order to be able to provide quality remarks on the grade, a comparison with a similar faculty and comparable study programme would be needed. However, the final thesis grade is very high.

The analysis shows that approximately half of the graduates are employed, which is less than expected, bearing in mind the full time/part time students ratio at the faculty (14% full time and 86% part time students in the graduates sample).

Around 70% of graduates in the observed period graduated according to plan. 25% graduate in less than 2 years, 49 % in less than three and 5 % later.

Average final thesis grades are very high: 47.4% get the grade 10, 37.6% the grade 9, and 15% the grades 7 and 8. Grade point average is more evenly distributed: 46.6% score between 8 and 8.99, 36.7% between 7 and 7.99, and 16.7% get other grades. In comparison with final thesis grades, the grade point average is more realistic.

2.3 Data from student questionnaires

Educational work was assessed by around 2000 students in 2007/08 and by more than 3000 students in 2008/09. The average grade for teachers and study courses are almost identical in the observed period; 1.8 and of 2.3 respectively (on a scale from 1 to 4, where 1 is the highest and 4 the lowest grade). The grade for other teaching staff (teaching assistants, instructors, invited lecturers) varies more: 1.7 in 2007/08 and 1.6 in the next year.

The satisfaction with the Faculty's organization and management was assessed with the grade 1.9 by 220 student in 2007/08 and with the grade 2 by 357 student in 2008/09.

The other criteria, e.g. the integration of the faculty in the environment, motivation of full-time students, coordination of work, support of the management to the students, teacher training and adequacy of study literature, were assessed with the grade 1.8 in 2007/08 and 1.6 in 2008/09.


After processing the student questioners and analysing the data from the student database we were able to establish who our students and graduates are and how satisfied they are with the faculty. However, the interpretations of analysis results for individual study years are unrelated and non-transparent. Analysts lack the tool for comparing the results, examining the characteristic statistical differences in the evaluation of individual criteria and for simulating the desired conditions.

In what follows, we will present the expert model for evaluating the studies at our Faculty, which was designed with the computer programme Dexi (Decision Expert), we will make a sample evaluation for two observed periods, interpret the results and propose the use of the method.

Dex (Jereb, E., Bohanec, M., Rajkovič, V. 2003) is the shell of the expert system, intended for multi-parameter assessment. It provides effective assistance for decision-making regarding more complex problems (Adelman, 1992). DEXi is a derivate of the tool used in education. We evaluate individual study years according to criteria in the model, which are hierarchically structured; the grades for primary criteria are calculated by aggregation of grades for subordinate criteria according to decision-making rules. Besides an in-depth report, which can be compiled in Dex, the graphs and the so-called »what-if« analysis can help make simple simulations of the desired conditions.

In our case we built a model (prototype) for evaluating the studies at the Faculty, based on key criteria like: student profile, graduate profile, satisfaction of students (Figure 1). Criteria for establishing the profile of students (Par. 2.1), the profile of graduates (Par. 2.2) and the satisfaction based on the student questionnaires (Par. 2.3) were used as subordinate criteria. Average grades for criteria, calculated from previous analyses for the years 2007/08 and 2008/09, will be included in the model; grades for individual years present an individual variant assessed in the model.

The model prototype was tried out by a questionnaire among the faculty workers (model validation). The sample included teachers, other Faculty staff and the management of the Faculty. The questionnaire was completed and handed in by 85.71% of the respondents. The discussion about the model validation can be extended and the model can be appropriately corrected. According to the collected opinions and suggestions, the Dex model may remain in the proposed form (Figure 1).

The grades for criteria in the years 2007/08 and 2008/09 were included in the model as variants. Two variants are enough to explain the usability of the method based on the interpretation of assessment results (Figure 2).

The evaluation of studies for both years is positive. However, there are certain differences which indicate whether or not the observed parameters are improving. Comparing the year 2008/09 with the previous one, the following differences may be established: the share of part time students increased (this is, according to employees, less favourable), the enrolment in the first year increased (which is desirable in the light of education work and the owners' interests), half of the graduates are employed and the other half are not (the previous year more graduates were employed), the integration of the faculty in the environment improved, as did the cooperation between teachers and students and between the management and students etc. If we were to examine grades for subordinate criteria, differences between both variants would be established. If the number of evaluated periods was higher (more variants) a trend could be observed - whether the evaluation of the study at the Faculty is improving or not; which differences demand special attention etc.

The tree of criteria Description of the criteria

Figure 1: The model of evaluation studies at the Faculty, made in computer program Dexi.

Criteria The year 2007/08 The year 2008/09

Figure 2: The result of evaluation of two consecutive academic years.


This paper presents the method of evaluation of studies at the Faculty with the help of expert modelling. We used the computer programme Dexi, which is a simple programme in public domain, but any other comparable tool could also be used.

Expert model is a knowledge database, which can be used for establishing, monitoring and assuring quality at the Faculty. It collects data from dispersed databases at a single location, thus making the interpretation of results more transparent and enabling direct comparison of evaluations for different evaluated periods. The transparency of the evaluated variants (study years) makes it possible to react quickly and introduce improvements or changes.

The model may always be corrected by changing the relations and influences of individual criteria (correct decision-making rules). It would be reasonable to name a team of experts responsible for quality of studies, which would deal with such researches.

The presented method does not aim to automate the procedures and tasks related to the evaluation of quality at the Faculty. It present a sample support system for decision making, which collects data at a single location and helps experts to observe information which could previously be overlooked due to the dispersion of data. This is the significance and the value of the method.