Evaluation of Research Methods for Education Research

Introduction

The purpose of this exercise is to explore and set out content for possible areas of consideration in a methodologies chapter for a Masters of Education thesis. Consequently, the following pages set out to present the primary and secondary research questions. Next, the paradigm type is defined and its selection is rationalised. Furthermore, this work explores the methodology outlining its strengths and limitations. In addition, an examination is made of the process of interviews, data analysis, validity reliability, reflexivity and ethical considerations.

Aim and Research Questions

Research is carried out for various reasons which include an investigation, evaluation or to progress practice, Thomas (2011), and a piece of research is built around a question it is not built around a method. The crucial points when posing research questions are what does the researcher want to find out and why is it essential to identify an answer to make a change to the world around us? (Bryman, 2012).

In 2017 the Department of Education and skills Circular letter 0011/2017 sought to increase allocation of teaching posts and also included an additional allocation to support school leadership. This allocation increased the intitlement of schools with over 700 pupils from one deputy principal to two deputy principals. This resulted in a large number of new deputy principals being appointed very quickly. These new appointees were promoted from the ranks of teachers with posts of responsibility and joined the ranks of school leadership with many having to move school in order to do so. A year later, very little if anything is known about how this substantial cohort of new additions to leadership, in our schools around the country, is thriving in their new roles.

The primary research question concerning this research is going to be ‘what are the challenges faced in their job by newly appointed deputy principals in post-primary sector?’ The aim is to find out, from their own perspective what are the biggest problems encountered by new deputy principals as they go about their work in second level schools.

The secondary questions that may arise are as follows:

• What are the greatest challenges you face day to day as deputy principal?
• What do other deputy principals identify as the greatest challenge they face in their school?
• Have you received enough and appropriate CPD in order to carry out your role since your appointment?
• Is there a particular area of need with regard to CPD?
• How has your relationship with colleagues changed since your appointment?
• How has your relationship with students changed since your appointment?
• What have been the biggest surprises that you have experienced in your role?
• What are your thoughts on the selection process?
• What do you consider to be your greatest competency?
• What do you think is the most important competency necessary to do the job?
• How important is consideration of complimentary competencies across the management team?
• How important is your relationship with principal/other deputy principals?
• What do you think is the department of Educations’ reasons for acceding to the addition of so many new deputy principals to second level schools?
• How do you see the role evolving? What changes are coming down the track?
• Are you happy/satisfied in your role?

Paradigm

The term ‘paradigm’ came from Thomas Khun’s 1970 The Structure of Scientific Revolutions work, the shared ideas and concepts that guide members of a given scientific community. A paradigm “is a technical word used to describe the ways we think about and research the world” (Thomas 2009, p.72)

The two distinct paradigms, of relevance here, are ‘positivist’ and ‘interpretive’ Thomas (2013). The positivist paradigm can be described as observable, controllable and measureable. (Basit, 2010). In social science research the positivist approach has been the most popular paradigm for decades in eliciting knowledge through scientific experiments and methods (Thomas, 2013). However, in recent times the positivist approach has been critiqued in relation to its inability to adequately elucidate how humans live and interpret the world (Savin-Baden and Major, 2013). As Thomas (2013) identifies, the specific approach chosen is more important than the actual approach itself. On the other hand qualitative research is used to try to comprehend, that very elusive aspect, peoples perceptions of the world (Bell, 2010; Cohen et al., 2011). Qualitative research, otherwise known as interpretivist research emphasises ‘the exploration of in depth quality data as opposed to quantifying it (Creswell, 2013). Qualitative research has the capacity to deliver complex textual accounts of how participants experience a given research issue and it can capture meanings and insights which quantitative methods frequently cannot (Bryman, 2012).

The strengths of Qualitative Research

There are a number of specific advantages of using qualitative research methods when studying educational research topics as proposed by Bryman (2012). Firstly, the researcher gets an opportunity to immerse themselves in the position and culture that the participants are speaking from in order to gain greater insight and a more accurate interpretation of the lived world they inhabit. This cannot be experienced through the use of statistical and numerical analysis (Bryman, 2012). Secondly, the researcher has the opportunity to interact with the participants on their own terms and in their own language. This gives the researcher primary data which describes existing phenomena and contemporaneous happenings in relation to the research topic (Bryman, 2012). Most importantly, qualitative research has the potential to produce outcomes which can contribute towards newer understandings as qualitative research can be flexible in the way data is collected and also through the interpretation and analysis processes (Bryman, 2012).

Limitations of Qualitative research

Literature suggests, Bryman (2012), that using qualitative research can raise some concerns in relation to its lack of formal processes. It can be very subjective and can have a higher risk of research bias. Other variables include, the experience of the interviewer which in this case is novice, and the sample size. Every effort will be made to ensure the sample size is large enough to have integrity and be representative of disparate school cultures but since participation is voluntary then invariably weaknesses, in that context, may materialise.

Methodology and Methods

This researcher has identified a qualitative research method as the most appropriate to help answer the primary and secondary research questions. Qualitative research is influenced by interpretivism and as a result seeks to allow researchers to observe events through the eyes of the persons participating in the research (Bryman, 2012). A qualitative research approach will allow the Deputy Principals, participating in the process, the freedom to speak about their personal experiences of working in the role rather than having their narrative directed, restricted or packaged.

Access

This study seeks to gain the perspectives of newly appointed deputy principals with regard to challenges that they encounter while carrying out their role. Prior to collecting any data ethical approval will be sought from University of Limerick. A specific sampling strategy will be employed in an effort to achieve a representative sample. Sampling involves taking a small portion of the whole in order to show quality. It looks for relationships that show variables (Thomas 2009). This researcher is choosing random sampling as a result. This strategy should help to minimise bias and has to be drawn from a large enough subset of the population in order to avoid finding a distorted picture (Thomas, 2009). The sample size is envisioned to contain from eight to ten participants. Initially in order to ensure a random varied sample the assistance of some school leadership organisations, such as the Centre for School Leadership, Principals and Deputy Principal Association and National Association of Principal and Deputies, will be sought as well as publically available information on school websites. GDPR considerations will be adhered to at all times. It is expected that a considerable amount of communication will be conducted through emails and phone calls. Permissions will be sought from principals(Gatekeepers) of schools where deputy principals are employed. They will receive an information letter and a letter of consent. Permission will be requested from the principal of this researchers school also in order to work with deputy principals and in order to use a school room as a venue for interviews. Deputy principals taking part in the focus group and/or interviews will each receive an information letter, letter of consent and a copy of the interview questions so that they can familiarise themselves with the content in advance. Confidentiality will be guaranteed at all times with the right to withdraw at any stage reiterated.

Semi-Structured interviews

Three types of interview can be utilised in qualitative research according to Thomas (2013). These are; structured, semi-structured and unstructured. Structured interviews can be conducted in a consistent fashion as the interviewer asks the same questions. The limitations of structured interviews however is, that very rigidity which does not allow the interviewer to follow up statements made by participants. Unstructured interviews are dictated by the interviewee so therefore they set the agenda and this limitation means that the topic can be changed on a whim making it necessary and forcing the interviewer to interrupt in order to steer the interview back on course (Thomas 2013). The third option is semi-structured face to face interviews which is deemed to be the most suitable data collection method in this case as it will facilitate the possibility of developing points made by the interviewee while maintaining structure to the overall process (Thomas 2011). Although semi-structured interviews allow this flexibility and at the same time they should by conducted like a “carefully controlled conversation” (Robson, 2007,p.74).

The strength of using semi-structured interviews is that it provides a methodical direction for participants, yet the interview remains primarily conversational (Cohen et al., 2011). Similarly, Thomas (2009) agrees that semi-structured interviews allow the author to keep their themes in mind without constricting the participants. The interviewer has the potential to follow up ideas, probe responses and investigate motives and feelings which the questionnaire can never do, according to (Bell, 2010). In addition it is possible to tune into tone of voice, hesitations in replies, body language and facial expressions.

The disadvantage of semi-structured interviews can be the desire of the participant to please the researcher with their answers (Patton, 2002). The researcher must be aware throughout the whole interview process that they are a human instrument that is capable of being biased and subjective in the process (Savin-Baden and Major, 2013). This researcher will naturally endeavour to make participants feel at ease during the interview process.

Pilot interview

A pilot interview is conducted “on the basis of convenience, access, and geographic proximity” (Creswell 2013, p.165). This researcher proposes to conduct a pilot interview in order to trial run the research interview and has communicated with two deputy principals in the school and has received confirmation that they will take part in a pilot interview in a school classroom. According to Merriam (2009) pilot interviews are essential as they allow the interviewer to refine the interview technique and tailor the questions. The interviewer gets to run through the process, iron out any problem areas and the location can be evaluated in terms of its suitability for digital audio recording (Creswell, 2013).

Focus group

The focus group is a form of group interview (Cohen et al., 2011), though not in the sense of a backwards and forwards interaction between the interviewer and group. The reliance is on the interaction between the group members to discuss a topic and as a result yields a collective rather than an individual view. They are very focused on a particular issue so can yield insights not otherwise gleaned from an interview and producing a large amount of data in a short space of time. The dynamic lends itself to gathering data on attitudes, values, opinions and can provide a useful element to triangulate with traditional forms of interviewing. When the ingredient of reflexivity is included into the mix then this explains one of the major reasons why this researcher is choosing to pursue this method. It is proposed to utilise two focus groups so that the outcomes can be further cross-referenced. Cohen et al., (2011), while acknowledging the considerable potential, point towards the facilitation of the group as the main source of problems. It is important to strike a balance between being too directive and allowing the group to veer off the point. Vigilance must also be maintained to prevent disagreement and conflict.

Data Analysis

It is proposed to interview at least eight to ten participants all going well with two groups of four or two groups of five in each focus group. Pseudonyms will be used to protect each participant’s anonymity. Participants will be informed of their right to withdraw from the research at any stage prior to data analysis along with the right to confidentiality throughout the entire process (Creswell, 2013). The data will be collected using an audio digital recorder to record semi-structured interviews and these will be transcribed by the researcher for final analysis. It is prudent to conduct data analysis concurrently with data collection, according to Merriam and Tisdell (2016), so this researcher will be allocating an adequate amount of time for this procedure and ensuring synchronisation with participant availability. On completion of the data collection, the data analysis will involve:

Organising, accounting for and explaining the data; in short making sense of the data interms of the participants’ definitions of the situation, noting patterns, themes catergories and regularities.

                                                                                                             (Cohen et al.,2007,p.461)

The interviews will be analysed using a thematic approach through the use of coding and thematic mapping, (Thomas, 2013), which seeks to explore reoccurring patterns and relationships within the collected data. It is proposed to personally transcribe the data and listen to each recording repeatedly known as the “constant comparative method” in order to enable the researcher to become aware of the language and the specific life experiences of each participant. The aim of data analysis is to foster the rich illuminative words of the participants in exploring their unique experiences (Thomas, 2013).

Validity and reliability

The term validity refers to certain requirements to which research has to adhere to in order ensure integrity. It “tells us whether an item or instrument  measures or describes what it is supposed to measure or describe” (Bell, 2010, p.119). it also deals with the question of how research findings match reality (Merriam and Tisdell, 2016). Thus, in this case, methods of data collection will include two focus groups and interviews with triangulation between them and reflexivity.

“Reliability is concerned with the question of whether the results of a study are repeatable” (Bryman, 2010). For that reason, careful consideration will be given to planning the questions so as to minimise interviewer bias (Bell, 2010). The voice from the interviewees perspective needs to be heard. To this end, questions will be worded uniformly and asked in the same sequence for all participants. Some level of interviewer personal bias is inevitable because of this researchers interest in the topic and intrinsic motivation to glean insights. In order to declare this aspect and capture other subjective responses a reflexive journal will be utilised. It will be used to record any relevant thought processes, thought evolutions and critiques as an interpretivist researcher must seek to state their own positionality (Thomas, 2013).

Reflexivity

The notion of reflexivity is further nailed down by, (Pillow, 2003), when she says that it is since interpretive research has come into vogue in the social sciences that the objectivity of research has come under the spotlight and issues of power in research relations begin to be acknowledged. It is in this context that reflexivity takes on an even larger and more significant role. In addition, she goes on to explain that, precisely because of its multiple uses, reflexivity has been associated with or indeed used as a measure of legitimacy and validity in recent qualitative research.

Listening and writing with reflexivity  are often described as tools to help situate oneself and be cognisant of the ways your personal history can influence the research process and yield more ‘accurate’, more valid research.

                                                                                                                  (Pillow, 2003, pg.179)

This researcher will implement a reflexive approach by using a research diary. All thoughts and observations before, during and after the process will form part of conclusions and will be included in appendix pages of thesis.

Ethical considerations

In all types of social research, ethical considerations are paramount. When conducting qualitative interviewing the main focus of ethical issues are informed consent, confidentiality and the consequences of the interviews (Cohen et al., 2007). Not forgetting the necessity to go through the process of seeking approval from an ethics committee in order to carry out the research in the first place (Guillemin and Gillam 2004). In this case it is proposed that consent will be established, pseudonyms will be applied and data treated confidentially. Interviews will be digitally recorded and transferred to a password protected laptop immediately. This researcher will transcribe each interview onto a password protected laptop. All data collected will be stored in a locked cabinet in Principal investigators office until such time as it is destroyed. Interviewees dignity and wellbeing will be of primary importance at all times.

Conclusion

This exercise outlined perspective primary and secondary research questions relating to the challenges faced by newly appointed deputy principals in second level schools. The interpretivist paradigm was selected and defended as the most suitable viewpoint from which to elucidate the research problem. The methodologies that flowed naturally from this for this researcher were those of interviews and focus groups for triangulation and reflexivity in order to position the researcher in the whole process. Following on from this the technicalities of data collection and analysis were looked at to a depth appropriate for the scope of this work. Finally, validity, reliability, reflexivity and ethical considerations were discussed.

Bibliography

• Bell, J. (2010) Doing your research project. A guide for first time researchers in education, health and social science, 5^th ed., Berkshire: University Press.
• Bryman, A. (2012) Social Research Methods, 4^th ed., Oxford: Oxford University Press.
• Cohen, L., Manion, L. and Morrison, K. (2011) Research Methods in Education, 6^th ed., London: Routledge.
• Cohen, L., Manion, L. and Morrison, K. (2013) Research Methods in Education, 7^th ed., London: Routledge.
• Creswell, J.W. (2013) Qualitative Inquiry and Research Design, choosing among five approaches, 3^rd ed., London: Sage Publications Ltd.
• Department of Education and Skills, (2017) Circular letter 0011/2017
• Guillemin, M. and Gillam, L. (2004) ‘Ethics, Reflexivity, and “Ethically Important Moments” in research’, Qualitative Inquiry [online],10(2), 261-280, available:
• http://journals.sagepub.com/doi/pdf/10.1177/1077800403262360[accessed 3 December 2018]
• Merriam, S.B. (2009) Qualitative Research: A Guide to Design and Implementation, San Francisco: Jossey-Bass.
• Merriam, S.B. and Tisdell, E.J. (2016) Qualitative Research: A Guide to Design and Implementation, San Francisco: Jossey-Bass.
• Patton, M. (2002) Qualitative Research and Evaluation Methods, 3^rd ed., London: Sage.
• Robson, C. (2007) How to do a Research Project; A guide for undergraduate Students, London: Blackwell Publishing.
• Savin-Baden, M. and Major, C.H. (2013) Qualitative Research: The Essential Guide to Theory and practice, Abingdon: Routledge.
• Thomas, G. (2013) How to do your Research project: A guide for Students and Researchers, London: Sage.
• Wanda, Pillow (2003) Confession, catharsis, or cure? Rethinking the uses of reflexivity as methodological power in qualitative research, International Journal of Qualitative Studies in Education, 16:2, 175-196, DOI: 10.1080/0951839032000060635 [accessed 1 Dec 2018]