A Critical Review of the methods used in data collection

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

This paper will be a critical analysis of the quality and appropriateness of the methods of data collection and analysis in the paper produced by Bosu et al. (2011). I will attempt to provide with reasoning, an understanding for the reasons of the methods used in the case studies analysed and make use of literature to provide supporting and opposing views on the data collection methods as well as its analysis.

The context of the paper is set in the African countries of Ghana and Tanzania, both considered 'emerging and developing economies' according to the International Monetary Fund (IMF, 2010, p. 152). Three case studies are scrutinised, each with the theme of educational leadership with undercurrents of social justice however it is not clear what the objectives of the researchers are, which is vital in a case study due to their being "broad in nature" (Best, 2010a, p. 1).

Each was an explanatory case study, investigating the results of decisions made by school leaders based upon data collected using an etic approach. In each case study there is a review of an action research campaign, where the facilitator was the head teacher of a school. Two of the head teachers, both from Ghana, attended an action research workshop where they were introduced to research techniques, there is no mention of the history of the Tanzanian head teacher, or if they had received similar training. There is no clarity of the researchers focus, be it the difference that attending the workshop makes to the effectiveness of action research, the difference caused by boarders or simply the resultant of varying methods of action research.

At this juncture I shall attempt to review the data collection methods of each individual case study, concluding by reviewing the data collection by the researchers, of the paper as a whole.

Case Study: School A - Ghana

This case study had the focus of attempting to reduce teenage pregnancies of members of the school in question. To achieve this, the head teacher attempted to gain information from the community. There is clear evidence of the use of interview techniques, notably the head teacher holding a 'discussion' with the PTA which was a focus-group interview. A focus group interview does seem appropriate in this case as they are designed to "explain and understand, the meanings, feelings, beliefs and attitudes that influence the behaviours of individuals who are assumed to share a common culture, problem or issue" (Best, 2010b, p. 23) which in this case seems to be pertinent despite a lack of information as to who is invited to the PTA and if they do have a knowledge of the area to be discussed. Maxwell (1997) explains the importance of purposive sampling - of which this is evidence - stating "particular settings, persons, or events are deliberately selected for the important information they can provide that cannot be gotten as well from other choices'' (p. 87). There is no clear indication of the structure, although using the word discussion, one could ascertain that the form of an in-depth interview was taken where the head teacher had compiled a list of the issues to be covered and the interview structure was of a guided conversation.

The announcement of progressive improvement in terms of numbers of pregnancies, and in reading skills by the head teacher was followed by a proclamation that the intervention was a success. There is also an indirect suggestion by the researchers that all communities and schools can work collaboratively facilitated by the head teacher to address social justice problems. This is clearly evidence of a classic feature of the literature and observations of school improvement, as evidenced by Coe (2009, p. 368), with teachers often declaring:

'My school was failing in A, B, C ways. We did X, Y, Z. Now the evidence of success is P, Q, R'

Coe warns against making summations that what is observed in one school may necessarily be true for another (ibid.)

Case Study: School B - Ghana

This case studies focus was that of attempting to reduce truancies of pupils who on market days absented from school to make money. According to the baseline data collected by the head teacher, using quantitative data sources such as registers and text books, there was a clear group of boys identified with low attendances and poor performance, this is an example of secondary analysis, which according to Lewis (2003, p. 76) is examining a set of data which was originally used for one purpose with a different purpose. There was a meeting between the head teacher and a teacher to decide upon an intervention, there is no mention of the format of this meeting or how the format of the intervention was taken. It was concluded that a focus group interview with the pupils followed by individual interviews. The quality of this data is arguably high, according to Best (2010b, p. 23) the group interview can potentially provide 'deeper and richer' data due to the social interaction and questioning of the group members. As with School A this is evidence of purposive sampling. The use of individual interviews also allows for personal issues to be expressed, which participants may feel uncomfortable with expressing in the group environment. This is evidence of use of method triangulation in that the head teacher used two different data collection methods on the same group of pupils in order to increase the validity of the data.

An opportunity to collect further information arose from the meetings with the parents, enabling the head teacher to highlight a group of three pupils as high risk due to parental indifference with regard to their children's education. The purpose of these meetings were clearly to educate the parents, however it seems unintentionally vital data has come from them.

Due to the intervention the head teacher provided data for attendance as being 100%, with increased levels in mathematics and English, collected from monthly tests. The quality of these tests could be questioned as well as the teaching methods used as there is no information provided. One could question if these children were taught specifically to perform in tests. The head teacher took this further collecting data on six of the boys after graduation to junior high school; the results according to the researchers were sustained.

Case Study: School C - Tanzania

The case of the Tanzanian head teacher differed from the previous cases due to the lack of history in relation to their experience in action research. The head teacher, investigating the effect of poverty and difficult home environments on the schooling of the pupils, attempted to answer some questions. The head teacher 'sought information' from teachers, the format of which is unspecified, however due to the prior preparation and clear understanding of the questions needed answering it could be suggested that there was an interview, using a structured or semi-structured method. In addition data was collected from quantitative sources including, daily attendance records and test and examination results, as well as qualitative sources like class assignments and homework. In combining multiple methods a researcher must be aware of balancing differing theoretical backgrounds with any observed relationships between the data sets (Brannen, 1992, p. 33). It is the view of Ritchie (2003, p. 38) that there can be benefit in harnessing the two methods provided that the two data sets can be described precisely.

There may be biographical accounts within the class assignments or the homework which can provide vital information and "sensitize a researcher to key issues" (Plummer, 2001, p. 130). This is supported by Angell and Freedman (1953) who explain that this form of data has great value at the beginning stages of research rather than the end stages in order to gain an understanding of an investigation, which is reflected in the methodology of this case study. They emphasise the value of the understanding that this type of data can provide with respect to the subject of investigation (p. 305). According to Best (2010c) the reliability of this approach is questionable however using a theoretical generalisation the data can be used to strengthen the validity of a chosen theory or in this case the form of an intervention (p. 5). It is important at this stage to emphasise the obvious ethical issues in this case, initially due to the nature of the study, investigating pupils personal lives, and secondly due to the collection of the biographical data of pupils.

Interviews were held with the pupils individually, presumably using the same set of questions. If this was the case it would be evidence of use of data triangulation by the researcher, in that the same data collection method was used on numerous respondents in an attempt to increase validity of the results. All pupils insisted they enjoyed school and they felt that they could "freely express their problems to teachers who were relatively more caring and considerate than many of their parents and guardians" (p. 75). One could question however if the responses were entirely truthful due to the nature of the relationship between the pupils and the head teacher and if the pupils would provide different responses to a stranger. There is evidence to suggest that interviewees are more willing to provide truthful answers to strangers, although this argument relates specifically to online settings rather than in face to face interviews (Paine et al., 2007, p. 533; Nguyen and Alexander, 1996, p. 104; Wellman and Gulia, 1999, p. 172 & 178). Harrison (2008, p. xl) highlights the importance of the 'subjective' nature of data collection as well as how the relationship between the researcher and participant may 'impinge' on its validity.

An intervention similar to that in School A was held, with fluctuating results, with pupils failing to attend and perform well in monthly tests. There is a clear contrast from the results of the previous case studies, however due to the nature of the description of the case studies it is unclear what the cause of variety in results is. The researchers attempt to salvage some sense of accomplishment by emphasising the sense of empowerment of the head teacher, where due to the circumstances, it would be likely that many would feel helpless. Provided is no evidence of the empowerment of the head teacher after the completion of the case study, as well as a lack of evidence for the insinuation that the circumstances would most likely generate a sense of helplessness, leading to questions of the quality of data analysis displayed in this case. In the main the data analysed was qualitative, which as Harrison (2008) notes, has a different approach to analysing quantitative data, although there are approaches which aim to achieve a comparable 'routinization' (that is a routine of analysis) (p. 51). Primarily there is the method of analysing the structure of the text and based on Glaser and Strauss' (1967) propositions, the identification of commonalities and relevant theory (p. 105-106). Evidence of use of this methodology is present with the researchers identifying typical cases in each of the case studies, making comparisons with literature and mentioning the observed differences between the Westernised literature and African reality.

Secondly there is the method of objective hermeneutics (that is the study of interpretation theory) developed by Oevermann et al. (1979, in Harrison, 2008, p. 51). It involves the interpretation of the motives behind an action, with researchers adhering to two principles (Soeffner, 1997, p. xiv; Harrison, 2008, p. 51). The researcher should produce a highly extensive interpretation, providing all points of view that the data may represent however unlikely, and investigate the motivation behind an action extensively before providing any opinion about the data. There is evidence that both of these principles have been neglected to some degree by the researchers; there is a largely imbalanced preference towards the positives that action research has provided in these cases despite its clear failure in the study of School C of which there is little mentioned. There is also evidence of the data being used to fit the argument, with a clear prior knowledge of practise in Ghana that relates to the research carried out.


Although the investigative approach of the researchers is by use of case studies, it is also biographical in nature as there is a clear narrative produced, reflecting upon the experiences of the head teachers. All three head teachers participate in action research that affects them directly, according to Erben (1993, p. 15) this is good practice for a biographical researcher; supported by Mills' argument, as explained by Best (2010c, p. 7) that often 'personal problems are public issues.'

It is clear that the researchers feel they have obtained vital data from the case studies that strengthens their stance on action research as a method of school improvement (p. 75-76). There is clear evidence of varied interview methods including individual and focus-group interviews with varying structure, with more than one method used in the cases. One could suggest that the principles of method triangulation could also apply for analysing the use of different interview approaches.

Although there is variation in the method of data collection, the literature suggests that there are data collection methods that were not utilised in this paper, such as questionnaires, visual analysis (photographs, videos, being present), online interviews or use of data analysis software to explicate qualitative data.

It could be suggested that a more complete, higher quality of data could have been accumulated had there been more variation in the techniques used by the head teachers in the case studies, questions can then be raised about the quality of the action research workshop. Bearing in mind the context of the case studies, it is understandable that in developing nations, there may not be as much access to computers and therefore email, and that people may not have as much time to spend on data collection because of a need to be at work to make money. The data that was collected however was useful to each head teacher, evidenced by a considered approach to each intervention justifying the data as appropriate for each case.

The conclusions drawn by the researchers focus on the positives that action research offers, and the opportunities it offers, in these cases to the head teachers, schools and communities (p. 75-76). The authors however are involved in the organisation at the head of the project which has future aims to provide service in developing nations (p. 69-70). The quality of the analysis is also questionable, Harrison (2008) highlights two approaches to analysing qualitative data, and the researchers have only displayed evidence of the use of one of these. Although the purpose of the investigation clearly has value, dealing with issues of social justice, one could assume that the authors have not taken a solely humanistic approach. Due to their involvement and the entirely affirmative nature of the conclusions drawn, it is right to question the objectivity of the researchers as there may be an element of politically oriented evaluation evident (Best, 2010d, p. 1-2).