Truancy is the unauthorized absence from school without parental consent or knowledge. This seemingly minor social issue has been identified as needing attention because it is the first sign in a series of deviant behaviours that may lead to negative outcomes such as sexual promiscuity, drug and alcohol use, delinquency, and attrition (Bell, Rosen, & Dynlacht, 1994; Van Petegem, 1994, as cited in Teasley, 2004). Thus the notion of prevention rather than cure has led to numerous early intervention programs being implemented in schools to reduce truancy.
The following school-police partnership in a Truancy Recovery Program is no different. The purpose of the following program was to return truants to school as fast as possible. The police were involved because of the recognition that it was their responsibility to identify the paths frequently leading to homicide and eliminate them through early intervention.
Participants. A sample of 178 truants from more than 15 different schools ranging from elementary to high school in Richmond, California were selected at random (White et al., 2001). The median age of the participants was 15.
Design and materials. A single-group pre-test post-test design was used. The independent variable was the program and the dependent variables were (1) school attendance, (2) academic grades, (3) school disciplinary incidents, and (4) detailed criminal and juvenile justice data.
Procedure. During the time the program was run, eight police agencies were given the authority to take temporary custody of suspected truants during school hours to transport them to their school offices and to contact their parents. Repeated offenders were subjected to in-school suspension where the goal was to reintegrate truants into the school rather than punishing their truancy with more time away from school (e.g., out-of-school suspension) and, depending on the severity, to the juvenile court which could impose sanctions (e.g., suspension of driving license, fines, community service). The program had been implemented in 1994 in other states and was only introduced to Richmond in the Fall semester of 1997.
Analysis and Results
The researchers did analyses for the pre-test post-test comparison, missing data, and grade-specific analysis. The pre-test post-test comparison yielded positive results with decreased unexcused absences (from 90% to 82%); slight improvement in grades (median number of D and F grades per student fell from 7 to 5); and decreased disciplinary incidents (median number fell from 5 to 1). The issue of missing data was investigated to evaluate if the positive results were because the data of poor performers were missing. Analysis suggested that the missing data was not related to the improved grade performance, attendance, and behaviour. The grade-specific analysis sought to check whether grade improvements were uniform across the sample or just among a particular group of students. Analysis of individual studentsââ‚¬â„¢ grades across the program period indicated that improvement was only minor and not significant. Put simply, the program appeared to be successful in improving school civility in terms of attendance and behaviour, but not grades.
Critiquing the Evaluation
Strengths. The evaluators assessed the sample to be fairly representative of schools throughout the district (White et al., 2001). They also noted that the male to female ratio was 2:1 and that there was great disparity in cultural backgrounds with 60% African-American, 25% Hispanic, 8% Asian, and 3% White.
Weaknessess. While the evaluators described the demographics rather well, they failed to mention explanations and possible implications for the gender and cultural disparity. The gender disparity could have been due to girls being better able to avoid the police, or bias on the part of the police officers who tended to seek out boys. Either way, this resulted in a 2:1 ratio which may not have been a true reflection of demographics of those playing truant. The girls who were not recorded in this sample may have provided valuable data.
As for the cultural disparity, according to Teasley (2004), the largest school systems in inner cities like Richmond, have student cohorts where African-Americans constitute about 70% and Hispanics nearly 20%. This statistic could rule out the possibility of cultural bias in the sample collected (e.g., cultural bias led police to pick up more African-Americans than Whites). This data could have further supported population validity in that the sample was truly representative of the actual schooling population.
Design and materials.
Strengths. The evaluators identified the limitation that the truancy sample construction and the qualitative nature of research prohibited a non-truant comparison sample (White et al., 2001). The evaluators recognised that the design made the study less like a formal program evaluation.
Weaknesses. Although commenting that the design did not allow causal relationships to emerge, the evaluators did not suggest alternative designs which might improve the program. For instance, given comparison samples were not viable, an interrupted time-series design (i.e. ABAB design) could have been used to control for maturation and history effects (Posavac, 2011, p. 187).
Analysis and Results
Strengths. The evaluation team analysed non-truant criminal arrests and found a 4% increase in number of truants arrested following the program (White et al., 2001). This information was not discussed by the original researchers who may have deliberately left it out. The evaluators attempted to explain the undesirable results of increased criminal arrests using the internal validity threat of maturation. According to White and colleagues (2004), given the median age of truants being 15, by the time the post-test period arrived, most of the truants had reached their 16th to 18th birthdays, which prior research suggests is the most active age group for delinquency and crime-committing. Thus the higher arrest rates were not a result of the program but rather the entering of late adolescence.
Weaknesses. Other validity threats that could have accounted for the undesirable finding of higher arrest rates were not discussed. One possible threat is history effect because the dot-com bubble recession from March to November 2001 had occurred during the post-test period and this might have resulted in more financially constraint teenagers resorting to criminal behaviour. Testing is another internal validity threat where the police may have become familiar with the truant teenagers, and thus may have patrolled areas frequented by them more often following the program, leading to higher arrests.
Strengths. The evaluators questioned whether the program purpose of reducing truancy was truly met because despite the improvement in attendance (from 42 to 23 days a year), students still missed more than four weeks of academic year. Furthermore, the negligible improvement in grades and the increased arrest rates were disconcerting findings pointed out which might have reduced the overall efficacy of the program.
Weaknesses. The fact that the evaluators were questioning if the program goals were achieved indicates that the final outcome was not explicit enough. Tangible criteria should have been set by the stakeholders (e.g., 50% cut in absenteeism). Intermediate outcomes should have been formulated to help assess whether the goals were met. Additionally, the dependent variables need to be more clearly defined. The program measured school attendance, behaviour, grades, and criminal arrests. The program goals should have defined which was to be the main variable to distinguish whether the program was successful or not.
Meeting Program Evaluation Standards
The following are comments on the evaluation based on the program evaluation standards listed by Yarbrough, Shulha, Hopson, and Caruthers (2011).
Evaluator credibility. The standard was met because the four evaluators were qualified individuals with experience and knowledge in criminal justice. White, Fyfe, and Goldkamp hold doctorate degrees in criminal justice and are all professors of criminal justice in different Universities. Campbell holds a masterââ‚¬â„¢s degree in criminal justice and is an assistant managing director of the City of Philadelphia.
Attention to stakeholders. The standard was not met because the evaluation focussed on the participants and failed to give attention to the other stakeholders (e.g., the police departments, the government, teachers, parents, etc.). The stakeholders and their needs could have been briefly recognised in the report so that any biases could be detected (e.g., The government that was probably sponsoring the program may have pressurized the researchers to ignore the program limitations which could explain why the increased arrest rate was not originally detected.)
Negotiated purposes. The standard was not met. The program did state that the purpose was to keep youth in school and off the streets during school hours. The belief was that this would close the paths to delinquency and criminal behaviour. However, despite students staying in school longer during the program, the criminal arrest rates still increased following the program. The evaluators should have negotiated the true purpose of the program which was to close the paths that lead to homicide. In doing so, stakeholders might have been able to identify other ways of defining the program purpose (e.g,. keep youth off the streets during school hours so they do not associate with bad company and criminal behaviour and so they have less time to engage in criminal behaviour).
Explicit values. The standard was met because the researchers established the underpinning value that recognizing homicide prevention was a key police responsibility best accomplished by recognizing paths leading to homicide and closing them. This explicit value was common to many of the stakeholders such as the government, the police, school administration, and the parents.
Relevant information. The evaluators were mostly able to keep to relevant information (e.g., background information was brief and redundant information like sampling procedure was omitted). Furthermore, the use of subheadings and graphs facilitated easy perusal. However, the section on describing analysis of the missing data seemed unnecessarily lengthy.
Timely and appropriate communicating and reporting. The evaluators did not mention how or when they communicated with and reported to stakeholders. If when the evaluation was published was when stakeholders got information, a 2001 evaluation for a program that ran for three-months in 1997 might have been too late. Understandably this late publishing date was to cater to using data 18 to 21 months after the program for the post-test comparison. However, it might be more desirable to reduce this time period or instead do multiple post-tests and release draft versions so that there can some results to report to the stakeholders.
Concern for consequences and influence. This standard seems to have been met because while the original researchers failed to report the negative finding of increased arrest rates, the evaluators did highlight it and even attempted to explain it. The evaluators also showed concern for consequences by weighing the benefits and costs of the program (i.e., improved attendance but increased arrest rates) in the discussion section and questioned whether the program was worth the costs.
Practical procedures. Evaluation procedures were practical in having a data collection method of low cost which was also non-disruptive to stakeholders. The only issue might be the large quantity of data required (e.g., data from three years prior to the program, year of the program, and 18 to 21 months after the program) which would have made data analysis more taxing.
Resource use. This standard was met because given that the dependent variable measures were cheap and easy to obtain, resource use for the evaluation was minimal. For example, school attendance and criminal justice data would have been collected by the school and police respectively. All that needed to be done was to obtain permission to access the data. This is as opposed to say holding an open forum to discuss issues with the police or administering surveys to the truant students.
Responsive and inclusive orientation. The standard was not met. As the stakeholders were not clearly identified in the report in the first place, there was no information on whether the evaluators were responsive and inclusive.
Human rights and respect. The program was implemented without consent of stakeholders such as parents or the participants themselves. As a result, their rights may have been violated. For instance, if an 18 year old student, who is legally an adult, decides to drop out of school, he or she should have the right to do so especially when the state has no compulsory education policy for high school and college.
Clarity and fairness. The standard may not have been met because the program was implemented only in certain states and not nationwide. The evaluators failed to mention this arguable injustice where students in Richmond have to be subjected to the humiliation of a police pickup while truants from other states do not. The police departments may also feel the injustice where they have the extra duty of having to patrol and pickup truants as well as the tedious paperwork to be completed.
Transparency and disclosure. The evaluation report disclosed detailed and complete descriptions of findings, limitations, and conclusions and was made available to all stakeholders by being published in a public journal Evaluation Review.
Conflicts of interests. The evaluators did somewhat address the conflict of interest of whether the program was worth it by highlighting the contradiction in wanting to decrease truancy to prevent further criminal behaviour at the cost of higher arrest rates amongst truants. Perhaps reducing truancy was a more desirable goal to the school board, while reducing criminal behaviour was preferred for the police department.
Justified conclusions and decisions. Given that truancy is a social issue common to nations across the globe with education systems (Sutphen, Ford, & Flaherty, 2010), the evaluation conclusions may be justified even though the cultural context was not stated.
Valid information. The evaluation did serve the intended purpose of a summative evaluation. The evaluators further analysed the validity of the original researchers findings and found results in support of the conclusion that the school attendance did increase following the program.
Reliable information. Evaluatorsââ‚¬â„¢ analyses yielded results that were consistent what that of the researchers. This might indicate that the evaluation procedures were sufficiently reliable. The evaluators should however have discussed reliability in greater detail.
Explicit program and context descriptions. The evaluation was considerably long with 25 pages. There was adequate documenting of the program and evaluation. There was appropriate detail in descriptions which was aided by six tables and two graphs.
There is value in the evaluation simply because it did show a positive association between improved school attendance and the Truancy Recovery Program. At present, there is a scarcity of evidence-based truancy programs, and even if a strong evidence base is established, it would be limited when programs are not evaluated to assess their effectiveness. A singe truancy program alone purporting significant positive results is insufficient for sponsors and governments to pour in resources to implement it. Program evaluation is needed to determine utility, relevance, practicality, and scientific validity. There is consideration of the opportunity cost because there are always other options for early interventions.