Critical Appraisal Skills Programme

This article discussed strengths and weaknesses of two reviews including one systematic review: Interventions for promoting the initiation of breastfeeding, and one tradition review: Breastfeeding Initiation and Duration: A 1900-2000 Literature review by using appraisal tool- CASP (Critical Appraisal Skills Programme – 10 questions to help you make sense of reviews).

The importance of quality assessment the of reviews

Hunt & Mckibbon (1997) stated that systematic reviews are a powerful and useful way to assemble evidence. However, just because a review has been done using systematic review methods does not guarantee that its results are credible. Regardless of the source, all systematic reviews (like all types of research evidence) require critical appraisal to determine their validity and to establish whether and how they will be useful in practice. What is critical appraisal? Young (2008) described that critical appraisal is a systematic process through which the strengths and weaknesses of a research study can be identified. This process enables the reader to assess the study’s usefulness and whether its findings are trustworthy and it provides a basis for decisions on whether to use the results of a study in clinical practice.

In order to practice evidence-based medicine, healthcare professionals need to apply the findings of scientific research to the situations of individual patients as part of their clinical decision-making process. For that reason, healthcare professionals must be able to select and appraise scientific literature that is relevant to their field, understand the implications of research findings for individual patients, elicit patients’ own preferences and develop an appropriate management plan based on the combination of this information (Young, 2008).

Get Help With Your Essay

If you need assistance with writing your essay, our professional essay writing service is here to help!
Find out more about our Essay Writing Service

Why choose CASP as appraisal tool?

The CASP appraisal tools are based on the guides produced by the Evidence Based Medicine Working Group, a group of clinicians at McMaster University, Hamilton, Canada, and colleagues across North America, published in the Journal of the American Medical Association. The tools were designed to address the epidemiological principles behind the study types with particular attention to assessing study validity. All the study tools are divided into three sections relating to internal validity, the results and the relevance to practice (PHRU Anon, 2007). And the purpose of using appraisal tool for review is to examine its validity, to analyze the result and to appraise its applicability and generalisability in clinical practice. The CASP tool assesses both internal and external validity. Therefore, I think it is suitable for appraising reviews.

The features of systematic and traditional review

A systematic review involves the application of scientific strategies, in ways that limit bias, to the assembly, critical appraisal, and synthesis of all relevant studies that address a specific clinical question. Systematic reviews can help healthcare professionals keep abreast of the medical literature by summarizing large bodies of evidence and helping to explain differences among studies on the same question and also high-quality systematic reviews can define the boundaries of what is known and what is not known and can help us avoid knowing less than has been proven (Cook, et al., 1997). By quantitatively combining the results of several small studies, metaanalyses can create more precise, powerful, and convincing conclusions (Cook, et al., 1997). It helps healthcare providers understanding a more comprehensive solution when they face a specific issue in clinical practice. By pooling together the results from several studies, the evidence drawn from systematic reviews can be also very powerful and influential in decision-making in clinical practice, research, and policy-making ( Ajetunmobi, 2002).

Most narrative review articles deal with a broad range of issues related to a given topic rather than addressing a particular issue in depth and they are less often useful in furnishing quantitative answers to specific clinical questions (Cook, et al., 1997). To sum up, traditional reviews provided a broader review in a subject while systematic review focuses on a specific question.

Critical appraisal for systematic review

Title: Interventions for promoting the initiation of breastfeeding

1. Did the review ask a clearly-focused question?

In the review, the population studied is all pregnant women, mothers of newborn infants and women who may decide to breastfeed in the future and the population subsets of women, such as from low-income or ethnic groups also included. The interventions given are any intervention target to promote the initiation of breastfeeding, which occur before the first breastfeeding. In addition, the outcomes considered in the review are specified in the objectives: 1. to identify and describe health promotion activity intended to increase the rate of initiation of breastfeeding; 2. to evaluate the effectiveness of different types of health promotion activity, in terms of changing the number of women who initiate breastfeeding; 3. to compare the effectiveness of different types of health promotion interventions as appropriate; 4. to assess the impact of these interventions on secondary outcomes, namely, duration of any or exclusive breastfeeding and any adverse outcomes as a result of the intervention.

The purpose of this review is to examine interventions which aim to encourage women to breastfeed, to evaluate their effectiveness on the number of women who initiate breastfeeding and to report any other effects (beneficial or adverse) of such interventions. From the population, intervention and outcomes mentioned, the review clearly established the question which also pointed out the purpose of the search.

2. Did the review include the right type of study?

To study the effects of interventions, it is necessary to compare a group of patients who have received the intervention (study group) with a comparable group who have not received the intervention (control group). A randomized controlled trial, which is a trial in which subjects are randomly allocated to the study or control groups, is usually the ideal design (Glasziou, 2001). The type of study the authors chosen in this review is randomised controlled trials, with or without blinding and no limitation of study by country of origin or language. In the paper, the authors examined more than 1400 titles and abstracts of studies and identified 83 potentially relevant studies and one author used a prescreen form to assess retrieved papers against the inclusion criteria and to classify included studies by the type of health promotion intervention.

3. Did the reviewers try to identify all relevant studies?

The authors clearly stated the inclusion and exclusion criteria about the types of participants and interventions. They searched the Cochrane Pregnancy and Childbirth Group’s Trials Register by contacting the Trials Search Co-ordinator including: 1. quarterly searches of the Cochrane Central Register of Controlled Trials (CENTRAL); 2. weekly searches of MEDLINE; 3. handsearches of 30 journals and the proceedings of major conferences; 4. weekly current awareness alerts for a further 44 journals plus monthly BioMed Central email alerts. The Trials Search Co-ordinator searches the register for each review using the topic list instead of keywords. The authors also scanned reference lists of all relevant papers retrieved and searched SIGLE, DHSS Data, and Dissertation Abstracts for grey literature. Furthermore, they contacted original authors to clarify or obtain relevant details of individual studies, particularly to request details of their randomisation processes. By thoroughly search, it can reduce the chance of publication bias and a large of possibly relevant studies could be identified.

4. Did the reviewers assess the quality of the included studies?

Empirical studies show that inadequate quality of trials may distort the results from systematic reviews and meta­analyses (Jüni, et al., 2001). In this paper, the validity assessment of each included study according to the criteria outlined in the Cochrane Handbook was done by two authors, which can minimize errors. They assessed selection bias on the basis of concealment of allocation: adequate; unclear; or inadequate. Performance bias, attrition bias and detection bias were rated as: adequate; unclear or partially adequate; or inadequate. The authors also explained which were the studies exclude and the reasons of their exclusion.

5. If the results of the studies have been combined, was it reasonable to do so?

The five studies in the paper evaluating the efficacy of health education interventions for increasing breastfeeding initiation rates combined by meta-analysis showed the interventions were effective overall (risk ratio (RR) 1.57, 95% confidence interval (CI) 1.15 to 2.15). Substantial statistical heterogeneity in this analysis is suggested by the heterogeneity score above 50% (I2 = 53.4%). The authors pointed out the studies were small, with the largest having 200 participants. All five evaluated studies were delivered in the USA to low-income women with different feeding intentions and where baseline breastfeeding rates are typically low. Moreover, programme components varied; however, all forms of health education included in this review seem to have increased breastfeeding rates. But these findings are based only on studies conducted in the USA; it raised some questions regarding generalisability to other settings.

6. How are the results presented and what is the main result?

Eleven studies were included in the review; statistical analyses were conducted on data from eight trials (1553 women). Five studies (582 women) in the USA with low incomes and typically low breastfeeding rates showed breastfeeding education had a significant effect on increasing initiation rates compared to standard care (risk ratio (RR) 1.57, 95% confidence interval (CI) 1.15 to 2.15, P = 0.005). Subset analyses revealed that one-to-one, needs-based, informal repeat education sessions and generic, formal antenatal education sessions are effective in increasing breastfeeding rates among women on low incomes regardless of ethnicity and feeding intention. Needs-based, informal peer support in the antenatal and postnatal periods was also shown to be effective in one study conducted among Latina women who were considering breastfeeding in the USA (RR 4.02, 95% CI 2.63 to 6.14, P < 0.00001).

7. How precise are these results?

If we want to know the range of values within which we can affirm with some confidence (usually 95%) that the estimated effect will occur in the general population, we must make use of the confidence intervals. The narrower the range included in the interval, the more precise the estimation of the result will be, and it will be possible to get a more reliable idea of the true effect of the treatment (Abalos, 2001). In this paper, Studies were analysed within four types of intervention: health education (risk ratio (RR) 1.57, 95% confidence interval (CI) 1.15 to 2.15), peer support (RR 4.02, 95% CI 2.63 to 6.14, P<0.00001), breastfeeding promotion packs (RR0.93, 95% CI 0.80 to 1.08, P=0.34), and early mother-infant contact (RR 1.05, 95% CI 0.94 to 1.17, P = 0.39). According to the data above, I consider the interventions of health education and peer support are likely to increase the improvement of promoting the initiation of breastfeeding.

8. Can the results be applied to the local population?

In the studies of health education and peer support intervention, all the research were conducted in low-income women in USA with different ethnic background and feeding intention. In this condition, the application is less likely to our population. The result of early mother-infant contact followed by complete separation until hospital discharge was not effective when targeting women with high breastfeeding rates living in poor urban areas in a low-income country setting. Nevertheless, in Ten Steps of the UNICEF/WHO Baby Friendly Hospital Initiative, mothers and infants remain together for 24 hours a day, is a global programme to support successful breastfeeding and demonstrated to increase initiation rates for all women in all settings. In this way, I assume that the intervention of early mother-infant contact is applicable to our population.

9. Were all important outcomes considered?

In the paper, the authors suggest that the generalisability of findings in health education and peer support intervention should be more considered due to small number of trials and the characteristics of patients of studies. Although the result of the intervention of early mother-infant contact in the paper was not as effective as we expected, according Ten Steps of the UNICEF/WHO Baby Friendly Hospital Initiative, we should consider that the intervention of early mother-infant contact as an implementation in our clinic practice.

10. Should policy or practice change as a result of the evidence contained in this review?

The authors imply the effectiveness of interventions reviewed in the paper needs to be assessed in diverse countries and settings, especially outside the USA and suggest future research should aim to evaluate the efficacy of the intervention to improve both the initiation and duration of exclusive breastfeeding at least up to six months to enable proper planning and implementation of interventions during pregnancy and the postnatal stage. Thus, according to the authors’ conclusion, I assume policy or practice should be remained.

Critical appraisal for traditional review

Title: Breastfeeding Initiation and Duration: A 1900-2000 Literature review

1. Did the review ask a clearly-focused question?

In the review, we only can find the outcome measures from the objective: to review the literature on breastfeeding initiation and duration and to delineate effective strategies for promoting positive breastfeeding behaviors. The author didn’t specify the type of population, for example, the specific characteristic of the patients in whom the intervention will be evaluated or the type of intervention. Thus, the paper did not establish a clearly-focused clinical question to which it related to.

2. Did the review include the right type of study?

The types of study the author selected in the paper are randomized controlled trials, meta-analyses, studies with the largest, most representative samples and investigations conducted in North America. The author chose randomized controlled trials as study which is reasonable, but the author didn’t explain the meaning of the largest, most representative samples and why select studies which located in North America.

3. Did the reviewers try to identify all relevant studies?

In this paper, the author uses MEDLINE, CINAHL, and the Cochrane Library as database search. However, the author neither mention the inclusion and exclusion criteria about the types of participants and interventions nor follow reference lists of all relevant papers retrieved. Besides, the author did not mentioned that she contacted original authors to clarify or obtain relevant details of studies or search for unpublished studies and non-English language studies. Because of so, there is high probability that some relevant studies may have been omitted or increase the chance of publication bias.

4. Did the reviewers assess the quality of the included studies?

The author did the data extraction but did not do the validity assessment for each included study. Abalos (2001) suggested, by answering the questions about methodological validity it is possible to determine the degree of reliability of the results of the review and hence to decide whether it is worth reading it or not. Therefore, we can not be convinced by the validity of the review’s result without assessing the quality of the included studies.

5. If the results of the studies have been combined, was it reasonable to do so?

The results of studies were classify by following headings: benefits of breastfeeding, breastfeeding initiation and duration, personal characteristics, attitudinal and intrapersonal characteristics, hospital policies and intrapartum experience, sources of support, breastfeeding interventions, and review implications. By doing so, we can’t notify the similarity and difference in characteristics of the studies.

6. How are the results presented and what is the main result?

The results of studies were presented by different headings. The main results found in the review are women least likely to breastfeed are those who are young, have a low income, belong to an ethnic minority, are less supported, full-time employed, decided to breastfeed during or late in pregnancy, have negative attitudes about breastfeeding, and have low confidence in their ability to breastfeed. Support from the mother’s partner or a nonprofessional greatly increases the positive breastfeeding behaviors. From the results of the review, the author implies that healthcare professionals can be a negative source of support if their lack of knowledge results in inaccurate or inconsistent advice. Although professional interventions that enhance the usual care mothers receive increase breastfeeding duration to 2 months, these supportive strategies have limited long-term effects. Peer support interventions also promote positive breastfeeding behaviors and should be considered.

7. How precise are these results?

In the paper, the author did not collect the original meta-analysis to make a more powerful quntatitive synthesis but put the original meta-analysis into related heading which made the review less convincing. The original meta-analyses below which are represented statistical significance in the study.

1. A retrospective cohort study of 434 women infected with HIV-1, breastfeeding was significantly associated with maternal vertical transmission, after controlling for the stage of maternal HIV-1 disease (odds ratio [OR] = 2. 95%, confidence interval [CI] = 1.3-3.8).

2. A study of 350 New Zealand women, Vogel, Hutchinson, and Mitchell reported that younger women were at a greater risk for shorter breastfeeding duration (relative risk [RR] = 2.33, 95% CI = 1.33-4.05).

3. A descriptive study of 198 pregnant women, O’Campo et al. examined 11 psychosocial and demographic variables and found maternal confidence to be 1 of 5 variables that significantly influenced breastfeeding duration. Women with low confidence in their perceived ability to breastfeed were at three times (RR = 3.1, 95% CI = 1.39-6.76) the risk of discontinuing breastfeeding when compared with very confident women who were breastfeeding.

4. Blomquist, Jonsbo, Serenium, and Persson followed 521 breastfed infants for 3 months and found that supplementary feedings with formula shortly after birth were independently associated with early cessation of breastfeeding (OR =3.9, 95% CI = 2.1-7.2) when compared with exclusive breastfeeding.

5. Women with low confidence in their perceived ability to breastfeed were at three times (RR = 3.1, 95% CI =1.39-6.76) the risk of discontinuing breastfeeding when compared with very confident women who were breastfeeding.

6. A study of 200 women, Giugliani et al. concluded that, regardless of maternal age, education level, ethnicity, and marital status, women who indicated that their partners preferred breastfeeding were significantly more likely to initiate breastfeeding when compared with women whose partners were ambivalent or preferred bottle-feeding (OR = 32.8, 95% CI = 6.7-159.5).

8. Can the results be applied to the local population?

In this paper, because without rigorous quality assessment, I assume the results can not be applied to our population. Have saying that, this review provided a panorama in personal characteristic/hospital/support/intervention strategy for promoting positive behaviors.

9. Were all important outcomes considered?

Without knowing the validity of the paper, I suggest that only the meta-analyses in the review might be considered.

10. Should policy or practice change as a result of the evidence contained in this review?

After appraising this paper, I think the policy or practice should not be changed due to the less explicit methodology.

Evaluation

For the two review papers, we can find that in systematic review, a clearly-focused question can formulated by particular population, intervention and outcomes which made a very explicit objective and target while in traditional review, the search object is vague. When the two papers both included randomized controlled trials (RCT) to evaluate the result, the systematic review exercise RCT as a primary data source and incorporate the data into meta-analysis which made the result more powerful and convincing. In contrast, the traditional review also uses RCT but only use in data synthesis which made the result less credible. In the systematic review, the author stated a clearly inclusion and exclusion criteria and used very detailed method to search relevant paper which can avoid the publication bias and omit the potential related research while the traditional review didn’t identify the relevant study explicitly.

In systematic review, the author assessed the validity of each included study whereas the traditional review did not. Because the latter review didn’t assess the quality of included studies made its result less compelling. In terms of results, the author of systematic review combined the original studies and produced the meta-analysis made the result more powerful while the results in traditional review are described context.

Conclusion

A systematic review usually follows rigorous methodology to identify, to interpret, and to evaluate the relevant studies. A good systematic review not only can specific in a particular issue and establish a significant finding but also generate a generalisability for healthcare providers, researchers and policy makers when they experience a very special circumstance for decision making.

For traditional reviews, they do not usually follow any specific rules. The principles in the traditional review are usually followed by the author’s viewpoint and it becomes more subjective. Yet, when a traditional view is conducted methodically and the data was synthesized with valid and planned, it can be developed a broader view for readers.

As healthcare professionals, we have to update ourselves with new technology and profound knowledge. When we face a clinical problem and seek for answer in research studies. We need to remind ourselves the paper we read is a good quality of research study or just another data combination. In light of this, we must learn how to appraise research study evidence critically to enhance our profession and decision making ability when we in clinical practice.

Reference

Abalos, E., Carroli, G., Mackey, M.E., & Bergel, E., 2001. Critical appraisal of systematic reviews. The WHO Reproductive Health Library, No 4, Geneva. The World Health Organization (WHO/RHR/01.6).

Ajetunmobi, O., 2002. Making sense of critical appraisal. New York: Oxford University Press.

CASP, 2003. Critical Appraisal Skills Programme (CASP): appraisal tools. Public Health Resource Unit, NHS.

Cook, D.J., Mulrow, C.D. & Haynes, R.B., 1997. Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions. Ann Intern Med.126, pp.376-380.

Glasziou, P., Irwig, L., Bain, C., & Colditz, G., 2001. Systematic review in healthcare: a practical guide. Cambridge: Cambridge University Press.

Greenhalgh, T., 1997. Papers that summarise other papers (systematic reviews and meta-analyses). British Medical Journal, 315, pp.672-675.

Hunt, D.L., & Mckbbon, K.A., 1997. Locating and Appraising Systematic Reviews. Annals of Internal Medicine, 126(7), pp.532-538.

Jüni, P., Altman D.G., & Egger, M., 2001. Systematic reviews in health care: Assessing the quality of controlled clinical trials. British Medical Journal, 323, pp.42-46.

Katrak, P., Bialocerkowski, A.E., Massy-Westropp, N., Kumar, S., & Grimmer, K.A., 2004. A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology, 4:22.

Public Health Resource Unit (PHRU), 2007. phru FAQ’s. Available at: http://www.phru.nhs.uk/pages/phd/FAQs.htm [Accessed 20 Dec 2005]

Sanderson, S., Tatt, I.D., & Higgins J.P., 2007. Tools for assessing quality and susceptibility

to bias in observational studies in epidemiology: a systematic review and annotated bibliography.

International Journal of Epidemiology. doi:10.1093/ije/dym018.

The Centre for Reviews and Dissemination, 2009. Systematic reviews: CRD’s guidance for undertaking reviews in health care. York: University of York.

Young, J.M., & Solomon, M.J., 2009. How to Critically Appraise an Article. Nature clinical practice Gastroenterology & Hepatology, 6(2), pp.82-91.