Programme evaluation is one of the key skills required of public health professionals. Evaluating healthcare interventions is important in order to allocate resources efficiently, assist decision-making and inform policy.It is also important in order for programme managers to monitor progress in achieving set objectives, improving programme activities, justify need for continuity and accountability to project funders.
The proposed evaluation would be carried out using the CDC framework of programme evaluation which has six steps.This evaluation would be somewhat pluralistic in nature as there would be an interplay of the perspectives of different groups.However, the dominant perspective would however be that of the policy makers-a managerial perspective.This evaluation would be carried out three years after the start of the programme. It would take a long time to achieve the overall programme aim, hence the need for reliance on short or intermediate indicators (Parry)cited in. An evaluation working team would be set up for the proposed evaluation and a leader would be appointed.
In this evaluation, the author would be the lead evaluator and would be responsible forselecting members of the evaluation team, scheduling meetingswith stakeholders, planning, and budgeting funds for the evaluation, addressing data collection needs, reporting evaluation findings, and working with consultants whose services may be required. Prioragreement would be on the reasons for carrying out the evaluation, the primary user of the evaluation findings, funding arrangements, time-line for completion of the evaluation and reporting of its findings.
The programme mainly targets postmenopausal women between 48 and 63 years as breast cancer commonly affects this population. However, the 40-47 age group would be included as early presentation occurs commonly in this population. The key stakeholders involved in the programme are the clients, the Federal Government of Nigeria, the Federal Ministry of Health, Oyo State Ministry of Health, the Ministry of Women Affairs, University College Hospital, Ibadan, programme managers, and staff. Professional associations-Nigerian Medical Association (NMA), Association of Radiologists of West Africa and the Medical Women Association ofNigeria (MWAN) -advocate for the implementation of a national breast cancer screening programme and policy which is currently non-existent[5, 6]. Other stakeholders are advocacy groups-the Genevieve Pink Ball Foundation , Breast Cancer Association of Nigeria (BRECAN),Medicines Plus; media groups-Silverbird Entertainment, Inspirational FM, City People, Genevieve magazine, Globacom Limited-a leading telecommunications firm.
The primary users of this proposed evaluation are the policy makers and the programme management.The findings would determine if the programme is worth its ‘value for money’ and would help inform decision on the whether the programme should be continued and if a screening policy would be implemented. The purpose of the evaluation would be made clear and agreed upon by all key stakeholders from the outset.
It would have been possible to develop the logic model for this evaluation using the theory of change approach with the stakeholders. Defining a theory of change which is agreed by all stakeholders can mitigate the effects of causal attribution.One representative would be selected from each key stakeholders group to form an evaluation working group. The stakeholders in the team would be asked about their vested interests in the programme, their expectations, and what resources they would put in during the actual implementation of the evaluation.
The Ibadan-Ibarapa Breast Cancer Screening Initiative
The programme was commissioned in December, 2006 in response to the growing concern for the need for early detection of breast cancer. The programme aims is to reduce incidence and mortality from breast cancer in women. Breast cancer screening is a secondary prevention programme that takes the form of physical examination using in combination with radiographic techniques-mammography. Health needs assessment using the epidemiological approach conducted prior to the implementation of the programme revealed that breast cancer is the most common cause of female cancer deaths in Nigeria.
Breast cancer is the most common cancer affecting women globally and in Nigeria [5, 8-11].It accounts for the greatest proportion of cancer deaths in women in Nigeria. Previous surveys showed a rising prevalence in breast cancer in Nigeria from 33.6 per 100,000 in 1992 to 116 per 100,000 in 2001.A retrospective review of hospital-diagnosed cases of cancer revealed 1,216 cases over a four-year period with breast cancer accounting for 13.9% of the cases .Differences exist between the aggressiveness and survival rates in Nigerian women when compared to other populations[13, 14]. The mean age at presentation is 48 years. The average age at menopause of Nigerian women is 48 years[15, 16]. The intervention targeted at women between 40 and 64 years as early presentation-before menopause-has been observed to be the dominant pattern in this population.
There is generally a low level of awareness and knowledge of early detection breast cancer among Nigerian women [8, 10, 17-19].Despite the burden, there is currently no national policy or programme for early breast cancer detection[5, 18]. There is a high mortality from breast cancer in Nigeria. This is because most present for treatment late[18, 20-22] and tumours are more aggressive, responding poorly to medical treatment.Cultural factors and religious influences also affect health seeking behaviours with some detected cases seeking healing from lay or traditional practitioners.
Risk factors for developing breast cancer include age at menopause, at age at first live birth and parity, history of first-degree relative with breast cancerand smoking. Research evidence supports that early detection of breast cancer leads to improved outcomes. Surveys investigating the knowledge, attitudes and practice of female health care workers showed that with the exception of physicians, there are gaps in knowledge of the risks of breast cancer and low practice of early detection methods [6, 25].
The activities being carried out in this programme are:
- Breast cancer seminars and workshops for nurses, physicians, community health workers, public health nurses and health promotion specialists in order to increase their knowledge of breast cancer risks and change their attitudes towards screening practices
- Clinical Breast Examination (CBE) trainings using breast models -donated by the University of Chicago Medical Centre-in order to enhance clinical skills in breast lump detection.
- Breast cancer education workshops and lectures for postmenopausal women and teaching skills in Breast Self Examination (BSE) to increase their knowledge of the benefits of early detection of breast cancer and to change their attitudes and beliefs on breast cancer
- Counselling sessions by breast cancer survivors who are volunteers trained to help motivate women in utilising the screening service and reassurance that the benefit of screening outweighs any discomfort felt during the mammography procedure
- Distribution of IEC (Information, education and communication) materials breast cancer facts leaflets and posters to increase knowledge of breast cancer risks and the benefits of early detection
- Media campaigns through paid and unpaid advertisements in newspapers and magazines widely read by women; radio jingles in English and the local dialect; role plays on television by volunteers; health promotion messages on television aired in English and Yoruba languages
- Provision of free Clinical Breast Examination (CBE) and free mammography screening to postmenopausal women between ages 40 to 63 years
Funding is mainly from the Federal Government with equal support from the Breast Cancer Research Foundation and the Genevieve Pink Ball Foundation. Shortage of health care staff and inadequately trained staff contribute to mortality from breast cancer. Cultural and religious beliefs also influence cancer health-seeking behaviours. There is increasing interest in cancer issues and many agencies are getting involved in breast cancer awareness activities.
Focusing the evaluation design
Evaluation design and perspective taken by an evaluation depends on who the primary user of the evaluation is.The proposed evaluation takes the managerial perspective. The primary users of this evaluation are the programme managers and the policy makers (health systems). At this stage of the programme, three years after its implementation, it is inappropriate to make judgement based on the achievement of long-term outcomes. It will thus assess the changes in the knowledge, attitudes, and beliefs of the clients concerning breast cancer and the utilisation of the screening service.
Purpose of the evaluation
In future, evaluating the programme would determine if the breast cancer awareness programme and screening leads to a reduction in incidence and mortality from breast cancer in women aged 40-63 years .The proposed evaluation would carried out in order for policy makers to decide the effectiveness of the programme in promoting health i.e. to evaluate the effectiveness of breast cancer screening as a public health policy; to make decisions about continuity of the service in light of its cost-effectiveness in the face of other healthcare needs competing for funds and to assist in policy making and implementation of a national breast cancer screening programme. It would also assist the programme manager in judging fidelity of implementing the programme activities.
The main evaluation questions to be dealt with can be thus can be summarised as follows:
- Effectiveness-‘Is the programme working as intended?’ i.e.is it achieving the objectives it set out in its mandate?
- Efficiency- ‘Are the programme activities being produced with minimal use of resources?’
The proposed evaluation would use the Before-After (Type 3) evaluation approach in assessing the effectiveness of the programme in producing the change in knowledge, attitudes and behaviour that would ultimately lead to achievement of the programme’s goal of reducing the incidence and mortality from breast cancer. This evaluation needs to be carried out in a relatively short period and funding for the evaluation is limited. While true randomised controlled trials are said to be the gold standard for evaluation, they would be unethical; expensive to conduct and take a really long time to complete. The main problem with this before-after design is the issue of causal attribution- that the results may not be conclusive in proving that the outcomes are as a result of the programme activities.The opinions and inputs of the key stakeholders would also be taken into account at this stage.Stakeholders opinion would be sought on how the evaluation would be funded ,how long it would be carried out for and what the reasonable level of achievement should be. The number of years the programme has been in existence would help set the standard against which progress is monitored.
Gathering credible evidence-Data collection
The mixture of quantitative and qualitative methods would be used in data collection.Observational methods would be used in collecting some primary data for the evaluation as there are few existing databases from which the relevant information for the evaluation can be obtained. Green and South stress that selecting methods for measuring outcomes of health promotion programmes should be based on concern for “reliability, validity, suitability for purpose, feasibility, consistency with the values and methods of working of the project and appropriateness for use with various groups”.Although validity is recognised as being very important in selecting the indicators for evaluation, practical feasibility should also be put into consideration. McNamara fallacy-‘making the measurable important rather than the important measurable’.
If you need assistance with writing your essay, our professional essay writing service is here to help!Find out more
Change in the knowledge, attitudes and practice of healthcare workers would be assessed using a structured self-administered questionnaire placed in the staff pigeon holes at the teaching hospital where the programme activities are carried out. The questionnaire would be pre-tested prior to its use for the evaluation. Pertinent questions would reflect social and demographic characteristics of the respondents, level of knowledge about breast cancer, risk factors for its development, symptoms, screening methods known as well as their individual practice of breast self examination (BSE) and screening using mammography as appropriate. The questions would be closed and pre-coded responses would be included in the questionnaire as these are quick to analyse.Careful attention would be paid to the wording of the questionnaire and how the information is coded during its design in order to ensure its reliability and validity. The healthcare workers would be sampled using the stratified random sampling technique based on age and gender. Responses would be anonymised to reduce the chances of reporting bias. The number of healthcare workers would be determined from the training register kept by the programme monitoring and evaluation staff and it can be estimated from the survey in the absence of fidelity in the recording of programme activities.
In order to assess if the healthcare workers have developed the appropriate clinical competence in Clinical Breast Examination (CBE).Participants would be observed carrying out the examinations. A hospital researcher skilled in ethnographic methods would be called upon to engage in this aspect of the evaluation. The limitation of this method of investigation is that observer bias is an issue to contend with and the method is not objective.To limit observer bias, a different observer would also assess the trained participants-inter-observer comparison.The contribution of the Hawthorne effect-a reactive effect which produces bias- is also a limitation worth noting .If healthcare staff are aware that they are being studied, they strive to demonstrate that they have the required level of competence in carrying out the breast examination.
Changes in the knowledge, attitudes, and practice of screening in the clients would also follow the before-after method of investigation. Study participants would be randomly selected from the community. Informed consent would be ensured before data collection. Data would be obtained using structured pre-coded questionnaires with allowance for open-ended questions delivered via face-to-face interviews trained interviewers. The interviewers would trained on the social skills of establishing good rapport with people in order to reduce potential bias.Information obtained would include socio-demographic variables- age, marital status, level of education, religion, occupation-potential risk factors for developing breast cancer-family history of breast cancer, parity, duration of breast-feeding age at onset of menstruation, age at menopause-knowledge of screening methods-Breast Self Examination(BSE),Clinical Breast Examination(CBE) and mammography and on utilisation of screening services.Barriers to accessing service can also be explored. Mailed questionnaires would have taken less time and would be cheaper to administer but consideration is given to the fact that most people in this community do not have mailing addresses and the literacy level is generally low.
Data collected from the surveys would be analysed using appropriate tests with statistical software. The main exposure variables of interest the educational component of the programme and the physician skills training in Clinical Breast Examination(CBE) while the main outcome of interest is the utilisation of mammography as a screening service. Secondary variables of interest are the media campaigns, healthcare workers training, Clinical Breast Examination Other factors that would be explored would be the effect of some key demographic characteristics like age, ethnicity, and level of education on the utilisation of the mammography service. In conducting the analysis, adjustment would be made for potential confounding by other influences.
The before-after status of the women receiving the educational programme would be compared. For the uptake of mammography as a screening method comparison before and after status would also be compared as though a few other mammography centres exist, none is currently running a similar awareness programme that may qualify its use as a non-random control.
The overall results obtained would be compared with the set standards as outlined in the programme objectives or in the absence of this, what was agreed upon at the stakeholders meeting as a measure of success
Ensuring use of the evaluation findings
The findings would be put together as a draft after representing the data in well-presented graphs and tables as appropriate and circulate among the stakeholders. After reviewing the draft with the stakeholders, the results would be disseminated widely in the hospital journals, as a government publication, in women magazines and other publications as defined by the stakeholders.
The results obtained are expected to help inform the development and implementation of a national breast screening policy and programme.
There are a number of frameworks that have been developed for investigating the quality of healthcare services. The Maxwell’s framework would be used for assessing the quality of the screening service.
Maxwell’s conceptualisation of health care quality is organised around the following six dimensions:
- Access to services-accessibility in terms of time ,distance and location of services, language and other cultural barriers
- Relevance to need (for the whole community)
- Effectiveness (for individual patients)
- Equity (fairness)
- Social acceptability(cultural competence)
- Efficiency and economy.
The effectiveness of mammography as a screening method has been proven to be beneficial by some studies. A case-control study reported a 50% reduction in mortality from breast cancer using mammography hence supporting its efficacy.A recent case control study also demonstrated similar results. Collette et alevaluated a breast cancer screening programme using different methodologies and found that early detection using mammography reduced mortality in the 50-64 age groups. Though a previous meta-analysis demonstrated no benefit to women in the 40-49 age group,the well cited Swedish trials have however suggested the need for screening in this age group.
Efficiency and economic evaluation
This would answer the evaluation question: ‘are we making the best use of limited resources?’ The economic evaluation of this programme would take the managerial standpoint to answer the policy makers question on if it is the best ‘value for money’. It is important in making resource allocation decisions. The overall summary of the programme can be succinctly stated as:
Activities- breast cancer education workshops, lectures, and seminars, Clinical Breast Examinations (CBE) for healthcare workers distribution of information leaflets and posters, media campaigns, mammography.
Outputs-Number of educational sessions held, number of healthcare workers trained, no of women reached by educational programme, number of women counselled, number of women screened, and number of IEC materials distributed
Outcomes -increased awareness and knowledge of breast cancer risks and screening methods (short-term outcome)
-increased awareness of early detection (intermediate outcome)
-increase in uptake of breast cancer screening (behaviour change)
Goal-reduction in incidence and mortality from breast cancer (long term impact)
Accessibility, equity, relevance, and acceptability
The issue of accessibility and social acceptability would be addressed in the questionnaire survey of clients after the intervention.Relevance to need was initially addressed in the epidemiological needs assessment prior to programme implementation. In further evaluating the need for the programme with respect to the community, focus group discussions with carefully selected members of the community would be held. Members of women groups who are opinion leaders in the community would be involved in these interviews. This technique enables the evaluator explore in-depth views on how social, cultural, religious, and other barriers that affect the uptake of screening service. However, they are time-consuming; data is difficult to analyse; confidentiality is compromised and interviewer bias is a key issue.
- Ovretveit, J., Evaluating health interventions : an introduction to evaluation of health treatments, services, policies and organizational interventions. 1998, Buckingham: Open University Press.
- Centers for Disease Control and Prevention. Framework for Program Evaluation in Public Health. 1999 [cited 2010 7 February]; Available from: http://www.cdc.gov/eval/framework.htm.
- Naidoo, J. and J. Wills, Evaluation in health promotion, in Foundations for health promotion. 2005, Baillière Tindall Edinburgh.
- Green, J. and J. South, Key Concepts for Public Health Practice: Evaluation. 2006, Maidenhead: Open University Press.
- Adesunkanmi, A.R.K., et al., The severity, outcome and challenges of breast cancer in Nigeria. The Breast, 2006. 15(3): p. 399-409.
- Akhigbe, A. and V. Omuemu, Knowledge, attitudes and practice of breast cancer screening among female health workers in a Nigerian urban city. BMC Cancer, 2009. 9(1): p. 203.
- Ferlay, J., et al., Global Burden of Breast Cancer. Li C et al. Breast Cancer Epidemiology, Springer Science, 2008.
- Okobia, M., et al., Knowledge, attitude and practice of Nigerian women towards breast cancer: A cross-sectional study. World journal of surgical oncology, 2006. 4(1): p. 11.
- Okobia, M.N. and U. Osime, Clinicopathological Study of Carcinoma of the Breast in Benin City. African Journal of Reproductive Health / La Revue Africaine de la SantÃ© Reproductive, 2001. 5(2): p. 56-62.
- Adebamowo, C.A. and O.O. Ajayi, Breast cancer in Nigeria. West African Journal of Medicine, 2000. 19(3): p. 179-91.
- Nggada, H.A., et al., Breast Cancer Burden in Maiduguri, North Eastern Nigeria. The Breast Journal, 2008. 14(3): p. 284-286.
- Ihekwaba, F.N., Breast cancer in Nigerian women. British Journal of Surgery, 1992. 79(8): p. 771-775.
- Ikpatt, O.F., et al., Breast cancer in Nigeria and Finland: epidemiological, clinical and histological comparison. Anticancer Research, 2002. 22(5): p. 3005-12.
- Huo, D., et al., Population Differences in Breast Cancer: Survey in Indigenous African Women Reveals Over-Representation of Triple-Negative Breast Cancer. Journal of Clinical Oncology, 2009. 27(27): p. 4515-4521.
- Okonofua, F.E., A. Lawal, and J.K. Bamgbose, Features of menopause and menopausal age in Nigerian women. International Journal of Gynecology & Obstetrics, 1990. 31(4): p. 341-345.
- OlaOlorun, F. and T. Lawoyin, Age at menopause and factors associated with attainment of menopause in an urban community in Ibadan, Nigeria. Climacteric, 2009. 12(4): p. 352 – 363.
- Okobia, M., et al., Knowledge, attitude and practice of Nigerian women towards breast cancer: A cross-sectional study. World journal of surgical oncology, 2006. 4: p. 1 – 9.
- Oluwatosin, O.A. and O. Oladepo, Knowledge of breast cancer and its early detection measures among rural women in Akinyele Local Government Area, Ibadan, Nigeria. BMC Cancer, 2006. 6(1): p. 271.
- Odusanya, O.O., Breast cancer: knowledge, attitudes, and practices of female schoolteachers in Lagos, Nigeria. Breast J, 2001. 7(3): p. 171-5.
- Ezeome, R.E., Delays in presentation and treatment of breast cancer in Nigeria. J Clin Oncol (Meeting Abstracts), 2009. 27(15S): p. 1527-.
- Ekanem, V.J. and J.U. Aligbe, Histopathological types of breast cancer in Nigerian women: a 12-year review (1993-2004). African Journal of Reproductive Health, 2006. 10(1): p. 71-5.
- Ukwenya, A., et al., Delayed treatment of symptomatic breast cancer: The experience from Kaduna, Nigeria. South African Journal of Surgery, 2008. 46(4): p. 106.
- Gukas, I.D., et al., Clinicopathological features and molecular markers of breast cancer in Jos, Nigeria. West African Journal of Medicine, 2005. 24(3): p. 209-13.
- Negri, E., et al., Risk factors for breast cancer: pooled results from three Italian case studies. American Journal of Epidemiology, 1988. 128(6): p. 1207-1215.
- O. Odusanya, O.O.T., Olumuyiwa, Breast Cancer Knowledge, Attitudes and Practice among Nurses in Lagos, Nigeria. Acta Oncologica, 2001. 40(7): p. 844-848.
- Black, N., Why we need observational studies to evaluate the effectiveness of health care. BMJ, 1996. 312(7040): p. 1215-1218.
- U.S. Department of Health and Human Services. Centers for Disease Control and revention. Office of the Director, O.o.S.a.I., Introduction to program evaluation for public health programs: A self-study guide 2005: Atlanta, GA.
- Bowling, A., Research methods in health:investigating health and health services. 2002, Buckingham: Open University Press
- Akhigbe, A.O. and V.O. Omuemu, Knowledge, attitudes and practice of breast cancer screening among female health workers in a Nigerian urban city. BMC Cancer, 2009. 9: p. 203.
- Gray, S., The contribution of health services to public health, in Public Health for the 21st Century:new perpectives on policy,practice and participation, J. Orme, et al., Editors. 2007, Open University Press: Maidenhead.
- Maxwell, R., Quality assessment in health. British Medical Journal, 1984. 288(6428): p. 1470-1.
- Walter, S.D., Mammographic screening: case-control studies. Annals of Oncology, 2003. 14(8): p. 1190-1192.
- Puliti, D., et al., Effectiveness of service screening: a case-control study to assess breast cancer mortality reduction. British Journal of Cancer, 2008. 99(3): p. 423-427.
- Collette, H.J., et al., Further evidence of benefits of a (non-randomised) breast cancer screening programme: the DOM project. Journal of Epidemiology and Community Health, 1992. 46(4): p. 382-386.
- Kerlikowske, K., et al., Efficacy of Screening Mammography: A Meta-analysis. JAMA, 1995. 273(2): p. 149-154.
Cite This Work
To export a reference to this article please select a referencing stye below:
Related ServicesView all
DMCA / Removal Request
If you are the original writer of this essay and no longer wish to have your work published on UKEssays.com then please: