SPECIAL REPORT Expanding the Scope of Humanitarian Program Evaluation

Center for Refugee and Disaster Studies, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland 21205, USA.
Prehospital and disaster medicine: the official journal of the National Association of EMS Physicians and the World Association for Emergency and Disaster Medicine in association with the Acute Care Foundation 10/2007; 22(5):390-5. DOI: 10.1017/S1049023X00005100
Source: PubMed


The effectiveness of humanitarian programs normally is evaluated according to a limited number of pre-defined objectives. These objectives typically represent only selected positive expected impacts of program interventions and as such, are inadequate benchmarks for understanding the overall effectiveness of aid. This is because programs also have unexpected impacts (both positive and negative) as well as expected negative impacts and expected positive impacts beyond the program objectives. The authors contend that these other categories of program impacts also should be assessed, and suggest a methodology for doing so that draws on input from the perspectives of beneficiaries. This paper includes examples of the use of this methodology in the field. Finally, the authors suggest future directions for improving this type of expanded assessment and advocate for its widespread use, both within and without the field of disaster response.

Download full-text


Available from: Sharon Mcdonnell,
30 Reads
  • Source
    • "There is great interest in measuring the effectiveness and impact of programs developed to assist populations Copyright © 2012 SciRes. IIM affected by disasters and to aid in their recovery [2] [3]. To evaluate the effectiveness or cost-effectiveness of a specific health intervention typically involves comparing two populations, one that has received the intervention and the other that has not received it. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Improving and sustaining successful public health interventions relies increasingly on the ability to identify the key components of an intervention that are effective, to identify for whom the intervention is effective, and to identify under what conditions the intervention is effective. Bayesian probability an "advanced" experimental design framework of methodology is used in the study to develop a systematic tool that can assist health care managers and field workers in measuring effectiveness of health program intervention and systematically assess the components of programs to be applied to design program improvements and to advocate for resources. The study focuses on essential management elements of the health system that must be in place to ensure the effectiveness of IMNCI intervention. Early experiences with IMNCI implemented led to greater awareness of the need to improve drug delivery, support for effective planning and management at all levels and address issues related to the organization of work at health facilities. The efficacy of IMNCI program from the experience of experts and specialists working in the state is 0.67 and probability of effective-ness of all management components in the study is 58%. Overall the standard assessment tool used predicts success of around 39% for the IMNCI intervention implemented in current situation in Rajasthan. Training management compo-nent carried the highest weight-age of 21% with 73% probability of being effective in the state. Human resource man-agement has weight-age of 13% with 53% probability of being effective in current scenario. Monitoring and evaluation carried a weight-age of 11% with only 33% probability of being effective. Operational planning carried a weight-age of 9% with 100% probability of being effectively managed. Supply management carried a weight-age of 8% with zero probability of being effective in the current field scenario. In the study, each question that received low score identifies it as a likely obstacle to the success of the health program. The health program should improve all sub-components with low scores to increase the likelihood of meeting its objectives. Public health interventions tend to be complex, pro-grammatic and context dependent. The evaluation of evidence must distinguish between the fidelity of the evaluation process in detecting the success or failure of the intervention, and relative success or failure of the intervention itself. We advocate management attributes incorporation into criteria for appraising evidence on public health interventions. This can strengthen the value of evidence and their potential contributions to the process of public health management and social development.
    Intelligent Information Management 01/2012; 04(06). DOI:10.4236/iim.2012.46029
  • Source
    • "Although the above connections imply the importance of treating depression for communities in developing countries, there have been no studies to date that explicitly illustrate the combined psychological, physical, and material importance of treating depression on communities as whole. None of the five other randomized controlled evaluations of psychotherapy-based treatments for depression in developing countries (Bolton et al. 2007; Patel et al., 2003; Araya et al., 2003; Araya et al., 2007; Rahman et al., 2008), nor any other trial of a psychotherapy treatment for depression have assessed the broader impacts of the treatment in the community. Research reviewed in this section suggests the value of broadening the horizon of treatment evaluations to include assessments of indirect and unanticipated community impacts. "
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: To assist field workers in program evaluation and to explicitly discuss program strengths and weaknesses, a practical method to estimate the effectiveness of public health interventions within the existing program capacity was developed. The method and materials were tested in seven countries (Afghanistan, Zimbabwe, Tanzania, Uganda, Guatemala, the Philippines, and Ghana). In this method, four core components are assessed using a questionnaire: (1) the efficacy of the intervention; (2) the level of existing human resources (i.e., quality of recruitment, training, and continuing education); (3) the infrastructure (i.e., supplies, salary, transportation, and supervision); and (4) the level of community support (i.e., access and demand). Using the assessment tool provided, program staff can determine if all necessary elements are in place for a successful program that can deliver the specific intervention. Based on the results of the assessment program, weaknesses can be identified, explicitly discussed, and addressed. The usefulness of this tool in humanitarian relief may be twofold: (1) to assess the design and implementation of effective programs; and (2) to highlight the inevitable need for capacity building as the disaster situation evolves.
    Prehospital and disaster medicine: the official journal of the National Association of EMS Physicians and the World Association for Emergency and Disaster Medicine in association with the Acute Care Foundation 10/2007; 22(5):396-405. DOI:10.1017/S1049023X00005112
Show more