Expanding the scope of humanitarian program evaluation.
ABSTRACT The effectiveness of humanitarian programs normally is evaluated according to a limited number of pre-defined objectives. These objectives typically represent only selected positive expected impacts of program interventions and as such, are inadequate benchmarks for understanding the overall effectiveness of aid. This is because programs also have unexpected impacts (both positive and negative) as well as expected negative impacts and expected positive impacts beyond the program objectives. The authors contend that these other categories of program impacts also should be assessed, and suggest a methodology for doing so that draws on input from the perspectives of beneficiaries. This paper includes examples of the use of this methodology in the field. Finally, the authors suggest future directions for improving this type of expanded assessment and advocate for its widespread use, both within and without the field of disaster response.
SourceAvailable from: Thomas D. Kirsch[Show abstract] [Hide abstract]
ABSTRACT: There exists little agreement on the choice of indicators to be used to assess the impact of humanitarian assistance. The 2004 Indian Ocean tsunami led to significant mortality and displacement in Aceh Province, Indonesia, as well as a nearly unprecedented humanitarian response. Six years after the disaster we conducted an impact assessment of humanitarian services rendered in Aceh using a comprehensive set of rights-based indicators and sought to determine modifiable predictors of improved outcomes in disaster-affected households. A sample of 597 returned and non-returned households in Banda Aceh and Meulaboh was selected using a multistage stratified cluster survey design. We employed principle components analysis and the Framework on Durable Solutions for Internally Displaced Persons to develop a comprehensive and rights-based approach to humanitarian impact measurement using multivariate regression models. The attainment of durable solutions was equivalent in both returned households 100.1 [CI] 97.63-102.5) and households that integrated elsewhere (99.37 [CI] 95.43-103.3, P = 0.781). Standard of living as well as education and health facility satisfaction increased significantly whereas monthly income decreased after the tsunami, from 2585241 IDR ([CI] 2357202-2813279 IDR) to 2038963 ([CI] 1786627-2291298 IDR, P < 0.001). Shelter (P = 0.007) and legal assistance (P < 0.001) were both significantly associated with positive durable solutions outcomes, whereas prolonged displacement duration was significantly associated with poorer outcomes (P < 0.001). Livelihood assistance received after one year was associated with higher odds of increasing or maintaining pre-tsunami income levels (OR = 3.02, P = 0.008), whereas livelihood assistance received within one year was associated with lower odds of attaining pre-tsunami income (OR = 0.52, P = 0.010). We find that after adjusting for pre-tsunami conditions and tsunami-related damages, the impact of sectoral responses can be assessed. The duration of displacement was the strongest negative predictive factor for the attainment of durable solutions, suggesting that measures to reduce displacement time may be effective in mitigating the long-term effects of disaster on households. The durable solutions framework is a novel and effective impact measurement tool and can be used to identify factors amenable to intervention and inform future disaster recovery efforts.BMC Public Health 11/2014; 14(1):1168. DOI:10.1186/1471-2458-14-1168 · 2.32 Impact Factor
[Show abstract] [Hide abstract]
ABSTRACT: Improving and sustaining successful public health interventions relies increasingly on the ability to identify the key components of an intervention that are effective, to identify for whom the intervention is effective, and to identify under what conditions the intervention is effective. Bayesian probability an "advanced" experimental design framework of methodology is used in the study to develop a systematic tool that can assist health care managers and field workers in measuring effectiveness of health program intervention and systematically assess the components of programs to be applied to design program improvements and to advocate for resources. The study focuses on essential management elements of the health system that must be in place to ensure the effectiveness of IMNCI intervention. Early experiences with IMNCI implemented led to greater awareness of the need to improve drug delivery, support for effective planning and management at all levels and address issues related to the organization of work at health facilities. The efficacy of IMNCI program from the experience of experts and specialists working in the state is 0.67 and probability of effective-ness of all management components in the study is 58%. Overall the standard assessment tool used predicts success of around 39% for the IMNCI intervention implemented in current situation in Rajasthan. Training management compo-nent carried the highest weight-age of 21% with 73% probability of being effective in the state. Human resource man-agement has weight-age of 13% with 53% probability of being effective in current scenario. Monitoring and evaluation carried a weight-age of 11% with only 33% probability of being effective. Operational planning carried a weight-age of 9% with 100% probability of being effectively managed. Supply management carried a weight-age of 8% with zero probability of being effective in the current field scenario. In the study, each question that received low score identifies it as a likely obstacle to the success of the health program. The health program should improve all sub-components with low scores to increase the likelihood of meeting its objectives. Public health interventions tend to be complex, pro-grammatic and context dependent. The evaluation of evidence must distinguish between the fidelity of the evaluation process in detecting the success or failure of the intervention, and relative success or failure of the intervention itself. We advocate management attributes incorporation into criteria for appraising evidence on public health interventions. This can strengthen the value of evidence and their potential contributions to the process of public health management and social development.Intelligent Information Management 01/2012; 04(06). DOI:10.4236/iim.2012.46029