Assessing local instrument validity and reliability: a field-based example from Northern Uganda

Dept. of Population and International Health, François-Xavier Bagnoud Center for Health and Human Rights, Harvard School of Public Health, Boston, MA 02115, USA.
Social Psychiatry (Impact Factor: 2.58). 02/2009; 44(8):685-92. DOI: 10.1007/s00127-008-0475-1
Source: PubMed

ABSTRACT This paper presents an approach for evaluating the reliability and validity of mental health measures in non-Western field settings. We describe this approach using the example of our development of the Acholi psychosocial assessment instrument (APAI), which is designed to assess depression-like (two tam, par and kumu), anxiety-like (ma lwor) and conduct problems (kwo maraco) among war-affected adolescents in northern Uganda. To examine the criterion validity of this measure in the absence of a traditional gold standard, we derived local syndrome terms from qualitative data and used self reports of these syndromes by indigenous people as a reference point for determining caseness. Reliability was examined using standard test-retest and inter-rater methods. Each of the subscale scores for the depression-like syndromes exhibited strong internal reliability ranging from alpha = 0.84-0.87. Internal reliability was good for anxiety (0.70), conduct problems (0.83), and the pro-social attitudes and behaviors (0.70) subscales. Combined inter-rater reliability and test-retest reliability were good for most subscales except for the conduct problem scale and prosocial scales. The pattern of significant mean differences in the corresponding APAI problem scale score between self-reported cases vs. noncases on local syndrome terms was confirmed in the data for all of the three depression-like syndromes, but not for the anxiety-like syndrome ma lwor or the conduct problem kwo maraco.

Download full-text


Available from: Ivelina Borisova, Jun 23, 2014
  • Source
    • "Depression/anxiety is a subscale of the Acholi Psychosocial Assessment Instrument (APAI). APAI is a 40-item field-based self-report questionnaire previously developed for use in Northern Uganda [41]. The measure comprises: depression/anxiety, conduct problems, pro-social behaviours, and somatic complaints without medical cause. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Globally, suicide is a public health burden especially in the aftermath of war. Understanding the processes that define the path from previous war experiences (WE) to current suicidal ideation (SI) is crucial for defining opportunities for interventions. We assessed the extent to which different types of previous WE predict current SI and whether post-war hardships and depression mediate the relations between WE and SI among former child soldiers (FCS) in Northern Uganda.Methods We performed cross-sectional analyses with a sample of 539 FCS (61% male) participating in an on-going longitudinal study. The influence of various types of previous WE on current SI and mediation by post-war hardships and depression were assessed by regression analyses.ResultsThe following types of war experiences: ¿witnessing violence¿, ¿direct personal harm¿, ¿deaths¿, ¿Involvement in hostilities¿, ¿sexual abuse¿ and ¿general war experiences¿ significantly predicted current SI in a univariable analyses whereas ¿direct personal harm¿, ¿involvement in hostilities¿, and ¿sexual abuse¿ independently predicted current SI in a multivariable analyses. General WE were linked to SI (ß¿=¿0.18 (95% CI 0.10 to 0.25)) through post-war hardships (accounting for 69% of the variance in their relationship) and through depression/anxiety (ß¿=¿0.18 (95% CI 0.10 to 0.25)) accounting for 65% of the variance in their relationship. The direct relationship between previous WE and current SI reduced but remained marginally significant (ß¿=¿.08, CI: (.01, .17) for depression/anxiety but not for post-war hardships (ß¿=¿.09, CI: (¿.03, .20).Conclusion Types of WE should be examined when assessing risks for SI. Interventions to reduce SI should aim to alleviate post-war hardships and treat depression/anxiety.
    BMC Psychiatry 09/2014; 14(1):271. DOI:10.1186/s12888-014-0271-2 · 2.24 Impact Factor
    • "In the current study, internal consistency was 0.557. Internalising symptoms, conduct problems and pro-social behaviour were assessed using the African Youth Psychosocial Assessment Instrument (AYPA) (Betancourt et al., 2009). This measure was developed in northern Uganda after extensive qualitative consultation with young people, caregivers and mental health workers. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Rural communities in the Haut-Uele Province of northern Democratic Republic of Congo live in constant danger of attack and/or abduction by units of the Lord's Resistance Army operating in the region. This pilot study sought to develop and evaluate a community-participative psychosocial intervention involving life skills and relaxation training and Mobile Cinema screenings with this war-affected population living under current threat. 159 war-affected children and young people (aged 7–18) from the villages of Kiliwa and Li-May in north-eastern DR Congo took part in this study. In total, 22% of participants had been abduction previously while 73% had a family member abducted. Symptoms of post-traumatic stress reactions, internalising problems, conduct problems and pro-social behaviour were assessed by blinded interviewers at pre- and post-intervention and at 3-month follow-up. Participants were randomised (with an accompanying caregiver) to 8 sessions of a group-based, community-participative, psychosocial intervention (n = 79) carried out by supervised local, lay facilitators or a wait-list control group (n = 80). Average seminar attendance rates were high: 88% for participants and 84% for caregivers. Drop-out was low: 97% of participants were assessed at post-intervention and 88% at 3 month follow-up. At post-test, participants reported significantly fewer symptoms of post-traumatic stress reactions compared to controls (Cohen's d = 0.40). At 3 month follow up, large improvements in internalising symptoms and moderate improvements in pro-social scores were reported, with caregivers noting a moderate to large decline in conduct problems among the young people. Trial Registration, Identifier: NCT01542398.
    Child abuse & neglect 07/2014; 38(7). DOI:10.1016/j.chiabu.2014.02.004 · 2.34 Impact Factor
  • Source
    • "Perhaps an optimistic interpretation of our mental health results is that the boys directly exposed to the sport-for-development intervention became more comfortable in expressing their problems and this created a response bias. However, the mental health assessment tool used was developed to be sensitive, valid and reliable for local constructs of depression- and anxiety-like syndromes [18,22]. Therefore, it is more likely that there was an inherent component of the GMKL that caused a deterioration in the mental health of the boys directly exposed to the programme. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Physical inactivity contributes to poor fitness and mental health disorders. This is of concern in post-conflict low-income settings where non-communicable diseases are emerging and there is limited evidence for physical activity interventions. We examined the effects of a sport-for-development programme on adolescent physical fitness and mental health in Gulu, Uganda. Methods We conducted a single-blinded RCT nested within an observational study with three unbalanced parallel groups. Participants were able-bodied adolescents aged 11–14 years. The intervention comprised an 11-week voluntary competitive sport-for-development football league. Participants who did not subscribe for the intervention formed a non-registered comparison group. Boys who registered for the sport-for-development programme were randomly allocated to the intervention or wait-listed. The girls programme subscription was insufficient to form a wait-list and all registrants received the intervention. Physical fitness was assessed by cardiorespiratory fitness (multi-stage fitness test), muscular power (standing broad jump) and body composition (BMI-for-age). Mental health was measured using the Acholi Psychosocial Assessment Instrument for local depression-like (DLS) and anxiety-like (ALS) syndromes. All randomisation was computer generated and assessors were masked to group allocation. An intention-to-treat analysis of adjusted effect size (ES) was applied. Results There were 1,462 adolescents in the study (intervention: boys = 74, girls = 81; wait-list: boys = 72; comparison: boys = 472, girls = 763). At four months follow-up there was no significant effect on the boys fitness when comparing intervention vs wait-listed and intervention vs non-registered groups. However, there was a negative effect on DLS when comparing boys intervention vs wait-listed (ES = 0.67 [0.33 to 1.00]) and intervention vs non-registered (ES = 0.25 [0.00 to 0.49]). Similar results were observed for ALS for boys intervention vs wait-listed (ES = 0.63 [0.30 to 0.96]) and intervention vs non-registered (ES = 0.26 [0.01 to 0.50]). There was no significant effect on the girls for any outcomes. Conclusions The sport-for-development league in this study had no impact on fitness and a negative effect on the mental health of participating boys. From this research, there is no evidence that voluntary competitive sport-for-development interventions improve physical fitness or mental health outcomes in post-conflict settings.
    BMC Public Health 06/2014; 14(1):619. DOI:10.1186/1471-2458-14-619 · 2.32 Impact Factor
Show more