How many mailouts? Could attempts to increase the response rate in the Iraq war cohort study be counterproductive?

King's Centre for Military Health, Research, Institute of Psychiatry, King's College London, London, UK.
BMC Medical Research Methodology (Impact Factor: 2.27). 02/2007; 7(1):51. DOI: 10.1186/1471-2288-7-51
Source: PubMed


Low response and reporting errors are major concerns for survey epidemiologists. However, while nonresponse is commonly investigated, the effects of misclassification are often ignored, possibly because they are hard to quantify. We investigate both sources of bias in a recent study of the effects of deployment to the 2003 Iraq war on the health of UK military personnel, and attempt to determine whether improving response rates by multiple mailouts was associated with increased misclassification error and hence increased bias in the results.
Data for 17,162 UK military personnel were used to determine factors related to response and inverse probability weights were used to assess nonresponse bias. The percentages of inconsistent and missing answers to health questions from the 10,234 responders were used as measures of misclassification in a simulation of the 'true' relative risks that would have been observed if misclassification had not been present. Simulated and observed relative risks of multiple physical symptoms and post-traumatic stress disorder (PTSD) were compared across response waves (number of contact attempts).
Age, rank, gender, ethnic group, enlistment type (regular/reservist) and contact address (military or civilian), but not fitness, were significantly related to response. Weighting for nonresponse had little effect on the relative risks. Of the respondents, 88% had responded by wave 2. Missing answers (total 3%) increased significantly (p < 0.001) between waves 1 and 4 from 2.4% to 7.3%, and the percentage with discrepant answers (total 14%) increased from 12.8% to 16.3% (p = 0.007). However, the adjusted relative risks decreased only slightly from 1.24 to 1.22 for multiple physical symptoms and from 1.12 to 1.09 for PTSD, and showed a similar pattern to those simulated.
Bias due to nonresponse appears to be small in this study, and increasing the response rates had little effect on the results. Although misclassification is difficult to assess, the results suggest that bias due to reporting errors could be greater than bias caused by nonresponse. Resources might be better spent on improving and validating the data, rather than on increasing the response rate.

Download full-text


Available from: Matthew Hotopf
  • Source
    • "An independent samples t-test was conducted to examine any differences by mean scores for those followed-up (M = 9.97, SD = 5.65) and those who were not (M = 9.71, SD = 5.81) and found no statistically significant difference; t(10151) = 1.3090, p = 0.0953. The response rates were 59% at phase 1 and 68% (6429 of 9395) at phase 2. There was no evidence of response bias by health status or alcohol misuse, but there was for age, sex and rank (Hotopf et al., 2006; Rona et al., 2010; Tate et al., 2007). At both phases, respondents completed a self-report questionnaire covering socio-demographics, service history, physical and mental health and alcohol use. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Objectives: We assessed changes in Alcohol Use Disorders Identification Test (AUDIT) scores over time. We investigated the impact of life events and changes in mental health status on AUDIT scores over time in UK military personnel. Methods: A random representative sample of regular UK military personnel who had been serving in 2003 were surveyed in 2004-2006 (phase 1) and again in 2007-2009 (phase 2). The impact of changes in symptoms of psychological distress, probable post-traumatic stress disorder (PTSD), marital status, serving status, rank, deployment to Iraq/Afghanistan and smoking was assessed between phases. Results: We found a statistically significant but small decrease in AUDIT scores between phases 1 and 2 (mean change=-1.01, 95% confidence interval=-1.14, -0.88). Participants reported a decrease in AUDIT scores if they experienced remission in psychological distress (adjusted mean -2.21, 95% CI -2.58, -1.84) and probable PTSD (adjusted mean -3.59, 95% CI -4.41, -2.78), if they stopped smoking (adjusted mean -1.41, 95% CI -1.83, -0.98) and were in a new relationship (adjusted mean -2.77, 95% CI -3.15, -2.38). On the other hand, reporting new onset or persistent symptoms of probable PTSD (adjusted mean 1.34, 95% CI 0.71, 1.98) or a relationship breakdown (adjusted mean 0.53, 95% CI 0.07, 0.99) at phase 2 were associated with an increase in AUDIT scores. Conclusions: The overall level of hazardous alcohol consumption remains high in the UK military. Changes in AUDIT scores were linked to mental health and life events but not with deployment to Iraq or Afghanistan.
    Full-text · Article · Sep 2015 · Drug and Alcohol Dependence
  • Source
    • "Military cohorts are characterized by unique experiences including deployments [17-21]. US military personnel are highly mobile and often more difficult to track and contact for long-term studies [22,23]. Deployment experience in support of the recent operations in Iraq and Afghanistan has not yet been studied as a determinant for enrollment in a longitudinal study. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Longitudinal cohort studies are highly valued in epidemiologic research for their ability to establish exposure-disease associations through known temporal sequences. A major challenge in cohort studies is recruiting individuals representative of the targeted sample population to ensure the generalizability of the study's findings. We evaluated nearly 350,000 invited subjects (from 2004-2008) of the Millennium Cohort Study, a prospective cohort study of the health of US military personnel, for factors prior to invitation associated with study enrollment. Multivariable logistic regression was utilized, adjusting for demographic and other confounders, to determine the associations between both deployment experience and prior healthcare utilization with enrollment into the study. Study enrollment was significantly greater among those who deployed prior to and/or during the enrollment cycles or had at least one outpatient visit in the 12 months prior to invitation. Mental disorders and hospitalization for more than two days within the past year were associated with reduced odds of enrollment. These findings suggest differential enrollment by deployment experience and health status, and may help guide recruitment efforts in future studies.
    Full-text · Article · Jul 2013 · BMC Medical Research Methodology
  • Source
    • "Because of these high non-participation rates, selection bias could be a serious issue in such studies. Research on factors associated with non-participation is quite rare because very often there is no information at all on those who did not participate [1,5,8,24]. However, if we are to use appropriate analytical methods that take non-participation into account, we need to understand its underlying mechanisms. "
    [Show abstract] [Hide abstract]
    ABSTRACT: A good response rate has been considered as a proof of a study’s quality. Decreasing participation and its potential impact on the internal validity of the study are of growing interest. Our objective was to assess factors associated with contact and response to a postal survey in a epidemiological study of the long-term outcome of IVF couples. The DAIFI study is a retrospective cohort including 6,507 couples who began an IVF program in 2000-2002 in one of the eight participating French IVF centers. Medical data on all 6,507 couples were obtained from IVF center databases, and information on long-term outcome was available only for participants in the postal survey (n = 2,321). Logistic regressions were used to assess firstly factors associated with contact and secondly factors associated with response to the postal questionnaire among contacted couples. Sixty-two percent of the 6,507 couples were contacted and 58% of these responded to the postal questionnaire. Contacted couples were more likely to have had a child during IVF treatment than non-contactable couples, and the same was true of respondents compared with non-respondents. Demographic and medical characteristics were both associated with probability of contact and probability of response. After adjustment, having a live birth during IVF treatment remained associated with both probabilities, and more strongly with probability of response. Having a child during IVF treatment was a major factor impacting on participation rate. Non-response as well as non-contact were linked to the outcome of interest, i.e. long-term parenthood success of infertile couples. Our study illustrates that an a priori hypothesis may be too simplistic and may underestimate potential bias. In the context of growing use of analytical methods that take attrition into account (such as multiple imputation), we need to better understand the mechanisms that underlie attrition in order to choose the most appropriate method.
    Full-text · Article · Jul 2012 · BMC Medical Research Methodology
Show more