Article

How many mailouts? Could attempts to increase the response rate in the Iraq war cohort study be counterproductive?

King's Centre for Military Health, Research, Institute of Psychiatry, King's College London, London, UK.
BMC Medical Research Methodology (Impact Factor: 2.17). 02/2007; 7(1):51. DOI: 10.1186/1471-2288-7-51
Source: PubMed

ABSTRACT Low response and reporting errors are major concerns for survey epidemiologists. However, while nonresponse is commonly investigated, the effects of misclassification are often ignored, possibly because they are hard to quantify. We investigate both sources of bias in a recent study of the effects of deployment to the 2003 Iraq war on the health of UK military personnel, and attempt to determine whether improving response rates by multiple mailouts was associated with increased misclassification error and hence increased bias in the results.
Data for 17,162 UK military personnel were used to determine factors related to response and inverse probability weights were used to assess nonresponse bias. The percentages of inconsistent and missing answers to health questions from the 10,234 responders were used as measures of misclassification in a simulation of the 'true' relative risks that would have been observed if misclassification had not been present. Simulated and observed relative risks of multiple physical symptoms and post-traumatic stress disorder (PTSD) were compared across response waves (number of contact attempts).
Age, rank, gender, ethnic group, enlistment type (regular/reservist) and contact address (military or civilian), but not fitness, were significantly related to response. Weighting for nonresponse had little effect on the relative risks. Of the respondents, 88% had responded by wave 2. Missing answers (total 3%) increased significantly (p < 0.001) between waves 1 and 4 from 2.4% to 7.3%, and the percentage with discrepant answers (total 14%) increased from 12.8% to 16.3% (p = 0.007). However, the adjusted relative risks decreased only slightly from 1.24 to 1.22 for multiple physical symptoms and from 1.12 to 1.09 for PTSD, and showed a similar pattern to those simulated.
Bias due to nonresponse appears to be small in this study, and increasing the response rates had little effect on the results. Although misclassification is difficult to assess, the results suggest that bias due to reporting errors could be greater than bias caused by nonresponse. Resources might be better spent on improving and validating the data, rather than on increasing the response rate.

Download full-text

Full-text

Available from: Matthew Hotopf, Sep 02, 2015
0 Followers
 · 
116 Views
 · 
35 Downloads
  • Source
    • "Military cohorts are characterized by unique experiences including deployments [17-21]. US military personnel are highly mobile and often more difficult to track and contact for long-term studies [22,23]. Deployment experience in support of the recent operations in Iraq and Afghanistan has not yet been studied as a determinant for enrollment in a longitudinal study. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Longitudinal cohort studies are highly valued in epidemiologic research for their ability to establish exposure-disease associations through known temporal sequences. A major challenge in cohort studies is recruiting individuals representative of the targeted sample population to ensure the generalizability of the study's findings. We evaluated nearly 350,000 invited subjects (from 2004-2008) of the Millennium Cohort Study, a prospective cohort study of the health of US military personnel, for factors prior to invitation associated with study enrollment. Multivariable logistic regression was utilized, adjusting for demographic and other confounders, to determine the associations between both deployment experience and prior healthcare utilization with enrollment into the study. Study enrollment was significantly greater among those who deployed prior to and/or during the enrollment cycles or had at least one outpatient visit in the 12 months prior to invitation. Mental disorders and hospitalization for more than two days within the past year were associated with reduced odds of enrollment. These findings suggest differential enrollment by deployment experience and health status, and may help guide recruitment efforts in future studies.
    BMC Medical Research Methodology 07/2013; 13(1):90. DOI:10.1186/1471-2288-13-90 · 2.17 Impact Factor
  • Source
    • "Because of these high non-participation rates, selection bias could be a serious issue in such studies. Research on factors associated with non-participation is quite rare because very often there is no information at all on those who did not participate [1,5,8,24]. However, if we are to use appropriate analytical methods that take non-participation into account, we need to understand its underlying mechanisms. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background A good response rate has been considered as a proof of a study’s quality. Decreasing participation and its potential impact on the internal validity of the study are of growing interest. Our objective was to assess factors associated with contact and response to a postal survey in a epidemiological study of the long-term outcome of IVF couples. Methods The DAIFI study is a retrospective cohort including 6,507 couples who began an IVF program in 2000-2002 in one of the eight participating French IVF centers. Medical data on all 6,507 couples were obtained from IVF center databases, and information on long-term outcome was available only for participants in the postal survey (n = 2,321). Logistic regressions were used to assess firstly factors associated with contact and secondly factors associated with response to the postal questionnaire among contacted couples. Results Sixty-two percent of the 6,507 couples were contacted and 58% of these responded to the postal questionnaire. Contacted couples were more likely to have had a child during IVF treatment than non-contactable couples, and the same was true of respondents compared with non-respondents. Demographic and medical characteristics were both associated with probability of contact and probability of response. After adjustment, having a live birth during IVF treatment remained associated with both probabilities, and more strongly with probability of response. Having a child during IVF treatment was a major factor impacting on participation rate. Conclusions Non-response as well as non-contact were linked to the outcome of interest, i.e. long-term parenthood success of infertile couples. Our study illustrates that an a priori hypothesis may be too simplistic and may underestimate potential bias. In the context of growing use of analytical methods that take attrition into account (such as multiple imputation), we need to better understand the mechanisms that underlie attrition in order to choose the most appropriate method.
    BMC Medical Research Methodology 07/2012; 12(1):104. DOI:10.1186/1471-2288-12-104 · 2.17 Impact Factor
  • Source
    • "Different strategies have been described to reduce response bias, such as repeated mailings following initial non-response [9,10] and the use of alternative, shortened versions of measurement instruments [11]. In our own study, the TRacking Adolescents Individual Lives’ Survey (TRAILS), extra recruitment effort at the first measurement wave consisted of one or two house visits after no response to both an initial and a reminder letter had been received, and offering a two-month reflection period if the initial participation request was at an inconvenient time [5]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Extensive recruitment effort at baseline increases representativeness of study populations by decreasing non-response and associated bias. First, it is not known to what extent increased attrition occurs during subsequent measurement waves among subjects who were hard-to-recruit at baseline and what characteristics the hard-to-recruit dropouts have compared to the hard-to-recruit retainers. Second, it is unknown whether characteristics of hard-to-recruit responders in a prospective population based cohort study are similar across age group and survey method. Methods First, we compared first wave (T1) easy-to-recruit with hard-to-recruit responders of the TRacking Adolescents’ Individual Lives Survey (TRAILS), a prospective population based cohort study of Dutch (pre)adolescents (at first wave: n = 2230, mean age = 11.09 (SD 0.56), 50.8% girls), with regard to response rates at subsequent measurement waves. Second, easy-to-recruit and hard-to-recruit participants at the fourth TRAILS measurement wave (n = 1881, mean age = 19.1 (SD 0.60), 52.3% girls) were compared with fourth wave non-responders and earlier stage drop-outs on family composition, socioeconomic position (SEP), intelligence (IQ), education, sociometric status, substance use, and psychopathology. Results First, over 60% of the hard-to-recruit responders at the first wave were retained in the sample eight years later at the fourth measurement wave. Hard-to-recruit dropouts did not differ from hard-to-recruit retainers. Second, extensive recruitment efforts for the web based survey convinced a population of nineteen year olds with similar characteristics as the hard-to-recruit eleven year olds that were persuaded to participate in a school-based survey. Some characteristics associated with being hard-to-recruit (as compared to being easy-to-recruit) were more pronounced among non-responders, resembling the baseline situation (De Winter et al.2005). Conclusions First, extensive recruitment effort at the first assessment wave of a prospective population based cohort study has long lasting positive effects. Second, characteristics of hard-to-recruit responders are largely consistent across age groups and survey methods.
    BMC Medical Research Methodology 07/2012; 12(1):93. DOI:10.1186/1471-2288-12-93 · 2.17 Impact Factor
Show more