Reliability of perceived neighborhood conditions and the effects of measurement error on self-rated health across urban and rural neighborhoods

Washington University School of Medicine, Department of Medicine, Division of Health Behavior Research, Box 8504, 4444 Forest Park Avenue, Suite 6700, St Louis, MO 63108, USA.
Journal of epidemiology and community health (Impact Factor: 3.5). 11/2010; 66(4):342-51. DOI: 10.1136/jech.2009.103325
Source: PubMed


Limited psychometric research has examined the reliability of self-reported measures of neighbourhood conditions, the effect of measurement error on associations between neighbourhood conditions and health, and potential differences in the reliabilities between neighbourhood strata (urban vs rural and low vs high poverty). We assessed overall and stratified reliability of self-reported perceived neighbourhood conditions using five scales (social and physical disorder, social control, social cohesion, fear) and four single items (multidimensional neighbouring). We also assessed measurement error-corrected associations of these conditions with self-rated health.
Using random-digit dialling, 367 women without breast cancer (matched controls from a larger study) were interviewed twice, 2-3 weeks apart. Test-retest (intraclass correlation coefficients (ICC)/weighted κ) and internal consistency reliability (Cronbach's α) were assessed. Differences in reliability across neighbourhood strata were tested using bootstrap methods. Regression calibration corrected estimates for measurement error.
All measures demonstrated satisfactory internal consistency (α ≥ 0.70) and either moderate (ICC/κ=0.41-0.60) or substantial (ICC/κ=0.61-0.80) test-retest reliability in the full sample. Internal consistency did not differ by neighbourhood strata. Test-retest reliability was significantly lower among rural (vs urban) residents for two scales (social control, physical disorder) and two multidimensional neighbouring items; test-retest reliability was higher for physical disorder and lower for one multidimensional neighbouring item among the high (vs low) poverty strata. After measurement error correction, the magnitude of associations between neighbourhood conditions and self-rated health were larger, particularly in the rural population.
Research is needed to develop and test reliable measures of perceived neighbourhood conditions relevant to the health of rural populations.

Download full-text


Available from: Mario Schootman
  • Source
    • "is the mean of the replicate measurements for each participant, and λ is the ICC reliability coefficient [50]. We compared the mean calibrated (regression calibration) odds ratio (OR) that corrects for measurement error to the naïve analysis (uncorrected for measurement error). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Although neighborhood characteristics have important relationships with health outcomes, direct observation often involves imperfect measurement. The African American Health (AAH) study included two observer neighborhood rating systems (5-item Krause and 22-item AAH Neighborhood Assessment Scale [NAS]). Good measurement characteristics were previously shown for both, although there was more rater variability than desired. In 2010 both measures were re-fielded, hypothesizing that enhanced training and field methods would have decreased rater variability. AAH included a poor inner city and more heterogeneous suburban areas. Four interviewers rated 483 blocks, with 120 randomly selected blocks rated by two interviewers. We tested the Krause scale (5--20 points), AAH NAS with 22 items, plus a brief NAS (7-items; 0--17 points). Retest reliability for items (kappa) and scales (Intraclass Correlation Coefficient [ICC]) were calculated overall and among pre-specified subgroups. Linear regression assessed interviewer effects on total scale scores. Concurrent validity was examined using linear regression on lung and lower body functions, and self-rated health (SRH). The effect of mismeasurement on predicting self-rated health was assessed. Scale scores were lower (better) in the suburban area than in the inner city. ICC was poor for the Krause scale overall (ICC=0.19), but was better if the retests were within 10 days (ICC=0.49). The brief AAH NAS scale had a much better ICC (0.56) overall, and was even higher (0.71) within 10 days. Interviewers demonstrated strikingly different raw scores for both scales, with 1--3 point differences (compared to the supervising rater). Concurrent validity for health outcomes was modest, with residents living in worse neighborhood conditions demonstrating worse function and SRH. Unadjusted estimates were smaller (biased towards the null) compared with measurement-error corrected estimates. Enhanced field protocols and rater training that reduced the opportunity for interrater variability failed to fully eradicate prior measurement concerns. Specifically, retest reliability and interviewer variability remained problematic. The error effect reduced, but did not eliminate, expected findings in the validation analyses, suggesting there are robust associations between neighborhood characteristics and health outcomes. We urge additional examination of measurement properties of environmental rating methods and thorough discussion of field protocols and training.
    Full-text · Article · Oct 2013 · BMC Public Health
  • Source
    • "Test-retest reliability range from moderate for community ladder to good for society and work ladders according to Kappa statistics; and according to the ICC the reliability was substantial for the three ladders. Low reliability may influence the performance of a measure, contribute to participant misclassification and attenuate associations with other variables [22]. In our results the lower reliability was observed for the community ladder which showed moderate kappa statistics. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background The MacArthur Scale of Subjective Social Status intend to measure the subjective social status using a numbered stepladder image. This study investigated the reliability of the MacArthur scale in a subsample of the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil). Method Three scales were employed using different references: 1) the overall socioeconomic position; 2) the socioeconomic situation of the participant’s closer community; 3) the workplace as a whole. A total of 245 of the ELSA participants from six states were involved. They were interviewed twice by the same person within an interval of seven to fourteen days. The reliability of the scale was assessed with weighted Kappa statistics and intraclass correlation coefficient (ICC), with their respective 95% confidence interval (CI). Results Kappa values were 0.62(0.58 to 0.64) for the society ladder; 0.58(0.56 to 0.61) for the community-related ladder; and 0.67(0.66 to 0.72) for the work-related ladder. The ICC ranged from 0.75 for the work ladder to 0.64 for the community ladder. These values differed slightly according to the participants’ age, sex and education category. Conclusion The three ladders showed good stability in the test-retest, except the community ladder that showed moderate stability. Because the social structure in Brazil is rapidly changing, future qualitative and longitudinal studies are needed to confirm and understand the construct underlying the MacArthur Scale in the country.
    Full-text · Article · Dec 2012 · BMC Public Health
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: To estimate the effect of hypothetical changes in modifiable predictors on the incidence of fair-to-poor self-rated health (SRH) in breast cancer survivors. In 2007-2008, we interviewed 832 breast cancer survivors 1 year after diagnosis (baseline) and 1 year later. First, multivariable logistic regression models estimated the association between the predictors (sociodemographic factors, access to medical care, comorbid conditions, psychosocial factors, perceived neighborhood conditions, cancer-related behaviors, clinical factors) and SRH. Second, we estimated the probabilities of fair-to-poor SRH for values of the predictors for each breast cancer survivor. Third, we estimated the population-wide effect of potential changes in modifiable predictors on the incidence of fair-to-poor SRH. A total of 7.6% of participants (92.4% white; mean age, 58.0 years) whose SRH was rated good-to-excellent at baseline reported fair-to-poor SRH 1 year later. The largest potential reduction in incidence of fair-to-poor SRH could be obtained by eliminating surgical side effects (27.8% reduction) and comorbidity (21.8% reduction) and by engaging in any physical activity (19.6% reduction). A significant portion of the decline in SRH can be avoided by reducing surgical side effects, preventing comorbidity, and improving physical activity with the use of evidence-based strategies.
    Full-text · Article · Feb 2012 · Annals of epidemiology
Show more