Using cognitive interviews to develop surveys in diverse populations.

Medical Effectiveness Research Center for Diverse Populations and the Center on Aging in Diverse Communities, University of California at San Francisco, San Francisco, California 94118-1944, USA.
Medical Care (Impact Factor: 2.94). 12/2006; 44(11 Suppl 3):S21-30. DOI: 10.1097/01.mlr.0000245425.65905.1d
Source: PubMed

ABSTRACT Conceptual equivalence of measures is essential in research that compares health across diverse racial/ethnic groups. Cognitive interviews are pretest methods to explore the conceptual equivalence of survey items. Systematic approaches for using these methods are emerging.
We describe an interaction analysis (IA) approach using qualitative data analysis software to analyze transcripts of cognitive interviews in a study to develop a survey instrument of the quality of interpersonal processes of care of diverse patients. Cognitive interviews included standard administration of the survey followed by retrospective probes for selected items.
Interviews were completed with 48 Latino, black, and non-Latino white respondents 18 years of age or older with at least one doctor's visit in the past 12 months. Participants averaged 45.8 years in age (standard deviation [SD] = 18.4), 58% were women, and mean education was 14.7 years (SD = 4.0).
Problems were identified in 126 of 159 items (79%). Behavior coding identified 32 problematic items (20%). IA of the transcript of the survey and retrospective probes identified 94 additional problematic items (59%). IA often revealed the nature of the problems, enabling decisions to modify or drop items based on respondents' comments. Behavior coding and IA identified ethnic and language similarities and differences in the use of response sets and the interpretation of items.
IA and behavior coding of cognitive interview transcripts can identify efficiently problems with items and their source to increase the likelihood of the revised items being conceptually equivalent across ethnic groups.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: In Germany, the number of people with dementia living in nursing homes is rapidly increasing. Providing adequate care for their special needs is a challenge for institutions and their staff members. Because of the growing number of people with dementia, changes to the conceptual orientation of nursing homes have occurred. These changes include specialized living arrangements and psychosocial interventions recommended for people with dementia. Until now, the provision of dementia care and its association to the residents' behavior and quality of life is not well investigated in Germany. The purpose of this study is to describe the provision of dementia care and to identify resident- as well as facility-related factors associated with residents behavior and quality of life. The DemenzMonitor study is designed as a longitudinal study that is repeated annually. Data will be derived from a convenience sample consisting of nursing homes across Germany. For the data collection, three questionnaires have been developed that measure information on the level of the nursing home, the living units, and the residents. Data collection will be performed by staff members from the nursing homes. The data collection procedure will be supervised by a study coordinator who is trained by the research team. Data analysis will be performed on each data level using appropriate techniques for descriptions and comparisons as well as longitudinal regression analysis. The DemenzMonitor is the first study in Germany that assesses how dementia care is provided in nursing homes with respect to living arrangements and recommended interventions. This study links the acquired data with residents' outcome measurements, making it possible to evaluate different aspects and concepts of care.
    BMC Geriatrics 11/2013; 13(1):123. · 2.00 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Abstract In this AMEE Guide, we consider the design and development of self-administered surveys, commonly called questionnaires. Questionnaires are widely employed in medical education research. Unfortunately, the processes used to develop such questionnaires vary in quality and lack consistent, rigorous standards. Consequently, the quality of the questionnaires used in medical education research is highly variable. To address this problem, this AMEE Guide presents a systematic, seven-step process for designing high-quality questionnaires, with particular emphasis on developing survey scales. These seven steps do not address all aspects of survey design, nor do they represent the only way to develop a high-quality questionnaire. Instead, these steps synthesize multiple survey design techniques and organize them into a cohesive process for questionnaire developers of all levels. Addressing each of these steps systematically will improve the probabilities that survey designers will accurately measure what they intend to measure.
    Medical Teacher 03/2014; · 2.05 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Background and objectives. Despite their rapid increase in number, workers in personal care and service occupations are underrepresented in research on psychoso- cial work characteristics and occupational health. Some of the research challenges stem from the high proportion of immigrants in these occupations. Language barriers, 15 low literacy, and cultural differences as well as their nontraditional work setting (i.e., providing service for one person in his/her home) make generic questionnaire measures inadequate for capturing salient aspects of personal care and service work. This study presents strategies for (1) identifying psychosocial work characteristics of home care workers that may affect their occupational safety and health and (2) 20 creating survey measures that overcome barriers posed by language, low literacy, and cultural differences. Design and results. We pursued these aims in four phases: (Phase 1) Six focus groups to identify the psychosocial work characteristics affecting the home care workers ’ occupational safety and health; (Phase 2) Selection of questionnaire items 25 (i.e., questions or statements to assess the target construct) and first round of cognitive interviews (n = 30) to refine the items in an iterative process; (Phase 3) Item revision and second round of cognitive interviews (n = 11); (Phase 4) Quantitative pilot test to ensure the scales ’ reliability and validity across three language groups (English, Spanish, and Cantonese; total n = 404). Analysis of the data from each phase informed the nature of 30 subsequent phases. This iterative process ensured that survey measures not only met the reliability and validity criteria across groups, but were also meaningful to home care workers. Conclusion. This complex process is necessary when conducting research with nontraditional and multilingual worker populations.
    Ethnicity and Health 12/2014; · 1.20 Impact Factor

Full-text (2 Sources)

Available from
May 31, 2014