Using Cognitive Interviews to Develop Surveys in Diverse Populations

Department of Medicine, University of California, San Francisco, San Francisco, California, United States
Medical Care (Impact Factor: 3.23). 12/2006; 44(11 Suppl 3):S21-30. DOI: 10.1097/01.mlr.0000245425.65905.1d
Source: PubMed


Conceptual equivalence of measures is essential in research that compares health across diverse racial/ethnic groups. Cognitive interviews are pretest methods to explore the conceptual equivalence of survey items. Systematic approaches for using these methods are emerging.
We describe an interaction analysis (IA) approach using qualitative data analysis software to analyze transcripts of cognitive interviews in a study to develop a survey instrument of the quality of interpersonal processes of care of diverse patients. Cognitive interviews included standard administration of the survey followed by retrospective probes for selected items.
Interviews were completed with 48 Latino, black, and non-Latino white respondents 18 years of age or older with at least one doctor's visit in the past 12 months. Participants averaged 45.8 years in age (standard deviation [SD] = 18.4), 58% were women, and mean education was 14.7 years (SD = 4.0).
Problems were identified in 126 of 159 items (79%). Behavior coding identified 32 problematic items (20%). IA of the transcript of the survey and retrospective probes identified 94 additional problematic items (59%). IA often revealed the nature of the problems, enabling decisions to modify or drop items based on respondents' comments. Behavior coding and IA identified ethnic and language similarities and differences in the use of response sets and the interpretation of items.
IA and behavior coding of cognitive interview transcripts can identify efficiently problems with items and their source to increase the likelihood of the revised items being conceptually equivalent across ethnic groups.

Download full-text


Available from: Jasmine Santoyo-Olsson, Apr 29, 2014
54 Reads
    • "Pretest A pretest with informal dementia carers was performed . We used cognitive interview techniques (Collins 2003, Napoles-Springer et al. 2006), namely the 'Think Aloud' method (Pr€ ufer & Rexroth 2000) to check the questionnaire's understandability and practicability . The results of the pretest helped to modify the content of the instrument and improve the visual "
    [Show abstract] [Hide abstract]
    ABSTRACT: The majority of people with dementia live at home, and informal carers assume the role of key care providers, often supported by formal services. The purpose of this pilot study was to assess home-based care arrangements, to illustrate utilisation of formal services over time and to identify factors associated with perceived stability of the care situation from the informal carer's perspective. A self-administered questionnaire (D-IVA 'Instrument for Assessing Home-Based Care Arrangements for People with Dementia') was developed and distributed in a provincial-rural setting in Germany as a cross-sectional survey. Data analysis used descriptive statistics, unbiased conditional inference trees and thematic analysis for open-ended questions. In total, 84 care arrangements were assessed. The majority of participants were direct relatives of the care-dependent person [mostly adult children (48.8%) or spouses (27.4%)]. Formal services were already sought in the first year after onset of memory problems. The most frequently used formal services were home care nursing services (53.0%), day care (49.4%) and respite care (29.6%), whereas 15.5% did not use any type of formal support. Companion home visit, home care nursing service and day care were used over the longest periods of time. The recruitment strategy used in this study may have recruited persons who were relatively more dependent on their informal carers. In this small sample, carers' perceived stability of the care situation was high, and this was associated with the country of origin and sex of the person with dementia (P = 0.004 and 0.023 respectively). Most care arrangements consisted of a mix of informal and formal services. However, informal carers assumed prime responsibility. The questionnaire D-IVA proved to be suitable. It remains a challenge to further examine factors associated with perceived stability and to explain the phenomenon in its whole complexity. Further research using the D-IVA should consider applying complementing quantitative measures as well as qualitative methods. © 2015 John Wiley & Sons Ltd.
    Health & Social Care in the Community 02/2015; DOI:10.1111/hsc.12202 · 1.15 Impact Factor
  • Source
    • "Typically, an interaction analysis attempts to describe and explain the ways in which people interpret and interact during a conversation, and this method can be applied during the administration of a cognitive interview to determine the meaning of responses (Napoles-Springer et al. 2006). Studies have demonstrated that the combination of coding and interaction analysis can be quite effective, providing more information about the “cognitive validity” of a new questionnaire (Napoles-Springer et al. 2006). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Abstract In this AMEE Guide, we consider the design and development of self-administered surveys, commonly called questionnaires. Questionnaires are widely employed in medical education research. Unfortunately, the processes used to develop such questionnaires vary in quality and lack consistent, rigorous standards. Consequently, the quality of the questionnaires used in medical education research is highly variable. To address this problem, this AMEE Guide presents a systematic, seven-step process for designing high-quality questionnaires, with particular emphasis on developing survey scales. These seven steps do not address all aspects of survey design, nor do they represent the only way to develop a high-quality questionnaire. Instead, these steps synthesize multiple survey design techniques and organize them into a cohesive process for questionnaire developers of all levels. Addressing each of these steps systematically will improve the probabilities that survey designers will accurately measure what they intend to measure.
    Medical Teacher 03/2014; 36(6). DOI:10.3109/0142159X.2014.889814 · 1.68 Impact Factor
  • Source
    • "Focus groups and quantitative expert ratings were also conducted to achieve content and face validity [33,34]. A multi-method pretest with intended users revealed problems with comprehensibility and practicability [35,36]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In Germany, the number of people with dementia living in nursing homes is rapidly increasing. Providing adequate care for their special needs is a challenge for institutions and their staff members. Because of the growing number of people with dementia, changes to the conceptual orientation of nursing homes have occurred. These changes include specialized living arrangements and psychosocial interventions recommended for people with dementia. Until now, the provision of dementia care and its association to the residents' behavior and quality of life is not well investigated in Germany. The purpose of this study is to describe the provision of dementia care and to identify resident- as well as facility-related factors associated with residents behavior and quality of life. The DemenzMonitor study is designed as a longitudinal study that is repeated annually. Data will be derived from a convenience sample consisting of nursing homes across Germany. For the data collection, three questionnaires have been developed that measure information on the level of the nursing home, the living units, and the residents. Data collection will be performed by staff members from the nursing homes. The data collection procedure will be supervised by a study coordinator who is trained by the research team. Data analysis will be performed on each data level using appropriate techniques for descriptions and comparisons as well as longitudinal regression analysis. The DemenzMonitor is the first study in Germany that assesses how dementia care is provided in nursing homes with respect to living arrangements and recommended interventions. This study links the acquired data with residents' outcome measurements, making it possible to evaluate different aspects and concepts of care.
    BMC Geriatrics 11/2013; 13(1):123. DOI:10.1186/1471-2318-13-123 · 1.68 Impact Factor
Show more