-
[show abstract]
[hide abstract]
ABSTRACT: OBJECTIVE: To produce reliable and informative health plan performance data by race/ethnicity for the Medicare beneficiary population and to consider appropriate presentation strategies. DATA SOURCES: Patient experience data from the 2008-2009 Medicare Advantage (MA) and fee-for-service (FFS) CAHPS surveys and 2008-2009 HEDIS data (MA beneficiaries only). STUDY DESIGN: Mixed effects linear (and binomial) regression models estimated the reliability and statistical informativeness of CAHPS (HEDIS) measures. PRINCIPAL FINDINGS: Seven CAHPS and seven HEDIS measures were reliable and informative for four racial/ethnic subgroups-Whites, Blacks, Hispanics, and Asian/Pacific Islanders-at sample sizes of 100 beneficiaries (200 for prescription drug plans). Although many plans lacked adequate sample size for reporting group-specific data, reportable plans contained a large majority of beneficiaries from each of the four racial/ethnic groups. CONCLUSIONS: Statistically reliable and valid information on health plan performance can be reported by race/ethnicity. Many beneficiaries may have difficulty understanding such reports, however, even with careful guidance. Thus, it is recommended that health plan performance data by subgroups be reported as supplemental data and only for plans meeting sample size requirements.
Health Services Research 04/2013; 48(2pt1):417-434. · 2.16 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: This report describes the findings from a demonstration conducted by RAND with the Florida Medicaid program to test applications of Consumer Assessment of Health Plans Study (CAHPS) consumer reports in the Medicaid sector. In collaboration with the Florida Agency for Health Care Administration (AHCA), we designed both paper and computer-based report formats, which were tested in field applications in Volusia County, Florida. This demonstration was the first of RAND's CAHPS demonstrations. Results of this work have been applied in subsequent demonstrations with the New Jersey and Iowa Medicaid programs.
11/2012;
-
[show abstract]
[hide abstract]
ABSTRACT: : To develop and evaluate survey questions that assess processes of care relevant to Patient-Centered Medical Homes (PCMHs).
: We convened expert panels, reviewed evidence on effective care practices and existing surveys, elicited broad public input, and conducted cognitive interviews and a field test to develop items relevant to PCMHs that could be added to the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician & Group (CG-CAHPS) 1.0 Survey. Surveys were tested using a 2-contact mail protocol in 10 adults and 33 pediatric practices (both private and community health centers) in Massachusetts. A total of 4875 completed surveys were received (overall response rate of 25%).
: We calculated the rate of valid responses for each item. We conducted exploratory factor analyses and estimated item-to-total correlations, individual and site-level reliability, and correlations among proposed multi-item composites.
: Ten items in 4 new domains (Comprehensiveness, Information, Self-Management Support, and Shared Decision-Making) and 4 items in 2 existing domains (Access and Coordination of Care) were selected to be supplemental items to be used in conjunction with the adult CG-CAHPS 1.0 Survey. For the child version, 4 items in each of 2 new domains (Information and Self-Management Support) and 5 items in existing domains (Access, Comprehensiveness-Prevention, Coordination of Care) were selected.
: This study provides support for the reliability and validity of new items to supplement the CG-CAHPS 1.0 Survey to assess aspects of primary care that are important attributes of PCMHs.
Medical care 11/2012; 50 Suppl:S2-S10. · 3.24 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: : Consumer assessment of health care is an important metric for evaluating quality of care. These assessments can help purchasers, health plans, and providers deliver care that fits patients' needs.
: To examine differences in reports and ratings of care delivered to adults and children and whether they vary by site.
: This observational study compares adult and child experiences with care at a large west coast medical center and affiliated clinics and a large mid-western health plan using Consumer Assessment of Healthcare Providers and Systems Clinician & Group 1.0 Survey data.
: Office staff helpfulness and courtesy was perceived more positively for adult than pediatric care in the west coast site. In contrast, more positive perceptions of pediatric care were observed in both sites for coordination of care, shared decision making, overall rating of the doctor, and willingness to recommend the doctor to family and friends. In addition, pediatric care was perceived more positively in the mid-west site for access to care, provider communication, and office staff helpfulness and courtesy. The differences between pediatric care and adult care were larger in the mid-western site than the west coast site.
: There are significant differences in the perception of care for children and adults with care provided to children tending to be perceived more positively. Further research is needed to identify the reasons for these differences and provide more definitive information at sites throughout the United States.
Medical care 11/2012; 50 Suppl:S35-9. · 3.24 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Using data from 335,249 Medicare beneficiaries who responded to the 2007 Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey, along with data from 22 cognitive interviews, we investigated the reliability and validity of an instrument designed to assess beneficiaries' experiences with their prescription drug plans. Composite measures derived from the instrument had acceptable internal consistency and sufficient plan-level reliability to inform consumer choice, quality improvement, and payor oversight. These measures were positively associated with members' overall rating of the plan and their willingness to recommend the plan. Moreover, each was independently useful in predicting beneficiaries' global ratings of their plan. This instrument can be an important tool for helping beneficiaries to choose a plan that best meets their needs.
Health care financing review 02/2009; 30(3):41-53. · 2.06 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The medical group survey from the CAHPS (formerly Consumer Assessment of Health Plans Study) project, G-CAHPS, focuses on patient experiences in receiving care from their medical group practice. We compared mail and telephone responses to the G-CAHPS survey in a sample of 880 patients from four physician groups. Patients were randomly assigned to mode. Analyses included comparison of response rates, missing data, internal consistency reliability of six multi-item scales, and mean scores. A total of 537 phone completes and 343 mail completes were obtained (54% response rate). There were no significant differences in internal consistency by mode. In addition, there was only one significant mode difference in item and composite means by mode of administration after adjusting for case-mix differences. This study indicates that mail and telephone modes of data collection for the G-CAHPS survey produce similar results.
Evaluation & the Health Professions 01/2006; 28(4):377-89. · 1.23 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: To describe how cognitive testing results were used to inform the modification and selection of items for the Consumer Assessment of Health Providers and Systems (CAHPS) Hospital Survey pilot test instrument.
Cognitive interviews were conducted on 31 subjects in two rounds of testing: in December 2002-January 2003 and in February 2003. In both rounds, interviews were conducted in northern California, southern California, Massachusetts, and North Carolina.
A common protocol served as the basis for cognitive testing activities in each round. This protocol was modified to enable testing of the items as interviewer-administered and self-administered items and to allow members of each of three research teams to use their preferred cognitive research tools.
Each research team independently summarized, documented, and reported their findings. Item-specific and general issues were noted. The results were reviewed and discussed by senior staff from each research team after each round of testing, to inform the acceptance, modification, or elimination of candidate items.
Many candidate items required modification because respondents lacked the information required to answer them, respondents failed to understand them consistently, the items were not measuring the constructs they were intended to measure, the items were based on erroneous assumptions about what respondents wanted or experienced during their hospitalization, or the items were asking respondents to make distinctions that were too fine for them to make. Cognitive interviewing enabled the detection of these problems; an understanding of the etiology of the problem informed item revisions. However, for some constructs, the revisions proved to be inadequate. Accordingly, items could not be developed to provide acceptable measures of certain constructs such as shared decision making, coordination of care, and delays in the admissions process.
Cognitive testing is the most direct way of finding out whether respondents understand questions consistently, have the information needed to answer the questions, and can use the response alternatives provided to describe their experiences or their opinions accurately. Many of the candidate questions failed to meet these standards. Cognitive testing only evaluates the way in which respondents understand and answer questions. Although it does not directly assess the validity of the answers, it is a reasonable premise that cognitive problems will seriously compromise validity and reliability.
Health Services Research 01/2006; 40(6 Pt 2):2037-56. · 2.16 Impact Factor
-
Takahiro Higashi,
Ron D Hays, Julie A Brown,
Caren J Kamberg,
Chau Pham,
David B Reuben,
Paul G Shekelle,
David H Solomon,
Roy T Young,
Carol P Roth,
John T Chang,
Catherine H MacLean,
Neil S Wenger
[show abstract]
[hide abstract]
ABSTRACT: While falls and urinary incontinence are prevalent among older patients, who sometimes rely on proxies to provide their health information, the validity of proxy reports of concern about falls and urinary incontinence remains unknown.
Telephone interviews with 43 consecutive patients with falls or fear of falling and/or bothersome urinary incontinence and their proxies chosen by patients as most knowledgeable about their health. The questionnaire included items derived from the Medical Outcomes Study Short Form 12 (SF-12), a scale assessing concerns about urinary incontinence (UI), and a measure of fear of falling, the Falls Efficacy Scale (FES). Scores were estimated using items asking the proxy perspective (6 items from the SF-12, 10 items from a UI scale, and all 10 FES items). Proxy and patient scores were compared using intraclass correlation coefficients (ICC, one-way model). Variables associated with absolute agreement between patients and proxies were explored.
Patients had a mean age of 81 years (range 75-93) and 67% were female while proxies had a mean age of 70 (range 42-87) and 49% were female. ICCs were 0.63 for the SF-12, 0.52 for the UI scale, and 0.29 for the FES. Proxies tended to understate patients' general health and incontinence concern, but overstate patients' concern about falling. Proxies who lived with patients and those who more often see patients more closely reflected patient FES scores compared to those who lived apart or those who saw patients less often. Internal consistency reliability of proxy responses was 0.62 for the SF-12, 0.86 for the I-QOL, and 0.93 for the FES. In addition, construct validity of the proxy FES scale was supported by greater proxy-perceived fear of falling for patients who received medical care after a fall during the past 12 months (p < .05).
Caution should be exercised when using proxies as a source of information about older patients' health perceptions. Questions asking about proxies' views yield suboptimal agreement with patient responses. However, proxy scales of UI and fall concern are internally consistent and may provide valid independent information.
Health and Quality of Life Outcomes 01/2005; 3:75. · 2.11 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: To assess the effects of CAHPS health plan performance information on plan choices and decision processes by New Jersey Medicaid beneficiaries.
The study sample was a statewide sample of all new Medicaid cases that chose Medicaid health plans during April 1998. The study used state data on health maintenance organization (HMO) enrollments and survey data for a subset of these cases.
An experimental design was used, with new Medicaid cases randomly assigned to experimental or control groups. The experimental group received a CAHPS report along with the standard enrollment materials, and the control group did not.
The HMO enrollment data were obtained from the state in June 1998, and evaluation survey data were collected from July to October 1998.
No effects of CAHPS information on HMO choices were found for the total sample. Further examination revealed that only about half the Medicaid cases said they received and read the plan report and there was an HMO with dominant Medicaid market share but low CAHPS performance scores. The subset of cases who read the report and did not choose this dominant HMO chose HMOs with higher CAHPS scores, on average, than did those in an equivalent control group.
Health plan performance information can influence plan choices by Medicaid beneficiaries, but will do so only if they actually read it. These findings suggest a need for enhancing dissemination of the information as well as further education to encourage informed choices.
Health Services Research 09/2002; 37(4):985-1007. · 2.16 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The focus of the literature review is consumer assessment of providers at the medical group level. The Internet provides access to a range of searchable databases. Without a search strategy, however, untold hours may be spent scouring different, sometimes overlapping bibliographies. For this reason, we devised a systematic search strategy to ensure a quality and comprehensive search. Time limits and other constraints on the search helped to combat diminishing returns. Thorough documentation is the best guarantee that references are not lost or unnecessarily duplicated. To maintain quality control in our literature review, we documented not only the reference of interest, but also any information about the search path that led to the reference. This helped to retrospectively analyze the comprehensiveness of our search and allows for the reproducibility of the search results. For each iteration of the literature search, we recorded the following information: (a) database searched, (b) key words and search option combinations, (c) specific organization or WWW site, (d) years used for the search, (e) number of returned references per search, and (f) number of selected references per search.
08/2000;
