[Show abstract][Hide abstract] ABSTRACT: Anonymous survey methods appear to promote greater disclosure of sensitive or stigmatizing information compared to non-anonymous methods. Higher disclosure rates have traditionally been interpreted as being more accurate than lower rates. We examined the impact of 3 increasingly private mailed survey conditions--ranging from potentially identifiable to completely anonymous, on survey response and on respondents' representativeness of the underlying sampling frame--completeness in answering sensitive survey items, and disclosure of sensitive information. We also examined the impact of 2 incentives ($10 versus $20) on these outcomes.
BMC Medical Research Methodology 07/2014; 14(1):90. · 2.21 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Prior research has examined clinical effects of performance measurement systems. To the extent that non-clinical effects have been researched, the focus has been on negative unintended consequences. Yet, these same systems may also have ancillary benefits for patients and providers-that is, benefits that extend beyond improvements on clinical measures. The purpose of this study is to identify and describe potential ancillary benefits of performance measures as perceived by primary care staff and facility leaders in a large US healthcare system.
In-person individual semistructured interviews were conducted with 59 primary care staff and facility leaders at four Veterans Health Administration facilities. Transcribed interviews were coded and organised into thematic categories.
Interviewed staff observed that local performance measurement implementation practices can result in increased patient knowledge and motivation. These effects on patients can lead to improved performance scores and additional ancillary benefits. Performance measurement implementation can also directly result in ancillary benefits for the patients and providers. Patients may experience greater satisfaction with care and psychosocial benefits associated with increased provider-patient communication. Ancillary benefits of performance measurement for providers include increased pride in individual or organisational performance and greater confidence that one's practice is grounded in evidence-based medicine.
A comprehensive understanding of the effects of performance measurement systems needs to incorporate ancillary benefits as well as effects on clinical performance scores and negative unintended consequences. Although clinical performance has been the focus of most evaluations of performance measurement to date, both patient care and provider satisfaction may improve more rapidly if all three categories of effects are considered when designing and evaluating performance measurement systems.
[Show abstract][Hide abstract] ABSTRACT: The purpose of this study was to investigate possible racial differences in opioid prescriptions among primary care patients with chronic, non-cancer pain receiving care in the Veterans Affairs (VA) healthcare system. This was a retrospective cohort study of 99,903 veterans with diagnoses of low back, neck, or joint pain, selected to participate in the VA Survey of the Healthcare Experiences of Patients (SHEP) in Fiscal Year 2006. The outcome was prescription of opioids in the year following the first pain diagnosis, obtained through electronic medical record data. Analyses incorporated fixed effects for race, most recent pain intensity rating, new or established primary care patient status, and an interaction between race and most recent pain intensity rating, together with random effects for healthcare facility and race within facility. The association between patient race and prescription of opioids was moderated by baseline level of pain intensity scores (assessed on a 0-10 scale) and patient age. Among patients under 65 years of age, blacks with moderate (4-6) or high levels (7-10) of pain were less likely to receive opioids than whites (p = 0.0025; p = 0.0011); however there were no significant differences between black and white patients with low levels of pain intensity (1-3) and those with pain intensity ratings of 0 (no pain). Among patients 65 and older with pain intensity ratings of zero, blacks were more likely than whites to receive opioid prescriptions (p = .0087), but there were no significant racial differences in opioid prescriptions in those with low to high levels of pain.
Among veterans under age 65 reporting moderate to high levels of chronic, non-cancer pain, blacks were less likely to be prescribed opioids than whites, even after controlling for clinical and system-level factors. Results underscore the challenges of eliminating racial differences in pain treatment, despite comprehensive system-wide improvement initiatives.
The journal of pain: official journal of the American Pain Society 01/2014; · 3.78 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: This study assessed whether postal follow-up to a web-based physician survey improves response rates, response quality, and representativeness. We recruited primary care and gastroenterology chiefs at 125 Veterans Affairs medical facilities to complete a 10-min web-based survey on colorectal cancer screening and diagnostic practices in 2010. We compared response rates, response errors, and representativeness in the primary care and gastroenterology samples before and after adding postal follow-up. Adding postal follow-up increased response rates by 20-25 percentage points; markedly greater increases than predicted from a third e-mail reminder. In the gastroenterology sample, the mean number of response errors made by web responders (0.25) was significantly smaller than the mean number made by postal responders (2.18), and web responders provided significantly longer responses to open-ended questions. There were no significant differences in these outcomes in the primary care sample. Adequate representativeness was achieved before postal follow-up in both samples, as indicated by the lack of significant differences between web responders and the recruitment population on facility characteristics. We conclude adding postal follow-up to this web-based physician leader survey improved response rates but not response quality or representativeness.
Evaluation & the Health Professions 12/2013; · 1.48 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Primary care providers frequently recommend, administer, or prescribe health care services that are unlikely to benefit their patients. Yet little is known about how to reduce provider overuse behavior. In the absence of a theoretically grounded causal framework, it is difficult to predict the contexts under which different types of interventions to reduce provider overuse will succeed and under which they will fail. In this article, we present a framework based on the theory of planned behavior that is designed to guide overuse research and intervention development. We describe categories of primary care provider beliefs that lead to the formation of intentions to assess the appropriateness of services, and propose factors that may affect whether the presence of assessment intentions results in an appropriate recommendation. Interventions that have been commonly used to address provider overuse behavior are reviewed within the context of the framework.
Medical Care Research and Review 08/2013; · 3.01 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: OBJECTIVES:: To determine whether black patients are less likely to be screened for pain than white patients. PARTICIPANTS:: A sample of 25,382 black and 220,122 non-Hispanic white Veterans Affairs (VA) patients was identified among the panel surveyed in the ambulatory care module of the 2007 Survey of Health Care Experiences of Patients. DESIGN:: This was a cross-sectional analysis of documentation of a pain score in the electronic medical record at the patient's Survey of Health Care Experiences of Patients index visit. Hierarchical logistic regression analyses were used to examine the association between race and documentation of pain screening. RESULTS:: After accounting for site and whether the patient was a new or established primary care patient, black VA patients were significantly less likely than their white counterparts to be screened for pain, odds ratio: 0.79, P<0.0001, with estimated screening rates of 78% and 82% for black and white established primary care patients at a typical VA site, respectively. Further adjusting for demographics, medical and psychological comorbidity, prescription of pain medication, and health care utilization reduced the odds ratio to 0.86, P<0.0001). Additional analyses revealed that this reduction in odds ratio was primarily explained by higher rates of outpatient visits to the VA in the previous 2 years among black patients, which was associated with lower rates of screening at the index visit. CONCLUSIONS:: Rates of screening were lower among black patients. The magnitude of this disparity was small and was explained, in part, by racial variation in prior health care utilization.
The Clinical journal of pain 11/2012; · 3.01 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: BACKGROUND: Policy-makers have called for efforts to reduce overuse of cancer screening tests, including colorectal cancer screening (CRCS). Overuse of CRCS tests other than colonoscopy has not been well documented. OBJECTIVE: To estimate levels and correlates of fecal occult blood test (FOBT) overuse in a national Veterans Health Administration (VHA) sample. DESIGN: Observational PARTICIPANTS: Participants included 1,844 CRCS-eligible patients who responded to a 2007 CRCS survey conducted in 24 VHA facilities and had one or more FOBTs between 2003 and 2009. MAIN MEASURES: We combined survey data on race, education, and income with administrative data on region, age, gender, CRCS procedures, and outpatient visits to estimate overuse levels and variation. We coded FOBTs as overused if they were conducted <10 months after prior FOBT, <9.5 years after prior colonoscopy, or <4.5 years after prior barium enema. We used multinomial logistic regression models to examine variation in overuse by reason (sooner than recommended after prior FOBT; sooner than recommended after colonoscopy, barium enema, or a combination of procedures), adjusting for clustering of procedures within patients, and patients within facilities. KEY RESULTS: Of 4,236 FOBTs received by participants, 885 (21 %) met overuse criteria, with 323 (8 %) sooner than recommended after FOBT, and 562 (13 %) sooner than recommended after other procedures. FOBT overuse varied across facilities (9-32 %, p < 0.0001) and region (12-23 %, p < .0012). FOBT overuse after prior FOBT declined between 2003 and 2009 (8 %-5 %, p = .0492), but overuse after other procedures increased (11-19 %, p = .0002). FOBT overuse of both types increased with number of outpatient visits (OR 1.15, p < 0.001), but did not vary by patient demographics. More than 11 % of overused FOBTs were followed by colonoscopy within 12 months. CONCLUSIONS: Many FOBTs are performed sooner than recommended in the VHA. Variation in overuse by facility, region, and outpatient visits suggests addressing FOBT overuse will require system-level solutions.
Journal of General Internal Medicine 07/2012; · 3.28 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Multilevel intervention research holds the promise of more accurately representing real-life situations and, thus, with proper research design and measurement approaches, facilitating effective and efficient resolution of health-care system challenges. However, taking a multilevel approach to cancer care interventions creates both measurement challenges and opportunities.
One-thousand seventy two cancer care articles from 2005 to 2010 were reviewed to examine the state of measurement in the multilevel intervention cancer care literature. Ultimately, 234 multilevel articles, 40 involving cancer care interventions, were identified. Additionally, literature from health services, social psychology, and organizational behavior was reviewed to identify measures that might be useful in multilevel intervention research.
The vast majority of measures used in multilevel cancer intervention studies were individual level measures. Group-, organization-, and community-level measures were rarely used. Discussion of the independence, validity, and reliability of measures was scant.
Measurement issues may be especially complex when conducting multilevel intervention research. Measurement considerations that are associated with multilevel intervention research include those related to independence, reliability, validity, sample size, and power. Furthermore, multilevel intervention research requires identification of key constructs and measures by level and consideration of interactions within and across levels. Thus, multilevel intervention research benefits from thoughtful theory-driven planning and design, an interdisciplinary approach, and mixed methods measurement and analysis.
[Show abstract][Hide abstract] ABSTRACT: Using data from an improvement collaborative, we examined whether facility-specific conclusions regarding the success of efforts to improve timely access could vary depending on the type of measure used. The sample was drawn from 21 Veterans Health Administration (VHA) medical facilities participating in a collaborative on timely diagnostic evaluation following positive fecal occult blood tests (FOBT+). We identified FOBT+ cases from participating facilities between September 2004 and August 2005 (precollaborative), and September 2006-August 2007 (postcollaborative). Dates of FOBT+ results, colonoscopy, and death were extracted from VHA medical records. We estimated the cumulative proportion receiving colonoscopy within 2 months of the FOBT+ (target measure established by collaborative), and compared facility-specific results regarding improvement on this measure to results from measures of the cumulative proportion receiving colonoscopy within 12 months, and average time-to-colonoscopy. In 12 facilities (57%), all measures suggested consistent results regarding pre-post collaborative changes in colonoscopy rates. In four facilities (19%), the target measure suggested less favorable change, and in five (24%), more favorable change than one or both other measures. Because conclusions drawn about the success of QI efforts can vary by the measure used, multiple measures should be employed to track progress toward timeliness goals.
[Show abstract][Hide abstract] ABSTRACT: Although benefits of performance measurement (PM) systems have been well documented, there is little research on negative unintended consequences of performance measurement systems in primary care. To optimize PM systems, a better understanding is needed of the types of negative unintended consequences that occur and of their causal antecedents.
(1) Identify unintended negative consequences of PM systems for patients. (2) Develop a conceptual framework of hypothesized relationships between PM systems, facility-level variables (local implementation strategies, primary care staff attitudes and behaviors), and unintended negative effects on patients. DESIGN, PARTICIPANTS, APPROACH: Qualitative study design using dissimilar cases sampling. A series of 59 in-person individual semi-structured interviews at four Veterans Health Administration (VHA) facilities was conducted between February and July 2009. Participants included members of primary care staff and facility leaders. Sites were selected to assure variability in the number of veterans served and facility scores on national VHA performance measures. Interviews were recorded, transcribed and content coded to identify thematic categories and relationships.
Participants noted both positive effects and negative unintended consequences of PM. We report three negative unintended consequences for patients. Performance measurement can (1) lead to inappropriate clinical care, (2) decrease provider focus on patient concerns and patient service, and (3) compromise patient education and autonomy. We also illustrate examples of negative consequences on primary care team dynamics. In many instances these problems originate from local implementation strategies developed in response to national PM definitions and policies.
Facility-level strategies undertaken to implement national PM systems may result in inappropriate clinical care, can distract providers from patient concerns, and may have a negative effect on patient education and autonomy. Further research is needed to ascertain how features of centralized PM systems influence whether measures are translated locally by facilities into more or less patient-centered policies and processes.
Journal of General Internal Medicine 10/2011; 27(4):405-12. · 3.28 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Improving colorectal cancer (CRC) screening rates represents a challenge for primary care providers. Some have argued that offering a choice of CRC screening modes to patients will improve the currently low adherence rates. Others have raised concerns that offering numerous CRC screening options in practice could overwhelm patients and thus dampen enthusiasm for screening. In this article we assemble evidence to critically evaluate the relative merit of these opposing views. We find little evidence to support the hypothesis that the number of options offered will affect adherence (either positively or negatively), or that expanding the modalities offered beyond FOBT and colonoscopy will improve patient satisfaction. Therefore, we assert future decisions about the number of CRC screening modes to offer would more productively be focused on considerations such as what benefit the health-care organization would derive from offering additional modes, and how this change would affect other critical components of a successful screening program such as timely diagnosis. In light of these organizational level considerations, we agree with the assertion made by others that a screening program limited to FOBT and colonoscopy is likely to be ideal in most settings.
Journal of General Internal Medicine 09/2011; 27(3):376-80. · 3.28 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: In 2005, the Veterans Health Administration initiated a yearlong Colorectal Cancer Care Collaborative (C4) to improve timely follow-up after positive fecal occult blood tests.
Twenty-one facilities formed local quality improvement (QI) teams. Teams received QI training, created process flow maps, implemented process changes, and shared learning through 2 face-to-face meetings, conference calls, and a discussion board. We evaluated pre-post change in the timeliness of follow-up among C4 facilities and 3 control facilities. Outcome measures included the proportion of patients receiving a follow-up colonoscopy within 1 year, the proportion receiving 60-day follow-up (the focus of C4 teams), and average days to colonoscopy. Survey data from C4 team members was analyzed to identify predictors of facility-level improvement.
Both C4 and control facilities improved on 1-year follow-up (10% and 9% increases, respectively, both P's<0.001). There was a statistically significant increase in the proportion receiving 60-day follow-up among C4 facilities (27% pre-C4 vs. 39% post-C4, P=0.008) but a nonsignificant decrease among control facilities (45% pre-C4 vs. 29% post-C4, P=0.14). Average days to colonoscopy decreased significantly among C4 facilities (129 pre-C4 vs. 103 post-C4, P=0.004) but increased significantly among control facilities (81 pre-C4 vs. 103 post-C4, P=0.04). Teams with the most improvement established clear roles/goals, had previous QI training, made more use of QI tools, and incorporated primary care education into their improvement work.
A Veterans Health Administration improvement collaborative modestly decreased time to colonoscopy after a positive colorectal cancer screening test but significant room for improvement remains and benefits of participation were not realized by all facilities.
Medical care 06/2011; 49(10):897-903. · 3.24 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Maximizing response rates is critically important in order to provide the most generalizable and unbiased research results. High response rates reduce the chance of respondents being systematically different from non-respondents, and thus, reduce the risk of results not truly reflecting the study population. Monetary incentives are often used to improve response rates, but little is known about whether larger incentives improve response rates in those who previously have been unenthusiastic about participating in research. In this study we compared the response rates and cost-effectiveness of a $5 versus $2 monetary incentive accompanying a short survey mailed to patients who did not respond or refused to participate in research study with a face-to-face survey.
1,328 non-responders were randomly assigned to receive $5 or $2 and a short, 10-question survey by mail. Reminder postcards were sent to everyone; those not returning the survey were sent a second survey without incentive. Overall response rates, response rates by incentive condition, and odds of responding to the larger incentive were calculated. Total costs (materials, postage, and labor) and incremental cost-effectiveness ratios were also calculated and compared by incentive condition.
After the first mailing, the response rate within the $5 group was significantly higher (57.8% vs. 47.7%, p<.001); after the second mailing, the difference narrowed by 80%, resulting in a non-significant difference in cumulative rates between the $5 and $2 groups (67.3% vs. 65.4%, respectively, p=.47). Regardless of incentive or number of contacts, respondents were significantly more likely to be male, white, married, and 50-75 years old. Total costs were higher with the larger versus smaller incentive ($13.77 versus $9.95 per completed survey).
A $5 incentive provides a significantly higher response rate than a $2 incentive if only one survey mailing is used but not if two survey mailings are used.
BMC Medical Research Methodology 05/2011; 11:81. · 2.21 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Evaluate a computer-assisted telephone counseling (CATC) decision aid for men considering a prostate specific antigen (PSA) test.
Eligible men were invited by their primary care providers (PCPs) to participate. Those consenting received an educational booklet followed by CATC. The counselor assessed stage of readiness, reviewed booklet information, corrected knowledge deficits and helped with a values clarification exercise. The materials presented advantages and disadvantages of being screened and did not advocate for testing or for not testing. Outcome measures included changes in stage, decisional conflict, decisional satisfaction, perceived vulnerability and congruence of a PSA testing decision with a pros/cons score. Baseline and final surveys were administered by telephone.
There was an increase in PSA knowledge (p<0.001), and in decisional satisfaction (p<0.001), a decrease in decisional conflict (p<0.001), and a general consistency of those decisions with the man's values. Among those initially who had not made a decision, 83.1% made a decision by final survey with decisions equally for or against screening.
The intervention provides realistic, unbiased and effective decision support for men facing a difficult and confusing decision.
Our intervention could potentially replace a discussion of PSA testing with the PCP for most men.
Patient Education and Counseling 02/2011; 82(2):193-200. · 2.60 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Four population-based studies of screening for CRC with fecal occult blood testing (FOBT) have shown that mortality can be significantly reduced. However, nearly half of all positive screening tests are not appropriately evaluated.
We evaluated whether an electronic record intervention improved the follow-up of patients with a positive FOBT (FOBT+) result.
We conducted a cluster randomized trial involving four Veteran's Affairs (VA) medical centers pair-matched by colonoscopy volume and randomized within the pair to receive the electronic intervention or usual care.
All patients with FOBT+ results at participating facilities during a matched pre- and post-intervention time period.
In the two intervention sites, an electronic consult that imported relevant clinical information was automatically submitted to the gastroenterology (GI) clinic for all FOBT+ patients at the time the result was recorded in the laboratory. In both intervention and control sites (usual care), PCPs continued to be notified of FOBT+ results in the usual manner
Pre- and post-intervention changes in the proportion of FOBT+ patients having: (1) a GI consult or (2) a GI consult plus complete diagnostic evaluation (CDE) of the colon within 30, 90 and 180 days were compared across intervention and control sites. Log rank tests were used to determine statistical significance.
The 30-, 90- and 180-day GI consult rates improved 21-33 % (p < 0.001) among intervention sites, but did not change in the usual care sites. Thirty-, 90- and 180-day CDE rates improved 9-31% (p < 0.03) in intervention sites, but did not significantly change in the usual care sites. Time to GI consult and CDE decreased significantly over time in the intervention sites (p < 0.001), but remained unchanged in the usual care sites.
The relatively simple electronic intervention evaluated can significantly improve the follow-up of FOBT+ results. Interventions such as this could improve patient care and may be applicable to other practice settings, as well as other types of tests.
Journal of General Internal Medicine 02/2011; 26(7):691-7. · 3.28 Impact Factor