Recognition of Patient Referral Desires in an Academic Managed Care Plan. Frequency, Determinants, and Outcomes
To determine the frequency and determinants of provider nonrecognition of patients' desires for specialist referral. Prospective study. Internal medicine clinic in an academic medical center providing primary care to patients enrolled in a managed care plan. Twelve faculty internists serving as primary care providers (PCPs) for 856 patient visits. Patients were given previsit and postvisit questionnaires asking about referral desire and visit satisfaction. Providers, blinded to patients' referral desire, were asked after the visit whether a referral was discussed, who initiated the referral discussion, and whether the referral was indicated. Providers failed to discuss referral with 27% of patients who indicated a definite desire for referral and with 56% of patients, who indicated a possible desire for referral. There was significant variability in provider recognition of patient referral desire. Recognition is defined as the provider indicating that a referral was discussed when the patient marked a definite or possible desire for referral. Provider recognition improved significantly (P <.05), when the patient had more than one referral desire, if the patient or a family member was a health care worker and when the patient noted a definite desire versus a possible desire for referral. Patients were more likely (P <.05) to initiate a referral discussion when they had seen the PCP previously and had more than one referral desire. Of patient-initiated referral requests, 14% were considered "not indicated" by PCPs. Satisfaction with care did not differ in patients with a referral desire that were referred and those that were nor referred. These PCPs frequently failed to explicitly recognize patients' referral desires. Patients were more likely to initiate discussions of a referral desire when they saw their usual PCP and had more than a single referral desire.
Recognition of Patient Referral Desires in an
Academic Managed Care Plan
Frequency, Determinants, and Outcomes
Gail A. Albertson, MD, C. T. Lin, MD, Jean Kutner, MD, Lisa M. Schilling, MD,
Susan N. Anderson, BS, Robert J. Anderson, MD
To determine the frequency and determinants
of provider nonrecognition of patients’ desires for specialist
Internal medicine clinic in an academic medical
center providing primary care to patients enrolled in a man-
aged care plan.
Twelve faculty internists serving as primary
care providers (PCPs) for 856 patient visits.
MEASUREMENTS AND MAIN RESULTS:
Patients were given
previsit and postvisit questionnaires asking about referral de-
sire and visit satisfaction. Providers, blinded to patients’ re-
ferral desire, were asked after the visit whether a referral was
discussed, who initiated the referral discussion, and whether
the referral was indicated. Providers failed to discuss referral
with 27% of patients who indicated a definite desire for refer-
ral and with 56% of patients, who indicated a possible desire
for referral. There was significant variability in provider rec-
ognition of patient referral desire. Recognition is defined as
the provider indicating that a referral was discussed when
the patient marked a definite or possible desire for referral.
Provider recognition improved significantly (
the patient had more than one referral desire, if the patient
or a family member was a health care worker and when the
patient noted a definite desire versus a possible desire for re-
ferral. Patients were more likely (
.05) to initiate a referral
discussion when they had seen the PCP previously and had
more than one referral desire. Of patient-initiated referral re-
quests, 14% were considered “not indicated” by PCPs. Satis-
faction with care did not differ in patients with a referral de-
sire that were referred and those that were nor referred.
These PCPs frequently failed to explicitly
recognize patients’ referral desires. Patients were more likely
to initiate discussions of a referral desire when they saw
their usual PCP and had more than a single referral desire.
managed care; patient satisfaction; referral
J GEN INTERN MED 2000;15:242–247.
he role of many primary care physicians (PCPs) has
altered in recent years in concert with market-driven
changes in health care. For example, in many managed
care systems, PCPs act as gatekeepers responsible for au-
thorizing access to specialty, emergency, and hospital
Authorization of access to specialty care has impor-
tant implications for health care utilization, expenditures,
and PCP function. On one hand, specialty referrals are of-
ten associated with significant economic implications and
do not always improve outcome.
On the other hand, pa-
tients have a relatively high degree of desire for specialist
and PCP failure to recognize these desires
have been associated with lack of clinical improvement
and dissatisfaction with care.
Despite the obvious importance of the managed care
referral process, there is remarkably little information on
patient–PCP interaction in this process.
vey revealed that many PCPs feel patients view managed
care PCPs as adversaries (66%), that managed care regu-
lation of specialist referrals negatively impacts patient
care (57%), and that managed care impairs physician–
patient communication (37%).
The present study was
therefore undertaken with several goals in mind. We pri-
marily wanted to determine the frequency with which
PCPs recognize patient referral desires and to see if we
could identify factors associated with increased PCP rec-
ognition of these desires in an academic managed care
setting. We also wanted to delineate circumstances in
which patients are more likely to initiate a referral discus-
sion with their PCP. Finally, we wanted to assess current
patient attitudes toward the referral process and the in-
fluence of the referral process on overall satisfaction with
care in the studied plan.
Patients, Providers, and Setting
The study population consisted of 856 consecutive
patients seen at the University Medical Group Practice
from September 1997 through December 1997. All pa-
tients were enrolled in a University of Colorado managed
care program (CU Gold), which utilizes PCPs as gatekeep-
ers for specialty access. Patients in this plan who visit the
study site include approximately 3,000 state employees,
health care workers, physicians, nurses, university-based
employees, and their family members. The University of
Colorado managed care plan, implemented in January
1996, represented a change from an open-access health
Received from the Department of Medicine, Division of General
Internal Medicine, University of Colorado Health Sciences
Center, Denver, Colo (GAA, CTL, JK, LMS, SNA, RJA).
Address correspondence and reprint requests to Dr. Albertson:
University of Colorado Health Sciences Center, 4200 E. 9th Ave.,
B180, Denver, CO 80262 (e-mail: Gail.Albertson@UCHSC.edu).
Volume 15, April 2000
care system. Patients enrolled in the study were present-
ing to the clinic for a new-patient, a return-patient, or an
episodic visit. Patients seeking referral for emergent or ur-
gent care, pregnancy, cancer, mental health, AIDS, an-
nual eye examination, or continuing therapy for any pre-
vious referral issued in the past 6 months were excluded.
The PCPs in this study consisted of 12 board-certified
general internists. Providers were aware of the research
study but were unaware of its objectives. All patients were
seen by a faculty physician without involvement of medi-
cal students or housestaff. The study was approved by
the Colorado Multiple Institutional Review Board.
Demographic data were obtained from the computer-
ized scheduling system used in the clinic. Patients com-
pleted a 1-page, self-administered previsit questionnaire
and were told their responses would be confidential and
not be made known to their physician. All patients were
asked to rate their overall health (poor, fair, good, very
good, or excellent). Patients were also asked to provide a
yes or no response to three questions: “Have you seen a
regular care provider at this office before?” “Is this ap-
pointment with your PCP?” “Are you or a family member a
health care worker?” In addition, patients were asked to
indicate the length of the relationship with their PCP
(never met, 1 visit, less than 1 year, or more than 1 year).
Patients were asked whether they needed a referral to a
specialist that same day (yes, possibly, or no). Patients in-
dicating either a definite or possible need for referral were
asked to provide a yes or no answer to whether they had
more than one referral desire. Patients indicating definite
or possible desire for specialist referral were also asked to
respond to two other statements regarding their referral
desire (“I am worried about this health concern” and “Be-
cause of this health concern, I am unable to function nor-
mally”) by indicating they strongly disagree, disagree, not
sure, agree, or strongly agree with the statements. A ran-
domly selected subset of patients who indicated either yes
or possible desire for specialist referral were contacted by
telephone 3 to 14 days after the visit. These patients were
asked to respond to three statements (“I am satisfied with
my medical care for my referral concern.” “I am satisfied
with the referral process for my referral desire.” “I think it
is a good idea to see my PCP before being referred to a
specialist”) on a strongly disagree, disagree, not sure,
agree and strongly agree scale.
The PCPs were asked to conduct patient visits during
the study period in their usual fashion. Providers were
asked to complete a brief, self-administered questionnaire
immediately after each patient visit. Providers were asked
to provide a yes or no answer to the question, “Did you
discuss a referral today?” If a referral was discussed dur-
ing the encounter, providers were asked to indicate (yes
or no) if a referral was made. When a referral was dis-
cussed, providers were asked to identify the main referral
concern and whether there was more than one concern.
Providers were also asked to indicate if any referral dis-
cussion was initiated by the patient or the provider. Pro-
viders were asked to provide a yes or no answer to the
question, “Was the referral indicated?” For referrals felt to
be not indicated, PCPs were asked to indicate why they
felt the referral was not indicated from a menu of possibil-
ities (PCP comfortable treating this condition; patient
seeks additional reassurance that is not indicated; re-
quested test or treatment not indicated; patient desires to
use all benefits of the plan; other). This menu of possibili-
ties was derived after discussion with several experienced
PCPs. Both patient and provider questionnaires were re-
viewed with a professional survey consultant and exten-
sively pretested before implementation.
Statistical Analysis and Definitions
For questions related to overall health, a numerical
score was obtained by assigning a score of 1 for poor; 2,
fair; 3, good; 4, very good; or 5, excellent. For statements
regarding worry, ability to function normally regarding the
referral desire, and opinion regarding satisfaction with
care, a score was assigned of 1 for strongly disagree; 2,
disagree; 3, not sure; 4, agree; or 5, strongly agree. For this
study, we defined that physician recognition of a patient
referral desire occurred when the physician noted that a
referral was discussed when patients indicated a desire
(yes or possible) for a referral. Analyses were performed on
SPSS-PC version 4.2 (SPSS Inc., Chicago, Ill). Initially,
continuous variables were tested using
tests and analysis
of variance where appropriate, and categorical variables
were compared using
tests. Because a relatively small
12) of physicians were involved in this study,
we did further analyses. In order to account for possible
clustering by physician, a mixed effects linear model was
used. These analyses were conducted using the Proc Mixed
procedure in the SAS (version 6.12) statistical package. The
values for the variables in Tables 1–3 are expressed in
terms of this mixed linear model. A
.05 is consid-
Frequency of Patient Referral Desire and
Complete data on 822 of 856 patients were available
on patient referral desire and physicians noting whether a
referral was discussed. The frequency of patient referral
desire and of PCP explicit referral discussion in these 822
visits is shown in Figure 1. The PCPs did not explicitly
discuss referral with 59% of patients indicating a possible
Albertson et al., Patients’ Referral Desires
referral desire or with 29% of those indicating a definite
PCP Recognition of Patient Referral Desire
Individual provider recognition of a patient referral de-
sire (patients answering yes or possibly) varied significantly
.02) within this group practice and is depicted in Figure
2. We were unable to find a significant relation between PCP
age, gender, years in practice, clinic workload, and PCP rec-
ognition of patient referral concern (data not shown).
Table 1 compares selected visit and patient charac-
teristics for patients with a referral desire in whom PCPs
did and did not explicitly recognize this desire. Providers
were significantly more likely to recognize a referral desire
in patients who were health care workers or family mem-
bers, in patients who had more than one referral desire,
and in patients who had indicated a definite desire versus
a possible desire for referral. No differences in patient
self-rating of overall health, worry regarding the referral
desire, or self-reported lower functional status related to
their referral concern were present when the PCP did rec-
ognize and did not recognize a referral desire. Duration of
patient-provider relationship and seeing the PCP for the
referral concern did not improve provider recognition of a
Comparison of Patient-Initiated and PCP-Initiated
Table 2 compares several variables when either the
patient or the PCP initiated a referral discussion. This set
of data refers to the 239 patients with whom a referral
discussion was held. Of these 239 patients, complete data
were available on 224. Patients were significantly more
likely to have initiated the referral discussion when they
had seen the PCP previously and had more than one re-
ferral desire. There was a trend for patient initiation of the
referral discussion when the patient had known the PCP
for more than a year (
.08 by bivariate analysis,
.113 by cluster analysis). Patient self-rating of overall
health and degree of worry regarding the referral desire
did not differ significantly when patient and PCP initiation
of a referral discussion were compared. The frequency
with which a referral was made was comparable with pa-
tient-initiated and PCP-initiated referral discussions.
PCP-Judged Nonindicated Referrals
Overall, 17.4% of referrals that were made were
judged by the involved PCPs to be “not indicated.” The in-
volved PCP felt that 13.4% of referrals that were made for
patient-initiated referrals were not indicated, while 4.0%
of PCP-initiated referrals were not indicated (
The main reason PCPs felt that referrals were not indi-
cated was that they felt comfortable treating the condition
(68.2%). Less often cited reasons included patient sought
additional reassurance that was not felt by the PCP to be
necessary (13.6%), miscellaneous (13.6%), and patient
desired a test or procedure not felt by the PCP to be indi-
cated (4.6%). There were no significant differences in pa-
tient self-rating of overall health, concern regarding the
referral issue, or functional status relative to the referral
issue when referrals judged to be indicated and nonindi-
cated by the PCP were compared (data not shown).
Patient Satisfaction with the Referral Process
Of the 351 patients indicating a referral desire (yes or
possibly), 150 were randomly selected for a follow-up tele-
phone survey. Of these 150 patients, 111 could be con-
tacted. Of these 111 patients, 80 were referred and 31
were not. Overall satisfaction with care and the referral
Figure 1. Patient referral desire and physician provider recog-
nition of these referral desires.
Figure 2. Individual physician provider recognition of patient
referral desire. The percentage of patients with a referral de-
sire that was recognized by individual physicians is on the ver-
tical axis. The solid bars represent individual providers. There
was significant interindividual physician variability in recogni-
tion of patient referral desire.
Volume 15, April 2000
process for these patients with a referral desire that were
referred and that were not referred are compared in Table
3. Patients who were not referred showed no significant
differences in their overall satisfaction with care or satis-
faction with the referral process. No significant difference
between referred and not-referred patients was found to
the statement, “I think it is a good idea to see my PCP be-
fore being referred to a specialist.” The mean scores for
both referred and nonreferred patients were, however, sig-
nificantly lower (
.025) when the responses as to
whether it is a good idea to see a PCP before specialist re-
ferral were compared with the rating of overall satisfac-
tion with care.
The present study demonstrates that general inter-
nist PCPs did not explicitly discuss a specialist referral in
29% to 59% of patients that answered either “yes” or
“possibly” to the query, “Do you need a referral to a spe-
cialist today?” Although these results may seem surpris-
ing, recent studies by Kravitz et al. indicate a very high
overall frequency of unmet patient desires for ambulatory
Also, analyses of data from two studies, not
designed specifically to examine the referral process, re-
veal a frequency of unmet patient desires for specialist re-
ferral of 24% and 58%, respectively.
present and previous studies indicate significant patient
desires for referral to a specialist that may not be directly
addressed in general medical ambulatory settings. More-
over, the study of Marple et al. found that a residual de-
sire for subspecialty referral was a powerful, independent
correlate of lack of patient satisfaction 2 weeks following
the ambulatory encounter (odds ratio, 2.9; 95% confidence
interval, 1.5 to 5.4,
Previous studies have emphasized high variability in
provider practice patterns including specialist referral
Given this variability, it is perhaps not sur-
prising that we found significant PCP variability with re-
gard to explicit recognition of patient referral desires. This
variability ranged from an average recognition of 68% to
24% of patient referral desires for individual PCPs. We
could not account for this variability by PCP age, gender,
years in practice or clinic workload. Three patient factors,
being a health care worker, having a definite referral de-
sire versus a possible referral desire, and having more
than a single referral desire, were the only factors that we
could ascertain to be associated with increased PCP rec-
ognition of patients’ referral desire. Given data on the fre-
quency with which patients desire specialist referral and
the implication of not addressing such desires,
PCP awareness and further studies to directly address
how to improve PCP recognition of patients’ referral de-
sires are warranted.
Inasmuch as PCPs frequently fail to recognize patients’
Table 1. Characteristics of Patients with a Referral Desire with Whom Primary Care Providers (PCPs) Did and
Did Not Discuss the Referral Concern
Patient Referral Desire
by PCP (
Appointment with usual PCP, % 67.2 68.7 .64
1 y, % 50.3 49.4 .99
Health care worker or family member, % 47.5 58.8 .025
More than one referral concern, % 20.5 33.5 .004
Patient noted “possible” referral need, % 56 44 NS
Patient noted “definite” referral need, % 27 73 .0001
Patient self-rating of overall health, mean
Patient self-rating of worry regarding
referral concern, mean
Patient self-rating of functional status
related to referral concern, mean
NS indicates not significant.
Table 2. Comparison of Patient Initiated and Primary Care
Provider (PCP)–Initiated Referral Discussions
Seen PCP at this office
previously, % 71.6 53.5 .05
1 y, % 58.1 42.3 .11
More than one referral
concern, % 44.2 19.7 .003
Referral made, % 53.3 46.7 .58
Self-rating of health,
Self-rating of worry
Self-rating of functional
status related to referral
Albertson et al., Patients’ Referral Desires
referral desires, we examined patient-initiated referral dis-
cussions. Our results indicate that continuity of care and
familiarity with their PCP is a significant correlate of pa-
tient initiation of a referral discussion (Table 2). Interest-
ingly, these variables are not associated with enhanced
PCP recognition of a patient’s referral desire (Table 1).
Our results are of interest with regard to another as-
pect of the referral desires. We found that the PCPs studied
felt that a modest proportion (14%) of patient-initiated re-
ferral desires were not indicated. Another study has indi-
cated moderate variability in provider responses to patient
requests for costly, unindicated services.
Our recent stud-
ies have found that patient need for reassurance, having
previously seen a specialist for the same or a similar prob-
lem, and the belief that the PCP did not have the requisite
expertise to handle the issue are the main factors underly-
ing most patient-initiated referral requests.
standing of factors that motivate patients to request refer-
rals may serve to develop better strategies to handle patient
requests felt to be not indicated. However, handling such
situations in a cost-effective manner while maintaining pa-
tient satisfaction is an area in need of further study.
Another result of note is our finding of slightly posi-
tive feelings in our population about the gatekeeper
model. This result contrasts somewhat with results re-
ported in 1998 from Israel.
Our results are based on a
relatively small number of patients, and further studies
are needed to better address this issue.
Some potential limitations of our study merit consid-
eration. It is possible that in some visits PCPs discussed
issues regarding the patient’s referral desire without ex-
plicitly discussing the need or lack thereof for referral.
Such a discussion would have been classified as PCP
“nonrecognition” of a patient referral desire in our study.
Thus, it is possible that our results overestimate PCP
nonrecognition of patient referral desires. Our data on
health care workers must be interpreted with caution be-
cause our definition may have resulted in a health care
worker population consisting of patients who have only
workplace contacts with health professionals or the
health care field. The expectations of the health care
workers, as a group, might become less distinguishable
from those of the other subjects if a more rigid definition
were applied. Our study did not explicitly differentiate pa-
tient desire from expectation in that we asked about
“need” for a referral. “Desire” refers to what patients want
before their PCP visit, and “expectation” refers to what pa-
tients feel they are likely to receive from their PCP. Expec-
tation rather than desire has been shown to impact pa-
Our study was undertaken in a highly
selected setting. Thus, our results may not be generaliz-
able. Moreover, physicians, while not knowledgeable about
the specific objectives of our study, could surmise that it
dealt with patient referral issues, and this knowledge
could have influenced our results. Our study included a
relatively modest number of PCPs. However, we utilized a
mixed effects linear model to account for any effects of
cluster randomization due to the modest number of phy-
sicians included in our study. We did not study telephone
referral requests, which in some systems are an impor-
tant avenue for specialist access. Our study also did not
evaluate the concordance of referral concerns between pa-
tients and providers when providers recognized patient re-
ferral concerns. Discordance between the patient agenda
and physician agenda is well recognized and has prompted
study of agenda-setting clinical tools.
In summary, our results demonstrate that these
PCPs frequently failed to explicitly discuss patients’ spe-
cialty referral desires. The PCPs were more likely to recog-
nize a referral desire when the patient was a health care
worker, expressed a definite need versus a possible need
for referral, and the patient had more than one referral
desire. Patients were more likely to initiate a referral dis-
cussion in a continuity of care setting in which they were
familiar with the PCP. Understanding factors that influ-
ence a patient’s request for referral and provider recogni-
tion of a referral desire may influence patient satisfaction.
This work was funded by University Hospital Board of Directors,
1. Franks P, Clancy CM, Nutting PA. Gatekeeping revisited–protect-
ing patients from overtreatment. N Engl J Med. 1992;327:424–9.
2. Forrest CB, Starfield B. The effect of first-contact with primary
care clinicians on ambulatory health expenditures. J Fam Pract.
3. Martin DP, Diehr P, Price KF, Richardson WC. Effect of a gate-
Table 3. Patients with a Referral Concern: Perceptions and Attitudes Toward Overall Care and the Referral Process
Variable Not Referred Referred
Patient rating of overall satisfaction with care, mean
Patient rating of satisfaction with the referral process, mean
Patient response to the statement, “It is a good idea to see my primary care
provider before specialist referral,” mean
⫾ 0.05 .51
Patient rating of strongly agree or agree with referral process, % 54.8 74.7 NS
*NS indicates not significant.
.05 versus patient rating of overall satisfaction with care.
JGIM Volume 15, April 2000 247
keeper plan on health services use and charges: a randomized
trial. Am J Public Health. 1989;79:1628–32.
4. Glenn JK, Lawler FK, Hoerl MS. Physician referrals in a competi-
tive environment: an estimate of the economic impact of a referral.
5. Greenfield S, Nelson E, Zubkoff M. Variations in resource utiliza-
tion among medical specialties and systems of care: results from
the Medical Outcomes Study. JAMA. 1992;267:1624–30.
6. Donohoe MT. Comparing generalist and specialty care: discrep-
ancies, deficiencies and excesses. Arch Intern Med. 1998;158:
7. Webb S, Lloyd M. Prescribing and referral in general practice: a
study of patients’ expectations and doctors’ actions. Br J Gen
8. Kravitz RL, Cope DW, Bhrany V, Leake B. Internal medicine pa-
tients’ expectations for care during office visits. J Gen Intern Med.
9. Hornberger J, Thom D, MaCurdy T. Effects of a self-administered
pre-visit questionnaire to enhance awareness of patients’ con-
cerns in primary care. J Gen Intern Med. 1997;12:597–606.
10. Kravitz RL, Callahan EJ, Azari R, Antonius D, Lewis CE. Assess-
ing patients’ expectations in ambulatory medical practice. J Gen
Intern Med. 1997;12:67–72.
11. Marple RL, Kroenke K, Lucey CR, Wilder J, Lucas CA. Concerns
and expectations in patients presenting with physical complaints.
Arch Intern Med. 1997;157:1482–8.
12. Tabenkin H, Revital G, Brammli S, Shvartzman P. Views of direct
access to specialists. JAMA. 1998;279:1943–8.
13. Rosenthal TC, Riemenschneider TA, Feather J. Preserving the pa-
tient referral process in the managed care environment. Am J
14. Borowsky SJ. What do we really need to know about consultation
and referral? J Gen Intern Med. 1998;13:497–8.
15. Kravitz RL, Callahan EJ, Paternitic D, Antonius D, Dunham M,
Lewis CE. Prevalence and sources of patients’ unmet expectations
for care. Ann Intern Med. 1996;125:730–7.
16. Kerr EA, Hays RD, Lee ML, Siu AL. Does dissatisfaction with ac-
cess to specialists affect the desire to leave a health plan? J Gen
Intern Med. 1996;11:77. Abstract.
17. Gallagher TH, Lo B, Chesney M, Christensen K. How do physi-
cians respond to patients’ requests for costly, unindicated ser-
vices? J Gen Intern Med. 1997;12:663–8.
18. Feldman DS, Novack DH, Gracely E. Effects of managed care on
physician-patient relationships, quality of care and the ethical
practice of medicine. Arch Intern Med. 1998;158:1626–32.
19. McLeod P, Tamblyn R, Gayton D, et al. Use of standardized pa-
tients to assess between physician variation in resource utiliza-
tion. JAMA. 1997;278:1164–8.
20. Calman NS, Hyman RB, Hecht W. Variability in consultation rates
and practitioner level of diagnostic certainty. J Fam Pract. 1992;
21. Franks P, Clancy CM. Referrals of adult patients from primary
care: demographic disparities and their relationship to HMO in-
surance. J Fam Pract. 1997;45:47–53.
22. Albertson G, Lin CT, Swaney R, Anderson S, Anderson RJ. Pa-
tient-provider interactions in the subspecialty referral process in a
gatekeeper model academic managed care plan. J Gen Intern
Med. 1998;13(suppl):103. Abstract.
JGIM Website — Visit us online today!
Please visit the JGIM World-Wide Website: