40 JANUARY 2019 www.ajmc.com
elemedicine visits have been used in the United States to
enhance access to healthcare, most notably for people who
live in remote and underserved areas.1,2 The increasing avail-
ability of personal technology (89% have internet access, 77% are
online daily3) oers patients and clinicians the opportunity to utilize
real-time virtual communication to enhance access for patients
when transportation challenges, schedules, or physical disability
make oce visits dicult in any geography.4 Although face-to-face
interactions may be preferred in some circumstances by patients or
clinicians, the convenience of accessing healthcare consultations
from the home or oce may save lost time at home or work, travel
time, and missed and rescheduled appointments.
perceived relative value of dierent modes of healthcare services may
help to shape the use of virtual or remote healthcare technologies.
Eective population health management is a balancing act that
requires consideration of patient needs and preferences for more
exible and timely access to consultation, accountability to payers
by managing high costs, and understanding how to leverage new
technologies.12,13 System learning that demonstrates the value of
dierent types of “visits” for the system and the patient is essential.
We initiated the Massachusetts General Hospital (MGH) TeleHealth
program in 2012, oering a range of telemedicine services in 15 clinical
departments. This paper describes experiences with virtual video
visits (VVVs): 2-way audiovisual synchronous videoconferencing
between the MGH clinician and patient. The research reported
here focuses on the patient and clinician experience of a VVV in
a full year of operation to understand its value and comparative
experience with VVVs and oce visits.
Study Setting and Telemedicine
Clinicians in 5 specialties (psychiatry, neurology, cardiology,
oncology, and primary care) were trained in how to provide a
VVV throughout the rst year and on a rolling basis. Oncology
and primary care VVVs were not implemented until late in the
data collection period. Clinicians oered VVVs as an option to
Patient and Clinician Experiences With
Telehealth for Patient Follow-up Care
Karen Donelan, ScD, EdM; Esteban A. Barreto, MA; Sarah Sossong, MPH; Carie Michael, SM; Juan J. Estrada, MSc, MBA;
Adam B. Cohen, MD; Janet Wozniak, MD; and Lee H. Schwamm, MD
OBJECTIVES: The increasing and widespread availability
of personal technology offers patients and clinicians the
opportunity to utilize real-time virtual communication
to enhance access to health services. Understanding the
perceived value of different modes of care may help to shape
the future use of technology.
STUDY DESIGN: Cross-sectional surveys of patients and
clinicians participating in telehealth virtual video visits
(VVVs) in an academic health system.
METHODS: We administered surveys to 426 unique
established patients and 74 attending physicians in our
hospital to measure perceptions of the comparative
experience of VVVs and office visits; 254 patients and
61physicians completed the surveys.
RESULTS: When comparing VVVs and office visits, 62.6%
of patients and 59.0% of clinicians reported no difference
in“the overall quality of the visit.” VVVs were vastly
preferred to office visits by patients for convenience and
travel time. A majority (52.5%) of clinicians reported higher
efficiency of a VVV appointment.
CONCLUSIONS: For established patients, VVVs may
provide effective follow-up and enhanced convenience when
compared with traditional office visits.
Am J Manag Care. 2019;25(1):40-44
FROM THE FIELD
VOL. 25, NO. 1 41
THE AMERICAN JOURNAL OF MANAGED CARE®
Virtual vs In-Person Office Visits
established patients based on their professional
assessment of the suitability of the mode of
visit for the individual patient’s situation (eg,
patient could communicate eectively in this
mode, physical examination was not critical
at the visit). Clinicians were compensated by
MGH for conducting these VVVs because they
were not covered by payers in Massachusetts.
In advance of the VVV, participating patients
received education, instruction, and phone-
based technology support and testing for
installation of the visit software. Patients were
not charged insurance co-payments for the visit.
The data reported here come from surveys of patients and clinicians
in the MGH TeleHealth program. This study was reviewed and
approved by the Partners Health Care Oce of Human Research.
The surveys we employed were developed by the MGH Center for
TeleHealth leadership and Mongan Institute Health Policy Center
research team, including experts in survey and health services
research, telemedicine, clinical medicine, and health management.
We included selected patient experience measures developed by
the Consumer Assessment of Healthcare Providers and Systems
(CAHPS)15 andaugmented with items developed for this mode of
visit. Surveys were pretested with patients and rened. Key domains
included technology and communication quality, visit quality and
experience, patient time and costs, and willingness to pay for a VVV.
Of 426 eligible patients, 254 (60%) completed surveys using a
secure web tool. Eligible patients had at least 1 VVV during the
accrual period and at least 1 oce visit in the 6-month period
prior. Initial recruitment was by email request within 1 week of
the VVV. Patients whose email addresses were not functional were
contacted by postal mail or telephone. Persistent nonresponders to
the survey were oered a $10 incentive after 4 weeks of attempts
without reaching the patient. Patients younger than 18 years were
not directly contacted; rather, surveys were sent to their parents.
Of 74 eligible clinicians (physicians, nurse practitioners, psycholo-
gists) who provided at least 1 VVV to program patients during the
study period, 61 (82%) completed surveys online, with recruitment
by email. An Amazon.com gift certicate valued at $50 was oered
to each physician as an honorarium for participation.
The results reported here are descriptive; subgroup comparisons
within patient and clinician populations use χ
or t test comparisons
as indicated, and analyses comparing patient and clinician responses
utilize 2-sample t tests of the dierence in proportions. All analyses
were conducted using SPSS version 23 (IBM Corp; Armonk, New York).
Patient characteristics are shown in the eAppendix Table (eAppendix
available at ajmc.com). VVV patients were diverse in age and gender
and were predominantly white and non-Hispanic, consistent with
patient demographics in our system. Patients receiving behavioral
health care are overrepresented due to the rapid uptake of VVV
by those clinicians and patients. No signicant dierences were
measured in characteristics of respondents and nonrespondents.
Clinician respondents were psychologists and psychiatrists
(34%), neurologists (38%), cardiologists (10%), oncologists (2%),
andprimary care clinicians (16%).
Patient Experience of VVVs
Patient experience with a VVV was measured in multiple items.
Responses are shown in the Table by specialty, comparing neurology
and cardiology patient responses with those of the psychiatry patients
(referent). We used 4 measures from the CAHPS Clinician Group visit
to assess patient experience with eciency and communication
during VVVs. Overall, 80% or more respondents answered “yes,
denitely” to these items; 82.3% responded “yes, denitely” to
whether they would recommend VVV to their family and friends.
On a scale of 0 to 10, where 0 is the worst possible visit and 10 is
the best visit, 68.5% rated the visit a 9 or 10.
We asked patients to consider whether the care provided during
their recent VVV could have been provided in another way and
still met their needs, asking “Do you think the health issues you
discussed with clinicians today could just as easily have been
addressed by [insert visit mode]?” Nearly 90% of patients agreed
that theirissues could also have been addressed in an oce visit
(denitely, 70.8%; somewhat, 17.8%) and nearly 60% by telephone
calls (denitely, 23.2%; somewhat, 36.7%). The results for secure
email (denitely, 7.2%; somewhat, 23.5%) and text messaging
(denitely, 3.6%; somewhat, 12.7%) suggest that they are less often
appropriate substitutes (data not shown).
Patient and Clinician Comparisons of Office Visits
The Figure shows patient and clinician responses to a series of
comparative questions about oce visits and VVVs that were asked
of both groups. Expanded data are shown in eAppendix Figures 1
and 2—one with patient data, one with clinician data—and include
Telemedicine visits have been used to provide healthcare access to more remote populations.
In a busy health system, telehealth visits were incorporated for established patients to allow
patient–clinician interaction in a new, more convenient mode. In the first full year of patient
visits in this new mode, we found that:
Patients rated these visits highly and the majority would recommend them to family and friends.
› Using standard measures of patient experience, most patients and clinicians perceived no
loss of communication in virtual video visits compared with office visits, although clinicians
were somewhat more likely to see loss of personal connection as a problem.
Patients perceived considerable added convenience, saved travel time, and expressed
willingness to pay co-payments for this visit option.
› Virtual visits are an important and useful option in clinical care.
42 JANUARY 2019 www.ajmc.com
TRENDS FROM THE FIELD
several items that were directed only to one
group or the other.
Most patients (62.6%) and clinicians (59.0%)
reported “no dierence” between virtual and
oce visits on “the overall quality of the visit.”
When rating “the personal connection felt
during the visit,” 32.7% of patients and 45.9%
of clinicians reported that the “oce visit is
better,” but more than half of the respondents
(patients, 59.1%; clinicians, 50.8%) said that
there was “no dierence.”
Patient Willingness to Pay for VVV
eAppendix Figure 3 shows data on patient
willingness to pay for VVV out of pocket. Patients
were not initially charged co-payments. However,
the majority of patients expressed a willingness
to pay a co-payment of up to $50. Among those
willing to bear the full cost of the VVV, more than
one-third had no current co-payment and all
had private (vs public) insurance. We conducted
bivariate analyses of the willingness to make
co-payments for VVV by both self-reported
travel time and cost of attending oce visits.
Among those who traveled more than 90 minutes
to an oce visit, 51.5% indicated they would
pay a co-payment of more than $50 for a VVV
compared with 30.4% of those who traveled less
than 30 minutes. Among patients who spent
$25 or more on travel to attend an oce visit,
73.2% would pay a co-payment of $26 to $50
for a VVV and 97.6% would pay a co-payment
of $10 to $25 (travel data not shown).
Increased interest in new strategies for managing
population health and episodic specialty care,
coupled with the widespread availability of
communications technologies, have encour-
aged the exploration of the appropriate roles
of dierent modes of clinical encounters or
Our data—gathered from patients and
clinicians during and following an initial full
year of experience with the MGH Center for
TeleHealth’s VVV implementation—show a high
degree of patient and clinician satisfaction, as
measured by both ratings of overall visit quality
and willingness to recommend the visits.
VVVs are perceived by the majority of patients
as the same as or better than oce visits in
convenience and cost, at the same level of
TABLE. Patient Experience With Virtual Video Visits
(N = 254)
(n = 113)
(n = 92)
(n = 30)
Thinking about your most recent virtual
video visit, please tell us how much you
agree with the following items:
I saw my clinician within 15 minutes of
my appointment time. Reference P = .609 P = .284
Yes, definitely agree 88.2% 90.3% 88.0% 83.3%
Yes, somewhat agree 6.3% 6.2% 4.3% 10.0%
No 3.5% 1.8% 4.3% 6.7%
Not answered 2.0% 1.8% 3.3% 0.0%
My clinician explained things in a way
that was easy to understand. Reference P = .046 P = .791
Yes, definitely agree 92.9% 95.6% 88.0% 96.7%
Yes, somewhat agree 3.9% 1.8% 7.6% 3.3%
No 0.8% 0.0% 1.1% 0.0%
Not answered 2.0% 2.7% 3.3% 0.0%
My clinician listened carefully to me. Reference P = .026 P = .791
Yes, definitely agree 92.5% 95.6% 87.0% 96.7%
Yes, somewhat agree 4.7% 1.8% 9.8% 3.3%
No 0.8% 0.0% 1.1% 0.0%
Not answered 2.0% 2.7% 2.2% 0.0%
My clinician spent enough time with me. Reference P = .002 P = .656
Yes, definitely agree 98.8% 94.7% 80.4% 96.7%
Yes, somewhat agree 7.1% 2.7% 15.2% 3.3%
No 1.2% 0.9% 1.1% 0.0%
Not answered 2.0% 1.8% 3.3% 0.0%
On a scale from 0 to 10, where 10 is your
BEST visit and 0 is your WORST possible visit,
how would you rate your virtual video visit?
Reference P = .771 P = .168
9-10 68.5% 66.3% 67.4% 80.0%
7-8 25.2% 27.5% 23.9% 20.0%
≤6 5.1% 5.3% 6.5% 0.0%
Would you recommend this clinician
toyour family and friends? Reference P = .034 P = .544
Yes, definitely agree 89.0% 93.8% 84.8% 96.7%
Yes, somewhat agree 7.5% 5.3% 8.7% 3.3%
No 1.6% 0.0% 2.2% 0.0%
Not answered 2.0% 0.9% 4.3% 0.0%
Would you recommend a virtual visit to
your family and friends? Reference P = .416 P = .076
Yes, definitely agree 82.3% 86.7% 82.6% 66.7%
Yes, somewhat agree 13.4% 11.5% 10.9% 33.3%
No 0.8% 0.0% 2.2% 0.0%
Not answered 3.5% 1.8% 4.3% 0.0%
Source: Data from the TeleHealth Patient Survey, February 2014-March 2015.
VOL. 25, NO. 1 43
THE AMERICAN JOURNAL OF MANAGED CARE®
Virtual vs In-Person Office Visits
quality and personal connection. Patients appear to value the
face-to-face interactive nature of VVVs; they perceive oce or
telephone encounters as the main alternatives to this new type
of visit, rather than email or structured questionnaires, likely
because they maintain a real-time personal connection. Even
though VVVs represent a great convenience, our data reinforce that
they are unlikely to be a useful substitute for an in-oce visit in
some clinical situations—for example, those with more complexity
and need for physical examination or observation. More research
is needed to study the association of the care delivery modality
(in-person vs telehealth) with total cost and clinical outcomes and to
understand settings in which telehealth aects healthcare value.
Theregulatory, administrative, and legal environment related to
the conduct of telehealth visits is in constant ux; delivery of care
to established patients across state lines adds further complexity.
The Chronic Care Act of 2017, enacted in 2018, takes some small
steps toward relaxing reimbursement and regulatory restrictions
in selected diseases and insurance markets; hopefully, this trend
Patient participants were selected for these visits by their clinicians
based on their suitability, as determined during prior oce visits
as established patients. This was not a controlled study. These data
were gathered in the rst full year of VVV implementation in our
system and are primarily from VVVs for a specialty, so they very likely
do not fully reect the challenges or opportunities of using VVVs
in primary care settings. Surveys are subject to sources of error and
bias; we attempted to minimize these with high-quality methods
and response rate enhancement methods. Lastly, Massachusetts
is not among the 31states that now routinely require third-party
reimbursement for telehealth visits; our health system chose to
support reimbursement to clinicians. It is unknown if the level of
reimbursement created positive or negative incentives. Further study
is needed, including improved models to compensate clinicians, to
measure and dene the appropriate mix of virtual and oce visits,
and to understand the role of other modes of care.
With the inexorable adoption of digital oerings to meet many of the
needs of today’s consumers, it is likely that telehealth will increasingly
be adopted over the next several years. Our data suggest that initial
experiences for patients and clinicians were positive and that, for
most encounters, these VVVs are just as clinically eective and less
expensive for both patient and provider compared with in-person
visits. The fears of distracted, overwhelmed providers and a loss of
human connection between patient and provider have been raised
repeatedly with the rising use of computers in the doctor’s oce.
Interestingly, this issue was not a central concern to participants in
our VVV program, perhaps due to the use of VVVs with established
patients. These visits are not just replacements for in-oce visits; they
hold the possibility of new avenues for care delivery, more frequent
but shorter encounters, and opportunity for earlier intervention.
Further studies are needed to test dierent provider compensation
models to measure and dene the appropriate mix of virtual, oce,
and other modes of care and to establish appropriateness criteria for
the use of telehealth encounters. These issues are all important in
the further use of this technology as part of eective and ecient
patient-centered care and population management.
FIGURE. Patient and Clinician Comparisons of Virtual and Office Visits
0% 20% 40% 60% 80%
Length of visit required to assess and treat patient (provider)
Amount of time I spend with my clinician (patient)
Comfort I feel sharing personal or private information (patient)
Confidence health concern can be taken care of (patient)
The ability to see a physical problem (provider)
Ability to show clinician a physical problem (patient)
Personal connection I feel with the patient (provider)
Personal connection I feel with clinician during the visit (patient)
Overall quality of the visit (provider)
Overall quality of the visit (patient)
Virtual visit is better No difference Office visit is better Does not apply to me/not answered
44 JANUARY 2019 www.ajmc.com
TRENDS FROM THE FIELD
should further the aims of improving the quality of healthcare
and addressing the Institute of Medicine domains of quality, with
special attention to overcoming existing barriers in access to care,
including the burden of time and nancial costs that patients and
families bear in attending traditional oce visits. n
Author Aliations: Mongan Institute Health Policy Center (KD, EAB, CM),
MGH Center for TeleHealth (JJE, LHS), Department of Neurology (JJE, LHS), and
Department of Psychiatry (JW), Massachusetts General Hospital, Boston, MA; Flare
Capital Partners (SS), Boston, MA; Applied Physics Laboratory, Johns Hopkins
University (ABC), Laurel, MD; Department of Neurology, Johns Hopkins Hospital
and Health System (ABC), Baltimore, MD.
Source of Funding: Massachusetts General Hospital Institutional Funds.
Author Disclosures: Dr Cohen is employed with and provided expert testimony
for Johns Hopkins University Applied Physics Laboratory, which develops health
technology, including telemedicine-related technology. Dr Wozniak’s spouse
has received royalties from UpToDate; consultation fees from Advance Medical,
FlexPharma, Merck, Otsuka, and Gerson Lehman Group; and research support
from RLS Foundation. Dr Schwamm has received a Patient-Centered Outcomes
Research Institute grant on tele–palliative care and is the director of the MGH
Center for TeleHealth. The remaining authors report no relationship or nancial
interest with any entity that would pose a conict of interest with the subject
matter of this article.
Authorship Information: Concept and design (KD, SS, CM, JJE, ABC, JW, LHS);
acquisition of data (KD, EAB, SS, CM, JJE, ABC, JW, LHS); analysis and interpretation
of data (KD, EAB, SS, CM, JJE, ABC, JW, LHS); drafting of the manuscript (KD, EAB,
CM, LHS); critical revision of the manuscript for important intellectual content
(KD, SS, CM, JJE, ABC, JW, LHS); statistical analysis (KD, EAB, CM, JJE, ABC, JW, LHS);
provision of patients or study materials (KD, SS, CM, JJE, ABC, JW, LHS); obtaining
funding (KD, SS, CM, JJE, ABC, JW, LHS); administrative, technical, or logistic support
(KD, SS, CM, JJE, ABC, JW, LHS); and supervision (KD, SS, CM, JJE, ABC, JW, LHS).
Address Correspondence to: Karen Donelan, ScD, EdM, Mongan Institute Health
Policy Center, Massachusetts General Hospital, 100 Cambridge St, Ste 1600, Boston,
MA 02114. Email: email@example.com.
1. Güler NF, Übeyli ED. Theory and applications of telemedicine. J Med Syst. 2002;26(3):199-220.
2. Mehrotra A, Jena AB, Busch AB, Souza J, Uscher-Pines L, Landon BE. Utilization of telemedicine among rural
Medicare beneficiaries. JAMA. 2016;315(18):2015-2016. doi: 10.1001/jama.2016.2186.
3. Perrin A, Jiang J. About a quarter of U.S. adults say they are ‘almost constantly’ online. Pew Research Center
constantly. Published March 14, 2018. Accessed July 12, 2018.
4. Perrin A, Duggan M. Americans’ internet access: 2000-2015. Pew Research Center website. pewinternet.org/
2015/06/26/americans-internet-access-2000-2015. Published June 26, 2015. Accessed December 21, 2018.
5. Agha Z, Schapira RM, Laud PW, McNutt G, Roter DL. Patient satisfaction with physician-patient communica-
tion during telemedicine. Telemed J E Health. 2009;15(9):830-839. doi: 10.1089/tmj.2009.0030.
6. Perle JG, Langsam LC, Randel A, et al. Attitudes toward psychological telehealth: current and future clinical
psychologists’ opinions of internet-based interventions. J Clin Psychol. 2013;69(1):100-113. doi: 10.1002/jclp.21912.
7. Polinski JM, Barker T, Gagliano N, Sussman A, Brennan TA, Shrank WH. Patients’ satisfaction with and
preference for telehealth visits. J Gen Intern Med. 2015;31(3):269-275. doi: 10.1007/s11606-015-3489-x.
8. Viers BR, Pruthi S, Rivera ME, et al. Are patients willing to engage in telemedicine for their care: a survey
of preuse perceptions and acceptance of remote video visits in a urological patient population. Urology.
2015;85(6):1233-1240. doi: 10.1016/j.urology.2014.12.064.
9. LeRouge CM, Garfield MJ, Hevner AR. Patient perspectives of telemedicine quality. Patient Prefer Adherence.
2015;9:25-40. doi: 10.2147/PPA.S67506.
10. LeRouge C, Hevner A, Collins R, Garfield M, Law D. Telemedicine encounter quality: comparing patient and
provider perspectives of a socio-technical system. Presented at: 37th Annual Hawaii International Conference
on System Sciences; January 5-8, 2004; Big Island, HI. doi: 10.1109/HICSS.2004.1265375.
11. Weinstein RS, Lopez AM, Krupinski EA, et al. Integrating telemedicine and telehealth: putting it all together.
Stud Health Technol Inform. 2008;131:23-38.
12. Meyers D, Peikes D, Genevro J, et al. The roles of patient-centered medical homes and accountable care
organizations in coordinating patient care [AHRQ publication no. 11-M0005-EF]. Agency for Healthcare
Research and Quality website. pcmh.ahrq.gov/sites/default/files/attachments/Roles%20of%20PCMHs%20
And%20ACOs%20in%20Coordinating%20Patient%20Care.pdf. Published December 2010. Accessed
December 21, 2018.
13. Bao Y, Casalino LP, Pincus HA. Behavioral health and health care reform models: patient-centered
medical home, health home, and accountable care organization. J Behav Health Serv Res. 2013;40(1):121-132.
14. Adler-Milstein J, Kvedar J, Bates DW. Telehealth among US hospitals: several factors, including
state reimbursement and licensure policies, influence adoption. Health Aff (Millwood). 2014;33(2):207-215.
15. An overview of version 3.0 of the CAHPS Clinician & Group Survey. Agency for Healthcare Research and
Quality website. ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance/cg/about/cg_3-0_overview.pdf.
Published June 17, 2015. Accessed December 21, 2018.
16. Kvedar J, Coye MJ, Everett W. Connected health: a review of technologies and strategies to improve patient
care with telemedicine and telehealth. Health Aff (Millwood). 2014;33(2):194-199.
17. Kim YS. Telemedicine in the USA with focus on clinical applications and issues. Yonsei Med J.
2004;45(5):761-775. doi: 10.3349/ymj.2004.45.5.761.
18. Health policy brief: telehealth parity laws. Health Affairs Blog website. healthaffairs.org/do/10.1377/
hblog20160815.056155/full. Published August 15, 2016. Accessed December 21, 2018.
19. Schwamm LH. Telehealth: seven strategies to successfully implement disruptive technology and transform
health care. Health Aff (Millwood). 2014;33(2):200-206. doi: 10.1377/hlthaff.2013.1021.
Visit ajmc.com/link/3588 to download PDF and eAppendix
eAppendix Table. Respondent Characteristics
(N = 254)
Service or department
Device used for virtual video visit
How long does it take you to travel to MGH for office visits?
eAppendix Figure 1. TeleHealth Patients’ Perceptions of Virtual Video Visits vs Office Visits
Source: Data from MGH TeleHealth Patient Survey.
0% 20% 40% 60% 80 % 100%
Overall quality of the visit
Amount of time I spend with my clinician
Personal connection I feel with clinician during the visit
Amount of time I wait for my clinician
Cost of the visit
Finding a convenient time for visit
Travel time to the visit
Virtual visit is better No di fference Office visit is better Does not apply to me Not answer ed
eAppendix Figure 2. TeleHealth Clinicians’ Perception of Virtual Video Visits vs Office Visits
Source: Data from MGH TeleHealth Clinician Survey.
0% 20% 40 % 60% 80% 100%
Overall quality of the visit
Personal connection I feel with the patient
The ability to see a physical problem
Ease of ordering tests for the patient
Ease of accessing patient records during the visit
Ease of scheduling followup visits
Workflow outside of the visit (before and after)
Length of visit required to assess and treat patient
Efficiency of the visit
Timely patient access to followup appointments
Virtual visit is bette r No difference Office visit is better Does not apply to me
eAppendix Figure 3. TeleHealth Patients’ Willingness to Pay for Virtual Video Visit
Source: Data from MGH TeleHealth Patient Survey.
020 40 60 80 100
Full cost of visit
Yes, Definitely Yes, Somewhat No Don't know/Not answered