Telemental Health Evaluations Enhance Access and Efficiency
in a Critical Access Hospital Emergency Department
Erik P. Southard, DNP, FNP-BC,1Jonathan D. Neufeld, PhD,2
and Stephanie Laws, MSN3
1Department of Advanced Practice Nursing, Indiana State
University, Terre Haute, Indiana.
2Upper Midwest Telehealth Resource Center, Terre Haute, Indiana.
3Richard G. Lugar Center for Rural Health, Terre Haute, Indiana.
Introduction: Mentally ill patients in crisis presenting to critical
access hospital emergency rooms often face exorbitant wait times to
be evaluated by a trained mental health provider. Patients may be
discharged from the hospital before receiving an evaluation or
boarded in a hospital bed for observation, reducing quality and in-
creasing costs. This study examined the effectiveness of an emer-
gency telemental health evaluation service implemented in a rural
hospital emergency room. Materials and Methods: Retrospective
data collection was implemented to consider patients presenting to
the emergency room for 212 days prior to telemedicine interventions
and for 184 days after. The study compared measures of time to
treatment, length of stay (regardless of inpatient or outpatient sta-
tus), and door-to-consult time. Results: There were 24 patients seen
before telemedicine was implemented and 38 seen using tele-
medicine. All patients had a mental health evaluation ordered by a
physician and completed by a mental health specialist. Significant
reductions in all three time measures were observed. Mean and
median times to consult were reduced from 16.2h (standard devi-
ation=13.2h) and 14.2h, respectively, to 5.4h (standard devia-
tion=6.4h) and 2.6h. Similar reductions in length of stay and
door-to-consult times were observed. By t tests, use of telemedicine
was associated with a statistically significant reduction in all three
outcome measures. Conclusions: Telemedicine appears to be an
effective intervention for mentally ill patients by providing more
timely access to mental health evaluations in rural hospital emer-
Key words: telemedicine, telehealth, business administration/eco-
ental health services have been provided via tele-
medicine for more than 50 years, but patients in rural
areas continue to face numerous barriers to accessing
mental health treatment.1–3Patients presenting to
critical access hospital (CAH) emergency rooms (ERs) with mental
health conditions often do not receive timely access to mental
health evaluations. These patients may be unnecessarily admitted
for observation or discharged before a mental health professional is
able to evaluate them, and patients often leave without a clear
treatment plan in place.4These patients rarely engage in follow up
and may continue to access care via the ER, resulting in systemwide
According to a telephone survey of 422 CAHs completed by the
Maine Rural Health Research Center in 2005, CAH ERs nationwide
reported on average 99 ER visits per week.4The survey also indicated
that 6.5% of these patient encounters had a mental health disorder
listed as the primary or secondary diagnosis.4Of the communities
surveyed, 42.1% had no mental health care coverage available lo-
cally, and only 2.1% of the communities had inpatient psychiatric
care available. The study also found that the mean travel time to
psychiatric services was 52min (with a maximum time of 4h).4
There is a growing consensus among practitioners that services
delivered via telemedicine are effective for behavioral health di-
agnoses, even though empirical evidence pertaining specifically to
rural populations is slow to accumulate. In general, telemedicine’s
clinical utility and overall effectiveness are largely recognized,5
and at least 27 different specialties or subspecialties are being
provided via telemedicine technology.6
In 1998, Ruskin et al.7studied inter-rater reliability between face-
to-face evaluations and evaluations conducted via video teleconfer-
encing. The most common diagnoses of depression, bipolar disorder,
panic disorder, and alcohol dependence were studied. Inter-rater re-
of the delivery mechanism.7
O’Reilly et al.8completed a randomized controlled equivalence
trial of psychiatric consultations and short-term follow-up via face-
to-face and by telepsychiatry. Consultations delivered via video were
found to be as effective as consultations delivered face-to-face.
Global severity index scores decreased equal amounts in the face-to-
face and the telepsychiatry groups.8
De Las Cuevas et al.9completed a randomized controlled trial to
assess the clinical efficacy and beneficence of telepsychiatry versus
face-to-face intervention. In this study, telepsychiatry was found to
be an effective means of delivering mental health services to outpa-
tients in remote areas. Clinical efficacy of the video intervention was
found to be indistinguishable from that of face-to-face intervention.
Overall,patients showedsignificant improvements onmultipleglobal
index scores, indicating that there were clear clinical improvements
regardless of delivery mechanism.9
ª MARY ANN LIEBERT, INC. ? VOL. 20NO. 7 ? JULY 2014
TELEMEDICINE and e-HEALTH 1
Telepsychiatry has also been reported to have a high level of ac-
curacy and value as an assessment tool by Singh et al.10in 2007. In
this study, substantial agreement was found between face-to-face
assessment and telepsychiatry, and the intermethod agreement for
Cohen’s kappa was high. Regardless of how service was delivered,
results for this study revealed that diagnosis, risk assessment, and
nondrug and drug intervention were all greater than 0.76 with a
combined overall accuracy of >0.81.10
Two meta-analyses have been conducted on the delivery of mental
results from 14 studies encompassing 500 total cases, to compare the
accuracy of in-person psychiatric evaluation with psychiatric evalua-
tion delivered using live two-way interactive video. No significant
differences in diagnostic accuracy were reported between the two
modalities, even when low versus high bandwidth conditions were
compared. The authors concluded that evaluations conducted via tele-
psychiatry were accurate enough to replace in-person evaluations in
In a more recent study, Osenbach et al.12conducted a meta-
analysis that included 14 studies of depression treatment delivered
via telehealth versus in-person. No significant systematic difference
in effect size was found between the two modalities, although studies
that indicated the use of ‘‘care as usual’’ as the comparison group
differed from those using a face-to-face comparison treatment, sug-
gesting that ‘‘care as usual’’ was inferior to treatment delivered using
telehealth technology.12Despite this growing consensus regarding its
utility, telemedicine adoption still faces significant hurdles with re-
et al.13revealed that only small numbers of emergency telemental
health evaluation programs are functioning in hospital emergency
departments nationwide, and most of these programs must rely on
grants to sustain them.
The current study evaluated a telemedicine program implemented
in arural hospital ER. The project compared time to treatment, length
of stay, and door-to-consult time among patients presenting to the
ER before and after implementation of an emergency telemental
health evaluation service. This article presents important descriptive
data on the impact of integrating telemental health services into the
work flow processes at a CAH ER.
Materials and Methods
occurring serial sample of patients. All those who presented in the ER
over the 13-month study period from January 1, 2009 through January
31, 2010 and who had an emergency mental health evaluation ordered
and completed were included in the study (n=62). These patients were
divided into a pretelemedicine group that contained 24 patients and a
telemedicine group that contained 38 patients.Timeto treatment,door-
to-consult time, and length of stay were extracted from charts and
electronic records and compared for the two groups of patients.
Time to treatment was defined as the time interval (in hours) from
the consultation was completed. Door-to-consult time was the time
interval from the time the patient arrived (triage date and time) to the
time the consult was completed. Length of stay was defined as the
and time) to the time the patient was discharged from the hospital,
including any time spent as a registered inpatient.
Data were gathered for 7 months prior to implementation of the
telemental health program and for 6 months following implementa-
tion. To ensure that preimplementation data and postimplementation
data were collected in a standardized fashion, data collection tem-
plates were developed and imported into the facility’s electronic
medical record. Following Institutional Review Board approval, the
emergency department manager and the project team used the elec-
tronic medical record to collect the researchers’ variables of interest
prior to project implementation to ensure that new clinical and doc-
umentation protocols presented minimal disruption to the facility’s
standard operating procedures.
Following collection of baseline data, interactive videocon-
ferencing equipment was installed, and all involved hospital staff
were trained in the use of the equipment. A contingency plan
was put in place should the technology fail. Extensive records of
the process were kept, and necessary documentation was pro-
vided in easy-to-locate, color-coded envelopes to facilitate ease
of use and staff buy-in. Training was not required for staff at the
community mental health center because they were already
highly skilled in rendering urgent mental health consults via
videoconferencing. Following implementation of the emergency
telemental health program, the same variables of interest were
collected on patients who had a mental health consult ordered by
the provider on duty.
Tests of normality were conducted on outcome variables to ensure
the assumptions for parametric tests were met. Welch two-sample
t tests were used to determine differences between the variables of
interest. Data manipulation took place in Excel? (Microsoft?,
Redmond, WA), whereas all calculations were performed using SPSS
version 17.0 (IBM, Armonk, NY) and SAS System 9.0 for Windows
(SAS Institute, Cary, NC).
In total, 67 patients received urgent mental health consultations
were excluded from the study because they presented in an atypical
through the ER. The mean age of the sample was 37.1 years, with a
median and standard deviation of 40.0 and 15.9 years, respectively.
The range was 8–76 years. The population was 56% female and 44%
male. Half of the sample was single, with 27% married, 21.0% di-
vorced, and 1.6% widowed. Forty-five percent were unemployed,
with 21.0% employed, 12.9% disabled, 12.9% students, and 8.1%
retired. The payer mix of the sample was 41.9% self-pay, 22.6%
commercial, 14.5% Medicare, and 21.0% Medicaid. Percentages in
SOUTHARD ET AL.
TMJ-2013-0257-ver9-Southard_1P.3d04/21/14 12:35pmPage 2
2 TELEMEDICINE and e-HEALTH
each of these categories did not differ significantly between the pre-
telemedicine and telemedicine groups.
Patients presented to the emergency department with a wide va-
riety of chief complaints and received a corresponding range of di-
Table 1). The largest patient group (46.8%) presented
because they had attempted suicide. Four patients (6.5%) came in
requesting a psychiatric evaluation. Remaining chief complaints
were scattered across a wide range.
Suicide attempt was the most frequently assigned diagnosis code
at 32.3%, corresponding to the principal chief complaint. Additional
primary diagnoses included depression at 21.0%, other psychiatric
symptoms at 14.5%, suicidal ideation at 6.5%, substance abuse at
4.8%, and anxiety at 3.2%.
CHANGES IN SHIFT DISTRIBUTION
Prior to the implementation of the telemedicine program, emer-
gency mental health evaluations were only available from 8:00 a.m.
was operational, ER staff could request evaluations at any time and
most were completed within a few hours. Although this was not a
factor specifically studied or targeted for change, a change in the
distribution of consults across shift times was observed.
shows how consultations were limted to the day shift in the pre-
telemedicine group but were later spread across all shifts through the
use of telemedicine, with the majority being completed during
evening and night shifts (combined).
The professional personnel conducting the mental health consul-
tations were of varying types and consisted of social workers, li-
censed mental health counselors, and counseling psychologists. The
same staff at the community mental health center conducted the
Table 1. Diagnostic Groups and Chief Complaints
Primary diagnosis codea
Suicide attempt20 (32%)11 (46%) 9 (24%)
Depression13 (21%) 1 (4%) 12 (32%)
Substance abuse3 (5%) 2 (8%)1 (3%)
Anxiety2 (3%)1 (4%)1 (3%)
Psych other 9 (15%)4 (17%) 5 (13%)
Suicidal ideation4 (6%) 0 (0%) 4 (11%)
Chest pain3 (5%) 1 (4%)2 (5%)
3 (5%)0 (0%) 3 (8%)
Other 5 (8%) 4 (17%)1 (3%)
Attempted suicide29 (47%) 13 (54%)16 (42%)
7 (11%) 2 (8%)5 (13%)
4 (6%) 2 (8%)2 (5%)
Chest pain 4 (6%) 1 (4%)3 (8%)
Depression 3 (5%)0 (0%)3 (8%)
Confusion 4 (6%)1 (4%) 3 (8%)
MVA2 (3%) 2 (8%)0 (0%)
Assaulted2 (3%)1 (4%) 1 (3%)
Pain nonspecific 2 (3%)0 (0%)2 (5%)
5 (8%)2 (8%) 3 (8%)
Data are number (valid percentage).
aSuicide attempt includes drug overdose, poisoning, attempted suicide, and
intentional overdose. Substance abuse includes drugs and/or alcohol. Psych
other includes psychosis, schizophrenia, conduct disorder, bipolar disorder, and
behavior problems. Abdominal complaints includes nausea, vomiting, and stab
wound. Other includes sexually transmitted disease testing, hypernatremia,
concussion, and diabetic ketoacidosis.
bPsych other includes hallucinations, anxiety, behavioral problems, stress, panic
attack, hopelessness, and worthlessness (each n=1).
cOther includes alcohol intoxication, sexually transmitted disease testing,
weakness, nausea, and vomiting (each n=1).
MVA, motor vehicle accident.
Table 2. Triage Shift, Consult Completion Shift,
Days 10 (42%)7 (18%)
Evenings12 (50%)20 (53%)
Nights2 (8%)11 (29%)
Shift when consult completed
Days24 (100%)18 (47%)
Evenings0 (0%)12 (32%)
Nights0 (0%) 8 (21%)
Inpatient observation24 (100)%15 (39%)
Home (with follow-up) 0 (0%) 11 (29%)
Tertiary care center0 (0%) 3 (8%)
Behavioral facility0 (0%) 9 (24%)
Data are number (%).
RURAL EMERGENCY TELEMENTAL HEALTH EVALUATIONS
ª M A R Y A N N L I E B E R T , IN C . ? VOL. 20NO. 7 ? JULY 2014
TELEMEDICINE and e-HEALTH 3
evaluations prior to, and after, the implementation of the telehealth
intervention. Each case was staffed individually with a psychiatrist
prior to rendering a treatment plan. The case experiences actually
rendered by each of the varying different types of health profes-
sionals were not recorded.
It should be noted that prior to the use of telemedicine, all patients
who ultimately received a consult were admitted for observation. Once
increasednumber ofdisposition optionscould besafely and practically
considered by ER providers. These options included discharging pa-
tients to home with follow-up instructions or transfering directly to a
mental health facility. Although none of the patients was dischargedto
home prior to the implementation of telemedicine, 11 were discharged
to home with follow-up instructions after receiving an evaluation us-
ing telemedicine. This suggests that the ER provided more rapid access
to appropriate levels of care with the availability of the evaluations
AND LENGTH OF STAY
Order-to-consult times before the implementation of telemedicine
ranged from 0.50 to 52.9h. The mean, median, and standard deviation
for this group were 16.2, 14.2, and 13.2h, respectively. There was one
outlier at 52.9h. Order-to-consult times for the telemedicine group
were 5.4, 2.6, and 6.4h, respectively. There were three outliers: 15.9,
24.7, and 25.1h.
Table 3 lists the average order-to-consult, door-to-
Tests of assumptions for the use of parametric statistics were
shown to be met, so multiple t tests were conducted to determine the
significance of differences observed. The alpha level was adjusted
accordingly to 0.015. Welch two-sample t tests confirmed that the
reductions observed in all three measures were statistically signifi-
cant: order to consult, t29.9=3.75, p<0.001; door to consult,
t41.5=3.97, p<0.001; and length of stay, t57.2=3.60, p<0.001.
This study examined changes in rural hospital emergency de-
partment wait times for patients for whom mental health evaluations
were ordered. Prior to the implementation of telemedicine, these
consults were provided by regional providers who traveled various
Exorbitant transit time and provider shortages frequently resulted in
missed opportunities and delayed access for patients in desperate
need of mental health services. The telemental health service, once
implemented, provided 24/7 access to mental health evaluations in
the rural hospital via telemedicine. Many of the same clinicians
provided these services, but the travel time variable related to the
hospital was eliminated.
patients to receive emergency mental health evaluations and to sub-
sequently be released from the rural hospital was significantly lower
after implementation of the telemedicine program. The telemedicine
night. When ‘‘anytime’’ access became available, the distribution of
findings suggest that using telemedicine to make mental health eval-
uations readily available around the clock may be viewed as a ‘‘best
mental health diagnoses or symptoms. These facilities provide critical
access to a wide range of valuable services but are often staffed by
practitioners who are uncomfortable dealing with mental health crises.
This lack of comfort can lead to the provision of expensive, overly
cautious, and less than optimal care. The implementation of a mech-
anism to provide timely and equitable access to specialized urgent
mental health consults appears likely to have a positive impact on the
quality and efficiency of ER services available at rural hospitals.
This study makes an important contribution to the body of evi-
mental health assessment in rural and underserved areas. As health-
care providers and administrators in critical access hospitals strive to
meet the needs of their patients while strugglingwith a myriad of
financial challenges, it is imperative that they work together to use
effective care delivery modalities. Future studies investigating lon-
gitudinal patient outcomes and the economic impact of the inter-
vention are warranted.
The groups in the project had relatively small sample sizes (n=24
and n=38); therefore statistical efficiency was less than optimal.
Randomization was not part of the project given the timing of the
Table 3. Average Time Interval Spent in the Emergency
RANGE MEAN (SD)MEDIAN
Order to consulta
Without TM0.5–52.9 16.2 (13.2)14.2
With TM0.02–25.1 5.4 (6.4)2.6
Door to consulta
Without TM 6.3–56.6 22.7 (12.6)19.6
With TM1.7–50.910.5 (10.2) 5.9
Length of staya
Without TM12.6–65.9 31.7 (14.1)26.3
With TM3.0–69.5 17.0 (18.0)8.2
Data are hours.
SD, standard deviation; TM, telemedicine.
SOUTHARD ET AL.
TMJ-2013-0257-ver9-Southard_1P.3d04/21/14 12:35pmPage 4
4 TELEMEDICINE and e-HEALTH
telemedicine intervention. Implementation of standardized docu- Download full-text
mentation templates prior to initiation of the project may have led to
subsequent reductions in time to consult in both groups because of
increased awareness of the performance improvement project and
independent of the implementation of telemedicine. This last limi-
tation may have distorted measures of the magnitude of the actual
differences in time between the two service delivery methods but
could not have accounted for the differences themselves.
An additional limitation of the study is the fact that not all patients
who presented to the facility and who needed a mental health evalu-
and who actually received that consult were included because of lim-
itations of the medical record system. The study does not capture pa-
tients who had a consultation ordered but did not receive it because of
signing out against medical advice, leaving the hospital, being taken
into custody, or any other reason. Capturing data from these patients
may have improved our estimates of the program’s overall utility.
The authors would like to thank the Indiana State Office of Rural
Health for funding this project, Union Hospital’s Richard G. Lugar
Center for Rural Health, The Hamilton Center, and the amazing staff
at the Sullivan County Community Hospital. Special thanks also goes
to Galen Goode and Dr. Robert ‘‘Robe’’ G. Fazekas for their com-
mitment and dedication to mental health diagnosis and treatment in
our great state.
No competing financial interests exist.
1. National Center for Health Statistics. National health interview survey.
Hyattsville, MD: National Center for Health Statistics, U.S. Department of Health
and Human Services, 1999.
2. Gamm LD, Hutchison LL. Rural health priorities in America: Where you stand
depends on where you sit. J Rural Health 2003;19:209–213.
3. Holzer C, Goldsmith H, Ciarlo J. Effects of rural-urban county type on the
availability of health and mental health care providers. In: Mental health United
States. Rockville, MD: Substance Abuse and Mental Health Service
Administration, U.S. Department of Health and Human Services, 1998:
4. Loux S, Hartley D. Smallest rural hospitals treat mental health emergencies.
Research & policy brief. Portland, ME: Maine Rural Health Research Center,
Edmund S. Muskie School of Public Service, University of Southern Maine,
5. Gamble J, Savage GT, Icenogle ML. Value-chain analysis of a rural health
program: Toward understanding the cost benefit of telemedicine applications.
Hosp Top 2004;82:10–17.
6. Telehealth Resource Center. Available at www.telehealthresourcecenter.org/
accessed July 20, 2013).
7. Ruskin PE, Reed S, Kumar R. Reliability and acceptability of psychiatric
diagnosis via telecommunication and audiovisual technology. Psychiatr Serv
8. O’Reilly R, Bishop J, Maddox K, Hutchinson L, Fisman M, Takhar J. Is
telepsychiatry equivalent to face-to-face psychiatry? Results from a
randomized controlled equivalence trial. Psychiatr Serv 2007;58:836–843.
9. De Las Cuevas C, Arrendondo T, Cabrera FM, Sulzenbacher H, Meise U.
Randomized clinical trial of telepsychiatry through videoconference versus
face-to-face conventional psychiatric treatment. Telemed J E Health
10. Singh S, Arya D, Peters T. Accuracy of telepsychiatric assessment of new routine
outpatient referrals. BMC Psychiatry 2007;7:55.
11. Hyler SE, Gangure DP, Batchelder ST. Can telepsychiatry replace in-person
psychiatric assessments? A review and meta-analysis of comparison studies.
CNS Spectr 2005;10:403–413.
12. Osenbach JE, O’Brien KM, Mishkind M, Smolenski DJ. Synchronous telehealth
technologies in psychotherapy for depression: A meta-analysis. Depress Anxiety
13. Williams M, Pfeffer M, Boyle J, Hilty DM. Telepsychiatry in the emergency
department: Overview and case studies. California Healthcare Foundation.
Available at www.chcf.org/publications/2009/12/telepsychiatry-in-the-
emergency-department-overview-and-case-studies (last accessed July 20,
Address correspondence to:
Erik P. Southard, DNP, FNP-BC
1433 North 6½ Street
Terre Haute, IN 47807
Received: July 27, 2013
Revised: October 14, 2013
Accepted: October 14, 2013
RURAL EMERGENCY TELEMENTAL HEALTH EVALUATIONS
TMJ-2013-0257-ver9-Southard_1P.3d 04/21/14 12:35pmPage 5
ª M A R Y A N N L I E B E R T , IN C . ? VOL. 20 NO. 7 ? JULY 2014
TELEMEDICINE and e-HEALTH 5