Content uploaded by Jason Alan Fries
Author content
All content in this area was uploaded by Jason Alan Fries on Sep 06, 2020
Content may be subject to copyright.
Brief Communications
Measure what matters: Counts of hospitalized patients
are a better metric for health system capacity planning for
a reopening
Sehj Kashyap ,
1,
* Saurabh Gombar,
1,3,
* Steve Yadlowsky,
2
Alison Callahan,
1
Jason Fries,
1
Benjamin A. Pinsky,
3
and Nigam H. Shah
1
1
Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA,
2
Deptartment of Electrical
Engineering, Stanford University, Stanford, California, USA, and
3
Department of Pathology and Medicine, Stanford University
School of Medicine, Stanford, California, USA
*These authors contributed equally.
Corresponding Author: Saurabh Gombar, Stanford Center for Biomedical Informatics Research, Medical School Office
Building, Room x235, 1265 Welch Road, Stanford, CA 94305-5479, USA; sgombar@stanford.edu
Received 17 April 2020; Revised 21 April 2020; Editorial Decision 22 April 2020; Accepted 25 April 2020
ABSTRACT
Objective: Responding to the COVID-19 pandemic requires accurate forecasting of health system capacity
requirements using readily available inputs. We examined whether testing and hospitalization data could help
quantify the anticipated burden on the health system given shelter-in-place (SIP) order.
Materials and Methods: 16,103 SARS-CoV-2 RT-PCR tests were performed on 15,807 patients at Stanford facili-
ties between March 2 and April 11, 2020. We analyzed the fraction of tested patients that were confirmed posi-
tive for COVID-19, the fraction of those needing hospitalization, and the fraction requiring ICU admission over
the 40 days between March 2nd and April 11th 2020.
Results: We find a marked slowdown in the hospitalization rate within ten days of SIP even as cases continued
to rise. We also find a shift towards younger patients in the age distribution of those testing positive for COVID-
19 over the four weeks of SIP. The impact of this shift is a divergence between increasing positive case confir-
mations and slowing new hospitalizations, both of which affects the demand on health systems.
Conclusion: Without using local hospitalization rates and the age distribution of positive patients, current mod-
els are likely to overestimate the resource burden of COVID-19. It is imperative that health systems start using
these data to quantify effects of SIP and aid reopening planning.
Key words; COVID-19, surge models, capacity planning, electronic health records
INTRODUCTION
In order to prepare for coronavirus disease 2019 (COVID-19), health
system leaders and policymakers need to forecast future healthcare
needs. A number of forecasting models have been developed and widely
shared to help healthcare facilities and governments predict upcoming
patient surges and plan accordingly.
1,2
These models take in a myriad of
inputs, including population demographics, currently admitted patients,
case doubling times, and the rate at which positive cases turn into hospi-
talizations, among others.
1,3–5
While there are enough models, guidance,
and data to provide accurate inputs to these models remain lacking.
6–
8
Currently, 95% of Americans have been asked to stay at home.
Schools, businesses, and community gathering activities have been
closed or curtailed, and plans of reopening hinge on demonstrated sus-
tained reduction in cases, availability of testing, and enough health sys-
V
CThe Author(s) 2020. Published by Oxford University Press on behalf of the American Medical Informatics Association.
All rights reserved. For permissions, please email: journals.permissions@oup.com
1126
Journal of the American Medical Informatics Association, 27(7), 2020, 1026–1131
doi: 10.1093/jamia/ocaa076
Advance Access Publication Date: 17 June 2020
Brief Communications
Downloaded from https://academic.oup.com/jamia/article/27/7/1026/5858301 by Stanford University Medical Center user on 06 September 2020
tem capacity to treat patients requiring hospitalization without resorting
to crisis standards of care.
9,10
Theneedtoaccuratelyplanhealthsystem
capacity is 1 of the 6 indicators cited to guide the state of California’s de-
cision making around reopening the state’s economy.
11
An important
first step is taking stock of the currently available hospitalization and
testing data and how it has evolved as the pandemic progresses.
Stanford Health Care (SHC), a large academic healthcare system,
serves patients in the San Francisco Bay Area. The major hospitals that
comprise SHC are located in Santa Clara County, which began mandated
shelter in place (SIP) on midnight March 16, 2020, 26 days before we
conducted the analysis presented here. SHC developed an in house Sars-
CoV-2 and had extensive testing capacity starting on March 4, and after-
ward, tested anyone who had influenza-like infection symptoms, anyone
who had known COVID-19–positive contact, and healthcare workers
with a known exposure, or those who were referred by physician discre-
tion. As of April 11, our laboratory had tested nearly 15,800 cases, and
tracked hospitalization data of the test-confirmed COVID-19 cases.
Given our dual access to testing and hospitalization data, we exam-
ined whether we could reliably quantify the effects of state-mandated
SIP using testing and hospitalization rates. This allowed us to quantify
the divergence between the rates of positive case confirmations and
hospitalizations. In addition, we identified a shift in the age distribution
of new COVID-19–positive cases over the duration of the study.
MATERIALS AND METHODS
A total of 16,103 severe acute respiratory syndrome coronavirus 2
(SARS-CoV-2) reverse-transcription polymerase chain reaction tests
were performed on 15,807 patients at Stanford facilities between March
2 and April 11, 2020. Of these, 8309 tests were performed on 7929
patients in facilities in which the patient would have been admitted to
our hospital if necessary. We analyzed the fraction of tested patients that
were confirmed positive for COVID-19, the fraction of those needing
hospitalization, and the fraction requiring intensive care unit (ICU) ad-
mission over the 40 days between March 2 and April 11, 2020.
Data were obtained from a combination of 2 sources: a daily
refreshed snapshot of the health system’s Enterprise Data Warehouse
(EDW) and a twice-daily refreshed extract of all reverse-transcription
polymerase chain reaction laboratory tests for SARS-CoV-2 at Stan-
ford.AccesstoaviewoftheEDWwassetuplastyearaspartofStan-
ford Medicine’s Program for Artificial Intelligence in Healthcare.
12
The
EDW view consisted of 33 tables, containing demographics, diagnoses,
procedures, labs, and orders. These tables were refreshed nightly with
updated 1-year historical and current hospitalization information of
patients admitted in the census of the hospital. The access to the labo-
ratory testing was provided as part of Stanford’s data science response
to COVID-19.
13
The laboratory testing data consisted of details about
the specimen collected, the result, the procedure of specimen collection,
identifiers to link with EDW tables, and symptoms documented on ad-
mission. The project to track SARS-CoV-2 test positivity and hospitali-
zation trends was initiated as a quality assurance project aimed at
enabling hospital capacity planning, and institutional review board ap-
proval (protocol # 55544) was obtained prior to this submission sum-
marizing the learning from this effort.
RESULTS
A total of 3.77%of COVID-19–positive patients required
an ICU admission
As shown in Figure 1, of these 7929 tested patients, 451 (5.68%)
tested positive for COVID-19, and of these 451 cases, 59 (13.08%)
were hospitalized following their test. Among the 59 hospitalized
cases, 17 (28.8% of hospitalized and 3.77% of all positive cases) re-
quired ICU care. Our observed case hospitalization rate is more in
line with the 12% reported nationally than the 25.5% reported in
New York City (as of April 7).
14,15
The higher case hospitalization
rate in New York City may be due to the fact that low testing capac-
ity to case numbers might have led to severe cases being prioritized
for testing. Our ICU rate among confirmed cases similarly matches
American national reports of 2.9%, compared with the 5% reported
in China or 12% in Lombardy (Italy).
16,17
The hospitalization rate slowed within 10 days of SIP,
even as confirmed cases continue to rise
Between March 2nd, 2020 and April 11th, 2020, we have continued
to see new COVID-19 cases, hospitalizations, and ICU admissions;
however, their rates have slowed (Figure 2). The doubling time for
each metric increased from under 5 days on March 16 to over 25
days as of April 11. The slowdown in hospital admissions began
within 10 days of SIP and was more dramatic than the slowdown of
confirmed cases. Forecasting models should incorporate the diver-
gence between the rate of new COVID-19 cases vs new hospitaliza-
tions to better forecast near-term demand on the health system.
Given that the test results and admissions data are available in
nearly real time, health systems–based monitoring of admission
rates and the doubling time of hospitalized patient counts can pro-
vide accurate data for both public health planning and epidemiologi-
cal modeling.
3
Recently detected COVID-19 cases were younger than
were those detected prior to social distancing measures
As shown in Figure 3, between weeks 11 and 14 of the epidemic,
there was no significant shift in the age distribution of patients
tested. However, the average age of COVID-19–positive patients de-
creased (P¼.0004) from 55.6 (95% confidence interval [CI], 53.0-
58.3) years of age prior to social distancing (March 16, week 11) to
49.8 (95% CI 47.9—51.7) years of age after 2 weeks of social dis-
tancing (March 30, week 14).
Characteristics of COVID-19 cases hospitalized later
were the same as those hospitalized earlier
Compared with before social distancing, the mean length of stay of
hospitalized cases and rate of ICU admission was not significantly dif-
ferent (1.73 [95% CI, -2.16 to 5.62] days shorter and 4.1% absolute
increase [95% CI, -30% to 38%], respectively) than 2 weeks after so-
cial distancing (difference in means P¼.378 and .810, respectively)
(Table 1). Because the length of stay was right-censored for patients
still in the hospital, these estimates were corrected for censoring.
DISCUSSION
While most epidemic simulation models use new case rates, no cur-
rently published model takes into account the shifts in demographics
of positive patients, which is a major determinant of future hospital
admission rates. If most new cases are younger, the corresponding
need for hospitalizations will also be lower. Over the course of 5
weeks, we did not find a significant change in the age distribution of
patients presenting with influenza-like illness (ILI) and who tested
for COVID-19; however, we did find a significant shift toward
younger patients in those testing positive for COVID-19. The sim-
plest explanation for the shift toward younger patients testing posi-
Journal of the American Medical Informatics Association, 2020, Vol. 27, No. 7 1127
Downloaded from https://academic.oup.com/jamia/article/27/7/1026/5858301 by Stanford University Medical Center user on 06 September 2020
tive could be relaxing testing criteria. For example, guidelines at our
institution went from requiring ILI symptoms and a known COVID-
19 exposure to expressing symptoms consistent with ILI, to medical
doctor discretion. However, this does not explain why there is no
change in the demographics of patients getting tested for COVID-
19. A more plausible explanation is that because of the SIP order,
the at-risk elderly population is protected and hence less likely to
contract COVID-19. This interpretation is supported by the fact
that despite seeing a larger number of younger COVID-19–positive
patients, those that need to be admitted have similar ages, rate of
ICU admission, and length of stay as before the SIP order; now, we
see fewer absolute numbers of such cases.
Our analysis spanning 26 days after SIP clearly demonstrates
that the rate of confirmed cases, hospitalizations, and ICU admis-
sions for COVID-19 has flattened. The decrease in the rate of new
hospitalizations began within 10 days after SIP was initiated and
continues today. Despite the decrease in hospitalizations, we con-
tinue to see new patients presenting with ILI and younger patients
testing positive for COVID-19, indicating ongoing community
spread but perhaps in a lower-risk population.
This analysis demonstrates how, compared with new case counts,
new hospitalizations is a better metric both for detecting the effect of SIP
and for estimating the anticipated burden on the health system.
10,11,18
Our findings also suggest that existing surge planning efforts should fre-
quently recompute hospitalization doubling time because the change
can be swift, as seen in our data.
19
Models that do not use local hospital-
ization rates as well as the age distribution of the positive patients are
likely to overestimate the resource burden of COVID-19.
Investments made in setting up data feeds prior to and immedi-
ately at the outset of this crisis were critical to accessing such data
quickly. Additionally, a unification of our information technology
organizations across the School of Medicine and the Healthcare Sys-
Figure 1. Test result as well as hospitalization outcomes of patients tested for severe acute respiratory syndrome coronavirus 2. Each box represents 10 patients.
ICU: intensive care unit.
1128 Journal of the American Medical Informatics Association, 2020, Vol. 27, No. 7
Downloaded from https://academic.oup.com/jamia/article/27/7/1026/5858301 by Stanford University Medical Center user on 06 September 2020
Figure 2. Slowed growth in total cases detected, hospitalized, and admitted to the intensive care unit (ICU) is seen after the shelter-in-place order (first dashed
line). The prolongation of the doubling time of hospitalizations (yellow) happens faster and earlier than cases detected (gray). The divergence between the rate of
cases detected and slower rate of hospitalizations is seen within 10 days of the shelter-in-place order. Doubling times calculated over a 7-day sliding window
show that by March 28 (second dashed line), cases were doubling every 9 days, but hospitalizations were doubling every 13 days.
Figure 3. Change in age distribution of patients with positive severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) test (brown) compared with those
tested for SARS-CoV-2 (gray) for 4 weeks after sheltering in place order.
Journal of the American Medical Informatics Association, 2020, Vol. 27, No. 7 1129
Downloaded from https://academic.oup.com/jamia/article/27/7/1026/5858301 by Stanford University Medical Center user on 06 September 2020
tem, which was completed in September 2019, proved to be im-
mensely valuable enabling the setup of data access (such as to the
laboratory testing data) in a timely manner.
CONCLUSION
Given the relative ease of obtaining testing and admission data at a
health system, these metrics can not only help quantify the effects of
state-mandated SIP, but also enable better planning of health system
capacity to aid any actions required to return to precrisis operations.
20
For any reopening scenario, accurate projections of near-term health
system capacity requirements are essential.
21
Therefore, we must start
using these easily available, and useful, inputs right away.
FUNDING
This work was supported by National Library of Medicine grant
R01LM011369-07 (SK, AC, JF, and NHS), the Stanford Health CEO Innova-
tion Fund, and the Debra and Mark Leslie endowment for AI in Healthcare.
AUTHOR CONTRIBUTIONS
All authors were involved in drafting or critically revising the presented work,
gave final approval of the submitted manuscript, and agree to be accountable
for ensuring the integrity of the work. SG, SK, NHS and SY drafted the manu-
script, which was reviewed and edited by all co-authors. SG, SK, NHS con-
ceived of the analysis design. BAP acquired, helped interpret and analyze lab
testing data. JAF and AC analyzed hospitalized patients’ admission character-
istics and presenting symptom severity. SG, SK and SY conducted statistical
analysis of the data, and SY conducted additional statistical analyses to pro-
duce the censor-adjusted estimates.
ACKNOWLEDGMENTS
We acknowledge support from the Departments of Medicine and Pathology
at Stanford.
CONFLICT OF INTEREST STATEMENT
The authors have no competing interests to declare.
REFERENCES
1. IHME COVID-19 Health Service Utilization Forecasting Team, Murray
CJL. Forecasting COVID-19 impact on hospital bed-days, ICU-days,
ventilator-days and deaths by US state in the next 4 months. medRxiv
10.1101/2020.03.27.20043752; 2020.
2. Goh G. Epidemic calculator. https://gabgoh.github.io/COVID/index.html
Accessed April 2020.
3. Ferstad JO, Gu AJ, Lee RY, et al. A model to forecast regional demand for
COVID-19 related hospital beds. medRxiv 10.1101/2020.03.26
.20044842; 2020. Accessed April 2020.
4. Srivastava I. COVID-19 hospital impact model for epidemics (CHIME).
https://oto.med.upenn.edu/2020/03/31/covid-19-hospital-impact-model-
for-epidemics-chime/ Accessed April 2020.
5. Flaxman S, Mishra S, Gandy A, et al. Report 13: estimating the number
of infections and the impact of non-pharmaceutical interventions on
COVID-19 in 11 European countries. https://spiral.imperial.ac.uk:8443/han-
dle/10044/1/77731 Accessed April 2020.
6. Yadlowsky S, Shah N, Steinhardt J. Estimation of SARS-CoV-2 Infection
Prevalence in Santa Clara County. medRxiv 10.1101/2020.03.24.
20043067; 2020.
7. Shah N. There are enough models, we need accurate inputs! https://me-
dium.com/@nigam/there-are-enough-models-we-need-accurate-inputs-
5a20aef22f01 Accessed April 2020.
8. Malhi S. Poor state reporting hampers pandemic fight. The Hill. https://
thehill.com/opinion/white-house/492025-poor-state-reporting-hampers-
pandemic-fight Accessed April 2020.
9. Mervosh S, Lu D, Swales V. See which states and cities have told residents
to stay at home. The New York Times. https://www.nytimes.com/interac-
tive/2020/us/coronavirus-stay-at-home-order.html?auth¼login-
email&login¼email Accessed April 2020.
10. American Enterprise Institute. National coronavirus response: A road map
to reopening. - AEI. https://www.aei.org/research-products/report/national-
coronavirus-response-a-road-map-to-reopening/ Accessed April 2020.
11. Kennedy M. California governor outlines how the state will decide to
open again. NPR. https://www.npr.org/sections/coronavirus-live-updates/
2020/04/14/834434343/california-governor-outlines-how-the-state-will-
decide-to-open-again Accessed April 2020.
12. Newby K. AI love you: Stanford artificial intelligence projects designed to
improve your health. Stanford Medicine. 2018. https://stanmed.stanford.edu/
2018summer/how-ai-enhances-doctor-patient-relationship.html Accessed April
2020.
13. COVID-19 research. Stanford Medicine. https://med.stanford.edu/
covid19/research.html Accessed April 2020.
14. Arentz M, et al. Characteristics and outcomes of 21 critically ill patients
with COVID-19 in Washington State. JAMA 2020; 323 (16): 1612–4. doi
: 10.1001/jama.2020.4326.
15. CDC COVID-19 Response Team. Severe outcomes among patients with
coronavirus disease 2019 (COVID-19) - United States, February 12-
March 16, 2020. MMWR Morb Mortal Wkly Rep. 2020; 69: 343–6.
16. Grasselli G, Pesenti A, Cecconi M. Critical care utilization for the
COVID-19 outbreak in Lombardy, Italy: early experience and forecast
during an emergency response. JAMA 2020; 323 (16): 1545–6. doi :
10.1001/jama.2020.4031.
17. Wu Z, McGoogan JM. Characteristics of and important lessons from the
coronavirus disease 2019 (COVID-19) outbreak in China: summary of a
report of 72 314 cases from the Chinese Center for Disease Control and
Prevention. JAMA 2020; 323 (13): 1239–42.
18. Lewnard JA, Liu VX, Jackson ML, et al. Incidence, clinical outcomes, and
transmission dynamics of hospitalized 2019 coronavirus disease among
9,596,321 individuals residing in California and Washington, United States:
a prospective cohort study. medRxiv 10.1101/2020.04.12.20062943; 2020.
Table 1. Characteristics of COVID-19–hospitalized cases around the state-mandated SIP ordered on March 16
Period Hospitalized Cases Age (y) Length of stay (d) % Hospitalized Admitted to ICU Transfer to ICU From Admission (d)
Pre-SIP 16 58.1 615.1 8.31 31.2 1.72 61.4
First 2 wk SIP 28 59 618.7 6.91 25 1.19 61.7
>2 wk SIP 15 61.4 619.1 6.58 33.3 0.76 60.65
Values are mean 6SD, unless otherwise indicated. Length of stay is adjusted for censoring using the Kaplan-Meier curve and 14 day restricted mean length of
stay.
COVID-19: coronavirus disease 2019; ICU: intensive care unit; SIP: shelter in place.
1130 Journal of the American Medical Informatics Association, 2020, Vol. 27, No. 7
Downloaded from https://academic.oup.com/jamia/article/27/7/1026/5858301 by Stanford University Medical Center user on 06 September 2020
19. Zhang T, McFarlane K, Vallon J, et al. A model to estimate bed demand
for COVID-19 related hospitalization. medRxiv 10.1101/2020.03.24.
20042762; 2020.
20. Koseff A, Allday E. California’s coronavirus reopening: Gavin Newsom’s
six-point plan will alter daily life. San Francisco Chronicle. https://www.
sfchronicle.com/politics/article/California-s-coronavirus-reopening-Gov-
Gavin-15200205.php Accessed April 2020.
21. Roy A. A new strategy for bringing people back to work during COVID-
19. FREOPP. https://freopp.org/a-new-strategy-for-bringing-people-
back-to-work-during-covid-19-a912247f1ab5 Accessed April 2020.
Journal of the American Medical Informatics Association, 2020, Vol. 27, No. 7 1131
Downloaded from https://academic.oup.com/jamia/article/27/7/1026/5858301 by Stanford University Medical Center user on 06 September 2020