ArticlePDF Available

Maximizing the Efficiency of Active Case Finding for SARS-CoV-2 Using Bandit Algorithms

Authors:

Abstract

Even as vaccination for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) expands in the United States, cases will linger among unvaccinated individuals for at least the next year, allowing the spread of the coronavirus to continue in communities across the country. Detecting these infections, particularly asymptomatic ones, is critical to stemming further transmission of the virus in the months ahead. This will require active surveillance efforts in which these undetected cases are proactively sought out rather than waiting for individuals to present to testing sites for diagnosis. However, finding these pockets of asymptomatic cases (i.e., hotspots) is akin to searching for needles in a haystack as choosing where and when to test within communities is hampered by a lack of epidemiological information to guide decision makers’ allocation of these resources. Making sequential decisions with partial information is a classic problem in decision science, the explore v. exploit dilemma. Using methods—bandit algorithms—similar to those used to search for other kinds of lost or hidden objects, from downed aircraft or underground oil deposits, we can address the explore v. exploit tradeoff facing active surveillance efforts and optimize the deployment of mobile testing resources to maximize the yield of new SARS-CoV-2 diagnoses. These bandit algorithms can be implemented easily as a guide to active case finding for SARS-CoV-2. A simple Thompson sampling algorithm and an extension of it to integrate spatial correlation in the data are now embedded in a fully functional prototype of a web app to allow policymakers to use either of these algorithms to target SARS-CoV-2 testing. In this instance, potential testing locations were identified by using mobility data from UberMedia to target high-frequency venues in Columbus, Ohio, as part of a planned feasibility study of the algorithms in the field. However, it is easily adaptable to other jurisdictions, requiring only a set of candidate test locations with point-to-point distances between all locations, whether or not mobility data are integrated into decision making in choosing places to test.
Technical Note
Medical Decision Making
1–8
ÓThe Author(s) 2021
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/0272989X211021603
journals.sagepub.com/home/mdm
Maximizing the Efficiency of Active Case
Finding for SARS-CoV-2 Using Bandit
Algorithms
Gregg S. Gonsalves , J. Tyler Copple, A. David Paltiel, Eli P. Fenichel,
Jude Bayham, Mark Abraham, David Kline, Sam Malloy, Michael F. Rayo,
Net Zhang, Daria Faulkner, Dane A. Morey, Frank Wu, Thomas Thornhill,
Suzan Iloglu , and Joshua L. Warren
Even as vaccination for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) expands in the United
States, cases will linger among unvaccinated individuals for at least the next year, allowing the spread of the corona-
virus to continue in communities across the country. Detecting these infections, particularly asymptomatic ones, is
critical to stemming further transmission of the virus in the months ahead. This will require active surveillance efforts
in which these undetected cases are proactively sought out rather than waiting for individuals to present to testing
sites for diagnosis. However, finding these pockets of asymptomatic cases (i.e., hotspots) is akin to searching for nee-
dles in a haystack as choosing where and when to test within communities is hampered by a lack of epidemiological
information to guide decision makers’ allocation of these resources. Making sequential decisions with partial infor-
mation is a classic problem in decision science, the explore v. exploit dilemma. Using methods—bandit algorithms—
similar to those used to search for other kinds of lost or hidden objects, from downed aircraft or underground oil
deposits, we can address the explore v. exploit tradeoff facing active surveillance efforts and optimize the deployment
of mobile testing resources to maximize the yield of new SARS-CoV-2 diagnoses. These bandit algorithms can be
implemented easily as a guide to active case finding for SARS-CoV-2. A simple Thompson sampling algorithm and
an extension of it to integrate spatial correlation in the data are now embedded in a fully functional prototype of a
web app to allow policymakers to use either of these algorithms to target SARS-CoV-2 testing. In this instance,
potential testing locations were identified by using mobility data from UberMedia to target high-frequency venues in
Columbus, Ohio, as part of a planned feasibility study of the algorithms in the field. However, it is easily adaptable
to other jurisdictions, requiring only a set of candidate test locations with point-to-point distances between all loca-
tions, whether or not mobility data are integrated into decision making in choosing places to test.
Keywords
bandit algorithms, reinforcement learning, SARS-CoV-2, surveillance, testing
Date received: December 31, 2020; accepted: May 10, 2021
Even as vaccinations against severe acute respiratory
syndrome coronavirus 2 (SARS-CoV-2) roll out in the
United States, new infections continue to mount across
the country.
1
However, while new infections will decrease
as more people are immunized, lingering cases of the cor-
onavirus will still exist in communities across the United
States, frustrating attempts to fully suppress transmission
and end the pandemic.
2
Even though multiple sites for
testing for SARS-COV-2 are available in many
Corresponding Author
Gregg S. Gonsalves, Department of Epidemiology of Microbial Dis-
eases, Yale School of Public Health, Public Health Modeling Unit, 350
George Street, 3rd Floor, New Haven, CT 06511, USA
(gregg.gonsalves@yale.edu).
communities, tracking down and detecting many cases of
the virus will require active surveillance, in which public
health workers seek out infections rather than waiting
for individuals to present for diagnosis.
3
While active surveillance efforts, particularly tied to
contact tracing, were stymied by the scale of the epidemic
in 2020 in the United States, other countries, particularly,
China, South Korea, Hong Kong, Singapore, Taiwan,
Australia, Vietnam, and New Zealand, achieved low
community transmission levels through robust control
measures that include extensive surveillance efforts.
4,5
In
fact, relying on presentation to health care facilities for
diagnosis cannot contain the pandemic, and community-
based surveillance and contact tracing are vital to early
detection of new cases and prevention of the resurgence
of the disease.
5,6
Community-based surveillance efforts in the United
States are now widespread and have taken SARS-CoV-2
testing out of the health care setting. There are different
types of off-site testing designs, retrofitting existing struc-
tures (e.g., a sporting venue) and using tents for pop-up
clinics and vans to take SARS-CoV-2 testing fully
mobile.
7
Our efforts are directed toward the last two
design choices, in which SARS-CoV-2 testing does not
depend on individuals seeking out a fixed site but can be
moved to seek out places in the community at higher risk
for transmission than others and reach vulnerable popu-
lations that may not be able to reach other venues at a
distance from their homes (e.g., the elderly, those with-
out transport).
These kinds of mobile testing opportunities have been
successful in increasing uptake of human immunodefi-
ciency virus (HIV) testing in the United States and
abroad, particularly targeting high-risk populations that
otherwise may not come forward for screening.
8–10
This
targeted testing among high-risk communities is now
also happening in the context of coronavirus disease
2019 (COVID-19).
11
Combining these types of testing
interventions with geospatial and phylogenetic data,
information on social and sexual networks has also been
proposed as a way to home in on hotspots of HIV,
increasing the yield of testing.
12
However, even with
these kinds of efforts, approximately 14% of Americans
living with HIV remain unaware of their HIV serostatus,
leaving the detection of undiagnosed infections as the
‘‘holy grail’’ of HIV control efforts, indicating additional
approaches are necessary to reach these individuals.
13
A
similar situation may persist with SARS-CoV-2 in the
United States, with many undiagnosed infections still
undetected across the country.
14
As happened with HIV infection, new platforms for
SARS-CoV-2 testing are emerging quickly, from at-
home tests to rapid antigen assays to supplement stan-
dard laboratory-based polymerase chain reaction (PCR)
diagnostics, with saliva-based alternatives to invasive
nasopharyngeal swabs for the comfort and convenience
of patients.
15
But these technologies cannot address the
simple question that underlies active surveillance efforts:
where can we find lingering cases of SARS-COV-2?
Identifying most, if not all, infections in the United
States through active case finding, contact tracing, isola-
tion of infected individuals, and quarantine of their con-
tacts is the ideal route to containing SARS-CoV-2.
16–18
Universal testing, contact tracing, and isolation strate-
gies for SARS-CoV-2 deployed in places like Wuhan,
China, and akin to the universal test-and-treat efforts for
HIV in some countries are expensive, are resource inten-
sive, and, in the context of SARS-CoV-2 control in
Wuhan, have raised human rights concerns.
19,20
If uni-
versal testing, contact tracing, and isolation are infeasible
for the United States, how can we maximize the yield of
new cases detected? While we might target pop-up or
mobile testing to places where we believe the prevalence
of undetected infection to be very high (e.g., apartment
buildings, nursing homes, police stations) in many cities
and towns, choosing between these venues may be diffi-
cult both in terms of their epidemiological value (i.e., the
Department of Epidemiology of Microbial Diseases, Yale School of
Public Health, New Haven, CT, USA (GSG, JTC, FW, TT, SI);
Department of Health Policy and Management, Yale School of Public
Health, New Haven, CT, USA (ADP); Yale School of the Environ-
ment, New Haven, CT, USA (EPF); Department of Agricultural and
Resource Economics, Colorado State University, Fort Collins, CO,
USA (JB); DataHaven, New Haven, CT, USA (MA); Center for Bios-
tatistics, Department of Biomedical Informatics, The Ohio State Uni-
versity, Columbus, OH, USA (DK); Battelle Center for Science,
Engineering, and Public Policy, John Glenn College of Public Affairs,
The Ohio State University, Columbus, OH, USA (SM, NZ); Integrated
Systems Engineering, The Ohio State University, Columbus, OH, USA
(MFR, DAM); College of Public Health, The Ohio State University,
Columbus, OH, USA (DF); Department of Biostatistics, Yale School
of Public Health, New Haven, CT, USA (JLW); Public Health Model-
ing Unit, Yale School of Public Health, New Haven, CT, USA (GSG,
JTC, ADP, FW, TT, SI). The author(s) declared no potential conflicts
of interest with respect to the research, authorship, and/or publication
of this article. The author(s) disclosed receipt of the following financial
support for the research, authorship, and/or publication of this article:
Financial support for this study was provided entirely by grants from
the National Institute on Drug Abuse DP2 (DA049282 to GSG), R37
(DA15612 to GSG and ADP), the National Institute of Allergy and
Infectious Diseases R01 (AI137093 to JLW), the National Science
Foundation Northeast Big Data Innovation Hub Subaward 4
(GG01486-02) PTE Federal Award (No. OAC-19165850) (EPF), Yale-
AWS Enterprise Agreement (EPF), and the Tobin Center for Economic
Policy at Yale University (EPF). The funding agreement ensured the
authors’ independence in designing the study, interpreting the data,
writing, and publishing the report.
2Medical Decision Making 00(0)
underlying prevalence at a site) and the willingness of
people passing through these locations to volunteer for
testing.
The elements of the predicament for screening for
SARS-CoV-2 in the United States are clear: many people
remain undiagnosed with SARS-CoV-2, and the pros-
pects of universal testing of entire communities are slim.
Thus, we want to maximize the number of cases detected
with the limited resources we do have while cognizant
that we also have imperfect information about where
these undetected infections are to be found. Policy-
makers have choices about how to address this problem.
In deciding where to deploy their testing resources, pol-
icymakers must choose between making the best possible
choice based on their current understanding of the evi-
dence and investing in improving their understanding of
the evidence in the hope that it will lead to even better
choices in subsequent periods. That is, policymakers can
go with what they know and target testing at places they
assume are high risk (e.g., nursing homes), but the kinds
and numbers of high-risk venues, which might yield the
most cases, may be large and diverse and shift over
time.
21
In addition, SARS-CoV-2 is an overdispersed
pathogen tending to spread in clusters with heterogeneity
and stochasticity in transmission, and targeting locations
based on simple assumptions about risk environments
may turn up to be dead-ends.
22
Beyond universal testing,
the alternative is to test randomly across a community,
with the hopes of finding ‘‘hotspots’’ but facing the pros-
pect that the number of positive diagnoses may wane at
promising locations as testing uncovers most of the
undiagnosed cases there or the epidemic moves on to
new places in a community. Both of these choices open
to policymakers present an iterative series of questions:
where do we test today, how long do we stay in that
location, when do we move on, and where do we go
next? How can policymakers best make these compli-
cated decisions between the choices open to them for
venues to test for SARS-CoV-2 on an ongoing basis and
in an evidence-based fashion? Here we describe how a
set of tools we have modified and adapted from the
sequential decision making field—namely, bandit
algorithms—may help solve the conundrum of SARS-
CoV-2 testing in the context of limited resources.
The Explore v. Exploit Dilemma
and Bandit Algorithms
The ‘‘explore v. exploit dilemma’’ is a classic problem,
where a limited resource must be deployed across alter-
native targets in a way that maximizes overall gain, when
the critical attribute of each target is only partially
known at the time of deployment but may become better
understood as a result of the deployment decision. This
is a problem that has been studied in depth in the fields
of operations research and decision science. How do you
mine your best current prospects (‘‘exploit’’) while keep-
ing your eye open to better opportunities (‘‘explore’’)?
The tools that are used to address this dilemma are
called bandit algorithms.
23
They are widely used to guide
sequential decisions under uncertainty in a range of set-
tings, from commercial applications (e.g., oil explora-
tion) to military efforts (e.g., searching for downed
airplanes).
24
We have previously studied the use of bandit algo-
rithms for HIV testing—to identify undiagnosed HIV
infection using mobile testing units—for several
years.
24,25
Using model-based simulation studies, we
have shown that bandit algorithms outperform more tra-
ditional approaches for deploying HIV testing resources,
including going this year where you found the most HIV
cases last year or sampling a large number of candidate
locations before settling down on the best place to
test.
24,25
The basic bandit algorithm—known as Thomp-
son sampling—outperforms these other methods.
26
Thompson sampling is an adaptive Bayesian approach.
First, it makes an inventory of all possible target settings.
Second, a policymaker offers an initial assessment of the
prevalence of undetected infection in each target setting.
This takes the form of a probability distribution and is,
by design, a subjective exercise that permits the policy-
maker to be as definitive or tentative as their prior infor-
mation directs them to be. These prior probability
distributions are updated as new information arrives.
Third, the decision to deploy testing resources on the
first day is made via a random selection from the various
prior distributions assigned to each candidate setting.
Fourth, on each day, a record is maintained in every
active testing setting of the total tests performed and the
number of positive cases detected. This information is
used to update the prior distribution for that site. Then,
the decision to deploy testing resources on the next day
is repeated via a random selection using the updated
priors (i.e., posteriors) and the process repeats.
At the outset, this strategy assigns greatest priority to
settings based entirely on the policymaker’s initial assess-
ment, which is based on the data available at the
moment. But as the testing campaign proceeds, this
strategy provides new information about the probability
of finding a case among the available locations by using
the daily test results (positive and negative) at each site
selected to refine the understanding of the prevalence at
each location. Over time, this continuous process of
Gonsalves et al. 3
‘‘learning while doing’’ homes in on the places with
highest potential yield of finding new cases faster than
other strategies. In addition, if the situation changes—
for instance, if one has saturated a given location and
depleted the number of undetected cases in that
location—the posterior distribution associated with that
location will reflect those shifts as well, making it less
likely that the site will be chosen in the future. The algo-
rithm is flexible in its accommodation of the time-value
of information. If, for example, there is reason to believe
that information acquired in previous rounds should
have diminishing influence over time (e.g., the site has
not been visited in a month) or some exogenous factor
has changed the underlying environment (e.g., emergence
of new viral strains or variants), one can apply a ‘‘dis-
count factor’’ that assigns less and less weight to prior
observations as time goes by. Alternatively, if there is
reason to privilege initial assumptions and to make deci-
sions increasingly resistant to new observations, one can
apply a different discount factor that assigns decreasing
weight to newer data.
27
In practical terms, this strategy
is meant on an ongoing basis to guide and draw those
performing testing in the field to the sites where more
people are willing to test and with a premium on loca-
tions with a higher prevalence of undetected infection.
That is, the goal is to maximize yield of positive cases,
not to estimate local prevalence of disease. The details
of the Thompson sampling strategy are described in
Table 1.
We have also developed a variation on Thompson
sampling to account for spatial correlation in the
prevalence of undetected infections between adjacent
geographical areas using a hierarchical Bayesian spatial
modeling framework employing an intrinsic conditional
autoregressive (ICAR) prior distribution for the spatial
random effects and exchangeable, normally distributed
random effects to account for nonspatial heterogene-
ity.
25,28
The details of the spatial algorithm are shown in
Table 2.
Adapting the Bandit Algorithm to SARS-CoV-2
Bandit algorithms, including Thompson sampling and
those that model spatial correlation, can be implemented
easily as a guide to active case finding and screening for
SARS-CoV-2. To provide priors to initialize the algo-
rithms for use with SARS-CoV-2, we defined a set of
highly trafficked candidate locations for daily testing.
For Columbus, Ohio, we used raw data from the Uber-
Media COVID-19 recovery data set (https://covid19.ub
ermedia.com/covid-19-recovery-insights/), which con-
tains pairs of individual smart devices within 5 meters of
each other within a 5-minute window. These data were
cleaned for obvious geolocation errors and to remove
contacts on roadways using the Census Tiger Lines.
These data were then spatially joined with Loveland
Landgrid (https://landgrid.com/) parcel data. We pro-
duced indices of contacts and unique contacts per parcel,
and we also rarified the window of the contact definition.
These various indices generally produced consistent
rankings. We chose highly trafficked locations as these
venues create more opportunities for casual encounters
for testing but also because these bandit algorithms are
sensitive to testing volume, and a low number of volun-
teers could hamper the effectiveness of this approach. In
fact, in our studies of bandit algorithms in the context of
HIV infection, at fewer than 10 tests per day, bandit
algorithms performed poorly.
24
Before deploying these
algorithms in field testing, establishing a floor for daily
testing volume at potential locations through simulation
using best estimates of local epidemiology of SARS-
CoV-2 will be an important operational consideration,
and performing more tests at fewer sites may be a trade-
off to weigh for those using these algorithms in practice.
We supplemented this list of highly trafficked locations
with residential settings (e.g., apartment buildings),
where close contacts are numerous and frequent and
where adherence to social distancing and other infection
control measures may be more difficult. These are likely
to be potential hotspots for disease transmission and
Table 1 Thompson Sampling Strategy for Identifying Testing Sites for Severe Acute Respiratory Syndrome Coronavirus 2.
Algorithm 1 Thompson sampling strategy
For each potential testing site i=1...,nset X
i
(0) =0, Y
i
(0) =0.
for each t=1,2...t
max
,do
For each testing site i=1...,n, sample p
i
(t– 1) from the Beta(a
i
+X
i
(t–1), b
i
+Y
i
(t–1)) distribution.
Select testing site j = argmax
i
p
i
(t– 1).
Perform mBernoulli trials in testing site jand observe x
j
successes and (m–x
j
) failures.
Let X
j
(t)=X
j
(t–1)+x
j
and Y
j
(t)=Y
j
(t–1)+(m–x
j
).
For all testing sites ij, let X
i
(t)=X
i
(t– 1) and Y
i
(t)=Y
i
(t– 1).
End
4Medical Decision Making 00(0)
could be important places to search for new cases, partic-
ularly in census tracts with no public locations for test-
ing. Because we are assessing the yield of testing at point
locations rather than areal zones, we use the Euclidean
distance between locations rather than map adjacencies
when defining spatial proximity. However, distances
between locations can be defined in other ways given
topographical considerations and local contexts. We set
prior distributions for the prevalences in these locations
with a Beta(0.50,0.50) distribution (i.e., Jeffreys prior),
indicating our lack of knowledge of SARS-CoV-2
prevalence in any of these areas in Columbus. With more
data on the epidemiology of SARS-CoV-2, these prior
distributions could be crafted to reflect more knowledge
of local epidemics. Potential testing sites can also be
determined in other ways beyond the use of cell phone
and epidemiological data. For instance, targeting indus-
tries that are low-work-from-home and demand high
physical proximity for workplace testing could be con-
sidered potential targeting sites where social gatherings
regularly still take place even in the context of the pan-
demic (e.g., houses of worship).
29,30
Table 2 Hierarchical Bayesian Spatial Strategy for Identifying Testing Sites for SARS-CoV-2.
Algorithm 2 BYM strategy
For each potential testing site i = 1, . . ., n set X
i
(0) =0, Y
i
(0) =0.
do while the number of unique potential testing sites is \10:
For each testing site i=1,2,...,n, sample p
i
(t– 1) from the Beta(a
i
+X
i
(t– 1), b
i
+Y
i
(t– 1)) distribution.
Select testing site j = argmax
i
p
i
(t– 1).
Perform mBernoulli trials in testing site jand observe x
j
successes and (m–x
j
) failures.
Let X
j
(t)=X
j
(t–1)+x
j
and Y
j
(t)=Y
j
(t–1)+(m–x
j
).
For all testing sites ij, let X
i
(t)=X
i
(t– 1) and Y
i
(t)=Y
i
(t– 1).
do while the number of unique visited zones is 10:
Fit the hierarchical Bayesian spatial logistic regression model to the complete set of data:
Xit1ðÞjpi;Binomial Yit1ðÞ+Xit1ðÞ,pi
ðÞ;
logit pi
ðÞ=b0+fi+ui
where Xit1ðÞis the total number of identified severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) cases in testing
site iup to time t–1,Yit1ðÞ+Xit1ðÞis the total number of administered tests in testing site iup to time t–1,pirepresents
the true but unobserved prevalence in testing site i,b0is an intercept parameter, fiis a spatial random effect that follows the
ICAR distribution, and uiis an exchangeable random effect (independent and normally distributed with variance
parameter, s2
u). Note that only previously sampled zones contribute data to the fitting of this model.
The intrinsic conditional autoregressive (ICAR) random effects are defined conditionally as
fijfi,s2
f;Normal Pn
j=1wijfj
Pn
j=1wij
,s2
f
Pn
j=1wij

where fi=f1,...,fi1,fi+1,...,fn
ðÞ
Tand wij describes the spatial proximity between spatial zones iand j(e.g., touching
borders, inverse distance weights) with wii =0for all i.
After fitting the model, we obtain posterior samples from f(pijXt1ðÞ,Yt1ðÞ) for each testing site (even those that have not
been visited yet) where Xt1
ðÞ
and Yt1
ðÞ
are the complete set of data from all zones (Xt1
ðÞ
=X1t1
ðÞ
,...,Xnt1
ðÞðÞ
T;
Yt1ðÞdefined similarly). We then randomly select a posterior sample from each testing site and define it as p
i
(t– 1).
Select testing site j = argmax
i
p
i
(t– 1).
Perform mBernoulli trials in testing site jand observe x
j
successes and (m–x
j
) failures.
Let X
j
(t)=X
j
(t–1)+x
j
and Y
j
(t)=Y
j
(t–1)+(m–x
j
).
For all zones ij, let X
i
(t)=X
i
(t– 1) and Y
i
(t)=Y
i
(t– 1).
End
Prior specifications:
b0;Normal 0,2:85ðÞ; results in Uniform(0,1) prior probabilities for each testing site a priori assuming no excess
variability in the data.
s2
f(variance parameter for the ICAR random effect) ;Inverse Gamma 3:00,2:00ðÞ.
s2
u(variance parameter for the exchangeable random effect) ;Inverse Gamma 3:00,2:00ðÞ.
Gonsalves et al. 5
We have now set up a fully functional prototype of a
web app to allow policymakers in Columbus to use
Thompson sampling to target SARS-CoV-2 testing in
the city (Figure 1).
It is straightforward to adapt the algorithm to other
jurisdictions, requiring only a set of candidate test loca-
tions with point-to-point distances between all locations.
Finally, the only inputs required for these bandit algo-
rithms between testing forays are the number of tests
performed on a given day in each location and the num-
ber of positive tests obtained on that day in that loca-
tion. This makes the algorithm easy to use by local
public health departments. The algorithm is simple
enough that it can make virtually instantaneous use of
new data to inform the deployment of testing units for
the next day’s effort. This would make this approach
best suited for rapid diagnostic tests, particularly rapid
lateral-flow antigen-based assays, which, although with
lower sensitivity than standard PCR, are well positioned
to detect individuals with high titers of SARS-CoV-2
and most likely to transmit in a given settings.
7
In fact,
while a pilot in Columbus, Ohio, is still in the planning
stage, we intend to use the BinaxNOW COVID-19 Ag
Card provided through our collaboration with the Ohio
Department of Health.
31
However, even with standard
PCR-based assays and the delays in reporting of SARS-
CoV-2 results in practice, these tools can be useful. In
this case, the updating of prior distributions for testing
locations will be lagged, so the allocation of new testing
assignments will only benefit from additional informa-
tion about potential yield of new diagnoses among loca-
tions as testing results from previous days become
available. Furthermore, bandit algorithms complement
test pooling strategies and can help to concentrate
Figure 1 Home page of the web app for targeting severe acute respiratory syndrome coronavirus 2 testing with mobile units
(https://netzissou.shinyapps.io/BanditDemo/).
6Medical Decision Making 00(0)
positives within pools to minimize the number of
second-round tests needed. Finally, although these algo-
rithms are simple to set up and run, the practical consid-
erations of deploying testing teams to multiple locations,
often shifting teams at least initially from day to day,
require the support and engagement of local public
health officials, the resources to mount and maintain a
mobile testing program over time, and a seamless inte-
gration of the algorithms into the normal workflow of
testing sites. In the context of SARS-CoV-2, ensuring
safety of those getting tested as well as staff at a given
testing location is also paramount.
7
Bandit algorithms could provide a useful, simple tool
to find needles in a haystack—the tens of thousands of
undiagnosed infections from coast to coast—when one
cannot test everyone, everywhere. As discussed above,
bandit algorithms are widely used to successfully address
the explore v. exploit dilemma in several other fields.
Deployment of bandit algorithms for SARS-CoV-2 may
provide useful answers, enable more cost-effective test-
ing, and offer a lifeline to policymakers trying to figure
out where to test next for SARS-CoV-2.
ORCID iDs
Gregg S. Gonsalves https://orcid.org/0000-0002-5789-9841
Suzan Iloglu https://orcid.org/0000-0003-1611-9850
Research Data
All code and data associated with the web app are available at
https://github.com/NetZissou/Bandit. The web app itself is
available at https://netzissou.shinyapps.io/BanditDemo/.
References
1. Sharma B, Mashal M. Covid-19: U.S. vaccinations
increase, but virus continues to spread. New York Times.
2021 Mar 19. Available from: https://www.nytimes.com/
live/2021/03/19/world/covid-vaccine-coronavirus-cases
2. Phillips N. The coronavirus is here to stay—here’s what
that means. Nature. 2021;590(7846):382–4.
3. Murray J, Cohen AL. Infectious disease surveillance. In:
Quah SR, ed. International Encyclopedia of Public Health.
2nd ed. Oxford, UK: Academic Press; 2017. p 222–9.
Available from: https://www.sciencedirect.com/science/
article/pii/B9780128036785005178
4. Clark E, Chiao EY, Amirian ES. Why contact tracing
efforts have failed to curb coronavirus disease 2019
(COVID-19) transmission in much of the United States.
Clin Infect Dis. 2021;72(9):e415–9.
5. Lokuge K, Banks E, Davis S, et al. Exit strategies: optimis-
ing feasible surveillance for detection, elimination, and
ongoing prevention of COVID-19 community transmis-
sion. BMC Med. 2021;19(1):50.
6. Ferguson N, Laydon D, Nedjati Gilani G, et al. Report 9:
impact of non-pharmaceutical interventions (NPIs) to reduce
COVID19 mortality and healthcare demand. 2020. https://
spiral.imperial.ac.uk/handle/10044/77482.
7. Network for Regional Healthcare Improvement. Off-site
COVID-19 testing toolkit. Available from: https://
www.nrhi.org/offsite-testing-toolkit/
8. Bassett IV, Govindasamy D, Erlwanger AS, et al. Mobile
HIV screening in Cape Town, South Africa: clinical impact,
cost and cost-effectiveness. PLoS One. 2014;9(1):e85197.
9. Rubin R. Have tent, will do pop-up HIV screening.
JAMA. 2018;320(20):2063–5.
10. Knight V, Gale M, Guy R, et al. A novel time-limited pop-
up HIV testing service for gay men in Sydney, Australia,
attracts high-risk men. Sex Health. 2014;11(4):345–50.
11. UC San Francisco. Rapid COVID-19 test shows promise
in community test setting. Available from: https://
www.ucsf.edu/news/2020/10/418761/rapid-covid-19-test-
shows-promise-community-test-setting
12. Burns DN, DeGruttola V, Pilcher CD, et al. Toward an
endgame: finding and engaging people unaware of their
HIV-1 infection in treatment and prevention. AIDS Res
Hum Retroviruses. 2014;30(3):217–24.
13. Eyawo O, Hogg RS, Montaner JS. The Holy Grail: The
search for undiagnosed cases is paramount in improving
the cascade of care among people living with HIV. Can J
Public Health. 2013;104(5):e418.
14. Stout RL, Rigatti SJ. Seroprevalence of SARS-CoV-2 anti-
bodies in the US adult asymptomatic population as of Sep-
tember 30, 2020. JAMA Netw Open. 2021;4(3):e211552.
15. Kevadiya BD, Machhi J, Herskovitz J, et al. Diagnostics for
SARS-CoV-2 infections. Nat Mater. 2021;20(5):593–605.
16. Li Z, Chen Q, Feng L, et al. Active case finding with case
management: the key to tackling the COVID-19 pandemic.
Lancet. 2020;396(10243):63–70.
17. Wilasang C, Sararat C, Jitsuk NC, et al. Reduction in effec-
tive reproduction number of COVID-19 is higher in coun-
tries employing active case detection with prompt isolation.
J Travel Med. 2020;27(5):taaa095.
18. Kretzschmar ME, Rozhnova G, van Boven M. Isolation
and contact tracing can tip the scale to containment of
COVID-19 in populations with social distancing. Front
Phys. 2021;8:677.
19. Havlir D, Lockman S, Ayles H, et al. What do the Univer-
sal Test and Treat trials tell us about the path to HIV epi-
demic control? J Int AIDS Soc. 2020;23(2):e25455.
20. Wee S-L, Wang V. Here’s how Wuhan tested 6.5 million
for coronavirus in days. New York Times. 2020 May 26.
Available from: https://www.nytimes.com/2020/05/26/world/
asia/coronavirus-wuhan-tests.html
21. Benzell SG, Collis A, Nicolaides C. Rationing social con-
tact during the COVID-19 pandemic: transmission risk and
social benefits of US locations. Proc Natl Acad Sci U S A.
2020;117(26):14642–4.
Gonsalves et al. 7
22. Althouse BM, Wenger EA, Miller JC, et al. Stochasticity
and heterogeneity in the transmission dynamics of SARS-
CoV-2. arXiv preprint arXiv:200513689. 2020.
23. Berry DA, Fristedt B. Bandit problems: sequential alloca-
tion of experiments. Available from: http://link.springer
.com/content/pdf/10.1007/978-94-015-3711-7.pdf
24. Gonsalves GS, Crawford FW, Cleary PD, Kaplan EH, Pal-
tiel AD. An adaptive approach to locating mobile HIV test-
ing services. Med Decis Making. 2018;38(2):262.
25. Gonsalves GS, Copple JT, Johnson T, Paltiel AD, Warren
JL. Bayesian adaptive algorithms for locating HIV mobile
testing services. BMC Med. 2018;16(1):155.
26. Thompson WR. On the likelihood that one unknown prob-
ability exceeds another in view of the evidence of two sam-
ples. Biometrika. 1933;25:285–94.
27. Davidson-Pilon C. Bayesian methods for hackers: prob-
abilistic programming and Bayesian inference. Available
from: https://books.google.com/books?hl=en&lr=&id=
rMKiCgAAQBAJ&oi=fnd&pg=PT14&dq=Probabilistic
+Programming+and+Bayesian+Methods+for+Hacke
rs:+Using+Python+and+PyMC.&ots=DLkm-jGzjs&sig=
xV7KouHpeL8hakwOCtP9d2pg2p8
28. Besag J, York J, Mollie
´A. Bayesian image restoration,
with two applications in spatial statistics. Ann Inst Stat
Math. 1991;43(1):1–20.
29. Mongey S, Pilossoph L, Weinberg A. Which Workers Bear
the Burden of Social Distancing Policies? (April 26, 2020).
University of Chicago, Becker Friedman Institute for Eco-
nomics Working Paper No. 2020-51. Available from:
https://ssrn.com/abstract=3586077 or http://dx.doi.org/
10.2139/ssrn.3586077
30. Liu T, Gong D, Xiao J, et al. Cluster infections play impor-
tant roles in the rapid evolution of COVID-19 transmis-
sion: a systematic review. Int J Infect Dis. 2020;99:374–80.
31. Pilarowski G, Marquez C, Rubio L, et al. Field perfor-
mance and public health response using the BinaxNOWä
rapid severe acute respiratory syndrome coronavirus 2
(SARS-CoV-2) antigen detection assay during community-
based testing. Clin Infect Dis. 2020 Dec 26. Available from:
https://doi.org/10.1093/cid/ciaa1890
8Medical Decision Making 00(0)
... How to optimize resource allocation over time is a well-studied problem in sequential decision-making and reinforcement learning. The introduction of a spatial component to these kinds of dilemmas has been applied in a variety of settings, from military search and rescue to oil exploration [8]. We have previously described the use of one set of tools, bandit algorithms, to address these kinds of problems for detection of HIV and SARS-CoV-2 in the community [8][9][10]. ...
... The introduction of a spatial component to these kinds of dilemmas has been applied in a variety of settings, from military search and rescue to oil exploration [8]. We have previously described the use of one set of tools, bandit algorithms, to address these kinds of problems for detection of HIV and SARS-CoV-2 in the community [8][9][10]. Up until now, these methods have only been evaluated in computer simulations. ...
... The algorithm used to direct pop-up SARS-CoV-2 testing for this project has been described in detail elsewhere [8][9][10]. The algorithm is based on Thompson sampling, which uses a Bayesian updating process involving iteratively sampling from prior probability distributions of all potential testing sites-the set of all locations at which testing is being considered-to home in on those with the highest probability over the long run in finding new cases of SARS-CoV-2 [16,17]. ...
Article
Full-text available
Background The Flexible Adaptive Algorithmic Surveillance Testing (FAAST) program represents an innovative approach for improving the detection of new cases of infectious disease; it is deployed here to screen and diagnose SARS-CoV-2. With the advent of treatment for COVID-19, finding individuals infected with SARS-CoV-2 is an urgent clinical and public health priority. While these kinds of Bayesian search algorithms are used widely in other settings (eg, to find downed aircraft, in submarine recovery, and to aid in oil exploration), this is the first time that Bayesian adaptive approaches have been used for active disease surveillance in the field. Objective This study’s objective was to evaluate a Bayesian search algorithm to target hotspots of SARS-CoV-2 transmission in the community with the goal of detecting the most cases over time across multiple locations in Columbus, Ohio, from August to October 2021. Methods The algorithm used to direct pop-up SARS-CoV-2 testing for this project is based on Thompson sampling, in which the aim is to maximize the average number of new cases of SARS-CoV-2 diagnosed among a set of testing locations based on sampling from prior probability distributions for each testing site. An academic-governmental partnership between Yale University, The Ohio State University, Wake Forest University, the Ohio Department of Health, the Ohio National Guard, and the Columbus Metropolitan Libraries conducted a study of bandit algorithms to maximize the detection of new cases of SARS-CoV-2 in this Ohio city in 2021. The initiative established pop-up COVID-19 testing sites at 13 Columbus locations, including library branches, recreational and community centers, movie theaters, homeless shelters, family services centers, and community event sites. Our team conducted between 0 and 56 tests at the 16 testing events, with an overall average of 25.3 tests conducted per event and a moving average that increased over time. Small incentives—including gift cards and take-home rapid antigen tests—were offered to those who approached the pop-up sites to encourage their participation. ResultsOver time, as expected, the Bayesian search algorithm directed testing efforts to locations with higher yields of new diagnoses. Surprisingly, the use of the algorithm also maximized the identification of cases among minority residents of underserved communities, particularly African Americans, with the pool of participants overrepresenting these people relative to the demographic profile of the local zip code in which testing sites were located. Conclusions This study demonstrated that a pop-up testing strategy using a bandit algorithm can be feasibly deployed in an urban setting during a pandemic. It is the first real-world use of these kinds of algorithms for disease surveillance and represents a key step in evaluating the effectiveness of their use in maximizing the detection of undiagnosed cases of SARS-CoV-2 and other infections, such as HIV.
... Other concentrated efforts consisted of finding optimal testing strategies that inform epidemic dynamics (Chatzimanolakis et al., 2020) and helping to reduce disease spread (Biswas et al., 2020;Jonnerby et al., 2020;Gonsalves et al., 2021;Du et al., 2021). In particular, Jonnerby et al. (2020) focuses on optimal allocations designed as a combination of group and segmented testing; segments of the population based on occupation, age and geographical location are given testing priority. ...
... In particular, Jonnerby et al. (2020) focuses on optimal allocations designed as a combination of group and segmented testing; segments of the population based on occupation, age and geographical location are given testing priority. Both Biswas et al. (2020) and Gonsalves et al. (2021) advocate for contextual bandits as a possible approach to the optimal testing allocation, with Biswas et al. (2020) additionally suggesting an utility-based active learning solution. On the other hand, Du et al. (2021) develop a probabilistic framework accounting for resource limitations, imperfect testing and the need for prioritizing higher risk patient populations. ...
Preprint
Strategic test allocation plays a major role in the control of both emerging and existing pandemics (e.g., COVID-19, HIV). Widespread testing supports effective epidemic control by (1) reducing transmission via identifying cases, and (2) tracking outbreak dynamics to inform targeted interventions. However, infectious disease surveillance presents unique statistical challenges. For instance, the true outcome of interest - one's positive infectious status, is often a latent variable. In addition, presence of both network and temporal dependence reduces the data to a single observation. As testing entire populations regularly is neither efficient nor feasible, standard approaches to testing recommend simple rule-based testing strategies (e.g., symptom based, contact tracing), without taking into account individual risk. In this work, we study an adaptive sequential design involving n individuals over a period of {\tau} time-steps, which allows for unspecified dependence among individuals and across time. Our causal target parameter is the mean latent outcome we would have obtained after one time-step, if, starting at time t given the observed past, we had carried out a stochastic intervention that maximizes the outcome under a resource constraint. We propose an Online Super Learner for adaptive sequential surveillance that learns the optimal choice of tests strategies over time while adapting to the current state of the outbreak. Relying on a series of working models, the proposed method learns across samples, through time, or both: based on the underlying (unknown) structure in the data. We present an identification result for the latent outcome in terms of the observed data, and demonstrate the superior performance of the proposed strategy in a simulation modeling a residential university environment during the COVID-19 pandemic.
Article
The COVID-19 pandemic has highlighted the need for increased and more dynamic access to healthcare resources. It has also revealed a novel complication to the effective delivery of health resources to communities, which we call the final inch problem. In our recent COVID-19 pop-up testing work with Columbus Public Health and the Ohio National Guard, we have observed that, even when a healthcare-related service is transported directly to community members, it is not a given that they will use that service. We argue that crossing this final inch will require us to reframe public health initiatives through the lens of joint activity: a partnership between healthcare institutions and the public. Our work focuses on three questions. How do we engage with the public and foster common ground between people and our healthcare providers? As part of this, how can we work with the community to determine where to dynamically direct our resources on a given day? Finally, when we show up at the “right” place, will the community join us? Our recent work creating and deploying the Flexible Algorithmic, Adaptive Surveillance Testing (FAAST) has generated promising insights to answer these questions. Throughout our initial tests, we observed a continuous increase in community participation as well as increased positivity through multiple iterations of the program. We consistently overrepresented traditionally underserved minority groups in all testing locations as well. Insights for convincing communities to participate in pop-up testing may yield repeatable, generalizable strategies by which public health officials and healthcare providers may cross the final inch. Through establishing and nurturing reliable community relationships, public health institutions working in partnership with their constituent communities can proactively monitor the health of their communities, thereby facilitating a more resilient response to emerging threats.
Article
Full-text available
This cross-sectional study of a sample of healthy adults assesses the seroprevalence of SARS-CoV-2 antibodies in the US asymptomatic population as of September 30, 2020.
Article
Full-text available
Background Following implementation of strong containment measures, several countries and regions have low detectable community transmission of COVID-19. We developed an efficient, rapid, and scalable surveillance strategy to detect remaining COVID-19 community cases through exhaustive identification of every active transmission chain. We identified measures to enable early detection and effective management of any reintroduction of transmission once containment measures are lifted to ensure strong containment measures do not require reinstatement. Methods We compared efficiency and sensitivity to detect community transmission chains through testing of the following: hospital cases; fever, cough and/or ARI testing at community/primary care; and asymptomatic testing; using surveillance evaluation methods and mathematical modelling, varying testing capacities, reproductive number (R) and weekly cumulative incidence of COVID-19 and non-COVID-19 respiratory symptoms using data from Australia. We assessed system requirements to identify all transmission chains and follow up all cases and primary contacts within each chain, per million population. Results Assuming 20% of cases are asymptomatic and 30% of symptomatic COVID-19 cases present for testing, with R = 2.2, a median of 14 unrecognised community cases (8 infectious) occur when a transmission chain is identified through hospital surveillance versus 7 unrecognised cases (4 infectious) through community-based surveillance. The 7 unrecognised community upstream cases are estimated to generate a further 55–77 primary contacts requiring follow-up. The unrecognised community cases rise to 10 if 50% of cases are asymptomatic. Screening asymptomatic community members cannot exhaustively identify all cases under any of the scenarios assessed. The most important determinant of testing requirements for symptomatic screening is levels of non-COVID-19 respiratory illness. If 4% of the community have respiratory symptoms, and 1% of those with symptoms have COVID-19, exhaustive symptomatic screening requires approximately 11,600 tests/million population using 1/4 pooling, with 98% of cases detected (2% missed), given 99.9% sensitivity. Even with a drop in sensitivity to 70%, pooling was more effective at detecting cases than individual testing under all scenarios examined. Conclusions Screening all acute respiratory disease in the community, in combination with exhaustive and meticulous case and contact identification and management, enables appropriate early detection and elimination of COVID-19 community transmission. An important component is identification, testing, and management of all contacts, including upstream contacts (i.e. potential sources of infection for identified cases, and their related transmission chains). Pooling allows increased case detection when testing capacity is limited, even given reduced test sensitivity. Critical to the effectiveness of all aspects of surveillance is appropriate community engagement, messaging to optimise testing uptake and compliance with other measures.
Article
Full-text available
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has spread to nearly every corner of the globe, causing societal instability. The resultant coronavirus disease 2019 (COVID-19) leads to fever, sore throat, cough, chest and muscle pain, dyspnoea, confusion, anosmia, ageusia and headache. These can progress to life-threatening respiratory insufficiency, also affecting the heart, kidney, liver and nervous systems. The diagnosis of SARS-CoV-2 infection is often confused with that of influenza and seasonal upper respiratory tract viral infections. Due to available treatment strategies and required containments, rapid diagnosis is mandated. This Review brings clarity to the rapidly growing body of available and in-development diagnostic tests, including nanomaterial-based tools. It serves as a resource guide for scientists, physicians, students and the public at large.
Article
Full-text available
SARS-CoV-2 has established itself in all parts of the world, and many countries have implemented social distancing as a measure to prevent overburdening of health care systems. Here we evaluate whether and under which conditions containment of SARS-CoV-2 is possible by isolation and contact tracing in settings with various levels of social distancing. To this end we use a branching process model in which every person generates novel infections according to a probability distribution that is affected by the incubation period distribution, distribution of the latent period, and infectivity. The model distinguishes between household and non-household contacts. Social distancing may affect the numbers of the two types of contacts differently, for example while work and school contacts are reduced, household contacts may remain unchanged. The model allows for an explicit calculation of the basic and effective reproduction numbers, and of exponential growth rates and doubling times. Our findings indicate that if the proportion of asymptomatic infections in the model is larger than 30%, contact tracing and isolation cannot achieve containment for a basic reproduction number ( ℛ 0 ) of 2.5. Achieving containment by social distancing requires a reduction of numbers of non-household contacts by around 90%. If containment is not possible, at least a reduction of epidemic growth rate and an increase in doubling time may be possible. We show for various parameter combinations how growth rates can be reduced and doubling times increased by contact tracing. Depending on the realized level of contact reduction, tracing and isolation of only household contacts, or of household and non-household contacts are necessary to reduce the effective reproduction number to below 1. In a situation with social distancing, contact tracing can act synergistically to tip the scale toward containment. These measures can therefore be a tool for controlling COVID-19 epidemics as part of an exit strategy from lock-down measures or for preventing secondary waves of COVID-19.
Article
Full-text available
Among 3,302 persons tested for SARS-CoV-2 by BinaxNOW TM and RT-PCR in a community setting, rapid assay sensitivity was 100%/98.5%/89% using RT-PCR Ct thresholds of 30, 35 and none. The specificity was 99.9%. Performance was high across ages and those with and without symptoms. Rapid resulting permitted immediate public health action.
Article
Full-text available
Objectives To summarize the major types of cluster infections of SARS-CoV-2 all over the world through a comprehensive systematic review. Methods We searched all of the studies published between January 1, 2020 and June 15, 2020, on the cluster infections of COVID-19 in the English electronic databases including PubMed, Embase, Web of Knowledge, and Scopus. All included studies were independently screened and evaluated by two authors, and information of each study was extracted using a standard form. Results A total of 65 studies were included in this study which involved 108 cluster infections from 13 countries, areas or territories. Out of the cluster infections, 72(66.7%) were reported in China. The major types of cluster infections include family cluster, community transmission, nosocomial infection, transmission in gathering activities, on transportations, in shopping malls, on conference, among tourists, in religious organizations, among workers, in prisons, office, and in nursing home. Conclusions The SARS-CoV-2 can be transmitted in various circumstances, and cluster infection plays important roles in the rapid evolution of COVID-19 transmission. Prevention and control measures such as social distance must be strictly implemented to contain the cluster infections.
Article
Full-text available
Countries that implemented liberal testing with active case finding and prompt isolation, combined with contact tracing and quarantine, were more successful in reducing the reproduction number compared to countries that primarily relied on social distancing and lockdown measures.
Article
Full-text available
To prevent the spread of coronavirus disease 2019 (COVID-19), some types of public spaces have been shut down while others remain open. These decisions constitute a judgment about the relative danger and benefits of those locations. Using mobility data from a large sample of smartphones, nationally representative consumer preference surveys, and economic statistics, we measure the relative transmission reduction benefit and social cost of closing 26 categories of US locations. Our categories include types of shops, entertainments, and service providers. We rank categories by their trade-off of social benefits and transmission risk via dominance across 13 dimensions of risk and importance and through composite indexes. We find that, from February to March 2020, there were larger declines in visits to locations that our measures indicate should be closed first.
Article
A Nature survey shows many scientists expect the virus that causes COVID-19 to become endemic, but it could pose less danger over time. A Nature survey shows many scientists expect the virus that causes COVID-19 to become endemic, but it could pose less danger over time.
Article
By late April 2020, public discourse in the U.S. had shifted toward the idea of using more targeted case-based mitigation tactics (e.g., contact tracing) to combat COVID-19 transmission while allowing for the safe "re-opening" of society, in an effort to reduce the social, economic, and political ramifications associated with stricter approaches. Expanded tracing-testing efforts were touted as a key solution that would allow for a precision approach, thus preventing economies from having to shut down again. However, it is now clear that many regions of the U.S. were unable to mount robust enough testing-tracing programs to prevent major resurgences of disease. This viewpoint offers a discussion of why testing-tracing efforts failed to sufficiently mitigate COVID-19 across much of the nation, with the hope that such deliberation will help the U.S. public health community better plan for the future.