Another look at Emergency Department HIV screening in practice: no need to revise expectations.

Abstract

A recent study reported a lower than expected specificity and positive predictive value of the rapid oral HIV test in the setting of routine emergency department (ED) screening. These results appeared inconsistent with the findings in another urban Emergency Department during the same time period.
To compare the specificity and positive predictive vale (PPV) of an oral rapid HIV test used in an ED screening program in Washington DC with that performed in the USHER clinical trial.
Period cross-sectional analysis of rapid oral HIV testing conducted in an ongoing HIV screening program emergency department patients. Setting: The George Washington University Emergency Department (Washington DC) from 7 February to 1 October 2007. Patients: 1,560 adults seen in the ED for non-HIV-related presenting complaints, who participated in the HIV screening program. Intervention: Rapid HIV testing with the OraQuick ADVANCE Rapid HIV-1/2 Antibody Test (OraSure Technologies, Bethlehem, Pennsylvania). Patients with reactive rapid test results were offered Western blot testing for confirmation. Measurements: Specificity and positive predictive value for the program were determined. Findings were compared to those found in the USHER trial.
Of 1,560 patients screened for HIV, 13 [0.8%, 95% CI 0.38% to 1.28%] had a reactive HIV screening test, and all were confirmed to be positive by Western Blot. The specificity was 100% (95% CI 99.6%-100%). Limitation: Since non-reactive tests were not confirmed, the test sensitivity cannot be determined.
Review of our data conflict with findings from the USHER study surrounding false positive OraQuick HIV screening. Our data suggest that rapid HIV screening protocols implemented in EDs outside of the clinical trial paradigm perform effectively without an excess of false positive results. Compared with other screening tests, HIV rapid screening should remain an essential component of ED practice.

Full text

Page 1

RESEARCHOpen Access
Another look at Emergency Department HIV
screening in practice: no need to revise
expectations
Jeremy Brown1*, Manya Magnus2, Maggie Czarnogosrki3, Vanessa Lee1
Abstract
Background: A recent study reported a lower than expected specificity and positive predictive value of the rapid
oral HIV test in the setting of routine emergency department (ED) screening. These results appeared inconsistent
with the findings in another urban Emergency Department during the same time period.
Objective: To compare the specificity and positive predictive vale (PPV) of an oral rapid HIV test used in an ED
screening program in Washington DC with that performed in the USHER clinical trial.
Design: Period cross-sectional analysis of rapid oral HIV testing conducted in an ongoing HIV screening program
emergency department patients.
Setting: The George Washington University Emergency Department (Washington DC) from 7 February to 1
October 2007.
Patients: 1,560 adults seen in the ED for non-HIV-related presenting complaints, who participated in the HIV
screening program.
Intervention: Rapid HIV testing with the OraQuick ADVANCE Rapid HIV-1/2 Antibody Test (OraSure Technologies,
Bethlehem, Pennsylvania). Patients with reactive rapid test results were offered Western blot testing for
confirmation.
Measurements: Specificity and positive predictive value for the program were determined. Findings were
compared to those found in the USHER trial.
Results: Of 1,560 patients screened for HIV, 13 [0.8%, 95% CI 0.38% to 1.28%] had a reactive HIV screening test, and
all were confirmed to be positive by Western Blot. The specificity was 100% (95% CI 99.6%-100%).
Limitation: Since non-reactive tests were not confirmed, the test sensitivity cannot be determined.
Conclusion: Review of our data conflict with findings from the USHER study surrounding false positive OraQuick
HIV screening. Our data suggest that rapid HIV screening protocols implemented in EDs outside of the clinical trial
paradigm perform effectively without an excess of false positive results. Compared with other screening tests, HIV
rapid screening should remain an essential component of ED practice.
Background
Since the Centers for Disease Control and Prevention
(CDC) recommended routine opt-out HIV screening in
emergency departments in 2006,[1] the George
Washington Emergency Department has been conduct-
ing routine HIV testing among all patients that present
for care[2]. This clinical program (rather than a research
trial) is funded by the DC Office of HIV/AIDS, the CDC
and unrestricted pharmaceutical grants. It operates in a
real-world setting among a diverse population of
patients seeking ED care.
In August 2008, the investigators for the Universal
Screening for HIV infection in the Emergency Room
(USHER) Trial published a report which described a lar-
ger than expected number of false positive tests [3].
* Correspondence: jbrown@mfa.gwu.edu
1Department of Emergency Medicine, The George Washington University
Medical Center, Washington DC, USA
Brown et al. AIDS Research and Therapy 2010, 7:1
http://www.aidsrestherapy.com/content/7/1/1
© 2010 Brown et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons
Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in
any medium, provided the original work is properly cited.

Page 2

This led the authors to suggest that expectations from
rapid HIV testing in the emergency department should
be revised. As evidence of the importance of this paper,
the journal in which these results were published
printed a one page patient summary [4]. The USHER
Trial analyzed data from 849 patients between 2/7/07
and 10/1/07, and found, of those who allowed confirma-
tory Western Blot testing, that 26 of their 39 prelimin-
ary positive results were not confirmed positive; this
revealed a false positive rate that exceeded expectations
that were based on parameters listed within the Ora-
Quick product insert. The purpose of this study was to
examine the impact of a real-world, non-clinical trial ED
HIV screening program during the same time span and
to examine the prevalence of false positives.
Methods
Clinical protocol and test site
The George Washington University Hospital is an aca-
demic urban medical centre in the District of Columbia.
The annual census in the emergency department is over
60,000. Of these, 53% are African-American and 33%
are white. Unlike the USHER trial, in which participants
were consented to participate in the study and then ran-
domized to receive rapid HIV testing in the ED by a
dedicated HIV counsellor or by a staff member at the
ED, all GWU patients between the ages of 13 and 64
years presenting to the emergency department are eligi-
ble for an HIV screening test if they speak either English
or Spanish. Patients who know they are HIV positive,
who have been tested for HIV in the last three months,
who have an altered mental status, or who require
urgent medical intervention are excluded from screen-
ing. Screening was offered by trained additional staff
and two screeners were assigned to periods of peak
activity in the ED. Screeners were available from 8 am-
midnight every day. The screening staff was made up of
extensively trained undergraduate health sciences
students.
Patients that were not critically ill, whether they were
ambulatory patients or had arrived by ambulance were
informed of the availability of a free HIV screening test
at the point of triage. They were given written informa-
tion about HIV disease, and were informed of the
importance of HIV testing by the triage nurse. However
the triage nurse did not ask the patient to whether or
not they would accept an HIV test when offered to
them. HIV screeners then approached the patients and
offered them a rapid HIV screening test. Patients who
accepted screening were tested with an oral swab using
the OraQuick ADVANCE Rapid HIV-1/2 Antibody test
(OraSure Technologies Inc, Bethlehem PA). Testing was
performed in parallel to the provision of standard ED
care. Results were available within 20-40 minutes and
negative results were relayed to the patient by the
screener. All patients with a negative screening test were
given HIV risk reduction materials, and the results were
noted on the ED record. All positive screening results
were reviewed by a second screener and the ED attend-
ing physician. If there were consensus surrounding the
positive result, the ED attending physician informed the
patient of the test result as well as the preliminary nat-
ure of the screening result in a confidential area. Unlike
the clinical trial setting found in the USHER study,
laboratory investigation with HIV RNA PCR and CD4
counts were not available to further assess patients. A
local testing algorithm was implemented to avert incon-
clusive and false positive results: patients who had a
weakly positive test were screened a second time using
whole blood. If the result were positive twice, it was
recorded as a preliminary positive test result in the ED
records. All patients with a preliminary positive test
were offered a Western Blot confirmatory test. They
were also linked to care with the hospital’s Division of
Infectious Diseases clinic where the results of the confir-
matory Western Blot test were disclosed, and further
care was arranged as needed. Data on age, gender, race,
zip code of residence, acceptance or refusal of HIV test-
ing, and the test results were collected for all patients by
the screening personnel.
To explore the possibility of a cluster of inaccurate
OraQuick test kits or other barriers to effective ED
screening, we reviewed the outcomes of the tests from 7
February to 1 October 2007, the same time span
reported in the USHER Trial paper [3]. This allows for
comparison between two sites as well as two testing
algorithms during the same time.
Role of the funding sources
Since its inception, the GWU ED HIV testing program
has been supported by the DC Department of Health,
the Centers for Disease Control, and from unrestricted
grants from Gilead Sciences. The decision about which
test kit to use was made by the DC Dept of Health,
which then provided the kits to the testing site. None of
these organizations had any role in the study design and
interpretation of the results, or in the decision to submit
the manuscript for publication.
Evaluation of the screening performance
The specificity of the screening test is defined as the
proportion of screened patients who had a negative
rapid test among non-HIV infected patients. The posi-
tive predictive value (PPV) is calculated as the propor-
tion of patients with a preliminary reactive rapid test
that are infected with HIV as confirmed by Western
Blot. Sensitivity cannot be calculated given that the
negative population seen in the GWU ED, as with the
USHER trial, do not contribute specimens for confirma-
tory testing, and patients with a negative screening test
Brown et al. AIDS Research and Therapy 2010, 7:1
http://www.aidsrestherapy.com/content/7/1/1
Page 2 of 6

Page 3

were not retested and did not have confirmatory Wes-
tern Blot or other HIV-related measures collected. Both
are consistent with standard HIV screening practice. All
95% confidence intervals were calculated on the basis of
the normal approximation of the binomial distribution
[5]. In order to assess test parameters under a variety of
assumptions regarding the proportion of both false
negatives and false positives, a sensitivity analysis was
performed. The scenarios looked at the resulting sensi-
tivity, specificity, and PPV had there been 0, 1, and 28
(1.2%) false negatives, 0, 2 (0.95% overall HIV preva-
lence), and 4 (1.1% overall HIV prevalence) additional
false positives among the negatives, ad the combination
of each of these. Data were analyzed using Stata version
9.0SE (College Station, TX).
Results
Sample
From 2/7/07 to 10/1/07, there were 41,889 visits to the
ED. Of these, 3,163 patients were offered an HIV
screening test, representing 7.5% of all ED patients seen
over this period of. A total of 1,560 (49%) agreed to be
tested and all received their results in the ED (Figure 1).
Of those tested, 1,547 were negative [99.2%, 95% CI
98.7% to 99.6%] and 13 [0.8%, 95% CI 0.38% to 1.28%]
were preliminary positive. There were no invalid tests.
Patients with a reactive test were more likely to be male
(p < 0.05) and black (p < 0.10), as shown in Table 1.
Otherwise, the demographic characteristics between
patients with a reactive and a non-reactive result were
similar.
Test performance
All 13 patients (out of 1,560 tested) with a reactive test
agreed to confirmatory testing, and all were confirmed
positive by Western Blot. Therefore, the occult HIV pre-
valence rate was 0.8% during the study period [95% CI
0.38% to 1.28%]. Assuming that the nonreactive tests
were truly negative, the test specificity was 100% (95%
CI 99.8%-100%). Although the observed sensitivity and
observed false negative rates could not be calculated
because non-reactive tests were not confirmed with a
Western Blot, a series of scenarios were developed to
place bounds on what might be expected. While specifi-
city remains constant in the face of false negatives, sen-
sitivity with 0, 1, and 28 false negatives ranges from
100.0% to 31.7% at each extreme. The primary concern
that observed negatives could have been false positives
(or that the use of the enhanced testing algorithm to
retest weak results could have obscured such a finding)
was explored by looking at sensitivity and specificity for
0, 2, and 4 additional false positives (out of observed
negatives). In this case, the sensitivity reduces to 86.7%
under the assumption of 2 false positives and 1.2% false
negatives. Specificity remains in the high 90% range
throughout (Table 2).
In order to expand the time period from that explored
by USHER, we also combined our sample described
here with our sample of patients previously described
elsewhere [6]. From the resulting combined sample of
4,046 patients at GWU, the specificity is 99.9% and the
PPV is 85% (Table 3).
Discussion
This study revealed that the oral rapid HIV screen per-
formed as expected, and that the number of false posi-
tive tests was as predicted by the manufacturer. The
earlier published report by Walensky of “a false-positive
rate fifteen times greater than the anticipated specificity
of 0.2%” was not supported by the evidence from our
clinical program involving almost twice as many
patients. Under extreme scenarios of false positives
among negatives as false negatives alone and combined,
the sensitivity remained at an acceptable level for a
rapid screening test administered in an ED setting, com-
parable with other screening tests used on a population
level.
There have been a number of reports addressing a
higher than expected false positive rate. In 2006 Delaney
reported the performance of the rapid oral HIV test in
four CDC studies and reported that there was a small
cluster of 16 false positive tests in one site [7]. This site
had a specificity of 99% which was lower than in the
other three studies, which reported a specificity of 99.6-
99.8%. Jafa et al described an increase in the false-posi-
tive rate that occurred in Minnesota in 2006 [8]. The
field investigation could not identify a cause for the
increase. More recently, a report in 2008 described two
episodes of an unexpected increase in the false positive
rate in New York City [9]. It remains unclear if these
Table 1 Characteristics of Patients tested for HIV
CharacteristicReactive test
(n = 13)
n(%)
34 (8)
11 (84)
Non reactive test
(n = 1547)
n(%)
35 (12.6)
681 (44)
Mean age (SD), yrs
Men, n (%)*
Race n(%)
White
Black**
Other
Ethnicity
Hispanic
Non-Hispanic
2 (15)
10 (77)
1 (8)
554 (36)
826 (53)
167 (11)
1 (7)
12 (93)
64 (4)
1483 (96)
*p < 0.05
**p < 0.10
Brown et al. AIDS Research and Therapy 2010, 7:1
http://www.aidsrestherapy.com/content/7/1/1
Page 3 of 6

Page 4

are randomly occurring clusters, user-end failure, test kit
failure, or other phenomena. Broader prospective studies
of the overall accuracy of the test are needed in real-
world settings, however, in order to fully understand
these clusters.
Limitations
The first explanation of these differences is that the two
sites use different screening protocols. At The George
Washington University ED, if the positive test line is
only weakly visible, the screeners are directed to ask the
patient for a sample of blood on which the test is run a
second time. If this second sample is positive (even if
weakly so) the test is considered to be reactive. If this
second sample is negative, the test is recorded as
negative. All reactive samples are considered preliminary
and require confirmation with a Western Blot. This
algorithm has been described elsewhere as likely to
reduce the number of false-positive cases [7,10]. It is
possible that the reactive results represent a subset of
those that would have been reported had we used the
algorithm in the product insert. However, since the
number of weakly reactive oral tests requiring a second
rapid test with whole blood is not recorded in our data-
base, we cannot determine the size of the total subset of
weakly reactive test. For this reason it is important to
use caution when comparing reports of testing out-
comes between sites, even if the product being used is
identical. Our sensitivity analysis attempts to measure
41,889
ED visits
1,603
patients declined testing
13 tested POSITIVE
Confirmed Negative
0
Confirmed Positive
13
1,547 tested NEGATIVE
1,560
patients accepted testing
3,163
patients offered
HIV screening
38,726 patients not tested:

Already known to be HIV positive
No screener available
Language barrier
Recently tested
Acute medical conditions taking
priority
Figure 1 Study flow.
Brown et al. AIDS Research and Therapy 2010, 7:1
http://www.aidsrestherapy.com/content/7/1/1
Page 4 of 6

Page 5

the impact of this testing algorithm and reveals that
false negatives and false positives would not have elimi-
nated the utility of the rapid HIV test except in the
most extreme cases.
Although the number of patients we report is consid-
erably larger than the USHER Trial report, the total
number of reactive results (13) was fewer than those
reported in the USHER Trial (39). Confidence intervals
for the frequency of false positives are wide for both set-
tings. In GW there were no false positives, with a 95%
CI of 0-4 cases. In the USHER Trial, there were 26 false
positives, with a 95% CI of 16-24 cases.
What are we to make of these findings? We believe
that they are best understood when compared to other
widely accepted screening tests that are performed. For
example the PPV of the newborn screen for hypothyr-
oidism is only 1.8%, and there are about 50 false positive
results for every true-positive result identified through
the US newborn screening program [11]. The false posi-
tive rate for a 40-year-old woman having her first mam-
mogram is 7-10% [12]. The PPV of first-screening
mammography (number of breast cancers detected per
abnormal examination) increases with age from 3% for
those aged 30 to 39 years up to 19% for those aged 70
years or older [13]. The USHER data demonstrated a
lower than expected specificity, but one that is neverthe-
less far higher than specificity of these other accepted
screening tests.
However, ED HIV screening is still in its infancy, and
many hospitals are still undecided about whether or not
to implement the CDC recommendation and screen ED
patients for HIV [14]. The prior published findings of
the USHER group are likely to dissuade such sites, given
their number of false positive screening tests (26/31)
and their low predictive values. Improved quality assur-
ance and immediate verification with a second rapid test
may have decreased the numbers of false positives in
the USHER Trial. Our protocol, and others that have
more recently been suggested [10] may be incorporated
into routine ED HIV testing with the aim of decreasing
the false positive rate. For this reason we believe that it
is critical that the test characteristics of rapid HIV
screening from other venues be reported and reviewed.
Conclusions
We remain enthusiastic about routine ED HIV screen-
ing. Our analysis of the test characteristics demonstrates
that the test outperforms many other commonly used
screening tests. Given the potential for improved indivi-
dual-level and population-level health as well as HIV
prevention through identification and treatment of HIV-
positive persons, there remains a great need for rapid
testing in EDs. The real challenge is not whether or not
the test performs as intended, but rather how to
increase the numbers of patients who agree to be
screened, which remains only 50-60% of all those eligi-
ble [15].
Author details
1Department of Emergency Medicine, The George Washington University
Medical Center, Washington DC, USA.2Department of Epidemiology and
Biostatistics, The George Washington University School of Public Health and
Table 2 Sensitivity Analysis
Assuming X of the negatives were actually false positives
0
(observed)
2
(0.95% prevalence
of positives overall)
Pr(+|A) 100%
(95% CI 75.3%-100%)
Pr(-|N) 99.9%
(95% CI 99.5%-100%)
Pr(A|+) 86.7%
(95% CI 59.5%-98.3%)
Pr(+|A) 92.9% (95%
CI 66.1%-99.8%)
Pr(-|N) 99.9%
(95% CI 99.5%-100%)
Pr(A|+) 86.7%
(95% CI 59.5%-98.3%)
Pr(+|A) 86.7%
(95% CI 59.5%-98.3%)
Pr(-|N) 100%
(95% CI 99.8%-100%)
Pr(A|+) 100%
(95% CI 75.3%-100%)
4
(1.1% prevalence
of positives overall)
Pr(+|A) 100%
(95% CI 75.3%-100%)
Pr(-|N) 99.7%
(95% CI 99.3%- 99.9%)
Pr(A|+) 76.5%
(95% CI 50.1%-93.2%)
Pr(+|A) 92.9%
(95% CI 66.1%-99.8%)
Pr(-|N) 99.7%
(95% CI 99.3%-99.9%)
Pr(A|+) 76.5%
(95% CI 50.1%-93.2%)
Pr(+|A) 24.3%
(95% CI 11.8%-41.2%)
Pr(-|N) 99.7%
(95% CI 99.3%-99.9%)
Pr(A|+) 69.2%
(95% CI 38.6%-90.9%)
False negatives
0
Pr(+|A) 100%
(95% CI 75.3%-100%)
Pr(-|N) 100%
(95% CI 99.8%-100%)
Pr(A|+) 100%
(95% CI 75.3%-100%)
Pr(+|A) 92.9%
(95% CI 66.1%-99.8%)
Pr(-|N) 100%
(95% CI 99.8%-100%)
Pr(A|+) 100%
(95% CI 75.3%-100%)
Pr(+|A) 31.7%
(95% CI 18.1%-48.1%)
Pr(-|N) 100%
(95% CI 99.8%-100%)
Pr(A|+) 100%
(95% CI 75.3%-100%)
1
28
(1.2% false negatives)
Brown et al. AIDS Research and Therapy 2010, 7:1
http://www.aidsrestherapy.com/content/7/1/1
Page 5 of 6

Page 6

Health Services, Washington DC, USA.3National Institute of Allergy
Infectious Diseases, Bethesda MD, USA.
Authors’ contributions
JB conceived the study and wrote the manuscript. MM performed the data
analysis and reviewed the manuscript. MC performed patient follow up
and reviewed the manuscript. VL performed data abstraction and
manuscript review.
All authors have read and approved the final manuscript.
Competing interests
JB has received lecture fees from OraSure.
Received: 15 July 2009
Accepted: 5 January 2010 Published: 5 January 2010
References
1.Branson BM, Handsfield HH, Lampe MA, et al: Revised recommendations
for HIV testing of adults, adolescents, and pregnant women in health-
care settings. MMWR Recomm Rep 2006, 55:1-17.
2.Brown J, Shesser R, Simon G, et al: Routine HIV screening in the
emergency department using the new US Centers for Disease Control
and Prevention Guidelines: results from a high-prevalence area. J
Acquir Immune Defic Syndr 2007, 46:395-401.
3.Walensky RP, Arbelaez C, Reichmann WM, et al: Revising expectations
from rapid HIV tests in the emergency department. Ann Intern Med
2008, 149:153-60.
4.
Rapid Oral Testing for HIV Infection. Annals of Internal Medicine 2008,
149:I-30.
5.Hanley JA, Lippman-Hand A: If nothing goes wrong, is everything all
right? Interpreting zero numerators. JAMA 1983, 249:1743-5.
6.Brown J, Simon G, Shesser R, Sikka N, Bahn M, Czarnogorski M: Opt-out
HIV Testing in the Emergency Department: Results from a High
Prevalence Setting Following the New CDC Guidelines. Acad Emerg Med
2007, 14:S14.
7.Delaney KP, Branson BM, Uniyal A, Kerndt PR, et al: Performance of an
oral fluid rapid HIV-1/2 test: experience from four CDC studies. AIDS
2006, 20(12):1655-60.
8. Jafa K, Patel P, Mackellar DA, et al: Investigation of false positive results
with an oral fluid rapid HIV-1/2 antibody test. PLoS One 2007, 2:e185.
9.
False-positive oral fluid rapid HIV tests–New York City, 2005-2008.
MMWR Morb Mortal Wkly Rep 2008, 57:660-5.
10.Bennett B, Branson B, Delaney K, Owen M, Pentella M, Werner B: HIV
Testing Algorithms: A Status Report. Silver Spring MD: Association of
Public Health Laboratories & The Centers for Disease Control & Prevention
2009.
11.Kwon C, Farrell PM: The magnitude and challenge of false-positive
newborn screening test results. Arch Pediatr Adolesc Med 2000, 154:714-
8.
12.Christiansen CL, Wang F, Barton MB, et al: Predicting the cumulative risk
of false-positive mammograms. J Natl Cancer Inst 2000, 92:1657-66.
13. Kerlikowske K, Grady D, Barclay J, Sickles EA, Eaton A, Ernster V: Positive
predictive value of screening mammography by age and family history
of breast cancer. JAMA 1993, 270:2444-50.
14.Kelen GD, Rothman RE: Emergency department-based HIV testing: too
little, but not too late. Ann Emerg Med 2009, 54:65-71.
15.Brown J, Kuo I, Bellows J, et al: Patient perceptions and acceptance of
routine emergency department HIV testing. Public Health Rep 2008,
123(Suppl 3):21-6.
doi:10.1186/1742-6405-7-1
Cite this article as: Brown et al.: Another look at Emergency
Department HIV screening in practice: no need to revise expectations.
AIDS Research and Therapy 2010 7:1.
Table 3 Oral rapid HIV test performance
Study
Total
tested
Negative
Reactive
(preliminary
positive)
Lost to
follow-up
True
Positive
False
Positive
95% CI for
false
positives
Specificity
PPV
Unconfirmed cases included as
HIV Negative
Specificity
HIV
Positive
Specificity
HIV Negative
PPV
Walensky [3]BOSTON
849
810
39
8
5
26
16-35
96.89%
16.13%
95.97%
96.89%
12.82%
Brown
DC
1560
1547
13
0
13
0
0-4
100%
100%
100%
100%
100%
Brown [9]
DC
2486
2460
26
13
9
4
1-8
99.84%
69.23%
99.31%
99.84%
40.91%
DC
Totals only
4046
4007
39
13
22
4
1-8
99.90%
84.62%
99.58%
99.90%
56.41%
Brown et al. AIDS Research and Therapy 2010, 7:1
http://www.aidsrestherapy.com/content/7/1/1
Page 6 of 6