HIV drug resistance testing: is the evidence really there?

Abstract

To assess the strength of evidence supporting the routine use of HIV drug resistance testing.
A critical review of all studies relating to the clinical utility of HIV resistance testing, with a focus on randomized trials.
Two cohort studies found no evidence of a difference in virological response in patients who had resistance testing compared with matched controls. We identified nine published randomized trials that were specifically designed to assess the clinical utility of drug resistance testing. In a meta-analysis of these trials, resistance testing increased the proportion of patients who achieved undetectable viral load by an average of 7% (95% confidence interval: 3-11%). However, this may be an over-estimate of the impact of resistance testing in clinical practice because of the idealized design and analytical approaches used in most of the studies.
The available evidence does not clearly demonstrate that HIV drug resistance testing is clinically effective. To optimize their value for health decision-making, future trials of HIV resistance testing should be carefully designed to mimic the circumstances of routine clinical practice.

Full text

Page 1

Objectives: To assess the strength of evidence supporting
the routine use of HIV drug resistance testing.
Design: A critical review of all studies relating to the clin-
ical utility of HIV resistance testing, with a focus on
randomized trials.
Results: Two cohort studies found no evidence of a
difference in virological response in patients who had
resistance testing compared with matched controls. We
identified nine published randomized trials that were
specifically designed to assess the clinical utility of drug
resistance testing. In a meta-analysis of these trials,
resistance testing increased the proportion of patients
who achieved undetectable viral load by an average of
7% (95% confidence interval: 3–11%). However, this may
be an over-estimate of the impact of resistance testing in
clinical practice because of the idealized design and
analytical approaches used in most of the studies.
Conclusions: The available evidence does not clearly
demonstrate that HIV drug resistance testing is clinically
effective. To optimize their value for health decision-
making, future trials of HIV resistance testing should be
carefully designed to mimic the circumstances of routine
clinical practice.
Review
HIV drug resistance testing: is the evidence really
there?
David T Dunn1*, Diana M Gibb1, Abdel G Babiker1, Hannah Green1, Janet H Darbyshire1and Ian VD
Weller2
1MRC Clinical Trials Unit, London, UK
2Department of Sexually Transmitted Diseases, Royal Free and University College Medical School, University College London, UK
*Corresponding author: Tel: +44 20 7670 4739; Fax: +44 20 7670 4818; E-mail: d.dunn@ctu.mrc.ac.uk
Antiviral Therapy 9:641–648
The development of resistance to antiretroviral drugs is
considered to be an important cause of treatment
failure in HIV infection. Several clinical guidelines have
recommended that drug resistance testing should be
used every time a patient switches antiretroviral
therapy to inform the selection of the new regimen
[1,2]. However, with direct, commercial costs of
around US$500 per genotypic test and US$900 per
phenotypic test (genotypic tests detect mutations in the
viral genome associated with resistance to specific
drugs; phenotypic tests measure drug susceptibility
directly but are less widely available and take longer to
perform) [3], decisions about whether to routinely use
resistance testing should be based on sound evidence
[4–6]. It is widely held that persuasive evidence already
exists; indeed one paper claimed ‘...there is no longer
any doubt about the advantage for clinical care
provided by drug-resistance testing’ [7]. The evidence
base for this assertion consists of retrospective studies
that have correlated virological response with baseline
resistance patterns, cohort studies with matched
controls and randomized trials specifically designed to
assess the clinical utility of resistance testing [7–9]. In
this review we critically assess the robustness of this
evidence, with a focus on the randomized studies.
Review of studies
We searched for relevant studies and randomized
trials through Medline (using search expression: exp
HIV and exp DRUG RESISTANCE), by discussion
with colleagues, by examining proceedings of major
HIV conferences and checking reference lists in review
articles [8–10].
Correlational studies
The presence of drug-associated mutations or reduced
phenotypic sensitivity has been found to correlate with
short-term virological response to a new antiretroviral
regimen in many studies [2,7,8]. In most of these
studies, the resistance test was performed retrospec-
tively on a stored plasma sample and therefore did not
Introduction
©2004 International Medical Press 1359-6535/02/$17.00 641

Page 2

influence the selection of the new regimen. Intuitively,
knowledge of the result of the resistance test in ‘real
time’ would have resulted in the avoidance of drugs
with reduced antiviral efficacy and thus in the selection
of a more potent regimen. However, this intuition is
over-simplistic for several reasons, including test inac-
curacy [11], the insensitivity of tests to detect minority
viral subspecies [4], evidence that drugs can exert a
strong effect despite the presence of key mutations [12]
and the ability to partly infer resistance from drug
history alone [8]. Although correlational studies play
an important role in increasing our understanding of
HIV drug resistance, they do not provide any direct
evidence that resistance testing is beneficial since they
lack a control group of patients who were not tested.
This is a fundamental limitation not shared by the
other types of study considered below.
Cohort studies
We identified two observational cohort studies that
assessed the effect of drug resistance testing on virolog-
ical and immunological response to a new antiretroviral
regimen.Inonestudy,495patientswhohadagenotypic
resistance test at treatment initiation or switch were
identified from a hospital database [13]. These patients
were matched on a number of factors, including
calendar time at antiretroviral initiation/switch, with
1–4 control patients who did not have a resistance test.
The resistance testing group was subdivided according
to whether or not at least three drugs to which the virus
was still sensitive could be used in the new regimen as
indicated by the resistance test. As expected, the group
with less resistance had a better virological response
than the group with more resistance, with the control
group showing an intermediate response. The key
analysis, a direct comparison of the control group and
the entire resistance testing group, was not shown, but
the data suggest very similar virological and immuno-
logical responses over 96 weeks of follow-up.
A similar design was used for subjects in the HIV
Ontario Observational Database, for whom the
Ontario Central Public Health Laboratory is the sole
provider of HIV viral load and genotypic resistance
testing [14]. In this study, cases (n=198) were defined
as patients whose plasma sample was sent for a viral
load measurement and a resistance test (freely available
at the request of the clinician) and controls whose
plasma sample was sent for a viral load measurement
only (n=328). Thus, patients were not necessarily
switching drugs at the point of study entry. The two
groups were matched for calendar date, number of
previous antiretroviral drugs and HIV viral load. The
main outcome measure was whether an undetectable
viral load (that is, below the sensitivity threshold of the
assay) was ever achieved 3–9 months after the index
test. Counter-intuitively, the proportion of subjects
who achieved this endpoint was lower in the resistance
testing group than in the control group (adjusted odds
ratio (OR)=0.42, all subjects; adjusted OR=0.70,
subgroup analysis of subjects who switched antiretro-
viral regimen), although these differences were not
statistically significant.
Although these two studies suggest that resistance
testing conferred no benefit, observational studies are
generally difficult to interpret because of potential
selection bias – patients who received resistance testing
may not be strictly comparable with contemporaneous
patients who did not. Although matching can partially
reduce selection bias, randomization is the only
method to exclude it with certainty.
Randomized trials
We identified nine published randomized controlled
trials that were specifically designed to assess the clin-
ical utility of drug resistance testing (Table 1). Eight of
the nine studies shared the same main design features
(Figure 1): (i) enrolment of patients about to switch
therapy due to therapeutic failure, (ii) a viral load
above a specified value (generally 1000–2000
copies/ml) because of requirements of resistance
assays, (iii) random allocation to have or not have a
resistance test at baseline to guide the selection of the
new drug regimen, and (iv) primary outcome measure
of attained viral load or change in viral load from base-
line. The studies differed in inclusion criteria (including
previous drug exposure), the use of genotypic or
phenotypic testing, the availability of repeat testing
after the initial test, the use of an expert panel to advise
on the selection of the new regimens and the duration
of follow-up (Table 1). Trials comparing genotypic and
phenotypic testing without a no-test group have not
been included.
Asummaryeffectofresistancetestingwasobtainedby
combining within-study estimates using the metan func-
tion (fixed effects) in Stata Release 8.0 (Stata
Corporation, College Station, Tex., USA). This excluded
theCERTtrial(seenext page)and,inthemainanalysis,
the genotypic and phenotypic groups in the NARVAL
study were combined. In a supplementary analysis,
separate estimates were derived for trials of genotypic
resistance testing and trials of phenotypic testing.
Randomized trials – results
The proportion of control group subjects with unde-
tectable viral load levels at the end of follow-up
(between 12–26 weeks) ranged from 14–48% (Table 1,
Figure 2). The general trend was towards increased
viral load suppression in the resistance testing group,
although four of the studies found little or no differ-
ence. On average, the rate of viral load undetectability
DT Dunn et al.
©2004 International Medical Press
642

Page 3

Antiviral Therapy 9:5
643
Effectiveness of HIV drug resistance testing
Table 1. Summary of randomized studies of the clinical utility of HIV resistance testing
Participants
Resistance testing
Expert advice on
Minimum VL at
Latest endpoint,
Percent with undetectable
Mean increase in CD4 count
Study
enrolled*
Intervention
after baseline
new regimen
entry, copies/ml
weeks†
VL‡
(cells/µl) from baseline
VIRADAPT [15]
108
GENO
3-monthly
Selected patients in
10 000
48 (26)
CONTROL 14, GENO 32
CONTROL 33, GENO 21 (NS)
both groups§
(P=0.07)
CPCRA 046 [16]
153
GENO
–
GENO group only
10 000
12
CONTROL 22, GENO 34
CONTROL 18, GENO 25
(P=0.10)
(P=0.67)
Melnick et al. [17]
115
PHENO
–
No
2000
16
Similar virological response
CONTROL 11, PHENO 53
(P=0.01)
ANRS 088,
541
GENO or
3-monthly
No
1000
36 (12)
CONTROL 19, PHENO 20
CONTROL 27,
NARVAL [18]
PHENO
(P=0.94), GENO 25 (P=0.27)
PHENO 40 (P=0.47), GENO 14
(P=0.45)
VIRA3001 [19]
272
PHENO
–
No
2000
16
CONTROL 34, PHENO 46
[Median] CONTROL 40,
(P=0.08)
PHENO 27 (P=0.77)
HAVANA [20]
274
GENO
–
Factorial design
1000
24
CONTROL 36, GENO 48
Not reported
(P=0.03)
ARGENTA [21]
174
GENO
–
Both groups
2000
26
CONTROL 17, GENO 21
CONTROL 22, GENO 15 (NS)
(P=0.47)
CCTG 575 [22]
238
PHENO
Whenever VL
Not stated
400
52 (26)
CONTROL 48, PHENO 48
Not reported
>400 copies/ml
(NS)
CERT [23]
450
GENO or
Whenever VL
No
None
52–156
Failure time endpoint (see text)
Not reported
PHENO
>1000 copies/ml
*Equal allocation to intervention groups except VIRADAPT (imbalance in favour of GENO group). †Value in parenthesis is latest reported or interpretable endpoint (used in the next two columns). ‡Threshold varied between 200–500
copies/ml. §Patients who failed more than one previous regimen. VL, viral load; GENO, genotypic testing; PHENO, phenotypic testing; NS, not significant.

Page 4

was 7% (95% CI: 3–11%) higher in the resistance
testing group compared with the control group, with
no indication of between-study heterogeneity (P=0.43).
A larger effect was observed for genotypic testing (8%,
95% CI: 3–14%) than for phenotypic testing (2%,
95% CI: –3–8%), but this difference is not statistically
significant (P=0.14). Only one study reported an effect
of resistance testing on immunological response as
measured by CD4 T-cell count [19].
The design of one study, the CERT trial, differed
from the others in several key aspects [23]. In this trial,
patients receiving combination antiretroviral therapy
were randomized to one of three study arms (routine
access to genotypic testing, routine access to pheno-
typic testing or no access to resistance testing)
irrespective of whether the patient was experiencing
therapeutic failure at the time. Testing was then avail-
able at each routine visit at which the viral load
exceeded 1000 copies/ml. The primary endpoint was a
composite one: time to second virological failure (a
measure of treatment durability), study discontinua-
tion or loss to follow-up. The median times to this
endpoint were similar (between approximately 600
and 700 days), and differences were not statistically
significant in the three study arms. Disconcertingly, the
number of subjects who discontinued participation
was similar to the number who reached the primary
efficacy endpoint.
Randomized trials – methodological considerations
Designing a trial to measure the clinical impact of a
diagnostic test is inherently more complex than
designing a conventional therapeutic trial, stemming
from the fact that the intervention does not have an
effect per se but merely provides information that may
influence clinical decisions [5,6]. It has been argued
that the trial design should pre-specify a clear link
between the test results and clinical actions contingent
upon the results [6]. Whilst this is a desirable principle
and improves the external validity of the trial, it is
impracticable in the context of HIV resistance testing
for two main reasons. Firstly, many other considera-
tions (for example, previous adverse drug reactions,
adherence, patient choice) influence the selection of the
new drug regimen. Secondly, as HIV resistance testing
is a comparatively new technology, the interpretation
of results, even among experts in the field, is constantly
evolving. In the absence of a clear scientific hypothesis
DT Dunn et al.
©2004 International Medical Press
644
Patient about to change drugs
following therapeutic failure
Randomize
Resistance test
Change drugs based on test
results and clinical history
Follow-up (12-26 weeks) and assess outcome by viral load endpoint
Change drugs based on
clinical history alone
No resistance test
Figure 1. Trial schema

Page 5

it is appropriate to design and analyse trials using a
‘pragmatic’ approach [24], that is, the resistance tests
should be used within the trial in a way that mirrors
clinical practice as closely as possible. In contrast, most
trials were designed and analysed in ways that may
have over-estimated the impact of testing.
Time point at randomization
Withoneexception[23],randomizationwasperformed
in all trials after a decision to switch therapy had been
made. However, in clinical practice the results of resis-
tance tests may be used, in conjunction other
information, to guide the decision as to whether to
switch therapy. In the HIV Ontario Observational
Database study, only 40% of resistance tests led to a
treatment switch [14]. Thus, as well as influencing the
selection of individual drugs, resistance testing could
affect the frequency of switching of therapy. To date, no
study has had sufficient follow-up to assess this effect.
Time origin for analysis
Following the standard analysis of therapeutic trials,
viral load was assessed at fixed times measured from
the date when therapy was switched. However, the
interval between blood collection and the reporting
of a resistance test (typically several weeks) meant
that the time from randomization to viral load
measurement tended to be longer in the testing group
than in the control group (Figure 3). This conflicts
with the principle that the time origin for analysis
should correspond to the point of randomization. An
alternative viewpoint is that the wait for the result is
an intrinsic part of the intervention which should not
be factored out. Insufficient information is provided
in the study reports to assess whether this was an
important bias, although it is of more concern the
shorter the follow-up. To circumvent this difficulty, a
number of studies performed a resistance test on all
participants and only randomized after the report
had been produced, keeping the control group blind
to the results [16,18–20]. However, this procedure
could be criticized as having artificially delayed the
change in therapy in the control group. A related
issue is the conflict between the desirability of
switching treatment as soon as viral load is
confirmed as detectable and the need to wait until
viral load is sufficiently high (median entry criterion
of 2000 copies/ml across trials) to be confident of
successful PCR amplification.
Interpretation systems and expert advice
To be clinically useful, the raw output of a genotypic
resistance test (a nucleotide sequence) needs to be
translated in terms of viral susceptibility to individual
drugs. Each trial used its own algorithm or interpreta-
tion system, which adds to the problem of
generalizability and further complicates comparisons
between studies. Also, phenotypic assays use different
cut-off levels to indicate ‘sensitive’ and ‘resistant’
viruses, a factor which could influence whether a drug
is prescribed. A more controversial issue is the use of
an expert panel to advise on the new drug regimen. In
some studies, this advice was given to participants in
both groups (based on drug history alone in the control
group) so that concern is about generalizability rather
than bias [15,21]. In one trial, the panel recommended
regimens for the resistance testing group only, making
it impossible to separate the effect of the resistance test
from that of the expert advice [16]. 74% of partici-
pants in the genotypic group were prescribed four or
more drugs compared with 46% in the control group –
was this due to resistance testing per se or because the
expert panel favoured a comparatively aggressive ther-
apeutic approach? A factorial design was used in one
study, thereby allowing separation of the two effects;
resistance testing but not expert advice was found to
confer a virological advantage, with no evidence of
statistical interaction [20].
Antiviral Therapy 9:5
645
Effectiveness of HIV drug resistance testing
Figure 2. Proportion of participants in resistance testing
(vertical axis) and control groups (horizontal axis) with an
undetectable HIV viral load after treatment switch
Time point ranges from 12–26 weeks after the treatment switch and the viral
load limit of detection between 200–500 copies/ml (see Table 1). The area of
the circle is proportional to number of participants enrolled. The 3-group ANRS
088 study [18] is represented twice; the CERT study [23] and the study of
Melnick et al. [17] are not represented because relevant quantitative results are
not available.
60
30
40
50
20
10
0
Control: undetectable VL (%)
Testing > control
010 20304050 60
Resistance testing: undetectable VL (%)
Control > testing
Phenotypic test
Genotypic test