Content uploaded by Chad Hazlett

Author content

All content in this area was uploaded by Chad Hazlett on Dec 11, 2018

Content may be subject to copyright.

J. Causal Infer. 2018; 20180019

Chad Hazlett*

https://doi.org/10.1515/jci-2018-0019

Received July 8, 2018; accepted November 9, 2018

Providing terminally ill patients with access to experimental treatments, as allowed by recent “right

to try” laws and “expanded access” programs, poses a variety of ethical questions. While practitioners and

investigators may assume it is impossible to learn the eects of these treatment without randomized trials,

this paper describes a simple tool to estimate the eects of these experimental treatments on those who take

them, despite the problem of selection into treatment, and without assumptions about the selection process.

The key assumption is that the average outcome, such as survival, would remain stable over time in the ab-

sence of the new treatment. Such an assumption is unprovable, but can often be credibly judged by reference

to historical data and by experts familiar with the disease and its treatment. Further, where this assumption

may be violated, the result can be adjusted to account for a hypothesized change in the non-treatment out-

come, or to conduct a sensitivity analysis. The method is simple to understand and implement, requiring just

four numbers to form a point estimate. Such an approach can be used not only to learn which experimental

treatments are promising, but also to warn us when treatments are actually harmful – especially when they

might otherwise appear to be benecial, as illustrated by example here. While this note focuses on experi-

mental medical treatments as a motivating case, more generally this approach can be employed where a new

treatment becomes available or has a large increase in uptake, where selection bias is a concern, and where

an assumption on the change in average non-treatment outcome over time can credibly be imposed.

Non-randomized trials, Observational studies, Clinical trials

On 30th May 2018, the United States established a federal “right to try” law, allowing terminally ill patients

to access experimental medical treatments that have cleared Phase 1 testing but were not yet approved by

the Food and Drug Administration (FDA).1Such laws extend pre-existing methods of gaining access to un-

approved treatments through “compassionate use” or “expanded access” programs, but sidestepping FDA

petition and oversight procedures. Numerous ethical objections have been made to these laws, pointing to

risks of dangerous side-eects that could shorten or worsen lives, raising false hopes among vulnerable pa-

tients, enabling quackery, undermining the FDA, and imposing a nancial burden on desperate patients and

their families.

Can we – and should we – learn anything about the ecacy and safety of drugs from those taking such

experimental treatments? The rst reaction of clinicians and statisticians alike may be an emphatic “no”:

without randomization into treatment and control groups, individuals will self-select into treatment, thus

little can be learned from the observational results. Indeed, one criticism of these laws have been that they

“will only make it more dicult to know if medication is eective or safe [1].

1This law federally enshrines analogous rights previously recognized by 40 US states.

Departments of Statistics and Political Science, University of California Los Angeles,

Los Angeles, United States, e-mail: chazlett@ucla.edu, URL: http://www.chadhazlett.com

Brought to you by | University of California - Los Angeles - UCLA Library

Authenticated

Download Date | 12/11/18 9:04 PM

| C. Hazlett, Estimating Causal Eects of New Treatments Despite Self-Selection

Yet, failing to carefully assess the eects of these treatment would both forgo the opportunity to learn

which are promising, and perhaps worse, could greatly amplify their harm. Clinicians and investigators will

surely make inferences about the eects of new treatments by comparing, at least casually, those who receive

them and those who do not. Estimates based on comparisons of this type are not just biased, they are dan-

gerous. As illustrated below, through the eects of self-selection, treatments that are actually harmful may

appear to be benecial in such naive comparisons, perversely encouraging more patients to take them.

We thus have a responsibility to learn the benets and harms of such treatments, and to avoid that worst

errors that naive comparisons would generate. Fortunately, a very simple technique outlined here allows

valid estimates of treatment eects under such circumstances in which individuals self-select into treatment

in unknown ways. Of course, such inference is not free. The critical assumption required is that in the absence

of the new treatment of interest, the average outcome would remain stable or change by a specied amount

across two time periods (one before the treatment is available, and one after). While not true in every case, this

assumption is straightforward to understand and can often be assessed by experts familiar with treatment of

the disease in question.

While this note takes experimental medical treatments as an illustration and motivation, the method

described here applies to a wide variety of circumstances where a treatment becomes newly available, in-

dividuals or units opt into taking that treatment, and the assumption of stability in average non-treatment

outcomes over time is reasonable.2The method may also be useful even in cases where a randomized trial

is possible or has been conducted, but that trial had strict eligibility criteria or low willingness to consent to

randomization, resulting in an estimate that may not generalize well to the population that would actually

elect to take the treatment.

Perhaps surprisingly, we can make inferences about how those who take a new treatment benet from it even

when patients select into taking treatment in unknown ways. Consider person i, with some observed outcome

Yi, such as whether or not they survive at one year post diagnosis. Using the potential outcomes framework[2]

and assuming no interference, we consider not only the observed outcome for person i, but also two potential

outcomes – the outcome she would have experienced had she been treated, Yi(1), and the outcome she would

have experienced had she not been treated at Yi(0).

We next consider how the average outcome people would experience without the treatment, 피[Yi(0)](or

simply 피[Y(0)]) changes from the time period before the new treatment becomes available to the time period

after,

피[Y(0)|T=1]−피[Y(0)|T=0]=δ(1)

where T=0 designates a time period before a treatment (D) is available, T=1 is a time period after it is made

available. Note that T∈{0,1}designates time periods or windows – perhaps a few years wide – and not single

points in time. This is important, as Yimay take time to measure (e. g. the proportion surviving for one year

post-diagnosis), and treatments may take time to take eect.

The core assumption required is on the value of δ. For simplicity of exposition and because it is likely

the primary use case, we consider rst the assumption that δ=0, which we call the stability of the aver-

age non-treatment outcome assumption. However, alternative assumed values of δcan be employed. Within

this setting, besides the practical assumption that at least some eligible individuals take the new treatment,

2Some examples of other cases where this approach may be applicable and where randomization is dicult or impossible in-

clude: What is the eect of a drug newly being used o-label by select physicians to treat a disease? What is the eect of warnings

sent to a patient’s doctor by a health monitoring system? Outside of medicine, what is the eect of an early release program from

jail, assigned by judges, on recidivism of those who are released? What is the eect of legal aid, given to those who most need

it, on legal outcomes? What is the eect of a television program on behaviors of those who choose to watch it, or the eect of

advertisements on those who receive them?

Brought to you by | University of California - Los Angeles - UCLA Library

Authenticated

Download Date | 12/11/18 9:04 PM

C. Hazlett, Estimating Causal Eects of New Treatments Despite Self-Selection |

Pr(D=1|T=1)>0, only an assumption on δis required to identify the average treatment eect among the

treated. No assumption is required on the treatment assignment mechanism.

Showing this identication result is straightforward: weknow the average non-treatment outcome among

the whole group in period one, because it is equal to the mean observed (non-treatment) outcome in period

zero (shifted by δif δ̸= 0). This group average is, in turn, a weighted combination of two other averages:

the average non-treatment outcome among the untreated, which we observe, and the average non-treatment

outcome among the treated, for which we can solve. That is, the average (non-treatment) outcome among the

non-treated, combined with knowledge of the average non-treatment outcome among the whole group, tells

us what the average non-treatment outcome among treated must be. Formally, by applying the law of iterated

expectations,

피[Y(0)|T=0]=피[Y(0)|T=1]−δ

=피[Y(0)|D=1,T=1]Pr(D=1|T=1)

+피[Y(0)|D=0,T=1]Pr(D=0|T=1)−δ

which we can re-arrange to identify 피[Y(0)|D=1,T=1]in terms of observables,

피[Y(0)|D=1,T=1]=피[Y(0)|T=0]−피[Y(0)|D=0,T=1]Pr(D=0|T=1)+δ

Pr(D=1|T=1)

=피[Y|T=0]−피[Y|D=0,T=1]Pr(D=0|T=1)+δ

Pr(D=1|T=1)(2)

Finally, the Average Treatment Eect on the Treated (ATT) is the dierence between the treatment and non-

treatment potential outcomes, taken solely among the treated, i. e. 피[Y(1)|D=1,T=1]−피[Y(0)|D=1,T=1].

While we directly observe an estimate of the rst quantity, the second term – the average outcome among the

treated had they not taken the treatment – has now been given by the strategy above (Equation 2). We have

thus identied the ATT,

ATT =피[Y(1)|D=1,T=1]−피[Y(0)|D=1,T=1]

=피[Y|D=1,T=1]−피[Y|T=0]−피[Y|D=0,T=1]Pr(D=0|T=1)+δ

Pr(D=1|T=1)(3)

When the stability assumption is maintained, δ=0 and can thus be simply removed. In that case, despite

appearing quite dierent, this estimator is equivalent to an instrumental variables approach in which “time”

is the instrument and non-compliance is one-sided (see Discussion).

A number of extensions are possible. First, investigators could use this tool to examine “what-if” scenarios:

if clinicians have beliefs about the four quantities required here or evidence from past cases, we can compute

ATT estimates to understand the underlying eect implied by those beliefs. Such an analysis is informal and

only as good as the guesses that are used as data. However, it correctly produces an ATT estimate subject

to those guesses, avoiding the errors that result from naive comparisons that may otherwise be made. An

illustration of such errors is given below.

Second, the assumption that δ=0 can be replaced with one that allows a hypothetical or modeled

change in the average non-treatment outcome over time such that δ̸= 0. One application for this is sensitiv-

ity analysis: we can hypothesize shifts in the average non-treatment outcome and compute the corresponding

eect, repeating this for dierent hypothesized shifts. This allows us to ask “how large a shift in the average

non-treatment outcome must be permitted in order for our conclusions about the benet (or harmfulness)

of a treatment to change?” If a seemingly implausible shift is required to change our substantive conclu-

sion – e. g. that non-treatment outcomes improved by 20 % despite no known changes in treatment protocols

or compositional shifts in those who get the disease – then we would be able to rule out such concerns. These

Brought to you by | University of California - Los Angeles - UCLA Library

Authenticated

Download Date | 12/11/18 9:04 PM

| C. Hazlett, Estimating Causal Eects of New Treatments Despite Self-Selection

analyses may or may not prove informative, but are an improvement over the naive comparisons that may

otherwise be attempted. Alternatively, the stability assumption can be replaced with a known or estimated

shift in non-treatment outcomes. If, for example, there are known compositional shifts or changes in available

care besides the treatment of interest, and if we can model these or make reasonable estimates of how they

may change average non-treatment outcomes, we can use this information to adjust the resulting treatment

eect estimate for the treatment of interest.

A simple illustration can demonstrate the method and how it can avoid the most dangerous errors that may

arise due to direct comparisons that practitioners may be tempted to make. Suppose an aggressive form of

cancer, once diagnosed at a certain stage, has only a 50% one-year survival rate as measured over a period

from 2005 to 2010 (

피[Y(0)|T=0]=0.5, where

피[⋅] designates a sample average). Between 2012 and 2016

suppose a new treatment becomes available, and that 30% of the group diagnosed with this cancer after

2010 attempt this new treatment (

Pr(D=1|T=1)=0.3). Among this group, suppose that the one-year

survival rate is 60 % (

피[Y(1)|D=1,T=1]=0.6), while the one-year survival rate among those who chose not

to take the treatment is only 40% (

피[Y(0)|D=0,T=1]=0.4). Under the stability assumption, we set δ=0

and can estimate the expected non-treatment outcome among those who took the treatment:

피[Y(0)|T=1,D=1]=피[Y|T=0]−피[Y|D=0,T=1]Pr(D=0|T=1)

Pr(D=1|T=1)

≈0.5−0.4(0.7)

0.3=0.73

To verify this result while reinforcing the intuition: rst, under the stability assumption, if we could see

how everybody in the later group fairs in the absence of the treatment, we know that (up to nite sample

error) 50 % would survive at one year. Among the 70% who did not take the treatment, only 40 % survive at

one year. This “drop” in survival rate among the non-treated signals that it must have been those who were

worse-o who chose not to take the treatment. Consequently, those who do take the treatment must have

had higher (non-treatment) survival in order to bring this 40 % up to the required 50 % for the whole group.

Specically, the 30 % of the group who took the treatment must have had an average non-treatment survival

of xin the equation (0.4)(0.7)+(.3)x=.5, which solves to x=0.73.

The observed survival rate of 60% under treatment thus no longer appears favorable compared to the

73 % who would have survived without treatment in this group, yielding an estimated ATT of 60% −73 % =

−13 %. This nding of a harmful eect of treatment emphasizes both that this technique can return counter-

intuitive results, and the ethical imperative to understand the impacts of experimental treatments. We em-

phasize that naive comparisons tell the opposite story: The treatment at rst appears benecial, with higher

survival among those who took the drug compared to those before the drug existed (60 % versus 50 %), and

higher survival among those who took it versus those who did not in the second period (60% versus 40 %).

This may persuade practitioners to recommend it, and patients to take it. Yet, it actually reduces survival by

13 % among those who take it. It would seem unethical not to make this information available to practitioners,

patients, and regulators. Furthermore, if the assumption that δ=0 cannot condently be defended, sensitiv-

ity analysis using a range of values for δwill characterize what we would conclude for any given assumption

of δ.

While motivated here by the problem of experimental medical treatments, this approach is quite general in

its applicability to situations in which new treatments become available and individuals self-select to receive

Brought to you by | University of California - Los Angeles - UCLA Library

Authenticated

Download Date | 12/11/18 9:04 PM

C. Hazlett, Estimating Causal Eects of New Treatments Despite Self-Selection |

them. The main concern investigators must keep in mind when choosing to apply this method is whether they

can justify the stability assumption, or employ some value of δother than zero. Fortunately, the stability of

non-treatment outcomes is a straightforward assumption to understand, and can be wellevaluated by experts

in many cases. It is most plausible, rst, if little has changed in the way the disease is treated over the time

in question. This includes any other treatments that might be administered or withdrawn due to taking the

new treatment in question. Second, a more subtle concern lies in compositional changes in the population

who acquires the disease, such as changes in population health or competing risks, as these could also drive

changes in the average non-treatment outcome. That said, if such compositional shifts do occur, they are

likely to be slow-moving and so may be possible to rule out as problematic in the short-run. While no test

can prove that the stability assumption (or any other assumption on δ) holds, investigators can check the

stability of average outcomes over the course of many years prior to the introduction of the new treatment,

which can boost the credibility that it remains stable thereafter. Altogether, if the outcome has been stable

for many years prior to the introduction of the treatment in question, and there are no known changes in the

use of other treatments or sudden changes in the composition of the group with the disease, then a strong

case can be made for the stability of the average non-treatment outcome.

Pragmatically, only four numbers are required to form a point estimate: the estimated average outcome

prior to treatment (피[Y|T=0]), the estimated average outcome among the treated and untreated after intro-

duction of the treatment (피[Y|D=1,T=1]and 피[Y|D=0,T=1]respectively), and the proportion who took

the treatment after introduction, Pr(D=1|T=1). In some cases, the required data may thus be available from

existing sources, such as electronic medical records. In other cases, investigators may choose to run a trial of

this type by design.

One alternative approach worth mentioning but less often applicable would be possible when we have a

disease such that the prognosis in the absence of any new treatment is virtually certain, and thus the non-

treatment outcome one would normally learn from a control group is already known. Suppose that nearly all

individuals with a certain cancer at a certain stage die within one year (and those whose cancers do remit,

if any, show no signs of their potential for remittance until it happens and thus would not have any basis

for self-selecting into treatment). If a group – selected by any means – takes a new medication and has a

50 % survival rate at one year, then the improvement can reasonably be attributed to the new treatment, as

we believe we know how those individuals would have faired under non-treatment, despite the absence of

a control group. While possibly workable in some scenarios, such an approach is limited to cases where the

outcome is nearly certain. By contrast, the approach here is more general, and recognizes that when outcomes

are uncertain (such as a 50 % one year survival rate), there is non-trivial scope for self-selection.3

Second, this method may bear a resemblance to the Dierence-in-Dierence (DID) approach, but can

operate in circumstances where DID is not possible, and provides a relaxation of DID in cases where it is pos-

sible. To conduct DID, we need either to measure each unit before and after (some are exposed to) treatment,

or we must be able to place individuals into larger groupings that persist over time (such as states), with treat-

ment being assigned at the level of those larger units at time T=1. By contrast, the present method works

even when there is no way to know if an individual observed at time T=0 would have chosen treatment had

they appeared at time T=1. This is useful in cases such as new medical treatments: among those diagnosed

with a given disease during T=0, there is no way to say which would have taken the treatment at time T=1,

as would be required by DID.

The method is thus particularly useful where DID is not possible, however in arrangements where DID

is possible (such as panel data), it provides an “adjustable” version of DID that allows prescribed deviations

3Another approach that may be feasible in some circumstances occurs when a new treatment becomes available and all the

individuals under study take it, leaving no control group at time T=1. In this case, the method proposed here specializes to a

simple cross-sectional “post-minus-pre” estimator, where identication is still achieved by assuming δ.

Brought to you by | University of California - Los Angeles - UCLA Library

Authenticated

Download Date | 12/11/18 9:04 PM

| C. Hazlett, Estimating Causal Eects of New Treatments Despite Self-Selection

from the parallel trends assumption. Specically, DID requires the parallel trends assumption, 피[Y(0)|T=

1,D=1]−피[Y(0)|T=0,D=1]=피[Y(0)|T=1,D=0]−피[Y(0)|T=0,D=0], whereas the present

method assumes 피[Y(0)|T=1]−피[Y(0)|T=0]=δfor some choice δ. To understand the connection,

consider that there are two ways to support a particular assumption of δ. The trend in average non-treatment

outcomes over time could be dierent for the would-be-treated group and the would-be-control group, with

the average of these trends (weighted by their population proportions) amounting to δ. Alternatively, we may

propose a given δby assuming that both the would-be-treated and would-be-control group changed by δ,

in turn ensuring that the average 피[Y(0)] across the two also changes by δ. This more restrictive claim is

precisely the parallel trends assumption. Further, if we do assume parallel trends, this means we can learn

the appropriate δfrom the change over time in the control group alone. Setting δto the estimated change

in the control group,

피[Y(0)|T=1,D=0]−

피[Y(0)|T=0,D=0], returns a value exactly equal to the DID

estimate. However, if we wish to make any assumption other than parallel trends, this method allows it: any

trends in the average non-treatment outcomes hypothesized for the treated and control group implies a choice

of δthrough their weighted average.

Third and perhaps most illuminating, in the case when δ=0 this procedure is identical to an instru-

mental variables approach in which “time” is the instrument. In the framework of [3], those at T=1 are

“encouraged” to take the treatment by its availability. When we assume δ=0, we assume 피[Y(0)] does not

change over time – thus the only way that the instrument (time) can inuence outcomes is by switching

some individuals (“compliers”) into taking the treatment, satisfying the “exclusion restriction”. The reader

can verify that when δ=0 the estimator in (3) is numerically equal to the Wald estimator for instrumental

variables. The proportion who take the treatment at T=1 is the “compliance rate” or rst stage eect. Deers

are assumed not to exist, and because of the unavailability of the treatment at time T=0, non-compliance

is known to be one-sided. As a consequence, the eect among the compliers is simply the average treatment

eect on the treated. Going beyond the usual instrumental variable arrangement, when δ̸= 0 is employed,

this corresponds to allowing a prescribed violation of the exclusion restriction.4Accordingly, the required

assumptions for this method (under the δ=0 assumption) can be partially represented by a Directed Acylic

Graph (DAG) encoding an instrumental variable relationships, as in Figure 1 [4]. As with instrumental vari-

ables in general, the additional assumption of monotonicity or “no-deers” on the T→Drelationship is not

represented on the DAG but must be stated. The absence of an edge between Tand Yin this graph encodes the

exclusion restriction corresponding to the “δ=0” case, though the method can allow for δ̸= 0, not encoded

on the non-parametric DAG.

T D Y

Graphical representation of time as an instrument. Note: Instrumental variables representation of the identication

requirements. T∈{0,1}is the time period, D∈{0,1}is treatment status, and Yis the outcome. The required absence of deers,

and the possibility that δ̸= 0, are not shown.

While I am not aware of any empirical or theoretical work describing the identication logic used here,

the equivalence to using time as an instrument connects to an emergent set of medical studies in which the

uptake of new treatments increases dramatically over time [5, 6, 7, 8, 9].5The approach described here helps to

4The instrumental variables interpretation also suggests that this procedure could be used for treatments that were available in

time T=0 but whose uptake changed dramatically in T=1, in which case the Wald estimator can be used to rescale the change

in outcomes between T=0 and T=1 (i. e. the reducedform) by the change in treatment uptake, Pr(D=1|T=1)−Pr(D=1|T=0)

(i. e. the compliance rate). This may also be a reasonable strategy in some cases. In this case, however, the local average treatment

eect identied is no longer the ATT.

5Brookhart et al. [10] also provides a guide to the use of instrumental variables generally in comparative safety and eectiveness

research, with a brief section discussing the use of calendar time as an instrument.

Brought to you by | University of California - Los Angeles - UCLA Library

Authenticated

Download Date | 12/11/18 9:04 PM

C. Hazlett, Estimating Causal Eects of New Treatments Despite Self-Selection |

give clarity to the identication assumptions required of such work and how they can be judged. In addition,

when identication depends upon an assumption on δas described here, covariate adjustment procedures

become unnecessary, or require explicit justication in terms of identication. Rather, simpler analyses us-

ing Equation 3, together with discussions of plausible values of δare called for. Further, sensitivity analysis

based on a range of these δvalues can be a valuable addition to any such work.

Even when RCTs are possible, investigators may worry about two important representational limitations. For

example if only a small fraction of people with a given disease are eligible or willing to consent, then how

might this group be dierent from the ultimate target group who will use the treatment once approved? Clin-

ical designs that allow partial self-selection in an eort to address these concern include “comprehensive

cohort studies” [11] and “patient preference trials” [12]. In comprehensive cohort studies, it is proposed that

those patients who refuse randomization be allowed to instead join the study but with the treatment of their

preference. The randomized arms are compared as in any experiment. The outcomes (as well as the pre-

treatment characteristics) of those in the preference groups are hoped to improve our understanding of gen-

eralizability, but it is unclear how to make reliable use of the information provided by these preference groups

given confounding biases. In later work, “patient preference trials” encapsulate a variety of research designs

in which patients’ preferences are elicited, some individuals are randomized, and some receive a treatment

of their choosing. In the recent proposal of [13], treatment preferences are elicited from all individuals, who

are randomized into one group that will have their treatment assigned at random, and one that can choose

its own treatment. This design allows for sharp-bound identication and sensitivity analysis for the average

causal eects among those who would choose a given treatment.

The present method has the primary benet of sidestepping the need for randomization, still required by

the above designs. However, the allowance for self-selection alters the estimand in ways that may be prefer-

able or complementary to an RCT, depending on the goals of the study. First, in retrospective work, the ATT

identied here may be the ideal quantity of interest if we would like to know what eect a past treatment

actually had. Second, if our goals are more prospective, the ATT from this method may say more about the

potential eects of a new treatment in the clinical population likely to take it, if the RCT was highly restrictive

in its eligibility criteria, or suered low consent rates. On the other hand, the ATT may not be ideal to inform

policy making decisions – such as promoting a new treatment to become a rst line therapy – if the group

likely to take the treatment under such a policy varies widely from those who have elected to take in the study

period.

In summary, this note describes a simple identication procedure allowing estimation of the ATT regardless

of self-selection into the sample. The simplest assumption – stability of average non-treatment outcomes

(δ=0) – may be reasonable when we know that (a) the composition of those who acquire a certain condition

has not changed, and (b) the availability and use of treatments have not changed, except for the new treat-

ment of interest. Where investigators are uncertain that the average non-treatment outcomes have remained

stable over time, one can model or propose a non-zero change (δ̸= 0), or show the sensitivity of results to a

range of δvalues. In contrast to DID, the method works when dierent individuals are present in the two time

periods, without any indication of who in the earlier time period would have been exposed to treatment had

they been observed in the latter time period. In the δ=0 case it corresponds to using time as an instrument,

clarifying the assumptions and required analyses of such an approach. In the δ̸= 0 case, it allows a pre-

scribed deviation from the exclusion restriction, as well as a sensitivity analysis. While the most obvious use

of this method is when randomized trials are not possible, another potential benet regards representation

or external validity: The ATT estimated here may be more informative than the average eect from an RCT,

depending upon our scientic goals, and who was able and willing to participate in the RCT.

Brought to you by | University of California - Los Angeles - UCLA Library

Authenticated

Download Date | 12/11/18 9:04 PM

| C. Hazlett, Estimating Causal Eects of New Treatments Despite Self-Selection

This method is broadly applicable where treatment become newly available or popular, and an assump-

tion on the stability of average non-treatment outcomes can be credibly made. Returning to the motivating

case of “right to try” and other access to experimental medical treatments, the availability of this method does

not change the deep and dicult set of ethical questions that must be answered about when and whether an

experimental treatment should be made available. Rather, given the current laws, we must consider our ethi-

cal responsibility to learn what we can from such treatment regimes – not only to determine which therapies

are promising for further trials, but to more quickly protect against harmful ones.

The author thanks Darin Christensen, Erin Hartman, Chris Tausanovitch, Mark Handcock,

Je Lewis, Aaron Rudkin, Maya Petersen, Arash Naeim, Onyebuchi Arah, Dean Knox, Ami Wulf, Paasha Mah-

davi, and members of the 2018 Southern California Methods workshop for valuable feedback and discussion.

1. Hambine J. The disingenuousness of right to try. The Atlantic. 2018.

2. Neyman J, Dabrowska D, Speed T. On the application of probability theory to agricultural experiments. Essay on principles.

Section 9. Stat Sci. 1923[1990];5(4):465–72.

3. Angrist JD, Imbens GW, Rubin DB. Identication of causal eects using instrumental variables. J Am Stat Assoc.

1996;91(434):444–55.

4. Pearl J. Causality. Cambridge university press; 2009.

5. Johnston K, Gustafson P, Levy A, Grootendorst P. Use of instrumental variables in the analysis of generalized linear models

in the presence of unmeasured confounding with applications to epidemiological research. Stat Med. 2008;27(9):1539–56.

6. Cain LE, Cole SR, Greenland S, Brown TT, Chmiel JS, Kingsley L, Detels R. Eect of highly active antiretroviral therapy on

incident aids using calendar period as an instrumental variable. Am J Epidemiol. 2009;169(9):1124–32.

7. Shetty KD, Vogt WB, Bhattacharya J. Hormone replacement therapy and cardiovascular health in the united states. Med

Care. 2009;600–5.

8. Mack CD, Brookhart MA, Glynn RJ, Meyer AM, Carpenter WR, Sandler RS, StürmerT. Comparative eectiveness of oxaliplatin

vs. 5-ourouricil in older adults: an instrumental variable analysis. Epidemiology (Camb, Mass). 2015;26(5):690.

9. Gokhale M, Buse JB, DeFilippo Mack C, Jonsson FunkM, Lund J, Simpson RJ, Stürmer T. Calendar time as an

instrumental variable in assessing the riskof heart failure with antihyperglycemic drugs. Pharmacoepidemiol Drug Saf.

2018;27(8):857–66.

10. Brookhart MA, Rassen JA, SchneeweissS. Instrumental variable methods in comparative safety and eectiveness research.

Pharmacoepidemiol Drug Saf. 2010;19(6):537–54.

11. Olschewski M, Scheurlen H. Comprehensive cohort study: an alternative to randomized consent design in a breast

preservation trial. Methods Inf Med. 1985;24(03):131–4.

12. Brewin CR, Bradley C. Patient preferences and randomised clinical trials. BMJ, Br Med J. 1989;299(6694):313.

13. Knox D, Yamamoto T, Baum MA, Berinsky A. Design, identication, and sensitivity analysis for patient preference trials.

Technical report, Working Paper. 2014.

Brought to you by | University of California - Los Angeles - UCLA Library

Authenticated

Download Date | 12/11/18 9:04 PM