PreprintPDF Available

Retrieved dropout imputation considering administrative study withdrawal

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) E9 (R1) Addendum provides a framework for defining estimands in clinical trials. Treatment policy strategy is the mostly used approach to handle intercurrent events in defining estimands. Imputing missing values for potential outcomes under the treatment policy strategy has been discussed in the literature. Missing values as a result of administrative study withdrawals (such as site closures due to business reasons, COVID-19 control measures, and geopolitical conflicts, etc.) are often imputed in the same way as other missing values occurring after intercurrent events related to safety or efficacy. Some research suggests using a hypothetical strategy to handle the treatment discontinuations due to administrative study withdrawal in defining the estimands and imputing the missing values based on completer data assuming missing at random, but this approach ignores the fact that subjects might experience other intercurrent events had they not had the administrative study withdrawal. In this article, we consider the administrative study withdrawal censors the normal real-world like intercurrent events and propose two methods for handling the corresponding missing values under the retrieved dropout imputation framework. Simulation shows the two methods perform well. We also applied the methods to actual clinical trial data evaluating an anti-diabetes treatment.
Content may be subject to copyright.
Retrieved dropout imputation considering administrative study
withdrawal
Rong Liu, Yongming Qu
Department of Global Statistical Sciences
Eli Lilly and Company, Indianapolis, Indiana, USA
October 10, 2024
Abstract
The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Hu-
man Use (ICH) E9 (R1) Addendum provides a framework for defining estimands in clinical trials. Treat-
ment policy strategy is the mostly used approach to handle intercurrent events in defining estimands.
Imputing missing values for potential outcomes under the treatment policy strategy has been discussed
in the literature. Missing values as a result of administrative study withdrawals (such as site closures
due to business reasons, COVID-19 control measures, and geopolitical conflicts, etc.) are often imputed
in the same way as other missing values occurring after intercurrent events related to safety or efficacy.
Some research suggests using a hypothetical strategy to handle the treatment discontinuations due to
administrative study withdrawal in defining the estimands and imputing the missing values based on
completer data assuming missing at random, but this approach ignores the fact that subjects might ex-
perience other intercurrent events had they not had the administrative study withdrawal. In this article,
we consider the administrative study withdrawal censors the normal real-world like intercurrent events
and propose two methods for handling the corresponding missing values under the retrieved dropout
imputation framework. Simulation shows the two methods perform well. We also applied the methods
to actual clinical trial data evaluating an anti-diabetes treatment.
Key words: Competing risks; Missing data; Multiple imputation; Normal real-world like treatment
discontinuation, Treatment policy strategy.
1 Introduction
Estimands and missing data are crucial considerations in clinical trial design and analysis. The ICH E9 (R1)
Addendum (2020) has established a framework for defining estimands with five key attributes: treatment
1
arXiv:2410.06774v1 [stat.AP] 9 Oct 2024
condition of interest, population,variable, handling of intercurrent events, and population-level summary.
The framework includes several strategies for addressing intercurrent events, such as treatment policy, hy-
pothetical, composite variable, while-on-treatment, and principal stratum strategies. The treatment policy
strategy, which uses the potential outcome despite intercurrent events to determine the treatment effect in
estimands, is gaining increasing attention in pharmaceutical statistics and has become a favored approach
by regulatory agencies for many disease areas. The intent of a treatment policy strategy is to estimate the
treatment effect as it would occur under real-world conditions, rather than in a hypothetical scenario where
all subjects would adhere to the treatment. To achieve this, the study should implement strategies designed
to systematically maximize subject retention and consistently minimize missing data, even after intercurrent
events (e.g. treatment discontinuation). In cases where missing data still arise despite these precautions,
the imputation of missing values will be conducted in a manner that reflects the likely real-world values
that would have been observed had the data been collected. Methods to impute missing values under the
treatment policy strategy have been discussed recently (Wang et al., 2023; He et al., 2023). These methods
impute all missing values in a clinical study uniformly using one single approach, in a fashion consistent with
what the values would have been had they been collected while off treatment.
The ICH E9 (R1) Addendum (2020) provides clear differentiation between treatment discontinuation
and study withdrawal. Lipkovich et al. (2020) suggest using the potential outcome framework in causal
inference to define estimands. In the spirit of the treatment policy strategy, the question of interest here
is that “what is the potential outcome under the treatment policy strategy if the administrative study
withdrawal does not occur?” or “what is the potential outcome when using the treatment policy strategy to
handle intercurrent events that similarly occur in real life and normal circumstances?”. Typically, reasons
for treatment discontinuation and study withdrawal are recorded on separate case report forms (CRFs)
in clinical trials. There are 2 main scenarios regarding treatment discontinuation and study withdrawal.
Firstly, study withdrawal can occur after treatment discontinuation. For example, in a 12-month study, a
subject might stop taking the study treatment at 3 months and then withdraw from the study entirely at
6 months. In this case, the reasons for the treatment discontinuation and study withdrawal are generally
different, such as discontinuing treatment due to an adverse event and withdrawing from the study due
to relocation. We consider treatment discontinuations in this scenario as normal real-world like (NRWL)
treatment discontinuations. Secondly, a subject might need to discontinue both the study and treatment at
the same time. There are two scenarios for such cases, depending on the reason for the study withdrawal:
1. Administrative study withdrawal: this occurs due to reasons beyond the participants’ control (e.g.,
pandemics, geopolitical conflicts, and natural disasters) or with clearly documented reasons that are not
related to treatment effect and disease progression (e.g., relocation, frequent traveling, and scheduling
conflicts). In this case, continued outcome collection is not feasible, but estimating the treatment effect
2
under such extraordinary conditions that do not reflect normal real life circumstances, or under the
treatment discontinuations that would not occur in the real life (e.g., relocation generally does not
prevent patients continuing the medication as they can get the medicine from the pharmacy in the new
location), is not of interest. Treatment discontinuations resulting from such study withdrawals are not
considered NRWL treatment discontinuations.
2. Study withdrawal possibly related to treatment or disease progression: This includes cases where the
study withdrawal is possibly related to study medication or disease progression, such as lost to follow-
up, or the withdrawal clearly due to reasons related to treatment or disease progression, such as
adverse event or lack of efficacy. To be conservative, treatment discontinuations resulting from such
study withdrawals are considered NRWL treatment discontinuations.
Darken et al. (2020) and Qu et al. (2021) consider using mixed strategies in handling different types of
intercurrent events in the same study. They propose using the treatment policy strategy to handle NRWL
intercurrent events while using the hypothetical strategy to other intercurrent events. One drawback to this
approach is that it overlooks the possibility that a subject may experience multiple intercurrent events, and
these events (including study withdrawals) may have competing risks. Had this subject not experienced an
intercurrent event or had not withdrawn from the study due to administrative reasons, this subject might
have experienced (other) intercurrent events, such as treatment discontinuation due to adverse events or lack
of efficacy.
In this article, we propose and discuss methods for handling missing values due to study withdrawals
when the treatment policy strategy is used to handle NRWL treatment discontinuations in defining estimands.
Section 2 describes the statistical methods. Section 3 outlines simulation schema and presents the results
of the simulation study. In Section 4, we apply this method to a clinical trial evaluating an anti-diabetes
medication. Finally, Section 5 provides a summary and discussion.
2 Methods
Let Yjk denote the longitudinal outcome at time tk(0 < t1< . .. < tK=d) for subject j, where dis the study
duration. Let i= 0,1 denote the control and experimental arms, respectively. Let Xjbe a vector of baseline
covariates (including the baseline value of the response variable), and Zjis the code for the assigned treatment
(0 for control and 1 for experimental treatment). Generally, YjK , the response at the final time point, is the
primary outcome measurement. We use the notation introduced by Lipkovich et al. (2020) for the potential
outcome under the hypothetical and treatment policy strategies. For each variable, we use “(i, i)” after the
variable name to denote the potential outcome under the hypothetical strategy (adhering to treatment i
when assigned to treatment i), and use “(i)” to denote the potential outcome under the treatment policy
3
strategy if assigned to treatment i. For example, Yjk(1,1) is the potential outcome under the hypothetical
strategy for subject jat time tkif this subject is adherent to the assigned experimental treatment, and Yjk (1)
is the potential outcome under the treatment policy strategy if assigned to the experimental treatment. Let
Uj(i) denote the random variable for the time to the NRWL treatment discontinuation for subject junder
treatment iif subject jwould continue the study and Vj(i) denote random variable for the time to study
withdrawal. Note Uj(i) is censored if Uj(i)> Vj(i). Let Dj(i) denote type of study discontinuation for subject
junder treatment i:Dj(i) = 1 indicates study discontinuation due to administrative study withdrawal (for
example site closure, geographic conflicts, pandemic) and Dj(i) = 0 represents study discontinuation due to
other reasons not clearly documented or reasons that could be related to study treatment or disease severity
(for example, lost to follow-up, study noncompliance). Let Aj k (i) denote the treatment adherence indicator
at time tkfor subject junder treatment isuch that Ajk (i) = 1 (adherent) if Uj(i)tkand Ajk (i) = 0
otherwise. Let Rjk be the missing data indicator for Yj k(i) such that Rjk(i) = 1 indicates the outcome is
missing and Rjk (i) = 0 means the outcome is observed. In general, Rj k (i) = 1 if Vj(t)< tkand Rj k (i)=0
otherwise.
In this section, we describe a procedure to estimate the treatment effect with the potential outcome
consistent with the estimand using the treatment regimen policy to handle intercurrent events.
Figure 1: The scenarios for missing data, the treatment discontinuation, and study withdrawal in a clinical trial of
fixed duration.
As illustrated in Figure 1, there are 5 scenarios of missing data, the treatment discontinuation, and study
withdrawal, regarding the treatment journey of a subject relative to the primary endpoint in a clinical trial
with fixed duration:
Scenario 1 The subject completes the study without treatment discontinuation and without missing the
primary outcome.
4
Scenario 2 The subject completes the study without treatment discontinuation but the primary outcome
is missing. The missing values in this scenario are likely due to logistic reasons, so the missing values
can be imputed using observed data across all times points from subjects that fall in Scenarios 1 and
2 under the missing at random (MAR) assumption.
Scenario 3 The subject discontinues the study treatment early but the primary outcome is collected.
Scenario 4 The subject discontinues the study treatment first, then either withdraws from or completes the
study later, and the primary outcome is missing. In this case, the study withdrawal is typically assumed
to be unlikely related to the potential primary outcome unless the primary outcome is correlated with
subjects’ ability (e.g., mobility) to continue the clinical trial. This assumption can be described in
mathematical form as
YjK (i) Vj(i)|Xj, Vj(i)> Uj(i).(1)
Under this assumption, the missing values can be imputed using the retrieved dropouts, that is, subjects
in Scenario 3.
Scenario 5 The subject withdraws from the study before NRWL treatment discontinuation, leading to a
missing primary outcome and censoring the time of treatment discontinuation. We can handle the
situation differently according to the reason for study withdrawal:
5.1 For study withdrawal not due to administrative reasons, or reasons not clearly documented, we can
consider that the NRWL treatment discontinuation occurs at the same time as study withdrawal,
that is,
Uj(i) = Vj(i)|Dj(i)=0.(2)
5.2 For administrative study withdrawals, we consider imputing missing data in a fashion consistent
with what the values would likely have been had the events leading to study withdrawl not
occured and the efficacy outcomes been collected. In this context, the study discontinuation time,
is independent of both the potential outcome for the time to the NRWL treatment discontinuation
and the potential outcome for the primary outcome. For example, if a subject could no longer
continue with the study or the treatment due to local war, their potential time to the NRWL
treatment discontinuation and potential primary outcome would not be related to their study
discontinuation time due to the war.
Vj(i) {Uj(i), Yj(i)}|Xj, Dj(i)=1.(3)
In this case, we evaluated two methods to impute the missing outcomes for YjK , considering that
some subjects that fall into this scenario could discontinue treatment due to adverse events or
lack of efficacy if they were to continue in the study.
5
5.2(a) Let ˆ
S(i)(t|X) denote the estimator for the survival function for Uj(i) conditional on baseline
covariates X. For each subject with an administrative study withdrawal, the estimated
probability of the NRWL treatment discontinuation from time Vj(i) to the study end dis
ˆpj=P(Vj(t)< Uj(i)< d|Uj(i)> Vj(i)) = ˆ
S(i)(Vj(t)|X)ˆ
S(i)(d|X)
ˆ
S(i)(Vj(t)|X).
A binary random number ξjcan be generated from a Bernoulli distribution with mean ˆpj.
We assume that if ξj= 1, the subject would have discontinued treatment, and we would use
retrieved dropouts to impute the missing value in YjK (i). Conversely, if ξj= 0, we assume
that the subject would continue study treatment until the study end and use subjects who
adhere to the study treatment throughout the study to impute YjK (i).
5.2(b) Use observed and imputed values at the endpoint for subjects in Scenarios 1, 2, 3, 4, and 5.1
to impute the missing value in YjK (i) for subjects in Scenario 5.2. Since the source data used
for imputation includes subjects with and without NRWL treatment discontinuations, the
imputed data approximately reflect that each subject in Scenario 5.2 has a certain probability
of NRWL treatment discontinuations.
Table 1 provides an overview of the imputation method for each of the five scenarios.
3 Simulations
We considered a 48-week clinical trial that evaluated anti-diabetes medication. The primary endpoint was
the change in Hemoglobin A1c (HbA1c) from baseline to 48 weeks, with measurements taken at baseline
and then subsequently every 12 weeks throughout the study. For simplicity, we assumed that the trial only
had two treatment arms: a control group and an experimental treatment group, each with a sample size of
200. For simulating a single trial, the process involved the following 6 steps:
Step 1 For each subject, we simulated the change in HbA1c from baseline across all time points under the
assumption that the subject was adherent to the study treatment from an exponential decay model:
Yjk (Zj, Zj) = {θZj+ (β0+Zjβ1)(Xjµx) + sj}{1eκtk}+ϵjk ,(4)
where jis the index for the subjects (j= 1,2, ..., 400); kis the index for the time points (k= 1,2,3,4),
representing Week 12, 24, 36 and 48, respectively; Zjis the treatment assignment with Zj= 0 for
j200 and Zj= 1 of j > 200; Xjis the baseline HbA1c for subject j;Zjis the randomized treatment
(0 for control and 1 for experimental treatment) assigned to the jth subject; and Yjk (Zj, Zj) is the
potential observation of the jth subject at the kth time point when assigned to treatment Zj. We
6
Table 1: Imputation procedure for each scenario by treatment group
Scenario (Treatment/Study Discontin-
uation; Missingness at Endpoint)
Assumption Methods to Handle Missing Values in YK
Scenario 1. No treatment or study dis-
continuation; no missing value
N/A N/A
Scenario 2. No treatment or study dis-
continuation; with missing value
MAR Use observed data from Scenarios 1 and 2 to
impute under the MAR assumption.
Scenario 3. Treatment discontinuation
but no study discontinuation; no miss-
ing value (retrieved dropouts)
N/A N/A
Scenario 4. Treatment discontinuation
followed by study discontinuation/com-
pletion later; with missing value
Equation (1) Missing values at endpoint will be imputed un-
der the assumption of multivariate normality
for baseline and the endpoint visit, based upon
data from Scenario 3 (retrieved dropouts).
Scenario 5. Study discontinuation re-
sulting in treatment discontinuation;
with missing value
5.1. Study withdrawal is possibly
related to study treatment
Equation (2) Use data in Scenario 3 (retrieved dropouts) to
impute by treatment group (refer to Scenario
4).
5.2: Study discontinuation is clearly
due to administrative reasons
Equation (3) Method 5.2(a): Impute the NRWL treat-
ment discontinuation status first; if imputed
the NRWL treatment discontinuation status
is ”yes”, the corresponding missing value in
YjK (i) is imputed using retrieved dropouts
(subjects in Scenario 3); if imputed the NRWL
treatment discontinuation status is ”no”, the
corresponding missing value in YjK (i) is im-
puted using observed data from Scenarios 1 and
2 under the MAR assumption.
Method 5.2(b): Use data (observed and im-
puted) at the endpoint in Scenarios 1 to 4, and
5.1 to impute the missing values at endpoint in
Scenario 5.2.
Abbreviations: MAR: missing at random; N/A: not applicable.
randomly drew Xj’s from a location-scaled Beta distribution with parameters Beta (1.5, 2), a location
parameter of 7 and a scale parameter of 3, resulting in a mean baseline µxof approximate 8.3. In this
model, θZjis the (theoretical) ultimate change in HbA1c for treatment Zjif we follow the subjects long
7
enough. We chose β0and β1to be 0.1 and 0.2, respectively. Random terms are assumed independent
and normally distributed among and within subjects: sjN(0, σ2
s) is the between-subject error, and
ϵjk N(0, σ2
e) is the within-subject error. We chose σ2
sand σ2
eto be 1 and 0.5, respectively.
Step 2 We simulated the treatment adherence status Ajk (Zj) for subject jin treatment group Zjat time
point kfrom a Bernoulli distribution with probability of
Pr(Ajk (Zj, Zj)=0|Yj,k1(Zj, Zj)) = exp(α0+α1Yj,k1(Zj, Zj))
1 + exp(α0+α1Yj,k1(Zj, Zj)) +ck(Zj).(5)
The first part of Equation (5) models the dropout due to lack of efficacy, and the second part models
the dropout due to adverse events or other reasons. If Ajk (Zj) = 0, subject jdiscontinues treatment
iright after the time point tk1. Then, the time to the NRWL treatment discontinuation for subject
jis given by ta,j = min{tk1:Ajk(Zj)=0}. Through this step, we essentially simulated the time for
the NRWL treatment discontinuation for all subjects. We chose c1(0) = c2(0) = c3(0) = c4(0) = 0.2,
c1(1) = 0.06, c2(1) = 0.06, c3(1) = 0.03, c4(1) = 0.02. Different values for α0and α1were specified for
2 simulation settings, which will be described later.
Step 3 For subjects with the NRWL treatment discontinuation occurring before the endpoint, we simulated
the potential outcome under the treatment policy strategy:
Yjk (Zj) = [θZj(θZjθ0)·min{max(tkta,j ,0),24}/24+(β0+Zjβ1)(Xjµx)+sj](1eκtk)+ϵjk.(6)
The above model assumes the effect of the experimental treatment on HbA1c will be washed out
in 24 weeks while the effect of the control treatment will stay the same after the NRWL treatment
discontinuation. It follows that
Yjk (Zj) = Yjk (Zj, Zj)+[(θZjθ0)·min{max(tkta,j ,0),24}/24](1 eκtk).
Figure 2 illustrates the change in the mean response for various time points of NRWL treatment
discontinuation.
Step 4 For all subjects, simulate the time to study withdrawal due to administrative reasons from an
exponential distribution assuming a constant hazard rate λfor subjects to discontinue from the study
due to administrative reasons. Through this step, we essentially simulated subjects for Scenario 5.2.
Note that we only generated study discontinuations due to administrative reasons (Scenario 5.2).
Given that study withdrawals due to non-administrative reasons (Scenario 5.1) are handled similarly to
treatment discontinuation in Scenario 4, we can consider these discontinuations as part of the treatment
discontinuations modeled in Equation (11). Subsequently, in the paper, we combined Scenarios 4 and
5.1, referring to them collectively as Scenarios 4 & 5.1.
8
Figure 2: Timecourse of mean response by various time of the NRWL treatment discontinuation
Table 2: The average number (%) of subjects in each scenario
Setting Treatment Group Scenario 1 Scenario 2 Scenario 3 Scenario 4 & 5.1 Scenario 5.2
1Control 132 (65.8%) 7.0 (3.5%) 8.8 (4.4%) 36.7 (18.4%) 15.9 (8.0%)
Experimental 136 (68.0%) 7.1 (3.6%) 7.9 (4.0%) 33.3 (16.7%) 15.7 (7.9%)
2Control 109 (54.7%) 5.8 (2.9%) 8.6 (4.3%) 40.6 (20.3%) 35.5 (17.8%)
Experimental 125 (62.6%) 6.7 (3.3%) 5.9 (3.0%) 23.5 (11.7%) 38.7 (19.4%)
Step 5 A probability of missingness of 0.05 was imposed to the observations at Week 48 from subjects who
had neither discontinued treatment nor withdrawal from the study. This simulated the subjects for
Scenario 2.
Step 6 A probability of missingness of 0.8 was imposed to the observations at Week 48 from subjects who
had discontinued treatment before the endpoint but stayed in the study until the endpoint. This helped
to control the percentage of subjects that fell into Scenario 3.
We considered 2 simulation settings, each assessed using a total of 5,000 trials simulated according to
Steps 1-6 outlined above. The average number and percentage of subjects by treatment group for each
setting are presented in Table 2.
Setting 1: We chose α0=3.5, α1= 1.5, and λ= 0.002. This setting mimics real clinical trials in
evaluating anti-diabetes treatments.
Setting 2: We chose α0=3.5, α1= 0, and λ= 0.005. This setting allows for the evaluation of
9
the performance of various methods under a scenario with a larger proportion of study withdrawals
and discontinuation from treatment is purely random not depending on efficacy from previous time
point(s).
For each simulated dataset, four estimators with different methods to handle the missing values due to
study withdrawal were calculated:
Method A: Using the non-missing values for those who were adherent to the assigned treatment to
impute the missing values due to study withdrawal.
Method B: Using the retrieved dropouts to impute the missing values due to study withdrawal. This
method was described in Wang et al. (2023).
Method C: Using Method 5.2 (a) to impute the missing values due to study withdrawal.
Method D: Using Method 5.2 (b) to impute the missing values due to study withdrawal.
To evaluate the performance of these methods, the true values for the parameters of interest, including
means under the treatment policy strategy for the control group, the experimental treatment group, and
the treatment difference, were calculated as the average of the estimated values across 20,000 simulated
completed datasets. These datasets were generated using the models described in Steps 1-3, before imposing
missing data through treatment and study discontinuations on the simulated datasets. The true means
under the treatment policy strategy for the control group, the experimental treatment group, and the treat-
ment difference are -0.001, 1.591, and 1.590, respectively in Setting 1, and 0.001, 1.497, and 1.498,
respectively in Setting 2.
Table 3 presents the simulation results using multiple imputation to handle missing values for Methods
A to D. In Setting 1, Methods A, C, and D seemed to perform well, with small bias and approximately
95% coverage probability for the 95% confidence interval. In Setting 2, as the proportion of administrative
study withdrawal increased, Method A showed some negative bias in treatment effect, suggesting Method
A is anti-conservative for estimating the treatment effect. As expected, the bias for Method B increased as
the proportion of administrative withdrawal increased. Methods C and D still performed well.
4 Application
We applied the four methods to estimate the change in HbA1c from baseline to 52 weeks in IMAGINE-5
Study (ClinicalTrials.gov Identifier: NCT01582451). IMAGINE-5 is a randomized study to compare the
efficacy and safety of insulin peglispro with insulin glargine in subjects with type 2 diabetes who were
previously treated with a basal insulin. There were 466 subjects randomly assigned to insulin glargine or
10
Table 3: Simulation results for the mean response for each treatment group and the treatment difference for
various methods for a study with a total sample size of 400 (based on 5,000 simulated datasets)
Group Method Setting 1 Setting 2
BIAS ESE ASE CP BIAS ESE ASE CP
Control
A -0.005 0.129 0.143 0.958 0.001 0.122 0.138 0.965
B 0.060 0.161 0.182 0.942 0.035 0.209 0.221 0.938
C 0.012 0.134 0.151 0.958 0.001 0.132 0.153 0.964
D 0.008 0.137 0.153 0.957 0.003 0.140 0.161 0.962
Treatment
A -0.012 0.141 0.142 0.931 -0.058 0.153 0.158 0.907
B 0.143 0.177 0.190 0.857 0.229 0.236 0.232 0.781
C 0.004 0.149 0.157 0.933 -0.012 0.163 0.164 0.923
D 0.022 0.151 0.154 0.931 0.022 0.181 0.188 0.932
Treatment
Difference
A -0.007 0.191 0.193 0.939 -0.058 0.197 0.200 0.927
B 0.084 0.240 0.245 0.932 0.195 0.322 0.292 0.862
C -0.008 0.200 0.209 0.947 -0.013 0.212 0.213 0.934
D 0.014 0.203 0.206 0.941 0.019 0.231 0.232 0.936
Note: BIAS, empirical bias; CP, 95% empirical coverage probability; ASE: mean standard error estimates
of the mean based upon Rubin’s rule (Rubin 1987); ESE: standard deviation of the estimates.
insulin peglispro with a 1:2 randomization. The study was conducted in accordance with the Declaration of
Helsinki guidelines on good clinical practices (World Medical Association, 2010). The primary results were
published in Buse et al. (2016).
Table 4 presents the the number and percent of subjects in each scenario (defined in Table 1) by treatment
group. Table 5 summarizes the results for the estimated mean change from baseline to 52 weeks for each
treatment group along with the treatment differences for Methods A, B, C, and D. The results based on the
four methods were very similar. This is likely because (1) the proportion of administrative study withdrawal
is small, and (2) the mean response in the change in HbA1c from baseline to 52 weeks between retrieved
dropouts and adherers was relatively small. The observed mean changes in HbA1c for adherers were 0.236
and 0.672 for insulin glargine and insulin peglispro, respectively. The observed mean changes in HbA1c
for retrieved dropouts were 0.060 and 0.031 for insulin glargine and insulin peglispro, respectively.
11
Table 4: Summary of Number (%) of subjects in each scenario (IMAGINE-5)
Arm Scenario 1 Scenario 2 Scenario 3 Scenarios 4/5.1 Scenario 5.2
Insulin Glargine (N=159) 130 (81.8%) 1 (0.6%) 5 (3.1%) 7 (4.4%) 9 (5.7%)
Insulin Peglispro (N= 307) 257 (83.7%) 0 (0.0%) 13 (4.2%) 19 (6.2%) 13 (4.2%)
Table 5: Summary of the estimated mean change in HbA1c from baseline to 52 weeks (IMAGINE-5)
Insulin Glargine Insulin Peglispro Treatment Difference
Method Mean (SE) Mean (SE) Mean (95% CI)
A -0.230 (0.061) -0.602 (0.045) -0.372 (-0.519,-0.224)
B -0.219 (0.061) -0.588 (0.045) -0.368 (-0.517, -0.219)
C -0.233 (0.061) -0.603 (0.044) -0.369 (-0.517, -0.221)
D -0.228 (0.061) -0.608 (0.045) -0.380 (-0.528, -0.232)
5 Summary and Discussion
Since the release of ICH E9 (R1), treatment policy strategy has become one of the most popular approaches to
handle intercurrent events including premature treatment discontinuations in defining estimands. The ratio-
nale for using data after treatment discontinuation in the analyses is that it reflects real clinical use. However,
treatment discontinuation caused by study withdrawal due to administrative reasons such as COVID-19 con-
trol measures, geopolitical conflicts, and site closures due to business reasons which are rare events in real
life and are unlikely the interest of the estimation. Relocation, frequent traveling, and scheduling conflicts,
which caused study withdrawal and treatment discontinuation, unlikely prevent patients from continuing
the medication in real life. Therefore, treatment effect under thess types of treatment discontinuations that
only present in clinical trials is unlikely the clinical interest. Therefore, a hypothetical strategy may be used
to handle these non-NWRL treatment discontinuations.
Hybrid estimands have recently been proposed to use mixed strategies to handle different types of inter-
current events in the same study. For example, the treatment policy strategy is used to handle treatment
discontinuations due to adverse events and a hypothetical strategy is used to handle treatment discontinua-
tions as a result of study withdrawal due to administrative reasons. However, the hybrid estimand ignores
the fact that treatment discontinuations due to different reasons are at competing risks. Sub jects may ex-
perience treatment discontinuations due to reasons related to efficacy or safety should they have not had
the administrative study withdrawal. In this research, we propose a new method to estimate the estimand
with NRWL intercurrent events handled by the treatment policy strategy assuming the administrative study
withdrawal censors the NRWL treatment discontinuations. For participants with administrative study with-
12
drawal, the potential status of NRWL intercurrent events censored by the study withdrawal is first imputed.
Then, the missing outcome is imputed based on the imputed status of the NRWL intercurrent events: if the
imputed intercurrent event status is “no”, we impute the missing value for the outcome using MAR; if the
imputed intercurrent event status is “yes”, we compute the missing value using retrieved dropouts. An alter-
native and simpler approach is to impute the missing values due to administrative study withdrawal using
all subjects with observed values (including those who are adherent and the retrieved dropouts), which nat-
urally implies a certain proportion of subjects with administrative study withdrawal have NRWL treatment
discontinuations should they have not withdrawn from the study. These approaches make the imputation
methods consistent with the potential outcome of interest under the treatment policy strategy for the NRWL
intercurrent events.
Simulation shows when the administrative treatment discontinuation rate is low [e.g., less than 10% in
Setting 1 of the simulation (Table 2)], imputing the missing data using adherers (Method A) produces little
bias (Table 3). This is likely because only a small proportion of the administrative withdrawals would have
NWRL treatment discontinuations had they not withdrawn from the study. Therefore, Method A, which is
much simpler, may be used if the proportion of the administrative study withdrawals is less than 10%.
Note administrative study withdrawals due to frequent traveling, and scheduling conflicts, which are
often reported by patients, are more subjective than those caused by site closures due to pandemics, natural
disasters, and geopolitical conflicts. In application of this method, the key stakeholders including sponsors
and regulators should get aligned on the definition of estimands, especially on what the intercurrent events
are handled by the treatment policy strategy.
To effectively use the proposed imputation method for estimands using the treatment policy strategy to
handle relevant intercurrent events, it is critical to collect the accurate reasons for treatment discontinuations
and study withdrawals. Qu et al. (2022) reviewed a set of clinical trials for the reasons of treatment/study
discontinuations and suggested ways to improve the collection of the reasons for treatment/study discontin-
uations. An ongoing PHUSE working group on “Implementation of Estimands (ICH E9 (R1)) using Data
Standards” is developing a new recommendation for the CRF for treatment and study discontinuations
(PHUSE, 2024).
The competing risk for different types of intercurrent events can also be applied to situations where a
hypothetical strategy is used for certain types of intercurrent events. For example, if in the estimand the
intercurrent event of using rescue medication is handled by a hypothetical strategy, the data collected after
the initiation of the rescue medication may be discarded. Instead of imputing the missing outcome based
on the MAR assumption, we may need to treat the rescue medication use as an event censoring the NRWL
treatment discontinuation. However, the probability of treatment discontinuation would likely be high had
rescue not been administered. Note that, in this case, it may not be appropriate to classifying the event of
13
using rescue medication as Scenario 5.2 and use the method proposed in this article to impute the missing
(or unobserved) values. Further research is needed on this topic.
In the estimation of the cumulative distribution function for the NRWL treatment discontinuation, the
proposed methods assume that the the occurrence of the NRWL treatment discontinuation is independent of
the potential outcome for the analysis variable. This assumption may not hold. For example, subjects with
a poor response at early time points may have a higher probability of discontinuing treatment. However,
modeling the NRWL treatment discontinuations conditional on the intermediate outcomes may be complex.
Sensitivity analyses are generally recommended to investigate the impact of underlying assumptions on
missing data.
On a practical note, “same time” for Scenario 5 does not imply the exact same calendar date for clinical
visits. A subject may decide to discontinue treatment and withdrawal from the study due to administrative
reasons at the same time, but the formal discontinuation from the study may require a follow-up visit (which
generally occurs later) to complete the discontinuation procedures. In practice, Scenario 5 can be identified
by the absence of office visits between the discontinuation of treatment and the study withdrawal.
Acknowledgement
We would like to thank Dr. Jianghao Li (Eli Lilly and Company) for reviewing the manuscript, the simulation
and data analysis program and for his valuable feedback. Additionally, we extend our gratitude to Dr. Yanyao
Yi (Eli Lilly and Company) for some discussions regarding this topic.
Disclosure statement
Both authors are employees and minor shareholders of Eli Lilly and Company. Not additional funds were
received for this research.
14
References
Buse, J. B., Rodbard, H. W., Trescoli Serrano, C., Luo, J., Ivanyi, T., Bue-Valleskey, J., Hartman, M. L.,
Carey, M. A., and Chang, A. M. (2016). Randomized clinical trial comparing basal insulin peglispro and
insulin glargine in patients with type 2 diabetes previously treated with basal insulin: Imagine 5. Diabetes
Care, 39(1):92–100.
Darken, P., Nyberg, J., Ballal, S., and Wright, D. (2020). The attributable estimand: A new approach to
account for intercurrent events. Pharmaceutical Statistics, 19(5):626–635.
He, J., Crackel, R., Koh, W., Chen, L.-W., Li, F., Zhang, J., and Rothmann, M. (2023). Retrieved-
dropout-based multiple imputation for time-to-event data in cardiovascular outcome trials. Journal of
Biopharmaceutical Statistics, 33(2):234–252.
ICH E9 (R1) Addendum (2020). Addendum on estimands and sensitivity analysis in clinical trials to the
guideline on statistical principles for clinical trials. EMA/CHMP/ICH/436221/2017, Step 5 (17 February
2020 ).
Lipkovich, I., Ratitch, B., and Mallinckrodt, C. H. (2020). Causal inference and estimands in clinical trials.
Statistics in Biopharmaceutical Research, 12(1):54–67.
PHUSE (2024). Implementation of Estimands (ICH E9 (R1)) using Data Standards.https:
//advance.phuse.global/display/WEL/Implementation+of+Estimands+%28ICH+E9+%28R1%29%29+
using+Data+Standards. Accessed on August 14, 2024.
Qu, Y., Shurzinske, L., and Sethuraman, S. (2021). Defining estimands using a mix of strategies to handle
intercurrent events in clinical trials. Pharmaceutical Statistics, 20(2):314–323.
Qu, Y., White, R. D., and Ruberg, S. J. (2022). Accurate collection of reasons for treatment discontinuation
to better define estimands in clinical trials. Therapeutic Innovation & Regulatory Science, pages 1–8.
Wang, Y., Tu, W., Kim, Y., Sinks, S., He, J., Cambon, A., Crackel, R., Hamilton, K., Kettermann, A., and
Clark, J. (2023). Statistical methods for handling missing data to align with treatment policy strategy.
Pharmaceutical statistics, 22(4):650–670.
World Medical Association (2010). World medical association declaration of Helsinki. http://www. wma.
net/en/30publications/10policies/b3/index. html.
15
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Recently, retrieved-dropout-based multiple imputation has been used in some therapeutic areas to address the treatment policy estimand, mostly for continuous endpoints. In this approach, data from subjects who discontinued study treatment but remained in study were used to construct a model for multiple imputation for the missing data of subjects in the same treatment arm who discontinued study. We extend this approach to time-to-event endpoints and provide a practical guide for its implementation. We use a cardiovascular outcome trial dataset to illustrate the method and compare the results with those from Cox proportional hazard and reference-based multiple imputation methods.
Article
Full-text available
Randomized controlled trials (RCTs) are the gold standard for evaluation of the efficacy and safety of investigational interventions. If every patient in an RCT were to adhere to the randomized treatment, one could simply analyze the complete data to infer the treatment effect. However, intercurrent events (ICEs) including the use of concomitant medication for unsatisfactory efficacy, treatment discontinuation due to adverse events, or lack of efficacy may lead to interventions that deviate from the original treatment assignment. Therefore, defining the appropriate estimand (the appropriate parameter to be estimated) based on the primary objective of the study is critical prior to determining the statistical analysis method and analyzing the data. The International Council for Harmonisation (ICH) E9 (R1), adopted on November 20, 2019, provided five strategies to define the estimand: treatment policy, hypothetical, composite variable, while on treatment, and principal stratum. In this article, we propose an estimand using a mix of strategies in handling ICEs. This estimand is an average of the “null” treatment difference for those with ICEs potentially related to safety and the treatment difference for the other patients if they would complete the assigned treatments. Two examples from clinical trials evaluating antidiabetes treatments are provided to illustrate the estimation of this proposed estimand and to compare it with the estimates for estimands using hypothetical and treatment policy strategies in handling ICEs.
Article
The International Council for Harmonization (ICH) E9(R1) addendum recommends choosing an appropriate estimand based on the study objectives in advance of trial design. One defining attribute of an estimand is the intercurrent event, specifically what is considered an intercurrent event and how it should be handled. The primary objective of a clinical study is usually to assess a product's effectiveness and safety based on the planned treatment regimen instead of the actual treatment received. The estimand using the treatment policy strategy, which collects and analyzes data regardless of the occurrence of intercurrent events, is usually utilized. In this article, we explain how missing data can be handled using the treatment policy strategy from the authors' viewpoint in connection with antihyperglycemic product development programs. The article discusses five statistical methods to impute missing data occurring after intercurrent events. All five methods are applied within the framework of the treatment policy strategy. The article compares the five methods via Markov Chain Monte Carlo simulations and showcases how three of these five methods have been applied to estimate the treatment effects published in the labels for three antihyperglycemic agents currently on the market.
Article
Background: Reasons for treatment discontinuation are important not only to understand the benefit and risk profile of experimental treatments, but also to help choose appropriate strategies to handle intercurrent events in defining estimands. The current case report form (CRF) commonly in use mixes the underlying reasons for treatment discontinuation and who makes the decision for treatment discontinuation, often resulting in an inaccurate collection of reasons for treatment discontinuation. Methods and results: We systematically reviewed and analyzed treatment discontinuation data from nine phase 2 and phase 3 studies for insulin peglispro. A total of 857 participants with treatment discontinuation were included in the analysis. Our review suggested that, due to the vague multiple-choice options for treatment discontinuation present in the CRF, different reasons were sometimes recorded for the same underlying reason for treatment discontinuation. Based on our review and analysis, we suggest an intermediate solution and a more systematic way to improve the current CRF for treatment discontinuations. Conclusion: This research provides insight and directions on how to optimize the CRF for recording treatment discontinuation. Further work needs to be done to build the learning into Clinical Data Interchange Standards Consortium standards. Clinical trials: Clinicaltrials.gov numbers: NCT01027871 (Phase 2 for type 2 diabetes), NCT01049412 (Phase 2 for type 1 diabetes), NCT01481779 (IMAGINE 1 Study), NCT01435616 (IMAGINE 2 Study), NCT01454284 (IMAGINE 3 Study), NCT01468987 (IMAGINE 4 Study), NCT01582451 (IMAGINE 5 Study), NCT01790438 (IMAGINE 6 Study), NCT01792284 (IMAGINE 7 Study).
Article
The term “intercurrent events” has recently been used to describe events in clinical trials that may complicate the definition and calculation of the treatment effect estimand. This paper focuses on the use of an attributable estimand to address intercurrent events. Those events that are considered to be adversely related to randomized treatment (eg, discontinuation due to adverse events or lack of efficacy) are considered attributable and handled with a composite estimand strategy, while a hypothetical estimand strategy is used for intercurrent events not considered to be related to randomized treatment (eg, unrelated adverse events). We explore several options for how to implement this approach and compare them to hypothetical “efficacy” and treatment policy estimand strategies through a series of simulation studies whose design is inspired by recent trials in chronic obstructive pulmonary disease (COPD), and we illustrate through an analysis of a recently completed COPD trial.
Article
The National Research Council’s report on the prevention and treatment of missing data (NRC, 2010) highlighted the need to clearly specify causal estimands. This focus fundamentally changed how the missing data problem was perceived and addressed in clinical trials. The recent ICH E9(R1) draft addendum (ICH, 2017) is another major step in promoting the use of the causal estimands framework that should further influence how clinical trial protocols and statistical analysis plans are written and implemented. The language of potential outcomes that is widely accepted in the causal inference literature is not widely recognized in the clinical trialists community and was not used in defining causal estimands in the NRC report or the ICH E9(R1). In this article, we attempt to bridge the gap between the causal inference community and clinical trialists to further advance the use of causal estimands in clinical trial settings. We illustrate how concepts from causal literature, such as potential outcomes and dynamic treatment regimens, can facilitate defining and implementing causal estimands and may provide a unifying language to describing the targets for both observational and randomized clinical trials.
Article
Objective: To evaluate the efficacy and safety of basal insulin peglispro (BIL) versus insulin glargine in patients with type 2 diabetes (hemoglobin A1c [HbA1c] ≤9% [75 mmol/mol]) treated with basal insulin alone or with three or fewer oral antihyperglycemic medications. Research design and methods: This 52-week, open-label, treat-to-target study randomized patients (mean HbA1c7.42% [57.6 mmol/mol]) to BIL (n = 307) or glargine (n = 159). The primary end point was change from baseline HbA1c to 26 weeks (0.4% [4.4 mmol/mol] noninferiority margin). Results: At 26 weeks, reduction in HbA1c was superior with BIL versus glargine (-0.82% [-8.9 mmol/mol] vs. -0.29% [-3.2 mmol/mol]; least squares mean difference -0.52%, 95% CI -0.67 to -0.38 [-5.7 mmol/mol, 95% CI -7.3 to -4.2; P < 0.001); greater reduction in HbA1c with BIL was maintained at 52 weeks. More BIL patients achieved HbA1c <7% (53 mmol/mol) at weeks 26 and 52 (P < 0.001). With BIL versus glargine, nocturnal hypoglycemia rate was 60% lower, more patients achieved HbA1c <7% (53 mmol/mol) without nocturnal hypoglycemia at 26 and 52 weeks (P < 0.001), and total hypoglycemia rates were lower at 52 weeks (P = 0.03). At weeks 26 and 52, glucose variability was lower (P < 0.01), basal insulin dose was higher (P < 0.001), and triglycerides and aminotransferases were higher with BIL versus glargine (P < 0.05). Liver fat content (LFC), assessed in a subset of patients (n = 162), increased from baseline with BIL versus glargine (P < 0.001), with stable levels between 26 and 52 weeks. Conclusions: BIL provided superior glycemic control versus glargine, with reduced nocturnal and total hypoglycemia, lower glucose variability, and increased triglycerides, aminotransferases, and LFC.
Addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials
ICH E9 (R1) Addendum (2020). Addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials. EMA/CHMP/ICH/436221/2017, Step 5 (17 February 2020 ).
Implementation of Estimands (ICH E9 (R1)) using Data Standards
  • Phuse
PHUSE (2024). Implementation of Estimands (ICH E9 (R1)) using Data Standards. https:
World medical association declaration of Helsinki
World Medical Association (2010). World medical association declaration of Helsinki. http://www. wma. net/en/30publications/10policies/b3/index. html.