Challenges in the transition to model-based development.

Abstract

Practitioners of the art and science of pharmacometrics are well aware of the considerable effort required to successfully complete modeling and simulation activities for drug development programs. This is particularly true because of the current, ad hoc implementation wherein modeling and simulation activities are piggybacked onto traditional development programs. This effort, coupled with the failure to explicitly design development programs around modeling and simulation, will continue to be an important obstacle to the successful transition to model-based drug development. Challenges with timely data availability, high data discard rates, delays in completing modeling and simulation activities, and resistance of development teams to the use of modeling and simulation in decision making are all symptoms of an immature process capability for performing modeling and simulation. A process that will fulfill the promise of model-based development will require the development and deployment of three critical elements. The first is the infrastructure--the data definitions and assembly processes that will allow efficient pooling of data across trials and development programs. The second is the process itself--developing guidelines for deciding when and where modeling and simulation should be applied and the criteria for assessing performance and impact. The third element concerns the organization and culture--the establishment of truly integrated, multidisciplinary, and multiorganizational development teams trained in the use of modeling and simulation in decision-making. Creating these capabilities, infrastructure, and incentivizations are critical to realizing the full value of modeling and simulation in drug development.

Full text

Page 1

Themed Issue: Population Pharmacokinetics - In Memory of Lewis Sheiner
Guest Editors - Peter Bonate and Diane Mould
Challenges in the Transition to Model-Based Development
Submitted: May 3, 2005; Accepted: May 5, 2005; Published: October 5, 2005
Thaddeus H. Grasela,1Jill Fiedler-Kelly,1Cynthia A. Walawander,1Joel S. Owen,1Brenda B. Cirincione,1
Kathleen E. Reitz,1Elizabeth A. Ludwig,1Julie A. Passarell,1and Charles W. Dement2
1Cognigen Corporation
2University at Buffalo, The State University at New York, Buffalo, NY
ABSTRACT
Practitioners of the art and science of pharmacometrics are
well aware of the considerable effort required to success-
fully complete modeling and simulation activities for drug
development programs. This is particularly true because of
the current, ad hoc implementation wherein modeling and
simulation activities are piggybacked onto traditional
development programs. This effort, coupled with the fail-
ure to explicitly design development programs around
modeling and simulation, will continue to be an important
obstacle to the successful transition to model-based drug
development. Challenges with timely data availability,
high data discard rates, delays in completing modeling and
simulation activities, and resistance of development teams
to the use of modeling and simulation in decision making
are all symptoms of an immature process capability for
performing modeling and simulation.
A process that will fulfill the promise of model-based de-
velopment will require the development and deployment of
three critical elements. The first is the infrastructure——the
data definitions and assembly processes that will allow
efficient pooling of data across trials and development
programs. The second is the process itself——developing
guidelines for deciding when and where modeling and
simulation should be applied and the criteria for assessing
performance and impact. The third element concerns the
organization and culture——the establishment of truly inte-
grated, multidisciplinary, and multiorganizational develop-
ment teams trained in the use of modeling and simulation
in decision-making. Creating these capabilities, infrastruc-
ture, and incentivizations are critical to realizing the full
value of modeling and simulation in drug development.
KEYWORDS: pharmacometrics, model-based develop-
ment, real-time data assembly
PERSONAL REFLECTIONS
During my 2-year clinical pharmacology fellowship with
Dr. Leinis Sheiner at the University of California, San
Francisco, starting in 1980, my primary research project
focused on an evaluation of the population pharmacoki-
netics (PKs) of procainamide and its metabolite, N-acetyl
procainamide, using plasma samples left over from routine
laboratory evaluations and timed urine collections in
patients treated for arrhythmias.1My experiences in one of
the earliest applications of nonlinear mixed-effect model-
ing (NONMEM) left an indelible impression as to the
potential value of population modeling, as well as the diffi-
culties in successfully executing a population analysis.
The project required that I keypunch data onto cards and
submit the NONMEM runs via a card reader to access
the Lawrence Livermore Laboratory mainframe. The NON-
MEM Project Group had its research account on this main-
frame, and Dr. Stuart Beal was responsible for monitoring
expenditures for mainframe time. To conserve resources, I
was asked to carefully select runs forsubmissionand to sub-
mit jobs after 9:00 PM, when cheaper computing rates ap-
plied. Parsimony proved to be a difficult principle to adhere
to, because punching data onto cards often resulted in inad-
vertent mistakes that were not detected until the run ended
in errors. Needless to say, the frequent rerunning of data
andattendant incurredcharges were notlostonDr.Beal.
At the time, NONMEM users were required to write their
own prediction subroutine (PRED) to generate the predic-
tions of drug concentrations at the time of the measured
values. The procainamide model involved plasma and
urine measurements of both procainamide and N-acetyl
procainamide. The predictive equations had to be written
in recursive formats that were then used to compute the
analytic form of the partial derivatives. The entry of the
equations into the program to compute the partial deriva-
tives and the subsequent hand entry of the derivatives onto
punch cards was also fraught with errors. I believe that
these trials and tribulations are some of the reasons why
we now have PRED PP (PRED Subroutine for Population
Pharmacokinetics).2
Many of the technical aspects of the application of mixed-
effect modeling in the analysis of PK and pharmacody-
Corresponding Author: Thaddeus H. Grasela, Cognigen
Corp, 395 Youngs Road, Buffalo, NY 14221-5831;
Tel: (716) 633-3463; Fax: (716) 633-7404; E-mail: ted.
grasela@cognigencorp.com
The AAPS Journal 2005; 7 (2) Article 49 (http://www.aapsj.org).
E488

Page 2

namic (PD) data have improved tremendously over the last
20 years. Other aspects have not changed, and these
obstacles represent frequent sources of frustration for sci-
entists seeking to use modeling and simulation to inform
drug development and regulatory decision-making. This
article describes the nature of the important obstacles that
must be resolved if modeling and simulation is to have
maximal impact on drug development.
INTRODUCTION
The application of population approaches to the investiga-
tion of PKs and PDs has had an important impact on drug
development. The role of mixed-effect modeling has grown
from early investigations of population PKs in patients
enrolled in clinical trials and the evaluation of covariates
as sources of PK variability to the development of PK and
PD models that provide a comprehensive, exposure-
response-based characterization of safety and efficacy.
Technically, the process for estimating population PK/PD
parameters using mixed-effect modeling has improved
dramatically since it was first proposed by Sheiner and
Beal.2The development of PRED PP, the growing body of
examples of mixed-effect modeling applied to specific
therapeutic areas, and the availability of fast and cheap
computer resources are all important milestones. These
advances, along with the growing recognition of the value
of modeling and simulation in drug development, have all
helped to contribute to the expanding list of development
programs that have successfully used modeling and simula-
tion in decision making.
Practitioners of the art and science of pharmacometrics are
well aware of the considerable effort required to success-
fully complete modeling and simulation for drug develop-
ment programs. This is particularly true because of the
current, ad hoc implementation, wherein modeling and
simulationactivities arepiggybackedontotraditionaldevel-
opment programs. The process used for implementing
modeling and simulation has evolved over time, and phar-
macometricians are at an important disadvantage as
we move toward model-based development, because this
process has not been explicitly designed to address critical
successfactors.
This paper focuses on two theses. The first thesis is that
modeling and simulation will play an increasingly impor-
tant role in drug development and that this role will shift
over time from a supportive function in the current
empiric-based development paradigm to a central role in a
fully model-based development paradigm. The second the-
sis is that the design, deployment, and maintenance of a
reliable and efficient process for pharmacometric analysis
represents a critical challenge that must be addressed if the
full value of modeling and simulation, even in a supportive
role, is to be realized.
A CHANGING DEVELOPMENT PARADIGM
The 1962 amendment to the Federal Pure Food, Drug, and
Cosmetic Act requires that manufacturers demonstrate the
safety and effectiveness of their products by conducting
adequate and well-controlled studies. The empiric-based
development paradigm that has evolved from this mandate
has been a powerful tool for assuring the safety and effec-
tiveness of marketed products. The randomized, double-
blind trial represents a clear and unambiguous standard for
demonstrating efficacy and provides the basis for a binary
yes or no approval decision during the review of study
results by regulatory agencies.
More recently, there have been growing concerns regard-
ing the rising costs of development and reduced pipeline
productivity. A recent Food and Drug Administration
(FDA) report states that new compounds entering phase I
have an 8% chance of reaching the market place today
versus a 14% chance 15 years ago. Additionally, the phase
III failure rate is now reported to be 50% versus 20% from
10 years ago. In the view of the FDA, this falling success
rate has occurred because the applied sciences needs for
medical product development have not kept pace with the
tremendous advances in the basic sciences. The new sci-
ence is not being used to guide the technology develop-
ment process in the same way that it is accelerating the
technology discovery process.3
At the same time, there has also been increased scrutiny
of the ability of the FDA to certify the safety of new
medicines. A number of drugs have been withdrawn from
the marketplace or required significant changes in the pro-
duct label because of drug safety concerns uncovered after
marketing.
The inviolable requirement to demonstrate the safety and
effectiveness of new medicines coupled with the critical
business challenge of dealing with rising development
costs and an increased risk of failure is creating a ??perfect
storm?? in the pharmaceutical industry. These challenges
have placed increased pressure on the industry and regula-
tory agencies to find new approaches to drug development.
The FDA has recently called for the implementation of a
Critical Path Initiative to develop and deploy ??new tools to
get fundamentally better answers about how the safety and
effectiveness of new products can be demonstrated in faster
time frames, with more certainty, and at lower costs.??3One
aspect of this initiative calls for the transition to a more
model-based development approach based on the learn-
and-confirm paradigm proposed by Sheiner.4
The application of the learn-and-confirm paradigm breaks
development into 2 major learn—confirm cycles. In the first
cycle of phases I and IIa, we learn what dose is tolerated in
normal subjects and subsequently confirm that this dose
has the potential to be effective in selected patients. At the
The AAPS Journal 2005; 7 (2) Article 49 (http://www.aapsj.org).
E489

Page 3

end of phase IIa, we reach a decision point as to whether
there is sufficient evidence of efficacy and a corresponding
lack of toxicity to justify additional investment. If so, the
second learn—confirm cycle of phases IIb and III/IV is
begun. The goal of the learning step of phase IIb is to learn
how to use the drug in representative patients to improve
the odds of an acceptable benefit-risk assessment. The goal
of the confirm step (phase III/IV) is to demonstrate, in a
large and representative patient population, that an accept-
able benefit/risk profile has been achieved.
This paradigm relies on the application of modeling and
simulation to assist in the transition through the learn—
confirm cycles. We incorporate the knowledge from one
step of development in an explicitly specified model and
formulate how the next step in development is to be per-
formed. For example, the end-of-phase IIa meetings
between sponsors and FDA allows for the review of all of
the available information in support of the dose for later-
stage development. Modeling of preclinical and early clini-
cal data and extrapolation of results via simulations to the
design and expected outcomes of phase III clinical trials
have become an important basis for decision making at
these meetings.
The real difference between empiric-based development
and model-based development is the role that models play
in the process of hypothesis formulation and confirmation.
In the current model-supported, empiric-based paradigm,
modeling and simulation play supportive roles in helping
to set the design characteristics of empirical clinical trials.
In a fully realized model-based development paradigm,
models will be both the instruments and aims of drug
development programs. There is a much more intimate
relationship among the premises, hypotheses, and theorems
that a model realizes or conveys, on the one hand, and
those that a clinical trial is meant to test. In other words,
the model-based paradigm will focus on the development
and support of models as the primary outcome of a devel-
opment program. This entails a much more iterative proc-
ess that is currently used and the need for a much more
rigorous and efficient ??industrialized?? process to support
timely decision-making.
PHARMACOMETRICS IN A MODEL-SUPPORTED
PARADIGM
In the current business and regulatory climate, modeling
and simulation will likely continue to grow in importance
as a supportive function. But even in this supportive role,
the successful application of modeling and simulation must
be accompanied by critical strategic, logistic, tactical, and
architectural infrastructure advances. The lack of these
critical infrastructure elements is a major impediment in
the successful deployment and sustainment of modeling
and simulation in drug development.
It is useful to examine the reasons why the execution of
traditional clinical trials within the empiric-based develop-
ment paradigm has been so successful. From an architec-
tural perspective, the entire process of clinical trial execu-
tion including study design, data collection, data scrubbing,
data management, data programming, analysis, and report-
ing is geared toward maximizing the efficiency, cost, and
schedule requirements of the empiric-based paradigm. The
design principles of clinical trials are well understood, and
textbooks with prototypical development strategies are
available to guide programmatics. The time and resource
requirements are well appreciated, and the informatic ele-
ments are straightforward and relatively easy to acquire.
Consequently, the production of analysis results is predict-
able in both cost and schedule. The process is reliable,
repeatable, and determinable. Once the analysis plan has
been specified, data analysis programs can be written and
verified during data collection so that results can be readily
generated once database lock has been accomplished.
Importantly, the outcome of the process, the p value, is
adequate for the purposes of the major stakeholders, in-
cluding regulatory agencies, development and marketing
teams, and even prescribers. These characteristics serve as
baseline performance measures that any new paradigm,
including one supported by modeling and simulation, must
exhibit.
Model-supported development, as it is currently imple-
mented, shares few of these critical programmatic charac-
teristics. There are important data availability and quality
issues that contribute to highly variable costs and schedule
requirements for PK/PD model development and analysis
activities. The process of performing a model-based analy-
sis is model-dependent and analyst-dependent. The process
is nonprogrammatic because model-based development is
both a ??hypothesis generator?? and a confirmator. Impor-
tantly, the outcomes of the analyses are interpretable in
different ways by different stakeholders and are overly
complex for historical stakeholder purposes. There is no
textbook that provides rigorous and well-accepted prin-
ciples for the design, implementation, and interpretation of
a model-based development program.
The lack of explicitly designed programmatics and proc-
esses for modeling and simulation has placed enormous
pressure on pharmacometricians to deliver results in a time
frame that corresponds with the delivery of traditional stat-
istical analyses, often defined as within 14 days of database
lock. The overall time frame for a typical exposure-
response evaluation, encompassing a variety of modeling
and simulation activities, can be weeks or months depend-
ing on the complexity of the analysis. Figure 1 is a pie
chart showing the relative proportion of time required for
the typical activities involved in an exposure-response
evaluation based on our collective experiences.5
The AAPS Journal 2005; 7 (2) Article 49 (http://www.aapsj.org).
E490

Page 4

If modeling and simulation results are to be delivered in a
timely manner so that they can inform development and
regulatory decision making, the industry must move
beyond ad hoc implementation and address infrastructure,
process, organizational, and culture issues with the same
consistency and efficiency currently attendant with em-
piric-based development. These issues bear on data assem-
bly, modeling and simulation activities, and communica-
tion strategies used by pharmacometricians.
Data Assembly
The lead time required for development of population PK
models and the performance of exposure-response model-
ing places enormous pressure on data management to
expedite data scrubbing and dataset assembly so that mod-
eling and simulation efforts can be initiated as quickly as
possible. The assembly of datasets for a population PK/PD
analysis is complicated by the complexity of both the
content and the structure of the required database. These
analyses typically required pooling disparate data, includ-
ing PK information, the drug dosing history, patient
demography, laboratory data, concomitant medicines, and
measures of efficacy and safety to create a time-ordered
sequence of relevant events for each patient from the time
of enrollment in a trial until its conclusion.
This information must be secured from numerous databases
managed by different functional groups, either internal or
external to the company, so accessing and pooling the data
can be cumbersome and time consuming. Moreover, the
definition data are inadequate to support efficiencies in the
assembly process, which generally entails a manual process
dependentontheskillsof thedatamanager.
Once the data are pooled, a myriad of problems can be
encountered during data scrubbing. A number of essential
data queries, particularly the determination of whether the
drug concentration values and the date and time of sam-
pling make sense in the context of the dosing history,
cannot be performed until the drug concentration database
is merged with the drug-dosing history for each patient.
Yet, it is common for these individual data elements to be
queried separately, so the important questions as to whether
data issues will impact on the quality of the results or pre-
clude any meaningful analysis may not be recognized in a
timely manner.6
Data assembly and querying procedures performed for one
recent study resulted in a 50% discard rate of the concen-
trations because of recording errors in sampling and dosing
time, incomplete data collection, and administrative errors
noted after merging the drug concentration and dosing
history databases. The time and expense associated with
discarded drug concentrations underscore the need for a
commitment to specialized monitoring for clinical trials
incorporating sparse sampling for mixed-effect modeling.
This effort in itself can be costly and time consuming,
but experience has shown that it is required to ensure the
quality of the results.7
Ideally, PK data should be queried during trial execution to
identify problems early so that they can be rectified by
appropriate interventions at the problem sites. Our experi-
ences in developing a process for real-time data assembly
during the delavirdine phase III clinical development
program are illustrative of the issues that arise and the
value provided by this strategy.8
Delavirdine Case Study
Delavirdine mesylate is a nonnucleoside reverse-transcrip-
tase inhibitor that is currently approved for the treatment of
HIV infection. During the design of the phase III develop-
ment program, concerns over potential safety issues associ-
ated with saturable metabolism prompted the sponsor to
initiate a program to monitor drug concentrations in all of
the patients who enrolled in two double-blind, randomized,
pivotal registration trials to allow dosing adjustments in
patients who were experiencing elevated concentrations of
delavirdine.8
Maintaining the study blind was an important issue with
respect to the data assembly and concentration monitoring
program. The procedures for maintaining the study blind
started with very strict conservative reporting standards.
Patient-specific information was presented with a blinded
identification number, and data summaries were only
presented if the sample size was larger than a predeter-
mined number for each display. Contacts to blinded study
Figure 1. Pie chart showing the relative proportion of time
required for the typical activities involved in an exposure-
response evaluation. The overall time frame for an exposure-
response evaluation encompassing these activities can be weeks
to months depending on the complexity of the program. Based
on data on file at Cognigen Corporation.
The AAPS Journal 2005; 7 (2) Article 49 (http://www.aapsj.org).
E491

Page 5

personnel were scripted to minimize the chance of inadver-
tent disclosures that would compromise study blind. In the
event that a dosage adjustment was required for a patient,
another patient who was randomized to the placebo was
also identified, and a similar, mock-dosage adjustment
was made.
The delavirdine sparse concentration time data collected
during the regular clinical visits were subsequently com-
bined with information regarding demography, delavirdine
dosing history, concomitant medications, adverse events,
and laboratory safety studies. The data assembly process
allowed for the expedited scrubbing of the drug concentra-
tion time data along with the corresponding dosing histor-
ies and facilitated monitoring of possible concentration-
related safety events and screens for drug-drug interactions
during the trial. This information was then used to prepare
summary reports that were submitted to the data safety
monitoring board. The compliance of each site with respect
to protocol requirements was monitored, and sites received
feedback to improve compliance. This evaluation also
served to identify and censor sites failing to meet the mini-
mum criteria for data validity and reliability.
This program had an important impact on patient recruit-
ment and safety monitoring. Patients who might have
been excluded based on concomitant medications were
included, enhancing recruitment, and patients subsequ-
ently found to have low exposures to delavirdine be-
cause of concomitant use of metabolic inducers were
identified and discontinued their study participation, ad-
dressing an important ethical issue. In addition, the data
assembly process allowed for a much more efficient and
timely completion of modeling and simulation activities
and the preparation of supportive material for the regu-
latory submission. The value of real-time data assembly
was subsequently recognized in the FDA Guidance on
Population Pharmacokinetics.9
The Modeling and Simulation Process
When first introduced into drug development, mixed-effect
modeling of sparse samples focused on estimating the pop-
ulation PK parameters of a drug. The goal was generally to
assess the influence of patient covariates, such as demogra-
phy, laboratory data, and concomitant medications, on the
typical values of PK parameters and the magnitude of
interindividual and intraindividual variability. Although
this is still a common component of population analysis,
the focus of mixed-effect modeling for the purposes of
decision-making has shifted to the estimation of patient-
specific measures of exposure and the incorporation of
these estimates into exposure-response evaluations. These
evaluations provide an assessment of the clinical impor-
tance of altered PK disposition and allow a more detailed
assessment of the determinants of efficacy and safety out-
comes. This assessment is one of the key value proposi-
tions of model-supported development.
Often the reason given for obtaining sparse samples in a
clinical trial is as an insurance policy in the event that the
study results are not as expected. In this situation, the
thought is that a subsequent exposure-response analysis
may offer an explanation for the findings and point the
way forward. This has created a chicken and egg conun-
drum, because the inadequate attention to sampling design
and data management coupled with the need to urgently
perform retrospectively designed analyses has limited the
value of modeling and simulation in decision-making.
Analyses conducted under these conditions may provide
insights of value for the immediate need, but they often
reveal the possibility of more important insights that can-
not be additionally explored because of inadequate designs
or data management procedures.
Because of the long lead time for modeling and simulation
activities, some pharmacometricians have been successful
at moving the development teams toward a more deliberate
and proactive implementation plan. Figure 2 shows the
tasks and timelines for modeling and simulation activities
superimposed on a linear development paradigm. This
commonly used strategy is focused on rapidly completing
the exploratory development so that a ??go/no-go?? decision
for full development can be made as soon as possible.
As the popularity of modeling and simulation increases,
pharmacometricians are going to be faced with the task of
triaging development programs to the appropriate applica-
tion of modeling and simulation. We must deal with ques-
tions such as how do we decide when and where modeling
and simulation should be applied? How do we assess the
Figure 2. Tasks and timelines for modeling and simulation
activities superimposed on a linear development strategy. This
implementation is often required to compensate for management
requirements that exploratory development be completed rapidly
so that a go/no-go decision for full development can be made as
soon as possible.
The AAPS Journal 2005; 7 (2) Article 49 (http://www.aapsj.org).
E492