New Benchmarks on Protocol Amendment Practices, Trends and their Impact on Clinical Trial Performance

Abstract

The Tufts Center for the Study of Drug Development (Tufts CSDD) conducted a follow-up study in 2022 to assess trends in protocol amendment experiences and the impact amendments have had on clinical trial performance, particularly during the COVID-19 pandemic. Sixteen pharmaceutical companies and contract research organizations provided data on 950 protocols and 2,188 amendments. The results show that, since 2015, the prevalence of amendments in phase I – IV protocols has increased substantially (from 57–76%) and the mean number of amendments per protocol has increased 60% to 3.3, up from 2.1. Phase I and III protocols saw the highest increases in the mean number of amendments implemented per protocol. A much higher percentage of amendments — 77% —were deemed unavoidable with regulatory agency requests and changes to the study strategy as the top reasons cited for amending a protocol. In addition, the total average time to implement an amendment has nearly tripled during the past decade. The time from identifying the need-to-amend to last oversight approval now takes an average of 260 days and the mean duration during which investigative sites operate with different versions of the clinical trial protocol spans 215 days. Protocols that implemented at least one amendment were more effective at increasing patient screening volume and reducing the actual number of patients enrolled relative to plan. Lastly, the prevalence and mean number of amendments was significantly higher for protocols conducted during the pandemic.

Full text

Page 1/17
New Benchmarks on Protocol Amendment Practices,
Trends and their Impact on Clinical Trial Performance
Kenneth Getz (  kenneth.getz@tufts.edu )
Tufts University School of Medicine https://orcid.org/0000-0003-0568-2541
Zachary Smith
Tufts University School of Medicine
Emily Botto
Tufts University School of Medicine
Elisabeth Murphy
Janssen Pharmaceutical Companies of Johnson and Johnson: Janssen Global Services LLC
Arnaud Dauchy
Sano
Research Article
Keywords: Protocol amendments, protocol design, protocol complexity, protocol design changes
Posted Date: July 28th, 2023
DOI: https://doi.org/10.21203/rs.3.rs-3168679/v1
License:   This work is licensed under a Creative Commons Attribution 4.0 International License.  Read Full License

Page 2/17
Abstract
The Tufts Center for the Study of Drug Development (Tufts CSDD) conducted a follow-up study in 2022 to assess trends
in protocol amendment experiences and the impact amendments have had on clinical trial performance, particularly
during the COVID-19 pandemic. Sixteen pharmaceutical companies and contract research organizations provided data on
950 protocols and 2,188 amendments. The results show that, since 2015, the prevalence of amendments in phase I – IV
protocols has increased substantially (from 57–76%) and the mean number of amendments per protocol has increased
60% to 3.3, up from 2.1. Phase I and III protocols saw the highest increases in the mean number of amendments
implemented per protocol. A much higher percentage of amendments — 77% —were deemed unavoidable with regulatory
agency requests and changes to the study strategy as the top reasons cited for amending a protocol. In addition, the total
average time to implement an amendment has nearly tripled during the past decade. The time from identifying the need-
to-amend to last oversight approval now takes an average of 260 days and the mean duration during which investigative
sites operate with different versions of the clinical trial protocol spans 215 days. Protocols that implemented at least one
amendment were more effective at increasing patient screening volume and reducing the actual number of patients
enrolled relative to plan. Lastly, the prevalence and mean number of amendments was signicantly higher for protocols
conducted during the pandemic.
Introduction
Although the implementation of protocol amendments are necessary for a variety of reasons (e.g., to protect study
volunteer safety; improve patient recruitment and retention; and enhance study strategies) they are also highly disruptive
and represent the largest single cause of unplanned delays and unbudgeted costs in clinical trials. Regardless of their
extensive internal review, renement and approval processes, most nalized protocols are amended multiple times
resulting in months of delay and several hundred thousand dollars ($US) in direct costs. This includes adaptive protocol
designs that must be amended despite pre-planned and pre-approved interim changes to the study design.
A 2010 study conducted by the Tufts Center for the Study of Drug Development (Tufts CSDD) found that the frequency of
protocol amendments varied widely by therapeutic area. In that study, 17 large and mid-sized pharmaceutical companies
provided data on 3,413protocols and 3,596 amendments. Nearly 60% of all protocols had at least one amendment with a
mean of 2.6 and 3.6 total amendments per phase II and III protocols respectively. The 2010 study also found that the top
reason for amending a protocol was to modify study volunteer eligibility criteria due to changes in study design strategy
and diculties recruiting patients and that the prevalence of amendments was highly associated with more complex
protocols that have a higher relative number of endpoints, procedures, and investigative sites located in more countries.
Finally, this study estimated that one-third of all protocol amendments could have been avoided.
In a follow-up 2015 study, 15 pharmaceutical companies and contract research organizations (CROs) provided data on
836 protocols and 984 amendments. The primary aims of this study were to update the 2010 study ndings and to
measure the impact of protocol amendments on clinical trial performance, cost and eciency. Tufts CSDD observed a
similar prevalence of protocols with at least one amendment (57%) though a much higher percentage of these
amendments (45%) were deemed avoidable. A positive association was again observed between amendment prevalence
and protocol complexity. This study also found that protocols with amendments were associated with longer timelines
for study initiation, study execution, and study close-out. Protocols with amendments, however, resulted in fewer actual
screened and enrolled patients relative to baseline plan than protocols without amendments.
In 2022, Tufts CSDD, in collaboration with 19 pharmaceutical companies and CROs conducted a follow-up study to
continue assessing trends in protocol amendment experiences, add to our understanding of their impact and identify new
opportunities to anticipate and prevent avoidable protocol amendments. This new study also offered insight into changes

Page 3/17
in amendment experience due to the global COVID-19 pandemic. The authors hope that these results will encourage
discussion and inform new strategies to optimize protocol design and execution.
Methods
In April 2022, Tufts CSDD kicked-off this most recent study with representatives from the following sponsors and CROs:
Amgen, AstraZeneca, Biogen, Bristol Myers Squibb, EMD Serono, Eli Lilly & Company, Gilead Sciences, IQVIA, Janssen
Pharmaceutical Companies of Johnson & Johnson, Jazz Pharmaceuticals, Merck, Parexel, Pzer, Regeneron,
Roche/Genentech, Sano, Seagen, and Veristat. Working collaboratively with participating companies, Tufts CSDD
determined the study sampling frame, and selected and dened study variables of interest based, in part, on past Tufts
CSDD studies. Enhancements were made to the study methodology including measuring protocol writing cycle time;
determining whether an amendment halted the study or paused recruitment during implementation; gathering more
granular data on oversight of an amendment at the site level (i.e., date of rst and last site oversight approval of an
amendment); and assessing the impact of COVID-19 on amendment experience.
The data collection process was initiated in June 2022. Participating companies were asked to provide data on a
representative sample of up to 100 randomly selected protocols -- with and without amendments. For those protocols
with amendments, participating companies provided additional data about the nature, causes and impact of those
amendments. A unique protocol identier (NCT Number, protocol number, or other identier, provided by the companies)
was used to associate amendment data with the correct protocols.
The sampling requirements for protocols included and evaluated in this study were as follows:
Phase I – IV protocols, including pharmacology studies;
Excluding protocols for devices;
Including Traditional, adaptive and master protocol designs;
Achieved database lock (DBL) or primary completion date (PCD) between 2016 and 2021, with no restrictions on
protocol approval year;
Randomly selected regardless of the presence of amendments, to be representative of company portfolios;
CROs should provide data on protocols from companies not already participating in the study.
Data were collected on protocol-specic characteristics including number of endpoints, procedures, countries,
investigative sites engaged and patients enrolled; screening, recruitment and enrollment completion rates; planned and
actual timelines and budgets. Data were also collected on substantial and country-specic amendments. Substantial
amendments were dened as any change to a protocol on a global level requiring internal approval followed by approval
by a regulatory authority and oversight body (e.g., ethical review committee); country-specic amendments included any
change to the protocol that did not apply to all global locations. More granular data were only collected on substantial
amendments and included: milestone dates of amendment implementation (e.g., internal approval, date of rst oversight
approval, etc.), primary and auxiliary amendment causes, specic changes made in response to an amendment, and
direct and indirect amendment costs. Amendment causes included:
New Data Available (Other Than Safety Data)
New Safety Data Available
Change in Standard of Care
Regulatory Agency Request
Recruitment Diculty

Page 4/17
Manufacturing Change
Investigator / Site Feedback
Change in Study Strategy
Protocol Design Flaw
Inconsistency and / or Other Error in Protocol
COVID-19 Pandemic
Protocol design areas that were changed in response to implementing an amendment included General Information; Trial
Background; Objectives and Purpose; Trial Design; Selection, Withdrawal, and Treatment; Assessments; Statistics;
Operational; and Other.
All participating companies collected and coded their own data. Questions about variable denitions and data collection
methods were handled by the Tufts CSDD team. Data collection was completed in October 2022. All collected data were
reviewed and cleaned by the Tufts CSDD team. Numerical errors and incomplete data elements were returned to
participating companies for clarication and correction.
Data were merged and analyzed using SAS 9.4. Any protocols falling outside of the sampling requirements were removed
from the analysis dataset. Descriptive statistics were calculated and chi-square and analyses of variance (ANOVA) were
conducted to test for signicant differences in the prevalence and mean number of amendments by clinical trial
characteristics including phase, therapeutic area, use of DCT methodologies, and New Molecular Entity (NME) Type (large
molecule, small molecule, or vaccine). Mean, coecient of variation, median, and range were calculated for cycle times.
Amendment impact was measured by comparing the difference between planned and actual performance outcomes
including cycle times (in days), protocol budget, scope, and enrollment for protocols with and without amendments.
Change in enrollment performance—including Screen Failure (total enrolled / total screened) and Completion Rate (total
enrolled / total completed)—was also calculated based on planned and actual raw enrollment numbers for protocols with
and without amendments. Actual Screen Failure, Completion, and Dropout Rate (total dropouts / total enrolled) were
calculated for protocols with amendments before (pre-pandemic) and during the pandemic. These changes were
measured as a percent change from plan to account for differences in protocol size.
Causes of amendments, and the associated changes made during implementation, were assessed overall; for oncology
compared to non-oncology clinical trials; for clinical trials conducted pre- and during the COVID-19 pandemic; and by key
milestones in the timing of amendment implementation: before First Patient First Visit (FPFV), between FPFV and Last
Patient First Visit (LPFV), and after Last Patient First Visit (LPFV)).
Similar to past Tufts CSDD studies, participating companies classied amendments into four categories based on
primary cause: Completely Avoidable (e.g., Protocol Design Flaw; Inconsistency and / or Other Error in Protocol),
Somewhat Avoidable (e.g., Recruitment Diculty; Investigator/Site Feedback), Somewhat Unavoidable (e.g., New Data
Available (Other than Safety Data); Change in Standard of Care; Change in Study Strategy), and Completely Unavoidable
(e.g., New Safety Data Available; Regulatory Agency Request; Manufacturing Change; due to the COVID-19 Pandemic).
Results
Sixteen companies provided data on 950 qualifying protocols with 2,188 qualifying amendments. Completion rates of
individual variables in our sample were generally high; 87% of protocols contained general characteristics and 72% had
data on study scope, time and budget. Protocol design variables had lower relative completion rates of approximately
50%. Protocol amendment timeline and impact variables had an average completion rate just below 40%. Amendment
causes and changes had average completion rates of 57%. Cost data proved extremely dicult for participating

Page 5/17
companies to provide, with direct cost variables having a very low completion rate of only 1%; no participating companies
were able to provide indirect cost data. As such, we are unable to analyze and report on amendment cost benchmarks or
trends.
[Insert Table1]

Page 6/17
Table 1
Protocol Sample and Amendment Sample Characteristics
Characteristic Protocol Sample Amendment Sample
n
Percent
n
Percent
Phase    
 Phase I 358 37.7% 685 31.3%
 Phase II 251 26.4% 686 31.4%
 Phase III 286 30.1% 757 34.6%
 Phase IV 55 5.8% 60 2.7%
NME Type    
 Large Molecule 358 44.8% 1,058 53.1%
 Small Molecule 386 48.3% 825 41.4%
 Vaccine 56 7.0% 111 5.6%
Orphan Designation    
 Yes 41 8.3% 106 9.4%
 No 451 91.7% 1,021 90.6%
Therapeutic Area    
 Oncology 249 26.2% 779 35.6%
 Immunology / Rheumatology 152 16.0% 378 17.3%
 Neurology 150 15.8% 226 10.3%
 Infectious Disease 107 11.3% 216 9.9%
 Gastroenterology / Endocrinology 70 7.4% 130 5.9%
 Vaccines 55 5.8% 109 5.0%
 Dermatology 40 4.2% 63 2.9%
 Hematology 39 4.1% 110 5.0%
 Cardiology 30 3.2% 77 3.5%
 Respiratory 20 2.1% 28 1.3%
 Musculoskeletal 4 0.4% 10 0.5%
 Orthopedics 1 0.1% 0 0.0%
 Other 33 3.5% 62 2.8%
Participant Population    
 Patients 657 72.1% 1,741 83.6%
 Healthy Volunteers 238 26.1% 310 14.9%
 Both 16 1.8% 31 1.5%

Page 7/17
Characteristic Protocol Sample Amendment Sample
n
Percent
n
Percent
Phase    
Participant Age    
 Adult (18+) 795 89.7% 1,908 89.2%
 Pediatric (< 18) 59 6.7% 171 8.0%
 All Ages 32 3.6% 59 2.8%
Company Type    
 Sponsor 915 96.3% 2,096 96.9%
 CRO 35 3.7% 67 3.1%
*Amendment sample includes multiple amendments per protocol
Table1 presents the sample size and distribution of protocols and amendments analyzed. Overall, our sample included a
strong mix of protocols and amendments by phase, therapeutic area and molecule size.
Table2 presents the prevalence of amendments and the mean number of amendments implemented where amendments
were present. Overall, more than three quarters of protocols required at least one amendment, with a mean of 3.3
amendments implemented per protocol when required. Protocols for large molecules had a higher prevalence of
amendments and required a higher mean number to be implemented than did protocols for small molecules or vaccines.
Oncology protocols had a higher prevalence and a higher mean number of amendments at 90% and 4.0 respectively.
Protocols targeting diseases in immunology / rheumatology and protocols testing vaccines showed the next highest
prevalence of amendments (79% and 75%, respectively), however they required implementing a smaller mean number of
amendments than the overall average at 3.2 and 2.7, respectively.

Page 8/17
Table 2
Prevalence and Number of Amendments
Protocol Characteristic
n
Percent with Amendments When Amendments are Present
n
Mean (CoV) Median
Overall 864 75.8% 690 3.3 (0.74) 3.0
Phase     
 Phase I 326 65.3% 228 3.1 (0.74) 3.0
 Phase II 221 88.2% 212 3.3 (0.71) 3.0
 Phase III 269 82.2% 224 3.5 (0.76) 3.0
 Phase IV 48 54.2% 26 2.4 (0.59) 2.0
NME Type     
 Large Molecule 346 81.5% 294 3.7 (0.69) 3.0
 Small Molecule 362 70.2% 277 3.1 (0.78) 2.0
 Vaccine 56 75.0% 42 2.6 (0.69) 2.0
Orphan Designation     
 Yes 41 70.7% 29 3.7 (0.74) 3.0
 No 451 81.6% 368 2.8 (0.61) 3.0
Therapeutic Area     
 Oncology 198 89.9% 205 4.0 (0.67) 3.0
 Immunology / Rheumatology 148 79.1% 121 3.2 (0.79) 3.0
 Vaccines 55 74.6% 41 2.7 (0.70) 2.0
 Infectious Disease 106 74.5% 80 2.9 (0.63) 3.0
 Hematology 39 74.4% 29 3.9 (0.77) 3.0
 Cardiology 30 70.0% 21 3.6 (0.64) 3.0
 Neurology 121 69.4% 87 2.6 (0.60) 2.0
 Dermatology 40 65.0% 26 2.1 (0.54) 2.0
 Respiratory 20 65.0% 13 2.1 (0.42) 2.0
 Gastroenterology / Endocrinology 69 62.3% 43 3.1 (0.97) 2.0
 Musculoskeletal 4 50.0% 2 5.0 (0.28) 5.0
 Orthopedics 1 0.0% 0 - (-) -
 Other 33 66.7% 22 3.5 (0.86) 3.0
Although two-thirds of implemented amendments resulted in the need to reconsent study volunteers, less than 2%
resulted in pausing recruitment or halting the clinical trial. And, with respect to amendments implemented in a single or
partial number of geographic areas, phase II and III clinical trials had the highest proportion of country-specic
amendments at 44.8% and 60.1% of all protocols respectively.

Page 9/17
[Insert Tables2 and 3]
Table 3
Amendment Prevalence and Frequency Trends
2010 Study 2015 Study 2022 Study
Protocol
Characteristic % Protocols
w/
Amendments
Mean
Amendments,
When
Amendments
are Present
% Protocols
w/
Amendments
Mean
Amendments,
When
Amendments
are Present
% Protocols
w/
Amendments
Mean
Amendments,
When
Amendments
are Present
Overall 69% 2.4 57% 2.1 76% 3.3
Phase I  2.0 52% 1.8 65% 3.1
Phase II  2.6 77% 2.2 88% 3.3
Phase III  3.6 66% 2.3 82% 3.5
Phase IV  2.3 25% 1.9 54% 2.4
Gastrointestinal  4.4  2.3 100% 3.5
Cardiovascular  3.4  1.9 70% 3.6
Oncology  3.2  2.6 90% 4.0
Respiratory  3.2  1.3 65% 2.1
Hematology  2.3  1.5 74% 3.9
Immunology /
Inammation  1.8  2.1 76% 3.1
While the overall prevalence and mean number of amendments per protocol decreased between the 2010 and 2015
studies, the prevalence and mean number of amendments implemented per protocol both increased considerably
between 2015 and the 2022 time period (refer to Table3). Amendment prevalence has increased in each phase since
2015, as has the average number of amendments implemented per protocol when amendments were present. Compared
to 2010, the average number of amendments implemented when amendments were present also increased for every
phase except Phase III, where the average decreased modestly (3.6 in 2010 and 3.5 in 2022). The average number of
amendments has increased since 2015 in all therapeutic areas measured in our study with the exceptions of
gastrointestinal (4.4 in 2010 and 3.5 in 2022) and respiratory (3.2 in 2010 and 2.1 in 2022) disease protocols.
[Insert Table4]

Page 10/17
Table 4
Trends in Amendment Timing
Before FPFV During Enrollment
(FPFV – LPFV)
During Study Maintenance (After
LPFV)
Phase 2010
Study 2015
Study 2022
Study 2015
Study 2022
Study 2015 Study 2022 Study
Overall 43% 23% 24% 62% 58% 15% 18%
Phase I 52% 40% 25% 50% 63% 10% 13%
Phase II 37% 18% 26% 70% 54% 12% 19%
Phase
III 30% 15% 22% 65% 57% 20% 21%
Phase
IV 38% 33% 28% 67% 56% 0% 16%
Table4 examines trends in the timing of amendment implementation. Overall, the percentage of amendments occurring
before FPFV has decreased since 2010, but there has been minimal change between 2015 and 2022. Several trends are,
however, observed in the timing of amendment implementation by phase. Among Phase I protocols there has been a
steady decrease in the percentage of amendments occurring before FPFV from 52% in 2010, to 40% in 2015, down to 25%
in 2022. A rising trend is observed in the percentage of amendments occurring during enrollment from 50% in 2015 to
63% in 2022. In both Phase II and Phase III protocols, the percentage of amendments occurring before FPFV decreased
between 2010 and 2015 but increased between 2015 and 2022, with both phases showing a decrease in the percentage
of amendments occurring during enrollment between 2015 and 2022.
The reported percentage of unavoidable amendments – including those deemed ‘Somewhat Unavoidable’ and
‘Completely Unavoidable’ – has increased substantially since the 2010 and 2015 Tufts CSDD studies (refer to Table 5).
Table5 also shows that changes in clinical trial strategy and regulatory agency requests are consistently among the most
common primary causes of amendments in 2022. Protocol design aws and recruitment diculties were among the top
ve reported reasons for amending protocols in the earlier 2010 and 2015 Tufts CSDD studies.

Page 11/17
Table 5
Trends in Amendment Causes and Avoidability
Avoidability 2010 Study 2015 Study 2022 Study
Completely Avoidable 18% 23% 17%
Somewhat Avoidable 19% 22% 6%
Somewhat Unavoidable 27% 30% 37%
Completely Unavoidable 39% 25% 40%
Top 5 Reasons for
Amendments New safety information
available Protocol design change /
clarication Regulatory agency request
Regulatory agency
request Administrative Change in study strategy
Change in strategy /
objective Change to analysis New data available (other
than safety)
Protocol design aw Recruitment diculty Inconsistency / error in
protocol
Recruitment diculty Feedback from sites New safety data available
[Insert Table5]
Amendment implementation, measured from internal approval of the amendment to the last required approval by an
ethical review committee or oversight body takes — on average — approximately 260 days, with a median of 190 days
(refer to Table6). The duration from internal approval of an amendment to rst patient reconsented is 89 days in the
2022 study, more than 2.5 times longer than the duration observed in our 2010 study. During amendment implementation,
the duration of time in which sites are operating with different versions of a clinical trial protocol (i.e., time between the
rst required approval by an ethical review committee to the last required approval by an ethical review committee) was
just over 7 months (215 days).
Table 6
Amendment Implementation Cycle Times
Cycle Time (in Days)
n
Mean (CoV) Median Range
Amendment Implementation
(Internal Approval to Last Oversight Approval)
391 259.7 (0.95) 190.0 1–1,599
Submission Preparation
(Internal Approval to First Oversight Submission)
352 37.8 (3.06) 14.0 0–1,147
Time to First Approval
(Internal Approval to First Oversight Approval)
372 61.9 (1.97) 37.0 0–1,177
Time to Reconsent
(Internal Approval to First Patient Reconsented)
71 88.8 (1.36) 52.0 0–666
Duration of Sites on Different Protocol Versions
(First Oversight Approval to Last Oversight Approval)
393 214.8 (1.15) 140.0 0–1,583
[Insert Table6 ]

Page 12/17
Table7 presents mean actual cycle times for primary clinical trial time points for protocols with and without amendments
and shows trends in these cycle times since 2015. Protocols with amendments generally have longer cycle times than do
those without amendments. Protocols with at least one amendment are associated with larger and more complex studies
having nearly 25% more endpoints and 16% more eligibility criteria than do those without an amendment. The average
number of countries and investigative sites per protocol were also signicantly greater for protocols with amendments.
Protocols without at least one amendment screen and enroll 46% and 37% fewer patients, respectively.
Table 7
Cycle Times for Protocols with and without Amendments
Cycle Time Without Amendments With Amendments
p
-value
n
Mean
n
Mean
Study Initiation
(Protocol Approval – FPFV)
2015 Study 12 154 14 181 .45
2022 Study 198 176 666 181 .7457
Study Conduct
(Protocol Approval – LPLV)
2015 Study 235 490 477 580 .0003
2022 Study 200 546 617 1,121 < .0001
Study Closeout
(LPLV – DBL)
2015 Study 33 140 127 230 .007
2022 Study 195 58 564 49 .3868
First Visit Duration
(FPFV – LPFV)
2015 Study 215 238 286 403 < .0001
2022 Study 190 234 625 516 < .0001
Enrollment Completion
(FPFV – LPLV)
2015 Study 225 352 439 437 < .0001
2022 Study 195 370 617 942 < .0001
With respect to cycle time trends, study initiation durations did not change signicantly since 2015. Study conduct, rst
visit duration, and enrollment completion cycle times all increased signicantly. Moreover, the difference in cycle time
lengths between protocols with and without amendments has also widened since 2015.
[Insert Table7]
Enrollment performance in clinical trials without protocol amendments tends to perform closer to plan than do those that
implement at least one amendment. The actual number of patients screened, enrolled and completing protocols without
amendments was consistently closer to plan or forecast (see Table8). The difference between plan and actual durations
overall and for individual time points are also substantially longer for protocols with at least one amendment. Protocols
with at least one amendment require screening more patients and have much higher screen failure rates than initially
planned. However, they recruit and enroll far fewer patients than originally planned.

Page 13/17
Table 8
Percent Change in Enrollment and Cycle Times for Protocols with and without Amendments
% Change in – No Amendments With Amendments
n
Mean (CoV) Interpretation
n
Mean (CoV) Interpretation
Participants Screened 60 0.7% (95.40) Actual > Planned 339 8.6% (7.35) Actual > Planned
Participants Enrolled 176 -6.1% (5.97) Actual < Planned 620 -7.8% (4.90) Actual < Planned
Participants Completed 56 -3.9% (9.89) Actual < Planned 329 -20.8% (2.56) Actual < Planned
Screen Failure Rate 46 -6.9% (11.84) Actual < Planned 297 31.0% (9.73) Actual > Planned
Completion Rate 54 -0.7% (45.12) Actual < Planned 327 -19.3% (2.48) Actual < Planned
Study Startup
(Approval to FPFV)
126 2.8% (10.17) Actual > Planned 593 18.2% (4.43) Actual > Planned
First Visit Cycle
(FPFV to LPFV)
114 2.2% (28.95) Actual > Planned 529 14.1% (5.84) Actual > Planned
LP Participation Cycle
(LPFV to LPLV)
110 45.9% (8.56) Actual > Planned 504 34.8% (6.24) Actual > Planned
Closeout
(LPLV to DBL)
72 27.6% (7.99) Actual > Planned 440 59.0% (4.01) Actual > Planned
Study Reporting
(Database Lock to CSR)
69 34.0% (4.33) Actual > Planned 433 32.5% (8.43) Actual > Planned
Enrollment Duration
(FPFV – LPLV)
123 2.4% (14.01) Actual > Planned 567 14.0% (5.66) Actual > Planned
Total Study Duration
(Approval to DBL)
77 3.8% (6.94) Actual > Planned 463 13.9% (3.26) Actual > Planned
[Insert Table8]
Protocols that completed their last patient’s last visit by March 1st, 2020 were considered to have occurred before the
onset of the COVID-19 pandemic. Those that completed LPLV after that date screened, recruited and enrolled patients
during the COVID-19 pandemic. Table9 presents comparisons between plan and actual scope and patient recruitment
and retention rates for protocols conducted before and during the pandemic. The results show signicantly higher relative
increases in the number of endpoints and countries added to protocols requiring amendments
during
the pandemic. The
mean number of amendments per protocol increased signicantly during the pandemic, from 3.0 to 3.5 (
p
= .0063). A
signicant decrease in screen failure rates (34.8–31.3%,
p
= .0362) and completion rates (71.5–65.0%,
p
= .0193), and a
signicant increase in dropout rates (18.8–29.6%,
p
< .0001) are associated with protocols that were amended during the
pandemic.

Page 14/17
Table 9
Comparing Protocols Before and During the Pandemic
Pre-Pandemic Pandemic 

n
Mean (CoV) Median
n
Mean (CoV) Median
p
-value
Amendments* 302 3.0 (0.74) 2.0 319 3.5 (0.70) 3.0 .0063
Screen Failure Rate 383 34.8% (0.69) 32.5% 380 31.3% (0.72) 28.6% .0362
Completion Rate 350 71.5% (0.49) 88.0% 323 65.0% (0.56) 79.6% .0193
Dropout Rate 313 18.8% (1.37) 8.9% 311 29.6% (1.14) 15.4% < .0001
Change in Scope*
(Planned vs Actual)
      
Countries 251 14.5% (9.70) 0.0% 259 56.7% (5.38) 0.0% .0448
Sites 275 -1.6% (24.81) 0.0% 296 -7.4% (7.86) -2.3% .1669
Endpoints 202 7.7% (4.24) 0.0% 218 17.5% (3.10) 0.0% .0245
Eligibility Criteria 203 3.9% (5.15) 0.0% 221 6.0% (5.55) 0.0% .4220
*When amendments are present 
Discussion
Despite high industry-wide awareness and recognition of the unplanned time and unbudgeted cost associated with
implementing protocol amendments, the prevalence and mean number of amendments per protocol have increased
across all clinical phases since 2015. Overall, the average number of amendments per protocol in 2022 increased nearly
60% between 2015 and 2022. More specically, phase I and III protocols saw the highest increases in the mean number of
amendments implemented per protocol over this seven-year period. This study found that more than 75% of protocols
require at least one substantial amendment with the highest prevalence (89%) observed in phase II protocols. And clinical
teams now implement an average of 3.3 amendments per protocol with the highest mean number (3.5) in Phase III
clinical trials. Compared to protocols without amendments, those with at least one amendment were larger in scope and
took signicantly longer overall as well as at each critical clinical trial segment (e.g., initiation, conduct, close out).
The upward trend in the prevalence and frequency of amendments may be explained, in part, by the strong positive
relationship between protocol complexity (e.g., higher relative number of endpoints, eligibility criteria, procedures,
countries and investigative sites) and amendments and the steady increase in the scientic and operating complexity of
protocol designs observed by Tufts CSDD over the past decade. The most notable factors driving complexity at this time
include the growing number of drugs in active clinical trials targeting rare diseases, more narrowly dened patient
populations and more logistically demanding and complicated clinical trial execution models involving more countries,
investigative sites, technology solutions and primary data sources (e.g., mobile and wearable devices; specialized
assessments), .
Although the gap between actual durations and those initially planned were wider for protocols with amendments than
for those without, protocols that implemented at least one amendment were more effective at increasing patient
screening volume and reducing the actual number of patients enrolled relative to plan.

Page 15/17
A much higher percentage of amendments (77%) were deemed unavoidable, up from 66% in the 2010 Tufts CSDD study.
This nding may reect changing sponsor perceptions that a growing majority of amendments are necessary. This trend
corresponds with the increase in the percentage of sponsor companies reporting that regulatory agency requests and
changes in study strategy were top reasons for amending a protocol. Intensifying competition from a larger community
of active sponsors during drug development through commercialization, increasing formal and informal sponsor-
regulatory agency interactions, and growing agency demand for patient-centric clinical trial designs and executional
models may help explain the prominence of these reasons at the present time for amending protocols.
The results of this study characterize the stunning toll that protocol amendment implementation takes on clinical trial
timelines and the potential for high levels of uncertainty between investigative sites administering the protocol. The total
average time to implement an amendment for example — from identifying the need-to-amend to last oversight approval —
was a remarkable 260 days. During implementation, the average duration during which investigative sites were operating
with potentially different versions of the clinical trial protocol was 215 days. And the results showed a 154% increase
since 2010 in the time between internal approval-to-implement-an- amendment and re-consent of the rst patient.
We were not surprised to observe an increase in the prevalence and mean number of amendments and the corresponding
increase in the number of endpoints and countries during the pandemic. The unique challenges of conducting clinical
trials during the pandemic — including the extra precautions taken and social distancing required, heightened levels of
remote interaction and concerns about transmitting the COVID-19 virus — all contributed to greater uncertainty and
anxiety among study volunteers and study staff. Indeed the results indicated that protocols conducted during the
pandemic had signicantly higher study volunteer dropout rates.
In the Tufts CSDD 2010 and 2015 studies we advocated for protocol simplication and a focus on preventing avoidable
amendments — particularly those implemented before study volunteers received the rst dose of the investigational drug
— as primary strategies to minimize the highly disruptive nature and delays associated with substantial amendments.
The results of this recent study suggest that these strategies may be only marginally effective. Protocol design
complexity has been rising and the proportion of avoidable amendments has been declining since 2015.
A separate but recent Tufts CSDD study, however, found that soliciting input into draft protocol designs from patients and
investigative sites did improve clinical trial performance and reduced the number of protocol amendments. Notable
differences were observed in protocols informed by a patient advisory board including faster study initiation, clinical trial
and study close-out durations, lower-than-planned study budgets, and a signicantly smaller average number of
substantial protocol amendments.
Results from the 2022 study also point to the importance of strategies that hold greater potential to drive more ecient
and shorter protocol amendment implementation timelines. Technology solutions that improve communication and
coordination between internal teams and collaborators (e.g., CROs, global investigative sites and oversight bodies)
through faster and more ecient data-sharing will play an important role. Over time, automation and AI-enabled
approaches may also help drive faster and more ecient implementation of amendments.
This study had limitations of note. The data collected came from a biased sample of primarily mid-sized and major
pharmaceutical companies. However, we have no data that evidences that protocol amendment experience are different
in clinical trials sponsored by small companies. Due to the timing of this study, we do not have results that characterize
protocol amendment experience post pandemic. Tufts CSDD plans to conduct a follow-up study in the coming years to
inform our understanding of experiences after the ‘ocial’ end of the global pandemic. We were also unable to collect
data on the cost to implement amendments in part due to the diculty that participating companies had in accessing
data from other functions in their organization and from vendors. Tufts CSDD is currently planning to do a follow-up
study focusing on gathering robust protocol amendment cost data.

Page 16/17
The highly disruptive and unplanned impact of substantial protocol amendments on clinical trial timelines and costs
combined with their growing prevalence and frequency point to a priority challenge, and an unprecedented opportunity, to
drive drug development eciency and speed.
Declarations
Acknowledgments
Thank you to the many companies that participated and supported this working group study: Amgen, AstraZeneca,
Biogen, Bristol Myers Squibb, EMD Serono, Eli Lilly & Company, Gilead Sciences, IQVIA, Janssen Pharmaceutical
Companies of Johnson & Johnson, Jazz Pharmaceuticals, Merck, Parexel, Pzer, Regeneron, Roche/Genentech, Sano,
Seagen, and Veristat.
Conict of Interest Statement
Kenneth Getz, Tufts CSDD, has nothing to disclose.
Zachary Smith, Tufts CSDD, has nothing to disclose.
Emily Botto, Tufts CSDD, has nothing to disclose.
Elisabeth Murphy, Janssen Pharmaceutical Companies of Johnson & Johnson,declares that she is an employee and has
nancial holdings in the company.
Arnaud Dauchy, Sano, declares that he is an employee and has nancial holdings in the company.
Author Contributions:
Kenneth Getz, Tufts CSDD, contributed to all four aspects (substantial contribution to conception, design, analysis,
interpretation; drafting and revising the work; nal approval of the version to be published; agreement to be accountable
for all aspects in ensuring accuracy and integrity of the work)
Zachary Smith, Tufts CSDD, contributed to all four aspects;
Emily Botto, Tufts CSDD, contributed to all four aspects;
Elisabeth Murphy, Janssen Pharmaceutical Companies of Johnson & Johnson,made substantial contribution to
conception, design, analysis, interpretation.
Arnaud Dauchy, Sano, made substantial contribution to conception, design, analysis, interpretation.
References
1. Getz K, Stergiopoulos S, Short M et al. The impact of protocol amendments on clinical trial performance and cost.
Therapeutic Innovation and Regulatory Science
. 2016; 50(4): 436–441.
2. Ibid
3. Kaitin K. Cite Impact Report
4. Getz K, Zuckerman R, Cropp A, Hindle A, Krauss R, Kaitin K. Measuring the incidence, causes, and repercussions of
protocol amendments.
Drug Information Journal
2011; 45: 265 − 75.

Page 17/17
5. Getz K, Stergiopoulos S, Short M et al. The impact of protocol amendments on clinical trial performance and cost.
Therapeutic Innovation and Regulatory Science
. 2016; 50(4): 436–441.
. Getz K, Smith Z, Kravet M. Protocol design and performance benchmarks by phase and by oncology and rare disease
subgroups. TIRS 2023; 57(1): 49–56.
7. Cite IQVIA Report
. Cite Getz – new benchmarks paper
9. Cite IQVIA and/or Evaluate Pharma Report
10. Getz K. Amplifying patient voices in protocol design.
Applied Clinical Trials
, 2021; 30 (9): 10–12.