Improving Clinical Trial Participant Prescreening With Artificial Intelligence (AI): A Comparison of the Results of AI-Assisted vs Standard Methods in 3 Oncology Trials

To read the full-text of this research, you can request a copy directly from the authors.


Background:: Delays in clinical trial enrollment and difficulties enrolling representative samples continue to vex sponsors, sites, and patient populations. Here we investigated use of an artificial intelligence-powered technology,, as a means of overcoming bottlenecks and potential biases associated with standard patient prescreening processes in an oncology setting. Methods:: was applied retroactively to 2 completed oncology studies (1 breast, 1 lung), and 1 study that failed to enroll (lung), at the Comprehensive Blood and Cancer Center, allowing direct comparison between results achieved using standard prescreening practices and results achieved with Outcome variables included the number of patients identified as potentially eligible and the elapsed time between eligibility and identification. Results:: For each trial that enrolled, use of resulted in a 24% to 50% increase over standard practices in the number of patients correctly identified as potentially eligible. No patients correctly identified by standard practices were missed by For the nonenrolling trial, both approaches failed to identify suitable patients. An average of 19 days for breast and 263 days for lung cancer patients elapsed between actual patient eligibility (based on clinical chart information) and identification when the standard prescreening practice was used. In contrast, ascertainment of potential eligibility using took minutes. Conclusions:: This study suggests that augmentation of human resources with artificial intelligence could yield sizable improvements over standard practices in several aspects of the patient prescreening process, as well as in approaches to feasibility, site selection, and trial selection.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... This would enable improved decision-making regarding: (i) medical treatments, especially oral supplements known to decrease progression risk, (ii) lifestyle interventions, particularly smoking cessation and dietary changes, and (iii) intensity of patient monitoring, e.g., frequent reimaging in clinic and/or tailored home monitoring programs [4][5][6][7][8] . It would also aid the design of future clinical trials, which could be enriched for participants with a high risk of progression events 9 . ...
... In addition, unlike the SSS, whose 5-year risk prediction becomes saturated at 50% 10 , the DL models enable ascertainment of risk above 50%. This may be helpful in justifying medical and lifestyle interventions 4,5,8 , vigilant home monitoring 6,7 , and frequent reimaging 39 , and in planning shorter but highly powered clinical trials 9 . For example, the AREDS-style oral supplements decrease the risk of developing late AMD by~25%, but only in individuals at higher risk of disease progression 4,5 . ...
Full-text available
By 2040, age-related macular degeneration (AMD) will affect ~288 million people worldwide. Identifying individuals at high risk of progression to late AMD, the sight-threatening stage, is critical for clinical actions, including medical interventions and timely monitoring. Although deep learning has shown promise in diagnosing/screening AMD using color fundus photographs, it remains difficult to predict individuals’ risks of late AMD accurately. For both tasks, these initial deep learning attempts have remained largely unvalidated in independent cohorts. Here, we demonstrate how deep learning and survival analysis can predict the probability of progression to late AMD using 3298 participants (over 80,000 images) from the Age-Related Eye Disease Studies AREDS and AREDS2, the largest longitudinal clinical trials in AMD. When validated against an independent test data set of 601 participants, our model achieved high prognostic accuracy (5-year C-statistic 86.4 (95% confidence interval 86.2–86.6)) that substantially exceeded that of retinal specialists using two existing clinical standards (81.3 (81.1–81.5) and 82.0 (81.8–82.3), respectively). Interestingly, our approach offers additional strengths over the existing clinical standards in AMD prognosis (e.g., risk ascertainment above 50%) and is likely to be highly generalizable, given the breadth of training data from 82 US retinal specialty clinics. Indeed, during external validation through training on AREDS and testing on AREDS2 as an independent cohort, our model retained substantially higher prognostic accuracy than existing clinical standards. These results highlight the potential of deep learning systems to enhance clinical decision-making in AMD patients.
Full-text available
The widespread adoption of electronic health records (EHRs) and the growing wealth of digitized information sources about patients is ushering in an era of 'Big Data' that may revolutionize clinical research in oncology. Research will likely be more efficient and potentially more accurate than the current gold standard of manual chart review studies. However, EHRs as they exist today have significant limitations: important data elements are missing or are only captured in free text or PDF documents. Using two case studies, we illustrate the challenges of leveraging the data that are routinely collected by the healthcare system in EHRs (e.g., real-world data), specific challenges encountered in the cancer domain and opportunities that can be achieved when these are overcome.
Full-text available
Concern about additional costs for direct patient care impedes efforts to enroll patients in clinical trials. But generalizable evidence substantiating these concerns is lacking. To assess the additional cost of treating cancer patients in the National Cancer Institute (NCI)-sponsored clinical trials in the United States across a range of trial phases, treatment modalities, and patient care settings. Retrospective cost study using a multistage, stratified, random sample of patients enrolled in 1 of 35 active phase 3 trials or phase 1 or any phase 2 trials between October 1, 1998, and December 31, 1999. Unadjusted and adjusted costs were compared and related to trial phase, institution type, and vital status. A representative sample of 932 cancer patients enrolled in nonpediatric, NCI-sponsored clinical trials and 696 nonparticipants with a similar stage of disease not enrolled in a research protocol from 83 cancer clinical research institutions across the United States. Direct treatment costs as measured using a combination of medical records, telephone survey, and Medicare claims data. Administrative and other research costs were excluded. The incremental costs of direct care in trials were modest. Over approximately a 2.5-year period, adjusted costs were 6.5% higher for trial participants than nonparticipants (35,418 dollars vs 33,248 dollars; P =.11). Cost differences for phase 3 studies were 3.5% (P =.22), lower than for phase 1 or 2 trials (12.8%; P =.20). Trial participants who died had higher costs than nonparticipants who died (17.9%; 39,420 dollars vs 33,432 dollars, respectively; P =.15). Treatment costs for nonpediatric clinical trial participants are on average 6.5% higher than what they would be if patients did not enroll. This implies total incremental treatment costs for NCI-sponsored trials of 16 million dollars in 1999. Incremental costs were higher for patients who died and who were in early phase studies although these findings deserve further scrutiny. Overall, the additional treatment costs of an open reimbursement policy for government-sponsored cancer clinical trials appear minimal.
Background: Only 3-5% of cancer patients participate in clinical trials even though up to 20% are eligible. Cognitive computing has promising potential to assist trial enrollment efficiency and accuracy by performing background analytics. The Watson for Clinical Trial Matching (CTM) cognitive system utilizes natural language processing to derive patient and tumor attributes from unstructured text in the electronic health record that can be matched to complex eligibility criteria in trial protocols. Screening patients for trials was performed on an ad hoc basis with traditional methods prior to implementation of the CTM system in the Mayo Clinic breast oncology practice. Methods: The Watson CTM system was trained by Mayo subject matter experts and implemented in July 2016. Systemic therapy trials enrolling breast cancer patients were included in the system. Clinical research coordinators validated Watson-derived clinical trial matches on the day prior to patient clinic visits. They gave the oncology providers a list of matched trials for each patient to facilitate treatment decision making at point of care. Enrollment and timing metrics were tracked and compared with manual screening methods. Results: Watson CTM facilitated screening of all breast cancer patients for systemic therapy trials with matches validated by coordinators in 40% of patients. Over the 18 month (mo) period following implementation, 6.3 patients/mo were enrolled to breast cancer systemic therapy trials compared with 3.5 patients/mo in the period prior. The average monthly enrollment increased by 80%. This was further increased to 8.1 patients/mo when including accruals to breast cancer cohorts of phase I trials within the experimental therapeutics program. Time to match patients to trials with the CTM system was faster than manual methods but variable depending on the role of the screener and the depth of the matching. Conclusions: Implementation of the Watson for CTM system with a screening coordinator team was associated with an increase in breast cancer clinical trial enrollment. The system enabled high volume screening in an efficient manner and promoted awareness of clinical trial opportunities within the breast oncology practice.
Background: Site identification, site selection, and study start-up have become the focus of improvement by organizations conducting clinical trials. Methods: To examine and measure the process from site identification through site activation, Tufts Center for the Study of Drug Development (CSDD) conducted a comprehensive survey among pharmaceutical organizations, biotech companies, and contract research organizations (CROs). Responses from over 400 unique companies were gathered and analyzed. Results: The results indicate that the start-up process is on average 5 to 6 months in total duration, and cycle times across all activities, including site identification, site selection, and study start-up, are faster for repeat sites than for new sites. Comparisons between sponsor and CROs indicate that CROs completed all site-related activities 6 to 11 weeks faster than sponsors. Other areas impacting cycle times were examined, including centralized versus decentralized functions, investment in technology, and organizational strategies that improve cycle time efficiency and performance. Conclusion: Tufts CSDD will explore this area in future research to gather additional insights into other factors that may be associated with speed and efficiency.
To investigate the barriers, modifiers, and benefits involved in participating in randomized controlled trials of cancer therapies as perceived by health care providers and patients. We conducted a systematic review of the literature to identify published and unpublished studies in any language using electronic databases searched from 1996 to 2004, contact with experts, and reference lists. All study designs were acceptable provided relevant data were reported. Two reviewers were involved in the selection of studies, data extraction, and quality assessment processes. Studies were combined in a narrative synthesis. Fifty-six studies met the inclusion criteria and represented the perspective of the patient or the health care provider or both. Although a range of barriers to trial participation were identified, a number of threats to the internal and external validity of the included studies limited interpretation of the evidence. The limitations within the evidence base do not permit a clear interpretation of the barriers, moderators, and benefits involved in participation in cancer trials. We recommend that trialists prospectively identify the issues relevant to a particular trial using the current research as a starting point. We report checklists to guide this process.
Impact of a cognitive computing clinical trial matching system in an ambulatory oncology practice
  • T Haddad
  • J Helgeson
  • K Pomerteau
Haddad T, Helgeson J, Pomerteau K, et al. Impact of a cognitive computing clinical trial matching system in an ambulatory oncology practice. Abstract presented at American Society of Clinical Oncology (ASCO) Annual Meeting, 2018.
Enrollment performance: weighing the "facts
  • K Getz
Getz K. Enrollment performance: weighing the "facts." Appl Clin Trials. 2012;21(5).