No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Improving Clinical Trial Participant Prescreening With Artificial Intelligence (AI): A Comparison of the Results of AI-Assisted vs Standard Methods in 3 Oncology Trials
Abstract
Background::
Delays in clinical trial enrollment and difficulties enrolling representative samples continue to vex sponsors, sites, and patient populations. Here we investigated use of an artificial intelligence-powered technology, Mendel.ai, as a means of overcoming bottlenecks and potential biases associated with standard patient prescreening processes in an oncology setting.
Methods::
Mendel.ai was applied retroactively to 2 completed oncology studies (1 breast, 1 lung), and 1 study that failed to enroll (lung), at the Comprehensive Blood and Cancer Center, allowing direct comparison between results achieved using standard prescreening practices and results achieved with Mendel.ai. Outcome variables included the number of patients identified as potentially eligible and the elapsed time between eligibility and identification.
Results::
For each trial that enrolled, use of Mendel.ai resulted in a 24% to 50% increase over standard practices in the number of patients correctly identified as potentially eligible. No patients correctly identified by standard practices were missed by Mendel.ai. For the nonenrolling trial, both approaches failed to identify suitable patients. An average of 19 days for breast and 263 days for lung cancer patients elapsed between actual patient eligibility (based on clinical chart information) and identification when the standard prescreening practice was used. In contrast, ascertainment of potential eligibility using Mendel.ai took minutes.
Conclusions::
This study suggests that augmentation of human resources with artificial intelligence could yield sizable improvements over standard practices in several aspects of the patient prescreening process, as well as in approaches to feasibility, site selection, and trial selection.
... This would enable improved decision-making regarding: (i) medical treatments, especially oral supplements known to decrease progression risk, (ii) lifestyle interventions, particularly smoking cessation and dietary changes, and (iii) intensity of patient monitoring, e.g., frequent reimaging in clinic and/or tailored home monitoring programs [4][5][6][7][8] . It would also aid the design of future clinical trials, which could be enriched for participants with a high risk of progression events 9 . ...
... In addition, unlike the SSS, whose 5-year risk prediction becomes saturated at 50% 10 , the DL models enable ascertainment of risk above 50%. This may be helpful in justifying medical and lifestyle interventions 4,5,8 , vigilant home monitoring 6,7 , and frequent reimaging 39 , and in planning shorter but highly powered clinical trials 9 . For example, the AREDS-style oral supplements decrease the risk of developing late AMD by~25%, but only in individuals at higher risk of disease progression 4,5 . ...
By 2040, age-related macular degeneration (AMD) will affect ~288 million people worldwide. Identifying individuals at high risk of progression to late AMD, the sight-threatening stage, is critical for clinical actions, including medical interventions and timely monitoring. Although deep learning has shown promise in diagnosing/screening AMD using color fundus photographs, it remains difficult to predict individuals’ risks of late AMD accurately. For both tasks, these initial deep learning attempts have remained largely unvalidated in independent cohorts. Here, we demonstrate how deep learning and survival analysis can predict the probability of progression to late AMD using 3298 participants (over 80,000 images) from the Age-Related Eye Disease Studies AREDS and AREDS2, the largest longitudinal clinical trials in AMD. When validated against an independent test data set of 601 participants, our model achieved high prognostic accuracy (5-year C-statistic 86.4 (95% confidence interval 86.2–86.6)) that substantially exceeded that of retinal specialists using two existing clinical standards (81.3 (81.1–81.5) and 82.0 (81.8–82.3), respectively). Interestingly, our approach offers additional strengths over the existing clinical standards in AMD prognosis (e.g., risk ascertainment above 50%) and is likely to be highly generalizable, given the breadth of training data from 82 US retinal specialty clinics. Indeed, during external validation through training on AREDS and testing on AREDS2 as an independent cohort, our model retained substantially higher prognostic accuracy than existing clinical standards. These results highlight the potential of deep learning systems to enhance clinical decision-making in AMD patients.
... The selection of effective investigation sites is one of the most crucial trial components. Site characteristics that can affect both research durations and data quality and integrity include administrative practices, resource availability, and clinicians with extensive experience and knowledge of the disease [16]. Clinical research organizations (CROs) can use AI technology to find target sites, qualified investigators, and priority candidates. ...
Clinical trials are essential for delivering novel medications, technology, and procedures to the market and clinical practice. Only 10% of these studies complete the entire procedure from the drug design to the four phases of development, because clinical trials are becoming more expensive and difficult to perform. The population's health, standard treatment, health economics, and sustainability suffered greatly from this low completion rate. Artificial intelligence (AI) is one of the tools that could streamline some of the processes which are the most tedious operations, like patient selection, matching, and enrollment; better patient selection could also minimize harmful treatment and its side effects. The widespread implementation of AI technology in clinical trials still faces many challenges and requires more high-quality prospective clinical validation. In this review, we discussed the prospective applications of AI in clinical research and patient care in the future.
... Various machine learning models have been employed in clinical trials to improve efficiency and scalability [143,144], from the protocol design and approval [145], the participant recruitment and the execution [146], to the results management and analysis [147]. For example, in a clinical trial, investigating the combined therapy with methotrexate and adalimumab for non-infectious non-anterior uveitis, Ana Rivase et al. [148] developed random forest prediction models with genetic and proteomic biomarkers to identify subgroups of participants with different treatment responses. ...
The eyes possess sophisticated physiological structures, diverse disease targets, limited drug delivery space, distinctive barriers, and complicated biomechanical processes, requiring a more in-depth understanding of the interactions between drug delivery systems and biological systems for ocular formulation development. However, the tiny size of the eyes makes sampling difficult and invasive studies costly and ethically constrained. Developing ocular formulations following conventional trial-and-error formulation and manufacturing process screening procedures is inefficient. Along with the popularity of computational pharmaceutics, non-invasive in silico modeling & simulation offer new opportunities for the paradigm shift of ocular formulation development. The current work first systematically reviews the theoretical underpinnings, advanced applications, and unique advantages of data-driven machine learning and multiscale simulation approaches represented by molecular simulation, mathematical modeling, and pharmacokinetic (PK)/pharmacodynamic (PD) modeling for ocular drug development. Following this, a new computer-driven framework for rational pharmaceutical formulation design is proposed, inspired by the potential of in silico explorations in understanding drug delivery details and facilitating drug formulation design. Lastly, to promote the paradigm shift, integrated in silico methodologies were highlighted, and discussions on data challenges, model practicality, personalized modeling, regulatory science, interdisciplinary collaboration, and talents training were conducted in detail with a view to achieving more efficient objective-oriented pharmaceutical formulation design.
... A crossmodal inference learning model technique may more successfully match patients to trials using EHR data by concurrently encoding enrollment criteria (text) and patient records (tabular data) into a shared latent space [86]. The utility of these procedures is questioned by the lack of peer-reviewed documentation of their development and performance measures [87]. Mendel.AI and Deep6AI are two businesses that provide comparable services. ...
Machine learning and Artificial Intelligence have significantly advanced in recent years owing to their potential to considerably increase the quality of life while reducing human workload. The paper demonstrates how AI and ML are used in the drug development process to shorten and enhance the overall timeline. It contains pertinent information on a variety of Machine Learning approaches and algorithms that are used across the whole drug development process to speed up research, save expenses, and reduce risks related to clinical trials. A range of QSAR analysis, hit finding, and de novo drug design applications are used in the pharmaceutical industry to enhance decision-making. As technologies like high-throughput screening and computation analysis of databases used for lead and target identification and development create and integrate vast volumes of data, machine learning and deep learning have grown in importance. It has also been emphasized how these cognitive models and tools may be used in lead creation, optimization, and thorough virtual screening. In this paper, problem statements and the corresponding state-of-the-art models have been considered for target validation, prognostic biomarkers, and digital pathology. Machine Learning models play a vital role in the various operations related to clinical trials embracing protocol optimization, participant management, data analysis and storage, clinical trial data verification, and surveillance. Post-development drug monitoring and unique industrially prevalent ML applications of pharmacovigilance have also been discussed. As a result, the goal of this study is to investigate the machine learning and deep learning algorithms utilised across the drug development lifecycle as well as the supporting techniques that have the potential to be useful.
... As investments in clinical trials increase in parallel, their failures are causing very large economic losses [2][3][4]. Unfortunately, translation from preclinical to clinical studies is poor, in contrast to the recent advances in computerized tools for increasing trial success rates [5,6]. Begley et al. pointed out that some discoveries could not even be replicated by the original researchers [7], and claims that most new biological discoveries would not withstand the test of time [8]. ...
Aims: Methodological rigour enhances reproducibility in preclinical studies and translation from preclinical to clinical studies. We aimed to investigate the prevalence and the trends of essential study design elements in preclinical urological studies, as well as key factors which may improve methodological rigour.
Methods and Results: PubMed database was searched, and all the resulting articles in preclinical urological articles published over the past 14-years were reviewed. Total 3768 articles met inclusion criteria. Data on study design elements and animal model used were collected. Citation density and journal impact factor was also examined as a surrogate marker of study influence. We performed analysis on prevalence of seven critical study design elements, and temporal patterns over 14 years. Randomization was reported in 50.0%, blinding in 15.0%, sample size estimation in 1.0%, inclusion of both sexes in 5.7%, statistical analysis in 97.1%, housing and husbandry in 47.7%, and inclusion/exclusion criteria in 5.0%. Temporal analysis showed that the implementation of these study design elements has increased, except for inclusion of both sexes and inclusion/exclusion criteria. Reporting study design elements were not associated with increased citation density.
Conclusions: The risk of bias is prevalent in 14-year publications describing preclinical urological research, and the quality of methodological rigour is poorly related to the journal impact factor or the citation of the article. Yet guidelines seem helpful in improving the research quality, because five study design elements (randomization, blinding, sample size estimation, statistical analysis, housing and husbandry) proposed by both NIH and ARRIVE guidelines have been either well reported or improved.
Systematic review registration: PROSPERO CRD42022233125
One-sentence summary: Research bias still exists in the fields in preclinical urology, but it is gradually improving.
Background:
The aim of this study is to provide a comprehensive understanding of the current landscape of AI for cancer clinical trial enrolment, and its predictive accuracy in identifying eligible patients for inclusion in such trials.
Methods:
Databases of PubMed, Embase and Cochrane CENTRAL were searched until June 2022. Articles were included if they reported on AI actively being used in the clinical trial enrolment process. Narrative synthesis was conducted amongst all extracted data-accuracy, sensitivity, specificity, positive predictive value, negative predictive value. For studies where the 2x2 contingency table could be calculated or supplied by authors, a meta-analysis to calculate summary statistics was conducted using the hierarchical summary receiver operating characteristics curve model.
Results:
Ten articles, reporting on over 50,000 patients in 19 datasets were included. Accuracy, sensitivity and specificity exceeded 80% in all but one dataset. Positive predictive value exceeded 80% in 5 of 17 datasets. Negative predictive value exceeded 80% in all datasets. Summary sensitivity was 90.5% (95% CI: 70.9%-97.4%); summary specificity was 99.3% (95% CI: 81.8%-99.9%).
Conclusions:
AI demonstrated comparable, if not superior performance to manual screening for patient enrolment into cancer clinical trials. As well, AI is highly efficient, requiring less time and human resources to screen patients. AI should be further investigated and implemented for patient recruitment into cancer clinical trials. Future research should validate the use of AI for clinical trials enrolment in less resource-rich regions, and to ensure broad inclusion for generalizability to all genders, ages and ethnicities.
The widespread adoption of electronic health records (EHRs) and the growing wealth of digitized information sources about patients is ushering in an era of 'Big Data' that may revolutionize clinical research in oncology. Research will likely be more efficient and potentially more accurate than the current gold standard of manual chart review studies. However, EHRs as they exist today have significant limitations: important data elements are missing or are only captured in free text or PDF documents. Using two case studies, we illustrate the challenges of leveraging the data that are routinely collected by the healthcare system in EHRs (e.g., real-world data), specific challenges encountered in the cancer domain and opportunities that can be achieved when these are overcome.
Concern about additional costs for direct patient care impedes efforts to enroll patients in clinical trials. But generalizable evidence substantiating these concerns is lacking.
To assess the additional cost of treating cancer patients in the National Cancer Institute (NCI)-sponsored clinical trials in the United States across a range of trial phases, treatment modalities, and patient care settings.
Retrospective cost study using a multistage, stratified, random sample of patients enrolled in 1 of 35 active phase 3 trials or phase 1 or any phase 2 trials between October 1, 1998, and December 31, 1999. Unadjusted and adjusted costs were compared and related to trial phase, institution type, and vital status.
A representative sample of 932 cancer patients enrolled in nonpediatric, NCI-sponsored clinical trials and 696 nonparticipants with a similar stage of disease not enrolled in a research protocol from 83 cancer clinical research institutions across the United States.
Direct treatment costs as measured using a combination of medical records, telephone survey, and Medicare claims data. Administrative and other research costs were excluded.
The incremental costs of direct care in trials were modest. Over approximately a 2.5-year period, adjusted costs were 6.5% higher for trial participants than nonparticipants (35,418 dollars vs 33,248 dollars; P =.11). Cost differences for phase 3 studies were 3.5% (P =.22), lower than for phase 1 or 2 trials (12.8%; P =.20). Trial participants who died had higher costs than nonparticipants who died (17.9%; 39,420 dollars vs 33,432 dollars, respectively; P =.15).
Treatment costs for nonpediatric clinical trial participants are on average 6.5% higher than what they would be if patients did not enroll. This implies total incremental treatment costs for NCI-sponsored trials of 16 million dollars in 1999. Incremental costs were higher for patients who died and who were in early phase studies although these findings deserve further scrutiny. Overall, the additional treatment costs of an open reimbursement policy for government-sponsored cancer clinical trials appear minimal.
Background: Only 3-5% of cancer patients participate in clinical trials even though up to 20% are eligible. Cognitive computing has promising potential to assist trial enrollment efficiency and accuracy by performing background analytics. The Watson for Clinical Trial Matching (CTM) cognitive system utilizes natural language processing to derive patient and tumor attributes from unstructured text in the electronic health record that can be matched to complex eligibility criteria in trial protocols. Screening patients for trials was performed on an ad hoc basis with traditional methods prior to implementation of the CTM system in the Mayo Clinic breast oncology practice. Methods: The Watson CTM system was trained by Mayo subject matter experts and implemented in July 2016. Systemic therapy trials enrolling breast cancer patients were included in the system. Clinical research coordinators validated Watson-derived clinical trial matches on the day prior to patient clinic visits. They gave the oncology providers a list of matched trials for each patient to facilitate treatment decision making at point of care. Enrollment and timing metrics were tracked and compared with manual screening methods. Results: Watson CTM facilitated screening of all breast cancer patients for systemic therapy trials with matches validated by coordinators in 40% of patients. Over the 18 month (mo) period following implementation, 6.3 patients/mo were enrolled to breast cancer systemic therapy trials compared with 3.5 patients/mo in the period prior. The average monthly enrollment increased by 80%. This was further increased to 8.1 patients/mo when including accruals to breast cancer cohorts of phase I trials within the experimental therapeutics program. Time to match patients to trials with the CTM system was faster than manual methods but variable depending on the role of the screener and the depth of the matching. Conclusions: Implementation of the Watson for CTM system with a screening coordinator team was associated with an increase in breast cancer clinical trial enrollment. The system enabled high volume screening in an efficient manner and promoted awareness of clinical trial opportunities within the breast oncology practice.
Background:
Site identification, site selection, and study start-up have become the focus of improvement by organizations conducting clinical trials.
Methods:
To examine and measure the process from site identification through site activation, Tufts Center for the Study of Drug Development (CSDD) conducted a comprehensive survey among pharmaceutical organizations, biotech companies, and contract research organizations (CROs). Responses from over 400 unique companies were gathered and analyzed.
Results:
The results indicate that the start-up process is on average 5 to 6 months in total duration, and cycle times across all activities, including site identification, site selection, and study start-up, are faster for repeat sites than for new sites. Comparisons between sponsor and CROs indicate that CROs completed all site-related activities 6 to 11 weeks faster than sponsors. Other areas impacting cycle times were examined, including centralized versus decentralized functions, investment in technology, and organizational strategies that improve cycle time efficiency and performance.
Conclusion:
Tufts CSDD will explore this area in future research to gather additional insights into other factors that may be associated with speed and efficiency.
To investigate the barriers, modifiers, and benefits involved in participating in randomized controlled trials of cancer therapies as perceived by health care providers and patients.
We conducted a systematic review of the literature to identify published and unpublished studies in any language using electronic databases searched from 1996 to 2004, contact with experts, and reference lists. All study designs were acceptable provided relevant data were reported. Two reviewers were involved in the selection of studies, data extraction, and quality assessment processes. Studies were combined in a narrative synthesis.
Fifty-six studies met the inclusion criteria and represented the perspective of the patient or the health care provider or both. Although a range of barriers to trial participation were identified, a number of threats to the internal and external validity of the included studies limited interpretation of the evidence.
The limitations within the evidence base do not permit a clear interpretation of the barriers, moderators, and benefits involved in participation in cancer trials. We recommend that trialists prospectively identify the issues relevant to a particular trial using the current research as a starting point. We report checklists to guide this process.
Impact of a cognitive computing clinical trial matching system in an ambulatory oncology practice
- T Haddad
- J Helgeson
- K Pomerteau
Haddad T, Helgeson J, Pomerteau K, et al. Impact of a cognitive
computing clinical trial matching system in an ambulatory oncology practice. Abstract presented at American Society of Clinical
Oncology (ASCO) Annual Meeting, 2018.
Enrollment performance: weighing the "facts
- K Getz
Getz K. Enrollment performance: weighing the "facts." Appl Clin
Trials. 2012;21(5).