Project

Deep Learning-Based Patient-Specific Multimodality Treatment Outcome Prediction for Gynecological Cancers using pre/post diagnostic image modalities and digital histopathology images

Goal: To develop a deep learning-based patient-specific multimodal treatment outcome prediction model and investigate how pretreatment patient characteristics of various data types influence treatment efficacy as measured by post-treatment response. Specific Aim A: Investigate the correlation between microscopic influence of cell spacing and nuclei size, extracted from Hematoxylin and Eosin(H&E) stained digital histopathological images, and gynecological cancer treatment outcomes (i.e., recurrence, toxicity, survival) in radiotherapy (external radiotherapy and brachytherapy).Specific Aim B: Developing a DL-based patient-specific treatment outcome prediction model that combines auto-segmented tumor radiomic signatures from diagnostic images (e.g., CT, MR, US) and digital histopathological images such as nuclei size & cell spacing distribution data, which will accurately predict outcomes for various treatment modality combinations for gynecological cancer patients.

Date: 1 September 2021

Updates
0 new
0
Recommendations
0 new
0
Followers
0 new
1
Reads
0 new
19

Project log

Yujing Zou
added 3 research items
Purpose/Objective: The McMedHacks workshop and presentation series was created to teach individuals from various backgrounds about deep learning (DL) for medical image analysis in May, 2021. Material/Methods: McMedHacks is a free and student-led 8-week summer program. Registration for the event was open to everyone, including a form to survey participants’ area of expertise, country of origin, level of study, and level of programming skills. The weekly workshops were instructed by 8 students and experts assisted by 20 mentors who provided weekly tutorials. Recent developments in DL and medical physics were highlighted by 21 leaders from industry and academia. A virtual grand challenge Hackathon took place at the end of the workshop series. All events were held virtually and recorded on Zoom to accommodate all time zones and locations. The workshops were designed as interactive coding demos and shared through Google Colab notebooks. Results: McMedHacks gained 356 registrations from participants of 38 different countries (Fig. 1) from undergraduates, to PhDs and MDs. A vast number of disciplines and professions were represented, dominated by medical physics students, academic, and clinical medical physicists (Fig. 2). Sixty-nine participants earned a certificate of completion by having engaged with at least 12 of all 14 events. The program received participant feedback average scores of 4.768, 4.478, 4.579, 4.292, 4.84 out of five for the qualities of presentation, workshop session, tutorial and mentor, assignments, and course delivery, respectively. The eight-week long workshop’s duration allowed participants to digest the taught materials in a continuous manner as opposed to bootcamp-style conference workshops. Conclusion: The overwhelming interest and engagement for the McMedHacks workshop series from the Radiation Oncology (RadOnc) community illustrates a demand for Artificial Intelligence (AI) education in RadOnc. The future of RadOnc clinics will inevitably integrate AI. Therefore, current RadOnc professionals, and student and resident trainees should be prepared to understand basic AI principles and its applications to troubleshoot, innovate, and collaborate. McMedHacks set an excellent example of promoting open and multidisciplinary education, scientific communication, and leadership for integrating AI education into the RadOnc community on an international level. Therefore, we advocate for implementation of AI curriculums in professional education programs such as Commission on Accreditation of Medical Physics Education Programs (CAMPEP). Furthermore, we encourage experts from around the world in the field of AI, or RadOnc, or both, to take initiatives like McMedHacks to collaborate and push forward AI education in their departments and lead practical workshops, regardless of their levels of education.
Purpose: To build a machine-learning (ML) classifier to predict the clinical endpoint of post-Radiation-Therapy (RT) recurrence of gynecological cancer patients, while exploring the outcome predictability of cell spacing and nuclei size pre-treatment histopathology image features and clinical variables. Materials and Methods: Thirty-six gynecological (i.e., cervix, vaginal, and vulva) cancer patients (median age at diagnosis = 59.5 years) with a median follow-up time of 25.7 months, nine of which (event rate of 25%) experienced post-RT recurrence, were included in this analysis. Patient-specific nuclei size and cell spacing distributions from cancerous and non-tumoral regions of pre-treatment hematoxylin and eosin (H&E) stained digital histopathology Whole-Slide-Images (WSI) were extracted. The mean and standard deviation of these distributions were computed as imaging features for each WSI. Clinical features of clinical and radiological stage at the time of radiation, p16 status, age at diagnosis, and cancer type were also obtained. Uniquely, a Tree-based Pipeline Optimization Tool (TPOT) AutoML approach, including hyperparameter tuning, was implemented to find the best performing pipeline for this class-imbalanced and small dataset. A Radial Basis Function Kernel (RBF) sampler (gamma = 0.25) was applied to combined imaging and clinical input variables for training. The resulting features were fed into an XGBoost (ie., eXtreme gradient-boosting) classifier (learning rate = 0.1). Its outputs were propagated as “synthetic features” followed by polynomial feature transforms. All raw and transformed features were trained with a decision tree classification algorithm. Results of model evaluation metrics from a 10-fold stratified shuffle split cross-validation were averaged. A permutation test (n=1000) was performed to validate the significance of the classification scores. Results: Our model achieved a 10-fold stratified shuffle split cross-validation scores of 0.87 for mean accuracy, 0.92 for mean balanced accuracy, 0.78 for precision, 1 for recall, 0.85 for F1 score, and 0.92 for Area Under the Curve of Receiver Operating Characteristics Curve, to predict our patient cohort’s post-RT recurrence binary outcome. A p-value of 0.036 was obtained from the permutation test. This implies real dependencies between our combined imaging and clinical features and outcomes which were learned by the classifier, and the primising model performance was not by chance. Conclusions: Despite the small dataset and low event rate, as a proof of concept, we showed that a decision-tree-based ML classification algorithm using an XGBoost algorithm is able to utilize combined (cell spacing & nuclei size) imaging and clinical features to predict post-RT outcomes for gynecological cancer patients.
Yujing Zou
added a project goal
To develop a deep learning-based patient-specific multimodal treatment outcome prediction model and investigate how pretreatment patient characteristics of various data types influence treatment efficacy as measured by post-treatment response. Specific Aim A: Investigate the correlation between microscopic influence of cell spacing and nuclei size, extracted from Hematoxylin and Eosin(H&E) stained digital histopathological images, and gynecological cancer treatment outcomes (i.e., recurrence, toxicity, survival) in radiotherapy (external radiotherapy and brachytherapy).Specific Aim B: Developing a DL-based patient-specific treatment outcome prediction model that combines auto-segmented tumor radiomic signatures from diagnostic images (e.g., CT, MR, US) and digital histopathological images such as nuclei size & cell spacing distribution data, which will accurately predict outcomes for various treatment modality combinations for gynecological cancer patients.