Thomas Walter’s research while affiliated with Mines Paris, PSL University and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (9)


Survival of NSCLC patients and Venn diagram summarizing the multimodal cohort
A OS and PFS Kaplan-Meier survival curve (solid lines) for the whole NSCLC cohort (n = 311 for OS and n = 316 for PFS) with a 95% confidence interval (shaded areas). Patients are stratified with respect to their first-line therapy, either pembrolizumab alone or pembrolizumab + chemotherapy. Log-rank p-values are reported to characterize the separation of the survival curves. B OS and PFS Kaplan-Meier survival curves (solid lines) with 95% confidence interval (shaded areas) and log-rank p-values for the patients with available PD-L1 expression (n = 295 for OS and n = 300 for PFS). Patients are stratified with respect to their PD-L1 status (positive vs negative). C OS Kaplan-Meier survival curves (solid lines) with 95% confidence interval (shaded areas) and log-rank p-values for the 43 patients with available TMB and the 174 patients with available TILs status. For the TMB, patients are stratified with a threshold of 15 mutations per megabase (see Methods). For TILs, patients are stratified with respect to their positive vs negative TILs status. D Overview of the multimodal cohort with a Venn diagram. The four data modalities and their intersections are represented (i.e., PET/CT images, clinical data, pathological slides, and bulk RNA-seq profiles). Source data are provided as a Source Data file.
Feature importance ranking for the prediction of overall survival, for clinical and transcriptomic modalities
Feature importance ranking was obtained by aggregating the SHAP values collected from both tasks (OS and 1-year death) and both approaches (linear and tree ensemble methods) (see Methods). Features that were significantly associated with 1-year death (one-sided permutation test with univariate AUCs) after Benjamini-Hochberg (BH) correction (α=0.05\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha=0.05$$\end{document}) are shown with a * on the left side, while features that were significantly associated with OS (one-sided permutation test with univariate C-index) after BH correction are annotated with a * on the right side. * corresponds to an adjusted p-value below 0.05. A Consensus feature importance ranking for the clinical data modality (left) and heatmap of correlations between consensus clinical features (right). Correlations were evaluated by Spearman correlation coefficients (for continuous feature vs continuous feature), AUCs rescaled to [−1,1]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$[-1,\,1]\,$$\end{document}(for continuous feature vs binary categorical feature), or Matthews correlation coefficient (for binary categorical feature vs binary categorical feature). B Consensus feature importance ranking for the RNA data modality (left) and heatmap of Spearman correlations between consensus RNA features (right). Source data are provided as a Source Data file.
Performance of all the possible multimodal combinations, with a late fusion strategy and tree ensemble methods
The bar height corresponds to the performance metric (either ROC AUC or C-index) averaged across the 100 cross-validation schemes, and the error bar corresponds to ± 1 standard deviation, estimated across the 100 cross-validation schemes. A ROC AUCs associated with the prediction of 1-year death with XGBoost algorithms (top) and estimated with n = 77 patients. C-indexes associated with the prediction of OS with Random Survival Forest algorithms (bottom) and estimated with n = 79 patients. B ROC AUCs associated with the prediction of 6-month progression with XGBoost algorithms (top) and estimated with n = 75 patients. C-indexes associated with the prediction of PFS with Random Survival Forest algorithms (bottom) and estimated with n = 80 patients. * C: clinical, R: radiomic, P: pathomic, RNA: transcriptomic. Source data are provided as a Source Data file.
Marginal contribution of each modality to the multimodal predictions for late fusion strategy and XGBoost classifiers
A Heatmap of the marginal contribution (i.e., Shapley value) of each modality to the 1-year death prediction using the C + R + RNA late fusion model with XGBoost classifiers. Marginal contributions indicate how each modality influences the prediction relative to a random baseline of 0.5. Patients are stratified based on the multimodal model’s final prediction (with a 0.5 threshold), where the positive class corresponds to those who died within 1 year, and the negative class corresponds to those who survived. B For each modality and patient in clusters 1 and 2 (see A), represented by vertical lines, this plot shows the feature with the highest SHAP value that aligns with the modality’s marginal contribution. The size of each triangle indicates the absolute SHAP value, while its orientation corresponds to its sign (up for positive values that increase the predicted probability of death within 1 year and down for negative values that decrease it). The color scale represents the associated feature value relative to the whole patient cohort. C Relationship between the unimodal predictions from clinical, radiomic, and RNA modalities (i.e., unimodal tree ensemble models). Each dot is colored according to the patient’s true label. *In these plots, all marginal contributions, SHAP values, and predictions were obtained for the 77 patients with complete multimodal profiles and available 1-year death labels across the cross-validation test sets. They were collected for each of the 100 cross-validation schemes (see Methods) and subsequently averaged for each patient. Source data are provided as a Source Data file.
Best unimodal and multimodal performances across all the possible combinations of modalities and predictive algorithms
The top barplot displays the performance of the best multimodal combination for each integration strategy, while the bottom barplot shows the performance of the best unimodal algorithm for each data modality. Bar heights and error bars correspond to the mean metric (AUC or C-index) and ± 1 standard deviation, respectively, estimated across the 100 cross-validation schemes (except for the dyam_optim models for which only 10 cross-validation schemes were used, due to computational constraints). A Best performance (AUC) for the prediction of 1-year death and 6-month progression (n = 77 for 1-year death and n = 75 for 6-month progression). B Best performance (C-index) for the prediction of OS and PFS (n = 79 for OS and n = 80 for PFS). Source data are provided as a Source Data file.

+3

Integration of clinical, pathological, radiological, and transcriptomic data improves prediction for first-line immunotherapy outcome in metastatic non-small cell lung cancer
  • Article
  • Full-text available

January 2025

·

57 Reads

·

1 Citation

Nicolas Captier

·

·

·

[...]

·

Immunotherapy is improving the survival of patients with metastatic non-small cell lung cancer (NSCLC), yet reliable biomarkers are needed to identify responders prospectively and optimize patient care. In this study, we explore the benefits of multimodal approaches to predict immunotherapy outcome using multiple machine learning algorithms and integration strategies. We analyze baseline multimodal data from a cohort of 317 metastatic NSCLC patients treated with first-line immunotherapy, including positron emission tomography images, digitized pathological slides, bulk transcriptomic profiles, and clinical information. Testing multiple integration strategies, most of them yield multimodal models surpassing both the best unimodal models and established univariate biomarkers, such as PD-L1 expression. Additionally, several multimodal combinations demonstrate improved patient risk stratification compared to models built with routine clinical features only. Our study thus provides evidence of the superiority of multimodal over unimodal approaches, advocating for the collection of large multimodal NSCLC datasets to develop and validate robust and powerful immunotherapy biomarkers.

Download

Figure 3: Benchmark of sCellST a. Overview of the benchmarking approach. Each slide is used for training and then the model is evaluated on the two remaining slides. b. H&E slides from the PDAC Visium dataset. c. Benchmark results: each boxplot represents the distribution of Pearson / Spearman correlation coefficient on all genes.
Figure 4: sCellST comparison with HoverNet labels: Each row corresponds to a Visium slide from cancer tissue: a, c, e: breast and b, d, f ovarian. a,b. Visium slides used for the experiments. c,d: Top differentially expressed genes when grouping cells by HoverNet labels. e,f: Distribution of known marker genes grouped with HoverNet labels.
sCellST: a Multiple Instance Learning approach to predict single-cell gene expression from H&E images using spatial transcriptomics

November 2024

·

90 Reads

Advancing our understanding of tissue organization and its disruptions in disease remains a key focus in biomedical research. Histological slides stained with Hematoxylin and Eosin (H&E) provide an abundant source of morphological information, while Spatial Transcriptomics (ST) enables detailed, spatially-resolved gene expression (GE) analysis, though at a high cost and with limited clinical accessibility. Predicting GE directly from H&E images using ST as a reference has thus become an attractive objective; however, current patch-based approaches lack single-cell resolution. Here, we present sCellST, a multiple-instance learning model that predicts GE by leveraging cell morphology alone, achieving remarkable predictive accuracy. When tested on a pancreatic ductal adenocarcinoma dataset, sCellST outperformed traditional methods, underscoring the value of basing predictions on single-cell images rather than tissue patches. Additionally, we demonstrate that sCellST can detect subtle morphological differences among cell types by utilizing marker genes in ovarian cancer samples. Our findings suggest that this approach could enable single-cell level GE predictions across large cohorts of H&E-stained slides, providing an innovative means to valorize this abundant resource in biomedical research.


figure 1. MGC density is a biomarker of prognosis in patients with HNSCC. A, Therapeutic sequence of the treatment-naive cohorts of patients with HNSCC from TCGA and GR. B, Flow chart of TCGA and GR cohorts of treatment-naive patients with HNSCC. Among the 946 patients (527 from TCGA and 419 from GR), 394 were included (110 from TCGA and 284 from GR). C, Non-keratinizing HNSCC (left) and keratinizing HNSCC (right), stained by H&E. WSI from TCGA (scale bar, 250 μm). D, Histograms showing the number of K Low and K High patients from TCGA and GR. E, Keratinizing SCC of the oral cavity of a patient from GR, with MGC-rich granulomas surrounding keratin debris. Left, low magnification of the carcinoma (scale bar, 2 mm). Middle, high magnification of a granuloma containing MGC and keratin (scale bar, 50 μm). Right, MGC are highlighted in yellow and keratin in red. f, Histograms showing the proportion of MGC high patients (TCGA and GR cohorts) among the keratinization groups (no, low, moderate, and high). G, OS curve of 110 TCGA patients stratified according to MGC density in their tumors. The vertical tick mark on the curves means that a patient was censored at this time. H, OS curve of 284 GR patients stratified according to MGC density in their tumors. i, OS curve of all patients (TCGA and GR) stratified according to MGC density in their tumors. J, PFI curve of all patients (TCGA and GR) stratified according to MGC density in their tumors. K, Therapeutic sequence of the induction chemotherapy (ICT)-treated cohort of patients with HNSCC from GR. L, Flow chart of the ICT-treated cohort of patients with HNSCC from GR. M, Keratinizing oral SCC from a patient treated at GR whose tumor responded well to ICT, showing areas of keratin surrounded by numerous MGC. Left, low magnification of the carcinoma (scale bar, 2 mm). Middle, high magnification of a granuloma containing MGC and keratin (scale bar, 50 μm). Right, MGC are highlighted in yellow and keratin in red. N, Histograms showing the percentage of patients with no residual tumor, and the percentage of patients with residual tumor being MGC High or MGC Low/Int , in treatment-naive (TCGA and GR) and ICT patients (GR). O, OS curve of 52 patients treated by ICT at GR, stratified by their tumor content and MGC density on surgical resection. The vertical tick mark on the curves means that a patient was censored at this time. P, Histograms showing the proportion of MGC High and MGC Low/Int patients according to the pathological response status of their tumors (poor vs. good/partial).
figure 3. Spatial transcriptomics reveals a unique MGC signature. A, Low magnification of a representative HES section of a carcinoma selected for spatial transcriptomic analysis. Inset, high magnification of an MGC. B, Same HES section with the overlay of the spots analyzed by Visium technology. One spot is covering a single MGC. C, Low magnification of a representative HES of an MGC High carcinoma from a patient in the GR cohort. D, Overlay of the seven cell populations analyzed by unsupervised clustering. E, Same HES section showing pathologist annotations of the tumor area (blue) and the MGC (yellow). f, Low magnification of a representative HES of an MGC Low carcinoma from a patient in the GR cohort. G, Overlay of the seven cell populations analyzed by unsupervised clustering. H, Same HES section showing pathologist annotations of the tumor area (blue). i, Projection of the Visium spots onto a UMAP space; spots from nine different patients. J, Histograms showing the number of spots capturing the different cell types in MGC High and MGC Low carcinomas. K, Volcano plot showing MGC RNA signature extracted from the DEG analysis between supervised MGC spots and all the other non-MGC spots. L, Volcano plot showing the DEG between MGC High and MGC Low tumors from patients in TCGA cohort (n = 108 patients). M, GSEA plot showing the enrichment of the Visium MGC signature in MGC High vs. MGC Low patients classified from TCGA. The green line represents the running enrichment score for the MGC signature, with the peak indicating maximum enrichment. The normalized enrichment score (NES) is 3.24, and the P value is 0.000522, demonstrating significant enrichment of the MGC signature in MGC High patients. The barcode plot shows the positions of the MGC signature genes within the ranked list of genes from the bulk RNA sequencing data, with a higher density of genes towards the left, indicating higher enrichment.
figure 4. MGC-related tumor microenvironment is enriched in CD4 TCM T cells and TREM2-expressing mononuclear macrophages. A, Low magnification of a representative HES section MGC High carcinoma selected for CosMx 64-plex protein panel analysis. Scale bar, 500 μm. Inset, high magnification of an area annotated by a pathologist, enriched in TILs (green) close to tumor cells (blue) and a granuloma with MGC (yellow). Scale bar, 50 μm. B, Low magnification of the same tumor section, stained by the CosMx protein panel, with the overlay of 20 cell populations (non-immune and immune). Scale bar, 500 μm. Inset, high magnification of the same area, with the annotation overlay of tumor cells, MGC (yellow), TCM CD4 T cells (red), B cells (light gray) and plasma cells (dark gray). C, Projection of the CosMx cell population onto a UMAP space. D, Heat maps showing the expression of immune markers across the 14 immune cell types. Red indicates a higher expression and blue a bottom expression. E, Box-plot showing the TCM CD4 T cells ratio between MGC Low (n = 4) and MGC High patients (n = 4). f, Box-plot showing TCM CD4 T cells degree centrality (left) and average clustering (right) between MGC Low (n = 4) and MGC High patients (n = 4). G, Heat maps of neighborhood enrichment scores showing the spatial colocalization of the 22 cell types across four MGC High patients. Yellow indicates a high enrichment and purple a low enrichment. White boxes are highlighting the proximity between (i) TCM CD4 T cells and B cells and (ii) MGC and MGC-MNP. H, Box plots showing B cells (left) and plasma cells ratio (right) between MGC Low (n = 4) and MGC High patients (n = 4). i, Box plots showing B cells (left) and plasma cells average clustering between MGC Low (n = 4) and MGC High patients (n = 4). J, Low magnification of a representative section stained by the CosMx protein panel, of two granulomas, with the annotation overlay of tumor cells (blue), stromal cells (green), MGC and MGC-MNP (yellow). Insets: high magnification showing MGC surrounded by numerous MGC-MNP. Scale bar, 800 μm. K, Box-plot showing the MGC-MNP ratio between MGC Low (n = 4) and MGC High patients (n = 4). L, Representative image of a single MGC stained by CD68 (pink), TREM2 (orange), and Hoechst (cyan). The merge image is shown (scale bar, 50 μm). M, Representative high magnification image of a small granuloma. The arrow indicates mononuclear macrophages (scale bar, 20 μm). N, Histograms showing the distance of TREM2 High MGC from keratin (left) and histograms showing the distance of TREM2 High mononuclear macrophages from keratin (right).
TREM2-Expressing Multinucleated Giant Macrophages Are a Biomarker of Good Prognosis in Head and Neck Squamous Cell Carcinoma

September 2024

·

85 Reads

·

3 Citations

Cancer Discovery

Patients with head and neck squamous cell carcinomas (HNSCC) often have poor outcomes due to suboptimal risk management and treatment strategies; yet integrating novel prognostic biomarkers into clinical practice is challenging. Here, we report the presence of multinucleated giant cells (MGC)—a type of macrophages—in tumors from patients with HNSCC, which are associated with a favorable prognosis in treatment-naive and preoperative chemotherapy–treated patients. Importantly, MGC density increased in tumors following preoperative therapy, suggesting a role of these cells in the antitumoral response. To enable clinical translation of MGC density as a prognostic marker, we developed a deep-learning model to automate its quantification on routinely stained pathological whole slide images. Finally, we used spatial transcriptomic and proteomic approaches to describe the MGC-related tumor microenvironment and observed an increase in central memory CD4 T cells. We defined an MGC-specific signature resembling to TREM2-expressing mononuclear tumor-associated macrophages, which colocalized in keratin tumor niches. Significance Novel individual biomarkers are needed to guide therapeutic decisions for patients with head and neck cancer. We report for the first time, granulomas of TREM2-expressing multinucleated giant macrophages in keratin-rich tumor niches, as a biomarker of favorable prognosis and developed a deep-learning model to automate its quantification on routinely stained pathological slides.


Integration of clinical, pathological, radiological, and transcriptomic data improves the prediction of first-line immunotherapy outcome in metastatic non-small cell lung cancer

June 2024

·

52 Reads

The survival of patients with metastatic non-small cell lung cancer (NSCLC) has been increasing with immunotherapy, yet efficient biomarkers are still needed to optimize patient care. In this study, we explored the benefits of multimodal approaches to predict immunotherapy outcome using multiple machine learning algorithms and integration strategies. We leveraged a novel multimodal cohort of 317 metastatic NSCLC patients treated with first-line immunotherapy, collecting at baseline positron emission tomography images, digitized pathological slides, bulk transcriptomic profiles, and clinical information. Most integration strategies investigated yielded multimodal models surpassing both the best unimodal models and established univariate biomarkers, such as PD-L1 expression. Additionally, several multimodal combinations demonstrated improved patient risk stratification compared to models built with routine clinical features only. Our study thus provided new evidence of the superiority of multimodal over unimodal approaches, advocating for the collection of large multimodal NSCLC cohorts to develop and validate robust and powerful immunotherapy biomarkers.


Image-based multiplex immune profiling of cancer tissues: translational implications. A report of the International Immuno-oncology Biomarker Working Group on Breast Cancer

January 2024

·

276 Reads

·

5 Citations

The Journal of Pathology

Recent advances in the field of immuno‐oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples. By establishing the phenotype of individual tumour cells when distributed within a mixed cell population, the identification of clinically relevant biomarkers with high‐throughput multiplex immunophenotyping of tumour samples has great potential to guide appropriate treatment choices. Moreover, the emergence of novel multi‐marker imaging approaches can now provide unprecedented insights into the tumour microenvironment, including the potential interplay between various cell types. However, there are significant challenges to widespread integration of these technologies in daily research and clinical practice. This review addresses the challenges and potential solutions within a structured framework of action from a regulatory and clinical trial perspective. New developments within the field of immunophenotyping using multiplexed tissue imaging platforms and associated digital pathology are also described, with a specific focus on translational implications across different subtypes of cancer. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.


Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: a report of the international immuno-oncology biomarker working group

August 2023

·

256 Reads

·

22 Citations

The Journal of Pathology

The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.


Figure 1. Raster versus vector spatial data structure. (A) An example of a TNBC specimen imaged by H&E, with high-resolution multi-color images obtained using mIF (Vectra platform). High-resolution images are used to obtain cell coordinates and phenotypes. (B) In raster-based spatial analysis, the tumor is divided into small subregions (usually by a rectangular grid) and spatial metrics are calculated independently across each subregion, allowing for analysis of spatial metrics such as average cell count, deviation/skewness, and hotspot analysis. (C and D) In vector-based spatial analysis, cells are annotated by their phenotype, (x, y) geographic location, and other attributes, such as PD-L1 expression. These data can then be analyzed using statistical software to calculate a variety of metrics.
Figure 2. Illustration of various hotspot metrics. Various methods of calculating IC hotspots have been described in the literature, and include methods based upon rank-ordering of IC density across subregions, or based upon inferential testing. (A) Rank-order-based approaches, which define hotspots as either 'top 3' (the three most densely infiltrated subregions) or 'top 30%' (the top 30% most densely infiltrated subregions). (B) The Getis-Ord Gi* method, which uses inferential statistical testing to estimate p values indicating the likelihood of each subregion being a hotspot or a coldspot. The Getis-Ord test statistic follows a normal distribution and can be thought of as a measure of local IC density in neighboring subregions, relative to overall IC density. IC, immune cell; TC, tumor cell.
Figure 4. sTILs score and Immunoscore: methodology and opportunities for spatial applications. (A) Segmentation step: for the sTILs score, the intraepithelial versus stromal tumor compartment are visually determined by a pathologist, whereas for Immunoscore, the IM versus tumor center is determined using an automated ML platform. (B) Sampling and density estimation steps: for the sTIL score, several representative subregions are visually selected and sTIL counts are estimated across each subregion, whereas for Immunoscore, the entirety of the tumor area is divided by a rectangular raster grid, and CD3 + and CD8 + ICs are counted for each raster cell. (C) Calculation step: for the sTIL score, the arithmetic mean of sTIL densities for each subregion is calculated, whereas for Immunoscore, the arithmetic mean of cohort-level percentile scores across the four cellular compartments is calculated (CD3 + IM, CD3 + TC, CD8 + IM, CD8 + TC). (D) Advanced ML, histologic imaging, and spatial analytic approaches can be applied to the sTIL score and Immunoscore to potentially improve predictive/prognostic utility.
Examples of spatial metrics and definitions.
Spatial analyses of immune cell infiltration in cancer: current methods and future directions. A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

August 2023

·

512 Reads

·

16 Citations

The Journal of Pathology

Modern histologic imaging platforms coupled with machine learning methods have provided new opportunities to map the spatial distribution of immune cells in the tumor microenvironment. However, there exists no standardized method for describing or analyzing spatial immune cell data, and most reported spatial analyses are rudimentary. In this review, we provide an overview of two approaches for reporting and analyzing spatial data (raster versus vector-based). We then provide a compendium of spatial immune cell metrics that have been reported in the literature, summarizing prognostic associations in the context of a variety of cancers. We conclude by discussing two well-described clinical biomarkers, the breast cancer stromal tumor infiltrating lymphocytes score and the colon cancer Immunoscore, and describe investigative opportunities to improve clinical utility of these spatial biomarkers. © 2023 The Pathological Society of Great Britain and Ireland.


Models including pathological and radiomic features vs clinical models in predicting outcome of patients with metastatic non-small cell lung cancer treated with immunotherapy.

June 2023

·

37 Reads

Journal of Clinical Oncology

e21164 Background: Overall survival of patients with metastatic non-small cell lung cancer (NSCLC) has increased with the use of anti-PD-1 immune checkpoint inhibitors. However, the duration of response remains highly variable between patients, and only 20-30% of patients are alive at 2 years. Thus, new biomarkers for predicting response to treatment and patient outcomes are still needed to guide therapeutic decision. In this study, we retrospectively investigated multimodal approaches that might improve the limited predictive power of clinical data. Methods: We studied a cohort of 317 patients with metastatic NSCLC treated with first-line immune checkpoint inhibitors alone or combined with platinum-based chemotherapy. Clinical data were collected for all patients, pathological slides (HES and PD-L1 staining) and baseline 18-FDG PET/CT scans were available in 237 and 130 patients respectively. An automatic cell type detection algorithm was applied to each pathological slide and pathomic features were extracted from the resulting annotations. After semi-automated segmentation of all tumor foci in the PET/CT scans, radiomic features were calculated for each tumor lesion and aggregated across all the lesions of each patient. Prognostic models were built using random forest and XGboost classifiers to predict patient survival at 12 months based on 1) features from single modalities (clinical, pathomic, or radiomic), 2) features from multiple modalities, where early fusion and late fusion strategies were investigated. The models were trained and tested with cross-validation and their performances were established using the area under the ROC curve (AUC) computed on the same 88 test patients for whom all the modalities were available. Results: Unimodal strategies yielded AUC of 0.62 ± 0.08 (1 std), 0.64 ± 0.07, 0.59 ± 0.08 for clinical, radiomic and pathomic features respectively. With late fusion, bimodal models consistently outperformed the clinical model, with the combination of radiomic and clinical features giving the best performance (AUC = 0.67 ± 0.07). The trimodal model outperformed all other modality combinations with an AUC of 0.69 ± 0.07; in particular, it was significantly superior to the clinical model (p-value < 0.001, paired t-test). The early fusion experiments confirmed the superiority of every bimodal approach over the clinical model. However, the trimodal model did not outperform the best bimodal model with early fusion. Validation will be performed on independent cohorts from external centers. Conclusions: Our study highlighted the potential of multimodal approaches for predicting the outcome of metastatic NSCLC patients treated with immunotherapy. Models integrating medical images and pathological slides usually collected from routine care outperformed a model trained on clinical data alone.


Assessment of algorithms for mitosis detection in breast cancer histopathology images

November 2014

·

540 Reads

·

462 Citations

Medical Image Analysis

The proliferative activity of breast tumors, which is routinely estimated by counting of mitotic figures in hematoxylin and eosin stained histology sections, is considered to be one of the most important prognostic markers. However, mitosis counting is laborious, subjective and may suffer from low inter-observer agreement. With the wider acceptance of whole slide images in pathology labs, automatic image analysis has been proposed as a potential solution for these issues. In this paper, the results from the Assessment of Mitosis Detection Algorithms 2013 (AMIDA13) challenge are described. The challenge was based on a data set consisting of 12 training and 11 testing subjects, with more than one thousand annotated mitotic figures by multiple observers. Short descriptions and results from the evaluation of eleven methods are presented. The top performing method has an error rate that is comparable to the inter-observer agreement among pathologists.

Citations (4)


... This is a time consuming process requiring pooling or separation of groups of spectra, or intense dimensionality reduction. The central hypothesis of this study is that the use of deep learning approaches on chemical image data, where spectral information is not separated from its spatial context, will lead to improved performance in predicting disease recurrence, where spatial context within the tumor microenvironment is known to be a contributory factor [49][50][51] . ...

Reference:

Prediction of post-treatment recurrence in early-stage breast cancer using deep-learning with mid-infrared chemical histopathological imaging
Image-based multiplex immune profiling of cancer tissues: translational implications. A report of the International Immuno-oncology Biomarker Working Group on Breast Cancer

The Journal of Pathology

... This is a time consuming process requiring pooling or separation of groups of spectra, or intense dimensionality reduction. The central hypothesis of this study is that the use of deep learning approaches on chemical image data, where spectral information is not separated from its spatial context, will lead to improved performance in predicting disease recurrence, where spatial context within the tumor microenvironment is known to be a contributory factor [49][50][51] . ...

Spatial analyses of immune cell infiltration in cancer: current methods and future directions. A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

The Journal of Pathology

... Tumor, stroma, and immune cells were identified in each case of H&E slides for the object and pixel classifiers. Guidelines established by the International Immuno-Oncology Biomarker Working Group on Breast Cancer were followed [11,[26][27][28]. This included enumerating TILs across the selected area of the slide, which excluded areas of artifacts, normal breast tissue, tertiary lymphoid structures, and others defined further in the guidelines. ...

Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: a report of the international immuno-oncology biomarker working group

The Journal of Pathology

... Potential use: The automated evaluation of immunohistochemistry (IHC) staining of receptors along with their percentage of staining, intensity and other parameters for scoring in breast carcinoma has attracted a lot of interest. Attempts have also been made to simultaneously predict biomarkers involved in carcinoma breast directly from H&E slides recently (18). These algorithms also need to consider the cost-savings aspect and the extent to which the objectivity and accuracy of AIassisted assessment adds value for clinical application. ...

Assessment of algorithms for mitosis detection in breast cancer histopathology images
  • Citing Article
  • November 2014

Medical Image Analysis