Dongjoo Lee’s research while affiliated with Seoul National University and other places

Background: Digital high-resolution H&E images provide valuable insights into tumor heterogeneity within the tumor microenvironment (TME). However, the ability to perform detailed cell typing solely based on H&E images remains limited. Recent advances in high-resolution spatial transcriptomics (ST) enable precise characterization of cell types and their spatial relationships within the TME, addressing challenges in understanding tumor heterogeneity. In this study, we developed AI models to predict cell types in breast cancer—including subtypes of lymphocytes that are challenging to differentiate visually—using large-scale image-based ST data aligned with H&E images. Methods: We established a breast cancer image-based ST database comprising 190 samples from 113 breast cancer patients, obtained from surgically resected primary tumors. Image-based ST data were generated using the Xenium platform with a 500-gene panel and matched with high-resolution H&E images. Cell type maps were constructed using reference single-cell RNA-seq data and transferred to ST data. The ST data were spatially registered to the corresponding H&E images, and cell type masks were generated at matching resolutions. AI models were trained to segment cell type masks, including epithelial cells, cancer cells, myeloid cells, fibroblasts, endothelial cells, T cells, and B cells. Additionally, refined cell type predictions were performed for dendritic cells, NK cells, CD4+ T cells, and CD8+ T cells. Model performance was validated using external whole-slide image datasets containing ST data from four independent cohorts. Results: Model performance was assessed using the area under the receiver operating characteristic (AUROC) curve for each cell type mask. Internal validation on four samples yielded AUROC values ranging from 0.90 to 0.95. External validation across independent whole-slide datasets demonstrated AUROC values between 0.88 and 0.97 for all cell type masks. The AI model successfully mapped TME cell types with high resolution, using only H&E images. Conclusion: By employing a self-supervised approach that integrates high-resolution H&E images with image-based ST data, we developed AI-driven tools for TME analysis. These models enable accurate identification of detailed cell types and their spatial relationships within the TME. This approach facilitates large-scale analysis of breast cancer TME and holds potential for advancing our understanding of tumor biology and therapeutic strategies. Citation Format: Haenara Shin, Dongjoo Lee, Yooeun Kim, Daeseung Lee, Kwon Joong Na, Chihwan David Cha, Hosub Park, Hongyoon Choi. A self-supervised AI model leveraging spatial omics for analyzing tumor microenvironment heterogeneity in breast cancer only with H&E [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Functional and Genomic Precision Medicine in Cancer: Different Perspectives, Common Goals; 2025 Mar 11-13; Boston, MA. Philadelphia (PA): AACR; Cancer Res 2025;85(5 Suppl):Abstract nr B038.

Purpose The potential of Large Language Models (LLMs) in enhancing a variety of natural language tasks in clinical fields includes medical imaging reporting. This pilot study examines the efficacy of a retrieval-augmented generation (RAG) LLM system considering zero-shot learning capability of LLMs, integrated with a comprehensive database of PET reading reports, in improving reference to prior reports and decision making. Methods We developed a custom LLM framework with retrieval capabilities, leveraging a database of over 10 years of PET imaging reports from a single center. The system uses vector space embedding to facilitate similarity-based retrieval. Queries prompt the system to generate context-based answers and identify similar cases or differential diagnoses. From routine clinical PET readings, experienced nuclear medicine physicians evaluated the performance of system in terms of the relevance of queried similar cases and the appropriateness score of suggested potential diagnoses. Results The system efficiently organized embedded vectors from PET reports, showing that imaging reports were accurately clustered within the embedded vector space according to the diagnosis or PET study type. Based on this system, a proof-of-concept chatbot was developed and showed the framework’s potential in referencing reports of previous similar cases and identifying exemplary cases for various purposes. From routine clinical PET readings, 84.1% of the cases retrieved relevant similar cases, as agreed upon by all three readers. Using the RAG system, the appropriateness score of the suggested potential diagnoses was significantly better than that of the LLM without RAG. Additionally, it demonstrated the capability to offer differential diagnoses, leveraging the vast database to enhance the completeness and precision of generated reports. Conclusion The integration of RAG LLM with a large database of PET imaging reports suggests the potential to support clinical practice of nuclear medicine imaging reading by various tasks of AI including finding similar cases and deriving potential diagnoses from them. This study underscores the potential of advanced AI tools in transforming medical imaging reporting practices.

Spatially resolved transcriptomics (ST) has revolutionized the field of biology by providing a powerful tool for analyzing gene expression in situ. However, current ST methods, particularly barcode-based methods, have limitations in reconstructing high-resolution images from barcodes sparsely distributed in slides. Here, we present SuperST, an algorithm that enables the reconstruction of dense matrices (higher-resolution and non-zero-inflated matrices) from low-resolution ST libraries. SuperST is based on deep image prior, which reconstructs spatial gene expression patterns as image matrices. Compared with previous methods, SuperST generated output images that more closely resembled immunofluorescence images for given gene expression maps. Furthermore, we demonstrated how one can combine images created by SuperST with computer vision algorithms. In this context, we proposed a method for extracting features from the images, which can aid in spatial clustering of genes. By providing a dense matrix for each gene in situ, SuperST can successfully address the resolution and zero-inflation issue.

Spatial transcriptomics is a cutting-edge technique that combines gene expression with spatial information, allowing researchers to study molecular patterns within tissue architecture. Here, we present IAMSAM, a user-friendly web-based tool for analyzing spatial transcriptomics data focusing on morphological features. IAMSAM accurately segments tissue images using the Segment Anything Model, allowing for the semi-automatic selection of regions of interest based on morphological signatures. Furthermore, IAMSAM provides downstream analysis, such as identifying differentially expressed genes, enrichment analysis, and cell type prediction within the selected regions. With its simple interface, IAMSAM empowers researchers to explore and interpret heterogeneous tissues in a streamlined manner.

The expression of PD-L1 on tumor cells (TCs) is used as an immunotherapy biomarker in lung cancer, but heterogeneous intratumoral expression is often observed. Using a Digital Spatial Profiling, we performed proteomic and whole-transcriptomic analyses of TCs and immune cells (ICs) in spatially matched areas based on tumor PD-L1 expression and the status of the immune microenvironment. Our findings were validated using immunohistochemistry, The Cancer Genome Atlas, and immunotherapy cohorts. ICs in areas with high PD-L1 expression on TCs showed more features indicative of immunosuppression and exhaustion than ICs in areas with low PD-L1 expression on TCs. TCs highly expressing PD-L1 within immune-inflamed (IF) areas show up-regulation of pro-inflammatory processes, whereas TCs highly expressing PD-L1 within immune-deficient (ID) areas show up-regulation of various metabolic processes. Using differentially expressed genes of TCs between the IF and ID areas, we identified a novel prognostic gene signature for lung cancer. In addition, a high ratio of CD8+ cells to M2 macrophages was found to predict favorable outcomes in patients with PD-L1-expressing lung cancer after immune checkpoint inhibitor therapy. This study demonstrates that TCs and ICs have distinct spatial features within the tumor microenvironment that are related to tumor PD-L1 expression and IC infiltration.

Background Triple-negative breast cancer (TNBC) poses unique challenges due to its complex nature and the need for more effective treatments. Recent studies showed encouraging outcomes from combining paclitaxel (PTX) with programmed cell death protein-1 (PD-1) blockade in treating TNBC, although the exact mechanisms behind the improved results are unclear. Methods We employed an integrated approach, analyzing spatial transcriptomics and single-cell RNA sequencing data from TNBC patients to understand why the combination of PTX and PD-1 blockade showed better response in TNBC patients. We focused on toll-like receptor 4 (TLR4), a receptor of PTX, and its role in modulating the cross-presentation signaling pathways in tumor-associated macrophages (TAMs) within the tumor microenvironment. Leveraging insights obtained from patient-derived data, we conducted in vitro experiments using immunosuppressive bone marrow-derived macrophages (iBMDMs) to validate if PTX could augment the cross-presentation and phagocytosis activities. Subsequently, we extended our study to an in vivo murine model of TNBC to ascertain the effects of PTX on the cross-presentation capabilities of TAMs and its downstream impact on CD8+ T cell-mediated immune responses. Results Data analysis from TNBC patients revealed that the activation of TLR4 and cross-presentation signaling pathways are crucial for the antitumor efficacy of PTX. In vitro studies showed that PTX treatment enhances the cross-presentation ability of iBMDMs. In vivo experiments demonstrated that PTX activates TLR4-dependent cross-presentation in TAMs, improving CD8+ T cell-mediated antitumor responses. The efficacy of PTX in promoting antitumor immunity was elicited when combined with PD-1 blockade, suggesting a complementary interaction. Conclusions This study reveals how PTX boosts the effectiveness of PD-1 inhibitors in treating TNBC. We found that PTX activates TLR4 signaling in TAMs. This activation enhances their ability to present antigens, thereby boosting CD8+ T cell antitumor responses. These findings not only shed light on PTX’s immunomodulatory role in TNBC but also underscore the potential of targeting TAMs’ antigen presentation capabilities in immunotherapy approaches.

While cancer-associated fibroblasts (CAFs) are crucial in influencing tumor growth and immune responses in lung cancer, we still lack a comprehensive understanding of their spatial organization associated with tumor progression and clinical outcomes. This gap highlights the need to elucidate how the intricate spatial arrangement of CAFs affects their interactions within the tumor microenvironment, ultimately shaping cancer progression and patient prognosis. Here, we unveil the spatial diversity of CAFs in lung squamous cell carcinoma (LUSC), a prevalent and aggressive lung cancer type, elucidating their impact on tumor progression and patient outcomes using spatial transcriptomics (ST). Image-based ST data from 33 LUSC patients demonstrated a significant association of spatial interactions of tumor epithelium and CAFs with tumor size and metabolic activity measured by [18F]fluorodeoxyglucose PET. Furthermore, the proximity of fibroblasts to tumor epithelial cells was linked to recurrence-free survival in LUSC patients. By characterizing CAFs based on their spatial relationship, we identified distinct molecular signatures related to spatially distinct fibroblast subpopulations. In addition, barcode-based ST data from 8 LUSC patients revealed spatially overlapping fibroblast regions characterized by upregulated glycolysis pathways. Our study underscores the importance of the complex spatial dynamics of the tumor microenvironment revealed by ST and its implications for patient outcomes in LUSC.

Introduction: The potential of Large Language Models (LLMs) in enhancing a variety of natural language tasks in clinical fields includes medical imaging reporting. This pilot study examines the efficacy of a retrieval-augmented LLM system considering zero-shot learning capability of LLMs, integrated with a comprehensive PET reading reports database, in improving referring previous reports and decision-making. Methods: We developed a custom LLM framework enhanced with retrieval capabilities, leveraging a database encompassing nine years of PET imaging reports from a single center. The system employs vector space embedding of the reports database to facilitate retrieval based on similarity metrics. Queries prompt the system to retrieve embedded vectors, generating context-based answers and identifying similar cases or differential diagnoses from the historical reports database. Results: The system efficiently organized embedded vectors from PET reading reports, showing that imaging reports were accurately clustered within the embedded vector space according to the diagnosis or PET study type. Based on this system, a proof-of-concept chatbot was developed and showed the framework's potential in referencing reports of previous similar cases and identifying exemplary cases for various purposes. Additionally, it demonstrated the capability to offer differential diagnoses, leveraging the vast database to enhance the completeness and precision of generated reports. Conclusions: The integration of a retrieval-augmented LLM with a large database of PET imaging reports represents an advancement in medical reporting within nuclear medicine. By providing tailored, data-driven insights, the system not only improves the relevance of PET report generation but also supports enhanced decision-making and educational opportunities. This study underscores the potential of advanced AI tools in transforming medical imaging reporting practices.

Large-scale single-cell RNA sequencing (scRNA-seq) and spatial transcriptomics (ST) have transformed biomedical research into a data-driven field, enabling the creation of comprehensive data atlases. These methodologies facilitate detailed understanding of biology and pathophysiology, aiding in the discovery of new therapeutic targets. However, the complexity and sheer volume of data from these technologies present analytical challenges, particularly in robust cell typing, integration and understanding complex spatial relationships of cells. To address these challenges, we developed CELLama (Cell Embedding Leverage Language Model Abilities), a framework that leverage language model to transform cell data into ’sentences’ that encapsulate gene expressions and metadata, enabling universal cellular data embedding for various analysis. CELLama, serving as a foundation model, supports flexible applications ranging from cell typing to the analysis of spatial contexts, independently of manual reference data selection or intricate dataset-specific analytical workflows. Our results demonstrate that CELLama has significant potential to transform cellular analysis in various contexts, from determining cell types across multi-tissue atlases and their interactions to unraveling intricate tissue dynamics.

Purpose The tumor microenvironment (TME) is crucial in colorectal cancer as it influences disease progression, treatment response, and patient outcomes, providing valuable insights for personalized therapies and prognostic assessments. Here, we have developed a deep learning model by integrating hematoxylin and eosin (H&E) stained images of colorectal cancer and image-based spatial transcriptomics (Xenium) to infer spatial mapping of cell types in TME only using H&E images. Methods A total of 30 H&E images of colorectal cancer obtained by tissue microarray were registered with image-based spatial transcriptomics data (Xenium). Utilizing a Variational Autoencoder (VAE) based model and leveraging reference single-cell data enables the acquisition of cell types for individual cells in image-based spatial transcriptomics. A convolutional neural network (CNN) model was developed using H&E image as inputs to predict cell types in H&E-stained tissue image patches of colorectal cancer collected from various patients. The model also estimated the cell types from H&E-stained whole slide tissue image of colorectal cancer of The Cancer Genome Atlas (TCGA-COAD). Results The accuracy of the model's predictions for cell types using H&E image patches was notably high and exhibited a significant concordance with the results obtained through the validation. The Intersection over Union (IoU) metric for image segmentation indicated a value of 0.66 for epithelial cells and 0.44 for TNK cells. The output of deep learning model for epithelial cells and T/NK cells from TCGA-COAD tissue images showed a correlation with human-labeled regions of cancer epithelium and immune cells. Conclusions Leveraging image-based spatial transcriptomics, we developed a deep learning model capable of discerning various cell types within the tumor microenvironment solely from H&E images. This clinically translatable approach is valuable for investigating tumor microenvironment to develop biomarkers associated with various cancer therapeutics particularly immuno-oncology drugs. This approach can yield objective deep learning-based models without human labels for characterizing the tumor microenvironment in single-cell resolution, particularly regarding spatial immune distribution. Citation Format: Seungho Cook, Dongjoo Lee, Myunghyun Lim, Jae Eun Lee, Daeseung Lee, Hyung-Jun Im, Jung-Soo Pyo, Kwon Joong Na, Hongyoon Choi. Development of a deep learning model for cell type mapping in colorectal cancer using H&E images leveraging image-based spatial transcriptomics data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 899.