Jakob Nikolas Kather

Jakob Nikolas Kather
Technische Universität Dresden | TUD · Else Kroener Fresenius Center for Digital Health

Professor

About

189
Publications
35,873
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
4,178
Citations
Citations since 2017
173 Research Items
4141 Citations
201720182019202020212022202305001,0001,500
201720182019202020212022202305001,0001,500
201720182019202020212022202305001,0001,500
201720182019202020212022202305001,0001,500
Introduction
Head of Clinical Artificial Intelligence at the Faculty of Medicine and the Faculty of Computer Science, Technical University Dresden, Germany. I develop computational modeling and deep learning methods for clinical problems in oncology
Additional affiliations
July 2018 - May 2022
RWTH Aachen University
Position
  • Junior Group Leader
September 2014 - August 2016
Universität Heidelberg
Position
  • Researcher
August 2014 - May 2016
Universität Heidelberg
Position
  • Master's Student

Publications

Publications (189)
Article
Full-text available
Solid tumors are rich ecosystems of numerous different cell types whose interactions lead to immune escape and resistance to immunotherapy in virtually all patients with metastatic cancer. Here, we have developed a 3D model of human solid tumor tissue that includes tumor cells, fibroblasts, and myeloid and lymphoid immune cells and can represent ov...
Article
Full-text available
Background For virtually every patient with colorectal cancer (CRC), hematoxylin–eosin (HE)–stained tissue slides are available. These images contain quantitative information, which is not routinely used to objectively extract prognostic biomarkers. In the present study, we investigated whether deep convolutional neural networks (CNNs) can extract...
Article
Full-text available
Microsatellite instability determines whether patients with gastrointestinal cancer respond exceptionally well to immunotherapy. However, in clinical practice, not every patient is tested for MSI, because this requires additional genetic or immunohistochemical tests. Here we show that deep residual learning can predict MSI directly from H&E histolo...
Article
Full-text available
The text-guided diffusion model GLIDE (Guided Language to Image Diffusion for Generation and Editing) is the state of the art in text-to-image generative artificial intelligence (AI). GLIDE has rich representations, but medical applications of this model have not been systematically explored. If GLIDE had useful medical knowledge, it could be used...
Preprint
Full-text available
Background DNA methylation biomarkers (e.g., methylation level at CpG sites) have the potential to improve prognostic accuracy for patients with colorectal cancer (CRC). We identified existing DNA methylation-based prognostic biomarkers and prediction models for CRC prognosis and validated them in a large external cohort. Methods Epigenome-wide stu...
Preprint
Full-text available
Objectives To identify existing DNA methylation-based prognostic biomarkers and prediction models for colorectal cancer (CRC) prognosis and to validate them in a large external cohort. Design Systematic review and external validation study. Data source Systematic search in PubMed and Web of Science until October 2022 to identify epigenome-wide stud...
Article
In addition to the traditional staging system in colorectal cancer (CRC), the Immunoscore® has been proposed to characterize the level of immune infiltration in tumor tissue and as a potential prognostic marker. The aim of this study was to examine and validate associations of an immune cell score analogous to the Immunoscore® with established mole...
Chapter
Tissue typology annotation in Whole Slide histological images is a complex and tedious, yet necessary task for the development of computational pathology models. We propose to address this problem by applying Open Set Recognition techniques to the task of jointly classifying tissue that belongs to a set of annotated classes, e.g. clinically relevan...
Article
Full-text available
Background: Tumor resection represents the only potentially curative therapy for patients with biliary tract cancer. Nevertheless, disease recurrence is observed in about 50% of patients, leading to a 5-years survival rate of less than 50%. The Golgi protein 73 (GP73), a type II Golgi transmembrane protein, exerts important functions of intracellu...
Article
Full-text available
Background Image-based cancer classifiers suffer from a variety of problems which negatively affect their performance. For example, variation in image brightness or different cameras can already suffice to diminish performance. Ensemble solutions, where multiple model predictions are combined into one, can improve these problems. However, ensembles...
Preprint
Artificial Intelligence (AI) has a multitude of applications in cancer research and oncology. However, the training of AI systems is impeded by the limited availability of large datasets due to data protection requirements and other regulatory obstacles. Federated and swarm learning represent possible solutions to this problem by collaboratively tr...
Preprint
Tissue typology annotation in Whole Slide histological images is a complex and tedious, yet necessary task for the development of computational pathology models. We propose to address this problem by applying Open Set Recognition techniques to the task of jointly classifying tissue that belongs to a set of annotated classes, e.g. clinically relevan...
Article
Full-text available
Artificial intelligence (AI) can predict the presence of molecular alterations directly from routine histopathology slides. However, training robust AI systems requires large datasets for which data collection faces practical, ethical and legal obstacles. These obstacles could be overcome with swarm learning (SL), in which partners jointly train AI...
Article
Background The overall survival of patients with advanced and refractory oesophageal squamous cell carcinoma, mostly aged 65 years and older, is poor. Treatment with PD-1 antibodies showed improved progression-free survival and overall survival. We assessed the safety and efficacy of combined nivolumab and ipilimumab therapy in this population. Me...
Article
Cholangiocarcinoma (CCA) is a malignant tumour arising from the biliary system. In Europe, this tumour frequently presents as a sporadic cancer in patients without defined risk factors and is usually diagnosed at advanced stages with a consequent poor prognosis. Therefore, the identification of biomarkers represents an utmost need for patients with...
Article
Full-text available
Background Due to their ability to solve complex problems, deep neural networks (DNNs) are becoming increasingly popular in medical applications. However, decision-making by such algorithms is essentially a black-box process that renders it difficult for physicians to judge whether the decisions are reliable. The use of explainable artificial intel...
Article
Artificial intelligence (AI) can extract visual information from histopathological slides and yield biological insight and clinical biomarkers. Whole slide images are cut into thousands of tiles and classification problems are often weakly-supervised: the ground truth is only known for the slide, not for every single tile. In classical weakly-super...
Article
Full-text available
Aims: Artificial intelligence (AI) provides a powerful tool to extract information from digitized histopathology whole slide images. In the last five years, academic and commercial actors have developed new technical solutions for a diverse set of tasks, including tissue segmentation, cell detection, mutation prediction, prognostication and predic...
Article
Full-text available
Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that a weakly supervised AI can be trained on data from a clinic...
Preprint
Full-text available
Artificial Intelligence (AI) can support diagnostic workflows in oncology by aiding diagnosis and providing biomarkers. AI applications are therefore expected to evolve from academic prototypes to commercial products in the coming years. However, AI applications are vulnerable to adversarial attacks, such as malicious interference with test data ai...
Article
Precision oncology relies on the identification of targetable molecular alterations in tumor tissues. In many tumor types, a limited set of molecular tests is currently part of standard diagnostic workflows. However, universal testing for all targetable alterations, especially rare ones, is limited by the cost and availability of molecular assays....
Article
Full-text available
In the last four years, advances in Deep Learning technology have enabled the inference of selected mutational alterations directly from routine histopathology slides. In particular, recent studies have shown that genetic changes in clinically relevant driver genes are reflected in the histological phenotype of solid tumors and can be inferred by a...
Article
Full-text available
Background and study aims: Multiple computer-aided systems for polyp detection (CADe) are currently introduced into clinical practice, with an unclear effect on examiner behavior. In this work, we aimed to measure the influence of a CADe system on reaction time, mucosa misinterpretations, and changes in visual gaze pattern. Patients and methods:...
Preprint
Full-text available
Several mathematical models to predict tumor growth over time have been developed in the last decades. A central aspect of such models is the interaction of tumor cells with immune effector cells. The Kuznetsov model (Kuznetsov et al. (1994), Bulletin of Mathematical Biology, vol. 56, no. 2, pp. 295-321) is the most prominent of these models and ha...
Article
Full-text available
Classical mathematical models of tumor growth have shaped our understanding of cancer and have broad practical implications for treatment scheduling and dosage. However, even the simplest textbook models have been barely validated in real world-data of human patients. In this study, we fitted a range of differential equation models to tumor volume...
Article
Full-text available
Characterizing the mode—the way, manner or pattern—of evolution in tumours is important for clinical forecasting and optimizing cancer treatment. Sequencing studies have inferred various modes, including branching, punctuated and neutral evolution, but it is unclear why a particular pattern predominates in any given tumour. Here we propose that tum...
Article
Full-text available
Clinical routine in hepatology involves the diagnosis and treatment of a wide spectrum of metabolic, infectious, autoimmune and neoplastic diseases. Clinicians integrate qualitative and quantitative information from multiple data sources to make a diagnosis, prognosticate the disease course, and recommend a treatment. In the last five years, advanc...
Article
Background and aims Patients with hepatocellular carcinoma (HCC) displaying overexpression of immune gene signatures are likely to be more sensitive to immunotherapy, however the use of such signatures in clinical settings remains challenging. We thus aimed, using artificial intelligence (AI) on whole-slide digital histological Images (WSI), to dev...
Article
Full-text available
Background: Microsatellite instability (MSI)/mismatch repair deficiency (dMMR) is a key genetic feature which should be tested in every patient with colorectal cancer (CRC) according to medical guidelines. Artificial intelligence (AI) methods can detect MSI/dMMR directly in routine pathology slides, but the test performance has not been systematic...
Preprint
Full-text available
The text-guided diffusion model GLIDE (Guided Language to Image Diffusion for Generation and Editing) is the state of the art in text-to-image generative artificial intelligence (AI). GLIDE has rich representations, but medical applications of this model have not been systematically explored. If GLIDE had useful medical knowledge, it could be used...
Preprint
Full-text available
Artificial intelligence (AI) provides a powerful tool to extract information from digitized histopathology whole slide images. In the last five years, academic and commercial actors have developed new technical solutions for a diverse set of tasks, including tissue segmentation, cell detection, mutation prediction, prognostication and prediction of...
Preprint
Full-text available
The interpretation of digitized histopathology images has been transformed thanks to artificial intelligence (AI). End-to-end AI algorithms can infer high-level features directly from raw image data, extending the capabilities of human experts. In particular, AI can predict tumor subtypes, genetic mutations and gene expression directly from hematox...
Article
Full-text available
Microsatellite instability (MSI) or deficient mismatch repair (dMMR) is a clinically important genetic feature affecting 10-15% of colorectal cancer (CRC) patients. Patients with metastatic MSI/dMMR CRC are eligible for therapy with immune checkpoint inhibitors, making MSI/dMMR the most important immuno-oncological biomarker in CRC. Gold standard t...
Preprint
Full-text available
Osteoarthritis (OA) is the most common joint disorder affecting substantial proportions of the global population, primarily the elderly. Despite its individual and socioeconomic burden, the onset and progression of OA can still not be reliably predicted. Aiming to fill this diagnostic gap, we introduce an unsupervised learning scheme based on gener...
Article
Full-text available
Background Over the past decade, the development of molecular high-throughput methods (omics) increased rapidly and provided new insights for cancer research. In parallel, deep learning approaches revealed the enormous potential for medical image analysis, especially in digital pathology. Combining image and omics data with deep learning tools may...
Preprint
Full-text available
Artificial Intelligence (AI) can extract clinically actionable information from medical image data. In cancer histopathology, AI can be used to predict the presence of molecular alterations directly from routine histopathology slides. However, training robust AI systems requires large datasets whose collection faces practical, ethical and legal obs...
Article
Full-text available
Background Histopathological assessment of transplant biopsies is currently the standard method to diagnose allograft rejection and can help guide patient management, but it is one of the most challenging areas of pathology, requiring considerable expertise, time, and effort. We aimed to analyse the utility of deep learning to preclassify histology...
Preprint
In this paper, a large dataset of 590 Non-Small Cell Lung Patients treated with either chemotherapy or immunotherapy was used to determine whether a game-theoretic model including both evolution of therapy resistance and cost of resistance provides a better fit than classical mathematical models of population growth (exponential, logistic, classic...
Article
The spread of early‐stage (T1 and T2) adenocarcinomas to loco‐regional lymph nodes is a key event in disease progression of colorectal cancer (CRC). The cellular mechanisms behind this event are not completely understood and existing predictive biomarkers are imperfect. Here, we used an end‐to‐end Deep Learning algorithm to identify risk factors fo...
Preprint
Full-text available
Classical mathematical models of tumor growth have shaped our understanding of cancer and have broad practical implications for treatment scheduling and dosage. However, even the simplest textbook models have been barely validated in real world-data of human patients. In this study, we fitted a range of differential equation models to tumor volume...
Article
Full-text available
Background: Lymph node status is a prognostic marker and strongly influences therapeutic decisions in colorectal cancer (CRC). Objectives: The objective of the study is to investigate whether image features extracted by a deep learning model from routine histological slides and/or clinical data can be used to predict CRC lymph node metastasis (LNM...
Article
Full-text available
Background Multiple studies have compared the performance of artificial intelligence (AI)–based models for automated skin cancer classification to human experts, thus setting the cornerstone for a successful translation of AI-based tools into clinicopathological practice. Objective The objective of the study was to systematically analyse the curre...
Article
Full-text available
Background One prominent application for deep learning–based classifiers is skin cancer classification on dermoscopic images. However, classifier evaluation is often limited to holdout data which can mask common shortcomings such as susceptibility to confounding factors. To increase clinical applicability, it is necessary to thoroughly evaluate suc...
Article
Full-text available
Background: Gastrointestinal cancers account for approximately 20% of all cancer diagnoses and are responsible for 22.5% of cancer deaths worldwide. Artificial intelligence ebased diagnostic support systems, in particular convolutional neural network (CNN)ebased image analysis tools, have shown great potential in medical computer vision. In this sy...
Article
Full-text available
Deep Learning is a powerful tool in computational pathology: it can be used for tumor detection and for predicting genetic alterations based on histopathology images alone. Conventionally, tumor detection and prediction of genetic alterations are two separate workflows. Newer methods have combined them, but require complex, manually engineered comp...
Preprint
Full-text available
Artificial intelligence (AI) can extract subtle visual information from digitized histopathology slides and yield scientific insight on genotype-phenotype interactions as well as clinically actionable recommendations. Classical weakly supervised pipelines use an end-to-end approach with residual neural networks (ResNets), modern convolutional neura...
Article
Full-text available
Background Response to immunotherapy in gastric cancer is associated with microsatellite instability (or mismatch repair deficiency) and Epstein-Barr virus (EBV) positivity. We therefore aimed to develop and validate deep learning-based classifiers to detect microsatellite instability and EBV status from routine histology slides. Methods In this r...
Article
Full-text available
Aim: Sentinel lymph node status is a central prognostic factor for melanomas. However, the surgical excision involves some risks for affected patients. In this study, we therefore aimed to develop a digital biomarker that can predict lymph node metastasis non-invasively from digitised H&E slides of primary melanoma tumours. Methods: A total of 4...
Article
Full-text available
The oncological role of the density of nerve fibers (NFs) in the tumor microenvironment (TME) in intrahepatic cholangiocarcinoma (iCCA) remains to be determined. Therefore, data of 95 iCCA patients who underwent hepatectomy between 2010 and 2019 was analyzed regarding NFs and long-term outcome. Extensive group comparisons were carried out and the a...
Article
Full-text available
The Cancer Genome Atlas (TCGA) is one of the largest biorepositories of digital histology. Deep learning (DL) models have been trained on TCGA to predict numerous features directly from histology, including survival, gene expression patterns, and driver mutations. However, we demonstrate that these features vary substantially across tissue submitti...
Article
Full-text available
Background: Recent years have been witnessing a substantial improvement in the accuracy of skin cancer classification using convolutional neural networks (CNNs). CNNs perform on par with or better than dermatologists with respect to the classification tasks of single images. However, in clinical practice, dermatologists also use other patient data...
Article
Full-text available
Rising cancer care costs impose financial burdens on health systems. Applying artificial intelligence to diagnostic algorithms may reduce testing costs and avoid wasteful therapy-related expenditures. To evaluate the financial and clinical impact of incorporating artificial intelligence-based determination of mismatch repair/microsatellite instabil...
Article
Full-text available
Introduction: Perihilar cholangiocarcinoma (pCCA) is a biliary tract cancer with a dismal prognosis, with surgery being the only chance of cure. A characteristic aggressive biological feature of pCCA is perineural growth which is defined by the invasion of cancer cells to nerves and nerve fibers. Recently, nerve fiber density (NFD) was linked to o...
Article
Full-text available
Background Clinicians and pathologists traditionally use patient data in addition to clinical examination to support their diagnoses. Objectives We investigated whether a combination of histologic whole slides image (WSI) analysis based on convolutional neural networks (CNNs) and commonly available patient data (age, sex and anatomical site of the...
Article
Full-text available
Therapy with immune checkpoint inhibitors (ICIs) can lead to durable tumor control in patients with various advanced stage malignancies. However, this is not the case for all patients, leading to an ongoing search for biomarkers predicting response and outcome to ICI. The B and T lymphocyte attenuator (BTLA) is an immune checkpoint expressed on imm...
Article
Full-text available
Background: Fibroblast growth factor receptor (FGFR) inhibitor treatment has become the first clinically approved targeted therapy in bladder cancer. However, it requires previous molecular testing of each patient, which is costly and not ubiquitously available. Objective: To determine whether an artificial intelligence system is able to predict...
Article
Full-text available
Severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) causes COVID-19 and is responsible for the ongoing pandemic. Screening of potential antiviral drugs against SARS-CoV-2 depend on in vitro experiments, which are based on the quantification of the virus titer. Here, we used virus-induced cytopathic effects (CPE) in brightfield microscopy o...
Article
Full-text available
Background Immune checkpoint inhibitors (ICIs) have led to a paradigm shift in cancer therapy, improving outcomes in the treatment of various malignancies. However, not all patients benefit to the same extend from ICI. Reliable tools to predict treatment response and outcome are missing. Soluble urokinase plasminogen activator receptor (suPAR) is a...