No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Artificial Intelligence in Nuclear Cardiology
Abstract
Artificial intelligence (AI) encompasses a variety of computer algorithms that have a wide range of potential clinical applications in nuclear cardiology. This article will introduce core terminology and concepts for AI including classifications of AI as well as training and testing regimens. We will then highlight the potential role for AI to improve image registration and image quality. Next, we will discuss methods for AI-driven image attenuation correction. Finally, we will review advancements in machine learning and deep-learning applications for disease diagnosis and risk stratification, including efforts to improve clinical translation of this valuable technology with explainable AI models.
... Uncovering STEMI patient phenotypes using unsupervised machine learning In recent years, machine learning (ML) has revolutionized health care by providing powerful tools for data analysis and predictive modeling. Many ML models utilize a supervised learning regimen, where the model is tasked with optimally predicting a specific outcome of interest [1]. For example, ML models can be trained to predict amputation-free survival in patients following vascular surgery [2] or recurrence of atrial fibrillation following ablation [3]. ...
... For example, ML models can be trained to predict amputation-free survival in patients following vascular surgery [2] or recurrence of atrial fibrillation following ablation [3]. However, it is also possible to train ML models using unsupervised learning where the model is not predicting a specific outcome [1]. Instead, the ML model is tasked with determining the underlying structure of the data. ...
... A flattening layer is often involved in achieving a simplified output of normal or abnormal by assigning weights to the variables involved in the decisionmaking process such as age, sex, and other individual patient characteristics. Forms of DL include convolutional neural networks (CNN), convolutional autoencoder, and U-Net [15,16]. Most common form of artificial neural network employed in nuclear imaging is CNN. ...
Purpose of Review
Single photon emission computed tomography (SPECT) and positron emission tomography (PET) cardiac imaging have evolved, providing innumerable data points for the clinician reader to analyze to achieve accurate diagnosis and guide management. The advent of artificial intelligence (AI) could play a pivotal role in better harnessing these data and improving nuclear cardiology workflows. In this review, we explored the current applications of AI in various aspects of nuclear cardiology.
Recent Findings
Innovative studies have explored the use of AI, particularly deep learning models, to identify ideal patient candidates for stress-only imaging to reduce radiation exposure and acquisition time. Furthermore, there is published evidence that deep learning can provide efficient methods to achieve reliable image segmentation, attenuation correction, and image registration. In addition, AI-based disease diagnosis and risk prediction models have been shown to perform similarly if not better than expert readers in some settings. Beyond coronary artery disease, there are promising results of deep learning algorithms to improve diagnostic imaging for cardiac sarcoidosis and amyloidosis.
Summary
Recent advancements in AI models provide an opportunity to refine the nuclear cardiology workflow ranging from patient selection to disease prediction and reporting. Promising results from these early studies need to be replicated in larger heterogenous patient populations to demonstrate generalizability prior to widespread adoption in clinical practice.
... This deep and nuanced analysis enables cardiologists to make more informed decisions, grounded in a comprehensive understanding of each patient's unique clinical profile. AI tools, such as predictive analytics, can forecast potential disease progression and response to treatments, allowing cardiologists to tailor their therapeutic strategies to individual patients [70][71][72][73][74][75][76]. For instance, AI algorithms can predict adverse cardiac events, helping cardiologists to preemptively modify treatment plans to prevent such outcomes [76,77]. ...
Background: Artificial intelligence (AI) in echocardiography represents a transformative advancement in cardiology, addressing longstanding challenges in cardiac diagnostics. Echocardiography has traditionally been limited by operator-dependent variability and subjective interpretation, which impact diagnostic reliability. This study evaluates the role of AI, particularly machine learning (ML), in enhancing the accuracy and consistency of echocardiographic image analysis and its potential to complement clinical expertise. Methods: A comprehensive review of existing literature was conducted to analyze the integration of AI into echocardiography. Key AI functionalities, such as image acquisition, standard view classification, cardiac chamber segmentation, structural quantification, and functional assessment, were assessed. Comparisons with traditional imaging modalities like computed tomography (CT), nuclear imaging, and magnetic resonance imaging (MRI) were also explored. Results: AI algorithms demonstrated expert-level accuracy in diagnosing conditions such as cardiomyopathies while reducing operator variability and enhancing diagnostic consistency. The application of ML was particularly effective in automating image analysis and minimizing human error, addressing the limitations of subjective operator expertise. Conclusions: The integration of AI into echocardiography marks a pivotal shift in cardiovascular diagnostics, offering enhanced accuracy, consistency, and reliability. By addressing operator variability and improving diagnostic performance, AI has the potential to elevate patient care and herald a new era in cardiology.
... With the rapid advances in computing technology and the ongoing refinement of statistical theory, machine learning (ML) has gradually been promoted and applied in clinical practice. For instance, ML can not only improve image quality, reduce misregistration, and simulate attenuation correction imaging in core cardiology [6], but also be used for cancer screening (detection of lesions), characterization and grading of tumors, and prognosis prediction, thus facilitating clinical decision-making [7]. Since fundus images are noncontact, noninvasive, low-cost, easily accessible, and easy to process, ML has been extensively used to diagnose common retinal diseases, including diabetic retinopathy [8][9][10], macular degeneration [10], and glaucoma [11][12][13]. ...
Background
In recent years, with the rapid development of machine learning (ML), it has gained widespread attention from researchers in clinical practice. ML models appear to demonstrate promising accuracy in the diagnosis of complex diseases, as well as in predicting disease progression and prognosis. Some studies have applied it to ophthalmology, primarily for the diagnosis of pathologic myopia and high myopia-associated glaucoma, as well as for predicting the progression of high myopia. ML-based detection still requires evidence-based validation to prove its accuracy and feasibility.
Objective
This study aims to discern the performance of ML methods in detecting high myopia and pathologic myopia in clinical practice, thereby providing evidence-based support for the future development and refinement of intelligent diagnostic or predictive tools.
Methods
PubMed, Cochrane, Embase, and Web of Science were thoroughly retrieved up to September 3, 2023. The prediction model risk of bias assessment tool was leveraged to appraise the risk of bias in the eligible studies. The meta-analysis was implemented using a bivariate mixed-effects model. In the validation set, subgroup analyses were conducted based on the ML target events (diagnosis and prediction of high myopia and diagnosis of pathological myopia and high myopia-associated glaucoma) and modeling methods.
Results
This study ultimately included 45 studies, of which 32 were used for quantitative meta-analysis. The meta-analysis results unveiled that for the diagnosis of pathologic myopia, the summary receiver operating characteristic (SROC), sensitivity, and specificity of ML were 0.97 (95% CI 0.95-0.98), 0.91 (95% CI 0.89-0.92), and 0.95 (95% CI 0.94-0.97), respectively. Specifically, deep learning (DL) showed an SROC of 0.97 (95% CI 0.95-0.98), sensitivity of 0.92 (95% CI 0.90-0.93), and specificity of 0.96 (95% CI 0.95-0.97), while conventional ML (non-DL) showed an SROC of 0.86 (95% CI 0.75-0.92), sensitivity of 0.77 (95% CI 0.69-0.84), and specificity of 0.85 (95% CI 0.75-0.92). For the diagnosis and prediction of high myopia, the SROC, sensitivity, and specificity of ML were 0.98 (95% CI 0.96-0.99), 0.94 (95% CI 0.90-0.96), and 0.94 (95% CI 0.88-0.97), respectively. For the diagnosis of high myopia-associated glaucoma, the SROC, sensitivity, and specificity of ML were 0.96 (95% CI 0.94-0.97), 0.92 (95% CI 0.85-0.96), and 0.88 (95% CI 0.67-0.96), respectively.
Conclusions
ML demonstrated highly promising accuracy in diagnosing high myopia and pathologic myopia. Moreover, based on the limited evidence available, we also found that ML appeared to have favorable accuracy in predicting the risk of developing high myopia in the future. DL can be used as a potential method for intelligent image processing and intelligent recognition, and intelligent examination tools can be developed in subsequent research to provide help for areas where medical resources are scarce.
Trial Registration
PROSPERO CRD42023470820; https://tinyurl.com/2xexp738
... This deep and nuanced analysis enables cardiologists to make more informed decisions, grounded in a comprehensive understanding of each patient's unique clinical profile. AI tools, such as predictive analytics, can forecast potential disease progression and response to treatments, allowing cardiologists to tailor their therapeutic strategies to individual patients [70][71][72][73][74][75][76]. For instance, AI algorithms can predict adverse cardiac events, helping cardiologists to preemptively modify treatment plans to prevent such outcomes [76,77]. ...
Artificial Intelligence (AI) in echocardiography represents a transformative leap in the realm of cardiology, refining diagnostic accuracy and mitigating human error. AI, often erroneously equated with robotics or automation, transcends mere mechanical operations. It encompasses the capacity of machines to emulate human decision-making processes and problem-solving abilities, augmenting rather than replacing human expertise. In echocardiography, AI has emerged as a pivotal tool, enhancing consistency and precision, elements crucial in cardiac diagnostics. Echocardiography is indispensable in the diagnosis of cardiovascular diseases, but it is traditionally hampered by operator-dependent variability and subjective interpretation. AI intervenes here, offering high-precision automation in echocardiographic analysis. This encompasses several key phases: from image acquisition and standard view classification to cardiac chamber segmentation, structural quantification, and functional assessment. AI algorithms have demonstrated expertise-level accuracy in these domains, notably in identifying cardiac conditions such as cardiomyopathies. The incorporation of AI into echocardiography tackles inherent challenges like variance in image capture and analysis. AI's prowess in machine learning (ML), a crucial subset of AI, specifically enhances image interpretation in echocardiography. This advancement is significant given echocardiography's reliance on the operator's subjective expertise, a reliance more pronounced than in other imaging modalities such as computed tomography (CT), nuclear imaging, and magnetic resonance imaging (MRI). AI technologies, through automated and consistent interpretations, promise to reshape echocardiographic diagnostics. This review delves into ML's role in augmenting echocardiographic image analysis and diagnostic performance. It also examines the existing literature, highlighting AI's value in echocardiography and its potential to elevate patient care. In summary, AI's integration into echocardiography marks a pivotal shift, offering enhanced diagnostic capabilities and heralding a new era in cardiovascular care.
... By integrating clinical, stress, and imaging data with machine learning, AI can improve the detection of obstructive coronary artery disease (CAD) and enhance risk prediction. Additionally, AI algorithms have been developed to simulate attenuation correction imaging, minimize misregistration, and improve overall image quality, making data science a critical driver of change in cardiovascular imaging [23]. ...
This article examines the transformative potential of artificial intelligence (AI) in shaping the future of healthcare. It highlights AI's capacity to revolutionize various medical fields, including diagnostics, personalized treatment, drug discovery, telemedicine, and patient care management. Key areas explored include AI's roles in cancer screening, reproductive health, cardiology, outpatient care, laboratory diagnosis, language translation, neuroscience, robotic surgery, radiology, personal healthcare, patient engagement, AI-assisted rehabilitation with exoskeleton robots, and administrative efficiency. The article also addresses challenges to AI adoption, such as privacy concerns, ethical issues, cost barriers, and decision-making authority in patient care. By overcoming these challenges and building trust, AI is positioned to become a critical driver in advancing healthcare, improving outcomes, and meeting the future needs of patients and providers
... ML also has the potential to facilitate clinical and imaging big-data integration for personalized risk stratification by efficiently combining large numbers of clinical and quantitative imaging variables to optimize diagnostic or prognostic utility [20,21]. The majority of the clinical studies on ML in nuclear cardiology focused on MPI related to CAD diagnosis, risk assessment, prediction of revascularization and LV ejection fraction (EF), and prognostication [22][23][24][25]. The incorporation of radiomics into the ML approach is even more scarce with a handful of studies focusing on CAD diagnosis, risk stratification, and cardiac contractile pattern recognition [26][27][28][29][30]. ...
Background
To assess the feasibility of a machine learning (ML) approach using radiomics features of perfusion defects on rest myocardial perfusion imaging (MPI) to detect the presence of hibernating myocardium.
Methodology
Data of patients who underwent 99mTc-sestamibi MPI and 18F-FDG PET/CT for myocardial viability assessment were retrieved. Rest MPI data were processed on ECToolbox, and polar maps were saved using the NFile PMap tool. The reference standard for defining hibernating myocardium was the presence of mismatched perfusion-metabolism defect with impaired myocardial contractility at rest. Perfusion defects on the polar maps were delineated with regions of interest (ROIs) after spatial resampling and intensity discretization. Replicable random sampling allocated 80% (257) of the perfusion defects of the patients from January 2017 to September 2022 to the training set and the remaining 20% (64) to the validation set. An independent dataset of perfusion defects from 29 consecutive patients from October 2022 to January 2023 was used as the testing set for model evaluation. One hundred ten first and second-order texture features were extracted for each ROI. After feature normalization and imputation, 14 best-ranked features were selected using a multistep feature selection process including the Logistic Regression and Fast Correlation-Based Filter. Thirteen supervised ML algorithms were trained with stratified five-fold cross-validation on the training set and validated on the validation set. The ML algorithms with a Log Loss of <0.688 and <0.672 in the cross-validation and validation steps were evaluated on the testing set. Performance matrices of the algorithms assessed included area under the curve (AUC), classification accuracy (CA), F1 score, precision, recall, and specificity. To provide transparency and interpretability, SHapley Additive exPlanations (SHAP) values were assessed and depicted as beeswarm plots.
Results
Two hundred thirty-nine patients (214 males; mean age 56 ± 11 years) were enrolled in the study. There were 371 perfusion defects (321 in the training and validation sets; 50 in the testing set). Based on the reference standard, 168 perfusion defects had hibernating myocardium (139 in the training and validation sets; 29 in the testing set). On cross-validation, six ML algorithms with Log Loss <0.688 had AUC >0.800. On validation, 10 ML algorithms had a Log Loss value <0.672, among which six had AUC >0.800. On model evaluation of the selected models on the unseen testing set, nine ML models had AUC >0.800 with Gradient Boosting Random Forest (xgboost) [GB RF (xgboost)] achieving the highest AUC of 0.860 and could detect the presence of hibernating myocardium in 21/29 (72.4%) perfusion defects with a precision of 87.5% (21/24), specificity 85.7% (18/21), CA 78.0% (39/50) and F1 Score 0.792. Four models depicted a clear pattern of model interpretability based on the beeswarm SHAP plots. These were GB RF (xgboost), GB (scikit-learn), GB (xgboost), and Random Forest.
Conclusion
Our study demonstrates the potential of ML in detecting hibernating myocardium using radiomics features extracted from perfusion defects on rest MPI images. This proof-of-concept underscores the notion that radiomics features capture nuanced information beyond what is perceptible to the human eye, offering promising avenues for improved myocardial viability assessment.
... The increasing speed of decision-making based on algorithms, automation bias, and over-trust in automated (often complex) systems could increase the risk of unintended errors (Johnson 2022;Rautenbach 2023). Experts point to previous failures of automated "dead hand" early warning systems, arguing that AI is "brittle" and could result in malfunctions which affect states' confidence in their second-strike capabilities (Horowitz 2019;Johnson 2020;Kallenborn 2022). Studies also highlight risks associated with AI-enabled remotely controlled nuclear delivery platforms (Boulanin et al. 2020), spoofing and adversarial attacks, as well as the role of AI in cyber-attacks, including on nuclear facilities (Johnson 2019a;Sharikov 2018). ...
Military applications of artificial intelligence (AI) are said to impact strategic stability, broadly defined as the absence of incentives for armed conflict between nuclear powers. While previous research explores the potential implications of AI for nuclear deterrence based on technical characteristics, little attention has been dedicated to understanding how policymakers of nuclear powers conceive of AI technologies and their impacts. This paper argues that the relationship between AI and strategic stability is not only given through the technical nature of AI, but also constructed by policymakers’ beliefs about these technologies and other states’ intentions to use them. Adopting a constructivist perspective, we investigate how decision-makers from the United States and Russia talk about military AI by analyzing US and Russian official discourses from 2014–2023 and 2017-2023, respectively. We conclude that both sides have constructed a threat out of their perceived competitors’ AI capabilities, reflecting their broader perspectives of strategic stability, as well as the social context characterized by distrust and feelings of competition. Their discourses fuel a cycle of misperceptions which could be addressed via confidence building measures. However, this competitive cycle is unlikely to improve due to ongoing tensions following the Russian invasion of Ukraine.
... Moreover, the implementation of AI may increase first-mover advantage. Particularly for countries without a secure second-strike capability, incentives to attack preemptively increase as the speed of combat increases (Horowitz, 2019). The advances in HGVs bring about a similar situation: They create risks that the target's strategic forces will be obliterated before they can be engaged (Speier et al., 2017). ...
What is the impact of emerging technologies on nuclear security and disarmament? Current rapid technological advances are taking place against the backdrop of increased investments in modernizing nuclear arsenals, rising tensions among great powers, and increased pressure on nuclear arms control agreements. Yet, the anticipated net effect of these emerging technologies on the nuclear landscape remains ambiguous. Through a survey with 85 experts and a series of elite interviews with 14 decision-makers, this article contends that while emerging technologies destabilize nuclear deterrence by increasing nuclear risk, they can also create fresh opportunities for nuclear disarmament. Given that new technologies are changing the nature of nuclear threats, this article also argues that we need to change the way we think about arms control if we want to respond effectively to the threats posed by emerging technologies.
Healthcare, where its integration into nuclear medicine is redefining how diseases are diagnosed, treated, and managed. Nuclear medicine, which uses small amounts of radioactive materials for diagnostic imaging and therapeutic purposes, has long been pivotal in detecting conditions neurological incorporation AI in this field promises to revolutionize its practice by optimizing workflows and enhancing despite its potential, also comes with its benefits. Nuclear medicine diagnostic precision. Analyze imaging data with remarkable accuracy, often surpassing human performance in detecting subtle patterns. For example, AI can identify minute abnormalities that might overlooked by human observers. This precision survival rates. Furthermore, AI's capacity for quantitative analysis allows it to provide objective measurements of tracer uptake in target tissues, reducing variability in interpretation and ensuring consistent diagnostic outcomes. Nuclear medicine potentially streamlines clinical workflows.
Objective: Chatbots have been frequently used in many different areas in recent years, such as diagnosis and imaging, treatment, patient follow-up and support, health promotion, customer service, sales, marketing, information and technical support. The aim of this study is to evaluate the readability, comprehensibility, and accuracy of queries made by researchers in the field of health through artificial intelligence chatbots in biostatistics. Methods: A total of 10 questions from the topics frequently asked by researchers in the field of health in basic biostatistics were determined by 4 experts. The determined questions were addressed to the artificial intelligence chatbots by one of the experts and the answers were recorded. In this study, free versions of most widely preferred ChatGPT4, Gemini and Copilot chatbots were used. The recorded answers were independently evaluated as “Correct”, “Partially correct” and “Wrong” by three experts who blinded to which chatbot the answers belonged to. Then, these experts came together and examined the answers together and made the final evaluation by reaching a consensus on the levels of accuracy. The readability and understandability of the answers were evaluated with the Ateşman readability formula, Sönmez formula, Çetinkaya-Uzun readability formula and Bezirci-Yılmaz readability formulas. Results: According to the answers given to the questions addressed to the artificial intelligence chatbots, it was determined that the answers were at the “difficult” level according to the Ateşman readability formula, “insufficient reading level” according to the Çetinkaya-Uzun readability formula, and “academic level” according to the Bezirci-Yılmaz readability formula. On the other hand, the Sönmez formula gave the result of “the text is understandable” for all chatbots. It was determined that there was no statistically significant difference (p=0.819) in terms of accuracy rates of the answers given by the artificial intelligence chatbots to the questions. Conclusion: It was determined that although the chatbots tended to provide accurate information, the answers given were not readable, understandable and their accuracy levels were not high.
At the moment, one of the most common causes of morbidity and mortality is coronary heart disease, which determines the need to develop methods for its diagnosis. Among diagnostic methods, non-invasive methods occupy a special place, in particular, determination of myocardial perfusion. One of the “gold standards” for assessing cardiac muscle perfusion is positron emission tomography combined with computed tomography (PET/CT) with 82Rb-chloride. Recently, attempts have been actively made to introduce the use of artificial intelligence in a variety of areas of medical clinical practice, including the development of medical decision support systems, as well as neural networks for assessing the results of diagnostic studies. In particular, there is information about attempts to use artificial intelligence in assessing myocardial perfusion using PET/CT with 82Rb-chloride. This paper analyzes the possibilities and prospects for using artificial intelligence in assessing the results of PET/CT with 82Rb-chloride. The use of well-trained neural networks and machine learning algorithms can significantly increase the accuracy of diagnosing coronary heart disease by improving the quality of images, analyzing the data obtained, or calculating characteristics and indicators, the quantitative interpretation of which may be difficult for a doctor. Neural networks are able to take into account in the prognosis both clinical and anamnestic data and additional parameters determined from research data, which the doctor may not pay attention to, which determines the relevance and prospects for the use of artificial intelligence in relation to the interpretation of 82Rb-PET/CT results.
This chapter introduces artificial intelligence (AI) and machine learning as major enablers of military innovation, especially regarding autonomy in weapons systems. It discusses the potential of AI where sensing, decision-making and acting are concerned. It also sheds light on the risks involved and questions claims about the effectiveness, reliability and trustworthiness of AI in military settings.
Background
Myocardial perfusion imaging (MPI) is frequently used to provide risk stratification, but methods to improve the accuracy of these predictions are needed.
Objectives
We developed an explainable deep learning (DL) model (HARD MACE [major adverse cardiac events]-DL) for the prediction of death or nonfatal myocardial infarction (MI) and validated its performance in large internal and external testing groups.
Methods
Patients undergoing single-photon emission computed tomography MPI were included, with 20,401 patients in the training and internal testing group (5 sites) and 9,019 in the external testing group (2 different sites). HARD MACE-DL uses myocardial perfusion, motion, thickening, and phase polar maps combined with age, sex, and cardiac volumes. The primary outcome was all-cause mortality or nonfatal MI. Prognostic accuracy was evaluated using area under the receiver-operating characteristic curve (AUC).
Results
During internal testing, patients with normal perfusion and elevated HARD-MACE-DL risk were at higher risk than patients with abnormal perfusion and low HARD-MACE-DL risk (annualized event rate, 2.9% vs 1.2%; P < 0.001). Patients in the highest quartile of HARD MACE-DL score had an annual rate of death or MI (4.8%) 10-fold higher than patients in the lowest quartile (0.48% per year). In external testing, the AUC for HARD MACE-DL (0.73; 95% CI: 0.71-0.75) was higher than a logistic regression model (AUC: 0.70), stress TPD (AUC: 0.65), and ischemic TPD (AUC: 0.63; all P < 0.01). Calibration, a measure of how well predicted risk matches actual risk, was excellent in both groups (Brier score, 0.079 for internal and 0.070 for external).
Conclusions
The DL model predicts death or MI directly from MPI, by estimating patient-level risk with good calibration and improved accuracy compared with traditional quantitative approaches. The model incorporates mechanisms to explain to the physician which image regions contribute to the adverse event prediction.
Background: Low-dose ungated CT attenuation correction (CTAC) scans are commonly obtained with SPECT/CT myocardial perfusion imaging. Despite characteristically low image quality of CTAC, deep learning (DL) can potentially quantify coronary artery calcium (CAC) from these scans in an automatic manner. We evaluated CAC quantification derived with a DL model including correlation with expert annotations and associations with major adverse cardiovascular events (MACE). Methods: We trained a convolutional long short-term memory DL model to automatically quantify CAC on CTAC scans using 6608 studies (2 centers) and evaluated the model in an external cohort of patients without known coronary artery disease (n = 2271) obtained in a separate center. We assessed agreement between DL and expert annotated CAC scores. We also assessed associations between MACE (death, revascularization, myocardial infarction, or unstable angina) and CAC categories (0; 1-100; 101-400; >400) for scores manually derived by experienced readers and scores obtained fully automatically by DL using multivariable Cox models (adjusted for age, sex, past medical history, perfusion, and ejection fraction) and net reclassification index (NRI). Results: In the external testing population, DL CAC was 0 in 908(40.0%), 1-100 in 596(26.2%), 100-400 in 354(15.6%), and >400 in 413(18.2%) patients. Agreement in CAC category by DL CTAC and expert annotation was excellent (linear weighted Kappa 0.80), but DL CAC was obtained automatically in <2 seconds compared to ~2.5-minutes for expert CAC. DL CAC category was an independent risk for MACE with hazard ratios in comparison to CAC of zero: CAC 1-100 (2.20, 95% CI 1.54 - 3.14, p<0.001), CAC 101-400 (4.58, 95% CI 3.23 - 6.48, p<0.001), and CAC > 400 (5.92, 95% CI 4.27 - 8.22, p<0.001). Overall NRI was 0.494 for DL CAC, which was similar to expert annotated CAC (0.503). Conclusion: DL CAC from SPECT/CT attenuation maps has good agreement with expert CAC annotations and provides similar risk stratification but can be obtained automatically. DL CAC scores improved classification of a significant proportion of patients as compared to myocardial perfusion SPECT alone.
Purpose
Artificial intelligence (AI) has high diagnostic accuracy for coronary artery disease (CAD) from myocardial perfusion imaging (MPI). However, when trained using high-risk populations (such as patients with correlating invasive testing), the disease probability can be overestimated due to selection bias. We evaluated different strategies for training AI models to improve the calibration (accurate estimate of disease probability), using external testing.
Methods
Deep learning was trained using 828 patients from 3 sites, with MPI and invasive angiography within 6 months. Perfusion was assessed using upright (U-TPD) and supine total perfusion deficit (S-TPD). AI training without data augmentation (model 1) was compared to training with augmentation (increased sampling) of patients without obstructive CAD (model 2), and patients without CAD and TPD < 2% (model 3). All models were tested in an external population of patients with invasive angiography within 6 months (n = 332) or low likelihood of CAD (n = 179).
Results
Model 3 achieved the best calibration (Brier score 0.104 vs 0.121, p < 0.01). Improvement in calibration was particularly evident in women (Brier score 0.084 vs 0.124, p < 0.01). In external testing (n = 511), the area under the receiver operating characteristic curve (AUC) was higher for model 3 (0.930), compared to U-TPD (AUC 0.897) and S-TPD (AUC 0.900, p < 0.01 for both).
Conclusion
Training AI models with augmentation of low-risk patients can improve calibration of AI models developed to identify patients with CAD, allowing more accurate assignment of disease probability. This is particularly important in lower-risk populations and in women, where overestimation of disease probability could significantly influence down-stream patient management.
Background:
We aim to develop an explainable deep learning (DL) network for the prediction of all-cause mortality directly from positron emission tomography myocardial perfusion imaging flow and perfusion polar map data and evaluate it using prospective testing.
Methods:
A total of 4735 consecutive patients referred for stress and rest 82Rb positron emission tomography between 2010 and 2018 were followed up for all-cause mortality for 4.15 (2.24-6.3) years. DL network utilized polar maps of stress and rest perfusion, myocardial blood flow, myocardial flow reserve, and spill-over fraction combined with cardiac volumes, singular indices, and sex. Patients scanned from 2010 to 2016 were used for training and validation. The network was tested in a set of 1135 patients scanned from 2017 to 2018 to simulate prospective clinical implementation.
Results:
In prospective testing, the area under the receiver operating characteristic curve for all-cause mortality prediction by DL (0.82 [95% CI, 0.77-0.86]) was higher than ischemia (0.60 [95% CI, 0.54-0.66]; P <0.001), myocardial flow reserve (0.70 [95% CI, 0.64-0.76], P <0.001) or a comprehensive logistic regression model (0.75 [95% CI, 0.69-0.80], P <0.05). The highest quartile of patients by DL had an annual all-cause mortality rate of 11.87% and had a 16.8 ([95% CI, 6.12%-46.3%]; P <0.001)-fold increase in the risk of death compared with the lowest quartile patients. DL showed a 21.6% overall reclassification improvement as compared with established measures of ischemia.
Conclusions:
The DL model trained directly on polar maps allows improved patient risk stratification in comparison with established methods for positron emission tomography flow or perfusion assessments.
Background
Assessment of coronary artery calcium (CAC) by computed tomographic (CT) imaging provides an accurate measure of atherosclerotic burden. CAC is also visible in CT attenuation correction (CTAC) scans, always acquired with cardiac positron emission tomographic (PET) imaging.
Objectives
The aim of this study was to develop a deep-learning (DL) model capable of fully automated CAC definition from PET CTAC scans.
Methods
The novel DL model, originally developed for video applications, was adapted to rapidly quantify CAC. The model was trained using 9,543 expert-annotated CT scans and was tested in 4,331 patients from an external cohort undergoing PET/CT imaging with major adverse cardiac events (MACEs) (follow-up 4.3 years), including same-day paired electrocardiographically gated CAC scans available in 2,737 patients. MACE risk stratification in 4 CAC score categories (0, 1-100, 101-400, and >400) was analyzed and CAC scores derived from electrocardiographically gated CT scans (standard scores) by expert observers were compared with automatic DL scores from CTAC scans.
Results
Automatic DL scoring required <6 seconds per scan. DL CTAC scores provided stepwise increase in the risk for MACE across the CAC score categories (HR up to 3.2; P < 0.001). Net reclassification improvement of standard CAC scores over DL CTAC scores was nonsignificant (−0.02; 95% CI: −0.11 to 0.07). The negative predictive values for MACE of zero CAC with standard (85%) and DL CTAC (83%) CAC scores were similar (P = 0.19).
Conclusions
DL CTAC scores predict cardiovascular risk similarly to standard CAC scores quantified manually by experienced operators from dedicated electrocardiographically gated CAC scans and can be obtained almost instantly, with no changes to PET/CT scanning protocol.
Purpose
We sought to evaluate inter-scan and inter-reader agreement of coronary calcium (CAC) scores obtained from dedicated, ECG-gated CAC scans (standard CAC scan) and ultra-low-dose, ungated computed tomography attenuation correction (CTAC) scans obtained routinely during cardiac PET/CT imaging.
Methods
From 2928 consecutive patients who underwent same-day ⁸²Rb cardiac PET/CT and gated CAC scan in the same hybrid PET/CT scanning session, we have randomly selected 200 cases with no history of revascularization. Standard CAC scans and ungated CTAC scans were scored by two readers using quantitative clinical software. We assessed the agreement between readers and between two scan protocols in 5 CAC categories (0, 1–10, 11–100, 101–400, and > 400) using Cohen’s Kappa and concordance.
Results
Median age of patients was 70 (inter-quartile range: 63–77), and 46% were male. The inter-scan concordance index and Cohen’s Kappa for readers 1 and 2 were 0.69; 0.75 (0.69, 0.81) and 0.72; 0.8 (0.75, 0.85) respectively. The inter-reader concordance index and Cohen’s Kappa (95% confidence interval [CI]) was higher for standard CAC scans: 0.9 and 0.92 (0.89, 0.96), respectively, vs. for CTAC scans: 0.83 and 0.85 (0.79, 0.9) for CTAC scans (p = 0.02 for difference in Kappa). Most discordant readings between two protocols occurred for scans with low extent of calcification (CAC score < 100).
Conclusion
CAC can be quantitatively assessed on PET CTAC maps with good agreement with standard scans, however with limited sensitivity for small lesions. CAC scoring of CTAC can be performed routinely without modification of PET protocol and added radiation dose.
Background
Transthyretin amyloidosis (ATTR) is a progressive disease which can be diagnosed non-invasively using bone avid [99mTc]-labeled radiotracers. Thus, ATTR is also an occasional incidental finding on bone scintigraphy. In this study, we trained convolutional neural networks (CNN) to automatically detect and classify ATTR from scintigraphy images. The study population consisted of 1334 patients who underwent [99mTc]-labeled hydroxymethylene diphosphonate (HMDP) scintigraphy and were visually graded using Perugini grades (grades 0–3). A total of 47 patients had visual grade ≥ 2 which was considered positive for ATTR. Two custom-made CNN architectures were trained to discriminate between the four Perugini grades of cardiac uptake. The classification performance was compared to four state-of-the-art CNN models.
Results
Our CNN models performed better than, or equally well as, the state-of-the-art models in detection and classification of cardiac uptake. Both models achieved area under the curve (AUC) ≥ 0.85 in the four-class Perugini grade classification. Accuracy was good in detection of negative vs. positive ATTR patients (grade < 2 vs grade ≥ 2, AUC > 0.88) and high-grade cardiac uptake vs. other patients (grade < 3 vs. grade 3, AUC = 0.94). Maximum activation maps demonstrated that the automated deep learning models were focused on detecting the myocardium and not extracardiac features.
Conclusion
Automated convolutional neural networks can accurately detect and classify different grades of cardiac uptake on bone scintigraphy. The CNN models are focused on clinically relevant image features. Automated screening of bone scintigraphy images using CNN could improve the early diagnosis of ATTR.
Rationale: Artificial intelligence may improve accuracy of myocardial perfusion imaging (MPI) but will likely be implemented as an aid to physician interpretation rather than an autonomous tool. Deep learning (DL) has high standalone diagnostic accuracy for obstructive coronary artery disease (CAD), but its influence on physician interpretation is unknown. We assessed whether access to explainable DL predictions improves physician interpretation of MPI. Methods: We selected a representative cohort of patients who underwent MPI with reference invasive coronary angiography. Obstructive CAD, defined as stenosis ≥ 50% in the left main artery or ≥70% in other coronary segments, was present in half of patients. We utilized an explainable DL model (CAD-DL), which was previously developed in a separate population from different sites. Three physicians interpreted studies first with clinical history, stress, and quantitative perfusion, then with all the data plus the DL results. Diagnostic accuracy was assessed using area under the receiver-operating characteristic curve (AUC). Results: In total, 240 patients were included with median age 65 (IQR 58 - 73). The diagnostic accuracy of physician interpretation with CAD-DL (AUC 0.779) was significantly higher compared to physician interpretation without CAD-DL (AUC 0.747, P = 0.003) and stress total perfusion deficit (AUC 0.718, p<0.001). With matched specificity, CAD-DL had higher sensitivity when operating autonomously compared to readers without DL results (p<0.001), but not compared to readers interpreting with DL results (P = 0.122). All readers had numerically higher accuracy with the use of CAD-DL, with AUC improvement 0.02 to 0.05, and interpretation with DL resulted in overall net reclassification improvement of 17.5% (95% CI 9.8% - 24.7%, p<0.001). Conclusion: Explainable DL predictions lead to meaningful improvements in physician interpretation; however, the improvement varied across the readers reflecting the acceptance of this new technology. This technique could be implemented as an aid to physician diagnosis, improving the diagnostic accuracy of MPI.
Purpose
Myocardial perfusion imaging (MPI) using single-photon emission computed tomography (SPECT) is widely used for coronary artery disease (CAD) evaluation. Although attenuation correction is recommended to diminish image artifacts and improve diagnostic accuracy, approximately 3/4ths of clinical MPI worldwide remains non-attenuation-corrected (NAC). In this work, we propose a novel deep learning (DL) algorithm to provide “virtual” DL attenuation–corrected (DLAC) perfusion polar maps solely from NAC data without concurrent computed tomography (CT) imaging or additional scans.
Methods
SPECT MPI studies (N = 11,532) with paired NAC and CTAC images were retrospectively identified. A convolutional neural network–based DL algorithm was developed and trained on half of the population to predict DLAC polar maps from NAC polar maps. Total perfusion deficit (TPD) was evaluated for all polar maps. TPDs from NAC and DLAC polar maps were compared to CTAC TPDs in linear regression analysis. Moreover, receiver-operating characteristic analysis was performed on NAC, CTAC, and DLAC TPDs to predict obstructive CAD as diagnosed from invasive coronary angiography.
Results
DLAC TPDs exhibited significantly improved linear correlation (p < 0.001) with CTAC (R² = 0.85) compared to NAC vs. CTAC (R² = 0.68). The diagnostic performance of TPD was also improved with DLAC compared to NAC with an area under the curve (AUC) of 0.827 vs. 0.780 (p = 0.012) with no statistically significant difference between AUC for CTAC and DLAC. At 88% sensitivity, specificity was improved by 18.9% for DLAC and 25.6% for CTAC.
Conclusions
The proposed DL algorithm provided attenuation correction comparable to CTAC without the need for additional scans. Compared to conventional NAC perfusion imaging, DLAC significantly improved diagnostic accuracy.
Objectives:
This study sought to develop and evaluate a novel, general purpose, explainable deep learning model (coronary artery disease-deep learning [CAD-DL]) for the detection of obstructive CAD following single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI).
Background:
Explainable artificial intelligence (AI) can be integrated within standard clinical software to facilitate the acceptance of the diagnostic findings during clinical interpretation.
Methods:
A total of 3,578 patients with suspected CAD undergoing SPECT MPI and invasive coronary angiography within a 6 month interval from 9 centers were studied. CAD-DL computes the probability of obstructive CAD from stress myocardial perfusion, wall motion, and wall thickening maps, as well as left ventricular volumes, age, and sex. Myocardial regions contributing to the CAD-DL prediction are highlighted to explain the findings to the physician. A clinical prototype was integrated using a standard clinical workstation. Diagnostic performance by CAD-DL was compared to automated quantitative total perfusion deficit (TPD) and reader diagnosis.
Results:
In total, 2,247 patients (63%) had obstructive CAD. In 10-fold repeated testing, the area under the receiver-operating characteristic curve (AUC) (95% CI) was higher according to CAD-DL (AUC: 0.83; 95% CI: 0.82-0.85]) than stress TPD (AUC: 0.78; 95% CI: 0.77-0.80]) or reader diagnosis (AUC: 0.71; 95% CI: 0.69-0.72]; P < 0.0001 for both). In external testing, the AUC in 555 patients was higher according to CAD-DL (AUC: 0.80; 95% CI: 0.76-0.84]) than stress TPD (AUC: 0.73; 95% CI: 0.69-0.77) or reader diagnosis (AUC: 0.65; 95% CI: 0.61-0.69; P < 0.001). The present model can be integrated within standard clinical software and generates results rapidly (<12 s on a standard clinical workstation) and therefore could readily be incorporated into a typical clinical workflow.
Conclusions:
The deep-learning model significantly surpasses the diagnostic accuracy of standard quantitative analysis and clinical visual reading for MPI. Explainable artificial intelligence can be integrated within standard clinical software to facilitate acceptance of artificial intelligence diagnosis of CAD following MPI.
Aims: Computed tomographic attenuation correction (CTAC) scans for single photon emission computed tomography myocardial perfusion imaging (SPECT-MPI) may reveal coronary artery calcification. The independent prognostic value of a visually estimated coronary artery calcium score (VECACS) from these low-dose, non-gated scans is not established.
Methods & Results: VECACS was evaluated in 4,720 patients undergoing SPECT-MPI with CTAC using a 4-point scale. Major adverse cardiac events (MACE) were defined as all-cause mortality, acute coronary syndrome , or revascularization > 90 days after SPECT-MPI. Independent associations with MACE were determined with multivariable Cox proportional hazards analyses adjusted for age, sex, past medical history , perfusion findings, and left ventricular ejection fraction. During a median follow up of 2.9 years (interquartile range 1.8-4.2), 494 (10.5%) patients experienced MACE. Compared to absent VECACS, patients with increased VECACS were more likely to experience MACE (all log-rank p < 0.001), and findings were similar when stratified by normal or abnormal perfusion. Multivariable analysis showed an increased MACE risk associated with VECACS categories of equivocal (adjusted hazard ratio [HR] 2.54, 95% CI 1.45-4.45, p = 0.001), present (adjusted HR 2.44, 95% CI 1.74-3.42, p < 0.001) and extensive (adjusted HR 3.47, 95% CI 2.41-5.00, p < 0.001) compared to absent. Addition of VECACS to the multivariable model improved risk classification (continuous net reclassification index 0.207, 95% CI 0.131-0.310).
Conclusion: VECACS was an independent predictor of MACE in this large SPECT-MPI patient cohort. VECACS from CTAC can be used to improve risk stratification with SPECT-MPI without additional radiation.
Background
Attenuation correction (AC) using CT transmission scanning enables the accurate quantitative analysis of dedicated cardiac SPECT. However, AC is challenging for SPECT-only scanners. We developed a deep learning-based approach to generate synthetic AC images from SPECT images without AC.MethodsCT-free AC was implemented using our customized Dual Squeeze-and-Excitation Residual Dense Network (DuRDN). 172 anonymized clinical hybrid SPECT/CT stress/rest myocardial perfusion studies were used in training, validation, and testing. Additional body mass index (BMI), gender, and scatter-window information were encoded as channel-wise input to further improve the network performance.ResultsQuantitative and qualitative analysis based on image voxels and 17-segment polar map showed the potential of our approach to generate consistent SPECT AC images. Our customized DuRDN showed superior performance to conventional network design such as U-Net. The averaged voxel-wise normalized mean square error (NMSE) between the predicted AC images by DuRDN and the ground-truth AC images was 2.01 ± 1.01%, as compared to 2.23 ± 1.20% by U-Net.Conclusions
Our customized DuRDN facilitates dedicated cardiac SPECT AC without CT scanning. DuRDN can efficiently incorporate additional patient information and may achieve better performance compared to conventional U-Net.
Patient motion during dynamic PET imaging can induce errors in myocardial blood flow (MBF) estimation. Motion correction for dynamic cardiac PET is challenging because the rapid tracer kinetics of 82Rb leads to substantial tracer distribution change across different dynamic frames over time, which can cause difficulties for image registration-based motion correction, particularly for early dynamic frames. In this paper, we developed an automatic deep learning-based motion correction (DeepMC) method for dynamic cardiac PET. In this study we focused on the detection and correction of inter-frame rigid translational motion caused by voluntary body movement and pattern change of respiratory motion. A bidirectional-3D LSTM network was developed to fully utilize both local and nonlocal temporal information in the 4D dynamic image data for motion detection. The network was trained and evaluated over motion-free patient scans with simulated motion so that the motion ground-truths are available, where one million samples based on 65 patient scans were used in training, and 600 samples based on 20 patient scans were used in evaluation. The proposed method was also evaluated using additional 10 patient datasets with real motion. We demonstrated that the proposed DeepMC obtained superior performance compared to conventional registration-based methods and other convolutional neural networks (CNN), in terms of motion estimation and MBF quantification accuracy. Once trained, DeepMC is much faster than the registration-based methods and can be easily integrated into the clinical workflow. In the future work, additional investigation is needed to evaluate this approach in a clinical context with realistic patient motion.
Coronary artery calcium is an accurate predictor of cardiovascular events. While it is visible on all computed tomography (CT) scans of the chest, this information is not routinely quantified as it requires expertise, time, and specialized equipment. Here, we show a robust and time-efficient deep learning system to automatically quantify coronary calcium on routine cardiac-gated and non-gated CT. As we evaluate in 20,084 individuals from distinct asymptomatic (Framingham Heart Study, NLST) and stable and acute chest pain (PROMISE, ROMICAT-II) cohorts, the automated score is a strong predictor of cardiovascular events, independent of risk factors (multivariable-adjusted hazard ratios up to 4.3), shows high correlation with manual quantification, and robust test-retest reliability. Our results demonstrate the clinical value of a deep learning system for the automated prediction of cardiovascular events. Implementation into clinical practice would address the unmet need of automating proven imaging biomarkers to guide management and improve population health.
Dynamic positron emission tomography (PET) imaging can provide information about metabolic changes over time, used for kinetic analysis and auxiliary diagnosis. Existing deep learning-based reconstruction methods have too many trainable parameters and poor generalization, and require mass data to train the neural network. However, obtaining large amounts of medical data is expensive and time-consuming. To reduce the need for data and improve the generalization of network, we combined the filtered back-projection (FBP) algorithm with neural network, and proposed FBP-Net which could directly reconstruct PET images from sinograms instead of post-processing the rough reconstruction images obtained by traditional methods. The FBP-Net contained two parts: the filtered back-projection (FBP) part and the denoiser part. The FBP part adaptively learned the frequency filter to realize the transformation from the detector domain to the image domain, and normalized the coarse reconstruction images obtained. The denoiser part merged the information of all time frames to improve the quality of dynamic PET reconstruction images, especially the early time frames. The proposed FBP-Net was performed on simulation and real dataset, and the results were compared with the state-of-art U-net and DeepPET. The results showed that FBP-Net did not tend to overfit the training set and had a stronger generalization.
Artifacts caused by patient breathing and movement during PET data acquisition affect image quality. Respiratory gating is commonly used to gate the list-mode PET data into multiple bins over a respiratory cycle. Non-rigid registration of respiratory-gated PET images can reduce motion artifacts and preserve count statistics, but it is time consuming. In this work, we propose an unsupervised non-rigid image registration framework using deep learning for motion correction. Our network uses a differentiable spatial transformer layer to warp the moving image to the fixed image and uses a stacked structure for deformation field refinement. Estimated deformation fields were incorporated into an iterative image reconstruction algorithm to perform motion compensated PET image reconstruction. We validated the proposed method using simulation and clinical data and implemented an iterative image registration approach for comparison. Motion compensated reconstructions were compared with ungated images. Our simulation study showed that the motion compensated methods can generate images with sharp boundaries and reveal more details in the heart region compared with the ungated image. The resulting normalized root mean square error (NRMS) was 24.3 ± 1.7% for the deep learning based motion correction, 31.1 ± 1.4% for the iterative registration based motion correction, and 41.9 ± 2.0% for ungated reconstruction. The proposed deep learning based motion correction reduced the bias compared with the ungated image without increasing the noise level and outperformed the iterative registration based method. In the real data study, both motion compensated images provided higher lesion contrast and sharper liver boundaries than the ungated image and had lower noise than the reference gate image. The contrast of the proposed method based on the deep neural network was higher than the ungated image and iterative registration method at any matched noise level.
PurposeAttenuation correction using CT transmission scanning increases the accuracy of single-photon emission computed tomography (SPECT) and enables quantitative analysis. Current existing SPECT-only systems normally do not support transmission scanning and therefore scans on these systems are susceptible to attenuation artifacts. Moreover, the use of CT scans also increases radiation dose to patients and significant artifacts can occur due to the misregistration between the SPECT and CT scans as a result of patient motion. The purpose of this study is to develop an approach to estimate attenuation maps directly from SPECT emission data using deep learning methods.Methods
Both photopeak window and scatter window SPECT images were used as inputs to better utilize the underlying attenuation information embedded in the emission data. The CT-based attenuation maps were used as labels with which cardiac SPECT/CT images of 65 patients were included for training and testing. We implemented and evaluated deep fully convolutional neural networks using both standard training and training using an adversarial strategy.ResultsThe synthetic attenuation maps were qualitatively and quantitatively consistent with the CT-based attenuation map. The globally normalized mean absolute error (NMAE) between the synthetic and CT-based attenuation maps were 3.60% ± 0.85% among the 25 testing subjects. The SPECT reconstructed images corrected using the CT-based attenuation map and synthetic attenuation map are highly consistent. The NMAE between the reconstructed SPECT images that were corrected using the synthetic and CT-based attenuation maps was 0.26% ± 0.15%, whereas the localized absolute percentage error was 1.33% ± 3.80% in the left ventricle (LV) myocardium and 1.07% ± 2.58% in the LV blood pool.Conclusion
We developed a deep convolutional neural network to estimate attenuation maps for SPECT directly from the emission data. The proposed method is capable of generating highly reliable attenuation maps to facilitate attenuation correction for SPECT-only scanners for myocardial perfusion imaging.
Background:
Epicardial adipose tissue (EAT) volume (cm3) and attenuation (Hounsfield units) may predict major adverse cardiovascular events (MACE). We aimed to evaluate the prognostic value of fully automated deep learning-based EAT volume and attenuation measurements quantified from noncontrast cardiac computed tomography.
Methods:
Our study included 2068 asymptomatic subjects (56±9 years, 59% male) from the EISNER trial (Early Identification of Subclinical Atherosclerosis by Noninvasive Imaging Research) with long-term follow-up after coronary artery calcium measurement. EAT volume and mean attenuation were quantified using automated deep learning software from noncontrast cardiac computed tomography. MACE was defined as myocardial infarction, late (>180 days) revascularization, and cardiac death. EAT measures were compared to coronary artery calcium score and atherosclerotic cardiovascular disease risk score for MACE prediction.
Results:
At 14±3 years, 223 subjects suffered MACE. Increased EAT volume and decreased EAT attenuation were both independently associated with MACE. Atherosclerotic cardiovascular disease risk score, coronary artery calcium, and EAT volume were associated with increased risk of MACE (hazard ratio [95%CI]: 1.03 [1.01-1.04]; 1.25 [1.19-1.30]; and 1.35 [1.07-1.68], P<0.01 for all) and EAT attenuation was inversely associated with MACE (hazard ratio, 0.83 [95% CI, 0.72-0.96]; P=0.01), with corresponding Harrell C statistic of 0.76. MACE risk progressively increased with EAT volume ≥113 cm3 and coronary artery calcium ≥100 AU and was highest in subjects with both (P<0.02 for all). In 1317 subjects, EAT volume was correlated with inflammatory biomarkers C-reactive protein, myeloperoxidase, and adiponectin reduction; EAT attenuation was inversely related to these biomarkers.
Conclusions:
Fully automated EAT volume and attenuation quantification by deep learning from noncontrast cardiac computed tomography can provide prognostic value for the asymptomatic patient, without additional imaging or physician interaction.
We propose a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach—Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say ‘dog’ in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g.VGG), (2) CNNs used for structured outputs (e.g.captioning), (3) CNNs used in tasks with multi-modal inputs (e.g.visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are robust to adversarial perturbations, (d) are more faithful to the underlying model, and (e) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show that even non-attention based models learn to localize discriminative regions of input image. We devise a way to identify important neurons through Grad-CAM and combine it with neuron names (Bau et al. in Computer vision and pattern recognition, 2017) to provide textual explanations for model decisions. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a ‘stronger’ deep network from a ‘weaker’ one even when both make identical predictions. Our code is available at https://github.com/ramprs/grad-cam/, along with a demo on CloudCV (Agrawal et al., in: Mobile cloud visual media computing, pp 265–290. Springer, 2015) (http://gradcam.cloudcv.org) and a video at http://youtu.be/COjUB9Izk6E.
Myocardial perfusion imaging is a non-invasive imaging technique commonly used for the diagnosis of Coronary Artery Disease and is based on the injection of radiopharmaceutical tracers into the blood stream. The patient’s heart is imaged while at rest and under stress in order to determine its capacity to react to the imposed challenge. Assessment of imaging data is commonly performed by visual inspection of polar maps showing the tracer uptake in a compact, two-dimensional representation of the left ventricle. This article presents a method for automatic classification of polar maps based on graph convolutional neural networks. Furthermore, it evaluates how well localization techniques developed for standard convolutional neural networks can be used for the localization of pathological segments with respect to clinically relevant areas. The method is evaluated using 946 labeled datasets and compared quantitatively to three other neural-network-based methods. The proposed model achieves an agreement with the human observer on 89.3% of rest test polar maps and on 91.1% of stress test polar maps. Localization performed on a fine 17-segment division of the polar maps achieves an agreement of 83.1% with the human observer, while localization on a coarse 3-segment division based on the vessel beds of the left ventricle has an agreement of 78.8% with the human observer. Our method could thus assist the decision-making process of physicians when analyzing polar map data obtained from myocardial perfusion images.
What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI.
Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.
Objectives:
This study evaluated the added predictive value of combining clinical information and myocardial perfusion single-photon emission computed tomography (SPECT) imaging (MPI) data using machine learning (ML) to predict major adverse cardiac events (MACE).
Background:
Traditionally, prognostication by MPI has relied on visual or quantitative analysis of images without objective consideration of the clinical data. ML permits a large number of variables to be considered in combination and at a level of complexity beyond the human clinical reader.
Methods:
A total of 2,619 consecutive patients (48% men; 62 ± 13 years of age) who underwent exercise (38%) or pharmacological stress (62%) with high-speed SPECT MPI were monitored for MACE. Twenty-eight clinical variables, 17 stress test variables, and 25 imaging variables (including total perfusion deficit [TPD]) were recorded. Areas under the receiver-operating characteristic curve (AUC) for MACE prediction were compared among: 1) ML with all available data (ML-combined); 2) ML with only imaging data (ML-imaging); 3) 5-point scale visual diagnosis (physician [MD] diagnosis); and 4) automated quantitative imaging analysis (stress TPD and ischemic TPD). ML involved automated variable selection by information gain ranking, model building with a boosted ensemble algorithm, and 10-fold stratified cross validation.
Results:
During follow-up (3.2 ± 0.6 years), 239 patients (9.1%) had MACE. MACE prediction was significantly higher for ML-combined than ML-imaging (AUC: 0.81 vs. 0.78; p < 0.01). ML-combined also had higher predictive accuracy compared with MD diagnosis, automated stress TPD, and automated ischemic TPD (AUC: 0.81 vs. 0.65 vs. 0.73 vs. 0.71, respectively; p < 0.01 for all). Risk reclassification for ML-combined compared with visual MD diagnosis was 26% (p < 0.001).
Conclusions:
ML combined with both clinical and imaging data variables was found to have high predictive accuracy for 3-year risk of MACE and was superior to existing visual or automated perfusion assessments. ML could allow integration of clinical and imaging data for personalized MACE risk computations in patients undergoing SPECT MPI.
Background:
We investigated fully automatic coronary artery calcium (CAC) scoring and cardiovascular disease (CVD) risk categorization from CT attenuation correction (CTAC) acquired at rest and stress during cardiac PET/CT and compared it with manual annotations in CTAC and with dedicated calcium scoring CT (CSCT).
Methods and results:
We included 133 consecutive patients undergoing myocardial perfusion (82)Rb PET/CT with the acquisition of low-dose CTAC at rest and stress. Additionally, a dedicated CSCT was performed for all patients. Manual CAC annotations in CTAC and CSCT provided the reference standard. In CTAC, CAC was scored automatically using a previously developed machine learning algorithm. Patients were assigned to a CVD risk category based on their Agatston score (0, 1-10, 11-100, 101-400, >400). Agreement in CVD risk categorization between manual and automatic scoring in CTAC at rest and stress resulted in Cohen's linearly weighted κ of 0.85 and 0.89, respectively. The agreement between CSCT and CTAC at rest resulted in κ of 0.82 and 0.74, using manual and automatic scoring, respectively. For CTAC at stress, these were 0.79 and 0.70, respectively.
Conclusion:
Automatic CAC scoring from CTAC PET/CT may allow routine CVD risk assessment from the CTAC component of PET/CT without any additional radiation dose or scan time.
Access to big datasets from e-health records and individual participant data (IPD) meta-analysis is signalling a new advent of external validation studies for clinical prediction models. In this article, the authors illustrate novel opportunities for external validation in big, combined datasets, while drawing attention to methodological challenges and reporting issues.
Introduction: To improve diagnostic accuracy, myocardial perfusion imaging (MPI) SPECT studies can use CT-based attenuation correction (AC) (CTAC). However, CTAC is not available for most SPECT systems in clinical use, increases radiation exposure, and is impacted by misregistration. We developed and externally validated a deep-learning model to generate simulated AC images directly from non-attenuation corrected (NC) SPECT, without the need for CT. Methods: SPECT MPI was performed using Tc-99m sestamibi or Tc-99m tetrofosmin on contemporary scanners with solid-state detectors. We developed a conditional generative adversarial neural network that generates simulated AC SPECT images (DeepAC). The model was trained with short-axis NC and AC images performed in one site (n = 4886) and was tested in patients from two separate external sites (n = 604). We assessed diagnostic accuracy of stress total perfusion deficit (TPD) obtained from NC, AC, and DeepAC images for obstructive coronary artery disease (CAD) with area under the receiver operating characteristic curve (AUC). We also quantified direct count change between AC, NC, and DeepAC images on a per-voxel basis. Results: DeepAC could be obtained in <1 second from NC images, AUC for obstructive CAD was higher for DeepAC TPD (0.79, 95% CI 0.72 - 0.85) compared to NC TPD (0.70, 95% Confidence Intervals (CI) 0.63 - 0.78, p<0.001), and similar to AC TPD (0.81, 95% CI 0.75 - 0.87, P = 0.196). The normalcy rate (defined as stress TPD <3%) in the LLK population was higher for DeepAC TPD (70.4%) and AC TPD (75.0%) compared to NC TPD (54.6%, p<0.001 for both). Positive count change (increase in counts) was significantly higher for AC vs NC (median 9.4, Inter Quartile Range (IQR) 6.0 - 14.2, p<0.001) than for AC vs DeepAC (median 2.4, interquartile range [IQR] 1.3 - 4.2). Conclusion: In an independent external dataset, DeepAC provides improved diagnostic accuracy for obstructive CAD similar to actual AC, as compared to NC images. DeepAC simplifies the task of artifact identification for physicians, avoids misregistration artifacts, and can be performed rapidly without the need for CT hardware and additional acquisitions.
Subject motion in whole-body dynamic PET introduces inter-frame mismatch and seriously impacts parametric imaging. Traditional non-rigid registration methods are generally computationally intense and time-consuming. Deep learning approaches are promising in achieving high accuracy with fast speed, but have yet been investigated with consideration for tracer distribution changes or in the whole-body scope. In this work, we developed an unsupervised automatic deep learning-based framework to correct inter-frame body motion. The motion estimation network is a convolutional neural network with a combined convolutional long short-term memory layer, fully utilizing dynamic temporal features and spatial information. Our dataset contains 27 subjects each under a 90-min FDG whole-body dynamic PET scan. Evaluating performance in motion simulation studies and a 9-fold cross-validation on the human subject dataset, compared with both traditional and deep learning baselines, we demonstrated that the proposed network achieved the lowest motion prediction error, obtained superior performance in enhanced qualitative and quantitative spatial alignment between parametric Ki and Vb images, and significantly reduced parametric fitting error. We also showed the potential of the proposed motion correction method for impacting downstream analysis of the estimated parametric images, improving the ability to distinguish malignant from benign hypermetabolic regions of interest. Once trained, the motion estimation inference time of our proposed network was around 460 times faster than the conventional registration baseline, showing its potential to be easily applied in clinical settings.
Background:
Accurately predicting which patients will have abnormal perfusion on MPI based on pre-test clinical information may help physicians make test selection decisions. We developed and validated a machine learning (ML) model for predicting abnormal perfusion using pre-test features.
Methods:
We included consecutive patients who underwent SPECT MPI, with 20,418 patients from a multi-center (5 sites) international registry in the training population and 9019 patients (from 2 separate sites) in the external testing population. The ML (extreme gradient boosting) model utilized 30 pre-test features to predict the presence of abnormal myocardial perfusion by expert visual interpretation.
Results:
In external testing, the ML model had higher prediction performance for abnormal perfusion (area under receiver-operating characteristic curve [AUC] 0.762, 95% CI 0.750-0.774) compared to the clinical CAD consortium (AUC 0.689) basic CAD consortium (AUC 0.657), and updated Diamond-Forrester models (AUC 0.658, p < 0.001 for all). Calibration (validation of the continuous risk prediction) was superior for the ML model (Brier score 0.149) compared to the other models (Brier score 0.165 to 0.198, all p < 0.001).
Conclusion:
ML can predict abnormal myocardial perfusion using readily available pre-test information. This model could be used to help guide physician decisions regarding non-invasive test selection.
Background:
The likelihood of ischemia on myocardial perfusion imaging is central to physician decisions regarding test selection, but dedicated risk scores are lacking. We derived and validated two novel ischemia risk scores to support physician decision making.
Methods:
Risk scores were derived using 15,186 patients and validated with 2,995 patients from a different center. Logistic regression was used to assess associations with ischemia to derive point-based and calculated ischemia scores. Predictive performance for ischemia was assessed using area under the receiver operating characteristic curve (AUC) and compared with the CAD consortium basic and clinical models.
Results:
During derivation, the calculated ischemia risk score (0.801) had higher AUC compared to the point-based score (0.786, p < 0.001). During validation, the calculated ischemia score (0.716, 95% CI 0.684- 0.748) had higher AUC compared to the point-based ischemia score (0.699, 95% CI 0.666- 0.732, p = 0.016) and the clinical CAD model (AUC 0.667, 95% CI 0.633- 0.701, p = 0.002). Calibration for both ischemia scores was good in both populations (Brier score < 0.100).
Conclusions:
We developed two novel risk scores for predicting probability of ischemia on MPI which demonstrated high accuracy during model derivation and in external testing. These scores could support physician decisions regarding diagnostic testing strategies.
Background
Outcome prediction following heart transplant is critical to explaining risks and benefits to patients and decision-making when considering potential organ offers. Given the large number of potential variables to be considered, this task may be most efficiently performed using machine learning (ML). We trained and tested ML and statistical algorithms to predict outcomes following cardiac transplant using the United Network of Organ Sharing (UNOS) database.
Methods
We included 59,590 adult and 8,349 pediatric patients enrolled in the UNOS database between January 1994 and December 2016 who underwent cardiac transplantation. We evaluated three classification and three survival methods. Algorithms were evaluated using shuffled 10-fold cross-validation (CV) and rolling CV. Predictive performance for one-year and 90 days all-cause mortality was characterized using the area under the receiver-operating characteristic curve (AUC) with 95% confidence interval.
Results
In total, 8,394 (12.4%) patients died within 1 year of transplant. For predicting 1-year survival, using the shuffled 10-fold CV, Random Forest achieved the highest AUC (0.893; 0.889–0.897) followed by XGBoost and logistic regression. In the rolling CV, prediction performance was more modest and comparable among the models with XGBoost and Logistic regression achieving the highest AUC 0.657 (0.647–0.667) and 0.641(0.631–0.651), respectively. There was a trend towards higher prediction performance in pediatric patients.
Conclusions
Our study suggests that ML and statistical models can be used to predict mortality post-transplant, but based on the results from rolling CV, the overall prediction performance will be limited by temporal shifts in patient and donor selection.
Background
Machine learning (ML) models can improve prediction of major adverse cardiovascular events (MACE), but in clinical practice some variables may be missing. We evaluated the influence of missing values in ML models for patient-specific prediction of MACE risk.
Methods
We included 20,179 patients from the multicenter REFINE SPECT registry with MACE follow-up data. We evaluated seven methods for handling missing values: 1) removal of variables with missing values (ML-Remove), 2) imputation with median and unique category for continuous and categorical variables, respectively (ML-Traditional), 3) unique category for missing variables (ML-Unique), 4) cluster-based imputation (ML-Cluster), 5) regression-based imputation (ML-Regression), 6) Miss-Ranger imputation (ML-MR), and 7) multiple imputation (ML-MICE). We trained ML models with full data and simulated missing values in testing patients. Prediction performance was evaluated using area under the receiver-operating characteristic curve (AUC) and compared with a model without missing values (ML-All), expert visual diagnosis and total perfusion deficit (TPD).
Results
During mean follow-up of 4.7 ± 1.5 years, 3,541 patients experienced at least one MACE (3.7% annualized risk). ML-All (reference model-no missing values) had AUC 0.799 for MACE risk prediction. All seven models with missing values had lower AUC (ML-Remove: 0.778, ML-MICE: 0.774, ML-Cluster: 0.771, ML-Traditional: 0.771, ML-Regression: 0.770, ML-MR: 0.766, and ML-Unique: 0.766; p < 0.01 for ML-Remove vs remaining methods). Stress TPD (AUC 0.698) and visual diagnosis (0.681) had the lowest AUCs.
Conclusion
Missing values reduce the accuracy of ML models when predicting MACE risk. Removing variables with missing values and retraining the model may yield superior patient-level prediction performance.
Aims
Optimal risk stratification with machine learning (ML) from myocardial perfusion imaging (MPI) includes both clinical and imaging data. While most imaging variables can be derived automatically, clinical variables require manual collection, which is time consuming and prone to error. We determined the fewest manually input and imaging variables required to maintain the prognostic accuracy for major adverse cardiac events (MACE) in patients undergoing single-photon emission computed tomography (SPECT) MPI.
Methods and Results
This study included 20,414 patients from the multicenter REFINE SPECT registry and 2,984 from the University of Calgary for training and external testing of the ML models, respectively. ML models were trained using all variables (ML-All) and all image-derived variables (including age and sex, ML-Image). Next, ML models were sequentially trained by incrementally adding manually input and imaging variables to baseline ML models based on their importance ranking. The fewest variables were determined as the ML models (ML-Reduced, ML-Minimum, and ML-Image-Reduced) that achieved comparable prognostic performance to ML-All and ML-Image. Prognostic accuracy of the ML models was compared with visual diagnosis, stress total perfusion deficit (TPD), and traditional multivariable models using area under the receiver-operating characteristic curve (AUC).
ML-Minimum (AUC 0.798) obtained comparable prognostic accuracy to ML-All (AUC 0.798, p = 0.18) by including 12 of 40 manually input variables and 11 of 58 imaging variables. ML-Reduced achieved comparable accuracy (AUC 0.795) with a reduced set of manually input variables and all imaging variables. In external validation, the ML models also obtained comparable or higher prognostic accuracy than traditional multivariable models.
Conclusion
Reduced ML models, including a minimum set of manually collected or imaging variables, achieved slightly lower accuracy compared to a full ML model, but outperformed standard interpretation methods and risk models. ML models with fewer collected variables may be more practical for clinical implementation.
Translational Perspective
A reduced machine learning model, with 12 out of 40 manually collected variables and 11 of 58 imaging variables, achieved >99% of the prognostic accuracy of the full model. Models with fewer manually collected features require less infrastructure to implement, are easier for physicians to utilize, and are potentially critical to ensuring broader clinical implementation. Additionally, these models can integrate mechanisms to explain patient-specific risk estimates to improve physician confidence in the machine learning prediction.
Background:
Stress-only myocardial perfusion imaging (MPI) markedly reduces radiation dose, scanning time, and cost. We developed an automated clinical algorithm to safely cancel unnecessary rest imaging with high sensitivity for obstructive coronary artery disease (CAD).
Methods and results:
Patients without known CAD undergoing both MPI and invasive coronary angiography from REFINE SPECT were studied. A machine learning score (MLS) for prediction of obstructive CAD was generated using stress-only MPI and pre-test clinical variables. An MLS threshold with a pre-defined sensitivity of 95% was applied to the automated patient selection algorithm. Obstructive CAD was present in 1309/2079 (63%) patients. MLS had higher area under the receiver operator characteristic curve (AUC) for prediction of CAD than reader diagnosis and TPD (0.84 vs 0.70 vs 0.78, P < .01). An MLS threshold of 0.29 had superior sensitivity than reader diagnosis and TPD for obstructive CAD (95% vs 87% vs 87%, P < .01) and high-risk CAD, defined as stenosis of the left main, proximal left anterior descending, or triple-vessel CAD (sensitivity 96% vs 89% vs 90%, P < .01).
Conclusions:
The MLS is highly sensitive for prediction of both obstructive and high-risk CAD from stress-only MPI and can be applied to a stress-first protocol for automatic cancellation of unnecessary rest imaging.
Aims:
Single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI) stress-only protocols reduce radiation exposure and cost but require clinicians to make immediate decisions regarding rest scan cancellation. We developed a machine learning (ML) approach for automatic rest scan cancellation and evaluated its prognostic safety.
Methods and results :
In total, 20 414 patients from a solid-state SPECT MPI international multicentre registry with clinical data and follow-up for major adverse cardiac events (MACE) were used to train ML for MACE prediction as a continuous probability (ML score), using 10-fold repeated hold-out testing to separate test from training data. Three ML score thresholds (ML1, ML2, and ML3) were derived by matching the cancellation rates achieved by physician interpretation and two clinical selection rules. Annual MACE rates were compared in patients selected for rest scan cancellation between approaches. Patients selected for rest scan cancellation with ML had lower annualized MACE rates than those selected by physician interpretation or clinical selection rules (ML1 vs. physician interpretation: 1.4 ± 0.1% vs. 2.1 ± 0.1%; ML2 vs. clinical selection: 1.5 ± 0.1% vs. 2.0 ± 0.1%; ML3 vs. stringent clinical selection: 0.6 ± 0.1% vs. 1.7 ± 0.1%, all P < 0.0001) at matched cancellation rates (60 ± 0.7, 64 ± 0.7, and 30 ± 0.6%). Annualized all-cause mortality rates in populations recommended for rest cancellation by physician interpretation, clinical selection approaches were higher (1.3%, 1.2%, and 1.0%, respectively) compared with corresponding ML thresholds (0.6%, 0.6%, and 0.2%).
Conclusion:
ML, using clinical and stress imaging data, can be used to automatically recommend cancellation of rest SPECT MPI scans, while ensuring higher prognostic safety than current clinical approaches.
Introduction:
The purpose of this work was to assess the feasibility of acquisition time reduction in MPI-SPECT imaging using deep leering techniques through two main approaches, namely reduction of the acquisition time per projection and reduction of the number of angular projections.
Methods:
SPECT imaging was performed using a fixed 90° angle dedicated dual-head cardiac SPECT camera. This study included a prospective cohort of 363 patients with various clinical indications (normal, ischemia, and infarct) referred for MPI-SPECT. For each patient, 32 projections for 20 seconds per projection were acquired using a step and shoot protocol from the right anterior oblique to the left posterior oblique view. SPECT projection data were reconstructed using the OSEM algorithm (6 iterations, 4 subsets, Butterworth post-reconstruction filter). For each patient, four different datasets were generated, namely full time (20 seconds) projections (FT), half-time (10 seconds) acquisition per projection (HT), 32 full projections (FP), and 16 half projections (HP). The image-to-image transformation via the residual network was implemented to predict FT from HT and predict FP from HP images in the projection domain. Qualitative and quantitative evaluations of the proposed framework was performed using a tenfold cross validation scheme using the root mean square error (RMSE), absolute relative error (ARE), structural similarity index, peak signal-to-noise ratio (PSNR) metrics, and clinical quantitative parameters.
Results:
The results demonstrated that the predicted FT had better image quality than the predicted FP images. Among the generated images, predicted FT images resulted in the lowest error metrics (RMSE = 6.8 ± 2.7, ARE = 3.1 ± 1.1%) and highest similarity index and signal-to-noise ratio (SSIM = 0.97 ± 1.1, PSNR = 36.0 ± 1.4). The highest error metrics (RMSE = 32.8 ± 12.8, ARE = 16.2 ± 4.9%) and the lowest similarity and signal-to-noise ratio (SSIM = 0.93 ± 2.6, PSNR = 31.7 ± 2.9) were observed for HT images. The RMSE decreased significantly (P value < .05) for predicted FT (8.0 ± 3.6) relative to predicted FP (6.8 ± 2.7).
Conclusion:
Reducing the acquisition time per projection significantly increased the error metrics. The deep neural network effectively recovers image quality and reduces bias in quantification metrics. Further research should be undertaken to explore the impact of time reduction in gated MPI-SPECT.
Lowering the administered dose in SPECT myocardial perfusion imaging (MPI) has become an important clinical problem. In this study we investigate the potential benefit of applying a deep learning (DL) approach for suppressing the elevated imaging noise in low-dose SPECT-MPI studies. We adopt a supervised learning approach to train a neural network by using image pairs obtained from full-dose (target) and low-dose (input) acquisitions of the same patients. In the experiments, we made use of acquisitions from 1,052 subjects and demonstrated the approach for two commonly used reconstruction methods in clinical SPECT-MPI: 1) filtered backprojection (FBP), and 2) ordered-subsets expectation-maximization (OSEM) with corrections for attenuation, scatter and resolution. We evaluated the DL output for the clinical task of perfusion-defect detection at a number of successively reduced dose levels (1/2, 1/4, 1/8, 1/16 of full dose). The results indicate that the proposed DL approach can achieve substantial noise reduction and lead to improvement in the diagnostic accuracy of low-dose data. In particular, at 1/2 dose, DL yielded an area-under-the-ROC-curve (AUC) of 0.799, which is nearly identical to the AUC = 0.801 obtained by OSEM at full-dose (
-value = 0.73); similar results were also obtained for FBP reconstruction. Moreover, even at 1/8 dose, DL achieved AUC = 0.770 for OSEM, which is above the AUC = 0.755 obtained at full-dose by FBP. These results indicate that, compared to conventional reconstruction filtering, DL denoising can allow for additional dose reduction without sacrificing the diagnostic accuracy in SPECT-MPI.
Aim:
To identify distinct phenotypic subgroups in a highly-dimensional, mixed-data cohort of individuals with heart failure (HF) with preserved ejection fraction (HFpEF) using unsupervised clustering analysis.
Methods and results:
The study included all Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist (TOPCAT) participants from the Americas (n = 1767). In the subset of participants with available echocardiographic data (derivation cohort, n = 654), we characterized three mutually exclusive phenogroups of HFpEF participants using penalized finite mixture model-based clustering analysis on 61 mixed-data phenotypic variables. Phenogroup 1 had higher burden of co-morbidities, natriuretic peptides, and abnormalities in left ventricular structure and function; phenogroup 2 had lower prevalence of cardiovascular and non-cardiac co-morbidities but higher burden of diastolic dysfunction; and phenogroup 3 had lower natriuretic peptide levels, intermediate co-morbidity burden, and the most favourable diastolic function profile. In adjusted Cox models, participants in phenogroup 1 (vs. phenogroup 3) had significantly higher risk for all adverse clinical events including the primary composite endpoint, all-cause mortality, and HF hospitalization. Phenogroup 2 (vs. phenogroup 3) was significantly associated with higher risk of HF hospitalization but a lower risk of atherosclerotic event (myocardial infarction, stroke, or cardiovascular death), and comparable risk of mortality. Similar patterns of association were also observed in the non-echocardiographic TOPCAT cohort (internal validation cohort, n = 1113) and an external cohort of patients with HFpEF [Phosphodiesterase-5 Inhibition to Improve Clinical Status and Exercise Capacity in Heart Failure with Preserved Ejection Fraction (RELAX) trial cohort, n = 198], with the highest risk of adverse outcome noted in phenogroup 1 participants.
Conclusions:
Machine learning-based cluster analysis can identify phenogroups of patients with HFpEF with distinct clinical characteristics and long-term outcomes.
Aims:
To optimize per-vessel prediction of early coronary revascularization (ECR) within 90 days after fast single-photon emission computed tomography (SPECT) myocardial perfusion imaging (MPI) using machine learning (ML) and introduce a method for a patient-specific explanation of ML results in a clinical setting.
Methods and results:
A total of 1980 patients with suspected coronary artery disease (CAD) underwent stress/rest 99mTc-sestamibi/tetrofosmin MPI with new-generation SPECT scanners were included. All patients had invasive coronary angiography within 6 months after SPECT MPI. ML utilized 18 clinical, 9 stress test, and 28 imaging variables to predict per-vessel and per-patient ECR with 10-fold cross-validation. Area under the receiver operator characteristics curve (AUC) of ML was compared with standard quantitative analysis [total perfusion deficit (TPD)] and expert interpretation. ECR was performed in 958 patients (48%). Per-vessel, the AUC of ECR prediction by ML (AUC 0.79, 95% confidence interval (CI) [0.77, 0.80]) was higher than by regional stress TPD (0.71, [0.70, 0.73]), combined-view stress TPD (AUC 0.71, 95% CI [0.69, 0.72]), or ischaemic TPD (AUC 0.72, 95% CI [0.71, 0.74]), all P < 0.001. Per-patient, the AUC of ECR prediction by ML (AUC 0.81, 95% CI [0.79, 0.83]) was higher than that of stress TPD, combined-view TPD, and ischaemic TPD, all P < 0.001. ML also outperformed nuclear cardiologists' expert interpretation of MPI for the prediction of early revascularization performance. A method to explain ML prediction for an individual patient was also developed.
Conclusion:
In patients with suspected CAD, the prediction of ECR by ML outperformed automatic MPI quantitation by TPDs (per-vessel and per-patient) or nuclear cardiologists' expert interpretation (per-patient).
Reduction of radiation exposure in SPECT-myocardial perfusion imaging (MPI) is critically important. However, lowering radiation dose significantly degrades image quality, which impacts the diagnostic accuracy in the clinic. Recent deep learning developments have shown promising results for denoising of low-dose imaging data in the field of computed tomography (CT). However, to the best of our knowledge, there are no studies implementing deep learning structures for denoising of low-dose SPECT-MPI images. This paper reports a deep learning method to predict standard-dose images from low-dose clinical SPECT-MPI images inspired by the recent work on low-dose CT. The proposed method is a 3D convolutional neural network based on stacked convolutional autoencoders, which is trained using pairs of clinical standard-dose and low-dose volume patches containing sections of heart images. Preliminary results show that image quality improves significantly in the low-dose studies over conventional noise reduction methods, which suggests that further dose reduction could be achieved if the proposed method is used for post-processing of reconstructed images. By the time of the conference, we expect to quantify the extent to which dose reduction can be achieved using the proposed model.
Combined analysis of SPECT myocardial perfusion imaging (MPI) performed with a solid-state camera on patients in two positions (semi-upright, supine) is routinely used to mitigate attenuation artifacts. We evaluated the prediction of obstructive disease from combined analysis of semi-upright and supine stress MPI by deep learning (DL) as compared to standard combined total perfusion deficit (TPD). Methods: 1160 patients without known coronary artery disease (64% males) were studied. Patients underwent stress 99mTc-sestamibi MPI with new generation solid-state SPECT scanners in four different centers. All patients had on-site clinical reads and invasive coronary angiography correlations within six months of MPI. Obstructive disease was defined as ≥70% narrowing of the 3 major coronary arteries and ≥50% for the left main coronary artery. Images were quantified at Cedars-Sinai. The left ventricular myocardium was segmented using standard clinical nuclear cardiology software. The contour placement was verified by an experienced technologist. Combined stress TPD was computed using gender- and camera-specific normal limits. DL was trained using polar distributions of normalized radiotracer counts, hypoperfusion defects and hypoperfusion severities and was evaluated for prediction of obstructive disease in a novel leave-one-center-out cross-validation procedure equivalent to external validation. During the validation procedure, four DL models were trained using data from three centers and then evaluated on the one center left aside. Predictions for each center were merged to have an overall estimation of the multicenter performance. Results: 718 (62%) patients and 1272 of 3480 (37%) arteries had obstructive disease. The area under the receiver operating characteristics curve for prediction of disease on a per-patient and per-vessel basis by DL was higher than for combined TPD (per-patient: 0.81 vs 0.78, per-vessel: 0.77 vs 0.73, P<0.001). With the DL cutoff set to exhibit the same specificity as the standard cutoff for combined TPD, per-patient sensitivity improved from 61.8% (TPD) to 65.6% (DL) (P<0.05), and per-vessel sensitivity improved from 54.6% (TPD) to 59.1% (DL) (P<0.01). With threshold matched to specificity of normal clinical read (56.3%) DL had sensitivity 84.8% vs 82.6% for on-site clinical read (P = 0.3). Conclusion: Deep learning improves automatic interpretation of MPI as compared to current quantitative methods.
Objectives:
The study evaluated the automatic prediction of obstructive disease from myocardial perfusion imaging (MPI) by deep learning as compared with total perfusion deficit (TPD).
Background:
Deep convolutional neural networks trained with a large multicenter population may provide improved prediction of per-patient and per-vessel coronary artery disease from single-photon emission computed tomography MPI.
Methods:
A total of 1,638 patients (67% men) without known coronary artery disease, undergoing stress99mTc-sestamibi or tetrofosmin MPI with new generation solid-state scanners in 9 different sites, with invasive coronary angiography performed within 6 months of MPI, were studied. Obstructive disease was defined as ≥70% narrowing of coronary arteries (≥50% for left main artery). Left ventricular myocardium was segmented using clinical nuclear cardiology software and verified by an expert reader. Stress TPD was computed using sex- and camera-specific normal limits. Deep learning was trained using raw and quantitative polar maps and evaluated for prediction of obstructive stenosis in a stratified 10-fold cross-validation procedure.
Results:
A total of 1,018 (62%) patients and 1,797 of 4,914 (37%) arteries had obstructive disease. Area under the receiver-operating characteristic curve for disease prediction by deep learning was higher than for TPD (per patient: 0.80 vs. 0.78; per vessel: 0.76 vs. 0.73: p < 0.01). With deep learning threshold set to the same specificity as TPD, per-patient sensitivity improved from 79.8% (TPD) to 82.3% (deep learning) (p < 0.05), and per-vessel sensitivity improved from 64.4% (TPD) to 69.8% (deep learning) (p < 0.01).
Conclusions:
Deep learning has the potential to improve automatic interpretation of MPI as compared with current clinical methods.
Background:
We developed machine-learning (ML) models to estimate a patient's risk of cardiac death based on adenosine myocardial perfusion SPECT (MPS) and associated clinical data, and compared their performance to baseline logistic regression (LR). We demonstrated an approach to visually convey the reasoning behind a patient's risk to provide insight to clinicians beyond that of a "black box."
Methods:
We trained multiple models using 122 potential clinical predictors (features) for 8321 patients, including 551 cases of subsequent cardiac death. Accuracy was measured by area under the ROC curve (AUC), computed within a cross-validation framework. We developed a method to display the model's rationale to facilitate clinical interpretation.
Results:
The baseline LR (AUC = 0.76; 14 features) was outperformed by all other methods. A least absolute shrinkage and selection operator (LASSO) model (AUC = 0.77; p = .045; 6 features) required the fewest features. A support vector machine (SVM) model (AUC = 0.83; p < .0001; 49 features) provided the highest accuracy.
Conclusions:
LASSO outperformed LR in both accuracy and simplicity (number of features), with SVM yielding best AUC for prediction of cardiac death in patients undergoing MPS. Combined with presenting the reasoning behind the risk scores, our results suggest that ML can be more effective than LR for this application.
The algorithms of machine learning, which can sift through vast numbers of variables looking for combinations that reliably predict outcomes, will improve prognosis, displace much of the work of radiologists and anatomical pathologists, and improve diagnostic accuracy.
There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at
http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net
.