ArticlePDF Available
154
Editorial
Artificial Intelligence and Cardiac Imaging: We need to talk
about this
José de Arimateia Batista Araujo-Filho1,2, Antonildes Nascimento Assunção Júnior1,3, Marco Antonio
Gutierrez3,Cesar Higa Nomura1,3
1Hospital Sirio Libanes, São Paulo, Brazil; 2Memorial Sloan-Kettering Cancer Center, New York, Estados Unidos; 3InCor-HFCMUSP, São Paulo, Brazil
Keywords
Artificial Intelligence; Machine Learning; Radiology.
Correspondence: José de Arimateia Batista Araujo-Filho •
E-mail: ariaraujocg@gmail.com
DOI: 10.5935/2318-8219.20190035
“Measure what is measurable, and make measurable what
is not so.”
Galileo Galilei (1564-1642)
With the rapid technological progress experienced by
medical imaging in recent years, the conversion of digital images
into high-dimensional data, that is, with a large number of
variables, has been driven by the concept that images contain
a myriad of underlying pathophysiological information that is
often difficult identify and comprehend using conventional
visual analysis.1 The quantitative analysis of these images and
the organization of these parameters in complex databases (Big
Data) — with large volume, variety and speed of information
generation — brought radiology closer to the new technological
frontiers, involving Artificial Intelligence (AI), Machine Learning
(ML) and Deep Learning (DL) (Figure 1).
“Images are more than pictures, they are data.”1 The
mantra of modern radiology portrays the potential of this new
understanding of imaging in the new age of precision medicine,
going far beyond diagnosis and having a decisive role in clinical
decision making. In this new and complex context, Cardiology
has been a broad and fertile ground for AI approaches, as many
heterogeneous and sufficiently prevailing diseases (ideal for large
databases), such as heart failure and coronary artery disease, are
yet to be sub-phenotyped in the constant pursuit of increasingly
customized treatments. Besides, problems with acquisition time,
high costs, efficiency and misdiagnosis are commonly observed
and thus expected to be mitigated with the promising new
applications of AI in cardiovascular propaedeutics.2
The first applications of AI in cardiac imaging were
based on the automated quantitative measurement of
anatomical parameters (such as stenosis and vascular
dilatations) and functional parameters (such as ventricular
ejection fraction) that were previously performed
manually and were often considered laborious and
time-consuming. The latest rapidly-evolving applications
include the prediction of myocardial ischemia from
automated coronary Fractional Flow Reserve (FFR) analysis
using computed tomography (CT)3 and identification of
vulnerable plaques by CT angiography using radiomics, a
tool of quantitative evaluation of images based on textural
analysis, that is, heterogeneity of an area of interest from
the distribution on pixels or gray levels of each voxel. Other
recent applications include the development of automatic
reconstruction algorithms, analysis of image quality with DL5
and identification of incidental cardiovascular findings in
CT and Magnetic Resonance Imaging (MRI) examinations.
In addition to these axial methods, potential uses of AI
tools in echocardiography are also comprehensive and
include automated functional assessment (including
ejection fraction and longitudinal strain), quantification
of segmental contractility anomalies, and recognition of
axes and structures with DL techniques.6 Our group has
been a pioneer in research on the subject in the country,
developing studies that include textural analysis of coronary
plaques and aortic valve by radiomics (Figure 2), prognostic
markers in cardiomyopathies with ML7 techniques and
ventricular function evaluation with DL8 tools.
Despite the encouraging results and the growing number
of publications on the subject, there is still a long way to go
before scientific evidence involving AI in cardiac imaging
is implemented in clinical practice. The methodology of
many of the latest studies differs significantly, and some
of them have used mathematical corrections, which can
lead to overly optimistic results — many without external
validation. Another important point to be considered,
when use AI to solve medical image problems, is the
limited number of annotated data available for training.
In most of the cases, this requires the involvement of well-
trained physicians in a time consuming task, which limits
the number of annotated data available. Other limitations
include frequent variations of acquisition protocols (which
may reduce data robustness), as well as the diversity of
the methodologies used to extract features (quantitative
information derived from images) and interpretation of
the statistical models used. Consequently, more studies
are needed to validate the potential of these techniques,
preferably with larger samples, interinstitutional cooperation
and consistent methodology and validation.
Notwithstanding the great enthusiasm of the scientific
community, numerous doubts about the effects of all
this great potential has generated excessive anxiety
among professionals dealing with medical imaging. It is
paramount to state that the use of AI tools in medical
diagnosis is neither a threat nor a strategy to replace the
role of the physician in the cardiovascular propaedeutics.
By providing new diagnostic, predictive and prognostic
data, with potential impact on the individualized therapy
of these patients, such tools undoubtedly represent a
potential strategy to increase the importance and precision
of our work. We believe that AI has great chances of
promoting optimization of workflow and support in the
155
Editorial
Arq Bras Cardiol: Imagem cardiovasc. 2019;32(3):154-156
Araujo-Filho et al.
Artificial Intelligence and Cardiac Imaging
diagnosis in Radiology, as well as reducing the amount of
stress and exhaustion among professionals and improving
the quality of patient care.
Finally, we are sure that the incorporation of AI into cardiac
imaging is not something to be feared or avoided, but discussed,
understood and encouraged, both in multidisciplinary medical
meetings and in the academic training of new professionals.
We believe that all clinical and pathophysiological knowledge
gained throughout our training and career will remain vital for
the balanced use and interpretation of the new data generated
by these tools.9 It is up to us to take a leading role in deciding
where and how to apply all this knowledge.
Figure 1 – Diagram illustrating the basic concepts of articial intelligence, machine learning and deep learning.
Figure 2 – Workow for characterization of coronary plaque using radiomics. First-order features were extracted from a heterogeneous plaque in the anterior
descending artery.
Image acquisition and selection
of the volume of interest
Lesion
segmentation
Histogram
construction and
analysis
Data extraction and
analysis
Relative frequncy
Mitigation
(HU)
Mean
SD
Skewness
Kurtosis
275
190
0.75
1.93
Data prediction
and validation
ARTIFICIAL INTELLIGENCE
Any technique or system that allows
computers to mimic human behavior (feeling,
thinking, acting and adapting)
MACHINE LEARNING
A technique used to provide articial
intelligence with the capacity to learn
DEEP LEARNING
Class of machine learning
algorithms characterized by
the use of complex neural
networks
156
Editorial
Arq Bras Cardiol: Imagem cardiovasc. 2019;32(3):154-156
Araujo-Filho et al.
Artificial Intelligence and Cardiac Imaging
Referências
1. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures,
They Are Data. Radiology. 2016;278(2):563-77.
2. Dey D, Slomka PJ, Leeson P, Comaniciu D, Shrestha S, Sengupta PP, et al.
Artificial Intelligence in Cardiovascular Imaging: JACC State-of-the-Art
Review. J Am Coll Cardiol.2019;73(11):1317-35.
3.
Gaur S, Ovrehus KA, Dey D, Leipsic J, Botker HE, Jensen JM, et al. Coronary plaque
quantification and fractional flow reserve by coronary computed tomography
angiography identify ischaemia-causing lesions. Eur Heart J. 2016;37(15):1220-7.
4. Kolossvary M, Karady J, Szilveszter B, Kitslaar P, Hoffmann U, Merkely
B, et al. Radiomic Features Are Superior to Conventional Quantitative
Computed Tomographic Metrics to Identify Coronary Plaques With
Napkin-Ring Sign. Circ Cardiovasc Imaging.2017;10(12). pii: e006843.
5. Soffer S, Ben-Cohen A, Shimon O, Amitai MM, Greenspan H, Klang E.
Convolutional Neural Networks for Radiologic Images: A Radiologist’s
Guide. Radiology. 2019;290(3):590-606.
6. Alsharqi M, Woodward WJ, Mumith JA, Markham DC, Upton R,
Leeson P. Artificial intelligence and echocardiography. Echo Res Pract.
2018;5(4):R115-R25.
7. Rocon C, Tabassian M, Tavares De Melo MD, Araujo Filho JA, Parga Filho
JR, Hajjar LA, et al. Biventricular imaging markers to predict outcome in
non-compaction cardiomyopathy: a machine learning study. Eur Heart J.
2018;39(suppl_1), ehy566.P6485.
8.
Moreno RA, Rebelo MFSdS, Carvalho T, Assunção AN, Dantas RN,
Val Rd, et al. A combined deep-learning approach to fully automatic
left ventricle segmentation in cardiac magnetic resonance imaging.
Proc. SPIE 10953, Medical Imaging 2019: Biomedical Applications
in Molecular, Structural, and Functional Imaging, 109531Y (15
March 2019).
9. Johnson KW, Torres Soto J, Glicksberg BS, Shameer K, Miotto R,
Ali M, et al. Artificial Intelligence in Cardiology. J Am Coll Cardiol.
2018;71(23):2668-79.
... Figure 15. Subsets of Artificial Intelligence (Adapted from [22,23]). ...
... Machine Learning is divided into 3 categories: supervised learning, unsupervised learning and reinforcement learning ( Figure 16) [22,23]. [22][23]). ...
... Machine Learning is divided into 3 categories: supervised learning, unsupervised learning and reinforcement learning ( Figure 16) [22,23]. [22][23]). ...
Preprint
Full-text available
The quality of yarn is essential in the control of the fabrics processes. There is some commercial equipment that measures the quality of yarn based on sensors, of different types, used for collecting data about some yarn characteristic parameters. The existing equipment, for the above-mentioned purpose, is characterized by its high size and cost, and for allowing the analysis of only few yarn quality parameters. Thus, this paper presents the development and results obtained with the design of a mechatronic prototype integrating a computer vision system that allows, among other parameters, the analysis and classification, in real time, of the hairs of the yarn using artificial intelligence techniques. The system also determines other characteristics inherent to the yarn quality analysis, such as: linear mass, diameter, volume, twist orientation, twist step, average mass deviation, coefficient of variation, hairiness coefficient, average hairiness deviation and standard hairiness deviation, as well as performs spectral analysis. A comparison of the obtained results with the designed system and a commercial equipment was performed validating the undertaken methodology.
... In the clustering of data, known as clustering algorithms (Figure 18), the data are grouped according to the similarities between them. In extracting information (Figure 20), the algorithm associates the information from other information obtained previously, such as, for example, a recommendation of a movie on a specific website, based on the history of previously watched movies [26,27]. ...
... In the clustering of data, known as clustering algorithms ( Figure 18), the data are grouped according to the similarities between them. In extracting information (Figure 19), the algorithm associates the information from other information obtained previously, such as, for example, a recommendation of a movie on a specific website, based on the history of previously watched movies [26,27]. • Reinforcement learning is the training of machine learning models to make a sequence of decisions from a bonus/punishment perspective. ...
Article
Full-text available
The quality of yarn is essential in the control of the fabrics processes. There is some commercial equipment that measures the quality of yarn based on sensors, of different types, used for collecting data about some textile yarn characteristic parameters. The irregularity of the textile thread influences its physical properties/characteristics and there may be a possibility of a break in the textile thread during the fabric manufacturing process. This can contribute to the occurrence of unwanted patterns in fabrics that deteriorate their quality. The existing equipment, for the above-mentioned purpose, is characterized by its high size and cost, and for allowing the analysis of only few yarn quality parameters. The main findings/results of the study are the yarn analysis method as well as the developed algorithm, which allows the analysis of defects in a more precise way. Thus, this paper presents the development and results obtained with the design of a mechatronic prototype integrating a computer vision system that allows, among other parameters, the analysis and classification, in real time, of the hairs of the yarn using artificial intelligence techniques. The system also determines other characteristics inherent to the yarn quality analysis, such as: linear mass, diameter, volume, twist orientation, twist step, average mass deviation, coefficient of variation, hairiness coefficient, average hairiness deviation, and standard hairiness deviation, as well as performing spectral analysis. A comparison of the obtained results with the designed system and a commercial equipment was performed validating the undertaken methodology.
Article
Full-text available
Aims Left ventricular non‐compaction cardiomyopathy (LVNC) is a genetic heart disease, with heart failure, arrhythmias, and embolic events as main clinical manifestations. The goal of this study was to analyse a large set of echocardiographic (echo) and cardiac magnetic resonance imaging (CMRI) parameters using machine learning (ML) techniques to find imaging predictors of clinical outcomes in a long‐term follow‐up of LVNC patients. Methods and results Patients with echo and/or CMRI criteria of LVNC, followed from January 2011 to December 2017 in the heart failure section of a tertiary referral cardiologic hospital, were enrolled in a retrospective study. Two‐dimensional colour Doppler echocardiography and subsequent CMRI were carried out. Twenty‐four hour Holter monitoring was also performed in all patients. Death, cardiac transplantation, heart failure hospitalization, aborted sudden cardiac death, complex ventricular arrhythmias (sustained and non‐sustained ventricular tachycardia), and embolisms (i.e. stroke, pulmonary thromboembolism and/or peripheral arterial embolism) were registered and were referred to as major adverse cardiovascular events (MACEs) in this study. Recruited for the study were 108 LVNC patients, aged 38.3 ± 15.5 years, 48.1% men, diagnosed by echo and CMRI criteria. They were followed for 5.8 ± 3.9 years, and MACEs were registered. CMRI and echo parameters were analysed via a supervised ML methodology. Forty‐seven (43.5%) patients had at least one MACE. The best performance of imaging variables was achieved by combining four parameters: left ventricular (LV) ejection fraction (by CMRI), right ventricular (RV) end‐systolic volume (by CMRI), RV systolic dysfunction (by echo), and RV lower diameter (by CMRI) with accuracy, sensitivity, and specificity rates of 75.5%, 77%, 75%, respectively. Conclusions Our findings show the importance of biventricular assessment to detect the severity of this cardiomyopathy and to plan for early clinical intervention. In addition, this study shows that even patients with normal LV function and negative late gadolinium enhancement had MACE. ML is a promising tool for analysing a large set of parameters to stratify and predict prognosis in LVNC patients.
Article
Full-text available
Echocardiography plays a crucial role in the diagnosis and management of cardiovascular disease. However, interpretation remains largely reliant on the subjective expertise of the operator. As a result inter-operator variability and experience can lead to incorrect diagnoses. Artificial intelligence (AI) technologies provide new possibilities for echocardiography to generate accurate, consistent and automated interpretation of echocardiograms, thus potentially reducing the risk of human error. In this review, we discuss a subfield of AI relevant to image interpretation, called machine learning, and its potential to enhance the diagnostic performance of echocardiography. We discuss recent applications of these methods and future directions for AI-assisted interpretation of echocardiograms. The research suggests it is feasible to apply machine learning models to provide rapid, highly accurate and consistent assessment of echocardiograms, comparable to clinicians. These algorithm are capable of accurately quantifying a wide range of features, such as the severity of valvular heart disease or the ischaemic burden in patients with coronary artery disease. However, the applications and their use are still in their infancy within the field of echocardiography. Research to refine methods and validate their use for automation, quantification and diagnosis are in progress. Widespread adoption of robust AI tools in clinical echocardiography practice should follow and have the potential to deliver significant benefits for patient outcome.
Article
Full-text available
Artificial intelligence and machine learning are poised to influence nearly every aspect of the human condition, and cardiology is not an exception to this trend. This paper provides a guide for clinicians on relevant aspects of artificial intelligence and machine learning, reviews selected applications of these methods in cardiology to date, and identifies how cardiovascular medicine could incorporate artificial intelligence in the future. In particular, the paper first reviews predictive modeling concepts relevant to cardiology such as feature selection and frequent pitfalls such as improper dichotomization. Second, it discusses common algorithms used in supervised learning and reviews selected applications in cardiology and related disciplines. Third, it describes the advent of deep learning and related methods collectively called unsupervised learning, provides contextual examples both in general medicine and in cardiovascular medicine, and then explains how these methods could be applied to enable precision cardiology and improve patient outcomes.
Article
Full-text available
Background—Napkin-ring sign (NRS) is an independent prognostic imaging marker of major adverse cardiac events. However, identification of NRS is challenging because of its qualitative nature. Radiomics is the process of extracting thousands of quantitative parameters from medical images to create big-data data sets that can identify distinct patterns in radiological images. Therefore, we sought to determine whether radiomic analysis improves the identification of NRS plaques. Methods and Results—From 2674 patients referred to coronary computed tomographic angiography caused by stable chest pain, expert readers identified 30 patients with NRS plaques and matched these with 30 non-NRS plaques with similar degree of calcification, luminal obstruction, localization, and imaging parameters. All plaques were segmented manually, and image data information was analyzed using Radiomics Image Analysis package for the presence of 8 conventional and 4440 radiomic parameters. We used the permutation test of symmetry to assess differences between NRS and nonNRS plaques, whereas we calculated receiver-operating characteristics’ area under the curve values to evaluate diagnostic accuracy. Bonferroni-corrected P<0.0012 was considered significant. None of the conventional quantitative parameters but 20.6% (916/4440) of radiomic features were significantly different between NRS and non-NRS plaques. Almost half of these (418/916) reached an area under the curve value >0.80. Short- and long-run low gray-level emphasis and surface ratio of high attenuation voxels to total surface had the highest area under the curve values (0.918; 0.894 and 0.890, respectively). Conclusions—A large number of radiomic features are different between NRS and non-NRS plaques and exhibit excellent discriminatory value.
Article
Full-text available
Aims: Coronary plaque characteristics are associated with ischaemia. Differences in plaque volumes and composition may explain the discordance between coronary stenosis severity and ischaemia. We evaluated the association between coronary stenosis severity, plaque characteristics, coronary computed tomography angiography (CTA)-derived fractional flow reserve (FFRCT), and lesion-specific ischaemia identified by FFR in a substudy of the NXT trial (Analysis of Coronary Blood Flow Using CT Angiography: Next Steps). Methods and results: Coronary CTA stenosis, plaque volumes, FFRCT, and FFR were assessed in 484 vessels from 254 patients. Stenosis >50% was considered obstructive. Plaque volumes (non-calcified plaque [NCP], low-density NCP [LD-NCP], and calcified plaque [CP]) were quantified using semi-automated software. Optimal thresholds of quantitative plaque variables were defined by area under the receiver-operating characteristics curve (AUC) analysis. Ischaemia was defined by FFR or FFRCT ≤0.80. Plaque volumes were inversely related to FFR irrespective of stenosis severity. Relative risk (95% confidence interval) for prediction of ischaemia for stenosis >50%, NCP ≥185 mm(3), LD-NCP ≥30 mm(3), CP ≥9 mm(3), and FFRCT ≤0.80 were 5.0 (3.0-8.3), 3.7 (2.4-5.6), 4.6 (2.9-7.4), 1.4 (1.0-2.0), and 13.6 (8.4-21.9), respectively. Low-density NCP predicted ischaemia independent of other plaque characteristics. Low-density NCP and FFRCT yielded diagnostic improvement over stenosis assessment with AUCs increasing from 0.71 by stenosis >50% to 0.79 and 0.90 when adding LD-NCP ≥30 mm(3) and LD-NCP ≥30 mm(3) + FFRCT ≤0.80, respectively. Conclusion: Stenosis severity, plaque characteristics, and FFRCT predict lesion-specific ischaemia. Plaque assessment and FFRCT provide improved discrimination of ischaemia compared with stenosis assessment alone.
Article
Full-text available
In the past decade, the field of medical image analysis has grown exponentially, with an increased number of pattern recognition tools and an increase in data set sizes. These advances have facilitated the development of processes for high-throughput extraction of quantitative features that result in the conversion of images into mineable data and the subsequent analysis of these data for decision support; this practice is termed radiomics. This is in contrast to the traditional practice of treating medical images as pictures intended solely for visual interpretation. Radiomic data contain first-, second-, and higher-order statistics. These data are combined with other patient data and are mined with sophisticated bioinformatics tools to develop models that may potentially improve diagnostic, prognostic, and predictive accuracy. Because radiomics analyses are intended to be conducted with standard of care images, it is conceivable that conversion of digital images to mineable data will eventually become routine practice. This report describes the process of radiomics, its challenges, and its potential power to facilitate better clinical decision making, particularly in the care of patients with cancer.
Article
Data science is likely to lead to major changes in cardiovascular imaging. Problems with timing, efficiency, and missed diagnoses occur at all stages of the imaging chain. The application of artificial intelligence (AI) is dependent on robust data; the application of appropriate computational approaches and tools; and validation of its clinical application to image segmentation, automated measurements, and eventually, automated diagnosis. AI may reduce cost and improve value at the stages of image acquisition, interpretation, and decision-making. Moreover, the precision now possible with cardiovascular imaging, combined with “big data” from the electronic health record and pathology, is likely to better characterize disease and personalize therapy. This review summarizes recent promising applications of AI in cardiology and cardiac imaging, which potentially add value to patient care.
Article
Deep learning has rapidly advanced in various fields within the past few years and has recently gained particular attention in the radiology community. This article provides an introduction to deep learning technology and presents the stages that are entailed in the design process of deep learning radiology research. In addition, the article details the results of a survey of the application of deep learning-specifically, the application of convolutional neural networks-to radiologic imaging that was focused on the following five major system organs: chest, breast, brain, musculoskeletal system, and abdomen and pelvis. The survey of the studies is followed by a discussion about current challenges and future trends and their potential implications for radiology. This article may be used as a guide for radiologists planning research in the field of radiologic image analysis using convolutional neural networks.
A combined deep-learning approach to fully automatic left ventricle segmentation in cardiac magnetic resonance imaging
  • R A Moreno
  • Rebelo Mfsds
  • T Carvalho
  • A N Assunção
  • R N Dantas
  • Val Rd
Moreno RA, Rebelo MFSdS, Carvalho T, Assunção AN, Dantas RN, Val Rd, et al. A combined deep-learning approach to fully automatic left ventricle segmentation in cardiac magnetic resonance imaging. Proc. SPIE 10953, Medical Imaging 2019: Biomedical Applications in Molecular, Structural, and Functional Imaging, 109531Y (15 March 2019).