ArticlePDF Available

Neural network-based integration of polygenic and clinical information: development and validation of a prediction model for 10-year risk of major adverse cardiac events in the UK Biobank cohort

Authors:

Abstract and Figures

Background In primary cardiovascular disease prevention, early identification of high-risk individuals is crucial. Genetic information allows for the stratification of genetic predispositions and lifetime risk of cardiovascular disease. However, towards clinical application, the added value over clinical predictors later in life is crucial. Currently, this genotype–phenotype relationship and implications for overall cardiovascular risk are unclear. Methods In this study, we developed and validated a neural network-based risk model (NeuralCVD) integrating polygenic and clinical predictors in 395 713 cardiovascular disease-free participants from the UK Biobank cohort. The primary outcome was the first record of a major adverse cardiac event (MACE) within 10 years. We compared the NeuralCVD model with both established clinical scores (SCORE, ASCVD, and QRISK3 recalibrated to the UK Biobank cohort) and a linear Cox-Model, assessing risk discrimination, net reclassification, and calibration over 22 spatially distinct recruitment centres. Findings The NeuralCVD score was well calibrated and improved on the best clinical baseline, QRISK3 (ΔConcordance index [C-index] 0·01, 95% CI 0·009–0·011; net reclassification improvement (NRI) 0·0488, 95% CI 0·0442–0·0534) and a Cox model (ΔC-index 0·003, 95% CI 0·002–0·004; NRI 0·0469, 95% CI 0·0429–0·0511) in risk discrimination and net reclassification. After adding polygenic scores we found further improvements on population level (ΔC-index 0·006, 95% CI 0·005–0·007; NRI 0·0116, 95% CI 0·0066–0·0159). Additionally, we identified an interaction of genetic information with the pre-existing clinical phenotype, not captured by conventional models. Additional high polygenic risk increased overall risk most in individuals with low to intermediate clinical risk, and age younger than 50 years. Interpretation Our results demonstrated that the NeuralCVD score can estimate cardiovascular risk trajectories for primary prevention. NeuralCVD learns the transition of predictive information from genotype to phenotype and identifies individuals with high genetic predisposition before developing a severe clinical phenotype. This finding could improve the reprioritisation of otherwise low-risk individuals with a high genetic cardiovascular predisposition for preventive interventions. Funding Charité–Universitätsmedizin Berlin, Einstein Foundation Berlin, and the Medical Informatics Initiative.
Content may be subject to copyright.
www.thelancet.com/digital-health Vol 4 February 2022
e84
Articles
Lancet Digit Health 2022;
4: e84–94
*Contributed equally
†Contributed equally
Department of Cardiology,
Campus Benjamin Franklin
(J Steinfeldt MD,
Prof U Landmesser
MD), Center
for Digital Health
(T Buergel MSc, L Loock
MSc,
P Kittner BSc, G Ruyoga,
J Upmeier zu Belzen
MSc,
S Sasse
BSc, H Strangalies
BSc,
L Christmann
MSc,
N Hollmann
BSc, B Wolf
BSc,
Prof R Eils
PhD), Charité–
Universitätsmedizin Berlin,
corporate member of Freie
Universität Berlin, Humboldt-
Universität zu Berlin, and
Berlin Institute of Health,
Berlin, Germany; Centre for
Naturally Randomized Trials,
University of Cambridge,
Cambridge, UK
(Prof B Ference
MD); Institute of
Cardiovascular Science,
University College London,
London, UK
(Prof J Deanfield
MD); Health
Data Science Unit, Heidelberg
University Hospital and
BioQuant, Heidelberg,
Germany (Prof R Eils)
Correspondence to:
Prof Roland Eils, Berlin Institute
of Health, Charité–
Universitätsmedizin Berlin,
Digital Health Center, Berlin
10117, Germany
roland.eils@bih-charite.de
Neural network-based integration of polygenic and clinical
information: development and validation of a prediction
model for 10-year risk of major adverse cardiac events in the
UK Biobank cohort
Jakob Steinfeldt*, Thore Buergel*, Lukas Loock, Paul Kittner, Greg Ruyoga, Julius Upmeier zu Belzen, Simon Sasse, Henrik Strangalies,
Lara Christmann, Noah Hollmann, Benedict Wolf, Brian Ference, John Deanfield†, Ulf Landmesser†, Roland Eils†
Summary
Background In primary cardiovascular disease prevention, early identification of high-risk individuals is crucial.
Genetic information allows for the stratification of genetic predispositions and lifetime risk of cardiovascular disease.
However, towards clinical application, the added value over clinical predictors later in life is crucial. Currently, this
genotype–phenotype relationship and implications for overall cardiovascular risk are unclear.
Methods In this study, we developed and validated a neural network-based risk model (NeuralCVD) integrating
polygenic and clinical predictors in 395 713 cardiovascular disease-free participants from the UK Biobank cohort. The
primary outcome was the first record of a major adverse cardiac event (MACE) within 10 years. We compared the
NeuralCVD model with both established clinical scores (SCORE, ASCVD, and QRISK3 recalibrated to the UK
Biobank cohort) and a linear Cox-Model, assessing risk discrimination, net reclassification, and calibration over
22 spatially distinct recruitment centres.
Findings The NeuralCVD score was well calibrated and improved on the best clinical baseline, QRISK3 (∆Concordance
index [C-index] 0·01, 95% CI 0·009–0·011; net reclassification improvement (NRI) 0·0488, 95% CI 0·0442–0·0534)
and a Cox model (∆C-index 0·003, 95% CI 0·002–0·004; NRI 0·0469, 95% CI 0·0429–0·0511) in risk discrimination
and net reclassification. After adding polygenic scores we found further improvements on population level
(∆C-index 0·006, 95% CI 0·005–0·007; NRI 0·0116, 95% CI 0·0066–0·0159). Additionally, we identified an interaction
of genetic information with the pre-existing clinical phenotype, not captured by conventional models. Additional high
polygenic risk increased overall risk most in individuals with low to intermediate clinical risk, and age younger than
50 years.
Interpretation Our results demonstrated that the NeuralCVD score can estimate cardiovascular risk trajectories for
primary prevention. NeuralCVD learns the transition of predictive information from genotype to phenotype and
identifies individuals with high genetic predisposition before developing a severe clinical phenotype. This finding
could improve the reprioritisation of otherwise low-risk individuals with a high genetic cardiovascular predisposition
for preventive interventions.
Funding Charité–Universitätsmedizin Berlin, Einstein Foundation Berlin, and the Medical Informatics Initiative.
Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND
4.0 license.
Introduction
Cardiovascular diseases, such as coronary heart disease,
are consistently among the leading causes of death world-
wide. A personalised risk assessment is fund amental to
targeted prevention, intervention, and therapy. The early
identification of high-risk individuals is crucial to
reducing the disease burden on the population and
increasing the eectiveness of interventions.1
Current prognostic models focus on prevalent classical
cardiovascular risk factors, such as age, sex, blood
pressure, cholesterol measurements, lifestyle factors
such as smoking status, and medical history, which are
analysed by linear models such as the semi-parametric
Cox model.2–4 Beyond these risk factors, many genetic
variants have been associated with cardiovascular
disease5 and leveraged in polygenic scores (PGSs),6
summarising genetic predisposition for cardiovascular
disease at the time of birth. It has been shown that
genetic risk captured in PGSs is associated with disease
frequency for coronary heart disease and stroke.7,8 The
promise of PGSs to leverage genetic information in
primary prevention for early disease detection has
sparked the interest of regulatory authorities.1,9
However, the general applicability and benefit of PGSs
for preventive cardiovascular medicine remain disputed.10
One objection against a broad application of PGSs in
Articles
e85
www.thelancet.com/digital-health Vol 4 February 2022
primary prevention is the low information content for
most individuals in the population. In the long-tailed
PGS distribution, only the individuals in the top
percentiles show big changes in the associated disease
frequencies. Five groups recently investigated the
potential benefits of combining PGSs with conventional
cardiovascular disease risk factors for cardiovascular risk
prediction. Mosley and colleagues11 found no additional
benefit in adding a PGS against coronary heart disease to
the features of the American Heart Association/
Atherosclerotic Cardiovascular Disease (AHA/ASCVD)
pooled cohort equation in two cohorts of US adults. Elliot
and colleagues12 found significant, yet modest
improvements in discrimination (ie, the ability to
dierentiate individuals at low and high risk) and
reclassification (ie, correct reclassification of predicted
cases and non-cases based on the known ground truth
compared with a baseline model) after adding their score
against coronary artery disease to the features of the
AHA/ASCVD pooled cohort equation and the QRISK3
score in the UK Biobank cohort. Sun and colleagues13
found only incremental improvements in discrimination
over the population, but notable reclassification after
adding two PGSs against coronary heart disease stroke,
respectively, to conven tional predictors. With the
additional information of genetic predisposition, the
authors estimate prevention of additional 7% cardio-
vascular disease events compared with conventional
scores based on the altered treatment recommendations.13
Most recently, McKay and colleagues14 developed a novel
integrative risk tool com bining a novel coronary artery
disease PGS with the Pooled Cohort Equations (PCE)
and QRISK3 score, respectively. The authors report a
benefit in coronary artery disease prediction and estimate
a net reclassification improvement of 0·137 at the 7·5%
10-year risk threshold for PCE, and 0·035 at the 10%
10-year threshold for QRISK3, and propose an eect of
age and sex on reclassification.
Although PGSs bear an enormous potential for preven-
tive medicine and risk modelling, their relationship to
clinical phenotypes and known predictors remains
elusive.15 PGSs incorporate a wide range of variants over
an individual’s genome, distinguishing variants solely by
eect size and dose, not by the mechanism of action. The
incorporation of single nucleo tide polymorphisms
(SNPs) acting on known risk factors in PGSs has raised
concerns about potential biases emerging from joint
analysis with those same risk factors.15 This concern calls
for tools to model complex interactions to correct for
these shortcomings in integrating PGSs and clinical
predictors.
Neural networks represent state-of-the-art survival
analysis.16–19 If applied to real-world medical data, the
model’s increased complexity could facilitate the inte-
gration of polygenic information for primary prevention
of cardiovascular disease by inherently accounting for
the interaction of the polygenic information and the
clinical parameters.
This study presents the development and validation of
a novel neural network-based cardiovascular disease risk
model, NeuralCVD, based on Deep Survival Machines,19
for primary prevention based on a set of established
cardiovascular disease risk factors. Comparing our model
against existing risk scores and a Cox proportional
hazards model20 trained on the same data over the entire
study population, we first demonstrated its discriminative
capabilities. We subsequently assessed the integration of
PGSs in risk modelling for primary cardiovascular
Research in context
Evidence before this study
In primary cardiovascular risk prediction, genetic predictors
have already been added to traditional risk prediction models.
Although neural networks have been applied previously on
clinical variables, to date, no study has investigated their
application for modelling the interaction of clinical and genetic
predictors. We gathered evidence before this study using
Google Scholar, searching all entries from the beginning of the
database records until April 16, 2021, with no language
restrictions. Relevant work identified by the Google Scholar
search was considered the current state-of-the-art (thus
reference material) for this study. To identify eligible studies, we
used the keywords “cardiovascular disease”, “polygenic scores”,
“survival analysis”, and “neural networks”.
Added value of this study
Neural network-based survival models represent the state-of-
the art in time-to-event modelling. This study is, to our
knowledge, the first to assess the applicability of these
approaches for cardiovascular risk modelling in primary
cardiovascular disease prevention. Furthermore, it is the first
study to directly model the interaction of clinical risk and
polygenic risk. We show that neural networks can model this
genotype–phenotype relationship which could have direct
consequence in the prioritisation of preventive interventions.
Implications of all the available evidence
Our proposed NeuralCVD score demonstrates that neural
network-based survival models can learn expressive
multimodal patient representations. Consequently, this paves
the way for integrative models for cardiovascular primary
prevention leveraging both clinical and polygenic information
and for a clinical application of neural network-based risk
models in general. Furthermore, this study motivates research
in genetic variants and polygenic scores which maximise the
residual information content over commonly assessed clinical
predictors.
Articles
www.thelancet.com/digital-health Vol 4 February 2022
e86
disease prevention by building on six well established
PGSs against coronary artery disease and stroke.7,8,21–24
After retraining our NeuralCVD risk score and the Cox
model on clinical covariates and the PGSs, we
demonstrate that our model can integrate the genetic
information and learns the residual predictive
contribution of the poly genic information over the
manifested clinical phenotype.
Methods
Data source and outcome
We used data from the UK Biobank—a cohort of
273 383 women and 229 122 men aged between 37 years
and 73 years at the time of their baseline assessment. The
cohort is a sample of the UK’s general population;
participants were enrolled in 22 recruitment centres
across the UK. Patients with pre-existing myocardial
infarction, stroke, or lipid-lowering therapy were
excluded from the analysis, but retained as auxiliary
training data for our NeuralCVD score.
The outcome was 10-year cardiovascular disease risk
defined by the earliest recorded event of fatal or non-fatal
myocardial infarction (International Classification of
Diseases [ICD]10 codes I21, I22, I23, I24, I25) or fatal or
non-fatal transient ischaemic attack or ischaemic stroke
(ICD10 codes G45, I63, I64) either in the primary care
records, the hospital episode statistics, or death records.
The study adhered to the transparent reporting of a
multivariable prediction model for individual prognosis
or diagnosis (TRIPOD) state ment for reporting.25 The
completed checklist can be found in the appendix (p 17).
Covariate selection
Predictors were selected to reflect traditional primary
prevention risk models2–4 (appendix p 16). Demographic
information was extracted from primary care records and
confirmed at the study’s recruitment interview. Lifestyle
information was extracted from the questionnaire at
recruitment. Physical measurements and laboratory
measures were taken at recruitment. Pre-existing
medical conditions were extracted from the questionnaire
or interview at recruitment, primary care records, and
hospital episode statistics. Medications were extracted
from the recruitment interview. PGSs (PGS000011,21
PGS000018,7 PGS000057,22 PGS000058,23 PGS00005924)
for coronary artery disease and PGS0000398 for stroke
were selected from the PGS catalog26 and calculated for
all participants.
Dataset partitions and imputation
For model development and testing, we split the dataset
into 22 spatially separated partitions based on the
location of the assessment centre at recruitment. We
analysed the data in 22-fold nested cross-validation,
setting aside one of the spatially separated partitions as a
test set, aggregating the remaining partitions and
randomly selecting 10% of the aggregated data as the
validation set. Within each of the 22 cross-validation
loops, the individual test set (ie, the spatially disjunct
partition) remained untouched throughout model
development, while the validation set was used to validate
the fitting progress and checkpoint selection. All
22 obtained models were then evaluated on their
respective test sets. We assumed missing data occurred
at random depending on the clinical variables and the
Data for 502505 people available in complete UK Biobank
395 713 included in 22-times nested cross-validation with
spatially separated test sets by UK Biobank assessment
centre
106792 excluded
16 withdrew consent
1 sex not available
106775 earlier records available for myocardial
infarction, stroke, or lipid-lowering
treatment
0 5
Observation time (years)
10 15
0
0·1
B
A
0·4
0·3
0·2
Density
Median: 11·7 years
Sex
Male
Female
0
169081
226632
5
162447
222587
10
154654
217260
15
0
0
Observation time (years)
0
85
90
95
C
100
Disease-free survival (%)
Male
Female
Sex
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
++
++
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+++
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
++
++
+
+
+
+
+
+
+
+
+
++
+
+
+
+
++
+
++
+
+
+
+
++
+
+
+
++
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
++
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
++
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
++
+
+
++
++
+
+
++
+
+
++
+
+
+
+
+
+
+
+
+++
+
+
+
+
+
+
+
+
+
++
+++
+
++++
+
+++
++
+
+
+
++
+
+++
+++
+
+
+
+
+
+
+++
+
+
+
++
+
+
+
+
+
++
++
++
++++
++
+
+
+
+
+
+
+++
+
+++
++
++
+
+
+
++
+
+
+++
+++++
+
++
+
++
+
++++++
++
+
+
+
++++
+
++++++
+
+
++
+
+
+++++
+
++
+++
++
+
++++++
++
+
+
+++
+
+
++++
++
+
+++
+
++
+
+
+
+++
++++
+
++++
++
+++
++++++
+++++
+++++
+++
++
+++
++++++++
+
++
+
++
+
+
++++++++++
+++
++++++++++
++++
+++++++++++++
++++++++
++
++
++++++++++++++++++++
++++
+++++
++
++++
+
++++++++++
+++++++++++++++++
+++++++++
++++++
+++++
++++++++++++
++++++
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
++
++
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
++
+
++
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
++
+
+
+
+
+
+
+
+++
+
+
+
++
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+++
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
++
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
++
+
+
+
+
++
+
+
++
++
+
+
+
+
+
+
+
+
+
++
+
+
++
+
+
+
+
++
+
+
+
+
++
+
+
+
+
+
+
++
+
++
+
+
+
+
+
+
+
+
+
+
++
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
++
+
+
++
+
+
++
+
+
+
+
++
+
+
+
+
++
+++
+
+
+
++
+
++
+
+
+
+
+++
+
+++
++
+
+++
++
+
+
+
+++
+
+
+
+
+
+
+
+
++
+
+
++
+
+
+
++
++
++
+
+
+++
+
+
+
+
+
+
++
+
+
++
+
++
+
+
++++
+++
+++
+
+
+
+
+
+
+++
+
+
+
+
+
++
+
++
++
+
+
+
+++
+
++
+
+
+++
+
++
+
+
+
++++++
++
+
+++
++++
+
+
+
+
++
++
++
++
+
+
++
++
++
++
+++
+
+
++
++
++++
+++
+++
+
++++
++
++
+++
+
++
+
+++
+
++
+++
++
++++++
++
+
+++++
+
++
+
+
+++++++++
+
+
+
++++
+
+
+++++
+
+++++++
+++++
+++
+
++++++++
++++++
+++
++++
++
+
++
+++
+
++
+++++++++++++++
+++++++++
++++++++++++
++++
++++
+++++++++++
++++++++++++++++++
++++
++++++++++++++++++++++++++++++
++++++
++++++++++++++++
++++++
++
++
+
+++++++++++++
+++++++++++
++
++++++++++++++
+++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Figure 1: Selection and characteristics of study population
(A) Individuals in the UK Biobank population who withdrew consent, with
missing information about their sex or with earlier records of incident
myocardial infarction or stroke or lipid-lowering treatment at baseline were
excluded. The remaining set was split into training, validation, and test sets in
22-fold nested cross-validation based on the assigned UK Biobank assessment
centre. (B) Distribution of observation times for the derived study population.
The median observation time was 11·7 years (IQR 11·0–12·3). (C) Kaplan-Meier
estimates for the disease-free survival function stratified by sex. (D) Numbers at
risk in 5-year intervals stratified by sex.
See Online for appendix
Articles
e87
www.thelancet.com/digital-health Vol 4 February 2022
cardiovascular events and performed multiple impu-
tations using chained equations with random forests.27
Continuous variables were standardised and mean
centred; categorical variables were one-hot encoded.
Imputation models were fitted on the training sets and
applied to the respective validation and test set.
Model development and evaluation
We developed models on two distinct covariate sets, one
including 29 cardiovascular risk factors in existing risk
scores (table 1), the other with the addition of the
computed values for the six PGSs. We constructed three
models for each covariate set: a linear Cox model, a Cox
model with interaction terms for age and each PGS, and
our NeuralCVD score. The Cox model with the interaction
terms allows to assess potential non-linear eects
between age and the genetic information. For each
assessment centre, and thus each cross-validation split,
models were trained on the respective training set, and
checkpoints selected on the respective validation set. For
the final evaluation, predictions were then made for all
participants in the test set. Harell’s C-index was calculated
with the lifelines package,28 for both the aggregated test
set and individual assessment centres. The net
reclassification improvement was calculated with the
nricens package.29 95% CIs were calculated based on
1000 bootstrapping runs and report the 2·5% and 97·5%
quantile borders. For details on the implementation of
NeuralCVD, the Cox models and the calibration, please
refer to the appendix (pp 1–2).
Calculation of PGSs and the PGSMETA score
The PGSs were developed on multiple external cohorts
and covered a diverse set of patients. Detailed information
is available in the appendix (p 2). PGSs were calculated
with the published weights from the PGS catalog,26 the
imputed genotype information from the UK Biobank,
and the R package PRSice-2.30 To analyse and visualise
our model predictions by overall genetic risk, we sum the
individual percentile ranks for each of the six PGS scores
and calculate a new aggregated percentile rank over the
sum to construct a polygenic meta score (PGSMETA). All
models are trained by adding the six individual PGS
scores, not the PGSMETA score.
Relative risk differences
To investigate the impact of the PGSs on individual
predictions, we calculate relative risk dierences between
models trained with and without the polygenic information.
By subtracting the clinical risk estimate from the model’s
prediction, which was trained on the clinical and polygenic
information, we obtained an absolute risk dierence. It is
positive if the PGSs resulted in a higher risk estimate and
negative if they lead to a lower risk estimate. Next, we
normalised the absolute risk dierence by dividing it by the
clinical risk estimate to calculate the relative risk dierences.
Because absolute risk dierences for individuals with
clinical risk below 1% are close to zero and resulting relative
risk dierences in this group are thus prone to numerical
instabilities in calibration, we did not calculate relative risk
dierences for these individuals. All patient data used
throughout this study has been subject to patient consent
as covered by the UK Biobank. All patient data used
throughout this study was covered by the general patient
consent of the UK Biobank, which applies to this study
through the Material Transfer Agreement (MTA) of
application 51157.Calibration was evaluated graphically by
comparing predicted and observed risks.
Role of the funding source
The funders had no role in data collection, analysis,
interpretation, writing, and the decision to submit.
Results
Participants were enrolled from March 13, 2006, to
Oct 1, 2010. We extracted information on the demo-
graphics, clinical records, and outcomes of the complete
Male (n=169 081) Female (n=226 632) Overall (N=395 713)
Age at recruitment 56 (48 to 62) 56 (49 to 62) 56 (49 to 62)
Ethnicity ·· ·· ··
Asian 3523 (2·1%) 3588 (1·6%) 7111 (1·8%)
Black 2782 (1·7%) 3871 (1·7%) 6653 (1·7%)
Chinese 491 (0·3%) 866 (0·4%) 1357 (0·3%)
Mixed 892 (0·5%) 1624 (0·7%) 2516 (0·6%)
White 158 761 (95%) 213 464 (96%) 372 225 (95%)
Missing 2632 3219 5851
Townsend deprivation
index
–2·16
(–3·67 to 0·53)
–2·19
(–3·67 to 0·37)
–2·18
(–3·67 to 0·44)
Missing 230 276 506
Overall health rating ·· ·· ··
Excellent 30 892 (18%) 42 310 (19%) 73 202 (19%)
Good 98 290 (59%) 136 866 (61%) 235 156 (60%)
Fair 32 890 (20%) 39 057 (17%) 71 947 (18%)
Poor 5813 (3·5%) 6970 (3·1%) 12 783 (3·3%)
Missing 1196 1429 2625
Smoking status ·· ·· ··
Current 21 454 (13%) 20 012 (8·9%) 41 466 (11%)
Previous 58 717 (35%) 69 074 (31%) 127 791 (32%)
Never 87 924 (52%) 136 326 (60%) 224 250 (57%)
Missing 986 122 2206
Body-mass index, mg/
kg
26·9 (24·7 to 29·5) 25·8 (23·2 to 29·3) 26·4 (23·8 to 29·4)
Missing 1161 118 2341
Weight, kg 84 (76 to 93) 68 (61 to 78) 75 (66 to 86)
Missing 1026 1105 2131
Standing height, cm 176 (172 to 181) 163 (158 to 167) 168 (162 to 175)
Missing 1017 959 1976
Systolic blood pressure,
mm Hg
138 (128 to 150) 132 (120 to 146) 135 (124 to 148)
Missing 10 193 13 679 23 872
(Table 1 continues on next page)
Articles
www.thelancet.com/digital-health Vol 4 February 2022
e88
UK Biobank cohort.31,32 16 participants who withdrew
their participation consent agreement; one participant
without information on sex; and 106 775 participants
with earlier records of myocardial infarction, stroke, or
lipid-lowering treatment were excluded (figure 1). The
remaining 395 713 participants had a median age of
56 years (IQR 49–62), with 95% being of White or
British ethnicity, a median Townsend deprivation index
of –2·18 (–3·67 to 0·44). 60% of the study population had
good self-reported overall health (table 1; appendix p 10).
The median follow-up time was 11·7 years
(IQR 11·0–12·3). 28 083 (7·1%) participants had a major
cardiovascular adverse event (MACE; defined as included
fatal and non-fatal myocardial infarction, fatal and non-
fatal transient ischaemic attack or stroke, and cardio-
vascular death; figure 1). Based on Deep Survival
Machines,19 we developed the NeuralCVD score (figure 2;
appendix p 1) on a set of 29 cardiovascular risk factors
used in well established scores, the ESC score,2 the AHA/
ASCVD score,3 and the QRISK3 score4 (table 1).
To determine whether neural networks improved risk
discrimination over conventional approaches, we
compared the NeuralCVD against established clinical
baselines and a linear Cox20 model trained on the same
29 cardiovascular risk factors. All scores were evaluated
independently on all 22 assessment centres of the UK
Biobank cohort with the Concordance Index and the
categorical net-reclassification-improvement at the 10%
threshold (following the NICE guidelines33) as metrics for
the risk discrimination. We found that the NeuralCVD
score outperformed SCORE with a dierence in C-index
of 0·037 (95% CI 0·034–0·039), ASCVD with 0·024
(0·023–0·026), and QRISK3 with 0·010 (0·009–0·011). At
the 10% risk threshold the NRI over SCORE was 0·1043
(95% CI 0·0981–0·1103), resulting in an additional 4828
of 23 786 cases correctly identified as high-risk and 1106
cases incorrectly down classified. For non-cases, 14 572 of
371 889 were correctly down-classified, while 33 972 were
incorrectly identified as high risk. The NRI of NeuralCVD
over ASCVD was 0·0704 (0·0648–0·0765) and 0·0488
(0·0442–0·0534) over QRISK3 (figure 2; table 2). Absolute
reclassification counts are provided in the appendix (pp
11, 15). Improvements were smaller over a linear Cox
model fitted with the same set of covariates with a
dierence in C-index of 0·003 (0·002–0·004) and NRI of
0469 (0·0429–0·0511). The discrimination is stable
over all 22 distinct assessment centres (appendix pp 4,
12). All models were well calibrated over the observed risk
spectrum (figure 2; appendix p 4).
To assess the composite of clinical and genetic predictors,
we rebuilt the model on an extended covariate set with six
established PGS. We evaluated it against the occurrence of
the first recorded MACE in the observation window.
To ensure the validity of the PGS in our cohort, we first
confirmed the association of the applied PGS with the
observed frequency of the MACE endpoint (appendix
p 6). To test the potential of the NeuralCVD model to
integrate polygenic information, we added six well
established PGSs against coronary artery disease7,21–24 and
stroke8 to the covariate set used in the previous analysis.
The coecients of the Cox model that included the PGS
can be found in the appendix (p 14). Furthermore, to
allow the Cox model to assess potential non-linear eects
between age and the genetic information we additionally
tested interaction terms between age and the PGSs.
Additionally, we compared our models with the ASCVD-
based model previously published by Sun and colleagues.13
Integrating PGSs in the NeuralCVD model improved
risk discrimination over the clinical covariates alone with
a dierence in C-index of 0·006 (95% CI 0·005–0·007)
and NRI of 0·0116 (95% CI 0·0066–0·0159; figure 2;
table 3; appendix pp 11, 15). Although we observed
improvements in discriminative performance for the Cox
model after addition of the PGSs as well, the NeuralCVD
model remained superior in C-index (COX plus PGS
002, 95% CI 0·002–0·003; COX plus PGS*age 0·002,
002–0·003) and NRI (COX plus PGS 0·0424,
95% CI 0·0383–0·0464; COX plus PGS*age 0·0359,
Male (n=169 081) Female (n=226 632) Overall (N=395 713)
(Continued from previous page)
Diastolic blood
pressure, mm Hg
84 (78 to 91) 80 (74 to 87) 82 (75 to 89)
Missing 10 192 13 678 2387
Total cholesterol,
mmol/L
5·72 (5·07 to 6·41) 5·93 (5·24 to 6·68) 5·84 (5·16 to 6·56)
Missing 1048 15 514 25 994
HDL cholesterol, mmol/L 1·26 (1·08 to 1·47) 1·57 (1·34 to 1·83) 1·43 (1·20 to 1·70)
Missing 22 622 34 937 57 559
LDL cholesterol,
mmol/L
3·67 (3·17 to 4·20) 3·66 (3·13 to 4·25) 3·67 (3·15 to 4·23)
Missing 10 835 15 858 206 693
Triglycerides, mmol/L 1·68 (1·16 to 2·43) 1·30 (0·94 to 1·84) 1·44 (1·02 to 2·09)
Missing 10 659 15 634 26 293
Familial history of heart
disease
57 025 (34%) 90 847 (40%) 147 872 (37%)
Antihypertensive
treatment
1697 (1·0%) 1610 (0·7%) 3307 (0·8%)
Aspirin 1456 (0·9%) 935 (0·4%) 2391 (0·6%)
Atypical antipsychotics 2086 (1·2%) 3865 (1·7%) 5951 (1·5%)
Glucocorticoids 122 (<0·1%) 233 (0·1%) 355 (<0·1%)
Type 1 diabetes 795 (0·5%) 586 (0·3%) 1381 (0·3%)
Type 2 diabetes 3379 (2·0%) 2440 (1·1%) 5819 (1·5%)
Chronic kidney disease 6052 (3·6%) 8253 (3·6%) 14 305 (3·6%)
Atrial fibrillation 2687 (1·6%) 2303 (1·0%) 4990 (1·3%)
Migraine 7027 (4·2%) 20 879 (9·2%) 27 906 (7·1%)
Rheumatoid arthritis 6052 (3·6%) 16 916 (7·5%) 22 968 (5·8%)
Systemic lupus
erythematosus
186 (0·1%) 725 (0·3%) 911 (0·2%)
Severe mental illness 14 303 (8·5%) 28 902 (13%) 43 205 (11%)
Erectile dysfunction 7731 (4·6%) 0 (0%) 7731 (2·0%)
Data are median (IQR) or n (%).
Table 1: Study population
Articles
e89
www.thelancet.com/digital-health Vol 4 February 2022
SCORE ASCVD QRISK3 NeuralCVD
clinical PGS
Cox
clinical
0·69
0·70
0·72
74
C-index
0·76
0·71
0·73
0·75
B
A
0 302010
0
Observed risk (%)
Predicted risk (%)
100
30
20
10
0
Density
0·100
SCORE
0·075
0·050
0·025
C
0 302010
Predicted risk (%)
ASCVD
0 302010
Predicted risk (%)
QRISK3
0 302010
Predicted risk (%)
Cox clinical
0 302010
Predicted risk (%)
NeuralCVD
clinical
Cox Sun
PGS
Cox
clinical
NeuralCVD
clinical
Cox clinical
PGS*age
NeuralCVD
clinical PGS
Cox clinical
PGS
0·69
0·70
0·72
74
C-index
0·76
0·71
0·73
0·75
D
0 302010
0
Observed risk (%)
Predicted risk (%)
100
30
20
10
0
Density
0·100
Cox Sun PGS
0·075
0·050
0·025
E
0 302010
Predicted risk (%)
Cox clinical PGS
0 302010
Predicted risk (%)
Cox clinical
PGS*age
0 302010
Predicted risk (%)
NeuralCVD
clinical PGS
29 clinical predictors
256
NeuralCVD score
Linear layer
SELU
128
Linear layer
SELU
100
4
4
4
4
Linear layer
Linear layer Linear layerLinear layer
SoftplusSoftmax Softplus
SELU
η
Πi σi Weibull (ηi, ƹi)
Deep survival machine
σ
ƹ
4
t
Incidence
t
Cumulative incidence
0·734
0·707
0·719
7410·743
∆–0·003
∆–0·01
∆–0·037
0·727
∆–0·022
741
∆–0·009
743
∆–0·006
0·747
∆–0·002
0·747
∆–0·002
0·749
∆–0·024
Articles
www.thelancet.com/digital-health Vol 4 February 2022
e90
0321–0·0394). Compared with the model proposed by
Sun and colleagues,13 we see improvements in the C-index
of 0·022 (95% CI 0·021–0·024) and in NRI of 0·0740
(95% CI 0·0687–0·0790). All models were well calibrated
over the full spectrum of risk (figure 2), and the
dierences were consistent in the spatially separated
assessment centres (appendix pp 4, 12).
To investigate the individual impact of the additional
genetic information in both the Cox and the NeuralCVD
models, we calculated relative risk dierences with and
without genetic information. The neural risk model
predicted relative risk dierences of up to 805% and
–84% compared with up to 152% and –63% (249% and
–72% with PGS*age interaction) for the Cox model in
our study cohort (figure 3).
To examine these sizable risk dierences in the
NeuralCVD model, we investigated associations between
the information added by the PGS (ie, the relative risk
dierence) and the observed clinical phenotype. Although
we did not find pronounced associations of individual
conventional risk factors with relative risk dierences at
10 years (appendix p 7), we observed an association with
the overall clinical risk (figure 3) and an association with
age (figure 3). For high genetic risk individuals (top 5%
PGSMETA), we saw pronounced risk dierences in the
predicted risk in younger individuals with low to
intermediate clinical risk. With increasing clinical risk and
age, the risk dierence was diminished. This eect was
predicted to be most pronounced for individuals with high
genetic risk, decreasing with lower genetic predisposition
(appendix p 8). These dierences were non-existent in the
linear Cox model without interaction terms, but observable
in the Cox model with the PGS*age interaction terms.
Furthermore, the eect reflected the predicted cardio-
vascular risk trajectories stratified by clinical risk and
age (figure 3). Young and low-risk individuals were
predicted to have the highest relative risk increase from
high genetic predisposition (RR[t10] 2·64, 95% CI
2·52–2·76). Patients between 50 years and 60 years at
intermediate clinical risk were predicted to have a lower
impact (RR[t10] 1·81, 1·78–1·85) and individuals older
than 60 years at already high clinical risk see the smallest
eect on their overall risk with the additional high
genetic risk information 1·40 (1·37–1·42). To substantiate
these findings, we calculated the number of events
stratified by clinical risk and age at the end of the
observation window for dierent genetic risk strata
(appendix p 9). The relative risk for high genetic risk (top
5%) was 1·93 (95% CI 1·62–2·25) in the young and low
risk subgroup, 1·65 (1·43–1·94) in the middle and
intermediate risk supgroup, and 1·49 (1·34–1·63) in the
older and high-risk subgroup at the end of the
observation window.
These findings suggest that the additional predictive
polygenic information depends on the clinical phenotype
(ie, clinical risk) and that our NeuralCVD score can
model this residual contribution. High genetic risk did
SCORE vs NeuralCVD clinical AHA/ASCVD vs NeuralCVD
clinical
QRISK3 vs NeuralCVD clinical Cox clinical vs NeuralCVD
clinical
NRI 0·1043 (0·0981 to 0·1103) 0·0704 (0·0648 to 0·0765) 0·0488 (0·0442 to 0·0534) 0·0469 (0·0429 to 0·0511)
Cases (n=23 786) 15·65% (15·06 to 16·23) 11·41% (10·86 to 12·01) 9·62% (9·17 to 10·06) 10·75% (10·35 to 11·17)
Non-cases (n=371 889) –5·22% (–5·33 to –5·11) –4·38% (–4·47 to –4·27) –4·74% (–4·83 to –4·65) –6·06% (–6·14 to –5·98)
Data are NRI (95% CI) or % (95% CI). We assessed the categorical net reclassification improvement of our NeuralCVD score at the clinically relevant 10% risk threshold
compared with the clinical baselines SCORE, ASCVD, QRISK3, and the linear Cox model. The NeuralCVD score substantially improves net reclassification and is particularly
sensitive in detecting high-risk cases. NRI=net reclassification improvement. AHA/ASCVD=American Heart Association/ Atherosclerotic Cardiovascular Disease.
Table 2: Categorical net reclassification improvement of NeuralCVD clinical at the 10% threshold
Cox Sun PGS vs NeuralCVD
clinical plus PGS
Cox clinical PGS vs NeuralCVD
clinical plus PGS
Cox clinical vs NeuralCVD clinical
plus PGS
Cox clinical PGS*age vs
NeuralCVD clinical plus PGS
NeuralCVD clinical vs
NeuralCVD clinical plus PGS
NRI 0·0740 (0·0678 to 0·0795) 0·0424 (0·0383 to 0·0464) 0·0585 (0·0538 to 0·0625) 0·0359 (0·0321 to 0·0394) 0·0116 (0·0066 to 0·0159)
Cases (n=23 790) 12·92% (12·39 to 13·47) 10·34% (9·89 to 10·76) 11·87% (11·42 to 12·27) 9·00% (8·65 to 9·34) 1·12% (0·62 to 1·54)
Non-cases (n=371 909) –5·52% (–5·64 to –5·42) –6·10% (–6·16 to –6·00) –6·01% (–6·11 to –5·92) –5·41% (–5·48 to –5·34) 0·05% (–0·03 to 0·12)
Data are NRI (95% CI) or % (95% CI). Categorical net reclassification improvement of the NeuralCVD score with PGS at the 10% 10-year risk threshold compared with the American Heart Association/
Atherosclerotic Cardiovascular Disease-based model proposed by Sun and colleagues,13 the linear Cox model with and without PGS addition, the non-linear Cox model with PGSs, and the NeuralCVD score
without PGS addition. The NeuralCVD score with PGS improves net reclassification over all other scores. PGS=polygenic scores. NRI=net reclassification improvement.
Table 3: Categorical net reclassification improvement of NeuralCVD clinical plus PGS at the 10% threshold
Figure 2: Comparison of the NeuralCVD score with established risk scores and
after addition of PGSs
(A) Our NeuralCVD score builds on the architecture of Deep Survival Machines,19
learning a patient representation from the input features to parameterise a
mixture of Weibull distributions to model the incidence function over a
continuous time scale. (B) Our NeuralCVD score outperformed existing
approaches in discrimination of major adverse cardiac event risk at 10 years
measured by bootstrapped C-index. Over the entire population, this
corresponded to an increment of 0·01 compared with the best-performing
baseline model, the QRISK3 score (appendix p 4). (C–E) Calibration curves at
10 years. PGS=polygenic score. SELU=scaled exponential linear unit.
Articles
e91
www.thelancet.com/digital-health Vol 4 February 2022
0 5 10 15 20 25
–100
Relative risk difference (Δ Risk/RiskClinical)
Clinical risk (%)
500
400
300
200
100
0
B
Bottom 5% genetic
risk
0 5 10 15 20 25
Clinical risk (%)
0 5 10 15 20 25
Clinical risk (%)
Median 5% genetic
risk
COX
clinical
PGS*age
COX
clinical
PGS*age
NeuralCVD
clinical PGS
NeuralCVD
clinical PGS
Cox
clinical
PGS
Cox clinical
PGS
Top 5% genetic risk
Age at recruitment
(years)
0
Relative risk
3
2
1
D
F
Low clinical risk (<5%)
Observation time (years) Observation time (years) Observation time (years)
Age <50 years
Intermediate clinical
risk (5–10%)
High clinical risk (>10%)
0 5 10 15 20 25
0
Relative risk
3
2
1
Age >60 years
0 5 10 15 20 25 0 5 10 15 20 25
0
Relative risk
Overall risk (%)Overall risk (%) Overall risk (%)
3
2
1
Age 50−60 years
0
100
50
40
30
20
10
E
Low clinical risk (<5%)
Observation time (years) Observation time (years) Observation time (years)
Age <50 years
Intermediate clinical
risk (5–10%)
High clinical risk (>10%)
0 5 10 15 20 25
Age >60 years
0 5 10 15 20 25 0 5 10 15 20 25
Age 50−60 years
–100 0 100 200 300 400 500 600 700 800
Genetic risk
Relative risk difference (Δ risk/riskclinical)
Median
5%
Bottom
5%
Top 5%
A
Genotypes
SNPs
Observed phenotypes
Clinical risk factors
Residual predictive information
Future phenotypes
MACE
40 50 60 70
Age at recruitment
(years)
Age at recruitment
(years)
–100
Relative risk difference (Δ Risk/RiskClinical)
500
400
300
200
100
0
C
Bottom 5% genetic
risk
40 50 60 70 40 50 60 70
Median 5% genetic
risk
Top 5% genetic risk
Cox
Cox interaction
NeuralCVD
0
100
50
40
30
20
10
0
100
50
40
30
20
10
Top 5%
Genetic risk
Bottom 5%Median 5%
Articles
www.thelancet.com/digital-health Vol 4 February 2022
e92
not significantly aect the overall risk in older individuals
when their clinical risk was already high. In contrast, the
risk in young individuals at low to intermediate clinical
risk sharply increased.
Discussion
PGSs have been shown to inform on an individual’s
genetic predisposition for many common diseases. Their
application in primary cardiovascular disease prevention
suggests great potential for early identification of high-
genetic-risk individuals and timely intervention before a
clinical phenotype is developed. However, PGSs are
approximations of the lifetime genetic risk and thus, to
be applied clinically, it is imperative to understand the
relationship between the information provided by PGSs,
the observed clinical phenotype, and the overall risk later
in life. Similarly, although neural networks represent the
state-of-the-art performance in survival modelling to
date, few medical studies exploit this potential.
In this study, we presented NeuralCVD, a novel neural-
network-based cardiovascular disease risk model for
primary cardiovascular disease prevention. On data from
the UK Biobank cohort, we show that an application of
NeuralCVD on phenotypic data improves discrimination
and reclassification at the 10% risk threshold over
currently available clinical scores and a Cox baseline
model. These findings encourage the use of neural
survival models in primary cardiovascular disease
prevention, as this improvement in discrimination does
not require any additional predictors. In agreement with
previous studies,12,13 we subsequently show that adding
genetic information further improves discrimination
and categorical reclassification at the 10% risk threshold
resulting in more high-risk cases detected.
Established methods integrate clinical predictors and
the polygenic information additively, irrespective of
biological mechanisms of action and mediatory eects on
the observed phenotype.13,14 Although the eect of PGS
addition on risk discrimination is small at the population
level, it is greater at the individual level. Through further
investigation of relative risk dierences, we found that our
NeuralCVD score accounted for the transition of predictive
information from the genotype to the composite clinical
phenotype by learning higher order interactions between
clinical risk factors and PGS variables. Thereby,
NeuralCVD captured an attenuating eect of observed
phenotypes with increasing clinical risk on information
gained by the PGS addition in the high genetic risk
strata. We found a similar but weaker interaction with age,
which could be modelled by interaction terms between
age and the PGS in the Cox model. These findings imply
that substantial parts of the genetic risk captured by PGS
act through phenotypic manifestation, and age alone is
not a sucient approximation. It is the residual
contribution of PGS information over the clinical risk
factors (figure 3), which is relevant in an applied clinical
setting.15 This transition of the predictive information
from the genotype to the clinical phenotype was first
hypothesised by Jannsen and colleagues.15 In their article,
the central idea was that, although independent at birth,
the eects of SNPs in the PGSs are mediated through
clinical factors (eg, LDL cholesterol, blood pressure, and
weight) and reduce the residual genetic risk contribution
later in life. Analysing event rates in the UK Biobank, we
can confirm this heterogeneity in residual genetic
information (appendix pp 9, 13).
The implications of the findings are two-fold. First,
PGSs allow for the identification of individuals who are
still most susceptible to their genetic predisposition
before developing a severe clinical phenotype. Second,
when the predictive information has already transitioned
from the genotype to the phenotype (ie, clinical risk), the
future overall risk trajectory is just modestly informed by
PGS.
Nevertheless, this study is subject to several limitations.
First, as shown previously,13 the UK Biobank study cohort
is of generally lower risk for cardiovascular events than
the general primary care population and recalibration
with a relevant data source—eg, the UK Clinical Practice
Research Datalink, should be performed before public
application. Second, although the model was validated in
spatially separated samples from the individual
assessment centres, and we did not observe any signs of
overfitting, the NeuralCVD model is yet to be evaluated
in an entirely independent cohort. This is of particular
importance for every model incorporating PGSs, as
generalisation to ancestrally distinct populations is
controversial.34
Figure 3: Differences in relative and overall risk as modelled by NeuralCVD
and the Cox models when stratified by age and clinical risk
(A) Distributions of the RRD for three genetic strata (bottom, median, and top
5% PGSMETA). Higher genetic risk increases the RRD for all models. The
distributions of RRDs for the NeuralCVD model are wide, with RRDs of up to
805% for the top 5% genetic stratum compared with the predicted risk based on
the clinical factors. (B) RRDs within the two Cox models and the NeuralCVD
score on PGS addition for the bottom, median, and top 5% of PGSMETA.
Increasing genetic risk yields positive RRDs for both the Cox models and the
NeuralCVD score. RRDs for the Cox model are constant over the spectrum of
clinical risk. In contrast, the NeuralCVD learned the residual contribution of the
polygenic risk over the clinical risk. In the high genetic risk group, RRDs were the
highest for the low-to-intermediate clinical risk group and declined with clinical
risk of more than 15%. (C) RRDs plotted against patient age at baseline.
(D) 25-year risk ratios stratified by genetic risk (bottom, median, and top 5%
PGSMETA), age, and clinical risk. Additional genetic information increased risk
most in individuals with low-to-intermediate clinical risk, and age younger than
50 years. (E) 25-year overall risk stratified by genetic risk (bottom, median, and
top 5% PGSMETA), age, and clinical risk. Risk ratios from (D) are reflected in the
cardiovascular disease risk trajectories and in the proportion of polygenic risk in
the overall risk. The difference in trajectories is most pronounced in individuals
with low-to-intermediate clinical risk and age younger than 50 years.
(F) Proposed mechanism for impact of polygenic information on overall risk,
adapted from Janssens.15 Parts of the SNPs included in PGS mediate through the
manifestation of a clinical phenotype. As conventional risk factors contain this
information, the information gained by PGS addition is the residual
information. RRD=relative risk difference. PGSMETA=polygenic meta score as
defined in methods. MACE=major adverse cardiac event. SNPs=single nucleotide
polymorphisms.
Articles
e93
www.thelancet.com/digital-health Vol 4 February 2022
Third, although discrimination, reclassification, and
calibration are crucial criteria for evaluating predictive
models and allow comparison over the established
baselines, they are not quantifying an absolute clinical
impact at the population level (eg, life-years saved by
identifying the correct individuals for early intervention).
This is relevant for primary prevention, because most
individuals in a population are not expected to show
strong risk modifications after adding PGSs to the
predictors. Here prospective studies are required to show
clinical utility. Additionally, clinical acceptance of genetic
risk modification could be facilitated by further validation
with phenotypic markers of subclinical disease.35
In summary, we introduced a clinically applicable
neural-network-based risk model for primary cardio-
vascular disease prevention that outperformed conven-
tional scores and learnt the residual genetic contribution
to identify individuals at the highest risk of cardiovascular
events. This opens up new opportunities for targeted
primary cardiovascular disease prevention, integrating
both clinical and genetic risk factors.
Contributors
RE, UL, and JD conceived, designed, and supervised the project. JS and
TB implemented models, and did tests and data analysis. LL, PK, GR,
HS, LC, and BW supported the analysis. JUzB, SS, and NH provided
support with polygenic score calculation. BF provided methodological
support and contributed to discussion of the results. JS, TB, RE, and UL
wrote and prepared the Article. JS, TB, LL, PK, GR, HS, LC, BW, JUzB,
SS, NH, RE, and UL had access to the raw data sets and verified the data.
BF and JD were not covered by Charité Berlin’s Material Transfer
Agreement for the UK Biobank data application (number 51157) and
therefore not permitted to access the raw data. JS, TB, JD, UL, and RE
were responsible for the decision to submit this Article. All authors had
access to the data presented, read, revised and approved the Article.
Declaration of interests
UL received grants from Bayer, Novartis, and Amgen; consulting fees
from Bayer, Sanofi, Amgen, and Novartis; Daiichi Sankyo and honoraria
from Novartis, Sanofi, Bayer, Amgen, and Daiichi Sankyo. JD received
consulting fees from GENinCode UK; honoraria from Amgen,
Boehringer Ingelheim, Merck, Pfizer, Aegerion, Novartis, Sanofi, Takeda,
Novo Nordisk, and Bayer. He holds an Einstein Professorship, serves as
fiduciary Senior Advisor and NHS Healthcheck Expert at Public Health
England and chairs the Review of the National Health Check
Programme at Public Health England. He is chief medical advisor to
Our Future Health. All other authors declare no competing interests.
Data sharing
UK Biobank data are available to bona fide researchers on application at
http://www.ukbiobank.ac.uk/using-the-resource/. Our code is available
on https://github.com/thbuerg/NeuralCVD.
Acknowledgments
This research was conducted using data from UK Biobank, a major
biomedical database, via application number 51157. This project was
funded by the Charité–Universitätsmedizin Berlin and the Einstein
Foundation Berlin. The study was supported by the Bundesministerium
für Bildung und Forschung-funded Medical Informatics Initiative
(HiGHmed, 01ZZ1802A, 01ZZ1802Z).
References
1 UK Department of Health and Social Care. Advancing our health:
prevention in the 2020s. 2019. https://www.gov.uk/government/
consultations/advancing-our-health-prevention-in-the-2020s/
advancing-our-health-prevention-in-the-2020s-consultation-
document (accessed Feb 5, 2021).
2 Conroy R. Estimation of ten-year risk of fatal cardiovascular disease
in Europe: the SCORE project. Eur Heart J 2003; 24: 987–1003.
3 Go D C, Lloyd-Jones D M, Bennett G, et al. 2013 ACC/AHA
guideline on the assessment of cardiovascular risk. Circulation 2014;
129: S49–73.
4 Hippisley-Cox J, Coupland C, Brindle P. Development and
validation of QRISK3 risk prediction algorithms to estimate future
risk of cardiovascular disease: prospective cohort study. BMJ 2017;
357: j2099.10.1136/bmj.j2099.
5 Nikpay M, Goel A, Won HH, et al. A comprehensive
1000 Genomes-based genome-wide association meta-analysis of
coronary artery disease. Nat Genet 2015; 47: 1121–30.
6 Khera AV, Chan M, Aragam KG, et al. Genome-wide polygenic
scores for common diseases identify individuals with risk
equivalent to monogenic mutations. Nat Genet 2018; 50: 1219–24.
7 Inouye M, Abraham G, Nelson CP, et al. Genomic risk prediction of
coronary artery disease in 480,000 adults: implications for primary
prevention. J Am Coll Cardiol 2018; 72: 1883–93.
8 Abraham G, Malik R, Yonova-Doing E, et al. Genomic risk score
oers predictive performance comparable to clinical risk factors for
ischaemic stroke. Nat Commun 2019; 10: 5819.
9 Khoury MJ, Mensah GA. Is it time to integrate polygenic risk scores
into clinical practice? Let’s do the science first and follow the
evidence wherever it takes us! 2019. https://blogs.cdc.gov/
genomics/2019/06/03/is-it-time/ (accessed Feb 5, 2021).
10 Torkamani A, Wineinger NE, Topol EJ. The personal and clinical
utility of polygenic risk scores. Nat Rev Genet 2018; 19: 581–90.
11 Mosley JD, Gupta DK, Tan J, et al. Predictive accuracy of a polygenic
risk score compared with a clinical risk score for incident coronary
heart disease. JAMA 2020; 323: 627–35.
12 Elliott J, Bodinier B, Bond TA, et al. Predictive accuracy of a
polygenic risk score-enhanced prediction model vs a clinical risk
score for coronary artery disease. JAMA 2020; 323: 636–45.
13 Sun L, Pennells LID, Kaptoge S, et al. Polygenic risk scores in
cardiovascular risk prediction: a cohort study and modelling
analyses. PLoS Med 2021; 18: e1003498.
14 Riveros-Mckay F, Weale M E, Moore R, et al. An integrated
polygenic tool substantially enhances coronary artery disease
prediction. Circ Genom Precis Med 2021; 14: e003304.
15 Janssens ACJW. Validity of polygenic risk scores: are we measuring
what we think we are? Hum Mol Genet 2019; 28: R143–50.
16 Luck M, Sylvain T, Lodi A, Bengio Y. Deep learning for patient-
specific kidney graft survival analysis. arXiv 2017; published online
May 29. https://arxiv.org/pdf/1705.10245 (preprint).
17 Gensheimer MF, Narasimhan B. A scalable discrete-time survival
model for neural networks. PeerJ 2019; 7: e6257.
18 Rietschel C, Yoon J, Van Der Schaar M. Feature selection for
survival analysis with competing risks using deep learning. 2018;
published online Nov 22. https://arxiv.org/abs/1811.09317 (preprint).
19 Nagpal C, Li XR, Dubrawski A. Deep survival machines: fully
parametric survival regression and representation learning for
censored data with competing risks. IEEE J Biomed Health Inform
2021; 25: 3163–75.
20 Cox DR. Regression models and life-tables.
J R Stat Soc Series B Stat Methodol 1972; 34: 187–202.
21 Tada H, Melander O, Louie JZ, et al. Risk prediction by genetic risk
scores for coronary heart disease is independent of self-reported
family history. Eur Heart J 2016; 37: 561–67.
22 Natarajan P, Young R, Stitziel NO, et al. Polygenic risk score
identifies subgroup with higher burden of atherosclerosis and
greater relative benefit from statin therapy in the primary
prevention setting. Circulation 2017; 135: 2091–101.
23 Morieri ML, Gao H, Pigeyre M, et al. Genetic tools for coronary risk
assessment in type 2 diabetes: a cohort study from the ACCORD
clinical trial. Diabetes Care 2018; 41: 2404–13.
24 Hajek C, Guo X, Yao J, et al. Coronary heart disease genetic risk
score predicts cardiovascular disease risk in men, not women.
Circulation 2018; 11: e002324.
25 Moons KGM, Altman DG, Reitsma JB, et al. Transparent reporting
of a multivariable prediction model for individual prognosis or
diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med
2015; 162: W1–73.
26 Lambert SA, Gil L, Jupp S, et al. The Polygenic Score Catalog: an open
database for reproducibility and systematic evaluation. bioRxiv 2020;
published online May 23. https://doi.org/10.1101/2020.05.20.20108217
(preprint).
Articles
www.thelancet.com/digital-health Vol 4 February 2022
e94
27 Wilson S. Miceforest. Github. https://github.com/
AnotherSamWilson/miceforest (accessed Jan 4, 2021).
28 Davidson-Pilon C. lifelines: survival analysis in Python.
J Open Source Softw 2019; 4: 1317.
29 Inoue E. NRI for risk prediction models with time to event and
binary response data. CRAN 2018; published online May 30.
https://cran.r-project.org/web/packages/nricens/nricens.pdf
(accessed Jan 16, 2021).
30 Choi SW, Mak TS-H, O’Reilly PF. Tutorial: a guide to performing
polygenic risk score analyses. Nat Protoc 2020; 15: 2759–72.
31 Sudlow C, Gallacher J, Allen N, et al. UK Biobank: an open access
resource for identifying the causes of a wide range of complex
diseases of middle and old age. PLoS Med 2015; 12: e1001779.
32 Bycroft C, Freeman C, Petkova D, et al. The UK Biobank resource
with deep phenotyping and genomic data. Nature 2018; 562: 203–09.
33 NICE. Overview, cardiovascular disease: risk assessment and
reduction, including lipid modification. Guidance. NICE. 2016.
https://www.nice.org.uk/guidance/cg181 (accessed Feb 3, 2021).
34 Schultz LM, Merikangas AK, Ruparel K, et al. Stability of polygenic
scores across discovery genome-wide association studies. bioRxiv 2021;
published online June 18. https://doi.org/10.1101/2021.09.10.459833
(preprint).
35 Figtree GA, Vernon ST, Nicholls SJ. Taking the next steps to
implement polygenic risk scoring for improved risk stratification
and primary prevention of coronary artery disease.
Eur J Prev Cardiol 2020; published online Nov 4. DOI:10.1093/
eurjpc/zwaa030.
... Importantly, our approach, based on routine health records, shows large discriminative improvements for the majority of diseases compared with conventionally tested biomarkers [55][56][57] and can be generalized across diverse health systems, populations, and ethnicities. However, we also see that including the medical history over age and sex deteriorated the performance for a subset of 0.7% (UK Biobank) and 5.5% (All Of Us cohort), respectively. ...
Article
Full-text available
The COVID-19 pandemic exposed a global deficiency of systematic, data-driven guidance to identify high-risk individuals. Here, we illustrate the utility of routinely recorded medical history to predict the risk for 1741 diseases across clinical specialties and support the rapid response to emerging health threats such as COVID-19. We developed a neural network to learn from health records of 502,489 UK Biobank participants. Importantly, we observed discriminative improvements over basic demographic predictors for 1546 (88.8%) endpoints. After transferring the unmodified risk models to the All of US cohort, we replicated these improvements for 1115 (78.9%) of 1414 investigated endpoints, demonstrating generalizability across healthcare systems and historically underrepresented groups. Ultimately, we showed how this approach could have been used to identify individuals vulnerable to severe COVID-19. Our study demonstrates the potential of medical history to support guidance for emerging pandemics by systematically estimating risk for thousands of diseases at once at minimal cost.
... Neural network survival analysis represents the most advanced technology currently available for survival analysis [13]. Notable examples of this include Deep-Surv, DeepHit, Logistic-Hazard, and others. ...
Article
Full-text available
Background Dementia is a major public health challenge in modern society. Early detection of high-risk dementia patients and timely intervention or treatment are of significant clinical importance. Neural network survival analysis represents the most advanced technology for survival analysis to date. However, there is a lack of deep learning-based survival analysis models that integrate both genetic and clinical factors to develop and validate individualized dynamic dementia risk prediction models. Methods and results This study is based on a large prospective cohort from the UK Biobank, which includes a total of 41,484 participants with an average follow-up period of 12.6 years. Initially, 364 candidate features (predictor variables) were screened. The top 30 key features were then identified by ranking the importance of each predictor variable using the Gradient Boosting Machine (GBM) model. A multi-model comparison strategy was employed to evaluate the predictive performance of four survival analysis models: DeepSurv, DeepHit, Kaplan–Meier estimation, and the Cox proportional hazards model (CoxPH). The results showed that the average Harrell's C-index for the DeepSurv model was 0.743, for the DeepHit model it was 0.633, for the CoxPH model it was 0.749, and for the Kaplan–Meier estimator model it was 0.500. In addition, the average D-Calibration Survival Measure was 6.014, 4408.086, 32274.743, and 1.508, respectively. The Brier score (BS) was used to assess the importance of features for the DeepSurv dementia prediction model, and the relationship between features and dementia was visualized using a partial dependence plot (PDP). To facilitate further research, the team deployed the DeepSurv dementia prediction model on AliCloud servers and designated it as the UKB-DementiaPre Tool. Conclusion This study successfully developed and validated the DeepSurv dementia prediction model for individuals aged 60 years and above, integrating both genetic and clinical data. The model was then deployed on AliCloud servers to promote its clinical translation. It is anticipated that this prediction model will provide more accurate decision support for clinical treatment and will serve as a valuable tool for the primary prevention of dementia.
... In each fold, the data were split by patient-level into training (70%), validation (10%), and test (20%) sets. Within each of the 5 cross-validation loops, the individual test set (that is, the spatially separated partition) remained untouched throughout model development and the validation set was used to validate the fitting progress and checkpoint selection 58,59 . The checkpoint yielding the highest 95% lower bound of estimated performance return on the validation set was selected as the final checkpoint for each model. ...
Article
Full-text available
Delirium can result in undesirable outcomes including increased length of stays and mortality in patients admitted to the intensive care unit (ICU). Dexmedetomidine has emerged for delirium prevention in these patients; however, optimal dosing is challenging. A reinforcement learning-based Artificial Intelligence model for Delirium prevention (AID) is proposed to optimize dexmedetomidine dosing. The model was developed and internally validated using 2416 patients (2531 ICU admissions) and externally validated on 270 patients (274 ICU admissions). The estimated performance return of the AID policy was higher than that of the clinicians’ policy in both derivation (0.390 95% confidence interval [CI] 0.361 to 0.420 vs. −0.051 95% CI −0.077 to −0.025) and external validation (0.186 95% CI 0.139 to 0.236 vs. −0.436 95% CI −0.474 to −0.402) cohorts. Our finding indicates that AID might support clinicians’ decision-making regarding dexmedetomidine dosing to prevent delirium in ICU patients, but further off-policy evaluation is required.
... This shows the efficacy of AI algorithms in accurately distinguishing between various cardiomyopathy types. Steinfeldt et al. 91 96 classified AD patients and healthy controls using GWAS and ResNet, achieving accuracies of 71.38% for AD classification and 92.65% for healthy control classification. The study also discovered novel genetic biomarkers for AD. ...
Article
Full-text available
Background The field of precision medicine endeavors to transform the healthcare industry by advancing individualised strategies for diagnosis, treatment modalities, and predictive assessments. This is achieved by utilizing extensive multidimensional biological datasets encompassing diverse components, such as an individual's genetic makeup, functional attributes, and environmental influences. Artificial intelligence (AI) systems, namely machine learning (ML) and deep learning (DL), have exhibited remarkable efficacy in predicting the potential occurrence of specific cancers and cardiovascular diseases (CVD). Methods We conducted a comprehensive scoping review guided by the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework. Our search strategy involved combining key terms related to CVD and AI using the Boolean operator AND. In August 2023, we conducted an extensive search across reputable scholarly databases including Google Scholar, PubMed, IEEE Xplore, ScienceDirect, Web of Science, and arXiv to gather relevant academic literature on personalised medicine for CVD. Subsequently, in January 2024, we extended our search to include internet search engines such as Google and various CVD websites. These searches were further updated in March 2024. Additionally, we reviewed the reference lists of the final selected research articles to identify any additional relevant literature. Findings A total of 2307 records were identified during the process of conducting the study, consisting of 564 entries from external sites like arXiv and 1743 records found through database searching. After 430 duplicate articles were eliminated, 1877 items that remained were screened for relevancy. In this stage, 1241 articles remained for additional review after 158 irrelevant articles and 478 articles with insufficient data were removed. 355 articles were eliminated for being inaccessible, 726 for being written in a language other than English, and 281 for not having undergone peer review. Consequently, 121 studies were deemed suitable for inclusion in the qualitative synthesis. At the intersection of CVD, AI, and precision medicine, we found important scientific findings in our scoping review. Intricate pattern extraction from large, complicated genetic datasets is a skill that AI algorithms excel at, allowing for accurate disease diagnosis and CVD risk prediction. Furthermore, these investigations have uncovered unique genetic biomarkers linked to CVD, providing insight into the workings of the disease and possible treatment avenues. The construction of more precise predictive models and personalised treatment plans based on the genetic profiles of individual patients has been made possible by the revolutionary advancement of CVD risk assessment through the integration of AI and genomics. Interpretation The systematic methodology employed ensured the thorough examination of available literature and the inclusion of relevant studies, contributing to the robustness and reliability of the study's findings. Our analysis stresses a crucial point in terms of the adaptability and versatility of AI solutions. AI algorithms designed in non-CVD domains such as in oncology, often include ideas and tactics that might be modified to address cardiovascular problems. Funding No funding received.
... 59 60 Recent studies have shown significant improvements in cardiovascular risk prediction using large data sets and machine learning methods. [60][61][62][63] However, these studies still only target one organ (the heart), and when compared with conventional statistical models, deep learning or other 'black box' methods are not as readily explainable or easily translatable to clinical use. 64 Several studies have tackled multidisease prediction. ...
Article
Full-text available
Objectives Despite rising rates of multimorbidity, existing risk assessment tools are mostly limited to a single outcome of interest. This study tests the feasibility of producing multiple disease risk estimates with at least 70% discrimination (area under the receiver operating curve, AUROC) within the time and information constraints of the existing primary care health check framework. Design Observational prospective cohort study Setting UK Biobank. Participants 228 240 adults from the UK population. Interventions None. Main outcome measures Myocardial infarction, atrial fibrillation, heart failure, stroke, all-cause dementia, chronic kidney disease, fatty liver disease, alcoholic liver disease, liver cirrhosis and liver failure. Results Using a set of predictors easily gathered at the standard primary care health check (such as the National Health Service Health Check), we demonstrate that it is feasible to simultaneously produce risk estimates for multiple disease outcomes with AUROC of 70% or greater. These predictors can be entered once into a single form and produce risk scores for stroke (AUROC 0.727, 95% CI 0.713 to 0.740), all-cause dementia (0.823, 95% CI 0.810 to 0.836), myocardial infarction (0.785, 95% CI 0.775 to 0.795), atrial fibrillation (0.777, 95% CI 0.768 to 0.785), heart failure (0.828, 95% CI 0.818 to 0.838), chronic kidney disease (0.774, 95% CI 0.765 to 0.783), fatty liver disease (0.766, 95% CI 0.753 to 0.779), alcoholic liver disease (0.864, 95% CI 0.835 to 0.894), liver cirrhosis (0.763, 95% CI 0.734 to 0.793) and liver failure (0.746, 95% CI 0.695 to 0.796). Conclusions Easily collected diagnostics can be used to assess 10-year risk across multiple disease outcomes, without the need for specialist computing or invasive biomarkers. Such an approach could increase the utility of existing data and place multiorgan risk information at the fingertips of primary care providers, thus creating opportunities for longer-term multimorbidity prevention. Additional work is needed to validate whether these findings would hold in a larger, more representative cohort outside the UK Biobank.
... The same conclusion was reached by the researchers also for the breast cancer and breast cancer subtypes in Chinese population in the work [6], although the difference in performance was less signi cant (AUC ROC of 0.601 for DNN and 0.598 for logistic ridge regression). Neural network-based approach has also proven effective in predicting other risks, including some heart conditions (myocardial infarction, stroke and others) [7], Alzheimer's disease [8,9] and 10 phenotypes from UK biobank [10]. In this study, we evaluated the potential of various machine-learning methods on simulated data with epistasis. ...
Preprint
Full-text available
Background Polygenic risk score (PRS) prediction is widely used to assess the risk of diagnosis and progression of many diseases. Routinely, the weights of individual SNPs are estimated by the linear regression model that assumes independent and linear contribution of each SNP to the phenotype. However, for complex multifactorial diseases such as Alzheimer's disease, diabetes, cardiovascular disease, cancer, and others, association between individual SNPs and disease could be non-linear due to epistatic interactions. The aim of the presented study is to explore the power of non-linear machine learning algorithms and deep learning models to predict the risk of multifactorial diseases with epistasis. Results First, we tested ensemble tree methods and deep learning neural networks against LASSO linear regression model on simulated data with different types and strength of epistasis. The results showed that with the increase of strength of epistasis effect, non-linear models significantly outperform linear. Then the higher performance of non-linear models over linear was confirmed on real genetic data for multifactorial phenotypes such as obesity, type 1 diabetes, and psoriasis. From non-linear models, gradient boosting appeared to be the best model in obesity and psoriasis while deep learning methods significantly outperform linear approaches in type 1 diabetes. Conclusions Overall, our study underscores the efficacy of non-linear models and deep learning approaches in more accurately accounting for the effects of epistasis in simulations with specific configurations and in the context of certain diseases.
Article
Full-text available
Background Polygenic risk score (PRS) prediction is widely used to assess the risk of diagnosis and progression of many diseases. Routinely, the weights of individual SNPs are estimated by the linear regression model that assumes independent and linear contribution of each SNP to the phenotype. However, for complex multifactorial diseases such as Alzheimer’s disease, diabetes, cardiovascular disease, cancer, and others, association between individual SNPs and disease could be non-linear due to epistatic interactions. The aim of the presented study is to explore the power of non-linear machine learning algorithms and deep learning models to predict the risk of multifactorial diseases with epistasis. Methods Simulated data with 2- and 3-loci interactions and tested three different models of epistasis: additive, multiplicative and threshold, were generated using the GAMETES. Penetrance tables were generated using PyTOXO package. For machine learning methods we used multilayer perceptron (MLP), convolutional neural network (CNN) and recurrent neural network (RNN), Lasso regression, random forest and gradient boosting models. Performance of machine learning models were assessed using accuracy, AUC-ROC, AUC-PR, recall, precision, and F1 score. Results First, we tested ensemble tree methods and deep learning neural networks against LASSO linear regression model on simulated data with different types and strength of epistasis. The results showed that with the increase of strength of epistasis effect, non-linear models significantly outperform linear. Then the higher performance of non-linear models over linear was confirmed on real genetic data for multifactorial phenotypes such as obesity, type 1 diabetes, and psoriasis. From non-linear models, gradient boosting appeared to be the best model in obesity and psoriasis while deep learning methods significantly outperform linear approaches in type 1 diabetes. Conclusion Overall, our study underscores the efficacy of non-linear models and deep learning approaches in more accurately accounting for the effects of epistasis in simulations with specific configurations and in the context of certain diseases.
Article
Recent advances have given rise to a spectrum of digital health technologies that have the potential to revolutionize the design and conduct of cardiovascular clinical trials. Advances in domain tasks such as automated diagnosis and classification, synthesis of high-volume data and latent data from adjacent modalities, patient discovery, telemedicine, remote monitoring, augmented reality, and in silico modelling have the potential to enhance the efficiency, accuracy, and cost-effectiveness of cardiovascular clinical trials. However, early experience with these tools has also exposed important issues, including regulatory barriers, clinical validation and acceptance, technological literacy, integration with care models, and health equity concerns. This narrative review summarizes the landscape of digital tools at each stage of clinical trial planning and execution and outlines roadblocks and opportunities for successful implementation of digital tools in cardiovascular clinical trials.
Article
Background Cardiovascular disease (CVD) remains a major cause of mortality in the UK, prompting the need for improved risk predictive models for primary prevention. Machine learning (ML) models utilizing electronic health records (EHR) offer potential enhancements over traditional risk scores like QRISK3 and ASCVD. Objective To systematically evaluate and compare the efficacy of ML models against conventional CVD risk prediction algorithms using EHR data for medium to long-term (5-10 years) CVD risk prediction. Methods A systematic review and random-effect meta-analysis were conducted according to PRISMA guidelines, assessing studies from 2010 to 2024. We retrieved 32 ML models and 26 conventional statistical models from 20 selected studies, focusing on performance metrics such as Area Under the Curve (AUC) and heterogeneity across models. Results ML models, particularly random forest and deep learning, demonstrated superior performance, with the highest recorded pooled AUCs of 0.865 (95% CI: 0.812-0.917) and 0.847 (95% CI: 0.766-0.927), respectively. These significantly outperformed the conventional risk score of 0.765 (95% CI: 0.734-0.796). However, significant heterogeneity (I² > 99%) and potential publication bias were noted across the studies. Conclusion While ML models show enhanced calibration for CVD risk, substantial variability and methodological concerns limit their current clinical applicability. Future research should address these issues by enhancing methodological transparency and standardization to improve the reliability and utility of these models in clinical settings. Implications This study highlights the advanced capabilities of ML models in CVD risk prediction and emphasizes the need for rigorous validation to facilitate their integration into clinical practice. This review was registered with PROSPERO (CRD42021250058).
Article
Full-text available
The COVID-19 pandemic exposed a global deficiency of systematic, data-driven guidance to identify high-risk individuals. Here, we illustrate the utility of routinely recorded medical history to predict the risk for 1883 diseases across clinical specialties and support the rapid response to emerging health threats such as COVID-19. We developed a neural network to learn from health records of 502,460 UK Biobank. Importantly, we observed discriminative improvements over basic demographic predictors for 1774 (94.3%) endpoints. After transferring the unmodified risk models to the All of US cohort, we replicated these improvements for 1347 (89.8%) of 1500 investigated endpoints, demonstrating generalizability across healthcare systems and historically underrepresented groups. Ultimately, we showed how this approach could have been used to identify individuals vulnerable to severe COVID-19. Our study demonstrates the potential of medical history to support guidance for emerging pandemics by systematically estimating risk for thousands of diseases at once at minimal cost.
Article
Full-text available
Polygenic scores (PGS) are commonly evaluated in terms of their predictive accuracy at the population level by the proportion of phenotypic variance they explain. To be useful for precision medicine applications, they also need to be evaluated at the individual level when phenotypes are not necessarily already known. We investigated the stability of PGS in European-American (EUR) and African-American (AFR)-ancestry individuals from the Philadelphia Neurodevelopmental Cohort and the Adolescent Brain Cognitive Development Study using different discovery GWAS for post-traumatic stress disorder (PTSD), type-2 diabetes (T2D), and height. We found that pairs of EUR-ancestry GWAS for the same trait had genetic correlations > 0.92. However, PGS calculated from pairs of same-ancestry and different-ancestry GWAS had correlations that ranged from <0.01 to 0.74. PGS stability was greater for height than for PTSD or T2D. A series of height GWAS in the UK Biobank suggested that correlation between PGS is strongly dependent on the extent of sample overlap between the discovery GWAS. Focusing on the upper end of the PGS distribution, different discovery GWAS do not consistently identify the same individuals in the upper quantiles, with the best case being 60% of individuals above the 80th percentile of PGS overlapping from one height GWAS to another. The degree of overlap decreases sharply as higher quantiles, less heritable traits, and different-ancestry GWAS are considered. PGS computed from different discovery GWAS have only modest correlation at the individual level, underscoring the need to proceed cautiously with integrating PGS into precision medicine applications.
Article
Full-text available
Background - There is considerable interest in whether genetic data can be used to improve standard cardiovascular disease risk calculators, as the latter are routinely used in clinical practice to manage preventative treatment. Methods - Using the UK Biobank (UKB) resource, we developed our own polygenic risk score (PRS) for coronary artery disease (CAD). We used an additional 60,000 UKB individuals to develop an integrated risk tool (IRT) that combined our PRS with established risk tools (either the American Heart Association/American College of Cardiology's Pooled Cohort Equations (PCE) or UK's QRISK3), and we tested our IRT in an additional, independent, set of 186,451 UKB individuals. Results - The novel CAD PRS shows superior predictive power for CAD events, compared to other published PRSs and is largely uncorrelated with PCE and QRISK3. When combined with PCE into an integrated risk tool, it has superior predictive accuracy. Overall, 10.4% of incident CAD cases were misclassified as low risk by PCE and correctly classified as high risk by the IRT, compared to 4.4% misclassified by the IRT and correctly classified by PCE. The overall net reclassification improvement for the IRT was 5.9% (95% CI 4.7-7.0). When individuals were stratified into age-by-sex subgroups the improvement was larger for all subgroups (range 8.3%-15.4%), with best performance in 40-54yo men (15.4%, 95% CI 11.6-19.3). Comparable results were found using a different risk tool (QRISK3), and also a broader definition of cardiovascular disease. Use of the IRT is estimated to avoid up to 12,000 deaths in the USA over a 5-year period. Conclusions - An integrated risk tool that includes polygenic risk outperforms current risk stratification tools and offers greater opportunity for early interventions. Given the plummeting costs of genetic tests, future iterations of CAD risk tools would be enhanced with the addition of a person's polygenic risk.
Article
Full-text available
Background Polygenic risk scores (PRSs) can stratify populations into cardiovascular disease (CVD) risk groups. We aimed to quantify the potential advantage of adding information on PRSs to conventional risk factors in the primary prevention of CVD. Methods and findings Using data from UK Biobank on 306,654 individuals without a history of CVD and not on lipid-lowering treatments (mean age [SD]: 56.0 [8.0] years; females: 57%; median follow-up: 8.1 years), we calculated measures of risk discrimination and reclassification upon addition of PRSs to risk factors in a conventional risk prediction model (i.e., age, sex, systolic blood pressure, smoking status, history of diabetes, and total and high-density lipoprotein cholesterol). We then modelled the implications of initiating guideline-recommended statin therapy in a primary care setting using incidence rates from 2.1 million individuals from the Clinical Practice Research Datalink. The C-index, a measure of risk discrimination, was 0.710 (95% CI 0.703–0.717) for a CVD prediction model containing conventional risk predictors alone. Addition of information on PRSs increased the C-index by 0.012 (95% CI 0.009–0.015), and resulted in continuous net reclassification improvements of about 10% and 12% in cases and non-cases, respectively. If a PRS were assessed in the entire UK primary care population aged 40–75 years, assuming that statin therapy would be initiated in accordance with the UK National Institute for Health and Care Excellence guidelines (i.e., for persons with a predicted risk of ≥10% and for those with certain other risk factors, such as diabetes, irrespective of their 10-year predicted risk), then it could help prevent 1 additional CVD event for approximately every 5,750 individuals screened. By contrast, targeted assessment only among people at intermediate (i.e., 5% to <10%) 10-year CVD risk could help prevent 1 additional CVD event for approximately every 340 individuals screened. Such a targeted strategy could help prevent 7% more CVD events than conventional risk prediction alone. Potential gains afforded by assessment of PRSs on top of conventional risk factors would be about 1.5-fold greater than those provided by assessment of C-reactive protein, a plasma biomarker included in some risk prediction guidelines. Potential limitations of this study include its restriction to European ancestry participants and a lack of health economic evaluation. Conclusions Our results suggest that addition of PRSs to conventional risk factors can modestly enhance prediction of first-onset CVD and could translate into population health benefits if used at scale.
Preprint
Full-text available
Polygenic [risk] scores (PGS) can enhance prediction and understanding of common diseases and traits. However, the reproducibility of PGS and their subsequent applications in biological and clinical research have been hindered by several factors, including: inadequate and incomplete reporting of PGS development, heterogeneity in evaluation techniques, and inconsistent access to, and distribution of, the information necessary to calculate the scores themselves. To address this we present the PGS Catalog (www.PGSCatalog.org), an open resource for polygenic scores. The PGS Catalog currently contains 192 published PGS from 78 publications for 86 diverse traits, including diabetes, cardiovascular diseases, neurological disorders, cancers, as well as traits like BMI and blood lipids. Each PGS is annotated with metadata required for reproducibility as well as accurate application in independent studies. Using the PGS Catalog, we demonstrate that multiple PGS can be systematically evaluated to generate comparable performance metrics. The PGS Catalog has capabilities for user deposition, expert curation and programmatic access, thus providing the community with an open platform for polygenic score research and translation.
Article
We present the Polygenic Score (PGS) Catalog (https://www.PGSCatalog.org), an open resource of published scores (including variants, alleles and weights) and consistently curated metadata required for reproducibility and independent applications. The PGS Catalog has capabilities for user deposition, expert curation and programmatic access, thus providing the community with a platform for PGS dissemination, research and translation.
Article
We describe a new approach to estimating relative risks in time-to-event prediction problems with censored data in a fully parametric manner. Our approach does not require making strong assumptions of constant proportional hazards of the underlying survival distribution, as required by the Cox-proportional hazard model. By jointly learning deep nonlinear representations of the input covariates, we demonstrate the benefits of our approach when used to estimate survival risks through extensive experimentation on multiple real world datasets with different levels of censoring. We further demonstrate advantages of our model in the competing risks scenario. To the best of our knowledge, this is the first work involving fully parametric estimation of survival times with competing risks in the presence of censoring.
Article
Coronary artery disease (CAD) remains the leading cause of death worldwide. The role of hypertension, cholesterol, diabetes mellitus, and smoking in driving disease has been well recognized at a population level and has been the target of primary prevention strategies for over 50 years with substantial impact. However, in many cases, these factors alone do not provide enough precision at the individual level to allow physicians and patients to take appropriate preventive measures and many patients continue to suffer acute coronary syndromes in the absence of these risk factors. Recent advances in user-friendly chip designs, high speed throughput, and economic efficiency of genome-wide association studies complemented by advances in statistical analytical approaches have facilitated the rapid development of polygenic risk scores (PRSs). The latest PRSs combine data regarding hundreds of thousands of single-nucleotide polymorphisms to predict chronic diseases including CAD. Novel CAD PRSs are strong predictors of risk and may have application, in a complementary manner with existing risk prediction algorithms. However, there remain substantial controversies, and ultimately, we need to move forward from observational studies to prospectively and rigorously assess the potential impact if widespread implementation is to be aspired to. Consideration needs to be made of ethnicity, sex, as well as age, and risk estimate based on existing non-genomic algorithms. We provide an overview and commentary on the important advances in deriving and validating PRSs, as well as pragmatic considerations that will be required for implementation of the new knowledge into clinical practice.
Article
A polygenic score (PGS) or polygenic risk score (PRS) is an estimate of an individual’s genetic liability to a trait or disease, calculated according to their genotype profile and relevant genome-wide association study (GWAS) data. While present PRSs typically explain only a small fraction of trait variance, their correlation with the single largest contributor to phenotypic variation—genetic liability—has led to the routine application of PRSs across biomedical research. Among a range of applications, PRSs are exploited to assess shared etiology between phenotypes, to evaluate the clinical utility of genetic data for complex disease and as part of experimental studies in which, for example, experiments are performed that compare outcomes (e.g., gene expression and cellular response to treatment) between individuals with low and high PRS values. As GWAS sample sizes increase and PRSs become more powerful, PRSs are set to play a key role in research and stratified medicine. However, despite the importance and growing application of PRSs, there are limited guidelines for performing PRS analyses, which can lead to inconsistency between studies and misinterpretation of results. Here, we provide detailed guidelines for performing and interpreting PRS analyses. We outline standard quality control steps, discuss different methods for the calculation of PRSs, provide an introductory online tutorial, highlight common misconceptions relating to PRS results, offer recommendations for best practice and discuss future challenges.
Article
Importance Polygenic risk scores comprising millions of single-nucleotide polymorphisms (SNPs) could be useful for population-wide coronary heart disease (CHD) screening. Objective To determine whether a polygenic risk score improves prediction of CHD compared with a guideline-recommended clinical risk equation. Design, Setting, and Participants A retrospective cohort study of the predictive accuracy of a previously validated polygenic risk score was assessed among 4847 adults of white European ancestry, aged 45 through 79 years, participating in the Atherosclerosis Risk in Communities (ARIC) study and 2390 participating in the Multi-Ethnic Study of Atherosclerosis (MESA) from 1996 through December 31, 2015, the final day of follow-up. The performance of the polygenic risk score was compared with that of the 2013 American College of Cardiology and American Heart Association pooled cohort equations. Exposures Genetic risk was computed for each participant by summing the product of the weights and allele dosage across 6 630 149 SNPs. Weights were based on an international genome-wide association study. Main Outcomes and Measures Prediction of 10-year first CHD events (including myocardial infarctions, fatal coronary events, silent infarctions, revascularization procedures, or resuscitated cardiac arrest) assessed using measures of model discrimination, calibration, and net reclassification improvement (NRI). Results The study population included 4847 adults from the ARIC study (mean [SD] age, 62.9 [5.6] years; 56.4% women) and 2390 adults from the MESA cohort (mean [SD] age, 61.8 [9.6] years; 52.2% women). Incident CHD events occurred in 696 participants (14.4%) and 227 participants (9.5%), respectively, over median follow-up of 15.5 years (interquartile range [IQR], 6.3 years) and 14.2 (IQR, 2.5 years) years. The polygenic risk score was significantly associated with 10-year CHD incidence in ARIC with hazard ratios per SD increment of 1.24 (95% CI, 1.15 to 1.34) and in MESA, 1.38 (95% CI, 1.21 to 1.58). Addition of the polygenic risk score to the pooled cohort equations did not significantly increase the C statistic in either cohort (ARIC, change in C statistic, −0.001; 95% CI, −0.009 to 0.006; MESA, 0.021; 95% CI, −0.0004 to 0.043). At the 10-year risk threshold of 7.5%, the addition of the polygenic risk score to the pooled cohort equations did not provide significant improvement in reclassification in either ARIC (NRI, 0.018, 95% CI, −0.012 to 0.036) or MESA (NRI, 0.001, 95% CI, −0.038 to 0.076). The polygenic risk score did not significantly improve calibration in either cohort. Conclusions and Relevance In this analysis of 2 cohorts of US adults, the polygenic risk score was associated with incident coronary heart disease events but did not significantly improve discrimination, calibration, or risk reclassification compared with conventional predictors. These findings suggest that a polygenic risk score may not enhance risk prediction in a general, white middle-aged population.
Article
Importance: The incremental value of polygenic risk scores in addition to well-established risk prediction models for coronary artery disease (CAD) is uncertain. Objective: To examine whether a polygenic risk score for CAD improves risk prediction beyond pooled cohort equations. Design, Setting, and Participants: Observational study of UK Biobank participants enrolled from 2006 to 2010. A case-control sample of 15 947 prevalent CAD cases and equal number of age and sex frequency–matched controls was used to optimize the predictive performance of a polygenic risk score for CAD based on summary statistics from published genome-wide association studies. A separate cohort of 352 660 individuals (with follow-up to 2017) was used to evaluate the predictive accuracy of the polygenic risk score, pooled cohort equations, and both combined for incident CAD. Exposures: Polygenic risk score for CAD, pooled cohort equations, and both combined. Main Outcomes and Measures: CAD (myocardial infarction and its related sequelae). Discrimination, calibration, and reclassification using a risk threshold of 7.5% were assessed. Results: In the cohort of 352 660 participants (mean age, 55.9 years; 205 297 women [58.2%]) used to evaluate the predictive accuracy of the examined models, there were 6272 incident CAD events over a median of 8 years of follow-up. CAD discrimination for polygenic risk score, pooled cohort equations, and both combined resulted in C statistics of 0.61 (95% CI, 0.60 to 0.62), 0.76 (95% CI, 0.75 to 0.77), and 0.78 (95% CI, 0.77 to 0.79), respectively. The change in C statistic between the latter 2 models was 0.02 (95% CI, 0.01 to 0.03). Calibration of the models showed overestimation of risk by pooled cohort equations, which was corrected after recalibration. Using a risk threshold of 7.5%, addition of the polygenic risk score to pooled cohort equations resulted in a net reclassification improvement of 4.4% (95% CI, 3.5% to 5.3%) for cases and −0.4% (95% CI, −0.5% to −0.4%) for noncases (overall net reclassification improvement, 4.0% [95% CI, 3.1% to 4.9%]). Conclusions and Relevance: The addition of a polygenic risk score for CAD to pooled cohort equations was associated with a statistically significant, yet modest, improvement in the predictive accuracy for incident CAD and improved risk stratification for only a small proportion of individuals. The use of genetic information over the pooled cohort equations model warrants further investigation before clinical implementation.