Using structural bioinformatics to investigate the impact of non synonymous SNPs and disease mutations: scope and limitations.

Abstract

Linking structural effects of mutations to functional outcomes is a major issue in structural bioinformatics, and many tools and studies have shown that specific structural properties such as stability and residue burial can be used to distinguish neutral variations and disease associated mutations.
We have investigated 39 structural properties on a set of SNPs and disease mutations from the Uniprot Knowledge Base that could be mapped on high quality crystal structures and show that none of these properties can be used as a sole classification criterion to separate the two data sets. Furthermore, we have reviewed the annotation process from mutation to result and identified the liabilities in each step.
Although excellent annotation results of various research groups underline the great potential of using structural bioinformatics to investigate the mechanisms underlying disease, the interpretation of such annotations cannot always be extrapolated to proteome wide variation studies. Difficulties for large-scale studies can be found both on the technical level, i.e. the scarcity of data and the incompleteness of the structural tool suites, and on the conceptual level, i.e. the correct interpretation of the results in a cellular context.

Full text

Page 1

BioMed Central
Page 1 of 10
(page number not for citation purposes)
BMC Bioinformatics
Open Access
Research
Using structural bioinformatics to investigate the impact of non
synonymous SNPs and disease mutations: scope and limitations
Joke Reumers, Joost Schymkowitz and Fréderic Rousseau*
Address: Switch Laboratory, VIB, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium
Email: Joke Reumers - joke.reumers@switch.vib-vub.be; Joost Schymkowitz - joost.schymkowitz@switch.vib-vub.be;
Fréderic Rousseau* - frederic.rousseau@switch.vib-vub.be
* Corresponding author
Abstract
Background: Linking structural effects of mutations to functional outcomes is a major issue in
structural bioinformatics, and many tools and studies have shown that specific structural properties
such as stability and residue burial can be used to distinguish neutral variations and disease
associated mutations.
Results: We have investigated 39 structural properties on a set of SNPs and disease mutations
from the Uniprot Knowledge Base that could be mapped on high quality crystal structures and
show that none of these properties can be used as a sole classification criterion to separate the
two data sets. Furthermore, we have reviewed the annotation process from mutation to result and
identified the liabilities in each step.
Conclusion: Although excellent annotation results of various research groups underline the great
potential of using structural bioinformatics to investigate the mechanisms underlying disease, the
interpretation of such annotations cannot always be extrapolated to proteome wide variation
studies. Difficulties for large-scale studies can be found both on the technical level, i.e. the scarcity
of data and the incompleteness of the structural tool suites, and on the conceptual level, i.e. the
correct interpretation of the results in a cellular context.
Background
The molecular phenotype of a coding non synonymous
SNP or disease associated mutation describes the func-
tional and structural properties of a protein that are
affected by a single amino acid substitution [1]. In this
study we want to address whether the concept of the in sil-
ico determined molecular phenotype can be employed for
large-scale classification of SNPs and disease mutations.
The attempt to classify a large set of mutations based on
an incomplete molecular phenotype may seem naive at
first glance, had it not been suggested that individual
properties such as protein stability, the accessibility of the
amino acid substitution site, and the location of variants
in surface pockets are predictive determinants of the phe-
notypic effect of a variation [2-5]. A comparative study of
protein stability predictors by Blundell and co-workers
from ECCB 2008 Workshop: Annotations, interpretation and management of mutations (AIMM)
Cagliari, Italy. 22 September 2008
Published: 27 August 2009
BMC Bioinformatics 2009, 10(Suppl 8):S9doi:10.1186/1471-2105-10-S8-S9
<supplement> <title> <p>Proceedings of the European Conference on Computational Biology (ECCB) 2008 Workshop: Annotations, interpretation and management of mutations (AIMM)</p> </title> <editor>Christopher JO Baker and Dietrich Rebholz-Schuhmann</editor> <note>Research</note> <url>http://www.biomedcentral.com/content/pdf/1471-2105-10-S8-info.pdf</url> </supplement>
This article is available from: http://www.biomedcentral.com/1471-2105/10/S8/S9
© 2009 Reumers et al; licensee BioMed Central Ltd.
This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0),
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Page 2

BMC Bioinformatics 2009, 10(Suppl 8):S9 http://www.biomedcentral.com/1471-2105/10/S8/S9
Page 2 of 10
(page number not for citation purposes)
demonstrated that although protein stability changes
caused by mutation can be relatively accurately estimated
in silico, these predictions by themselves do not yield accu-
racy on large-scale classification between benign and dis-
ruptive mutations [6-8].
Furthermore, computational analyses rely heavily on the
quality of the data under scrutiny and the computational
methods used to evaluate these data. Before investigating
39 structural properties of proteins and amino acid substi-
tutions for their predictive power regarding SNP classifica-
tion, we have investigated what major liabilities are
encountered when implementing an structural approach
to SNP annotation and classification. The results are com-
pared with those achieved by the best performers among
the state-of-the-art tools.
Results and discussion
In this study we have identified the common issues that
are encountered when performing large-scale analyses of
structural properties of human coding variation. The first
issue concerns the availability of structural data for nsS-
NPs and disease mutations, while the second involves the
availability of computational tools to predict structural
properties. The last issue concerns the quality of classifica-
tion: are the training and evaluation data sets used in the
analyses sufficient to extrapolate results for larger studies,
and do the properties used have sufficient predictive
power to separate the two data sets?
Structural coverage of human genetic variation
Despite structural genomics projects, the gap between
sequence and structural information is still wide, and the
coverage of variation data with structural data is estimated
to be as low as 14% [5]. We have investigated the bound-
aries of structural coverage by varying the quality require-
ments on the structural model (Supplementary Figure S1A
in Additional file 1), the sequence identity between query
sequence and modelled structure (Figure S1B), the per-
centage of the wild type sequence covered by the structural
model (Figure S1C), and the length of the alignment
between query and target (Figure S1D). Circa 12% of all
nsSNPs present in the Ensembl Variation Database
(release 44) can be mapped on a structural model, in
accordance with the estimate cited previously. However,
this percentage is valid only when no restrictions regard-
ing sequence identity, sequence coverage or structure
quality are applied.
In Figure S1A we see that the number of SNPs covered by
structural data drops after 40% sequence identity.
Requirements on sequence identity sufficient for predic-
tion are different for various methods. Yue & Moult [5]
found a sequence identity of 40% sufficient for accurate
prediction, while Chasman & Adams [2] obtained the best
results with identities higher than 60%. However, these
methods do not use full atomic detail to assess the struc-
tural properties of an amino acid substitution, and thus to
do not require high sequence identity to be able to model
the substitution. We use the FoldX force field to model
amino acid variation on structural models, which uses an
all-atom representation of the structure. Although this
introduces high accuracy of stability estimation [7,9,10],
it also requires high quality structural models. Our stand-
ard restrictions on building high-confidence structural
models using the FoldX force field are X-ray structures
with a resolution lower than 2.5 Å and sequence identity
higher than 80%. Applying these restrictions to the
Ensembl data results in a data set of 5416 nsSNPs (circa
4% of the data, Figure S1B).
Other factors in fluencing the structural coverage of SNPs
is the length of the alignment and the percentage of cov-
erage between the query sequence and the structural
model. A realistic criterion for human proteins to apply
would be to request the structural alignment to be about
100 amino acids long, or, for proteins shorter than 100
residues, to cover more than 80% of the query sequence.
When this criteria are combined with the need for high
quality structural data, we find that 8238 nsSNPs remain
in the data set. A summary of the number of SNPs covered
by high quality structural data, in combination with crite-
ria regarding the reliability of the nsSNP data, is shown in
Table 1. In this table we see that the application of strin-
gent criteria will result in the structural mapping of very
few nsSNP data.
Predictability of structural properties
The second issue for a large-scale structural bioinformatics
approach is the structural properties that are predictable
with state of the art tools: how well can we describe the
structural behaviour of a protein and its mutants? Previ-
ous structural studies have identified protein stability,
aggregation and misfolding as determinants of correct
functioning on the single protein level [8,11,12]. Muta-
tions affecting the functional sites of a protein, such as
DNA, ligand and protein interaction sites, are not consid-
ered within this scope, but the investigation of these sites
will most certainly be of great importance to assess the
impact of amino acid substitutions.
Tools have been developed that describe the structure and
dynamics of a protein: stability, aggregation, amyloidosis,
and folding. We have used computational methods that
are capable of assessing the effects of a mutation on pro-
tein stability (FoldX), aggregation (Tango) and amyloido-
sis (Waltz). Although algorithms exist that can predict
folding of small single domain proteins (e.g. Rosetta [13],
FoldX [14], SimFold [15]), to date no computational
method exists that can predict folding events on large

Page 3

BMC Bioinformatics 2009, 10(Suppl 8):S9 http://www.biomedcentral.com/1471-2105/10/S8/S9
Page 3 of 10
(page number not for citation purposes)
multi-domain proteins, or that is applicable in genome
wide studies.
Although we have limited ourselves to the analysis of
structural features of single protein molecules, and have
not investigated protein-protein interactions in this study,
we have included an analysis of the binding of proteins to
molecular chaperones, as it is directly related to correct
folding of the protein. The high abundance of chaperones
in the cell emphasises their crucial role in the protein
quality control system [16], but this is not reflected in the
availability of computational tools for chaperone bind-
ing. We have used the only available tool, the Hsp70
binding predictor Limbo [17], to assess chaperone bind-
ing variation caused by amino acid alteration.
The predictive power of structural properties
Following the recommendations of Care et al [18], we
have used the SwissProt annotated disease and polymor-
phism data (SwissProt Variation Index release 52) as the
evaluation data for our analyses. Mapping of these vari-
ants on high quality structural models (X-ray structures
with resolution  2.5 Å, sequence identity with the model
above 80%) yielded a data set of 240 positive (disease-
associated) mutations and 400 negative variations (neu-
tral nsSNPs) in 98 proteins. To ensure that the analyses are
comparable, we applied the sequence based predictors to
the same small data set as the predictors that use 3D struc-
tures or structural models.
Before we evaluated the discriminative power of the indi-
vidual structural parameters, we wanted to assess whether
our data showed distinguishable patterns for three impor-
tant parameters. The first two criteria, stability difference
and the degree of burial of the mutation site, have previ-
ously been identified as providing information about the
severity of a mutation [5,19]. The third criterion is differ-
ence in aggregation propensity, which has been cited as
likely to be an important factor in disease susceptibility
[12,20] but thus far has not been applied in a proteome
wide mutation analysis.
Figure 1 shows the distributions for the stability differ-
ences (A) and differences in aggregation propensity (B)
between wild type and variant proteins, and the burial of
the mutation site (C). The first observation of both the sta-
bility and the aggregation analysis is that the observed
changes are not discrete but follow a smooth distribution
from negative to positive change. Second, there are notice-
able differences between SNPs and disease mutations, but
they cannot be distinguished by a simple cut-off value on
the output, as there is large overlap between the distribu-
tions. This is confirmed by the P-values obtained from
paired student t-tests, which are 0.96 for the stability dis-
tributions, 0.99 for the aggregation distributions, and
0.99 for the burial distributions, respectively. For the sta-
bility distributions, we see that disease mutations are gen-
erally more destabilising than SNPs, but their
distributions overlap largely. A similar analysis has been
performed on SwissProt variants using the Site Directed
Mutator stability predictor [8], and the distributions of
stability differences of disease mutations and neutral var-
iations resemble our findings.
In a first series of properties to test as classifiers, we have
investigated 13 properties of the amino acid substitution
site that contribute to the assessment of the effect of the
mutation using the FoldX algorithm (Table 2). Cut off val-
ues were generated that varied between the minimal and
maximal values measure for the specific property, and the
true and false positive rate, and the Matthews correlation
coefficient (MCC) were calculated for each cut-off value.
Table 2 lists the data for both the best MCC and the
MCC90, i.e. the coefficient that is measured at high specif-
icity (true negative rate = 90%). The corresponding ROC
Table 1: Summary of structural coverage of SNP data.
Properties # SNPs% SNPs
nsSNPs covered by high quality structural data
No additional criteria
Sequence coverage > 80 or alignment length > 100
Sequence identity > 80
Sequence coverage > 80 or alignment length > 100, and sequence identity > 80
Highly reliable nsSNPs covered by high quality structural data
Doublehit validation status, MAF > 0.01
Doublehit validation status, MAF > 0.01, sequence identity > 80
Doublehit validation status, MAF > 0.01, sequence coverage > 80 or alignment length > 100
Doublehit validation status, MAF > 0.01, sequence coverage > 80 or alignment length > 100, and sequence identity > 80
9877
8238
5416
5318
7.4
6.2
4.1
4.0
680
229
446
209
0.51
0.17
0.33
0.16
Several criteria resulting from the above analyses are applied to assess the structural coverage and reliability of that coverage of human SNPs in the
Ensembl database, as well as the overlap of the structural coverage with quality parameters for the validation and frequency status of the
polymorphism data.

Page 4

BMC Bioinformatics 2009, 10(Suppl 8):S9 http://www.biomedcentral.com/1471-2105/10/S8/S9
Page 4 of 10
(page number not for citation purposes)
Distributions for the major structural criteria in the disease and polymorphism datasets
Figure 1
Distributions for the major structural criteria in the disease and polymorphism datasets. White = disease muta-
tions, grey = polymorphisms. A. Stability difference as calculated by the FoldX force field (in kcal.mol-1). B. Difference in aggre-
gation propensity as calculated by the Tango algorithm. Values close to neutral changes (in the range [-50, 50]) are left out for
display purposes. C. Distribution of degree of burial of the amino acid substitution site.
0
5
10
15
20
0 0.10.2 0.3 0.4 0.50.6 0.70.80.91
Side chain burial (ratio)
% of variations
0
5
10
15
20
25
30
-10-9 -8 -7-6-5-4-3 -2-10123456789 10.
% of variations
FoldX ??G
0
0.2
0.4
0.6
0.8
1
1.2
-1250 -1000 -750-500-250-100-50050 100 2505007501000 1250.
Tango score difference
% of variations
A
B
C

Page 5

BMC Bioinformatics 2009, 10(Suppl 8):S9 http://www.biomedcentral.com/1471-2105/10/S8/S9
Page 5 of 10
(page number not for citation purposes)
curves for these analyses can be found in Supplementary
Figure S2 in Additional file 1.
The same strategy was then applied to predicted values of
structural differences between mutant and wild type pro-
teins (24 properties). Statistics were calculated for stability
and entropy parameters, as well as for differences concern-
ing protein aggregation, amyloidosis and chaperone bind-
ing (Table 3, Supplementary Figure S3 in Additional file
1).
The results obtained from these detailed analyses are
unanimous: none of the parameters evaluated can be used
to separate the data. All MCC values are close to zero, and
thus the predictions are no better than a random predictor
would perform on the data. The high accuracy of FoldX
for quantitative stability prediction has been proven in
various studies [7,9,10], so we have high confidence in
our stability estimations. In accordance with the analyses
of Worth and co-workers [8], we find that high stability
differences alone are no sufficient criterion to distinguish
deleterious mutations and neutral variation. These results
show that the dominant effect of for instance stability that
was proposed in earlier large-scale studies [5,21] can not
always be generalised for other data.
The fact that none of the properties representing confor-
mational differences between wild type and variant pro-
tein contain enough information to distinguish neutral
and deleterious variation implies that large-scale classifi-
cation based on singular structural properties is not feasi-
ble and requires a better understanding of how the
complex interplay between biophysical and biochemical
properties of a protein conspire to different tolerance for
mutations in different proteins. Although we can predict
the structural and functional impact of a mutation of a
protein, it is not always feasible to translate this into a pre-
diction of the overall phenotypic effect, i.e. will the muta-
tion result in a disease phenotype or not.
Recent studies that combine structural and evolutionary
information using machine learning techniques are able
to classify relatively large data sets obtained for the Swiss-
Prot database successfully (summarised in Supplementary
Table S2). Although the combination of these two types of
information improves classification of disease mutations
greatly, the incorporation of sequence conservation meas-
ures may obscure the mechanism underlying disease. Low
frequency substitutions at conserved positions suggest
that the mutation will not be tolerated, but will not teach
us what the underlying reason of disease is. Although
knowing that an amino acid is critical for correct function
is of course useful, in a structural bioinformatics approach
the focus is more on the molecular mechanism underly-
ing disease.
A simple combination of the SNPeffect structural bioin-
formatics toolsuite on our evaluation data set showed that
in our case, at least a linear combination of these methods
is not sufficient to classify the data (TPR = 0.73, TNR =
Table 2: Predictive power of structural properties of the modeled variant proteins.
PropertyFPRTPRBest MCC ThresholdMCC90
FoldX energy evaluation
Overall stability of residue
Backbone H bond
Sidechain H bond
Electrostatics
Entropy side chain
Entropy main chain
Van der Waals contribution
Solvation hydrophobic
Solvation polar
Van der Waals clash
Side chain burial
Main chain burial
Entropy by sampling of possible side chain conformations
Entropy side chain
14
32
99
86
59
13
25
10
42
18
51
59
33
72
100
93
80
27
47
22
70
33
67
83
0.22
0.40
0.07
0.11
0.22
0.18
0.23
0.16
0.28
0.17
0.16
0.26
1.61
-1.05
-1.76
-0.10
0.32
1.96
-0.98
-0.6
1.5
0.22
0.43
0.73
0.19
0.22
<0
-0.01
0.05
0.10
0.15
0.16
0.06
0.15
-0.1
0.05
72 840.150.930
The false positive rate (FPR = 1 - specificity) and the true positive rate (TPR = sensitivity) for the threshold on the specific property that gave the
best Matthews correlation coefficient (MCC) are shown. MCC90 is the Matthews correlation coefficient for a specificity of 90% (i.e. 10% false
positive rate). The ROC curves corresponding with the evaluation of all properties can be found in Supplementary Figure S2 in Additional file 1.
FoldX was used to evaluate both the overall stability contribution of the amino acid substitution site in the modeled structure and the various
factors involved in this stability. The entropy of the variant amino acid was calculated using a sampling strategy to assess the possible side chain
conformations allowed at the substitution site. Both stability and entropy were calculated for all mutations and for a subset of buried mutations
(side chain burial < 0.5) and surface mutations (side chain burial  0.5). Corresponding ROC curves are shown in Supplementary Figure S3 in
Additional file