ArticlePDF Available

An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks and other empirical models

Authors:

Abstract and Figures

Sensitivity analysis applied to Artificial Neural Networks (ANNs) as well as to other types of empirical ecological models allows assessing the importance of environmental predictive variables in affecting species distribution or other target variables. However, approaches that only consider values of the environmental variables that are likely to be observed in real-world conditions, given the underlying ecological relationships with other variables, have not yet been proposed. Here, a constrained sensitivity analysis procedure is presented, which evaluates the importance of the environmental variables considering only their plausible changes, thereby exploring only ecological meaningful scenarios. To demonstrate the procedure, we applied it to an ANN model predicting fish species richness, as identifying relationships between environmental variables and fish species occurrence in river ecosystems is a recurring topic in freshwater ecology. Results showed that several environmental variables played a less relevant role in driving the model output when that sensitivity analysis allowed them to vary only within an ecologically meaningful range of values, i.e. avoiding values that the model would never handle in its practical applications. By comparing percent changes in MSE between constrained and unconstrained sensitivity analysis, the relative importance of environmental variables was found to be different, with habitat descriptors and urbanization factors that played a more relevant role according to the constrained procedure. The ecologically constrained procedure can be applied to any sensitivity analysis method for ANNs, but obviously it can also be applied to other types of empirical ecological models.
Content may be subject to copyright.
RESEARCH ARTICLE
An ecologically constrained procedure for
sensitivity analysis of Artificial Neural
Networks and other empirical models
Simone FranceschiniID
1
*, Lorenzo Tancioni
1
, Massimo LorenzoniID
2
, Francesco Mattei
1
,
Michele Scardi
1
1Department of Biology, University of Rome Tor Vergata, Rome, Italy, 2Department of Chemistry, Biology
and Biotechnology, Perugia, Italy
*smn.franceschini@gmail.com
Abstract
Sensitivity analysis applied to Artificial Neural Networks (ANNs) as well as to other types of
empirical ecological models allows assessing the importance of environmental predictive
variables in affecting species distribution or other target variables. However, approaches
that only consider values of the environmental variables that are likely to be observed in
real-world conditions, given the underlying ecological relationships with other variables,
have not yet been proposed. Here, a constrained sensitivity analysis procedure is pre-
sented, which evaluates the importance of the environmental variables considering only
their plausible changes, thereby exploring only ecological meaningful scenarios. To demon-
strate the procedure, we applied it to an ANN model predicting fish species richness, as
identifying relationships between environmental variables and fish species occurrence in
river ecosystems is a recurring topic in freshwater ecology. Results showed that several
environmental variables played a less relevant role in driving the model output when that
sensitivity analysis allowed them to vary only within an ecologically meaningful range of val-
ues, i.e. avoiding values that the model would never handle in its practical applications. By
comparing percent changes in MSE between constrained and unconstrained sensitivity
analysis, the relative importance of environmental variables was found to be different, with
habitat descriptors and urbanization factors that played a more relevant role according to
the constrained procedure. The ecologically constrained procedure can be applied to any
sensitivity analysis method for ANNs, but obviously it can also be applied to other types of
empirical ecological models.
1. Introduction
Fish assemblage diversity in freshwater ecosystems constitutes a valuable natural resource in
economic, scientific, cultural and educational terms [1]. Its conservation and management
face threats as overexploitation of inland waters, flow modification, water pollution, habitat
degradation and invasion by exotic species [2], [3]. Identifying the relationships between fish
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 1 / 15
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
OPEN ACCESS
Citation: Franceschini S, Tancioni L, Lorenzoni M,
Mattei F, Scardi M (2019) An ecologically
constrained procedure for sensitivity analysis of
Artificial Neural Networks and other empirical
models. PLoS ONE 14(1): e0211445. https://doi.
org/10.1371/journal.pone.0211445
Editor: Thilo Gross, University Of Bristol, UNITED
KINGDOM
Received: October 23, 2017
Accepted: January 15, 2019
Published: January 30, 2019
Copyright: ©2019 Franceschini et al. This is an
open access article distributed under the terms of
the Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
Data Availability Statement: All relevant data are
within the paper and its Supporting Information
files.
Funding: The author(s) received no specific
funding for this work.
Competing interests: The authors have declared
that no competing interests exist.
species richness and habitat complexity at a local scale is one of the primary concerns in under-
standing how environmental descriptors actually affect fish biodiversity [4], [5], [6].
In this respect, the ecological variables that can be taken into account are often character-
ized by complex and non-linear dependencies [7]. Ecological models have been increasingly
applied in the management and conservation of freshwater fish communities, especially to pre-
dict spatial patterns of fish occurrence [8], [9]. In particular, Artificial Neural Networks
(ANNs) modeling has proved to be a valuable method in order to assess whether predictable
relationship between environmental descriptors and fish species richness exist in small stream
environments [10], [11], [12].
While in the past ANNs were defined as “black boxes” since the computational processes
taking place inside them are not easy to untangle, at present several methodologies have been
developed to assess the contribution of each variable to the prediction process. For deeper elu-
cidations, Olden et al. [13] provided a comprehensive review and comparison of these
methodologies.
In particular, sensitivity analysis is the term used to define a collection of methods that
evaluate how sensitive model output is to changes in the values of predictive variables [14]. In
ecology, the main sensitivity analysis methods applied to ANNs can be classified into four cate-
gories: (i) the Lek’s profiles method [15], [16]; (ii) the Perturbation method [17], [18]; (iii) the
Partial Derivatives method [19], [20], [21], [22]; (iv) the Weights method, developed by Gar-
son [23] and then implemented by Olden & Jackson [24]. Lek’s profiles study each input vari-
able by keeping all other parameters at fixed values, while in Perturbation method each input
variable is perturbed according to empirically established ranges while all others are kept
untouched. The Partial Derivatives method involves small changes in each input variable and
the evaluation of their relative contribution by computing the partial derivatives of the ANN
output with respect to changes in the input. In the Weights method the connection weights of
the ANN model are partitioned to evaluate the relative importance of each input variable and
its positive or negative contribution to the model output. In the application of the first three
methods, the values assigned to input variables can be devoid of real ecological meaning, i.e.
they can be out of the range that is likely to be observed in real-world conditions. In these
cases, environmental variables are forced to values that are only aimed at evaluating the model
output, with no attention to the actual probability of recording those values given the (fixed)
values of all the other variables. In fact, while of course the above-mentioned methods may
provide valuable information about the way the “black-box” model works, the role of ecologi-
cal relationships in constraining the multidimensional space where meaningful data patterns
exist is not fully taken into account. With regard to the Weights method instead, the estimation
of the input variables importance based on the connection weights may result unbalanced in
certain cases where constrained training procedure may be applied to the ANN model for opti-
mization purposes [25] (NB: in this sentence the term constrained is referred to the training
procedure developed by Scardi [25] and it has nothing to do with the constrained perturbation
of input variables here illustrated).
Therefore, although all those methodologies proved to be means of determining the overall
numerical influence of each predictor variable to the model output, approaches that only con-
sider changes consistent with the ecological relationships among environmental variables have
never been proposed. It is well known in ecology that most environmental variables are far
from independent of each other [26], [27]and therefore not all the combinations of their values
are likely to occur (e.g. river slope tends to increase with elevation, as does the water oxygen
concentration, and cannot be very steep in a floodplain). As these relationships constrain each
variable in the complex multidimensional space that represents the abiotic conditions found in
an ecosystem, some combinations of values are more easily found, while others just cannot
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 2 / 15
occur. In fact, for instance, it would be highly unlikely for the maximum width of a stream
channel to occur in a headwaters reach.
These issues raise the question: what is the point of perturbing or fixing variables at values
which are ecologically meaningless? Evaluating the model output response in areas of the mul-
tidimensional space where environmental descriptors take far-fetched values may not be useful
from an ecological perspective. Indeed it would make more sense to evaluate how sensitive
model output is to changes in predictive variables values taking into account only plausible
perturbations, i.e. changes which are consistent with the ecological relationships between envi-
ronmental variables.
This study demonstrates an example of a new type of sensitivity analysis, using a case study
about an ANN model aimed at predicting fish species richness in central Italian rivers. The
goal of this work is to evaluate the real contribution of each predictive variable to species rich-
ness estimates by taking into full account the underlying ecological relationships and con-
straints. This way, all the perturbations applied to predictive variables reflect plausible
environmental conditions, thus evaluating shifts in fish species richness only among ecological
meaningful scenarios.
2. Material and methods
2.1. Study area and data collection
Data have been obtained from 368 sites that have been sampled from 2009 to 2014 in central
Italy [28], [29] (Fig 1). Most rivers in this area are characterized by a Mediterranean climate,
hydrological regimes affected by rainfall variability and strong seasonal discharge variation,
with high flows in spring and fall, and droughts in summer [30].
Fish sampling and environmental data acquisition were carried out according to the official
Italian sampling protocol [31]. It generally consists of electrofishing sampling using a standard
electro-fish shoulder-bag (4KW, 0.3–6 Ampere, 150–600 Volt). All available habitats were
sampled along a stream channel 40–70 m long (the transect length was about 20 times the
width of the wetted channel). Field activities were carried out beyond parks or protected areas.
No endangered or protected species were involved and no specimen were harmed during the
study nor collected. The occurrence of 55 fish species and values for 27 environmental vari-
ables (Table 1) were recorded at each site during sampling activities. Most of these variables
had been already considered in previous studies [9], [32], [33].
Channel width was always less than 20 m, since sample sites were primarily located within
foothills and mountain zones. Thus, sampling methods (electrofishing) was standardized
across sites, where wider river widths would have required nets or other gears.
2.2. Data set processing
All quantitative or semi-quantitative environmental data were normalized in the [0, 1], interval
while qualitative data (e.g. wetlands or islands presence) were coded as binary values (0–1).
Data normalization is a common procedure in ANNs model development [16], [17], since it
transposes the predictive input variables into the data range on which sigmoid activation func-
tions are based, thereby helping to approach to global minima at the error surface. As very
steep slopes were only observed at two sites (13.4% and 23.4% respectively), slope data were
normalized, omitting these two values, relative to third steepest slope value (9%). The maxi-
mum normalized value, i.e. 1, was assigned these outliers after normalization. This solution
was adopted to prevent the compression of the normalized slope values into a very narrow
range because of a couple of cases that cannot be regarded as part of a continuum. Species rich-
ness values were also normalized in the [0, 1] interval.
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 3 / 15
The whole data set was divided into three subsets (i.e. training, validation and test). The
training set included 50% of records, while validation and test set included both 25% of rec-
ords. Records were assigned to each subset by sorting all data according to ascending values of
fish species richness and by dividing the resulting ordered sequence into groups of four rec-
ords. Then the first and third record in each group of four records were assigned to the train-
ing set, while the second assigned to the validation test and the fourth to the test set. This
procedure allowed to avoid unbalanced levels of species richness in the three data subsets.
2.3. Artificial neural network modeling
In this study, a three-layered feedforward network with bias has been trained in order to predict
species richness. The optimal number of neurons in the hidden layer was determined by com-
paring the performance of different networks with 1 to 30 hidden neurons. A sigmoid transfer
function was used both for hidden and output layers, thus enabling the network to learn non-
linear relationships between input and output vectors [34]. Mean Square Error (MSE) was com-
puted for the validation set to quantify the goodness of fit of the ANNs during training. The
training procedure was terminated as soon as the MSE stopped decreasing monotonically, thus
preventing the overtraining of the model during the learning process. This approach favors bet-
ter generalization of ANN models while predicting new cases, as previously described in several
ecological papers [25], [26]. Several values of learning rate and momentum (range 0.1–0.5) were
tested to optimize learning performances. ANNs training and testing were performed in R envi-
ronment [35] by using the functions of the package h2o [36].
Fig 1. Sampling sites. Elevation map of the river basins of latium and umbria administrative regions in central Italy.
Black dots mark the position of sampling sites. The image was obtained by using QGIS 2.18 (http://www.qgis.org).
https://doi.org/10.1371/journal.pone.0211445.g001
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 4 / 15
2.4. Constrained sensitivity analysis
In order to use a sensitivity analysis aimed at perturbing environmental predictive variables in
an ecologically sound perspective, the dependencies between all environmental variables were
first investigated.
In particular, for each j
th
environmental variable, the following steps were performed:
A Euclidean distance matrix was computed between the test set observations taking into
account all the environmental variables but excluding the j
th
variable.
For the i
th
observation, neighboring observations were selected by taking those within the
first quartile of the (d
max
d
min
) distribution, where d
max
and d
min
were respectively the
maximum and the minimum distance between the i
th
observation and all other
observations.
The minimum (j
min
) and maximum (j
max
) values of the j
th
environmental variable were
selected within the neighboring, i.e. most similar, observations. This defined the range of
values that the j
th
variable can take for the i
th
observation.
The j
th
variable was perturbed in the [j
min,
j
max
] range while all other variables were kept
untouched.
Table 1. Environmental variables used as input to the ANN model. All environmental data have been obtained according to the official Italian sampling protocol [31].
Variable Label Min Max Mean Median
Slope (%) SLP 0 23.4 1.46 0.83
Channel width (m) CHW 0.8 20 6.04 4
Elevation (m) ELV 0 973 236.61 212.5
Depth (m) DET 0.05 20 0.49 0,35
Runs (area %) RUN 0 100 50.78 50
Pools (area %) POL 0 100 23.99 20
Riffles (area %) RIF 0 100 24.32 15
Wetlands (0/1) WEL 0 1 0.07 0
Bars & islands (0/1) BAS 0 1 0.05 0
Boulders (area %) BOL 0 70 9.86 0
Rocks & pebbles (area %) ROK 0 80 30.38 30
Gravel (area %) GRV 0 90 24.47 20
Sand (area %) SAD 0 80 20.20 20
Silt & clay (area %) SIT 0 100 15.17 0
Velocity (0–5) VEO 0 5 1.89 2
Vegetation cover (area %) VEC 0 90 13.84 10
Shade (area %) SHD 0 90 41.2 40
Anthropic disturbance (0–4) AND 0 4 2.36 2
Upstream barrier (Km, 0–100, 100 if no barrier) UPB 0.01 100 62.41 100
Downstream barrier (0/1) DOB 0 1 0.51 1
Upstream lake (Km, 0–50, 50 if no lake) UPL 0.2 50 45.85 50
Temperature (˚C) TEP 2.56 28.3 14.89 14.65
pH PHP 4.88 9.45 8.04 8.09
Conductivity (mS/cm) COD 229.2 1659 639.02 594.5
O2 (%) O2O 7.31 160 88.76 93.08
Source distance (km) SOD 0.01 233 21.49 11.99
Sampled area (m
2
) SAA 30 3500 530.35 400
https://doi.org/10.1371/journal.pone.0211445.t001
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 5 / 15
Five perturbed values in the [j
min,
j
max
] range for each predictive variable were then
passed to the data pattern fed to the ANN model, whose output was compared to the tar-
get (i.e. observed) fish species richness.
The same process was iterated for each observation (i.e. for each sampling site in the test
set).
The results of this constrained sensitivity analysis were then compared to those obtained
from simple input perturbation, i.e. by adding white noise in the [-0.5, 0.5] range to each input
variable while keeping all the others untouched.
The method was entirely implemented in R programming language [35]. An example code
is provided in the S1 File.
3. Results and discussion
3.1. Artificial neural network model
The best ANN architecture for predicting fish species richness on the basis of our environmen-
tal predictive variables had 8 hidden neurons and therefore a 27-8-1 structure. It explained a
fairly large share of variance, ranging from R
2
= 0.771 for the training/validation set to R
2
=
0.675 for the test set (Fig 2).
The MSE (obtained from normalized data) varied correspondingly: MSE = 0.00756 for the
training/validation set and MSE = 0.01001 for the test set. It seems that very low observed val-
ues of species richness are hardly reproduced by the model, possibly because the absence of
species that could have been found on the basis of their ecological niche might depend on
other factors (e.g. pressures not described by the available environmental variables) in species-
Fig 2. Predicted vs. observed species richness. Values on axes refer to normalized species richness. The determination coefficient for the ANN model was R
2
= 0.771
for the training/validation set and R
2
= 0.675 for the test set.
https://doi.org/10.1371/journal.pone.0211445.g002
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 6 / 15
poor situations. On the contrary, the highest values in the training set are slightly underesti-
mated, while they match the observed values in the test set. However, the overall agreement
between observed and predicted values is quite good with both data sets and is comparable to
the level obtained in similar cases [12], [37], [38].
The average residuals relative to the normalized training data set as well as those relative to
the normalized test set were very small (0.0017 and 0.0016, respectively), thus showing that the
model was not systematically biased. In fact, when compared to the test set data, model predic-
tions about species richness differed in only one species in 46% of the cases.
Although all the levels of species richness were included in both training and test data set,
the model was less accurate when the highest species richness values were involved. This effect
was most likely related to the difficulty of the ANN in identifying less frequent patterns (those
with high species richness in this case), as already evidenced by Ozesmi et al. [39], thereby
more easily leading to incorrect estimations. In fact, species richness values higher than 11
(normalized value = 0.631) were not frequently found, amounting to less than 5% of the whole
data set.
3.2. Sensitivity analysis
3.2.1. Constrained perturbations. All the methods for analyzing the sensitivity of ANNs
relative to predictive variables are based on the assessment of changes in output values
obtained as a consequence of known changes in input values. The procedure we present here
has been implemented by constraining the random permutation method [17], [18], but its
rationale (i.e. the same constraints) can be applied to any other method [21], [24].
In order to outline the differences between the way input data are perturbed by any uncon-
strained procedure and the way they are by our constrained approach, Fig 3 shows observed
(dark circle) and perturbed (light circle) values for three environmental variables (Slope, Riffles
and Conductivity) in scatter plots against elevation. Elevation is obviously not independent of
some environmental variables and constrains their values according to the procedure outlined
in section 2.4. In particular, in this example, constrained ranges are clearly visible on slope
(positively correlated to elevation) and conductivity (negatively correlated to elevation), while
perturbations of riffles values are very close to the maximum potential range in the ether upper
quartiles of the elevation range, as a consequence of a much looser dependence of this variable
from elevation.
Fig 3. Constrained perturbations for slope, riffles and conductivity vs. elevation values. Perturbed values were obtained by applying the procedure outlined in
section 2.4. The effect of the constraint is more evident for Slope and Conductivity, given their stronger dependence from elevation, than for Riffles, where it only limits
the variability at low Elevation. Both observed (dark dots) and perturbed (lighter dots) values are shown.
https://doi.org/10.1371/journal.pone.0211445.g003
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 7 / 15
The effect of random perturbations of slope and conductivity (i.e. complete independence
between variables) would have been to fill up all graphs, while points representing perturba-
tions of the two environmental variables occupy only a portion of the two-dimensional space,
thus showing that some combinations of values are very unlikely to be observed. Perturbations
showed in Fig 3 consist only of values that are more likely to be found in real-world conditions,
although their range is large enough to allow assessing their impact on model behavior. Fig 3 is
obviously depicting a very simplified set of relationships (only 3 out of 27 predictive variables).
In practice, however, the same concept was applied to an n-dimensional space, where nis the
number of environmental predictive variables used for the model development, thus defining
an n-dimensional envelope that constrains the random perturbation of each environmental
variable, excluding very unlikely patterns (e.g. very steep slope at very low elevation) from the
sensitivity analysis.
3.2.2. MSE percentages differences. The percent increase in MSE obtained by con-
strained perturbation of each variable for the test set is shown in Fig 4 versus the percent
increase obtained by unconstrained perturbation. Unconstrained perturbations obviously
induce larger increases in MSE, as they modify known data patterns to a larger extent.
Although ANNs may respond to changes in a single input variable in a non-monotonic way,
thus potentially making a large change in an input value less influential than a smaller one, in
practice larger changes in input variables are clearly associated with larger increases in MSE.
However, very large increases in MSE obtained from data patterns that are unlikely to occur in
practical applications of the model are not useful–and possibly misleading–when it comes to
the very purpose of sensitivity analysis, i.e. at inferring the role each input variable plays rela-
tive to the target variable.
While all the input variables are more sensitive to unconstrained perturbations, some show
negligible differences between the two perturbation strategies, while others exhibit sharp dif-
ferences. According to changes in MSE, the input variables that showed largest differences
between the two perturbation methods were Slope (101.1% and 17.1%, for unconstrained and
constrained perturbations, respectively), pH (57.8%; 9.3%), Source distance (52.7%; 14.5%)
and Sampled area (38.5%; 10.1%).
Variables whose perturbations affected the model to a very limited extent (less than 10%
increase in MSE), i.e. those in the lower left corner of Fig 4, do not deserve any further com-
ment, because they certainly play a less important role. Other variables, however, are associ-
ated with changes in MSE between 10% and 30% and their constrained perturbation in some
cases (e.g. Conductivity, Pools and Anthropic disturbance) induces changes in MSE almost as
large as unconstrained and even more than the constrained perturbation of the “most influen-
tial” unconstrained (Slope, pH, Source distance and Sampled area).
In ecology, it is well known that fish species composition in lotic ecosystems tends to follow
a typical longitudinal pattern [4] (i.e. differences in fish guilds occurrences and abundances)
and generally fish species richness generally tends to increase with the distance from the river
source [40]. Of course, there are field conditions that can be regarded as exceptions to this gen-
eral trend. In fact, habitat features [41], [42], hydrological factors [43] or urbanization [44]
may highly affect fish species diversity. It is clear that environmental variables like slope, pH or
distance from source may provide information about the riverine trait where a site to be mod-
eled is located (e.g. mountain or hilly region) [45], thereby providing valuable input informa-
tion to the ANN model about expected species richness and inducing large changes in MSE
when their values are perturbed. However, unconstrained perturbations, especially with those
variables, may result in combinations of values, e.g. a steep slope too far from the source, that
are unlikely or even impossible to occur in real-world situations, but that could trigger large
changes in MSE.
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 8 / 15
Sensitivity analysis based on unconstrained perturbations can be deeply affected by this
problem and the reason is that any model (and ANNs are no exception) is fitted to known data
patterns, which obviously include only the combination of input values that actually occur in
real-world situations. Extreme values may occur, but only in combination with a narrow range
of values for other variables. Moreover, environmental variables are often strongly correlated
with each other and their correlations make the range of ecologically meaningful variation in
their values even narrower. For instance, pH usually decreases as the distance from river
source increases, while conductivity increases [46]. These relationships make perturbations
for Slope, pH, Sampled area, Source distance and Elevation strictly related to the ecological
context, thereby defining a narrower, but more realistic range of values that can be safely
used in practical applications of the model. Therefore, the MSE increase associated to large
Fig 4. Percent increase in MSE obtained by constrained vs. unconstrained perturbations. Constrained sensitivity analysis clearly
reduces maximum perturbations for the environmental variables, thus resulting in smaller increases in MSE for all of them (all points
are below the unit slope line). However, the effect of the constraint is larger for some variables (e.g. Slope, SLP; pH, PHP; Source
distance, SOD; Sampled area, SAA). See Table 1 for the names of environmental variables corresponding to other point labels.
https://doi.org/10.1371/journal.pone.0211445.g004
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 9 / 15
perturbations of these variables has very little importance relative to real world applications of
the model.
3.2.3. Importance of the environmental variables. Changes in MSE after perturbation of
each environmental variable were sorted in decreasing order after the application of a conven-
tional scheme for sensitivity analysis and after the application of the constrained procedure.
The outcome relative to the unconstrained procedure can be regarded as a different and sim-
plified view relative to Fig 4. In fact, the bar diagram in Fig 5, just shows the increase in MSE
caused by the perturbation of each variable. On the left after unconstrained perturbation and
on the right after constrained perturbation. MSE% scales show the percent increase in MSE
and are different in the two cases, as constrained perturbation cannot induce a level of increase
in MSE as large as that induced by unconstrained perturbations.
In fact, all variables were obviously associated with smaller changes in MSE when the con-
strained procedure for sensitivity analysis was applied and the largest differences in the rank of
variable importance occurred for Slope, Conductivity, pH, Sampled area, Pools and Anthropic
disturbance, while less important environmental variables showed only minor shifts in their
relative importance. pH was one of the most important variables according to the conventional
Fig 5. Unconstrained and constrained sensitivity analysis compared. Bars show the percent increase in MSE caused by
the perturbation of each variable. Black bars (left) are for unconstrained perturbations while blue bars (right) are for
constrained ones. Environmental variables are ordered according to the rank of their importance in the unconstrained
sensitivity analysis. As the increase in percent MSE was smaller in constrained sensitivity analysis, the MSE axis was scaled
accordingly to better show the relative length of the bars.
https://doi.org/10.1371/journal.pone.0211445.g005
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 10 / 15
procedure of sensitivity analysis based on unconstrained variable perturbation, but it only
ranked eighth in sensitivity analysis based on constrained perturbations. Similar downgrades
in importance were also observed for Slope and Sampled area. They are not surprising, as they
occurred because of the narrower range of perturbed values these variables can assume under
the constrained procedure for sensitivity analysis. In fact, this procedure takes only into
account an amount of variability that is consistent with the observed relationships between
variables and with the environmental context of each data pattern. As a consequence, environ-
mental variables that had an intermediate relative importance according to the unconstrained
procedure (e.g. Conductivity, Pools and Anthropic disturbance), gained a more relevant role
as potential drivers of the local fish species richness.
While this result cannot be formally validated, as the true relative importance of the envi-
ronmental variables is obviously unknown, it demonstrated an important feature of the con-
strained sensitivity analysis. The unconstrained procedure suggested a ranking of variables
importance that showed what made the ANN model learn to recognize the riverine trait where
sampling sites are located, thus obtaining estimates for fish species richness. However, species
richness was also affected by variables that convey information about some relevant local con-
ditions, like habitat features, hydrologic factors or urbanization. As a matter of fact, several
studies evidenced that, at local scale, urbanization and/or flow regulation may strongly modify
the expected fish species richness [40], [47]. Results obtained from the constrained sensitivity
analysis showed indeed how, at any given site, fish species diversity is highly affected by envi-
ronmental factors as habitat descriptors (e.g. Pools; Bars & islands) and anthropic disturbance
(Conductivity; Anthropic disturbance). As conductivity can be considered as an indirect mea-
sure of water pollution [48], [49] and anthropic disturbance in most cases is related to urbani-
zation, it is reasonable that they had a strong impact on fish assemblage diversity and
composition.
In this work we focused on the estimation of variables importance taking into account first-
order effects, as one input variable at a time was perturbed, while all other variables were kept
untouched. Estimating the model output response to two-way [22] or more complex interac-
tions between variables is certainly feasible in a constrained sensitivity analysis, but the prob-
lems related to the complexity of the procedure remain unsolved, making the analysis of
higher order interactions between predictive variables practical only when their number is
very small.
A very common goal in ANN modeling is the reduction of the number of input variables.
The reason for that reduction is twofold: it might reduce the cost of predictive information
and it might help to fight the curse of dimensionality [50]. The first problem depends on the
way predictive information is collected: if all predictive data are already available, or if they are
collected with no additional costs, e.g. during the same field activities, then the overall cost of
predictive information will not be affected. The second problem is strictly related to the ratio
between the number of available records and the number of input variables. According to
Theodoridis & Koutroumbas [51], acceptable values for that ratio are in the 2 to 10 range, with
smaller values that might result in a reduced prediction ability of the model.
As our data set was already available and all the predictive variables are routinely included
in monitoring activities, no reduction in the cost of information could be achieved. Moreover,
the number of available records (N = 368) is quite large relative to the number of predictive
variables (p = 27) and therefore the ratio between the two (N/p = 13.63) is even larger than the
upper limit of the above-mentioned range. Therefore, reducing the number of input variables
was not needed, while preserving the full set allowed testing the constrained sensitivity analysis
on a wider spectrum of variables. Moreover, preserving the full set of input variables allowed
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 11 / 15
to exploit all the potential high-order relationships between variables that a trained ANN is
able to capture and embed in its synaptic weights.
However, selecting the most important variables on the basis of a sensitivity analysis can be
needed in data-limited scenarios and therefore we checked the effects of a reduced set of input
variables, selected through a constrained sensitivity analysis, on the performance of the result-
ing ANN model. A subset of input variables was selected, including only those whose con-
strained perturbation induced increases in MSE larger than 10% (Fig 5), i.e. conductivity,
pools, slope, anthropic disturbance, source distance and sampled area. Then a new ANN
model with a 6-4-1 structure was trained and the determination coefficient for the test set was
R
2
= 0.44. Even if model accuracy in predicting fish species richness values considerably
decreased, the variance explained by the model using the selected variables was still acceptable,
especially in the light of the exclusion of 21 variables out of 27.
As far as we know, problems related to the scaling of ANN input variables (e.g. because of
heterogeneity in their units) have been already tackled [52], [53], but methods aimed at defin-
ing to what an extent normalized input variable can be perturbed or changed in a sensitivity
analysis, while preserving reasonable quantitative relationships with each other have never
been implemented. From an ecological point of view, the method we propose showed what
environmental variables, in real-world conditions (i.e. with values that vary within a realistic
range) may actually induce changes in fish species richness. Looking at the results from a con-
servation perspective, assigning the highest degree of importance to variables that are very
unlikely to change at local scale (e.g. slope) would be meaningless, while considering as more
influential variables that may have a real impact on the fish assemblage richness, such as the
level of water pollution or alterations of river traits due to urbanization [54], [55] is certainly
more appropriate.
4. Conclusions
While several methods are available to test the sensitivity of ANNs or of any other type of
model, we based our analysis on the perturbation method, because it is the one that most
closely matches the rationale of the procedure we propose. However, the same rationale may
be adapted to any other method (e.g. Partial Derivatives or Lek’s profiles method), as its only
goal is to avoid data patterns that are not likely to occur in real-world conditions and that
therefore are not really useful to open the ANN “black-box” as well as any other type of empiri-
cal model and to elucidate the way it worked and the ecological relationships it captured.
Of course, it was not possible to validate the approach we proposed by means of statistical
analyses or by any other method. However, it showed that variables that influence fish species
richness according to a procedure that takes into account only combinations of values that are
likely to occur in real-world situations are not the same that would have been selected accord-
ing to a procedure that does not take the ecological relationships between environmental vari-
ables into due account. Thus, our constrained approach to sensitivity analysis can be regarded
as more realistic way to look into the model behavior, focusing on a meaningful subset of the
multidimensional space in which the model can be theoretically applied. In fact, investigating
how a model behaves in a region of its potential input space that will never be used in practical
applications seems definitely pointless.
Needless to say, the procedure we proposed is only aimed at demonstrating a concept and
therefore further developments can be imagined in its future applications, particularly as
regards the selection of the number of neighboring observations or the maximum distance to
them, thus investigating the effect of different levels of constrained perturbations and their
effects in the resulting ranking of environmental variables importance.
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 12 / 15
Supporting information
S1 File. Constrained sensitivity analysis algorithm. Here, the R code algorithm of the con-
strained sensitivity analysis is provided.
(R)
S2 File. Data set. Data set used for the Artificial Neural Network modeling. All values were
normalized as described in the Material and Methods section.
(CSV)
Author Contributions
Conceptualization: Lorenzo Tancioni, Michele Scardi.
Data curation: Simone Franceschini, Lorenzo Tancioni, Massimo Lorenzoni, Francesco
Mattei.
Formal analysis: Simone Franceschini, Michele Scardi.
Investigation: Simone Franceschini.
Methodology: Simone Franceschini, Michele Scardi.
Project administration: Michele Scardi.
Supervision: Lorenzo Tancioni, Michele Scardi.
Validation: Simone Franceschini.
Visualization: Simone Franceschini.
Writing – original draft: Simone Franceschini.
Writing – review & editing: Simone Franceschini, Michele Scardi.
References
1. Dudgeon D, Arthington AH, Gessner MO, Kawabata Z-I, Knowler DJ, Le
´vêque C, et al. Freshwater bio-
diversity: importance, threats, status and conservation challenges. Biological Reviews.2006 May; 81
(2):163–82. https://doi.org/10.1017/S1464793105006950 PMID: 16336747
2. Postel S, Richter B. Rivers for Life: Managing Water for People and Nature. Island Press; 2012.
3. Revenga C, Campbell I, Abell R, Villiers, Bryer M. Prospects for monitoring freshwater ecosystems
towards the 2010 targets. Philosophical Transactions of the Royal Society of London B: Biological Sci-
ences. 2005 Feb 28; 360(1454):397–413. https://doi.org/10.1098/rstb.2004.1595 PMID: 15814353
4. Jackson DA, Peres-Neto PR, Olden JD. What controls who is where in freshwater fish communities: the
roles of biotic, abiotic, and spatial factors. Can J Fish Aquat Sci. 2001 Jan 1; 58(1):157–70.
5. Albaret J-J, Simier M, Darboe FS, Ecoutin J-M, Raffray J. Fish diversity and distribution in the Gambia
Estuary, West Africa, in relation to environmental variables. Aquat Living Resour. 2004 Jan 1; 17(1):35–
46.
6. Radinger J, Ho
¨lker F, Horky
´P, Slavı
´k O, Dendoncker N, Wolter C. Synergistic and antagonistic interac-
tions of future land use and climate change on river fish assemblages. Glob Change Biol. 2016 Apr 1;
22(4):1505–22.
7. Levin SA. Ecosystems and the Biosphere as Complex Adaptive Systems. Ecosystems. 1998 Sep 1; 1
(5):431–6.
8. Joy MK, Death RG. Predictive modelling and spatial mapping of freshwater fish and decapod assem-
blages using GIS and neural networks. Freshwater Biology. 2004 Aug 1; 49(8):1036–52.
9. Franceschini S, Gandola E, Martinoli M, Tancioni L, Scardi M. Cascaded neural networks improving
fish species prediction accuracy: the role of the biotic information. Scientific Reports. 2018 Mar 15; 8
(1):4581.
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 13 / 15
10. Oberdorff T, Gue
´gan J-F, Hugueny B. Global Scale Patterns of Fish Species Richness in Rivers. Eco-
graphy. 1995; 18(4):345–52.
11. Gue
´gan J-F, Lek S, Oberdorff T. Energy availability and habitat heterogeneity predict global riverine fish
diversity. Nature. 1998 Jan 22; 391(6665):382–4.
12. Olaya-Marı
´n EJ, Martı
´nez-Capel F, Vezza P. A comparison of artificial neural networks and random for-
ests to predict native fish species richness in Mediterranean rivers. Knowl Managt Aquatic Ecosyst.
2013;(409):07.
13. Olden JD, Joy MK, Death RG. An accurate comparison of methods for quantifying variable importance
in artificial neural networks using simulated data. Ecological Modelling. 2004 Nov 1; 178(3):389–97.
14. Schwartzman GL, Kaluzny SP 1987. Ecological simulation primer. New York: Macmillan. 1987
15. Lek S, Belaud A, Baran P, Dimopoulos I, Delacoste M. Role of some environmental variables in trout
abundance models using neural networks. Aquatic Living Resources. 1996 Jan; 9(1):23–9.
16. Lek S, Delacoste M, Baran P, Dimopoulos I, Lauga J, Aulagnier S. Application of neural networks to
modelling nonlinear relationships in ecology. Ecological Modelling. 1996 Sep 1; 90(1):39–52.
17. Scardi M, Harding LW. Developing an empirical model of phytoplankton primary production: a neural
network case study. Ecological Modelling. 1999 Aug 17; 120(2):213–23.
18. Mattei F, Franceschini S, Scardi M. A depth-resolved artificial neural network model of marine phyto-
plankton primary production. Ecological Modelling. 2018 Aug 24; 382:51–62.
19. Dimopoulos Y, Bourret P, Lek S. Use of some sensitivity criteria for choosing networks with good gener-
alization ability. Neural Process Lett. 1995 Dec 1; 2(6):1–4.
20. Dimopoulos I, Chronopoulos J, Chronopoulou-Sereli A, Lek S. Neural network models to study relation-
ships between lead concentration in grasses and permanent urban descriptors in Athens city (Greece).
Ecological Modelling. 1999 Aug 17; 120(2):157–65.
21. Gevrey M, Dimopoulos I, Lek S. Review and comparison of methods to study the contribution of vari-
ables in artificial neural network models. Ecological Modelling. 2003 Feb 15; 160(3):249–64.
22. Gevrey M, Dimopoulos I, Lek S. Two-way interaction of input variables in the sensitivity analysis of neu-
ral network models. Ecological Modelling. 2006 May 15; 195(1):43–50.
23. Garson GD. Interpreting Neural-network Connection Weights. AI Expert. 1991 Apr; 6(4):46–51.
24. Olden JD, Jackson DA. Illuminating the “black box”: a randomization approach for understanding vari-
able contributions in artificial neural networks. Ecological Modelling. 2002 Aug 15; 154(1):135–50.
25. Scardi M. Advances in neural network modeling of phytoplankton primary production. Ecological Model-
ling. 2001 Dec 1; 146(1):33–45.
26. Lek S, Scardi M, Verdonschot PFM, Descy J-P, Park Y-S. Modelling Community Structure in Freshwa-
ter Ecosystems. Springer Science & Business Media; 2005. 526 p.
27. Lecours V, Brown CJ, Devillers R, Lucieer VL, Edinger EN. Comparing Selections of Environmental
Variables for Ecological Studies: A Focus on Terrain Attributes. PLOS ONE. 2016 Dec 21; 11(12):
e0167128. https://doi.org/10.1371/journal.pone.0167128 PMID: 28002453
28. Larsen S, Mancini L, Pace G, Scalici M, Tancioni L. Weak Concordance between Fish and Macroinver-
tebrates in Mediterranean Streams. PLOS ONE. 2012 Dec 10; 7(12):e51115. https://doi.org/10.1371/
journal.pone.0051115 PMID: 23251432
29. Carosi A, Ghetti L, Forconi A, Lorenzoni M. Fish community of the river Tiber basin (Umbria-Italy): tem-
poral changes and possible threats to native biodiversity. Knowl Manag Aquat Ecosyst. 2015;(416):22.
30. Sarrocco S, Maio G, Celauro D, Tancioni L. "Carta della biodiversita’ ittica delle acque correnti del
Lazio. Analisi della fauna ittica".— Roma: Agenzia Regionale Parchi. 2012
31. Scardi M, Tancioni M, Martone M. Protocollo di campionamento e analisi della fauna ittica dei sistemi
lotici. APAT, Rome. 2007
32. Olden JD, Jackson DA. Fish–Habitat Relationships in Lakes: Gaining Predictive and Explanatory
Insight by Using Artificial Neural Networks. Transactions of the American Fisheries Society. 2001 Sep
1; 130(5):878–97.
33. Joy MK, Death RG. Predictive modelling of freshwater fish as a biomonitoring tool in New Zealand.
Freshwater Biology. 2002 Nov 1; 47(11):2261–75.
34. Lek S, Guegan J-F. Artificial Neuronal Networks: Application to Ecology and Evolution. Springer Berlin
Heidelberg; 2000. 296 p.
35. R Development Core Team. R: A language and environment for statistical computing. R Foundation for
Statistical Computing, Vienna, Austria. 2008. ISBN 3-900051-07-0, URL http://www.R-project.org.
36. The H2O.ai Team. h2o: R Interface for H2O. Version 3.14.0.3. 2017. URL https://cran.r-project.org/
web/packages/h2o/index.html
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 14 / 15
37. Joy MK, Death RG. Modelling of freshwater fish and macro-crustacean assemblages for biological
assessment in New Zealand. In: Modelling Community Structure in Freshwater Ecosystems. Springer,
Berlin, Heidelberg; 2005. p. 76–89.
38. Gevrey M, Park YS, Oberdoff T, Lek S. Predicting fish assemblages in France and evaluating the influ-
ence of their environmental variables. In: Modelling Community Structure in Freshwater Ecosystems.
Springer, Berlin, Heidelberg; 2005. p. 54–63.
39. O
¨zesmi SL, Tan CO, O
¨zesmi U. Methodological issues in building, training, and testing artificial neural
networks in ecological applications. Ecological Modelling. 2006 May 15; 195(1):83–93.
40. Ibarra AA, Park Y-S, Brosse S, Reyjol Y, Lim P, Lek S. Nested patterns of spatial diversity revealed for
fish assemblages in a west European river. Ecology of Freshwater Fish. 2005 Sep 1; 14(3):233–42.
41. Hoeinghaus DJ, Winemiller KO, Birnbaum JS. Local and regional determinants of stream fish assem-
blage structure: inferences based on taxonomic vs. functional groups. Journal of Biogeography. 2007
Feb 1; 34(2):324–38.
42. Teresa FB, Casatti L. Influence of forest cover and mesohabitat types on functional and taxonomic
diversity of fish communities in Neotropical lowland streams. Ecology of Freshwater Fish. 2012 Jul 1; 21
(3):433–42.
43. Lamouroux N, Poff NL, Angermeier PL. Intercontinental Convergence of Stream Fish Community Traits
Along Geomorphic and Hydraulic Gradients. Ecology. 2002 Jul 1; 83(7):1792–807.
44. Cunico AM, Allan JD, Agostinho AA. Functional convergence of fish assemblages in urban streams of
Brazil and the United States. Ecological Indicators. 2011 Sep 1; 11(5):1354–9.
45. Giller P, Malmqvist B. The Biology of Streams and Rivers. Oxford, New York: Oxford University Press;
1998. 304 p. (Biology of Habitats Series).
46. Cushing CE, Allan JD. Streams: Their Ecology and Life. Gulf Professional Publishing; 2001. 392 p.
47. Ferreira FC, Petrere M. Anthropic effects on the fish community of Ribeirão Claro, Rio Claro, SP, Brazil.
Braz J Biol. 2007 Feb; 67(1):23–32. PMID: 17505746
48. Vega M, Pardo R, Barrado E, Deba
´n L. Assessment of seasonal and polluting effects on the quality of
river water by exploratory data analysis. Water Research. 1998 Dec 1; 32(12):3581–92.
49. Morrison G, Fatoki OS, Persson L, Ekberg A. Assessment of the impact of point source pollution from
the Keiskammahoek Sewage Treatment Plant on the Keiskamma River—pH, electrical conductivity,
oxygen- demanding substance (COD) and nutrients. Water SA. 2001 Jan 1; 27(4):475–80.
50. Bellman RE. Dynamic Programming. Princeton: Princeton University Press; 1957.
51. Theodoridis S, Konstantinos K. Pattern Recognition - 4th Edition. Burlington: Academic Press; 2008.
52. Ming Lu, AbouRizk S. M., Hermann U. H. Sensitivity Analysis of Neural Networks in Spool Fabrication
Productivity Studies. Journal of Computing in Civil Engineering. 2001 Oct 1; 15(4):299–308.
53. Nourani V, Sayyah Fard M. Sensitivity analysis of the artificial neural network outputs in simulation of
the evaporation process at different climatologic regimes. Advances in Engineering Software. 2012
May 1; 47(1):127–46.
54. Morgan RP, Cushman SF. Urbanization effects on stream fish assemblages in Maryland, USA. Journal
of the North American Benthological Society. 2005 Sep 1; 24(3):643–55.
55. Nelson KC, Palmer MA, Pizzuto JE, Moglen GE, Angermeier PL, Hilderbrand RH, et al. Forecasting the
combined effects of urbanization and climate change on stream ecosystems: from impacts to manage-
ment options. J Appl Ecol. 2009 Feb; 46(1):154–63. https://doi.org/10.1111/j.1365-2664.2008.01599.x
PMID: 19536343
An ecologically constrained procedure for sensitivity analysis of Artificial Neural Networks
PLOS ONE | https://doi.org/10.1371/journal.pone.0211445 January 30, 2019 15 / 15
... With regard to phytoplankton, ML has not been explicitly applied within ESMs but has been used on phytoplankton observations (Bourel et al., 2017;Flombaum et al., 2020;Kruk and Segura, 2012;Mattei et al., 2018;Olden, 2000;Rivero-Calle et al., 2015;Scardi, 1996Scardi, , 2001Scardi and Harding, 1999) and has used ESM output as input for a ML model trained on phytoplankton observations (Flombaum et al., 2020). Rivero-Calle et al. (2015) used random forest (RF) to identify the drivers of coccolithophore abundance in the North Atlantic through feature importance measures and partial dependence plots. ...
... With regard to phytoplankton, ML has not been explicitly applied within ESMs but has been used on phytoplankton observations (Bourel et al., 2017;Flombaum et al., 2020;Kruk and Segura, 2012;Mattei et al., 2018;Olden, 2000;Rivero-Calle et al., 2015;Scardi, 1996Scardi, , 2001Scardi and Harding, 1999) and has used ESM output as input for a ML model trained on phytoplankton observations (Flombaum et al., 2020). Rivero-Calle et al. (2015) used random forest (RF) to identify the drivers of coccolithophore abundance in the North Atlantic through feature importance measures and partial dependence plots. ...
... ML has been used to examine apparent relationships of phytoplankton in the environment (Flombaum et al., 2020;Rivero-Calle et al., 2015;Scardi, 1996Scardi, , 2001, and it is reasonable to assume that ML could find intrinsic relationships when provided a new independent dataset from laboratory growth experiments. However, it has yet to be determined under what circumstances the apparent relationships captured by ML have significantly different functional forms com-pared to the intrinsic relationships that actually control phytoplankton growth. ...
Article
Full-text available
A key challenge for biological oceanography is relating the physiological mechanisms controlling phytoplankton growth to the spatial distribution of those phytoplankton. Physiological mechanisms are often isolated by varying one driver of growth, such as nutrient or light, in a controlled laboratory setting producing what we call “intrinsic relationships”. We contrast these with the “apparent relationships” which emerge in the environment in climatological data. Although previous studies have found machine learning (ML) can find apparent relationships, there has yet to be a systematic study examining when and why these apparent relationships diverge from the underlying intrinsic relationships found in the lab and how and why this may depend on the method applied. Here we conduct a proof-of-concept study with three scenarios in which biomass is by construction a function of time-averaged phytoplankton growth rate. In the first scenario, the inputs and outputs of the intrinsic and apparent relationships vary over the same monthly timescales. In the second, the intrinsic relationships relate averages of drivers that vary on hourly timescales to biomass, but the apparent relationships are sought between monthly averages of these inputs and monthly-averaged output. In the third scenario we apply ML to the output of an actual Earth system model (ESM). Our results demonstrated that when intrinsic and apparent relationships operate on the same spatial and temporal timescale, neural network ensembles (NNEs) were able to extract the intrinsic relationships when only provided information about the apparent relationships, while colimitation and its inability to extrapolate resulted in random forests (RFs) diverging from the true response. When intrinsic and apparent relationships operated on different timescales (as little separation as hourly versus daily), NNEs fed with apparent relationships in time-averaged data produced responses with the right shape but underestimated the biomass. This was because when the intrinsic relationship was nonlinear, the response to a time-averaged input differed systematically from the time-averaged response. Although the limitations found by NNEs were overestimated, they were able to produce more realistic shapes of the actual relationships compared to multiple linear regression. Additionally, NNEs were able to model the interactions between predictors and their effects on biomass, allowing for a qualitative assessment of the colimitation patterns and the nutrient causing the most limitation. Future research may be able to use this type of analysis for observational datasets and other ESMs to identify apparent relationships between biogeochemical variables (rather than spatiotemporal distributions only) and identify interactions and colimitations without having to perform (or at least performing fewer) growth experiments in a lab. From our study, it appears that ML can extract useful information from ESM output and could likely do so for observational datasets as well.
... For example, Worland et al. (2018) used feature importance (FI) to identify primary drivers of annual low stream flows, then used partial dependence plots (PDPs) to visualize the shapes of the effects of each primary driver. Similar approaches have been used to assess the relationships between newly proposed hydrologic metrics and depth to water table (Belitz et al., 2019); regional-to-local lake properties and water quality (Read et al., 2015); and potential drivers of groundwater nitrate concentrations (Ransom et al., 2017), fish reproductive success (Buston and Elith, 2011), and fish species richness (Franceschini et al., 2019). The shapes of relationships in PDPs have also been used to suggest thresholds in stream and watershed disturbance that trigger regime shifts in stream temperatures (Hill et al., 2013). ...
... Incorporation of limnological knowledge can improve prediction accuracy and help ML models earn trust. For example, Franceschini et al. (2019) used knowledge of fish species interactions to develop an NN model that more accurately predicted the occurrence probabilities of a target fish species by including two meaningful intermediate variables, the occurrence probabilities of two secondary fish species. Noori et al. (2020) passed outputs from a process-based water quality model into an ANN to achieve more accurate predictions of riverine nutrient loads in and around Atlanta, Georgia. ...
Preprint
This chapter provides an overview of machine learning models and their applications to the science of inland waters. Such models serve a wide range of purposes for science and management: predicting water quality, quantity, or ecological dynamics across space, time, or hypothetical scenarios; vetting and distilling raw data for further modeling or analysis; generating and exploring hypotheses; estimating physically or biologically meaningful parameters for use in further modeling; and revealing patterns in complex, multidimensional data or model outputs. An important research frontier is the injection of limnological knowledge into machine-learning models, which has shown great promise for increasing such models’ accuracy, trustworthiness, and interpretability. Here we describe a few of the most powerful machine learning tools, describe best practices for employing these tools and injecting knowledge guidance, and give examples of their applications to advance understanding of inland waters.
... The results showed that cognitive improvements were not limited to training with action games and that different games improved different aspects of cognition. Action VGs have even been used to treat dyslexic children [46,47]. Only 12 h of action VGs, for nine sessions of 80 min per day, significantly improved reading and attentional skills [48]. ...
... Moreover, several studies emphasize the educational and academic potential of VGs that may become effective and 'exemplary teachers' [12,82] providing fun and motivating contexts for deep learning in a wide range of content [12], such as school learning [96], rehabilitation activities [46,47], new health care and protection behavior development and the enhancement of specific skills [97,99,100]. Similarly, the literature data document that the intensive use of VGs results in generalized improvements in cognitive functions or specific cognitive domains, and in behavioral changes [1]. ...
Article
Full-text available
A growing number of children and adolescents play video games (VGs) for long amounts of time. The current outbreak of the Coronavirus pandemic has significantly reduced outdoor activities and direct interpersonal relationships. Therefore, a higher use of VGs can become the response to stress and fear of illness. VGs and their practical, academic, vocational and educational implications have become an issue of increasing interest for scholars, parents, teachers, pediatricians and youth public policy makers. The current systematic review aims to identify, in recent literature, the most relevant problems of the complex issue of playing VGs in children and adolescents in order to provide suggestions for the correct management of VG practice. The method used searches through standardized search operators using keywords related to video games and the link with cognition, cognitive control and behaviors adopted during the pandemic. Ninety-nine studies were reviewed and included, whereas twelve studies were excluded because they were educationally irrelevant. Any debate on the effectiveness of VGs cannot refer to a dichotomous approach, according to which VGs are rigidly ‘good’ or ‘bad’. VGs should be approached in terms of complexity and differentiated by multiple dimensions interacting with each other.
... The findings of the model's sensitivity analysis might well be interpreted as being more adaptive to certain parameters than others [44]. By eliminating input from the training model, the sensitivity analysis was submitted to the test. ...
Article
Full-text available
This paper addresses the application of an adaptive neuro-fuzzy inference system (ANFIS) to assign the optimal dose of propofol as a vital anesthetic drug considering patient needs. The purpose of this research was to explore the factors that influence the propofol dosage needed to sedate patients. This paper estimates the drug dose to regulate the depth of anesthesia by administrating propofol. In this regard, two artificial intelligence approaches; a feedforward neural network and ANFIS are applied to predict the propofol dose. Introducing an estimator to control automatically might provide remarkable advantages for the patient in reducing the risk for under- and over-dosing. The suggested estimations are compared with results extracted from the classical model revised method and then evaluated patients undergoing surgery in a Mashhad’s hospital to identify a research innovation. The propofol doses are optimized using a genetic algorithm. Sensitivity analysis methods are used to test the estimator using a collection of patient models consisting of some populations. Finally, during anesthesia, an optimal dose estimator allows for a rapid period of induction with reasonable overshoot and adequate disturbance rejection results. The novelty of this study is in estimating without using Bi-spectral Index signal and also there is a significant reduction in anesthesia costs by optimizing the drug dose. The results of the optimization model show a 14.06% saving of propofol dose with MSE 5.3 × 10⁻⁶.
... Sensitivity analysis of the model can be interpreted as the results are more sensitive to some parameters than others [61]. Before starting a simulation process, determining the input variables' influence on the estimation of DOA in the FFNN model, a sensitivity analysis has been carried out. ...
Article
Full-text available
This study aims to estimate the depth of anesthesia (DOA) at a safe and appropriate level taking into account the patient characteristics during the induction phase. Bi-spectral Index signal (BIS) as a common approach of controlling DOA generates noise and delays in the initial phase of induction. This may lead to useless information in the process of controlling. Moreover, using the BIS index entails a time-consuming process, high equipping costs, and a lack of accessibility to device accessories. To overcome these problems, we propose a new model of controlling DOA with no need for the use of such an index. Hence, an estimation strategy for DOA is developed applying a feedforward neural network and an adaptive neuro-fuzzy inference estimation model. This model estimates the dose of intravenous anesthetic drugs concerning the patients’ needs resulting in optimal drug dose and stable anesthesia depth. The proposed estimations are tested by sensitivity analysis being compared with real data obtained from the classical model (PK-PD) revised approach and BIS approach on 13 patients undergoing surgery. The results show an accuracy of 0.999, indicative of a high-validated model. Compared to BIS, our proposed model not only controls DOA accurately but also achieves outcomes in practice successfully. Some practical implications for future research and clinical practice are also suggested.
... B Because different basins of attraction correspond to specific regions in state space (Walker et al., 2004;Westley et al., 2011), they are likely to be under-represented in regions of input space which fall outside of these domains. Consequently, there will be a higher degree of uncertainty in neural network predictions in these regions (see Section 2.3), as less data will be available to train the network (Franceschini et al., 2019). ...
Article
Observation of global smart city trends shows a shift in focus from sector-based interventions towards integrated decision-making informed by Big Data. This move towards integration is evident in the emergence of Integrated City Management Platforms (ICMPs). Despite the deluge of data generated by ICMPs and the accompanying growth in computing power, limited research has been conducted on exploring the use of this data to develop quantitative tools for integrated smart city planning and management. In this study, the Design Science Research process was followed to develop and evaluate a modelling framework aimed at exploring the use of ICMP data to identify synergies and dependencies across smart city sectors. This paper provides a summary of framework design and implementation and discusses the Design Science Knowledge gained from the exercise.
... After testing the abovementioned MLP, we performed a sensitivity analysis named Perturbation method (Scardi and Harding, 1999;Franceschini et al., 2019). In this procedure, an increasing amount of white noise was added to the values of each predictive variable while those of the other variables were not altered. ...
Article
Marine litter has significant ecological, social and economic impacts, ultimately raising welfare and conservation concerns. Assessing marine litter hotspots or inferring potential areas of accumulation are challenging topics of marine research. Nevertheless, models able to predict the distribution of marine litter on the seabed are still limited. In this work, a set of Artificial Neural Networks were trained to both model the effect of environmental descriptors on litter distribution and estimate the amount of marine litter in the Central Mediterranean Sea. The first goal involved the use of self-organizing maps in order to highlight the importance of environmental descriptors in affecting marine litter density. The second goal was achieved by developing a multilayer perceptron model, which proved to be an efficient method to estimate the regional quantity of seabed marine litter. Results demonstrated that machine learning could be a suitable approach in the assessment of the marine litter issues.
... Moreover, the back-propagation (BP) neural network, which can approximate any nonlinear function and is widely applied in predictions, optimizations, assessments and classifications in China, has been used less in ecological security evaluations [27]. A cultivated land ecological system is a complex multivariable nonlinear dynamic system, that has posed many limitations on traditional methods for assessing and analyzing, and the BP neural network has strong nonlinear approximation ability and the capabilities to handle unclear, disordered and complex information [28]. Its characteristics are appropriate for overcoming the shortcomings of traditional approaches, thereby reaching a higher accuracy in a short time. ...
Article
Full-text available
The purpose of this study was to explore the correlations between land use/cover change and cultivated land ecological security in flatland and mountainous areas. Firstly, the spatiotemporal variation characteristics of land use/cover change are described in conjunction with ArcGIS10.5 software based on remote sensing images of 2005 and 2015. Then, by establishing a pressure-support framework as an assessment indicator system and developing an improved BP neural network model via a genetic algorithm with the help of MATLAB2016a, the spatiotemporal dynamic changes of cultivated land ecological security in Yuxi City from 2005 to 2015 are evaluated. The results showed that the transformation of farmland area accounted for a large proportion of increased constructive land and land use/cover spatial variations were significantly different among counties, which manifested the changes in farmland and the construction land in flatland areas but also facilitated a mutual transformation of forest and grass in mountainous areas. Moreover, ecological security status presented a clear difference among counties due to their different land use/cover changes. The ecological security state of the flatland expressed a higher ecological pressure and lower ecological support, so the security grade was IV. Otherwise, the ecological security was superior and the security grade was level II or I in the mountainous areas. Thus, protection strategies for ecological security should be differentiated in the flatland areas and mountainous areas due to their different ecological security status brought by land use/cover change.
Article
Full-text available
Classical sensitivity analysis of machine learning regression models is a topic sparse in literature. Most of data-driven models are complex black boxes with limited potential of extracting mathematical understanding of underlying model self-arranged through the training algorithm. Sensitivity analysis can uncover erratic behaviour stemming from overfitting or insufficient size of the training dataset. It can also guide model evaluation and application. In this paper, our work on data-driven sensitivity analysis of complex machine learning models is presented. Rooted in one-at-a-time method it utilizes training, validation and testing datasets to cover the hyperspace of potential inputs. The method is highly scalable, it allows for sensitivity analysis of individual as well as groups of inputs. The method is not computationally expensive, scaling linearly both with the available data samples, and in relation to the quantity of inputs and outputs. Coupled with the fact that calculations are considered embarrassingly parallel, it makes the method attractive for big models. In the case study, a regression model to predict inclinations using recurrent neural network was employed to illustrate our proposed sensitivity analysis method and results.
Article
Enhancing the understanding of marine phytoplankton primary production is paramount due to the relationships with oceanic food webs, energy fluxes, carbon cycle and Earth's climate. As field measurements of this process are both expensive and time consuming, indirect approaches, which can estimate primary production from remotely sensed imagery, are the only viable large-scale solution. We boosted the quality of phytoplankton primary production estimates, with respect to a previously developed model, by embedding ecological knowledge into the training of an artificial neural network. In order to achieve this goal, we drove the training procedure on the basis of both theoretical and data-derived ecological knowledge about phytoplankton primary production. A “single peak” constraint exploits the theoretical knowledge about the vertical shape of the production profile; a “depth-weighted error” procedure was based on available information about the production magnitude along the water column; a variable learning rate and momentum approach allowed to better exploit the available data to train our artificial neural network. Thanks to this customized procedure, we improved the quality of the primary production estimates from both a theoretical and a numerical point of view. Accordingly, the new artificial neural networks not only provided ecologically sounder estimates, but also explained up to 4% more variance with respect to the traditional error back-propagation solution. This result was achieved exclusively through an ecologically-driven customization of the basic algorithm, since the dataset and predictive variables were the same utilized in the conventional counterpart, trained with a classic error back-propagation algorithm. We suggest that the proposed rationale could lead to improved performances in similar modelling applications.
Article
Full-text available
Species distribution is the result of complex interactions that involve environmental parameters as well as biotic factors. However, methodological approaches that consider the use of biotic variables during the prediction process are still largely lacking. Here, a cascaded Artificial Neural Networks (ANN) approach is proposed in order to increase the accuracy of fish species occurrence estimates and a case study for Leucos aula in NE Italy is presented as a demonstration case. Potentially useful biotic information (i.e. occurrence of other species) was selected by means of tetrachoric correlation analysis and on the basis of the improvements it allowed to obtain relative to models based on environmental variables only. The prediction accuracy of the L. aula model based on environmental variables only was improved by the addition of occurrence data for A. arborella and S. erythrophthalmus. While biotic information was needed to train the ANNs, the final cascaded ANN model was able to predict L. aula better than a conventional ANN using environmental variables only as inputs. Results highlighted that biotic information provided by occurrence estimates for non-target species whose distribution can be more easily and accurately modeled may play a very useful role, providing additional predictive variables to target species distribution models.
Article
Full-text available
Selecting appropriate environmental variables is a key step in ecology. Terrain attributes (e.g. slope, rugosity) are routinely used as abiotic surrogates of species distribution and to produce habitat maps that can be used in decision-making for conservation or management. Selecting appropriate terrain attributes for ecological studies may be a challenging process that can lead users to select a subjective, potentially sub-optimal combination of attributes for their applications. The objective of this paper is to assess the impacts of subjectively selecting terrain attributes for ecological applications by comparing the performance of different combinations of terrain attributes in the production of habitat maps and species distribution models. Seven different selections of terrain attributes, alone or in combination with other environmental variables, were used to map benthic habitats of German Bank (off Nova Scotia, Canada). 29 maps of potential habitats based on unsupervised classifications of biophysical characteristics of German Bank were produced, and 29 species distribution models of sea scallops were generated using MaxEnt. The performances of the 58 maps were quantified and compared to evaluate the effectiveness of the various combinations of environmental variables. One of the combinations of terrain attributes–recommended in a related study and that includes a measure of relative position, slope, two measures of orientation , topographic mean and a measure of rugosity–yielded better results than the other selections for both methodologies, confirming that they together best describe terrain properties. Important differences in performance (up to 47% in accuracy measurement) and spatial outputs (up to 58% in spatial distribution of habitats) highlighted the importance of carefully selecting variables for ecological applications. This paper demonstrates that making a subjective choice of variables may reduce map accuracy and produce maps that do not adequately represent habitats and species distributions, thus having important implications when these maps are used for decision-making.
Book
This book considers classical and current theory and practice, of supervised, unsupervised and semi-supervised pattern recognition, to build a complete background for professionals and students of engineering. The authors, leading experts in the field of pattern recognition, have provided an up-to-date, self-contained volume encapsulating this wide spectrum of information. The very latest methods are incorporated in this edition: semi-supervised learning, combining clustering algorithms, and relevance feedback. Thoroughly developed to include many more worked examples to give greater understanding of the various methods and techniques Many more diagrams included--now in two color--to provide greater insight through visual presentation Matlab code of the most common methods are given at the end of each chapter An accompanying book with Matlab code of the most common methods and algorithms in the book, together with a descriptive summary and solved examples, and including real-life data sets in imaging and audio recognition. The companion book is available separately or at a special packaged price (Book ISBN: 9780123744869. Package ISBN: 9780123744913) Latest hot topics included to further the reference value of the text including non-linear dimensionality reduction techniques, relevance feedback, semi-supervised learning, spectral clustering, combining clustering algorithms Solutions manual, powerpoint slides, and additional resources are available to faculty using the text for their course. Register at www.textbooks.elsevier.com and search on "Theodoridis" to access resources for instructor. Published: December 2010.
Chapter
Biological assessment of flowing water has undergone a conceptual change from the use of biological indicators of water quality to the assessment of 'biological quality' (Wright et al. 2000) or 'biological integrity' (Karr 1991). This change from biological indicators to the assessment of biological integrity marks a shift towards ecosystem level evaluation and these approaches are often referred to as measures of 'ecosystem health' (Chessman et al. 1999). These new concepts come from a recognition that water quality is the result of many factors including biological interactions, flow regimes and habitat structure and that these are all dependant on modification by human activities (Karr 1991, 1995). It follows from this movement to an ecosystem approach that impacts on the biological condition of rivers exposed to human impacts can be judged by comparing the river biota with that from relatively unimpacted reference ecosystems. The prerequisite is that the sites occur in similar geomorphological and climatic settings referred to as reference conditions (Chessman et al. 1999). Thus, we can assess the condition of human disturbed sites by measuring their structural attributes and then comparing them with relevant reference conditions that are pristine or, at worst, relatively undisturbed (Hughes et al. 1986). This reference site method of river assessment has been in use for some time although different approaches have been used in different parts of the world. In the USA, a reference site approach has been applied with 'indices of biotic integrity' (e.g. Karr 1981). Whereas in the United Kingdom and more recently Australia, multivariate reference site predictive models have been used (Simpson and Norris 2000). The predictive reference condition approach (RIVPACS: River InVertebrate Prediction And Classification System) and its derivatives developed originally by Wright et al. (1984) and later advanced by Reynoldson et al. (1995), Simpson and Norris (2000) and Hawkins et al. (2000b) have been successfully applied to streams worldwide. The output from the models is a list of taxa expected to be at a site in the absence of human impacts predicted from a suite of environmental variables. The predictions come from a database of least impacted sites selected to cover all stream types within a region. In the bioassessment of a site, the final output is a measure of the relationship between the fauna collected at a site and that predicted. That is the observed number of taxa (O) is compared to the number expected (E) in the absence of stress as a measure of departure from expected conditions, and is thus, a measure of biological impairment. Although the predictive models described above have been developed using macroinvertebrates they have the potential for use with other biotic groups (Reynoldson and Wright 2000) and have been developed for use with fish (Joy and Death 2002), diatoms (Chessman et al. 1999) and stream habitat features (Davies et al. 2000). New Zealand has a limited native fish and macro-crustacean fauna of 39 recognised species, dominated by Galaxiidae and Eleotridae as well as a non-migratory crayfish and a diadromous shrimp. The fauna is characterised by a high proportion of diadromous species such that there are marked longitudinal trajectories of fish distribution with species richness reducing with elevation (Joy et al. 2000). These distributional patterns negate the application of bioassessment methods using fish employed elsewhere in the world such as the index of biotic integrity (IBI). These index approaches would be problematical in New Zealand because they rely on relationships between community metrics and habitat quality and would not account for the overriding longitudinal distributional patterns caused by diadromy (McDowall and Taylor 2000). Migratory fish and crustacea can however be used in bioassessment if the method used takes into account their longitudinal distribution patterns. A predictive modelling approach using reference sites allows for the migration driven distributional patterns to be incorporated in the bioassessment of fish and macrocrustaceans (Joy and Death 2002). We took a reference site predictive modelling approach to the use of fish and macrocrustaceans in the assessment of biological quality in the rivers of Auckland, New Zealand. To achieve this we modeled the presence or absence of 10 fish and 2 macro-crustacean species using individual artificial neural network models, one for each taxon based on environmental variables. To make the predictions from the model independent of human impacts we used only environmental variables unlikely to be influenced by human impacts and used only data from minimally disturbed reference sites. The predictions from these models were combined to predict the fish and macro-crustacean assemblage to be expected at sites.