Technical ReportPDF Available

Abstract and Figures

"WG4 Instruments and Uncertainty Questionnaire: Results" presents the results of a questionnaire conducted in 2019 within the SENSECO COST action. But it is much more than that. This work starts off with a well-written description of the concept of uncertainty. Author Luc Sierro has well risen to the challenge of presenting the usually dry topic of uncertainty to an audience that will, typically, not only include hardened statisticians but regular practitioners of field spectroscopy and environmental monitoring. Consequently, the need to deal with uncertainties should become evident to all readers. A section on remote sensing as applied to environmental monitoring gives a good introduction about the processes of calibration, characterisation, validation and traceability. The main part deals with the SENSECO Questionnaire by presenting the survey results per question. The discussion synthesises these results and provides a critical interpretation.
Content may be subject to copyright.
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 1 of 27
WG4 Instruments and
Uncertainty Questionnaire:
Results
Version: 1.0
Date: 14.09.2020
Status: Approved
Authors: L. Sierro (UZH)
Editor: A. Hueni (UZH, SENSECO COST Action WG4 Lead)
File: SENSECO WG4 - Uncertainty and Instrument Questionnaire - Analysis and
Conclusions.docx
Pages: 27
Classification:
Distribution: Field Spectroscopy Users
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 2 of 27
1 Contents
!!"#$%&$%'()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(*!
*!+#,&-#,.()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(/!
/!0$%,#.12%3#$()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(4!
/)!!52#6&(#7(%8&'3'())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(4!
/)*!9&'&:,28(;1&'%3#$'())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(4!
/)/!<,=:$3>:%3#$(#7(%8&'3'()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(?!
@!A$2&,%:3$%B()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(C!
@)!!D377&,&$%(E3$.'(#7(1$2&,%:3$%B())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(C!
@)*!A$2&,%:3$%B(3$(&$F3,#$G&$%:H(G#$3%#,3$=()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(C!
!"#"$!%&'()*+,&*-.,&./(+01)(/(&*.+&2./32(44,&5!......................................................................................!8!
!"#"#!632(44,&5.37.1&'()*+,&*-!............................................................................................................................!9!
!"#"8!93//1&,'+*,3&.37.1&'()*+,&*,(0!.............................................................................................................!9!
4!9&G#%&(5&$'3$=(3$(I$F3,#$G&$%:H(J#$3%#,3$=())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(!K!
4)!!9&G#%&('&$'3$=(3$(=&$&,:H())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(!K!
4)*!L66H32:%3#$(7#,(&$F3,#$G&$%:H(G#$3%#,3$=())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(!K!
:"#"$!9+4,;)+*,3&.+&2.'<+)+'*(),=+*,3&!.........................................................................................................!10!
:"#"#!>+4,2+*,3&!.......................................................................................................................................................!11!
:"#"8!?)+'(+;,4,*-!...................................................................................................................................................!11!
?!5IM5I"<(N1&'%3#$$:3,&()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(!*!
?)!!0$'%,1G&$%:%3#$(O(6,#2&''3$=('#7%-:,&()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(!*!
?)*!":H3P,:%3#$(:$.(28:,:2%&,3>:%3#$()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(!@!
?)/!A$2&,%:3$%B(:$:HB'3'(Q%,:2&:P3H3%BR())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(!S!
C!D3'21''3#$()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(*K!
C)!!0$%&,6,&%:%3#$(#7(%8&(,&'1H%'(7,#G(%8&(5IM5I"<(;1&'%3#$$:3,&()))))))))))))))))))))))))))))))))))))))(*K!
C)*!"#$%,:'%3$=(5IM5I"<(,&'1H%'(-3%8(73$.3$='(7,#G(H3%&,:%1,&()))))))))))))))))))))))))))))))))))))))))))))(*!!
C)/!L$'-&,3$=(%8&(,&'&:,28(;1&'%3#$'())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(**!
C)@!T3G3%:%3#$(#7(%83'(%8&'3'()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(**!
S!"#$2H1'3#$())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(*@!
U!9&7&,&$2&'())))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))(*4!
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 3 of 27
2 Foreword
WG4 Instruments and Uncertainty Questionnaire: Results
presents the results of a
questionnaire conducted in 2019 within the SENSECO COST action. But it is much more than
that. This work starts off with a well-written description of the concept of uncertainty. Author
Luc Sierro has well risen to the challenge of presenting the usually dry topic of uncertainty to an
audience that will, typically, not only include hardened statisticians but regular practitioners of
field spectroscopy and environmental monitoring. Consequently, the need to deal with
uncertainties should become evident to all readers.
A section on remote sensing as applied to environmental monitoring gives a good introduction
about the processes of calibration, characterisation, validation and traceability.
The main part deals with the SENSECO Questionnaire by presenting the survey results per
question. The discussion synthesises these results and provides a critical interpretation.
It is evident that fostering the understanding of the concept of uncertainty and its practical
application to the field of spectroscopy are critical steps to be taken. The work carried out by
WG4 will aid this process, but further initiatives are clearly needed and applied handling of
uncertainty should be included in all curricula of natural sciences per default.
The credit of this work goes to Luc Sierro, who chose this topic for his BSc thesis at the Remote
Sensing Laboratories, UZH.
May your results be traceable!
Andy Hueni
SENSECO WG4 Lead
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 4 of 27
List of Abbreviations
ASD Analytical Spectral Devices, a Malvern PANalytical company
COST European Cooperation in Science and Technology
DVI Difference Vegetation Index
EA Environmental Assessment
EIS Environmental Impact Statement
EPP Environmental Protection Plan
EVI Enhanced Vegetation Index
FWHM Full Width Half Maximum
GUM Guide to the expression of Uncertainty in Measurement
IPCC Intergovernmental Panel on Climate Change
LAI Leaf Area Index
LiDAR Light Detection And Ranging
LPU Law of Propagation of Uncertainties
NDVI Normalized Differenced Vegetation Index
NPL National Physical Laboratory
RADAR RAdio Detection And Ranging
SENSECO Optical synergies for spatiotemporal SENsing of Scalable ECOphysiological traits
SI International System of Units (
Système international (d'unités))
SNR Signal-to-Noise Ratio
WG Working Group
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 5 of 27
3 Introduction
In a constantly changing environment, it is of great importance that we recognize changes and
trends early on so that we can react to them in time (Glasson, Therivel and Chadwick, 2012).
This is exactly what the environmental monitoring community has specialized in (Mazzotti,
Hughes and Harvey, 2007). Due to a rapidly changing environment (IPCC, 2013), in recent
decades, remote sensing has become a very important research tool for observing changes and
trends in the environmental monitoring community (Donoghue, 2002; Cracknell and Varotsos,
2011). With improved spatial and temporal resolution, the applicability of remote sensing
approaches has increased significantly. Today it is not only possible to carry out large-scale
environmental monitoring with satellite data, but we can also observe phenomena on a smaller
scale.
After several decades of environmental monitoring using remote sensing approaches, it is still
claimed that the satellite systems currently in use are one magnitude off from being able to
monitor changes properly; therefore, it is essential to communicate clearly regarding calibration
of instruments, validation of the data and traceability of uncertainties in research (Lees
et al.
,
2016). Meiller and Hueni (2019) conducted a not yet published questionnaire to shed some light
on the current situation in this field of science. While uncertainties are omnipresent in science
(Feynman, 1988), analyses thereof and traceability of results is not yet wide spread. Many
authors give input to what a framework for coping with uncertainties in measurements should
look like. Saltelli and Tarantola (2002) for example state that sensitivity analysis regarding
uncertainty should be global, meaning that all the input distribution should be taken into
account and that no assumptions on the model’s functional relationship to the inputs should be
necessary. While this is a very broad idea of a framework, there have also been very detailed
propositions made by global institutions (Joint Committee For Guides In Metrology, 2008; World
Meteorological Organisation, 2017). While it is widely acknowledged that solid uncertainty
analyses are an important factor for research, currently there is still a lack of implementation of
norms and standards.
3.1 Scope of thesis
This thesis serves as an introduction to the topic of calibration, validation, uncertainty and
traceability in environmental monitoring using remote sensing approaches. By synthesizing the
questionnaire conducted by the SENSECO COST Action and contrasting it to recent literature in
this field of research, it compiles the currently used remote sensing technology in environmental
monitoring and sheds light upon the current situation regarding calibration, validation and
traceability. Of special interest for this thesis is the question whether today there is an
increased use of uncertainty analysis and traceability in research concerned with environmental
monitoring.
3.2 Research questions
This thesis aims at answering the following research questions:
What is the state-of-the-art instrumentation and how is data processed?
Is there an increased use of uncertainty analysis in research concerned with environmental
monitoring?
Is there an increase in traceability of data quality?
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 6 of 27
3.3 Organization of thesis
This thesis is organized in six main sections: After a general overview over scientific
uncertainties and an insight into uncertainties specific to environmental monitoring, the reader
is briefly introduced to remote sensing. Following is a closer look at remote sensing methods for
environmental monitoring and a literature review on calibration, validation, and traceability. In a
next section, results from a survey on calibration and validation practices are summarized. The
discussion recapitulates the
status quo
on uncertainties and traceability in remote sensing for
environmental monitoring using the results from the questionnaire and contrasts these with a
literature review. Furthermore, the research questions are answered, followed by a reflection on
the limitations of this thesis. The conclusion summarizes the findings and provides an outlook
on potential future studies and suggests improvements that could be done working with
uncertainties in environmental monitoring using remote sensing approaches.
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 7 of 27
4 Uncertainty
As Feynman (1988), an American theoretical physicist stated, scientific knowledge can never be
absolutely certain. Therefore uncertainties are of great importance in research. Not only do they
provide information on the accuracy of measurements and models, but well-documented
uncertainties allow for the comparability of data from different studies over space and time
(Joint Committee For Guides In Metrology, 2008). The difficulty here is not only to quantify
uncertainties, but also to document them in a comprehensible and understandable way. In
environmental monitoring, this documentation plays a particularly important role, as it is a
highly interdisciplinary field of research and studies from a wide variety of subjects come
together. To ensure comparability, an analysis of existing uncertainties should be carried out
regularly at all levels and specifically every time a decision has to be taken based on the results
from measurements (Damasceno and Couto, 2018).
Not only can uncertainties lead to erroneous measurements or models, but when not disclosed
in communication with decision-makers, they can also be responsible for poor policy decisions,
so the 2007 Intergovernmental Panel on Climate Change (IPCC) report for example falsely
states, that 56% of the Netherlands lies below sea level, when in fact, not more than 26% of
the Netherlands is below sea level (IPCC, 2007, p. 574; Cracknell and Varotsos, 2011). Such
mistakes can have serious effects on health, the economy as well as the environment and can
deal massive damage to the credibility of scientific studies (Wardekker
et al.
, 2008).
4.1 Different kinds of uncertainty
Uncertainties are generally understood as a "lack of knowledge". But Povey and Grainger
(2015) go a little further and distinguish between "known and unknown unknowns". They,
thereby, indicate that some uncertainties are known and can be quantified, whereas others
might be known but not yet assessable or even quantifiable. For the “quantifiable known
unknowns” (Povey and Grainger, 2015) scientists find ways of calculating potential errors and
can follow the propagation of these uncertainties throughout their studies. Unquantifiable
known unknows can at least be discussed and provided for research purposes. Furthermore,
there is a third group of uncertainties included in the description of Povey and Grainger (2015):
the “unknown unknowns” of whose existence the scientific community might be yet completely
ignorant. While researchers can try their best to quantify known unknowns, there is always a
principle uncertainty in science, as we can never be certain of everything (Feynman, 1988).
4.2 Uncertainty in environmental monitoring
Uncertainties may arise on several levels. For example, there may already be uncertainty in the
formulation of research questions, then in the measurements taken resulting in flaws in the
dataset, in modelling and above all in the evaluation and understanding of results. Last but not
least, in the case of Environmental Assessment (EA) papers, Environmental Impact statements
(EI) or Environmental Protection Plans (EPP), unclear communication can again lead to errors
(Cracknell and Varotsos, 2011; Lees
et al.
, 2016).
EA’s, EIS’s and EPP’s are established to inform decision makers about potential short- and long-
term environmental impacts of new or existing projects. Based on these assessments, the
benefits of the projects planned can be weighed up against potential environmental impacts
and appropriate strategies and procedures can be developed (Therivel
et al.
, 1999; Glasson,
Therivel and Chadwick, 2012). As today, proponents do often not clearly communicate
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 8 of 27
regarding uncertainties, in fear of not getting the needed approval by the decision-makers
(Tennøy, Kværner and Gjerstad, 2006; Lees
et al.
, 2016). Therefore, proponents should be
encouraged to acknowledge and provide detailed information about uncertainties related to the
projects. One way of doing so could be by adapting the legislation, regulating, providing
guidelines and by setting policies (Aksamit
et al.
, 2020).
Although the Guide to the expression of Uncertainty in Measurement (GUM) recommends a
uniform approach to the analysis of uncertainty and the communication of it, and even
proposes a framework (Joint Committee For Guides In Metrology, 2008), many authors do not
yet make use of it for their research. The GUM was first published in 1993 before the founding
of the Joint Committee for Guides in Metrology (JCGM) which was formed in 1997 and took
over the responsibility for updating and distributing the GUM. The JCGM on a regularly basis
revises the GUM and proposes changes. The guide is based on the most frequent
advancements and principles in statistical mathematics for the propagation of sources of errors
to the final results. The GUM is widely accepted, as it rather provides general rules for
evaluating and expressing uncertainty in measurement than precise and detailed specific
instructions for each field of research (JCGM, 2012; World Meteorological Organisation, 2017).
4.2.1
Uncertainty in measurement and modelling
Uncertainty in measurements could be described as doubts about the accuracy of measuring
instruments. An uncertainty analysis, for example, makes statements about the confidence level
of measurements, i.e. describes how certain researchers are of their results. In most cases,
statistical tests are carried out and a confidence interval is given (Bell, 2001). The international
vocabulary of metrology defines uncertainty in measurement as a “non-negative parameter
characterizing the dispersion of the quantity values being attributed to a measurand, based on
the information used” (JCGM, 2012). Thus, uncertainty in measurements is generally expressed
as a quantitative value of a potential deviation of the measurement. It is important to
understand that measurement uncertainties go hand in hand with metrological traceability. Only
by disclosing the uncertainties associated with measurements comparability with other data can
be achieved. Damasceno and Couto (2018), based on the GUM state, that as a first step the
measurand must be defined and input qualities assessed. Therefore, identifying the variables
directly or indirectly influencing the measurand is crucial. This procedure is called
characterization of the measuring instrument. In the laboratory, experiments are carried out to
determine how the measured value of the object under investigation changes due to the
influence of e.g. temperature fluctuations or changes in signal magnitude and various other
factors (Bouvet
et al.
, 2019; Pacheco-Labrador
et al.
, 2019). Using the findings from
characterizations, the mostly linear and normally distributed random fluctuations can be
represented by the standard deviation of their distribution, which makes propagation of
uncertainties much easier.
Errors are generally divided into two categories, random errors and systematic errors (Povey
and Grainger, 2015). In addition there sometimes is the category of gross errors (Shi, 2010).
Random errors can be detected if irregular measured values are obtained under constant
conditions during several measurements of the same object. If the measurement is carried out
often enough, random errors can be statistically calculated and thus be quantified. Systematic
errors on the other hand correspond to a certain pattern of occurrence. They are often
associated with a lack in the performance of measuring instruments and incorrect
understanding of facts. They usually have a larger impact on measurement results than random
errors. Gross errors (e.g. misidentification of the measuring target) are usually introduced by
humans, for example by negligence in measurements or errors in the processing of measured
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 9 of 27
values (Shi, 2010). These errors have the largest impact on studies but are easier to avoid than
random and systematical errors.
One of the main problems in environmental monitoring is the lack of reproducibility of
measurements in a constantly changing environment. Jiménez
et al.
(2018) point out the most
important sources of uncertainty induced by environmental factors in measuring spectral
reflectance assessed by field spectroscopy, which are the sun’s position, the cloud coverage of
the sky, the presence of atmospheric water vapor and aerosols and adjacency effects caused by
surrounding elements. To better understand and evaluate data, it is therefore important to
archive or publish data with full disclosure of the environmental conditions during the
measurement (Milton, 1987).
During the generation of models, researchers usually have to approximate and simplify data
inputs and calculations. These are made
inter alia
to simplify the processing of the data. For
example, the state of the atmosphere is approximated by some top of the atmosphere
measurements, which cannot possibly reflect the entirety of the complex structure and
composition of the atmosphere. These approximations and simplifications in models lead to
systematic errors in the data generated (Povey and Grainger, 2015).
4.2.2
Modelling of uncertainty
An uncertainty model is a mathematical model which brings together the uncertainties of the
individual inputs and calculates them with an algorithm. An example of such an algorithm is the
law of propagation of uncertainty. The National Physical Laboratory (NPL) proposes eight steps
to establish an uncertainty model in early stages (Jiménez
et al.
, 2018). To understand the
problem, one should describe the traceability chain, write down the calculation equations and
consider the existing sources of uncertainty. Thereafter the formal relationships should be
determined by creating the measurement equation, determining the sensitivity coefficients, and
assigning uncertainties to each element of the calculation. After these six phases the next two
stages are about propagation of uncertainties by combining and propagating uncertainties and
by expanding the uncertainties present (Woolliams, Hueni and Gorrono, 2014).
4.2.3
Communication of uncertainties
Not only is it difficult to grasp uncertainties in EA, since high complexity results in high
uncertainties, but also the communication of these uncertainties confronts proponents of
projects with the unpleasant task of informing decision-makers about a lack of knowledge. EA
practice is often used by proponents for project approval, therefore they are less likely to
inform about uncertainties and other information that may seem unfavorable to administrative
authorization or the public (Cowan and Gadenne, 2005).
The true purpose of Environmental Impact Statements (EIS) is to help decision makers better
understand the possible consequences of their decisions, therefore, these papers serve as a
decision-making aid for policy makers. In these papers, it is essential to draw attention to
uncertainties (Lees et al., 2016). Several such EIS have been analysed by Tennøy, Kværner and
Gjerstad (2006) and Lees et al. (2016) and they agree in that uncertainties are not addressed
clearly enough and that the processes towards forecasts and recommendations can often not
be traced back. EA practice is commonly used by proponents for project approval, therefore
they are less likely to inform about uncertainties and other information that may seem
unfavorable to administrative authorization or the public (Cowan and Gadenne, 2005). When
uncertainties were not directly addressed, but only vaguely paraphrased (e.g. using terms like
may, could, probably or maybe), it was not possible for Lees et al. (2016) to determine whether
the EA was intending to disclose or hide gaps in knowledge. It is moreover not unusual for
whole groups of projects to deliberately leave out and distract from uncertainties in order to get
projects approved (Aksamit
et al.
, 2020).
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 10 of 27
5 Remote Sensing in Environmental Monitoring
5.1 Remote sensing in general
Remote sensing methods offer the possibility to conduct measurements over large areas and
with high temporal resolution while being characterized by comparatively low costs. Active and
passive systems are thereby distinguished. While active systems emit an inherent radiation and
measure the reflection of this radiation, passive systems measure the solar radiation reflected
by an objects surface and the inherent radiation emitted by the reflecting surface. Remote
sensing sensors can be airborne or spaceborne systems, depending on their platform (e.g.
drones, airplanes or satellites). In satellite-based remote sensing, it can further be differentiated
between geostationary satellites, which rotate synchronously with their corresponding planet
(typically Earth) and thus always show the same section, and polar orbiting satellites which
move around the planet crossing the poles (Lillesand and Kiefer, 2000).
5.2 Application for environmental monitoring
For the application of remote sensing systems in environmental monitoring researchers try to
optimize the benefits from different sensors. As it is a highly interdisciplinary field of research,
many different remote sensing approaches are used, and their data get linked together. For
example satellite-borne systems are used by the IPCC to gain information about the large-scale
distribution of atmospheric carbon dioxide on the planet, as satellite-borne sensors are able to
assimilate data all over the planet and with a relatively high temporal resolution (for example
the European Space Agency’s Sentinel 2 with a revisit time of 2-3 days in mid-latitude regions
(European Space Agency, 2000)). Furthermore, not only atmospheric properties can be
assessed, but also land use change can be monitored. Remote sensing approaches allow for
researchers to better understand large-scale processes and land cover dynamics (Donoghue,
2002).
In order to use remote sensing systems and obtain reliable data from the measurements, the
sensors must be calibrated and characterized on a regular basis. This can be particularly difficult
if one has no longer access to the sensor because it is for example in space on its orbit around
the world. To solve this problem, some space-borne systems carry on-board calibration
instruments (Bouvet
et al.
, 2019).
5.2.1
Calibration and characterization
In order to deliver useful, reliable measurements, instruments must regularly be calibrated, as
their responses may evolve over time (Povey and Grainger, 2015). The calibration is defined as
an “operation that, under specified conditions, in a first step, establishes a relation between the
quantity values with measurement uncertainties provided by measurement standards and
corresponding indications with associated measurement uncertainties and, in a second step,
uses this information to establish a relation for obtaining a measurement result from an
indication” (JCGM, 2012). The calibration process often requires a characterization of the
instrumentation. A characterization report describes the behavior of the instrument to different
induced effects, such as temperature variations.
Satellite radiometers are therefore characterized before launch. Nevertheless, the launch into
space can strongly influence the sensors due to the powerful forces acting on the space-borne
sensors during launch. Therefore the statements of the characterization can only be considered
as a first estimate (Kummerow
et al.
, 2000; Povey and Grainger, 2015).
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 11 of 27
5.2.2
Validation
Remote sensing models and predictions often lack "ground truth" data to be validated. If this
possibility does not exist, remote sensing data which are not critically examined can lead to
misinterpretation in environmental change (Donoghue, 2002). With increasing uncertainty and
traceability requirements demanded for field spectroscopy, ground-truth data acquired by field
spectroscopy takes in a major role in sensor calibration and validation for earth observation
satellites (Jiménez et al., 2018).
5.2.3
Traceability
In order to ensure complete traceability and comparability of data, analyses of uncertainties
must be carried out at each data level. Remote sensing data are divided into four main levels
(Chase, 1986 in: Povey and Grainger, 2015, p. 4700):
Table 1 Satellite data processing levels (Povey and Grainger, 2015, p. 4700)
Level 0
Reconstructed, unprocessed instrument data at full resolution.
Level 1A
Reconstructed, unprocessed instrument data, time-referenced and annotated
with ancillary information such as radiometric and geometric calibration
coefficients and geolocation parameters. Data may be at full resolution or an
average over some retrieval area.
Level 1B
Level 1A data that have been converted to physical units (e.g. brightness
temperature rather than voltage). Not all instruments will have a Level 1B
equivalent.
Level 2
Derived environmental variables (e.g. ocean wave height, soil moisture) at the
same resolution and location as the Level 1 source data.
Level 3
Variables mapped onto uniform space-time grid scales, usually with some
corrections for completeness and consistency (e.g. interpolation of missing
points, interlacing multiple orbits).
The JCGM (2012) defines metrological traceability as the “property of a measurement result
whereby the result can be related to a reference through a documented unbroken chain of
calibrations, each contributing to the measurement uncertainty”. Combining this definition of
metrological traceability with the data processing levels in remote sensing, this means, that for
traceability of level three data, uncertainties from all the previous data levels must have been
propagated through the calculations using for example the Law of Propagation of Uncertainties
(LPU) and an unbroken chain of calibrations must be provided.
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 12 of 27
6 SENSECO Questionnaire
The optical synergies for spatiotemporal SENsing of Scalebale ECOphysiological traits
(SENSECO) is an action brought to life by the European Cooperation in Science & Technology
(COST). COST is an intergovernmental funding organization for the bottom-up creation of
research networks (COST Actions). The actions enable collaboration among European scientists
and offer European Union funding complementing national research funds (European
Cooperation in Science and Technology, 2020).
The SENSECO COST Action focuses on assessing the dynamic response of vegetation to
changing environmental conditions and is representative of the larger environmental monitoring
community. Its main objectives are to close the gap from leaf to satellite measurements, to
improve the time-series processing of satellite sensor data, to improve synergies between
passive optical earth observation domains and to ensure measurement comparability across
different scales, space and time (SENSECO, 2018a). SENSECO is structured into four working
groups (WG); while the first three working groups are concerned with closing scaling and
temporal gaps and realizing synergy between passive earth observation spectral domains WG 4
tries to establish data quality through traceability and uncertainty (SENSECO, 2018b).
In order to compile the currently used remote and proximal sensing technology used in
environmental monitoring and to assess the actual situation concerning calibration, validation,
uncertainty and traceability, the SENSECO COST Action conducted a questionnaire in 2019. The
survey was conducted online using the SurveyMonkey tool. The questionnaire was designed by
Meiller and Hueni (2019) from the Remote Sensing Laboratories at the University of Zurich.
Starting at the end of March to mid-May 2019, 23 researchers from various institutions
anonymously replied to the questionnaire. On average, answering the 44 questions took 17
minutes and 36 seconds. Among other institutions represented were the National
Meteorological Administration (ANM) in Romania, the national Institute of Aerospace
Technology (INTA) in Spain, the University of Tasmania in Australia and the Federal Technical
Highschool Zurich (ETHZ) in Switzerland to name just a few. The following three subsections of
this thesis compile the results of this questionnaire.
6.1 Instrumentation / processing software
The first nine questions were to assess the currently used instrumentation in environmental
monitoring and to see, what data levels are covered. In the following section this thesis refers
to data levels as explained in chapter 3.2.3. Question number 43 asked for the processing
software, the respondents worked with. For most questions multiple answers were allowed.
Table 2 Answers to the first question "Instrumentation" (data from Meiller and Hueni, 2019)
Answer choices
Responses (23)
Hyperspectral (Spectroradiometers): Fixed Point
78.26%
Hyperspectral (Spectroradiometers): Imaging
Spectrometer
52.17%
Multispectral: Fixed Point
21.74%
Multispectral: Imaging Spectrometer
30.43%
Broadband (e.g. Pyranometer)
17.39%
Other (please specify):
4.35%
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 13 of 27
The answering scheme to the first question shows up the importance of spectroscopy as a key
technology in spatiotemporal sensing of scaleable ecophysiological traits. By far most of the
respondents use hyperspectral instruments, some also use multispectral or broadband
instruments. Furthermore, it is to mention, that one respondent also uses RADAR and LiDAR
technology for his/her research. Therefore, more than one answer could be selected by each
participant.
As auxiliary instrumentation, meteorological sensors (e.g. temperature, pressure, wind speed)
and structural sensors (e.g. LAI, ground based lidar) are particularly popular. All the available
spatial scales are used starting from millimeter (needle) up to kilometer (landscape). However,
most answers ranged from leaf size to canopy scale. While the spectra from the visual spectrum
up to the thermal infrared spectrum are widely covered (with a maximum in the visual and
near-infrared spectrum VNIR), not yet much work is done in the ultraviolet range (Meiller and
Hueni, 2019).
Figure 1 Data level used (see chapter 3.2.3 in this thesis) (data from Meiller and Hueni, 2019)
As can be seen in figure 1, the questionnaire found that all the data levels are being used. Level
3 data however is strongly underrepresented compared to the other levels. On level 1, every
single respondent is working with radiance data, while only about half of the researchers use
irradiance for their studies. On level 2 extensive use is made of reflectance and emissivity data,
about a third of the respondents use transmittance data. Also backscatter and surface
temperature data is used. Those using level 3 data often use vegetation indices like DVI, LAI,
NDVI or EVI. Moreover, data for evapotranspiration, canopy height and structure, fertilization
maps or tree health condition classes are being utilized (Meiller and Hueni, 2019).
For processing purposes mostly own, custom-built software is utilized, some also use available
opensource software based on Python or R (e.g. R FieldSpectroscopyDP, R
FieldSpectroscopyCC, R-Statistica or Gnumeric). Only few make use of commercial products like
Spectrawiz, Agisoft, IDL, Atcor or manufacturer software. For the custom-built software often
Python, IDL, Fortran, C, R and Matlab are the means of choice. One respondent makes use of
the Java-based spectrum database SPECCHIO (Meiller and Hueni, 2019).
Table 3 gives an overview of the answers to the question regarding the type of first
instrumentation:
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Level!0 Level!1 Level!2 Level!3
Percentage
Data!Level
Usage!of!data!level
Respondents!(100%!=!23)
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 14 of 27
Table 3 Unchanged answers to “first instrumentation” (data from Meiller and Hueni, 2019)
spectroradiometer
Telops Hypercam LW
Non-imaging point spectrometer ASD FSIII
Field spectroradiometer ASD Fieldspec 3
QE PRO OceanOptics
ADS Field Spec
MicaSense Red Edge M (multispec)
Monitoring field spectrometer
Ximea multispectral camera
Hyperspectral
Hyperspecrtal Spectroradiometer
pushbroom and whiskbroom hyperspectral
airborne imagers
Multispectral uav imager
HR-1024 Spectra Vista
Airborne LWIR FTIR imager
ASD FieldSpec 4+ASD PlantProbe
QE65000 sectrometer
ASD FR3
Imaging spectrometer APEX
Hyperspectral point sensor
Field spectro-radiometer (ASD FieldSpec-3)
Asd
Again, we can see the high popularity of spectrometers and especially of field
spectroradiometers issued by the market leader Analytical Spectral Devices (ASD). Most of
these primarily used instruments named by the respondents are utilized
in-situ
, only few are
airborne.
6.2 Calibration and characterization
As we have seen in chapter 5.2.1 calibration practices of instruments and the characterization
of which plays an important role in research. In order to uncover current patterns in the field of
environmental monitoring, about 30 questions of the questionnaire dealt with this subject area.
Care must be taken evaluating the answers, as not all respondents utilize primary, secondary or
even tertiary instrumentation. If not used, they were asked to skip the questions.
Figure 2 Calibration (data from Meiller and Hueni, 2019)
As we can see in figure 2, for calibration, radiance takes up a very prominent position.
Simultaneously, wavelength does not have the same magnitude, even though the pattern from
0%
20%
40%
60%
80%
100%
Radiance
Irradiance
Temperature
Transmittance
Emissivity
Wavelength
Reflectance:!8…
Reflectance:!0-…
Reflectance:!BRF
Reflectance:…
Percentage
Calibration
Calibration
1st!instrumentation!(100%!=!23) 2nd!instrumentation!(100%!=!13)
3rd!instrumentation!(100%!=!6)
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 15 of 27
first to third instrumentation looks similar. Emissivity, although used by more than half of the
respondents on data level 2 is not often calibrated. Surprisingly, reflectance is rarely calibrated.
Figure 3 Calibration provider (data from Meiller and Hueni, 2019)
Figure 3 shows the results to questions regarding the calibration providers. It is evident, that
most instruments are calibrated by the manufacturer. Also, own calibration in the lab is often
done. Furthermore, the field calibration seems to be applied in practice. Less frequently than
the tertiary instruments, the primary and secondary instruments are calibrated by a third party.
It is also interesting to note that there is no uncertainty about the calibration provider for the
primary instrumentation, while for the other instruments it is sometimes unclear where the
calibration originates from.
Calibration intervals for the primary instrument are in a wide range. Field-spectrometer get
calibrated all five measurements in one case, yearly with another respondent and others
calibrate their instrument every three to five years. Moreover, push- and whiskbroom
hyperspectral airborne imagers get calibration before and after each flight campaign season or
even before each flight line is acquired. The availability of funds for calibration also plays an
important role to one respondent. The few answers given to the calibration interval of the
second and third instrumentation suggest that the situation is very similar for these
instruments.
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Manufacturer
Third-party!calibration
Own!lab
In-field
Not!known
Percentage
Provider
Calibration!provider
1st!instr.!(100%!=!23)
2nd!instr.!(100%!=!13)
3rd!instr.!(100%!=!7)
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 16 of 27
Figure 4 Uncertainty and traceability in calibration (data from Meiller and Hueni, 2019)
Although a calibration is done at least once for each instrument, Figure 4 shows that there is a
great deal of uncertainty about the quality of the calibration. For example, only about half of
the survey respondents can trace the calibration back to SI level. It is also noticeable that not
even half of the calibrations contain uncertainties. Moreover, a disproportional high number of
the respondents do not know about the uncertainty and traceability of the calibration of the
instruments.
Figure 5 Calibration provided in own lab (data from Meiller and Hueni, 2019)
To analyze in-house calibration in more detail, figure 5 provides a graph summarizing the
survey results regarding calibration and traceability by providing calibration in the own lab. The
graph looks very similar to figure 4, with one main difference in that the third instrumentation
calibration does not include uncertainty. It is noticeable that even in-house calibrations do not
necessarily involve uncertainties and traceability, especially since on average almost 20% of the
respondents calibrating in the own lab, do not know whether uncertainties were taken into
account and whether their calibration can be traced back to SI standard.
In order to figure out how the instruments react to induced effects, characterization
experiments are implemented. Figure 6 compiles the characterization parameters used for
analyses. It is remarkable that hardly anybody carries out experiments to investigate the
behavior of the instruments in regard to spectral straylight. The most commonly used
characterization parameters are linearity, SNR, the spectral resolution (FWHM) and the dynamic
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Calibration
includes
uncertainty
Calibration!is
traceble!to!SI
Not!known
Percentage
Uncertainty!and!Traceability
Calibration:!Uncertainty!and!traceability
1st!instrumentation!(100%!=!23)
2nd!instrumentation!(100%!=!13)
3rd!instrumentation!(100%!=!7)
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Calibration!includes!uncertainty Calibration!is!traceable!to!SI Not!known
Percentage
Calibration!and!Traceability
In-house!calibration
1st!instrumentation!(100%!=!10) 2nd!instrumentation!(100%!=!7) 3rd!instrumentation!(100%!=!2)
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 17 of 27
range. Also, experiments examining the reaction of instruments to temperature effects are
conducted.
Figure 6 Characterization parameters (data from Meiller and Hueni, 2019)
As can be seen in figure 7, the characterization of the instrumentation used is often conducted
by the manufacturer. Around half of the respondents conduct their own characterization
experiments in their proper laboratory, generally in addition to the manufacturer’s
characterization. Offers from third parties are hardly used and literature values are only used in
exceptional cases.
Figure 7 Characterization source (data from Meiller and Hueni, 2019)
Further questions in the survey investigated the reasons for characterizing instruments. Figure 8
gives an overview over the results to the questions what the characterization is utilized for.
0.00%
50.00%
100.00%
Linearity
Dynami…
Temper…
SNR
Spectral…
Spectral…
Other…
Percentage
Parameter
Characterization!parameters
1st!instrumentation!(100%!=!23) 2nd!instrumentation!(100%!=!12)
3rd!instrumentation!(100%!=!7)
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Manufacturer Third-party!service Literature Own!lab
Percentage
Source
Characterization!source
1st!instrumentation!(100%!=!23) 2nd!instrumentation!(100%!=!13)
3rd!instrumentation!(100%!=!7)
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 18 of 27
Figure 8 Usage of characterization (data from Meiller and Hueni, 2019)
It can be clearly seen that the characterization is primarily used for the calibration of data and
the correction of biases. Only few respondents make use of the characterization process for
propagating uncertainties.
6.3 Uncertainty analysis (traceability)
While the characterization of instrumentation is barely used for uncertainty propagation, the
questionnaire also asked directly for the provision of uncertainty for different information levels.
Figure 9 shows, that around 80% of the researchers provide uncertainty analyses for level 1
data, while for level 2 the number decreases to around 60% and for the most complex data
level (i.e. level 3) the value decreases to a minimum of only around one third of the
respondents.
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Data!calibration!(bias!correction) Uncertainty!propagation
Percentage
Usage
Characterization!usage
1st!instrumentation!(100%!=!22) 2nd!instrumentation!(100%!=!11)
3rd!instrumentation!(100%!=!6)
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 19 of 27
Figure 9 Supply of uncertainty for different information levels (data from Meiller and Hueni,
2019)
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Level!1
(radiometrically
calibrated)
Level!2!(reflectance,
transmittance,!etc)
Level!3!(higher-level
products)
Percentage
Information!Level
Providing!uncertainty!for!information!levels
Respondents!(100%!=!16)
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 20 of 27
7 Discussion
7.1 Interpretation of the results from the SENSECO questionnaire
The evaluation of the survey conducted by the SENSECO WG4 provides a deeper insight into
the current situation regarding calibration, validation and traceability in environmental
monitoring using remote sensing approaches, but at the same time, raises new questions which
are discussed below.
Spectrometry as a first instrumentation is very popular among researchers and therefore acts as
a key technology for environmental monitoring, but it is not apparent from the answers, what
exactly the instruments are used for. It remains open whether the collected data is used
directly for research, whether they are used to calibrate remote sensing systems or if they are
utilized for validation purposes. A further open question is what the auxiliary instrumentation is
used for. It remains to be seen whether the particularly popular tools from the meteorological
field are used independently to collect data usable for further research or whether they are
highly valued for documenting the environmental conditions during a measurement in order to
enable data comparison. As all spatial scales are in use, there is a high probability, that data
assessed by spectrometers get used both on their own and for calibrating/validating airborne
and spaceborne systems. Not surprisingly, the spectra used in environmental monitoring largely
covered the traditional visual to the infrared spectrum of light. More astonishingly, the
ultraviolet and the thermal infrared spectrum gain in importance, even though their use is not
yet widespread.
When looking at Figure 1, it is noticeable that level three data is only used by around half of the
respondents, while levels zero to two are well-covered. Conversely, it can be concluded that
only slightly more than half of the data is used to compute higher level products. This could
serve as an indication that the lower data levels collected are used for calibration and validation
purpose rather than for receiving more sophisticated data level products.
Since environmental monitoring is still an evolving field of research, it is not surprising that
most researchers rely on self-developed software. In particular, the programs Matlab, Python, R
and C, which are frequently used in environmental research, are used for programming. Third-
party software is rarely used. This is most likely related to usage costs that are preferred to be
used elsewhere in the studies budgets.
Figure 2, showing parameters for calibration, summarizes that most instruments used by the
respondents to the questionnaire are calibrated regarding radiance. Potentially there is a
knowledge gap here, as fewer respondents clicked for wavelength calibration than for radiance,
even though one would assume the numbers to be equal.
According to the results shown in figure 3, over 80% of the researchers send their two main
instruments to the manufacturer for calibration. For the third instrumentation, it is only about
half. A large proportion of the respondents perform calibrations in their own lab and directly in
the field. It is noticeable in the distribution of the answers that it seems that the calibration of
the first and second instrumentation becomes more important than that of the third. For
example, there is no confusion about the calibration providers for the first instrumentation,
while there is a lack of knowledge for the second and third instrumentation. No clear statement
can be made with regard to the calibration interval, as the answers vary greatly. However, the
maximum time between two calibrations is 5 years. Most of the answers, though, are within a
shorter period of time. Someone answered the question that it depends on the availability of
funding. From this it is concluded that calibrations are costly and therefore likely to be
performed at irregular intervals.
From figure 5 and 6 (calibration and traceability) it becomes clear, that uncertainty and
traceability are not yet widespread. Only about half of the calibrations include uncertainty and
half of them are traceable to SI. Furthermore, for around 30% for the secondary, 40% for the
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 21 of 27
primary and even more than 50% for the third instrumentation there is a lack of knowledge
regarding uncertainty disclosure and traceability to an international standard. This lack of
knowledge is also potentially reflected in the fact that calibrations have been stated to be
traceable to SI levels, but at the same time have no indication of uncertainties. This may be a
clue to why today, there is still a lack of uncertainty disclosure in environmental monitoring. The
lack of knowledge about uncertainties and traceability is also reflected in the fact that
respondents who carry out their calibrations in-house provide little information about
uncertainties and less than half of them are traceable to international standards. About one
third of them do not even know whether their own calibrations are traceable and if they take
uncertainties into account. These results indicate clearly that many researchers are not aware
of traceability and uncertainties. Consequently, a general need to render these topics more
visible in environmental monitoring using remote sensing approaches can be identified.
The answers concerning the source of the characterization show a similar pattern as the
answers concerning the source of calibrations. This can be seen as an indication that the
characterization of instruments plays a significant role in the calibration. This assumption is
confirmed in figure 8, where it is clearly seen that instrument characterization is primarily used
for calibration purposes. However, the uncertainty propagation is not considered. Again, most
respondents rely directly on the manufacturer or characterize their instruments in their own
laboratory.
Regarding traceability it is interesting, that more than 80% of the respondents provide
uncertainty for level one data, around 60% for level two data and only one third of the
respondents furthermore provide information on uncertainty for level three data. With an
increase in data complexity, we therefore see a decrease of uncertainty disclosure. However,
the question comes up on how it is possible for 80% of the respondents to provide uncertainty
for level one data, while only around half of them make use of uncertainty in calibration or are
able to trace calibration to SI.
7.2 Contrasting SENSECO results with findings from literature
In accordance with the results of the survey, in current primary literature on the use of remote
sensing in environmental monitoring we often find analyses on calibration, validation and
uncertainties for lower-level data. Among others, Aasen et al. (2019), Cendrero-Mateo et al.
(2019) and Pacheco-Labrador et al. (2019) serve as excellent examples in regard to
instrumental considerations by presenting their works on sun-induced fluorescence (paper
divided into three parts). Interest in the measurement of sun-induced fluorescence has risen
significantly in recent years after it was announced that the European Space Association is
planning a mission to map vegetation fluorescence in order to improve the understanding of the
way carbon moves between plants and the atmosphere and to analyze the effects of
photosynthesis on the carbon and water cycle.
However, if we speak of more sophisticated data based on complex models often used in the
process of environmental assessment, we recognize a lack of uncertainty disclosure (Lees
et al.
,
2016; Aksamit
et al.
, 2020). While the authors accuse the project proponents of intentionally
leaving out existing uncertainties, the survey of the SENSECO WG4 rather suggests that authors
of EA's and EIS's do not even have the necessary information to state uncertainties. The results
of the questionnaire showed that uncertainties from data level zero to level two are often still
guaranteed, but that the disclosure of uncertainty decreases significantly at level three. Since a
further direct evaluation of the survey's results with literature in this field of research cannot be
carried out without reservation, reference is made to the limitations of this thesis.
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 22 of 27
7.3 Answering the research questions
Ø What is the state-of-the-art instrumentation and how is data processed?
Modern field spectrometers are very popular among researchers in the field of
environmental monitoring, and other spectral measuring instruments are also in use. In
addition, satellite-based data are used to make large-scale measurements with high
temporal resolution. The processing of the data is mainly carried out with self-designed
programs adapted to the desired product. Mostly, open-source programming
environments such as Python, R and C are used. Furthermore, Matlab enjoys great
popularity.
Ø Is there an increased use of uncertainty analysis in research concerned with
environmental monitoring?
There is in fact an increased use of uncertainty analysis in environmental monitoring.
Nevertheless, the use of uncertainty analysis is mainly applied to the handling of low-
level data. As the data becomes more complex, uncertainties are more often hidden
because they are difficult to capture. However, especially in environmental assessment
practice, uncertainties are still deliberately concealed to obtain approval for projects
even though they represent potential risks. Promises are often made that future
developments will be monitored, and that action will be taken if undesirable effects
occur.
Ø Is there an increase in traceability of data quality?
The traceability of data quality resembles that of uncertainty analysis. In the area of
low-level data, traceability of the data is provided by an uncertainty analysis. For more
complex data this is rarely the case. Although frameworks have been proposed to
improve the handling of uncertainties in data quality traceability, they are rarely used.
7.4 Limitation of this thesis
The concept of environmental monitoring is a collection of studies of all kinds and on all
environmental processes, so it is difficult to set clear boundaries in such an interdisciplinary field
of research. Especially since there was a time limit for this work, not all issues could be
addressed and only the most important aspects could be highlighted.
The SENSECO questionnaire has caused some ambiguity among the respondents, for example,
questions concerning the calibration interval of corresponding instruments (see questions 13, 24
and 35 in Meiller and Hueni, 2019) were frequently answered by providing a range of
wavelengths. Furthermore, multiple respondents commented on difficulties at understanding
some questions. Which according to these comments even led to random selection of one of
the provided answers by the interrogators.
Furthermore, the circle of participants of the survey was potentially biased, assuming that the
respondents to the questionnaire were likely to be from the same research environment than
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 23 of 27
Meiller and Hueni, while the pool of participants was restricted to maximum 23 (some questions
even down to 2) and therewith limiting the significance of the responses.
Nevertheless, its main parts corresponded well with the literature.
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 24 of 27
8 Conclusion
Despite the difficulties in the analysis of uncertainties in calibration, characterization, and
validation, it is of great importance to analyze and document the degree of uncertainty. This is
the only way to ensure reliable comparability of data. Uncertainty analyses should therefore
follow a globally applicable pattern and be provided for all data levels using the law of
propagation of uncertainties. For example, the guide to the expression of uncertainty in
measurement, which proposes a universal procedure for the expression of uncertainties, is a
useful tool. Today we have a solid foundation on which we can build. Researchers seem aware
of the importance of calibration, validation, and uncertainty, but there is still a massive lack of
implementation in research. This deficiency is probably due to the fact that operators are still
too little engaged in the characterization of their instruments. In order to motivate researchers
to document uncertainties more frequently, they should be encouraged to overcome their fear
of uncertainty. Because closing eyes to the insecure will not help in finding solutions to
minimize uncertainties. On a policy level, especially for the use in environmental assessment,
there should be regulations and laws implemented. While the evaluation of the SENSECO
survey shed light on practices regarding calibration, characterization and uncertainty, it also
raised some new questions. Further research could be done regarding the usage of auxiliary
data, to assess whether it is used for describing the environmental conditions during
measurement or if it is used as independent model input variable.
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 25 of 27
9 References
Aasen, H.
et al.
(2019) ‘Sun-induced chlorophyll fluorescence II: Review of passive measurement setups,
protocols, and their application at the leaf to canopy level’,
Remote Sensing
, 11(8). doi:
10.3390/rs11080956.
Aksamit, C. K.
et al.
(2020) ‘Sources of uncertainties in environmental assessment: Lessons about
uncertainty disclosure and communication from an oil sands extraction project in Northern Alberta’,
Journal of Environmental Planning and Management
. Routledge, 63(2), pp. 317334. doi:
10.1080/09640568.2019.1579973.
Bell, S. (2001) ‘Good Practice Guide No. 11 - Introductory Guide to Uncertainty of Measurement’,
Measurement Good Practice Guide
, (2), p. 41.
Bouvet, M.
et al.
(2019) ‘RadCalNet: A radiometric calibration network for earth observing imagers
operating in the visible to shortwave infrared spectral range’,
Remote Sensing
, 11(20). doi:
10.3390/rs11202401.
Cendrero-Mateo
et al.
(2019) Sun-Induced Chlorophyll Fluorescence III: Benchmarking Retrieval
Methods and Sensor Characteristics for Proximal Sensing’,
Remote Sensing
, 11(8), p. 962. doi:
10.3390/rs11080962.
Cowan, S. and Gadenne, D. (2005) ‘Australian corporate environmental reporting: a comparative analysis
of disclosure practices across voluntary and mandatory disclosure systems’,
Journal of Accounting &
Organizational Change
, 1(2), pp. 165179. doi: 10.1108/18325910510635344.
Cracknell, A. P. and Varotsos, C. A. (2011) ‘New aspects of global climate-dynamics research and remote
sensing’,
International Journal of Remote Sensing
, 32(3), pp. 579600. doi:
10.1080/01431161.2010.517807.
Damasceno, J. C. and Couto, P. R. G. (2018) ‘Methods for Evaluation of Measurement Uncertainty’, in Anil
Akdogan (ed.)
Metrology
. IntechOpen, pp. 928. doi: 10.5772/intechopen.74873.
Donoghue, D. N. M. (2002) ‘Remote sensing: Environmental change’,
Progress in Physical Geography
,
26(1), pp. 144152. doi: 10.1191/0309133302pp329pr.
European Cooperation in Science and Technology (2020)
About COST
. Available at:
https://www.cost.eu/who-we-are/about-cost/.
European Space Agency (2000)
ESA- Sentinel-2
. Available at:
https://sentinel.esa.int/web/sentinel/missions/sentinel-2.
Feynman, R. P. (1988)
‘What Do You Care What Other People Think?’: Further Adventures of a Curious
Character
. W. W. Norton & Co Inc.
Glasson, J., Therivel, R. and Chadwick, A. (2012)
Introduction to Environmental Impact Assessment
. 4th.
editi. Routledge.
IPCC (2007)
Climate Change 2007: Impacts, Adaptation and Vulnerability
,
Cambridge University Press
.
Edited by M. Parry et al. Cambridge University Press.
IPCC (2013) ‘Klimaänderung 2013/2014 - Naturwissenschaftliche Grundlagen’,
Zusammenfassungen für
politische Entscheidungsträger. Beiträge der drei Arbeitsgruppen zum Fünften Sachstandsbericht des
Zwischenstaatlichen Ausschusses für Klimaänderungen (IPCC). Deutsche Übersetzungen durch Deutsche
IPCC-Koordinierungsstelle, Österreich
, p. 28. Available at: http://www.de-ipcc.de/media/content/AR5-
WGI_SPM.pdf.
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 26 of 27
JCGM (2012) ‘International vocabulary of metrology - Basic and general concepts and associated terms
(VIM 3rd edition, 2008 version with minor corrections)’, 58(36), pp. 34933495. doi:
10.1016/j.tetlet.2017.07.069.
Jiménez, M.
et al.
(2018) ‘Towards a complete spectral reflectance uncertainty model for Field
Spectroscopy’,
The 5th International Symposium on Recent Advances in Quantitative Remote Sensing
,
(October). Available at:
https://www.researchgate.net/publication/328171692_Towards_a_complete_spectral_reflectance_uncert
ainty_model_for_Field_Spectroscopy.
Joint Committee For Guides In Metrology (2008) ‘Evaluation of measurement data Guide to the
expression of uncertainty in measurement’,
International Organization for Standardization Geneva ISBN
,
50(September), p. 134. doi: 10.1373/clinchem.2003.030528.
Kummerow, C.
et al.
(2000) ‘The status of the tropical rainfall measuring mission (TRMM) after two years
in orbit’,
Journal of Applied Meteorology
, 39(12 PART 1), pp. 19651982. doi: 10.1175/1520-
0450(2001)040<1965:tsottr>2.0.co;2.
Lees, J.
et al.
(2016) ‘Analysis of uncertainty consideration in environmental assessment: an empirical
study of Canadian EA practice’,
Journal of Environmental Planning and Management
. Taylor & Francis,
59(11), pp. 20242044. doi: 10.1080/09640568.2015.1116980.
Lillesand, T. M. and Kiefer, R. W. (2000)
Remote Sensing and Image Interpretation
. 4th edition. John
Wiley & Sons, Inc.
Mazzotti, F. J., Hughes, N. and Harvey, R. G. (2007) Why do we need environmental monitoring for
Everglades restoration?’, (November), pp. 13. Available at:
https://edis.ifas.ufl.edu/pdffiles/UW/UW28300.pdf.
Meiller, C. and Hueni, A. (2019). Instrument and CAL/VAL Practices. Questionnaire, SENSECO COST
Action WG4.
Milton, E. J. (1987) ‘Review article: Principles of field spectroscopy’,
International Journal of Remote
Sensing
, 8(12), pp. 18071827. doi: 10.1080/01431168708954818.
Pacheco-Labrador, J.
et al.
(2019) ‘Sun-induced chlorophyll fluorescence I: Instrumental considerations
for proximal spectroradiometers’,
Remote Sensing
, 11(8). doi: 10.3390/rs11080923.
Povey, A. C. and Grainger, R. G. (2015) ‘Known and unknown unknowns: Uncertainty estimation in
satellite remote sensing’,
Atmospheric Measurement Techniques
, 8(11), pp. 46994718. doi:
10.5194/amt-8-4699-2015.
Saltelli, A. and Tarantola, S. (2002) ‘On the relative importance of input factors in mathematical models:
Safety assessment for nuclear waste disposal’,
Journal of the American Statistical Association
, 97(459),
pp. 702709. doi: 10.1198/016214502388618447.
SENSECO (2018a)
About SENSECO
. Available at: https://www.senseco.eu/about-senseco/.
SENSECO (2018b)
SENSECO Working Groups
. Available at: https://www.senseco.eu/working-groups/.
Shi, W. (2010)
Principles of Modeling Uncertainties in Spatial Data and Spatial Analyses
. CRC Press, an
imprint of Taylor and Francis Group, LLC.
Tennøy, A., Kværner, J. and Gjerstad, K. I. (2006) ‘Uncertainty in environmental impact assessment
predictions: The need for better communication and more transparency’,
Impact Assessment and Project
Appraisal
, 24(1), pp. 4556. doi: 10.3152/147154606781765345.
Therivel, R.
et al.
(1999)
Strategic environmental assessment
. Earthscan Publications Limited.
SENSECO Instrument and Uncertainty Questionnaire Results
SENSECO WG4 - Instruments and Uncertainty Questionnaire - Results.docx
Version 1.0 / 14.09.2020 Page 27 of 27
Wardekker, J. A.
et al.
(2008) ‘Uncertainty communication in environmental assessments: views from the
Dutch science-policy interface’,
Environmental Science and Policy
. Elsevier Ltd, 11(7), pp. 627641. doi:
10.1016/j.envsci.2008.05.005.
Woolliams, E., Hueni, A. and Gorrono, J. (2014) ‘Intermediate Uncertainty Analysis for Earth Observation
- Instrument Calibration Module - Training Course Textbook’,
National Physical Laboratory NPL
, (January
2014), p. 131. doi: 10.5167/uzh-98386.
World Meteorological Organisation (2017)
Guidelines for the Assessment of Uncertainty for Hydrometric
Measurement
. Geneva: WMO.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Vicarious calibration approaches using in situ measurements saw first use in the early 1980s and have since improved to keep pace with the evolution of the radiometric requirements of the sensors that are being calibrated. The advantage of in situ measurements for vicarious calibration is that they can be carried out with traceable and quantifiable accuracy, making them ideal for interconsistency studies of on-orbit sensors. The recent development of automated sites to collect the in situ data has led to an increase in the available number of datasets for sensor calibration. The current work describes the Radiometric Calibration Network (RadCalNet) that is an effort to provide automated surface and atmosphere in situ data as part of a network including multiple sites for the purpose of optical imager radiometric calibration in the visible to shortwave infrared spectral range. The key goals of RadCalNet are to standardize protocols for collecting data, process to top-of-atmosphere reflectance, and provide uncertainty budgets for automated sites traceable to the international system of units. RadCalNet is the result of efforts by the RadCalNet Working Group under the umbrella of the Committee on Earth Observation Satellites (CEOS) Working Group on Calibration and Validation (WGCV) and the Infrared Visible Optical Sensors (IVOS). Four radiometric calibration instrumented sites located in the USA, France, China, and Namibia are presented here that were used as initial sites for prototyping and demonstrating RadCalNet. All four sites rely on collection of data for assessing the surface reflectance as well as atmospheric data over that site. The data are converted to top-of-atmosphere reflectance within RadCalNet and provided through a web portal to allow users to either radiometrically calibrate or verify the calibration of their sensors of interest. Top-of-atmosphere reflectance data with associated uncertainties are available at 10 nm intervals over the 400 nm to 1000 nm spectral range at 30 minute Remote Sens. 2019, 11, 2401 2 of 25 intervals for a nadir-viewing geometry. An example is shown demonstrating how top-of-atmosphere data from RadCalNet can be used to determine the interconsistency between two sensors.
Article
Full-text available
The interest of the scientific community on the remote observation of sun-induced chlorophyll fluorescence (SIF) has increased in the recent years. In this context, hyperspectral ground measurements play a crucial role in the calibration and validation of future satellite missions. For this reason, the European cooperation in science and technology (COST) Action ES1309 OPTIMISE has compiled three papers on instrument characterization, measurement setups and protocols, and retrieval methods (current paper). This study is divided in two sections; first, we evaluated the uncertainties in SIF retrieval methods (e.g., Fraunhofer line depth (FLD) approaches and spectral fitting method (SFM)) for a combination of off-the-shelf commercial spectrometers. Secondly, we evaluated how an erroneous implementation of the retrieval methods increases the Remote Sens. 2019, 11, 962 2 of 31 uncertainty in the estimated SIF values. Results show that the SFM approach applied to high-resolution spectra provided the most reliable SIF retrieval with a relative error (RE) ≤6% and <5% for F687 and F760, respectively. Furthermore, although the SFM was the least affected by an inaccurate definition of the absorption spectral window (RE = 5%) and/or interpolation strategy (RE = 15%-30%), we observed a sensitivity of the SIF retrieval for the simulated training data underlying the SFM model implementation.
Conference Paper
Full-text available
To improve the reliability of spectral libraries acquired by field spectroscopy, the uncertainty of the spectral reflectance measured must be estimated and reported, taken into account as many important sources of uncertainty as possible. This work presents the initial approach to establish a complete model of uncertainty of field spectroscopy measurements that ensures the traceability chain. The first two steps towards this objective are presented: on one hand, the framework and the sources of uncertainty to create the model are reviewed, on the other hand, with the intention to be aware of the uncertainty values that can result due to environmental conditions, some empirical data using two field spectroradiometers were undertaken. Type A standard statistical methods were applied for spectral reflectance measurements acquired with a dual system of two ASD FieldSpec3 spectroradiometers over two uniform and invariant surfaces. The reproducibility in environmental ambient were evaluated measuring under variable of solar and atmospheric conditions. For VNIR region, where spectroradiometer performs better, an average of 5% of uncertainty were gathered. For SWIR region both sources, environmental and instrumental rise to 12% of uncertainty.
Article
Full-text available
Identifying and communicating uncertainty is core to effective environmental assessment (EA). This study evaluates the extent to which uncertainties are considered and addressed in Canadian EA practice. We reviewed the environmental protection plans, follow-up programs, and panel reports (where applicable) of 12 EAs between 1995 and 2012. The types of uncertainties and levels of disclosure varied greatly. When uncertainties were acknowledged, practitioners adopted five different approaches to address them. However, uncertainties were never discussed or addressed in depth. We found a lack of suitable terminology and consistency in how uncertainties are disclosed, reflecting the need for explicit guidance, and we present recommendations for improvement. Canadian Environmental Impact Statements are not as transparent with respect to uncertainties as they should be, and uncertainties in EA need to be better considered and communicated.
Article
Full-text available
This paper discusses a best-practice representation of uncertainty in satellite remote sensing data. An estimate of uncertainty is necessary to make appropriate use of the information conveyed by a measurement. Traditional error propagation quantifies the uncertainty in a measurement due to well-understood perturbations in a measurement and in auxiliary data – known, quantified "unknowns". The under-constrained nature of most satellite remote sensing observations requires the use of various approximations and assumptions that produce non-linear systematic errors that are not readily assessed – known, unquantifiable "unknowns". Additional errors result from the inability to resolve all scales of variation in the measured quantity – unknown "unknowns". The latter two categories of error are dominant in under-constrained remote sensing retrievals, and the difficulty of their quantification limits the utility of existing uncertainty estimates, degrading confidence in such data. This paper proposes the use of ensemble techniques to present multiple self-consistent realisations of a data set as a means of depicting unquantified uncertainties. These are generated using various systems (different algorithms or forward models) believed to be appropriate to the conditions observed. Benefiting from the experience of the climate modelling community, an ensemble provides a user with a more complete representation of the uncertainty as understood by the data producer and greater freedom to consider different realisations of the data.
Article
WEC-241, a 3-page illustrated fact sheet by Frank J. Mazzotti, Nicola Hughes, and Rebecca G. Harvey, explains what environmental monitoring is, its scientific basis, how it relates to adaptive management, its cost effectiveness, and what it means for the Everglades Restoration Plan. Includes references. Published by the UF Department of Wildlife Conservation and Ecology, November 2007. WEC241/UW283: Why do we need environmental monitoring for Everglades restoration? (ufl.edu)
Article
This study investigates practices of uncertainty disclosure and communication in Canadian environmental assessment (EA) in the context of the Joslyn North Oil Sands Mine project. Nineteen interviews with project stakeholders were conducted, revealing significant uncertainties about the project, attributed to multiple factors including lack of clarity in the terms of reference and requirements of the proponent; the project’s predicted impacts and proponent commitments to mitigation; cumulative effects and the potential for effects interaction with other projects; Aboriginal engagement, including engagement processes and broader socio-political context; and poor uncertainty disclosure and communication practices. Some uncertainties were disclosed but at times downplayed to render the project more palatable through the EA process. Informants stated that this is not an uncommon occurrence in oil sands EA. Recommendations to improve uncertainty disclosure and communication in EA and enhance the consideration of uncertainties in decision-making are provided.
Book
When compared to classical sciences such as math, with roots in prehistory, and physics, with roots in antiquity, geographical information science (GISci) is the new kid on the block. Its theoretical foundations are therefore still developing and data quality and uncertainty modeling for spatial data and spatial analysis is an important branch of that theory. Principles of Modeling Uncertainties in Spatial Data and Spatial Analyses outlines the foundational principles and supplies a firm grasp of the disciplines' theoretical underpinnings. Comprehensive, Systematic Review of Methods for Handling Uncertainties The book summarizes the principles of modeling uncertainty of spatial data and spatial analysis, and then introduces the developed methods for handling uncertainties in spatial data and modeling uncertainties in spatial models. Building on this foundation, the book goes on to explore modeling uncertainties in spatial analyses and describe methods for presentation of data as quality information. Progressing from basic to advanced topics, the organization of the contents reflects the four major theoretical breakthroughs in uncertainty modeling: advances in spatial object representation, uncertainty modeling for static spatial data to dynamic spatial analyses, uncertainty modeling for spatial data to spatial models, and error description of spatial data to spatial data quality control. Determine Fitness-of-Use for Your Applications Modeling uncertainties is essential for the development of geographic information science. Uncertainties always exist in GIS and are then propagated in the results of any spatial analysis. The book delineates how GIS can be a better tool for decision-making and demonstrates how the methods covered can be used to control the data quality of GIS products.