Content uploaded by Norman Kerle
Author content
All content in this area was uploaded by Norman Kerle on Oct 01, 2020
Content may be subject to copyright.
1/2020
7
DEval Discussion Paper
1/2020
A PROOF-OF-CONCEPT OF INTEGRATING
MACHINE LEARNING, REMOTE SENSING,
AND SURVEY DATA IN EVALUATIONS
The measurement of disaster resilience in the Philippines
2020
Malte Lech
Saman Ghaffarian
Norman Kerle
Gerald Leppert
Raphael Nawrotzki
Kevin Moull
Sven Harten
Abstract
Disaster resilience is a topic of increasing importance for policy makers in the context of climate change.
However, measuring disaster resilience remains a challenge as it requires information on both the physical
environment and socio-economic dimensions. In this study we developed and tested a method to use remote
sensing (RS) data to construct proxy indicators of socio-economic change. We employed machine-learning
algorithms to generate land-cover and land-use classifications from very high-resolution satellite imagery to
appraise disaster damage and recovery processes in the Philippines following the devastation of typhoon
ed RS-
region surrounding Tacloban City in the central east of the Philippines. We then combined the RS-based proxy
indicators with detailed socio-economic information collected during a rigorous-impact evaluation by DEval
d that fastest post-disaster recovery occurred in urban
barangays that received sufficient government support (subsidies), and which had no prior disaster
experience. In general, socio- –
–
recovery performance only to some extent. Rather than providing an in-depth statistical analysis, this study
is intended as a proof-of-concept. We have been able to demonstrate that high-resolution RS data and
machine-learning techniques can be used within a mixed-methods design as an effective tool to evaluate
disaster impacts and recovery processes. While RS data have distinct limitations (e.g., cost, labour intensity),
they offer unique opportunities to objectively measure physical, and by extension socio-economic, changes
over large areas and long time-scales.
Keywords: remote sensing, disaster risk management, socio-economic change, machine learning, climate
change
Zusammenfassung
Zunehmende Wetterextreme und Naturkatastrophen sind Folgen des Klimawandels. Aufgrund dieser
steigenden Risiken rückt die Resilienz der Bevölkerung im Katastrophenfall als zentrales Thema in den
Vordergrund und hat zunehmende Bedeutung für politische Entscheidungstragende. Dennoch bleibt die
Messung des mehrdimensionalen Konzepts der Katastrophenresilienz eine Herausforderung, da sie
Informationen sowohl über die physische Umgebung als auch sozioökonomische Faktoren erfordert. In
dieser Studie wird eine Methode entwickelt, um aus Fernerkundungsdaten (RS-Daten) Indikatoren zu
entwickeln, die Aspekte des sozioökonomischen Wandels approximieren und somit messbar machen (Proxy-
Indikatoren).
Zu diesem Zweck wurden Algorithmen des maschinellen Lernens eingesetzt. Mit Hilfe dieser Algorithmen
wurden aus hochauflösenden Satellitenbildern Klassifizierungen für Landstruktur und Landnutzung
konstruiert, um Katastrophenschäden und Wiederaufbauprozesse auf den Philippinen nach der Zerstörung
ssen. Aus den RS-
Barangays (Dörfer) in der Region um die Stadt Tacloban im zentralen Osten der Philippinen berechnet. Diese
auf RS-Daten basierenden Indikatoren wurden mit detaillierten sozioökonomischen Informationen
kombiniert, die für eine DEval-
Analyse zeigen, dass der schnellste Wiederaufbau nach der Katastrophe in städtischen Barangays zu
beobachten war, die ausreichend staatliche Unterstützung (Subventionen) erhielten und über keine
Katastrophenerfahrung verfügten. Im Vergleich hatten soziodemografische Faktoren allgemein stärkere
--
bedingter Bezug zwischen der deutschen Entwicklungszusammenarbeit und den Wiederaufbauerfolgen
festgestellt werden.
Diese Studie versteht sich als Nachweis der Machbarkeit, weniger als detaillierte statistische Analyse. Sie
belegt, dass hochauflösende RS-Daten und Techniken des maschinellen Lernens innerhalb eines integrierten
Methodendesigns als effektives Werkzeug zur Bewertung von Katastrophenauswirkungen und
Wiederherstellungsprozessen eingesetzt werden können. Trotz spezifischer Einschränkungen (hohe Kosten,
Arbeitsintensität etc.) bieten RS-Daten einzigartige Möglichkeiten sowohl Umweltbedingungen als auch
sozioökonomische Veränderungen über große Gebiete und lange Zeiträume hinweg objektiv messen zu
können.
Keywords: Fernerkundung, Katastrophenrisikomanagement, Sozioökonomischer Wandel, Maschinelles
Lernen, Klimawandel
Imprint
Authors
Dr Malte Lech1
Saman Ghaffarian2
Prof Dr Norman Kerle3
Dr Gerald Leppert4
Raphael Nawrotzki5
Kevin Moull6
Dr Sven Harten7
Responsible
Dr Sven Harten
Design
MedienMélange:Kommunikation!, Hamburg
www.medienmelange.de
Editing
Jannet King
Bibliographical reference
Lech, M., S. Ghaffarian, N. Kerle, G. Leppert, R.
Nawrotzki, K. Moull, S. Harten (2020): A Proof-of-
Concept of Integrating Machine Learning, Remote
Sensing, and Survey Data in Evaluations. The
Measurement of Disaster Resilience in the
Philippines. DEval Discussion Paper
German Institute for Development Evaluation
(DEval), Bonn.
© German Institute for Development Evaluation
September
ISBN 978-3-96126-114-7 (PDF)
Published by
German Institute for Development
Evaluation (DEval)
Fritz-Schäffer-
-
E-Mail: info@DEval.org
www.DEval.org
The German Institute for Development Evalua-
tion (DEval) is mandated by the German Federal
Ministry for Economic Cooperation and Devel-
opment (BMZ) to independently analyse and as-
sess German development interventions.
DEval Discussion Papers present the results of the
ongoing scientific study of evaluation and the
effectiveness of development cooperation, thus
contributing to relevant expert debates on evalu-
ation, social science methods and development
cooperation. The discussion papers are geared
towards academics and practitioners in the field
of evaluation, methodology research and devel-
opment cooperation.
DEval Discussion Papers are written by DEval
evaluators and external guest authors. In con-
trast to our evaluation reports, they do not con-
tain any direct recommendations for German and
international development organizations.
Although DEval Discussion Papers are internally
peer-reviewed, the views expressed in them are
only those of the authors and – unlike our evalu-
ation reports – do not necessarily reflect those of
DEval.
All DEval Discussion Papers can be downloaded as
a PDF file from the DEval website:
http://www.deval.org/en/discussion-papers.html
1 Dr Malte Lech, Former Evaluator at the German Institute for Development Evaluation (DEval). Contact: malte.lech@gmail.com.
2 Saman Ghaffarian, University of Twente. Contact: s.ghaffarian@utwente.nl.
3 Prof Dr Norman Kerle, University of Twente. Contact: n.kerle@utwente.nl.
4 Dr Gerald Leppert, Senior Evaluator – Team Leader at the German Institute for Development Evaluation (DEval). Contact:
gerald.leppert@DEval.org.
5 Dr Raphael Nawrotzki, Evaluator at the German Institute for Development Evaluation (DEval). Contact: raphael.nawrotzki@DEval.org.
6 Kevin Moull, Evaluator at the German Institute for Development Evaluation (DEval). Contact: kevin.moull@DEval.org.
7 Dr Sven Harten, Deputy Director at the German Institute for Development Evaluation (DEval). Contact: sven.harten@DEval.org.
The measurement of disaster resilience in the Philippines |
CONTENTS
Abbreviations and Acronyms ........................................................................................................................... vii
Introduction ...............................................................................................................................................
Using Remote Sensing Data to Measure Socio-Economic Change and Climate Change ...........................
The Proxy-based Approach to Measure Socio-Economic Change .............................................................
Case Study of Disaster Risk Management in the Philippines .....................................................................
Study Context ..................................................................................................................................
Climate Change in the Philippines ......................................................................................
Disaster Risk Management in the Philippines ....................................................................
Study Region .......................................................................................................................
Objectives ...........................................................................................................................
Data and Methods ...........................................................................................................................
Data .....................................................................................................................................
Data Processing and Variable Construction .....................................................................
Statistical Approach ..........................................................................................................
Case Study Results and Discussion ................................................................................................
Land Use and Land Cover Proxies .....................................................................................
Integration of RS Analysis into Evaluative Work ..............................................................
Discussion and Conclusion .......................................................................................................................
Summary of Case Study Results ....................................................................................................
Outlook: Strengths and Limitations of the Approach ....................................................................
References ...............................................................................................................................................
Annex .. ....................................................................................................................................................
The measurement of disaster resilience in the Philippines |
Figures
Impact of climate change and socio-economic change ......................................................
The conceptual framework for post-disaster recovery assessment using remote sensing-
based proxies ......................................................................................................................
Seven municipalities selected for the case study ...............................................................
..............................
Tables
Examples of proxy indicators computed from RS data ......................................................
An overview of the relevant goals of the EnRD programme for DRM measures ...............
Summary statistics of relevant socio-economic and demographic variables ..................
......................
Summary statistics of selected RS-based socio-economic proxies across all study
barangays employed in the statistical analysis.................................................................
Damage by socio-economic status, disaster risk management and local governance ....
Recovery status by socio-economic status .......................................................................
Recovery performance by disaster risk management ......................................................
Recovery performance by local governance ....................................................................
Recovery performance by donor support ........................................................................
The measurement of disaster resilience in the Philippines | vii
ABBREVIATIONS AND ACRONYMS
BMZ German Federal Ministry for Economic Cooperation and Development
CLUP Comprehensive Land-Use Plan
CNN Convolutional neural networks
DEval German Institute for Development Evaluation
DILG Department for the Interior and Local Governance
DMSP Defense Meteorological Satellite Program
DRM Disaster risk management
EnRD Environment and Rural Development (Programme)
GBM Gradient-boosting method
GEE Google Earth Engine
GIS Geographic Information System
GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit GmbH
IRA Internal Revenue Allotment
ITC University of Twente’s Faculty for Geo-Information Science and Earth Observation
LBP Local binary patterns
LCLU Land cover and land use
NRG Natural resource governance
PHP Philippine Pesos
RS Remote sensing
SVM Support vector machines
UAV Unmanned aerial vehicle
VHR Very high resolution
VIIRS Visible infrared imaging radiometer suite
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 1
Deval Discussion
1. INTRODUCTION
International development cooperation is increasingly concerned with addressing climate change and
assisting vulnerable populations in adapting to its adverse impacts. Evaluating adaptation to climate change
is unique because changes in the natural environment (climate change) and the socio-economic situation
(adaptation) both need to be measured.
This leads to numerous challenges. First, climate change manifests itself as a heterogeneous group of
environmental impacts. These range from an increase in sudden-onset natural disasters such as tropical
storms to more gradual changes such as desertification and sea-level rise. These environmental changes can
impact large geographical areas and develop over long time periods. Second, human responses to climate
change are highly diverse, including changes in agricultural production techniques (e.g., drought-resistant
seeds, irrigation systems), infrastructure, and built environment (e.g., storm shelters, dams, sea-walls), or
mobility patterns (e.g., climate-related human mobility). Assisting populations in adapting to these
challenges requires a diverse set of programmes and strategies.
Surveys and censuses have been used by evaluators to measure social and economic changes. However, such
methods of data collection are both labour and cost intensive and not very flexible in terms of their scope
and application. For example, it is often not possible to collect relevant baseline data prior to a natural
disaster, making it difficult to evaluate disaster recovery. Similarly, it would require multiple rounds of survey
data collection to capture gradual climatic changes and related impacts that develop over decades. However,
recent technological developments in remote sensing (RS) methods and data availability may facilitate the
evaluation of climate-change adaptation and post-disaster recovery processes.
RS is a technology used to detect, collect, and monitor the earth’s surface by measuring its reflection and
emission of radiation. This is done by a collection platform that is remote from the target being observed,
such as a satellite, aeroplane, or unmanned aerial vehicle (UAV). RS sensor systems are able to collect
information ranging from visible to invisible (e.g. radar and infrared) features that can be traced using their
specific spectral signature. As such, RS data are ideal to measure changes in the physical environment,
including climate change. Moreover, some changes in the physical environment may be used as proxy
indicators or indirect measures of socio-economic changes. For example, RS data can be used to measure the
extent of slums as an indicator of poverty in a given region. However, the most comprehensive approach
would be a mixed-methods evaluation design that combines RS data with survey data to provide a complete
picture of the impacts of climate change as well as adaptation and recovery processes. As such, this study8
has two main objectives:
develop a (semi) automated approach to generate RS-based proxies for socio-economic change,
which can be used to assess post-disaster recovery;
combine these RS-based proxies with survey data to contextualize the recovery processes.
These objectives highlight the exploratory nature of this study. Specifically, it can be seen as a “proof of
concept” of using machine-learning-based RS data analysis to measure environmental as well as socio-
economic changes and of combining RS with survey data for a more comprehensive analysis of recovery
processes.
We first develop a conceptual framework for the generation and use of RS-based proxies of socio-economic
change. Second, this framework is applied to the case of post-disaster recovery in the Philippines following
the devastation of typhoon Haiyan.9
8 This study was designed as a joint project by the University of Twente’s Faculty for Geo-Information Science and Earth Observation (ITC) Earth
Systems Analysis Unit and the German Institute for Development Evaluation (DEval).
9 Typhoon Haiyan, which is the subject of this study, devastated large parts of the central Philippine archipelago on –
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 2
Deval Discussion
context for the use of RS data to measure socio-
the proxy-based approach to measure socio-
paper. Based on his study, a recent publication highlights the benefits and limitations of
using remote sensing to support evaluation work.
2. USING REMOTE SENSING DATA TO MEASURE SOCIO-ECONOMIC
CHANGE AND CLIMATE CHANGE
One of the most pressing questions of our time is whether humanity can find effective answers to climate
change. Current state-of-the-art prognoses anticipate a likelihood that certain extreme weather events will
increase in magnitude and frequency . In addition to these
irregularly occurring natural shocks, overarching long-term climate-change phenomena exist. Examples of
these so-called stressors are desertification, an increasing global temperature, and rising sea-levels. Both
shocks and stressors induce social stress and may cause lasting damage to the human environment (Turner
.
Adaptation processes play a central role in the search for solutions to climate change. The term ”adaptation”
refers to the process of adjusting to climate change and its effects in order to moderate or avert the harm to
human systems caused by stressors and shocks . However, human response to stressors is
different from the human response to shocks. While stressors require adaptation of human and natural
systems, typically through increased adaptive capacities or established enabling environments (i.e. system-
level changes in the environmental, socio-economic, and
but different response strategy.
While a proper response to shocks also benefits from an increase in adaptive capacities and economic
capabilities, it requires, in addition, systematic disaster risk management (DRM) to increase the preparedness
for likely disasters. If proper proactive DRM is absent in the event of a disaster, ad hoc reactive risk
management (i.e. risk coping) is required, which is typically less effective (Matthew.
The performance of post-disaster recovery depends on all response measures: the level of adaptive
capacities and economic capabilities, and, in particular, established proactive DRM.
Interdependencies between experienced shocks and vulnerability also impact disaster-related losses and
damages, as well as recovery performance. Bunched or repeated shocks can result in increased vulnerability
for future crises . A high degree of exposure to shocks and stressors may
reduce communities’ future adaptive capacities . By contrast,
resilient communities show a high capacity for adaptation and are able to maintain or retain their essential
functions even in the case of extreme weather events .
The adaptation process underlies a dynamic logic of social progress and is therefore directly or
indirectly connected to a range of social and economic factors
climate-related shocks and stressors may trigger adaptation and risk-management activities and may
ultimately lead to socio-economic change. In this sense, socio-economic change refers to “the way the state
of a system changes over time (…)” due to adaptation . The scale and dynamics of socio-
economic changes are highly variable, from the seemingly static traditional rural communities to the
dynamic, growing megacities that are in perpetual development and change.
In response to shocks and stressors, climate-change adaptation and risk management can trigger socio-
economic change. With reference to the previous example of a typhoon, affected communities might
implement early-warning systems and evacuation plans, the construction of sea-walls and, ultimately,
relocation from hazard zones – activities that increase their resilience with respect to future shocks and
stressors. However, as mentioned above, experienced shocks may also reduce the future adaptive capacity
of communities. If a typhoon were to completely destroy the infrastructure of a community, a social system
might struggle to adapt in a timely manner. As a consequence of the lack of adaptation, people’s livelihoods
deteriorate.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 3
Deval Discussion
Furthermore, vulnerability to climate change arises from certain socio-economic conditions, such as the
economic capability of households or the existing infrastructure . While a typhoon
may completely destroy the harvest of a rural farmer, resulting in debt and poverty, an urban resident whose
shop is destroyed may find employment in a local factory. These examples of typhoon recovery show the
complex relationship of shocks and stressors on one side and socio-economic change on the other side. Their
relationship is mediated by the extent of human adaptation and (proactive) risk management, which in itself
depends on current adaptive capacities and economic capabilities.
Figure 1 Impact of climate change and socio-economic change
Source: own figure.
For these reasons, measuring changes in socio-economic conditions is highly important for politicians, urban
planners, and policy makers alike. Frequently, censuses or population surveys are employed to measure
changes in socio-economic conditions . Yet, these forms of data collection are expensive
measure dynamic socio-economic changes, particularly if these changes are caused by sudden-onset climate
events such as tropical storms. An emerging alternative to measuring and tracking socio-economic changes
is the use of remote-sensing (RS) geospatial data.
RS data are collected via some type of sensor (similar to a camera) attached to a satellite, aeroplane or drone
. RS techniques can be used to measure most structural features of the outer layer of the earth,
including elevation and topography, vegetation cover, and infrastructure, to mention just a few. This makes
RS ideally suited to map the land cover over large areas. Moreover, satellites orbit the earth on regular and
short temporal intervals, which allows for an effective monitoring of changes in surface features (e.g.,
changes in building structures or vegetation coverage). Satellites have been orbiting the earth for decades
now, and the archived RS data allows a detailed study of changes over long periods of time.
as an effective tool
to measure disaster impacts and recovery. For example, the spatial resolution has increased from –m
necessary to assess structural damage and land-cover changes. In addition, the revisit time (minimum time
between two observations) has declined from several weeks to a few days, meaning that data are now often
available within hours after an event.
In DRM, optical images are the primary source of information as they allow detailed information on structural
changes and damages. Usually these RS images are captured during daylight hours but, recently, night-time
images have emerged as a useful source of information. The Defense Meteorological Satellite Program
(DMSP) generates night-time images that provide insightful time series of mainly anthropogenic light
emissions (mainly corresponding to exterior illumination such as street lamps). Drastic declines in local light
Impact of
climate change
Stressors
Shocks
E.g. increasing
number /frequency
of severe weather
events and natural
disasters
E.g. increasing global
temperature, water
shortages, droughts
Socio-economic
change
Change of state in
economic and social
systems. Systemic
change impacts
adaptive capacities.
Adaptation and
risk management
Extent of human adaptation or
proactive risk management to
respond to actual or anticipated
impacts of climate change.
mitigates
triggers leads to
capacitates
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 4
Deval Discussion
emission have been linked to disaster damage , while the subsequent increase in light may be
used as a proxy for recovery .
As an alternative to optical images, thermal sensors can be used to measure irregular sea-surface
temperature increases, which have been linked to stronger tropical storms. Similar, ambient temperatures
can be used to monitor the threat of heatwaves and wildfires. In addition to optical and thermal images,
electromagnetic radiation can be used to measure detailed topographic features. As an example, in the
Philippines, researchers used detailed topographic information to recalculate the storm-surge hazard
following a major tropical storm .
The fact that RS primarily captures the physical environment can be seen as a severe limitation when the
socio-economic situation or changes therein are of interest. However, many socio-economic parameters can
be readily linked to physical proxies that are in turn observable in RS data. For example, the number of
of
repair (as far as visible), building density etc. can be linked to the socio-economic condition of their
inhabitants . Likewise, progress has been made to approximate crops grown, the
method of planting and care, and crop yield from RS data and to link
it to the socio-economic reality on the ground; automated monitoring systems exist for some of these.
For example, in response to adverse climate change, RS data may be used to measure crop destruction and
productivity losses in flooded areas, or the declining number of livestock as a result of prolonged drought.
Moreover, recent studies have used RS data to identify and characterize slums and similarly deprived areas
. Being able to locate such sub-standard dwellings in turn can
be linked to both physical and social vulnerability .
However, correctly classifying RS data to capture socio-economic conditions and changes is methodologically
challenging. One promising approach that we explore in this study is the use of machine-learning algorithms.
These algorithms allow a computer to “learn” how to recognize certain patterns from a large set of examples,
including patterns not even known to the researcher. For example, to capture evidence of disaster damage,
we may provide the computer with a large number of damage examples. This enables the machine-learning
algorithm to understand the desired damage class in its varying forms and allows it to accurately classify
disaster damage across diverse geographic locations and settings .
3. THE PROXY-BASED APPROACH TO MEASURE SOCIO-ECONOMIC
CHANGE
As briefly outlined in Section -
economic changes. As such, this study employs “proxies”, or indirect measurements, following initial studies
to assess vulnerability , resilience , damage (Bevington et al.,
, and recovery (Rubi.
-based approach to measuring post-disaster recovery. Two
main steps can be distinguished. The first step constitutes the damage assessment. In this step, evaluators
extract RS data shorof
damage caused by the disaster. In the second step, evaluators then extract RS data directly after the disaster
took place time (Tn), which then allows the detection of the degree of
post-disaster recovery. The subscript “n” indicates that recovery may be measured at multiple time points,
as reconstruction may
take varying amounts of time.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 5
Deval Discussion
Figure 2 The conceptual framework for post-disaster recovery assessment using remote sensing-based
proxies
Source: own figure.
The conceptual framework of the proxy-based approach was developed generically to be applicable across
study sites, disaster types, and geo-spatial contexts (Kerle . Changes in different physical features
are associated with different changes in the socio-
provides a selection of proxies as well as a brief description of what is measured, and the socio-economic
dimension that is approximated.
Table 1 Examples of proxy indicators computed from RS data
Proxy Proxy Name Description of measurement Approximation of socio-economic
dimension
1 Buildings The area covered by buildings.
Changes in this measure permit an
estimation of the damage inflicted and
the scale of reconstruction needed
Economic loss and recovery
2 Proportion of
built-up area
Area covered by buildings expressed
as proportion of total area
Measure of the level of
urbanization
3 Impervious
surface
Changes in this measure permit a
computation of the amount of debris
covering the surface
Debris as obstacle for economic
business operations
4 Large-scale
industry
Measurement of buildings associated
with large scale-industry
Indicator of industrial capacity and
local labour market
5 Vehicles The number of vehicles Economic activity
6 Boats The number of boats Level of economic activity in
fisheries sector
7 Arable land Measurement of the area that can be
used for agricultural production
Use of agriculture for livelihood
recovery
8 Proportion
vegetated area
Area covered by vegetation expressed
as a proportion of total area
Measure of access to natural
resources
Proxies map extracted
just after disaster
Proxies map extracted
before disaster
Proxies map extracted
after disaster
Time
Disaster
T0 T1 Tn
Change-detection for
damage assessment
Change-detection for
recovery assessment
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 6
Deval Discussion
Proxy Proxy Name Description of measurement Approximation of socio-economic
dimension
9 Roof material Measurement of the texture and
colour of roofs
Roof material is associated with
housing quality and income level of
occupants. It may be used as
indicator of slum areas and
presence of poverty.
Source: own table.
A few examples help to illustrate the proxy-based approach . The difference in
the area covered by buildings (Proxy
structural damage in a given community. When the average economic value of buildings in a neighbourhood
is known, we can assess the economic losses caused by a natural disaster
then use this information to approximate the economic losses due to an obstruction of important
transportation networks . In terms of recovery processes,
on a given s
population fully recovers from a disaster, we would anticipate that at least the same number of boats and
vehicles are present after a sufficiently long recovery period . Similarly,
RS data can
indicators can be used as a proxy to measure the expansion of squatter settlements or slums (de Almeida et
.
4. CASE STUDY OF DISASTER RISK MANAGEMENT IN THE PHILIPPINES
4.1 Study Context
4.1.1 Climate Change in the Philippines
Climate change will lead to an increase in the frequency and severity of extreme weathers events such as
tropical cyclones and floods, associated with substantial socio-economic cost and in most cases also loss of
life (Combest-. Recently, Löw highlighted that total economic losses caused by
Asia-
fatalities .
Given these figures, it is not surprising that the Philippines is especially affected by the burdens of natural
disasters and severe weather events . For decades, the Philippines has been among the top
five countries in terms of annual number of natural disasters. Its geographic location makes it highly exposed
natural disasters have affected the Philippine
. While
not every occurrence of a typhoon is a catastrophic event, and not all Philippine regions are exposed to the
same degree to the potential adverse effects, the numbers underline the burden caused by such events.
According to figures by Jha et al. based on the EM-
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 7
Deval Discussion
4.1.2 Disaster Risk Management in the Philippines
It is thus not surprising that DRM plays an important role in the Philippine public administration. As a reaction
to these challenging conditions, national policies for disaster risk mitigation were formulated (Government
and have consequently been mainstreamed into guidelines for urban and land-use
planning - . Furthermore, the process of
implementing functional DRM was supported by a multitude of international donor organizations, which
contributed to capacity development for DRM in addition to local stakeholders, provided equipment and
machinery, and engaged in larger technical measurements to reduce disaster risk, specifically for coastal
hazards. While this was applied throughout the Philippines, the Eastern Visayas Region has been a focal area
for these support measures.
The relatively strong focus on improving capacities in the field of DRM in the Visayas area is therefore not
surprising. Tacloban City and its adjacent municipalities are frequently affected by tropical cyclones passing
through the Philippine archipelago. One of the most severe events occurred on –
typhoon Haiyan made landfall in Eastern Samar, before passing through San Pablo Bay and Leyte. Official
estimates approximate the numb(Del Rosario,
. Following the destruction caused by the typhoon, the region received the lion’s share of post-disaster
reconstruction support (both nationally and internationally). However, even before typhoon Haiyan triggered
a substantial development effort in DRM, the Philippines, and specifically the socio-economically
disadvantaged Visayas region, were recipients of international and German donor-support schemes.
The German Gesellschaft für Internationale Zusammenarbeit (GIZ) and its predecessor organizations have
been the main German development organizations involved in capacity development of Philippine
municipalities and provincial planning authorities in the field of land-use planning, natural resource
the Philippines were implemented through the Environment and Rural Development (EnRD) programme of
Philippine–German cooperation. It was mandated by the German Federal Ministry for Economic Cooperation
foster
sustainable rural development in the Philippine Visayas Region by means of support in policy fields such as
Environmental Protection, Food and Nutrition, DRM, and improved municipal land-use planning .
The programme was later complemented by a post-Haiyan reconstruction support component to provide
While the Natural Resource Governance (NRG) component of the EnRD programme aimed at integrating
DRM-sensitive planning into provincial and municipal land-use planning, the DRM component included more
immediate measures, such as the implementation of flood early warning systems and mitigation measures
-use planning can significantly enhance the
mitigation of disaster risks, by imposing regulations on settlement in disaster prone areas, and by enforcing
building standards, making constructions able to withstand the effects of natural disasters (Becker et al.,
. At the same time, planning-related measures usually take time to materialize and do not show
immediate results .
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 8
Deval Discussion
Table 2 An overview of the relevant goals of the EnRD programme for DRM measures
Disaster Risk Management
(DRM)
Natural Resource Governance
(NRG)
Support for reconstruction in Haiyan-
affected areas
Establishment of Flood Early
Warning Systems, including
precipitation pattern data
Development and approval of
Provincial Development and
Physical Framework Plans
Development and revision of disaster
and conflict preventive reconstruction
plans
Establishment of Risk
Mitigation measures such as
Slope Stabilization and
Reforestation
Development approval of
Comprehensive Land-Use
Plans (CLUP)
Implementation of reconstruction
plans in cooperation with
governmental, civil and non-
government agencies and
stakeholders. Reconstruction efforts
include, for example, shelter, streets,
schools, health and evacuation
centres, waste management, energy
and water supply, and the
revitalization of the economy
Increased stakeholders’
awareness for DRM
Development and approval of
-year Barangay Development
and Annual Investment Plans
Capacitated LGU include DRM
in munici-pal land-use and
development planning
Establishment of Capacity
Development Networks
Source: GIZ .
4.1.3 Study Region
Three barangays10 in each of seven municipalities in the greater Tacloban area were selected. These
barangays were covered in DEval’s impact evaluation of enhanced land-use planning.
The survey of the impact evaluation covered a large area (the whole province of Leyte as well as neighbouring
provinces). In the study region, many municipalities were affected by typhoon Haiyan to some extent.
However, we limit this study to seven municipalities due to the following reasons. First, we needed to ensure
that for the geographic area very high-resolution images were available. Second, RS data come at a cost, and
budget constraints required us to narrow the selection. Despite these limitations, the case-study region and
the selected barangays offer a unique combination in terms of
and their statd a combination of multi-
temporal RS imagery, ranging from before typhoon Haiyan to the phase immediately after the event, as well
as imagery of the reconstruction phase in the following years. More details on the RS imagery is provided in
section
10 A barangay is the Philippine equivalent of a village or a part
administrative hierarchy. The barangay captain is the official head of the village, elected for a period of three-years, and responsible for all
administrative duties in the village.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 9
Deval Discussion
Figure 3 Seven municipalities selected for the case study
4.1.4 Objectives
The objective of the case study is to demonstrate the process involved in analysing high-resolution RS
imagery to measure socio-economic changes such as disaster resilience. We explore the potential for using
automated methods (e.g., machine learning), and for integrating RS analysis into existing evaluation
approaches.
4.2 Data and Methods
4.2.1 Data
Remote Sensing Data
The imagery used in this case
- and Pleiades
(Satellite Imaging four
seven municipalities at a cost of about
USD . The multi-spectral images are very high resolution,
. See Annex A for detailed source information
regarding the purchased images.
Survey Data
To demonstrate the application of RS data in evaluations, we linked the geospatial information with survey
data collected during the DEval impact evaluation on enhanced land-use planning in the Philippines (Leppert
. For the evaluation, extensive panel survey data at the household, barangay, and municipal levels
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 10
Deval Discussion
11 Control municipalities were selected from all available
municipalities in the two study regions using a propensity-score matching technique (Garcia Schustereder et
. Within each municipality, three barangays were randomly selected. Either the barangay captain
or, in their absence,
barangay-level interviews. In each of the barang
at household level was negligible at (e.g., barangay, municipality).
For more detail on the survey data see Leppert et al. . In the following analysis, we will mostly refer to
the data collected at the barangay (village) level, while other analyses refer to municipal-level data.
With regard to the RS data
to budget constraints and the complicated processing structure. This number was further reduced to a
shadow rendering the
computation of some RS-based proxies infeasible. We purposely selected barangays that had been badly
affected by typhoon Haiyan. While we tried to include both intervention (SIMPLE-treated municipalities) and
control municipalities in the selection for the case study, because of the way the GIZ intervention had rolled
out, with many intervention municipalities located near Tacloban City, SIMPLE-supported barangays are
over-represented in our case study.
The survey data at the barangay-level contain extensive information on planning, DRM and previous
exposure to disasters. Several items provided information on the barangays’ affectedness by typhoon Haiyan.
The barangay dataset was completed by socio-economic and demographic information derived from the
Department for the Interior and Local Governance (DILG), including structural characteristics of barangays in
-demographic information.
4.2.2 Data Processing and Variable Construction
Remote Sensing Data Processing and Development
For the measurement of disaster effects and recovery processes with very high-resolution RS imagery, we
needed to classify a very large number of pixels. In order to detect disaster-related patterns in land cover and
land use (e.g., debris, structural damage to buildings), we employed machine-learning algorithms.
Specifically, we used a certain type of Gradient Boosting Method (GBM). GBM is a supervised classification
technique that belongs to regression and classification tree models . The GBM-based
methods are widely used for RS data analysis, such as scene classification
et al. However, the traditional GBMs need to be tuned for several parameters and are more likely to
suffer from overfitting than other machine-learning algorithms such as Support Vector Machines (SVM). In
(Chen and Guestr developed the Extreme Gradient Boosting (XGBoost) method, which can be
considered a regularized version of GBM that overcomes most of its limitations. The superiority of the
XGBoost-based approaches for LCLU classification of very high-resolution RS images was shown in recent
studies . Given these advantages, we used the XGBoost method to
compute proxy maps from very high-resolution satellite images in this study .
The computation of the RS-based proxies of social and economic change was a four-step procedure, as
are: pre-processing, land-cover and land-use
classification, post-processing, and extraction of proxies .
-processing: Satellite images obtained at different time points may have different inclination angles
of the sensor as well as inaccuracies in geo-referencing, leading to the coordinates not necessarily
matching perfectly. We therefore had to rectify and register the images to allow for a correct comparison
between LCLU maps obtained at different points in time. In addition, we merged satellite image tiles
11 “switched” to the intervention
group. The size of control g
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 11
Deval Discussion
(mosaicking) to completely cover the particular region of interest (ROI). Finally, we employed shapefiles
for ROIs to clip the rectangular raster images to the barangay boundaries, and used those for the image
classification described next. All pre-processing steps were conducted using ArcGIS software.
: We employed the XGBoost algorithm for the land-cover and land-use classification.
In order to recognize patterns for an accurate classification, the XGBoost algorithm requires “training
areas”. These training areas were selected using Environment for Visualizing Images (ENVI) image analysis
software, and used as input for the main classification model, which was developed in a programming
environment (Python script).12 We improved the classification precision by providing the XGBoost
algorithm not only with spectral information (e.g., different colour bands) but also with information on the
texture of a given land-cover and land-use (LCLU) class. To this end, we employed Local Binary Patterns
(LBPs) as an effective textural information extraction method .
Based on a grey-level co-, such
as mean, variance, homogeneity, and entropy for each image. To differentiate building types, the roof
colours were detected, based on the brightness values of the pixels.
-processing: Post-processing involved removal of pixels in areas obscured by clouds as well as the
modification of mixed classes.
: We first obtained the number of pixels for each LCLU class for a given barangay.
We then computed the coverage area (in m) for relevant LCLU classes and expressed this number as a
percentage of the total barangay area. However, some proxies, such as the presence of vehicles and boats,
which could not be extracted from the LCLU maps, were extracted manually using both panchromatic
(grayscale) and multispectral (multicolour) images.13
and
and thr Section
points allows us to assess disaster damage and recovery processes.
Although the developed approach provided overall good accuracies for LCLU classifications, we observed
some inaccuracies in the results due to a number of reasons. First, the presence of clouds led to
misclassifications. Similarly, the algorithm had difficulties in correctly classifying pixels in areas where cloud
shadow influenced the visual appearance of the landscape. Second, different radiometric and spectral values
in the merged images for one barangay led to inaccurate classification results for trees, crops, and grass
lands.
Survey Data
The survey data used in this study was collected during DEval’s impact evaluation on comprehensive land-
use planning in the Philippines . For the impact evaluation, the survey data was cleaned
and processed (for details see . mmary statistics of the relevant socio-
economic and demographic variables that we obtained from and employed in this study.
12 The selection of the training areas was challenging for some classes, particularly for the land-use classes. For examples, distinguishing large-scale
industrial buildings from formal buildings without using auxiliary data was difficult. Hence, we used our knowledge of the area and the
panchromatic images, in addition to multispectral images, to select the training areas as accurately as possible.
13 The vehicles and boats were counted manually because there is not yet an efficient and accurate method for automated extraction.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 12
Deval Discussion
Table 3 Summary statistics of relevant socio-economic and demographic variables
Variable N Unit Mean SD
Structural characteristics
Municipal income 20 1 = High, 0 = Low 0.85 0.366
Level of urbanization 20 1 = City, 0 = Rural 0.150 0.366
Subsidies (Internal Revenue
Allotment)
20 Philippine Pesos 115,000,000 15,400,000
DRM-related characteristics
Perceived disaster preparedness 20 Scale: 1 = Low, 10 = High 7.3 2.27
Previous disaster experience 19 1 = Yes, 0 = No 0.737 0.452
Reconstruction support 15 Philippine Pesos 16,100,000 23,300,000
DRM activities 20 Scale: 0 = Low, 10 = High 5.41 3.20
Local governance
Experienced Barangay Captain 20 1 = Yes, 0 = No 0.550 0.510
Experience of Mayor 20 Years in office 3.4 2.50
Perceived corruption 20 Scale: 0 = Low, 10 = High 4.55 2.781
Donor support
SIMPLE intervention 20 1 = Yes, 0 = No 0.850 0.366
SIMPLE intervention start 17 1 = Late, 0 = Early 0.471 0.514
Recipient of other donor-support 20 1 = Yes, 0 = No 0.850 0.366
Source: own table.
We made use of a number of variables outlining the structural characteristics of the barangay. First, we
classified barangays according to the income classification of their respective municipality as high (PHP
million) and low (< PHP million). We further expressed the level of urbanization by classifying barangays
as located in a city or rural area. The urbanization classification was derived from official Philippine statistical
classification and allowed a distinction to be made between barangays located in the provincial capital of
Tacloban City and adjacent “rural” municipalities. In addition, the variable “subsidies” captured the amount
of DILG Internal Revenue Allotment (IRA) subsidies that a particular municipality received.
A second group of variables captured DRM-related characteristics. These variables are derived from the DEval
-Haiyan reconstruction support). We used the self-reported
information to try to assess the perceived degree of preparedness of barangays for future disasters. The
variables measure, for a given barangay,
reconstruction support received (in Philippine Pesos), and an index of technical and planning-related disaster
Governance-related variables seek to differentiate barangays based on administrative capacity. These are
approximated by work (years in office) and political experience (more than one tenure) of mayors and
barangay captains, as well as their self-reported ass
Lastly, control variables related to donor-support try to allow for a differentiation between “supported” and
“not-supported” municipalities and barangays, based on GIZ’s land-use planning and disaster risk
management intervention “SIMPLE”. We accounted for the time spent on project implementation and
potential achievements by classifying barangays according to the start of the intervention as early (pre-
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 13
Deval Discussion
and red other donor support received by a barangay from
other international agencies (not including GIZ).
4.2.3 Statistical Approach
In order to analyse the relationship between changes in LCLU patterns derived from RS data and socio-
economic characteristics, factors related to local governance, DRM, and donor support, we relied on basic
statistical tests. Specifically, we employed group-mean T-test statistics and ordinary least squares (OLS)
regressions with robust standard errors. Using these tests, we could assess which factors and local conditions
were statistically associated with damage caused by typhoon Haiyan, and which factors were associated with
disaster resilience and recovery performance.
Because all variables derived from the RS analysis (dependent variable) have a continuous level of
measurement, the type of socio-economic variable (independent variable) determined the applied statistical
test. We used OLS regressions with robust standard errors in cases where both the dependent and
independent variables were continuous. Group-mean T-tests were applied in cases where the independent
variable was nominal to detect significant statistical differences between the two groups. Due to the small
ed
using Stata
4.3 Case Study Results and Discussion
4.3.1 Land Use and Land Cover Proxies
Using the XGBoost machine-learning algorithm, we extracted nine RS-based social and economic proxies for
es between the various time points permitted us to assess
recovery performance in the statistical analysis. As an illustrative example, we have given the results of the
proxy computation for one barangay. We north of Tacloban City) because it
classification.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 14
Figure 4 LCLU classification of barangay 69 before and after typhoon Haiyan
(a) Original image
before disaster (T0)
(b) LCLU classification
before disaster (T0)
Source: own figure.
The
most ur: red). While no debris was
detectable by debris shortly after Haiyan
ies (for a description of the proxies and what socio-economic dimension
is approximated, Section
statistics. For example, difference in the area covered by buildings provides insights into structural damages.
-
rebuilt
Large-, leaving
regular buildings, and three years after -disaster industry had
been rebuilt. As such, we can assume that only a fraction of the former workplaces were available to the
population, with potentially severe consequences for livelihoods and income-generation opportunities.
Deval Discussion
Building
Impervious surface
Bareland
Inland water
Tree/Flattened tree
Non-tree Vegetation
Rubble
2% 0% 9%
4%
20%
21%
44%
19%
6% 1%
2%
39%
33%
(c) LCLU classification
directly after disaster (T1)
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 15
Deval Discussion
either from the removal of debris on the roads
or the reconstruction of new ones, plus an , indicated
a positive recovery of transportation facilities and road network.
- d a
positive recovery of livelihood. Similarly, more arable land was available for livelihood pursuits at the end of
the recovery period, perhaps as a result of areas formerly used for industry and buildings being repurposed
for agricultural production.
Those areas identified as slums (based on a certain type of roof material such as cardboard or metal sheets)
near the coast were entirely wiped out by the storm, although other slum neighbourhoods suffered only
Slum areas re-emerged, -
number of people living in informal settlements can be
considered a positive outcome; it most likely results from government interventions or aid provided by
international donors or organizations.
Table 4 The extracted results for the selected proxies for Barangay 69, Tacloban
Proxy No.
Proxy Name
Unit
T0
T1
T2
T3
1 Buildings m2 36,076 3,236 25,236 26,852
2 Proportion of built-up area 26 2 28 29
3 Impervious surface 12,132 3,236 28,482 28,270
4 Large-scale industry 12,396 1,036 2,564 2,840
5 Vehicles count 9 1 15 10
6 Boats count 13 2 32 22
7 Arable land 2,422 16,740 13,998 6,106
8 Proportion vegetated area 71 41 61 64
9 Roof material m2 (slum area) 4,284 0 3,138 3,484
–
.
Source: own table.
While all nine RS-based proxies were constructed during the analysis, we employ only a selection of three
proxies as –
buildings (damaged buildings), impervious surface (debris), and large-scale industry (damaged industrial
areas) and expressed these changes in percentage of the total area. “Damaged buildings” as well as “debris”
are used to approximate damage to urban areas. The proxy indicator “damaged industrial areas” serves to
gain insights into the extent to which the regional economy might be affected. For example, directly after
ed us to capture growth rates of building
and yphoon
Haiyan struck and two years later, the building stock had increased by , indicating a sizeable damage
recovery-based proxies.
m2
m2
m2
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 16
Deval Discussion
Table 5 Summary statistics of selected RS-based socio-economic proxies across all study barangays
employed in the statistical analysis
Variables
∆ Time
N
Unit
Mean
SD
Damaged Buildings T0 - T1 13 54.1 29.4
Debris T0 - T1 14 21.4 20.4
Damaged industrial areas T0 - T1 14 47.9 28.7
Building reconstruction T1 - T2 15 347.7 496.3
Building reconstruction T2 - T3 17 18.0 33.4
Industry recovery T1 - T2 13 industrial area 397.9 652.9
Industry recovery T2 - T3 13 25.0 38.9
-
after disaster.
Source: own table.
4.3.2 Integration of RS Analysis into Evaluative Work
Damage Assessment
As the first part of our analysis, we investigated the association between disaster damage and socio-
our outcome measures (damaged
buildings, debris, and damaged industrial areas) and various socio-economic determinants, including
municipal income, previous disaster experience, perceived disaster preparedness, and the level of experience
of the barangay captain.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 17
Deval Discussion
Table 6 Damage by socio-economic status, disaster risk management and local governance
Variable
name
Damaged buildings (n=12) Debris (n=13) Damaged industrial areas (n=9)
beta Mean
(m2) T (sig.) Mean
(m2) T (sig.) beta Mean
(m2) T (sig.)
Municipal income
High 63 -2.3
26 - †
Low 25 8
Level of urbanization
City 93 -3.9
50 -3.7
96 -
Rural 42 14 55
Previous disaster experience
Yes 52 1.0 (n.s.) 22 0.5 (n.s.) 67 -0.0 (n.s.)
No 70 29 67
Perceived disaster preparedness
3.7 1.1 (n.s.) 1.6 0.7 (n.s.) 4.2 1.4 (n.s.)
Experienced captain
Yes 62 -1.3
(n.s.)
28 -1.4
(n.s.)
77 -1.2 (n.s.)
No 41 14 57
disaster.
Source: own table.
We found a highly significant difference in the extent of typhoon damage by level of urbanization. Generally,
urban areas experienced much more destruction than rural areas. We also found some statistically significant
relationships by wealth level, with the largest destruction in the wealthiest barangays. These relationships
the south. The wind speed and rotation of the storm caused a substantial storm surge in the San Pablo bay
area and was the main driver for extensive damage observable in the provincial capital. The extent to which
buildings and industrial infrastructure was damaged by the storm was not related to the experience and
perception of the barangay administrators. As such, the physical force of the storm determined entirely the
damage caused, and good governance or prior experience and perception of the barangay leadership could
not mitigate these effects.
Recovery Performance by Socio-Economic Status
After an assessment of damage caused by typhoon Haiyan, this sub-section focuses on determinants of
recovery performance. We generally differentiated –
recovery pha-ed on building reconstruction and industry recovery as outcome variables.
We found few significant socio-economic determinants on recovery performance, and these were for the
–
much stronger in urban areas than in rural areas, while there was no significant difference in the later
–nd
transport hub, and its coordination function in the immediate recovery process, might explain its stronger
recovery performance. For example, Tacloban City’s airport functioned as the main hub for international
relief goods and support.
beta
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 18
Deval Discussion
Table 7 Recovery status by socio-economic status
Variable
name
Building reconstruction (n=15) Industry recovery (n=13)
T1
T2 T2
T3 T1
T2 T2
T3
beta
Mean
growth
rate
(%)
T (sig.)
beta
Mean
growth
rate
(%)
T (sig.)
beta
Mean
growth
rate (%)
T (sig.)
beta
Mean
growth
rate (%)
T (sig.)
Municipal income
High 407 -0.9
(n.s.)
19 0.9
(n.s.)
505 -1.1
(n.s.)
28 -0.5
(n.s.)
Low 111 36 40 15
Level of urbanization
City 1109 -4.7
9 0.9
(n.s.)
704 -0.9
(n.s.)
30 -0.2
(n.s.)
Rural 158 26 306 24
Subsidies
a
2.2 2.6
-0.054 -2.5
1.03 1.3
(n.s.) 0.008 0.2
(n.s.)
a
Source: own table.
Municipal income had no influence on recovery performance, but the amount of subsidies, IRA, was
positively associated with building recovery in the early recovery phase. However, for the later recovery
–
financial performance, the subsidies are likely to have played an important role in financing immediate
reconstruction efforts. Yet, the subsidies were unable to support recovery in later years. Industry recovery
was not determined by municipal income, urbanization level, or availability of subsidies.
Recovery Performance by Disaster Risk Management (DRM)
In this section, we focus on a group of variables that reflect disaster risk management (DRM) capacities,
including previous disaster experience, perceived disaster preparedness, financial reconstruction support,
and DRM-
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 19
Deval Discussion
Table 8 Recovery performance by disaster risk management
Variable
name
Building reconstruction (n=15) Industry recovery (n=13)
T1
T2 T2
T3 T1
T2 T2
T3
beta
Mean
growth
rate (%)
T (sig.)
beta
Mean
growth
rate (%)
T
(sig.) beta
Mean
growth
rate (%)
T (sig.)
beta
Mean
growth
rate (%)
T (sig.)
Previous disaster experience
Yes
222 2.5
13
370 0.7
(n.s.)
12 3.3
No
910 42 728 87
Perceived disaster preparedness
17.0 0.5
(n.s.) -2.7 -0.8
(n.s.) -93.4 -1.2
(n.s.) 6.9 1.3
(n.s.)
Reconstruction
supporta
-0.3 -0.23
(n.s.) -1.0 -5.7
-1.1 -0.5
(n.s.) -0.3 -0.4
(n.s.)
DRM activities
-19.0 -0.7
(n.s.) 0.5 0.2
(n.s.) -73.4 -1.3
(n.s.) -6.3 -1.6
(n.s.)
Source: own table.
Previous disaster experience was significantly associated with both building reconstruction and industry
recovery. Barangays with previous disaster experience generally showed lower recovery rates.
Financial reconstruction support led to significantly slower building reconstruction in the later recover phase
– uilding policies. It is possible that government reconstruction
support was coupled with policies that required storm-proof, high-quality construction material and building
types. Compared to light-build housing construction in other regions without reconstruction support, high-
quality buildings took more time, resulting in the negative association.
Neither perceived disaster preparedness nor DRM activities in the barangay influenced the rate of building
reconstruction or industry recovery.
Recovery Performance by Local Governance
Quality and performance of local government officials can make a large difference in the recovery
performance of municipalities in a post-disaster situation. Good governance supports the implementation of
planning according to guidelines and safety standards, and improves reliability of public governance.
Furthermore, well-implemented and enforced DRM-planning requires coherent and reliable leadership as
well as political commitment to the public good.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 20
Deval Discussion
Table 9 Recovery performance by local governance
Variable
name
Building reconstruction (n=15) Industry recovery (n=13)
T1
T2 T2
T3 T1
T2 T2
T3
beta
Mean
growth
rate (%)
T (sig.)
beta
Mean
growth
rate (%)
T (sig.) beta
Mean
growth
rate (%)
T (sig.)
beta
Mean
growth
rate (%)
T (sig.)
Experienced barangay captain
Yes
317 0.3
(n.s.)
18 0.6
(n.s.)
195 1.2
(n.s.)
32 0.7
(n.s.)
No
394 27 634 17
Perceived corruption
-8.2 -2.5
0.6
-7.1 -1.5
(n.s.) 0.2 1.3
(n.s.)
Experience of Mayor
-87.3
-
-3.4 -1.1
(n.s.) -125.8 -
-0.6 -0.1
(n.s.)
*
Source: own table.
However,
example, an increase in the experience of the local mayor (years in office) led to slower building
reconstruction and slower industry recovery. Similarly, during the early years, recovery was strongest in those
barangays with less perceived corruption. This highlights the importance of trust and accountability of
government officials in the early recovery years. Yet, in the later years of adaptation, the sign of the
relationship switched, and barangays perceived as more corrupt demonstrated higher building
reconstruction rates. Generally, the Philippines is still characterized by relatively high rates of corruption
among local governments. At the same time, local governments possess substantial regulatory autonomy
with regards to the Local Government Code .
Recovery Performance by Donor Support
International donor support played an important role in technical-planning capacity development in the
Tacloban area. This included also a focus on planning for natural-disaster events. We investigate the influence
of donor support separately for the SIMPLE intervention and support by other (not GIZ) donor interventions
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 21
Deval Discussion
Table 10 Recovery performance by donor support
Variable
name
Building reconstruction (n=15) Industry recovery (n=13)
T1
T2 T2
T3 T1
T2 T2
T3
beta
Mean
growth
rate (%)
T (sig.)
beta
Mean
growth
rate (%)
T (sig.)
beta
Mean
growth
rate (%)
T (sig.)
beta
Mean
growth
rate (%)
T (sig.)
SIMPLE intervention
Yes
337 0.2
(n.s.)
16 2.2
† †
No
416 61
SIMPLE intervention start
Late 524
-
16 0.0
(n.s.)
787 -2.2
7 1.3
(n.s.)
Early 120 17 50 27
Recipient of other donor
-support
Yes 85 1.3
(n.s.)
28 -0.3
(n.s.)
870
-
-16 2.5
No 443 21 256 37
*
Source: own table.
We found slower building reconstruction rates among barangays that received the German-supported
SIMPLE intervention. While the general trend is similar for both time periods, the effect was statistically
–
intervention, barangays that adopted the programme later showed stronger building and industry recovery
rates. The evolving nature of the programme and the increasingly climate-risk-aware political context may
have led to higher quality and better outcomes in the later phase of implementation.
The existence of other donor support proved effective for industry recovery but only for the early recovery
–signs of a switch in the relationship, with those
barangays that did not receive other donor support showing higher recovery rates.
5. DISCUSSION AND CONCLUSION
5.1 Summary of Case Study Results
In this discussion paper we have demonstrated the integration of RS data and related machine-learning-
based analysis with survey data for complex evaluations. Using this method, we were able to evaluate
damage and the effectiveness of recovery processes following typhoon Haiyan, which made landfall on the
areas with large shares of disadvantaged populations (usually the coastal barangays).
Our analysis of recovery performance showed that the barangays that recovered fastest where those that
had no prior disaster experience and were provided with sufficient government support (subsidies). We
found that the GIZ-supported SIMPLE programme was related to recovery performance only to some extent.
We generally observed strong recovery during the first two years, but substantially slower recovery effects
for later years. In addition, we found a stronger recovery performance for industrial areas than for buildings.
This tendency may be explained by the strong focus of the government and international organizations on
economic capacity and performance.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 22
Deval Discussion
5.2 Outlook: Strengths and Limitations of the Approach
Combining remote sensing and machine learning with survey data offers many advantages but also has some
limitations. These have important implications for future research . In general,
we see the most promising applications of RS technologies and data within mixed-methods and multi-method
evaluation designs. Rather than a stand-alone tool, RS is best seen as a promising addition to the evaluators’
toolbox that allows viewing problems from a different angle and thereby broadens the evidence base of
evaluations. With this preamble in mind, we now discuss advantages and disadvantages of the use of RS in
evaluations:
Advantages: As a first advantage, machine-learning-based analysis of RS data permits the measurement of
physical changes over comparably large areas and across multiple time points. Natural disasters are hard to
predict, and if the goal is to evaluate disaster-recovery processes and programmes, it is almost impossible to
collect baseline survey data before the disaster event. In such a case, RS data offers a convenient way to
obtain relevant pre-disaster information on local conditions. Moreover, satellites now orbit the earth with
increasing frequency, and their wide coverage makes it possible to measure change processes more precisely
and across larger areas. Given the wide coverage of most RS data products, cross-regional, or even cross-
country evaluations are possible. This case study has shown that RS data can be used to construct proxy
indicators of social and economic conditions and to use these proxies for machine-learning-based
classification. This is of particular relevance to the evaluation community, as politicians and policy makers
are frequently concerned with the socio-economic conditions of the population in relatively large areas.
Moreover, as we have shown in this article, it is possible to join RS data with data from surveys and qualitative
interviews. Such a fusion allows the contextualization of the RS data and an exploration of the underlying
mechanism of change, and also helps to gain more human-centred insights into patterns of change.
Limitations and future prospects: Our study also revealed a number of disadvantages of RS data. First, the
very high-D per km for a single
time-period, resulting in substantial costs for large areas. Usually, it is not possible for implementing agencies
to spend a sizeable amount of their limited budget on the purchase of RS data. Even in our analysis we were
only able to purchase RS data for a small selection of barangays for which we had survey data available. This
limited the sample size of our analysis and prevented us from using advanced statistical methods. However,
it was the goal of this study to demonstrate the use of RS data in evaluations rather than to make
comprehensive use of all survey data available.
It is important to mention that many geo-data sources are freely available. For example, the European Space
Agency makes multi-
changes in physical environment. Many other RS data products are freely available and often have specific
applications. This includes night-time lights , vegetation indices, topographic information, and
land cover and land use (LCLU) classifications of the US Geographical Survey (USGS). New data providers are
also entering the market. For example, Planet Labs recently
CubeSat satellites that provide approximately daily imagery free for non-commercial use at a spatial
A second limitation of the use of RS data by the average evaluator is the steep learning curve involved in
working with the specialized software and format of RS data. Future research should develop user-friendly
statistical tools, enabling evaluators and practitioners to easily generate the relevant proxy indicators
employed in this study.
A third limitation is the substantial time commitment required for pre- and post-processing of the RS data.
Currently, this work requires a GIS expert for geo-rectification of the images, merging of image tiles
(mosaicking), detection and removal of parts of images that are contaminated by clouds or cloud shadows.
Similarly, it is necessary in a post-processing step to manually check that the LCLU classifications are assigned
correctly. Given the complexity of these tasks and geospatial analysis in general, an evaluation team wishing
to integrate RS data would have to employ a GIS expert, which could substantially increase staff costs. Future
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 23
Deval Discussion
work should aim at automating pre- and post-processing steps, since automation for large areas can become
economical, especially when compared with expensive field-based studies. In the meantime, the
collaboration with GIS experts at universities or research institutions may be an alternative. While such
collaboration comes with their own potential challenges (alignment of research interests and evaluation
objectives, different priorities and time schedules, time required for supervision), costs may be lower
compared to employing a GIS expert.
A fourth limitation is the problem of distinguishing and classifying structural rubble (which represents
evidence of building damage) from debris (which includes flotsam and other washed-up material that can be
quickly cleaned up). The potential for misclassification remains, and if damage is overestimated, the rate of
recovery is also overestimated, since recovery is defined as the change vector from the post-disaster damage
state.
Finally, RS data are one type of so-called “Big Data” resources. It is often not possible to work on regular
desktop computers with this type of data. Rather, multi-core servers or super-computers with large amounts
of RAM are necessary, particularly when computing intense machine-learning algorithms (e.g. Convolutional
Neural Networks) to assign LCLU classifications and to generate the relevant proxy indicators. Rather than
purchasing costly hardware, an emerging alternative is computing platforms such as Google Earth Engine
(GEE) . GEE, for example, is based on cloud computing, meaning that the analysis
procedures are programmed in the web interface, and the processing is done in the cloud, eliminating the
need to download vast amounts of data. Moreover, because the actual image data are not made available
for download, fewer copyright restrictions apply. GEE is designed to be used by non-experts in RS data
analysis and may become a viable resource for evaluators. However, without additional costs, GEE provides
only medium-resolution datasets that are primarily suitable for analyses on the national, regional, and global-
level scale.
Outlook: The above discussion has shown that integrating RS data has distinct strengths and weaknesses.
Yet, satellite technology, cloud computing, and processing speed and power are improving every year, and
most of the above limitations will likely be overcome within the next decade. It is therefore the right time for
evaluation units and institutes to explore uses and applications of RS data. RS data will be a valuable resource
to measure programme impacts that leave a mark on the surface of the earth. Yet, similar or more important,
machine-learning-based analysis of RS data can be used to approximate social and economic change.
Enhanced machine-learning algorithms and standardized packages, as well as sets of validated proxies for
socio-economic change, will increase automation and thus allow for the classification of larger land areas.
This discussion paper was intended to provide a real-world example of machine-learning-based analysis of
RS data in evaluation and, thus, to sensitize the evaluation community to the potentials and uses of RS data.
The future will be technological and driven by “Big Data” and we hope that this paper helps to encourage the
use of RS data and technologies in evaluations.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 24
Deval Discussion
6. REFERENCES
de Almeida, L. Q. et al. (2016), “Disaster Risk Indicators in Brazil: A Proposal Based on the World Risk Index”,
International Journal of Disaster Risk Reduction–
Andersson, M. et al. (2015), “
Night-Time Light Data and Vegetation Index: Recovery from the Indian Ocean Tsunami”,
Geographical Research–
Becker, J. S. et al. (2010), “A Synthesis of Challenges and Opportunities for Reducing Volcanic Risk through
Land Use Planning in New Zealand”, The Australasian Journal of Disaster and Trauma Studies, No. 1.
Bevington, J. et al. (2010), “Uncovering Community Disruption Using Remote Sensing: As Assessment of Early
Recovery in Post-Earthquake Haiti”, University of Delaware, Disaster Research Center, Delaware.
Bolton, D. K. and M. A. Friedl (2013), “Forecasting Crop Yield Using Remotely Sensed Vegetation Indices and
Crop Phenology Metrics”, –
Bradshaw, S. (2004), “Socio-Economic Impacts of Natural Disasters: A Gender Analysis”,
Brown, D. L. et al. (2010), “Disaster Recovery Indicators: Guidelines for Monitoring and Evaluation”.
Burton, C. G. (2012), “The Development of Metrics for Community Resilience to Natural Disasters”, Doctoral
Dissertation, University of South Carolina.
Chan, J. C.-W. and D. Paelinckx (2008), “Evaluation of Random Forest and Adaboost Tree-Based Ensemble
Classification and Spectral Band Selection for Ecotope Mapping Using Airborne Hyperspectral
Imagery”, Remote Sensing of Environment–
Chen, T. and C. Guestrin (2016), “XGBoost: A Scalable Tree Boosting System”, Nd
g, KDD ACM, New York,
NY, USA, pp. –
Chen et al. (2003), “Damage Pattern Mining in Hurricane Image Databases”,
, presented at the Proceedings Fifth IEEE Workshop
on Mobile Computing Systems and Applications, pp. –
Combest-Friedman, C. et al. (2012), “Household Perceptions of Coastal Hazards and Climate Change in the
Central Philippines”, Journal of Environmental Management–
Davies, M. et al. (2009), “Climate Change Adaptation, Disaster Risk Reduction and Social Protection:
Complementary Roles in Agriculture and Rural Growth?”,
Development Studies.
Del Rosario, E. D. (2013), “Final Report Re Effects of Typhoon “Yolanda” (Haiyan)”, Report, National Disaster
Risk Reduction and Management Council (NDRRMC), Quezon City, Philippines.
Duarte, D. et al. (2018), “Satellite Image Classification of Building Damages Using Airborne and Satellite
Image Samples in a Deep Learning Approach”,
Spatial Information Sciences, Vol. IV–pp. –
Ebert, A. et al. (2009), “Urban Social Vulnerability Assessment with Physical Proxies and Spatial Metrics
Derived from Air- and Spaceborne Imagery and GIS Data”, Natural Hazards–
Edenhofer, O. (Ed.) (2014),
,
Cambridge Univ. Press, New York, NY.
Field, C. B. and IPCC (Eds.) (2012),
, Cambridge
U
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 25
Deval Discussion
Flax, L. K. et al. (2002), “Community Vulnerability Assessment Tool Methodology”, Natural Hazards Review,
–
Friedman, J. H. (2001), “Greedy Function Approximation: A Gradient Boosting Machine”, The Annals of
Statistics–
Garcia Schustereder, M. et al. (2016), “Donor-Assisted Land-Use Planning in the Philippines: Insights from a
Multi-Level Survey”, Deutsches Evaluierungsinstitut der Entwicklungszusammenarbeit (DEval),
Bo
Georganos, S. et al. (2018), “Very High Resolution Object-Based Land Use–Land Cover Urban Classification
Using Extreme Gradient Boosting”, IEEE Geoscience and Remote Sensing Letters
–
Gevaert, C. M. et al. (2017), “Informal Settlement Classification Using Point-Cloud and Image-Based Features
from UAV Data”, –
GFDRR (2017), “Philippines”, ,
https://www.gfdrr.org/philippines.
Ghaffarian, S. et al. (2018), “Remote Sensing-Based Proxies for Urban Disaster Risk Management and
Resilience: A Review”, Remote Sensing–
GIZ (2014), “Results Chain - Environment and Rural Development Program (Based on Amended Offer, Version
-
Gorelick, N. et al. (2017), “Google Earth Engine: Planetary-Scale Geospatial Analysis for Everyone”, Remote
Sensing of Environment, Big Remot
–
Government of the Philippines (2010), “
Republic Act No.
Jha, S. et al. (2018), “Natural Disaster, Public Spending, and Creative Destruction: A Case Study of the
Philippines”,
Kerle, N. (2015), “Disasters: Risk Assessment, Management, and Post-Disaster Studies Using Remote
Sensing”, in Thenkabail, P. S. (ed.), ,
Boca Raton, Florida, pp. –
Kerle, N. et al. (2019), “Evaluating Resilience-Centered Development Interventions with Remote Sensing”,
Remote Sensing
Kerle, N. and R. R. Hoffman (2013), “Collaborative Damage Mapping for Emergency Response: The Role of
Cognitive Systems Engineering”, –
Kohli, D. et al. (2012), “An Ontology of Slums for Image-Based Classification”,
–
Kohli, D. et al. (2013), “Transferability of Object-Oriented Image Analysis Methods for Slum Identification”,
Remote Sensing–
Kuffer, M. et al. (2016), “Slums from Space - ”, Remote
Sensing
Lagmay, A. M. and N. Kerle (2015), “Storm-Surge Models Helped for Hagupit: Typhoons”, Nature
–
Lech, M. et al. (2018), “Improving International Development Evaluation through Geospatial Data and
Analysis”, International Journal of Geospatial and Environmental Research, Vol. 5, No. 2.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 26
Deval Discussion
Leppert, G. (2015), Social Risk Management Strate
Ghana and Malawi, Lit Verlag, Münster.
Leppert, G. et al. (2018), “Impact, Diffusion and Scaling-Up of a Comprehensive Land-Use Planning Approach
in the Philippines. From Development Cooperation to National Policies”, German Institute for
Development Evaluation (DEval), Bonn.
Li, X. et al. (2018), “Long-
”, Remote Sensing, Vol.
Löw, P. (2018), “ - The Year in Figures”, Munich RE,
https://www.munichre.com/topics-online/en/climate-change-and-natural-disasters/natural-
--in-figures.html.
Matthews, T. (2012), “Responding to Climate Change as a Transformative Stressor through Metro-Regional
Planning”, Local Environment–
Mboga, N. et al. (2017), “Detection of Informal Settlements from VHR Images Using Convolutional Neural
Networks”, Remote Sensing, Vol. 9, No. 11, p. 1106.
Morrow, B. H. (1999), “Identifying and Mapping Community Vulnerability”, Disasters–
MSU-Iligan Institute of Technology et al. (2015), “The Comprehensive Land Use Plan of Iligan City and the
Disaster Risk Reduction and Management Framework of the Philippines”, Journal of Government and
–
Mulla, D. J. (2013), “Twenty Five Years of Remote Sensing in Precision Agriculture: Key Advances and
Remaining Knowledge Gaps”, –
Newell, B. et al. (2005), “A Conceptual Template for Integrative Human-Environment Research”, Global
–
Platt, S. et al. (2016), “Measuring Resilience and Recovery”, International Journal of Disaster Risk Reduction,
–
Ren, X. et al. (2017), “A Novel Image Classification Method with CNN-XGBoost Model”, in Kraetzer, C., Y.-Q.
Shi, J. Dittmann and H. J. Kim (eds.), , Springer International
–
Rubin, C. B. et al. (1985), “Community Recovery from a Major Natural Disaster”, FMHI Publications, Louis de
la Parte Florida Mental Health Institute (FMHI), Florida.
Shackleton, S. et al. (2014), “A Gendered Perspective of Vulnerability to Multiple Stressors, Including Climate
Change, in the Rural Eastern Cape, South Africa”, Agenda, –
Sliuzas, R. and M. Kuffer (2008), “Analysing the Spatial Heterogeneity of Poverty Using Remote Sensing:
Typology of Poverty Areas Using Selected RS Based Indicators”, in Carsten, J. (ed.), Remote Sensing -
- ,
Bochum, Germany.
Toye, J. (2017), , Oxford, UK.
Turner, B. L. et al. (2003), “A Framework for Vulnerability Analysis in Sustainability Science”, eedings of
–
Vetrivel, A. et al. (2018), “
Cloud Features Derived from Very High Resolution Oblique Aerial Images, and Multiple-Kernel-
Learning”, –
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 27
Deval Discussion
Villanueva, P. S. (2010), “Learning to ADAPT: Monitoring and Evaluation Approaches in Climate Change
Adaptation and Disaster Risk Reduction – Challenges, Gaps and Ways Forward”, SCR Discussion
Yilmaz, S. and V. Venugopal (2013), “Local Government Discretion and Accountability in Philippines”, Journal
of International Development–
Zhang, F. et al. (2016), “Scene Classification via a Gradient Boosting Random Convolutional Network
Framework”, IEEE Transactions on Geoscience and Remote Sensing–02.
A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations 28
Deval Discussion
7. ANNEX
Annex A: Specifications and Acquisition Times of the Images Used for Proxy Extraction
Study area Satellite Acquired date
Spatial resolution
Multispectral
image (m)
Panchromatic
image (m)
Tacloban WorldView2 2013-03-17 2 0.5
Tacloban WorldView2 2017-03-18 2 0.5
Tacloban GeoEye1 2013-11-10 2 0.5
Tacloban GeoEye1 2013-11-12 2 0.5
Tacloban GeoEye1 2013-11-13 2 0.5
Tacloban GeoEye1 2016-04-24 2 0.5
South Leyte WorldView2 2013-03-25 2 0.5
South Leyte WorldView2 2013-04-02 2 0.5
South Leyte WorldView2 2014-01-07 2 0.5
South Leyte WorldView2 2014-07-16 2 0.5
South Leyte WorldView2 2014-09-11 2 0.5
South Leyte WorldView2 2014-10-21 2 0.5
South Leyte WorldView2 2014-12-01 2 0.5
South Leyte WorldView2 2016-01-24 2 0.5
South Leyte WorldView2 2016-06-24 2 0.5
Basey WorldView2 2013-05-18 2 0.5
Basey WorldView2 2013-05-18 2 0.5
Basey WorldView2 2013-09-01 2 0.5
Basey WorldView2 2013-11-19 2 0.5
Basey WorldView2 2013-11-21 2 0.5
Basey WorldView3 2014-12-09 1.3 0.31
Basey WorldView3 2015-01-10 1.3 0.31
Basey WorldView2 2016-06-04 2 0.5
Basey WorldView2 2016-06-04 2 0.5
Lawaan WorldView2 2013-05-18 2 0.5
Lawaan WorldView2 2013-05-18 2 0.5
Lawaan WorldView2 2014-01-07 2 0.5
Lawaan WorldView2 2014-01-07 2 0.5
Lawaan WorldView3 2014-10-07 1.3 0.31
Lawaan WorldView2 2015-11-24 2 0.5
Source: own figure.
