ArticlePDF Available

COINr: An R package for developing composite indicators

October 2022
The Journal of Open Source Software 7(78):4567

DOI:10.21105/joss.04567

License
CC BY 4.0

Authors:

William Becker

BlueFox Data

Giulio Caperna

European Commission

Maria Del Sorbo

European Commission

Hedvig Norlen

European Commission

Show all 6 authorsHide

An overview of the "COINr" R package, which is for building and analysing composite indicators.

National connectivity scores broken down into component scores and sorted from highest to lowest.

…

Correlations between sustainability indicators, with colouring thresholds. Only correlations within aggregation groups are shown.

…

Figures - uploaded by William Becker

Content may be subject to copyright.

Content uploaded by William Becker

Content may be subject to copyright.

Content uploaded by William Becker

Content may be subject to copyright.

COINr: An R package for developing composite

indicators

William Becker 1¶, Giulio Caperna2, Maria Del Sorbo 3, Hedvig Norlén1,

Eleni Papadimitriou2, and Michaela Saisana 2

1Freelance consultant, Ispra, Italy 2European Commission, Joint Research Centre, Italy 3European

Innovation Council and SMEs Executive Agency, Belgium ¶Corresponding author

DOI: 10.21105/joss.04567

Software

•Review

•Repository

•Archive

Editor: Mehmet Hakan Satman

Reviewers:

•@bauer-alex

•@paulrougieux

Submitted: 06 July 2022

Published: 11 October 2022

License

Authors of papers retain copyright

and release the work under a

Creative Commons Attribution 4.0

International License (CC BY 4.0).

Summary

Composite indicators (CIs) are aggregations of indicators that aim to measure complex, multi-

dimensional and typically socio-economic concepts such as sustainable development (Conceição,

2020), innovation (Dutta et al., 2020), globalisation (William Becker et al., 2021), gender

equality (Equal Measures 2030, 2019), and many more. CIs are very widely used in policy-

making, by international organisations, and they are equally well-covered in academic literature

(El Gibari et al., 2019;David Lindén et al., 2021;Stefana et al., 2021). They are often used to

rank and benchmark countries or regions to help direct policy making, but are also frequently

used for advocacy (Cobham et al., 2015).

The COINr package, introduced in this article, aims to provide a harmonised development

environment for composite indicators that includes all common operations from indicator

selection, data treatment and imputation up to aggregation, presentation of results and

sensitivity analysis. COINr enables development, visualisation and exploration of methodological

variations, and encourages transparency and reproducibility.

Statement of need

Existing tools

Some dedicated tools for composite indicators exist: in Microsoft Excel, the COIN Tool is a

spreadsheet-based system which allows users to build and analyse a composite indicator (W.

Becker et al., 2019). In MATLAB, there are some packages addressing specic parts of index

development: the CIAO package uses a nonlinear regression and optimisation approach to

tune weights to agree with expert opinions (D. Lindén et al., 2021). In R (R Core Team, 2022)

there is an existing package for composite indicator development, called compind (Fusco et

al., 2018), focusing on weighting and aggregation, although this gives no special consideration

to hierarchical structures, uncertainty and sensitivity analysis, and so on.

The Python library CIF gives a number of tools for building composite indicators, from loading

data to aggregation and visualisation (Vraná, 2022). This is focused in particular on Business

Cycle Analysis. Finally, there is a recently launched Web-based tool called the MCDA Index

Tool (Cinelli et al., 2021). This is mostly focused on multi-criteria decision analysis, and

doesn’t include dierent levels of aggregation.

Why COINr

COINr is a signicant step beyond existing composite indicator tools in many respects. COINr

wraps all composite indicator data, analysis and methodological choices into a single S3 class

Becker et al. (2022). COINr: An R package for developing composite indicators. Journal of Open Source Software,7(78), 4567. https:

//doi.org/10.21105/joss.04567.1

object called a “coin”. A coin is a structured list including:

• Indicator data sets for each processing step (e.g. imputation and normalisation).

•

Metadata pertaining to indicators and units (e.g. names and weights, but also the

hierarchical structure of the index).

• A record of the COINr functions applied in constructing the coin.

This enables a neat and structured environment, simplies the syntax of functions, and also

allows comparisons between dierent versions of the same index, as well as global sensitivity

analysis along the lines of Saisana et al. (2005) (for the distinction between “local” and “global”

sensitivity analysis, see e.g. Saltelli et al. (2019)). COINr also supports time-indexed (panel)

data, represented by the “purse” class (a data frame containing a time-indexed collection of

coins). For more information on coins and purses, see the coins vignette.

All major COINr functions have methods for coins, and many have methods for purses, data

frames, and numerical vectors. This means that COINr can be used either as an integrated

development environment via coins and purses, but equally as a toolbox of functions for other

related purposes.

COINr also oers a far wider range of functions and methodological options than any existing

package. It not only includes a range of options for treating, imputing, normalising and

aggregating indicator data (among others), but also has a suite of analysis tools to check

data availability and perform correlation/multivariate analysis. Moreover, it has many options

for plotting and visualising data using wrapper functions for the ggplot2 package (Wickham,

2016). Many core COINr functions are written with hooks to link with other packages, for

example allowing other imputation or aggregation packages to be used with coins.

Features

Primarily, COINr is used for building composite indicators: In practice this involves assembling

a set of indicators (usually from dierent sources) and accompanying metadata, and assembling

them into data frames that can be read by COINr to build a “coin” (see vignette). After that,

the composite scores are calculated by applying COINr functions to the coin, which specify the

methodological steps to apply, and how to apply them.

To give a avour of COINr, we present a very short example using the built-in “ASEM” data

set which comprises two data frames (one of indicator data, and the other of metadata). To

build a coin, the new_coin() function is called:

# load COINr

library(COINr)

# build a coin with example data set

coin <- new_coin(iData = ASEM_iData, iMeta = ASEM_iMeta)

To see how these data frames are formatted, one can use e.g.

str(ASEM_iData)

View(ASEM_iData) and refer to the coins vignette.

In the most simple case, we could build a composite indicator by normalising the indicators

(bringing them onto a common scale), and aggregating them (using weighted averages to

calculate index scores). This can be done in COINr using the

Normalise()

and

Aggregate()

functions respectively:

# normalise (scale) each indicator onto [0, 100] interval

coin <- qNormalise(coin, dset = ”Raw”,f_n = ”n_minmax”,

f_n_para = list(l_u = c(0,100)))

# aggregate using weighted arithmetic mean

Becker et al. (2022). COINr: An R package for developing composite indicators. Journal of Open Source Software,7(78), 4567. https:

//doi.org/10.21105/joss.04567.2

# Note: weights are input in data frames when calling new_coin()

coin <- Aggregate(coin, dset = ”Normalised”,f_ag = ”a_amean”)

Both of these functions allow any other function to be passed to them, allowing more

complex types of normalisation and aggregation. Here, the code simply uses the “min-max”

normalisation method (scaling indicators onto the

[0, 100]

interval), and aggregates using the

weighted arithmetic mean, following the hierarchical structure and weights specied in the

iMeta argument of new_coin().

To see the results in a table form, one can call the get_results() function:

# generate data frame with results at highest aggregation level (index)

get_results(coin, dset = ”Aggregated”)|> head()

uCode Index Rank

1DEU 75.23 1

2GBR 68.94 2

3FRA 65.92 3

4CHE 62.61 4

5NLD 61.24 5

6SWE 60.59 6

We may also visualise the same results using a bar chart - here we see how countries rank on

the “connectivity” sub-index (see Figure 1).

plot_bar(coin, dset = ”Aggregated”,iCode = ”Conn”,stack_children = TRUE)

Figure 1: National connectivity scores broken down into component scores and sorted from highest to

lowest.

As a nal example, we show one of the analysis features of COINr: the possibility to plot and

analyse correlations.

plot_corr(coin, dset = ”Normalised”,iCodes = list(”Sust”),

grouplev = 2,flagcolours = T, text_colour = ”darkblue”)

Becker et al. (2022). COINr: An R package for developing composite indicators. Journal of Open Source Software,7(78), 4567. https:

//doi.org/10.21105/joss.04567.3

Figure 2: Correlations between sustainability indicators, with colouring thresholds. Only correlations

within aggregation groups are shown.

The correlation plot in Figure 2 illustrates where e.g. negative correlations exist within ag-

gregation groups, which may lead to poor representation of indicators in the aggregated

scores.

COINr includes far more features than those shown here. Remaining features (with vignette

links) include:

Building:

•Denomination by other indicators

•Screening units by data requirements

•Imputation of missing data

•Outlier treatment using Winsorisation and nonlinear transformations

•

Weighting using either manual weighting, PCA weights or correlation-optimised weights.

Analysis:

•

Analysis via indicator statistics, data availability, correlation analysis and multivariate

analysis (e.g. PCA).

•Adjustments and Comparisons: checking the eects of methodological variations.

•

Global uncertainty and sensitivity analysis of the impacts of uncertainties in weighting

and many methodological choices

Others:

•

A range of visualisation options, including statistical plots, bar charts and correlation

plots

• Automatic import from the COIN Tool and fast export to Microsoft Excel.

For the full range of COINr features, see COINr documentation which is accessible at COINr’s

website.

Becker et al. (2022). COINr: An R package for developing composite indicators. Journal of Open Source Software,7(78), 4567. https:

//doi.org/10.21105/joss.04567.4

Acknowledgements

COINr was initally developed under contract for the European Commission’s Joint Research

Centre, and this is gratefully acknowledged for enabling the bulk of the initial design.

References

Becker, W., Benavente, D., Dominguez Torreiro, M., Tacao Moura, C., Neves, A. R., Saisana,

M., & Vertesy, D. (2019). COIN Tool User Guide. European Commission, Joint Research

Centre. https://doi.org/10.2760/523877

Becker, William, Domınguez-Torreiro, M., Neves, A. R., Moura, C. T., & Saisana, M. (2021).

Exploring the link between Asia and Europe connectivity and sustainable development.

Research in Globalization,3, 100045. https://doi.org/10.1016/j.resglo.2021.100045

Cinelli, M., Spada, M., Kim, W., Zhang, Y., & Burgherr, P. (2021). MCDA index tool: An

interactive software to develop indices and rankings. Environment Systems and Decisions,

41(1), 82–109. https://doi.org/10.1007/s10669-020-09784-x

Cobham, A., Janskỳ, P., & Meinzer, M. (2015). The Financial Secrecy Index: Shedding

new light on the geography of secrecy. Economic Geography,91(3), 281–303. https:

//doi.org/10.1111/ecge.12094

Conceição, P. (2020). The 2020 Human Development Report. United Nations Development

Programme. ISBN: 978-92-1-126442-5

Dutta, S., Lanvin, B., & Wunsch-Vincent, S. (2020). The Global Innovation Index 2020:

Who will nance innovation? World Intellectual Property Organisation. https://www.

globalinnovationindex.org/gii-2020-report

El Gibari, S., Gómez, T., & Ruiz, F. (2019). Building composite indicators using multicriteria

methods: A review. Journal of Business Economics,89(1), 1–24. https://doi.org/10.1007/

s11573-018-0902-z

Equal Measures 2030. (2019). Harnessing the power of data for gender equality: Introducing

the 2019 EM2030 SDG gender index. Equal Measures 2030. https://www.data.em2030.

org/2019-global-report

Fusco, E., Vidoli, F., & Sahoo, B. K. (2018). Spatial heterogeneity in composite indicator: A

methodological proposal. Omega,77, 1–14. https://doi.org/10.1016/j.omega.2017.04.007

Lindén, D., Cinelli, M., Spada, M., Becker, W., & Burgherr, P. (2021). Composite Indicator

Analysis and Optimization (CIAO) tool, v.2.https://doi.org/10.13140/RG.2.2.14408.75520

Lindén, David, Cinelli, M., Spada, M., Becker, W., Gasser, P., & Burgherr, P. (2021). A

framework based on statistical analysis and stakeholders’ preferences to inform weighting

in composite indicators. Environmental Modelling & Software, 105208. https://doi.org/10.

1016/j.envsoft.2021.105208

R Core Team. (2022). R: A language and environment for statistical computing. R Foundation

for Statistical Computing. https://www.R-project.org/

Saisana, M., Saltelli, A., & Tarantola, S. (2005). Uncertainty and sensitivity analysis techniques

as tools for the quality assessment of composite indicators. Journal of the Royal Statistical

Society: Series A (Statistics in Society),168(2), 307–323. https://doi.org/10.1111/j.

1467-985x.2005.00350.x

Saltelli, A., Aleksankina, K., Becker, W., Fennell, P., Ferretti, F., Holst, N., Li, S., &

Wu, Q. (2019). Why so many published sensitivity analyses are false: A systematic

Becker et al. (2022). COINr: An R package for developing composite indicators. Journal of Open Source Software,7(78), 4567. https:

//doi.org/10.21105/joss.04567.5

review of sensitivity analysis practices. Environmental Modelling & Software,114, 29–39.

https://doi.org/10.1016/j.envsoft.2019.01.012

Stefana, E., Marciano, F., Rossi, D., Cocca, P., & Tomasoni, G. (2021). Composite indicators

to measure quality of working life in Europe: A systematic review. Social Indicators

Research,157(3), 1047–1078. https://doi.org/10.1007/s11205- 021-02688-6

Vraná, L. (2022). Composite Indicators Framework (CIF) for business cycle analysis. Python

Software Foundation. https://pypi.org/project/cif/

Wickham, H. (2016). ggplot2: Elegant graphics for data analysis. Springer-Verlag New York.

ISBN: 978-3-319-24277-4

Becker et al. (2022). COINr: An R package for developing composite indicators. Journal of Open Source Software,7(78), 4567. https:

//doi.org/10.21105/joss.04567.6

Software for building and measuring the quality of composite indicators using ordered weighted averaging: So-called S-CI-OWA

Article

Full-text available

Mar 2024

Composite indicators are one-dimensional measures that help decision-makers understand complex realities. Existing software builds composite indicators using different methods, also providing a measure of the stability of their structure. Considering this, it is possible to point out that there is a gap for software that provides more quality measures and composite indicators capable of guiding decision-makers on their usability. The software for building and measuring the quality of composite indicators using Ordered Weighted Averaging, so-called SCI-OWA, fills this gap. The S-CI-OWA is based on Ordered Weighted Averaging, which assigns weights according to the input value, solving hierarchical evaluation problems considering the fuzzy nature of scoring, weighting, and aggregation operations. The S-CI-OWA offers three new features over existing software: one, versatility in driving composite indicator scores up or down; two, flexibility in defining this intensity of emphasis, allowing more positive or negative indicators (sub-indicators) to be highlighted in decision-making units; three a broader set of quality measures of composite indicators, notably: ranking uncertainty, ratio of atypical measurements, discriminating, explanatory and informational power.

The forest biodiversity index (FOBI): monitoring forest biodiversity potential over space and time

Article

Full-text available

Aug 2024

Public forest agencies are obligated to take steps to conserve and where possible enhance biodiversity, but they often lack information and tools that support and evidence their decision making. To help inform and monitor impact of management actions and policies aimed at improving forest biodiversity, we have co-developed a quantitative, transparent and repeatable approach for assessing the biodiversity potential of the United Kingdom's (UK) publicly owned forests over space and time. The FOrest Biodiversity Index (FOBI) integrates several forest biodiversity indicators or 'metrics' , which characterise management-sensitive woodland and landscape features associated with biodiversity. These are measured or modelled annually using spatially comprehensive forest survey data and other well-maintained spatial environmental datasets. Following metric normalisation and a correlation analysis, a statistically robust selection of these metrics is aggregated using a hierarchical procedure to provide composite index scores. The FOBI metric and index results are provided for every individual public forest, and can be summarised across any reporting region of interest. Compared to existing indicators that rely on sample-based forest data, the results thus better support decisions and obligations at a range of scales, from locally targeted action to national, long-term biodiversity monitoring and reporting. We set out how the FOBI approach and associated bespoke online interfaces were co-developed to meet public forest agency needs in two constituent countries of the UK (England and Scotland), whilst providing a conceptual framework that can be adapted and transferred to other geographic areas and private forests. Example results are reported for England's public forests for four annual timestamps between 2014 and 2021, which indicate improvements to the biodiversity potential of public forests and surrounding landscapes over this time via increases in their diversity, extent, condition and connectivity.

Green growth as a pathway toward sustainable development: A systems thinking on the Qinghai-Tibet Plateau in China

Article

Full-text available

Sep 2023
SUSTAIN DEV

Nyangchak Niangjijia

Green growth gained traction as a global climate change strategy and pathway toward sustainable development. China's green growth has been on the rise since the turn of the century, yet it is little understood in the context of its provinces. Previous studies focus on ranking green growth across countries and regions, not on assessing individual provinces over time. This study employs systems thinking and constructs an index framework to assess the environmental, economic, and social dimensions of green growth as a pathway toward sustainable development in Qing-hai on the Qinghai-Tibet Plateau. The study finds that green growth has steadily increased between 2000 and 2021 despite a volatile growth rate. The 10th-13th Five-Year Plans showed similar trends. Short-term green growth performance fluctuated in its dimensions and pillars, while long-term performance increased steadily. Qinghai is well-positioned to achieve sustainable development and build a circular economy. The study further discusses sustainable policy implications.

A tidy framework and infrastructure to systematically assemble spatio-temporal indexes from multivariate data

Preprint

Full-text available

Jan 2024

Indexes are useful for summarizing multivariate information into single metrics for monitoring, communicating, and decision-making. While most work has focused on defining new indexes for specific purposes, more attention needs to be directed towards making it possible to understand index behavior in different data conditions, and to determine how their structure affects their values and variation in values. Here we discuss a modular data pipeline recommendation to assemble indexes. It is universally applicable to index computation and allows investigation of index behavior as part of the development procedure. One can compute indexes with different parameter choices, adjust steps in the index definition by adding, removing, and swapping them to experiment with various index designs, calculate uncertainty measures, and assess indexes' robustness. The paper presents three examples to illustrate the pipeline framework usage: comparison of two different indexes designed to monitor the spatio-temporal distribution of drought in Queensland, Australia; the effect of dimension reduction choices on the Global Gender Gap Index (GGGI) on countries' ranking; and how to calculate bootstrap confidence intervals for the Standardized Precipitation Index (SPI). The methods are supported by a new R package, called tidyindex.

Cultural and Creative Cities Monitor 2023 update

Technical Report

Full-text available

Sep 2023

The Cultural and Creative Cities Monitor (hereafter ‘Monitor’) is a monitoring and benchmarking tool first developed in 2017 to allow European cities to compare and contrast their areas of excellence and improvement in terms of culture and creativity. Since its first launch, the Monitor has served as a guidance tool for policy makers at local, national and European level. The 2023 update of the Monitor provides a methodological improvement and a revision of the selected indicators, so to ensure the tool remains reliable and coherent, thus enabling meaningful comparisons of cities’ performance over time. The new results are available for three different reference years, with a focus on the most recent, 2019. These latest results provide a snapshot of the situation just before the COVID-19 pandemic. An updated statistical assessment of the Monitor provided in this report allows the user to use the Monitor consciously, encouraging an informed use of the data and monitoring tools provided. The update has also been applied to the online platform which, on top of the pre-existing tools, also allows users to compare and contrast the performance of 196 European cities at three different points in time.

A Senior Liveability Index for European Cities

Article

Jan 2023

The demographic challenge of the ageing population in European countries needs to be assessed in terms of sustainability and effectiveness of public policies improving the quality of life of the elderly. The European Commission has monitored the quality of life in European cities since 2004, through a survey on citizens’ perception of their liveability. Given that the assessment of life quality embeds a multitude of positive and negative aspects, we develop a composite indicator that we call Senior Liveability Index (SLI) to rank the performance of cities, to monitor the rank changes over time and to explore possible reasons.

A Tidy Framework and Infrastructure to Systematically Assemble Spatio-temporal Indexes from Multivariate Data

Article

Jul 2024

Indexes are useful for summarizing multivariate information into single metrics for monitoring, communicating, and decision-making. While most work has focused on defining new indexes for specific purposes, more attention needs to be directed towards making it possible to understand index behavior in different data conditions, and to determine how their structure affects their values and the variability therein. Here we discuss a modular data pipeline recommendation to assemble indexes. It is universally applicable to index computation and allows investigation of index behavior as part of the development procedure. One can compute indexes with different parameter choices, adjust steps in the index definition by adding, removing, and swapping them to experiment with various index designs, calculate uncertainty measures, and assess indexes’ robustness. The paper presents three examples to illustrate the usage of the pipeline framework: comparison of two different indexes designed to monitor the spatio-temporal distribution of drought in Queensland, Australia; the effect of dimension reduction choices on the Global Gender Gap Index (GGGI) on countries’ ranking; and how to calculate bootstrap confidence intervals for the Standardized Precipitation Index (SPI). The methods are supported by a new R package, called tidyindex. Supplemental materials for the article are available online.

A Case Study of the Sustainable Society Index (SSI)

Chapter

Jan 2024

In this chapter, we show the development and further improvements of the Sustainable Society Index (SSI) according to the process presented in Chap. 4. The aim of this analysis is to provide a practical example on the one hand and to illustrate the complexity of the development of index systems on the other hand.

Development of Smart Mobility Infrastructure in Saudi Arabia: A Benchmarking Approach

Article

Full-text available

Feb 2023

Fayez Khalaf Alanazi

Smart mobility systems offers solutions for traffic congestion, transport management, emergency, and road safety. However, the success of smart mobility lies in the availability of intelligent transportation infrastructure. This paper studied smart mobility systems in three Asia-Pacific countries (South Korea, Singapore, and Japan) to highlight the major strategies leading their successful journey to become smart cities for aspiring countries, such as the Kingdom of Saudi Arabia (KSA), to emulate. A robust framework for evaluating smart mobility systems in the three countries and Saudi Arabia was developed based on the indicators derived from the smart mobility ecosystem and three major types of transport services (private, public, and emergency). Sixty indicators of smart mobility systems were identified through a rigorous search of the literature and other secondary sources. Robots, drones, IoT, 5G, hyperloop tunnels, and self-driving technologies formed part of the indicators in those countries. The study reveals that the three Asia-Pacific countries are moving head-to-head in terms of smart mobility development. Saudi Arabia can join these smarter countries through inclusive development, standardization, and policy-driven strategies with clear commitments to public, private, and research collaborations in the development of its smart mobility ecosystem. Moreover, cybersecurity must be taken seriously because most of the smart mobility systems use wireless and IoT technologies, which may be vulnerable to hacking, and thus impact system safety. In addition, the smart mobility system should include data analytics, machine learning, and artificial intelligence in developing and monitoring the evaluation in terms of user experience and future adaptability.

The Global Innovation Index 2021

Book

Full-text available

Jan 2021

A framework based on statistical analysis and stakeholders’ preferences to inform weighting in composite indicators

Article

Full-text available

Sep 2021
ENVIRON MODELL SOFTW

Composite Indicators (CIs, a.k.a. indices) are increasingly used as they can simplify interpretation of results by condensing the information of a plurality of underlying indicators in a single measure. This paper demonstrates that the strength of the correlations between the indicators is directly linked with their capacity to transfer information to the CI. A measure of information transfer from each indicator is proposed along with two weight-optimization methods, which allow the weights to be adjusted to achieve either a targeted or maximized information transfer. The tools presented in this paper are applied to a case study for resilience assessment of energy systems, demonstrating how they can support the tailored development of CIs. These findings enable analysts bridging the statistical properties of the index with the weighting preferences from the stakeholders. They can thus choose a weighting scheme and possibly modify the index while achieving a more consistent (by correlation) index.

Composite Indicators to Measure Quality of Working Life in Europe: A Systematic Review

Article

Full-text available

Apr 2021
SOC INDIC RES

In the last two decades, Quality of Working Life (QWL) has become a core element of the European social model and the European Employment Strategy. "More and better jobs" is a strategic goal promoted within Europe for emphasising the attention in QWL. However, there is a large debate in the literature on the definition of QWL, its dimensions, and consequently on the methods to use for its measurement. To the best of our knowledge, the systematic reviews currently available in the literature on QWL measurement in European organisations investigate only a particular industry and/or working population. Moreover, they do not focus specifically on composite indicators, although they appear promising in facilitating QWL understanding and comparisons for supporting decision-makers and policy makers. To overcome these gaps, we conducted a systematic review to identify composite indicators for measuring QWL in European organisations. The review returned 19 studies that are analysed based on a set of factors related to QWL locutions, index name, geographical area, industry or population, level of analysis, dimensions, type of data, inputs, outputs, and test and/or validation. The results highlight a significant heterogeneity among the indicators, confirming the lack of an agreed upon QWL composite indicator for Europe. Such heterogeneity concerns also QWL dimensions. A critical comparison of the different composite indicators is provided, along with a unifying proposal of QWL macro-dimensions. Several gaps in the literature are pointed out suggesting directions for future research.

Exploring the link between asia-europe connectivity and sustainable development

Article

Full-text available

Apr 2021

Asia and Europe have made connectivity between people, businesses and institutions a top political priority in the frame of the Asia-Europe Meeting (ASEM) intergovernmental cooperation forum. In the ASEM context, policy leaders agreed that improving connectivity between countries should contribute to achieve the Sustainable Development Goals. Connectivity is a complex concept involving multiple dimensions. Yet in order to provide clear data-driven evidence to support policymaking on Asia-Europe sustainable connectivity, this concept calls for a measurement framework. The mismatch between the politically adopted definition of sustainable connectivity and existing metrics, led to the development of the two indexes described in this paper. Taken together, they help to explore and understand the relationship between connectivity and sustainability. The results suggest that connectivity and sustainability policy agendas have the potential to mutually reinforce one another. Higher values in connectivity are strongly associated with higher values in the social dimension of sustainability. There are evident gaps between European and Asian nations in terms of connectivity and sustainability. However, both continents are underpinned by the common challenge of reducing the environmental impacts of connectivity without neglecting economic/financial sustainability aspects, while at the same time ensuring that benefits will accrue to society at large.

Composite Indicator Analysis and Optimization (CIAO) Tool, v.2

Code

Full-text available

Jan 2021

The Composite Indicator Analysis and Optimization (CIAO) tool v.2 is an expansion of the automated Matlab menu-version of the approach presented by Becker et al. (2017)* for the advanced assessment of Composite Indicators (CIs). The CIAO tool allows users to: 1. Perform a detailed examination of the linear and nonlinear relationships among (i) the set of indicators and (ii) between the indicators and the Composite Indicator (CI). 2. Assess how the indicators are “balanced” within the CI, i.e. to what extent the influence of each indicator matches its assigned weight. 3. Tune the weights of the indicators so that the influence of each one matches pre-defined importance values. *Becker, W., M. Saisana, P. Paruolo, and I. Vandecasteele. 2017. Weights and importance in composite indicators: Closing the gap. Ecological Indicators 80:12-22.

MCDA Index Tool: an interactive software to develop indices and rankings

Article

Full-text available

Mar 2021

A web-based software, called MCDA Index Tool (https://www.mcdaindex.net/), is presented in this paper. It allows developing indices and ranking alternatives, based on multiple combinations of normalization methods and aggregation functions. Given the steadily increasing importance of accounting for multiple preferences of the decision-makers and assessing the robustness of the decision recommendations, this tool is a timely instrument that can be used primarily by non-multiple criteria decision analysis (MCDA) experts to dynamically shape and evaluate their indices. The MCDA Index Tool allows the user to (i) input a dataset directly from spreadsheets with alternatives and indicators performance, (ii) build multiple indices by choosing several normalization methods and aggregation functions, and (iii) visualize and compare the indices’ scores and rankings to assess the robustness of the results. A novel perspective on uncertainty and sensitivity analysis of preference models offers operational solutions to assess the influence of different strategies to develop indices and visualize their results. A case study for the assessment of the energy security and sustainability implications of different global energy scenarios is used to illustrate the application of the MCDA Index Tool. Analysts have now access to an index development tool that supports constructive and dynamic evaluation of the stability of rankings driven by a single score while including multiple decision-makers’ and stakeholders’ preferences.

COIN Tool User Guide

Technical Report

Full-text available

Nov 2019

The COIN Tool is a free Microsoft Excel-based tool designed to help users from research institutions, international organisations, European Union institutions, national and local governments, among others, in the process of building and analysing composite indicators. It was developed by the European Commission's Competence Centre on Composite Indicators and Scoreboards (COIN), at the Joint Research Centre.

Building composite indicators using multicriteria methods: a review

Article

Full-text available

Feb 2019
JBE

Composite indicators are increasingly recognised as a useful tool in policy analysis and public communication. They provide simple comparisons of units that can be used to illustrate the complexity of our dynamic environment in wide-ranging fields, such as competitiveness, governance, environment, press, development, peacefulness, tourism, economy, universities, etc. Their construction has been dealt with from several angles. Some authors claim that MCDM techniques are highly suitable in multidimensional frameworks when aggregating single indicators into a composite one, since this process involves making choices when combining criteria of different natures, and it requires a number of steps in which decisions must be made. In this paper, we conduct a literature review of papers published after 2002 in leading international journals indexed in a recognised database (JCR), in order to identify the different MCDM methods used for aggregating single indicators into composite ones. They have been classified in five categories: the elementary methods, the value and utility based methods, the outranking relation approach, the data envelopment analysis based methods and the distance functions based methods. In general, our review has shown a clear tendency towards an increasing number of papers that use MCDM methods to construct composite indicators since 2014.

Why So Many Published Sensitivity Analyses Are False. A Systematic Review of Sensitivity Analysis Practices

Article

Full-text available

Apr 2019
ENVIRON MODELL SOFTW

Sensitivity analysis provides information on the relative importance of model input parameters and assumptions. It is distinct from uncertainty analysis, which addresses the question ‘How uncertain is the prediction?’ Uncertainty analysis needs to map what a model does when selected input assumptions and parameters are left free to vary over their range of existence, and this is equally true of a sensitivity analysis. Despite this, many uncertainty and sensitivity analyses still explore the input space moving along one-dimensional corridors leaving space of the input factors mostly unexplored. Our extensive systematic literature review shows that many highly cited papers (42% in the present analysis) fail the elementary requirement to properly explore the space of the input factors. The results, while discipline-dependent, point to a worrying lack of standards and recognized good practices. We end by exploring possible reasons for this problem, and suggest some guidelines for proper use of the methods.

R: A Language and Environment for Statistical Computing

Book