Available via license: CC BY
Content may be subject to copyright.
This is an open-access article distributed under the terms of the Creative Commons Attribution License.
This paper may be copied, distributed, displayed, transmitted or adapted for any purpose, even commercially, if provided,
in a clear and explicit way, the name of the journal, the edition, the year and the pages on which the paper was originally
published, but not suggesting that RAM endorses paper reuse. This licensing term should be made explicit in cases
of reuse or distribution to third parties.
Este artigo pode ser copiado, distribuído, exibido, transmitido ou adaptado para qualquer fim, mesmo que comercial, desde
que citados, de forma clara e explícita, o nome da revista, a edição, o ano e as páginas nas quais o artigo foi publicado
originalmente, mas sem sugerir que a RAM endosse a reutilização do artigo. Esse termo de licenciamento deve ser
explicitado para os casos de reutilização ou distribuição para terceiros.
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
Special Issue, https://doi.org/10.1590/1678-6971/eRAMD220077.en
Submitted: Apr. 3, 2022 | Approved: Aug. 18, 2022
Challenges of impact measurement in
social innovation: Barriers and
interventions to overcome1
Desafios na medição do impacto da inovação social:
Barreiras e intervenções para superar
Jorge Cunha1, Wellington Alves2, and Madalena Araújo1
1 ALGORITMI Research Center, University of Minho, Guimarães, Portugal
2 Polytechnic Institute of Porto, Felgueiras, Portugal
Authors notes
Jorge Cunha is now an associate professor at the Engineering School of University of Minho;
Wellington Alves is now an adjunct professor at the Higher School of Technology and
Management of Polytechnic Institute of Porto; Madalena Araújo is now a full professor at the Engi-
neering School of University of Minho.
Correspondence concerning this article should be addressed to Jorge Cunha, Avenida da Universi-
dade, campus de Azurém, Guimarães, Portugal, ZIP code 4800-058. Email: jscunha@dps.uminho.pt
To cite this paper: Cunha, J., Alves, W., & Araújo, M. Challenges of impact measurement in social innova-
tion: Barriers and interventions to overcome. Revista de Administração Mackenzie, 23(6), 1–32. https://doi.
org/10.1590/1678-6971/eRAMD220077.en
1 This work has been supported by Fundação para a Ciência e Tecnologia (FCT) within the Project
Scope: PTDC/EGE-OGE/31635/2017.
2
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
ABSTRACT
Purpose: This paper reviews extant research related to the impact assess-
ment of social innovation and identifies several barriers to this assessment
.
Following this is a proposal of interventions to overcome these barriers.
Originality/value: Social innovation has played an essential role in
enhancing positive changes for society. Nonetheless, measuring its
impact is a very significant challenge due to the many barriers faced in
selecting metrics that fit its definition and goal. Recognizing these chal-
lenges, the main contribution of this paper was to identify the most
common barriers and to suggest how these barriers can be overcome.
Design/methodology/approach: Two approaches were considered in this
research. On the one hand, a bottom-up approach was applied to review
relevant literature related to impact metrics for social innovation and
good practices toward social innovation impact assessment. On the
other hand, we used a top-down approach through collecting and ana-
lyzing research projects related to identifying metrics of social innova-
tion impact, broadly disseminated and well-consolidated in the current
literature.
Findings: This research offers valuable insights to academic researchers,
policy decision-makers, and practitioners working in the field of social
innovation by identifying and classifying the main barriers faced to
measuring the impact of social innovation, namely lack of stakeholder
awareness in the field of social innovation, difficulties in selecting the
metrics to assess social innovation, problems in selecting criteria to
identify best-fitted indicators to social innovation, lack of beneficiaries
engagement, lack of financial and public support, and lack of consensus
in the social innovation definition.
Keywords: social innovation, impact measurement, barriers, social
impact, social value
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
3
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
RESUMO
Objetivo: Este artigo revisa pesquisas existentes relacionadas à avaliação
do impacto da inovação social e identifica várias barreiras para essa
avaliação. Em seguida, são propostas intervenções para superar essas
barreiras.
Originalidade/valor: A inovação social tem desempenhado um papel
importante na promoção de mudanças positivas para a sociedade. No
entanto, medir seu impacto é um desafio muito significativo devido às
muitas barreiras enfrentadas na seleção de métricas que se encaixam em
sua definição e objetivo. Reconhecendo esses desafios, a principal con-
tribuição deste artigo foi identificar as barreiras mais comuns e sugerir
como elas podem ser superadas.
Design/metodologia/abordagem: Duas abordagens foram consideradas
nesta pesquisa. Por um lado, aplicou-se uma abordagem bottow-up para
revisar a literatura relevante relacionada a métricas de impacto para ino-
vação social e boas práticas para avaliação de impacto de inovação social.
Por outro, utilizou-se uma abordagem top-down, por meio da coleta e
análise de projetos de pesquisa relacionados à identificação de métricas
de impacto da inovação social, amplamente difundidos e bem consoli-
dados na literatura atual.
Resultados: Esta pesquisa oferece insights valiosos para pesquisadores
acadêmicos, decisores de políticas e profissionais que trabalham no
campo da inovação social, identificando e classificando as principais bar-
reiras enfrentadas para medir o impacto da inovação social, ou seja, a
falta de conscientização das partes interessadas no campo da inovação
social, dificuldades para selecionar as métricas para avaliar a inovação
social, dificuldades para selecionar critérios para identificar indicadores
mais adequados à inovação social, falta de engajamento dos beneficiá-
rios, falta de apoio financeiro e público e falta de consenso na definição
de inovação social.
Palavras-chave: inovação social, medição de impacto, barreiras,
impacto social, valor social
4
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
INTRODUCTION
In the coming decades, the use of new technologies, such as digitaliza-
tion, industry 5.0, and sustainable and clean energy, will play a key role in
supporting governments and industries to overcome pressing social prob-
lems faced by society. Problems related to the scarcity of resources, the tran-
sition to sustainable energy, and demographic change, are the main concerns
for the next decades. In this sense, social innovation (SI) can be seen as an
important approach to contributing to responding to these societal chal-
lenges (Mildenberger et al., 2020).
The concept of social innovation has emerged as an important activity to
enhance social value creation for both companies and communities and
thereby contribute to socio-economic inclusion (Weaver & Marks, 2017).
Over the last years, the concept of SI has been increasingly popular in the
policy and public debate due to the relevance that it can play in generating
inclusive growth as well as empowering people towards enhancing positive
changes for societies (von Jacobi & Chiappero-Martinetti, 2017).
As a key driver for social change, SI is believed to lead to sustainable
outcomes for society. The topic’s relevance has become even more impor-
tant due to the possible contribution of SI initiatives to support sustainable
development and foster actions toward the framework of the United Nations’
sustainable development goals (SDGs). In this direction, the breakthrough
of SI can contribute to meeting the objectives of the SGDs in different areas;
it can be justified due to the possibility of fomenting initiatives in different
areas and sectors of activity (Nylund et al., 2021).
However, being a relatively new and complex concept, measuring the
impact of SI actions and practices is a very significant challenge for researchers
.
Also, empirical evidence on the potential social impact of social innovation
is still scarce for guiding investigation in this field (Antadze & Westley, 2012;
Organisation for Economic Co-operation and Development – OECD, 2010).
Despite the interest and popularity of measuring the social impact of SI
initiatives, the development of reliable and shared measurement practices
has emerged as a barrier to the widespread adoption of those SI initiatives
(Unceta et al., 2020). According to Rawhouser et al. (2019), the use of met-
rics to assess social impact aims to measure the magnitude of its implica-
tions in a particular context, ranging from research and development to sus-
tainable initiatives. Nonetheless, the authors also argue that quantifying
these initiatives requires a precise specification to which social outcomes
are compared and a robust specification of the measures used to evaluate
the context.
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
5
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
The current literature has various metrics for measuring social impact,
and each method presents different approaches and characteristics. Yet, the
choice for the metrics depends on the context that will be used, and also
the type of impact analyzed, which configures a gap inherent to the process
of measuring the impact and value created in the domain of SI (Perrini
et al., 2021).
Thus, the main objective of this paper is to identify the most common
barriers discussed in the previous literature in this field and then to propose
a set of interventions that help to overcome those barriers. Moreover, for
both the identified barriers and interventions, a typology is proposed that
allows them to be classified and makes their interpretation more useful and
meaningful for researchers, policymakers, and practitioners. Therefore, the
contribution of this paper to the literature is twofold. Firstly, by providing
empirical evidence on barriers to measuring the social impact of social inno-
vation, namely in terms of different contexts, financial support, the com-
plexity of defining SI, selection of criteria to select indicators, and awareness
in the field of SI, among others. Secondly, by suggesting an empirical inter-
vention to overcome the listed barriers, focusing on a set of research related
to SI available in the current literature.
The remainder of this paper is organized as follows. The second section
begins with a literature review presenting this study’s relevance and the
challenges of measuring the social impact of SI. The third section outlines
the methodological approach for this paper. Key results are described and
discussed in the fourth section. Finally, the last section presents the main
conclusions highlighting the lessons learned.
THEORETICAL BACKGROUND
According to Bund et al. (2015), the term innovation originated from
industrialization, mainly related to technological inventions. Therefore, over
the years, efforts have been made to operationalize the concept in evidence-
based policymaking to make the term innovation more tangible. The growing
importance of social issues, mainly the concern with a more inclusive and
sustainable development, brought together the terms social and innovation
within policy and academic circles.
In the last decades, technological and economic innovations have been
seen as one of the most important contributions to societal well-being
through the generation of employment and economic growth (Rehfeld et al.,
2015).
6
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Nonetheless, to tackle the social and economic challenges that society is
facing today (namely, what has been known as the great challenges of the
21st century), that kind of innovation is not enough. Several authors (e.g.,
Mulgan, 2019; Banerjee et al., 2019; Benneworth & Cunha, 2015; Gabriel
et al., 2015) claim that addressing those societal challenges calls for a new
type of innovation: social innovation. This can be considered a tool to
empower society due to the SI process expectations of producing effects
in the societal changes or at least putting it under pressure (von Jacobi &
Chiappero-Martinetti, 2017).
Nowadays the SI process has gained importance due to the possibility of
overcoming social problems not deep-rooted by traditional solutions. Social
needs and solvency problems became mainstream regarding education,
social mobility, trust, and community life. SI has been seen as an alternative
to overcome these social issues (Dainiene
∙ & Dagiliee
∙ , 2016).
According to Cunha and Benneworth (2020), the current literature on
the idea of social innovation has grown sharply over the last decade, with
researchers seeking to define its concept by presenting several examples of
successful social innovations (Cunha & Benneworth, 2020). SI literature
has been mainly seen as a practical led field of research.
Despite being a complex issue to address, SI has the potential to deal
with social and environmental problems where conventional frameworks
have been ineffective (Antadze & Westley, 2012). Over the last decades,
researchers, who have been investigating the field of social impact and SI,
have brought different methods to light that aim to measure social impact,
some of which are well-known and useful to be applied to a range of sec-
tors. Each approach offers advantages and disadvantages for social impact
measurement (Perrini et al., 2021). Regarding the process of measuring
social impact, these authors suggest a set of steps that can be used as a guide
to evaluate social impact, summarized in Table 1.
Table 1
Process for measuring social impact
Steps Measurement process Analyzing benefits
Set the
objectives
It involves the definition of the main
subject to be analyzed. It may consider a
specific project, society, or enterprise.
Inputs List the resources, costs,
and investments incurred
in the process.
(continue)
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
7
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Steps Measurement process Analyzing benefits
Set
stakeholders
The analysis must consider comparing
the ex-ante and the ex-post situations. In
this case, there is a need to identify the
change produced, and it is necessary to
map the main stakeholders.
Activities Point out interventions
that will be carried out to
improve people´s lives.
Set the
appropriate
metrics
Understand the context of the
application, analyze the possibility of
getting data, and link the context
undertaken and the possible impact
created. It can consider quantitative and
qualitative metrics.
Outputs List the expected results
of the activity provided.
Measure Evaluate the outcomes obtained through
the selected metrics.
Outcomes Represent the possible
changes achieved for the
beneficiaries.
Report the
results
Communicate the results to external and
internal stakeholders, and compare the
change obtained.
Impacts Put in evidence the
outcomes of the change
that would have
happened regardless of
the social value.
Source: Adapted from Perrini et al. (2021).
The process in Table 1 summarizes steps to be followed when measuring
social impact. Therefore, several barriers can be faced in the process. Namely,
lack of data and subjective judgment are the main aspects that need attention
in the process (Bozsik et al., 2021; Bund et al., 2015; Gasparin et al., 2021).
In the case of social innovation initiatives, it also exerts pressure on
social forces, predicting when exactly their effect will happen. However, pre-
dicting when institutional change could happen (Antadze & Westley, 2012)
is difficult. Due to many societal problems faced by modern society, such as
access to public services, inequalities, climate change, and demographic
change, the evaluation of the impact of social initiatives in these areas has
become a significant aspect in tackling the challenge of understanding the
social impact of SI (Mildenberger et al., 2020).
In this sense, to assess the impact of social innovation initiatives or pro-
jects, it is important to remember that this process has different lifecycles
and requires different evaluation times. For example, Benneworth and Cunha
(2015) proposed a model to understand the social innovation process involving
Table 1 (conclusion)
Process for measuring social impact
8
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
a series of interlinked stages inspired by the non-linear technological inno-
vation model in the most generic sense. This model captures the overlap,
interaction, and different ordering of activities, the variety of sources and
inputs, and the multiple relationships underlying the innovation process
(Russell & Williams, 2002).
SI has been considered a key driver of economic and development growth
(Ates et al., 2019; Vasin et al., 2017), that is, to achieve sustainable develop-
ment. Yet, SI is a complex, dynamic and socio-economic phenomenon that
needs to be approached holistically to be adequately measured and assessed
(Carayannis et al., 2018).
Von Jacobi and Chiappero-Martinetti (2017) argue that the assessment
of the consequences which SI initiatives can deliver is based on two key
aspects, namely: 1. the benefits which will be generated at both social and
individual levels and 2. the importance to provide a broader account of the
potential impact generated by SI focusing tangible and intangible effects.
Also, Bund et al. (2015) suggest that to measure SI impact, different
perspectives should be taken into account, such as the innovation perfor-
mance of projects and the innovativeness of the organizations. Furthermore,
the innovativeness of spatial units, such as the societies, should be accounted
for, which can be analyzed at national (macro), regional (meso), or munici-
pal (micro) levels.
In a similar line of reasoning, Cunha and Benneworth (2020) propose a
conceptual framework model to measure the impact of SI. These authors
claim that this framework helps to identify the most significant indicators
for capturing and assessing the effects of SI while recognizing that the selec-
tion of these indicators should be seen as an iterative process, establishing
cause and effect relationships between actions and results and simplifying
the complexity of the measurement process. In the conceptual model pro-
posed, the impact of SI is conceived as a set of results that manifests through
different periods, at different spatial scales, and must consider the value
experienced by beneficiaries and all stakeholders involved, which implies a
large set of indicators, categorized in several dimensions, to capture the
impact of SI completely. In turn, Cunha et al. (2019) investigate the litera-
ture regarding SI impact assessment and discuss the challenges posed by
measuring that impact and how these measurement approaches may change
the assessment process. Their analysis found that methodologies for meas-
uring the impact of SI have been mainly undertaken in Europe and con-
firmed the lack of SI frameworks, methodologies, and metrics capable of
measuring the social impact of SI.
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
9
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
As SI is considered a forefront approach, the current literature still fails
to deliver frameworks or methodologies that measure the impact of SI ini-
tiatives. However, some recent initiatives are available. For instance, the
Simpact project aimed to analyze several European projects in the area of SI
and proposed a methodological tool to analyze the impact of the selected
projects (Simpact, 2014); CrESSI is a project that aimed to examine the
effect of projects focusing on initiatives related to an inclusive and sustainable
society in Europe (Nicholls, 2017); SI-DRIVE was a project that investigated
over 1,000 cases worldwide associated with SI, where the main output of
the projects focused on contributing to improve the theoretical and empiri-
cal context of SI (Howaldt & Schwarz, 2016).
Yet, despite all these projects significantly contributing to a better
understanding of the importance of measuring the impact of SI, they mostly
take place in organizations or projects with social goals. However, Gasparin
et al. (2021) claim that SI can also be used as a driver to support competitive
advantages from different sectors, ranging from technology, science, and
companies. Once the sector responds positively to social needs and seeks to
contribute to societal change, the action and its impact should be investi-
gated in the light of SI definition (Mongelli & Rullani, 2017; Nicolopoulou
et al., 2017).
In this comprehensive background, the current literature offers different
streams for SI. However, three of them seem clear. Firstly, the importance of
the topic (social innovation) is to overcome the barriers related to social
changes. Secondly, the linkage of SI and SGDs is still little explored in the
literature. Thirdly, the challenge of the impact measuring of SI is precisely
the focus of this research (García-Jurado et al., 2021; Rodrigo & Palacios,
2021). Notwithstanding the complex challenge of developing metrics to
measure the impact of SI initiatives, it is evident that these initiatives have
been contributing to improving people’s lives, which means that it is even
more important to investigate SI metrics, only thus it will be possible to
understand the real impact of SI on the society (Mihci, 2020).
METHODOLOGY
Two paradigms were considered in this research to address the problem
being studied. On the one hand, a bottom-up approach is widely used to
analyze individual concepts from a global perspective to a specific one, pre-
cisely the case of this research. This approach was applied considering the
relevant literature on impact measures for SI and good practices for SI impact
10
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
assessment. On the other hand, a top-down approach relies on looking for-
ward to analyzing a big picture of the concept of SI to a smaller one, namely
measurement and practices related to this topic. In this research, a top-
down approach was used to collect and analyze research projects related to
identifying metrics of SI impact, broadly disseminated and well-consolidated
in the current literature.
The data were then analyzed in the light of content analysis. According to
Bengtsson (2016), this approach is used in qualitative research to organize
and prompt data implications from data collected and then draw new findings
.
The use of content analysis can also be considered a useful research strategy
that allows researchers to investigate previous analyses, to get further results
from the empirical findings. It is also an alternative to the traditional narra-
tive of research studies (Hsieh & Shannon, 2005). As the results come from
different individual sources, a content analysis was conducted to analyze
and categorize them systematically.
Figure 1 summarizes the stages carried out to develop this research as
well as the methodological approach applied.
Figure 1
Methodological approach
Social impact of SI: setting strategies
Literature review
Source:
Web of Science; Scopus; Database of EU Funded
Research and Innovation Projects
Survey of SI measures
Selected projects:
Resindex; Inobasque; Sinergiak; Nesta;
European Barcamp; Tepsie; Simpact; Blueprint
Content analysis
Barriers faced to
measure SI
Sharing expirences
Interventions to
overcome barries
Good practices to be considered on
measuring social impact of SI
Social impact assessment of SI
1
2
3
4
5
Source: Elaborated by the authors.
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
11
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Stage 1 is based on the literature review, which provides the fundamen-
tals of the research. Stage 2 focuses on the main identified impact measures
for SI discussed in the selected projects. Stage 3 starts picturing a meta-
analysis of the results obtained from stages 1 and 2 and summarizes barriers
faced to measure SI impact. Stage 4 presents a set of good practices that we
recommend being considered when evaluating SI impact. Finally, Stage 5
proposes interventions to overcome when developing metrics to assess SI.
Survey of social impact measures
The first step developed in this research aimed to select a set of studies
undertaken in different countries to assess the impact of SI. These works
were mainly research projects widely recognized in the current literature
regarding SI. The research relied on an extensive literature review, where
reports, scientific papers, and projects related to social impact metrics for SI
were consulted. The works were selected by resorting to scientific databases
such as Scopus and Web of Science and the database from European Union
(EU) funded research and innovation projects, focusing on outputs of pro-
jects related to impact assessment of SI. These platforms were chosen for
their disciplinary coverage and due to data availability. The selection of these
databases as the basis of our study focuses on the importance of such tools
as a source of documentation to support the work of academic researchers.
When searching for academic works within various contributions, efficiency
becomes a priority. Being able to search in a trustworthy and authoritative
database saves valuable time that would otherwise be spent cross-checking
multiple databases and having to confirm results (Sánchez et al., 2017).
After this screening process, and based on the previous work developed
by Cunha et al. (2019), the research focused on the following works:
Inobasque (Unceta et al., 2016), Resindex (Sinnergiak, 2013), Nesta (Inno-
vation Mapping Team – Nesta, 2019), European Barcamp (Dainiene
∙ &
Dagiliee
∙ , 2015), Tepsie (Mendes et al., 2012), Sinnergiak (Sinnergiak, 2013),
Simpact (Simpact, 2014), and Blueprint (Bund et al., 2013).
Barriers faced to measuring the social impact of innovation
Based on the literature review, this step identified the main barriers faced
to measuring the social impact of SI (further details and outputs of Step 3
can be found in Cunha & Benneworth, 2020). The identification of rele -
vant research was a prerequisite to analyzing those barriers. Considering
12
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
the difficulties highlighted by the research projects reviewed (in Step 1), the
identified barriers were selected to develop metrics for assessing the impact
of SI. This step focused on listing the main barriers to measuring SI’s social
impact, which are presented in tables 4 to 9.
Sharing experiences
Step 4 aimed to examine examples of good practices on measuring the
social impact of SI, insights from the reviewed projects, and experiences
identified in Step 3 to identify important issues and methodological chal-
lenges, learn from the research analyzed, and propose actions to overcome
barriers identified in Step 2.
Interventions to overcome
According to the results of the previous steps and the previous studies
by Cunha and Benneworth (2020) and Cunha et al. (2019), measuring the
social impact of SI faces several challenges. For that, Step 5 focused on sug-
gesting a set of interventions to be followed by future works to overcome
the barriers to measuring the social impact of SI initiatives.
KEY FINDINGS
This section presents and discusses the key findings from the literature
review on challenges posed by measuring the impact of SI. The discussion of
results considers three main interrelated aspects: insights from the reviewed
projects, barriers faced to measure the impact of SI, and interventions to
overcome the listed barriers.
Insights from the reviewed projects
The insights presented in this subsection are based on projects that
have been researched in SI. The results allowed us to understand the main
projects discussing the barriers faced to developing frameworks and meth-
odologies for measuring SI impact. Thus, a set of the main projects dis-
cussed in the current literature and available at EU-funded research and
innovation projects were selected, contributing to several barriers identified
when dealing with SI impact assessment.
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
13
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
The analysis puts in evidence the role of Tepsie, European Barcamp, and
Inobasque projects as the primary research on the main challenges faced to
measure the SI impact. The case of Tepsie is a research project supported by
the European Commission entitled “The Theoretical, Empirical and Policy
Foundations for Building Social Innovation in Europe.” This project outlines
several barriers to measuring the impact of SI (summarized in tables 4 to 9)
and suggests a conceptual framework to overcome the identified obstacles.
European Barcamp is research supported by Italiacamp, which has been
working on developing networks for SI processes. The European Barcamp has
created the ES + Methodology to measure the impact of SI. This methodology
aims to map the innovation and local entrepreneurship ecosystems by iden-
tifying innovative business models and disseminating stories and best prac-
tices. This research has identified several barriers to measuring the impact
of SI (summarized in tables 4 to 9).
The Inobasque (Basque Innovation Agency) is a non-profit company
that acts as a regional innovation partnership with Resindex (Regional Inno-
vation Index –Sinnergiak, 2013) and Simpact projects. These works have
been leading research on SI seeking to foster collaborative actions in the
region of the Basque country. Altogether, these projects aimed to investigate
SI in social cohesion, competitiveness, and sustainability of societies. The
results of these projects summarize a set of challenges they faced in developing
metrics to assess SI.
Studies on barriers and challenges in measuring the social impact of SI
are not yet widely discussed in the current literature. Although recognizing
the need to explore these first results further, the findings should be able to
support researchers and decision-makers to understand better developing
metrics to assess the social impact of SI.
Barriers faced to measuring the impact of SI
Based on the projects examined, a set of barriers were identified and
served as bases to suggest interventions to overcome them. The main barri-
ers were analyzed, and based on that, we propose their classification into the
following categories, as presented in Table 2.
14
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 2
Proposed categories for barriers faced by SI
Acronym Definition
PS Public support
FS Financial support
DSI Definition of SI
SM Selection of metrics to measure SI
SC Selection of criteria to select indicators
AMSI Assessment and measurement of social impact
BE Beneficiaries’ engagement
AIFSI Awareness in the field of social impact
Source: Elaborated by the authors.
As shown in Figure 2, the main barriers are related to 1. difficulties in
assessing and measuring the social impact of SI (AMSI), 2. awareness in the
field of social impact (AIFSI), and 3. selection of metrics to measure social
impact (SM).
Figure 2
Barriers faced to measuring SI impact
Indentified barriers
10
9
8
7
6
5
4
3
2
1
FS
0
DSI SM SC AMSI BE AIFSI PS
Source: Elaborated by the authors.
Based on the research projects analyzed and the identified barriers,
Figure 3 presents insights from the reviewed projects versus the barriers
faced. The results showed that financial support (FS) was a barrier high-
lighted only by the Tepsie project. According to this project, there is a lack
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
15
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
of funding devoted to SI compared to technological innovation. If this
shortage of funding is overcome, the number of investments in SI initiatives
would probably increase, and it may result in benefits for society and stake-
holders. Aiming to overcome this barrier, the project suggests better sup-
port from foundations and public agencies for SI initiatives.
Figure 3
Insights versus barriers faced
FS DSI SM SC AMSI BE AIFSI PS
Number of barriers
5
4
3
2
1
0
Resindex Inobasque Sinergiak Nesta European Barcamp Tepsie Simpact Blueprint
Source: Elaborated by the authors.
The definition of SI (DSI) was highlighted as an important barrier to be
faced by almost all projects consulted in this research. As presented in
Figure 3, Inobasque was the project which offered the highest number of
barriers for DSI. For instance, this project pointed out a lack of understanding
of SI, a lack of knowledge of social innovation and its impact meaning, and
difficulties defining the goal of SI. The results agree with the current litera-
ture, which discusses the challenges in defining SI (Agostini et al., 2017).
For the case of the selection of metrics for SI (SM), the results indicated
that it was considered a common barrier for all the reviewed projects. Tepsie
and European Barcamp were the ones that presented several difficulties in
this field, such as a lack of agreement on specific configuration rules to
select indicators, lack of understanding about the potential capacity of indi-
cators to measure the social impact of SI, lack of data sources, and lack of
networks.
Regarding the selection of criteria to select indicators for SI (SC), results
from the current literature show that it is under-discussed; some attempts
are presented for other sectors, such as sustainability. Still, in the case of SI,
as argued by Gault et al. (2014), Krlev et al. (2014), and Kleverbeck et al.
(2019), it persists as a gap.
16
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Nonetheless, the results presented here showed that the European
Barcamp is at the forefront of this discussion, offering some difficulties in
selecting criteria to select indicators for SI, namely the lack of strategies
focusing on empowerment and local inclusion.
Almost all the reviewed projects listed the assessment and measuring
social impact (AMSI) as a challenging task, and some problems related to
AMSI were pointed out. European Barcamp, Simpact, and Tepsie were the
ones that present a set of important barriers to be overcome in this field,
namely lack of experience and motivation in measuring the social impact of
SI, difficulties in quantifying the effect of SI, difficulties in determining the
decision process, and difficulties in reaching regional exchange.
Concerning the beneficiaries engagement (BE), although several pro-
jects have mentioned it, it was mainly addressed by Tepsie. The difficulties
presented by the project are related to aspects such as lack of engagement,
the definition of boundaries and players of SI, lack of understanding of the
role of stakeholders, lack of collaboration, and lack of networks between
the people involved in these initiatives. The results showed that despite the
growing social needs, there is a lack of understanding about what SI can
deliver to society. This demonstrates the need for a better beneficiary engage-
ment in developing SI initiatives in this area (Wittmayer et al., 2019).
The barriers listed by the reviewed projects in the category awareness in
the field of SI (AiFSI) are related to aspects such as lack of initiatives to dis-
seminate SI, low integration between stakeholders, lack of initiatives to
identify gaps in measuring SI, and lack of engagement and raising awareness
with the civil society. These barriers were mainly addressed by Inobasque
and European Barcamp, which can be justified due to the efforts made by
these projects to increase the awareness of SI among stakeholders.
In the case of public support (PS), it was widely regarded by the projects
as an important barrier faced by researchers and practitioners working in SI.
Nesta, European Barcamp, and Tepsie listed a set of difficulties related to PS
in supporting SI, for instance, lack of SI public policies, lack of evaluation,
and investment of previous initiatives.
Interventions to overcome
The results obtained allowed us to select and understand a set of chal-
lenges to SI impact assessment which will be summarized in tables 4 to 9.
Based on these challenges, this research proposes a set of interventions to
overcome these barriers. The suggested interventions are classified into five
main categories, as presented in Table 3.
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
17
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 3
Proposed interventions to be considered for overcoming barriers to SI projects
Acronomyn Definition
CDSI Clear definition of SI
FPS Fostering public support for SI
IDMSI Increase the development of metrics for SI
PBESSI Promotion of a better engagement between researchers, public/private
organizations, and practitioners working in the field of SI
DPBSI Dissemination of the potential benefits of SI initiatives for beneficiaries
Source: Elaborated by the authors.
The preliminary results indicate that in the long term, to overcome bar-
riers to SI impact assessment, an important step for researchers and decision-
makers would be to get support to foster SI between public agencies, define
and refine the goal of evaluation and identification of macro, meso and micro
indicators to evaluate SI initiatives or practices.
It is worth mentioning that, as SI push into different contexts, the social
impact assessment of these initiatives can also change the lives of communi-
ties and organizations.
The main results of this research are summarized in tables 4 to 9. In
these tables, the potential benefits of the implementation of the interven-
tions suggested are highlighted (act), where the left columns bring the pro-
posed categories and main barriers selected from the reviewed projects,
which means that those barriers need to be carefully identified (track) in SI
projects, middle columns propose the interventions to overcome (spot)
these barriers according to lessons learned from the reviewed research/pro-
jects, referenced in the right column.
The results presented in tables 4 to 9 evidence the challenging task of
measuring the social impact of SI initiatives. This is particularly evident in
the case of barriers: assessment and measurement of social influence (AMSI),
awareness in the field of social impact (AiFSI), and selection of metrics to
measure SI (SM), which were the categories with the highest numbers of
identified barriers (11 and 7, respectively). The suggested interventions to
overcome the barriers and the potential benefits proposed in the tables
should be considered as a first attempt to bring together different stakeholders,
such as public institutions, communities, and researchers, as fundamental
drivers to overcome these barriers through the development of suitable
practices in different categories.
18
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 4
Streamlining for barriers and interventions for public support
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Barriers faced to measuring
the social impact of social
innovation (track)
Interventions to overcome
(spot) Potential benefit (ACT) Pessons learned from
Public support (PS)
Lack of public support Foster social innovation
between public agencies (FPS)
Empowering democratizing,
raising awareness
Inobasque (Unceta et al., 2016) (Dainiene
•
& Dagiliene
• , 2015)
Lack of social innovation
public policies
Social policy experimentation
(DPBSI)
Support a systematical
evaluation of the social
impact of SI on social
ventures
Inobasque (Unceta et al., 2016) (Dainiene
•
& Dagiliene
• , 2015)
Lack of evaluation and
investment in previous
policies
Support of progressive
foundations (FPS)
Assessment and
measurement of impact
Tepsie / European Barcamp (Dainiene
• &
Dagiliene
• , 2015) (Mendes et al., 2012)
Lack of interest from public
organizations to measure
the social impact of SI
Further discussion between
stakeholders in the field of SI
(PBESSI)
Funding growth and
diffusion of innovation
Tepsie / European Barcamp / Nesta
(Dainiene
• & Dagiliene
• , 2015) (Innovation
Mapping Team – Nesta, 2019) (Mendes
et al., 2012)
Fonte: Elaborated by the authors.
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
19
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 5
Streamlining for barriers and interventions for assessment and measurement of social impact
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Barriers faced to measuring
the social impact of social
innovation (track)
Interventions to overcome (spot) Potential benefit (ACT) Pessons learned from
Assessment and measurement of social impact (AMSI)
Difficulties in reaching
regional exchange
Special economic zone for social
innovation (FPS)
Gaining systematic insight into
social enterprises
Inobasque (Unceta et al., 2016)
(Dainiene
• & Dagiliene
• , 2015)
Lack of an appropriate level
of analysis for measuring
the social impact of SI
A better discussion of the
assessment level (meso/macro)
(IDMSI)
Dissemination of social
innovation indicators at the
organizational level
Resindex (Sinnergiak, 2013)
(Unceta et al., 2016)
Lack of experiences and
motivations in measuring
the social impact of SI
To express an integrated vision of
social innovation (DPBSI)
Making a clear statement on the
importance of social innovation
and putting it on the agenda
Sinnergiak (Sinnergiak, 2013)
Difficulties in differentiating
the potentiality and
realization of social
innovations
Identification of indicators that can
measure the consequences for the
development of indicators and also
for the design of public incentives
for social innovation (IDMSI)
Exploring different capacities
such as 1. knowledge capacities,
2. learning capacities
Resindex (Sinnergiak, 2013)
(Unceta et al., 2016)
Difficulties in quantifying
the impact of social
innovation
Identification of macro indicators
which can result from surveys and
gathering of information that do
refer to social innovation (IDMSI)
Provide an approach to the
conditions of context in which
social innovations occur
Tepsie (Mendes et al., 2012)
Risk or uncertainty
presented during the social
innovation process
A common approach to defining steps
to build a framework to measure the
social impact of SI (IDMSI)
Reduce failure in the innovation
process
Tepsie (Mendes et al., 2012)
(continue)
20
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Barriers faced to measuring
the social impact of social
innovation (track)
Interventions to overcome (spot) Potential benefit (ACT) Pessons learned from
Assessment and measurement of social impact (AMSI)
Complexity and resilience
of social systems
Cooperation across multi-stakeholder
environments to tackle those
problems (PBESSI)
Reduce the complexity of the
social challenges addressed
Tepsie (Mendes et al., 2012)
Difficulties in measuring
outcomes of SI
Predefine which outcomes should
be achieved (IDMSI)
Reduce failure in the innovation
process
Simpact / European Barcamp
(Innovation Mapping Team –
Nesta, 2019) (Simpact, 2014)
Determinization of the
causation
To relate the outcomes to the
inputs (IDMSI)
Assessment and measurement
of impact
Simpact / Eurpean Barcamp
(Mendes et al., 2012) (Simpact,
2014)
To calculate the impact
of SI
The impact assessment should be
accompanied by a set of tests to
check the results counterfactual
nature. For each of the impacts, the
degree of uncertainty (likelihood)
should be estimated (IDMSI)
Assessment and measurement
of impact
Simpact /Tepsie / Euopean
Barcamp (Dainiene
• & Dagiliene
• ,
2015) (Mendes et al., 2012)
(Simpact, 2014)
Difficulties in defining the
decision process
The impact assessment should be
presented and discussed with the
stakeholders (PBESSI)
Raising awareness in the field
of social impact
Simpact/Tepsie/SINNERGIAK
(Sinnergiak, 2013) (Mendes
et al., 2012) (Simpact, 2014)
Fonte: Elaborated by the authors.
Table 5 (conclusion)
Streamlining for barriers and interventions for assessment and measurement of social impact
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
21
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 6
Streamlining for barriers and interventions for Beneficiaries’ engagement
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Arriers faced to measuring
the social impact of social
innovation (track)
Interventions to overcome (spot) Potential benefit (ACT) Lessons learned from
Beneficiaries engagement (BE)
Definition of the type of
organization (profit or non-profit)
to measure the social impact
Define and refine the goal of the
evaluation (CDSI)
Effective measurement Resindex (Sinnergiak, 2013)
(Unceta et al., 2016)
Lack of engagement between
beneficiaries
Projects to involve beneficiaries of
SI (DPBSI)
Beneficiaries
engagement
Tepsie (Mendes et al., 2012)
Lack of definition for the
boundaries and players of social
innovation
A common framework to define important
sectors and players (PBESSI)
Raising awareness in
the field of social impact
Tepsie / Nesta (Innovation
Mapping Team – Nesta,
2019) (Mendes et al., 2012)
Lack of awareness about the
social impact delivered by social
innovation
Development of workshops and seminars
bringing together stakeholders (DPBSI)
Raising awareness in
the field of social impact
Tepsie / European Barcamp
(Dainiene
• & Dagiliene
• , 2015)
(Mendes et al., 2012)
To determine the role of
stakeholders
Clear definition of who will play a role
in the assessment process, when, and
how (DPBSI)
Reduce the complexity
of the social challenges
addressed
Simpact /Tepsie (Mendes
et al., 2012) (Simpact, 2014)
Lack of collaboration and
networks
Take into account the systemic and
collaborative emphasized generation of
innovation measurement by including
“Firms with national/international
collaboration on innovation” (DPBSI)
Assessment and
measurement of impact
Blueprint / Tepsie (Mendes
et al., 2012) (Bund et al.,
2013)
Fonte: Elaborated by the authors.
22
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 7
Streamlining for barriers and interventions for Selection of metrics to measure SI
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Barriers faced to measuring
the social impact of social
innovation (track)
Interventions to overcome (spot) Potential benefit (ACT) Lessons learned from
Selection of metrics to measure SI (SM)
Lack of agreement on
specific configuration rules
to select indicators
To define the characteristics and
degree of generalization of a social
problem (IDMSI)
Reduction of conflicting
agencies, contexts, and
opportunities in the field of SI
Inobasque (Sinnergiak, 2013)
(Unceta et al., 2016)
Lack of agreement about
the potential capacity of
indicators to me asure the
social impact of SI
To define the knowledge, learning,
and development capacity of the
indicator (IDMSI)
Absorptive capacity and social
innovation at the
organizational level
Resindex (Sinnergiak, 2013)
(Unceta et al., 2016)
Lack of approaches based
on macro and comparable
indicators
Identification of macro indicators
which can be resulted from surveys
and gathering of information that
do refer to social innovation (IDMSI)
Provide an approach to the
conditions of context in which
social innovations occur
Sinnergiak (Sinnergiak, 2013)
Lack of data sources Building up social organization
surveys and indicators (IDMSI)
Knowledge exchange and
network data from open and
web data
Nesta (Innovation Mapping
Team – Nesta, 2019)
Lack of data Lack of a clear definition of the
concept of social innovation before
selecting measures (IDMSI)
Provide an approach to the
conditions of context in which
social innovations occur
Tepsie (Mendes et al., 2012)
(continue)
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
23
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 7 (conclusion)
Streamlining for barriers and interventions for Selection of metrics to measure SI
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Barriers faced to measuring
the social impact of social
innovation (track)
Interventions to overcome (spot) Potential benefit (ACT) Lessons learned from
Selection of metrics to
measure SI (SM)
Lack of networks Further discussion between
stakeholders in those fields (DPBSI)
Knowledge exchange and
network data from open
and web data
Tepsie / European Barcamp / Nesta
(Dainiene
• & Dagiliene
• , 2015)
(Innovation Mapping Team – Nesta,
2019) (Mendes et al., 2012)
Lack of basic assumptions
from existing innovation
metrics
To relate the outcomes to the
inputs (IDMSI)
Assessment and measurement
of impact
Blueprint / Simpact / Tepsie (Dainiene
•
& Dagiliene
• , 2015) (Mendes et al.,
2012) (Bund et al., 2013)
Fonte: Elaborated by the authors.
24
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 8
Streamlining for barriers and interventions for selection of criteria to select indicators
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Barriers faced to measuring the social
impact of social innovation (track) Interventions to overcome (spot) Potential
benefit (ACT) Lessons learned from
Selection of
criteria to select
indicators (SC)
Lack of strategies to select the indicator to
measure empowerment and local inclusion
Active involvement of the community
(local and across-sectors) (IDMSI)
Social inclusion European Barcamp (Dainiene
•
& Dagiliene
• , 2015)
Fonte: Elaborated by the authors.
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
25
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Table 9
Streamlining for barriers and interventions for awareness in the field of social impact
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Barriers faced to measuring the
social impact of social
innovation (track)
Interventions to overcome
(spot) Potential benefit (ACT) Lessons learned from
Awareness in the field of social impact (AIFSI)
Identification of social
innovation opportunities
Accepting risks and diffusing
good practices (DPBSI)
Dissemination of knowledge
across the society
Inobasque (Unceta et al., 2016)
(Dainiene
• & Dagiliene
• , 2015)
Low integration between
stakeholders
Identification of Social Innovation
Cluster/ Park (PBESSI)
Mapping the actors in the area Inobasque (Unceta et al., 2016)
(Dainiene
• & Dagiliene
• , 2015)
Dissemination of SI initiatives Scaling-up, cross-regional
exchange, and systemic change
(DPBSI)
Making a clear statement on the
importance of social innovation
and putting it on the agenda
Inobasque (Unceta et al., 2016)
(Dainiene
• & Dagiliene
• , 2015)
Identification gaps in measures
for social innovation systems
and new opportunities for
collaboration
Recombination through network
analyses and complexity science
(IDMSI)
Disseminate the results of these
analyses via interactive data
visualizations and dashboards,
search engines, open databases,
and open-source software that
others can build on
Sinnergiak (Sinnergiak, 2013)
Lack of research mapping
opportunities and challenges
for selecting measures for the
social impact of SI
Defining what the research is
trying to achieve, the audiences,
projects, methods, and outputs
(DPBSI)
Provide an approach to the
conditions of context in which
social innovations occur
Nesta (Innovation Mapping
Team, 2019)
(continue)
26
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Streamline for barriers to measuring the social impact of SI
Category for
interventions
Barriers faced to measuring the
social impact of social
innovation (track)
Interventions to overcome
(spot) Potential benefit (ACT) Lessons learned from
Awareness in the field of
social impact (AIFSI)
Lack of engagement and
raising awareness with the
civil society
Dissemination and promotion of
the initiative involving the
community (PBESSI)
Assessment and measurement
of impact
European Barcamp (Dainiene
• &
Dagiliene
• , 2015)
Lack of understanding of the
scope of innovations systems
Publications that deal primarily
with the measurement of
innovation (regardless of whether
they focus on a specific type of
innovation (DPBSI)
Raising awareness in the field of
social impact
Blueprint / European Barcamp
(Dainiene
• & Dagiliene
• , 2015)
(Bund et al., 2013)
Fonte: Elaborated by the authors.
Table 9 (conclusion)
Streamlining for barriers and interventions for awareness in the field of social impact
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
27
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
CONCLUSION
This work is part of ongoing research, and the experiences revised in
this paper provided a broad review of good practices developed by researchers
who have been working on the impact measurement of SI in different regions
worldwide (for further information, see also Cunha et al., 2019). Related
literature was analyzed, covering a set of projects of a successful application
which also contributed to enhancing the discussion about the challenges of
measuring the social impact of SI.
This research offers valuable insights to academic researchers, policy
decision-makers, and practitioners working in the field of SI by identifying
and classifying the main barriers faced to measuring the impact of SI, namely
lack of stakeholder awareness in the area of SI, difficulties in selecting the
metrics to assess SI, problems to establish criteria to identify best-fitted
indicators to SI, lack of beneficiaries engagement, lack of financial and public
support and lack of consensus in the SI definition.
Moreover, to overcome these barriers and challenges, one significant
contribution of this paper is listing and classifying possible interventions
and their positive benefits derived from the lessons learned from the research
projects reviewed and analyzed. The results obtained can be considered a
point of departure for future research regarding the important issue of cor-
rectly measuring the impact of SI. It can also be helpful to policymaking,
companies, or non-governmental organizations when implementing new SI
initiatives and demonstrating their actual value to society.
Further, the results presented in this research offer some clues regarding
the challenges of measuring SI. The results put in evidence the urgent need
to develop metrics in this direction to overcome the barriers related to the
unknown impact of SI on society. The linkage between SI and SGDs was also
discussed in this research, and the results showed that it is still little explored
in the literature. Yet, despite the difficult task of developing metrics to measure
the impact of SI initiatives, it is evident that these actions have been con-
tributing to improving social change, meaning that it is urgent to investigate
new SI metrics, only thus it will be possible to comprehend the overall
impact of SI on the society entirely.
Although it is aninitial research, this approach can be central for future
scientific development in studying metrics for SI. The work is now pro-
ceeding with selecting indicators to assess SI’s social impact.
28
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
REFERENCES
Agostini, M. R., Vieira, L. M., Tondolo, R. P., & Tondolo, V. G. (2017). An
Overview On Social Innovation Research: Guiding Future Studies. Brazilian
Business Review, 14(4), 385–402. https://doi.org/10.15728/bbr.2017.14.4.2
Ates, S. A., Ateş, M., & Yülek, M. A. (2019). Going Beyond GDP: The Role
of Social Innovation in Building a Welfare State. In Handbook of Research on
Digital Marketing Innovations in Social Entrepreneurship and Solidarity Economics
(pp. 241–258). https://doi.org/10.4018/978-1-5225-8939-6.ch013
Antadze, N., & Westley, F. R. (2012). Impact Metrics for Social Innovation:
Barriers or Bridges to Radical Change? Journal of Social Entrepreneurship,
3(2), 133–150. https://doi.org/10.1080/19420676.2012.726005
Banerjee, S., Stephen, C., & Hulgard, L. (2019). People-Centered Social Innova-
tion: Global Perspectives on an Emerging Paradigm. Routledge. https://doi.org/
10.4324/9781351121026
Bengtsson, M. (2016). How to plan and perform a qualitative study using
content analysis. NursingPlus Open, 2, 8–14. https://doi.org/10.1016/j.
npls.2016.01.001
Benneworth, P., & Cunha, J. (2015). Universities’ contributions to social
innovation: reflections in theory & practice. European Journal of Innovation
Management, 18(4), 508–527. https://doi.org/10.1108/EJIM-10-2013-
0099
Bozsik, S., Szeman, J., & Musinszki, Z. (2021). How to Measure the Perfor-
mance of Social Innovation? Case Study of Hungarian Social Cooperatives.
The Business and Management Review, 12(01), 10–12. https://doi.org/10.
24052/bmr/v12nu01/art-24
Bund, E., Gerhard, U., Hoelscher, M., & Mildenberger, G. (2015). A methodo-
logical framework for measuring social innovation. Historical Social Research,
40(3), 48–78. https://doi.org/10.12759/hsr.40.2015.3.48-78
Bund, E., Hubrich, D.-K., Schmitz, B., Mildenberger, G., & Krlev, G. (2013).
Blueprint of social innovation metrics – contributions to an understanding
of opportunities and challenges of social innovation measurement. In Tepsie
[Issue September 2015. http://www.tepsie.eu/images/documents/D2.4_
final.pdf
Carayannis, E. G., Goletsis, Y., & Grigoroudis, E. (2018). Composite inno-
vation metrics: MCDA and the Quadruple Innovation Helix framework.
Technological Forecasting and Social Change, 131, 4–17. https://doi.org/10.
1016/j.techfore.2017.03.008
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
29
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Cunha, J., & Benneworth, P. (2020). How to measure the impact of social
innovation initiatives? International Review on Public and Nonprofit Marketing,
17, 59–75. https://doi.org/10.1007/s12208-019-00240-4
Cunha, J., Alves, W., Araújo, M., & Benneworth, P. (2019). An investigation
of existing social impact measures for social innovation. Working Papers
Series on Social Responsibility, Ethics and Sustainable Business, 35.
Dainiene
∙ , R., & Dagiliene
∙ , L. (2015). A TBL Approach Based Theoretical
Framework for Measuring Social Innovations. Procedia: Social and Behavioral
Sciences, 213, 275–280. https://doi.org/10.1016/j.sbspro.2015.11.537
Dainiene
∙ , R., & Dagiliene
∙ , L. (2016). Measurement of Social Innovation at
Organisation’s Level: Theoretical Issues. Economics and Business, 29(1),
96–103. https://doi.org/10.1515/eb-2016-0027
García-Jurado, A., Pérez-Barea, J. J., & Nova, R. (2021). A new approach to
social entrepreneurship: A systematic review and meta-analysis. Sustainability
(Switzerland), 13(5), 1–16. https://doi.org/10.3390/su13052754
Gasparin, M., Green, W., Lilley, S., Quinn, M., Saren, M., & Schinckus, C.
(2021). Business as unusual: A business model for social innovation. Journal
of Business Research, 125, 698–709. https://doi.org/10.1016/j.jbusres.
2020.01.034
Gabriel, M., Simon, J., Nicholls, A., Macmillan, P., Howaldt, J., Kopp, R.,
& Schwarz, M. (2015). New Frontiers in Social Innovation Research.
In A. Nicholls, J. Simon, & M. Gabriel (eds.), New Frontiers in Social Innovation
Research Palgrave Macmillan UK. https://doi.org/10.1057/9781137506801
Gault, F., Mulgan, G., Joseph, K., & Norman, W. (2014). Indicators for social
innovation. In Handbook of Innovation Indicators and Measurement, Reeder,
(pp. 420–438). https://doi.org/10.4337/9780857933652.00030
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative con-
tent analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.
org/10.1177/1049732305276687
Howaldt, J., & Schwarz, M. (2016). Social innovation and its relationship to
social change: Verifying existing Social Theories in reference to Social Innovation
and its Relationship to Social Change. SI drive.
https://d-nb.info/1113875666/34
Innovation Mapping Team (Nesta) (2019). Innovation mapping now [Issue
March].
Kleverbeck, M., Krlev, G., Mildenberger, G., Strambach, S., Thurmann, J.-F.,
Terstriep, J., & Wloka, L. (2019). Indicators for Measuring Social Innovation
(pp. 98–101).
30
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Krlev, G., Bund, E., & Mildenberger, G. (2014). Measuring What Matters-
Indicators of Social Innovativeness on the National Level. Information
Systems Management, 31(3), 200–224. https://doi.org/10.1080/10580530.
2014.923265
Mendes, A., Batista, A., & Fernandes, L. (2012). Barriers to Social Innovation.
A deliverable of the project: “The theoretical, empirical and policy foundations for
building social innovation in Europe.Tepsie.
Mihci, H. (2020). Is measuring social innovation a mission impossible?
Innovation: The European Journal of Social Science Research, 33(3), 337–367.
https://doi.org/10.1080/13511610.2019.1705149
Mildenberger, G., Schimpf, G. C., & Streicher, J. (2020). Social innovation
assessment? Reflections on the impacts of social innovation on society-
outcomes of a systematic literature review. European Public and Social
Innovation Review, 5(2), 1–13. https://doi.org/10.31637/epsir.20-2.1
Mongelli, L., & Rullani, F. (2017). Inequality and marginalisation: Social
innovation, social entrepreneurship and business model innovation: The
common thread of the DRUID Summer Conference 2015. Industry and
Innovation, 24(5), 446–467. https://doi.org/10.1080/13662716.2017.
1295365
Mulgan, G. (2019). Social Innovation (1st ed.). Policy Press. https://doi.org/
10.2307/j.ctvs89dd3 https://doi.org/10.1787/9789264086913-en
Nicholls, A. (2017). Creating Economic Space for Social Innovation. Oxford Uni-
versity Press.
Nicolopoulou, K., Karataş-Özkan, M., Vas, C., & Nouman, M. (2017). An
incubation perspective on social innovation: The London Hub – a social
incubator. R&D Management, 47(3), 368–384. https://doi.org/10.1111/
radm.12179
Organisation for Economic Co-operation and Development – OECD (2010).
Guidance on Sustainability Impact Assessment. OECD Publishing.
Perrini, F., Costanzo, L. A., & Karatas-Ozkan, M. (2021). Measuring impact
and creating change: A comparison of the main methods for social enter-
prises. Corporate Governance, 21(2), 237–251. https://doi.org/10.1108/
CG-02-2020-0062
Rawhouser, H., Cummings, M., & Newbert, S. L. (2019). Social Impact
Measurement: Current Approaches and Future Directions for Social Entre-
preneurship Research. Entrepreneurship Theory and Practice, 43(1), 82–115.
https://doi.org/10.1177/1042258717727718
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
31
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Rehfeld, D., Terstriep, J., Welschhoff, J., & Alijani, S. (2015). Comparative
Report on Social Innovation Framework. Simpact Project.
Rodrigo, L., & Palacios, M. (2021). What antecedent attitudes motivate
actors to commit to the ecosystem of digital social innovation? Technological
Forecasting and Social Change, 162, 120394. https://doi.org/10.1016/j.
techfore.2020.120394
Russell, S., & Williams, R. (2002). Social shaping of technology: Frame-
works, findings and implications for policy. Shaping Technology, Guiding
Policy: Concepts, Spaces and Tools. In K. H. Sørensen, & R. Williams (Eds.),
Shaping Technology, Guiding Policy: Concepts, Spaces and Tools (pp. 37–132).
Edward Elgar.
Sánchez, A., Rama, M., García, J.(2017). Bibliometric analysis of publica-
tions on wine tourism in the databases Scopus and WoS. European Research
on Management and Business Economics, 23, 8–15.
Simpact (2014). Social Innovation Evaluation Toolbox. Washington University
Law Review, 92(1), 1–71. https://doi.org/10.2796/27492
Sinnergiak (2013). Resindex: A regional index to measure social innovation. http://
www.simpact-project.eu/publications/indicators/2014_RESINDEX_
eng.pdf
Unceta, A., Castro-Spila, J., & García Fronti, J. (2016). Social innovation
indicators. Innovation, 29(2), 192–204. https://doi.org/10.1080/13511610.
2015.1127137
Unceta, A., Luna, Á., Castro, J., & Wintjes, R. (2020). Social Innovation
Regime: an integrated approach to measure social innovation. European
Planning Studies, 28(5), 906–924. https://doi.org/10.1080/09654313.2019.
1578338
von Jacobi, N., & Chiappero-Martinetti, E. (2017). Social Innovation, Indi-
viduals and Societies: An Empirical Investigation of Multi-layered Effects.
Journal of Social Entrepreneurship, 8(3), 271–301. https://doi.org/10.1080/
19420676.2017.1364288
Vasin, S. M., Gamidullaeva, L. A., & Rostovskaya, T. K. (2017). The chal-
lenge of social innovation: Approaches and key mechanisms of develop-
ment. European Research Studies Journal, 20(2), 25–45.
Weaver, M. P., & Marks, B. M. (2017). Social innovation resourcing strate-
gies and transformation pathways : a first-cut typology. In TRANSIT: EU
SSH.2013.3.2-1 Grant agreement no. 613169 [Issue 613169]. http://www.
transitsocialinnovation.eu/content/original/Book covers/Local PDFs/264
TRANSIT_WorkingPaper 11_Resourcing Strategies-pmw.pdf
32
Challenges of impact measurement in social innovation: Barriers and interventions to overcome
ISSN 1678-6971 (electronic version) • RAM, São Paulo, 23(6), eRAMD220077, 2022
https://doi.org/10.1590/1678-6971/eRAMD220077.en
Wittmayer, J. M., Backhaus, J., Avelino, F., Pel, B., Strasser, T., Kunze, I., &
Zuijderwijk, L. (2019). Narratives of change: How social innovation initia-
tives construct societal transformation. Futures, 112, 102433, https://doi.
org/10.1016/j.futures.2019.06.005
EDITORIAL BOARD
Editor-in-chief
Gilberto Perez
Associate editor
Eduardo Raupp de Vargas
Technical support
Vitória Batista Santos Silva
EDITORIAL PRODUCTION
Publishing coordination
Jéssica Dametta
Language editor
Bardo Editorial
(Irina Migliari & Andrew Benson)
Layout designer
Emap
Graphic designer
Libro
