No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
What about the issues in using social return on investment as an evaluation tool?
... Although there are other social evaluation methods (eg, social accounting and auditing and the Global Reporting Initiative), they do not embed stakeholder engagement as a requirement, which is key to understand the changes that they experience as a result of an intervention and to measure what really matters. 3,20 Also, although the rigor of application can vary (eg, lower levels inform decision makers within organizations), high levels can be achieved with the quality assurance of an external evaluation by Social Value International. 21 Another benefit is that the model can be applied before a program or activity has taken place to predict the value created by their outcomes (forecast SROI), or it can be used to measure and value the results of a program or activity that has already taken place (evaluative SROI). ...
... Twenty-five documents measured "mental health," including the indicators of reduced depression (12,14,15,22,26,39,(48)(49)(50)(51)(52)(53)(54)(55)(56), reduced anxiety (22,26,(48)(49)(50)(51)(52)(53)(54)(55)(56), reduced schizophrenia (48-56), reduced stress (19,28,26,44), carers' respite (4,44), reduced anger, alcohol use, and problem gambling (22), and suicide prevention (22,38,47). Other studies measured the generic improvement in mental health (1,2,20,23,38,44,47) and one negative outcome, that is, increased stress (19). ...
... Twenty-one studies measured "improved educational attainment" (11, 12, 13, 15, 17, 23, 24, 28, 39, 40, 41, 44, and 48-56), and 10 documents (24, 40, 48-54, and 56) measured the "reduction of school absenteeism." Twenty-one documents (1, 4, 6, 7, 9, 10, 19, 20, 21, 22, 23, 24, 28, 29, 35, 38, 39, 42, 43, 44, and 45) measured "skills acquisition" with indicators such as learning new skills and gaining personal development (6) or skills in public speaking (20). Finally, "other impacts from improved education" were measured in 35 documents and referred to those outcomes making participants more employable. ...
Background: Physical activity and sport (PAS) have been related to health and social benefits, but their monetary value remains unclear. This systematic review on the social return on investment of PAS aimed to find what are the social outcomes measured in previous PAS literature and how are these measured and valued. Methods: A systematic search was conducted on WoS, PubMed, and EconLit. Articles in English, measuring the social value of any type of PAS in monetary terms and utilizing a social return on investment framework, were included. Risk of bias was evaluated using the Drummond checklist. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed. Results: Fifty-five documents (2010–2022), from all continents except America, were included; only 8 were published in peer-reviewed journals, whereas 47 were reports. Most studies evaluated the benefits of specific programs, and 6 measured the engagement in PAS at the population level based on national or community surveys. The social outcomes identified were health (94.5%), crime (50.9%), education (83.6%), subjective well-being (89.1%), social capital (60%), and other (3.6%–23.6%). The valuation methods included willingness to pay, well-being valuation, the cost of an activity that could result in the same outcome, and cost databases associating outcomes with a monetary value. Conclusions: This study updates a previous review and widens the scope by answering the question of how social outcomes are measured and valued in previous PAS literature. Given the heterogeneity found in the application of the method, this review will inform a Delphi study to reach a Global Consensus Statement on the measurement of social value and PAS.
... Although there are other social evaluation methods (eg, social accounting and auditing and the Global Reporting Initiative), they do not embed stakeholder engagement as a requirement, which is key to understand the changes that they experience as a result of an intervention and to measure what really matters. 3,20 Also, although the rigor of application can vary (eg, lower levels inform decision makers within organizations), high levels can be achieved with the quality assurance of an external evaluation by Social Value International. 21 Another benefit is that the model can be applied before a program or activity has taken place to predict the value created by their outcomes (forecast SROI), or it can be used to measure and value the results of a program or activity that has already taken place (evaluative SROI). ...
... Twenty-five documents measured "mental health," including the indicators of reduced depression (12,14,15,22,26,39,(48)(49)(50)(51)(52)(53)(54)(55)(56), reduced anxiety (22,26,(48)(49)(50)(51)(52)(53)(54)(55)(56), reduced schizophrenia (48-56), reduced stress (19,28,26,44), carers' respite (4,44), reduced anger, alcohol use, and problem gambling (22), and suicide prevention (22,38,47). Other studies measured the generic improvement in mental health (1,2,20,23,38,44,47) and one negative outcome, that is, increased stress (19). ...
... Twenty-one studies measured "improved educational attainment" (11, 12, 13, 15, 17, 23, 24, 28, 39, 40, 41, 44, and 48-56), and 10 documents (24, 40, 48-54, and 56) measured the "reduction of school absenteeism." Twenty-one documents (1, 4, 6, 7, 9, 10, 19, 20, 21, 22, 23, 24, 28, 29, 35, 38, 39, 42, 43, 44, and 45) measured "skills acquisition" with indicators such as learning new skills and gaining personal development (6) or skills in public speaking (20). Finally, "other impacts from improved education" were measured in 35 documents and referred to those outcomes making participants more employable. ...
Background : Physical activity and sport (PAS) have been related to health and social benefits, but their monetary value remains unclear. This systematic review on the social return on investment of PAS aimed to find what are the social outcomes measured in previous PAS literature and how are these measured and valued. Methods : A systematic search was conducted on WoS, PubMed, and EconLit. Articles in English, measuring the social value of any type of PAS in monetary terms and utilizing a social return on investment framework, were included. Risk of bias was evaluated using the Drummond checklist. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed. Results : Fifty-five documents (2010–2022), from all continents except America, were included; only 8 were published in peer-reviewed journals, whereas 47 were reports. Most studies evaluated the benefits of specific programs, and 6 measured the engagement in PAS at the population level based on national or community surveys. The social outcomes identified were health (94.5%), crime (50.9%), education (83.6%), subjective well-being (89.1%), social capital (60%), and other (3.6%–23.6%). The valuation methods included willingness to pay, well-being valuation, the cost of an activity that could result in the same outcome, and cost databases associating outcomes with a monetary value. Conclusions : This study updates a previous review and widens the scope by answering the question of how social outcomes are measured and valued in previous PAS literature. Given the heterogeneity found in the application of the method, this review will inform a Delphi study to reach a Global Consensus Statement on the measurement of social value and PAS.
... Indeed, financial proxies from a different country may not reflect the value of outcomes for the actual beneficiaries. One approach for future SROI studies to improve the robustness of financial valuations may be to use the Delphi method, which triangulate valuations of beneficiaries with a panel of "experts" [50]. ...
... A SROI analysis is a multifaceted evaluation tool, involving mapping, reporting and monetising outcomes to establish the impact of an intervention in relation to its costs. Making robust financial valuation when monetising outcomes is often cited as one of the main challenges of the method [16,50,53]. However, robust financial valuation can only partly improve the robustness of the SROI if outcomes are not properly measured and if deadweight is not robustly established using a proper study design. ...
... Indeed, none of the included studies used a control group, only one used a comparison group, less than 20% used a before-and-after design and most studies used opinions and assumptions of stakeholders to estimate deadweight. To accurately determine the impact of interventions through SROI studies, Banke-Thomas et al. [13] emphasised the importance of using objective study designs such as case-control, before-and-after comparisons or a stepped-wedged cluster randomised trial [50]. Given organisational, financial and ethical constraints, non-academic organisations face significant limitations in adopting such study designs. ...
Background:
Physical Activity and Sport (PAS) interventions can reduce the social and economic burden of non-communicable diseases and improve the wellbeing of the population. Social return on investment (SROI) has the capacity to measure broader socio-economic outcomes in a singular monetary ratio to help identify the most impactful and cost-beneficial intervention. This review aimed to systematically identify and review studies using the SROI method within the field of PAS and assess their quality.
Methods:
Peer-reviewed and grey literature SROI studies were identified through a systematic search of six databases. Two reviewers independently assessed the identified studies to determine eligibility. Study quality was assessed using the Krelv et al. 12-point framework. For each included study, information was extracted and classified into summary tables. Extracted information included study and participant characteristics, type of outcomes and SROI ratio. The PRISMA guidelines were followed.
Results:
Seventeen studies published between 2010 and 2018 met the inclusion criteria. Most studies (94%) were non-peer reviewed publicly available reports, primarily conducted in the UK (76%), by private consulting firms (41%) and included all types of stakeholders (76%). PAS interventions included Primary prevention (47%), Sport for development (29%), Secondary and tertiary prevention (18%) and High-performance sport (6%). SROI ratios, which report the social value created in relation to the cost of an intervention, vary between 3:1 and 124:1 for the high-quality studies.
Conclusions:
The SROI framework can be a useful tool to inform policy-making relating to PAS investment as it can account for the wide societal benefits of PAS. The quality of studies in the field would benefit from the employment of an impact map (or logic model), reporting negative outcomes and using objective study designs. The application of the SROI method in the PAS field is relatively recent, and thus further research would be beneficial to promote its potential for policy-making bodies in the field.
... While an evaluative SROI analysis based on real-world evidence may provide a more realistic foundation for decision-making, this forecast-type SROI analysis may be used to identify specific and relevant data to monitor in a future evaluative SROI analysis. 16,39 Second, using a one-year time horizon may not reveal long-term cost-effective and/or cost-saving interventions. Alternatively, a long-term SROI analysis may provide a better basis for decision-making (eg, prioritizing smoking cessation programs with a large initial investment that may be cost-effective and/or cost-saving after 10 years of their implementation). ...
Purpose:
To define a set of proposals that would improve the current management of chronic obstructive pulmonary disease (COPD) within the Spanish National Healthcare System (SNHS) from a comprehensive multidisciplinary perspective and to assess the impact of its implementation from clinical, healthcare, economic, and social perspectives.
Patients and methods:
A group of 20 stakeholders related to COPD (healthcare professionals, patients, and informal caregivers, among others) participated in an online Delphi process to agree on a set of 15 proposals that would improve the current management of COPD within the SNHS in four areas: diagnosis, risk stratification, management of exacerbations, and management of stable COPD. A one-year forecast-type social return on investment (SROI) analysis was used to estimate the impact that implementing the set of proposals would have in relation to the investment required. A sensitivity analysis was used to test the strength of the model when varying assumption-based data-points.
Results:
The hypothetical implementation of the complete set of 15 proposals would require a €668 million investment and would generate a €2079 million social impact concerning savings for the SNHS and quality of life improvements for patients and their informal caregivers, among others. Accordingly, for every euro invested in the set of proposals, a social return of €3.11 would be generated (€2.71 in the worst-case scenario and €3.62 in the best-case scenario) of both tangible (32.56%) and intangible nature (67.44%).
Conclusion:
Altogether, implementing this set of 15 proposals would generate a positive social impact, threefold the required investment. The results may inform decisions relative to healthcare policy and practice regarding COPD management within the SNHS, further contributing to reduce the large burden of COPD.
... In contrast, an evaluative SROI analysis based on real-world evidence may provide a more robust foundation for decisionmaking. Nevertheless, the forecast-type SROI analysis has been recommended to identify relevant data to be included in the process of an evaluative SROI analysis [20,61]. Moreover, within the SROI framework, a certain degree of bias associated with the configuration of the MEC, the financial proxies used to account for intangible returns, or the assumptions made for missing or unclear data points may be introduced. ...
Background
Haemophilia A (HA) has been associated with poor health-related quality of life and a large economic burden, accentuated by severity, arthropathy, and inhibitors. To meet global standards of care, the management of HA should align with the principles of care outlined by the World Federation of Haemophilia. The aims of the present study were to establish a set of proposals to improve HA management within the Spanish National Health System (SNHS) and to estimate the impact its hypothetical implementation would generate from a clinical, healthcare, economic, and social perspective.
Methods
A multidisciplinary group of experts agreed on a set of 15 proposals to improve HA management within the SNHS. Thereafter, a forecast-type Social Return on Investment analysis was carried out to estimate the impact of implementing this set of proposals within the SNHS over a one-year timeframe, in relation to the required investment.
Results
This study estimated that the implementation of the complete set of 15 proposals would require a total investment of 2.34 M€ and have a total impact of 14.60 M€. Accordingly, every euro invested in the complete set of 15 proposals would yield a social return of €6.23 (€3.37 in the worst-case scenario and €9.69 in the best-case scenario) of both tangible and intangible nature in similar proportions (45.71 and 54.29%, respectively).
Conclusions
These results can be used to inform policy and practice such that interventions that may potentially improve current public health challenges associated with the management of HA may be implemented.
... Campbells Law gilt selbst im Falle von per se valide fundierten Sozialindikatoren, potenziert sich aber wenn die Messung von Impacts auf dem bloßen Fundament von Erfassungen der Einschätzungen von Praktiker*innen oder anderen Stakeholdern erfolgt. Die verbreitete Tendenz zu dieserTrivialform der Impact-Messung ist von einer breiten Reihe von Vertreter*innen des SROI-Ansatzes selbst kritisiert worden (Banke- Thomas et al., 2015;Banke-Thomas, 2018;Gair, 2009;Fujiwara, 2015), die auf die Notwenigkeit verweisen auf verlässliche Outcome-Daten und reliable Befunde zu rekurrieren. Während die Kritik an der Fundierung von Wirkungen auf der Basis querschnittlich erhobener Aussagen von Betroffenen und ,Stakeholder< überzeugt, ist die Zuverlässigkeit der Wirkungsaussagen von SROI auch dann nicht sichergestellt, wenn die Analysen den offensichtlichen Fehlschluss vermeiden, Wirkungsaskriptionen von Programmbeteiligten als reale Zustandsveränderungen zu interpretieren. ...
... An issue that has been reported in the literature is stakeholder creep where diverse stakeholders are deemed beneficiaries and included in the SROI analysis. A robust stakeholder analysis that categorizes stakeholders into one of funders, implementers, promoters, or beneficiaries can address this issue (Banke-Thomas, 2018), while an analysis that includes direct and indirect can always be conducted at two levels to include only direct beneficiaries as well as combine direct and indirect beneficiaries. It is important to resist the temptation of including funders, promoters, and implementers in SROI analyses. ...
Compared with traditional approaches such as cost-effectiveness, cost–utility, and cost–benefit analyses, the social return on investment methodology has the unique capacity to account for the broader social value and value for money of interventions while capturing perspectives of multiple stakeholders and relating this to the cost of implementing the intervention in a singular ratio. This case study describes the comprehensive assessment of and assesses the social impact and value for money of an emergency obstetric care training intervention for health care providers implemented in Kenya. Critical insights are shared into the practicalities of using the methodology within the maternal and newborn health area. Insights will be relevant to any other researcher who is keen to apply the methodology in any area of health.
... An issue that has been reported in the literature is stakeholder creep where diverse stakeholders are deemed beneficiaries and included in the SROI analysis. A robust stakeholder analysis that categorizes stakeholders into one of funders, implementers, promoters, or beneficiaries can address this issue (Banke-Thomas, 2018), while an analysis that includes direct and indirect can always be conducted at two levels to include only direct beneficiaries as well as combine direct and indirect beneficiaries. It is important to resist the temptation of including funders, promoters, and implementers in SROI analyses. ...
Compared with traditional approaches such as cost-effectiveness, cost–utility, and cost–benefit analyses, the social return on investment methodology has the unique capacity to account for the broader social value and value for money of interventions while capturing perspectives of multiple stakeholders and relating this to the cost of implementing the intervention in a singular ratio. This case study describes the comprehensive assessment of and assesses the social impact and value for money of an emergency obstetric care training intervention for health care providers implemented in Kenya. Critical insights are shared into the practicalities of using the methodology within the maternal and newborn health area. Insights will be relevant to any other researcher who is keen to apply the methodology in any area of health.
... However, the sensitivity of the value game is also a limitation. 44 Data for financial valuation were mostly obtained from the grey literature. In this study, discussion with in-country experts who had the local knowledge required to decipher the sensibility of the financial proxies retrieved was triangulated with financial valuations obtained from the literature, before incorporating these into the SROI analysis. ...
Introduction Emergency obstetric care (EmOC) training is considered a key strategy for reducing maternal and perinatal morbidity and mortality. Although generally considered effective, there is minimal evidence on the broader social impact and/or value-for-money (VfM). This study assessed the social impact and VfM of EmOC training in Kenya using social return on investment (SROI) methodology.
Methods Mixed-methods approach was used, including interviews (n=21), focus group discussions (n=18) incorporating a value game, secondary data analysis and literature review, to obtain all relevant data for the SROI analysis. Findings were incorporated into the impact map and used to estimate the SROI ratio. Sensitivity analyses were done to test assumptions.
Results Trained healthcare providers, women and their babies who received care from those providers were identified as primary beneficiaries. EmOC training led to improved knowledge and skills and improved attitudes towards patients. However, increased workload was reported as a negative outcome by some healthcare providers. Women who received care expected and experienced positive outcomes including reduced maternal and newborn morbidity and mortality. After accounting for external influences, the total social impact for 93 5-day EmOC training workshops over a 1-year period was valued at I745 000 for 2965 healthcare providers trained. The cost per trained healthcare provider per day was I$50.23 and SROI ratio was 12.74:1. Based on multiple one-way sensitivity analyses, EmOC training guaranteed VfM in all scenarios except when trainers were paid consultancy fees and the least amount of training outcomes occurred.
Conclusion EmOC training workshops are a worthwhile investment. The implementation approach influences how much VfM is achieved. The use of volunteer facilitators, particularly those based locally, to deliver EmOC training is a critical driver in increasing social impact and achieving VfM for investments made.
Purpose
The study aims to explore the complexities and challenges of measuring social impact, with a particular emphasis on the practical application of the social return on investment (SROI) methodology. By investigating social enterprises in Georgia, the study seeks to understand how social value is quantified and the implications of such measurements for policy and practice.
Design/methodology/approach
This study uses a mixed-methods approach, centered on the SROI methodology, to measure the social impact of 11 social enterprises in the Republic of Georgia. It incorporates qualitative interviews and quantitative financial analysis, engaging stakeholders from enterprises, beneficiaries and local authorities. The methodology adapts SROI to the Georgian context, addressing challenges such as data scarcity and cultural sensitivity.
Findings
Findings reveal significant challenges in social impact measurement, including the complexity of quantifying diverse impacts, lack of standardized methodologies and resource constraints. The application of SROI in Georgia demonstrates its flexibility and the critical role of stakeholder engagement, yet underscores the need for context-specific adaptations and rigorous data collection.
Research limitations/implications
The study is limited by its geographic focus on Georgia, which may affect the generalizability of findings. In addition, the reliance on stakeholder-reported data introduces potential biases. These limitations highlight the necessity for broader, cross-cultural studies and methodological advancements in social impact measurement.
Practical implications
The study offers practical insights for organizations implementing SROI, emphasizing the importance of stakeholder engagement, cultural sensitivity and adapting methodologies to local contexts. It suggests strategies for overcoming data limitations and enhancing the credibility of social impact assessments.
Social implications
The research underscores the transformative potential of social enterprises in addressing societal challenges. By quantifying social impact, organizations can better articulate their contributions to societal well-being, informing policy decisions and fostering a culture that values social over mere economic returns.
Originality/value
This study contributes to the literature on social impact measurement by detailing the application of SROI in a novel context – Georgia. It addresses a significant gap in understanding how social impact can be measured in transitional economies and offers valuable insights into the methodological and practical challenges involved.
Background
Physical activity and sport (PAS) have been related to many health outcomes and social benefits. The main aim of this research is to build a Social Return on Investment (SROI) model of PAS based on experts’ opinion to clarify the domains of impact and how to measure and value them.
Methods and analysis
A Delphi method will be employed with a systematic review on the SROI framework applied to PAS and initial interviews with experts informing the design of the Delphi survey statements. Three iterative rounds of communication with the expert panel will be carried out. Participants will indicate their level of agreement with each statement on a five-point Likert scale. During the second and third iterative rounds, experts will reappraise the statements and will be provided with a summary of the group responses from the panel. A statement will have reached consensus if ≥70% of the panel agree/strongly agree or disagree/strongly disagree after round 3. Finally, group meetings (3–4 experts) will be conducted to ask about the measurement and valuation methods for each domain.
Discussion
The final goal of this project will result in the design of a toolkit for organizations, professionals, and policymakers on how to measure the social benefits of PAS.
Objective:
The aim of this study was to reach a consensus on a set of proposals to optimise the disease management of Multiple myeloma (MM) within the Spanish National Health System (SNHS) and to apply the Social Return on Investment (SROI) method to estimate their social impact.
Methods:
A Multidisciplinary Working Team (MWT) including MM main stakeholders was organised. A survey was administered to gather information from patients regarding the impact of MM on different life domains. A forecast-type SROI analysis, with a 1-year timeframe, was applied.
Results:
Fifteen proposals were selected, to optimise MM management, including actions for early diagnosis, psychological support, improvement of information for patients and quick access to palliative care, among others. The implementation of these proposals would benefit patients, their informal caregivers and the SNHS. The investment required would amount to 10.32 million euros with a social return of 43.31 million euros: 4.2 euros for each euro invested. According to the sensitivity analysis, this ratio could range from 3.38 to 5.20 euros from the worst to the best-case scenario.
Conclusions:
The current management of MM could be optimised by implementing a set of proposals that would most likely result in an overall positive social return.
In einer jüngst veröffentlichten Analyse zum Social Return on Investment (SROI) des Mentoren-Programms ‘Balu und Du’ haben neun Mentor*innen angegeben, mit – als ‚Moglis‘ bezeichneten – Grundschulkindern Brettspiele zu spielen. Brettspiele, so argumentiert die Studie sinngemäß, seien geeignet, die Konzentrationsfähigkeit zu erhöhen. Diese Fähigkeit sei nun ohne Zweifel für den Schulerfolg wichtig, Schulerfolg sei eine Bedingung für beruflichen Erfolg und wer beruflich erfolgreich sei, zahle Steuern statt Sozialleistungen zu konsumieren.
Social return on investment (SROI) is an approach built
on cost benefit analysis and is used in the evaluation
of projects with social benefits, as an alternative to
cost benefit analysis and theory-based evaluation. This
paper provides an analysis of SROI as an evaluation
tool compared to theory-based evaluation, based on
an evaluation of a community based mental health
rehabilitation program in regional South Australia. The
paper describes the process of constructing a SROI
impact map and identifies the issues at each stage.
Establishing the resources used, the activities and the
outputs appears relatively straightforward. Arriving at
an agreed theory of change is much more contested,
even when using a high level of involvement of the
service beneficiaries. The single greatest difficulty is to
find the indicators and the financial proxies to value the
outcomes. Outcomes such as improved wellbeing are
difficult to value. It is particularly difficult to establish
the level of outcomes immediately after or during an
intervention. The paper concludes with an analysis of
landscapes where SROI is unlikely to fit.
Responding to increasing demands to demonstrate value-for-money (VfM) for maternal and newborn health interventions, and in the absence of VfM analysis in peer-reviewed literature, this paper reviews VfM components and methods, critiques their applicability, strengths and weakness and proposes how VfM assessments can be improved. VfM comprises four components: economy, efficiency, effectiveness and cost-effectiveness. Both ‘economy’ and ‘efficiency’ can be assessed with detailed cost analysis utilising costs obtained from programme accounting data or generic cost databases. Before-and-after studies, case–control studies or randomised controlled trials can be used to assess ‘effectiveness’. To assess ‘cost-effectiveness’, cost-effectiveness analysis (CEA), cost-utility analysis (CUA), cost-benefit analysis (CBA) or social return on investment (SROI) analysis are applicable. Generally, costs can be obtained from programme accounting data or existing generic cost databases. As such ‘economy’ and ‘efficiency’ are relatively easy to assess. However, ‘effectiveness’ and ‘cost-effectiveness’ which require establishment of the counterfactual are more difficult to ascertain. Either a combination of CEA or CUA with tools for assessing other VfM components, or the independent use of CBA or SROI are alternative approaches proposed to strengthen VfM assessments. Cross-cutting themes such as equity, sustainability, scalability and cultural acceptability should also be assessed, as they provide critical contextual information for interpreting VfM assessments. To select an assessment approach, consideration should be given to the purpose, data availability, stakeholders requiring the findings and perspectives of programme beneficiaries. Implementers and researchers should work together to improve the quality of assessments. Standardisation around definitions, methodology and effectiveness measures to be assessed would help.
Increased scarcity of public resources has led to a concomitant drive to account for value-for-money of interventions. Traditionally, cost-effectiveness, cost-utility and cost-benefit analyses have been used to assess value-for-money of public health interventions. The social return on investment (SROI) methodology has capacity to measure broader socio-economic outcomes, analysing and computing views of multiple stakeholders in a singular monetary ratio. This review provides an overview of SROI application in public health, explores lessons learnt from previous studies and makes recommendations for future SROI application in public health.
A systematic review of peer-reviewed and grey literature to identify SROI studies published between January 1996 and December 2014 was conducted. All articles describing conduct of public health SROI studies and which reported a SROI ratio were included. An existing 12-point framework was used to assess study quality. Data were extracted using pre-developed codes: SROI type, type of commissioning organisation, study country, public health area in which SROI was conducted, stakeholders included in study, discount rate used, SROI ratio obtained, time horizon of analysis and reported lessons learnt.
40 SROI studies, of varying quality, including 33 from high-income countries and 7 from low middle-income countries, met the inclusion criteria. SROI application increased since its first use in 2005 until 2011, declining afterwards. SROI has been applied across different public health areas including health promotion (12 studies), mental health (11), sexual and reproductive health (6), child health (4), nutrition (3), healthcare management (2), health education and environmental health (1 each). Qualitative and quantitative methods have been used to gather information for public health SROI studies. However, there remains a lack of consensus on who to include as beneficiaries, how to account for counterfactual and appropriate study-time horizon. Reported SROI ratios vary widely (1.1:1 to 65:1).
SROI can be applied across healthcare settings. Best practices such as analysis involving only beneficiaries (not all stakeholders), providing justification for discount rates used in models, using purchasing power parity equivalents for monetary valuations and incorporating objective designs such as case-control or before-and-after designs for accounting for outcomes will improve robustness of public health SROI studies.
The stepped wedge cluster randomised trial is a novel research study design that is increasingly being used in the evaluation of service delivery type interventions. The design involves random and sequential crossover of clusters from control to intervention until all clusters are exposed.It is a pragmatic study design which can reconcile the need for robust evaluations with political or logistical constraints. While not exclusively for the evaluation of service delivery interventions, it is particularly suited to evaluations that do not rely on individual patient recruitment. As in all cluster trials, stepped wedge trials with individual recruitment and without concealment of allocation (or blinding of the intervention) are at risk of selection biases.In a stepped wedge design more clusters are exposed to the intervention towards the end of the study than in its early stages. This implies that the effect of the intervention might be confounded with any underlying temporal trend. A result that initially might seem suggestive of an effect of the intervention may therefore transpire to be the result of a positive underlying temporal trend. Sample size calculations and analysis must make allowance for both the clustered nature of the design and the confounding effect of time.The stepped wedge cluster randomised trial is an alternative to traditional parallel cluster studies, in which the intervention is delivered in only half the clusters with the remainder functioning as controls. When the clusters are relatively homogeneous (that is, the intra-cluster correlation is small), parallel studies tend to deliver better statistical performance than a stepped wedge trial. However, if substantial cluster-level effects are present (that is, larger intra-cluster correlations) or the clusters are large, the stepped wedge design will be more powerful than a parallel design, even one in which the intervention is preceded by a period of baseline control observations.
Interventions and activities that influence health are often concerned with intangible outcomes that are difficult to value despite their potential significance. Social Return on Investment is an evaluation framework that explores all aspects of change and expresses these in comparable terms. It combines qualitative narratives and quantitative measurements with a financial approach to enable outcomes that can otherwise be overlooked or undervalued to be incorporated appropriately. This article presents Social Return on Investment as an effective tool for supporting the development of a holistic appreciation of how interventions impact on the health and well-being of individuals, communities and societies.
© The Author(s) 2014.
The Delphi method has proven a popular tool in information systems research for identifying and prioritizing issues for managerial decision-making. However, many past studies have not adopted a systematic approach to conduct a Delphi study. This article provides rigorous guidelines for the process of selecting appropriate experts for the study and gives detailed principles for making design choices during the process that ensure a valid study. A detailed example of a study to identify key factors affecting the diffusion of e-commerce in Sub-Saharan Africa illustrates the design choices that may be involved. We conclude with suggestions for theoretical applications.
This paper outlines recent advances in the methods of cost-benefit analysis (CBA). Economic evaluations in healthcare can be criticised for, amongst other things, the inappropriate use of incremental cost-effectiveness ratios and the reporting of benefits in terms of cost savings, such as treatment costs averted. Many such economic evaluations are, according to the ‘scientific’ definition, CBAs. The ‘balance-sheet’ (or opportunity cost) approach is a form of CBA which can be used to identify who bears the costs and who reaps the benefits from any change. Whilst the next stage in a CBA, as defined in health economics, would require that all costs and benefits be valued in monetary terms, the balance-sheet approach, however, advocates that available monetary values can be augmented by other measures of cost and benefit. As such, this approach, which has a theoretical basis, is proposed as a practical prescription for CBA and highlights the notion that unquantified benefits are important and can be included within CBAs even when monetarisation is not possible.
Recent methodological developments in monetary valuation for use in CBA are the development of the technique of willingness to pay, the use of conjoint analysis (CA) to elicit willingness-to-pay (WTP) values and advances in the debate on the inclusion of production gains in CBAs. Whilst acknowledging that there have been developments in each of these areas, it is claimed there has also been progress in using CBA as a framework for evaluation, as reflected by the balance-sheet approach.
The paper concludes by stating that almost all types of economic evaluation have an element of the ‘cost-benefit’ approach in them. The important issue is to focus on the policy question to be addressed and to outline the relevant costs and benefits in a manner which assists the evaluation of welfare changes resulting from changes in healthcare delivery. The focus should not be on moulding a question to fit a hybrid definition of an analytical technique.
Social enterprises are being promoted as responsive and innovative way to deliver public services. As part of this promotion, these organizations are being required to demonstrate the social and economic value they generate. Social return on investment (SROI) is a performance measurement tool currently being encouraged to capture this impact. This paper draws on survey and interview data to analyse how SROI is used and understood in health and social care settings. It indicates that despite being accepted as an internationally recognized measurement tool for social enterprise, SROI is underused and undervalued due to practical and ideological barriers.
Social Return on Investment (SROI): State-of-the-art and perspectives
- G Krlev
- R Münscher
- K Mülbert
A guide to social return on investment
- J Nicholls
- E Lawlor
- E Neitzert
- T Goodspeed
- S Cupitt
- S Durie
- J Inglis
- K Leathem
- T Lumley
Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2012). A guide to social return on
investment (S. Cupitt, S. Durie, J. Inglis, K. Leathem, T. Lumley, & R. Piper, Eds.) (2nd
ed.). Liverpool, UK: The Cabinet Office.
Social Return on Investment (SROI): State-ofthe-art and perspectives
- G Krlev
- R Münscher
- K Mülbert
Krlev, G., Münscher, R., & Mülbert, K. (2013). Social Return on Investment (SROI): State-ofthe-art and perspectives. Heidelberg, Germany: Heidelberg University.
Social Return on Investment (SROI) and performance measurement: The opportunities and barriers for social enterprises in health and social care
- R Millar
- K Hall
Millar, R., & Hall, K. (2013). Social Return on Investment (SROI) and performance measurement: The opportunities and barriers for social enterprises in health and social care. Public
Management Review, 15, 923-941.
The road to dignity by 2030: Ending poverty, transforming all lives and protecting the planet: Synthesis report of the Secretary-General on the post-2015 sustainable development agenda
United Nations General Assembly. (2014). The road to dignity by 2030: Ending poverty, transforming all lives and protecting the planet: Synthesis report of the Secretary-General on
the post-2015 sustainable development agenda. In 69th Session of the General Assembly.
New York, NY: United Nations.