Access to this full-text is provided by Springer Nature.
Content available from BMC Health Services Research
This content is subject to copyright. Terms and conditions apply.
Thusinietal. BMC Health Services Research (2022) 22:1492
https://doi.org/10.1186/s12913-022-08832-3
RESEARCH
© The Author(s) 2022, corrected publication 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0
International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you
give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes
were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated
otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To
view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver
(http:// creat iveco mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a
credit line to the data.
Open Access
The development oftheconcept
ofreturn-on-investment fromlarge-scale quality
improvement programmes inhealthcare:
anintegrative systematic literature review
S’thembile Thusini1*, Maria Milenova1, Noushig Nahabedian2, Barbara Grey2, Tayana Soukup1,
Kia‑Chong Chua1 and Claire Henderson1
Abstract
Background: Return on Investment (ROI) is increasingly being used to evaluate financial benefits from healthcare
Quality Improvement (QI). ROI is traditionally used to evaluate investment performance in the commercial field. Little
is known about ROI in healthcare. The aim of this systematic review was to analyse and develop ROI as a concept and
develop a ROI conceptual framework for large‑scale healthcare QI programmes.
Methods: We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google Scholar
using ROI or returns‑on‑investment concepts (e.g., cost–benefit, cost‑effectiveness, value). We combined this
terms with healthcare and QI. Included articles discussed at least three organisational QI benefits, including financial
or patient benefits. We synthesised the different ways in which ROI or return‑on‑investment concepts were used and
discussed by the QI literature; first the economically focused, then the non‑economically focused QI literature. We
then integrated these literatures to summarise their combined views.
Results: We retrieved 10 428 articles. One hundred and two (102) articles were selected for full text screening. Of
these 34 were excluded and 68 included. The included articles were QI economic, effectiveness, process, and impact
evaluations as well as reports and conceptual literature. Fifteen of 68 articles were directly focused on QI programme
economic outcomes. Of these, only four focused on ROI. ROI related concepts in this group included cost‑effective‑
ness, cost–benefit, ROI, cost‑saving, cost‑reduction, and cost‑avoidance. The remaining articles mainly mentioned effi‑
ciency, productivity, value, or benefits. Financial outcomes were not the main goal of QI programmes. We found that
the ROI concept in healthcare QI aligned with the concepts of value and benefit, both monetary and non‑monetary.
Conclusion: Our analysis of the reviewed literature indicates that ROI in QI is conceptualised as value or benefit as
demonstrated through a combination of significant outcomes for one or more stakeholders in healthcare organisa‑
tions. As such, organisations at different developmental stages can deduce benefits that are relevant and legitimate as
per their contextual needs.
*Correspondence: s’thembile.thusini@kcl.ac.uk
1 King’s College London, London, UK
Full list of author information is available at the end of the article
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 2 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
Trial registration: Review registration: PROSPERO;
CRD42021236948.
Keywords: ROI, Quality Improvement, Value, Benefit,
Mental health
Introduction
Quality Improvement (QI) programmes are being
increasingly used to improve care quality in healthcare
organisations [1]. QI is defined as a methodical approach
to making improvements in a number of healthcare
service aspects [1–3]. In particular, large-scale QI pro-
grammes are used to effect organisational level outcomes
e.g., financial, and patient outcomes or even health sys-
tem level outcomes e.g., population health [4, 5]. Large-
scale QI combines various strategic elements into a
coherent improvement process to improve safety, qual-
ity, capability, and capacity of organisations [4, 5]. Some
QI programmes are delivered through local, national, or
international collaboratives. A QI collaborative (QIC)
combines multidisciplinary teams from different organi-
sations to test solutions, and share learning in a specific
clinical or operational area [6]. Whatever the type, QI
programmes can have significant cost implications [7].
Investing in a QI programme may redirect money
from other healthcare initiatives. In economic terms, this
is called an “opportunity cost”, roughly meaning a lost
opportunity for alternative investment [8]. For this rea-
son, healthcare leaders must justify investments made
or proposed for particular programmes. is is called
makinga QI business case. Return on Investment (ROI)
is one such justification tool, used to decide how best to
allocate limited healthcare funds [9, 10]. Investment allo-
cation decisions have ethical, moral, political, and equity
implications [11, 12]. us, the need to understand the
meaning of ROI in the context of healthcare QI.
ROI is a financial tool that forecasts financial returns or
profit from an investment [13, 14]. e forecasting pro-
cess is called ROI analysis. is uses a methodology to
convert (monetise) costs and benefits into ROI [15–17].
ROI is reported as metric (percentage or a ratio), e.g.,
ROI = 1:1 means a 100% return was made. ROI is one
of many financialmetrics used to judge efficiency of an
investment [32]. ROI can be viewed independently, in
comparison to other programmes, or against the coun-
terfactual (doing nothing) [18]. In healthcare, ROI has
been used to evaluate financial value of a programme
postimplementation [16].ROI has also been used com-
mercially as an economic performance measure for meet-
ing product quality specifications [19]. ROI is sometimes
used as a performance management tool, that is to ensure
that organisations achieve their desired strategic goals
[20]. ese traditional definitions and uses of ROI are not
disputed here. However, as ROI moved from commerceto
healthcare frontlines, it became more than a metric. It
became a concept of returns or gains from an investment.
ROI’s introduction into healthcare has caused concern
[11, 21]. As well as the rationale for ROI being to justify
investment business cases [22], the language also used is
to “defend” against disinvestment [11, 23]. In many indus-
tries, including healthcare, there are several ROI technical
and philosophical challenges [21, 24–28]. e major con-
cerns are ROI’s de-emphasizing of wider organisational
benefits. Modifications of ROI methodology have been
attempted, for example detailing non-monetisable pro-
gramme benefits as additional (not primary) benefits [9,
16]. is was in recognition that only a small fraction of
QI benefits are actually monetisable [9]. However, there is
stilla general belief that only monetisable benefits should
be seen as ROI [9, 16, 29]. is is in-spite of the recog-
nition that non-monetisable benefits are highly valued by
most organisations [9, 16]. is has created scepticism as
to the extent of theinfluence of non-monetisable benefits
on investment decision-making [11, 30, 31].
Unsurprisingly, some deem the current ROI approach
as aesthetic and synthetic [23],an insincere “placebo” and
an oversimplification[27,32]. As such, some industries
appear to have rebranded ROI. In marketing and com-
mercial service industries there is return-on-quality [33],
and value-on-investment [34]. In healthcare, Leggat [35],
called for a return-to-care, and Fischer & Duncan [36], a
return-to-value. Healthcare researchers have also been
slow to embrace ROI. Currently, many published QI-ROI
studies involve small projects, often in health promotion,
public health, or back-office services like laboratories
(e.g., [21, 37–40]. is calls for an understanding and rec-
onciliation of healthcare views on ROI. is endeavour
must be based on a logical assessment of ROI as a con-
cept, not as a metric. Further, this must be driven by a
logic of contextual appropriateness [41].
Studying concepts such as ROI in context invites the
understanding of institutional logics. Logics are socially
constructed sets of assumptions, values, and beliefs that
are used to ascribe meaning, as well as frame reasoning
and legitimise choices [42]. As such, they reflect embed-
ded cultures. Healthcare is a complex social environ-
ment, filled with sub-cultures [43]. us here, complexity
constitutes multiple actions and interactions of not only
humans, but technologies, processes, and systems [44,
45]. Healthcare has various stakeholders or groups and
individuals that affect and are affected by healthcare [46].
ese stakeholders have multiple, at times conflicting
objectives and values [47]. erefore, contextual interac-
tions also entail multiple embedded theories, cognitive
or symbolic systems [41, 47, 48]. For this reason, QI in
healthcare is a complex intervention, with varied emer-
gent and unpredictable outcomes [49].
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 3 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
Healthcare is at the juncture of many logics, primar-
ily scientific, clinical, social, and economic logics [41,
47, 48]. Medical professionals may use science logic by
emphasising a curing by focusing on evidence-based
medicine, and or a care logic by focusing on interper-
sonal aspects. Managers may use an economic logic and
focus on competition, markets, and financial outcomes.
Further,some may use societal logics and focus on popu-
lation health and socio-economic outcomes [41, 47, 48].
e presence of multiple logics explains the multiple
ways used to define healthcare quality [48]. For example,
some may describe quality as that which save costs; eco-
nomic logic [50], is evidence based; science logic, or pri-
oritises positive patient experience; care logic [51].
In the current study, the interest is the conceptualisa-
tion of ROI as a concept that is meaningful for health-
care stakeholders, particularly healthcare leaders as
decision-makers. Concepts are mental abstractions
which package complex meaning [52], and must be
unpacked (or analysed) for effective application [53]. In
this endeavour, concept analysis must be part of devel-
opment of testable and practical theories [54]. rough
concept analysis, scholars can produce evidence of
their best estimate of the ‘probable truth’ about con-
cepts [55]. is is a complex entangled task of concept
analysis and development. In modern philosophy such
as Critical Realism [56], this undertaking assumes that
concepts are contextual and changeable [54, 55]. Fur-
ther, this presupposes moderate philosophies about the
nature (Ontology) of concepts [54].
Moderate philosophies are different from traditional
philosophies where the ‘truth’ is seen as absolute and or
residing on one end of the spectrum. In Realism, reality
exists regardless of human perception, whilst Relativism
views reality as based on human perception andsocially
constructed. Lack of clarity about concepts can lead to
poor communication, poor application in research and in
practice [52, 57]. As healthcare organisations are complex
dynamic contexts, modern philosophy insights could sup-
port relations between QI implementers and investors. A
scientific study of ROI as a concept grounded on amoder-
ate philosophy may help engage QI researchers, improve
practical application for practitioners, and improve com-
munication amongst improvement stakeholders.
Aims
e aim of this study was to learn how the concept of
a return-on-investment for healthcare large-scale QI
is understood, and how this differs from related con-
cepts. We first analysed, then developed the large-scale
QI-ROI concept for healthcare based on the systematic
literaturereview. We thenproposed a framework for
analysis of return-on-investment from QI programmes.
Methods
is paper is part of a larger integrative systematic liter-
aturereview on the conceptualisation of ROI in health-
care QI. Our review is registered with PROSPERO,
CRD42021236948. A link to our PRISMA reporting
checklist [58] can be found in the supplementary files.
We followed review guidance by Whittemore and Knafl
[59] and conceptual analysis and development by Hupcey
and Penrod [55] and Jabareen [53]. is led to 8 sepa-
rate review stages. Stage 1; clarifying research question,
involved background reading as discussed in our protocol
on PROSPERO. e remainder of the stages are reported
here. Stages 2–3 involved searching and selecting litera-
ture. In stage 4 we assessed the quality of research stud-
ies, stages 5–8 are reported in the synthesis, analysis, and
results sections below.
Search strategy
e identification of suitable search terms was an itera-
tive processes. To compile a list of ROI-like terms, we
referred to the National Institute of Health and Care
Excellence (NICE) ROI guide. NICE [60] views ROI as
a term for various economic evaluation tools and pro-
cesses used to evaluate value-for-money of healthcare
programmes. Economic evaluation is the comparative
analysis of alternative courses of action in relation to both
their costs and consequences [61]. Economic evaluation
methods include cost–benefit analysis (CBA), cost-effec-
tiveness analysis (CEA), and cost consequence analysis
(CCA). Search terms were also derived from background
reading in healthcare and commercial literature. e final
search terms were in three categories (Table1): (i) con-
text, (ii) QI methods, and (iii) QI outcomes. Category 2
terms were the most frequently mentioned QI methods
in literature. Category 3 terms denote some form of out-
come (return, benefit) derived from some form of input
(investment, cost, resource).
We searched Medline, Embase, Global health, PsycInfo,
EconLit, NHS EED, Web of Science, Google, Google
scholar, organisational journals, as well hand-searched
citations. No language and date limits were set to enable
us to note any changes in QI-ROI conceptualisation over
time. e search ended January 30, 2021. An example of
the search strategy for Web of Science has been provided
as Supplementary Table1. A link to more of our search
strategies can be found in the Supplementary files. e
main search terms are defined in Table2.
Eligibility
As ROI is an investment allocation decision tool, our
stakeholder of interest were the healthcare leaders, and
level of analysis the organisation, where decision-making
outcomes are assessed. During our initial search, many
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 4 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
articles identified themselves as large-scale QI pro-
grammes. However, at closer inspection, some of these
only impacted a small part of an organisation and were
therefore equivalent to a small organisational unit inter-
vention. To focus our selection criteria, we developed a
preliminary ROI conceptual framework (Fig.1). e pre-
liminary framework contained various needs and obliga-
tions of healthcare organisations [71, 72]. We assumed
these to signal desired organisational outcomes.
e Framework had four criteria: 1) organisational per-
formance (patients and financial outcomes), 2) organisa-
tional capacity and capability, 3) external relations (e.g.,
accreditation), and 4) unintended consequences (posi-
tive/negative). Organisational performance is a marker
of how well organisations perform on delivering value for
its stakeholders [73]. us, in a way it includes external
relations, e.g., population health. However, external out-
comes were isolated here to deduce some unique external
outcomes and obligations towards external stakehold-
ers. We included negative outcomes as potential indica-
tors of the lack of ROI positive returns. We then used this
framework to decide on eligibility.
To be included, the literature that had to mention at
least three QI organisational goals or benefits, two of
which had to be patient or financial outcomes. rough
this, we sought to isolate articles that discussed a range
of QI outcomes, with patient and financial outcomes as
basic organisational goals. In addition, articles had to
mention use of at least one QI method, and involvement
of various stakeholders, in at least two organisational
units. Altogether, this denoted a three-dimensional cri-
teria: depth, breath, and complexity of QI programmes
per organisation. We included literature on discussions of
large-scale QI programmes outcomes across healthcare
globally. Table3 has included/excluded article types.
Screening andselection ofarticles
Data were managed using Endnote citation manager [74]
and Ryann systematic review app [75]. Screening and
Table 1 Search terms
CONTEXT AND QI METHODS AND QI OUTCOMES
Health Quality improvement
OR QI OR statistical
process control OR
Lean OR Six sigma
OR Lean Six‑sigma
OR Audit and
feedback OR Model
for improvement OR
Root cause analysis
OR Process mapping
OR Define Measure
Analyse Improve
Control OR DMAIC
OR Plan do study act
OR PDSA OR PDCA
OR Driver diagram OR
Theory of change OR
Logic model OR SPC
OR statistical quality
control OR SQC
Return on investment
OR Rate of return OR
Payback OR Business
case OR Benefit cost
OR Risk benefit OR
Cost benefit OR Cost
consequence OR
Cost reduction OR
Cost containment
OR Cost control OR
Cost avoidance OR
Cost saving OR cost
outcome OR Value
on investment OR
Value care OR Value
for money OR Value
improvement OR
Improvement out‑
come OR Resource
outcome OR
Resource benefit
Table 2 Definitions of terms
Terms Description
CEA Cost‑effectiveness analysis: Achieving more of the outcome for the same cost or achieving the same outcome for less
cost, expressed in incremental benefits on Quality Adjusted Life Years (QALY), or incremental cost‑effectiveness ratio
(ICER)
CUA Cost‑utility analysis: Similar to CEA but for multiple outcome measures in quality‑of‑life units (QoL)
CBA Cost–benefit analysis: Financial expression of costs and benefits from a programme in a cost–benefit ratio (CBR)
CBA is the basis for ROI and SROI; CBA and SROI are societal perspectives, ROI is managerial/investor focused
ROI Return on Investment: Expression of costs and benefits from a programme expressed in an ROI metric
SROI Social Return on Investment: Expression of costs and benefits from a programme expressed in a ROI metric
Includes benefits for society, environment, and others. Engages various stakeholders in the calculation process
CCA Cost consequence analysis: comparing alternative interventions or programs in which the components of incremental
costs and consequences without aggregating these results
Economic terms sources: [17, 60–63]
Value
Value for money Any outcome seen to be of importance, utility, or usefulness. [64]
Obtaining the most useful (utility), most effective, and less wasteful (efficient) from your service or purchase [60]
Benefit Any outcome that produces useful, helpful, or advantageous outcomes [65]
Outcome A result or consequence of an action or process [66]
QI methods Methods used to improve organisational processes and behaviours e.g., PDSA, Lean, Six‑Sigma, Lean‑Six Sigma, Audit &
Feedback. [67–69]
Healthcare organisation (UK) A unique framework of authority within which a person or persons act or are designated to act towards some purpose as
a direct provider of healthcare services (preventative, curative, rehabilitative, or palliative). Includes Local Authorities with
Social care working in cooperation with the NHS [70]
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 5 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
selection were performed by two independent reviewers,
ST, and MM. To refine our selection criteria, five articles
were initially selected and discussed to clarify any uncer-
tainties. e two reviewers then completed the screening
and selection of the remaining articles independently: ST
100%, MM 5%. Overall agreement was over 90%. Disa-
greements were discussed and settled by ST and MM,as
well as with co-authors CH and K-CC.
Fig. 1 Preliminary QI‑ROI Conceptual Framework
Table 3 Eligibility criteria and selected article types
Eligibility Outcomes
QI Effectiveness or process outcomes e.g., goals achieved
QI economic outcomes e.g., savings
Clinical outcomes e.g., symptoms
Organisational outcomes e.g., development
Short‑term, intermediate, long‑term, and impacts
ROI concepts
Cost‑effectiveness
Cost–benefit
Value
Benefits
QI outcomes/consequences
Level of analysis
Organisation
Type of literature
Empirical and non‑empirical reports
Conceptual and Grey literature
Included Large scale complexity, depth, and breadth
At least one QI method used
At least three organisational outcomes
At least two organisational departments engaged
Excluded Articles where one department was engaged, two or less organisational outcomes were reported, and pre‑prints
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 6 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
Quality assessment
For researchers of integrative reviews and conceptual
development, quality assessment is optional as the qual-
ity of studies has little or no bearing on concept develop-
ment [53, 59]. As such, there was no intention to exclude
articles based on their quality. However, to understand
the scientific context in which QI benefits are discussed,
we assessed all empirical studies using specific quality
assessment and reporting tools. For reviews, we usedthe
Critical Appraisal Skills Programme (CASP) tool [76], for
mixed methods, we usedthe Mixed Methods Appraisal
Tool (MMAT) [77], for implementation studies, we
used Standards for Reporting Implementation Stud-
ies (STaRI) [78]. For economic evaluations, we usedthe
Consolidated Health Economic Evaluation Reporting
Standards (CHEERS) [79], and for QI, we usedthe Stand-
ards for QUality Improvement Reporting Excellence
(SQUIRE) tool [80]. As these are different tools, there
was no single criteria to judge collective study quality.
We therefore assessed the number of appropriate items
reported or addressed as per respective study’s tool. We
assigned good if 80–100% items were addressed, moder-
ate if 50–79% of items were addressed, and poor if less
than 50%.
Data extraction
Data extraction was performed using words and phrases
in the preliminary conceptual framework as well as out-
comes in the reviews search terms. We searched for these
from all parts of an article where QI benefits, outcomes,
and goals may be discussed. Articles were tabulated
according to type of article, type of focus, country, set-
ting, programme type, and outcomes discussed. e data
collection tool can be found as Supplementary Table2.
Data synthesis andanalysis
e synthesis and analysis section forms stages 5–7 of the
integrative review process (integrate, synthesise, analyse).
e synthesis, analysis, and framework development
were performed iteratively by ST. All steps in the synthe-
sis and analysis were discussed with co-authors CH and
K-CC. We used the principle-based analysis method [55],
to assess the maturity of the QI-ROI concept in health-
care literature. is involved asking four principle-based
questions: 1) Epistemological principle: is the concept
clearly defined and well differentiated from other con-
cepts? 2) Pragmatic principle: is the concept applicable
and useful within the scientific realm of inquiry? Has it
been operationalised? 3) Linguistic principle: is the con-
cept used consistently and appropriately within context?
4) Logical principle: does the concept hold its boundaries
through theoretical integration with other concepts?
Once the ROI concept maturity was established, we
followed Jabareen’s [53] conceptual framework develop-
ment process. Jabareen describes a conceptual frame-
work as “a network…of interlinked concepts that…
provide a comprehensive understanding of a phenom-
enon or phenomena” (p.51). A framework was developed
by identifying and naming concepts, describingconcepts.
Concepts were then categorised according to their onto-
logical, epistemological, or methodological role. is was
followed by synthesising, sense-making, and integration
of similar concepts into one new concept, the QI-ROI.
We also contextualise the ROI concept by highlighting
how the concept is defined in the healthcare context,
the alternative explanations afforded by the new concept
which are not normally enabled by similar concepts, and
the patterns in which the QI-ROI concept appear in the
healthcare context [54].
Results
A total of 10 428 articles were retrieved, 10 327 were
excluded for various reason as seen in Fig.2. One hun-
dred and two (102) articles were eligible, 34 were
excluded and 68 included. Included articles were: Con-
ceptual n = 24, Quantitative n = 19, including three eco-
nomic evaluations (CEA n = 1, Economic Impact n = 1,
ROI n = 1), Qualitative n = 3, Mixed-Methods n = 8,
Systematic Reviews n = 8 (2 economic; 1 SROI), Lit-
erature reviews n = 2, Brief Report n = 4, irty three of
the excluded articles engaged a single department and/
or discussed two or less QI outcomes/goals. irteen of
these were collaboratives. ere was one pre-print. A link
to the excluded studies document is available on the sup-
plementary files.
Article characteristics
Included articles covered different healthcare levels and
disciplines globally. Primary care included public health,
child and maternal health, and mental health. Secondary
or tertiary healthcare included mental health, medical
and surgical care, critical care, accident and emergency
and acute care services, paediatrics and neonatal care,
outpatients, pharmacy, and laboratories. One article cov-
ered both health and social care, and anotherwas about a
charitable organisation. Global regions were Africa, Asia,
Europe, Australia, and Canada, with the US and UK the
mostly represented. e summary of included studies can
be found in supplementary files (Supplementary Table3).
Quality ofstudies
From the 68 articles, 30 were not subject to quality assess-
ment. is included conceptual articles, unsystematic
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 7 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
literature reviews, brief reports. irty eight articles were
subject to quality assessment: 19 quantitative studies,
three qualitative studies, eight mixed-methods studies,
and eight systematic reviews. Of the 38, 39% reported or
addressed 80%-100% all items required, 43% reported on
50%-79% the data required, and 18% reported below 50%
of items by their respective reporting tool. irty out of
thirty eight studies were primary studies. In these, there
were three main areas of poor reporting and or poor rig-
our on all types of studies: ethics (29%), statistical analy-
sis methods (75%), discussion of study limitations and
their management (42%), integration of quantitative and
qualitative data unclear (29%). Reporting of funding and
affiliations missing on three of all 35 studies. erefore,
the quality of the studies was summed up as moderate.
e quality assessment summary is available as Supple-
mentary Table4.
Synthesis summary
QI Economically‑focused literature
Economically-focused literature were those articles
whose specific focus was on either studying or discussing
QI economic benefits. is made up 15 articles of the 68
QI literature. Amongst these were four conceptual litera-
ture, (three of which were business case discussions) [22,
36, 81, 82], four QI evaluation frameworks [83–86], two
systematic reviews of economic evaluations [87, 88], one
article discussed cost effectiveness of QI programmes
[89], one article calculated an organisation’s QI related
cost-savings [90], and two were economic evaluations
[91, 92]. One article discussed cost–benefit analysis [93].
Of the total 15 articles, ROI was a specific subject of only
four [81, 84, 87, 91].
(1) Business case conceptual literature
Conceptual literature authorsdiscussed QI business
case development [22, 36, 81, 82]. eir views were
based on literature reviews, expert opinions, case
studies, or all three. is literature were mostly
sources of information for how to develop business
cases that justify QI programme from a financial
benefit perspective. However, they also highlighted
multiple QI objectives and stakeholders. As such,
there was a requirement to present QI outcomes
as a monetised ratio, and oruse of ROI as a finan-
cialperformance measurement method [81].How-
ever, this literature also highlighted multiple QI
objectives as well as multiple stakeholders as QI
beneficiaries.
(2) ROI Frameworks
ese authors advanced on the conceptual literature
by developing QI business case frameworks that
incorporate monetary and non-monetary benefits
[83–86, 93]. QI was seen to serve various organi-
sational interests, for various internal and external
stakeholders. As such, business case frameworks
centred around these principles. Swensen etal. [86]
focused on four organisational interests: patient’s
needs, organisational reputation, pride, and finan-
cial returns. ere was also aninterest in organisa-
tional productivity and efficiency.Bailit and Dyer
[83], described 10 business case arguments that
Fig. 2 PRISMA Flow‑chart
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 8 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
combine financial, strategic, and internal organisa-
tional objectives.
Shah and course [85] hada six category framework
containing three as financial measures (revenue,
cost-reduction cost-avoidance) and one measure
forpatients, family andcarersexperience, and one
for staff experience, and lastly one forproductivity
and efficiency. Financial objectives and outcomes
included cost-avoidance such future legal costs [86].
Internal outcomes also included capacity building,
whilst external outcomes examples included market
share [84–86].
(3) QI economic evaluation literature
ere were three economic evaluations [89, 91, 92],
and two systemic reviews of QI economic evaluations
[87, 88]. By the virtue of their study foci, their measure
of ROI was the monetary. ese authors saw savings as
an important QI outcome, however also discussed a wide
range of QI benefits. For example, Crawley-stout et al.
[91] considered internal outcomes (e.g., cost-reduction,
productivity, and time savings) and external benefits
(e.g., patient costs and carer time). Crawley-stout et al.
described ROI as a performance measure used to evalu-
ate investment efficiency in financial terms. de la Perrele
etal. reported a lack of QI economic evaluations in their
review. ey concluded that collaboratives are potentially
cost-saving. However, theyfound that studies used vari-
able methods to assess cost and effectiveness, and that
studies did not report negative findings. ey recom-
mended that future research should include societal per-
spectives of costs and savings [88]. Banke-omas etal.
stated that SROI (a societal version of ROI)can be used
across healthcare [87]. However, there were challenges
with inadequate skills for ROI evaluation, lack of credible
financial proxies, a lack of consensus on; who to include
as beneficiaries, how to account for counterfactual and
appropriate study-time horizon [87].
QI non‑economically focused literature
ese made up 53 [94–147] of the selected 68 articles.
ese articles included QI effectiveness, process, and
impact evaluations as well as discussions of QI achieve-
ments over time e.g., [94, 110, 116, 117, 130, 138]. ese
articles did not focus on ROI or economic measures, but
nonetheless highlighted financial outcomes as impor-
tant benefits for consideration. Some QI implementa-
tion studies assessed their implementation costs [109,
140, 145], as part of their study reporting guidelines [78,
148]. Authors here also discussed improving QI effective-
ness determinants such as staff and safety culture devel-
opment. Authors discussed or mentioned financial value
or benefit, financial returns or outcomes, cost savings,
reduction, containment, and economic impact, as well
as productivity, efficiency, value, and benefits. Of these,
cost-saving was the most frequently used term. ese
articles considered ROI as one of many organisational
outcomes [103, 130, 135].
ere were three QI evaluation frameworks [103, 104,
124]. ese frameworks also considered various elements
of organisational benefits. Chow-Chua and Goh [103]
combined existing organisational performance tools;
the Singapore Quality Award (SQA) model (modelled
after Baldrige Award) and balanced scorecard (BSC) to
develop a performance and quality improvement evalua-
tion framework for hospitals. Four strategic components
were seen as crucial: the drivers of QI (e.g., leadership),
internal performance, knowledge management, and QI
outcomes. McLees etal.’s [124] framework for QI in pub-
lic health was described as a performance management
tool, andfocused on two key constructs: efficiency and
effectiveness. Ciarniene et al.’s [104] framework envi-
sionedbroad value creation through QI.
Integrated synthesis
Morganti etal. [126] remarked that there is a lack of an
agreed concept of QI success, and by extension, ROI.
is was seen in how authors gave priority to certain
outcomes. Van den Heuwel etal. [141] for an example,
referred to quality improvement as business improve-
ment, viewed ROI quantitatively, and expected qual-
ity improvement to be a valuable “side effect” of value
improvement (often a euphemism financial improve-
ment). Alternatively, others proclaimed to value patient
safety and quality first, and saw financial matters as the
valuable “side-effect” of QI [85]. Hunter etal. [116] con-
sidered “cost savings or increased efficiency “helpful by-
products” (p. 129).
Swensen etal. [86] QI business case discussion stated
that their QI investment decisions were never based
purely on positive ROIs but on broader qualitative con-
siderations. A similar view was held by O’Sullivan et al.
[130] and Shah and Course (2020). Bailit and Dyer [83],
advocated for broad business cases that embrace differ-
ent rationales for QI investment. Fischer and Duncan [36]
stated that some interventions are purely designed to pro-
duce health outcomes. ey also called for a broader QI
outcome definition that acknowledges the utility of differ-
ing projects and value for all stakeholders. e review also
indicated that even failed goal attainment can be useful
in providing insights and legacies like building capacities
and safety cultures [94, 110, 116, 117, 130, 138, 146].
Overall, financial outcomes were not the primary or
commonly sought goal or addressed outcome. How-
ever, it was seen as directly or indirectly significant by
the majority of the authors. e perception that QI is an
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 9 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
expense used for revenue generation was seen to be due
to faulty assumptions by someauthors [86]. For example,
authors suggested that profit-seeking through QI first
emerged as an optional strategy to increase revenue and
market-shares by for-profit healthcare organisations [22,
83, 86]. However, grey areas on views existed and views
appear to have shifted towards integrating or emphasis-
ing non-monetary outcomes over time. e literature
also agreed that QI does not always save cost, and finan-
cial outcomes are not the only organisational objectives
[22, 36, 82, 83, 89, 93]. ese literatures portrayed ROI
as any value or benefit from QI for various stakeholders.
e reviewed literature illustrated five main ROI uses
related to QI: ROI as 1) a strategic business case develop-
ment tool, 2) an investment performance measure, 3) a
comparative evaluation tool, 4) a cost management tool,
and 5) a performance management tool. ROI was also
used tocreate fiscal awareness [81]. Some of this ROI
usewas similar to commercial ROI use. Various concepts
were used to denote areturn-on-investment. ese con-
cepts were used in relation to changes and improvements
in various organisational outcomes including patients,
staff, financial, and overall organisational development.
Both economic and non-economic focused literature
used almost identical concepts to denote an investment
and a return as seen in Fig.3.
However, profit, revenue, and market share were
mainly found in the few economic focused literature. e
majority of literature put greater emphasis on non-mon-
etary QI benefits. is indicates that although different
logics were applied in the conceptualisation of healthcare
QI-ROI, the dominant logic was that of health and social
care and not economics or markets.
At face value, there were two broad QI-ROI phi-
losophies: the economic and the healthcare ROI phi-
losophies. However, our synthesis indicated a merging
meaning of ROI from different schools of thought.
rough markets logic, the economic philosophy per-
spective views outcomes in terms of their tangible,
quantitative, or financial offerings. e economic phi-
losophy is related to managerial logics as managers’
roles require them to monitor organisational financial
performance. Alternatively, the healthcare perspec-
tive suggested a more dulled though increasing finan-
cial focus. Healthcare logics (medical, societal) tended
to view ROI qualitatively, primarily from a patient and
staff perspectives but also wider internal and external
stakeholders. e integrated literature indicated that
Fig. 3 ROI‑like concepts
Table 4 ROI concept perspectives
Type of QI outcome: ROI-like concepts Stakeholder
perspective Dominant logic Philosophical
perspective
Organisational outcomes:
Improvement, efficiency, productivity, effectiveness, profit, financial return, ROI,
SROI, CBA, CEA, economic impact, cost saving, cost avoidance, cost reduction,
market share, reputation, organisational development, performance (organisa‑
tional), management (cost)
Managerial Markets/
Economic Realism
Patients, family, carer, and societal outcomes:
improvements, value, benefits, impacts
(SROI, CBA)
Managerial
Patients
Society
Medical
Societal Relativism/
Interpretivism
Sta outcomes:
improvements, value, benefits, impacts Managerial
Staff Medical
Societal Relativism/
Interpretivism
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 10 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
healthcare leaders value these perspectives too. Table4
illustrates this finding.
Discussion
Although concept analysis, concept development, and
conceptual framework development are traditionally
separate steps [55], they have been performed concur-
rently in this review. Firstly, we sought to understand the
nature of the ROI concept in healthcare QI. Using the
Hupcey and Penrod [55] principle-based concept analysis
method, we found that the QI-ROI concept is not clearly
defined or developed (principle 1), the ROI application
method is unclear (principle 2), its appropriate use is
unestablished (principle 3), and conceptual boundaries
are unclear (principle 4). ese determinations answered
the first part of our research question; the nature of the
ROI conceptin healthcare QI.
We then sought to answer the second part of our
research question; the definition of the QI-ROI con-
cept and its relationship with similar concepts. We did
this by following the concept development and concep-
tual framework development guide described earlier by
Hupcey and Penrod [55], and Jabareen [53]. Our review
confirmed that various concepts and terms are used to
denote returns or investment vs returns in healthcare
QI. Some of these concepts (e.g., CEA, CBA, value) were
identified in the background literature and used as the
review’s search terms. ese concepts, including the dif-
ferences between costs, investments, and revenue have
been discussed in more details in economic literature
(e.g., [61, 149–151]. We refer interested readers to such
text for nuanced descriptions.
(1) QI-ROI concept development
To develop the QI-ROI concept, we first eliminated
similar concepts by differentiating QI-ROI from other
ROI-like concepts and terms. To start with, financial or
economic returns are alternative ways of saying ROI[60].
Other terms used to a lesser extent in the review were
economic impact, which denotes only the cost of ill-
ness [152], and cost management which is a process of
managing and controlling costs of a programme to fit
desired criteria, e.g., to reduce costs [153]. Cost-reduc-
tion is described as resulting from providing the same
or better quality for a lower cost, through new ways of
working that eliminate waste ([178], p.4). Cost avoid-
ance is cost-reduction related to preventing future costs
[178]. Very few articles mentioned profit or revenue.
Publicly funded healthcare such as the UK NHS does not
customarily engage in profit-making. Net profit is what
remains after all expenses have been subtracted from net
revenue (income generated minus costs).e remaining
concepts: SROI, cost–benefit, cost-effectiveness, produc-
tivity, efficiency, value, and benefit were seen as the most
mimicking ROI. ese concepts have varying degrees of
association with ROI depending on the ultimate objective
sought through a QI programme.
(1a) ROI vs other economic evaluation methods
In health economics, cost-effectiveness is assessed
using CEA and whilst costs-benefits are assessed using
CBA. CEA and CBA goals are to ensure that fixed allo-
cated quantity of healthcare resources result in most
health outcomes improvement (CEA) or maximum
social advantage (CBA) [61]. CEA and CBA denote cost
vs goal achievement. Although conceptually different,
in practice, researchers do conflate the different types
of evaluation [154, 155] or see them as interchangeable
[60]. Comparative ROI mimics CEA, but ROI reports
an aggregated cost–benefit metric similar to CBA’s
cost–benefit ratio (CBR). Alternatively, CEA reports an
incremental cost-effectiveness ratio (ICER) per health
outcomes. Incremental benefits using ROI of new QI
programmes have been compared by some research-
ers [156, 157]. However, given the multiple healthcare
objectives, these metrics represent only a fraction of pro-
gramme benefits or consequences [15]. is point is sup-
ported by the current literature review.
CBA is the basis of ROI and SROI. SROI and CBA are
monetisation of broad programme benefits and costs
such as societal costs and benefits. SROI extends CBA by
including environmental and other stakeholder benefits
[37]. Alternatively, ROI generally focuses on programme
specific costs and benefits from a managerial perspective
[9, 16]. As returns-on-investments evaluation methods,
CEA, CBA, SROI, and traditional ROI are too narrowly
focused as they all ultimately only emphasise monetary
focus. According to Bridges [101], CBA, does not account
for how care is produced, and thus excludes many crucial
organisational outcomes. Bridges suggested that what is
needed is a systematic value assessment approach.
(1b) ROI vs input–output based measures
CEA, CBA, productivity, and efficiency are similar as
their goals are using resources without waste. However,
they are all a single focus outcomes. CEA/CBA are an
input vs goal measures, efficiency and productivity are
input vs output measures. Productivity and efficiency are
ROI-like as they denote a return (output) of an invest-
ment (input) [158]. Inputs and outputs may be both
monetary and non-monetary. Productivity is the quantity
of outputs per investment/input. Efficiency is achieving
those outputs with least or no waste (e.g., in time, money,
effort). erefore, unlike CBA and CEA, efficiency and
productivity are related to exactly how care is produced.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 11 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
For example, increasing productivity by increasing
patients seen (output) per clinician (input), whilst provid-
ing quality care without wasting resources (efficiency).
Efficiency is divided into allocative, productive, and
technical efficiency [61]. Simply put, allocative efficiency
refers to allocation of healthcare resources such that the
most benefits are delivered [159]. Productive efficiency
is increasing output per given resource/input/invest-
ment (e.g., seeing more patients by same staff member).
If this is done such that more is obtained from the same
resource, or less resource is required for the same out-
put, it is technically efficient [151]. is description also
fits CEA, with outputs being effectiveness.It also mimics
the concept of value-for-money (VfM), used to describe
the optimal balance betweenefficiency, economy (lowest
cost), and effectiveness [60]. Efficiency and productivity
are crucial in healthcare as profit-based ROI is deemed
improbable [160]. Efficiency can translate to both mon-
etary ROI (e.g., savings), and non-monetary benefits (e.g.,
improved staff work experience).
Productivity and efficiency are often used to measure
performance of healthcare organisations [150, 158]. Pro-
ductivity may enable allocative efficiency of funds or bet-
ter time allocation for tasks by staff. Productivity can be
an efficiency measure (input/output) [158]. It can also be
a combined effectiveness and efficiency measure (goal/
input/output), or of all that makes an organisation func-
tion better [150]. e latter is what the reviewed literature
indicated QI-ROI to be. Effectiveness through attainment
of goals alone is therefore also insufficient to describe QI-
ROI. Goals may be achieved, but inefficiently. In a bal-
anced productivity-efficiency-effectiveness relationship,
all three contribute to the overall QI-ROI [161]. is may
then mean avoiding, reducing, and containing costs, and
thus saving costs.
(1c) ROI vs cost saving
Cost-saving is also a more likely outcome than hard-
cash profit in healthcare QI [16]. Cost-saving was a
particularly prevalent term in the reviewed literature
and ROI in healthcare has been called savings [21]. e
current desire to save costs is thought to have driven
the change in focus from cost-effectiveness studies to
ROI [162]. Cost-saving means saving money that would
have otherwise been spent. Savings (time/money) often
result from better efficiency and productivity. Similar
terms such as cost-containment, cost-minimisation,
cost-avoidance, cost-reduction are also not seen here as
complete representations of QI-ROI. Here, these terms
are seen as representing outputs, initial or intermedi-
ate outcomes that lead to savings. ese terms (includ-
ingcost-management) can also be processes or abilities
that enable cost-saving or profit-making. Together, these
terms refer to mechanisms (or processes that enable an
outcome) [163] through which long-term financial ROI
may be achieved. Alternatively, some may see these initial
outcomes as benefits themselves.
(2) QI-ROI framework development
For some organisations, initial outputs and intermedi-
ate outcomes may be the intended outcomes and there-
fore may represent a form of ROI. In Phillips etal. [16]
for example, productivity and efficiency were viewed
as final intended outcomes of improvements. In other
instances, cost-effectiveness may be the intended goal.
Often in healthcare the ultimate objective is to achieve
higher goals, such as financial stability. In such cases,
implementing QI leads to changeand development and
possibly improvement in desired outcomes. Improve-
ment may result in improved productivity and efficiency.
is in turn improves abilities to better avoid, reduce, or
manage costs, leading to savings, and potentially mone-
tary ROI. All of this is a benefit in and of itself. Monetary
ROI depends on each output, ability, or outcome, most
of which is non-monetary. is conceptualisation of QI-
ROI is illustrated in Fig.4 below.
e QI-ROI conceptualisation can be translated as fol-
lows: value is any outcome seen to be of importance, util-
ity, or usefulness [64]; attaining a return-on-investment
whatever that/those are, is valued and therefore of ben-
efit. A benefit is any outcome that produces useful, help-
ful, or advantageous outcomes [164]. Any benefit is of
value in of itself. Based on this review, a fuller description
of QI-ROI is suggested as follows: QI-ROI is any value or
benefit (or any valued benefit) derived from or contrib-
uted to by QI programmes. is value or benefit maybe
in a form of an improved output, process, ability, out-
come, and or overall impacts, depending on an organisa-
tion’s values and objectives.is description reflects the
Healthcare Financial Management Association (HFMA),
NHS England, and NHS Improvement (NHSE&I)’s view.
e NHSE&I differentiates from cash-releasing benefits
that enable resource allocation, and non-cash-releasing
benefits that provide economic benefit, without releasing
cash [178].
is definition is different from that which supports
the economic logics. For example, Phillip etal. [16], and
Solid [9] who discussed value and ROI as separate (pre-
sumed to mean non-monetary and monetary value).
Here, only monetary value is seen as ROI. is assertion
is based on viewing ROI as a purely quantitative metric.
However, Solid’s writing also refers to value as being sub-
jective (p. 5). e definition of ROI concept as any ben-
efit may be deemed an abuse of ROI [16, 29]. However,
this definition was based on the review of numerous
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 12 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
healthcare stakeholders’ views. is may be an indica-
tion that new constructive conversations about how to
integrate economic and accounting concepts in health-
care are needed [57, 165]. After-all, value was not always
seen as a ratio between perceived benefits and perceived
costs [166]. It was initially viewed as multi-dimensional
by earlier economic scholars [57, 166–168]. erefore,
other QI programme effects that are regarded as valuable
must not only be considered [61], but prioritised where
appropriate.
e view of QI-ROI in healthcare as a broad and no-
specific concept, encompassing both monetary and
non-monetary outcomes, opens ROI to being a con-
text-specific and dynamic concept. is is in-line with
the moderate ontological expectations of modern con-
cepts [54, 55]. is suggests abilities to compromise
and accommodate varied logics that govern healthcare
[169]. e concepts of ROI, value, and benefit, denote
the beliefs about what is a true return, value, or benefit
(Ontology) as well as what seen asa moral good (Axiol-
ogy) in healthcare QI. As such, the QI-ROI can be seen as
a value-based ROI. Traditional ROI is a monetary meas-
ure (Realism), benefits in general tend to be described as
non-monetary (Relativism), and value can denote either a
monetary or a non-monetary outcome (Critical Realism).
ese beliefs then influence how evidence is created,
viewed, and studied (Epistemology and Methodology)
[53].
e lack of convincing vocabulary to argue against the
logics of the markets in healthcare was seen as the limit-
ing factor by Bozeman[170]. In the current review this
has also been demonstrated. In non-economic literature,
financial outcomes mentions appeared to be nebulous, or
in general use of terms as in everyday language. Authors
here focused on non-monetary outcomes. ey dis-
cussed aspirations to raise fiscal awareness and encour-
age financial outcomes focus on QI evaluations. is can
be contrasted with economic focussed literature where
for example, economic evaluations referred to ROI as
a specific scientific quantitative measure. Traditional
ROI is portrayed as a rational measure of objectively
assessed inputs leading to objective outputs [171]. is
suggests that the scientific language of healthcare stake-
holders for ROI is currently underdeveloped. It reflects
general challenges of legitimising and aligning qualitative
benefits with specific scientific measures that are seen as
valid and trusted [48].
e QI literature discussed the use of ROI in several
ways, including to create fiscal awareness. Such a use
for ROI was noted by Botchkarev & Andru [29] in their
analysis of ROI definitions. eir typology included the
use of ROI as a persuasive device to gain credibility for a
Fig. 4 QI‑ROI conceptual framework
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 13 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
desired programme [9]. Healthcare leaders need credible
recourse or language to articulate large-scale QI benefits
[85, 170]. If we accept that reality is socially constructed,
then we can view various logics as both coercive and
emancipating [41]. at is, although political and market
logics may constrain freedoms of local expressions, the
mere tendency for humans to create their own meanings
has potential to liberate from such constraints. Logics
‘in situ’ provide symbolic systems and vocabularies for
expression. Hence, the prevailing logic both shapes and is
shaped by contexts. Scientifically developing concepts for
healthcare is essential to support this.
Establishing ways of expressing QI-ROI from health-
care programmes is crucial to avoid missing oppor-
tunities for essential healthcare improvements [30].
Additionally, insisting on inflexible use of acertain (ROI)
policy may lead to data manipulation in bids to increase
credibility [172, 173]. e view of ROI as both monetary
and non-monetary benefits reflects the multi-stake-
holder healthcare context. e lean towards non-mon-
etary benefits is influenced by persistent healthcare and
societal logics [11, 23, 47]. ese logics emphasise relief
of suffering and ethical principles such as beneficence
(benefiting others) and non-maleficence (do no harm)
[174]. It is therefore important to differentiate ROI as
concept and as a metric. A concept is more than a metric,
it encapsulates mental abstractions about how it is per-
ceived by those using it and influences the decisions that
then may follow [175].
Strengths andlimitations
is review has a few strengths and limitations. Concept
analysis, concept development, and conceptual frame-
work development are traditionally separate steps [55],
unlike in this review. It is however accepted that these
processes are intertwined [55]. However, we based our
analysis on intensive background reading as well as a
large review of different QI literature. is enabled us
to gain some understanding of the current “state of the
science” [55] surrounding the ROI concept as used in
healthcare QI. We then followed a well-recognised Hup-
cey and Penrod [55], andJabareen [53] development pro-
cess to start to develop the concept QI-ROI in healthcare.
Secondly, productivity and efficiency proved to be cru-
cial parts of the QI-ROI concept. ese concepts were
not included as search terms, however the large amount
of literature retrieved means that it is unlikely that this
made a significant difference in the review. Alternatively,
it could be argued that our inclusion of specific ROI-like
concepts in our search terms constitutes sampling bias.
However, this strategy helped identify relevant literature
for a more in-depth review. irdly, asignificant amount
of the literature reviewed was non-empirical in nature.
Although this literature lacks a scientific focus, it was
nonetheless very insightful in understanding the nature
of the QI-ROI concept. Fourthly,some of the literature
is quite dated, however newer literature suggest continu-
ance of some trends and issues in QI-ROI and business
case development. Lastly, subjectivity in the synthe-
sis and analysis cannot be ruled out. As Parkinson etal.
[176] put it “…findings are a consequence of intersubjec-
tive meaning-making through imagination, interpreta-
tion, and conceptual input…” (p15).
Implications forresearch andpractice
Implementation and Improvement Sciences are faced
with the challenge of developing the ROI concept that is
theoretically sound, and scientifically valid. is means a
QI-ROI framework must clearly isolate constructs that
can and should be included in an evaluation tool. e
development of the QI-ROI concept and its conceptual
framework must also ensure it is fit for purpose by incor-
porating both monetary and non-monetary benefits. is
means finding more innovative and accessible ways for
evaluating the QI-ROI aspects that are hard to measure
and or monetise. Developing the QI-ROI concept in this
way will enable the field to progress and take ownership
in QI fiscal matters, and leaders to justify investments.
is is crucial as justification for investment is unavoid-
able and necessary in the current economic climate.
e review indicated that the use of reporting tools is
having a positive effect on the quality of QI studies. How-
ever, there remains room for improvement. QI research-
ers have a responsibility to show more transparency
on ethical aspects of their studies. QI studies may not
require ethical permissions, and if so, it must be stated
as such. Current QI reporting tools allow for this [78,
148]. QI studies must also be clear about theirstatisti-
cal analysis methods use.Another area of improvement
is the integration of qualitative and quantitative data in
their analysis. is is important in strengthening research
findings [177]. Further, reporting of study limitations
was limited in the reviewed literature. e knowledge of
QI implementation or research challenges can help arm
other researchers and field practitioners in their QI initi-
atives. is is crucial for developing a stronger evidence-
base as we develop the QI-ROI concept.
Conclusion
Return-on-investment is an important tool with great
potential to communicate QI benefits not covered by
CEA and CBA. However, in its traditional form, ROI
does not take advantage of this potential use. Ignoring
the paradoxes contained within the traditional ROI use
in healthcare may continue to keep ROI in the fringes of
QI evaluation or cause conflict amongst stakeholders if
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 14 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
enforced. erefore, continued application of ROI must
be based on its conceptualisation within healthcare QI
and must be grounded on scientific inquiry that considers
relevance to practice and policy. If QI-ROI is developed
in this way, its legitimacy within healthcare stakeholders
may be established and increased. In this review, we have
begun to unpack what ROI is and means for healthcare
stakeholders in QI at the organisational level. We hope to
continue to develop this framework into a practical tool
that is meaningful to its users: the QI teams and health-
care leaders, and QI investors.
Abbreviations
QI: Quality Improvement; ROI: Return on Investment; SROI: Social Return on
Investment; QI‑ROI: Return on Investment from healthcare quality improve‑
ment; CEA: Cost Effectiveness Analysis; CUA : Cost Utility Analysis; CBA: Cost
Benefit Analysis.
Supplementary Information
The online version contains supplementary material available at https:// doi.
org/ 10. 1186/ s12913‑ 022‑ 08832‑3.
Additional le1:Supplementary Table1. Example search strat‑
egy. Supplemental Table2. Data extraction tool. Supplementary
Table3. Included studies. Supplementary Table4. Summary of Quality
assessment.
Acknowledgements
The authors would like to acknowledge Professor Judith E. Hupcey for her
advice on concept development. The authors would also like to thank Mr
Andy Healy for his advice on economic evaluation.
Authors’ contributions
Two reviewers ST and MM worked independently under the guidance of
co‑authors CH and KC‑C. MM reviewed 5% of articles from search to synthesis,
and ST 100% of all stages. Agreement in the co‑review stages was over 90%.
The synthesis and analysis was performed iteratively by ST. All steps in the
synthesis and analysis were discussed with co‑authors CH and K‑CC. Any
disagreements were discussed with N.N, B.G, T.S and KC‑C, and C.H. S.T wrote
the manuscript, compiled all the tables and figures in this manuscript. All
authors advised, reviewed, and approved the development of this manuscript,
its tables, and figures.
Funding
This work is supported by the Economic and Social Research Council, grant
number ES/P000703/1. TS is supported by the Welcome Trust (219425/Z/19/Z)
and Diabetes UK (19/0006055).
Availability of data and materials
The datasets used and/or analysed during the current study are available
from the corresponding author on reasonable request. Some data has been
included in this article as its supplementary information files.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
TS received funding from Cancer Alliance and Health Education England for
training cancer multidisciplinary teams (MDTs) in assessment and quality
improvement methods in the United Kingdom. TS received consultancy fees
from Roche Diagnostics. The other authors declare that they have no compet‑
ing interests.
Author details
1 King’s College London, London, UK. 2 South London and Maudsley NHS
Foundation Trust, London, UK.
Received: 24 May 2022 Accepted: 14 November 2022
Published: 7 December 2022
References
1. Ross S, Naylor C. Quality improvement in mental health. King’s Fund. 2017.
2. Braithwaite J, Marks D, Taylor N. Harnessing implementation science to
improve care quality and patient safety: a systematic review of targeted
literature. Int J Qual Health Care. 2014;26(3):321–9.
3. Storkholm MH, Mazzocato P, Savage M, Savage C. Money’s (not) on my
mind: a qualitative study of how staff and managers understand health
care’s triple Aim. BMC Health Serv Res. 2017;17(1):98.
4. Benn J, Burnett S, Par A, Pinto A, Isk S, et al. Studying large‑scale pro‑
grammes to improve patient safety in whole care systems: challenges
for research. Soc Sci Med. 2009;69(12):1767–76.
5. Øvretveit J, Klazinga N. Learning from large‑scale quality improvement
through comparisons. Int J Qual Health Care. 2012;24(5):463–9.
6. Schouten LMT, Grol RPTM, Hulscher MEJL. Factors influencing success
in quality‑improvement collaboratives: development and psychometric
testing of an instrument. Implement Sci. 2010;5(1):84.
7. Dopp AR, Narcisse M‑R, Mundey P, Silovsky JF, Smith AB, Mandell D,
et al. A scoping review of strategies for financing the implementation
of evidence‑based practices in behavioral health systems: State of the
literature and future directions. Implementation Research and Practice.
2020;1:2633489520939980.
8. Danzon PM, Drummond MF, Towse A, Pauly MV. Objectives, budgets,
thresholds, and opportunity costs—a health economics approach: an
ISPOR Special Task Force report [4]. Value in Health. 2018;21(2):140–5.
9. Solid CA. Return on investment for healthcare quality improvement.
Springer International Publishing; 2020.
10. World Health Organization. Making the investment case for mental
health: a WHO/UND guidance note. Bahrain: 2019.
11. Masters R, Anwar E, Collins B, Cookson R, Capewell S. Return on invest‑
ment of public health interventions: a systematic review. J Epidemiol
Community Health. 2017;71(8):827.
12. Nystrand C, Gebreslassie M, Ssegonja R, Feldman I, Sampaio F. A
systematic review of economic evaluations of public health interven‑
tions targeting alcohol, tobacco, illicit drug use and problematic
gambling: Using a case study to assess transferability. Health Pol.
2021;125(1):54–74.
13. Botchkarev A. Estimating the accuracy of the Return on Investment
(ROI) performance evaluations. Int J Inform, Knowledge, Manage.
2015;10;217–33.
14. Gosselin V, Boccanfuso D, Laberge S. Social return on investment
(SROI) method to evaluate physical activity and sport interventions: a
systematic review. The international journal of behavioral nutrition and
physical activity. 2020;17(1):26.
15. Bukhari H, Andreatta P, Goldiez B, Rabelo L. A framework for determin‑
ing the return on investment of simulation‑based training in health
care. INQUIRY: The Journal of Health Care Organization, Provision, and
Financing. 2017;54. https:// doi. org/ 10. 1177/ 00469 58016 687176.
16. Phillips PP, Phillips JJ, Edwards LA. Measuring the success of coaching:
a step‑by‑step guide for measuring impact and calculating ROI. Ameri‑
can Society for Training and Development; 2012.
17. Pokhrel S. Return on investment (ROI) modelling in public health:
strengths and limitations. Eur J Pub Health. 2015;25(6):908–9.
18. Parand A, Dopson S, Renz A, Vincent C. The role of hospital managers
in quality and patient safety: a systematic review. BMJ Open. 2014;4(9):
e005055.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 15 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
19. Otake T, Min KJ. Inventory and investment in quality improve‑
ment under return on investment maximization. Comput Oper Res.
2001;28(10):997–1012.
20. Hewko SJ, Cummings GG. Performance management in healthcare: a
critical analysis. Leadersh Health Serv. 2016;29(1):52–68.
21. Price CP, McGinley P, John AS. What is the return on investment for
laboratory medicine? The antidote to silo budgeting in diagnostics. Br J
Healthc Manag. 2020;26(6):1–8.
22. Leatherman S, Berwick D, Iles D, Lewin LS, Davidoff F, Nolan T, et al. The
business case for quality: case studies and an analysis. Health affairs
(Project Hope). 2003;22(2):17.
23. Brousselle A, Benmarhnia T, Benhadj L. What are the benefits and risks
of using return on investment to defend public health programs? Prev
Med Rep. 2016;3:135–8.
24. Bontis N, Fitz‐enz J. Intellectual Capital ROI: A causal map of human
capital antecedents and consequents. J Intellect Cap. 2002;3(3):223–47.
25. Boonstra A, Broekhuis M. Barriers to the acceptance of electronic
medical records by physicians from systematic review to taxonomy and
interventions. BMC Health Serv Res. 2010;10(1):231.
26. Boyd J, Epanchin‑Niell R, Siikamäki J. Conservation Planning: A Review
of Return on Investment Analysis. Review of Environmental Economics
and Policy. 2015;9(1):23–42.
27. Dearden J. CASE AGAINST ROI CONTROL. Harvard Business Review.
1969. [Available https:// hbr. org/ 1969/ 05/ the‑ case‑ again st‑ roi‑ contr ol]
28. Ozminkowski RJ, Serxner S, Marlo K, Kichlu R, Ratelis E, Van de Meule‑
broecke J. Beyond ROI: Using Value of Investment to Measure Employee
Health and Wellness. Popul Health Manag. 2016;19(4):227–9.
29. Botchkarev A, Andru P. A Return on Investment as a Metric for Evaluat‑
ing Information Systems: Taxonomy and Application. Interdiscip J Inf
Knowl Manag. 2011;6:245–69.
30. Anderson JA. Evolution of the health care quality journey: from cost
reduction to facilitating patient safety. J Leg Med. 2010;31(1):59–72.
31. Lurie N, Somers SA, Fremont A, Angeles J, Murphy EK, Hamblin A. Chal‑
lenges to using a business case for addressing health disparities. Health
Aff. 2008;27(2):334–8.
32. Andru P, Botchkarev A. Return on investment: A placebo for the Chief
Financial Officer… and other paradoxes. Journal of MultiDisciplinary
Evaluation. 2011;7(16):201–6.
33. Rust RT, Zahorik AJ, Keiningham TL. Return on quality (ROQ): Making
service quality financially accountable. J Mark. 1995;59(2):58–70.
34. Phelps CE, Madhavan G. Using Multicriteria Approaches to Assess the
Value of Health Care. Value Health. 2017;20(2):251–5.
35. Leggat SG. Information management: the limitations of ROI. Australian
Health Review. 2007;31:488.
36. Fischer HR, Duncan SD. The business case for quality improvement. J
Perinatol. 2020;40(6):972–9.
37. Arvidson M, Battye F, Salisbury D. The social return on investment in
community befriending. Int J Pub Sector Manage. 2014;27(3):225–40.
38. Astrella JA. Return on Investment: Evaluating the Evidence Regarding
Financial Outcomes of Workplace Wellness Programs. J Nurs Adm.
2017;47(7/8):379–83.
39. Baxter S, Sanderson K, Venn AJ, Blizzard CL, Palmer AJ. The relation‑
ship between return on investment and quality of study methodol‑
ogy in workplace health promotion programs. Am J Health Promot.
2014;28(6):347–63.
40. Chalutz B‑G. An ROI‑based review of HR analytics: practical implemen‑
tation tools. Pers Rev. 2019;48(6):1429–48.
41. Thornton PH, Ocasio W. Institutional logics. The Sage handbook of
organizational institutionalism. 2008;2008(840):99–128.
42. Thornton PH, Ocasio W, Lounsbury M. The institutional logics per‑
spective: A new approach to culture, structure and process. Oxford:
Oxford University Press on Demand; 2012.
43. Fulop N, Robert G. Context for successful quality improvement. London:
The Health Foundation; 2015.
44. Braithwaite J, Churruca K, Ellis LA, Long J, Clay‑Williams R, Damen N,
et al. Complexity science in healthcare. Sydney: Australian Institute of
Health Innovation, Macquarie; 2017.
45. Cresswell KM, Worth A, Sheikh A. Actor‑Network Theory and its role in
understanding the implementation of information technology devel‑
opments in healthcare. BMC Med Inform Decis Mak. 2010;10(1):1–11.
46. Gasparski WW. The stakeholder organisation theory and its systemic
foundation. Int J Gen Syst. 2009;38(6):659–65.
47. Besharov ML, Smith WK. Multiple institutional logics in organizations:
Explaining their varied nature and implications. Acad Manag Rev.
2014;39(3):364–81.
48. Dunn MB, Jones C. Institutional logics and institutional pluralism:
The contestation of care and science logics in medical education,
1967–2005. Adm Sci Q. 2010;55(1):114–49.
49. Braithwaite J, Herkes J, Ludlow K, Testa L, Lamprell G. Association
between organisational and workplace cultures, and patient outcomes:
systematic review. BMJ Open. 2017;7(11): e017708.
50. Øvretveit J. Does improving quality save money. A review of evidence
of which improvements to quality reduce costs to health service pro‑
viders London: The Health Foundation. 2009;95.
51. Secretary of State for Health, editor. High quality care for all: NHS next
stage review final report. Volume 7432. 2008.
52. Morse JM, Hupcey JE, Mitcham C, Lenz ER. Concept analysis in nursing
research: a critical appraisal. Sch Inq Nurs Pract. 1996;10(3):253–77.
53. Jabareen Y. Building a Conceptual Framework: Philosophy, Definitions,
and Procedure. Int J Qual Methods. 2009;8(4):49–62.
54. Risjord M. Rethinking concept analysis. J Adv Nurs. 2009;65(3):684–91.
55. Hupcey J, Penrod J. Concept Analysis: Examining the State of the Sci‑
ence. Res Theory Nurs Pract. 2005;19:197–208.
56. Heeks R, Ospina AV, Wall P, editors. Combining pragmatism and critical
realism in ICT4D research: an e‑Resilience Case Example. International
Conference on Social Implications of Computers in Developing Coun‑
tries. Cham: Springer; 2019. p. 14–25
57. Svoboda F. In search of value: Vienna School of Art History, Austrian
value theory and the others. J Socio‑Econ. 2011;40(4):428–35.
58. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow
CD, The PRISMA, et al. statement: an updated guideline for reporting
systematic reviews. BMJ. 2020;2021:372.
59. Whittemore R, Knafl K. The integrative review: updated methodology. J
Adv Nurs. 2005;52(5):546–53.
60. National Institute for Health and Care Excellence (NICE). Supporting
investment in public health: Review of methods for assessing cost
effectiveness, cost impact and return on investment. Proof of Concept
Report. London; 2011.
61. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW.
Methods for the Economic Evaluation of Health Care Programmes.
Oxford, UNITED KINGDOM: Oxford University Press; 2015.
62. Berdot S, Korb‑Savoldelli V, Jaccoulet E, Zaugg V, Prognon P, Lê LMM,
et al. A centralized automated‑dispensing system in a French teaching
hospital: return on investment and quality improvement. Int J Qual
Health Care. 2019;31(3):219–24.
63. Mason J, Freemantle N, Nazareth I, Eccles M, Haines A, Drummond M.
When Is It Cost‑effective to Change the Behavior of Health Profession‑
als? JAMA. 2001;286(23):2988–92.
64. Viner J. The utility concept in value theory and its critics. J Polit Econ.
1925;33(6):638–59.
65. Cambridge Dictionary. Benefit definition 2022 [Available from:
https:// dicti onary. cambr idge. org/ dicti onary/ engli sh/ benefi t].
66. Merriam‑Webster Dictionary. Outcome Definition 2022 [Available from
https:// www. merri am‑ webst er. com/ dicti onary/ outco me? utm_ campa
ign= sd& utm_ medium= serp& utm_ source= jsonld]
67. Chartier LB, Cheng AH, Stang AS, Vaillancourt S. Quality improvement
primer part 1: preparing for a quality improvement project in the
emergency department. Canadian Journal of Emergency Medicine.
2018;20(1):104–11.
68. Jones B, Vaux E, Olsson‑Brown A. How to get started in quality improve‑
ment. BMJ. 2019;364: k5408.
69. Healthcare Quality Improvement Partnership (HQIP). A guide to quality
improvement methods. Det Norske Veritas and Germanischer Lloyd
(DNV‑GL); 2015.
70. Department of Health and Social Care. The health and care system
explained. U.K; 2013.[Available https:// www. gov. uk/ gover nment/ publi
catio ns/ the‑ health‑ and‑ care‑ system‑ expla ined/ the‑ health‑ and‑ care‑
system‑ expla ined]
71. Gartner J‑B, Lemaire C. Dimensions of performance and related key per‑
formance indicators addressed in healthcare organisations: A literature
review. Int J Health Plan Manage. 2022;37(4):1941–52.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 16 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
72. Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder‑DeWan
S, et al. High‑quality health systems in the Sustainable Devel‑
opment Goals era: time for a revolution. Lancet Glob Health.
2018;6(11):e1196–252.
73. Elg M, Broryd KP, Kollberg B. Performance measurement to drive
improvements in healthcare practice. Int J Oper Prod Manage.
2013;33(11/12):1623–51.
74. The EndNote Team, Clarivate. Endnote. Version: Endonote X9. Type:
64bt. Philadelphia; 2013.
75. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web
and mobile app for systematic reviews. Syst Rev. 2016;5(1):1–10.
76. Critical Appraisal Skills Programme (CASP) . CASP 2019 [https:// casp‑
uk. net/]
77. Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al.
Mixed methods appraisal tool (MMAT), version 2018. Registration of
copyright. 2018;1148552:10.
78. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ,
et al. Standards for Reporting Implementation Studies (StaRI) State‑
ment. BMJ. 2017;356: i6795.
79. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg
D, et al. Consolidated Health Economic Evaluation Reporting Standards
(CHEERS) statement. BMJ : British Medical Journal. 2013;346: f1049.
80. Ogrinc G, Mooney SE, Estrada C, Foster T, Goldmann D, Hall LW, et al.
The SQUIRE (Standards for QUality Improvement Reporting Excellence)
guidelines for quality improvement reporting: explanation and elabora‑
tion. Qual Saf Health Care. 2008;17(Suppl 1):i13‑32.
81. Moody M, Littlepage L, Paydar N. Measuring Social Return on Invest‑
ment. Nonprofit Manag Leadersh. 2015;26(1):19–37.
82. Perencevich EN, Stone PW, Wright SB, Carmeli Y, Fisman DN, Cosgrove
SE, et al. Raising standards while watching the bottom line: making
a business case for infection control. Infect Control Hosp Epidemiol.
2007;28(10):1121–33.
83. Bailit M, Dyer MB. Beyond bankable dollars: establishing a business case
for improving health care. Issue Brief (Commonw Fund). 2004;754:1–12.
84. Mer y G, Dobrow MJ, Baker GR, Im J, Brown A. Evaluating investment in
quality improvement capacity building: A systematic review. BMJ Open.
2017;7(2): e012431.
85. Shah A, Course S. Building the business case for quality improvement:
a framework for evaluating return on investment. Future healthcare
journal. 2018;5(2):132–7.
86. Swensen SJ, Dilling JA, Mc Carty PM, Bolton JW, Harper CM Jr. The
business case for health‑care quality improvement. J Patient Saf.
2013;9(1):44–52.
87. Banke ‑Thomas AO, Madaj B, Charles A, van den Broek N. Social Return on
Investment (SROI) methodology to account for value for money of public
health interventions: a systematic review. BMC Public Health. 2015;15(1):582.
88. de la Perrelle L, Radisic G, Cations M, Kaambwa B, Barbery G, Laver
K. Costs and economic evaluations of Quality Improvement Col‑
laboratives in healthcare: a systematic review. BMC Health Serv Res.
2020;20(1):155.
89. Gandjour A, Lauterbach KW. Cost‑effectiveness of quality improvement
programs in health care. Med Klin. 2002;97(8):499–502.
90. Comtois J, Paris Y, Poder TG, Chausse S. [The organizational benefits
of the Kaizen approach at the Centre Hospitalier Universitaire de
Sherbrooke (CHUS)]. L’approche Kaizen au Centre Hospitalier Universi‑
taire de Sherbrooke (CHUS) : un avantage organisationnel significatif.
2013;25 2:169–77.
91. Crawley‑Stout LA, Ward KA, See CH, olph G. Lessons Learned From
Measuring Return on Investment in Public Health Quality Improvement
Initiatives. JPHMP. 2016;22(2):E28–37.
92. Schouten LMT, Niessen LW, van de Pas JWAM, Grol RPTM, Hulscher
MEJL. Cost‑effectiveness of a quality improvement collaborative focus‑
ing on patients with diabetes. Med Care. 2010;48(10):884–91.
93. Rogers PJ, Stevens K, Boymal J. Qualitative cost–benefit evaluation of
complex, emergent programs. Eval Program Plann. 2009;32(1):83–90.
94. Care Quality Commission (CQC), editor. Quality improvement in
hospital trusts: Sharing learning from trusts on a journey of QI. In: Care
Quality Commission, Newcastle Upon Tyne; 2018.
95. Beers LS, Godoy L, John T, Long M, Biel MG, et al. Mental Health Screen‑
ing Quality Improvement Learning Collaborative in Pediatric Primary
Care. Pediatrics. 2017;140(6):e20162966.
96. Benning A, Ghaleb M, Suokas A, Dixon‑Woods M, Dawson J, Bar‑
ber N, et al. Large scale organisational intervention to improve
patient safety in four UK hospitals: mixed method evaluation. BMJ.
2011;342(7793):369.
97. Bevan H, Plsek P, Winstanley L. Part 1: Leading large scale change: A
practical guide What the NHS Academy for Large Scale Change learnt
and how you can apply these principles within your own health and
healthcare setting. NHS Institute for Innovation and Improvement. In:
Improvement. NHS Academy for Large Scale Change. Coventry. 2011.
98. Bielaszka‑DuVernay C. Innovation profile: Redesigning acute care
processes in Wisconsin. Health Aff. 2011;30(3):422–5.
99. Bosse G, Abels W, Mtatifikolo F, Ngoli B, Neuner B, Wernecke K‑D,
et al. Perioperative Care and the Importance of Continuous Quality
Improvement‑A Controlled Intervention Study in Three Tanzanian
Hospitals. Plos One. 2015;10(9).
100. Botros S, Dunn J. Implementation and spread of a simple and effective
way to improve the accuracy of medicines reconciliation on discharge:
a hospital‑based quality improvement project and success story. BMJ.
2019;8(3):e000363‑e.
101. Bridges JFP. Lean systems approaches to health technology assess‑
ment: a patient‑focused alternative to cost‑effectiveness analysis.
Pharmacoeconomics. 2006;24:101–9.
102. Brink AJ, Messina AP, Feldman C, Richards GA, van den Bergh D, Netcare
AS. From guidelines to practice: a pharmacist‑driven prospective
audit and feedback improvement model for peri‑operative antibiotic
prophylaxis in 34 South African hospitals. J Antimicrob Chemother.
2017;72(4):1227–34.
103. Chow‑Chua C, Goh M. Framework for evaluating performance and
quality improvement in hospitals. Managing Service Quality: An Inter‑
national Journal. 2002;12(1):54–66.
104. Ciarniene R, Vienazindiene M, Vojtovic S. Process Improvement for Value
Creation: a Case of Health Care Organization. Inzinerine Ekonomika‑
Engineering Economics. 2017;28(1):79–87.
105. Collins B, Fenney D. Improving patient safety through collaboration A
rapid review of the academic health science networks’ patient safety
collaboratives. King’s Fund: 2019.
106. Crema M, Verbano C. Lean Management to support Choosing Wisely in
healthcare: the first evidence from a systematic literature review. Inter‑
national journal for quality in health care : journal of the International
Society for Quality in Health Care. 2017;29(7):889–95.
107. de Miranda Costa MM, Santana HT, Hern S, ez PJ, Carvalho AA,
da Silva Gama ZA, et al. Results of a national system‑wide quality
improvement initiative for the implementation of evidence‑based
infection prevention practices in Brazilian hospitals. J Hosp Infect.
2020;105(1):24–34.
108. DelliFraine JL, Langabeer JR 2nd, Nembhard IM. Assessing the evidence
of Six Sigma and Lean in the health care industry. Qual Manag Health
Care. 2010;19(3):211–25.
109. Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, et al.
Implementation outcomes of evidence‑based quality improvement for
depression in VA community based outpatient clinics. Implement Sci.
2012;7(1):30.
110. The Health Foundation. Learning Report: Safer Patients Initiative: Les‑
sons from the first major improvement programme addressing patient
safety in the UK. The Health Foundation, editor. London; 2011.
111. Furukawa PdO, Cunha ICKO, Pedreira MdLG. Avaliação de ações eco‑
logicamente sustentáveis no processo de medicação. Rev Bras Enferm.
2016;69(1):23–9.
112. Goodridge D, Rana M, Harrison EL, Rotter T, Dobson R, Groot G, et al.
Assessing the implementation processes of a large‑scale, multi‑year
quality improvement initiative: survey of health care providers. BMC
Health Services Res. 2018;18(237).
113. Hatcher IB. Reducing sharps injuries among health care workers: a
sharps container quality improvement project. Jt Comm J Qual Improv.
2002;28(7):410–4.
114. Heitmiller ES, Hill RB, Marshall CE, Parsons BJ, Berkow LC, Barrasso CA,
et al. Blood wastage reduction using Lean Sigma methodology. Trans‑
fusion. 2010;50(9):1887–96.
115. Honda AC, Bernardo VZ, Gerolamo MC, Davis MM. How Lean Six Sigma
Principles Improve Hospital Performance. Qual Manag J. 2018;25(2):70–82.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 17 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
116. Hunter DJ, Erskine J, Hicks C, McGovern T, Small A, Lugsden E, Whitty
P, Steen IN, Eccles M. A mixed‑methods evaluation of transformational
change in NHS North East. Southampton (UK): NIHR J Library ; 2014.
117. NHS Institute for Innovation and Improvement. The Productive Ward:
Releasing time to careTM Learning and Impact Review Final report. In:
Institute IfIaIN, editor. 2011.
118. Jones B, Horton T, Warburton W. The improvement journey: Why
organisation‑wide improvement in health care matters, and how to get
started. The Health Foundation: London; 2019.
119. Kanamori S, Sow S, Castro MC, Matsuno R, Tsuru A, Jimba M. Implemen‑
tation of 5S management method for lean healthcare at a health center
in Senegal: a qualitative study of staff perception. Glob Health Action.
2015;8:27256.
120. Lavoie‑Tremblay M, O’Connor P, Lavigne GL, Frechette J, Briand A, et al.
The Effects of the Transforming Care at the Bedside Program on Per‑
ceived Team Effectiveness and Patient Outcomes. Health Care Manag.
2017;36(1):10–20.
121. MacVane PF. Chasing the golden fleece: Increasing healthcare quality,
efficiency and patient satisfaction while reducing costs. International
Journal of Health Governance. 2019;24(3):182–6.
122. Masso M, Robert G, McCarthy G, Eagar K. The Clinical Services Redesign
Program in New South Wales: Perceptions of senior health managers.
Aust Health Rev. 2010;34(3):352–9.
123. McGrath BA, Lynch J, Bonvento B, Wallace S, Poole V, Farrell A, Diaz C,
Khwaja S, Robertson DW. Evaluating the quality improvement impact
of the Global Tracheostomy Collaborative in four diverse NHS hospitals.
BMJ Open Qual Improve Rep. 2017;6(1):bmjqir‑u220636.
124. McLees AW, Nawaz S, Thomas C, Young A. Defining and Assessing
Quality Improvement Outcomes: A Framework for Public Health. Am J
Public Health. 2015;105:S167–73.
125. Moraros J, Lemstra M, Nwankwo C. Lean interventions in healthcare:
do they actually work? A systematic literature review. Int J Qual Health
Care. 2016;28(2):150–65.
126. Morganti KG, Lovejoy S, Haviland AM, Haas AC, Farley DO. Measuring
success for health care quality improvement interventions. Med Care.
2012;50(12):1086–92.
127. Morrow E, Robert G, Maben J, Griffiths P. Implementing large‑scale
quality improvement: lessons from The Productive Ward: Releasing
Time to Care. Int J Health Care Qual Assur. 2012;25(4):237–53.
128. Neri RA, Mason CE, Demko LA. Application of Six Sigma/CAP meth‑
odology: controlling blood‑product utilization and costs. Journal of
healthcare management / American College of Healthcare Executives.
2008;53(3):183–6.
129. Niemeijer GC, Trip A, de Jong LJ, Wendt KW, Does RJ. Impact of 5 years
of lean six sigma in a University Medical Center. Qual Manag Health
Care. 2012;21(4):262–8.
130. O’Sullivan Owen P, Chang Nynn H, Baker P, Shah A. Quality improve‑
ment at East London NHS Foundation Trust: the pathway to embed‑
ding lasting change. Governance IJoH, editor: International Journal of
Health Governance; 2020.
131. Pearson M, Hemsley A, Blackwell R, Pegg L, Custerson L. Improving
Hospital at Home for frail older people: insights from a quality improve‑
ment project to achieve change across regional health and social care
sectors. BMC Health Services Res. 2017;17(387).
132. Power M, Brewster L, Parry G, Brotherton A, Minion J, Ozieranski P, McNi‑
col S, Harrison A, Dixon‑Woods M. Multimethod study of a large‑scale
programme to improve patient safety using a harm‑free care approach.
BMJ Open. 2016;6(9):e011886.
133. Robert G, Sarre S, Maben J, Griffiths P, Chable R. Exploring the sustain‑
ability of quality improvement interventions in healthcare organisa‑
tions: a multiple methods study of the 10‑year impact of the ‘Productive
Ward: Releasing Time to Care’ programme in English acute hospitals.
BMJ Qual Saf. 2020;29(1):31.
134. Roney JK, Whitley BE, Long JD. Implementation of a MEWS‑Sepsis
screening tool: Transformational outcomes of a nurse‑led evidence‑
based practice project. Nurs Forum. 2016;55(2):144–8.
135. Sermersheim ER, Moon MC, Streelman M, McCullum‑Smith D, Fromm
J, Yohannan S, et al. Improving Patient Throughput With an Electronic
Nursing Handoff Process in an Academic Medical Center A Rapid
Improvement Event Approach. J Nurs Adm. 2020;50(3):174–81.
136. Sibthorpe B, Gardner K, Chan M, Dowden M, Sargent G, McAullay D.
Impacts of continuous quality improvement in Aboriginal and Torres
Strait islander primary health care in Australia: A scoping systematic
review. J Health Organ Manag. 2018;32(4):545–71.
137. Staines A, Thor J, Robert G. Sustaining Improvement? The 20‑Year
Jonkoping Quality Improvement Program Revisited. Qual Manag
Health Care. 2015;24(1):21–37.
138. Stephens TJ, Peden CJ, Pearse RM, Shaw SE, Abbott TEF, Jones EL, et al.
Improving care at scale: process evaluation of a multi‑component
quality improvement intervention to reduce mortality after emergency
abdominal surgery (EPOCH trial). Implement Sci. 2018;13(1):142.
139. Strauss R, Cressman A, Cheung M, Weinerman A, Waldman S, Etchells E,
et al. Major reductions in unnecessary aspartate aminotransferase and
blood urea nitrogen tests with a quality improvement initiative. BMJ
Qual Saf. 2019;28(10):809–16.
140. Thursky K, Lingaratnam S, Jayarajan J, Haeusler GM, Teh B, Tew M, et al.
Implementation of a whole of hospital sepsis clinical pathway in a
cancer hospital: impact on sepsis management, outcomes and costs.
BMJ. 2018;7(3):e000355‑e.
141. van den Heuvel J, Does RJMM, Bogers AJJC, Berg M. Implement‑
ing Six Sigma in The Netherlands. Jt Comm J Qual Patient Saf.
2006;32(7):393–9.
142. Wells S, Tamir O, Gray J, Naidoo D, Bekhit M, Goldmann D. Are quality
improvement collaboratives effective? A systematic review. BMJ Qual
Saf. 2018;27(3):226–40.
143. White M, Wells JS, Butterworth T. The transition of a large‑scale
quality improvement initiative: a bibliometric analysis of the
Productive Ward: Releasing Time to Care Programme. J Clin Nurs.
2014;23(17–18):2414–23.
144. Williams B, Hibberd C, Baldie D, Duncan EAS, Elders A, Maxwell M,
Rattray JE, Cowrie J, Strachan H, Jones MC. Evaluation of the impact of
an augmented model of The Productive Ward: Releasing Time to Care
on staff and patient outcomes: a naturalistic stepped‑wedge trial. BMJ
Qual Safety. 2020;30(1):27–37.
145. Wood J, Brown B, Bartley A, Margarida Batista Custodio Cavaco A, Rob‑
erts AP, Santon K, et al. Reducing pressure ulcers across multiple care
settings using a collaborative approach. BMJ. 2019;8(3):e000409.
146. Worrall A, Ramsay A, Gordon K, Maltby S, Beecham J, King S, Shaw I,
Walshe K, Lelliot P, Worrall A. Evaluation of the Mental Health Improve‑
ment Partnerships programme. National Co‑ordinating Centre for NHS
Service Delivery and Organisation R&D (NCCSDO); 2008.
147. Yamamoto J, Abraham D, Malatestinic B. Improving insulin distribution
and administration safety using lean six sigma methodologies. Hosp
Pharm. 2010;45(3):212–24.
148. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D.
Standards for QUality Improvement Reporting Excellence 2.0: revised
publication guidelines from a detailed consensus process. J Surg Res.
2016;200(2):676–82.
149. Hollingsworth B. The measurement of efficiency and productivity of
health care delivery. Health Econ. 2008;17(10):1107–28.
150. Linna P, Pekkola S, Ukko J, Melkas H. Defining and measuring produc‑
tivity in the public sector: managerial perceptions. Int J Public Sect
Manag. 2010;23(5):479–99.
151. Palmer S, Torgerson DJ. Definitions of efficiency Bmj.
1999;318(7191):1136.
152. Byford S, Torgerson DJ, Raftery J. Cost of illness studies. BMJ.
2000;320(7245):1335.
153. Goldberg AJ, Fleming WP. Cost‑containment and cost‑management
strategies. J Healthc Manag. 2010;55(5):308–11.
154. Evans RW, Pagani FD. Economic evaluation in health care: A modern
day quagmire. J Heart Lung Transplant. 2014;33(4):344–6.
155. Bridges JFP. Future challenges for the economic evaluation of health‑
care. Pharmacoeconomics. 2005;23(4):317–21.
156. Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, et al.
Scaling‑up treatment of depression and anxiety: a global return on
investment analysis. The Lancet Psychiatry. 2016;3(5):415–24.
157. Rost K, Smith JL, Dickinson M. The effect of improving primary care
depression management on employee absenteeism and productivity.
A randomized trial Med Care. 2004;42(12):1202–10.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Page 18 of 18
Thusinietal. BMC Health Services Research (2022) 22:1492
•
fast, convenient online submission
•
thorough peer review by experienced researchers in your field
•
rapid publication on acceptance
•
support for research data, including large and complex data types
•
gold Open Access which fosters wider collaboration and increased citations
maximum visibility for your research: over 100M website views per year
•
At BMC, research is always in progress.
Learn more biomedcentral.com/submissions
Ready to submit your research
Ready to submit your research
? Choose BMC and benefit from:
? Choose BMC and benefit from:
158. Sheiner L, Malinovskaya A. Measuring productivity in healthcare: an
analysis of the literature. Hutchins center on fiscal and monetary policy
at Brookings. 2016.
159. Consortium YHE. 2016 [Available from: https:// yhec. co. uk/ gloss ary/ effic
iency/.
160. Russ‑Eft D, Preskill H. In Search of the Holy Grail: Return on Investment
Evaluation in Human Resource Development. Adv Dev Hum Resour.
2005;7(1):71–85.
161. Hjeltnes TA, Hansson B. Cost effectiveness and cost efficiency in
e‑learning. Norway: QUIS‑Quality, Interoperability and Standards in
e‑learning; 2005.
162. Edwards RT, Charles JM, Lloyd‑Williams H. Public health economics: a
systematic review of guidance for the economic evaluation of public
health interventions and discussion of key methodological issues. BMC
Public Health. 2013;13(1):1001.
163. Lewis CC, Klasnja P, Powell BJ, Lyon AR, Tuzzio L, Jones S, et al. From
Classification to Causality: Advancing Understanding of Mechanisms
of Change in Implementation Science. Frontiers in public health.
2018;6:136‑.
164. Cambridge Dictionary. Defition of benefit 2021 [Available from: https://
dicti onary. cambr idge. org/ dicti onary/ engli sh/ benefi t].
165. O’Flynn J. From New Public Management to Public Value: Paradig‑
matic Change and Managerial Implications. Aust J Public Adm.
2007;66(3):353–66.
166. Grassl W. Toward a Unified Theory of Value: From Austrian Economics to
Austrian Philosophy. Axiomathes. 2017;27(5):531–59.
167. Eabrasu M. A Praxeological Assessment of Subjective Value. Q J Austrian
Econ. 2011;14(2):216–41.
168. Wieser F. The Austrian School and the Theory of Value. Econ J.
1891;1(1):108–21.
169. Andersson T, Liff R. Co‑optation as a response to competing institu‑
tional logics: Professionals and managers in healthcare. Journal of
Professions and Organization. 2018;5(2):71–87.
170. Bozeman B. Public‑Value Failure: When Efficient Markets May Not Do.
Public Adm Rev. 2002;62(2):145–61.
171. Millar R, Hall K. Social Return on Investment (SROI) and Performance
Measurement. Public Manag Rev. 2013;15(6):923–41.
172. Anderson JE, Chang DC. Using electronic health records for sur‑
gical quality improvement in the era of big data. JAMA Surg.
2015;150(1):24–9.
173. Gao J, Chan HS, Yang K. Gaming in performance management reforms
and its countermeasures: Symposium introduction. Public Perform
Manag Rev. 2021;44(2):243–9.
174. Hem MH, Gjerberg E, Husum TL, Pedersen R. Ethical challenges when
using coercion in mental healthcare: a systematic literature review. Nurs
Ethics. 2018;25(1):92–110.
175. Gelman SA, Kalish CW. Conceptual development. Handbook Child
Psychol. 2007;1(2).
176. Parkinson S, Eatough V, Holmes J, Stapley E, Midgley N. Framework
analysis: a worked example of a study exploring young people’s experi‑
ences of depression. Qual Res Psychol. 2016;13(2):109–29.
177. Moffatt S, White M, Mackintosh J, Howel D. Using quantitative and
qualitative data in health services research–what happens when mixed
method findings conflict?[ISRCTN61522618]. BMC Health Serv Res.
2006;6(1):1–10.
178. The Healthcare Financial Management Association, NHS England,
and NHS Improvement (NHSE&I). Bristol: NHS value and efficiency
map; 2022.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in pub‑
lished maps and institutional affiliations.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com
Available via license: CC BY 4.0
Content may be subject to copyright.
Content uploaded by S’thembile Thusini
Author content
All content in this area was uploaded by S’thembile Thusini on Dec 24, 2022
Content may be subject to copyright.
