Effect of results-based financing on facility-based maternal mortality at birth: an interrupted time-series analysis with independent controls in Malawi

Abstract

Introduction
The aim of this study was to assess the impact of a results-based financing (RBF) programme on the reduction of facility-based maternal mortality at birth. Malawi is a low-income country with high maternal mortality. The Results-Based Financing For Maternal and Newborn Health (RBF4MNH) Initiative was introduced at obstetric care facilities in four districts to improve quality and utilisation of maternal and newborn health services. The RBF4MNH Initiative was launched in April 2013 as a combined supply-side and demand-side RBF. Programme expansion occurred in October 2014.

Methods
Controlled interrupted time series was used to estimate the effect of the RBF4MNH on reducing facility-based maternal mortality at birth. The study sample consisted of all obstetric care facilities in 4 intervention and 19 control districts, which constituted all non-urban mainland districts in Malawi. Data for obstetric care facilities were extracted from the Malawi Health Management Information System. Facility-based maternal mortality at birth was calculated as the number of maternal deaths per all deliveries at a facility in a given time period.

Results
The RBF4MNH effectively reduced facility-based maternal mortality by 4.8 (−10.3 to 0.7, p<0.1) maternal deaths/100 000 facility-based deliveries/month after reaching full operational capacity in October 2014. Immediate effects (changes in level rather than slope) attributable to the RBF4MNH were not statistically significant.

Conclusion
This is the first study evaluating the effect of a combined supply-side and demand-side RBF on maternal mortality outcomes and demonstrates the positive role financial incentives can play in improving health outcomes. This study further shows that timeframes spanning several years might be necessary to fully evaluate the impact of health-financing programmes on health outcomes. Further research is needed to assess the extent to which the observed reduction in facility-based mortality at birth contributes to all-cause maternal mortality in the country.

Full text

1
De AllegriM, etal. BMJ Glob Health 2019;4:e001184. doi:10.1136/bmjgh-2018-001184
Effect of results-based nancing on
facility-based maternal mortality at
birth: an interrupted time-series
analysis with independent controls
in Malawi
Manuela De Allegri,1 Rachel P Chase,1 Julia Lohmann,1 Anja Schoeps,1
Adamson S Muula,2 Stephan Brenner1
Research
To cite: De AllegriM, ChaseRP,
LohmannJ, etal. Effect
of results-based nancing
on facility-based maternal
mortality at birth: an interrupted
time-series analysis with
independent controls in
Malawi. BMJ Glob Health
2019;4:e001184. doi:10.1136/
bmjgh-2018-001184
Handling editor Kerry Scott
►Additional material is
published online only. To view
please visit the journal online
(http:// dx. doi. org/ 10. 1136/
bmjgh- 2018- 001184).
Received 20 September 2018
Revised 13 March 2019
Accepted 16 March 2019
1Heidelberg Institute of Global
Health, University Hospital and
Medical Faculty, Heidelberg
University, Heidelberg, Germany
2Community Health, University
of Malawi College of Medicine,
Blantyre 3, Malawi
Correspondence to
Dr Stephan Brenner;
stephan. brenner@ uni-
heidelberg. de
© Author(s) (or their
employer(s)) 2019. Re-use
permitted under CC BY-NC. No
commercial re-use. See rights
and permissions. Published by
BMJ.
Key questions
What is already known?
►Many low-income countries have adopted perfor-
mance payments in the form of results-based -
nancing (RBF) programmes to improve quality and
utilisation of maternal health services.
►Current evidence on RBF impact is largely focused
on immediate or intermediate health service out-
comes (ie, service utilisation, service quality), but
remains so far rather inconclusive.
What are the new ndings?
►Our ndings suggest that RBF programs with a
strong focus on quality of service delivery and in
combination with demand-side interventions can
play a role in reducing maternal mortality, in settings
with high utilisation of facility-based childbirth ser-
vices but inadequate service quality.
►This study provides further insight into how health--
nancing interventions implemented in low-income
settings require several years before reaching full
operational capacity.
What do the new ndings imply?
►Selection and evaluation of performance incentives
might require a stronger focus on their actual contri-
bution to population health outcomes.
ABSTRACT
Introduction The aim of this study was to assess the
impact of a results-based nancing (RBF) programme
on the reduction of facility-based maternal mortality at
birth. Malawi is a low-income country with high maternal
mortality. The Results-Based Financing For Maternal and
Newborn Health (RBF4MNH) Initiative was introduced at
obstetric care facilities in four districts to improve quality
and utilisation of maternal and newborn health services.
The RBF4MNH Initiative was launched in April 2013 as a
combined supply-side and demand-side RBF. Programme
expansion occurred in October 2014.
Methods Controlled interrupted time series was used
to estimate the effect of the RBF4MNH on reducing
facility-based maternal mortality at birth. The study
sample consisted of all obstetric care facilities in 4
intervention and 19 control districts, which constituted
all non-urban mainland districts in Malawi. Data for
obstetric care facilities were extracted from the Malawi
Health Management Information System. Facility-based
maternal mortality at birth was calculated as the number
of maternal deaths per all deliveries at a facility in a given
time period.
Results The RBF4MNH effectively reduced facility-
based maternal mortality by 4.8 (−10.3 to 0.7, p<0.1)
maternal deaths/100 000 facility-based deliveries/
month after reaching full operational capacity in October
2014. Immediate effects (changes in level rather than
slope) attributable to the RBF4MNH were not statistically
signicant.
Conclusion This is the rst study evaluating the effect
of a combined supply-side and demand-side RBF on
maternal mortality outcomes and demonstrates the
positive role nancial incentives can play in improving
health outcomes. This study further shows that
timeframes spanning several years might be necessary to
fully evaluate the impact of health-nancing programmes
on health outcomes. Further research is needed to assess
the extent to which the observed reduction in facility-
based mortality at birth contributes to all-cause maternal
mortality in the country.
INTRODUCTION
Although maternal deaths have decreased
globally, sub-Saharan Africa (SSA) remains
the region with the highest maternal mortality
ratio (MMR), with 546 deaths per 100 000
live births in 2015.1 The majority of maternal
deaths are attributable to direct obstetric
causes, such as haemorrhage, eclampsia,
puerperal sepsis or obstructed labour.2 In
most African settings, lack of access to care
(due to financial and distance barriers) and
poor health service delivery are key factors
on 22 June 2019 by guest. Protected by copyright.http://gh.bmj.com/BMJ Glob Health: first published as 10.1136/bmjgh-2018-001184 on 22 June 2019. Downloaded from

2De AllegriM, etal. BMJ Glob Health 2019;4:e001184. doi:10.1136/bmjgh-2018-001184
BMJ Global Health
hampering countries’ ability to adequately address the
underlying clinical causes of maternal mortality.3
In recent years, results-based financing (RBF) has
caught traction as a health system strengthening approach
in improving both utilisation and quality of health
services in low-income countries (LICs).4 RBF refers
to a set of financial arrangements linking payments to
defined healthcare outputs (eg, performance payments
for service providers) or health-seeking behaviours (eg,
conditional cash transfers (CCT) or vouchers for service
users).5 Many LIC health systems therefore adopted
RBF to gain further improvements in the utilisation and
quality of primary care services, especially those related to
maternal and newborn health (MNH). Current evidence
on the effect of RBF is rather inconclusive given the
differences in implementation contexts, and furthermore
focused on immediate or intermediate health service
outcomes, such as service utilisation, health worker moti-
vation, patient satisfaction and clinical quality.6 7 While
few authors have looked at the impact of RBF on ultimate
MNH outcomes, such as mortality, this work has almost
exclusively addressed demand-side RBF programmes (ie,
use of RBF to improve service utilisation).8–10 It follows
that, to date, there is no clear evidence available on
the causality between supply-side RBF programmes (ie,
use of RBF to improve service provision) and maternal
mortality reduction in SSA.
Our study contributes towards filling this knowledge
gap by presenting results from a quasi-experimental
impact evaluation of an RBF intervention on facili-
ty-based maternal mortality in Malawi. Our work is based
on the assumption that the current evidence on RBF falls
short to gauge the ultimate role RBF programmes can
play in improving MNH outcomes in LIC. As a result, we
postulate that comprehensive assessments of RBF ought
to include analyses of its impact on relevant mortality
indicators. Moreover, using exclusively routine data for
our analysis, we demonstrate the feasibility of secondary
data for RBF impact evaluations.
METHODS
Study setting
Malawi is an LIC in SSA with an estimated MMR of
439 deaths per 100 000 live births in 2015.11 Obstetric
care services are provided through the country’s essen-
tial health package offered free of charge at public and
contracted not-for-profit health facilities.12 In 2015, 91%
of births occurred in health facilities, with 90% of births
attended by a skilled provider.11 In 2014, unmet need for
emergency obstetric care (EmOC) among women with
obstetric complications was estimated at 75%, given the
majority of health facilities failed to fully meet the appli-
cable EmOC standards.13 Shortages in human resources
and stock-outs of essential drugs and supplies further
challenge the health system’s ability to reliably provide
EmOC.
Intervention design
In April 2013, the Ministry of Health launched the
Results-Based Financing For Maternal and Newborn
Health (RBF4MNH) Initiative in four districts (Balaka,
Dedza, Mchinji, Ntcheu) to improve quality and utili-
sation of facility-based childbirth care services.14 15 The
RBF4MNH includes two components: (1) performance
contracts with facilities and district health management
teams (DHMTs) linked to defined childbirth care quality
targets; and (2) CCT to pregnant women linked to giving
birth and spending a 48-hour postpartum observation
period at their respective catchment facility.14 15 District
selection was non-random and a result of a political deci-
sion specifically supporting districts with weaker maternal
health outcomes and EmOC structures.
The RBF4MNH was rolled out at the facility level.
Initially, 18 non-randomly selected EmOC facilities (4
hospitals, 14 health centres) across the 4 districts received
RBF (ie, intervention phase 1). In October 2014, 15 addi-
tional EmOC facilities (3 hospitals, 12 health centres)
were added within the same districts (ie, intervention
phase 2). Facilities received performance payments
in addition to their usual budget allocation. As part of
RBF, most facilities also benefited from upfront invest-
ments in minor infrastructure repair or essential equip-
ment procurement (eg, renovation of labour rooms,
purchase of disinfectants, replacement of blood pressure
machines). Previous research related to the RBF4MNH
demonstrated positive effects of the programme on clin-
ical performance and supply chain management,16 an
overall positive but statistically non-significant impact on
effective coverage of pregnant women with obstetric care
services,17 a significant improvement in the timelines of
care-seeking for women with pregnancy-related compli-
cations,18 and no evidence for the erosion of overall
intrinsic health worker motivation.19
Study design and outcome variable
Our study adopted a quasi-experimental approach based
on an interrupted time series (ITS) design with inde-
pendent controls.20 We used monthly data on the number
of direct infacility maternal deaths (ie, occurred during
intrapartum or early postpartum period) and deliveries
(ie, excluding abortions and miscarriages) reported by
obstetric care facilities into the District Health Informa-
tion System version 2 (DHIS-2)-based national health
management information system and computed facili-
ty-based maternal mortality at time of birth as the outcome
variable. Beyond the lack of reliable population-based
maternal mortality data in our study setting, we preferred
this facility-based outcome because it better reflects the
RBF4MNH theory of change, which targeted specifically
effective childbirth care coverage at time of birth. Our
outcome variable, facility-specific maternal mortality at
birth per month, was calculated as the following:
Number of maternal deaths at birth per month in facility
Number of deliveries per month in facility ×
100, 000
on 22 June 2019 by guest. Protected by copyright.http://gh.bmj.com/BMJ Glob Health: first published as 10.1136/bmjgh-2018-001184 on 22 June 2019. Downloaded from

De AllegriM, etal. BMJ Glob Health 2019;4:e001184. doi:10.1136/bmjgh-2018-001184 3
BMJ Global Health
To estimate the impact of the RBF4MNH on facility-based
maternal deaths at birth, we compared monthly mortality
ratios over consecutive time points between all obstetric
care facilities in the 4 intervention districts (Balaka,
Dedza, Mchinji, Ntcheu) and all obstetric care facilities in
19 out of the country’s remaining 25 districts as controls
(district of ‘Nkhata Bay and Likoma’ treated as two sepa-
rate districts). We excluded six control districts due to
lack of a priori comparability: the four urban districts of
Blantyre, Lilongwe, Mzuzu and Zomba, and the island
districts of Likoma and Mwanza. Based on the 2015/2016
Democratic Health Survey, averages across districts for
both use of facility-based delivery services and accessi-
bility of skilled birth attendants are comparable between
intervention and control districts (93.5% vs 93.7% and
90.5% vs 90.3%, respectively).11
We decided to compare estimates aggregated at the
district rather than at the facility level for two reasons:
first, to account for the substantially higher number
of deaths reported by hospitals compared with health
centres, as risk profiles inevitably differ across levels of
care; second, to account for the fact that RBF4MNH
performance contracts in the four intervention districts
also targeted each DHMT, linking incentives to quality of
service delivery in the districts at large. Given this partic-
ular intervention feature, we postulated the existence of
a district effect due to spillover to non-RBF facilities.
Data extraction and cleaning
For each facility, monthly data points were extracted
from the Health Management Information System (HMIS) for
a total period of 57 consecutive months starting July 2012
(ie, 9 months before RBF4MNH launch) and ending
March 2017 (ie, 48 months after RBF4MNH launch).
During data preparation, we omitted all data points
that were of irretrievably poor quality (eg, number of
maternal deaths reported higher than number of deliv-
eries), three individual data points judged as outliers (ie,
extremely high numbers of maternal deaths observed in
two control districts during the preintervention period)
and single facilities with reported numbers of deliveries
missing for more than 40% of time points. The propor-
tion of omitted facilities was higher in the control (39%)
compared with the intervention districts (18%). We
further conducted sensitivity analyses comparing how
different data cleaning decisions might have affected the
resulting estimates, and found that results only very mini-
mally differed (data not shown) and thus not affected the
overall findings of the study as reported here.
Data analysis
We used multiple-group segmented linear regression to
analyse the ITS21 comparing maternal mortality at birth
between intervention and control districts, and between
the preintervention (July 2012 until March 2013), early
postintervention (April 2013 until September 2014) and
late postintervention (October 2014 until March 2017)
periods, according to the following model:
y
t
=β
0
+β
1
T
t
+β
2
z+β
3
zT
t
+β
4
x
1
+β5x1Tt+β6zx1+β7zx1Tt+β8x
2
+β9x2Tt+β10zx2+β11 zx2Tt+εt,
where yt represents the maternal mortality outcome vari-
able measured at each monthly time point t, Tt a contin-
uous variable representing the months since observation
start, x1 and x2 dummy variables representing each study
period (x1=0 for t in the preintervention period; x1=1
for t in the early and late postintervention periods; x2=0
for t in the preintervention and early postintervention
periods; x2=1 for t in the late postintervention period),
and z a dummy variable representing the treatment
group (0=control, 1=RBF). In this model,
β2
and
β3
indi-
cate the estimated differences in level (intercept) and
slope (trend), respectively, of maternal mortality between
treated and controls prior to the intervention;
β6
and
β7
represent the estimated difference-in-differences in level
and slope, respectively, attributable to the intervention
during the early intervention period; and
β10
and
β11
represent the estimated difference-in-differences in level
and slope, respectively, attributable to the intervention
during the late intervention period.
We defined two interruption points in our analysis
to reflect the beginning of the two RBF4MNH inter-
vention phases in April 2014 and October 2014. Our
model estimates the respective coefficients by Ordinary
Least Squares (OLS) regression using Newey-West SEs to
handle autocorrelation and potential heteroskedasticity.
The Cumby-Huizinga test for autocorrelation22 demon-
strated the presence of serial autocorrelation up to a
lag of 1; hence, we adjusted the model accordingly. In
two separate sensitivity analyses (see online supplemen-
tary appendix), we adjusted the model to account for
seasonality (hypothesised to affect labour patterns due
to climate variability), and we estimated a more parsimo-
nious model based on district matching. Stata V.14.2 was
used for all analyses.
Patient and public involvement statement
This study did not involve any patients.
RESULTS
Sample characteristics are shown in table 1. The 23
districts (4 intervention and 19 controls) contained a
total of 456 health facilities offering obstetric care services
for which HMIS data were available for more than 40%
of observation points. Over the 57-month study period, a
total of 23 964 complete observation points were included
in the analysis. The average number of deliveries per
month differed significantly between groups (p<0.01),
but reported mortality rates were statistically not signif-
icantly different between groups and thus comparable
during the preintervention period.
For the entire time series, the distribution of monthly
observations by intervention and control districts is shown
in the scatterplot in figure 1. Regression lines depict the
predicted values of maternal mortality for each period.
on 22 June 2019 by guest. Protected by copyright.http://gh.bmj.com/BMJ Glob Health: first published as 10.1136/bmjgh-2018-001184 on 22 June 2019. Downloaded from

4De AllegriM, etal. BMJ Glob Health 2019;4:e001184. doi:10.1136/bmjgh-2018-001184
BMJ Global Health
Table 1 Sample distribution and sample characteristics
Characteristics Intervention Control Total
Total number of districts 4 19 23
Total number of health facilities 63 245 308
Total number of complete observations across
all included facilities (entire study period)*
4948 18 980 23 964
Mean (SD) and median of monthly number of
facility-based deliveries (entire study period)
65.4 (104.3)†, 37 61.4 (91.5)†, 34 62.2 (94.3), 35
Mean (SD) of monthly facility-based maternal
deaths per 100 000 facility-based deliveries
Preintervention period 158.0 (54.0) 120.7 (22.5) 139.3 (44.5)
Postintervention period 1 158.4 (53.3)† 103.7 (21.8)† 131.0 (81.1)
Postintervention period 1 123.5 (66.0)† 83.5 (24.9)† 103.5 (53.4)
*Complete information on both indicators (ie, number of monthly facility-based deliveries and monthly facility-based direct maternal deaths)
feeding into outcome indicator for the entire study period.
†Difference in means statistically signicant at 0.05 level (based on two-group t-test).
0 100 200 300
Maternal deaths at birth per 100,000 deliveries
Jul 2012
Aug 2012
Sep 2012
Oct 2012
Nov 2012
Dec 2012
Jan 2013
Feb 2013
Mar 2013
Apr 2013
May 2013
Jun 2013
Jul 2013
Aug 2013
Sep 2013
Oct 2013
Nov 2013
Dec 2013
Jan 2014
Feb 2014
Mar 2014
Apr 2014
May 2014
Jun 2014
Jul 2014
Aug 2014
Sep 2014
Oct 2014
Nov 2014
Dec 2014
Jan 2015
Feb 2015
Mar 2015
Apr 2015
May 2015
Jun 2015
Jul 2015
Aug 2015
Sep 2015
Oct 2015
Nov 2015
Dec 2015
Jan 2016
Feb 2016
Mar 2016
Apr 2016
May 2016
Jun 2016
Jul 2016
Aug 2016
Sep 2016
Oct 2016
Nov 2016
Dec 2016
Jan 2017
Feb 2017
Mar 2017
Months
Intervention districts: Actual Predicted
Control districts: Actual Predicted
Regression with Newey−West standard errors − lag(1)
Figure 1 Time trends of facility-based maternal mortality by district and months. Dots represent maternal mortality ratios
averaged across facilities within each study arm (ie, intervention vs control); lines represent predicted maternal mortality ratio
trends for each period based on linear regression.
Dashed vertical lines indicate the two interruption points.
Intervention and control districts experienced similar
declines in mortality levels and slopes between prein-
tervention and first postintervention periods. For both
groups, mortality trends (slopes) in the first postinterven-
tion period were only slightly lower than the estimated
mortality levels at the end of the postintervention period.
Going from the end of the first to the beginning of the
second postintervention period, neither intervention
nor control districts experienced a statistically significant
drop in estimated mortality rates. However, the inter-
vention districts experienced a marginally significantly
greater decline in maternal mortality over the course of
the later compared with the earlier intervention period.
Table 2 displays the results from the controlled ITS
model. Maternal mortality at observation start (ie, July
2012) was estimated at 111.4 deaths/100 000 facility-based
deliveries for the control and 134.8 deaths/100 000 facil-
ity-based deliveries for the intervention districts. During
the 9-month preintervention period, maternal mortality
on 22 June 2019 by guest. Protected by copyright.http://gh.bmj.com/BMJ Glob Health: first published as 10.1136/bmjgh-2018-001184 on 22 June 2019. Downloaded from

De AllegriM, etal. BMJ Glob Health 2019;4:e001184. doi:10.1136/bmjgh-2018-001184 5
BMJ Global Health
Table 2 Effect of the RBF4MNH on facility-based maternal
mortality
Estimated maternal deaths
per 100 000 facility-based
deliveries (95% CI)
Preintervention period
Control level (July 2012) 111.4 (100.8 to
122.0)*
Difference in levels,
intervention vs control
(July 2012)
23.4 (−37.6 to 84.5)
Control monthly trend 2.3 (−2.0 to 6.6)
Difference of intervention
vs control in trend change
3.5 (−12.2 to 19.1)
Effects related to phase 1
(postintervention period 1)
Control level change −29.8 (−63.9 to 4.2)*
Difference of intervention
vs control in level change
−28.9 (−119.4 to 61.7)
Control monthly trend
change
−2.2 (−6.7 to 2.4)
Difference of intervention
vs control in trend change
−0.1 (−17.8 to 17.7)
Effects related to phase 2
(postintervention period 2)
Control level change −10.8 (−34.7 to 13.1)
Difference of intervention
vs control in level change
−26.9 (−91.0 to 37.1)
Control monthly trend
change
−0.9 (−2.4 to 0.6)
Difference of intervention
vs control in trend change
−4.8 (−10.3 to 0.7)*
Estimates based on interrupted time-series analysis.
*P<0.1
RBF4MNH, Results-Based Financing For Maternal and Newborn
Health Initiative.
increased by 2.3 deaths/100 000 facility-based deliveries
per month in the control and by 5.8 deaths/100 000
facility-based deliveries per month in the intervention
districts. The differences between levels and trends were
statistically not different, indicating that control and
intervention districts were sufficiently comparable prior
to the RBF4MNH intervention. During the first interven-
tion period, we observed a slope reduction attributable
to the RBF4MNH of 0.1 fewer maternal deaths/100 000
facility-based deliveries per month with an immediate
reduction in maternal mortality levels (comparing the
end of the preintervention with the beginning of the first
postintervention period) attributable to the RBF4MNH
of 28.9 deaths/100 000 facility-based deliveries. These
effects, however, are not statistically significant. During
the second postintervention period, we observed a
marginally significant negative trend effect of 4.8 fewer
deaths/100 000 facility-based deliveries per month
attributable to the RBF4MNH, coupled with a statisti-
cally non-significant immediate reduction in maternal
mortality attributable to the RBF4MNH of 26.9 deaths for
every 100 000 facility-based deliveries. The results of the
sensitivity analyses confirm the patterns observed in the
primary analysis.
DISCUSSION
Statement of principal ndings
Our study makes a unique contribution to the existing
literature being the first to assess the impact of a
combined supply-side and demand-side RBF interven-
tion on facility-based maternal mortality at birth. The
significant reduction by 4.8 deaths/100 000 deliveries
(CI −10.3 to 0.7, p<0.1) per month attributable to the
RBF4MNH intervention is remarkable considering that
the intervention had been operative for only 4 years at
the time of evaluation.
Strengths and weaknesses of the study
With under-reporting of maternal deaths being likely in
both intervention and control facilities, our mortality
estimates are probably rather conservative. Throughout
the study period, monthly average ratios of facility-based
maternal mortality were higher and more fluctuating
in the intervention compared with the control districts
(figure 1). In fact, the numbers of birth-related deaths
varied greatly for any given facility when measured
monthly. This fluctuation was more pronounced in the
four intervention districts given their smaller sample size
compared with the control arm. The higher mortality in
the four intervention districts might be explained by the
non-random selection of the RBF4MNH districts,14 and
the RBF4MNH incentives to improve HMIS reporting,
perhaps reducing previous under-reporting of birth-re-
lated maternal deaths in the intervention facilities, might
explain the higher mortality observed in the four inter-
vention districts.15
The similarity in trends and levels in both study groups
during the preintervention and first postinterven-
tion periods likely demonstrates a pre-existing general
decline in maternal mortality that continued far into the
initial programme phase (April 2013–September 2014).
This could be an indication of both the programme’s
limited capacity to produce any measurable effects in
its early phase and the existence of a nationwide secular
trend. In fact, early intervention was characterised by
several adjustments to the initial design, eventually
improving the programme’s operational capacity prior
to expansion.15 23 Coexistence of many independent
MNH programmes across Malawi during the pre-2015
period could explain the presence of a secular trend.24
Also, given the relatively high mortality rates observed in
the intervention districts, we cannot determine to what
extent the observed effect size would have been different
in scenarios with higher or lower baseline levels prior to
intervention start.
on 22 June 2019 by guest. Protected by copyright.http://gh.bmj.com/BMJ Glob Health: first published as 10.1136/bmjgh-2018-001184 on 22 June 2019. Downloaded from

6De AllegriM, etal. BMJ Glob Health 2019;4:e001184. doi:10.1136/bmjgh-2018-001184
BMJ Global Health
Strengths and weaknesses in relation to other studies
The size of the impact on mortality identified in this
study was surprising given our prior analyses of the
programme’s effect on intermediate outcomes based on a
controlled pre–post test design using primary data. While
we found significant improvements in equipment main-
tenance and selected drugs and consumables (ranging
between 9% and 52% point increases for selected items),
RBF4MNH effects were less conclusive or less extensive
in terms of birth attendants’ adherence to obstetric treat-
ment protocols (for instance, non-significant increases
between 8% and 21% points in activities related to infec-
tion prevention, accompanied by decreases between
18% and 46% points for activities related to postpartum
haemorrhage prevention)16 and effective childbirth care
coverage (increase by 7.1% points with a p=0.07 in effec-
tive coverage).17
Three factors may explain this discrepancy. First, our
prior analyses used endline data collected only 2 years
after the RBF4MNH launch compared with 48 months in
this ITS analysis. This analysis confirms that RBF4MNH
gains were not yet realised in the 2 years after programme
launch and were mainly accrued later once the interven-
tion had reached its full operational capacity. Second,
the RBF4MNH might have produced changes in service
quality early on that our previous studies failed to capture,
and those early changes led to remarkable reduction in
maternal mortality later on. Third, with 62% of maternal
deaths in Malawi occurring during the early postpartum
period,25 the combined demand-side and supply-side
effect of the RBF4MNH encouraging both women and
providers to remain at facilities for the 48-hour post-
partum observation period15 likely removed pre-existing
delays in postpartum care-seeking (not focus of our prior
work).26
Meaning of the study
The observed reduction in maternal mortality is highly
relevant from a policy point of view. Although remarkable
reductions,27 Malawi continues to experience high rates
of maternal mortality.11 About 71% of maternal deaths in
Malawi occur around the time of birth and 63% among
women who delivered in a facility.25 A recent survey indi-
cated that 62% of maternal deaths occurred at health
facilities and an additional 21% among mothers who
just returned home after delivering in a facility.26 Hence,
reducing facility-based maternal deaths at birth by acting to
improve quality of service delivery and extending women’s
in-hospital stays is likely to bear an important impact on the
country’s overall maternal mortality, considering Malawi’s
situation with over 90% of women giving birth at a facility11
in the context of poor obstetric care quality.28 29 However,
we cannot fully appraise the mortality reduction produced
by the RBF4MNH in relation to other maternal care inter-
ventions due to the current lack of comparable studies.
Unanswered questions and future research
Our study inevitably suffers from a number of limita-
tions. First, reliance on HMIS data implied that overall
reductions in population-based maternal mortality could
not be estimated. Given the crucial role quality obstetric
care plays in shaping maternal health outcomes beyond
the early postpartum period, it is plausible to assume that
the RBF4MNH is likely to have produced broader impacts
on overall maternal mortality. Further research relying
on other data sources is needed to test this hypothesis.
Second, due to extremely poor quality of HMIS data
with extreme proportions of missing values on newborn
outcomes, we were unable to assess the impact of the
RBF4MNH on neonatal mortality. While the DHIS-2 plat-
form likely contributed to improved HMIS data quality
in Malawi (especially for Millennium Development
Goal-relevant indicators, such as facility-based deliveries
and related direct maternal deaths),30 31 and although
the assuring findings of our sensitivity analysis regarding
our data cleaning approach, the fact that we still had to
exclude single facilities due to poor data quality might
have biased our findings. This is unfortunate given that
improving delivery and early neonatal care is likely to
bear a more visible impact on neonatal than maternal
mortality.32
Third, while the quasi-experimental application of
the ITS allowed us to establish causality between the
RBF4MNH and maternal mortality, additional non-ob-
served confounders (eg, maternal health programmes
with local or regional effects) might have biased our
estimates. To our knowledge, however, the only other
large programme likely to have produced changes in
health system structures capable of inducing changes in
maternal mortality is the Support for Service Delivery
Integration, a United States Agency for International
Development-funded programme implemented in
parallel to the RBF4MNH,33 which we think is unlikely to
have shaped results since it was implemented in one of
four RBF districts but in 14 controls. If so, our analysis is
likely to have produced lower bound estimates of the true
effect of the RBF4MNH Initiative.
Fourth, in spite of the observed completeness of
HMIS data on maternal deaths, we need to acknowledge
the possibility that providers may under-report deaths.
Again, however, such under-reporting does not invali-
date our analysis, since we have no reason to imagine that
under-reporting differs systematically between interven-
tion and control facilities and/or districts.
Last, we need to acknowledge the limited generalis-
ability of our findings to other RBF settings. Unlike most
other RBF programmes where payments linked to quan-
tity aspects of service delivery dominate,34 the RBF4MNH
kept a stronger focus on payments linked to quality of
care processes, such as drug and supply procurement,
equipment maintenance, routine death audits and
selected aspects of clinical case management. We there-
fore need to caution the reader when extrapolating our
results to other RBF settings.
Acknowledgements The authors are grateful to the Ministry of Health of Malawi,
in particular the Reproductive Health Unit and its leader Fannie Kachale for the
support received in accessing data when rst expressing interest in conducting
on 22 June 2019 by guest. Protected by copyright.http://gh.bmj.com/BMJ Glob Health: first published as 10.1136/bmjgh-2018-001184 on 22 June 2019. Downloaded from

De AllegriM, etal. BMJ Glob Health 2019;4:e001184. doi:10.1136/bmjgh-2018-001184 7
BMJ Global Health
this analysis. The authors are also indebted towards the staff at the RBF4MNH
Implementation Unit and Options Consultancy Services, in particular Matthew
Nviiri, Mabvuto Mndau, Brigitte Jordan and Corinne Grainger, and towards the
KfW staff, in particular Kai Gesing, for facilitating the study and assisting with the
interpretation of the ndings.
Contributors SB, MDA and RPC made substantial contributions to the conception
and design of the study. SB, MDA, RPC and JL made substantial contributions
to the acquisition and analysis of the data for this study. All authors contributed
equally to the interpretation of the data for this study. SB, MDA and RPC drafted
the initial manuscript. All authors equally contributed to critically revising this draft
for important intellectual content. All authors approved the nal version of the
manuscript to be published. All authors are accountable for all aspects of the work
in ensuring that questions related to the accuracy or integrity of any part of the
work are appropriately investigated and resolved. The corresponding author attests
that all listed authors meet the authorship criteria and that no others meeting the
criteria have been omitted.
Funding The analysis presented in this manuscript was conducted by the authors
beyond the scope of the impact evaluation based on primary data collection
funded jointly by the USAID/TRAction (cooperative agreement number GHS-A-
00-09- 00015-00) and the Royal Norwegian Embassy in Malawi (Programme
Title MWI 12/0010). The authors devoted own time towards analysis and writing
while employed as staff at the Heidelberger Institut für Global Health, Heidelberg
University, or at their respective institutions.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The study obtained ethical approval from the Faculty of Medicine
of Heidelberg University as part of the overall RBF4MNH impact evaluation protocol
(protocol number S-256/2012) and approval for data use from the Malawi Ministry
of Health.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Requests for access to data should be addressed to
the corresponding author.
Open access This is an open access article distributed in accordance with the
Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which
permits others to distribute, remix, adapt, build upon this work non-commercially,
and license their derivative works on different terms, provided the original work is
properly cited, appropriate credit is given, any changes made indicated, and the
use is non-commercial. See:http:// creativecommons. org/ licenses/ by- nc/ 4. 0/.
REFERENCES
1. World Health Organization. Trends in maternal mortality: 1990 to
2015. Estimates by who, UNICEF, UNFPA, world bank group, and
the United Nations population division, 2015.
2. Graham W, Woodd S, Byass P, etal. Diversity and divergence:
the dynamic burden of poor maternal health. The Lancet
2016;388:2164–75.
3. Kyei-Nimakoh M, Carolan-Olah M, McCann TV. Access barriers to
obstetric care at health facilities in sub-Saharan Africa—a systematic
review. Syst Rev 2017;6.
4. Meessen B, Soucat A, Sekabaraga C, etal. Performance-based
nancing: Just a donor fad or a catalyst towards comprehensive
health-care reform? Bull World Health Organ 2011;89:153–6.
5. Savedoff WD. Basic economics of Results-Based nancing in health.
Bath, Maine: : Bath, Maine: Social Insight, 2010.
6. Witter S, Fretheim A, Kessy FL, etal. Paying for performance to
improve the delivery of health interventions in low- and middle-
income countries. Cochrane Database Syst Rev 2012;2.
7. Das A, Gopalan SS, Chandramohan D. Effect of pay for performance
to improve quality of maternal and child care in low- and middle-
income countries: a systematic review. BMC Public Health 2016;16.
8. Lim SS, Dandona L, Hoisington JA, etal. India's Janani Suraksha
Yojana, a conditional cash transfer programme to increase births in
health facilities: an impact evaluation. The Lancet 2010;375:2009–23.
9. Powell-Jackson T, Neupane BD, Tiwari S, etal. The impact of
Nepal's National incentive programme to promote safe delivery
in the District of Makwanpur. Adv Health Econ Health Serv Res
2009;21:221–49.
10. Bowser D, Gupta J, Nandakumar A. The effect of Demand- and
Supply-side health nancing on infant, Child, and maternal mortality
in low- and middle-income countries. Health Systems & Reform
2016;2:147–59.
11. National statistical ofce Malawi. Malawi demographic and Health
survey 2015-16. Zomba, Malawi, 2017.
12. Ministry of Health Malawi. Malawi health sector strategic plan
2011-2016. moving towards equity and quality. Lilongwe: Ministry of
Health (Malawi), 2011.
13. Ministry of health Malawi. Malawi 2014 emergency obstetric and
newborn care needs assessment. Ministry of Health Malawi, 2015.
14. Ministry of health Malawi. Inception report results based nancing
for maternal and neonatal health (RBF4MNH initiative) 2012-2014.
Lilongwe, Malawi/London, United Kingdom: : RBF4MNH Options
Ofce 2012.
15. White-Kaba M. How rewards improve health practice in Malawi:
Learnings from a maternal and newborn health Initiative. Bonn,
Germany: Federal Ministry for Economic Cooperation and
Development (BMZ), 2017.
16. Brenner S, Wilhelm D, Lohmann J, etal. Implementation research to
improve quality of maternal and newborn health care, Malawi. Bull
World Health Organ 2017;95:491–502.
17. Brenner S, Mazalale J, Wilhelm D, etal. Impact of results-based
nancing on effective obstetric care coverage: evidence from a
quasi-experimental study in Malawi. BMC Health Serv Res 2018;18.
18. Chinkhumba J, De Allegri M, Mazalale J, etal. Household costs and
time to seek care for pregnancy related complications: the role of
results-based nancing. Plos One 2017;12:e0182326.
19. Lohmann J, Muula AS, Houlfort N, etal. How does Performance-
based nancing affect health workers' intrinsic motivation? A Self-
Determination Theory-based mixed-methods study in Malawi. Social
Science & Medicine 2018;208:1–8.
20. Wagenaar BH, Sherr K, Fernandes Q, etal. Using routine health
information systems for well-designed health evaluations in low- and
middle-income countries. Health Policy Plan. 2016;31:129–35.
21. Linden A. Conducting interrupted time-series analysis for single- and
multiple-group comparisons. The Stata Journal 2015;15:480–500.
22. Cumby RE, Huizinga J. Testing the autocorrelation structure of
disturbances in ordinary least squares and instrumental variables
regressions. Econometrica 1992;60:185–95.
23. Wilhelm DJ, Brenner S, Muula AS, etal. A qualitative study
assessing the acceptability and adoption of implementing a results
based nancing intervention to improve maternal and neonatal
health in Malawi. BMC Health Serv Res 2016;16.
24. WHO. Who country cooperation strategy, 2008-2013, Malawi.
Brazzaville, Republic of Congo: WHO Regional Ofce for Africa,
2009.
25. Mataya R. Report on the Condential enquiry into maternal deaths in
Malawi (2008-2012. Ministry of Health (Malawi).
26. Mgawadere F, Unkels R, Kazembe A, etal. Factors associated with
maternal mortality in Malawi: application of the three delays model.
BMC Pregnancy Childbirth 2017;17.
27. Colbourn T, Lewycka S, Nambiar B, etal. Maternal mortality in
Malawi, 1977–2012. BMJ Open 2013;3:e004150.
28. Gunawardena N, Bishwajit G, Yaya S. Facility-Based maternal death
in Western Africa: a systematic review. Front. Public Health 2018;6.
29. Kongnyuy EJ, Mlava G, van den Broek N. Facility-based maternal
death review in three districts in the central region of Malawi: an
analysis of causes and characteristics of maternal deaths. Womens
Health Issues 2009;19:14–20.
30. Perin N, Anderson R, Mwanyika H, etal. The health information
systems programme, nal report. Seattle, USA: PATH, 2011.
31. Braa J, Sahay S. The DHIS2 open source software platform:
evolution over time and space. In: Global health informatics.
Principles of eHealth and mHealth to improve quality of care. The
MIT Press, 2017.
32. Chinkhumba J, De Allegri M, Muula AS, etal. Maternal and perinatal
mortality by place of delivery in sub-Saharan Africa: a meta-analysis
of population-based cohort studies. BMC Public Health 2014;14.
33. Senlet P, Kachiza C, Katekaine J, etal. USAID/Malawi support for
service delivery – integration performance evaluation. Lilongwe,
Malawi, and Vienna, VA, USA: IBTCI, 2014.
34. Renmans D, Holvoet N, Criel B. Combining Theory-Driven Evaluation
and Causal Loop Diagramming for Opening the 'Black Box' of an
Intervention in the Health Sector: A Case of Performance-Based
Financing in Western Uganda. Int J Environ Res Public Health
2017;14. doi:10.3390/ijerph14091007
on 22 June 2019 by guest. Protected by copyright.http://gh.bmj.com/BMJ Glob Health: first published as 10.1136/bmjgh-2018-001184 on 22 June 2019. Downloaded from