Figure - available from: Implementation Science
This content is subject to copyright. Terms and conditions apply.
![Mozambique with focal provinces of Sofala and Manica outlined in red. Figure sourced from Fernandes QF et al. (2014) [19]](publication/361117878/figure/fig1/AS:1164160809345037@1654568974895/Mozambique-with-focal-provinces-of-Sofala-and-Manica-outlined-in-red-Figure-sourced-from.png)
Mozambique with focal provinces of Sofala and Manica outlined in red. Figure sourced from Fernandes QF et al. (2014) [19]
Source publication
Background
Significant investments are being made to close the mental health (MH) treatment gap, which often exceeds 90% in many low- and middle-income countries (LMICs). However, limited attention has been paid to patient quality of care in nascent and evolving LMIC MH systems. In system assessments across sub-Saharan Africa, MH loss-to-follow-up...
Similar publications
Objective The aim of the study is to identify and prioritize early intervention (EI) stakeholders' perspectives of supports and barriers to implementing the Young Children's Participation and Environment Measure (YC-PEM), an electronic patient-reported outcome (e-PRO) tool, for scaling its implementation across multiple local and state EI programs....
Citations
... Eligible facilities were public sector Ministry of Health clinics with ≥100 average annual outpatient mental health consultations from 2019 to 2020 and that had a minimum of one psychiatric technician and one psychologist allocated by the government. 10 Data collection and management This study included a census of all patients presenting for intake or follow-up psychiatric care at 16 health facilities from 4 February 2022 to 19 September 2022, as a part of the baseline data collection period for a cluster RCT study to evaluate an implementation strategy (SAIA-MH) to optimise the MNS care cascade. 10 Primary and secondary MH diagnoses were recorded using the International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10), which is the standard practice by the Mozambican Ministry of Health. ...
... 10 Data collection and management This study included a census of all patients presenting for intake or follow-up psychiatric care at 16 health facilities from 4 February 2022 to 19 September 2022, as a part of the baseline data collection period for a cluster RCT study to evaluate an implementation strategy (SAIA-MH) to optimise the MNS care cascade. 10 Primary and secondary MH diagnoses were recorded using the International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10), which is the standard practice by the Mozambican Ministry of Health. Psychiatric technicians at all 16 facilities attended a 2-day training on using an enhanced patient registry, patient charting tools and the WHODAS 2.0. ...
... More information is available in the clinical trial protocol. 10 Facility staff received monthly supervision visits to answer questions and evaluate compliance with documentation procedures. Hard copy patient charts were completed by psychiatric technicians during routine service delivery and then digitised by study research assistants during monthly clinic visits using CommCare. ...
... The most common diagnosis was generalized anxiety disorder followed by major depressive disorder [47]. A vast majority of these cases go undiagnosed as the current mental healthcare system does not currently have sufficient capacity, but efforts to expand mental healthcare systems in Mozambique are currently ongoing [59,60]. ...
Drinking water access and water and food insecurity have been linked to mental well-being, but few studies have comprehensively assessed potential pathways linking these associations. Understanding these mediation pathways is particularly important among pregnant women, as prenatal stress and poor mental well-being have been shown to negatively impact fetal development. In this study, we address this gap by analyzing the relationships between drinking water source and water and food insecurity with mental well-being amongst pregnant women living in low-income, urban neighborhoods of Beira, Mozambique. Data for this cross-sectional analysis were collected among third-trimester, pregnant women (n = 740) from February 2021 through October 2022 as part of a matched cohort study. Validated, cross-cultural measures of mental well-being and household water and food insecurity were administered in the survey. Drinking water source was determined by presence of a household drinking water source on-premises. We used logistic regression to characterize the associations between drinking water source, water and food insecurity, and mental well-being and causal mediation analysis to determine mediation by food and water insecurity along these pathways. We found evidence that water insecurity (OR 1.44; 95%CI 1.02, 2.02) and food insecurity (OR 2.27; 95%CI 1.57, 3.34) were individually associated with adverse mental well-being. Drinking water source was not associated with mental well-being (OR 1.00; 95%CI 0.71, 1.39), water insecurity (OR 0.86; 95%CI 0.60, 1.24), or food insecurity (OR 1.02; 95%CI 0.71, 1.47). Food insecurity may also mediate the relationship between water insecurity and mental well-being (ACME 0.05; 95%CI 0.02, 0.07; ADE 0.04; 95%CI -0.04, 0.13). Our findings support growing literature that water and food insecurity are important to mental well-being, a key aspect of overall health. Further research is needed to confirm causality along these pathways and determine specific mechanisms through which these interactions take place.
... During initial testing, SAIA was applied to optimize the prevention of mother-tochild transmission of HIV in three sub-Saharan African countries and demonstrated dramatic cascade improvements, as well as high penetration, acceptability, and feasible integration into routine service management activities [33,34]. SAIA has been adapted and applied to multiple care cascades in the USA and multiple countries in sub-Saharan Africa including but not limited to pediatric HIV testing [35], HIV testing in family planning services [36], cervical cancer screening and prevention [37], severe mental illness diagnosis and management [38], malaria diagnosis and treatment, and Naloxone distribution [39]. SAIA's application to the hypertension cascade among PLHIV was assessed in a recent cluster randomized trial in central Mozambique (SAIA-HTN: R01HL142412, NCT04088656). ...
Background
Undiagnosed and untreated hypertension is a main driver of cardiovascular disease and disproportionately affects persons living with HIV (PLHIV) in low- and middle-income countries. Across sub-Saharan Africa, guideline application to screen and manage hypertension among PLHIV is inconsistent due to poor service readiness, low health worker motivation, and limited integration of hypertension screening and management within HIV care services. In Mozambique, where the adult HIV prevalence is over 13%, an estimated 39% of adults have hypertension. As the only scaled chronic care service in the county, the HIV treatment platform presents an opportunity to standardize and scale hypertension care services.
Low-cost, multi-component systems-level strategies such as the Systems Analysis and Improvement Approach (SAIA) have been found effective at integrating hypertension and HIV services to improve the effectiveness of hypertension care delivery for PLHIV, reduce drop-offs in care, and improve service quality. To build off lessons learned from a recently completed cluster randomized trial (SAIA-HTN) and establish a robust evidence base on the effectiveness of SAIA at scale, we evaluated a scaled-delivery model of SAIA (SCALE SAIA-HTN) using existing district health management structures to facilitate SAIA across six districts of Maputo Province, Mozambique.
Methods
This study employs a stepped-wedge design with randomization at the district level. The SAIA strategy will be “scaled up” with delivery by district health supervisors (rather than research staff) and will be “scaled out” via expansion to Southern Mozambique, to 18 facilities across six districts in Maputo Province. SCALE SAIA-HTN will be introduced over three, 9-month waves of intensive intervention, where technical support will be provided to facilities and district managers by study team members from the Mozambican National Institute of Health. Our evaluation of SCALE SAIA-HTN will be guided by the RE-AIM framework and will seek to estimate the budget impact from the payer’s perspective.
Discussion
SAIA packages user-friendly systems engineering tools to support decision-making by frontline health workers and to identify low-cost, contextually relevant improvement strategies. By integrating SAIA delivery into routine management structures, this pragmatic trial will determine an effective strategy for national scale-up and inform program planning.
Trial registration
ClinicalTrials.gov NCT05002322 (registered 02/15/2023).
... Our current work aims to facilitate future adaptations while maintaining reproducibility. Specific work exploring mechanisms of action (and the relative contributions of individual components of SAIA) is underway and will build upon the generalizability of the SAIA, including through the use of longitudinal structural equation modeling [36]. ...
Background
Healthcare systems in low-resource settings need simple, low-cost interventions to improve services and address gaps in care. Though routine data provide opportunities to guide these efforts, frontline providers are rarely engaged in analyzing them for facility-level decision making. The Systems Analysis and Improvement Approach (SAIA) is an evidence-based, multi-component implementation strategy that engages providers in use of facility-level data to promote systems-level thinking and quality improvement (QI) efforts within multi-step care cascades. SAIA was originally developed to address HIV care in resource-limited settings but has since been adapted to a variety of clinical care systems including cervical cancer screening, mental health treatment, and hypertension management, among others; and across a variety of settings in sub-Saharan Africa and the USA. We aimed to extend the growing body of SAIA research by defining the core elements of SAIA using established specification approaches and thus improve reproducibility, guide future adaptations, and lay the groundwork to define its mechanisms of action.
Methods
Specification of the SAIA strategy was undertaken over 12 months by an expert panel of SAIA-researchers, implementing agents and stakeholders using a three-round, modified nominal group technique approach to match core SAIA components to the Expert Recommendations for Implementing Change (ERIC) list of distinct implementation strategies. Core implementation strategies were then specified according to Proctor’s recommendations for specifying and reporting, followed by synthesis of data on related implementation outcomes linked to the SAIA strategy across projects.
Results
Based on this review and clarification of the operational definitions of the components of the SAIA, the four components of SAIA were mapped to 13 ERIC strategies. SAIA strategy meetings encompassed external facilitation, organization of provider implementation meetings, and provision of ongoing consultation. Cascade analysis mapped to three ERIC strategies: facilitating relay of clinical data to providers, use of audit and feedback of routine data with healthcare teams, and modeling and simulation of change. Process mapping matched to local needs assessment, local consensus discussions and assessment of readiness and identification of barriers and facilitators. Finally, continuous quality improvement encompassed tailoring strategies, developing a formal implementation blueprint, cyclical tests of change, and purposefully re-examining the implementation process.
Conclusions
Specifying the components of SAIA provides improved conceptual clarity to enhance reproducibility for other researchers and practitioners interested in applying the SAIA across novel settings.
... The SAIA-SCALE study, testing SAIA for PMTCT services in Mozambique, measured fidelity through a tablet-based survey completed by clinic staff to track number of SAIA cycles, attendants at each cycle meeting, and the number, content, and results of micro-interventions tested [33]. Two additional studies in Mozambique using SAIA for hypertension and mental health services assessed fidelity as number and frequency of SAIA cycles completed [10] and adherence to the 5-step SAIA cycle [34]. All three studies then categorized clinics as high or low performing and used focus group data to assess features of successful implementation. ...
Background
In Kenya, HIV incidence is highest among reproductive-age women. A key HIV mitigation strategy is the integration of HIV testing and counseling (HTC) into family planning services, but successful integration remains problematic. We conducted a cluster-randomized trial using the Systems Analysis and Improvement Approach (SAIA) to identify and address bottlenecks in HTC integration in family planning clinics in Mombasa County, Kenya. This trial (1) assessed the efficacy of this approach and (2) examined if SAIA could be sustainably incorporated into the Department of Health Services (DOHS) programmatic activities. In Stage 1, SAIA was effective at increasing HTC uptake. Here, we present Stage 2, which assessed if SAIA delivery would be sustained when implemented by the Mombasa County DOHS and if high HTC performance would continue to be observed.
Methods
Twenty-four family planning clinics in Mombasa County were randomized to either the SAIA implementation strategy or standard care. In Stage 1, the study staff conducted all study activities. In Stage 2, we transitioned SAIA implementation to DOHS staff and compared HTC in the intervention versus control clinics 1-year post-transition. Study staff provided training and minimal support to DOHS implementers and collected quarterly HTC outcome data. Interviews were conducted with family planning clinic staff to assess barriers and facilitators to sustaining HTC delivery.
Results
Only 39% (56/144) of planned SAIA visits were completed, largely due to the COVID-19 pandemic and a prolonged healthcare worker strike. In the final study quarter, 81.6% (160/196) of new clients at intervention facilities received HIV counseling, compared to 22.4% (55/245) in control facilities (prevalence rate ratio [PRR]=3.64, 95% confidence interval [CI]=2.68–4.94). HIV testing was conducted with 60.5% (118/195) of new family planning clients in intervention clinics, compared to 18.8% (45/240) in control clinics (PRR=3.23, 95% CI=2.29–4.55). Interviews with family planning clinic staff suggested institutionalization contributed to sustained HTC delivery, facilitated by low implementation strategy complexity and continued oversight.
Conclusions
Intervention clinics demonstrated sustained improvement in HTC after SAIA was transitioned to DOHS leadership despite wide-scale healthcare disruptions and incomplete delivery of the implementation strategy. These findings suggest that system interventions may be sustained when integrated into DOHS programmatic activities.
Trial registration
ClinicalTrials.gov (NCT02994355) registered on 16 December 2016.
... Other systematic reviews highlight deficits in the quality of published mediation analyses in implementation science to date and have called for increased and improved use of the method [5,18]. Reflecting its growing importance within the field, mediation analyses feature prominently in several implementation research protocols published in the field's leading journal, Implementation Science, during the last year [19][20][21][22]. Chashin et al. [23] recently published guidance for reporting mediation analyses in implementation studies, including the importance of determining required sample sizes for mediation tests a priori. ...
Background
Statistical tests of mediation are important for advancing implementation science; however, little research has examined the sample sizes needed to detect mediation in 3-level designs (e.g., organization, provider, patient) that are common in implementation research. Using a generalizable Monte Carlo simulation method, this paper examines the sample sizes required to detect mediation in 3-level designs under a range of conditions plausible for implementation studies.
Method
Statistical power was estimated for 17,496 3-level mediation designs in which the independent variable (X) resided at the highest cluster level (e.g., organization), the mediator (M) resided at the intermediate nested level (e.g., provider), and the outcome (Y) resided at the lowest nested level (e.g., patient). Designs varied by sample size per level, intraclass correlation coefficients of M and Y, effect sizes of the two paths constituting the indirect (mediation) effect (i.e., X→M and M→Y), and size of the direct effect. Power estimates were generated for all designs using two statistical models—conventional linear multilevel modeling of manifest variables (MVM) and multilevel structural equation modeling (MSEM)—for both 1- and 2-sided hypothesis tests.
Results
For 2-sided tests, statistical power to detect mediation was sufficient (≥0.8) in only 463 designs (2.6%) estimated using MVM and 228 designs (1.3%) estimated using MSEM; the minimum number of highest-level units needed to achieve adequate power was 40; the minimum total sample size was 900 observations. For 1-sided tests, 808 designs (4.6%) estimated using MVM and 369 designs (2.1%) estimated using MSEM had adequate power; the minimum number of highest-level units was 20; the minimum total sample was 600. At least one large effect size for either the X→M or M→Y path was necessary to achieve adequate power across all conditions.
Conclusions
While our analysis has important limitations, results suggest many of the 3-level mediation designs that can realistically be conducted in implementation research lack statistical power to detect mediation of highest-level independent variables unless effect sizes are large and 40 or more highest-level units are enrolled. We suggest strategies to increase statistical power for multilevel mediation designs and innovations to improve the feasibility of mediation tests in implementation research.
... Our current work aims to facilitate future adaptations while maintaining reproducibility. Specific work exploring mechanisms of action (and the relative contributions of individual components of SAIA) is underway and will build upon the generalizability of the SAIA, including through the use of longitudinal structural equation modeling [36]. ...
Background: The Systems Analysis and Improvement Approach (SAIA) is an evidence-based, multi-component implementation strategy that engages service providers in the use of routinely-available service data to optimize service delivery cascades and promote systems-level thinking. SAIA was originally developed to address bottlenecks in HIV care in low-and middle-income countries, but has since been adapted and applied to a variety of care systems including: cervical cancer screening, mental health treatment, hypertension management, family planning, and community-based naloxone distribution. These projects have been implemented across a variety of settings in sub-Saharan Africa and the United States. Given the diversity of implementation experience, our consortium aimed to define the core elements of SAIA, to improve reproducibility, guide future adaptations, and lay the groundwork to evaluate mechanisms of action.
Methods: Specification of the SAIA strategy was undertaken over 12 months by an expert panel of SAIA researchers, implementing agents and stakeholders, using a three-round, modified nominal group technique approach to match core SAIA components to the Expert Recommendations for Implementing Change (ERIC) list of distinct implementation strategies. Core implementation strategies were then specified according to Proctor’s recommendation for specifying and reporting, followed by synthesis of data on related implementation outcomes linked to the SAIA strategy across projects.
Results: The four components of the SAIA strategy: (1) SAIA strategy meetings; (2) cascade analysis; (3) process mapping; and (4) continuous quality improvement, mapped to 13 distinct ERIC strategies. The SAIA strategy meetings component mapped to external facilitation, organization of provider implementation meetings, and provision of ongoing consultation. Cascade analysis mapped to facilitating relay of clinical data to providers, use of audit and feedback, and modelling and simulation of change. Process mapping tied to local needs assessment, local consensus discussions, and assessment of readiness and identification of barriers and facilitators. Continuous quality improvement encompassed tailoring strategies, developing a formal implementation blueprint, cyclical tests of change and purposefully re-examining the implementation process.
Conclusions: Formally specifying the core components of SAIA provides improved conceptual clarity to enhance reproducibility for other researchers and practitioners interested in applying the SAIA across novel settings. Furthermore, this work provides a structured framework to examine potential mechanisms of SAIA and its component implementation strategies.
Background
Following the conclusion of a stepped-wedge cluster randomized trial of the Systems Analysis and Improvement Approach (SAIA) to optimize the Prevention of Mother-to-Child HIV Transmission (PMTCT) cascade in Manica Province, Mozambique, we conducted a natural experiment to test the sustainability of the delivery model with limited financial inputs.
Methods
District nurse supervisors were encouraged to continue to facilitate SAIA cycles in subordinate health facilities and provided phone credit and tablet access to upload implementation data. No additional resources (e.g. funds for transport, refreshments, or supplies) were provided. Barriers to implementation were collected via conversations with district supervisors.
Results
Monthly facilitation of SAIA cycles continued in 11 out of 12 (92%) districts and 13 out of 36 (36%) facilities through 12 months post-trial, which declined to 10 districts and 10 facilities by the end of the 15-month post-trial period. Despite interest among district supervisors to continue implementation, logistical and financial barriers prevented visits to facilities not in close proximity to district management offices. Turnover of district supervisors resulted in replacements not having knowledge and experience facilitating SAIA. The lack of refreshments for facility staff and limited supplies (pens and paper) were cited as additional barriers.
Conclusion
Despite the scalability of the SAIA model, it is susceptible to implementation decay without sufficient health system resources. Additional research is needed to test sustainment strategies that address identified barriers and enable continued delivery of implementation strategy core components at a sufficient level of fidelity to maintain desired health system improvements and patient-level outcomes.
Background
Depression is a major contributor to morbidity and mortality in sub-Saharan Africa. Due to low system capacity, three in four patients with depression in sub-Saharan Africa go untreated. Despite this, little attention has been paid to the cost-effectiveness of implementation strategies to scale up evidence-based depression treatment in the region. In this study, we investigate the cost-effectiveness of two different implementation strategies to integrate the Friendship Bench approach and measurement-based care in non-communicable disease clinics in Malawi.
Methods
The two implementation strategies tested in this study are part of a trial, in which ten clinics were randomly assigned (1:1) to a basic implementation package consisting of an internal coordinator acting as a champion (IC-only group) or to an enhanced package that complemented the basic package with quarterly external supervision, and audit and feedback of intervention delivery (IC + ES group). We included material costs, training costs, costs related to project-wide meetings, transportation and medication costs, time costs related to internal champion activities and depression screening or treatment, and costs of external supervision visits if applicable. Outcomes included the number of patients screened with the patient health questionnaire 2 (PHQ-2), cases of remitted depression at 3 and 12 months, and disability-adjusted life-years (DALYs) averted. We compared the cost-effectiveness of both packages to the status quo (ie, no intervention) using a micro-costing-informed decision-tree model.
Findings
Relative to the status quo, IC + ES would be on average US1349–119 vs 210 for IC + ES; IC-only dominated by the status quo at 12 months). Neither package would be cost-effective under the willingness-to-pay threshold of $65 per DALY averted currently used by the Malawian Ministry of Health. However, the IC + ES package would be cost-effective in relation to the commonly used threshold of three times per-capita gross domestic product per DALY averted.
Interpretation
Investing in supporting champions might be an appropriate use of resources. Although not currently cost-effective by Malawian willingness-to-pay standards compared with the status quo, the IC + ES package would probably be a cost-effective way to build mental health-care capacity in resource-constrained settings in which decision makers use higher willingness-to-pay thresholds.
Background
The Systems Analysis and Improvement Approach (SAIA) is an evidence-based package of systems engineering tools originally designed to improve patient flow through the prevention of Mother-to-Child transmission of HIV (PMTCT) cascade. SAIA is a potentially scalable model for maximizing the benefits of universal antiretroviral therapy (ART) for mothers and their babies. SAIA-SCALE was a stepped wedge trial implemented in Manica Province, Mozambique, to evaluate SAIA’s effectiveness when led by district health managers, rather than by study nurses. We present the results of a qualitative assessment of implementation determinants of the SAIA-SCALE strategy during two intensive and one maintenance phases.
Methods
We used an extended case study design that embedded the Consolidated Framework for Implementation Research (CFIR) to guide data collection, analysis, and interpretation. From March 2019 to April 2020, we conducted in-depth individual interviews (IDIs) and focus group discussions (FGDs) with district managers, health facility maternal and child health (MCH) managers, and frontline nurses at 21 health facilities and seven districts of Manica Province (Chimoio, Báruè, Gondola, Macate, Manica, Sussundenga, and Vanduzi).
Results
We included 85 participants: 50 through IDIs and 35 from three FGDs. Most study participants were women (98%), frontline nurses (49.4%), and MCH health facility managers (32.5%). An identified facilitator of successful intervention implementation (regardless of intervention phase) was related to SAIA’s compatibility with organizational structures, processes, and priorities of Mozambique’s health system at the district and health facility levels. Identified barriers to successful implementation included (a) inadequate health facility and road infrastructure preventing mothers from accessing MCH/PMTCT services at study health facilities and preventing nurses from dedicating time to improving service provision, and (b) challenges in managing intervention funds.
Conclusions
The SAIA-SCALE qualitative evaluation suggests that the scalability of SAIA for PMTCT is enhanced by its fit within organizational structures, processes, and priorities at the primary level of healthcare delivery and health system management in Mozambique. Barriers to implementation that impact the scalability of SAIA include district-level financial management capabilities and lack of infrastructure at the health facility level. SAIA cannot be successfully scaled up to adequately address PMTCT needs without leveraging central-level resources and priorities.
Trial registration
ClinicalTrials.gov, NCT03425136. Registered on 02/06/2018.
