ArticlePDF Available

A Perspective on Implementation Outcomes and Strategies to Promote the Uptake of COVID-19 Vaccines


Abstract and Figures

Recent articles have highlighted the importance of incorporating implementation science concepts into pandemic-related research. However, limited research has been documented to date regarding implementation outcomes that may be unique to COVID-19 vaccinations and how to utilize implementation strategies to address vaccine program-related implementation challenges. To address these gaps, we formed a global COVID-19 implementation workgroup of implementation scientists who met weekly for over a year to review the available literature and learn about ongoing research during the pandemic. We developed a hierarchy to prioritize the applicability of “lessons learned” from the vaccination-related implementation literature. We identified applications of existing implementation outcomes as well as identified additional implementation outcomes. We also mapped implementation strategies to those outcomes. Our efforts provide rationale for the utility of using implementation outcomes in pandemic-related research. Furthermore, we identified three additional implementation outcomes: availability, health equity, and scale-up. Results include a list of COVID-19 relevant implementation strategies mapped to the implementation outcomes.
published: 20 May 2022
doi: 10.3389/frhs.2022.897227
Frontiers in Health Services | 1May 2022 | Volume 2 | Article 897227
Edited by:
Nick Sevdalis,
King’s College London,
United Kingdom
Reviewed by:
Denise F. Lillvis,
University at Buffalo, United States
Meagan Pilar
Specialty section:
This article was submitted to
Implementation Science,
a section of the journal
Frontiers in Health Services
Received: 15 March 2022
Accepted: 22 April 2022
Published: 20 May 2022
Pilar M, Elwy AR, Lushniak L,
Huang G, McLoughlin GM, Hooley C,
Nadesan-Reddy N, Sandler B,
Moshabela M, Alonge O, Geng E and
Proctor E (2022) A Perspective on
Implementation Outcomes and
Strategies to Promote the Uptake of
COVID-19 Vaccines.
Front. Health Serv. 2:897227.
doi: 10.3389/frhs.2022.897227
A Perspective on Implementation
Outcomes and Strategies to Promote
the Uptake of COVID-19 Vaccines
Meagan Pilar 1
*, A. Rani Elwy 2,3 , Larissa Lushniak 1, Grace Huang 1,
Gabriella M. McLoughlin 4,5 , Cole Hooley 6, Nisha Nadesan-Reddy 7, Brittney Sandler 8,
Mosa Moshabela 7, Olakunle Alonge 9, Elvin Geng 1and Enola Proctor 10
1Department of Infectious Diseases, Washington University School of Medicine, Washington University in St. Louis, St. Louis,
MO, United States, 2Department of Psychiatry and Human Behavior, Warren Alpert Medical School, Brown University,
Providence, RI, United States, 3Center for Healthcare Organization and Implementation Research, VA Bedford Healthcare
System, Bedford, MA, United States, 4College of Public Health, Temple University, Philadelphia, PA, United States,
5Implementation Science Center for Cancer Control and Prevention Research Center, Brown School, Washington University
in St. Louis, St. Louis, MO, United States, 6School of Social Work, Brigham Young University, Provo, UT, United States,
7School of Nursing and Public Health, University of KwaZulu-Natal, Durban, South Africa, 8Bernard Becker Medical Library,
Washington University in St. Louis School of Medicine, St. Louis, MO, United States, 9Department of International Health,
Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, United States, 10 Shanti K. Khinduka Distinguished
Professor Emerita, Brown School, Washington University in St. Louis, St. Louis, MO, United States
Recent articles have highlighted the importance of incorporating implementation
science concepts into pandemic-related research. However, limited research has been
documented to date regarding implementation outcomes that may be unique to
COVID-19 vaccinations and how to utilize implementation strategies to address vaccine
program-related implementation challenges. To address these gaps, we formed a global
COVID-19 implementation workgroup of implementation scientists who met weekly
for over a year to review the available literature and learn about ongoing research
during the pandemic. We developed a hierarchy to prioritize the applicability of “lessons
learned” from the vaccination-related implementation literature. We identified applications
of existing implementation outcomes as well as identified additional implementation
outcomes. We also mapped implementation strategies to those outcomes. Our
efforts provide rationale for the utility of using implementation outcomes in pandemic-
related research. Furthermore, we identified three additional implementation outcomes:
availability, health equity, and scale-up. Results include a list of COVID-19 relevant
implementation strategies mapped to the implementation outcomes.
Keywords: implementation science, COVID-19, vaccine, implementation outcomes, implementation strategies
The coronavirus disease 2019 (COVID-19) pandemic vividly exemplifies an implementation crisis:
life-saving remedies exist, but their adoption and spread have lagged world-wide. Many longed for
vaccines and heralded their development. However, many medical and public health experts were
surprised at the skepticism, hesitancy, and outright resistance to available vaccines, even as the
pandemic led to sky-rocketing rates of mortality and morbidity and compromised even the most
resource-rich health systems. Despite recommendations from the World Health Organization, we
are still far from achieving the proposed goal of 70% vaccination coverage globally (1). Though
Pilar et al. COVID-19 Implementation Outcomes and Strategies
there is some debate as to whether 70% vaccination is still
sufficient due to the emergence of new variants (2), the challenges
with vaccine program implementation remain.
Implementation science is well-suited to tackle crises like
the COVID-19 pandemic. But what specifically does it offer?
This Perspective article addresses one part of this question by
demonstrating the importance of conceptualizing vaccine roll-
out through the lens of two implementation science concepts:
implementation outcomes and implementation strategies. These
concepts help clarify some of the most pressing questions
about the implementation challenges surrounding vaccination
programs: What needs to be achieved, and how do we
get there? This article is not meant to be an exhaustive
review of current research; rather, it is intended to provide
the reader with guidance on how we might more clearly
conceptualize the implementation outcomes—both anticipated
and actual (3)—that are most relevant for increasing the
uptake of COVID-19 vaccines. Additionally, we will provide
examples of implementation strategies, developed to ensure
successful implementation outcomes, to better aid researchers
and practitioners in evaluating the implementation of COVID-
19 vaccines.
Implementation science is the “study of methods to promote
the systematic uptake of research findings and other evidence-
based practices into routine practice” (4). This field seeks to
provide guidance in cases where evidence-based interventions
exist but are poorly implemented or, in some instances, not
implemented at all. In the case of the COVID-19 pandemic, an
intervention—a vaccine—exists, but it has been underutilized for
myriad reasons (e.g., issues surrounding supply and distribution,
mistrust, misinformation, vaccine hesitancy). Implementation
science provides an opportunity to apply existing methods to
study these challenges and improve the uptake of the COVID-
19 vaccine.
Implementation science is inherently pragmatic and
involves real-world, diverse populations, data collection that is
meaningful and actionable, and a focus on the application of
an evidence-based practice (e.g., vaccination) in local contexts
(5). Thus, ensuring the successful uptake of COVID-19 vaccines
globally requires a pragmatic, low burden assessment of
stakeholders’ perceptions of implementation outcomes related to
vaccination (e.g., acceptability, cost) (6), as well as specific and
operationalized implementation strategies to address barriers to
vaccine uptake (7).
Previous articles have underscored the importance of
incorporating implementation science concepts into pandemic-
related research (810). For example, it is vital to engage
stakeholders from project inception and consider context during
implementation, as factors such as available resources, policy
support, health system and population characteristics can impact
the uptake of an intervention (9). Additionally, implementation
science theories and methods can help inform the equitable
development, implementation, and evaluation of interventions
to address health disparities and promote health equity (11
14). However, there has been a limited research focus to date
regarding implementation outcomes that may be unique to
COVID-19 vaccinations and how to utilize implementation
strategies to address vaccine program-related implementation
challenges. To address these gaps, we formed a global COVID-
19 implementation workgroup of implementation scientists who
met weekly for over a year, to review the available literature
and learn about ongoing research during the pandemic. Our
efforts resulted in a list of implementation outcomes that can
be used to evaluate the implementation of vaccine programs
globally. Likewise, we have compiled a list of implementation
strategies, which can be mapped onto the aforementioned
outcomes to address common challenges related to vaccine
program implementation.
Given this novel disease and the unprecedented times,
evidence directly related to COVID-19 is still developing. As a
result, our recommendation for implementation strategies have
been informed by COVID-19 evidence, as well as previous public
health efforts, which we believe are transferable or applicable to
this pandemic. To aid in this conceptualization of evidence, our
team constructed the following hierarchy by which to prioritize
recommendations and “lessons learned.” We drew from existing
evidence hierarchies [e.g., GRADE (15) and AGREE II (16)] and
used a consensus-based approach among the authors similar to
that espoused for developing guideline recommendations [e.g.,
the DECIDE framework (1719)]. The implementation science
base of evidence to guide strategy selection for COVID-19
mitigation ranges from scant to very strong.
1. Direct evidence from COVID-19 vaccination: scant and
emergent, especially early in the pandemic.
2. Evidence from other preventative strategies used during
COVID-19 (e.g., masking, social distancing): emergent.
3. Evidence from vaccinations of non-COVID-19 diseases that
are respiratory, novel, or heterogeneous in severity, or during
a pandemic (e.g., influenza): solid to strong.
4. Evidence from vaccination for other infections (e.g., measles,
polio): strong.
5. Evidence from other health conditions: very strong.
Implementation outcomes provide a means to evaluate the
implementation success of interventions, treatments, policies,
and protocols and are distinct from other, traditionally measured
outcomes, such as service system and clinical outcomes (6).
In the case of COVID-19 prevention and treatment, service
system outcomes include the timeliness and efficiency of the
health system, and perceived equity and patient-centeredness of
treatments, while clinical outcomes reflect population and patient
health and safety, as well as satisfaction with treatment options,
including vaccines.
In 2011, Proctor and colleagues developed the
Implementation Outcomes Framework as a way to conceptualize
Frontiers in Health Services | 2May 2022 | Volume 2 | Article 897227
Pilar et al. COVID-19 Implementation Outcomes and Strategies
and measure eight distinct implementation outcomes—
acceptability, adoption, appropriateness, cost, feasibility, fidelity,
penetration, and sustainability (6). This framework has since
been widely cited and applied within the implementation science
community. However, based on our review of the literature,
there are additional outcomes that could—and should—be
considered within the context of the COVID-19 pandemic to
provide a more comprehensive evaluation of implemenation
of vaccine programs. As a result, we have operationalized the
eight well-known implementation outcomes based on their
relevance to the COVID-19 pandemic and have proposed three
additional outcomes (availability, health equity, and scale-up)
for consideration (Table 1).
Evaluations of COVID-19 mitigation efforts could have
been—and still have potential to be—more precise and robust
with a greater focus on implementation outcomes. These
outcomes can play three important roles in relation to COVID-
19 control. First, they allow researchers and implementers to
focus efforts on assessing the baseline or starting point, thereby
quantifying the gap between desired and achieved outcomes.
Vaccines need to “have an efficacy of at least 70% to prevent an
epidemic and of at least 80% to largely extinguish an epidemic
without any other measures (e.g., social distancing)” (29). Within
the context of COVID-19 vaccination, where many countries
included other mitigation measures in their efforts to control
the pandemic, such as mask wearing and social distancing,
this could include monitoring the vaccination status of a
population. In addition to quantifying gaps in implementation,
incorporating implementation outcomes into vaccine program
planning and evaluation also provides a source of accountability
for implementers at the national-, state-, or local-levels. The
additional implementation outcomes of availability and health
equity are essential for increasing vaccine uptake. Vaccines can
be utilized only when they are available, and the sufficiency of
their availability depends on supply, the size of population to
be covered, and the size of priority groups receiving vaccines.
Similarly, vaccine equity requires consideration of sub-groups,
including groups with priority needs for vaccination and those
that differ by social, economic, demographic, and geographical
factors linked to systemic disadvantage.
Second, implementation outcomes provide a direction for
implementation efforts. For instance, factors such as the
feasibility and sustainability of public health programs may be
prioritized when implementation outcomes are incorporated into
the planning and evaluation process. These outcomes can also
highlight potential barriers or facilitators that may arise during
implementation and help researchers identify implementation
strategies to address potential challenges. As we have seen over
the past 2 years, myriad challenges can arise during vaccine
development, distribution, and implementation. For example,
factors such as unfamiliar technology or a lack of staff buy-in may
decrease the feasibility of implementing a COVID-19 vaccine
program. The additional implementation outcome of scale-up
we have defined illustrates how vaccine scale-up is affected by
factors such as vaccine hesitancy, supply shortages, financial
and technological resources, and existing infrastructure (e.g.,
cold-chain system infrastructure and ample workforce). Scale-up
differs from the original penetration implementation outcome,
which refers to the degree to which an intervention has infiltrated
a service system. Scale-up, on the hand, is broader, encompassing
multiple service systems thereby requiring different strategies
at higher socioecological levels. Additionally, the deluge of
information disseminated through social media has increasingly
delivered misinformation impacting acceptability of COVID-
19 vaccines globally. As a result, specific strategies can be
selected based on the implementation context. Challenges with
technology could be addressed through the provision of technical
assistance, while buy-in could be encouraged by identifying
champions within the organization. Finally, vaccine-related
misinformation could be counteracted through a targeted public
health campaign emphasizing the transparency of the vaccine
development process, and using jargon-free messaging that takes
into account socioeconomic and cultural factors, as well as
personal and media sources needed for communicating with
specific populations (30).
Third, implementation outcomes can increase the precision
of evaluation efforts surrounding public health programs. Too
often, public health systems’ data collection is limited to clinical
outcomes (e.g., number of COVID-19 cases, hospitalizations,
and mortality). However, measuring intermediate outcomes,
such as acceptabilty, adoption and fidelity, paints a more
complete picture of program implementation and illuminates
the challenges to current implementation efforts. For example,
several global surveys conducted throughout the pandemic have
assessed the degrees of acceptability of COVID-19 vaccines
over time (31,32). Important findings from these surveys have
revealed that those who trust their governments and their
messaging (31) and receive information and guidance from
healthcare providers (32) reported that they will be more likely
to become vaccinated against COVID-19. In another project,
COVID-19 vaccine acceptability among patients and employees
of a large integrated healthcare system in the United States
was assessed through surveys (33) and interviews (34). Specific,
tailored communication strategies were then created to overcome
identified barriers to receiving a vaccine and were disseminated
widely across the healthcare system, for use in one-to-one
conversations between trusted providers and patients, and
among employees (34). The additional information regarding,
for example, program acceptability can enable implementers to
better evaluate program efforts and adapt practices to promote
the uptake of COVID-19 vaccines.
Implementation strategies are defined as “methods or techniques
used to enhance the adoption, implementation, and sustainability
of a clinical program or practice” (7). Implementation scientists
have emphasized the importance of identifying and compiling
evidence-based strategies (35), selecting strategies based on
implementation context and barriers (36), and specifying the
use of implementation strategies in research (7). Building on
this body of work, our team set out to map implementation
strategies onto specific implementation outcomes and challenges
Frontiers in Health Services | 3May 2022 | Volume 2 | Article 897227
Pilar et al. COVID-19 Implementation Outcomes and Strategies
TABLE 1 | Operationalization and addition of implementation outcomes unique to COVID-19.
Implementation outcome Definition Application to COVID-19
Acceptability*The perception among stakeholders that a given treatment,
service, practice, or innovation is agreeable, palatable, or
Vaccine implementation requires a minimum level of acceptability among
policy makers, healthcare providers, community leaders,
parents/caregivers, and those eligible for the vaccine. Vaccine acceptability
is undermined by misinformation, lack of trust in governments and health
systems, and anti-vaccine beliefs and attitudes.
Adoption*The intention, initial decision, or action to try or employ a new
treatment or innovation—also referred to as “uptake”
Vaccine adoption can be impacted by external mandates, vaccine supply,
organizational culture and climate, and healthcare providers’ willingness to
recommend and administer the vaccine.
Appropriateness*The perceived fit, relevance, or compatibility of the treatment
or innovation for a given practice setting, provider, or
consumer; and/or the perceived fit of the innovation to
address a particular issue or problem
The appropriateness of vaccines is influenced by organizational- and
individual-level factors, such as existing resources within an organization,
healthcare providers’ knowledge and perceptions of the vaccine, and
patients’ ability and willingness to receive the vaccine.
Availability** The supply of a new treatment or innovation at any one time Vaccines can be utilized only when they are available, and the sufficiency of
their availability depends on supply, the size of population to be covered,
and the size of priority groups receiving vaccines.
Cost*The cost impact of an implementation effort The cost of vaccine implementation includes multiple levels, such as the
cost to procure vaccine doses (e.g., national financing), administer doses
(e.g., organizations’ costs to pay employees, restructure existing processes,
etc.), incentivize doses (e.g., organizations paying employees to be
vaccinated), and receive doses (e.g., individuals’ costs to travel, take time
off work, etc.).
Feasibility*The extent to which a new treatment or innovation can be
successfully used or carried out in a given setting
The feasibility of vaccine implementation is affected by supply, as well as
geographic factors, transportation and storage structures, and the
capabilities, time, and training of vaccine providers.
Fidelity*The degree to which a treatment or innovation was
implemented as it was prescribed in the original protocol or
as was intended by program developer
Vaccines must be stored, distributed, and delivered with strict fidelity to
dosing and safety requirements.
Health Equity** Fair access to a treatment or innovation without avoidable or
remediable differences among groups of people
Vaccine equity requires consideration of sub-groups, including groups with
priority needs and those that differ by social, economic, demographic, and
geographical factors linked to systemic disadvantage.
Penetration*The integration of a treatment or innovation within a service
setting and its subsystem
The penetration of a vaccine program is impacted by myriad factors,
including policies, organizational culture, personal beliefs, and the supply of
the vaccine itself.
Scale-up** Deliberate efforts to increase the impact of successfully
tested health innovations so as to benefit more people and to
foster policy and program development on a lasting basis (20)
The potential for scale-up is affected by factors such as vaccine hesitancy,
supply shortages, financial and technological resources, political will, and
existing infrastructure (e.g., cold-chain system infrastructure and ample
Sustainability*The extent to which a treatment or innovation is maintained or
institutionalized within a service setting’s ongoing, stable
The sustainability of vaccination efforts and programs must be incorporated
into existing both national and organizational structures (e.g., allocating
financial and human resources), workflows (e.g., restructuring employee
roles and responsibilities), and/or programs (e.g., bundling with existing
*Definitions are adapted from the Proctor et al. 2011 paper (6). **Team added constructs—availability (2124), health equity (2527), and scale-up (28)—based on theoretical and
practical application.
to provide guidance for researchers and practitioners to increase
the uptake of vaccines and aid in the scale-up and spread of
successful vaccination programs.
After identifying and conceptualizing critical implementation
outcomes to assess to determine the effectiveness of COVID-
19 vaccine implementation globally (Table 1), our research
group considered some of the most common challenges
encountered during vaccine program implementation (e.g.,
mistrust, misinformation, vaccine hesitancy, supply issues).
We then compiled a range of implementation strategies to
determine which of these may be most useful in overcoming
common implementation challenges and increasing uptake of
COVID-19 vaccines. We present the implementation outcomes,
implementation challenges, and the implementation strategies
our team identified, and details on these strategies (e.g.,
specified actions or tools) identified in the literature to improve
implementation success (Table 2).
In many cases, we found that several implementation
strategies were needed to address these complex, multi-
level implementation challenges (5), especially prevalent in
vaccination programs. For example, Rajkumari and colleagues
identified two barriers related to the uptake of a measles-
rubella vaccine in India—limited knowledge of the vaccine
and geographic inaccessibility—which impacted the vaccine
program’s acceptability and feasibility, respectively (43). As a
result, the team selected implementation strategies to address
Frontiers in Health Services | 4May 2022 | Volume 2 | Article 897227
Pilar et al. COVID-19 Implementation Outcomes and Strategies
TABLE 2 | Implementation outcomes, challenges, and strategies applicable to the COVID-19 vaccine program implementation.
Implementation challenge Implementation strategy Example
Acceptability Vaccine hesitancy (broad) Plan: Identify hesitancy in the
population or subgroups and tailor
intervention efforts to reach them
Used the Guide to Tailoring Immunization Programs
(37,38) to identify subgroups with low immunization
rates, diagnose factors impacting vaccine hesitancy, and
tailor programs to addresses the factors leading to low
vaccine acceptance in the subgroup (39)
Used the Social Mobilization Network (SMNet) to target
resistance at multiple levels through effective,
personalized health communication (40)
Low vaccine demand Educate: Receive recommendation
from a trusted source
Promoted demand for vaccines through a personalized
recommendation from a trusted healthcare provider in
communities of color (41)
Limited knowledge or awareness of
Plan: Design and implement a health
Created a vaccination program through the Cameroon
Baptist Convention Health Services that targeted
schools, clinics, churches, and regarding HPV and
cervical cancer (42)
Implemented a national vaccination campaign through
school and community outreach sessions (43)
Used nation-wide campaign, National Immunization
Days (NIDs), to administer vaccines in locations across
the country (44)
Adoption Low levels of vaccine awareness Educate: Train health
workers/community volunteers in
information, education and
communication tactics
Harnessed the power of social networks and trained
community volunteers to and increase awareness of and
support for HIV vaccine research in minority populations
Misconceptions about the vaccine
and its effects
Plan/Educate: Use health
communication strategies to address
Recommendations to create positive vaccine narratives
and use positive emotional appeals (e.g., hope and job in
receiving a vaccine) to counteract negative emotions
(e.g., fear, anger, mistrust) surrounding vaccination (46)
Conducted a series of town hall meetings to address
concerns and misinformation raised by healthcare
workers and staff (47)
Vaccine hesitancy (mistrust in science
or the vaccine)
Plan/Educate: Create health
communication materials
Created a digital infographic to promote trust in science,
reduce the believability of misinformed narratives, and
increase the likelihood of engaging in preventive
behaviors (48)
Appropriateness Low vaccine demand and mistrust in
the community
Plan: Tailor outreach efforts and
communication strategies to
subgroups (e.g., race/ethnicity,
gender, rural areas)
Recommendations regarding outreach—Efforts should
be led by physicians reflecting the diversity of the
subgroups (e.g., Black physicians affiliated with
historically Black medical institutions to target
communities of color; physicians from well-respected
medical institutions in Republican-leaning states to target
conservative states) (49)
Vaccine hesitancy (broad) Plan: Create persuasive public health
communication plans tailored to an
individual’s level of vaccine hesitancy
Segmented portions of the population and target health
communication efforts to their identified barriers (50)
Utilized vaccine messaging that address the personal
benefits of vaccination (e.g., prevention of chronic illness)
to target the hesitant population (51)
Vaccine hesitancy (broad) Plan: Use effective mass
communication strategies
Recommendations to emphasize transparency regarding
vaccine-related health communications (e.g., safety,
efficacy, vaccine development, distribution, and cost) (52)
Vaccine hesitancy (mistrust in science
or the vaccine)
Educate: Provide training to promote
cultural competence
Recommendations to train and equip healthcare
providers, particularly when working with historically
marginalized groups (53)
Availability Limited number of suppliers Finance; Alter incentive structure by
developing advance purchase
Need an integrated policy approach that preserves
incentives for market entry and innovation in the vaccine
industry while addressing vaccine concerns and
increasing immunization funding and reimbursement for
both providers and patients (54)
High research and development and
production costs
Finance: Access new funding through
government subsidies for basic
vaccine research
Increased funding through the US Biomedical Research
and Development Authority, resulting in over $19.3 billion
to facilitate COVID-19 vaccine development (55)
Frontiers in Health Services | 5May 2022 | Volume 2 | Article 897227
Pilar et al. COVID-19 Implementation Outcomes and Strategies
TABLE 2 | Continued
Implementation challenge Implementation strategy Example
Safety problems leading to increased
regulatory requirements.
Policy context: Change liability laws to
provide protection for manufacturers
Established a COVID-19 vaccine injury no-fault
compensation scheme in South Africa to facilitate
COVID-19 vaccine rollout (56)
Storage availability Policy context: Identify barriers and
facilitators and test new workflows
Increased the frequency of transport capacity to reduce
storage bottlenecks and increase vaccine availability (57)
Cost Limited economic resources Finance: Engage and mobilize
stakeholders and payers
Recommendations to ensure adequate operational
funds are mobilized in readiness for the vaccination
exercise based on the country micro-plans (58)
Limited economic resources Finance: Include COVID-19 vaccine
strategy government budgets
Recommendations to estimate funding needs and align
cost plans with existing resources while minimizing
fragmentation for existing programs (58)
Limited economic resources Finance: Provide financial incentives
(in settings with low immunization
Utilized trusted vaccine “ambassadors,” SMS reminders,
and low-cost incentives (i.e., mobile phone credit) to
increase vaccine uptake (59)
Limited economic resources Finance: Identify potential new
sources of revenue
Recommendations to facilitate dialogue and alignment
with the budget and planning departments of the
Ministry of Health, Ministry of Finance, and the funding
partners (58)
Feasibility Low vaccine demand Restructure: Bundle vaccine efforts
with existing community programs
Recommendations regarding how vaccination programs
could be offered alongside existing services and valued
community initiatives, such as nutritional support and
food supplementation programs (60)
Recommendations to consider other factors when
bundling programs: similarities in target groups, logistical
requirements, skill levels required for healthcare staff (61)
Geographic inaccessibility Restructure: Decrease geographic
barriers to vaccine uptake
Included community organizations, such as schools, as
centers for vaccine campaign administration (43)
Employed strategies such as door-to-door visits to
spread awareness of vaccine program goals and local
vaccination sites (60)
Fidelity Compliance with public health
Policy context: Develop enforcement
policies regarding vaccination
Recommendations to consider factors such as control
aversion, trust in the government, and the degree of
enforcement when designing enforcement vs. voluntary
policies (62)
Recommendations for policymakers to develop
programs that optimize identification and treatment of
those with disease while minimizing the use of invasive
measures, such as involntary detention of noncompliant
patients or forced administration of vaccinations (63)
Fidelity Inability to track disease spread and
report cases
Quality management: Strengthen
surveillance systems and establish
robust system for capturing and
tracking cases
Examples: Provide supervision; use
desk and field reviews to assess
quality of AFP surveillance
Recommendations to improve surveillance system’s
functioning, sensitivity, and quality despite challenges
such as a large national geographic expanse, zones with
chronic insecurity and inaccessibility, and a lack of
capacity and infrastructure (64)
Health equity Structural racism Educate: Provide equity training for
Recommendations to provide training, education, or
opportunities for reflection in health equity, addressing
structural racism, and/or promoting antiracism
approaches with respect to our research, institutions,
and community partnerships (e.g., Public Health Critical
Race Praxis) (65)
Unequal power dynamics with
Plan: Include early and ongoing
engagement with stakeholders in
both decision-making and
Recommendations to promote transparency,
consideration of power dynamics, equitable sharing of
resources, respect of community values, and inclusion of
racially/ethnically diverse partners as equitable
decision-makers early and often (65)
Frontiers in Health Services | 6May 2022 | Volume 2 | Article 897227
Pilar et al. COVID-19 Implementation Outcomes and Strategies
TABLE 2 | Continued
Implementation challenge Implementation strategy Example
Vaccine hesitancy (broad) Plan: Engage community partners to
promote vaccine-related
information-sharing and build trust
with marginalized communities
Increased vaccine acceptance by waiting for safety data
to be more robust, knowing more about the vaccine, and
receiving a recommendation to take the vaccine from a
trusted healthcare provider (41)
Engaged youth group members and significantly
enhanced the ability of vaccination teams to vaccinate
chronically missed children (66)
Engaged pastors as trusted messengers; created
partnerships with shared responsibility and power; and
co-created solutions with faith leaders and their
community, governments and institutions (67)
Utilized local community members to spread information
about vaccination events, which was more effective than
mass media advertisements (68)
Low vaccine demand Vaccine
hesitancy (mistrust in science or
the vaccine)
Plan: Create micro-plans with
hard-to-reach communities at center
of plans
Elicited immunization preferences for six program
characteristics (e.g., location, use of incentives, bundling
with existing services) to create a targeted approach for
implementation (60)
Utilized local and religious leaders to enhance
community knowledge of vaccination campaigns (43)
Low vaccine demand Plan: Utilize social networks to
increase reach and uptake
Used social network methods to identify and recruit to
provide access to high-risk youth who may be critical
recipients of a future vaccine (38)
Penetration Low vaccine demand Restructure: Enable community
health workers to promote vaccine
Utilized existing community structures such as churches
to spread preventive care messages and facilitate
vaccine promotion (69)
Low vaccine demand Quality management: Use
patient-held web-based portals and
computerized reminders increase
immunization coverage rates
Used text messaging, immunization campaign websites,
patient-held web-based portals, and computerized
reminders and standing orders for physicians to increase
immunization coverage rates (70)
Limited advocacy for an
Plan/Quality management: Identify
and engage policy entrepreneurs and
champions in various levels of
government, user organizations, and
the broader community
Recruited individuals who were highly motivated to move
forward with innovations and advocate for their
promotion and adaptation at organizational or
bureaucratic levels (20)
Scale-up Lack of dynamic partnerships Quality management: Assess the
strengths and weaknesses of the user
organization (e.g., public sector
health service system, NGO, etc.) and
develop strategies to build capacity
Recommendations to identify how user organizations’
resources, staffing, organizational culture, and leadership
structures will affect program scale-up (20)
Limited organizational capacity to
Plan/Educate: Ensure the team has
necessary skills and capacities to
implement a vaccine program
Recommendations to conduct program evaluation,
management, training, economic evaluation, fundraising,
health communication, and writing while emphasizing
the importance of cultural knowledge (20)
Limited consideration of external
influences when developing
implementation program goals
Quality management/Restructure:
Identify the environmental factors
influencing scaling up and understand
how they affect the process
Recommendations to consider how policy/politics,
bureaucracy, and socio-economic/cultural contexts will
directly impact vaccination program scale-up prior to
implementation (20)
Sustainability Vaccine hesitancy (broad) Plan/Educate: Prepare materials for
healthcare workers to better integrate
into routine practice
Recommendations to (1) prepare a list of common
vaccine questions; (2) develop a list of effective
responses; and (3) train and practice with staff to
response to patients’ concerns (50)
Limited organizational capacity to
Restructure: Bundle vaccine efforts
with existing community programs
Recommendations to consider multi-level factors when
bundling programs: similarities in the availability of
funding, logistical requirements, political support, and
level of burden (i.e., to ensure that bundling does not
disrupt service delivery) (61)
Low vaccine demand Vaccine
hesitancy (mistrust in science or
the vaccine)
Plan: Build buy-in with stakeholders Engaged trusted community figures, such as community
influencers, local religious leaders, healthcare providers,
and parents (41,49,52,71)
Frontiers in Health Services | 7May 2022 | Volume 2 | Article 897227
Pilar et al. COVID-19 Implementation Outcomes and Strategies
these barriers, which included creating a national health
campaign and administering vaccines through community
organizations, such as schools (43). Momplaisir et al., explored
attitudes and beliefs related to COVID-19 vaccinations
within Black communities in the United States (41). Though
participants reported low demand and high levels of vaccine
hesitancy, they also identified that recommendations from
a healthcare provider might increase their trust in the
vaccine’s safety and efficacy (41). These findings illustrate
the importance of engaging community partners and promoting
trust, particularly in historically underserved communities,
to increase the acceptability and equitable distribution of the
COVID-19 vaccine.
The COVID-19 pandemic is an evolving situation, and
implementation science needs to respond accordingly. Many of
our theories and study designs have an implicit long period of
time that’s required to assess implementation (72). However, this
situation requires rapid assessments and evaluations, as well as
the measurement of implementation outcomes that may have
been previously overlooked (73).
Our team’s activity provided evidence that the original Proctor
implementation outcomes are still essential for examining the
success of COVID-19 vaccination programs globally; yet three
outcomes—those of availability, health equity and scale-up—are
welcome additions to the implementation outcomes framework.
Addressing implementation challenges related to availability, for
example, allows governments and policymakers to focus on the
earlier, or pre-implementation, factors that support widespread
vaccine scale-up, such as increasing vaccine suppliers and
providing incentives and additional funding structures to address
high research and development costs. In certain contexts, policies
may also need to change regarding manufacturing liability, which
is essential for increasing the availability and eventual scale-up of
vaccines. Vaccine availability is also impacted by storage issues
and can be mitigated by additional infrastructure support, such
as increasing transportation, which is needed for future scale-up.
Health equity, a critical part of implementation science,
and fair access to COVID-19 vaccines has been a continual
challenge globally. Health equity is also closely related to
the implementation outcomes of availability and scale-up. For
example, when there is limited vaccine supply or organizational
capacity for vaccine program implementation, power struggles
that exist among stakeholders with varying social, economic,
demographic, and geographical differences may lead to an
inequitable distribution of vaccines (e.g., vaccines only being
available to groups whose power aligns with the organizations
distributing vaccines). Additionally, greater distrust in science
and vaccines within historically underserved populations may
further impede vaccine acceptance and uptake. This underscores
the importance of considering the myriad external influences that
lead to misinformation campaigns and distrust when assessing
the potential for vaccine program scale-up.
Through engagement of a collaborative working group on
global vaccine implementation, our team was able to apply a well-
known implementation science framework to current and past
literature, policies, and country-level knowledge about vaccines.
As a result, we identified three additional implementation
outcomes and compiled a list of specific implementation
strategies that can be applied at multiple levels to increase
vaccine uptake.
The original contributions presented in the study are included
in the article/supplementary material, further inquiries can be
directed to the corresponding author/s.
EP, EG, MM, and OA contributed to conception of the study.
MP, AE, and EP wrote the first draft of the manuscript. GH and
LL wrote portions of the manuscript. All authors contributed to
manuscript revision, read, and approved the submitted version.
This work was supported in part by the National Institute of
Allergy and Infectious Diseases (No. K24AI134413). EP effort
was supported by the Implementation Science-Entrepreneurship
Unit of the Washington University Institute of Clinical
and Translational Sciences (No. UL1TR002345) from the
National Center for Advancing Translational Sciences of the
National Institutes of Health; the National Institute of Mental
Health (No. R25MH080916); National Cancer Institute (No.
P50CA244431); National Institute of Mental Health (No.
5P50MH113662); Patient-Centered Outcomes Research Institute
Award (No. TRD-1511-33321).
We would like to acknowledge the support of the Washington
University Center for Dissemination and Implementation.
1. World Health Organization. Strategy to Achieve Global Covid-19 Vaccination
by mid-2022 (2021). Available online at:
strategy-achieve- global-covid-19- vaccination-mid-2022 (accessed March 31,
2. Aizenman N. The goal: Vaccinate 70% of the world against COVID. In:
Scientists are proposing a reboot. NPR. (2022).
3. Damschroder LJ, Reardon CM, Opra Widerquist MA, Lowery J.
Conceptualizing outcomes for use with the Consolidated Framework
for Implementation Research (CFIR): the CFIR outcomes addendum. J
Article Implement Sci. (2022) 17:1–10. doi: 10.1186/s13012-021-01181-5
4. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne
AM. An introduction to implementation science for the non-
specialist. BMC Psychol. (2015) 3:32–32. doi: 10.1186/s40359-015-
Frontiers in Health Services | 8May 2022 | Volume 2 | Article 897227
Pilar et al. COVID-19 Implementation Outcomes and Strategies
5. Glasgow RE. What does it mean to be pragmatic? Pragmatic
methods, measures, and models to facilitate research translation.
Health Educ Behav. (2013) 40:257–65. doi: 10.1177/10901981134
6. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A,
et al. Outcomes for implementation research: Conceptual distinctions,
measurement challenges, and research agenda. Admin Policy Mental Health
Mental Health Serv Res. (2011) 38:65–76. doi: 10.1007/s10488-010-0319-7
7. Proctor EK, Powell BJ, McMillen JC. Implementation strategies:
recommendations for specifying and reporting. Implement Sci. (2013)
8:139. doi: 10.1186/1748-5908-8-139
8. Means AR, Wagner AD, Kern E, Newman LP, Weiner BJ. Implementation
science to respond to the COVID-19 pandemic perspective. Front Public
Health. (2020) 8:462. doi: 10.3389/fpubh.2020.00462
9. Chambers DA. Considering the intersection between implementation
science and COVID-19. Implement Res Pract. (2020)
1:0020764020925994. doi: 10.1177/0020764020925994
10. Wensing M, Sales A, Armstrong R, Wilson P. Implementation science in times
of Covid-19. Implement Sci. (2020) 15:42. doi: 10.1186/s13012-020-01006-x
11. Jacobson TA, Smith LE, Hirschhorn LR, Huffman MD. Using
implementation science to mitigate worsening health inequities in the
United States during the COVID-19 pandemic. Int J Equity Health. (2020)
19:170. doi: 10.1186/s12939-020-01293-2
12. Galaviz KI, Breland JY, Sanders M, Breathett K, Cerezo A, Gil
O, et al. Implementation Science to Address Health Disparities
During the Coronavirus Pandemic. Health Equity. (2020)
4:463–7. doi: 10.1089/heq.2020.0044
13. Woodward EN, Singh RS, Ndebele-Ngwenya P, Melgar Castillo A, Dickson
KS, Kirchner JE, et al. more practical guide to incorporating health
equity domains in implementation determinant frameworks. Implement Sci
Commun. (2021) 2:61. doi: 10.1186/s43058-021-00146-5
14. Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to
enhance sustainability: addressing dynamic context and promoting health
equity over time. Front Public Health. (2020) 8:134. doi: 10.3389/fpubh.2020.
15. Siemieniuk R, Guyatt G. What is GRADE? BMJ Publishing Group. (2022).
16. AGREE: Advancing the science of practice guidelines. AGREE II
Instructions. (2022).
17. Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brozek J,
et al. Developing and evaluating communication strategies to support
informed decisions and practice based on evidence (DECIDE): protocol and
preliminary results. Implement Sci. (2013) 8:6–6. doi: 10.1186/1748-5908-8-6
18. Guldbrandsson K, Stenström N, Winzer R. The DECIDE evidence to
recommendation framework adapted to the public health field in Sweden.
Health Promot Int Dec. (2016) 31:749–54. doi: 10.1093/heapro/dav060
19. Glenton C, Lewin S, Gülmezoglu AM. Expanding the evidence
base for global recommendations on health systems: strengths and
challenges of the OptimizeMNH guidance process. Implement Sci. (2016)
11:98. doi: 10.1186/s13012-016-0470-y
20. World Health Organization. Practical Guidance for scaling up health
service innovations. (2009). Available online at:
(accessed March 31, 2022).
21. Haldane V, Chuah FLH, Srivastava A, Singh SR, Koh GCH, Seng CK, et al.
Community participation in health services development, implementation,
and evaluation: a systematic review of empowerment, health, community,
and process outcomes. PLoS ONE. (2019) 5. doi: 10.1371/journal.pone.02
22. Jackson JA, Spilman SK, Kingery LK, Oetting TW, Taylor MJ, Pruett
WM, et al. Implementation of high-flow nasal cannula therapy outside the
intensive care setting. Respir Care. (2021) 66:357. doi: 10.4187/respcare.
23. Wang T, Lurie M, Govindasamy D, Mathews C. The Effects of school-
based condom availability programs (CAPs) on condom acquisition, use
and sexual behavior: a systematic review. AIDS Behav. (2018) 22:308–
20. doi: 10.1007/s10461-017-1787-5
24. Tanahashi T. Health service coverage and its evaluation. Bull World Health
Organ. (1978) 56:295–303.
25. Brownson RC, Kumanyika SK, Kreuter MW, Haire-Joshu D. Implementation
science should give higher priority to health equity. Implement Sci. (2021)
16:28. doi: 10.1186/s13012-021-01097-0
26. Baumann AA, Cabassa LJ. Reframing implementation science to
address inequities in healthcare delivery. BMC Health Serv Res. (2020)
20:190. doi: 10.1186/s12913-020-4975-3
27. Odeny B. Closing the health equity gap: a role for implementation science?
PLoS Med. (2021) 18:e1003762–e1003762. doi: 10.1371/journal.pmed.
28. Milat AJ, Bauman A, Redman S. Narrative review of models and success
factors for scaling up public health interventions. Implement Sci. (2015)
10:113. doi: 10.1186/s13012-015-0301-6
29. Bartsch SM, O’Shea KJ, Ferguson MC, Bottazzi ME, Wedlock PT, Strych U,
et al. Vaccine efficacy needed for a COVID-19 coronavirus vaccine to prevent
or stop an epidemic as the sole intervention. Am J Prev Med Oct. (2020)
59:493–503. doi: 10.1016/j.amepre.2020.06.011
30. Quinn SC, Jamison AM, Freimuth V. Communicating effectively about
emergency use authorization and vaccines in the COVID-19 pandemic. Am
J Public Health. (2021) 111:355–8. doi: 10.2105/AJPH.2020.306036
31. Lazarus JV, Ratzan SC, Palayew A, Gostin LO, Larson HJ, Rabin K, et al.
global survey of potential acceptance of a COVID-19 vaccine. Nat Med. (2021)
27:225–8. doi: 10.1038/s41591-020-1124-9
32. Solís Arce JS, Warren SS, Meriggi NF, Scacco A, McMurry N, Voors M,
et al. COVID-19 vaccine acceptance and hesitancy in low- and middle-income
countries. Nat Med. (2021) 27:1385–94. doi: 10.1101/2021.03.11.21253419
33. Jasuja GK, Meterko M, Bradshaw LD, Carbonaro R, Clayman
ML, LoBrutto L, et al. Attitudes and intentions of US veterans
regarding COVID-19 vaccination. JAMA Netw Open. (2021)
4:e2132548. doi: 10.1001/jamanetworkopen.2021.32548
34. Elwy AR, Clayman ML, LoBrutto L, Miano D, Ann Petrakis B, Javier S, et al.
Vaccine hesitancy as an opportunity for engagement: A rapid qualitative study
of patients and employees in the U.S. Veterans Affairs healthcare system.
Vaccine X Dec. (2021) 9:100116. doi: 10.1016/j.jvacx.2021.100116
35. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM,
et al. refined compilation of implementation strategies: results from the Expert
Recommendations for Implementing Change (ERIC) project. Implement Sci.
(2015) 10:21. doi: 10.1186/s13012-015-0209-1
36. Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al.
Methods to improve the selection and tailoring of implementation strategies. J
Behav Health Serv Res Apr. (2017) 44:177–94. doi: 10.1007/s11414-015-9475-6
37. Dubé E, Leask J, Wolff B, Kickler B, Balaban V, Hosein E,
et al. The WHO tailoring immunization programmes (TIP)
approach: review of implementation to date. Vaccine. (2018)
36:1509–15. doi: 10.1016/j.vaccine.2017.12.012
38. Butler R, MacDonald NE. Diagnosing the determinants of vaccine hesitancy in
specific subgroups: the Guide to Tailoring Immunization Programmes (TIP).
Vaccine. (2015) 33:4176–9. doi: 10.1016/j.vaccine.2015.04.038
39. MacDonald NE, Butler R, Dubé E. Addressing barriers to
vaccine acceptance: an overview. Hum Vaccin Immunother. (2018)
14:218–24. doi: 10.1080/21645515.2017.1394533
40. Siddique AR, Singh P, Trivedi G. Role of Social Mobilization (Network) in
Polio Eradication in India. Indian Pediatr. (2016) 53:S50–6.
41. Momplaisir F, Haynes N, Nkwihoreze H, Nelson M, Werner RM, Jemmott J.
Understanding drivers of coronavirus disease 2019 vaccine hesitancy among
blacks. Clin Infect Dis. (2021) 73:1784–9. doi: 10.1093/cid/ciab102
42. Wamai RG, Ayissi CA, Oduwo GO, Perlman S, Welty E, Manga S, et al.
Assessing the effectiveness of a community-based sensitization strategy in
creating awareness about HPV, cervical cancer and HPV vaccine among
parents in north west cameroon. J Community Health. (2012) 37:917–
26. doi: 10.1007/s10900-012-9540-5
43. Rajkumari B, Keisam A, Haobam D, Thounaojam T, Haobam DS. Evaluation
of vaccination coverage of measles-rubella campaign in Imphal East District,
Manipur: a cross-sectional study. Indian J Public Health. (2020) 64:173–7.
doi: 10.4103/ijph.IJPH_361_19
44. Bonu S, Rani M, Razum O. Global public health mandates in a diverse
world: the polio eradication initiative and the expanded programme on
immunization in sub-Saharan Africa and South Asia. Health Policy. (2004)
70:327–45. doi: 10.1016/j.healthpol.2004.04.005
Frontiers in Health Services | 9May 2022 | Volume 2 | Article 897227
Pilar et al. COVID-19 Implementation Outcomes and Strategies
45. Kelley RT, Hannans A, Kreps GL, Johnson K. The Community Liaison
Program: a health education pilot program to increase minority awareness
of HIV and acceptance of HIV vaccine trials. Health Educ Res Aug. (2012)
27:746–54. doi: 10.1093/her/cys013
46. Chou W-YS, Budenz A. Considering emotion in COVID-
19 vaccine communication: addressing vaccine hesitancy and
fostering vaccine confidence. Health Commun. (2020) 35:1718–
22. doi: 10.1080/10410236.2020.1838096
47. Berry SD, Johnson KS, Myles L, Herndon L, Montoya A, Fashaw
S, et al. Lessons learned from frontline skilled nursing facility staff
regarding COVID-19 vaccine hesitancy. J Am Geriatr Soc. (2021) 69:1140–
6. doi: 10.1111/jgs.17136
48. Agley J, Xiao Y, Thompson EE, Golzarri-Arroyo L. COVID-
19 misinformation prophylaxis: protocol for a randomized trial
of a brief informational intervention. JMIR Res Protoc. (2020)
9:e24383. doi: 10.2196/24383
49. SteelFisher GK, Blendon RJ, Caporello H. An uncertain public - encouraging
acceptance of Covid-19 vaccines. N Engl J Med. (2021) 384:1483–
7. doi: 10.1056/NEJMp2100351
50. Wood S, Schulman K. Beyond politics - promoting Covid-
19 vaccination in the United States. N Engl J Med. (2021)
384:e23. doi: 10.1056/NEJMms2033790
51. Freeman D, Loe BS Yu L-M, Freeman J, Chadwick A, Vaccari C, Shanyinde M,
et al. Effects of different types of written vaccination information on COVID-
19 vaccine hesitancy in the UK (OCEANS-III): a single-blind, parallel-
group, randomised controlled trial. Lancet Public health. (2021) 6:e416–
27. doi: 10.1016/S2468-2667(21)00096-7
52. Khubchandani J, Sharma S, Price JH, Wiblishauser MJ, Sharma
M, Webb FJ. COVID-19 vaccination hesitancy in the united
states: a rapid national assessment. J Commun Health. (2021)
46:270–7. doi: 10.1007/s10900-020-00958-x
53. Volpp KG, Loewenstein G, Buttenheim AM. Behaviorally informed strategies
for a national COVID-19 vaccine promotion program. JAMA. (2021)
325:125–6. doi: 10.1001/jama.2020.24036
54. Muzumdar JM, Cline RR. Vaccine supply, demand, and policy: a primer. J Am
Pharm Assoc. (2009) 49:e87–99. doi: 10.1331/JAPhA.2009.09007
55. Frank R, Dach L, Lurie N. It was the government that produced
COVID-19 vaccine success. Health Affairs May. (2021) 14:2021.
doi: 10.1377/forefront.20210512.191448
56. Zuma ND. Disaster Management Act, 2002: draft amendments to regulations
issued in terms of section 27(2) of the disaster management act, 2002. In:
Governance DoC-O, editor. (2021).
57. Haidari LA, Connor DL, Wateska AR, Brown ST, Mueller LE, Norman BA,
et al. Augmenting transport versus increasing cold storage to improve vaccine
supply chains. PLoS ONE. (2013) 8:e64303. doi: 10.1371/journal.pone.0064303
58. World Health Organization. Guidance on developing a national deployment
and vaccination plan for COVID-19 vaccines: Interim guidance. (2021).
Available online at:
59. Banerjee A, Chandrasekhar AG, Dalpath S, Duflo E, Floretta J, Jackson
MO, et al. Selecting the most effective nudge: evidence from a large-
scale experiment on immunization. In: NBER Working Papers 28726.
(2021). doi: 10.3386/w28726
60. Ozawa S, Zhou M, Wonodi C, Chen H-H, Bridges JFP. Parents’ preferences
for interventions to improve childhood immunization uptake in northern
Nigeria. Vaccine. (2018) 36:2833–41. doi: 10.1016/j.vaccine.2018.03.073
61. World Health Organization. Working together: an integration resource guide
for immunization services throughout the life course. Licence: CC BY-NCSA
3.0 IGO. (2018). Available online at:
276546 (accessed March 31, 2022).
62. Schmelz K. Enforcement may crowd out voluntary support for COVID-
19 policies, especially where trust in government is weak and in a
liberal society. In: Proceedings of the National Academy of Sciences
of the United States of America. (2021) 118. doi: 10.1073/pnas.20163
63. Bayer R, Dupuis L. Tuberculosis, public health, and civil liberties. Annu Rev
Public Health. (1995) 16:307–26. doi: 10.1146/annurev.pu.16.050195.001515
64. Alleman MM, Meyer SA, Mulumba A, Nyembwe M, Riziki Y, Mbule A,
et al. Improved acute flaccid paralysis surveillance performance in the
Democratic Republic of the Congo, 2010-2012. J Infect Dis. (2014) 210:S50–
61. doi: 10.1093/infdis/jit670
65. Shelton RC, Adsul P, Oh A, Moise N, Griffith DM. Application
of an antiracism lens in the field of implementation science
(IS): Recommendations for reframing implementation research
with a focus on justice and racial equity. Implement Res
Pract. (2021) 2:26334895211049482. doi: 10.1177/263348952110
66. Musa A, Mkanda P, Manneh F, Korir C, Warigon C, Gali E, et al.
Youth group engagement in noncompliant communities during supplemental
immunization activities in kaduna, Nigeria, in 2014. J Infect Dis. (2016)
213:S91–5. doi: 10.1093/infdis/jiv510
67. Privor-Dumm L, King T. Community-based strategies to engage pastors can
help address vaccine hesitancy and health disparities in black communities.
J Health Commun. (2020) 25:827–30. doi: 10.1080/10810730.2021.18
68. Lahariya C, Khandekar J, Ray TK, Meenakshi, Pradhan SK. Role of an area
specific approach to increase community participation in pulse polio program
in a locality of south Delhi. J Commun Dis. (2007) 39:245–8.
69. Wesevich A, Chipungu J, Mwale M, Bosomprah S, Chilengi R. Health
promotion through existing community structures: a case of churches’ roles
in promoting rotavirus vaccination in rural zambia. J Prim Care Commun
Health. (2016) 7:81–7. doi: 10.1177/2150131915622379
70. Odone A, Ferrari A, Spagnoli F, Visciarelli S, Shefer A, Pasquarella C, et al.
Effectiveness of interventions that apply new media to improve vaccine
uptake and vaccine coverage. Hum Vaccin Immunother. (2015) 11:72–
82. doi: 10.4161/hv.34313
71. Ratzan S, Schneider EC, Hatch H, Cacchione J. Missing the point—how
primary care can overcome Covid-19 vaccine “Hesitancy”. New Engl J Med.
(2021) 384:e100. doi: 10.1056/NEJMp2106137
72. Khan S, Chambers D, Neta G. Revisiting time to translation: implementation
of evidence-based practices (EBPs) in cancer control. Cancer Causes Control
Mar. (2021) 32:221–30. doi: 10.1007/s10552-020-01376-z
73. Proctor Enola K, Geng E, A. new lane for science. Science. (2021) 374:659–
659. doi: 10.1126/science.abn0184
Author Disclaimer: The findings and conclusions in this paper are those of the
authors and do not necessarily represent the official positions of the National
Institutes of Health. Additionally, the views expressed in this article are those of
the authors and do not necessarily reflect the position or policy of the Department
of Veterans Affairs or the United States government.
Conflict of Interest: The authors declare that the research was conducted in the
absence of any commercial or financial relationships that could be construed as a
potential conflict of interest.
Publisher’s Note: All claims expressed in this article are solely those of the authors
and do not necessarily represent those of their affiliated organizations, or those of
the publisher, the editors and the reviewers. Any product that may be evaluated in
this article, or claim that may be made by its manufacturer, is not guaranteed or
endorsed by the publisher.
Copyright © 2022 Pilar, Elwy, Lushniak, Huang, McLoughlin, Hooley, Nadesan-
Reddy, Sandler, Moshabela, Alonge, Geng and Proctor. This is an open-access article
distributed under the terms of the Creative Commons Attribution License (CC BY).
The use, distribution or reproduction in other forums is permitted, provided the
original author(s) and the copyright owner(s) are credited and that the original
publication in this journal is cited, in accordance with accepted academic practice.
No use, distribution or reproduction is permitted which does not comply with these
Frontiers in Health Services | 10 May 2022 | Volume 2 | Article 897227
... There is a range of implementation frameworks that exist to guide the design of a robust hybrid trial, including process frameworks, determinant frameworks, and evaluation frameworks. 6 Below we have listed a few of the most frequently used outcomes within implementation frameworks, with definitions adapted from a landmark paper by Proctor et al., 7 and COVID-specific updates by Pilar et al., 8 and encourage the readership to continue their learning via these papers. Please note that below, each outcome is framed relative to an EBP, but could also be framed relative to an implementation strategy (e.g., acceptability of a strategy). ...
... • Health equity involves fair access to treatment or innovation without avoidable or remediable differences among groups of people. 8 • Penetration (also known as reach) is the degree to which an EBP is integrated into the desired setting and subunits of that setting (e.g., the percentage of wards in a hospital using an EBP, or the percentage of eligible patients who receive an EBP within a primary care system). ...
Full-text available
Background: The COVID-19 pandemic impacted healthcare delivery worldwide, including pediatric cancer care, with a disproportionate effect in resource-limited settings. This study evaluates its impact on existing quality improvement (QI) programs. Methods: We conducted 71 semi-structured interviews of key stakeholders at five resource-limited pediatric oncology centers participating in a collaborative to implement Pediatric Early Warning System (PEWS). Interviews were conducted virtually using a structured interview guide, recorded, transcribed, and translated into English. Two coders developed a codebook of a priori and inductive codes and independently coded all transcripts, achieving a kappa of 0.8-0.9. Thematic analysis explored the impact of the pandemic on PEWS. Results: All hospitals reported limitations in material resources, reduction in staffing, and impacts on patient care due to the pandemic. However, the impact on PEWS varied across centers. Identified factors that promoted or limited ongoing PEWS use included the availability of material resources needed for PEWS, staff turnover, PEWS training for staff, and the willingness of staff and hospital leaders to prioritize PEWS. Consequently, some hospitals were able to sustain PEWS; others halted or reduced PEWS use to prioritize other work. Similarly, the pandemic delayed plans at all hospitals to expand PEWS to other units. Several participants were hopeful for future expansion of PEWS post-pandemic. Conclusion: The COVID-19 pandemic created challenges for sustainability and scale of PEWS, an ongoing QI program, in these resource-limited pediatric oncology centers. Several factors mitigated these challenges and promoted ongoing PEWS use. These results can guide strategies to sustain effective QI interventions during future health crises.
Full-text available
Background The challenges of implementing evidence-based innovations (EBIs) are widely recognized among practitioners and researchers. Context, broadly defined as everything outside the EBI, includes the dynamic and diverse array of forces working for or against implementation efforts. The Consolidated Framework for Implementation Research (CFIR) is one of the most widely used frameworks to guide assessment of contextual determinants of implementation. The original 2009 article invited critique in recognition for the need for the framework to evolve. As implementation science has matured, gaps in the CFIR have been identified and updates are needed. Our team is developing the CFIR 2.0 based on a literature review and follow-up survey with authors. We propose an Outcomes Addendum to the CFIR to address recommendations from these sources to include outcomes in the framework. Main text We conducted a literature review and surveyed corresponding authors of included articles to identify recommendations for the CFIR. There were recommendations to add both implementation and innovation outcomes from these sources. Based on these recommendations, we make conceptual distinctions between (1) anticipated implementation outcomes and actual implementation outcomes, (2) implementation outcomes and innovation outcomes, and (3) CFIR-based implementation determinants and innovation determinants. Conclusion An Outcomes Addendum to the CFIR is proposed. Our goal is to offer clear conceptual distinctions between types of outcomes for use with the CFIR, and perhaps other determinant implementation frameworks as well. These distinctions can help bring clarity as researchers consider which outcomes are most appropriate to evaluate in their research. We hope that sharing this in advance will generate feedback and debate about the merits of our proposed addendum.
Full-text available
Background Despite the promise of implementation science (IS) to reduce health inequities, critical gaps and opportunities remain in the field to promote health equity. Prioritizing racial equity and antiracism approaches is critical in these efforts, so that IS does not inadvertently exacerbate disparities based on the selection of frameworks, methods, interventions, and strategies that do not reflect consideration of structural racism and its impacts. Methods Grounded in extant research on structural racism and antiracism, we discuss the importance of advancing understanding of how structural racism as a system shapes racial health inequities and inequitable implementation of evidence-based interventions among racially and ethnically diverse communities. We outline recommendations for explicitly applying an antiracism lens to address structural racism and its manifests through IS. An anti-racism lens provides a framework to guide efforts to confront, address, and eradicate racism and racial privilege by helping people identify racism as a root cause of health inequities and critically examine how it is embedded in policies, structures, and systems that differentially affect racially and ethnically diverse populations. Results We provide guidance for the application of an antiracism lens in the field of IS, focusing on select core elements in implementation research, including: (1) stakeholder engagement; (2) conceptual frameworks and models; (3) development, selection, adaptation of EBIs; (4) evaluation approaches; and (5) implementation strategies. We highlight the need for foundational grounding in antiracism frameworks among implementation scientists to facilitate ongoing self-reflection, accountability, and attention to racial equity, and provide questions to guide such reflection and consideration. Conclusion We conclude with a reflection on how this is a critical time for IS to prioritize focus on justice, racial equity, and real-world equitable impact. Moving IS towards making consideration of health equity and an antiracism lens foundational is central to strengthening the field and enhancing its impact. Plain language abstract There are important gaps and opportunities that exist in promoting health equity through implementation science. Historically, the commonly used frameworks, measures, interventions, strategies, and approaches in the field have not been explicitly focused on equity, nor do they consider the role of structural racism in shaping health and inequitable delivery of evidence-based practices/programs. This work seeks to build off of the long history of research on structural racism and health, and seeks to provide guidance on how to apply an antiracism lens to select core elements of implementation research. We highlight important opportunities for the field to reflect and consider applying an antiracism approach in: 1) stakeholder/community engagement; 2) use of conceptual frameworks; 3) development, selection and adaptation of evidence-based interventions; 4) evaluation approaches; 5) implementation strategies (e.g., how to deliver evidence-based practices, programs, policies); and 6) how researchers conduct their research, with a focus on racial equity. This is an important time for the field of implementation science to prioritize a foundational focus on justice, equity, and real-world impact through the application of an anti-racism lens in their work.
Full-text available
Importance Compared with the general population, veterans are at high risk for COVID-19 and have a complex relationship with the government. This potentially affects their attitudes toward receiving COVID-19 vaccines. Objective To assess veterans’ attitudes toward and intentions to receive COVID-19 vaccines. Design, Setting, and Participants This cross-sectional web-based survey study used data from the Department of Veterans Affairs (VA) Survey of Healthcare Experiences of Patients’ Veterans Insight Panel, fielded between March 12 and 28, 2021. Of 3420 veterans who were sent a link to complete a 58-item web-based survey, 1178 veterans (34%) completed the survey. Data were analyzed from April 1 to August 25, 2021. Exposures Veterans eligible for COVID-19 vaccines. Main Outcomes and Measures The outcomes of interest were veterans’ experiences with COVID-19, vaccination status and intention groups, reasons for receiving or not receiving a vaccine, self-reported health status, and trusted and preferred sources of information about COVID-19 vaccines. Reasons for not getting vaccinated were classified into categories of vaccine deliberation, dissent, distrust, indifference, skepticism, and policy and processes. Results Among 1178 respondents, 974 (83%) were men, 130 (11%) were women, and 141 (12%) were transgender or nonbinary; 58 respondents (5%) were Black, 54 veterans (5%) were Hispanic or Latino, and 987 veterans (84%) were non-Hispanic White. The mean (SD) age of respondents was 66.7 (10.1) years. A total of 817 respondents (71%) self-reported being vaccinated against COVID-19. Of 339 respondents (29%) who were not vaccinated, those unsure of getting vaccinated were more likely to report fair or poor overall health (32 respondents [43%]) and mental health (33 respondents [44%]) than other nonvaccinated groups (overall health: range, 20%-32%; mental health: range, 18%-40%). Top reasons for not being vaccinated were skepticism (120 respondents [36%] were concerned about side effects; 65 respondents [20%] preferred using few medications; 63 respondents [19%] preferred gaining natural immunity), deliberation (74 respondents [22%] preferred to wait because vaccine is new), and distrust (61 respondents [18%] did not trust the health care system). Among respondents who were vaccinated, preventing oneself from getting sick (462 respondents [57%]) and contributing to the end of the COVID-19 pandemic (453 respondents [56%]) were top reasons for getting vaccinated. All veterans reported the VA as 1 of their top trusted sources of information. The proportion of respondents trusting their VA health care practitioner as a source of vaccine information was higher among those unsure about vaccination compared with those who indicated they would definitely not or probably not get vaccinated (18 respondents [26%] vs 15 respondents [15%]). There were no significant associations between vaccine intention groups and age (χ²4 = 5.90; P = .21) or gender (χ²2 = 3.99; P = .14). Conclusions and Relevance These findings provide information needed to develop trusted messages used in conversations between VA health care practitioners and veterans addressing specific vaccine hesitancy reasons, as well as those in worse health. Conversations need to emphasize societal reasons for getting vaccinated and benefits to one’s own health.
Full-text available
Although COVID-19 vaccines have been available to many U.S. Veterans Affairs (VA) healthcare system employees and Veteran patients since early 2021, vaccine receipt data indicates some groups are not receiving them. Our objective was to conduct a rapid qualitative assessment of Veterans' and VA employees' views on COVID-19 vaccination to inform clinical leaders' ongoing efforts to increase vaccine uptake across the VA. We employed semi-structured interviews and a focus group involving employees and Veterans as part of a quality improvement project between January and June 2021 at three VA medical centers. Thirty-one employees and 27 Veterans participated in semi-structured interviews; 5 Veterans from a national stakeholder organization participated in a focus group. Data were analyzed using directed content analysis, involving an a priori coding framework comprised of four domains with subcodes under each: contextual influences, barriers and facilitators, vaccine-specific issues, and VA/military experiences. We then classified initial codes into five categories of hesitancy: vaccine deliberation, dissent, distrust, indifference and skepticism. A subset of Veterans (n=14) and employees (n=8) identified as vaccine hesitant. Vaccine hesitancy categories were represented by subcodes of religion, culture, gender or socio-economic factors, perceptions of politics and policies, role of healthcare providers, and historical influences; (contextual influences); knowledge or awareness of vaccines, perceived susceptibility to COVID-19, and beliefs and attitudes about health and illness (barriers and facilitators); vaccine development process (vaccine-specific issues) and military experiences (VA/military factors). Facilitators involved talking with trusted others, ease of vaccine access, and perceptions of family and societal benefits of vaccines. Vaccine hesitancy is multi-faceted and likely requires multiple strategies for engaging in conversations to address Veteran and VA employee concerns. Messages should involve patient-centered communication strategies delivered by trusted healthcare providers and peers and should focus on addressing expected benefits for family, friends, and society.
Full-text available
Beryne Odeny discusses strategies to improve equity in health care and health research.
Full-text available
Widespread acceptance of COVID-19 vaccines is crucial for achieving sufficient immunization coverage to end the global pandemic, yet few studies have investigated COVID-19 vaccination attitudes in lower-income countries, where large-scale vaccination is just beginning. We analyze COVID-19 vaccine acceptance across 15 survey samples covering 10 low- and middle-income countries (LMICs) in Asia, Africa and South America, Russia (an upper-middle-income country) and the United States, including a total of 44,260 individuals. We find considerably higher willingness to take a COVID-19 vaccine in our LMIC samples (mean 80.3%; median 78%; range 30.1 percentage points) compared with the United States (mean 64.6%) and Russia (mean 30.4%). Vaccine acceptance in LMICs is primarily explained by an interest in personal protection against COVID-19, while concern about side effects is the most common reason for hesitancy. Health workers are the most trusted sources of guidance about COVID-19 vaccines. Evidence from this sample of LMICs suggests that prioritizing vaccine distribution to the Global South should yield high returns in advancing global immunization coverage. Vaccination campaigns should focus on translating the high levels of stated acceptance into actual uptake. Messages highlighting vaccine efficacy and safety, delivered by healthcare workers, could be effective for addressing any remaining hesitancy in the analyzed LMICs. Survey data collected across ten low-income and middle-income countries (LMICs) in Asia, Africa and South America compared with surveys from Russia and the United States reveal heterogeneity in vaccine confidence in LMICs, with healthcare providers being trusted sources of information, as well as greater levels of vaccine acceptance in these countries than in Russia and the United States.
Full-text available
Background Due to striking disparities in the implementation of healthcare innovations, it is imperative that researchers and practitioners can meaningfully use implementation determinant frameworks to understand why disparities exist in access, receipt, use, quality, or outcomes of healthcare. Our prior work documented and piloted the first published adaptation of an existing implementation determinant framework with health equity domains to create the Health Equity Implementation Framework. We recommended integrating these three health equity domains to existing implementation determinant frameworks: (1) culturally relevant factors of recipients, (2) clinical encounter or patient-provider interaction, and (3) societal context (including but not limited to social determinants of health). This framework was developed for healthcare and clinical practice settings. Some implementation teams have begun using the Health Equity Implementation Framework in their evaluations and asked for more guidance. Methods We completed a consensus process with our authorship team to clarify steps to incorporate a health equity lens into an implementation determinant framework. Results We describe steps to integrate health equity domains into implementation determinant frameworks for implementation research and practice. For each step, we compiled examples or practical tools to assist implementation researchers and practitioners in applying those steps. For each domain, we compiled definitions with supporting literature, showcased an illustrative example, and suggested sample quantitative and qualitative measures. Conclusion Incorporating health equity domains within implementation determinant frameworks may optimize the scientific yield and equity of implementation efforts by assessing and ideally addressing implementation and equity barriers simultaneously. These practical guidance and tools provided can assist implementation researchers and practitioners to concretely capture and understand barriers and facilitators to implementation disparities.
Full-text available
Background The effectiveness of the COVID-19 vaccination programme depends on mass participation: the greater the number of people vaccinated, the less risk to the population. Concise, persuasive messaging is crucial, particularly given substantial levels of vaccine hesitancy in the UK. Our aim was to test which types of written information about COVID-19 vaccination, in addition to a statement of efficacy and safety, might increase vaccine acceptance. Methods For this single-blind, parallel-group, randomised controlled trial, we aimed to recruit 15 000 adults in the UK, who were quota sampled to be representative. Participants were randomly assigned equally across ten information conditions stratified by level of vaccine acceptance (willing, doubtful, or strongly hesitant). The control information condition comprised the safety and effectiveness statement taken from the UK National Health Service website; the remaining conditions addressed collective benefit, personal benefit, seriousness of the pandemic, and safety concerns. After online provision of vaccination information, participants completed the Oxford COVID-19 Vaccine Hesitancy Scale (outcome measure; score range 7–35) and the Oxford Vaccine Confidence and Complacency Scale (mediation measure). The primary outcome was willingness to be vaccinated. Participants were analysed in the groups they were allocated. p values were adjusted for multiple comparisons. The study was registered with ISRCTN, ISRCTN37254291. Findings From Jan 19 to Feb 5, 2021, 15 014 adults were recruited. Vaccine hesitancy had reduced from 26·9% the previous year to 16·9%, so recruitment was extended to Feb 18 to recruit 3841 additional vaccine-hesitant adults. 12 463 (66·1%) participants were classified as willing, 2932 (15·6%) as doubtful, and 3460 (18·4%) as strongly hesitant (ie, report that they will avoid being vaccinated for as long as possible or will never get vaccinated). Information conditions did not alter COVID-19 vaccine hesitancy in those willing or doubtful (adjusted p values >0·70). In those strongly hesitant, COVID-19 vaccine hesitancy was reduced, in comparison to the control condition, by personal benefit information (mean difference –1·49, 95% CI –2·16 to –0·82; adjusted p=0·0015), directly addressing safety concerns about speed of development (−0·91, –1·58 to –0·23; adjusted p=0·0261), and a combination of all information (−0·86, –1·53 to –0·18; adjusted p=0·0313). In those strongly hesitant, provision of personal benefit information reduced hesitancy to a greater extent than provision of information on the collective benefit of not personally getting ill (−0·97, 95% CI –1·64 to –0·30; adjusted p=0·0165) or the collective benefit of not transmitting the virus (−1·01, –1·68 to –0·35; adjusted p=0·0150). Ethnicity and gender were found to moderate information condition outcomes. Interpretation In the approximately 10% of the population who are strongly hesitant about COVID-19 vaccines, provision of information on personal benefit reduces hesitancy to a greater extent than information on collective benefits. Where perception of risk from vaccines is most salient, decision making becomes centred on the personal. As such, messaging that stresses the counterbalancing personal benefits is likely to prove most effective. The messaging from this study could be used in public health communications. Going forwards, the study highlights the need for future health campaigns to engage with the public on the terrain that is most salient to them. Funding National Institute for Health Research (NIHR) Oxford Biomedical Research Centre and NIHR Oxford Health Biomedical Research Centre.
A recent Science editorial on the social and political headwinds that have blunted, obfuscated, and confused public behavior in the United States' COVID-19 response cautioned both politicians who appoint themselves scientists and scientists-including virologists and epidemiologists-to stay in their lanes. The warning raises an important question: Should science add another lane?