Journal of the National Cancer Institute Monographs, No. 44, 2012 127
DOI: 10.1093/jncimonographs/lgs019 Published by Oxford University Press 2012.
Clinical science is making significant strides and changing how
clinicians care for patients along the entire cancer care continuum.
These scientific advances are applied within an increasingly com-
plex social, organizational, and environmental context. Current
approaches to intervention research may be insufficient to address
this complexity. It is essential that we rethink the manner and
mode of interventions along the care continuum and their impli-
cations for affecting the quality, cost, and outcomes of the care
The previous 12 chapters in this monograph have attempted to
fill this need by illuminating and evaluating the role of multilevel
intervention (MLI) research in cancer care delivery. These chapters
were commissioned for this monograph either in preparation for
or in response to the National Cancer Institute (NCI)–organized
conference, Multilevel Interventions in Health Care: Building
Foundations for Future Research, held March 4–5, 2011 in Las
Vegas, Nevada. Guided by a multidisciplinary scientific program
committee drawn from government, academia, and health-care
delivery organizations, and attended by more than 160 researchers
and clinicians, the conference aimed to 1) describe the state of the
science in MLIs, 2) clarify the issues in the conceptualization of
multilevel effects, and 3) identify areas for building the foundation
of MLI research (eg, taxonomy, measurement, intervention
design). By addressing these goals, participants hoped to identify
promising areas of application throughout the cancer control
In this concluding chapter, we draw from the reviews and find-
ings of the preceding MLI research chapters to 1) assess the added
value of MLI research; 2) reflect on what has been learned to date
about the success and challenges of MLI research in cancer care
delivery; and 3) identify specific ways to improve the scientific
soundness, feasibility, and policy relevance of MLI research in can-
cer control and its contribution to the NCI research agenda (Box 1).
Multilevel Intervention Research: Lessons Learned and
Steven B. Clauser, Stephen H. Taplin, Mary K. Foster, Pebbles Fagan, Arnold D. Kaluzny
Correspondence to: Steven Clauser, PhD, Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute,
Executive Plaza North, Rm 4086, Bethesda, MD 28092-7344 (e-mail:email@example.com).
This summary reflects on this monograph regarding multilevel intervention (MLI) research to 1) assess its added value; 2)
discuss what has been learned to date about its challenges in cancer care delivery; and 3) identify specific ways to improve its
scientific soundness, feasibility, policy relevance, and research agenda. The 12 submitted chapters, and discussion of them at
the March 2011 multilevel meeting, were reviewed and discussed among the authors to elicit key findings and results address-
ing the questions raised at the outset of this effort. MLI research is underrepresented as an explicit focus in the cancer literature
but may improve implementation of studies of cancer care delivery if they assess contextual, organizational, and environmental
factors important to understanding behavioral and/or system-level interventions. The field lacks a single unifying theory,
although several psychological or biological theories are useful, and an ecological model helps conceptualize and communicate
interventions. MLI research designs are often complex, involving nonlinear and nonhierarchical relationships that may not be
optimally studied in randomized designs. Simulation modeling and pilot studies may be necessary to evaluate MLI interven-
tions. Measurement and evaluation of team and organizational interventions are especially needed in cancer care, as are atten-
tion to the context of health-care reform, eHealth technology, and genomics-based medicine. Future progress in MLI research
requires greater attention to developing and supporting relevant metrics of level effects and interactions and evaluating MLI
interventions. MLI research holds an unrealized promise for understanding how to improve cancer care delivery.
J Natl Cancer Inst Monogr 2012;44:127–133
Box 1. Our working definition of multilevel interventions
As noted in the introductory chapter, an intervention is a
specified strategy or set of strategies designed to change the
knowledge, perceptions, skills, and/or behavior of individuals,
groups, or organizations, with the goal of improving health
outcomes. The term “multilevel intervention” refers to an inter-
vention targeted to influence more than one contextual level
(individual, group, organization, and community). Our primary
interest is in MLI research that influences at least individual,
group, organizational, and societal contexts in the United
States that influence health-care delivery. The purpose of mul-
tilevel interventions is to affect the critical contextual issues and
create a more efficient, effective, and coordinated cancer care
delivery system that achieves relevant patient outcomes,
including improved survival, health-related quality of life, and
patient experience with care, at a reduced cost to all involved.
by guest on May 24, 2012
128 Journal of the National Cancer Institute Monographs, No. 44, 2012
The Added Value of MLI Research
One theme from the preceding chapters is that MLI research adds
richness to intervention science in cancer care delivery. As Stange
et al. and Charns et al. note, cancer has a significant intervention
research tradition, but it is almost exclusively focused on single-
level intervention studies, primarily targeted to patient and/or
physician behavior (1,2). Charns et al. (2) and Yano et al. (3) argue
that MLI can consider other factors and levels that, if left unad-
dressed, serve as potential unmeasured barriers or facilitators to
health-care improvements. By emphasizing the importance of con-
text, the Zapka et al. (4) and Yano et al. (3) articles illustrate another
way in which MLI research deepens our understanding of interven-
tion science. Most cancer care intervention studies lack details
about the multilevel context in which care is provided, making it
difficult to understand how or why the intervention succeeds or
fails or was not adopted (1,2). Environmental or policy factors,
whether at the organizational, community, state, or national level,
are often important to specify as mediators and moderators of
behavioral or system-level interventions. Even when they are not
modifiable, these factors frame the intervention problem at the
appropriate level of complexity necessary to understand how the
impact, direction, and sustainability of intervention effects are
shaped. Although MLIs are often more complex than single-level
interventions, they actually may be less costly to health-care systems
in the long run if they facilitate successful and sustainable improve-
ments in the quality of care.
Reviews by Stange et al. (1) and Sheinfeld Gorin et al. (5) illus-
trate that a few good MLI research examples exist, especially in
community-based intervention research like tobacco control, and
these examples demonstrate the potential power of MLI research
to facilitate positive and sustainable change. However, these stud-
ies are the exception rather than the rule in cancer intervention
research. In particular, almost all authors recognized that few
MLI research studies are done in actual care delivery settings and
that organizational- and policy-level interventions are systemati-
cally underrepresented in this literature, though they offer great
potential. Studies that intervene and measure effects on three or
more levels of the ecological model are virtually nonexistent (1,2).
The challenge remains to understand how to take advantage of
MLI research’s scientific potential to inform cancer care delivery
across the entire continuum.
Conceptual and Theoretical Issues
In the introduction to this monograph, we asked how theories and
definitions of context could be brought to bear on an MLI frame-
work (6). At least three frameworks for defining levels are used in
the monograph, including an ecological model (6), a psychological
model (7), and, as presented in the commentary by Scott, the con-
cept of an organizational field (8). These frameworks provide a
useful heuristic either explicitly or implicitly and are consistently
cited as useful frames of reference for conceptualizing MLIs. Yet,
no single unified theory explains how the community context
affects the behaviors of individuals seeking health care, the health
professionals providing care, and/or organizations providing health
care services, and no single conceptual framework predominates
The authors also note important gaps in the application of
theory to MLI studies. The cancer care intervention literature is
most developed at the individual level (ie, patients, consumers or
specific individual health-care providers) (1,2). This level is guided
by strong behavioral and psychological theories of individual
behaviors. Two areas found to be underdeveloped in cancer inter-
vention research that are replete in the general health-care litera-
ture are “teams” and “organizations” (9–13). Team-based care has
been associated with mortality reductions in hospital settings (12),
improvements in geriatric inpatient care (13), and as a critical ele-
ment in successful collaborations to improve the quality of depres-
sion, hypertension, and diabetic care in ambulatory settings (14).
Success with team approaches to care has become so well docu-
mented that the National Quality Forum has issued a recommen-
dation to “establish a proactive systematic, organization-wide
approach to developing team-based care through team-work training”
(15), and expand the Medical Home Model as a “team-based
approach to delivering care led by a personal physician” (16).
Organizations are a third level within our framework that needs
further consideration in cancer intervention research. In his
commentary, Scott (8) recognizes the centrality of organizations in
modern societal systems and suggests that MLI occurs within an
organization field. A field is heuristically composed of actors
(patients, families, providers, and organizations), institution logics
(norms, values, culture), and prevailing governance structures that
exercise control and formalize relationships (8,9). Organizations
are central and could include a single office practice with multiple
provider teams or a set of practices that each includes a set of
teams. The characteristics of the most common health-care orga-
nizations are that the activities of the members are organized
around achieving a specific set of goals, consistent with prevailing
norms and values, and that the structure is highly formalized. In
health care, formalization includes a financial and legal structure
that manages and controls reimbursement and the dispersion of
resources. By this definition, health-care organizations as part of
an organizational field could include a single practitioner’s office
or clinics within a managed care system. These organizations must
facilitate the activities of teams within them that actually deliver
care. Organizations provide some of the inputs that affect the pro-
cesses of teams and their outputs, so understanding and evaluating
how organizations facilitate teamwork is critical to improving
Although several monograph authors noted the importance of
health-care teams and organizations in MLI studies, none of them
developed the connection between these constructs and improved
quality of care. Much more of what is known about how health-
care teams and organizations function is in the management, social
psychology, and sociology literature rather than the medical liter-
ature. This separation of knowledge is a critical limitation to
improvements in the quality of health care. Donabedian (17) noted
long ago that quality was determined by both technical and inter-
personal aspects, where the latter were both contextual issues and
issues of interactions between individuals. Recently, these issues
have been the focus of a separate supplement on the interfaces
of primary and subspecialty oncology care (18). A critical area for
by guest on May 24, 2012
Journal of the National Cancer Institute Monographs, No. 44, 2012 129
future research is to bridge the lessons from management and the
technical aspects of care to solve some of the problems that we face
in delivering high-quality care. For example, Donald Berwick
argues that improving the US health-care system requires simulta-
neous pursuit of three aims: improving the experience of care,
improving the health of populations, and reducing per capita costs
of health care (19). Preconditions for this include interventions on
multiple levels, including the patient and family, organizational
redesign of physician practices to align financial management with
population health management, and macro-system integration so
payment incentives reward organizations that improve cost and
quality of care. The goal of improving the health of populations is
influenced by many forces well beyond the clinical encounter such
as public health efforts related to health behaviors and control of
environmental exposure. However, examining cancer care organi-
zations in the context of policy and practice is essential as well.
Nowhere is this need more clear than with multimodality cancer
care where new practice systems, training, and reimbursement
incentives are needed to help cancer specialists work more effec-
tively across the disciplines of surgery, radiation therapy, and
chemotherapy to optimize care delivery for cancer patients (20).
Viewing the problem as a whole rather than reducing it to its
individual parts conceptualizes care delivery as a complex adaptive
system. The opening article to this supplement argues this
approach, and some think it is not possible to address quality issues
without taking this holistic approach (6,21).
A source of conceptual confusion at the conference was how
to determine whether a study is multilevel or single level when
it targets physicians and patients. Traditional organizational and
psychological theory suggests that individuals are one level of
analysis that could include patients and/or the individual care pro-
vider. The care provider interacts with people seeking care, and
Engel acknowledged this in his ecological model by recognizing
the “physician–patient” dyad as a level of care above the individual
(22). People seeking care along the cancer care continuum interact
with individuals, groups, and organizations in the course of their
care. Care providers interact with each other, with patients, with
groups and organizations when delivering care. The ecological and
organizational framework helps conceptualize the levels and rele-
vant components, and the perspective that interpersonal interaction
is one mechanism of effect between individuals and the group
and between individuals and the organization. Thus, the number
of levels in a study depends on the number of contextual levels
targeted (ie, what social units are being targeted—individuals,
teams, organizations, and/or communities).
A holistic approach can be intimidating. Stepping back and
realizing that everything is connected in an interactive web of
communication and relationships may lend itself to paralysis
rather than intervention building. No single theory explains every-
thing, and no single disciplinary approach is sufficient. At some
point, the complex web must be broken into manageable parts, and
investigators need to assemble a group that can design an interven-
tion. Breaking the system into individual, team, and organizational
levels may be a place to begin. Then, build approaches that account
for time, consider organizational and team effects, conceptualize
the mechanism of the intervention, and conceive of the appropriate
Methodological Issues of Design and Measurement
Because of our interest in the causal relationships between the timing
of MLIs and health outcomes, we consider MLI research designs
emphasizing longitudinal data collection and interventions inher-
ently superior to cross-sectional approaches (23). Randomized
designs have long been the “gold standard” in cancer intervention
research, and cluster randomized designs are often effective when
measuring and intervening at multiple levels, especially when the
effects of organizations, groups, and individuals need to be sepa-
rately estimated (24). However, Cleary et al. note that in many MLI
research applications, randomized approaches are often infeasible
(eg, randomizing interventions among physicians within a group
practice) or inferior (eg, where generalizability or scalability of
MLIs are the priority research question) (25). Multimethod designs
(eg, using qualitative and quantitative scientific approaches), alter-
native evaluation models, and sophisticated statistical designs show
promise, especially in MLI research related to implementation
science and quality improvement research (3–5).
Many studies suffer from statistical limitations in estimating
effects across levels. However, existing statistical design principles
and approaches enable examination of how multilevel effects interact
on outcomes simultaneously (eg, path analysis, hierarchical linear
[or nonlinear] modeling, or other structural equation estimation
methods and modeling) (25). Still, collecting the breadth of data
necessary to support these complex models requires a major invest-
ment in data, often without theoretical confidence that the data
will support the complexities involved in these models or produce
expected results. In this case, Morrissey et al. make a good case that
simulation models may be a method to pretest intervention models
(although sufficient data to assess critical assumptions and model
sensitivity are still needed) or to examine the anticipated scale
effects of the MLI (26).
In the near term, Charns et al. note that the real challenge to MLI
research may be the limited experience in deploying measures
across levels, especially when applied to group-, organizational-,
and community-level measurement, in cancer care delivery
research (2). All statistical models presume the accurate and com-
plete measurement of the multiple contexts that are hypothesized
to influence cancer outcomes in various ways. Investigators need
to draw from other fields to begin testing and modifying these
measures to fit the circumstances of cancer care delivery. The
consequence of these limitations is that in the short term, it may
be necessary to improve multilevel measurements of effects, rather
than conduct MLIs.
The chapter by Charns et al. (2) identified several measurement
issues related to effective execution of MLI studies. These issues
include the need for congruency between theory, construct, and
measures; the lack of independence of some measures; potentially
nonhierarchical and nonlinear relationships between variables; the
need to confirm that intended effects have been/are achieved over
time and that unintended effects and barriers to implementation have
been/are captured over time; and the need to capture nontraditional
aspects of interventions (eg, practicality, feasibility, cost efficiency).
Few MLI research applications in cancer have used sophisticated
approaches to measure group- or organizational-level constructs,
such as leadership and team cohesion, although these measures
are well developed and used extensively in the management, social
by guest on May 24, 2012
130 Journal of the National Cancer Institute Monographs, No. 44, 2012
psychology, and general health literature (2). As discussed earlier,
modern cancer care is increasingly a team activity. Patients and
families work together to assure that informal supports are ade-
quate to cope with and manage their cancer (27). Treatments are
becoming multimodal, which involves multiple specialists (and
increasingly, primary care physicians) engaging in the execution of
complex sequencing of therapies that, for many tumors, last
months if not years (28). Measurement is not yet able to funda-
mentally capture and quantify the reality of patient/family “teams”
interacting with clinical teams, and this is an area of real need in
cancer care intervention research. Measures of policy influences at
multiple levels (organizational, state, and national) also are lacking.
In particular, they lack the sensitivity to the characteristics of
policy (eg, its restrictiveness or leniency) that serve as barriers or
facilitators of organizational, clinical, or patient self-management
approaches that lead to improved health outcomes (2). With the
ability to assess individual, group, organization, and community
effects, MLI research is uniquely suited to deploy such measures to
support these studies.
A final methodological issue is the argument that the complexity,
and thus cost of MLI research is beyond the capacity (ie, design,
recruitment, organizational, analytical, and funding capacity) of the
current grant funding approaches of most research organizations.
Cleary et al. (25), Alexander et al. (23), and Morrissey et al. (26) note
that many design options are suited to a variety of intervention
research issues. Simulation modeling, mixed-method designs, sophis-
ticated experimental designs that test multiple interventions and their
interactions rather than a single intervention at a time, and interdisci-
plinary action research designs, which engage key constituents in
rapid learning cycles, can provide pragmatic and practical approaches
to MLI research (29). The challenges of selecting theory and
evidence-based interventions may be addressed by using any of a
variety of conceptual frameworks (eg, Put Prevention into Practice;
Agency for Healthcare Research and Quality checklists; Strengths,
Weaknesses, Opportunities, Threats; force field analysis; Chronic
Care Model; and Reach, Efficacy/Effectiveness, Adoption,
Implementation and Maintenance) (30–37). The use of Internet-
based computer adaptive technology platforms also can reduce
the cost of standardized data collection considerably (38). Many
studies that target multiple measures at multiple levels in multiple
organizations will be expensive. MLI research may be an area
especially suited to conducting smaller pilot projects that test
various aspects of the broader model before funders are approached
with large-scale multisite intervention studies. Weiner et al. (7)
describe methods for more efficient selection and sequencing of
interventions in MLI studies.
Building Capacity, Systems, and MLI Research
Building capacity to move the field toward adopting, implementing,
evaluating, and sustaining MLI research must be deliberate and
comprehensive and may require some specialized infrastructure and
skills. This will require disciplines to work synergistically and across
traditional boundaries to examine the relevance of team, systems,
transdisciplinary science, and participatory research. Consideration
needs to be given to identifying crucial stakeholders from the
managerial, clinical, and policy communities to build capacity and
develop strategies for shifting organizational culture, norms, and
values to adopt and sustain MLIs.
Identifying Key Stakeholders
Identifying key stakeholders may seem simple or obvious. However,
in research and with the diffusion of a novel and complex MLI idea,
it is important to think carefully about the multiplicity of stakeholders
who must engage in all phases of the research process. This is
especially important in MLI research given the frequent need to get
buy in from organizational leaders and policy makers. As Yano et al.
(3) noted, if we neglect to invite appropriate stakeholders to
planning, implementation, and evaluation processes associated with
MLIs, then we risk reinventing the wheel, wasting resources,
or missing an opportunity to have expertise that will facilitate the
success of MLIs.
Working Synergistically to Build MLI Capacity
Although challenges exist both conceptually and methodologically,
there is precedence for addressing the cancer research enterprise in
the context of MLI research. We have learned many lessons from
funded research platforms about interventions, their context, and
the pros and cons of models of collaborative science.
For example, NCI funds multiple networks and systems to sup-
port intervention and epidemiological studies, some of which
include MLI approaches relevant to understanding cancer care.
These platforms include the Cancer Intervention and Surveillance
Modeling Network, Cancer Research Network (CRN), NCI
Comprehensive/Designated Cancer Centers, NCI Community
Cancer Centers Program (NCCCP), Community Clinical
Oncology Program, Patient Navigation Research Program,
Cancer Consortium for Outcomes Research and Surveillance,
the Breast Cancer Surveillance Consortium, and Centers of
Excellence for Cancer Communication Research Studies
(39–46). These research platforms include multiple partners
who take into consideration how team, systems, transdisci-
plinary science, and participatory research interact and support
the goals and objectives of their research. Yet, NCI-sponsored
practice-based networks like the CRN and the NCCCP mostly
support intervention studies at one or two levels. With the exception
of the Veterans Health Administration, few delivery-based studies
include three levels of intervention. Knowledge from multiple fields
will be needed to respond effectively to the new demands and expec-
tations of funders on research networks engaged in MLI research.
Furthermore, health-care organizations are facing increasing
costs, dwindling resources, and greater accountability from the
federal to local levels. Funder demands on the various delivery
systems through the grant or contract process may be incompati-
ble with developing research designs given existing operations or
prevailing norms. Therefore, as Yano et al. (3) note, engaging
stakeholders in the process at the onset is critical to implementa-
tion and sustainability. The delivery organizations may have to
identify new resources at the onset to facilitate change toward MLI
approaches. Although the research is critical, the practical compo-
nents of data collection in clinical settings are as critical if the goal
is to sustain change to support successful MLIs.
For example, we have learned from the field of tobacco control that
MLIs (comprehensive tobacco control) can reduce the prevalence
by guest on May 24, 2012
Journal of the National Cancer Institute Monographs, No. 44, 2012 131
rate of tobacco use and secondhand smoke exposure (47). The
challenge with this evidence-based MLI approach is that state
resources have diminished and funds set aside for comprehensive
control are now being used to fill state budget gaps. Comprehensive
tobacco control is no longer being implemented at the recom-
mended levels and declines in smoking rates in the United States
have stalled (48,49). Therefore, as we think about how to move
forward in building MLI capacity for the continuum of cancer care
delivery, it is important to consider strategies for sustaining and
optimizing research infrastructures and strategies for carrying out
and sustaining the practice of MLIs.
Funders and health/community systems also need to work syn-
ergistically and equitably with providers, patients, and consumers
of MLIs. Patients and consumers may include those who have
not benefited from single-level interventions, have been slow to
benefit, or who fall through the cracks within existing delivery
systems. Working with providers, consumers, and patients requires
participatory processes that engage them in research conceptuali-
zation, design, implementation, and evaluation. Furthermore,
communities and health system organizations will need to develop
communication and policy strategies to ensure patient access, use,
and benefits. This requires a clear understanding of the patients’
access to insurance, interpersonal factors, interpersonal relation-
ships, social support systems, and their history, culture, geography,
and perceptions about different interventions. Researchers need to
consider theories, measures, and methods that fit the populations
targeted for MLI.
Shifting Organizational Culture, Norms, and Values for
Adoption and Sustainability
Changing organizational cultures, norms, and values involves mul-
tiple processes and channels by which consumers and members of
social systems affected by MLIs are informed, engaged in the idea
that these interventions have the potential to enhance cancer care
for the system and its populations, and involved in their implemen-
tation. The acknowledgement of the process by which “innovation”
is diffused and disseminated is critical (50), particularly in an envi-
ronment where single-level approaches are standard, resources
are scarce, risk aversion is common practice, and stakeholders
have competing ideas on how to best provide cancer care across
The NCI consulted with leading members of the extramural
research community to develop a plan for “diffusing” consider-
ation of MLI science among multiple stakeholders, including
potential funders and researchers. This plan included developing
this monograph, hosting the meeting in March 2011, and engaging
new and existing NCI/NIH leadership in discussions to increase
knowledge on the value and benefits of MLI approaches. As with
any initiative of this size and scope, the process of persuading deci-
sion makers within the NCI and the larger research community of
the need to adopt this idea was met with mixed reactions. Clearly,
the NCI acknowledges that additional stakeholders, including
policy makers, health-care systems, insurers, patients, funders,
journals, providers, and many others, are needed to influence
thinking around the utility of MLIs. It is important to carefully
Table 1. MLI cancer research: issues and opportunities*
Research design, measurement,
Issues No overarching theory, although several
No standard definition of level of context
Unclear how to select interventions
Cancer MLI studies stress psychological,
social psychology, or biological theories
Randomized models are
feasible (eg, cluster randomized
designs) but may not be optimal
for all MLI applications
Team and organizational levels
Most intervention studies are single
level, single target, at the individual
Few delivery-oriented MLI studies
MLI studies limited to prevention and
Current research infrastructure not
aligned to support MLI studies
MLI research is expensive
OpportunitiesChoose theories or collection of theories
appropriate to research question
Consider a variety of established
conceptual frameworks: PPIP,
PRECEDE, AHRQ checklists, SWOT,
force field analysis, E2D2, Chronic
Care Model, RE-AIM
Use standardized definition of levels
based on types of human
aggregations: individual, team,
organization, community, nation
Let intervention target define
appropriate level; use theory and
causal modeling approaches to select
and package interventions and metrics
Consider a variety of promising
nonrandomized designs: for
quasi-experimental, rapid learning,
Consult management and
sociological sciences for
measures of teams and
Measure nontraditional aspects of
interventions (eg, practical,
feasible, scalable, cost-efficient)
Consider hierarchical linear (and
nonlinear) modeling; structural
equation modeling and
Build team-, organization-, and
into MLI designs; consider policy
interventions at multiple levels
Engage primary/specialty care
organizations to participate in MLI
studies related to improving cancer
diagnosis, treatment, survivorship
and end of life care
Develop multidisciplinary, team-based
MLI training curricula
Engage broad stakeholder community to
develop constituency for MLI research
Consider simulation modeling and
pilot studies to pretest intervention
* AHRQ = Agency for Healthcare Research and Quality; E2D2 = Exposure, Exploitation, and Data Dissemination; MLI = multilevel intervention; PPIP = Put
Prevention into Practice; PRECEDE = Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation; RE-AIM = Reach, Efficacy/
Effectiveness, Adoption, Implementation, and Maintenance; SWOT = Strengths, Weaknesses, Opportunities, Threats.
by guest on May 24, 2012
132 Journal of the National Cancer Institute Monographs, No. 44, 2012
consider the existing diversity of social norms around cancer care
in interacting systems, the perceived costs/benefits of MLIs, orga-
nizational culture and decision-making processes, and particular
obstacles and facilitators within systems that affect the successful
adoption and sustainability of MLIs.
For example, changing social norms regarding how best to
facilitate cancer care may require system-wide policy changes
among funders and individuals within the larger clinical, research,
and care delivery community. This may include training for peer
reviewers of grants; development of new study sections; and train-
ing and educational efforts among researchers, journal editors,
providers, and staff to familiarize stakeholders with MLI concepts,
methodologies, and design. In addition, the implementers must
consider short- and long-term costs/benefits because MLIs are
challenging, could initially be disruptive to systems, and gains to
the patient may take years.
An MLI Research Agenda
An important cross-cutting theme in this supplement is that MLI
research should be conceptualized, planned, and executed carefully
and will require enhanced infrastructure, training, and innovative
partnerships throughout the policy, clinical, and research commu-
nities to effectively build a sustainable approach to its widespread
application. The issues and opportunities identified in this supple-
ment for pursing these conceptual, methodological, and research
infrastructure challenges are summarized in Table 1. It is clear that
this transition will take time. Yet, major policy and research
changes are emerging in cancer care delivery that may serve as
leverage points to facilitate this transformation. The development
of genomic-based, individual targeted therapies is a potential “game
changer” in cancer care delivery because it fundamentally changes
early detection, diagnosis, and therapeutic processes associated with
cancer (51). Khoury et al. (52) argue that genomic medicine is
an excellent area for implementing MLI research, and point out
how little translational research has been conducted in this growing
area of medicine. They illustrate by describing how specific
genomic applications related to conditions such as Lynch Syndrome
are especially affected by influences at the state health policy, orga-
nizational, provider team, and patient and family levels. Similarly,
Flood et al. (53) illustrate that MLI research applications are espe-
cially promising for health reform, especially in terms of assessing
optimal approaches to delivering cancer care in new delivery
mechanisms, such as affordable care organizations. Other opportu-
nities for MLI research are present in eHealth interventions, which
have rapidly proliferated and can materialize simultaneously at the
patient, organizational, and community levels (54). We live in an age
of rapidly growing e-technology and policy change at the national,
state, and local levels. Reaching the tipping point can occur rapidly
when appropriate channels of communications are used to market
and diffuse the idea of MLIs in an effort to reach a critical mass of
change agents, decision makers, and patients.
The next generation of research must address three persistent chal-
lenges to progress in MLIs addressing health-care delivery: 1)
explaining interactions between the levels theoretically and
practically, 2) measuring the interactions and contextual effects in
quantifiable ways that build theory and contribute to stronger
interventions over time, and 3) developing the research infrastruc-
ture and training opportunities for MLI investigators. This is a
long-term agenda. In the short run, it will be important to define
levels, or organizational frames preferably as units of human aggre-
gation, develop measures, and show more evidence regarding the
relative effects of MLIs on health-care delivery to individuals and
the larger population being served. Consideration needs to be given
to the context of health-care reform, eHealth technology, and
genomics-based medicine that hold extraordinary opportunities to
improve cancer care outcomes. We also need to consider how this
context interacts with research initiatives to support the implemen-
tation and sustainability of MLIs.
1. Stange KC, Breslau ES, Dietrich AJ, Glasgow RE. State-of-the-art and
future directions in multilevel interventions across the cancer control
continuum. J Natl Cancer Inst Monogr. 2012;44:20–31.
2. Charns MP, Foster MK, Alligood EC, et al. Multilevel interventions:
measurement and measures. J Natl Cancer Inst Monogr. 2012;44:67–77.
3. Yano EM, Green LW, Glanz K, et al. Implementation and spread of
interventions into the multilevel context of routine practice and policy:
implications for the cancer care continuum. J Natl Cancer Inst Monogr.
4. Zapka J, Taplin SH, Ganz P, Grunfeld E, Sterba K. Multilevel factors
affecting quality: examples from the cancer care continuum. J Natl Cancer
Inst Monogr. 2012;44:11–19.
5. Sheinfeld Gorin S, Badr H, Krebs P, Prabhu Das I. Multilevel interven-
tions and racial/ethnic health disparities. J Natl Cancer Inst Monogr.
6. Taplin SH, Anhang Price R, Edwards HM, et al. Introduction: under-
standing and influencing mulitlevel factors across the cancer care continuum.
J Natl Cancer Inst Monogr. 2012;44:2–10.
7. Weiner BJ, Lewis MA, Clauser SB, Stitzenberg KB. In search of synergy:
strategies for combining interventions at multiple levels. J Natl Cancer Inst
8. Scott WR. Embedding the examination of multilevel factors in an organi-
zation field context. J Natl Cancer Inst Monogr. 2012;44:32–33.
9. Scott WR. Institutions and Organizations.Thousand Oaks, CA: Sage
10. West MA, Borrill CS, Dawson JF, Brodbeck F, Shapiro DA, Haward B.
Leadership clarity and team innovation in health care. Leadersh Q. 2003;
11. Salas E, Diaz Granados D, Weaver SJ, King H. Does team training work?
Principles for health care. Acad Emerg Med. 2008;15(11):1002–1009.
12. West MA, Guthrie JP, Dawson JF, Borrill CS, Carter M. Reducing
patient mortality in hospitals: the role of human resource management.
J Organ Behav. 2006;27(7):983–1002.
13. Scott J, Gade G, Venohr I, McKenzie M. Cooperative health care clinics:
a group approach to individual care. Geriatrics. 1998;53(5):68.
14. Lemieux-Charles L, McGuire WL. What do we know about health care
team effectiveness? A review of the literature. Med Care Res Rev. 2006;
15. National Quality Forum. Nursing Performance Measurement and Reporting:
A Status Report. Washington, DC: National Quality Forum; 2007.
16. Rosenthal TC. The medical home: growing evidence to support a new
approach to primary care. J Am Board Fam Med. 2008;21(5):427–440.
17. Donabedian A. Evaluating the quality of medical care. Milbank Q. 2005;
18. Taplin SH, Rodgers AB. Toward improving the quality of cancer care:
addressing the interfaces of primary and oncology-related subspecialty
care. J Natl Cancer Inst Monogr. 2010;40:3–10.
19. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and
cost. Health Aff (Milwood). 2008;27(3):759–769.
by guest on May 24, 2012
Journal of the National Cancer Institute Monographs, No. 44, 2012 133 Download full-text
20. Bach PB, Mirkin JN, Luke JJ. Episode-based payment for cancer care:
a proposed pilot for Medicare. Health Aff (Milwood). 2011;30(3):
21. Pronovost PJ, Goeschel CA. Time to take health delivery research seriously.
22. Engel GL. The need for a new medical model: a challenge for biomedicine.
Fam Syst Med. 1992;10(3):317–331.
23. Alexander J, Prabhu Das I, Johnson TP. Time issues in multilevel inter-
ventions for cancer treatment and prevention. J Natl Cancer Inst Monogr.
24. Donner A, Klar N. Design and Analysis of Cluster Randomization Trials in
Health Research. London, UK: Arnold; 2000.
25. Cleary PD, Gross CP, Zaslavsky AM, Taplin SH. Multilevel interventions:
study design and analysis issues. J Natl Cancer Inst Monogr. 2012;44:
26. Morrissey JP, Hassmiller Lich K, Anhang Price R, Mandelblatt J.
Computational modeling and multilevel cancer control interventions.
J Natl Cancer Inst Monogr. 2012;44:56–66.
27. Institute of Medicine; Adler NE, Page AEK, eds. Cancer Care for the Whole
Patient: Meeting Psychosocial Health Needs. Washington, DC: The National
Academies Press; 2008.
28. Fennell ML, Prabhu Das I, Clauser SB, Petrelli N, Salner A. The orga-
nization of multidisciplinary care teams: modeling internal and external
influences on cancer care quality. J Natl Cancer Inst Monogr. 2010;40:
29. Eisenberg EM, Baglia J, Pynes JE. Transforming emergency medicine
through narrative: qualitative action research at a community hospital.
Health Commun. 2006;19(3):197–208.
30. Prevention and chronic care resources. Agency for Healthcare Research
and Quality Prevention and Care Management Web site. http://www.
ahrq.gov/clinic/ppipix.htm. Accessed September 29, 2011.
31. Ko H, Turner T, Finnigan M. Systematic review of safety checklists for
use by medical care teams in acute hospital settings—limited evidence of
effectiveness. BMC Health Serv Res. 2011;11(1):211–211.
32. Hill T, Westbrook R. SWOT analysis: it’s time for a product recall. Long
Range Plan. 1997;30(1):46–52.
33. Helms MM, Nixon J. Exploring SWOT analysis—where are we now?: a
review of academic research from the last decade. J Strategy Manag. 2010;
34. Lewin K. Field Theory in Social Science and Selected Theoretical Papers. New
York, NY: Harper; 1951.
35. The chronic care model. Improving Chronic Illness Care Web site. http://
Model&s=2. Accessed September 25, 2011.
36. Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A.
Improving chronic illness care: translating evidence into action. Health Aff
(Milwood). 2001;20(6):64 –78.
37. Glasgow RE. RE-AIMing research for application: ways to improve
evidence for family medicine. J Am Board Fam Med. 2006;19(1):11–19.
38. Wright KB. Researching internet-based populations: advantages and
disadvantages of online survey research, online questionnaire authoring
software packages, and web survey services. J Computer-Mediated Commun.
39. Cronin KA, Feuer EJ, Clarke LD, Plevritis SK. Chapter 15: impact of
adjuvant therapy and mammography on U.S. mortality from 1975 to 2000:
comparison of mortality results from the CISNET breast cancer base case
analysis. J Natl Cancer Inst Monogr. 2006;36:112–121.
40. Wagner EH, Greene SM, Hart G, et al. Building a research consortium
of large health systems: the cancer research network. J Natl Cancer Inst
41. Cancer centers list. National Cancer Institute Office of Cancer Centers
Web site. http://cancercenters.cancer.gov/cancer_centers/cancer-centers-
list.html. Accessed September 29, 2011.
42. Clauser SB, Johnson MR, O’Brien DM, Beveridge JM, Fennell ML,
Kaluzny AD. Improving clinical research and cancer care delivery in com-
munity settings: evaluating the NCI community cancer centers program.
Implement Sci. 2009;4:63–63.
43. Ayanian JZ, Chrischilles EA, Fletcher RH, et al. Understanding cancer
treatment and outcomes: the Cancer Care Outcomes Research and
Surveillance Consortium [Erratum in J Clin Oncol. 2004;22(24):5026].
J Clin Oncol. 2004;22(15):2992–2996.
44. National Cancer Institute. Breast Cancer Surveillance Consortium: Evaluating
Screening Performance in Practice. NIH Publication 04-5490. http://breast
screening.cancer.gov/espp.pdf. Published April 2004. Accessed September 15,
45. Health communication and informatics research. National Cancer
Institute Web site. http://cancercontrol.cancer.gov/hcirb/index.html.
Updated May 4, 2011. Accessed September 29, 2011.
46. Centers of excellence in cancer communication research initiative.
National Cancer Institute Web site. http://cancercontrol.cancer.gov/hcirb/
ceccr/ceccr-index.html. Updated March 9, 2011. Accessed September 29,
47. Babb S, Tynan M, MacNeil A. State preemption of local smoke-free laws
in government work sites, private work sites, and restaurants – United
States, 2005-2009. MMWR. 2010;59(4):105–108.
48. King B, Dube S, Kaufmann R, Shaw L, Pechacek T. Vital signs: current
cigarette smoking among adults aged ≥ 18 years—United States, 2005-2010.
49. Griffin M, Babb SD, Tynan M, MacNeil AE. State preemption of local
tobacco control policies restricting smoking, advertising, and youth
access—United States, 2000-2010. MMWR. 2011;60(33):1124–1127.
50. Rogers EM. Diffusion of Innovations. 3rd ed. New York, NY: The Free
51. Collins FS. The Language of Life: DNA and the Revolution in Personalized
Medicine. New York, NY: Harper; 2010.
52. Khoury MJ, Coates RJ, Fennell ML, et al. Multilevel research and the
challenges of implementing genomic medicine. J Natl Cancer Inst Monogr.
53. Flood AB, Fennell ML, Devers KJ. Health reforms as examples of multilevel
interventions in cancer care. J Natl Cancer Inst Monogr. 2012;44:80–85.
54. Clauser SB, Wagner EH, Aiello Bowles EJ, Tuzzio L, Greene SM.
Improving modern cancer care through information technology. Am J
Prev Med. 2011;40:S198–S207.
This project has been funded in whole or in part with federal funds from the
National Cancer Institute, National Institutes of Health, under Contract No.
HHSN261200800001E. The content of this publication does not necessarily
reflect the views or policies of the Department of Health and Human Services,
nor does mention of trade names, commercial products, or organizations imply
endorsement by the US government.
Affiliations of authors: Applied Research Program (SBC) and Behavioral
Research Program (SHT) Division of Cancer Control and Population
Sciences, National Cancer Institute, Bethesda, MD; Earl G. Graves School
of Business and Management, Morgan State University, Baltimore, MD
(MKF); Cancer Prevention and Control Program, University of Hawaii Cancer
Center, Honolulu, HI (PF); Gillings School of Global Public Health, University
of North Carolina, Chapel Hill, NC (ADK).
by guest on May 24, 2012