A Multilevel Analysis of the Effects of the Practice Environment Scale of the Nursing Work Index on Nurse Outcomes

Abstract

Few researchers have examined how the components of the Practice Environment Scale of the Nursing Work Index (PES-NWI) relate to nurses' well-being at multiple organizational levels. The objective of the study was to perform a multilevel assessment of the relationships of the PES-NWI subscales with three nurse outcomes: job satisfaction, emotional exhaustion, and turnover intentions. Additionally, we tested the multilevel factor structure of the PES-NWI. In a sample of 699 full-time registered nurses in 79 units and 9 branches of a hospital system, relationships of the NWI with nurse outcomes were fairly consistent across levels of analysis. However, subscales contributed differently to the three outcomes, demonstrating the complexity of environmental influences on nurses' work experience. © 2013 Wiley Periodicals, Inc. Res Nurs Health.

Full text

Research in Nursing & Health
2013; 36:567–581
A Multilevel Analysis of the Effects of
the Practice Environment Scale of the
Nursing Work Index on
Nurse Outcomes
Allison S. Gabriel,
1*
Rebecca J. Erickson,
2**
Christina M. Moran,
3y
James M. Diefendorff,
4z
Gail E. Bromley
5§
1
Department of Management, Virginia Commonwealth University, 301 West Main Street PO Box 844000, Richmond,
Virginia 23284-4000, agabriel2@vcu.edu
2
Department of Sociology, The University of Akron, Akron, Ohio
3
PRADCO, Chagrin Falls, Ohio
4
Department of Psychology, The University of Akron, Akron, Ohio
5
College of Nursing, Kent State University, Kent, Ohio
Accepted 6 August 2013
Abstract: Few researchers have examined how the components of the
Practice Environment Scale of the Nursing Work Index (PES-NWI) relate
to nurses’ well-being at multiple organizational levels. The objective of
the study was to perform a multilevel assessment of the relationships
of the PES-NWI subscales with three nurse outcomes: job satisfaction,
emotional exhaustion, and turnover intentions. Additionally, we tested
the multilevel factor structure of the PES-NWI. In a sample of 699 full-
time registered nurses in 79 units and 9 branches of a hospital system,
relationships of the NWI with nurse outcomes were fairly consistent
across levels of analysis. However, subscales contributed differently
to the three outcomes, demonstrating the complexity of environmental
influences on nurses’ work experience. ß2013 Wiley Periodicals, Inc.
Res Nurs Health 36:567–581, 2013
Keywords: job satisfaction; emotional exhaustion; turnover intentions; multilevel
analysis; nurse practice environment; Practice Environment Scale; NursingWork Index
Although the recent recession within the United
States led to some easing of the decade-long
nursing shortage due to some part-time nurses
shifting to full-time hours, others delaying
retirement plans, and some patients choosing to
defer elective surgeries (Buerhaus, Auerbach, &
Staiger, 2009; Larson, 2009; Tri-Council for
Nursing, 2010), recent projections indicate that
The research reported here is an analysis of data from a larger study, “Identity
and Emotional Management Control in Health Care Settings,” funded by the
National Science Foundation (SES-1024271). A previous version of this article was
presented at the 72nd annual meeting of the Academy of Management.
Correspondence to Allison S. Gabriel
*
Assistant Professor of Management.
**
Professor of Sociology.
y
Management Consultant.
z
Associate Professor of Psychology.
§
Associate Dean for Academics.
Published online 30 September 2013 in Wiley Online Library
(wileyonlinelibrary.com). DOI: 10.1002/nur.21562

C2013 Wiley Periodicals, Inc.

the demand for registered nurses (RNs) will
continue to rise faster than other professions in
the coming decade (U.S. Department of Labor,
2012). This trend will likely be exacerbated by
the implementation of the Affordable Care Act
in 2014. Although demand is increasing, recent
studies suggest that RNs continue to leave their
jobs at a high rate (Kalisch, Lee, & Rochman,
2010) and that factors influencing dissatisfac-
tion, burnout, and turnover must be better
understood if these trends are to be slowed
(Brewer, Kovner, Green, & Cheng, 2009;
Kovner et al., 2007). Indeed, Aiken et al.
(2001), specified that 41% of nurses within the
United States were dissatisfied with their pres-
ent jobs, 43% of nurses scored in the high range
of job burnout, and 23% were planning on leav-
ing their current nursing jobs within the next
year.
Such trends, as well as concerns over
patient safety (Institute of Medicine, 2003),
have led researchers to investigate ways to
improve the nursing practice environment, par-
ticularly for those working in acute care hospi-
tals (Lake, 2007). The Practice Environment
Scale of the Nursing Work Index (PES-NWI;
Lake, 2002) continues to be the most widely
used measure of the nursing environment both
within the United States and internationally
(Warshawsky & Havens, 2011). The PES-NWI
targets features of acute care work environments
that have been shown to be associated with
attracting and retaining high quality nurses
(Lake, 2002) in a parsimonious scale that is
psychometrically sound at the individual and
hospital levels (Choi, Bakken, Larson, Du, &
Stone, 2004; Lake, 2007). Lake’s (2002) origi-
nal version of the PES-NWI contains five
dimensions: Nurse Participation in Hospital
Affairs, Nursing Foundations for Quality of
Care, Managerial Support (encompassing man-
ager ability, leadership, and support of nurses),
Staffing Aaequacy (also encompassing resource
adequacy), and Collegial Nurse-Physician Rela-
tionships. The characteristics of the work envi-
ronment measured by the PES-NWI have
consistently been linked to nursing outcomes
such as burnout (Vahey, Aiken, Sloane, Clarke,
& Vargas, 2004), emotional exhaustion (Van
Bogaert, Clarke, Vermeyen, Meulemans, & Van
den Heyning, 2009; Van Bogaert, Clarke, Roe-
lant, Meulemans, & Van de Heyning, 2010),
and job satisfaction (Foley, Kee, Minick, Harvey,
& Jennings, 2002). As articulated in greater
detail below, in this article, we seek to advance
methodological knowledge about the PES-NWI
and substantive knowledge about the multilevel
effects of the practice environment on emotional
exhaustion, job satisfaction, and turnover inten-
tions. In the current study, emphasis was placed
on the subscales that focus on the experiences
of staff nurses that distinguish between units
(Aiken & Patrician, 2000; Lake, 2002). As
a result, we did not include the subscale of
Nursing Foundations for Quality of Care.
The PES-NWI and Nurse Outcomes:
Advantages of Multilevel Analysis
Much work has focused on examining psycho-
metric properties of the PES-NWI (e.g., Lake,
2002, 2007; Lake & Friese, 2006; Leiter &
Laschinger, 2006) or the relationships of the
scale with outcomes in different contexts (e.g.,
psychiatric nurses, Hanrahan, 2007; global
usage, Warshawsky & Havens, 2011; Asian
cultures, Chiang & Lin, 2009; Liou & Cheng,
2009). However, fewer researchers have
explored how the PES-NWI operates within
multilevel contexts. For instance, some have
used the scale at the individual level (e.g.,
Budge, Carryer, & Woods, 2003; Gabriel,
Diefendorff, & Erickson, 2011; Rafferty, Ball,
& Aiken, 2001), while others have used it at the
unit level (Vahey et al., 2004; Van Bogaert
et al., 2009) or hospital level (e.g., Kutney-Lee,
Lake, & Aiken, 2009; Lake & Friese, 2006).
Such differences create problems for research-
ers, given that inconsistency in levels of analy-
sis can create a disconnect between the theory
surrounding a construct and the actual level of
analysis used to assess the construct (Chan,
1998; Klein, Danereau, & Hall, 1994; Kozlow-
ski & Klein, 2000). Because structures are
nested within organizations, researchers must
model multilevel effects when they exist
(Bliese, 2002). Unmodeled nesting can result in
increases in Type I or Type II errors, both of
which result in incorrect conclusions (e.g., con-
cluding a relationship exists when it does not,
or vice versa). Thus, not taking into account
nesting structures may obscure how the aspects
of the practice environment assessed in the
PES-NWI influence nurse well-being.
For instance, much of the work on the
PES-NWI has ignored the fact that nurses
are often nested within units, suggesting that
multilevel confirmatory factor analysis (CFA)
is necessary to understand the measurement
properties of the subscales of the PES-NWI
(see Gajewski, Boyle, Miller, Oberhelman, &
Research in Nursing & Health
568 RESEARCH IN NURSING & HEALTH

Dunton, 2010, as an exception). Multilevel CFA
of nested observations is necessary because (a)
the observations are correlated (rather than
independent) violating a key assumption of fac-
tor analysis, and (b) the nature of a construct
can differ across levels of analysis, making it
critical to test whether a scale has the same
dimensional properties at each level (Dyer,
Hanges, & Hall, 2005). Moreover, in their
research on Belgian nurses, Van Bogaert et al.
(2010) noted that “standard” regression analysis
“ignore[s] the correlated structure of the obser-
vations in clustered data and will underestimate
standard errors and increase the likelihood
of Type-I error” (p. 1667). Using multilevel
modeling, they found that high scores on the
PES-NWI subscales were related to less burnout
and improved quality of care outcomes. How-
ever, they did not examine effects at multiple
levels of analysis (e.g., both individual and unit
levels). This assumes that the relationships
between the PES-NWI and constructs are con-
sistent (i.e., homologous) across levels. Until
potential variations across levels are tested
empirically, the relative strength and scope of
the PES-NWI subscales at different levels of
analysis will remain unspecified.
As Bliese, Chen, & Ployhart (2007)
observed, “there has been a tendency in organi-
zational research to consider aggregated higher
level variables as faithful representations of the
lower-level variables used in the composition”
(p. 553). Researchers may thus be assuming
that the effects of independent variables tested
at one level of analysis will also be observed at
another level of analysis; that is, that the effects
are homologous (Bliese et al., 2007; Morgeson
& Hofmann, 1999). Yet, conducting such tests
may demonstrate that there are different effects
at different levels of analysis. For example,
Gibson (2001) found at the individual level of
analysis that nurses’ self-efficacy was related
to effectiveness (i.e., performance), as was
training. However, at the nursing team (i.e.,
unit) level of analysis, only group efficacy was
related to effectiveness. These findings suggest
that the relationship between training and effec-
tiveness was not homologous across levels.
Extending this to the PES-NWI, certain sub-
scales may predict nurse outcomes at the indi-
vidual level, but not at the unit level, and vice
versa. If so, theory related to the PES-NWI will
need to account for the presence of different
(i.e., emergent) effects across levels.
To advance knowledge of the PES-NWI,
we used multilevel CFA to systematically
examine the PES-NWI’s factor structure and
test for alternate and potentially more parsimo-
nious structures, including the viability of an
overall PES-NWI score (i.e., a one-factor solu-
tion; Lake, 2007; Lake & Friese, 2006). This
approach joins with recent work by Gajewski
et al. (2010) using multilevel factor analyses
with the PES-NWI and with other nursing
researchers who have begun to use multilevel
modeling (e.g., Estabrooks, Midodzi, Cum-
mings, & Wallin, 2007). We then conducted
multilevel modeling to examine relationships
of the dimensions of the practice environment
with nursing outcomes at the unit and individual
levels. Our approach is consistent with original
work outlined by Aiken and Hage (1968),
who were among the first to utilize multilevel
frameworks to examine how features of a
health care organization’s environment influence
its processes and behavior (e.g., how the num-
ber of joint programs within an organization
affects outcomes such as innovation, complex-
ity, and need for resources). Our emphasis was
on testing multilevel homology, or consistency,
between individual- and unit-level relationships,
an important component of building multi-
level theory (Bliese et al., 2007; Chen, Bliese,
& Mathieu, 2005). In conducting this test,
we analyzed relationships of subscales of the
PEW-NWI to outcomes of practical importance
and frequently studied by researchers:
emotional exhaustion, turnover intentions,
and job satisfaction (e.g., Aiken et al., 2001;
Aiken & Patrician, 2000; Van Bogaert et al.,
2010).
We proposed that Participation in Hospital
Affairs, Staffing Adequacy, Managerial Support,
and Collegial Nurse-Physician Relationships are
negatively related to nurse emotional exhaustion
(Hypothesis 1), negatively related to nurse turn-
over intentions (Hypothesis 2), and positively
related to nurse job satisfaction (Hypothesis 3).
To examine the extent to which the four PES-
NWI subscales have homologous or emergent
effects across different levels of analysis,
Hypotheses 1–3 were tested at the individual
level and the unit level. The wording of our
hypotheses reflects our expectation that high
scores on each of the PES-NWI subscales
would have beneficial effects on nurse outcomes
across both levels of analysis (i.e., homologous
effects). Hypothesis 4 was that individual-
and unit-level relationships among the PES-
NWI subscales and emotional exhaustion,
turnover intentions, and job satisfaction are
homologous.
Research in Nursing & Health
MULTILEVEL ANALYSIS OF THE PES-NWI/ GABRIEL ET AL. 569

Method
Data were collected from full-time RNs
employed in a Midwestern hospital system
as part of a larger study during Spring 2011.
Hospital managers identified nurses who were
full-time (managers’ criteria for eligibility);
additionally, managers identified the units to
which nurses belonged. This resulted in an
initial distribution to 1,702 nurses. Four weeks
after distribution, reminders were sent. Nurses
who had not completed the survey 8 weeks after
the initial mailing were sent a second copy
of the survey. Out of 1,702 surveys sent, 762
surveys were returned (response rate ¼44.7%),
comparable to similar studies of U.S. nurses
(Lucero, Lake, & Aiken, 2010). Of the original
762 surveys, our sample was reduced to 699
nurses; 9 were removed for missing all items
for all 4 subscales [n¼6], or all items for one
subscale [n¼3]), 2 were removed for not hav-
ing data on any of the 3 dependent variables in
the current study, and 51 were removed because
fewer than 3 responded from that unit (multi-
level analyses require that at least 3 individuals
are included in each unit).
Our final sample of 699 full-time RNs
was nested within 79 units (average number of
nurses per unit ¼8.85) and 9 branches of the
hospital (average number of units per hospital
branch ¼8.78). The average age was 40.41 (SD
¼12.58). The sample was predominately female
(91.3%) and white (87.9%). Although we were
unable to collect information for nurses who did
not complete the survey, compared to the hospi-
tal system we collected from, our sample was
fairly representative (Female: 91.5%; Cauca-
sian: 83.9%; average hours scheduled to work
¼35.4). Our demographics also fit with other
large-scale nursing surveys (e.g., Aiken, Clarke,
Sloane, Sochalski, & Silber, 2002) and nursing
demographic projections (e.g., Buerhaus, Staiger,
& Auerbach, 2009).
Average job tenure was 13.16 years (SD ¼
11.78), with an organizational tenure of 9.45
years (SD ¼9.34) and unit tenure of 7.33 years
(SD ¼7.38). Nurses had worked with their unit
managers for 3.44 years (SD ¼3.50) on average.
Nurses worked an average of 39.80 (SD ¼5.79)
hours weekly and saw an average of 4.90
patients per shift (SD ¼5.96). Sixteen nurses
indicated they had 0 patients per shift, 33 left
this statement blank, and 2 reported extreme
values (200 patients, 300 patients). We ran sup-
plemental analyses to see whether the removal
of these nurses altered results; results were no
different. Thus, we retained all 699 nurses for
our analyses.
Measures
Means, standard deviations, and correlations
among variables can be found in Table 1.
Nursing practice environment. Twenty-
one items from the PES-NWI (Lake, 2002)
were included in the four subscales. The first
was Nurse Participation in Hospital Affairs (8
items; a¼.86; e.g., “Staff nurses are involved
in the internal governance of the hospital”). The
item “A chief nurse officer equal in power and
authority to other top level hospital executives”
was removed because it has been found to be
the most weakly correlated with other scale
items (Lake, 2002) and it did not fit the larger
goal of understanding how staff nurses perceive
the responsiveness of nursing administration to
their needs. Staffing Adequacy had four items
(a¼.87; e.g., “I work with nurses who are
clinically competent”). Managerial Support had
five items (a¼.89; e.g., “My unit manager uses
mistakes as learning opportunities, not criti-
cism”). Collegial Nurse-Physician Relationships
had three items (a¼.91; e.g., “There is a lot of
teamwork between nurses and physicians”).
Items were on a 4-point scale (1 strongly dis-
agree;4strongly agree).
Emotional exhaustion. Wharton’s (1993;
see also Erickson & Ritter, 2001) seven-item
emotional exhaustion scale was used to assess
the extent to which individuals experienced
chronic emotional and interpersonal stressors at
work, on a 5-point scale (1 not at all;2about
once a month;3a few times a month;4about
once a week;5almost every day). An example
is, “I feel emotionally drained from my work.”
a¼.93.
Turnover intentions. Three items mea-
sured the extent to which nurses intend to leave
their jobs (Cropanzano, James, & Konovsky,
1993) on a 4-point Likert scale (1 strongly dis-
agree;4strongly agree). An example item is,
“I intend to leave this organization within the
next year.” a¼.80.
Job satisfaction. Five items adapted from
Quinn and Staines (1979) assessed the extent to
which individuals felt satisfied with their jobs,
on a 4-point scale (1 very dissatisfied;4very
satisfied). Participants rated satisfaction with
current work hours, control over the work done,
control over daily work routine, coworkers, and
job in general. a¼.75.
Research in Nursing & Health
570 RESEARCH IN NURSING & HEALTH

Analytic Approach
As an initial step, we partitioned the variance of
each variable (i.e., from the a priori four dimen-
sion solution) into the three levels of analysis
(nurse, unit, hospital) and calculated ICC(1) and
ICC(2) values. ICC(1) is the proportion of vari-
ance in the lower-level responses (individual)
that is accounted for by a higher-order nesting
factor (unit, hospital). ICC(2) is the reliability
of the higher-level scores (unit, hospital), or
the extent to which the higher-level scores can
be reliably differentiated. For this analysis, we
were only making a theoretical case for unit-
level constructs for the independent variables;
we were not concerned with the statistics
supporting aggregation for our dependent
variables.
Although there is no set cut-off for ICCs,
James (1982) reported a median ICC(1) value
of .12, and Glick (1985) suggested a minimum
ICC(2) of .60 to support aggregating predictor
variables. As shown in Table 2, all ICC values
for our independent variables reached these lev-
els, supporting aggregation. It is important to
point out that the amount of variance at the hos-
pital level of analysis was low, but meaningful,
suggesting that we should account for that layer
of nesting. However, we did not model any pre-
dictors at the hospital level, given the small
n-size at that level.
We also conducted two additional tests for
supporting aggregation of the PES-NWI sub-
scales at the unit level. First, we considered
design effects, which allow for researchers to
measure the extent to which standard errors
are underestimated in multilevel models (Kish,
1965; Maas & Hox, 2004). According to
Muthe
´n and Satorra (1995), when design effects
are less than 2, single-level analyses (i.e., stan-
dard regression without nesting) will not gener-
ate results that are overly misleading or
misspecified. Using the following formula for
design effects: 1 þd(n1), where dequals the
ICC(1) value for each of the four PES-NWI
subscales and nequals the average cluster size
(i.e., 8.85), we found the following design
effects: Participation in Hospital Affairs ¼2.33,
Staffing Adequacy ¼3.12, Managerial Support
¼2.57, and Collegial Nurse–Physician Rela-
tionships ¼2.88. Given that these design effects
for the subscales were greater than 2, we con-
tinued to support aggregation to the unit level.
As a final test, we calculated the r
wg(j)
statistic
(Lindell & Brandt, 1999) for each unit, which
reflects the unit’s level of agreement on scale
Table 1. Means, Standard Deviations, Correlations, and Internal Consistency of PES-NWI Subscales and Outcomes
Variable
Individual
Mean
Individual
SD
Unit
Mean
Unit
SD1234567
1. Participation in Hospital Affairs 2.69 0.52 2.66 0.30 (.86) .25.59 .37 .44 .31 .19
2. Staffing Adequacy 2.29 0.67 2.35 0.42 .36 (.87) .36 .05 .59 .49 .72
3. Managerial Support 2.78 0.72 2.78 0.42 .61 .35 (.89) .03 .50 .45 .40
4. Collegial Nurse–Physician Relationships 2.83 0.67 2.81 0.44 .36 .21 .23 (.91) .28.15 .02
5. Job satisfaction 2.98 0.50 3.00 0.25 .45 .52 .45 .36 (.75) .55 .69
6. Turnover intentions 1.94 0.68 1.91 0.29 .33 .29 .34 .16 .44 (.80) .48
7. Emotional exhaustion 3.29 1.05 3.21 0.54 .29 .55 .30 .19 .56 .39 (.93)
Note. Reliabilities (Cronbach a) are along the diagonal in parentheses. Correlations below the diagonal are at the individual level of analysis; variables have been centered
within-unit and within-hospital. Correlations above the diagonal are at the unit level of analysis; variables have been within-hospital-centered. Because of low hospital-level
variance, no correlations were calculated at that level. Individual level: n¼699; unit level: n¼79; hospital level: n¼9.
p<.05.
p<.01.
Research in Nursing & Health
MULTILEVEL ANALYSIS OF THE PES-NWI/ GABRIEL ET AL. 571

items. The average r
wg(j)
values across units
were .69 for Managerial Support, .80 for Colle-
gial Nurse-Physician Relationships, .76 for
Participation in Hospital Affairs, and .78 for
Staffing Adequacy. These findings helped sup-
port our decision to aggregate the PES-NWI
subscales to the unit level.
We conducted a multilevel CFA based
upon guidelines set by Dyer et al. (2005). Mul-
tilevel CFAs differ from traditional CFAs in that
multilevel CFAs model factor structures that
account for both between- and within-group
factor structures. Further, as articulated by Dyer
et al. (2005), multilevel analyses like CFA
are necessary when data are nested (as in the
current study) because the observations are cor-
related (rather than independent), and the under-
lying structure of the factors may vary from one
level of analysis to the next. In order to enhance
the accuracy and stability of the parameter esti-
mates, we created three item parcels for each
dimension of the PES-NWI when possible (e.g.,
Hall, Snell, & Foust, 1999). According to Hall
et al. (1999), parcels reduce the number of
parameters to be estimated, resulting in more
stable and accurate estimates when working
with smaller sample sizes (which is the case for
the unit level in the current article), and because
they tend to be more reliable and normally
distributed. Bandalos (2002) wrote that “for sol-
utions in which items have a well-known factor
structure, parceling together items that are
known a priori to be unidimensional appears to
result in less bias in structural parameters than
use of the individual items when items are
coarsely categorized, nonnormally distributed,
or both” (p. 101). The items that were used in
our item parcels loaded onto the factors in the
same manner as previously articulated in the
literature. We ran supplemental analyses with
individual items (once using within-unit cen-
tered data to remove variance associated with
units; once modeling individual-level items only
in a multilevel CFA). In both of these instances,
individual items loaded correctly onto their
respective subscales and mirrored the parcel
results.
In assessing model fit, we utilized the x
2
goodness of fit statistic, the root mean square
error approximation (RMSEA; Steiger, 1990),
the comparative fit index (CFI; Bentler, 1990),
the Tucker Lewis Index (TLI; Tucker & Lewis,
1973), and the standardized root mean square
residual (SRMR; Bentler, 1990), in addition to
changes in CFI (Cheung & Rensvold, 2002).
We used the following cut-off values as criteria
Table 2. ICC(1) and ICC(2) Values for PES-NWI Subscales and Outcomes
Unit Level Hospital Level % of Total Variance That Is
ICC(1) ICC(2) ICC(1) ICC(2) Between Persons (%) Between Units (%) Between Hospitals (%)
Participation in Hospital Affairs .17 .65 .31 .80 88.23 8.05 3.72
Staffing Adequacy .27 .77 .19 .67 77.23 15.99 6.78
Managerial Support .20 .69 .16 .63 84.23 12.53 3.24
Collegial Nurse–Physician Relationships .24 .74 .28 .78 81.28 11.62 7.10
Emotional exhaustion .08 .43 .17 .63 93.68 4.82 1.50
Job satisfaction .05 .32 .00 .01 96.25 3.74 0.01
Turnover intentions .05 .30 .06 .37 95.44 4.55 0.01
Note. ICC(1) and ICC(2) at the hospital level were calculated with data aggregated to the unit level; individual level data were used to calculate ICC(1) and ICC(2) for the unit
level. ANOVA (analysis of variance) was used to calculate ICC(1) and ICC(2) variables. The % of total variance that is at each level is calculated in a two-step procedure for
three-level models: (1) partitioning of the variance at the between-persons level (s
2
/[s
2
þt
00
]) and subtracting from 1.00 to obtain the remaining variance, where s
2
equals
within-unit variance and t
00
equals between-unit variance, and (2) partitioning the remaining variance into between-units and between-hospitals by utilizing s
2
/s
2
þt
00
to calcu-
late the between-units variance and multiplying it by the remaining variance from Step 1. Remaining variance from Step 2 is between-hospitals variance. All levels equal 100%.
Research in Nursing & Health
572 RESEARCH IN NURSING & HEALTH

for adequate fit: values close to .95 for TLI and
CFI, values less than .06 for RMSEA, values
less than .08 for SRMR (Hu & Bentler, 1999),
and change smaller than or equal to .01 for
change in CFI (Cheung & Rensvold, 2002).
These statistics were used since the x
2
statistic
can be sensitive to sample size suggesting poor
fit, even when the model is appropriate for the
data (Anderson & Gerbing, 1998; Bentler &
Bonett, 1980). We used Mplus 5.21 [Muthe
´n&
Muthe
´n, 19982007] for multilevel CFA.
To test the multilevel relationships of the
PES-NWI dimensions with outcomes, we used
hierarchical linear modeling (HLM 6.0; Rau-
denbush & Bryk, 2002). (We also tested the
hypothesized relationships in multilevel struc-
tural equation modeling using Mplus 5.21
[Muthe
´n & Muthe
´n, 1998–2007]. Results were
replicated, so for simplicity, we only present
the HLM analyses.) Two separate sets of
multilevel analyses were conducted to assess
the homology of effects across levels (Chen
et al., 2005). For individual-level effects, indi-
vidual-level data were entered at Level 1, with
fixed coefficients at Level 2 for units and Level
3 for hospitals. Although we did not initially
propose to test hospital-level relationships (i.e.,
testing homology between the individual and
unit levels of analysis), we did model this
higher-level nesting because of small but mean-
ingful variance between hospitals.
Individual-level data were unit-mean cen-
tered to remove unit-level variance in analyses
(Enders & Tofighi, 2007). For unit-level analy-
ses, following the approach of Chen et al.
(2005), individual-level data were aggregated
to the unit-level and nested within hospitals.
In these analyses, Level 1 represented hospital-
mean-centered unit-level effects (Enders &
Tofighi, 2007), with fixed coefficients at Level 2
(hospital level). Centering around the unit or
hospital mean for our analyses (i.e., group mean
centering) was appropriate given that we were
mainly concerned with the relationship between
the independent and dependent variables at
Level 1 in our analyses (Enders & Tofighi,
2007).
Results
Multilevel Confirmatory Factor
Analysis
A four-factor model fit the data well (see
Table 3). We considered alternative theoretical
structures in which the scales were collapsed
into three-, two-, and one-factor models. We
tested two three-factor models. The first three-
factor model was built around the theoretical
idea that Collegial Nurse–Physician Relation-
ships and Managerial Support may constitute
a single social support factor. Gabriel et al.
(2011) had previously conceptualized Collegial
Nurse–Physician Relations as a measure of
social support, and Managerial Support has
clear overlap with the idea of social support in
the workplace (e.g., Cohen & Willis, 1985).
Our second three-factor model involved a factor
of work environment job characteristics, com-
bining Participation in Hospital Affairs and
Staffing Adequacy. Whereas social support
may be construed as an emotional resource for
employees to draw upon, Participation in Hospi-
tal Affairs and Staffing Adequacy may reflect a
different type of resource for nurses to draw
upon that aid in their performance of daily
work tasks (e.g., cognitive resources; Bakker &
Demerouti, 2007). Based upon decisions made
for the two three-factor models, our two-factor
Table 3. Tests of Alternative Structures of the PES-NWI in Multilevel Confirmatory Factor Analysis
Model x
2
df RMSEA SRMR
w
SRMR
b
TLI CFI DCFI
Four factors 191.34 96 .04 .04 .09 .98 .98 —
Three factors—Work environment 807.89 102 .10 .09 .15 .86 .82 .16
Three factors—Social support 1271.13 102 .13 .15 .19 .78 .71 .27
Two factors 1873.92 106 .15 .18 .24 .66 .58 .40
One factor 2335.99 108 .17 .14 .24 .57 .48 .50
Note. Three factors—Work environment was comprised of Managerial Support, Collegial Nurse–Physic ian Relations,
and a “work characteristics” factor (Participation in Hospital Affairs, Staffing Adequacy). Three factors—Soci al
support was comprised of Participation in Hospital Affairs, Staffing Adequacy, and a “support” factor (Managerial
Support and Collegial Nurse–Physician Relationships combined). The two-factor model was comprised of the same
“support” construct and a “work environment” construct (Participation in Hospital Governance and Staffing
Adequacy combined). The one-factor model representing the overall NWI scale did not converge. SRMR
w
, SRMR
within (individual level); SRMR
b
, SRMR between (unit level).
p<.001.
Research in Nursing & Health
MULTILEVEL ANALYSIS OF THE PES-NWI/ GABRIEL ET AL. 573

model was based upon our work environment
and support factors. Finally, we examined
a one-factor solution, because Lake (2007;
p. 109S) reported that “the PES-NWI subscales
can be combined into a composite measure of
the practice environment, as either a continuous
variable or a three category variable,” and Lake
and Friese (2006) wrote that a single composite
score could be created from averaging together
the subscales of the PES-NWI, suggesting that
it would be important to consider a one-factor
structure. As shown in Table 3, each model fit
worse than the four-factor model. Thus, we
retained our four-factor PES-NWI scale. The
factor loadings for the parcels at the individual
level ranged from .69 to .92, and at the unit-
level ranged from .73 to .99 (see Table 4).
We do note that the x
2
statistic was signifi-
cant for our retained model, which could indi-
cate lack of model fit (Browne & Cudeck,
1993). However, as indicated by multiple
authors (Browne & Cudeck, 1993; Hu &
Bentler, 1999; Kline, 2005), and encouraged by
Jackson, Gillaspy, and Purc-Stephenson (2009),
we utilized multiple cut-off values with highly
stringent criteria, and clearly noted the cut-off
values we used. Thus, although the x
2
statistic
is significant, based upon the additional fit indi-
ces and the large body of literature supporting
the use of fit indices in settling on an acceptable
model (Browne & Cudeck, 1993; Hu & Bentler,
1999; Kline, 2005), we felt confident moving
forward with hypothesis testing.
Individual-Level Effects
All results for the individual-level hypothesis
tests (controlling for unit-level and hospital-
level nesting) can be found in Table 5. Hypothe-
sis 1 predicted that each of the four PES-NWI
subscales would inversely relate to emotional
exhaustion. Results were partially supportive
of hypotheses, with Participation in Hospital
Affairs and Staffing Adequacy exhibiting inverse
relationships with emotional exhaustion, but
there was no such effect for Managerial Support
or Collegial Nurse–Physician Relationships.
Hypothesis 2 predicted that each of the
four PES-NWI subscales would inversely relate
to turnover intentions. Results indicated that
Participation in Hospital Affairs, Staffing Ade-
quacy, and Managerial Support all inversely
related to turnover intentions; no relationship
was found for Collegial Nurse–Physician
Relationships.
Finally, Hypothesis 3 predicted that each
of the four PES-NWI subscales would posi-
tively relate to job satisfaction, which garnered
full support.
Unit-Level Effects
Hypotheses 1 through 3 were all tested at the
unit level of analysis (controlling for hospital-
level nesting), with results again presented in
Table 5. For Hypothesis 1 (emotional exhaus-
tion), results indicated that only Staffing
Adequacy was inversely related to emotional
exhaustion, with the effect of Participation
in Hospital Affairs, Managerial Support, and
Collegial Nurse–Physician Relationships being
unsupported. For Hypothesis 2 (turnover inten-
tions), results were largely supportive, with
Staffing Adequacy, Managerial Support, and
Collegial Nurse–Physician Relationships inversely
relating to turnover intentions; Participation
In Hospital Affairs was not supported. Finally,
for Hypothesis 3 (job satisfaction), Staffing
Adequacy, Managerial Support, and Collegial
Nurse–Physician Relationships were all posi-
tively related to job satisfaction; there was no
support for Participation in Hospital Affairs.
Homology Between Individual and
Unit Levels
Our data allowed us to test for configural
similarity at the individual and unit levels of
Table 4. Factor Loadings for Item Parcels of the
PES-NWI Within and Between Units
Within Units Between Units
Participation in Hospital Affairs
Parcel 1 .88 .97
Parcel 2 .78 .91
Parcel 3 .69 .80
Staffing Adequacy
Parcel 1 .90 .99
Parcel 2 .83 .92
Parcel 3 .75 .84
Managerial Support
Parcel 1 .83 .90
Parcel 2 .88 .98
Parcel 3 .83 .97
Collegial Nurse–Physician Relationships
Parcel 1 .85 .93
Parcel 2 .92 .93
Parcel 3 .72 .73
Note. Values are standardized factor loadings for each
parcel. Parcels were created based upon the subscale
the PES-NWI items should have loaded on.
Research in Nursing & Health
574 RESEARCH IN NURSING & HEALTH

analysis (Hypothesis 4). To do so, “researchers
simply need to test the significance levels of
parameter estimates at each relevant level of
analysis and compare the relative ‘hit rate”’
(Chen et al., 2005, p. 387). If significance tests
and the direction of effects are consistent,
researchers can conclude that there is configural
similarity. As shown in Table 5, at the individ-
ual level of analysis, 9 out of our 12 sub-
hypotheses were supported; at the unit level of
analysis, we supported 7 out of 12 sub-hypothe-
ses. All findings for Participation in Hospital
Affairs were inconsistent across levels, in that
significant effects were found at the individual
level, with no effects at the unit level. Thus,
when considering Staffing Adequacy, Manage-
rial Support, and Collegial Nurse–Physician
Relationships, our results were almost entirely
congruent across levels with one exception:
Collegial Nurse–Physician Relationships were
inversely related to turnover intentions at the
unit level, but not the individual level. Further
support for homology came from the correla-
tions in Table 1, where correlations are in the
same direction and of the same general magni-
tude at the two levels (i.e., the correlation
between Staffing Adequacy and job satisfaction
is .52 at the individual level and .59 at the unit
level). In sum, findings were largely consistent
across levels, supporting Hypothesis 4.
Discussion
Since the inception of the PES-NWI, research-
ers have sought to understand how aspects of
the practice environment relate to key nurse
outcomes. The current project extends theoreti-
cal and empirical knowledge of the PES-NWI
by examining the extent to which such relation-
ships were consistent across different levels of
analysis. The results show that while nursing
practice environments are multifaceted and sen-
sitive to context (Cho, Mark, Yun, & June,
2011; Friese, Lake, Aiken, Silber, & Sochalski,
2008; Lake, 2007) some work environment
characteristics may have more consistent effects
across levels of analysis and context than others.
Table 5. Multilevel Coefficient Model Predicting the Dependent Variables at the Individual and Unit
Levels
Hypothesized Predictor (PES-NWI Subscale)
Dependent Variable
Individual
Level,
Coefficient (SE)
Unit Level,
Coefficient (SE)
Homologous
Effects?
1a: Participation in Hospital Affairs and emotional
exhaustion
.24(.10) .24 (.25) No
1b: Staffing Adequacy and emotional exhaustion .70 (.07) .84 (.14) Yes
1c: Managerial Support and emotional exhaustion .13 (.07) .23 (.15) Yes
1d: Collegial Nurse–Physician Relationships and
emotional exhaustion
.12 (.07) .03 (.13) Yes
Total variance explained 28.49% 40.20%
2a: Participation in Hospital Affairs and turnover
intentions
.30 (.07) .19 (.15) No
2b: Staffing Adequacy and turnover intentions .10(.05) .16(.08) Yes
2c: Managerial Support and turnover intentions .17 (.05) .19(.09) Yes
2d: Collegial Nurse–Physician Relationships and
turnover intentions
.01 (.05) .16(.08) No
Total variance explained 15.03% 35.87%
3a: Participation in Hospital Affairs and job
satisfaction
.15 (.04) .10 (.12) No
3b: Staffing Adequacy and job satisfaction .27 (.03) .24 (.08) Yes
3c: Managerial Support and job satisfaction .15 (.03) .18(.07) Yes
3d: Collegial Nurse–Physician Relationships and
job satisfaction
.14 (.03) .16(.07) Yes
Total variance explained 39.71% 42.61%
Note. Individual-level n¼699 (except for emotional exhaustion [n¼693] and job satisf action [n¼691]), unit-level
n¼79, hospital-level n¼9. For individual-level analyses, effects were modeled at the individual level (Level-1) only.
For unit-level analyses, effects were only modeled at the unit level (Level-1). SE ¼standard erro r. Homology is sup-
ported if the same relationships were found at the individual and unit levels of analysis.
p<.05.
p<.001.
Research in Nursing & Health
MULTILEVEL ANALYSIS OF THE PES-NWI/ GABRIEL ET AL. 575

Our most interesting findings surround
nurses’ Participation In Hospital Affairs. At the
individual level, Participation in Hospital
Affairs was related to all three outcomes, with
no support for effects at the unit level. Such
findings demonstrate the need to consider
multiple levels of analysis when studying the
PES-NWI, and draws further attention to the
homology problem identified by Chen et al.
(2002, 2005). Thus, Participation In Hospital
Affairs may take on a different, less salient
meaning in the aggregate. At the individual
level, the results suggest that variation in
nurses’ perception of opportunities to partici-
pate in hospital affairs may provide insight into
the quality of relationships among staff nurses
and their superiors. In this case, the subscale
may be particularly useful for identifying the
range and correlates of contexts in which nurses
may have more or less voice in hospital affairs.
Recognizing that this individual-level sense
of inclusion tends to be related to feelings of
satisfaction, emotional exhaustion, and turnover
intention may be of use to managers seeking
ways to promote the well-being of highly val-
ued staff members. Aggregating this informa-
tion to the unit level, however, may not be as
informative since individual-level differences
may be less noticeable if the unit as a whole
has a relatively high participation rate.
Our results also affirm complex relation-
ships between the other subscales and outcomes
(Cho et al., 2011; Gajewski et al., 2010; Lake,
2007). First, it is important to note that Staffing
Adequacy was related to all three dependent
variables across both levels. This fits with previ-
ous work that has stressed the importance of
adequate staffing in terms of nurse well-being
(Aiken et al., 2001; Aiken et al., 2002; Clark,
Leddy, Drain, & Kaldenberg, 2007; Lake,
2002). Researchers have also documented that
nurses face many job demands during each shift
(e.g., direct and indirect patient care tasks;
Gabriel et al., 2011) and adequate staffing might
present one way to enable nurses to fulfill these
competing and potentially conflicting job
demands.
The other subscales, Collegial Nurse–
Physician Relationships and Managerial
Support, were only related to certain outcomes,
with effects largely consistent across levels of
analysis. Collegial Nurse–Physician Relation-
ships were related to job satisfaction across
both levels of analysis. Additionally, Collegial
Nurse–Physician Relationships were inversely
associated with turnover intentions at the unit
level of analysis only, suggesting that unit-level
feelings of physician collegiality may foster a
more supportive climate, perhaps encouraging
nurses to remain on the unit. However, when
considering these results as a whole, collegial
relationships may foster positive environments
that make nurses more satisfied, but the results
suggest that these relationships may do little to
assist with emotional exhaustion and intent to
leave. Such findings fit with previous work by
Van Bogaert et al. (2010) who found nonsignifi-
cant relationships between Collegial Nurse–
Physician Relationships and both turnover
intentions and quality of care, though they did
find a positive relationship with job satisfaction.
In considering how collegial nurse–physician
relationships may operate, Gabriel et al. (2011)
treated these as a buffering effect, finding that
high levels of collegiality weakened the nega-
tive relationship between nurses’ not being able
to complete indirect care tasks (e.g., charting,
patient history reviews) and negative affect.
Therefore, while collegial nurse–physician rela-
tionships may not directly alleviate negative
outcomes, they may act as a buffer of job
demands. Given that collegial nurse–physician
relationships had similar effects at the unit
level, and that collaborative relationships are
considered central to patient safety and care
(O’Daniel & Rosenstein, 2008), researchers may
want to consider investigating the ways that unit
characteristics, policies, and practices may pro-
mote interdisciplinary collaborations that are
likely to yield these types of positive outcomes.
Finally, Managerial Support was inversely
related to turnover intentions and positively
related to job satisfaction, but no relationship
was found with emotional exhaustion. Thus,
although Managerial Support may be beneficial
to help nurses feel satisfied with their jobs and
want to remain with the organization, it may
not be enough to offset the demands that
increase the experience of emotional exhaus-
tion. Given that social support is theorized to
operate as a moderator of stress on well-being
(Pearlin, 1999), future researchers may want
to examine how the predictors (e.g., workload,
job autonomy, equity) of emotional exhaustion
may be moderated by high levels of Managerial
Support, in addition to further examining the
possibility that it may have a direct effect on
indicators of individual and unit well-being.
The consistency in results for Staffing
Adequacy across both levels of analysis and the
more complex results reported for Participation in
Hospital Governance, Collegial Nurse–Physician
Research in Nursing & Health
576 RESEARCH IN NURSING & HEALTH

Relationships, and Managerial Support also
have implications for the potential of multilevel
modeling to contribute to theoretical develop-
ment related to the nurse practice environment.
In particular, our results suggest that researchers
may want to carefully consider the nomological
network (Cronbach & Meehl, 1955) associated
with the theoretical constructs underlying the
PES-NWI subscales and the level of analysis
at which they are being operationalized and
observed. For example, the results reported here
demonstrate that results at one level (e.g., indi-
vidual nurses) may not extend to higher levels
(e.g., units, hospitals) and vice versa.
Thus, although concepts may hold analyti-
cal relationships to one another, such that they
are expected to covary, these types of theoreti-
cal propositions cannot be generalized from one
level to the next without assessing the extent to
which they receive empirical support. Making
this mistake could result in an ecological fallacy
(Robinson, 1950), when scholars draw conclu-
sions about relationships at one level of analysis
and assume this relationship transcends all other
levels. Based on our findings, we urge research-
ers to be cognizant of this problem by either (a)
testing models across both levels of analysis, or
(b) being careful not to generalize results
beyond the scope of the study. For instance, had
we only considered our analyses at the individ-
ual level of analysis, we may have felt com-
pelled to conclude that Participation in Hospital
Affair is beneficial for well-being-related out-
comes for nursing units. However, our study
suggests that this finding may be reserved for
the individual level, and therefore advise that
others may want to proceed with caution as
well.
Limitations and Future Directions
Our sample was fairly homogenous (female,
Caucasian), and we were unable to determine
how the sample that responded to our survey
compared to the population of interest and those
that did not respond to the survey. However, our
sample demographics are similar to those in
many nurse studies (e.g., Aiken et al., 2002;
Buerhaus et al., 2009; Gabriel et al., 2011),
helping minimize our concern. Regardless, we
hope future research can consider more diverse
samples of RNs.
We used single-time assessments of the
PES-NWI. Future research should consider
multi-staged data collections to test perceptions
of the subscales over time, in addition to
conditions under which nurses’ perceptions of
these environmental constructs fluctuate across
time. Research on other organizational con-
structs has demonstrated that significant within-
person variability can exist (Dalal & Hulin,
2008) and that nursing work environments are
highly dynamic (Cummings, Hayduk, &
Estabrooks, 2006). Additionally, future multi-
level work should consider trying to link self-
reported multilevel data to data from other sour-
ces (e.g., manager ratings of performance,
patient satisfaction, patient health indicators).
Researchers using the PES-NWI also may
want to consider measurement invariance
(Cheung & Rensvold, 2002; Vandenberg &
Lance, 2000) to see if the latent structure of
items is different depending upon group mem-
bership (i.e., different units, different demo-
graphic groups). Further, researchers may
choose to model a method effect, to account for
common-method bias in confirmatory factor
analytic work (Podsakoff, MacKenzie, Lee, &
Podsakoff, 2003). Although we did not find this
necessary or appropriate in the current context
(see Richardson, Simmering, & Sturman, 2009,
for a demonstration of why modeling method
effects may not be advisable), future researchers
may want to consider this. Finally, we did not
consider any moderators of the relationships.
De Gieter, Hofmans, and Pepermans (2011)
suggested that individual differences might be
important in assessing relationships between
nurses; perhaps personality or other disposi-
tional traits may play a role in how PES-NWI
subscales influence outcome variables. Addi-
tionally, recent work on subcultures in hospitals
(Mallidou, Cummings, Estabrooks, & Giovannetti,
2011) presents interesting implications for the
PES-NWI, such that subcultures could moderate
the relationships between the PES-NWI sub-
scales and outcomes.
Conclusion
In the current article we expand the PES-NWI
literature by reporting relationships between the
PES-NWI subscales and key nursing outcomes.
Results demonstrated that the PES-NWI sub-
scales operate differently depending on the par-
ticular nursing outcome under consideration,
suggesting that certain aspects of the nursing
practice environment are more critical than
others when improving nurse well-being. Man-
agers and health care administers may want to
take these results into consideration as they
Research in Nursing & Health
MULTILEVEL ANALYSIS OF THE PES-NWI/ GABRIEL ET AL. 577

make decisions about how best to design the
nursing practice environment to maximally ben-
efit the well-being of nurses. Moreover, by
modeling effects at the individual and unit level
of analysis, we were able to demonstrate the
robust nature of most effects, as they remain
fairly consistent across levels of analysis.
References
Aiken, L., Clarke, S., Sloane, D., Sochalski, J.,
Busse, R., Clarke, H., … Shamian, J. (2001).
Nurses’ reports on hospital care in five countries.
Health Affairs,20, 43–53. doi:10.1377/hlthaff.
20.3.43
Aiken, L. H., Clarke, S. P., Sloane, D. M., Sochalski,
J., & Silber, J. H. (2002). Hospital nurse staffing
and patient mortality, nurse burnout, and job dis-
satisfaction. JAMA: The Journal of the American
Medical Association,288, 1987–1993. doi:
10.1001/jama.288.16.1987
Aiken, L., & Patrician, P. (2000). Measuring organi-
zational traits of hospitals: The revised nursing
work index. Nursing Research,49, 146–153.
Aiken, M., & Hage, J. (1968). Organizational inter-
dependence and intra-organizational structure.
American Sociological Review,33, 912–930.
doi:10.2307/2092683
Anderson, J., & Gerbing, D. (1998). Structural
equation modeling in practice: A review and
recommended two-step approach. Psychological
Bulletin,103, 411–423. doi:10.1037/0033-2909.
103.3.411
Bakker, A. B., & Demerouti, E. (2007). The job
demands-resources model: State of the art.
Journal of Managerial Psychology,22, 309–328.
doi:10.1108/02683940710733115
Bandalos, D. L. (2002). The effects of item parceling
on goodness-of-fit and parameter estimate bias
in structural equation modeling. Structural
Equation Modeling,9, 78–102. doi:10.1207/
S15328007SEM0901_5
Bentler, P. (1990). Comparative fit indexes in struc-
tural models. Psychological Bulletin,107, 238–
246. doi:10.1037/0033-2909.107.2.238
Bentler, P., & Bonett, D. (1980). Significance tests
and goodness of fit in the analysis of covariance
structures. Psychological Bulletin,88, 588–606.
doi:10.1037/0033-2909.88.3.588
Bliese, P. D. (2002). Multilevel random coefficient
modeling in organizational research. In F. Drasgow
& N. Schmitt (Eds.), Measuring and analyzing
behavior in organizations (pp. 401–445). San Fran-
cisco, CA: Jossey-Bass.
Bliese, P., Chan, D., & Ployhart, R. (2007). Multi-
level methods: Future directions in measurement,
longitudinal analyses, and nonnormal outcomes.
Organizational Research Methods,10, 551–563.
doi:10.1177/1094428107301102
Budge, C., Carryer, J., & Wood, S. (2003). Health
correlates of autonomy, control, and professional
relationships in the nursing work environment.
Journal of Advanced Nursing,42, 260–268.
doi:10.1046/j.1365-2648.2003.02615.x
Buerhaus, P., Auerbach, D., & Staiger, D. (2009).
The recent surge in nurse employment: Causes
and implications. Health Affairs,28, 657–668.
doi:10.1377/hlthaff.28.4.w657
Buerhaus, P., Staiger, D., & Auerbach, D. (2009).
The future of the nursing workforce in the United
States: Data, trends and implications. Sudbury,
MA: Jones & Bartlett Publishers.
Brewer, C., Kovner, C., Green, W., & Cheng, Y.
(2009). Predictors of RNs intent to work and work
decisions: 1 year later in a U.S. national sample.
International Journal of Nursing Studies,46, 940–
956. doi:10.1016/j.ijnurstu.2008.02.003
Browne, M. W., & Cudeck, R. (1993). Alternative
ways of assessing model fit. In K. A. Bollen &
J. S. Long (Eds.), Testing structural equation mod-
els (pp. 136–162). Newbury Park, CA: Sage.
Chan, D. (1998). Functional relations among con-
structs in the same content domain at different lev-
els of analysis: A typology of composition models.
Journal of Applied Psychology,83, 234–246.
doi:10.1037/0021-9010.83.2.234
Chen, G., Bliese, P., & Mathieu, J. (2005). Concep-
tual framework and statistical procedures for delin-
eating and testing multilevel theories of homology.
Organizational Research Methods,8, 375–409.
doi:10.1177/1094428105280056
Chen, G., Webber, S., Bliese, P., Mathieu, J., Payne,
S., Born, D., & Zaccaro, S. (2002). Simultaneous
examination of the antecedents and consequences
of efficacy beliefs at multiple levels of analysis.
Human Performance,15, 381–409. doi:10.1207/
S15327043HUP1504_05
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating
goodness-of-fit indexes for testing measurement
invariance. Structural Equation Modeling,9, 233–
255. doi:10.1207/S15328007SEM0902_5
Chiang, H. Y., & Lin, S. Y. (2009). Psychometric
testing of the Chinese version of nursing practice
environment scale. Journal of Clinical Nursing,18,
919–929. doi:10.1111/j.1365-2702.2008.02433.x
Cho, S. H., Mark, B. A., Yun, S. C., & June, K. J.
(2011). Differences in intensive care unit work
environments among and within hospitals using
subscales and a composite measure of the Revised
Nursing Work Index. Journal of Advanced
Nursing,67, 2637–2648. doi:10.1111/j.1365-
2648.2011.05713.x
Choi, J., Bakken, S., Larson, E., Du, Y., & Stone, P.
(2004). Perceived nursing work environment of criti-
cal care nurses. Nursing Research,53, 370–378.
Clark, P. A., Leddy, K., Drain, M., & Kaldenberg, D.
(2007). State nursing shortages and patient satis-
faction: More RNs-better patient experiences.
Journal of Nursing Care Quality,22, 119–127. doi:
10.1097/01.NCQ.0000263100.29181.e3
Research in Nursing & Health
578 RESEARCH IN NURSING & HEALTH

Cohen, S., & Willis, T. (1985). Stress, social support,
and the buffering hypothesis. Psychological Bulle-
tin,98, 310–357. doi:10.1037/0033-2909.98.2.310
Cronbach, L., & Meehl, P. (1955). Construct validity
in psychological tests. Psychological Bulletin,52,
281–302. doi:10.1037/h0040957
Cropanzano, R., James, K., & Konovsky, M. (1993).
Dispositional affectivity as a predictor of work atti-
tudes and job performance. Journal of Organiza-
tional Behavior,14, 595–606. doi:10.1002/
job.4030140609
Cummings, G. G., Hayduk, L., & Estabrooks, C. A.
(2006). Is the nursing work index measuring up?
Moving beyond estimating reliability to testing
validity. Nursing Research,55, 82–93.
Dalal, R., & Hulin, C. (2008). Motivation for what?
A multivariate dynamic perspective of the crite-
rion. In R. Kanfer, G. Chen, & R. Pritchard (Eds.),
Work motivation: Past, present, and future (pp. 63–
100). New York, NY: Routledge.
De Gieter, S., Hofmans, J., & Pepermans, R. (2011).
Revisiting the impact of job satisfaction and orga-
nizational commitment on nurse turnover intention:
An individual differences analysis. International
Journal of Nursing Studies,48, 1562–1569.
doi:10.1016/j.ijnurstu.2011.06.007
Dyer, N., Hanges, P., & Hall, R. (2005). Applying
multilevel confirmatory factor analysis techniques
to the study of leadership. The Leadership Quar-
terly,16, 149–167. doi:10.1016/j.leaqua.2004.
09.009
Enders, C., & Tofighi, D. (2007). Centering predictor
variables in cross-sectional multilevel models: A
new look at an old issue. Psychological Methods,
12, 121–138. doi:10.1037/1082-989X.12.2.121
Erickson, R., & Ritter, C. (2001). Emotional labor,
burnout, and inauthenticity: Does gender matter?
Social Psychology Quarterly,64, 146–163.
doi:10.2307/3090130
Estabrooks, C. A., Midodzi, W. K., Cummings,
G. G., & Wallin, L. (2007). Predicting research use
in nursing organizations: A multilevel analysis.
Nursing Research,56, S7–S23.
Foley, B., Kee, C., Minick, P., Harvey, S., & Jen-
nings, B. (2002). Characteristics of nurses and hos-
pital work environments that foster satisfaction and
clinical expertise. Journal of Nursing Administra-
tion,31, 274–282.
Friese, C. R., Lake, E. T., Aiken, L. H., Silber, J. H.,
& Sochalski, J. (2008). Hospital nurse practice
environments and outcomes for surgical oncology
patients. Health Services Research,43, 1145–1163.
doi:10.1111/j.1475-6773.2007.00825.x
Gabriel, A., Diefendorff, J., & Erickson, R. (2011).
The relations of daily task accomplishment satis-
faction with changes in affect: A multilevel study
in nurses. Journal of Applied Psychology,96,
1095–1104. doi:10.1037/a0023937
Gajewski, B. J., Boyle, D. K., Miller, P. A., Oberhel-
man, F., & Dunton, N. (2010). A multilevel
confirmatory factor analysis of the Practice Envi-
ronment Scale. Nursing Research,59, 147–153.
Gibson, C. (2001). Me and us: Differential relation-
ships among goal setting, training, efficacy, and
effectiveness at the individual and team level.
Journal of Organizational Behavior,22, 789–808.
doi:10.1002/job.114
Glick, W. (1985). Conceptualizing and measuring
organizational and psychological climate: Pitfalls
in multilevel research. Academy of Management
Review,10, 601–616. doi:10.5465/AMR.1985.
4279045
Hall, R., Snell, A., & Foust, M. S. (1999). Item
parceling strategies in SEM: Investigating the sub-
tle effects of unmodeled secondary constructs.
Organizational Research Methods,2, 233–256.
doi:10.1177/109442819923002
Hanrahan, N. P. (2007). Measuring inpatient psychi-
atric environments: Psychometric properties of the
Practice Environment Scale-Nursing Work Index
(PES-NWI). The International Journal of Psychiat-
ric Nursing Research,12, 1521–1528.
Hu, L., & Bentler, P. (1999). Cutoff criteria for
fit indexes in covariance structure analysis:
Conventional criteria versus new alternative. Struc-
tural Equation Modeling,6, 1–55. doi:10.1080/
10705519909540118
Institute of Medicine. (2003). Health professions edu-
cation: A bridge to quality. Retrieved from http://
www.iom.edu/Reports/2003/Health-Professions-
Education-A-Bridge-to-Quality.aspx
Jackson, D. L., Gillaspy, J. A., Jr., & Purc-Stephen-
son, R. (2009). Reporting practices in confirmatory
factor analysis: An overview and some recom-
mendations. Psychological Methods,14, 6–23.
doi:10.1037/a0014694
James, L. (1982). Aggregation bias in estimates of
perceptual agreement. Journal of Applied Psychol-
ogy,67, 219–229. doi:10.1037/0021-9010.67.2.219
Kalisch, B. J., Lee, H., & Rochman, M. (2010). Nurs-
ing staff teamwork and job satisfaction. Journal of
Nursing Management,18, 938–947. doi:10.1111/
j.1365-2834.2010.01153.x
Kline, R. (2005). Principles and practice of struc-
tural equation modeling. New York: Guilford.
Klein, K., Dansereau, F., & Hall, R. (1994). Levels
issues in theory development, data collection, and
analysis. Academy of Management Review,19,
195–229. doi:10.5465/AMR.1994.9410210745
Kish, L. (1965). Survey sampling. New York, NY:
Wiley.
Kovner, C., Brewer, C., Fairchild, S., Poornima, S.,
Kim, H., & Jadjukic, A. (2007). A better under-
standing of newly licensed RNs and their employ-
ment patterns is crucial to reducing turnover rates.
American Journal of Nursing,107, 58–70.
Kozlowski, S., & Klein, K. (2000). A levels approach
to theory and research in organizations. In K.
Klein & S. Kozlowski (Eds.), Multi-level theory,
research, and methods in organizations:
Research in Nursing & Health
MULTILEVEL ANALYSIS OF THE PES-NWI/ GABRIEL ET AL. 579

Foundations, extensions, and new directions
(pp. 3–90). San Francisco, CA: Jossey-Bass.
Kutney-Lee, A., Lake, E., & Aiken, L. (2009). Devel-
opment of the Hospital Nurse Surveillance Capac-
ity Profile. Research in Nursing & Health,32,
217–228. doi:10.1002/nur.20316
Lake, E. (2002). Development of the Practice Envi-
ronment Scale of the Nursing Work Index.
Research in Nursing & Health,25, 176–188.
doi:10.1002/nur.10032
Lake, E. (2007). The Nursing Practice Environment:
Measurement and evidence. Medical Care
Research and Review,64, 104S–122S.
doi:10.1177/1077558707299253
Lake, E., & Friese, C. (2006). Variations in nursing
practice environments: Relation to staffing and
hospital characteristics. Nursing Research,55, 1–9.
Larson, J. (2009). The recession’s effect on jobs for
new nurses. Retrieved from http://www.nursezone.
com/nursing-news-events/more-news/The-Recession
%E2%80%99sEffect-on-Jobs-for-New-Nurses_
28968.aspx
Leiter, M. P., & Laschinger, H. K. S. (2006). Rela-
tionships of work and practice environment to pro-
fessional burnout: Testing a causal model. Nursing
Research,55, 137–146.
Liou, S. R., & Cheng, C. Y. (2009). Using the prac-
tice environment scale of the nursing work index
on Asian nurses. Nursing Research,58, 218–225.
Lindell, M., & Brandt, C. (1999). Assessing interrater
agreement on the job relevance of a test: A com-
parison of the CVI, T, rwg(j), and rwg(j) indexes.
Journal of Applied Psychology,84, 640–647.
doi:10.1037/0021-9010.84.4.640
Lucero, R., Lake, E., & Aiken, L. (2010). Nursing
care quality and adverse events in US hospitals.
Journal of Clinical Nursing,19, 2185–2195.
doi:10.1111/j.1365-2702.2010.03250.x
Maas, C. J. M., & Hox, J. J. (2004). The influence of
violations and assumptions on multilevel parameter
estimates and their standard errors. Computational
Statistics & Data Analysis,46, 427–440.
doi:10.1016/j.csda.2003.08.006
Mallidou, A. A., Cummings, G. G., Estabrooks,
C. A., & Giovannetti, P. B. (2011). Nurse specialty
subcultures and patient outcomes in acute care hos-
pitals: A multiple-group structural equation model-
ing. International Journal of Nursing Studies,48,
81–93. doi:10.1016/j.ijnurstu.2010.06.002
Muthe
´n, L., & Muthe
´n, B. (1998 –2007). Mplus
user’s guide (5th ed.). Los Angeles, CA: Author.
Muthe
´n, B. O., & Satorra, A. (1995). Complex sam-
ple data in structural equation modeling. In P.
Marsden (Ed.), Sociological methodology (pp.
267–316). Washington, DC: American Sociological
Association.
Morgeson, F. P., & Hofmann, D. A. (1999). The
structure and function of collective constructs:
Implications for multilevel research and theory
development. Academy of Management Review,24,
249–255. doi:10.5465/AMR.1999.1893935
O’Daniel, M., & Rosenstein, A. H. (2008). Profes-
sional communication and team collaboration. In
R. G. Hughes (Ed.), Patient safety and quality: An
evidence-based handbook for nurses. Rockville,
MD: Agency for Healthcare Research and Quality.
Pearlin, L. I. (1999). The stress process revisited:
Reflections on concepts and their interrelation-
ships. In C. S. Aneshensel & J. C. Phelan (Eds.),
Handbook of the sociology of mental health
(pp. 395–415). New York, NY: Kluwer Academic/
Plenum Publishers.
Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., &
Podsakoff, N. P. (2003). Common method biases
in behavioral research: A critical review of the
literature and recommended remedies. Journal of
Applied Psychology,88, 879–903. doi:10.1037/
0021-9010.88.5.879
Quinn, R., & Staines, G. (1979). The 1977 Quality of
Employment Survey. Ann Arbor, MI: Institute for
Social Research.
Rafferty, A., Ball, J., & Aiken, L. (2001). Are team-
work and professional autonomy compatible, and
do they result in improved hospital care? Quality
Health Care,10, 32–37. doi:10.1136/qhc.0100032
Raudenbush, S., & Bryk, A. (2002). Hierarchical
linear models. Thousand Oaks, CA: Sage.
Richardson, H. A., Simmering, M. J., & Sturman,
M. C. (2009). A tale of three perspectives examin-
ing post hoc statistical techniques for detection
and correction of common method variance.
Organizational Research Methods,12, 762–800.
doi:10.1177/1094428109332834
Robinson, W. S. (1950). Ecological correlations and
the behavior of individuals. American Sociological
Review,15, 351–357.
Steiger, J. (1990). Structural model evaluation and
modification: An interval estimation approach.
Multivariate Behavioral Research,25, 173–180.
doi:10.1207/s15327906 mbr2502_4
Tri-Council for Nursing. (2010). Joint statement from
the Tri-council for Nursing on recent registered
nurse supply and demand projections. Retrieved
from http://www.aacn.nche.edu/Media/NewsRe-
leases/2010/tricouncil.html
Tucker, L., & Lewis, C. (1973). A reliability coeffi-
cient for maximum likelihood of factor analysis.
Psychometrika,38, 1–10. doi:10.1007/BF02291170
U.S. Department of Labor, Bureau of Labor Statis-
tics. (2012). The 30 occupations with the largest
projected employment growth, 2010–20. Retrieved
from http://www.bls.gov/news.release/ecopro.t06.
htm
Vahey, D., Aiken, L., Sloane, D., Clarke, S., &
Vargas, D. (2004). Nurse burnout and patient satis-
faction. Medical Care,42, 1157–1166.
Van Bogaert, P., Clarke, S., Roelant, E., Meulemans,
H., & Van de Heyning, P. (2010). Impacts of unit-
level nurse practice environment and burnout on
nurse-reported outcomes: A multilevel modeling
approach. Journal of Clinical Nursing,19, 1664–
1674. doi:10.1111/j.1365-2702.2009.03128.x
Research in Nursing & Health
580 RESEARCH IN NURSING & HEALTH

Van Bogaert, P., Clarke, S., Vermeyen, K., Meule-
mans, H., & Van de Heyning, P. (2009). Practice
environments and their associations with nurse-
reported outcomes in Belgian hospitals: Develop-
ment and preliminary validation of a Dutch adapta-
tion of the Revised Nursing Work Index.
International Journal of Nursing Studies,46,
55–65. doi:10.1016/j.ijnurstu.2008.07.009
Vandenberg, R., & Lance, C. (2000). A review and
synthesis of the measurement invariance literature:
Suggestions, practices, and recommendations for
organizational research. Organizational Research
Methods,3, 3–70. doi:10.1177/109442810031002
Warshawsky, N., & Havens, D. (2011). Global use of
the practice environments scale of the nursing
work index. Nursing Research,60, 17–31.
Wharton, A. (1993). The affective consequences of
service work: Managing emotions on the job. Work
and Occupations,20, 205–232. doi:10.1177/
0730888493020002004
Research in Nursing & Health
MULTILEVEL ANALYSIS OF THE PES-NWI/ GABRIEL ET AL. 581