Content uploaded by Teniell L. Trolian
Author content
All content in this area was uploaded by Teniell L. Trolian on Oct 26, 2017
Content may be subject to copyright.
(IIHFWVRI'LYHUVLW\([SHULHQFHVRQ&ULWLFDO7KLQNLQJ
6NLOOV2YHU<HDUVRI&ROOHJH
Ernest T. Pascarella, Georgianna L. Martin, Jana M. Hanson, Teniell L. Trolian, Benjamin
Gillig, Charles Blaich
Journal of College Student Development, Volume 55, Number 1, January
2014, pp. 86-92 (Article)
3XEOLVKHGE\7KH-RKQV+RSNLQV8QLYHUVLW\3UHVV
DOI: 10.1353/csd.2014.0009
For additional information about this article
Access provided by The University of Iowa Libraries (14 Aug 2015 03:28 GMT)
http://muse.jhu.edu/journals/csd/summary/v055/55.1.pascarella.html
86 Journal of College Student Development
Research in Brief
Effects of Diversity Experiences on Critical
Thinking Skills Over 4 Years of College
Ernest T. Pascarella Georgianna L. Martin Jana M. Hanson
Teniell L. Trolian Benjamin Gillig Charles Blaich
e benefits of student engagement in diversity
experiences on a range of college outcomes
have been well documented (e.g., Chang,
Denson, Saenz, & Misa, 2006; Gurin, Dey,
Hurtado, & Gurin, 2002; Hurtado, 2001;
Jayakumar, 2008; Kuklinski, 2006). However,
the potential influence of involvement in
diversity experiences during college on the
cognitive and intellectual outcomes of post-
secondary education is only beginning to be
understood (Bowman, 2010). Gurin et al.
(2002) made a convincing argument for why
exposure to diversity experiences might foster
the development of more complex forms
of thought, including the ability to think
critically. Drawing on research that spoke to
the social aspects of cognitive development,
they pointed out that students will be more
likely to engage in effortful and complex
modes of thought when they encounter new
or novel situations that challenge current and
comfortable modes of thinking. is often can
happen in classroom settings, but also can occur
in other contexts when students encounter
others who are unfamiliar to them, when these
encounters challenge students to think or act
in new ways, when people and relationships
change and produce unpredictability, and
when students encounter others who hold
different expectations for them.
Consistent with the argument by Gurin
et al. (2002), a series of studies by Dey (1991),
Chang et al. (2006), Gurin (1999), Hurtado
(2001), and Kim (1996) have suggested that
exposure to racial and cultural diversity during
college is significantly linked to such outcomes
as student self-reported gains in “problem
solving,” “critical thinking,” “cognitive devel-
opment,” and “complexity of thinking.” is
important initial work alerted scholars to
the possibility that a considerable range of
cognitive/intellectual growth during college
might be fostered by a student’s exposure
to diversity experiences. However, although
student self-reported gains can be revealing
and important outcomes, there are some
serious concerns about their actual validity
(e.g., Bowman, 2011). Inquiry that attempts to
estimate the impact of diversity experiences on
the development of cognitive and intellectual
skills using more objective standardized
measures than student self-reported gains is
extremely limited. Two early investigations,
analyzing the first year of the nearly 20-year-
old National Study of Student Learning
longitudinal database, addressed the link
Ernest T. Pascarella is Professor and the Petersen Chair in Higher Education at the University of Iowa. Georgianna
L. Martin is Assistant Professor of Student Affairs/Higher Education at the University of Southern Mississippi. Jana
M. Hanson is a Research Analyst at Kirkwood Community College in Iowa City, IA. Teniell L. Trolian is a doctoral
student in the Higher Education and Student Affairs (HESA) Program at the University of Iowa. Benjamin Gillig
is a doctoral student in the HESA Program at the University of Iowa. Charles Blaich is Director, Center of Inquiry
at Wabash College and Director of the Higher Education Data Sharing Consortium. Research supported by a grant
from the Center of Inquiry in the Liberal Arts at Wabash College to the Center for Research on Undergraduate
Education at e University of Iowa.
J ◆ 87
Research in Brief
between diversity experiences and critical
thinking skills—as measured by the Critical
inking Test of the Collegiate Assessment
of Academic Proficiency (Pascarella, Palmer,
Moye, & Pierson, 2001; Terenzini, Springer,
Yeager, Pascarella, & Nora, 1994). These
early studies indicated that, net of important
confounding variables (such as precollege
critical thinking skills), individual diversity
experiences such as attending a racial–cultural
workshop and making friends with someone
of a different race were significantly and
positively linked to first-year gains in critical
thinking scores. e Pascarella et al. (2001)
investigation, however, also suggested that the
positive effects of involvement in such diversity
experiences on growth in critical thinking were
more pronounced for White students than for
students of color.
The most recent work estimating the
influence of diversity experiences on the
development of cognitive skills during
college analyzed the first year (2006–2007)
of the Wabash National Study of Liberal
Arts Education (WNS), a longitudinal study
focusing on the effects of liberal arts education
(Loes, Pascarella, & Umbach, 2012). Loes
et al. (2012) found that, when important
confounding experiences (e.g., precollege
critical thinking skills and tested academic
preparation) were taken into account, students’
diversity experiences had no overall significant
link with first-year gains on a standardized
measure of critical thinking skills. However,
consistent with the earlier findings of Pascarella
et al. (2001), the positive effect of diversity
experiences on critical thinking was significantly
more pronounced for White students than for
students of color. Loes et al. also reported that
the effects of diversity experiences were more
important for the students least academically
prepared for college, as indicated by relatively
low ACT scores.
e present study sought to determine if
the effects of exposure to diversity experiences
on critical thinking skills extended beyond
the first year of college and if these effects
were detectable after 4 years of postsecondary
education. Specifically, we sought to determine
if the 4-year effects of diversity experiences on
critical thinking were general (i.e., the same
for all students) or conditional (differing
in magnitude for students with different
precollege characteristics).
RESEARCH METHODS
Sample and Data Collection
We analyzed the 4th-year follow-up of the
WNS, which is a longitudinal pretest–posttest
investigation of the effects of liberal arts
experiences on cognitive and noncognitive
college outcomes. Our analyses were conducted
on 949 fourth-year, full-time undergraduates
attending 17 4-year colleges and universities
(11 liberal arts colleges, three research uni-
ver sities, and 3 regional institutions) who
completed the early Fall 2006 and the late
Spring 2010 assessments of the WNS. e
949 students completing both assessments
represented a 44.8 percent follow-up response
rate from the 2006 precollege assessment. To
adjust for response bias, a weighting algorithm
was developed from population data supplied
by each institution. Within each institution
the sample was weighted up to the population
characteristics by sex, race, and ACT (or SAT
equivalent) score.
Dependent and Independent
Variables
e dependent variable was the 40-minute, 32-
item Critical inking Test (CTT) from the
Collegiate Assessment of Academic Proficiency.
Developed by the American College Testing
Program (ACT), the CTT was designed to
measure a student’s ability to clarify, analyze,
evaluate, and extend arguments. It consists
88 Journal of College Student Development
Research in Brief
of four passages in a variety of formats (e.g.,
case studies, debates, dialogues editorials,
experimental results, statistical arguments)
followed by a set of multiple-choice test items.
e internal consistency reliabilities for the
CTT range between .81 and .82 (ACT, 1990),
and it correlates .75 with Watson Glaser
Critical inking Appraisal (Pascarella, Bohr,
Nora, & Terenzini, 1995). The CTT was
administered during both the 2006 and
the 2010 WNS assessments. The primary
independent variable in the study was the
Interactional Diversity Scale (IDS; Loes et al.,
2012). With an internal consistency reliability
of .80, the IDS consists of nine items that
assess the extent of student participation in
diversity-oriented experiences and discussions
with diverse peers and university staff during
college. Representative items include how
often a respondent: had serious conversations
with students from a different race or ethnicity;
participated in a racial or cultural awareness
workshop; had serious conversations with
students whose religious beliefs, political
opinions, or personal values were very different
from the respondent; shared personal feelings
and problems with diverse students; and had
discussions regarding intergroup relations
with diverse students. (e total set of items
is available from the first author or from
Loes et al., 2012.) e IDS was administered
during the 2010 WNS assessment, but prior
to students completing the CTT.
Analytic Model and Control Variables
In developing our regression specifications,
we were guided by a number of longitudinal
conceptual models for studying the impact of
college on students (e.g., Astin, 1993; Pascarella
& Terenzini, 1991). ese conceptual models
argue that, to validly estimate the net impact
of any single experience, one must take
into account at least three additional sets of
influences: student precollege characteristics,
the institutional context (if the data are multi-
institutional), and other college experiences that
might co-vary with the particular experience in
question and potentially confound its effects.
e student precollege variables in our study
included: demographics, such as race, sex, and
parental education; ACT (or SAT equivalent)
score (provided by each institution); a measure
of high school social/academic involvement;
and precollege measures of critical thinking
skills (the Fall 2006 CTT score) and aca-
demic motivation. Institutional type was
repre sented by a dummy variable indicating
whether or not the individual attended a
liberal arts college. Finally, other college
experiences included major field of study
(social sciences/humanities major; science,
technology, engineering, or math major; or
other major) and six measures of good/high
impact practices in undergraduate education
(Chickering & Gamson, 1987; Kuh, 2008).
ese six good/high impact practices included
the following: working individually with a
faculty member on a research project, a 23-
item (α = .92) scale measuring good teaching
and high quality interactions with faculty, a
31-item (α = .88) scale measuring academic
challenge and high expectations, a nine-
item (α = .85) scale focusing on cocurricular
involvement and positive peer interactions,
a nine-item (α = .83) scale measuring extent
of interactions with faculty and student
affairs staff, and a four-item (α = .70) scale
measuring involvement in cooperative learning
experiences. Detailed definitions of the good
practice scales are available from the author or
from Seifert, Pascarella, Goodman, Salisbury,
and Blaich (2010).
Analyses
e data analyses were carried out in two
steps. To estimate the general net effect of
interactional diversity experiences on critical
thinking skills, we regressed 2010 CTT
J ◆ 89
Research in Brief
scores on the IDS plus all other control
variables described above. To determine the
presence of conditional effects of diversity
experiences on growth in critical thinking
skills, we created cross-product terms involving
five student precollege variables—sex, race,
academic motivation, 2006 CTT score, and
ACT score—and the IDS. If, when added
to the general effects equation, these five
cross-product terms were associated with
a significant increase in explained variance
in 2010 CTT scores, it would indicate the
presence of conditional effects (Pascarella
& Terenzini, 1991). Individually significant
cross-products could then be examined and
interpreted substantially. All analyses are based
on weighted sample estimates adjusted to the
actual sample size for correct standard errors.
Because our regression models were detailed
and had more variables than there were
individual sampling units (i.e., 17 institutions)
in the data, we could not employ procedures
to statistically adjust standard errors for the
nesting effect in our data. Consequently we
TABLE 1.
Regression Estimates for General and Conditional Effects of
Interactional Diversity on End-of-Fourth-Year Critical Thinkinga
Good/High Impact Practice
General Effects
Model (N = 949)
Conditional Effects Models
Low ACT Scoreb
(n = 444)
High ACT Scorec
(n = 505)
Coefcient
(SE)t Ratio
Coefcient
(SE)t Ratio
Coefcient
(SE)t Ratio
Working With a Faculty
Member on a Research
Project
.101 (.051) 1.99 .186 (.082) 2.29 .036 (.058) 0.62
Good Teaching and High-
Quality Interactions With
Faculty
.047 (.025) 1.81 .087 (.035) 2.49 .006 (.037) 0.16
Academic Challenge and High
Expectations .021 (.031) 0.68 .007 (.045) 0.15 .010 (.041) 0.25
Co-Curricular Involvement
and Positive Peer Interactions .010 (.024) 0.41 .0003 (.035) 0.01 .029 (.030) 0.97
Interactions With Faculty
Student Affairs Staff –.128 (.058) –2.21 –.140 (.084) –1.67 –.125 (.074) –1.68
Cooperative Learning
Experiences .033 (.064) 0.51 –.002 (.094) –0.02 .063 (.080) 0.79
Interactional Diversity .090 (.028) 3.26** .189 (.041) 4.65** –.033 (.035) –0.96
Model R2.621** .513** .477**
a Regression equations also include controls for: race (person of color vs. White), sex, parental education, ACT
(or SAT equivalent) score, high school social/academic involvement, precollege critical thinking skills (Fall
2006 Critical Thinking Test score), precollege academic motivation, attendance at a liberal arts college (vs.
research university or regional institution), and undergraduate major eld of study. Coefcients for other
conditional effects were: Sex × Interactional Diversity (ID) = .085 (p > .20), Race × ID = –.205 (p > .03),
Academic Motivation × ID = .021 (p > .50), and Precollege Critical Thinking × ID = .030 (p > .50).
b ACT or ACT equivalent ≤ 27.
c ACT or ACT equivalent > 27.
**p < .001.
90 Journal of College Student Development
Research in Brief
used a more stringent alpha level (p < .01)
for statistical significance to guard against a
Type I error (Raudenbush & Bryk, 2001).
All continuous variables, including the
dependent variable, were standardized. us,
the regression coefficients for those variables
can be considered as beta coefficients, or
estimated effect sizes (i.e., the faction of
standard deviation increase or decrease in
critical thinking for a one standard deviation
increase in the independent variable).
RESULTS
The regression estimates of the effects of
interactional diversity experiences on end-
of–fourth-year critical thinking skills are
summarized in Table 1. Only the results for
the seven good/high impact practices are
shown. e control variables in the equations
are listed in footnote a. e general effects
estimates for the aggregate sample are shown in
the “General effects model” columns in Table
1. As indicated in the table, net of all other
influences, only interactional diversity had a
significant positive estimated effect on 4th-year
critical thinking skills. e magnitude of the
general effect of interactional diversity was
slightly less than .10 of a standard deviation.
e estimated general effect of interactional
diversity on critical thinking, however, was
somewhat misleading. Our test for conditional
effects was associated with a significant increase
in explained variance (1.3 percent increase,
F = 6.57 with 5/925 df,p < .001), and the
only significant individual conditional effect
was ACT Score × Interactional Diversity
(coefficient = .210,p < .001). To determine
the nature of the conditional effect, we split
the sample at approximately the mean ACT
score and reran the general effects equation
separately for the “low ACT score group”
(ACT = 27 or less) and the “high ACT score
group” (ACT = 28 or higher). e results of
those disaggregated analyses are summarized
in the “Low ACT score” and “High ACT
score” columns in Table 1. As the data in
those columns illustrate, interactional diversity
experiences had a significantly stronger positive
effect on end-of-4th-year critical thinking for
the low ACT group (about .19 of a standard
deviation,p < .001) than for the high ACT
group (–.033 of a standard deviation, p > .30).
Although none of the other conditional
effects met the critical level for significance
(p < .01), the Race × Interactional Diversity
conditional effect could be considered
marginally significant at p < .05. Consistent
with previous findings from the initial year of
postsecondary education, the positive effects
of diversity experiences on the development of
critical thinking skills over 4 years of college
appeared somewhat more pronounced for
White students (coefficient = .090) than for
students of color (coefficient = –.053).
DISCUSSION
Our findings with an objective, standardized
measure of critical thinking skills essentially
support the conceptual argument of Gurin
et al. (2002) that exposure to diversity
experiences fosters the development of
cognitive growth and more complex modes
of thought. e cognitive effect of diversity
experiences appears to be sustained during
4 years of college and may even increase
in magnitude over time. For example, in
the earlier study by Loes et al. (2012), the
general effect of interactional diversity on
first-year critical thinking skills was small and
not significant (.04 of a standard deviation).
ough still modest in absolute terms, the
general cognitive effect of diversity experiences
after 4 years was estimated at .09 of a standard
deviation, which was significant at p < .001.
Moreover, the 4-year effect in our findings
persisted even in the presence of controls, not
J ◆ 91
Research in Brief
only for important precollege experiences and
traits (e.g., precollege critical thinking scores
and ACT scores), but also for a wide range of
other important college experiences.
As with previous research, however, the
cognitive impact of interactional diversity over
4 years of college would appear to be more
complicated than the general effects estimates
discussed above would suggest. Most notably,
the 4-year impact of interactional diversity in
our study would appear to be contingent on the
level of tested academic preparation with which
a student enters postsecondary education.
Students entering college with relatively low
ACT (or equivalent) scores derived substantially
greater 4-year critical thinking benefits
from engagement in interactional diversity
experiences than did their counterparts in the
upper half of the ACT distribution. is suggests
that, in terms of fostering the development of
critical thinking skills over 4 years of college,
interactional diversity may act in a compensatory
manner for those students who are relatively
less prepared to acquire critical thinking skills
from undergraduate academic experiences when
they enter college.
Although we did not find a conditional
effect involving race and interactional
diversity meeting the a priori level of statistical
significance (p < .01), the effect was significant
at p < .05. The nature of this marginally
significant conditional effect was consistent
with previous findings suggesting that the most
pronounced cognitive benefits of diversity
experiences during college may actually accrue
to White students.
From the broader perspective of college
impact research, our findings underscore the
fact that student characteristics may often shape
the developmental influence of postsecondary
education. This study contributes to the
mounting evidence of the educational impact
of engagement in diversity experiences during
college. At the same time, however, it suggests
that purposefully programming exposure to
diversity into the undergraduate experience
may not yield the same benefits to all students.
Indeed, as the undergraduate population of
American colleges and universities becomes
more diverse, we might legitimately anticipate
an increase in such conditional impacts. Future
research on the developmental impacts of
diversity experiences might well profit from
the routine estimation of conditional effects
based on measurable student characteristics.
Such inquiry could be helpful in identifying
additional student subgroups for whom
engagement in diversity experiences may be
particularly beneficial.
Correspondence concerning this article should be
addressed to Ernest Pascarella, EPLS, N491 Lindquist
Center, University of Iowa, Iowa City, IA 52242; ernest-
pascarella@uiowa.edu
92 Journal of College Student Development
Research in Brief
REFERENCES
American College Testing Program. (1990). Report on the
technical characteristics of CAAP; Pilot year 1: 1988-89.
Iowa City, IA: Author.
Astin, A. W. (1993). What matters in college? Four critical years
revisited. San Francisco, CA: Jossey-Bass.
Bowman, N. (2010). College diversity experiences and cognitive
development: A meta-analysis. Review of Educational Research,
80(1), 4-33.
Bowman, N. (2011). e validity of college student self-reported
gains at diverse institutions. Educational Researcher, 40(1),
22-24.
Chang, M. J., Denson, N., Saenz, V., & Misa, K. (2006). e
educational benefits of sustaining cross-racial interaction
among undergraduates. Journal of Higher Education, 77,
430-455.
Chickering, A., & Gamson, Z. (1987). Seven principles for
good practice in undergraduate education. AAHE Bulletin,
39(7), 3-7.
Dey, E. (1991). Community service and critical thinking: An
exploratory analysis of collegiate influences. Paper presented
at the conference on Setting the Agenda for an Effective
Research Strategy for Combining Service and Learning in
the 1990s, Racine, WI.
Gurin, P. (1999). Expert report of Patricia Gurin. In University of
Michigan (Ed.), e compelling need for diversity in education,
Gratz et al., v Bollinger et al., No. 97-75237 and Grutter et al.
v Bollinger et al., No. 97-75928. Ann Arbor, MI: University
of Michigan. Retrieved from http://www.umich.edu/~urel/
admissions/legal/expert/gurintoc.html
Gurin, P., Dey, E., Hurtado, S., & Gurin, G. (2002). Diversity
and higher education: eory and impact on educational
outcomes. Harvard Educational Review, 72, 330-366.
Hurtado, S. (2001). Linking diversity and educational purpose:
How diversity affects the classroom environment and student
development. In G. Orfield (Ed.), Diversity challenged:
Evidence on the impact of affirmative action (pp. 187-203).
Cambridge, MA: Harvard Educational Publishing Group.
Jayakumar, U. M. (2008). Can higher education meet the
needs of an increasingly diverse and global society? Campus
diversity and cross-cultural workforce competencies. Harvard
Educational Review, 78, 615-651.
Kim, M. (1996, April). e effectiveness of women-only colleges
for intellectual development. Paper presented at the annual
meeting of the American Educational Research Association,
New York, NY.
Kuh, G. (2008). High-impact educational practices: What they
are, who has access to them, and why they matter. Washington,
DC: Association of American Colleges and Universities.
Kuklinski, J. H. (2006). The scientific study of campus
diversity and students’ educational outcomes. Public Opinion
Quarterly, 70(1), 99-120.
Loes, C., Pascarella, E., & Umbach, P. (2012). e effects of
diversity experiences on critical thinking skills: Who benefits?
Journal of Higher Education, 83, 1-25.
Pascarella, E., Bohr, L., Nora, A., & Terenzini, P. (1995).
Cognitive effects of 2-year and 4-year colleges: New evidence.
Educational Evaluation and Policy Analysis, 17(1), 83-96.
Pascarella, E., Palmer, E., Moye, M., & Pierson, C. (2001).
Do diversity experiences influence the development of
critical thinking? Journal of College Student Development,
42, 257-271.
Pascarella, E. T., & Terenzini, P. T. (1991). How college affects
students: Findings and insights from twenty years of research.
San Francisco, CA: Jossey-Bass.
Raudenbush, S., & Bryk, A. (2001). Hierarchical linear models: Appli-
cations and data analysis methods. ousand Oaks, CA: SAGE.
Seifert, T., Pascarella, E., Goodman, K., Salisbury, M., &
Blaich, C. (2010). Liberal arts colleges and good practices
in undergraduate education: Additional evidence. Journal of
College Student Development, 51, 1-22.
Terenzini, P., Springer, L., Yaeger, P., Pascarella, E., & Nora, A.
(1994, November). e multiple influences on students’ critical
thinking skills. Paper presented at the annual meeting of the
Association for the Study of Higher Education, Orlando, FL.
