Content uploaded by Kathleen M Goodman
Author content
All content in this area was uploaded by Kathleen M Goodman
Content may be subject to copyright.
The Effects of Liberal Arts Experiences on Liberal Arts
Outcomes
Tricia A. Seifert ÆKathleen M. Goodman ÆNathan Lindsay Æ
James D. Jorgensen ÆGregory C. Wolniak ÆErnest T. Pascarella Æ
Charles Blaich
Received: 21 June 2007 / Published online: 30 October 2007
Springer Science+Business Media, LLC 2007
Abstract Despite scholars’ praise of liberal arts education as a model form, very little
research has examined the actual impact of liberal arts education on learning outcomes.
The elaborate rhetoric and anecdotal support, long used to advance liberal arts education as
the premier type of education with value for all, is no longer sufficient. The practices and
conditions that lead to outcomes of a liberally educated student remain an empirical black
box. Guided by the work of Pascarella et al. [2005,Liberal arts colleges and liberal arts
education: New evidence on impacts. ASHE Higher Education Report, 31(3)], this study
examined the extent to which an institutional ethos, that values student–student and stu-
dent–faculty interaction within a supportive environment characterized by high
expectations for developing the intellectual arts, manifests in the lived experiences of
students and predicts the development of outcomes theoretically associated with the liberal
arts. Specifically, we investigated the construct and predictive validity of the liberal arts
experience scale relative to liberal arts outcomes. Using data from the first phase of the
Wabash National Study of Liberal Arts Education, net of student background character-
istics and institution attended, we found liberal arts experiences had a positive effect on
four of six liberal arts outcomes, including intercultural effectiveness, inclination to inquire
and lifelong learning, well-being, and leadership.
Keywords Liberal arts education Learning outcomes Liberal arts
Liberally educated
T. A. Seifert (&)K. M. Goodman J. D. Jorgensen E. T. Pascarella
N491 Lindquist Center, The University of Iowa, Iowa City, IA 52242, USA
e-mail: tricia-seifert@uiowa.edu
N. Lindsay
The University of North Carolina Wilmington, Wilmington, NC, USA
G. C. Wolniak
National Opinion Research Center at the University of Chicago, Chicago, IL, USA
C. Blaich
The Center of Inquiry in the Liberal Arts at Wabash College, Crawfordsville, IN, USA
123
Res High Educ (2008) 49:107–125
DOI 10.1007/s11162-007-9070-7
Since Aristotle (350 B.C.E) differentiated ‘‘liberal’’ from ‘‘illiberal’’ education more than
2,000 years ago, liberal arts education
1
has been widely praised as a model form of
education, especially in the United States. Numerous publications describe the virtues of
liberal arts education, including the renowned Yale Report in 1828 (see Turner 1996),
which determined that a traditional liberal arts curriculum is the best means to prepare for a
changing society. The more contemporary Association of American Colleges & Univer-
sities [AAC&U] (2002) report, Greater Expectations: A New Vision for Learning as a
Nation Goes to College, named liberal education as the best means to prepare students for
the multiple demands of life in the 21st century. Advocates of liberal arts education claim
that it produces ‘‘broad and deep learning…’’ allowing students to acquire ‘‘…a rich fund
of meaningful knowledge’’ (AALE 2003, p. 7).
However, as society and our system of higher education evolve, educators struggle to
determine not only the purpose of a college education but in what manner that purpose is
best achieved. Many argue for liberal education as opposed to practical education, defining
the distinction as ‘‘the college way versus the university way, tradition or sentiment against
size and money, the finishing school and the trade school’’ (Matthews 1997, p. 106). Others
advocate a liberal education emphasizing practical skills, regardless of field of study or
intended career (AAC&U 2002). The recent report of the Spelling’s Commission (U.S.
Department of Education 2006) focused national attention on college access and quality of
learning, without consideration of pedagogy, curriculum, or specific educational practices.
The popularity of the U.S. News and World Report college rankings have sparked public
discourse on what makes a given college or university education valuable (Farrell and Van
Der Werf 2007).
These varied opinions and public debates demonstrate the need to find objective criteria
to assess the value of specific types of education as they relate to desired learning and
developmental outcomes. The elaborate rhetoric and anecdotal support, long used to
advance liberal arts education as the premier type of education with value for all, is no
longer sufficient. The institutional practices and conditions that lead to outcomes associ-
ated with a liberally educated student remain an empirical black box. The purpose of the
present study is to examine the extent to which an institutional ethos, that values student–
student and student–faculty interaction within a supportive environment characterized by
high expectations for developing the intellectual arts, manifests in the lived experiences of
students and thus predicts the development of the intellectual arts and skills necessary for a
life of substance and achievement (Center of Inquiry in the Liberal Arts 2006). Specifi-
cally, we aim to shine a light into the black box of liberal arts education to better
understand what institutional practices and conditions lead to the development of liberal
arts outcomes.
Review of the Literature
Liberal Arts Colleges
Despite the long-held status and assumed benefits of liberal arts education, little research
has carefully examined the impact of either the structure or the practices until recently.
1
As noted in Pascarella et al. (2005), we recognize the debate and distinctions that have been drawn
between ‘‘liberal education’’ and ‘‘liberal arts education’’. We use the term ‘‘liberal arts education’’ in this
paper.
108 Res High Educ (2008) 49:107–125
123
Pascarella et al. (2005) published an extensive analysis examining the institutional effects
of attending a liberal arts college on student learning. They found that attending a liberal
arts college, compared to a research university or regional institution, yielded mixed effects
on student learning outcomes (i.e., positive effects on openness to diversity/challenge and
learning for its own sake but negative effects on mathematics and science reasoning).
Controlling for a host of demographic, precollege, and college characteristics, including a
pretest on every outcome, attending a liberal arts college promoted the development of
some outcomes, inhibited the development of others, and largely had no significant effect
on the development of many of the learning outcomes under examination.
Liberal Arts Ethos and Practices
As part of their study of liberal arts education, Pascarella et al. (2005) also analyzed the
effects of a combination of teaching practices and institutional conditions that capture the
basic environmental elements of many liberal arts colleges on student learning. The Center
of Inquiry in the Liberal Arts working definition of a liberal arts education informed the
selection of the particular combination of institutional practices and conditions. The Center
of Inquiry’s working definition holds that a liberal arts education is characterized by an
institutional ethos that values: (a) the development of a set of intellectual arts (e.g.,
intellectual openness to inquire and discover; and the ability and desire to adopt a critical
perspective of one’s and other’s beliefs) more than professional or vocational skills; (b)
curricular and environmental structures that work in combination to create a coherent
integrity to students’ intellectual experience; and (c) an institutional tradition of student–
student and student–faculty interaction both in and out of the classroom (Blaich et al.
2004).
Although institutional type had inconsistent effects on the learning outcomes under
investigation, the practices and conditions students experienced positively affected their
development on a wide range of learning outcomes, from valuing literacy and learning for
its own sake to scientific reasoning and critical thinking (Pascarella et al. 2005). These
practices and conditions also acted in a compensatory manner, resulting in an even greater
positive influence on student learning for academically at-risk students, women, and stu-
dents of color (Pascarella et al.). Moreover, this combination of practices and conditions
effectively promoted these outcomes regardless of the kind of institution students attended.
In other words, student learning flourished where students experienced these powerful
practices and conditions, but faltered in their absence.
Although these findings confirm the assertions made by AAC&U (1998,2002) that a
liberal arts education is not institution specific, or even discipline specific, Pascarella and
colleagues found that students who attended liberal arts colleges were more likely to
experience these powerful practices and conditions than their peers at other types of
institutions. In three separate analyses, the practices and conditions under examination
strongly differentiated liberal arts colleges from their research and regional counterparts
(Pascarella et al. 2005).
Because the practices and conditions under examination displayed adequate discrimi-
nant validity by identifying known groups, Pascarella and his colleagues (2005) defined the
practices and conditions as those that appeared to empirically characterize liberal arts
education and then created commensurate scales for measurement purposes. The liberal
arts emphasis (i.e., a scale aggregated at the institutional level) and the liberal arts expe-
riences (i.e., a scale at the individual student level) scales were developed to measure the
Res High Educ (2008) 49:107–125 109
123
practices and conditions characteristic of liberal arts education. These variables included
student self-reports of the following institutional practices and conditions: scholarly/
intellectual emphasis of campus; number of essay exams; cumulative credit hours taken;
extracurricular involvement; ratio of liberal arts courses to vocational courses; quality of
non-classroom interactions with faculty; faculty interest in teaching and student devel-
opment; instructional skill/clarity; academic effort/involvement; supportive relationships
with students, staff, and faculty; quality of interactions with students; integration of ideas;
course challenge/effort; and instructional organization and preparation.
The liberal arts emphasis and liberal arts experience scales built on Chickering and
Gamson’s (1987,1991,1999) ‘‘good practices in undergraduate education’’ and the
National Survey of Student Engagement’s benchmarks (Kuh 2001,2003). Substantial
research has found these good practices and benchmarks associate positively with student
learning (Chickering and Reisser 1993; Cruce et al. 2006; Kuh et al. 2005; Pascarella and
Terenzini 1991,2005). However, this research has focused largely on the individual impact
of one good practice or benchmark on student learning. Rather than separate indicators, the
liberal arts emphasis and liberal arts experiences variables in the Pascarella et al. (2005)
study captured the seamlessness of the collegiate learning experience. This is especially
beneficial given that ‘‘the sources of influence on student development are themselves
holistic’’ (Terenzini et al. 1996, p. 149) and that multiple, diverse, interdependent, and
reinforcing experiences or conditions influence change (Pascarella and Terenzini 2005;
Terenzini et al. 1996). Thus, it is the seamless or holistic interconnectedness of the liberal
arts experiences that conceptually separates them from the more general ‘‘good practices.’’
In addition, the label of liberal arts experiences is appropriate, given research showing that
students in liberal arts colleges and liberal arts disciplines are exposed to ‘‘good practices’’
more frequently than students in other colleges and disciplines (Astin 2000; Pascarella
et al. 2004,2005).
The notion of a seamless learning environment is not new to higher education. In light
of the historical liberal arts tradition of student-focused education dating back to Socrates
and Plato, one can argue that the best practices in undergraduate education actually stem
from a liberal arts context. More recently, a host of scholars have extolled the benefits of an
environment in which those areas once believed to be separate and distinct (e.g., in-class
versus out-of-class; curricular versus cocurricular) are interwoven into a fluid and con-
tinuous whole (Kuh 1996; Kuh et al. 1991,2005). The liberal arts emphasis and experience
scales attempt to measure the practices and conditions that contribute to a seamless
institutional ethos, which values student–student and student–faculty interaction within a
supportive environment characterized by high expectations for developing a set of intel-
lectual arts and habits of mind more than professional or vocational skills.
Liberal Arts Educational Outcomes
The Wabash National Study of Liberal Arts Education seeks to extend the findings of
Pascarella et al. (2005) by enriching the range of measured outcomes, institutional con-
ditions, and teaching practices through the use of multiple methods of inquiry (Center of
Inquiry in the Liberal Arts 2006). The liberal arts emphasis and experience scales exam-
ined by Pascarella et al., while positively associated with the development of numerous
student learning outcomes, were not tested in terms of predicting outcomes specifically
associated with liberal arts education. The educational outcomes particularly tied to the
liberal arts were determined through an extensive literature review and analysis (see King
110 Res High Educ (2008) 49:107–125
123
et al. in press). As expected, the liberal arts outcomes under investigation overlap some-
what with many general educational outcomes. However, King et al. explain that the
distinctiveness of the liberal arts outcomes lies in their integrated connections with the
other outcomes, as well as their holistic characteristics that span the cognitive, intraper-
sonal, and interpersonal domains. Collectively, the liberal arts outcomes embody ‘‘a
cultivation of the whole human being for the functions of citizenship and life generally’’
(Nussbaum 1997, p. 9).
The lack of research on liberal arts outcomes noted by Pascarella et al. (2005) served as
the foundation for the basic research question that continues to guide our work: after
controlling for an array of background characteristics and institution type, to what extent, if
any, do students’ liberal arts experiences influence liberal arts outcomes?
Methods
Sample
The sample consisted of students from the four institutions participating in the pilot phase
of the Wabash National Study of Liberal Arts Education. The institutions represented three
states, differed in Carnegie classification, institutional control, and selectivity; we selected
them because of their willingness to participate in piloting the data collection processes.
We included a research university, a regional institution with limited graduate programs, a
liberal arts college, and a community college.
We randomly sampled and invited students from each institution to participate in the
study and sought approximately 200 students from each institution evenly distributed
across years in school. In the event students of color did not comprise 10% of the insti-
tution’s undergraduate student body, we oversampled students of color. Students received a
$50 cash stipend for participating. In all, we invited 3,820 students to participate and 909
registered (a 23.8% response rate). We developed a sample weighting algorithm to adjust
for sample bias by gender and year in school for each institution.
Data Collection
We collected data in three phases. In the first phase, students completed a registration form
with basic demographic information either online or by mail. After registration, students
then received a paper copy of the college experiences questionnaire, which measured a
range of in- and out-of-classroom experiences as well as the openness to diversity and
positive attitude toward literacy scales. Among those who registered, 723 completed the
questionnaires. Finally, students attended a monitored session in which they completed one
of two assessment batteries. Based on a matrix sample, we randomly assigned students to
an assessment group. Assessment Group A completed the Reasoning and Current Issues
[RCI] test (Wood et al. 2002), the Intercultural Developmental Inventory [IDI] (Hammer
and Bennett 2001), and the Scales of Psychological Well-being (Ryff 1989; Ryff and
Keyes 1995), while Assessment Group B completed the Defining Issues Test-2 [DIT-2]
(Rest et al. 1999), the Intercultural Developmental Inventory (Hammer and Bennett), Need
for Cognition (Cacioppo et al. 1984), and the Socially Responsible Leadership Scale
(Tyree 1998). A total of 601 students completed the matrix of assessment instruments (285
students in Assessment Group A and 316 in Assessment Group B). Because of attrition
Res High Educ (2008) 49:107–125 111
123
between completing the questionnaire and the assessments, the matrix sampling procedure,
and the different models estimated, the samples for these analyses vary from 708 to 279.
We specify the sample size for each outcome in the tables that follow.
Variables
The main independent variable of interest in these analyses was the ‘‘liberal arts experi-
ences’’ variable. Our conceptually-based scale draws upon the work of Pascarella et al.
(2005) but differs in some elements. We present the differences between the operational
definitions of the two liberal arts experiences scales in Table 1. We derived the scale for
the present study from student reports of the following college experiences that we
believed to be most closely aligned with the Center of Inquiry’s definition of a liberal arts
education (Blaich et al. 2004): positive and influential student–faculty contact; faculty
interest in teaching and student development; instructional clarity, organization, and
preparation; academic effort and challenge; degree to which the institution is supportive;
positive influence of interactions and relationships with peers; integration of ideas through
class activities and assignments; challenging classroom environment characterized by high
expectations; instructor feedback to students; emphasis on higher-order examinations and
assignments; frequency of engaging in cooperative learning activities; frequency of faculty
contact; frequency of student affairs contact; overall diversity experiences and interactions;
academically meaningful out-of-class experiences; involvement with active learning;
diversity courses; and out-of-class research with faculty member. The areliability of the
liberal arts experience scale was .842.
Because we were interested in estimating the net effect of liberal arts experiences on
liberal arts outcomes, we included students’ age, race (a dichotomous variable for White
versus student of color),
2
gender (with female as the reference category), parents’ edu-
cation and household income, high school GPA, a scaled measure of high school
involvement, precollege academic ability, educational aspirations, and the racial compo-
sition of high school to serve as controls for student background characteristics. We also
added a series of dichotomous variables representing type of institution attended, with the
liberal arts college serving as the reference category. We added students’ political attitude
as a control to two of our analyses, because of its effect on the development of intercultural
effectiveness.
Drawing from the breadth of literature on liberal arts education, we selected a number of
outcomes we believe conceptually and theoretically relate to liberal arts education (Center
of Inquiry in the Liberal Arts 2006; King et al. in press). Based on our review of the
literature, we estimated the effects of students’ liberal arts experiences on the following
liberal arts outcomes: moral reasoning, effective reasoning and problem solving, inter-
cultural effectiveness, inclination to inquire and lifelong learning, well-being, and
leadership. These outcomes are theoretically and conceptually-related to liberal arts edu-
cation and as such required separate operationalization for their appropriate measurement.
We measured the liberal arts outcomes using a host of quantitative instruments. We
measured students’ moral reasoning by using the Defining Issues Test, version 2 [DIT-2]
(Rest et al. 1999). We used the Reasoning and Current Issues [RCI] test (Wood et al. 2002)
to measure students’ effective reasoning and problem solving. We measured the outcome
2
Because we had small sample numbers by racial/ethnic groups and that the variable serves solely as a
control in the analyses, we have chosen to retain the dichotomous racial/ethnic distinction.
112 Res High Educ (2008) 49:107–125
123
intercultural effectiveness through the use of two measures: the Intercultural Develop-
mental Inventory [IDI] (Hammer and Bennett 2001) and the openness to diversity/
challenge scale (Pascarella et al. 2005). We also used two measures to gauge students’
Table 1 Operationalizations of liberal arts experiences scale
Constituent scales and items as
defined by Pascarella et al. (2005)
Liberal arts
experiences
(Pascarella
et al. 2005)
Liberal arts
experiences
(present study)
Constituent scales and items as
defined in present study
Scholarly/intellectual emphasis of
campus
9
Number of essay exams 9
Cumulative credit hours taken 9
Extracurricular involvement scale 9
Coursework ratio of liberal arts
courses to vocational/technical
courses
9
Quality of non-classroom
interactions with faculty
99 Positive and influential student–
faculty contact
Faculty interest in teaching and
students’ development
99 Faculty interest in teaching and
students’ development
Instructional skill/clarity 99 Instructional clarity, organization,
and preparation
Academic effort/involvement 99 Academic effort and challenge
Supportive relationships 99 Degree to which institution is
supportive
Quality of interactions with students 99 Positive influence of interactions and
relationships with peers
Integration of ideas 99 Integration of ideas through class
activities and assignments
Course challenge/effort 99 Challenging classroom environment
characterized by high expectations
Instructional organization &
preparation
99 Instructional clarity, organization,
and preparation
9Instructor feedback to students
9Emphasis on higher-order
examinations and assignments
9Frequency of engaging in
cooperative learning activities
9Frequency of faculty contact
9Frequency of student affairs contact
9Overall diversity experiences and
interactions
9Academically meaningful out-of-
class experiences
9Involvement with active learning
9Diversity courses
9Out-of-class research with faculty
member
Res High Educ (2008) 49:107–125 113
123
inclination to inquire and lifelong learning: the Need for Cognition (Cacioppo et al. 1984)
and the positive attitude toward literacy scale (Pascarella et al.). We measured well-being
using the six scales of psychological well-being identified by Ryff (1989) and colleagues
(Ryff and Keyes 1995). Finally, we measured leadership by using the eight scales
developed by Tyree (1998). The eight critical values of the social change model of
leadership development (Higher Education Research Institute 1996) serves as the basis for
the Socially Responsible Leadership Scale [SRLS] (Tyree). We present detailed descrip-
tive information for variables in the analyses in Table 2.
Analyses
In an effort to maximize our statistical power, we retained the largest sample for each
dependent measure. Therefore, we have a different sample size for each of our analyses,
which we recognize can make interpretation of the findings across different analytic
samples challenging. In order to account for any possible significant variation in the
demographic, precollege, and institutional variables across the different models, we
compared the descriptive statistics of these variables between the largest models (n= 708)
and the smaller models (n= 279). We note differences in the analytic samples on the
demographic, precollege, and college experience variables, where appropriate, in Table 2,
as well as the sample size for each dependent variable.
We used Ordinary Least Squares (OLS) regression to conduct the analyses. Using
weighted data, we estimated total and direct general effects models predicting either
overall measures or sub-scales, depending on the properties of the instrument. In the total
effects model, we regressed the liberal arts outcomes on the background characteristics and
the dichotomous institutional variables. The direct effects model was similar to the total
effects model but we added the liberal arts experience variable into the regression speci-
fication. This allowed us to examine the unique effect of students’ liberal arts experiences
on the various liberal arts outcomes.
3
Our results report the direct effects standardized
regression coefficient (b) of student reports of their liberal arts experiences on the out-
comes. Thus, brepresents the standard deviation change of the dependent variable that is
associated with a standard deviation increase in the independent variable (Cohen et al.
2003). We present the results of the direct effects models in Table 3.
Results
Overall, net of an extensive battery of student background characteristics and institution
attended, we found students’ liberal arts experiences positively affected four of the six
liberal arts outcomes. Adding the liberal arts experience variable significantly changed the
amount of explained variation in a host of the liberal arts outcomes’ measures from slightly
more than 1% to more than 14%. The magnitude of the statistically significant effects of
3
Because the sample from this study was not randomly assigned to treatment conditions (college versus
other post-high school experience), we are not able to use the term ‘effect’ in the experimental sense.
‘Effect’ is used rather in its statistical sense in that variables which may have a confounding influence on the
relationship under examination have been statistically controlled, leaving that part of the variance in ythat
can be attributed to x(Shadish et al. 2002).
114 Res High Educ (2008) 49:107–125
123
Table 2 Descriptive information for all variables in models
a
Operational definition of variable Sample
size
areliability Mean SD Statistically significant
differences in analytic
samples
Dependent variables—liberal arts outcomes
Moral reasoning
Defining issues test of moral
reasoning (DIT-2)
postconventional score
A 5-dilemma/story instrument that measures, among
other constructs, the extent to which one prefers post-
conventional moral thinking (i.e., using moral ideals
and/or theoretical frameworks for resolving complex
moral issues). Scoring algorithm developed by the
Center for Ethical Development at the University of
Minnesota. Reliability information from Steve
Thoma (personal communication, September 26,
2007)
279 Ranges from upper
.60s to .70s in
restricted samples
34.98 14.94
Effective reasoning and problem solving
Reasoning and current issues
(RCI) test reflective judgment
score
A 9-item instrument that measures the capacity to
recognize and endorse statements that reflect the
attributes of reflective thinking applied to current
issues. Scoring algorithm developed by Wood et al.
(2002)
279 a= .41 4.97 0.68
Intercultural effectiveness
Intercultural developmental
inventory (IDI) developmental
score
A 50-item instrument that measures intercultural
sensitivity along a developmental continuum from
ethnocentrism to ethnorelativism. Scoring algorithm
developed by Hammer and Bennett (2001).
Reliability information from Mitchell Hammer
(personal communication, September 26, 2007)
588 a= .83 89.29 14.65
Openness to diversity/challenge An individual’s score on a seven-item Likert-type scale
(5 = strongly agree to 1 = strongly disagree) that
assesses openness to cultural, racial, and values
diversity as well as the extent to which one enjoys
being challenged by different perspectives
708 a= .85 26.69 4.48
Res High Educ (2008) 49:107–125 115
123
Table 2 continued
Operational definition of variable Sample
size
areliability Mean SD Statistically significant
differences in analytic
samples
Inclination to inquire and lifelong learning
Need for cognition An 18-item scale that measures the degree to which one
enjoys engaging in effortful cognitive activities
309 a= .89 65.67 11.38
Positive attitude toward literacy
scale
An individual’s score on a seven-item Likert-type scale
(5 = strongly agree to 1 = strongly disagree) that
assessed a positive attitude toward reading and
writing
708 a= .65 @0.01 4.11
Well-being
Scales of psychological well-
being
All scales contain nine items 279
Autonomy The extent to which one, is self-determined and
independent, is able to resist social pressures to think
and act in certain ways, regulates behavior from
within, and evaluates one’s self by personal standards
a= .77 40.54 6.57
Positive relationship with
others
The extent to which one has warm, satisfying, and
trusting relationships with others, is concerned about
others’ welfare, is capable of strong empathy,
affection, and intimacy, and understands give and
take of human relationships
a= .83 43.24 7.71
Environmental mastery The extent to which one has a sense of mastery and
competence in managing the environment, controls
complex arrays of external activities, makes effective
use of surrounding opportunities, is able to choose or
create contexts suitable to personal needs and values
a= .80 40.30 6.84
116 Res High Educ (2008) 49:107–125
123
Table 2 continued
Operational definition of variable Sample
size
areliability Mean SD Statistically significant
differences in analytic
samples
Personal growth The extent to which one has a feeling of continued
development, sees self as growing and expanding, is
open to new experiences, has a sense of realizing his
or her potential, sees improvement in self and
behavior over time, and is changing in ways that
reflect more self-knowledge and effectiveness
a= .70 45.61 5.24
Life purpose The extent to which one has goals in life and a sense of
directedness, feels there is meaning to present and
past life, holds beliefs that give life purpose, and has
aims and objectives for living
a= .78 44.28 6.19
Leadership
Socially responsible leadership
scales
310
Consciousness of self A 10-item scale that measures being aware of the
beliefs, values, attitudes and emotions that motivate a
person to take action
a= .82 50.65 6.49
Congruence An 8-item scale that measures thinking, feeling and
behaving with consistency, genuineness, authenticity,
and honesty towards others
a= .80 48.45 5.59
Commitment A 9-item scale that measures having the energy that
motivates an individual to serve and that drives the
collective effort
a= .87 55.54 5.98
Collaboration An 11-item scale that measures working with others in a
common effort
a= .81 47.48 5.40
Common purpose An 11-item scale that measures having shared goals and
values when working with others
a= .82 52.07 5.38
Res High Educ (2008) 49:107–125 117
123
Table 2 continued
Operational definition of variable Sample
size
areliability Mean SD Statistically significant
differences in analytic
samples
Controversy with civility An 11-item scale that measures believing in two
fundamental realities of any creative effort: (1) that
differences in viewpoint are inevitable, and (2) that
such differences must be aired openly but with
civility
a= .75 52.00 5.93
Citizenship A 12-item scale that measures believing in a process
whereby an individual and/or group becomes
responsibly connected to the community and to
society through some activity
a= .92 55.41 7.81
Change An 11-item scale that measures believing in the
importance of making a better world and a better
society for oneself and others
a= .84 45.86 5.75
Independent variable
Liberal arts experiences (LAE) variable Defined in body of text a= .84 50.06 9.81
Control variables
Community College (1) versus others (0) 0.25 0.43
Liberal Arts College (1) versus others (0) 0.25 0.43
Research University (1) versus others (0) 0.25 0.43
Regional University (1) versus others (0) 0.25 0.43
Age in 2005 22.26 4.78
Male (1) versus female (0) 0.44 0.50 Higher proportion
of women
in RCI and
well-being
samples*
118 Res High Educ (2008) 49:107–125
123
Table 2 continued
Operational definition of variable Sample
size
areliability Mean SD Statistically significant
differences in analytic
samples
White (1) versus students of color (0) 0.80 0.40
Parents’ total level of education Sum of student-reported mother and father’s
education level, coded 1 = did not finish high
school; 2 = high school/GED; 3 = college, no
degree; 4 = vocational/technical certificate/
diploma; 5 = associate/2–year degree;
6 = bachelor’s/4-year degree; 7 = master’s;
8 = law; 9 = doctorate
9.39 3.75 Higher level of parents’
education in DIT-2
and SRLS samples*
Household income Student’s report of family household income, coded
1 = less than $10,000; 2 = $10,000–14,999;
3 = $15,000–19,999; 4 = $20,000–24,999;
5 = $25,000–29,999; 6 = $30,000–39,999;
7 = $40,000–49,999; 8 = $50,000–59,999;
9 = $60,000–74,999; 10 = $75,000–99,999;
11 = $100,000–149,999; 12 = $150,000–
199,999; 13 = $200,000–249,999; 14 = $250,000
or more
7.46 3.46
High school grade point average Student’s self-report of estimated high school grade
point average, coded 1 = below D; 2 = D to C-;
3 = C- to C; 4 = C to B-; 5 = B- to B; 6 = B to
A-; 7 = A- to A
6.24 1.07
Racial composition of high school Student’s self-report of high school racial
composition, coded 1 = almost all white students;
2 = mostly white students; 3 = roughly half white
students and half minority students; 4 = mostly
minority students; 5 = almost all minority
students
1.97 1.01
Res High Educ (2008) 49:107–125 119
123
Table 2 continued
Operational definition of variable Sample
size
areliability Mean SD Statistically significant
differences in analytic
samples
Precollege academic ability Pre-college academic ability, constructed from
standardized values (Mean = 100, SD = 1.0) of
either, composite ACT score, composite SAT
score, or the average of these scores
99.92 0.88
Highest degree expected to obtain Students’ self-report of the highest degree they
expect to obtain, coded 1 = vocational/technical
certificate or diploma; 2 = associate degree
(A.A., A.S., or equivalent); 3 = bachelor’s degree
(B.A., B.S., etc.); 4 = master’s degree (M.A.,
M.S., M.B.A., etc.); 5 = law (J.D.); 6 = doctorate
(Ph.D., Ed.D., M.D.)
4.12 1.23
High school involvement An individual’s score on a seven-item Likert-type
scale (5 = very often to 1 = never) that assessed
the frequency with which a student engaged in the
following during the year of high school: (1)
studying or doing homework in groups; (2)
socializing with friends; (3) talking with teachers
outside of class; (4) performing community
service or volunteer work; (5) exercising or
participation in sports; (6) participation in
extracurricular activities; and (7) using the
internet for schoolwork
a= .68 23.38 4.53
Political attitudes Student’s self-description of political attitudes,
coded 1 = far left; 2 = liberal; 3 = middle-of-the-
road; 4 = conservative; 5 = far right
2.74 0.83
a
Descriptive statistics for the independent and control variables are based on the largest analytic sample (n= 708). Any statistically significant differences between this
sample and those of other dependent variables are noted.
*p\.05
120 Res High Educ (2008) 49:107–125
123
liberal arts experiences on liberal arts outcomes ranged from .13 to .43 of a standard
deviation, as measured by b.
One can consider the liberal arts outcomes in terms of their cognitive and affective
orientation, although one may argue that each of these outcomes has multiple components.
The more cognitively-oriented liberal arts outcomes include moral reasoning, effective
reasoning and problem solving, and the inclination to inquire and lifelong learning.
Intercultural effectiveness, well-being, and leadership reflect affective outcomes. Among
Table 3 Estimated weighted standardized regression coefficients for the effects of liberal arts experiences
on liberal arts outcomes
a
General instruments, scales, or measures for
liberal arts outcomes
Sub-scales or measures R
2
change bSig.
Moral reasoning
Defining issues test (DIT-2) of moral
reasoning
Postconventional score 0.000 –0.01 NS
Effective reasoning and problem solving
Reasoning and current issues test of
reflective judgment
Reflective judgment score 0.000 –0.02 NS
Intercultural effectiveness
Intercultural development inventory (IDI)
b
Developmental score 0.012 0.138 **
Openness to diversity/challenge
b
Overall score 0.143 0.428 **
Inclination to inquire and lifelong learning
Need for cognition Overall score 0.038 0.235 **
Positive attitude toward literacy Overall score 0.052 0.258 **
Well-being
Scales of psychological well-being Autonomy 0.017 0.148 *
Positive relationships with others 0.025 0.180 **
Environmental mastery 0.013 0.130 *
Personal growth 0.044 0.239 **
Life purpose 0.042 0.235 **
Self-acceptance 0.029 0.194 **
Leadership
Socially responsible leadership scale (SRLS) Consciousness of self 0.046 0.261 **
Congruence 0.030 0.209 **
Commitment 0.064 0.306 **
Collaboration 0.067 0.315 **
Common purpose 0.053 0.280 **
Controversy with civility 0.071 0.324 **
Citizenship 0.101 0.386 **
Change 0.066 0.313 **
a
All analyses control for students’ age, race, gender, parents’ education and household income, if student
was financially dependent on parents, high school GPA, a scaled measure of high school involvement,
precollege academic ability, educational aspirations, the racial composition of high school, plus a series of
dummy variables representing institution attended
b
Include all controls in ‘‘a’’ plus political attitude, as political attitude may influence students’ intercultural
maturity
*p\.05; ** p\.01
Res High Educ (2008) 49:107–125 121
123
the cognitively-oriented liberal arts outcomes, we found that liberal arts experiences had no
significant effect on our measures of moral reasoning or effective reasoning and problem
solving. Specifically, the liberal arts experience scale was not related to students’ post-
conventional moral reasoning as measured by the DIT-2 or reflective judgment as
measured by the RCI. On the other hand, we found significant positive effects of liberal
arts experiences on both measures of the inclination to inquire and lifelong learning. Net of
confounding influences, students’ liberal arts experiences affected their need for cognition
and positive attitude toward literacy by .24 and .26 SD, respectively.
Students’ liberal arts experiences consistently predicted all of the affective liberal arts
outcomes. We found positive effects of students’ liberal arts experiences on both measures
of intercultural effectiveness, but to differing degrees. Controlling for all other factors,
students’ liberal arts experiences had a positive influence on students’ openness to
diversity/challenge by .43 SD where the effect of liberal arts experiences on the devel-
opmental score of the Intercultural Development Inventory was .14 SD. Students’ liberal
arts experiences had a positive effect on all of the dimensions of psychological well-being,
with the effects having the greatest magnitude for the personal growth (.24 SD) and life
purpose (.24 SD) scales. Finally, we found positive effects of liberal arts experiences on the
eight scales of socially responsible leadership (Tyree 1998). The liberal arts experiences
variable had the strongest relationship to the scales for citizenship and civility with
increases of .39 and .32 of a standard deviation, respectively, controlling for student
background characteristics and institution attended.
Discussion and Implications
Pascarella et al. (2005) found mere attendance at a liberal arts college did not consistently
influence student learning and development and where attendance did influence student
learning, the effect was not always positive. In contrast, they found a consistent, positive
relationship between students’ liberal arts experiences and several measures of student
learning. In the current study, we further tested the construct and predictive validity of the
liberal arts experiences variable by examining its relationship with outcomes theoretically
and conceptually-associated with liberal arts education. Given that we found significant
positive relationships between four of the six liberal arts outcomes (consisting of 18 out of
20 separate measures), our results suggest the liberal arts experience variable is a valid
construct in measuring liberal arts education as defined by Blaich et al. (2004).
It is interesting that the liberal arts experience variable failed to predict either post-
conventional moral reasoning or reflective judgment. We posit several reasons for our non-
significant findings. First, it is possible that fostering moral reasoning in college students
requires different liberal arts experiences and practices than those included in the current
operationalization of the liberal arts experiences variable. Second, with regard to the
reflective judgment measure from the Reasoning and Current Issues test, its low reliability
made finding any statistically significant relationships difficult. The low reliability
(a= .41) could have been an artifact of the data collection conditions or this particular
sample.
4
For centuries, advocates of liberal arts education have asserted that it is the best means
of education. The current study provided empirical evidence for the benefits of liberal arts
4
Internal consistency estimates for the RCI score are .61 for freshmen and .67 for seniors (Kitchener et al.
in preparation).
122 Res High Educ (2008) 49:107–125
123
education and an operational model for measuring it. Based on our findings, the practices
and conditions embodied in the liberal arts experiences variable are indeed those that
promoted the development of students’ intercultural effectiveness, inclination to inquire
and learn for a lifetime, psychological well-being, and leadership. Consequently, colleges
and universities are more likely to foster the development of these liberal arts outcomes in
students by creating an ethos that (a) values the intellectual arts rather than professional or
vocational skills; (b) integrates curricular and environmental structures coherently; and (c)
cultivates a culture that values student–student and student–faculty interactions both in and
out of the classroom (Blaich et al. 2004).
It is rare for a student to experience aspects of the campus environment like interaction
with faculty and peers or class challenge in isolated segments. These dimensions of the
environment overlap and blend together. We suggest this holistic ‘‘overlap and blending’’
is a key feature of the array of experiences, practices, and conditions that characterizes
liberal arts education. Conceptually speaking, what sets the liberal arts experience variable
apart from other good practices (Chickering and Gamson 1987,1991) and student
engagement benchmarks (Kuh 2001,2003) is that it attempts to capture the holistic and
seamless nature of this learning environment within a single scale.
We believe that our results are good news for colleges and universities. Despite the
preliminary nature of the findings, the connection between liberal arts experiences and
liberal arts outcomes is noteworthy because although it may be virtually impossible for a
college or university to change its institutional type, any institution can implement the
practices that foster rich and integrated learning environments. Like Pascarella and col-
leagues (2005), we found students’ liberal arts experiences influenced learning outcome
development, net of the institution attended. Our findings demonstrate that liberal arts
experiences and outcomes are not the exclusive domain of small liberal arts colleges.
Consistent with previous research (Astin 1993; Chickering and Gamson 1987,1991; Kuh
et al. 2005, Pascarella and Terenzini 1991,2005), our results lend further support to the
evidence that an institutional focus on good teaching and student engagement in an active,
collaborative, and supportive environment positively affects student learning and devel-
opment. Our evidence suggests that any institution, possessing the organizational will to
place student learning at the center, can create a culture that maximizes liberal arts
experiences and thus, the development of liberal arts outcomes for all students.
This study may be limited in several important ways. First, it may be limited due to our
operationalization of the liberal arts experience variable. It is possible that different op-
erationalizations of the independent measure would have yielded different findings. This is
particularly possible given the non-significant findings with regard to the measure of
postconventional moral reasoning. Additionally, the external validity of this study may be
limited in that our sample represented college students from four institutions. Although we
made an effort to diversify our institutional sample by Carnegie classification, selectivity,
and region of the country, the institutions in this study likely do not represent the vastness
of the U.S. higher education landscape. Finally, the cross-sectional nature of the findings
from the pilot phase of the Wabash National Study of Liberal Arts Education may limit the
study’s interpretation. Students, who report higher levels of liberal arts experiences, may
also be those who are predisposed to higher scores on the liberal arts outcome measures
before entering college (Pascarella 2006). Without a pretest, we are not able to take into
account this confounded predisposition. We hope to have a more robust test of the liberal
arts experiences variable and its effects on liberal arts outcomes in the panel portion of the
Wabash National Study of Liberal Arts Education. We believe this next phase of research
will enhance the present findings because it will use a longitudinal pretest–posttest design,
Res High Educ (2008) 49:107–125 123
123
with a national sample that will follow 4,500 students from 19 institutions throughout their
college career.
Acknowledgment This research was supported by a grant from the Center of Inquiry in the Liberal Arts at
Wabash College.
References
American Association for Liberal Education (2003). The academy’s education standards. Retrieved April
19, 2005 from http://www.aale.org/highered/edstand.htm.
Aristotle (350 B.C.E.). Politics, book eight. Translated by Benjamin Jowett. Retrieved September 24, 2007
from http://classics.mit.edu/Aristotle/politics.8.eight.html.
Association of American Colleges & Universities (1998, October). Statement on liberal learning. Retrieved
November 5, 2003, from http://www.aacu-edu.org/About/liberal_learning.cfm.
Association of American Colleges and Universities (AAC&U) (2002). Greater expectations: A new vision
for learning as a nation goes to college. Retrieved April 29, 2006 from http://www.greater
expectations.org/.
Astin A. (1993). What matters in college? San Francisco: Jossey-Bass.
Astin, A. W. (2000). How the liberal arts college affects students. In S. Koblik & S. Graubard (Eds.),
Distinctly American: The residential liberal arts colleges. New Brunswick: Transaction Publishers.
Blaich, C., Bost, A., Chan, E., & Lynch, R. (2004). Defining liberal arts education. Unpublished manuscript.
Retrieved September 24, 2007 from http://www.wabash.edu/cila/docs/DefLibArtEdFinal.pdf.
Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of
Personality Assessment, 48(3), 306–307.
Center of Inquiry in the Liberal Arts at Wabash College (2006). Wabash National Study of Liberal Arts
Education. Retrieved on April 29, 2006 from http://www.liberalarts.wabash.edu/nationalstudy.
Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education.
AAHE Bulletin, 39(7), 3–7.
Chickering, A. W., & Gamson, Z. F. (1991). Applying the seven principles for good practice in higher
education. New Directions for Teaching and Learning, 1991(47), 1–104.
Chickering, A. W., & Gamson, Z. F. (1999). Development and adaptations of the seven principles for good
practice in undergraduate education. New Directions for Teaching and Learning, 1999(80), 75–81.
Chickering, A., & Reisser, L. (1993). Education and identity (2nd ed.). San Francisco: Jossey-Bass.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis
for the behavioral sciences (3rd ed.). Mahwah: Erlbaum.
Cruce, T., Wolniak, G. C., Seifert, T. A., Pascarella, E. T., & Blaich, C. (2006). Impacts of good practices on
cognitive development, learning orientations, and graduate degree plans during the first year of college.
Journal of College Student Development, 47(4), 365–383.
Farrell, E. F., & Van Der Werf, M. (2007). Playing the rankings game. The Chronicle of Higher Education,
53(38), A11.
Hammer, M. R., & Bennett, M. J. (2001). The intercultural development inventory (IDI) manual. Portland:
Intercultural Communication Institute.
Higher Education Research Institute (1996). Social change model of leadership development.LosAngeles:UCLA.
King, P., Kendall Brown, M., Lindsay, N., & VanHecke, J. (in press). Liberal arts student learning out-
comes: An integrated perspective. About Campus.
Kitchener, K. S, Wood, P. K., & Jensen, L. (in preparation). Individual differences in gains in reflective
judgment and their relationship to college experiences.
Kuh, G. D. (1996). Guiding principles for creating seamless learning environments for undergraduates.
Journal of College Student Development, 37, 135–148.
Kuh, G. D. (2001). Assessing what really matters to student learning: Inside the National Study of Student
Engagement. Change, 33(3), 10–17, 66.
Kuh, G. D. (2003). What we’re learning about student engagement from NSSE. Change, 35(2), 24–32.
Kuh, G. D., Kinzie, J., Schuh, J. H., Whitt, E. J., & Associates (2005). Student success in college: Creating
conditions that matter. San Francisco: Jossey-Bass.
Kuh, G. D., Schuh, J. H., Whitt, E. J., & Associates (1991). Involving colleges: Successful approaches to
fostering student learning and personal development. San Francisco: Jossey-Bass.
Matthews, A. (1997). Bright college years: Inside the American college campus today. New York: Simon &
Schuster.
124 Res High Educ (2008) 49:107–125
123
Nussbaum, M. C. (1997). Cultivating humanity: A classical defense of reform in liberal education. Cam-
bridge: Harvard University Press.
Pascarella, E. T. (2006). How college affects students: Ten directions for future research. Journal of College
Student Development, 47(5), 506–520.
Pascarella, E., & Terenzini, P. (1991). How college affects students. San Francisco: Jossey-Bass.
Pascarella, E., & Terenzini, P. (2005). How college affects students: Vol. 2. A third decade of research. San
Francisco: Jossey-Bass.
Pascarella, E., Wolniak, G., Cruce, T., & Blaich, C. (2004). Do liberal arts colleges really foster good
practices in undergraduate education? Journal of College Student Development, 45(1), 57–74.
Pascarella, E., Wolniak, G., Seifert, T., Cruce, T., & Blaich, C. (2005). Liberal arts colleges and liberal arts
education: New evidence on impacts. ASHE Higher Education Report, 31(3).
Rest, J., Narvaez, D., Thoma, S. J., & Bebeau, M. J. (1999). DIT2: Devising and testing a new instrument of
moral judgment. Journal of Educational Psychology, 91(4), 644–659.
Ryff, C. D. (1989). Happiness is everything, or is it? Explorations on the meaning of psychological well-
being. Journal of Personality and Social Psychology, 57, 1069–1081.
Ryff, C. D., & Keyes, C. L. M. (1995). The structure of psychological well-being visited. Journal of
Personality and Social Psychology, 69, 719–727.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi experimental designs for
generalized causal inference. Boston: Houghton Mifflin.
Terenzini, P. T., Pascarella, E. T., & Blimling, G. S. (1996). Students’ out-of-class experiences and their
influence on learning and cognitive development: A literature review. Journal of College Student
Development, 37(2), 149–162.
Turner F. M. (Ed.) (1996). The idea of the university, John Henry Newman. New Haven: Yale University
Press.
Tyree, T. M. (1998). Designing an instrument to measure the socially responsible leadership using the social
change model of leadership development. Dissertation Abstracts International, 59(06), 1945.
U.S. Department of Education (2006). A test of leadership: Charting the future of U.S. higher education.
Retrieved May 21, 2007, from http://www.ed.gov/about/bdscomm/list/hiedfuture/reports/final-report.pdf
.
Wood, P. K., Kitchener, K. S., & Jensen, L. (2002). Considerations in the design, evaluation of a paper-and-
pencil measure of reflective thinking. In B. Hofer & P. Pintrich (Eds.), Personal epistemology: The
psychology of beliefs about knowledge and knowing. Mahwah: Lawrence Erlbaum Associates.
Res High Educ (2008) 49:107–125 125
123
