Content uploaded by Viviane M. J. Robinson
Author content
All content in this area was uploaded by Viviane M. J. Robinson on Oct 29, 2014
Content may be subject to copyright.
Content uploaded by Viviane M. J. Robinson
Author content
All content in this area was uploaded by Viviane M. J. Robinson
Content may be subject to copyright.
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
The Impact of Leadership on Student
Outcomes: An Analysis of the Differential
Effects of Leadership Types
Viviane M. J. Robinson Claire A. Lloyd Kenneth J.
Rowe
Authors’ Note: This article was completed with the financial support
of the Iterative Best Evidence Synthesis program of the New
Zealand Ministry of Education (http://education-
counts.edcentre.govt.nz/goto/BES?). An earlier version of this
article was presented in April 2007 at the annual meeting of the
American Educational Research Association in Chicago (Division A
Symposium: Developing a Knowledge Base for the Leadership of
Teaching and Learning). The authors thank John Hattie for his
statistical advice. Address correspondence to Viviane Robinson at
vmj.robinson@auckland.ac.nz.
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
The Impact of Leadership on Student
Outcomes: An Analysis of the Differential
Effects of Leadership Types
Abstract
Purpose: The purpose of this study was to examine the relative
impact of different types of leadership on students’ academic and
nonacademic outcomes.
Research Design: The methodology involved an analysis of findings
from 27 published studies of the relationship between leadership and
student outcomes. The first meta- analysis, including 22 of the 27
studies, involved a comparison of the effects of transfor- mational and
instructional leadership on student outcomes. The second meta-
analysis involved a comparison of the effects of five inductively
derived sets of leadership prac- tices on student outcomes. Twelve of
the studies contributed to this second analysis.
Findings: The first meta-analysis indicated that the average effect of
instructional lead- ership on student outcomes was three to four times
that of transformational leadership. Inspection of the survey items
used to measure school leadership revealed five sets of leadership
practices or dimensions: establishing goals and expectations;
resourcing strategically; planning, coordinating, and evaluating
teaching and the curriculum; pro- moting and participating in teacher
learning and development, and ensuring an orderly and supportive
environment. The second meta-analysis revealed strong average
effects for the leadership dimension involving promoting and
participating in teacher learning and development and moderate
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
effects for the dimensions concerned with goal setting and planning,
coordinating, and evaluating teaching and the curriculum.
Conclusions and Implications for Research and Practice: The
comparisons between transformational and instructional leadership
and between the five leadership dimen- sions suggested that the
more leaders focus their relationships, their work, and their learning
on the core business of teaching and learning, the greater their
influence on student outcomes. The article concludes with a
discussion of the need for leadership research and practice to be
more closely linked to the evidence on effective teaching and
effective teacher learning. Such alignment could increase the
impact of school leadership on student outcomes even further.
Keywords: leadership; principal; leadership theory;
achievement; outcomes; meta-analysis
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
There is unprecedented international interest in the
question of how educational leaders influence a range of
student outcomes. In conse- quence, at least five reviews of
empirical research on the direct and indirect effects of leadership
on student outcomes have appeared recently (Bell, Bolam, &
Cubillo, 2003; Leithwood, Day, Sammons, Harris, & Hopkins,
2006; Leithwood, Seashore Louis, Anderson, & Wahlstrom, 2004;
Marzano,
Waters, & McNulty, 2005; Witziers, Bosker, & Krüger, 2003).
A major reason for the interest in the links between leadership
and student outcomes is the desire of policy makers in many
jurisdictions to reduce the persistent disparities in educational
achievement between various social and ethnic groups, and their
belief that school leaders play a vital role in doing so (Organisation
for Economic Co-operation & Development, 2001). The con-
fidence of the public and politicians in the capacity of school
leaders to make a considerable difference to student outcomes is
supported by qualitative research on the impact of leadership on
school effectiveness and improve- ment. Case studies of “turn
around” schools and of interventions into teaching and
learning invariably credit school and district leadership with
considerable responsibility for school and teaching effectiveness
(Edmonds, 1979; Maden, 2001; Scheurich, 1998). The literature
on sustainability also sees the quality of school leadership as a
key to continued organizational learning and improvement
(Datnow, 2005; Hargreaves & Fink, 2006).
However, the picture one gains from the qualitative evidence
for the impact of leadership is very different from that gained
from quantitative analyses of the direct and indirect effects of
leadership on students’ acade- mic and social outcomes. In a
meta-analysis of 37 multinational studies of the direct effects of
leadership on student outcomes, Witziers reports an average
effect (reported as a z score) of 0.02, an estimate that is
typically interpreted as indicating no or a very weak impact
(Witziers et al., 2003).
Most subsequent quantitative research has conceptualized the
relation- ship between leadership and student outcomes as
indirect, with leaders establishing the conditions (e.g., provision
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
of teacher professional learning opportunities, forms of student
grouping) through which teachers make a more direct impact on
students. In the only published meta-analysis of such research,
Marzano reports an average effect of approximately 0.4 between
leadership and student academic outcomes (Marzano et al.,
2005).
1
There are several possible reasons why the estimate from the
Marzano meta-analysis is considerably greater than that of
Witziers. First, the latter analysis included both direct and indirect
effects of leadership and because leadership effects are typically
modeled as indirect, the Marzano studies were more likely to
capture how leaders make a difference. Second, the Marzano
work included only U.S. studies and the Witziers studies were
multinational. Because the impacts of leadership are typically
found to be stronger in the United States than in international
studies, these contrasting research sampling strategies could
explain some of the difference. Finally, 60 of the 70 studies
included in the Marzano meta-analysis were unpub- lished U.S.
theses and dissertations that have not been subject to the same
peer review processes as published work.
The typical conclusion drawn by quantitative leadership
researchers is that school leaders have small and indirect effects
on student outcomes that are essentially mediated by teachers
(Hallinger & Heck, 1998).
Thus, there seems to be a contradiction between the evidence
that leaders have a weak indirect effect on student outcomes and
the expectations of the public and policy makers that leaders
make a substantial difference. What explains this paradox? Do
public expectations reflect attribution bias and a romantic view
of leadership (Meindl, 1998)? Do quantitative researchers
systematically underestimate the impact of leadership through
research designs and assessment tools that miss the ways in
which particular prac- tices of particular leaders are powerful? Is
it possible that both views are partially correct?
The purpose of this article is to address the paradoxical
differences between the qualitative and quantitative evidence on
leadership impacts by taking a fresh approach to the analysis of
the quantitative evidence. Rather than conduct a further meta-
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
analysis of the overall impact of leadership on student outcomes,
we focus on identifying the relative impact of different types of
leadership. By focusing on types of leadership, rather than on
lead- ership as a unitary construct, we are recognizing that
leaders’ impact on student outcomes will depend on the particular
leadership practices in which they engage. If empirical research
indicates that some leadership prac- tices have stronger
impacts on student outcomes than others, then both researchers
and practitioners can move beyond a general focus on the impact of
leadership, to examining and increasing the frequency and
distribution of those practices that make larger positive
differences to student outcomes.
Two quite different strategies were used to identify types of
leadership and their impact. The first strategy involved a
comparison between the impact of transformational and
instructional leadership. These two leader- ship theories were
chosen because they dominate empirical research on
educational leadership and their research programs are mature
enough to have yielded sufficient evidence for analysis. Although
there have been sev- eral reviews published that include
discussions of the evidence about the impact on students of these
two types of leadership, those reviews have not quantified the
impact, and thus it has been difficult to compare them sys-
tematically against this criterion (Hallinger, 2005; Hallinger &
Heck, 1998; Leithwood & Jantzi, 2005; Leithwood, Tomlinson, &
Genge, 1996).
The second strategy for identifying types of leadership
involved a more inductive approach based on a detailed analysis
of the meaning of items included in the measures of leadership
used in studies of the leadership- outcome relationship. All
survey items, regardless of the underpinning leadership theory,
were listed and grouped to reflect common sets of leader- ship
practices. Five groupings or leadership dimensions emerged and
their relationship with student outcomes calculated.
We turn now to a brief discussion of the literature on
instructional and transformational leadership.
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
Instructional Leadership
Instructional leadership theory has its empirical origins in
studies under- taken during the late 1970’s and early 80’s of
schools in poor urban com- munities where students succeeded
despite the odds (Edmonds, 1979). As reported by Bossert,
Dwyer, Rowan, and Lee (1982), these schools typi- cally had
strong instructional leadership, including a learning climate free
of disruption, a system of clear teaching objectives, and high
teacher expec- tations for students.
Early formulations of instructional leadership assumed it to be
the respon- sibility of the principal. Hence, measures of such
leadership, such as the Principals’ Instructional Management
Rating Scale (PIMRS) (Hallinger & Murphy, 1985), focused only
on the principal and neglected the contribution of other staff to
instructional goal setting, oversight of the teaching programs, and
the development of a positive academic and learning culture. The
exclu- sive focus on the principal reinforced a heroic view of the
role that few were able to attain (Hallinger, 2005). Recent
research has a more inclusive focus with many instructional
leadership measures now embracing principals and their
designees (Heck, 1992; Heck, Larsen, & Marcoulides, 1990;
Heck, Marcoulides, & Lang, 1991), those in positions of
responsibility (Heck, 2000; Heck & Marcoulides, 1996), and
shared instructional leadership (Marks & Printy, 2003).
The most recent review of the impact of instructional
leadership on student outcomes concluded as follows: “The
size of the effects that principals indirectly contribute toward
student learning, though statistically significant is also quite
small” (Hallinger, 2005, p. 229). This conclusion was reached
as part of a literature review and discussion of research on
instructional leadership rather than as a result of the calculation of
the effect size statistic for each relevant study.
Transformational Leadership
Transformational leadership has its origins in James McGregor
Burns’s 1978 publication in which he analyzed the ability of
some leaders, across many types of organizations, to engage with
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
staff in ways that inspired them to new levels of energy,
commitment, and moral purpose (Burns, 1978). It was argued
that this energy and commitment to a common vision trans-
formed the organization by developing its capacity to work
collaboratively to overcome challenges and reach ambitious
goals.
Burns’s theory was extended further by Bass and colleagues
who devel- oped survey instruments to assess transformational
leadership (Bass & Avolio, 1994). Variations of these instruments
have been used in many pub- lished empirical studies of
transformational leadership in education, though few have
investigated the impact of such leadership on students’ academic
or social outcomes. Of 33 studies reviewed by Leithwood and
Jantzi (2005), about half were judged to show that
transformational leadership had a small indirect influence on
academic or social student outcomes. But this review did not
involve calculation of effect size statistics.
METHOD
The overarching methodology within which this study can be
located is that of meta-analysis. Meta-analysis is an empirical,
knowledge-building strategy that enables the results of
quantitative studies of the relationship between two constructs to
be aggregated so that an estimate of the average magnitude of the
impact of one on the other can be derived (Glass, McGaw, &
Smith, 1981; Hedges & Olkin, 1985; Lipsey & Wilson, 2001). In
meta-analyses, comparison of findings derived from different
analytic and statistical techniques is made possible by their
conversion to a common metric in the form of an effect size
statistic, usually expressed as standard deviation or a z score.
Although there are many different forms of effect size statistics,
it can be defined as a standardized measure of the magnitude of an
effect (Field, 2005).
The advantage of a meta-analysis over a qualitative literature
review is that it requires systematic treatment of relevant
studies and produces a measure of overall impact of the
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
construct of interest. It does not preclude, however, the need for
careful qualitative analysis of the relevant literature as the theory
and design of the constituent studies, and knowledge of relevant
contextual factors must be brought to bear on the
interpretation of the individual and overall effect size
statistics. These interpretive considera- tions were of particular
importance in the present meta-analysis as the constituent
studies used varying designs, theoretical approaches, and mea-
surement tools.
One of the most frequent criticisms of meta-analysis is
inappropriate aggregation across studies employing very different
theoretical or method- ological approaches (Lipsey & Wilson,
2001). Increasingly, meta-analysts are responding to this
criticism by conducting comparisons between sub- sets of
studies rather than aggregating across studies, which take very
dif- ferent approaches to the relationship in question. We have
taken this comparative approach by analyzing the impact of
different types of leader- ship instead of producing an estimate of
the impact of an undifferentiated overall leadership construct.
Search Strategies
The synthesis began with a search of the international literature
for pub- lications in English that empirically examined the
links between school leadership and academic or nonacademic
student outcomes. Thus, any study that examined relationships
between empirical measures of leader- ship (however theorized)
and measures of student outcomes was included. An inclusive
approach was taken to the concept of leadership, with super-
intendent, principal, teacher, and total school-based leadership
admissible. The first search strategy involved examining
electronic databases using a combination of keywords around
leadership (leaders, principal, teacher leadership) and student
outcomes (achievement, achievement gains, social outcomes).
The second strategy involved hand or electronic searches of the
tables of contents and abstracts of educational leadership
journals. The third search strategy involved careful screening
of the reference lists of relevant articles, technical reports, and
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
chapters in international journals and handbooks to identify any
further relevant studies.
Two types of potentially relevant studies were excluded.
Unpublished theses and conference papers were omitted
because they had not been subject to peer review processes.
Furthermore, some apparently relevant studies were excluded
because the same data sets were used in multiple publications.
The search yielded 27 studies, published between 1978 and
2006, that provided evidence about the links between leadership
and student outcomes (Table 1). The majority of studies in Table
1 (18 of 27) were conducted in
U.S. schools. Two studies were conducted in Canada and one in
each of Australia, England, Hong Kong, Israel, the Netherlands,
New Zealand, and Singapore.
Sixteen studies examined leadership in elementary school
contexts, four in high schools, and seven studies included a mix
of elementary, middle, and high schools. Fifteen of the 27 studies
confined their analysis of school leadership to the principal only,
whereas twelve took a broader, more dis- tributed view of
leadership.
Although these studies have examined the impact of
leadership on a wide range of student outcomes, academic
outcomes (mathematics, reading and language) predominated.
Twenty-two studies examined only academic outcomes, four
studies included only social and attitudinal outcomes, and one
study included both types of outcome. Without close
inspection of assessment items in the various standardized tests
used, it is difficult to evaluate the intellectual depth of the skills
and knowledge being assessed. Critical thinking, intellectual
challenge, and problem solving were features of at least some of
the assessments. The four studies examining leadership impact on
students’ social and personal well-being included measures of
students’ attitudes to school, academic self-concept, and
engagement with and participation in schooling.
The thoroughness of this search can be assessed by
comparing it with the number of studies included in two
recent literature reviews on the impact of leadership on
student outcomes. A synthesis by the London Institute of
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
Education found only eight studies (Bell et al., 2003), whereas
the meta-analysis of Marzano et al. (2005) located 70 studies, 60
of which were unpublished theses or conference papers. In short,
both these efforts yielded fewer than a dozen publications. A
meta-analysis reported in 2003 on the direct effects of leadership
on students included 15 published studies (Witziers et al., 2003).
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types. Educational
Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1
Individual Studies of the Effects of Leadership on Student Outcomes
Reference
Schools
Leadership
Theory
Leadership
Measure
Who Is Leader?
Measure of
Student Outcomes
Magnitude of Effects
Alig-Mielcarek &
Representative
Instructional
Survey of teacher
Principal only
School average
ES for math = 0.32
Hoy, 2005; USA
sample of 146
leadership
perceptions of
scores over 2
ES for reading =
elementary
instructional
years in 4th
0.16
schools
leadership
grade reading
and math (Ohio
proficiency
exams)
Andrews & Soder,
33 elementary
Instructional
18-item
Principal only
Gains over 2 years
Gains in schools with
1987; USA
schools
leadership
instructional
in individual
strong instructional
leadership
normal curve
leadership 2-3
survey
equivalent
times larger than
scores on CAT
under weak
in reading and
instructional
math
leadership
*Bamburg &
Andrews 1991;
USA
*Brewer, 1993;
USA
10otherwise
comparable
high-achieving
and 10 low-
achieving
elementary
schools
Representative
national sample
of 1,100 high
schools
Instructional
leadership
Instructional
leadership
19strategic
interactions of
principal
assessed by
teachers
a
Administrator and
teacher survey,
plus principal
ranking of acad-
emic excellence
Principal only
Principal only
Gain scores on
CAT in math
only
Gain scores over a
2-year period on
test of verbal and
quantitative ability
Mean ES for
math = 1.01
(n = 19)
Mean ES for
ability = 0.42
(n = 7)
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Leadership
Leadership
Measure of
Reference
Schools
Theory
Measure
Who Is Leader?
Student Outcomes
Magnitude of Effects
Cheng, 1994;
Hong Kong
*Eberts & Stone,
1986; USA
*Friedkin, &
Slater, 1994;
USA
Sample of 164
elementary
schools
Nationally
representative
sample of
approximately
300 elementary
schools
20California
elementary
schools
Four leadership
frames of
Bolman & Deal
(1991)
Instructional
leadership
Social network
theory
30-item teacher
survey
comprising four
generic
leadership
frames and one
additional
educational
leadership
dimension
Teacher and
principal survey
Teacher survey of
persons in school
(a) with whom
issues are
discussed, (b)
from whom
advice is sought,
and (c) who are
close personal
friends
Principal only
Principal only
Principal and
teachers can be
included in
network
Student survey
about
self-concept
and attitudes to
school, teachers,
and learning
Pre-post test scores
on standardized
math test
4-year average of
school perfor-
mance on math,
reading, and
language on
CAP adjusted
for socio
economic status
Mean ES for
affective
outcomes = 0.27
(n = 35)
Mean ES for
math = 0.14
(n = 8)
Mean ES for
combined
achievement =
0.44
(n = 6)
(continued)
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Leadership
Leadership
Measure of
Reference
Schools
Theory
Measure
Who Is Leader?
Student Outcomes
Magnitude of Effects
Goldring &
Pasternak, 1994;
Israel
34elementary
schools
Principal's control
and coordination
of the teaching
program
Principal's allocation
of time to set tasks,
degree of
influence over
teaching, and
importance of
certain goals;
teacher reports of
degree of goal
consensus
Principal only
5th grade math and
reading scores and
6th grade reading
Standardized
discriminant
coefficients showed
that principals' task
emphasis on involving
parents (0.42) and
implementing
innovations (-0.51)
discriminated between
more and less
effective schools;
principals' goal
emphasis on personal
growth and potential
(+ve), moral and social
values (-ve)
discriminated more
and less effective
schools; and staff
agreement about
educational goals was
strongest discriminator
(+ve)
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Leadership
Leadership
Measure of
Reference
Schools
Theory
Measure
Who Is Leader?
Student Outcomes
Magnitude of Effects
Griffith, 2004;
USA
Hallinger,
Bickman, &
Davis, 1996;
USA
117 urban elementary
schools
87Tennessee
elementary schools
participating in a
state program
Transformational
leadership
Instructional
leadership
3 domains of
transformational
leadership:
charisma,
individualized
consideration, and
intellectual
stimulation
18 items on
instructional
leadership as part
of Connecticut
School
Effectiveness
Questionnaire
Principal only
Principal only
(a) Individual level
analysis: student
report of grade
levels achieved
converted to GPA;
(b) school level
analysis: residual
standardized test
scores
Gain scores on 3rd
and 6th grade
reading tests
(BSFT)
ES for school
grades = 0.68
ES for
reading = 0.22
*Heck, 1992; USA
23 high-achieving
Instructional
Teacher survey of
Principal or
CAP scores
Primary schools:
elementary, 17
leadership
three domains of
designee
Mean ES for
high-achieving
high schools
instructional
leadership
achievement = 1.1
(n = 8)
High schools:
Mean ES for
achievement =
0.42 (n = 8)
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Leadership
Leadership
Measure of
Reference
Schools
Theory
Measure
Who Is Leader?
Student Outcomes
Magnitude of Effects
Heck, 2000; USA
(Hawaii)
*Heck, Larsen, &
Marcoulides,
1990; USA
*Heck &
Marcoulides,
1996; Singapore
122 elementary
schools
comprising all
eligible schools
in Hawaii
30otherwise
comparable high-
and low-
achieving
elementary and
high schools
A convenience
sample of 26
high schools
Instructional
leadership
Instructional
leadership
Transformational
leadership
b
Teacher survey
includes
instructional
leadership
Teachers reported
on frequency of
implementation
of 22
instructional
leadership
behaviors
Leadership as part
of managerial
processes
including
resource
availability,
responsiveness to
teachers'
(unspecified)
problems, and
visionary and
collaborative
leadership
Principal plus
Principal or
designee
School
administrators
Total scaled scores
for reading,
language, and
math on SAT
CAP scores on
combined math
and reading (and
language in high
schools)
National test on a
variety of
curriculum areas
ES for combined
achievement =
0.41
ES for combined gains
= 0.37
Mean ES for
combined
achievement = 0.86
(n = 22)
Mean ES for combined
achievement =
–0.22 (n = 3)
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Leadership
Leadership
Measure of
Reference
Schools
Theory
Measure
Who Is Leader?
Student Outcomes
Magnitude of Effects
*Heck,
Marcoulides, &
Lang, 1991; USA
&
Marshall Islands
USA: 32
elementary and
high schools;
Marshall Islands: 3
elementary and 1
high school
Instructional
leadership
Teachers reported
on frequency of
implementation
of 22
instructional
leadership
behaviors
Principal or
designee
California: CAP
scores;
Marshall Islands:
national test scores
in reading and
math
California:
Mean ES for
combined
achievement = 0.51
(n = 22)
Marshall Islands:
Mean ES for
combined
achievement = 0.33
(n = 22)
*Hoy, Tarter, &
58 high schools
Leadership
(a) Principal
Principal only
Reading and math
Mean ES for
Bliss, 1990;
theorized as part
supportiveness
achievement,
combined
USA
of (a)
and directiveness
New Jersey
achievement =
organizational
(within OCDQ-
HSPT
0.42 (n = 7)
climate or (b) a
RS); (b)
Parsonian
principal
concept of
influence,
organizational
academic
health
emphasis,
consideration,
initiating
structure, and
resource support
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Reference
Schools
Leadership
Theory
Leadership
Measure
Who Is Leader?
Measure of
Student Outcomes
Magnitude of Effects
Leithwood & 94 elementary Transformational 53-item teacher Principal only for
Student
ES for identification =
Jantzi, 1999;
Canada
schools
and transactional
leadership
survey
transformational
leadership
identification 0.30
with and ES for participation =
participation in 0.20
school measured
by Student
Engagement and
Family
Educational
Culture Survey
Leithwood & 110 elementary and Transformational Teacher survey Principal and
Student
Principal
Jantzi, 2000;
Canada
high schools and transactional
leadership
teacher
leadership
separately
assessed
engagement with
school measured
by Student
Engagement and
Family
Educational
Culture Survey
transformational
leadership:
ES for participation =
0.08;
ES for identification = 0.16
Teacher leadership:
ES for participation =
0.20;
ES for identification =
-0.08
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Leadership
Leadership
Measure of
Reference
Schools
Theory
Measure
Who Is Leader?
Student Outcomes
Magnitude of Effects
Leithwood &
Jantzi, 2006;
UK
*Leitner, 1994;
USA
Marks & Printy,
2003; USA
256 elementary
schools for
literacy and 258
for numeracy
27urban
elementary
schools
24elementary,
middle, and high
schools
Transformational
leadership
Instructional
leadership
Integrated
leadership,
comprising high
transformational
and high shared
instructional
leadership
Teacher survey
tailored to
implementation
of literacy and
numeracy
strategies
Measured by
Hallinger’s
PIMRS
Indices of each
leadership type
derived from
items in teacher
survey and
coding of
interviews and
observations;
instructional
leadership
measure includes
degree of focus
Distributed to
“those in
positions of
responsibility in
your school”
Principal only
Transformational
leadership
mostly Principal
only;
instructional
leadership
measure
combined
teacher and
Principal
influence
Gain scores on
Key Stage 2
tests
Gain scores over
one year for
reading, math,
and language
Student
achievement on
math and social
studies
assignments
marked against
three standards
of intellectual
quality
Impact of
transformational
leadership on
student gains in
literacy and
numeracy is “not
significantly
different from
zero.”
Mean ES for
combined
achievement =
0.02 (n = 60)
ES for combined
achievement =
0.56
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Reference
*May &
Wagemaker,
1993; NZ
Ogawa & Hart,
1985; USA
Schools
175 primary
schools
124 elementary and
151 high schools
Leadership
Theory
Instructional
leadership
Leadership as
incumbent
Leadership
Measure
on and influence
over teaching,
curriculum, and
assessment
Principal’s
involvement in
evaluation and
development of
teachers with
respect to
reading
Change in
principalship
Who Is Leader?
Principal only
Principal only
Measure of
Student Outcomes
IEA (1990)
measure of
reading
achievement and
extent of
voluntary
reading
activities
Math and reading
scores on CAP
achievement test
over a 6-year
period
Magnitude of Effects
ES for reading =
0.12
Elementary schools:
From 6% to 8% of
variance in
achievement attributed
to principal after
controlling for year
and school effects
High schools: similar
effect for reading but
smaller (3%) for
math
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Reference
Schools
Leadership
Theory
Leadership
Measure
Who Is Leader?
Measure of
Student Outcomes
Magnitude of Effects
Pounder, Ogawa, 35 elementary and Leadership as an Amount of Principal only, (a) SAT adjusted Principal leadership:
& Adams, 1995;
USA
25 high schools
organizational
quality
influence
exercised by
people in four
school secretary,
single staff
member, and
school average ES for achievement =
over prior three -0.20
years;
different
collective group
(b) student
Silins & Mulford,
2002; Australia
Van de Grift &
Houtveen, 1999;
Netherlands
96 high schools
383 elementary
schools
completed
survey; 174
elementary
schools assessed
students
Transformational
leadership
Instructional
leadership
leadership roles
Survey of teacher
perceptions of
their principal's
transformational
leadership
Measured by
teacher survey of
instructional
leadership on 15-
item Rasch scale
of staff
c
Principal and
teacher
leadership
measured
separately
Principal only
absenteeism
(a) Student
participation in
school,
(b) student
engagement
with school, and
(c) academic self-
concept
Student
achievement on
180-item test of
language,
arithmetic, and
information
processing
ES for participation =
0.10
ES for engagement =
0.30
ES for self-concept =
0.16
Instructional
leadership has
small but
significant effect on
student
achievement
outcomes
(continued)
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi: 10.1177/0013161X08321509
TABLE 1 (Continued)
Reference
Schools
Leadership
Theory
Leadership
Measure
Who Is Leader?
Measure of
Student Outcomes
Magnitude of Effects
*Wellisch,
9 successful and 13
Instructional
Teachers reports of
Principal plus
Grades 3, 4, & 5 in
ES for combined
MacQueen,
nonsuccessful
leadership
Principal's
reading and
achievement =
Carriere, &
elementary
concern about
math over 2
0.55 (n = 6)
Duck, 1978;
schools based on
instruction,
years on CAT
USA
number of
coordination of
grades/subjects
instructional
showing
program, and
improvements in
feedback on
one year
teacher
performance
NOTE: An asterisk against authors' names indicates those studies that contributed to the analysis of the impact of leadership dimensions. BSFT = Basic
Skills First Test; CAP = California Assessment Project; CAT = California Achievement Test; ES = effect size; GPA = grade point average; HSPT = High
School Proficiency Test; IEA = International Association for the Evaluation of Educational Achievement; OCDQ-RS = Organizational Climate
Description Questionnaire-Rutgers Secondary; PIMRS = Principal Instructional Management Rating Scale; SAT = Stanford Achievement Test. The bold
entries in the column labeled “Magnitude of Effects” indicate those studies for which it was possible to report the relationship between leadership and
student outcomes as an effect size statistic. (+ve) means that there was a positive relationship between principal emphasis on the goal (personal growth
and potential) and student outcomes and (-ve)means that there was a negative relationship between principal emphasis on the goal (moral and social val-
ues) and student outcomes.
a. An additional 18 items measured other aspects of leadership. Only 6 of these were described in sufficient detail to be included in the dimensional analy-
sis.
b. Of the three leadership variables included in this study, only one was described in sufficient detail to contribute to the dimensional analysis.
c. Even though the impact of four different leadership roles are assessed, not all results are reported in a manner that enables calculation of an effect size
statistic.
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis
of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635-
674. doi: 10.1177/0013161X08321509
Analytic Strategies
Relevant information from the 27 studies identified was entered into a
spreadsheet under the following headings: sample characteristics (jurisdic-
tion, type, and number of schools, inclusion/exclusion criteria, sampling of
persons within schools, and sample attrition); leadership theory and instru-
mentation, including whose leadership was assessed; student outcomes and
assessment tools; contextual variables (student background, school commu-
nity context); indirect leadership effects (e.g., on school climate or teachers’
work); study design and analysis techniques (e.g., path analysis, multilevel
modeling, discriminant analysis, regression techniques); and main findings,
including the magnitude of direct and indirect effects of leadership on student
outcomes. In nearly every study, the design included some control for student
background effects, either through the use of gain scores or covariates.
It was possible to record or calculate an effect size statistic for 22 of 27
studies, as recorded in Table 1. Nonreporting of critical data or the impossi-
bility of statistical conversion to an effect size statistic (e.g., when results are
reported as percentage of variance explained) accounted for the noninclusion of
the remaining five studies in the meta-analysis. These studies are explicitly
considered in the subsequent discussion of the quantitative analyses.
Statistical measures of the relationship between types of leadership and
student outcomes were converted to z scores. This particular effect size
statistic was chosen as it is readily derived from the variety of statistics
employed in the original studies, including regression, path and correlation
coefficients, and a variety of t tests.
Some of the studies included in Table 1 embed leadership in a wider
model of how various organizational, cultural, and/or community variables
influence student and school performance (Heck, 2000; Heck &
Marcoulides, 1996; Hoy, Tarter, & Bliss, 1990; May & Wagemaker, 1993). In
these studies, the relevant data on direct and indirect leadership effects
(usually regression coefficients) were extracted from the path models and
converted to z scores.
The last column of Table 1 reports the magnitude of the effect of lead-
ership on student outcomes in each of the 22 studies included in the first
meta-analysis. More than one effect size statistic is listed for a single study if
the authors reported leadership-outcome relationships for different types of
leaders (Leithwood & Jantzi, 2000), multiple school types, (Heck, 1992),
different educational jurisdictions (Heck et al., 1991), or multiple outcomes
(Alig-Mielcarick & Hoy, 2005; Heck, 2000; Leithwood & Jantzi, 1999, 2000;
Silins & Mulford, 2002). For some studies a single effect size is reported
and for others a mean effect along with the number of contributing effect
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis
of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635-
674. doi: 10.1177/0013161X08321509
size statistics is reported. The inclusion of a mean effect size for a single
study indicates that we were able to calculate separate effect sizes for the
components of a composite leadership variable. These component effect
sizes were used in the second meta-analysis, in which we calculated the
relative impact of different dimensions of leadership.
The first research question about the relative impact of instructional and
transformational leadership was answered by categorizing each study
according to the theoretical framework that informed the conceptualization
and measurement of leadership. Fourteen studies employed an instructional
leadership framework, twelve of which could be included in the meta-
analysis. Six studies used a transformational leadership framework, five of
which could be included in the meta-analysis. The remaining seven studies
employed a variety of theories, which are noted in the third column of Table
1. Five of those studies reported statistics that could be included in the
meta-analysis. The average effect size for studies in each of the three cate-
gories was then calculated.
The second research question about the impact of different leadership
dimensions was addressed by using specific leadership practices rather than
broad leadership theories as the unit of analysis. By disaggregating compos-
ite leadership variables and calculating measures of impact for each leader-
ship component, we were able to estimate the impact of different types of
leadership practice on student outcomes. Twelve of the 22 studies included in
the first meta-analysis contributed to this second analysis. Those studies are
indicated with an asterisk before the author listing in Table 1. The remaining 10
studies either used unitary leadership constructs, or it was not possible to
calculate effect sizes for the components of the leadership variables.
A separate effect size for every leadership variable or construct for
which there were available data was calculated. For example, the instruc-
tional leadership studies of Heck and colleagues (Heck, 1992; Heck et al.,
1990; Heck et al., 1991) all employ a similar instructional leadership survey in
which teachers report the frequency with which their principal or other
school leaders engage in particular behaviors. It was possible to calculate a
separate effect size statistic for each item in these surveys. In other studies,
where data were reported against component leadership constructs rather
than actual survey items, it was also possible to calculate an effect size
statistic for each component construct.
The 199 component leadership survey items and constructs were
recorded verbatim in a spreadsheet, read repeatedly, and grouped together to
reflect broadly similar meanings. This inductive strategy contrasts with the
more deductive approach used in the study reported by Witziers et al.
(2003), in which the instructional leadership categories of the PIMRS were
used as a basis for categorization. Five categories or dimensions of leader-
ship practice were derived from the 199 listed survey items or constructs.
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student outcomes. An analysis
of the differential effects of leadership types. Educational Administration Quarterly, 44(5), 635-
674. doi: 10.1177/0013161X08321509
Each listed item was then coded against one of the dimensions and the mean
effect size and standard error for each leadership dimension calculated, as
presented in Table 2.
FINDINGS
The results of our comparison of transformational leadership and
instructional leadership are presented first, followed by the analysis of the
impact of particular leadership dimensions.
The Impact of Transformational, Instructional, and Other Types of
Leadership
Figure 1 presents the mean effect size estimates and standard errors for the
impact of transformational leadership (ES 0.11), instructional leader- ship
(ES 0.42), and other types of leadership (ES 0.30) on student out- comes.
The first point to note is the considerable difference in mean effect size
between the three leadership types. This confirms the utility of analyzing the
impact of types of leadership rather than of leadership in general. The second
point is that the mean effect size estimates for the impact of instruc- tional
leadership on student outcomes is three to four times greater than that of
transformational leadership.
Of the 11 transformational leadership effect size statistics reported in
Table 1, 10 fell in the range that we interpret as weak to small impact.
2
The
remaining study by Griffith (2004), which examined principal leadership in
117 U.S. elementary schools, showed that principals had a moderate to large
indirect effect on school-level residual test scores via their influence on staff
satisfaction. This is an interesting finding, given other transformational
leadership research indicating that although it has an effect on staff attitudes,
those effects do not usually follow through to student outcomes.
It is also worth noting that leadership effects are not always positive. The
mean estimate for transformational leadership was slightly reduced by the
results of two studies that found a weak to small negative effect of teacher
leadership on student identification (Leithwood & Jantzi, 2000) and a small
negative effect of school administrator leadership on student achievement
(Heck & Marcoulides, 1996).
There was less consistency in the reported impacts of instructional lead-
ership, with about half of the 16 effects in Table 1 indicating weak or small
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
Figure 1. Mean Effect Sizes for Impact of Transformational
Leadership (13 effects from 5 studies), Instructional
Leadership (188 effects from 12 studies), and Other
Leadership Approaches (50 effects from 5 studies) on
Student Outcomes
NOTE: Bars indicate mean-point estimates bounded by 1 standard error.
impacts and 8 moderate to large impacts. On the whole, the
large effect sizes were found in studies that involved
between-group designs or analyses. The comparison groups
comprised schools in which students performed consis-
tently better or worse than schools that served students
from similar social backgrounds (Bamburg & Andrews,
1991; Heck, 1992; Heck et al., 1990; Heck et al., 1991;
Mean Effect
Size estimate
0.45
0.40
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.00
Transformational
Leadership
Instructional
Leadership
Other
Leadership
Leadership Types
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
Wellisch, MacQueen, Carriere, & Duck, 1978). In general,
the comparisons showed that there are substantial
differences between the leadership of otherwise similar
high- and low-performing schools, and that those
differences matter for student academic outcomes. The
leadership in the higher performing schools was reported
by teachers to be, among other things, more focused on
teaching and learning, to be a stronger instructional
658 Educational Administration Quarterly
resource for teachers, and to be more active participants in
and leaders of teacher learning and development.
Despite the apparently strong difference in the impact
of transforma- tional and instructional leadership, cautious
interpretation is warranted. As already indicated, there is a
considerable range of effects for instructional leadership.
Furthermore, the outcome measures used in the
transformational leadership studies were predominantly of
social outcomes, whereas instruc- tional leadership
researchers tended to focus on academic ones. Two trans-
formational leadership studies, however, did employ
academic outcomes, and showed widely differing impacts of
transformational leadership (Griffith, 2004; Heck &
Marcoulides, 1996). In addition, Leithwood and Jantzi’s
(2006) study of the effect of transformational leadership on
student gains in literacy and numeracy in English
elementary schools is relevant, even though it could not
be included in the meta-analysis. The authors concluded that
transformational leadership explained very little of the
variance in students’ gains in literacy and numeracy.
Effect sizes for the five studies included in the “other”
category of lead- ership theory, ranged from -0.20 (Pounder,
Ogawa, & Adams, 1995) to 0.56 (Marks & Printy, 2003).
The latter study is particularly relevant as school leadership
was assessed on measures of both instructional and
transforma- tional leadership. The authors concluded that an
“integrated” form of leader- ship, incorporating a strong
capacity for developing shared instructional leadership
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
combined with qualities associated with transformational
leader- ship, was the best predictor of the intellectual
quality of student work in both math and social studies.
In summary, although caution is needed in
interpreting the evidence presented in Figure 1, it suggests
that the impact of instructional leadership on student
outcomes is notably greater than that of transformational
leader- ship. It is noted that in general, abstract leadership
theories provide poor guides to the specific leadership
practices that have greater impacts on student outcomes.
In the next section, we outline the findings of our
second analysis that was designed to understand the impact
of specific sets of leadership practices, which we called
leadership dimensions.
The Impact of Particular Leadership Dimensions
Table 2 presents the 5 inductively derived leadership
dimensions, their definitions, and the average effect size
and standard errors associated with each dimension. It is
important to stress that these 5 dimensions reflect the
conceptual and measurement frameworks employed in the
12 studies that have an asterisk against the author entries in
Table 1, and that different dimen- sions could emerge from
future research.
The list of dimensions is unusual in that it does not
include the typical distinction between leading through
tasks and organization and leading through relationships
and people. Leithwood et al. (2004) for example,
organize their literature review on “How Leadership
Influences Student Learning” under three headings: setting
direction, developing people, and redesigning the
organization. The task–relationship distinction has been
eschewed here because relationship skills are embedded in
every dimension. In goal setting, for example, effective
leadership involves not only deter- mining the goal content
(task focus) but doing so in a manner that enables staff to
understand and become committed to the goal
(relationships). What works, it seems, is careful
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
integration of staff considerations with task requirements.
Effective leaders do not get the relationships right and then
tackle the educational challenges—they incorporate both sets
of constraints into their problem solving. The remainder of
this section briefly reviews the evidence relevant to each of
the five dimensions.
Dimension 1: Establishing goals and expectations. Seven
of the 12 studies used in the dimensional analysis provided
evidence of the importance of goals and expectations.
Twenty-one indicators of this dimension yielded an average
effect size of 0.42 standard deviations, which can be
interpreted as a moderately large, and certainly as an
educationally significant effect.
Goal setting, like all the leadership dimensions discussed
here, has indirect effects on students by focusing and
coordinating the work of teachers and, in some cases,
parents. With student background factors controlled,
leadership made a difference to students through the degree of
emphasis on clear acade- mic and learning goals (Bamburg
& Andrews, 1991; Brewer, 1993; Heck et al., 1991). This
effect was found even in schools where leaders did not make
academic goals the top priority. For example, in their study of
Israeli commu- nity schools, Goldring and Pasternak (1994)
found that academic excellence was not one of the top five
goals in either low- or high-performing schools, but the
principals in the latter group still gave it significantly more
importance than the former.
In schools with higher achievement or higher
achievement gains, acade- mic goal focus is both a property
of leadership (e.g., “the principal makes student
achievement the school’s top goal”) and a quality of school
organi- zation (e.g., “schoolwide objectives are the focal
point of reading instruc- tion in this school”).
3
If goals are
to function as influential coordinating mechanisms, they
need to be embedded in school and classroom routines and
procedures (Robinson, 2001). Successful leadership
influences teaching and learning both through face-to-face
relationships and by structuring the way that teachers do
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
their work (Ogawa & Bossert, 1995).
The importance of relationships in this leadership
dimension is apparent from the fact that leaders in higher
performing schools tend to give more emphasis to
communicating goals and expectations (Heck et al.,
1990; Heck et al., 1991), informing the community of
academic accomplishments and recognizing academic
achievement (Heck et al., 1991). There was also some
evidence that the degree of staff consensus about school
goals was a significant discriminator between otherwise
similar high- and low-performing schools (Goldring &
Pasternak, 1994).
Goal content is as important as the generic process of
goal setting. The instructional leadership studies were more
likely than transformational leadership to include leadership
indicators that asked teachers to report the leaders’
emphases on particular goals, rather than the extent to
which the school leadership provided a generic direction.
The greater alignment between leadership indicators and
outcome variables in the instructional leadership research
may partially account for its stronger leadership effects in
comparison to those of transformational leadership.
A similar point has been made by Leithwood and Jantzi
(2006) in their discussion of the results of the role of
transformational leadership in the English national literacy
and numeracy reforms. They found that the degree of
transformational leadership explained the extent to which
teachers changed, but the extent of teacher change bore no
relationship to students’ achievement gains in either literacy
or numeracy. The present authors agree with the call of
Leithwood and Jantzi (2006) for leadership researchers to
focus more strongly on what changes leaders encourage
and promote, rather than merely on the extent to which they
promote unspecified changes or innovation. Leithwood and
Jantzi (2006) write:
There is a significant gulf between classroom practices
that are “changed” and practices that actually lead to
greater pupil learning; the potency of lead- ership for
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
increasing student learning hinges on the specific classroom
prac- tices that leaders stimulate, encourage and promote.
(p. 223)
In the context of goal setting, this means that what
leaders and leader- ship researchers need to focus on is not
just leaders’ motivational and direc- tion-setting activities
but on the educational content of those activities and their
alignment with intended student outcomes.
The importance of goal setting is also suggested from
findings of a meta- analysis of research on the direct effects
of leadership on students’ academic achievements reported
by Witziers et al. (2003). Although the overall impact of
leadership on students was negligible, they found that the
direction-setting role of the leader had more direct impact
on student outcomes than any of the other six dimensions of
leadership on which data were available.
4
A long tradition of research in social psychology helps
explain why goal setting is so powerful (Latham & Locke,
2006). Goals provide a sense of purpose and priority in an
environment where a multitude of tasks can seem equally
important and overwhelming. Clear goals focus attention and
effort and enable individuals, groups, and organizations to
use feedback to regulate their performance.
Dimension 2: Resourcing strategically. The word
“strategic” in the description of this dimension signals that
the leadership activity is about securing resources that are
aligned with instructional purposes, rather than leadership
skill in securing resources per se. Thus, this measure should
not be interpreted as an indicator of skill in fundraising,
grant writing, or part- nering with business, as those skills
may or may not be applied in ways that serve key
instructional purposes.
Seven studies provided evidence for how principals can
influence student achievement through their decisions about
staffing and teaching resources. Eleven indicators of this
dimension yielded an average effect size of 0.31 standard
deviations, suggesting that this type of leadership has a small
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
indi- rect impact on student outcomes.
In one study involving two separate jurisdictions,
there was a small relationship between leaders’ ability to
secure instructional resources and student achievement in
California schools, and a large relationship in a second
sample of Marshall Island schools (Heck et al., 1991). The
stronger finding for the Marshall Islands probably reflects a
context with relatively scarcer teaching resources. In a
second study of 20 U.S. elementary schools, there was an
interesting interaction between principals’ control of teacher
selection and the ambitiousness of their academic goals
(Brewer, 1993). For principals with high academic goals,
student achievement was higher in those schools where
they themselves had appointed a greater percentage of
their current staff. For principals with low academic
goals, the reverse was apparent.
These findings are sketchy and more needs to be known
about the knowl- edge and skills needed by school
leadership to link resource recruitment and allocation to
specific pedagogical goals.
Dimension 3: Planning, coordinating, and evaluating
teaching and the curriculum. Eighty indicators of this
dimension across nine studies showed that this type of
leadership has a moderate impact on student outcomes
(ES 0.42). Leaders in higher performing schools are
distinguished from their counterparts in otherwise similar
lower performing schools by their personal involvement in
planning, coordinating, and evaluating teaching and
teachers. Four interrelated subdimensions are involved in this
leadership dimension. First, teachers in higher performing
schools report that their leaders are actively involved in
collegial discussion of instructional matters, including how
instruction impacts student achievement (Heck et al., 1991).
Second, the leadership of higher performing schools is
distinguished by its active oversight and coordination of
the instructional program. School leaders and staff work
together to review and improve teaching—an idea
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
captured by that of shared instructional leadership (Heck et
al., 1990; Heck et al., 1991; Marks & Printy, 2003). In
high-performing schools, the lead- ership was more
directly involved in coordinating the curriculum across
year levels than in lower performing schools. This included
such activities as developing progressions of teaching
objectives for reading across year levels (Heck et al.,
1991).
Third, the degree of leader involvement in classroom
observation and subsequent feedback was also associated
with higher performing schools. Teachers in such schools
reported that their leaders set and adhered to clear
performance standards for teaching (Andrews & Soder,
1987; Bamburg & Andrews, 1991) and made regular
classroom observations that helped them improve their
teaching (Bamburg & Andrews, 1991; Heck, 1992; Heck et
al., 1990).
Fourth, there was greater emphasis in higher
performing schools on ensuring that staff systematically
monitored student progress (Heck et al., 1990) and that test
results were used for the purpose of program improve-
ment (Heck et al., 1991). For one study in Hawaiian
primary schools, use of achievement data involved both
principal-led schoolwide examination of data and teacher-
led classroom-based monitoring of students (Heck, 2000).
Teachers’ use of data to evaluate student progress, adjust
their teaching, plan their weekly program, and give students
feedback was a strong indica- tor of school quality, and
level of school quality had a significant influence on student
achievement in reading and math.
It is important to consider whether these findings are
equally applicable to elementary and high schools. The
greater size, more differentiated struc- tures, and specialist
teaching culture of high schools would suggest that the
degree of principal influence, in particular, may be
attenuated (Siskin & Little, 1995). The present analysis
provides some evidence relevant to this issue. Using a
sample of 23 elementary and 17 high schools, Heck (1992)
found that the mean frequency of instructional leadership
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
activity in both higher and lower performing schools was
lower in the high school group.
The mean effect size for the influence of the principal or
designee was 1.1 standard deviations in elementary schools
compared to 0.42 in high schools. This suggests that
leaders’ oversight of teaching and the curricu- lum has
more impact in elementary than in high schools. Clearly,
this is an area in which further research, using identical
indicators across elementary and high schools, is needed.
In sum, among higher performing schools, leaders
work directly with teachers to plan, coordinate, and
evaluate teachers and teaching. They are more likely than
their counterparts in lower performing schools to provide
evaluations that teachers describe as useful, and to
ensure that student progress is monitored and the results
used to improve teaching programs.
Dimension 4: Promoting and participating in teacher
learning and development. This leadership dimension is
described as both promoting and participating because more
is involved than just supporting or sponsoring other staff in
their learning. The leader participates in the learning as
leader, learner, or both. The contexts for such learning are both
formal (staff meetings and professional development) and
informal (discussions about specific teaching problems).
Seventeen effect sizes from six studies were calculated
for this dimen- sion yielding an average effect size of 0.84
standard deviations. This is a large effect and provides
some empirical support for calls to school leaders to be
actively involved with their teachers as the “leading
learners” of their school. With student background factors
controlled, the more that teachers report their school leaders
(usually the principal) to be active participants in teacher
learning and development, the higher the student outcomes
(Andrews
& Soder, 1987; Bamburg & Andrews, 1991). Leaders in
high-performing schools are also more likely to be
described by their teachers as participating in informal staff
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
discussion of teaching and teaching problems (Heck et al.,
1990; Heck et al., 1991).
The principal is also more likely to be seen by staff as a
source of instruc- tional advice, which suggests that they
are both more accessible and more knowledgeable about
instructional matters than their counterparts in other- wise
similar lower achieving schools. In one study that used a
social network rather than instructional leadership theory,
teachers were asked to indicate who they approach for
advice about their teaching (Friedkin & Slater, 1994).
Principals were significantly more likely to be nominated
as sources of advice in higher achieving schools. In
contrast, the extent to which teachers identified principals
as close personal friends or as participants in discus- sions
was not significantly related to school performance. The
authors suggest that leaders who are perceived as sources of
instructional advice and expertise gain greater respect from
their staff and hence have greater influence over how they
teach. In addition, the principals’ central position in school
communication networks means that their advice is more
likely to have a coordinating influence across the school
(Friedkin & Slater, 1994).
Dimension 5: Ensuring an orderly and supportive
environment. Instructional leadership also includes creating
an environment for both staff and students that makes it
possible for important academic and social goals to be
achieved. In an orderly environment, teachers can focus on
teaching and students can focus on learning. This dimension
was derived from 42 effect sizes derived from 8 studies.
The mean effect size of those 20 indicators was a small
0.27 standard deviations.
These findings suggest that the leadership of effective
schools is distin- guished by emphasis on and success in
establishing a safe and supportive environment through
clear and consistently enforced social expectations and
discipline codes (Heck et al., 1991). In one study that
surveyed teachers, parents, and students (Heck, 2000), there
were consistent reports across all three groups of the
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
extent to which they felt safe, comfortable, and cared for.
The more positive these reactions, the higher the school qual-
ity and the higher its achievement levels when student
background factors were controlled.
The leadership in higher performing schools is also
judged by teachers to be significantly more successful than
the leadership of lower performing schools in protecting
teachers from undue pressure from education officials and
from parents (Heck, 1992; Heck et al., 1991). This finding
was partic- ularly strong in high school samples.
An orderly and supportive environment is also one in
which staff conflict is quickly and effectively addressed. In
one study, principal ability to iden- tify and resolve conflict,
rather than allow it to fester, was strongly associated with
student achievement in mathematics (Eberts & Stone, 1986).
A second variable, measuring differences between teacher
and principal perceptions of the latter’s ability to identify
and resolve conflict, discriminated even more strongly
between higher and lower performing schools.
DISCUSSION
The purpose of this study was to examine the impact of
particular types of leadership on student outcomes. Two
analyses of different types of lead- ership provided
essentially the same answer—the closer educational leaders
get to the core business of teaching and learning, the more
likely they are to have a positive impact on students’
outcomes.
Before elaborating on these conclusions, we need to
acknowledge some limitations of this study. First, only 27
published studies were available for analysis and 5 of these
could not be included in the first meta-analysis, which
compared the effects of instructional, transformational,
and other types of leadership. The second meta-analysis,
which calculated average effects for 5 different leadership
dimensions, was based on only 12 studies, as the remaining
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
studies used unitary leadership constructs or did not report
the data required to calculate the effects of the components of
their composite leadership variables. The second limitation
is our treatment of educational outcomes. Ideally, we would
have conducted separate analyses of the impact of leadership
on academic and nonacademic outcomes, but the number
of available studies was too small to make this practical.
Our findings of both moderate and strong effects for
particular leader- ship dimensions contrast with the meta-
analysis reported by Witziers et al. (2003). Witziers and
colleagues’s findings of from no effects to weak effects
can be explained by the fact that, at that time, there were
few if any studies of indirect effects of leadership on student
outcomes. The size of the leadership effects we report are
much more comparable with those reported by Marzano et
al. (2005), but it should be remembered that this latter meta-
analysis was largely based on unpublished evidence.
The comparison between instructional and
transformational leadership showed that the impact of the
former is three to four times that of the latter. The reason is
that transformational leadership is more focused on the rela-
tionship between leaders and followers than on the
educational work of school leadership, and the quality of
these relationships is not predictive of the quality of student
outcomes. Educational leadership involves not only
building collegial teams, a loyal and cohesive staff, and
sharing an inspira- tional vision. It also involves focusing
such relationships on some very spe- cific pedagogical
work, and the leadership practices involved are better
captured by measures of instructional leadership than of
transformational leadership.
Research on the construct validity of transformational
leadership helps explain why transformational leadership
may tell us more about leader–staff relations than about
leaders’ impact on student outcomes. Brown and Keeping
(2005) showed that subordinate ratings of transformational
leader- ship are strongly influenced by the degree to which
they “like” their leader. Indeed, when the degree of liking
was controlled, the impact of transforma- tional leadership
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
on organizational outcomes was significantly reduced. If
transformational leadership measures are capturing
subordinates’ liking of their leader rather than actual
leadership practices, then proponents of trans- formational
leadership have to argue that it is this affective response
rather than particular leadership practices that links
leadership to student out- comes. Given the technical
complexity of adding value to student outcomes, this
explanation of leadership influence seems far less
plausible than one, like instructional leadership, which
specifies the leadership practices that create the conditions
for enhanced teaching and learning.
It is important to note, however, that educational
researchers on transfor- mational leadership are increasingly
modifying the original generic assess- ment tools to include
more explicitly educational items (e.g., Leithwood &
Jantzi, 2006). At the level of leadership assessment,
therefore, if not at the level of leadership theorizing, there
is an increasing convergence between transformational and
instructional leadership research in education. There is at
least one empirical study that has assessed leadership against
both frame- works. In their study of 24 U.S. elementary,
middle, and high schools, Marks and Printy (2003) assessed
both principal transformational leadership and the degree
of shared instructional leadership and combined the two
into a measure of “integrated leadership” (Table 1).
Student achievement was higher in those schools with
higher integrated leadership. Their analyses of leadership
impact on pedagogical quality and student outcomes
employed the combined integrated leadership measure and
so no conclusions can be drawn about the relative
contribution of each. Nevertheless, they do suggest that
transformational leadership is a necessary but not sufficient
condition for shared instructional leadership.
Clearly, the types of motivational, collaborative, and
interpersonal skills that are emphasized in transformational
leadership research are essential to leaders’ ability to
improve teaching and learning. The critical question is
whether one needs transformational leadership theory to
Author copy - submitted and accepted version of the following publication:
Robinson, V. M. J., Lloyd, C. & Rowe, K. (2008). The impact of leadership on student
outcomes. An analysis of the differential effects of leadership types.
Educational Administration Quarterly, 44(5), 635-674. doi:
10.1177/0013161X08321509
study and develop this aspect of leadership. In our view
one does not. As discussed earlier, instructional leadership
measures are increasingly integrating an interper- sonal
and task focus into their indicators. The five leadership
dimensions derived from the published research all
include leadership practices that require the integration of
task and relationship considerations.
Our findings about the relative impact of the five
leadership dimensions provide more detailed guidance,
than does the prior analysis, about the types of leadership
that make a difference to student outcomes. Such lead-
ership involves the determined pursuit of clear goals, which
are understood by and attractive to those who pursue them.
Goal setting is a powerful lead- ership tool in the quest for
improving valued student outcomes because it signals to
staff that even though everything is important, some activities
and outcomes are more important than others. Without
clear goals, staff effort and initiatives can be dissipated in
multiple agendas and conflicting priori- ties, which, over
time, can produce burnout, cynicism, and disengagement.
Because considerably more happens in schools than the
pursuit of explicit goals, even the most goal-focused leaders
will need to skillfully manage the constant distractions that
threaten to undermine their best intentions. Such
distractions, in the form of new policy initiatives, school
crises, calls for goal revision or abandonment, and the need
to maintain school routines that are not directly goal related,
all threaten to undermine goal pursuit. A shared goal focus
enables leaders and staff to recognize that they are
being distracted and to consciously decide what to do about
it. Without that focus, there is no distraction to recognize
and the routines and crises come to dominate leaders’
work.
Clarity around educational goals makes strategic
resourcing possible. Although this leadership dimension had
a small impact on student outcomes, resourcing goal pursuit
is one of the conditions required for goal achieve- ment.