Re-Examining the Role of Teacher Quality In the Educational Production Function

Abstract

This study uses administrative data linking students and teachers at the classroom level to estimate teacher value-added to student test scores. We find that variation in teacher quality is an important contributor to student achievement more important than has been implied by previous work. This result is attributable, at least in part, to the lack of a ceiling effect in the testing instrument used to measure teacher quality. We also show that teacher qualifications are almost entirely unable to predict value-added. Motivated by this result, we consider whether it is feasible to incorporate value-added into evaluation or merit pay programs.

Full text

Page 1

Re-Examining the Role of Teacher Quality
In the Educational Production Function

Cory Koedel
University of Missouri

Julian R. Betts*
University of California, San Diego

Revised April 2007


Abstract

This study uses administrative data linking students and teachers
at the classroom level to estimate teacher value-added to student
test scores. We find that variation in teacher quality is an
important contributor to student achievement – more important
than has been implied by previous work. This result is
attributable, at least in part, to the lack of a ceiling effect in the
testing instrument used to measure teacher quality. We also show
that teacher qualifications are almost entirely unable to predict
value-added. Motivated by this result, we consider whether it is
feasible to incorporate value-added into evaluation or merit pay
programs.







* The authors thank Andrew Zau and many administrators at San Diego Unified School District,
in particular Karen Bachofer and Peter Bell, for helpful conversations and assistance with data
issues. We also thank Yixiao Sun, Julie Cullen, Nora Gordon, Mark Appelbaum and participants
at the 2006 SOLE conference, particularly our formal discussant Eric Hanushek, for their useful
comments and suggestions as well as the Spencer Foundation for research support. The
underlying project that provided the data for this study has been funded by the Public Policy
Institute of California.

Page 2

1
I.
Introduction
It has been well established that education plays an important role in determining both economic
growth and individual life outcomes (for example, see Katz and Murphy, 1992). This has led to
an ongoing interest in the determinants of student achievement, including teacher quality.
However, researchers have historically struggled to capture the role of teacher quality in the
educational production function. Given the importance of education and the undeniable role
played by teachers, how much does variation in teacher quality affect student performance?

The vast majority of the empirical work on teacher quality has relied on observable teacher
qualifications to measure teacher quality. As a whole, this body of research suggests that these
qualifications are only weakly related to student performance.1 Therefore, we shift our focus
away from teacher qualifications and instead measure teacher quality by value-added to student
test scores.2 Although value-added has been criticized by some, it continues to gain traction
among both researchers and policy makers. In fact, proposals to base teacher evaluations on
value-added, sometimes involving pay incentives, are becoming increasingly common.3

We analyze teacher value-added to student performance on math and reading standardized exams
using micro-level data from San Diego elementary schools linking students and teachers at the
classroom level. Our results indicate that variation in teacher quality, measured by value-added,
is considerably larger than previous research has implied. Our larger variance estimates are

1 For reviews of this literature, see Hanushek (1986, 1996).
2 There is a small literature that has shifted its focus to teacher value-added. Recent studies include Rivkin,
Hanushek and Kain (2005), Hanushek et al. (2005), Aaronson, Barrow and Sander (2007), Nye, Konstantopoulos
and Hedges (2004), McCaffrey et al. (2003), Harris and Sass (2006) and Koedel (2007).
3 For example, see Gordon, Kane and Staiger (2006). Other examples include non-profit groups like Battelle for
Kids in Columbus, OH, which has set up a three-year pilot program that uses value-added as an evaluation tool for
teachers in Ohio and the state of Florida.

Page 3

2
attributable, at least in part, to the lack of a ceiling effect in the testing instrument that we use to
measure teacher quality. Test-score ceilings inhibit students‟ performance gains as test-score
levels rise. These ceilings are quite common in practice and have two important implications for
value-added analysis. First, in the presence of a test-score ceiling, estimating teacher effects
from a typical value-added specification can lead to an understatement of the variance of teacher
quality and, in turn, of the importance of teacher quality as an educational resource. Second, a
test-score ceiling will influence individual teachers‟ value-added estimates. This latter issue is of
particular concern if value-added is to be used to evaluate teacher performance.

We relate our value-added measures of teacher quality to the qualifications that primarily
determine teacher recruitment, retention and salaries. Our results support previous research
indicating that these qualifications are poor predictors of teacher performance. Even upper-
bound estimates of the ability of observable teacher qualifications to predict variation in
outcome-based teacher quality are very small. Similarly, teachers‟ salaries are virtually
uncorrelated with their value-added to student test scores.

Motivated by the weak link between teacher performance and teacher qualifications, and the
growing interest in value-added more generally, we consider the role that value-added estimates
might play in determining teacher accountability. When compared to the current standards by
which most teachers are judged (observable qualifications), a value-added approach offers a
significant improvement in terms of rating teachers based on their contributions to actual student
performance. However, in both math and reading, estimation error constitutes a considerable

Page 4

3
portion of the individually estimated teacher effects. Therefore, value-added modeling may be
better suited as just one component of a more comprehensive system of teacher evaluation.

II.
Empirical Strategy

We estimate teacher fixed effects from a value-added model of student achievement in the
reduced form:4
(1)
( 1)

ijkst iijks tit ititit
TestScore TestScoreZipCode

XZC


()

()()
J teacher
it
D
K grade
it
D
S school
it
D
it



Equation (1) describes the test-score performance of student i taught by teacher j in grade k at
school s in year t. The model controls for heterogeneity in student ability by including student
fixed effects (denoted by
i  ).5 The vectors Xit, Zit and Cit contain time-varying student-, school-
and classroom-level characteristics, respectively. The variables included in these vectors are
listed in Table 1. Vectors of indicator variables for teachers, grade levels and schools are also
included in the student-achievement specification.

In addition to student fixed effects, our model includes school and zip-code fixed effects.
Together, these sets of fixed effects ensure that the model evaluates variation in teacher quality
within schools, ignoring any between-school variance. Our methodology is supported by
previous empirical work indicating that most of the variation in teacher quality occurs within

4 Value-added is often modeled in terms of test-score gains. The gainscore specification is a specific case of the
general value-added specification in equation (1). We do consider gainscore models in our analysis. As would be
expected, teacher-effect estimates from a gainscore model that is analogous to equation (1) are highly correlated
with our estimates.
5 Students and teachers in San Diego are non-randomly assigned to classrooms within schools, highlighting the
importance of controlling for student ability in the model of student achievement. In an omitted exercise that is
available from the authors upon request, we reject the hypothesis that, within schools and grades, current teachers do
not predict previous test-score performance. This result additionally implies that students may be sorted along
dimensions that are unobserved.

Page 5

4
schools as opposed to between schools (Hanushek et al., 2005; Nye, Konstantopoulos and
Hedges, 2004). This is likely to be the case because the degree of sorting of teacher quality
across schools, which would drive any between-school variation in teacher quality, is largely
dependent on the success of schools in identifying and hiring the best teachers.6 The empirical
evidence suggests that schools may find it very difficult to identify the best teachers and that
even if they do, they may choose not to hire them.7 In our model-sensitivity analysis in Section
V, we show that essentially all of the variation in teacher quality in San Diego elementary
schools exists within schools.

The potential influence of peer effects is possibly the most worrisome confounding factor in any
analysis of teacher quality. To address this issue, our model controls for the year (t-1)
achievement of classroom-level peers. We also note that the effect of any systematic ability
grouping experienced by students will be largely absorbed at the student level because the
student fixed effect will pick up the average peer effect experienced by a given student over the
course of the panel. Similarly, we control for class size to prevent variation in class size from
being misinterpreted as variation in teacher quality.

As it is written, the model in (1) will produce biased estimates of the coefficients of interest
because the demeaned error term will be correlated with the demeaned lagged dependent
variable. Therefore, we adopt the method of Anderson and Hsiao (1981) to estimate the

6 Teachers‟ preferences for better schools could also affect teacher sorting. However, hiring restrictions imposed by
the labor contract between San Diego Unified School District and the teacher‟s union should substantially limit the
effects of teachers‟ preferences on teacher sorting. This will be discussed in more detail in Section V.
7 Section VIII of this paper shows that observable teacher qualifications are virtually uncorrelated with outcome-
based teacher quality. In addition, numerous studies have documented the weak link between observable teacher
qualifications and student performance. See, for example, Aaronson et al. (2007), Angrist and Guryan (2003), Betts
(1995), Betts, Zau and Rice (2003), Hanushek (1986, 1996), Kane, Rockoff and Staiger (2006). Also, Ballou (1996)
argues that schools may choose not to hire the most qualified teachers even when given the opportunity.