Content uploaded by Mareike Kunter
Author content
All content in this area was uploaded by Mareike Kunter on Apr 15, 2015
Content may be subject to copyright.
Homework Works if Homework Quality Is High: Using Multilevel
Modeling to Predict the Development of Achievement in Mathematics
Swantje Dettmers
Max Planck Institute for Human Development
Ulrich Trautwein and Oliver Lu¨dtke
Max Planck Institute for Human Development
and University of Tuebingen
Mareike Kunter and Ju¨rgen Baumert
Max Planck Institute for Human Development
The present study examined the associations of 2 indicators of homework quality (homework selection
and homework challenge) with homework motivation, homework behavior, and mathematics achieve-
ment. Multilevel modeling was used to analyze longitudinal data from a representative national sample
of 3,483 students in Grades 9 and 10; homework effects were analyzed at the student and the class level
simultaneously. Students who perceived their homework assignments to be well selected reported higher
homework motivation, and homework behavior at both the student and the class level predicted later
achievement at the class level. Homework assignments perceived to be cognitively challenging were
differentially associated with achievement at the student and the class level. Students who perceived their
homework to be challenging (student level) showed relatively poor performance, but homework chal-
lenge was positively related to achievement at the class level.
Keywords: homework quality, mathematics achievement, multilevel modeling
Does homework enhance school effectiveness? More precisely,
under which conditions does homework effectively supplement in-
school learning? In most countries around the world, homework
accounts for a substantial proportion of study time (Cooper, 1989;
Cooper, Lindsay, Nye, & Greathouse, 1998; Xu, 2005). At the same
time, the effectiveness of homework is a topic of much discussion,
and studies investigating the relationship between homework and
achievement have produced mixed results. Most previous homework
research has focused on homework time and analyzed the relationship
between homework and achievement (Cooper, Robinson, & Patall,
2006). However, according to Corno (1996), homework is a complex
process influenced by a variety of factors. Much empirical research
fails to reflect the complexity of the variables involved in homework
assignment and homework completion (see Cooper, 1989), and the
methodological shortcomings of many studies make it difficult to
draw firm conclusions about the strength of the homework–
achievement relationship (Trautwein, 2007).
The present article builds on the homework model proposed by
Trautwein, Lu¨dtke, Schnyder, and Niggli (2006). The model predicts
that homework motivation, homework behavior, and achievement are
influenced by characteristics including homework quality (i.e., well
prepared and adequately challenging assignments). To date, however,
empirical research on homework quality has been sparse. For in-
stance, it was only a peripheral issue in Cooper et al.’s (2006)
state-of-the art review of homework studies. This paucity of research
is surprising, given that homework quality has been a subject of lively
debate among teachers, parents, and students for decades now. To
address this research deficit, this article examines how homework
quality is associated with homework motivation, homework behavior,
and mathematics achievement.
Homework as an Opportunity to Learn
Homework is defined as “tasks assigned to students by school
teachers that are meant to be carried out during non-school hours”
(Cooper, 1989, p. 7). Homework involves different actors (teach-
ers, students, and parents), serves different purposes (e.g., enhanc-
ing student performance and self-regulation), impacts the organi-
zation of lessons (e.g., discussing, checking, and grading
homework), and involves tasks at different levels of challenge
(e.g., routine vs. complex tasks). Thus, homework is a complex
issue that warrants investigation within a broad theoretical frame-
work derived from research on learning and instruction (Trautwein
&Ko¨ller, 2003).
Swantje Dettmers, Mareike Kunter, and Ju¨rgen Baumert, Center for
Educational Research, Max Planck Institute for Human Development,
Berlin, Germany; Ulrich Trautwein and Oliver Lu¨dtke, University of
Tuebingen, Tuebingen, Germany, and Center for Educational Research,
Max Planck Institute for Human Development.
The research reported in this article is based on data from the Professional
Competence of Teachers, Cognitively Activating Instruction, and the Devel-
opment of Students’ Mathematical Literacy study (COACTIV), directed by
Ju¨ rgen Baumert, Max Planck Institute for Human Development; Werner
Blum, Kassel University, Kassel, Germany; and Michael Neubrand, Carl von
Ossietzky University, Oldenburg, Germany. The project was funded by the
German Research Foundation (Grant BA 1461/2-2) as part of its priority
Program on School Quality.
Correspondence concerning this article should be addressed to Swantje
Dettmers, Center for Educational Research, Max Planck Institute for
Human Development, Lentzeallee 94, 14195 Berlin, Germany. E-mail:
dettmers@mpib-berlin.mpg.de
Journal of Educational Psychology © 2010 American Psychological Association
2010, Vol. 102, No. 2, 467–482 0022-0663/10/$12.00 DOI: 10.1037/a0018453
467
Models of school learning (e.g., Bloom, 1976; Carroll, 1963,
1989) propose that time is an important determinant of degree of
learning. Homework contributes substantially to time on task in
core subjects and thus provides an additional opportunity to learn.
One of the main reasons for assigning homework is thus to
increase the total study time (Paschal, Weinstein, & Walberg,
1984; Walberg & Paschal, 1995). Indeed, most homework studies
investigate out-of-school learning as a function of time or quantity
of homework (e.g., Cooper et al., 2006). Studies conducted in the
United States point to a positive overall association between home-
work time and achievement, but methodological shortcomings in
most of these studies have been noted (Cooper et al., 2006;
Trautwein, 2007). Major criticisms include the lack of control for
other important predictors of achievement, the failure to ade-
quately model the multilevel structure inherent in homework stud-
ies, the reliance on cross-sectional data, uncertainty about the
reliability of the homework measures used, and the absence of a
theoretical model of homework assignment and homework behav-
ior (Cooper, 1989; Trautwein & Ko¨ller, 2003).
In an attempt to overcome some of the limitations of prior home-
work research, Trautwein and colleagues (for a detailed description,
see Trautwein, Lu¨dtke, Kastens, & Ko¨ller, 2006; Trautwein, Lu¨ dtke,
Schnyder, & Niggli, 2006) proposed a theoretical model that com-
bines elements of expectancy–value theory (Eccles, 1983; Eccles &
Wigfield, 2002), self-determination theory (Deci & Ryan, 2002;
Grolnick & Slowiaczek, 1994; Ryan & Deci, 2000), and research on
learning and instruction (Brophy & Good, 1986; Weinert & Helmke,
1995a, 1995b). The model takes into account the three protagonists in
the homework process (students, teachers, and parents) and covers six
major groups of variables (achievement, homework behavior, home-
work motivation, student characteristics, parental behavior, and the
learning environment). The model predicts that the effort students
spend on their homework assignments (i.e., doing their best to solve
the tasks assigned) is positively related to their achievement. In line
with expectancy–value theory, homework effort is conceptualized as
strongly influenced by expectancy and value beliefs, representing two
aspects of homework motivation. The expectancy component reflects
a student’s belief in being able to complete a given homework
assignment successfully (Bandura, 1998; Pintrich, 2003). The value
component describes students’ reasons for doing a task (Eccles &
Wigfield, 2002; Pintrich, 2003; Pintrich & De Groot, 1990) in terms
of the importance of succeeding in a specific domain, the enjoyment
of engaging in the activity, the utility of the activity, and the costs
associated with it. The model further predicts that family character-
istics and the quality of parental homework assistance are associated
with homework expectancy and value beliefs and with homework
effort. Furthermore, student characteristics such as prior knowledge,
cognitive abilities, and conscientiousness are predicted to affect
homework motivation (expectancy and value beliefs) and effort. Fi-
nally, the model comprises core characteristics of homework, includ-
ing homework frequency, homework length, homework control, and
homework quality. Homework quality is at the core of the present
investigation.
Homework Quality, Homework Motivation,
Homework Behavior, and Achievement
Based on Astleitner (2007), homework can be regarded as a set
of tasks or problems that are supposed to support learning (e.g., by
activating prior knowledge, intensifying comprehension, or apply-
ing knowledge to new tasks or problems). High homework quality
thus requires the careful selection and preparation of appropriate
and, to some extent, interesting tasks that reinforce classroom
learning (Trautwein & Lu¨dtke, 2007). Further, homework assign-
ments must be cognitively challenging but not overtaxing. Home-
work assignments of low cognitive challenge simply require stu-
dents to recall information, whereas challenging tasks require them
to synthesize ideas, for example, or to combine strategies or
knowledge areas.
In stark contrast to research on general instructional quality
(Kunter & Baumert, 2006a; Weinert, Schrader, & Helmke, 1989),
research on the relationship between homework quality and stu-
dent achievement is—as noted above—surprisingly scarce. In-
stead, the clear focus of homework research has been on home-
work quantity (Warton, 2001). Yet the time needed to learn a given
criterion is, in part, a function of the quality of instruction and of
the students’ ability to understand that instruction (Gettinger,
1989). Warton (2001) argued that “the quality and type of home-
work tasks vary to such an extent both within and between subject
areas, ability, and grade level that to focus on time variables alone
seems an oversimplification” (p. 157).
The few empirical studies to date that have included homework
quality variables indicate beneficial effects of homework quality.
For instance, using structural equation modeling, Keith and Cool
(1992) found that high-quality instruction (in terms of average
student ratings of quality of instruction, school reputation, and
teachers’ interest in students) was positively associated with a
higher motivation, which led to more academic coursework, which
in turn was positively related to achievement. Moreover, the au-
thors found that higher quality of instruction and higher motivation
resulted in higher homework times.
Trautwein and colleagues (Trautwein & Lu¨dtke, 2007, 2009;
Trautwein, Lu¨dtke, Schnyder, & Niggli, 2006) used student reports
about the quality of their homework assignments as predictors of
homework expectancy beliefs, homework value, and homework
effort. The homework quality items used in these studies covered
several aspects of homework assignments, such as teachers’ ad-
vance preparation of assignments, integration in lessons, and dif-
ficulty level. Higher self-reported homework expectancy and value
beliefs and higher homework effort were found among students
who had a more favorable perception of homework quality than
their classmates (student-level effect) and in classes where the
aggregated perception of homework quality was higher than in
other classes (class-level effect). Unfortunately, achievement was
not considered in these studies. Although it did not target home-
work quality directly, a study by Trautwein, Niggli, Schnyder, and
Lu¨dtke (2009) is also of relevance in the present context. The
authors asked teachers about their homework attitudes and behav-
iors. Overall, they found a relatively low emphasis on drill and
practice tasks and a high emphasis on motivation was associated
with favorable developments in students’ homework effort and
achievement.
Cooper (1989) distinguished between homework containing
same-day tasks and homework including elements of practice and
or preparation. The former is cognitively less demanding, consist-
ing primarily of repetitive exercises. Practice or preparation home-
work is cognitively more demanding, involving material that has
not been covered fully in class or material dealt with previous
468 DETTMERS, TRAUTWEIN, LU
¨DTKE, KUNTER, AND BAUMERT
lessons. Reviewing eight studies, Cooper found an average effect
size of d⫽0.14 favoring cognitively more demanding homework
assignments. Lipowsky, Rakoczy, Klieme, Reusser, and Pauli
(2004) analyzed the predictive power of homework assignments
for mathematics achievement. They found that students in classes
where homework was perceived to be cognitively demanding
(“Our math teacher sets homework tasks that make us think about
new things”) showed greater achievement gains than their peers in
other classes.
Overall, the few available studies indicate that homework qual-
ity matters. More research is necessary, however. In particular, the
link between homework quality and later achievement is far from
established, despite the intuitive assumption that homework qual-
ity matters. Moreover, research would benefit from a deeper un-
derstanding of different facets of quality and whether they can be
reliably measured. The present study is a step in this direction. We
focused on two indicators of homework quality, which were col-
lected via student reports. The first, homework selection, taps the
selection of appropriate and interesting homework tasks. Do
the tasks selected by teachers enhance students’ understanding?
Are they interesting? Is homework well integrated into lessons?
The homework task selection indicator reflects students’ general
evaluation of homework quality. The measure has similarities to
scales used in previous research (e.g., Trautwein & Lu¨dtke, 2009),
the findings of which generally point to positive associations with
homework motivation and behavior.
The second indicator, homework challenge, measures students’
perceptions of the cognitive challenge inherent in the homework
tasks. Are they easy to solve or do they require mental effort? This
indicator targets the individually perceived difficulty level of
homework. Cognitively activating instruction has been found to be
positively associated with student achievement at the class level
(Kunter & Baumert, 2006b), whereas repetitive tasks and easy
homework assignments have been found in some studies to be
negatively related to student achievement (Cooper, 1989;
Trautwein, Ko¨ller, Schmitz, & Baumert, 2002).
Assessing Homework Quality Using Student Reports
The focus of the present study is on homework quality as a
characteristic of the learning environment. At least three data
sources are regularly used to assess classroom environments
(Anderson, 1982; Fraser, 1991; Turner & Meyer, 2000): observer
ratings, teacher ratings, and student ratings. Each perspective has
specific methodological and theoretical advantages and disadvan-
tages. Observer ratings are very cost and labor intensive. Teachers,
who can be considered experts on different instructional ap-
proaches and are responsible for guiding the instructional process,
might seem to be the ideal source of information. However,
self-serving strategies and teaching ideals may compromise the
validity of their ratings. In the present study, student ratings were
used to assess homework quality. Students can also be considered
experts on the learning environment. They are exposed to a variety
of teachers in different subjects and thus have the opportunity to
compare different teaching styles. From a phenomenological point
of view, students’ ratings are the most appropriate source of data
for assessing the learning environment: A given student’s behavior
is likely to be more affected by his or her interpretation of the
classroom context than by any objective indicator of that context.
At the same time, given the idiosyncratic nature of students’
perceptions of their learning environment, the reliability of student
report data has been questioned (Aleamoni, 1999; Marsh & Roche,
1997; but see also Marsh, 2001; Marsh & Roche, 2000). Concep-
tual and methodological challenges therefore need to be addressed
before student ratings can properly be used to gauge the effects of
characteristics of the learning environment (see Lu¨dtke, Robitzsch,
Trautwein, & Kunter, 2009).
Most important, perceived homework quality can be conceptu-
alized at two different levels: the student and the class level. At the
student level, ratings represent individual perceptions of home-
work quality that may differ across the students in a class, depend-
ing, for instance, on their prior knowledge. At this level, the focus
of interest is whether individual students’ perceptions of their
classroom or teacher are related to individual differences in moti-
vational, cognitive, and behavioral outcomes. Conversely, data
aggregated at the class level yield a measure of the shared percep-
tion of the learning environment; idiosyncrasies in individual
perceptions tend to be canceled out by the averaging process.
The literature on multilevel modeling (e.g., Raudenbush &
Bryk, 2002; Robinson, 1950) has clearly demonstrated that rela-
tions between variables often vary across the different levels of
analysis. The associations found at a higher, aggregate level (e.g.,
class level) do not allow conclusions to be drawn about relation-
ships at the lower level of analysis (e.g., student level), and vice
versa. The main reason for differential effects at different levels of
analysis is that entirely different factors might operate at the
individual and the group level. Such diverging patterns of associ-
ations are highly interesting from a theoretical and practical point
of view, and one of the main accomplishments of multilevel
analyses is to document them. Prominent examples of differential
relations at different levels of analysis in educational research are
effects of ability grouping (e.g., the big-fish-little-pond effect;
Trautwein, Lu¨dtke, Marsh, Ko¨ ller, & Baumert, 2006; Trautwein,
Lu¨dtke, Marsh, & Nagy, 2009) and the amount of time spent on
homework (Trautwein, 2007).
In the present study, we looked at two analytical levels: the
student level and the class level. At the student level, we were
interested in interindividual differences between students’ percep-
tions of their homework assignments in the same class and their
associations with different outcome variables. At the class level,
which reflects the shared environment, we aimed at analyzing the
overall effect of the quality of the homework assigned to a class.
Recent studies have confirmed that class-mean ratings provide a
reliable indicator of homework quality, with multilevel models
showing that between 12% and 21% of the total variance in
students’ perceptions of homework quality was located between
classes (Trautwein & Lu¨dtke, 2009; Trautwein, Lu¨ dtke, Schnyder,
& Niggli, 2006).
The Present Study
The present study examined whether homework quality predicts
homework motivation (homework expectancy and value beliefs),
homework behavior (time on homework and homework effort),
and achievement in mathematics (see Figure 1 for a graphical
illustration). We used data obtained from a nationally representa-
tive sample of 3,483 high school students at two points of mea-
surement over the course of a year in the context of the German
469
HOMEWORK WORKS IF HOMEWORK QUALITY IS HIGH
extension to the PISA 2003 study (Prenzel, Carstensen, Scho¨ps, &
Maurischat, 2006).
We investigated four research questions: two concerning rela-
tionships at the student level and two pertaining to the class level.
Our first research question addressed the predictive power of
perceived homework selection at the student level. Generally, we
expected to find a positive association between homework task
selection and the outcome variables. More specifically, at the
student level, we expected high ratings of homework selection to
predict high homework expectancy and value beliefs, homework
effort, and time spent on homework. Moreover, we expected
students who perceived that homework was well selected and
interesting to show greater achievement gains than their peers.
Our second research question concerned the role of homework
challenge. A student who reports a high level of homework chal-
lenge relative to his or her classmates may feel overtaxed by the
homework assignments and exhibit less favorable outcomes. We
thus hypothesized that homework challenge is negatively related to
homework expectancy beliefs at the student level. Moreover, be-
cause individual perceptions of homework challenge depend to
some extent on students’ cognitive abilities and prior achievement,
we expected high homework challenge ratings to negatively pre-
dict mathematics achievement. No specific hypotheses were for-
mulated for the association between homework challenge and
homework value beliefs, homework time, and homework effort.
According to Good and Brophy (1990), homework assignments
must be of appropriate difficulty for students to perceive them as
valuable. Assignments that are either too easy or too difficult may
be perceived as a waste of time. According to expectancy–value
theory, low value beliefs are likely to result in low effort.
Our third research question concerned the role of homework
task selection at the class level. We hypothesized a positive asso-
ciation between aggregated homework selection ratings and home-
work motivation (expectancy and value beliefs) and homework
behavior (time spent on homework and homework effort). In
addition, we expected to find greater achievement gains in classes
in which the mean perception of homework selection was com-
paratively high.
The fourth research question addressed the role of homework
challenge at the class level. The meaning of this variable might be
quite different at the student and the class level. At the class level,
a high level of perceived challenge might indicate that the teacher
considers it important to assign cognitively challenging tasks.
Paralleling our hypotheses at the student level, we expected the
relationship between homework challenge and homework expect-
ancy beliefs to be negative. No specific hypotheses were formu-
lated for the relationship between homework challenge and home-
work value beliefs, homework time, and homework effort.
However, we expected to find differential associations with math-
ematics achievement at the student and the class level. The aggre-
gated perceived level of challenge is a proxy of how cognitively
challenging homework assignments are perceived to be within a
class. In line with research on instructional quality (Kunter &
Baumert, 2006a), which has found cognitively activating elements
of instruction to be positively associated with student achievement,
we assumed that perceived level of challenge positively predicts
mathematics achievement at the class level.
Methods
Data Source and Sample
The Programme for International Student Assessment (PISA)
was initiated by the Organisation for Economic Cooperation and
Development (OECD) to study and compare student achievement
in the OECD and in some non-OECD countries (Organisation for
Economic Cooperation and Development [OECD], 2004a). The
analyses reported are part of the PISA-I-Plus study, the German
extension to the 2003 cycle of the PISA study. The study was
conducted during the school years 2003 (when students were in
Grade 9) and 2004 (when students were in Grade 10). The German
PISA extension study focused on intact classes in schools partic-
ipating in the international PISA 2003 study. The main goal of this
extension study was to examine effects of variables at the student,
parent, teacher, and school level on learning gains in mathematics
(Prenzel, Drechsel, Carstensen, & Ramm, 2004). A multistage
Figure 1. Condensed version of the homework model (see Trautwein, Lu¨dtke, Schnyder, & Niggli, 2006).
470 DETTMERS, TRAUTWEIN, LU
¨DTKE, KUNTER, AND BAUMERT
sampling procedure was implemented to ensure high representa-
tiveness of the data. The full dataset consisted of 4,567 students in
194 classes. The dataset used in the analyses reported here was
restricted in two respects: to students participating at both mea-
surement points and to classes with the same mathematics teacher
at both measurement points. Thus, the final data set consisted of
3,483 students (56.8% female; mean age at first measurement
point: 15.1 years) in 155 classes (average class size: 22.47 stu-
dents). The students participating in the PISA-I-Plus study were
administered additional tests the day after the international PISA
assessment.
Instruments
With the exception of the second mathematics achievement test,
all instruments analyzed in the present study were administered in
Grade 9 (T1). Further, all homework instruments referred specif-
ically to homework assignments in mathematics.
Variables at the student level. In sum, the following eight
variables and four control variables were analyzed at the student
level.
T1 achievement. We used students’ Grade 9 mathematics
literacy and reading literacy scores in the international PISA 2003
assessment to control for prior achievement.
T2 mathematics achievement. Mathematics achievement at
T2 was assessed by a test covering the standard content stipulated
in the federal states’ curricula for Grade 10 mathematics (see
Baumert et al., in press). The test was administered to all students
in the present sample. The correlation between the national and the
international test was .92. Mathematics test scores were generated
using item response theory techniques (for details, see Prenzel et
al., 2006). The resulting test score distribution had a mean of M⫽
500 (SD ⫽100); test reliability was r⫽.79 (T2). Because PISA
used a multimatrix design to assess mathematics achievement,
each individual score is based on a small sample of tasks. It was
therefore not possible to estimate Cronbach’s alpha. Instead, we
report reliability in terms of the correlation between independent
plausible value draws (Adams & Wu, 2002; OECD, 2004b).
Homework behavior. Homework effort was measured by five
items (e.g., “I do my best in my mathematics homework”; “I
always try to do my complete mathematics homework”). Students
scoring high on this scale do their homework assignments care-
fully and to the best of their ability. A 4-point Likert response scale
(from 1 ⫽totally disagree to 4 ⫽totally agree) was used. Internal
consistency (Cronbach’s alpha) was ␣⫽.69. Homework time was
assessed using one open-ended item that required students to state
how much time (in hours) they spent on mathematics homework
per week (see Baumert et al., 2006).
Homework motivation. Two scales assessed the expectancy
component (“If I make an effort, I can do all my mathematics
homework”; three items; ␣⫽.66) and the value component (“Our
mathematics homework takes a lot of time and is of little use to
me” [reverse scored]; two items; ␣⫽.72). A 4-point Likert
response scale (from 1 ⫽totally disagree to 4 ⫽totally agree) was
used. Students who score high on homework expectancy beliefs
are optimistic about their capability to work successfully on the
task assigned. The items tapping the value component focused on
the facets of utility and cost (see Baumert et al., 2006).
Homework quality. Two scales were used to describe home-
work quality. Perceived quality of homework task selection was
measured by five items (e.g., “Our mathematics teacher almost
always chooses homework assignments really well”; ␣⫽.83). The
scale assesses how well-prepared and interesting homework as-
signments were perceived to be. Perceived homework challenge
was measured by four items (e.g., “Our mathematics homework
assignments are often too easy” [reverse scored]; ␣⫽.74). The
scale assesses the extent to which homework assignments are
perceived to be cognitively challenging. Both homework quality
scales are reported in the appendix. In both cases, a 4-point Likert
response scale (from 1 ⫽totally disagree to 4 ⫽totally agree) was
used (see Baumert et al., 2006).
1
Control variables. Four variables, which were specified in the
individual background model used in Baumert et al. (in press), were
included in the multilevel models to control for possible confounds at
the student level. First, we included a measure of basic cognitive
abilities.The Figure Analogies subscale of the Cognitive Ability Test
4 –12 ⫹R (Heller & Perleth, 2000), a German version of Thorndike
and Hagen’s (1993) Cognitive Abilities Test, was used. Because the
Figure Analogies subscale taps highly g-loaded ability components
(Carroll, 1993), it is frequently used as a parsimonious test of cogni-
tive abilities. Second, we controlled for students’ socioeconomic
status (SES), assessed using the International Socio-Economic Index
(ISEI) developed by Ganzeboom, de Graaf, Treiman, and de Leeuw
(1992). We used the highest ISEI score in the family in our analyses.
Third, we created six dummy variables to control for the parental
educational background.
2
Fourth, we controlled for the sex of the
students.
Variables at the class level. At the class level, we included
two measures of homework quality and school track as a control
variable in our models.
Homework quality. The two homework quality scales, per-
ceived quality of homework task selection and perceived home-
work challenge, were aggregated at the class level to give a
measure of homework quality effects.
1
We also conducted a confirmatory factor analysis with the five home-
work scales (value, expectancy, effort, selection, and challenge). This
model revealed an acceptable fit, with
2
(3) ⫽1,007.31, root-mean-square
error of approximation ⫽.043, and CFI ⫽.929. On average, the standard-
ized factor loadings were .52 (effort), .64 (value), .76 (expectancy), .70
(selection), and .55 (challenge). Two of the scales (homework effort and
perceived homework challenge) included both positively and negatively
worded items, potentially leading to method factors. For instance, a
negative-item effect occurs if there are systematic residual covariations
among the responses to the negatively worded items and if this so-called
correlated uniqueness cannot be explained by the postulated latent factor.
We thus included a total of three correlated uniquenesses among two pairs
of negatively worded items and one pair of positively worded items to
address this issue (see Marsh, 1996).
2
The six dummy variables were Parental Educational Background
(PEB) 1 (no apprenticeship, with or without Hauptschule certificate; lowest
educational background), PEB 2 (apprenticeship, with or without Haupts-
chule certificate), PEB 3 (apprenticeship and Realschule certificate), PEB
4(Hauptschule or Realschule certificate and technical college), PEB 5
(technical college/Gymnasium certificate, no higher education), PEB 6
(degree qualification; highest educational background). PEB 4 (intermedi-
ate educational background) served as the reference category.
471
HOMEWORK WORKS IF HOMEWORK QUALITY IS HIGH
Control variables. At the class level, we controlled for school
track. After completing primary school (at the age of 10 or, in
some states, 12), students in Germany are typically assigned to
either a comprehensive school or to one of three secondary tracks:
Gymnasium, the highest track; Realschule, the intermediate track;
or Hauptschule, the least academically demanding track. Because
Hauptschule students graduate after Grade 9, they were not ana-
lyzed in the present study. Some German states have a combined
Mittelschule (catering for Realschule and Hauptschule students).
Thus, four school types are included in the following analyses
(Gymnasium,Realschule,Mittelschule, and comprehensives).
Dummy variables were created for each track to control for effects
of school type. Realschule was used as the reference category.
Statistical Analyses
Analyzing hierarchical data. As in most research conducted
in school settings, students in this study were nested within classes.
Students within a class are typically more similar to each other
than are two students randomly selected from the whole sample.
For the present research, it is important to note that the meaning of
a variable at the student level may not bear any straightforward
relation to its meaning at the class level. Whenever major variables
represent different levels of analysis, it is important to use a
statistical method that takes the nested structure into account.
Multilevel modeling provides a powerful framework for analyzing
data collected in the school context by accounting for nonindepen-
dence of the observations (Raudenbush & Bryk, 2002). A detailed
description of multilevel modeling is beyond the scope of the
present investigation and is available elsewhere (e.g., Goldstein,
1995; Hox, 2002; Raudenbush & Bryk, 2002; Snijders & Bosker,
1999).
When aggregated individual data are used to assess effects of
group characteristics, the observed group average score (e.g., ag-
gregated student ratings of homework quality) may be a rather
unreliable measure of the unobserved “true” group mean. As has
been shown, the reliability of these aggregated ratings depends on
the number of students per class and the extent to which students’
ratings vary across classes (Lu¨ dtke, Trautwein, Kunter, &
Baumert, 2006; Raudenbush & Bryk, 2002). We adopted the
multilevel latent covariate approach (MLC) implemented in Mplus
to control for the unreliability of aggregated student ratings of their
learning environment (Lu¨dtke et al., 2008; Muthe´ n & Muthe´n,
1998 –2007). The MLC approach corrects for the unreliable as-
sessment of the group mean when estimating group effects by
taking into account that only a finite number of students provided
ratings of the quality of their homework. Lu¨dtke et al. (2008)
suggested that research designs involving student ratings of the
learning environment are ideally suited to the MLC approach,
because each student’s perception of homework quality reflects a
construct at the class level (i.e., quality of the homework assigned
to a class). Thus, variation within each class can be regarded to
some extent as unreliability in the measurement of homework
quality. In the present study, a series of multilevel models were
specified using Mplus 5.1 (Muthe´n & Muthe´ n, 1998 –2007) to
predict homework motivation, homework behavior, and mathe-
matics achievement. Correlations and residual correlations were
freely estimated; thus, all models were saturated.
Centering student-level predictor variables. One critical is-
sue in multilevel modeling is the centering of student-level pre-
dictor variables (see Enders & Tofighi, 2007; Kreft, de Leeuw, &
Aiken, 1995). Student ratings of the learning environment can be
adjusted either to the cluster to which the student belongs (group-
mean centering) or to the mean ratings of the whole sample
(grand-mean centering). The decision to center student-level pre-
dictors at the group mean or the grand mean can affect the
interpretation of the parameters estimated (Enders & Tofighi,
2007) and must be driven by the research questions addressed. In
the present study, individual and aggregated ratings of the learning
environment (e.g., homework selection, homework challenge)
were entered simultaneously as predictors in the multilevel models
(see Lu¨dtke et al., 2009). Grand-mean centering would control for
interindividual differences in student ratings among classes and
would thus eliminate an essential component of the aggregated
student ratings. Because we were primarily interested in the effects
of homework selection and homework challenge as features of the
learning environment, we treated both classroom features as class-
level variables and decided to center student ratings at their group
mean. This approach allows us to differentiate between-class from
within-class variation in perceived homework characteristics (e.g.,
Karabenick, 2004). The other Level 1 predictor variables, which
are primarily defined at the individual level (e.g., value and ex-
pectancy beliefs), were centered at the grand mean. Thus, interin-
dividual differences among students are taken into account when
estimating effects of homework assignment at the class level.
Missing values. The present analyses are part of a larger
assessment, the PISA-I-Plus study. Due to time constraints, the
students in a class were randomly administered different versions
of the questionnaire. All students were administered the items
tapping homework motivation and homework behavior, but the
homework selection and homework challenge items were admin-
istered in only one of two booklets. Thus, approximately 50% of
the homework selection and homework challenge data are missing
by design. Planned missing data designs (Graham, Taylor, Ol-
chowski, & Cumsille, 2006) are well established as a research
strategy and have been applied in several large-scale assessment
studies to increase cost effectiveness and design efficiency (Gra-
ham et al., 2006). The average percentage of missing data was
otherwise 3.5%.
In the methodological literature on missing data (Peugh &
Enders, 2004; Schafer & Graham, 2002), there is growing consen-
sus that multiple imputation of missing data is superior to tradi-
tional pairwise and listwise deletion methods. Even when 50% of
the data are missing by design, methods such as multiple imputa-
tion allow researchers to obtain reliable parameter estimates and
standard errors. In multiple imputation, missing values are pre-
dicted from the observed values of each participant, with random
noise added to maintain a correct amount of variability in the
imputed data (Schafer & Graham, 2002). According to Schafer and
Graham (2002), estimation of five values provides highly suffi-
cient estimates for a moderate amount of missing data; increasing
the number of estimations increases the accuracy only marginally.
We therefore produced five data sets in which missing data were
replaced with values estimated by the PAN algorithm implemented
in the R software (Schafer, 2008). The PAN algorithm was devel-
oped to impute multivariate panel data or clustered data. PAN uses
a multivariate extension of a two-level linear regression model
472 DETTMERS, TRAUTWEIN, LU
¨DTKE, KUNTER, AND BAUMERT
commonly applied to multilevel data (Schafer, 2001). Each im-
puted data set was analyzed separately, and the resulting estimates
were combined using the formulas given by Rubin (1987) and
implemented in the Mplus software.
Results
Descriptives and Zero-Order Correlations
Table 1 presents means, standard deviations, and missing values
for the variables analyzed. Students’ average ratings of homework
effort were close to the scale midpoint. On average, students
reported typically spending 2.60 hr per week on their mathematics
homework assignments. Homework motivation ratings were mod-
erately above the scale midpoint (expectancy: M⫽2.96; value:
M⫽3.04), indicating rather high motivation levels. Finally, the
students perceived their mathematics homework assignments as
somewhat difficult (homework challenge: M⫽2.25) and the
quality of homework selection of middle to high quality (home-
work selection: M⫽2.69).
Table 2 presents zero-order correlations among the variables
analyzed. To take the multilevel structure into account, we esti-
mated all correlations using the Mplus option “type ⫽complex.”
As shown in Table 2, we found differential associations between
the homework characteristics and the achievement indicators.
Homework selection was positively associated with homework
expectancy and value beliefs, homework effort, and homework
time. Moreover, there was a positive correlation with achievement
at T2. Homework challenge was negatively associated with home-
work expectancy and value beliefs, homework effort, and mathe-
matics achievement at both points of measurement.
Homework Quality: A Class-Level Variable?
One major precondition for using student ratings of the
classroom environment is that the aggregated ratings must be
sufficiently reliable. In multilevel modeling, the reliability of
aggregated individual student judgments is estimated by the
intraclass correlation coefficients ICC
1
and ICC
2
(Bliese, 2000;
Raudenbush & Bryk, 2002). The ICC
1
reflects the proportion of
variance attributable to differences between classes; the higher
the ICC
1
, the more similar are the ratings of the students in a
given class. The ICC
1
for homework selection was .18; that for
homework challenge was .11. Thus, 18% of the variance in
homework selection and 11% of the variance in homework
challenge was located between classes. These findings are
consistent with those of prior studies based on students’ ratings
of their learning environment (e.g., Frenzel, Pekrun, & Goetz,
2007; Kunter, Baumert, & Ko¨ ller, 2007). The somewhat lower
ICC
1
for homework challenge indicates larger differences in
individual perceptions of this variable than of homework selec-
tion. Perceptions of homework challenge may be more depen-
dent on interindividual differences in, for instance, cognitive
abilities or prior knowledge. Whereas the ICC
1
indicates the
reliability of an individual student’s rating, the ICC
2
provides
an estimate of the reliability of the class-mean rating. It is
calculated by applying the Spearman-Brown prophecy formula
(Nunnally, 1978) to the ICC
1
. A satisfactory ICC
2
is a neces-
sary precondition for detecting associations between variables
at the class level (see Bliese, 2000; Lu¨ dtke et al., 2006).
Drawing on the classical test theory literature, we regarded .70
as a reasonable lower bound for acceptable reliability of aggre-
gated ratings. The ICC
2
for homework selection was .83; that
for homework challenge was .74, indicating satisfactory reli-
ability of the class-mean ratings.
In addition, we examined the agreement among the students
in each class on homework selection and challenge. The aver-
age deviation index (AD
M
) proposed by Burke, Finkelstein, and
Dusig (1999) and Burke and Dunlap (2002) indicates individual
students’ deviation from the class mean. A cutoff point of .73
has been proposed for a 4-point Likert scale with more than 20
raters per cluster, with lower values indicating better agreement
(see Smith-Crowe & Burke, 2003). Averaging the single AD
M
indices across classes indicated sufficient agreement between
students within classes on homework selection (M⫽0.63;
mean AD
M
value across classes) and homework challenge (M⫽
0.69). In sum, computation of the ICC
1
, ICC
2
, and AD
M
con-
firmed the reliability and within-group agreement of the aggre-
gated student perceptions of the learning environment, indicat-
ing that it is appropriate to use the aggregated data as class-level
variables.
Predicting Homework Motivation, Homework
Behavior, and Mathematics Achievement
Using Mplus 5.1, we specified several multilevel models to
test our hypotheses. Tables 3 to 5 present the results. In all
models, we controlled for potential confounding variables and
Table 1
Means, Standard Deviations, and Percentage of Missing Values
Variable NMSD% Missing
Class level
Gymnasium
a
155 36.77 0.49 0.00
Mittelschule
a
155 11.61 0.31 0.00
Realschule
a
155 43.87 0.50 0.00
Comprehensive schools 155 7.74 0.24 0.00
HW selection 155 2.69 0.34 0.00
HW challenge 155 2.25 0.31 0.00
Student level
Achievement T1
b
3,462 0.23 0.87 0.60
Achievement T2
b
3,324 571.22 79.44 4.57
Reading achievement 1,831 0.35 1.12 47.43
Cognitive abilities 3,436 0.26 1.35 1.35
Socioeconomic status 3,407 52.83 15.96 2.18
Immigration status 3,483 0.80 0.40 0.00
Male 3,483 0.43 0.50 0.00
HW time 3,390 2.60 1.84 2.67
HW effort 3,408 2.69 0.63 2.15
HW value 3,407 3.04 0.83 2.18
HW expectancy 3,408 2.96 0.70 2.15
HW selection 1,864 2.69 0.68 46.48
HW challenge 1,854 2.25 0.70 46.77
Note. HW ⫽homework.
a
Means are based on dummy coded school indicators. The means refer to
the population of students in the specific school track.
b
Means are above
the midpoint because students attending the academically least demanding
track were excluded from all analyses.
473
HOMEWORK WORKS IF HOMEWORK QUALITY IS HIGH
mathematics achievement at T1. M1 models present the rela-
tionships between homework motivation (expectancy and value
beliefs), homework behavior (effort and time spent on home-
work), and mathematics achievement. M2 models additionally
include the homework characteristics task selection and chal-
lenge (both class-mean centered) but exclude the mediator
variables homework expectancy and value beliefs (Tables 4
and 5). M3 models include all variables, allowing us to inves-
Table 2
Intercorrelations Among All Variables
Variable 1 2 3 4 5 678910111213141516
1. Mittelschule (lower track) —
2. Gymnasium (highest track) ⫺0.27 —
3. Comprehensive schools ⫺0.09 ⫺0.20 —
4. Achievement T1 ⫺0.16 0.43 ⫺0.19 —
5. Achievement T2 ⫺0.16 0.43 ⫺0.17 0.49 —
6. Reading achievement ⫺0.18 0.33 ⫺0.17 0.27 0.25 —
7. Cognitive abilities ⫺0.14 0.36 ⫺0.17 0.36 0.39 0.19 —
8. SES ⫺0.16 0.35 ⫺0.10 0.06 0.09 0.04 0.03 —
9. Migration status ⫺0.07 ⫺0.11 0.06 ⫺0.05 ⫺0.04 ⫺0.10 ⫺0.0
a
⫺0.18 —
10. Male 0.03 ⫺0.01 0.03 0.16 0.20 0.00
a
0.07 0.06 ⫺0.03 —
11. HW time ⫺0.07 ⫺0.05 ⫺0.02 ⫺0.14 ⫺0.19 ⫺0.06 ⫺0.13 ⫺0.06 0.04 ⫺0.18 —
12. HW effort ⫺0.01
a
⫺0.05 ⫺0.02 0.17 0.11 0.13 0.07 0.00
a
⫺0.01
a
⫺0.06 0.15 —
13. HW value ⫺0.06 0.01 ⫺0.01 ⫺0.01
a
⫺0.05 0.01
a
⫺0.03 ⫺0.02
a
0.04 ⫺0.08 0.14 0.26 —
14. HW expectancy 0.00
a
⫺0.04 0.02 0.20 0.12 0.07 0.10 0.05 0.01
a
0.11 ⫺0.03 0.29 0.17 —
15. HW selection ⫺0.01
a
⫺0.04 ⫺0.01 0.03 ⫺0.02 0.00
a
⫺0.02
a
⫺0.04 0.06 0.00
a
0.10 0.20 0.40 0.28 —
16. HW challenge 0.00
a
⫺0.05 ⫺0.02 0.26 0.24 0.16 0.20 0.06 ⫺0.03 0.15 ⫺0.17 0.24 0.07 0.37 0.15 —
Note. N ⫽3,483. The Mplus option “type ⫽complex” was used to correct for clustering effects. All items and variables were z-standardized before the
correlations were calculated. Correlations ⬎.023 are statistically significant at p⬍.05. SES ⫽socioeconomic status; HW ⫽homework.
a
Nonsignificant correlations.
Table 3
Predicting Homework Expectancy and Homework Value: Results From Multilevel Modeling
Variable
HW Expectancy (B) HW Value (B)
M1 M2 M1 M2
Level 2: Classes
School type (Reference category: middle track)
Mittelschule (lower track) 0.01 ⫺0.01 ⫺0.21
ⴱ
⫺0.14
Gymnasium (highest track) ⫺0.24
ⴱⴱⴱ
⫺0.10 ⫺0.03 ⫺0.01
Comprehensives 0.07 0.06 ⫺0.08 0.00
HW selection 0.10
ⴱ
0.48
ⴱⴱⴱ
HW challenge ⫺0.35
ⴱⴱⴱ
0.03
Level 1: Students
Male 0.19
ⴱⴱⴱ
0.11
ⴱⴱⴱ
⫺0.16
ⴱⴱⴱ
⫺0.18
ⴱⴱⴱ
SES 0.01 0.00 ⫺0.01 ⫺0.02
Cognitive abilities 0.06
ⴱⴱ
0.04 ⫺0.01 0.00
Achievement T1 0.06
ⴱⴱ
0.02 ⫺0.02 ⫺0.02
Reading achievement T1 0.02 ⫺0.01 0.04 0.02
Immigration status 0.14 0.07 0.18 0.03
PEB 1 0.03 ⫺0.01 ⫺0.08 ⫺0.07
PEB 2 0.06 0.04 0.03 ⫺0.01
PEB 3 0.01 ⫺0.01 0.04 0.00
PEB 5 0.17
ⴱⴱ
0.12 0.04 0.03
PEB 6 ⫺0.07 ⫺0.02 ⫺0.05 ⫺0.05
HW selection 0.24
ⴱⴱⴱ
0.43
ⴱⴱⴱ
HW challenge ⫺0.32
ⴱⴱⴱ
⫺0.03
R
2
Level 2 0.00 52.94 9.72 65.28
R
2
Level 1 3.02 18.73 1.39 17.70
Note. N ⫽3,483. M1 ⫽models without homework characteristics; M2 ⫽models with homework characteristics; HW ⫽homework; SES ⫽
socioeconomic status; PEB ⫽parental educational background (reference category: PEB 4); PEB 1 ⫽no apprenticeship, with or without Hauptschule
certificate (lowest parental educational background); PEB 2 ⫽apprenticeship, with or without Hauptschule certificate; PEB 3 ⫽apprenticeship and
Realschule certificate; PEB 4 ⫽Hauptschule or Realschule certificate and technical college; PEB 5 ⫽technical college or Gymnasium certificate, no higher
education; PEB 6 ⫽degree qualification (highest educational background).
ⴱ
p⬍.05.
ⴱⴱ
p⬍.01.
ⴱⴱⴱ
p⬍.001.
474 DETTMERS, TRAUTWEIN, LU
¨DTKE, KUNTER, AND BAUMERT
tigate the mediating role of homework motivation (Tables 4 and
5). The results are presented in two sections. First, we present
the results for homework expectancy and value beliefs, home-
work effort, and homework time; second, we describe the
findings for mathematics achievement.
Predicting homework motivation and homework behavior.
In a first step (see Table 3), we specified multilevel models
predicting homework expectancy beliefs (first two columns of
Table 3) and homework value beliefs (last two columns of Table
3). We first describe our findings for homework expectancy be-
liefs. We expected homework selection to positively predict ex-
pectancy beliefs at the student and the class level. Moreover, we
expected to find a negative association between homework chal-
lenge and expectancy beliefs at both analytical levels. As shown in
Table 3, homework expectancy beliefs were associated with T1
mathematics achievement and gender. At the class level, relative to
the reference category Realschule, we found Gymnasium students
to report lower expectancy beliefs (Gymnasium:M⫽2.92, SD ⫽
0.72; Realschule:M⫽2.99, SD ⫽0.70; Mittelschule:M⫽2.96,
SD ⫽0.67; comprehensive school: M⫽3.02, SD ⫽0.59).
Explained variance
3
was low at both the class level (0.00%) and
the student level (3.02%; M1). In the next step (M2), homework
characteristics were included in the model. As shown in Table 3,
at the student level a high rating of the quality of homework
selection was indeed associated with high homework expectancy
beliefs. This result is in line with recent findings of a positive
relationship between homework quality and homework motivation
(see Trautwein, Lu¨dtke, Schnyder, & Niggli, 2006). Moreover, in
line with parts of our second hypothesis, homework challenge was
negatively related to homework expectancy beliefs at the student
level. Hence, students who perceived their homework assignments
to be demanding had less belief in being able to complete them
than did other students. At the class level, we found homework
selection ratings to be positively related to homework expectancy
beliefs, whereas homework challenge was negatively related to
homework expectancy beliefs. Hence, students in classes with
higher average perceptions of homework demands had lower
homework expectancy beliefs than did students in other classes.
Inclusion of homework task selection and homework challenge in
the model considerably increased the amounts of variance ex-
plained at the class level (52.94%) and moderately increased the
amounts explained at the student level (18.73%).
3
Explained variance was computed using the variance components of
the unconditional means model and the residual variances estimated by
Mplus.
Table 4
Predicting Homework Time and Homework Effort: Results From Multilevel Modeling
Variable
HW time T1 (B) HW effort T1 (B)
M1 M2 M3 M1 M2 M3
Level 2: Classes
School type (Reference category: middle track)
Mittelschule (lower track) ⫺0.30
ⴱⴱⴱ
⫺0.27
ⴱⴱⴱ
⫺0.26
ⴱⴱⴱ
⫺0.02 ⫺0.04 ⫺0.01
Gymasium (highest track) 0.05 ⫺0.06 ⫺0.06 ⫺0.21
ⴱⴱⴱ
⫺0.21
ⴱⴱⴱ
⫺0.20
ⴱⴱⴱ
Comprehensive schools ⫺0.28
ⴱ
⫺0.23 ⫺0.24
ⴱ
⫺0.03 0.00 ⫺0.02
HW selection 0.23
ⴱⴱ
0.16 0.28
ⴱⴱⴱ
0.19
ⴱⴱ
HW challenge 0.34
ⴱⴱⴱ
0.39
ⴱⴱⴱ
⫺0.10 ⫺0.02
Level 1: Students
Male ⫺0.27
ⴱⴱⴱ
⫺0.26
ⴱⴱⴱ
⫺0.25
ⴱⴱⴱ
⫺0.17
ⴱⴱⴱ
⫺0.21
ⴱⴱⴱ
⫺0.20
ⴱⴱⴱ
SES ⫺0.01 ⫺0.01 ⫺0.01 ⫺0.03 ⫺0.04 ⫺0.03
Cognitive abilities ⫺0.07
ⴱⴱⴱ
⫺0.06
ⴱⴱ
⫺0.06
ⴱⴱ
0.00 0.00 ⫺0.01
Achievement T1 ⫺0.14
ⴱⴱⴱ
⫺0.12
ⴱⴱⴱ
⫺0.12
ⴱⴱⴱ
0.08
ⴱⴱ
0.06
ⴱⴱ
0.06
ⴱⴱ
Reading achievement T1 ⫺0.03 ⫺0.02 ⫺0.02 0.08
ⴱⴱⴱ
0.07
ⴱⴱⴱ
0.07
ⴱⴱⴱ
Migration status 0.08 0.06 0.06 0.09 0.11 0.09
PEB 1 0.05 0.06 0.07 0.22
ⴱⴱⴱ
0.19
ⴱ
0.20
ⴱⴱ
PEB 2 ⫺0.04 ⫺0.05 ⫺0.05 0.02 0.03 0.03
PEB 3 0.06 0.05 0.05 0.09 0.08 0.08
PEB 5 ⫺0.05 ⫺0.03 ⫺0.03 0.11
ⴱ
0.12
ⴱ
0.09
PEB 6 ⫺0.06 ⫺0.09 ⫺0.09 0.06 0.06 0.07
HW selection 0.12
ⴱⴱⴱ
0.07
ⴱⴱ
0.19
ⴱⴱⴱ
0.06
ⴱ
HW challenge 0.13
ⴱⴱⴱ
0.14
ⴱⴱⴱ
⫺0.20
ⴱⴱⴱ
⫺0.13
ⴱⴱⴱ
Value 0.13
ⴱⴱⴱ
0.11
ⴱⴱⴱ
0.23
ⴱⴱⴱ
0.20
ⴱⴱⴱ
Expectancy ⫺0.01 0.03 0.26
ⴱⴱⴱ
0.20
ⴱⴱⴱ
R
2
Level 2 29.41 47.06 50.00 51.43 54.29 62.86
R
2
Level 1 8.21 9.06 10.02 15.38 10.29 17.05
Note. N ⫽3,483. M1 ⫽models without homework characteristics; M2 ⫽models without homework expectancies and homework value; M3 ⫽complete
models; HW ⫽homework; SES ⫽socioeconomic status; PEB ⫽parental educational background (reference category: PEB 4); PEB 1 ⫽no apprenticeship,
with or without Hauptschule certificate (lowest parental educational background); PEB 2 ⫽apprenticeship, with or without Hauptschule certificate;
PEB 3 ⫽apprenticeship and Realschule certificate; PEB 4 ⫽Hauptschule or Realschule certificate and technical college; PEB 5 ⫽technical college or
Gymnasium certificate, no higher education; PEB 6 ⫽degree qualification (highest educational background).
ⴱ
p⬍.05.
ⴱⴱ
p⬍.01.
ⴱⴱⴱ
p⬍.001.
475
HOMEWORK WORKS IF HOMEWORK QUALITY IS HIGH
The next two columns in Table 3 present the results for the
prediction of homework value beliefs. We expected to find a
positive association between homework selection and value beliefs
at the student and the class level. No specific hypotheses were
formulated for the predictive power of homework challenge. In
Model 1, we found that male students reported higher value beliefs
than other students. Moreover, Mittelschule students reported
lower value beliefs than Realschule students (Gymnasium:M⫽
3.04, SD ⫽0.78; Realschule:M⫽3.06, SD ⫽0.85; Mittelschule:
M⫽2.90, SD ⫽0.89; comprehensive school: M⫽3.03, SD ⫽
0.86). Explained variance was low at both the class level (9.27%)
and the student level (1.39%; M1). The last column in Table 3
presents the results after the inclusion of homework selection and
homework challenge into the model. Consistent with parts of our
first and third hypotheses, students who perceived that their home-
work assignments were well selected reported higher value beliefs
than did other students. Furthermore, students in classes where
homework was generally perceived to be well selected scored
higher on the value beliefs scale than did students in other classes.
Finally, no statistically significant association was found between
homework challenge and value beliefs. The variance explained
increased in Model 2 at both the class (65.28%) and the student
level (17.70%).
Table 4 presents the results for predicting homework time and
homework effort; we first describe the findings for homework
time. In Model 1, being male, having high cognitive abilities, and
having high achievement at T1 negatively predicted time on home-
work. Conversely, high value beliefs were positively related to
homework time. At the class level, Mittelschule and comprehen-
sive school students reported statistically significantly shorter
homework times than did Realschule students (Gymnasium:M⫽
2.49, SD ⫽1.76; Realschule:M⫽2.79, SD ⫽1.86; Mittelschule:
M⫽2.26, SD ⫽1.19; comprehensive school: M⫽2.45, SD ⫽
1.86). The amount of variance explained was low at the student
level (8.21%) and moderate at the class level (29.41%). When
homework expectancy and value beliefs were replaced by the two
homework characteristics in Model 2, homework selection
was—as expected—positively associated with homework time at
both levels of analysis. Moreover, homework challenge was like-
wise positively associated with homework time at both levels.
Thus, students who perceived their homework assignments to be
well selected and cognitively demanding spent more time com-
Table 5
Predicting Mathematics Achievement: Results From Multilevel Modeling
Variable
Achievement (B)
M1 M2 M3
Level 2: Classes
School type (reference category: middle track)
Mittelschule (lower track) ⫺0.07 ⫺0.01 ⫺0.04
Gymasium (highest track) 0.33
ⴱⴱⴱ
0.20
ⴱⴱⴱ
0.26
ⴱⴱⴱ
Comprehensive schools ⫺0.23
ⴱ
⫺0.16
ⴱ
⫺0.20
ⴱ
HW selection 0.21
ⴱⴱⴱ
0.19
ⴱ
HW challenge 0.25
ⴱⴱⴱ
0.27
ⴱⴱ
Level 1: Students
Male 0.10
ⴱⴱⴱ
0.09
ⴱⴱⴱ
0.09
ⴱⴱⴱ
SES ⫺0.01 ⫺0.01 ⫺0.01
Cognitive abilities 0.18
ⴱⴱⴱ
0.17
ⴱⴱⴱ
0.17
ⴱⴱⴱ
Achievement T1 0.35
ⴱⴱⴱ
0.35
ⴱⴱⴱ
0.34
ⴱⴱⴱ
Reading achievement T1 0.13
ⴱⴱⴱ
0.13
ⴱⴱⴱ
0.13
ⴱⴱⴱ
Migration status ⫺0.08 ⫺0.06 ⫺0.07
PEB 1 ⫺0.04 ⫺0.04 ⫺0.05
PEB 2 ⫺0.02 0.00 ⫺0.01
PEB 3 0.06 0.07 0.06
PEB 5 0.07 0.07 0.06
PEB 6 0.04 0.05 0.04
HW selection 0.01 ⫺0.02
HW challenge ⫺0.10
ⴱⴱⴱ
⫺0.06
ⴱⴱⴱ
Value ⫺0.01 ⫺0.01
Expectancy 0.08
ⴱⴱⴱ
0.07
ⴱⴱⴱ
HW time ⫺0.03
ⴱ
⫺0.02
HW effort 0.07
ⴱⴱⴱ
0.06
ⴱⴱⴱ
R
2
Level 2 87.77 91.85 89.95
R
2
Level 1 32.19 30.99 32.48
Note. N⫽3,483. M1 ⫽models without homework characteristics; M2 ⫽models without homework expectancies
and homework value; M3 ⫽complete models; HW ⫽homework; SES ⫽socioeconomic status; PEB ⫽parental
educational background (reference category: PEB 4); PEB 1 ⫽no apprenticeship, with or without Hauptschule
certificate (lowest parental educational background); PEB 2 ⫽apprenticeship, with or without Hauptschule certifi-
cate; PEB 3 ⫽apprenticeship and Realschule certificate; PEB 4 ⫽Hauptschule or Realschule certificate and technical
college; PEB 5 ⫽technical college or Gymnasium certificate, no higher education; PEB 6 ⫽degree qualification
(highest educational background).
ⴱ
p⬍.05.
ⴱⴱ
p⬍.01.
ⴱⴱⴱ
p⬍.001.
476 DETTMERS, TRAUTWEIN, LU
¨DTKE, KUNTER, AND BAUMERT
pleting their homework than did other students. In Model 2, the
variance explained increased at the class level (47.06%) but re-
mained almost the same at the student level (9.06%). When the
mediator variables (expectancy and value beliefs) were reintro-
duced in Model 3, the regression coefficient for homework selec-
tion decreased at the student and the class level, revealing the
mediating role of homework motivation.
In terms of homework effort, in addition to several control
variables, homework motivation (expectancy and value beliefs)
positively predicted homework effort at the student level (M1).
Moreover, on average, Gymnasium students reported less home-
work effort than did Realschule students (Gymnasium:M⫽2.65,
SD ⫽0.64; Realschule:M⫽2.74, SD ⫽0.64; Mittelschule:M⫽
2.67, SD ⫽0.56; comprehensive school: M⫽2.66, SD ⫽0.56).
The variables in Model 1 explained 15.38% of the variance at the
student level and 51.43% of the variance at the class level. In line
with parts of our first and third hypotheses, Model 2 showed a
strong association between homework selection and homework
effort at the student and the class level. Hence, students who
perceived that their homework was well selected reported invest-
ing more effort in homework completion than did other students.
Moreover, students in classes with high average perceptions of
homework selection put more effort into homework completion
than did students in other classes. Furthermore, as indicated by the
decreased regression coefficient for homework selection (B⫽
0.06), the association between homework selection and homework
effort was largely mediated by homework expectancy and value
beliefs. With respect to homework challenge, however, we found
that students who perceived their homework assignments to be
cognitively demanding put less effort into homework completion
than other students (B⫽– 0.20). This result may be attributable to
low expectancy beliefs. Indeed, inclusion of the mediator variables
homework expectancy and value beliefs in Model 3 resulted in a
decreased regression coefficient for homework challenge. The
variance explained increased slightly at the class level (54.29%)
but decreased at the student level (10.29%) in Model 2, empha-
sizing the role of homework expectancy and value beliefs for
homework effort at the student level. In Model 3, the variance
explained increased moderately at the student level (17.05%) and
at the class level (62.86%).
In sum, we found homework selection and homework challenge
to be strongly related to homework motivation and homework
behavior. Including the two variables in the multilevel models
increased the amounts of variance, providing further support for
the view that student ratings of the classroom environment should
be analyzed within a multilevel framework.
Predicting mathematics achievement. Table 5 presents our
results for mathematics achievement. Controlling for potential
confounding variables and achievement at T1, we found home-
work expectancy beliefs and homework effort to positively predict
mathematics achievement at the student level (M1). Hence, stu-
dents who were confident in their ability to complete their home-
work assignments successfully and students who did their best to
complete their assignments scored higher in the mathematics
achievement test than did other students. In contrast, homework
time proved to be a negative predictor of student achievement,
indicating that long homework times might reflect inefficient study
habits. Gymnasium students showed greater achievement gains in
mathematics than did other students (Gymnasium:M⫽613.63,
SD ⫽68.79; Realschule:M⫽550.79, SD ⫽72.47; Mittelschule:
M⫽535.83, SD ⫽67.75; comprehensive school: M⫽512.12,
SD ⫽84.36). The variance explained was 32.19% at the student
level and 87.77% at the class level. In Model 2, homework task
selection and homework challenge were introduced as predictor
variables. Contrary to our first hypothesis, homework selection did
not positively predict mathematics achievement at the student
level. In line with our second hypothesis, however, homework
challenge was negatively related to mathematics achievement at
the student level to a statistically significant degree.
4
Thus, stu-
dents who perceived homework assignments to be cognitively
challenging showed lower achievement gains than did other stu-
dents. In line with our third and fourth hypotheses, both homework
quality indicators positively predicted mathematics achievement at
the class level. Students in classes with higher average perceptions
of homework selection showed greater achievement gains than did
students in other classes (B⫽0.19), as did students in classes with
higher average perceptions of homework challenge (B⫽0.27).
5
Thus, in general, high homework quality was positively related to
mathematics achievement. The amount of variance explained in
Model 2 increased slightly at the class level (91.85%) and re-
mained stable at the student level (30.99%). The inclusion of the
mediator variables in Model 3 led to a decreased regression coef-
ficient for homework challenge at the student level, pointing to the
mediating role of homework motivation and homework behavior.
Finally, we tested for cross-level interactions between student
characteristics (gender, cognitive abilities, socioeconomic status,
achievement at T1, reading achievement, homework challenge,
homework selection, homework value, homework expectancy, and
homework effort) and homework challenge. These additional anal-
yses revealed a statistically significant cross-level interaction be-
tween students’ cognitive abilities and homework challenge (B⫽
– 0.10, p⬍.05). Hence, challenging homework assignments are
less important for highly intelligent students than for their class-
mates.
4
It might be hypothesized that the relationship between homework
challenge and achievement is nonlinear, with an optimum level of home-
work challenge. However, an additional analysis did not reveal a statisti-
cally significant nonlinear relationship between the two variables.
5
In some additional analyses, we transformed the two homework quality
indicators (selection and challenge) to obtain results based on grand-mean
centering. For homework challenge, the resulting beta coefficient was ⫽
0.33; for homework selection we found a beta coefficient of ⫽0.21.
Hence, the direction of results for the present data did not change; on the
contrary, the beta coefficients increased slightly, indicating a rather stron-
ger relationship between the two homework quality variables and mathe-
matics achievement with grand-mean centering. These results show that
our findings are not sensitive to the centering decision. We further tested
whether this result might be due to the different school tracks. Using the
Mplus software with a multiple group multilevel model, we examined
whether the association between homework challenge and school track was
of the same magnitude in each track. The chi-square test did not reveal a
statistically significant difference in the relationship across the school
tracks.
477
HOMEWORK WORKS IF HOMEWORK QUALITY IS HIGH
Discussion
Homework Works
The primary aim of the present study was to analyze the effects
of homework quality on students’ learning and homework behav-
ior. Two indicators of homework quality were examined. Repli-
cating the results of recent studies (Trautwein & Lu¨dtke, 2007,
2009; Trautwein, Lu¨dtke, Schnyder, & Niggli, 2006), we found the
first indicator, homework selection, to be positively associated
with homework motivation (expectancy and value beliefs) and
homework behavior (effort and time). Moreover, we found first
evidence for a positive relationship between high-quality home-
work selection and mathematics achievement at the class level.
The second indicator, homework challenge, proved to be nega-
tively related to homework expectancy beliefs and homework
effort (the latter only at the student level). We found students in
classes with higher average perceptions of homework challenge
showed greater achievement gains than students in other classes,
even when prior knowledge was controlled. Finally, we found
differential effects for homework selection and homework chal-
lenge at the student and the class level, indicating that different
factors might operate at the individual and the class level. In the
following, we outline the main contributions of the study to home-
work research and describe possible educational implications. Fi-
nally, we identify the limitations of the present study and make
recommendations for future research.
Contribution of the Present Study to Homework
Research
Our findings supplement the existing research in several re-
spects. First, our study was based on a large, nationally represen-
tative sample of 3,483 students from 155 classes. Second, home-
work quality was measured on two scales: one assessing the
quality of homework selection and one assessing how cognitively
demanding homework assignments were perceived to be. Al-
though we found higher student agreement on homework selec-
tion, both homework quality scales proved to be reliable indicators
of the learning environment and can thus be used as student-level
and as class-level variables. The homework selection scale shares
similarities with scales used in prior research (see Trautwein &
Lu¨dtke, 2009); to our knowledge, however, the present study is the
first to address the role of perceived challenge of homework
assignments. Third, we examined the predictive power of the two
homework quality scales for mathematics achievement. Most pre-
vious studies have focused on the role of homework quality for
homework motivation and behavior (see Trautwein & Lu¨dtke,
2007, 2009; Trautwein, Lu¨dtke, Schnyder, & Niggli, 2006). We
were able to confirm the importance of homework selection and
homework challenge for student achievement at the class level,
even when controlling for prior knowledge and potential con-
founding variables. Hence, students in classes given well-chosen
and challenging homework assignments learn more than their
peers in other classes (teacher- or class-level effect).
A different picture emerges at the student level. We found no
statistically significant association between homework selection
and mathematics achievement. Moreover, we found that students
with high homework challenge ratings showed lower achievement
gains than other students. At first glance, these results seem
somewhat counterintuitive. However, it is well established in the
literature on multilevel modeling that variables can show different
relations at different levels of analysis (Raudenbush & Bryk,
2002). For example, Trautwein and colleagues (e.g., Trautwein et
al., 2002; Trautwein & Lu¨dtke, 2007) have found time spent on
homework to be negatively related to achievement at the student
level but positively related at the class level. A plausible explana-
tion is that measures of homework time typically conflate total
time and active time. Thus, an individual student’s report of high
homework time is not necessarily a sign of great studiousness but
may reflect problems of motivation or concentration. At the class
level, however, reports of high homework time indicate that the
teacher sets frequent or long homework assignments. In a similar
vein, student ratings of homework challenge can be interpreted in
two ways: First, students’ individual perceptions of homework
challenge can be assumed to be affected by their prior knowledge
and cognitive abilities. In other words, they are not only a function
of the homework assigned by the teacher, but also reflect individ-
ual differences among students. Second, at the class level, the
class-average response can be interpreted as students’ shared per-
ception of homework challenge, in which individual idiosyncrasies
are averaged out. Class-average ratings of homework challenge
thus reflect differences among classes in the homework assigned.
In sum, the differential effects found at the student and the class
level underline the importance of analyzing homework quality data
within a multilevel framework. Had we not differentiated between
the student and the class level, the two effects would have been
confounded, producing different results.
Fourth, student achievement was measured by a highly reliable
and valid assessment of mathematics achievement. Many previous
studies have used school grades as an indicator of student achieve-
ment. Grades seem problematic as outcome measures, however,
because they partly reflect individual student effort. Research that
relies on class grades as outcomes might thus overestimate the
influence of homework (Cooper, 1989).
Educational Implications
What are the implications of our findings for homework prac-
tice? Our study provides strong evidence that interesting and
well-selected homework assignments are associated with higher
expectancy and value beliefs and with higher homework effort and
that they are effective for learning. Teachers may thus be able to
improve the effectiveness of their instruction by optimizing the
homework assignments they set, and it may well be worth focusing
more on the quality of homework assignments in teacher training.
However, the picture that emerges for homework challenge is
more complex. We found that challenging homework assignments
were negatively related to homework expectancy beliefs and
homework effort and found differential effects for mathematics
achievement. In our view, the complexity of our results reflects the
difficulties teachers face in their daily routine: What is the ideal
balance between cognitively activating instruction (i.e., challeng-
ing homework assignments) and instruction that caters for the
low-achieving students in the class?
Given these results, how should homework assignments be
designed to enhance homework motivation, homework effort, and
achievement? Achievement motivation research suggests that
478 DETTMERS, TRAUTWEIN, LU
¨DTKE, KUNTER, AND BAUMERT
tasks of moderate difficulty are most likely to enhance student
motivation (Astleitner, 2007). Likewise, Good and Brophy (1990)
suggested that only homework assignments perceived as ade-
quately difficult elicit high value beliefs and high effort. From the
constructivist point of view, however, highly complex tasks can be
expected to be effective for learning (Astleitner, 2007)—provided
that students are adequately instructed (e.g., through scaffolding;
Anghileri, 2006). This hypothesis is in line with the homework
challenge effect we found at the class level, which showed that
challenging homework assignments generally foster achievement.
In sum, the current literature suggests that teachers should assign
adequately difficult tasks to improve students’ motivation and
effort and challenging but well-structured tasks to foster students’
performance.
But how should teachers go about designing homework tasks
that are both adequately difficult and challenging? One possibility
may be to assign individualized homework tasks that challenge but
do not overtax individual students. Moreover, assigning homework
tasks that match individual students’ interests may help to increase
motivation and homework effort and, at the same time, enhance
achievement. Such individualized assignments are rare in the Ger-
man school system, however (Rossbach, 1995; Schoenbrunn,
1989). Individualized assignments make intensive demands on
teachers’ time and resources. Moreover, they may in fact increase
within-class differences in achievement (see Trautwein & Ko¨ller,
2003). A further possibility would be to assign tasks that can be
solved by various methods of differing levels of complexity. Such
tasks have the potential to challenge— but not overchallenge—
most of the students in a class, depending on the approach chosen.
Limitations and Future Research
Several limitations of the present research must be mentioned.
First, student reports were our sole source of information, even on
shared aspects of the learning environment. Empirical studies
assessing characteristics of the learning environment may draw on
external observers, teacher reports, student reports, or a combina-
tion of data sources (Anderson, 1982; Fraser, 1991; Turner &
Meyer, 2000). Each perspective can be assumed to assess at least
slightly different aspects of the construct in question. A combina-
tion of methods might provide deeper insights into the effects of
homework quality for homework motivation, homework behavior,
and achievement in future research (De Jong & Westerhof, 2001;
Kunter & Baumert, 2006b).
Second, we were not able to address the issue of causation
satisfactorily. The homework model implies that teachers’ home-
work practices affect students’ homework motivation, homework
behavior, and achievement. However, we cannot rule out the
possibility that further confounding variables were omitted from
the present study: Unobserved predictor variables may also impact
homework behavior, homework motivation, and achievement. The
only way of addressing causality would be a carefully designed
intervention study in which teachers were assigned to different
treatments. To date, few studies have systematically evaluated
homework intervention programs (e.g., Perels, Gu¨rtler, & Schmitz,
2005; Zimmerman, Bonner, & Kovach, 1996). Intervention studies
could complement the present research, providing valuable in-
sights into the mechanisms of homework and its influence on
homework motivation, homework behavior, and achievement.
Third, it might be worthwhile for future research to assess
further aspects of homework quality. More specifically, items
tapping the opportunity for students to apply and combine different
strategies or to generate new ideas and items assessing the varia-
tion in tasks and their potential to challenge beliefs might help to
shed light on the association between the perceived level of chal-
lenge and value beliefs. However, it is unclear to what extent
students are able to evaluate such complex constructs.
Fourth, the generalizability of our results remains uncertain. Our
results apply specifically to ninth graders and to mathematics.
Previous research has identified differences in the homework–
achievement relationship across grades (Cooper, 1989). Future
studies should therefore analyze homework quality effects in dif-
ferent grade levels and in different subjects. It is also possible that
cultural differences affect the results. Cross-cultural studies are
thus also necessary to test the generalizability of the effects found
in the current study.
To conclude, the present study added to prior research by
demonstrating the importance of homework quality for student
achievement. Moreover, it confirmed the predictive power of
homework challenge as a further indicator of the learning envi-
ronment. In sum, the findings extend the scientific understanding
of the circumstances under which students invest effort in home-
work completion and of how homework assignments can enhance
student achievement in mathematics.
References
Adams, R., & Wu, M. (2002). PISA 2000: Technical report. Paris, France:
Organisation for Economic Cooperation and Development.
Aleamoni, L. M. (1999). Student rating myths versus research facts from
1924 to 1998. Journal of Personnel Evaluation in Education, 13, 153–
166.
Anderson, C. S. (1982). The search for school climate. Review of Educa-
tional Psychology, 52, 368 – 420.
Anghileri, J. (2006). Scaffolding practices that enhance mathematics learn-
ing. Journal of Mathematics Teacher Education, 9, 33–52.
Astleitner, H. (2007). Theory: Designing task-based learning sequences. A
categorical model of task attributes. In H. Astleitner & H.-J. Herber
(Eds.), Task- and standard-based learning: An instructional psychology
perspective (pp. 9 –34). Frankfurt am Main, Germany: Lang.
Bandura, A. (1998). Self-efficacy: The exercise of control. New York, NY:
Freeman.
Baumert, J., Blum, W., Neubrand, M., Klusmann, U., Brunner, M., Jordan,
A., . . . Lo¨ wen, K. (2006). Professionswissen von Lehrkra¨ ften, kognitiv
aktivierender Matehmatikunterricht und die Entwicklung von mathema-
tischer Kompetenz (COACTIV): Dokumentation der Erhebungsinstru-
mente [Professional competence of teachers, cognitively activating in-
struction, and the development of students’ mathematical literacy
(COACTIV): Documentation of the instruments]. Berlin, Germany: Max
Planck Institute for Human Development.
Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., . . .
Tsai, Y. (in press). Teachers’ mathematical knowledge, cognitive acti-
vation in the classroom, and student progress. American Educational
Research Journal.
Bliese, P. D. (2000). Within-group agreement, nonindependence, and re-
liability: Implications for data aggregation and analysis. In K. J. Klein &
S. W. Kozlowski (Eds.), Multilevel theory, research, and methods in
organizations (pp. 349 –381). San Francisco, CA: Jossey-Bass.
Bloom, B. S. (1976). Human characteristics and school learning. New
York, NY: McGraw-Hill.
Brophy, J., & Good, T. (1986). Teacher behavior and student achievement.
479
HOMEWORK WORKS IF HOMEWORK QUALITY IS HIGH
In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp.
328 –375). New York, NY: Macmillan.
Burke, M. J., & Dunlap, W. P. (2002). Estimating interrater agreement with
the average deviation index: A user’s guide. Organizational Research
Methods, 5, 159 –172.
Burke, M. J., Finkelstein, L. M., & Dusig, M. S. (1999). On average
deviation indices for estimating interrater agreement. Organizational
Research Methods, 2, 49 – 68.
Carroll, J. B. (1963). A model of school learning. Teachers’ College
Record, 64, 723–733.
Carroll, J. B. (1989). The Carroll model: A 25-year retrospective and
prospective view. Educational Researcher, 18, 26 –31.
Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic
studies. Cambridge, England: Cambridge University Press.
Cooper, H. (1989). Homework. White Plains, NY: Longman.
Cooper, H., Lindsay, J. J., Nye, B., & Greathouse, S. (1998). Relationships
among attitudes about homework, amount of homework assigned and
completed, and student achievement. Journal of Educational Psychol-
ogy, 90, 70 – 83.
Cooper, H., Robinson, J. C., & Patall, E. A. (2006). Does homework
improve academic achievement? A synthesis of research, 1987–2003.
Review of Educational Research, 76, 1– 62.
Corno, L. (1996). Homework is a complicated thing. Educational Re-
searcher, 25, 27–30.
Deci, E. L., & Ryan, R. M. (Eds.). (2002). Handbook of self-determination
research. New York, NY: University of Rochester Press.
De Jong, R., & Westerhof, K. J. (2001). The quality of student ratings of
teacher behavior. Learning Environments Research, 4, 51– 85.
Eccles, J. S. (1983). Expectancies, values, and academic choice: Origins
and changes. In J. Spence (Ed.), Achievement and achievement motiva-
tion (pp. 87–134). San Francisco, CA: Freeman.
Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and
goals. Annual Review of Psychology, 53, 109 –132.
Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in
cross-sectional multilevel models: A new look at an old issue. Psycho-
logical Methods, 12, 121–138.
Fraser, B. J. (1991). Two decades of classroom environment research. In
B. J. Fraser & H. J. Wahlberg (Eds.), Educational environments: Eval-
uation, antecedents and consequences (pp. 3–27). London, England:
Pergamon.
Frenzel, A., Pekrun, R., & Goetz, T. (2007). Perceived learning environ-
ment and students’ emotional experiences: A multilevel analysis of
mathematics classrooms. Learning and Instruction, 17, 478 – 493.
Ganzeboom, H. B. G., de Graaf, P. M., Treiman, D. J., & de Leeuw, J.
(1992). A standard international socio-economic index of occupational
status. Social Science Research, 21, 1–56.
Gettinger, M. (1989). Effects of maximizing time spent and minimizing
time needed for learning on pupil achievement. American Educational
Research Journal, 26, 73–91.
Goldstein, H. (1995). Multilevel statistical models. London, England:
Arnold.
Good, T. L., & Brophy, J. E. (1990). Educational psychology: A realistic
approach. White Plains, NY: Longman.
Graham, J. W., Taylor, B. T., Olchowski, A. E., & Cumsille, P. E. (2006).
Planned missing data designs in psychological research. Psychological
Methods, 11, 323–343.
Grolnick, W. S., & Slowiaczek, M. L. (1994). Parents’ involvement in
children’s schooling: A multidimensional conceptualization and moti-
vational model. Child Development, 65, 237–252.
Heller, K. A., & Perleth, C. (2000). Kognitiver Fa¨higkeitstest fu¨r 4.-12.
Klassen, Revision (KFT 4 –12 ⫹R) [Cognitive Abilities Test, Revised
Version (KFT 4 –12 ⫹R)]. Go¨ ttingen, Germany: Hogrefe.
Hox, J. (2002). Multilevel analysis: Techniques and applications. Mahwah,
NJ: Erlbaum.
Karabenick, S. A. (2004). Perceived achievement goal structure and col-
lege student help seeking. Journal of Educational Psychology, 96, 569 –
581.
Keith, T. Z., & Cool, V. A. (1992). Testing models of school learning:
Effects of quality of instruction, motivation, academic coursework, and
homework on academic achievement. School Psychology Quarterly, 7,
207–226.
Kreft, I. G. G., de Leeuw, J., & Aiken, L. S. (1995). The effect of differing
forms of centering in hierarchical linear models. Multivariate Behavioral
Research, 30, 1–21.
Kunter, M., & Baumert, J. (2006a). Linking TIMSS to research on learning
and instruction: A re-analysis of the German TIMSS and TIMSS video
data. In S. J. Howie & T. Plomp (Eds.), Contexts of learning mathemat-
ics and science: Lessons learned from TIMSS (pp. 335–351). London,
England: Routledge.
Kunter, M., & Baumert, J. (2006b). Who is the expert? Construct and
criteria validity of student and teacher ratings of instruction. Learning
Environments Research, 9, 231–251.
Kunter, M., Baumert, J., & Ko¨ller, O. (2007). Effective classroom man-
agement and the development of subject-related interest. Learning and
Instruction, 17, 494 –509.
Lipowsky, F., Rakoczy, K., Klieme, E., Reusser, K., & Pauli, C. (2004).
Hausaufgabenpraxis im Mathematikunterricht: Ein Thema fu¨r die Un-
terrichtsqualita¨tsforschung? [Homework practice in mathematics in-
struction: A topic for instructional quality research?]. In J. Doll & M.
Prenzel (Eds.), Bildungsqualita¨t von Schule: Lehrerprofessional-
isierung, Unterrichtsentwicklung und Schu¨lerfo¨rderungen als Strategien
derq Ualita¨tsverbesserung (pp. 250 –266). Mu¨nster, Germany: Wax-
mann.
Lu¨dtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T.,
& Muthe´n, B. (2008). The multilevel latent covariate model: A new,
more reliable approach to group-level effects in contextual studies.
Psychological Methods, 13, 203–229.
Lu¨dtke, O., Robitzsch, A., Trautwein, U., & Kunter, M. (2009). Assessing
the impact of learning environments: How to use student ratings of
classroom characteristics in multilevel modeling. Contemporary Educa-
tional Psychology, 34, 120 –131.
Lu¨dtke, O., Trautwein, U., Kunter, M., & Baumert, J. (2006). Reliability
and agreement of student ratings of the classroom environment: A
reanalysis of TIMSS data. Learning Environments Research, 9, 215–
230.
Marsh, H. W. (1996). Positive and negative global self-esteem: A substan-
tively meaningful distinction or artifacts? Journal of Personality and
Social Psychology, 70, 810 – 819.
Marsh, H. W. (2001). Distinguishing between good (useful) and bad
workload on students’ evaluations of teaching. American Educational
Research Journal, 38, 183–212.
Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of
teaching effectiveness effective: The critical issues of validity, bias, and
utility. American Psychologist, 52, 1187–1197.
Marsh, H. W., & Roche, L. A. (2000). Effects of grading leniency and low
workloads on students’ evaluations of teaching: Popular myth, bias,
validity or innocent bystanders? Journal of Educational Psychology, 92,
202–228.
Muthe´n, L. K., & Muthe´n, B. O. (1998–2007). MPlus user’s guide. Los
Angeles, CA: Muthe´n & Muthe´n.
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY:
McGraw-Hill.
Organisation for Economic Cooperation and Development. (2004a).
Learning for tomorrow’s world: First results from PISA 2003. Paris,
France: Author.
Organisation for Economic Cooperation and Development. (2004b). PISA
2003 technical report. Paris, France: Author.
Paschal, R. A., Weinstein, T., & Walberg, H. J. (1984). The effects of
480 DETTMERS, TRAUTWEIN, LU
¨DTKE, KUNTER, AND BAUMERT
homework on learning: A quantitative synthesis. Journal of Educational
Research, 78, 97–104.
Perels, F., Gu¨rtler, T., & Schmitz, B. (2005). Training of self-regulatory
and problem-solving competence. Learning and Instruction, 15, 123–
139.
Peugh, J. L., & Enders, C. K. (2004). Missing data in educational research:
A review of reporting practices and suggestions for improvement. Re-
view of Educational Research, 74, 525–556.
Pintrich, P. R. (2003). A motivational science perspective on the role of
student motivation in learning and teaching contexts. Journal of Edu-
cational Psychology, 95, 667– 686.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated
learning components of classroom academic performance. Journal of
Educational Psychology, 82, 33– 40.
Prenzel, M., Carstensen, C., Scho¨ps, K., & Maurischat, C. (2006). Die
Anlage des La¨ngsschnitts bei PISA 2003 [The longitudinal design of the
PISA 2003 study]. In M. Prenzel, J. Baumert, W. Blum, R. Lehmann, D.
Leutner, M. Neubrand, . . . U. Schiefele (Eds.), PISA 2003. Untersu-
chungen zur Kompetenzentwicklung im Verlauf eines Schuljahres [PISA
2003: Analyses of literacy development over a school year] (pp. 29 – 62).
Mu¨nster, Germany: Waxmann.
Prenzel, M., Drechsel, B., Carstensen, C. H., & Ramm, G. (2004). PISA
2003: Eine Einfu¨hrung [PISA 2003: An introduction]. In M. Prenzel, J.
Baumert, W. Blum, R. Lehmann, D. Leutner, M. Neubrand, . . . U.
Schiefele (Eds.), PISA 2003. Der Bildungsstand der Jugendlichen in
Deutschland—Ergebnisse des zweiten internationalen Vergleichs [PISA
2003: The educational attainment of young people in Germany—Results
of the second international comparison] (pp. 13– 46). Mu¨nster, Ger-
many: Waxmann.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models (2nd
ed.). Thousand Oaks, CA: Sage.
Robinson, W. S. (1950). Ecological correlations and the behavior of
individuals. American Sociological Review, 15, 351–357.
Rossbach, H.-G. (1995). Hausaufgaben in der Grundschule [Homework in
primary schools]. Die Deutsche Schule, 87, 103–112.
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New
York, NY: Wiley.
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the
facilitation of intrinsic motivation, social development, and well-being.
American Psychologist, 55, 68 –78.
Schafer, J. L. (2001). Multiple imputation with PAN. In L. M. Collins &
A. G. Sayer (Ed.), New methods for the analysis of change (pp. 357–
377). Washington, DC: American Psychological Association.
Schafer, J. L. (2008). The PAN package. Retrieved from http://cran
.r-project.org/web/packages/pan/pan.pdf
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state
of the art. Psychological Methods, 7, 147–176.
Schoenbrunn, G. (1989). Hausaufgaben in der paedagogischen Diskussion
[Homework in the educational debate]. Der Mathematikunterricht, 35,
5–21.
Smith-Crowe, K., & Burke, M. J. (2003). Interpreting the statistical sig-
nificance of observed AD interrater agreement values: Correction to
Burke and Dunlap (2002). Organizational Research Methods, 6, 129 –
131.
Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An intro-
duction to basic and advanced multilevel modeling. London, England:
Sage.
Thorndike, R. L., & Hagen, E. (1993). The Cognitive Abilities Test: Form
5. Itasca, IL: Riverside.
Trautwein, U. (2007). The homework–achievement relation reconsidered:
Differentiating homework time, homework frequency, and homework
effort. Learning and Instruction, 17, 372–388.
Trautwein, U., & Ko¨ller, O. (2003). The relationship between homework
and achievement: Still much of a mystery. Educational Psychology
Review, 15, 115–145.
Trautwein, U., Ko¨ller, O., Schmitz, B., & Baumert, J. (2002). Do home-
work assignments enhance achievement? A multilevel analysis in 7th
grade mathematics. Contemporary Educational Psychology, 27, 26 –50.
Trautwein, U., & Lu¨dtke, O. (2007). Students’ self-reported effort and time
on homework in six school subjects: Between-students differences and
within-student variation. Journal of Educational Psychology, 99, 432–
444.
Trautwein, U., & Lu¨dtke, O. (2009). Predicting homework motivation and
homework effort in six school subjects: The role of person and family
characteristics, classroom factors, and school track. Learning and In-
struction, 19, 243–258.
Trautwein, U., Lu¨dtke, O., Kastens, C., & Ko¨ller, O. (2006). Effort on
homework in Grades 5–9: Development, motivational antecedents, and
the association with effort on classwork. Child Development, 77, 1094 –
1111.
Trautwein, U., Lu¨dtke, O., Marsh, H. W., Ko¨ller, O., & Baumert, J. (2006).
Tracking, grading, and student motivation: Using group composition and
status to predict self-concept and interest in ninth-grade mathematics.
Journal of Educational Psychology, 98, 788 – 806.
Trautwein, U., Lu¨dtke, O., Marsh, H. W., & Nagy, G. (2009). Within-
school social comparison: How students’ perceived standing of their
class predicts academic self-concept. Journal of Educational Psychol-
ogy, 101, 853– 866.
Trautwein, U., Lu¨dtke, O., Schnyder, I., & Niggli, A. (2006). Predicting
homework effort: Support for a domain-specific multilevel homework
model. Journal of Educational Psychology, 98, 438 – 456.
Trautwein, U., Niggli, A., Schnyder, I., & Lu¨dtke, O. (2009). Between-
teacher differences in homework assignments and the development of
students’ homework effort, homework emotions, and achievement. Jour-
nal of Educational Psychology, 101, 176 –189.
Turner, J. C., & Meyer, D. K. (2000). Studying and understanding the
instructional contexts of classrooms: Using our past to forge the future.
Educational Psychologist, 35, 69 – 85.
Walberg, H. J., & Paschal, R. A. (1995): Homework. In L. W. Anderson
(Ed.), International Encyclopaedia of Teaching and Teacher Education
(pp. 268 –271). Oxford, England: Elsevier.
Warton, P.-M. (2001). The forgotten voices in homework: Views of
students. Educational Psychologist, 36, 155–165.
Weinert, F. E., & Helmke, A. (1995a). Interclassroom differences in
instructional quality and interindividual differences in cognitive devel-
opment. Educational Psychologist, 30, 15–20.
Weinert, F. E., & Helmke, A. (1995b). Learning from wise Mother Nature
or Big Brother instructor: The wrong choice as seen from an educational
perspective. Educational Psychologist, 30, 135–142.
Weinert, F. E., Schrader, F.-W., & Helmke, A. (1989). Quality of instruc-
tion and achievement outcomes. International Journal of Educational
Research, 13, 895–914.
Xu, J. (2005). Purposes for doing homework reported by middle and high
school students. Journal of Educational Research, 99, 46 –55.
Zimmerman, B. J., Bonner, R., & Kovach, R. (1996). Developing self-
regulated learners: Beyond achievement to self-efficacy. Washington,
DC: American Psychological Association.
(Appendix follows)
481
HOMEWORK WORKS IF HOMEWORK QUALITY IS HIGH
Appendix
Homework Selection Scale
Our mathematics teacher often sets interesting homework as-
signments.
Our mathematics teacher knows what homework to set to help
us understand the material covered in the lesson.
Our mathematics homework assignments really help us to un-
derstand our mathematics lessons.
Our mathematics teacher almost always chooses homework
assignments really well.
Our mathematics homework assignments are always well inte-
grated into the lessons.
Homework Challenge Scale
Our mathematics homework assignments are often quite diffi-
cult and really make us think.
Our mathematics homework assignments are often too easy.
Our mathematics homework assignments are often too difficult.
Our mathematics homework assignments are usually fairly easy.
Received March 23, 2009
Revision received November 23, 2009
Accepted December 1, 2009 䡲
E-Mail Notification of Your Latest Issue Online!
Would you like to know when the next issue of your favorite APA journal will be available
online? This service is now available to you. Sign up at http://notify.apa.org/ and you will be
notified by e-mail when issues of interest to you become available!
482 DETTMERS, TRAUTWEIN, LU
¨DTKE, KUNTER, AND BAUMERT
