Content uploaded by Thomas L Good
Author content
All content in this area was uploaded by Thomas L Good on May 17, 2019
Content may be subject to copyright.
Content uploaded by Douglas A. Grouws
Author content
All content in this area was uploaded by Douglas A. Grouws on Oct 03, 2014
Content may be subject to copyright.
Journal
of
Educational Psychology
1979,
Vol.
71, No. 3,
355-362
The
Missouri Mathematics Effectiveness Project:
An
Experimental
Study
in
Fourth-Grade Classrooms
Thomas
L.
Good
and
Douglas
A.
Grouws
University
of
Missouri—Columbia
The
study
investigated
the
effectiveness
of an
experimental mathematics
teaching
program.
The
treatment program
was
primarily based upon
a
large,
naturalistic study
of
relatively
effective
mathematics teachers. Students were
tested
before
and
after with
a
standardized
test
and a
content
test
(posttest
only),
which
had
been designed
to
approximate
the
actual instructional con-
tent
that
students
had
received during
the
treatment.
Observational mea-
sures revealed
that
teachers generally implemented
the
treatment,
and
analy-
ses of
product
data
showed
that
students
of
treatment teachers generally out-
performed
those
of
control teachers
on
both
the
standardized
and
content
tests.
Since strong
efforts
were made
to
control
for
Hawthorne
effects,
it
seems reasonable
to
conclude
that
teachers
and/or
teaching
methods
can
exert
a
significant
difference
on
student
progress
in
mathematics.
The
purpose
of
this
study
was to
exam-
ine
the
effectiveness
of a
mathematics
teaching program
on
student
achievement.
The
behaviors comprising
the
program
evolved
largely
from
a
correlational study
of
effective
fourth-grade mathematics teachers.
The
focus
of the
program
was
entirely
on
instructional behavior. There
was no at-
An
earlier
version
of
this article
was
read
by the first
author
at the
annual
meeting
of the
American
Educa-
tional
Research
Association,
Toronto, Canada, 1978.
This research
was
partially supported
by
National
Institute
of
Education Grant
NIE-G-77-0003.
The
opinions
expressed herein
do not
necessarily reflect
the
position
or
policy
of the
National Institute
of
Education,
and no
official
endorsement should
be
inferred.
The
authors
also
acknowledge
the
support
provided
by the
Center
for
Research
in
Social Behavior, University
of
Missouri—Columbia,
and the
typing
of the
manuscript
by
Sherry Kilgore.
The
authors
also
want
to ac-
knowledge
colleagues
who
helped with
the
research:
Howard
Ebmeier,
Terrill Beckerman, Dianne Hunter,
Sharon Schneeberger,
and
Harris Cooper.
The
authors
would
also
like
to
acknowledge
the
support
of
participating
teachers
and
principals
in the
Tulsa
Public School System
and
Jack
Griffin,
Associate
Superintendent,
and J, W.
Hosey, Mathematics Coor-
dinator,
for
their consistent
encouragement
and
support
throughout
the
project.
Requests
for
reprints should
be
sent
to
Thomas
L.
Good, Center
for
Research
in
Social Behavior,
111
Stewart Road,
University
of
Missouri,
Columbia,
Mis-
souri
65211.
tempt
to
modify
the
mathematics
curric-
ulum.
Background Correlational
Study
For the
correlational
study,
a
school dis-
trict
was
chosen
in
which
the
elementary
schools
used
a
common
mathematics
text-
book
and the
student population
was
rela-
tively
homogeneous across schools. These
characteristics were
useful
because
the
purpose
of the
study
was to
make inferences
about desirable teacher behavior.
By
con-
trolling
for
differences
in
curriculum mate-
rials
and
student
home backgrounds,
it
would
be
possible
to
argue with greater
confidence
that
any
observed differences
in
student achievement
were
due to
teachers
and not to
context
effects.
The
second
step
was to
identify
a
group
of
teachers
who
were consistent
and
relatively
effective
or
ineffective
in
obtaining
student
achievement results.
To
estimate
teachers'
effectiveness,
residual
gain scores were
computed
for
their students using their
pretest
and
posttest
scores
on a
standardized
achievement
test
(the
student's
pretest
score
was
used
as a
covariate
in a
linear
model).
It was
deemed important
to
select
an ob-
servational sample
of
teachers
who
were
Copyright 1979
by the
American
Psychological
Association, Inc.
0022-0663/79/7103-0355100.75
355
356
THOMAS
L.
GOOD
AND
DOUGLAS
A.
GROUWS
consistent
over consecutive years
in
their
impact upon
student
achievement
and who
were also notably different
in
their impact.
That
is,
teachers
who
regularly
obtained
more (and teachers
who
obtained consider-
ably less) achievement
from
students than
did
other
teachers
who
taught
similar stu-
dents
under similar circumstances were
se-
lected
for
observation.
After
observing
target
teachers repeatedly,
a
behavioral profile
was
constructed
for the
relatively effective
and
ineffective
teachers.
A
set of
factors
that
consistently separated
the
more
and
less
effective
teachers emerged.
These
naturalistic findings were integrated
with
the
recent research
of
others
and
translated
into
an
instructional
program.
The
variables
that
were derived
from
the
large
correlational study
and
those
that
were
suggested
by the
experimental studies
of
others
are
noted
in
Table
1.
Detailed vari-
able
descriptors
may be
found elsewhere
(Good
et
al,
Note
1).
Method
Although certain
of the
instructional variables
had
been
tested
individually
in
other settings,
the
first
ex-
perimental
test
of the
entire
instructional
program
commenced
in the
fall
of
1977. With
the
active assis-
tance
of
administrators
and
principals
in the
Tulsa
Public
School system,
it was
possible
to
recruit
a
vol-
unteer
sample
of 40
classroom teachers
who
used
the
semidepartmental
plan.1
The
decision
was
made
to do
the
research
within
this
organizational
pattern because
it
afforded
a
classroom structure that
was
most com-
parable
with
the
classroom organization
in
which
the
correlational
research
was
conducted
(e.g.,
no
class-
rooms
that were completely
individualized).
Choice
of
this
structure also provided
a
rough control
for in-
structional time, since teachers
did not
keep
the
same
students
for the
entire day. Hence,
for
most
of the
teachers
there
was
pressure
to end the
mathematics
class
at a set
time,
and
reteaching later
in the day was
impossible.
Teacher Training
On
September
20, we met
with
all
teachers
and all
school
principals
who had
volunteered
to
participate
in
the
project. Fourth-grade teachers
who
taught
using
a
semidepartrnentalized
structure
were
invited
to
par-
ticipate
in the
project. (Eighty percent
of the
available
population
volunteered
for the
program.)
Most
of the
semidepartrnentalized schools were
in low
socioeco-
nomic
status
(SES) areas.
At
this
workshop,
all 40
teachers
were
told
that
the
program
was
largely based upon
an
earlier observation
of
relatively
effective
and
ineffective
fourth-grade
mathematics teachers.
Teachers
were
told
that
al-
though
we
expected
the
program
to
work,
the
earlier
research
was
correlational
and the
present project
was
a
test
of
those
ideas.
After
a
brief
introduction,
the
teachers
(drawn
from
27
schools)
and
their principals
were
divided into
two
groups: treatment
and
control.
Schools
were
used
as the
unit
for
random
assignment2
to
experimental conditions. This
was
done
to
eliminate
the
difficulties
that
would
doubtlessly
follow
by
implementing
the
treatment
in one
class
but not an-
other
in the
same school.
Teachers
in the
treatment group were given
an ex-
planation
of the
program (the training lasted
for 90
minutes).
After
the
training session, treatment
teachers
were
given
the
45-page manual along with
the
instructions
to
read
it and to
begin
to
plan
for
imple-
mentation.
In
this
manual definitions
and
rationales
were
presented
for
each
part
of the
lesson, along with
detailed descriptions
of how to
implement
the
teaching
ideas.
Space limitations prohibit
a
presentation
of the
def-
inition,
rationale,
and
teaching practice statement
for
each part
of the
lesson. However,
it is
useful
to
sum-
marize
the
distinctive
aspects
of the
treatment:
(a) The
program,
in
total, represents
a
system
of
instruction;
(b)
instructional activity
is
initiated
and
reviewed
in the
context
of
meaning;
(c)
students
are
prepared
for
each
lesson stage
so as to
enhance involvement
and to
mini-
mize
student
performance
errors;
(d) the
principles
of
distributed
and
successful
practice
are
built into
the
program;
(e)
teaching presentations
and
explanations
are
emphasized.
Two
weeks after
the
treatment
began
we
returned
to
meet
with
treatment teachers.
The
purpose
of
this
90-minute
meeting
was to
respond
to
questions
that
teachers
had
about
the
meaning
of
certain teaching
behaviors
and to
react
to any
difficulties
that
the
teachers might have encountered.
Thus,
the
treatment
consisted
of two
90-minute training sessions
and a
45-page
manual
that
detailed
the
treatment
and
pro-
vided
a
base
for
teacher reference
as
necessary.
Control
teachers were told
that
they
would
not get the
details
of the
instructional program until February 1978.
Furthermore,
they were told
that
it was
hoped
that
this
information
might
be
especially
useful
to
them
then
because
at
that time they
would
receive individual
in-
formation
about their
own
classroom behavior
and re-
fined
information about
the
program itself. Finally,
control
teachers
were told
that
their immediate role
in
the
project
was to
continue
to
instruct
in
their
own
style.
1
A
semidepartrnentalized structure calls
for
teachers
to
teach only
two or
three
different
subjects
a
day.
2
Using information provided
by
school officials,
an
attempt
was
made
to
match schools
in
terms
of
student
SES,
and
then
one
school
from
each pair
was
assigned
to the
experimental
condition.
In the
earlier correla-
tional
study,
teachers
used only
one
textbook.
In
this
project,
teachers used
one of two
textbooks,
and it was
impossible
to
match
on
both
SES
characteristics
and
textbooks.
MISSOURI MATHEMATICS EFFECTIVENESS PROJECT
357
Table
1
Summary
of
Key
Instructional
Behaviors
Daily Review
(first
8
minutes except Mondays)
(a)
review
the
concepts
and
skills
associated with
the
homework
(b)
collect
and
deal
with homework
assignments
(c)
ask
several mental computation exercises
Development (about
20
minutes)
(a)
briefly
focus
on
prerequisite skills
and
concepts
(b)
focus
on
meaning
and
promoting student understanding
by
using
lively
explanations, demonstrations, process
explanations,
illustrations,
etc.
(c)
assess
student
comprehension
(1)
using process/product questions (active interaction)
(2)
using controlled practice
(d)
repeat
and
elaborate
on the
meaning portion
as
necessary
Seatwork
(about
15
minutes)
(a)
provide uninterrupted successful practice
(b)
momentum—keep
the
ball
rolling—get
everyone
involved,
then sustain involvement
(c)
alerting—let
students
know their work
will
be
checked
at end of
period
(d)
accountability—check
the
students'
work
Homework Assignment
(a)
assign
on a
regular basis
at the end of
each math class except Fridays
(b)
should involve about
15
minutes
of
work
to be
done
at
home
(c)
should include
one or two
review problems
Special Reviews
(a)
weekly review/maintenance
(1)
conduct during
the
first
20
minutes each Monday
(2)
focus
on
skills
and
concepts covered during
the
previous
week
(b)
monthly review/maintenance
(1)
conduct every fourth Monday
(2)
focus
on
skills
and
concepts covered since
the
last monthly
review
Given
that
control teachers knew
that
the
research
was
designed
to
improve
student
achievement,
that
the
school district
was
interested
in the
research,
and
that
they were being observed,
we
feel
reasonably confident
that
a
strong Hawthorne control
was
created.
To the
extent
that
a
strong Hawthorne condition
was
created,
it
could
be
argued
that
differences
in
performance
be-
tween
control
and
treatment groups were
due to the
program,
not to
motivational variables. However,
at
the
other extreme,
it was not
intended
to
create
so
strong
a
"press"
that
teachers (because
of
concern)
would seek
out
information from
treatment
teachers
or
would
alter their instructional style trying
to
guess what
the
experimenters wanted.
If
control
teachers
changed
their instructional behavior,
then
differences could
be
due
to the
fact
that
they were using
a
poorly thought
out
or
inconsistent
pattern
of
instruction.
Process
and
Product Measures
The
treatment
program
started
on
October
3,
1977,
and was
terminated
on
January
25,
1978. During
the
course
of the
project
all 40
teachers (with
few
excep-
tions) were observed
on six
occasions. Observers col-
lected information using both low-
and
high-inference
process
measures (see Good
&
Grouws, 1977,
Note
2).
Students were administered
the
mathematics
subtest
of
a
standardized achievement
test
(Science Research
Associates; SRA,
Short
Form
E,
blue level)
in
late
September
and in mid
December
and a
content
test
in
mid
January
(a
mathematics
achievement
test
con-
structed
by
Robert
E.
Reys
at the
University
of
Missouri
that
measured
the
content
that
students
had
been
ex-
posed
to
during
the
program
of
research). Further-
more,
an
instrument measuring student
learning
styles
and
preferences
in
attitude toward mathematics
was
administered
in
September
and in
January.
Results
At
the
debriefing
session
in
February,
control
teachers
consistently
indicated
that
(a)
they
did
think more about mathematics
instruction this year than previously,
(b)
they
did not
feel
that
they
had
altered their
behavior
in any
major way,
and (c)
directly
or
indirectly they
had not
been exposed
to
program information. Hence,
the
Haw-
thorne control condition appeared
to
have
been
satisfactorily implemented.
Implementation
The
second finding
is
that
treatment
teachers implemented
the
program
reason-
ably well.
If one is to
argue
that
a
program
works
or is
responsible
for a
change,
it is
important
to
show
that
teachers
exhibited
358
THOMAS
L.
GOOD
AND
DOUGLAS
A.
GROUWS
many
more
of the
classroom behaviors
re-
lated
to the
treatment
than
did
control
teachers. There were only
2 of the 21
treatment teachers
who
exhibited uniformly
low
implementation scores. Development
appears
to be the
only variable
that
teachers,
as
a
group,
had
consistent trouble
in
imple-
menting.
The
reason
for the low
level
of
implementation
may be due to
teachers'
fo-
cusing
on the
many other teaching requests
that
were
perhaps
easier
to
implement.
Alternatively,
teachers might
not
have
had
the
knowledge base necessary
to
focus
on
development
for
relatively long periods
of
time. Another possibility
is
that
some
of the
other components required more time
and
preparation
than
we
anticipated
and
thus
development
was
given
insufficient
attention
by
the
teachers.
These
issues need further
study,
but it is
clear
that
the
experimental
series
of
studies
in
which development alone
was
manipulated suggest
that
a
development
component
is
important. More work needs
to be
done
on the
development component,
and
this
may
involve more
and
different
types
of
training.
Given
the
complexity (several different
behavioral
requests involving sequences
of
behaviors)
of the
treatment,
it is
difficult
to
provide
a
single, precise
statement
about
the
extent
to
which
the
treatment
was
imple-
mented. Initially implementation
was es-
timated
by
using
a
summary checklist
that
observers
filled
out at the end of
each
ob-
servation.
The
information
on the
checklist
provides good,
but not
total,
coverage
of
treatment behaviors.
Using
the
checklist, eight
different
im-
plementation scores have been generated.
Multiple definitions
of
implementation were
used because
it is
possible
to
score program
implementation
in
absolute
and
relative
ways. However,
on all
scoring definitions,
treatment
teachers performed more
of the
treatment
behaviors
than
did
control
teachers.
Table
2
summarizes
those
be-
haviors
included
on the
checklist which were
used
to
estimate
the
extent
to
which teachers
implemented
the
treatment.
For
example,
in
91% of the
observations
treatment
teach-
ers
were found
to
conduct
a
review, whereas
control teachers were found
to
conduct
a
Table
2
Mean
Occurrence
(in
percent)
of
Selected Implementation Variables
for
Treatment
and
Control
Group
Teachers
and
Correlation
of
These
Variables
With Teachers' Residualized Gain Scores
on
SRA
Mathematics Test
Occurrence
Correlation
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Variable
Did
the
teacher conduct
review?
Did
development take place
within
review?
Did
the
teacher check
homework?
Did
the
teacher
work
on
mental computation?
Did
the
teacher
summarize
previous
day's materials?
There
was a
slow transition
from
review.
Did
the
teacher spend
at
least
5
minutes
on
development?
Were
the
students held
accountable
for
controlled practice
during
the
development phase?
Did
the
teacher
use
demonstrations
during
presentation?
Did
the
teacher conduct seatwork?
Did
the
teacher actively engage students
in
seatwork
(first
I'/a
minutes)?
Was the
teacher available
to
provide
immediate help
to
students
during
seatwork
(next
5
minutes)?
Were
students held accountable
for
seatwork
at the end
of
seatwork phase?
Did
seatwork directions take
longer
than
I
minute?
Did
the
teacher make
homework
assignments?
Treat-
ment
91%
51%
79%
69%
28%
7%
45%
33%
45%
80%
71%
68%
59%
18%
66%
Con-
trol
62%
37%
20%
6%
25%
4%
51%
20%
46%
56%
43%
47%
31%
23%
13%
P
.0097
.16
.0001
.001
.69
.52
.52
.20
.87
.004
.0031
.02
.01
.43
.001
r
.37
.10
.54
.48
.20
-.02
-.08
.12
-.15
.27
.32
.28
.35
-.02
.49
P
.04
.57
.001
.005
.26
.91
.65
.50
.41
.13
.07
.11
.05
.92
.004
Note.
SRA
=
Science Research Associates.
MISSOURI
MATHEMATICS EFFECTIVENESS PROJECT
359
Table
3
Preproject
and
Postproject
Means
and
Standard
Deviations
for
Experimental
and
Control
Classes
on the SRA
Mathematics
Achievement
Test
Preproject
data
Postproject
dat