(PDF) The impacts of an early mathematics curriculum on oral language and literacy

This article appeared in a journal published by Elsevier. The attached

copy is furnished to the author for internal non-commercial research

and education use, including for instruction at the authors institution

and sharing with colleagues.

Other uses, including reproduction and distribution, or selling or

licensing copies, or posting to personal, institutional or third party

websites are prohibited.

In most cases authors are permitted to post their version of the

article (e.g. in Word or Tex form) to their personal website or

institutional repository. Authors requiring further information

regarding Elsevier’s archiving and manuscript policies are

encouraged to visit:

http://www.elsevier.com/copyright

Author's personal copy

Early

Childhood

Research

Quarterly

27 (2012) 489–

502

Contents

lists

available

at

SciVerse

ScienceDirect

Early

Childhood

Research

Quarterly

The

impacts

of

an

early

mathematics

curriculum

on

oral

language

and

literacy

Julie

Saramaa, Alissa

A.

Langeb, Douglas

H.

Clementsa,∗,

Christopher

B.

Wolfec

aUniversity

at

Buffalo,

State

University

of

New

York,

Department

of

Learning

and

Instruction,

505

Baldy

Hall,

Buffalo,

NY

14260,

United

States

bRutgers

University,

United

States

cIndiana

University

– Kokomo,

United

States

a

r

t

i

c

l

e

i

n

f

o

Article

history:

Received

15

September

2010

Received

in

revised

form

13

November

2011

Accepted

6

December

2011

Keywords:

Preschool

Mathematics

Literacy

Language

Early

childhood

Scale-up

Randomized

Curriculum

a

b

s

t

r

a

c

t

Competence

in

early

mathematics

is

crucial

for

later

school

success.

Although

research

indicates

that

early

mathematics

curricula

improve

children’s

mathematics

skill,

such

curricula’s

impacts

on

oral

language

and

early

literacy

skills

are

not

known.

This

project

is

the

ﬁrst

to

investigate

the

effects

of

an

intensive

pre-kindergarten

mathematics

curriculum,

Building

Blocks,

on

the

oral

language

and

letter

recognition

of

children

participating

in

a

large-scale

cluster

randomized

trial

project.

Results

showed

no

evidence

that

children

who

were

taught

mathematics

using

the

curriculum

performed

differently

than

control

children

who

received

the

typical

district

mathematics

instruction

on

measures

of

letter

recognition,

and

on

two

of

the

oral

language

(story

retell)

subtests,

sentence

length

and

inferential

reasoning

(emotive

content).

However,

children

in

the

Building

Blocks

group

outperformed

children

in

the

control

group

on

four

oral

language

subtests:

ability

to

recall

key

words,

use

of

complex

utterances,

willingness

to

reproduce

narratives

independently,

and

inferential

reasoning

(practical

content).

Children

who

live

in

poverty

and

who

are

members

of

linguistic

and

ethnic

minority

groups

demonstrate

signiﬁcantly

lower

lev-

els

of

academic

achievement,

and

this

pernicious

process

begins

in

the

earliest

years

(Denton

&

West,

2002;

National

Research

Council,

2001;

Natriello,

McDill,

&

Pallas,

1990).

Preschool

edu-

cation

is

often

provided

to

address

early

experiential

differences.

Unfortunately,

many

children

from

lower-resource

communities

attend

preschools

that

are

not

of

high

quality.

For

example,

they

are

more

likely

than

children

from

higher-resource

communities

to

be

taught

by

teachers

with

fewer

qualiﬁcations

(Clifford

et

al.,

2005).

This

is

unfortunate

as

high-quality

programs

can

help

these

chil-

dren

achieve

greater

school

readiness

upon

entry

into

kindergarten

(Clements

&

Sarama,

2008a;

Magnuson,

Meyers,

Rathbun,

&

West,

2004;

National

Research

Council,

2001,

2009;

Sarama

&

Clements,

2009a;

Shonkoff

&

Phillips,

2000).

In

particular,

there

is

evidence

that

high-quality,

research-based

preschool

mathematics

curric-

ula

can

improve

early

mathematics

achievement

(e.g.,

Clements

&

Sarama,

2007c,

2008a,

2011a;

Sarama,

Clements,

Starkey,

Klein,

&

Wakeley,

2008).

However,

with

increasing

pressure

on

educa-

tors

to

achieve

benchmarks

across

multiple

areas

of

learning,

it

is

important

to

know

what,

if

any,

impacts

these

early

mathematics

∗Corresponding

author.

Permanent

address:

114

Carriage

Circle,

Williamsville,

NY

14221-2164,

United

States.

Tel.:

+1

716

689

3788;

fax:

+1

716

645

6721.

E-mail

addresses:

jsarama@buffalo.edu

(J.

Sarama),

alissa.lange@rutgers.edu

(A.A.

Lange),

clements@buffalo.edu

(D.H.

Clements),

chbwolfe@iuk.edu

(C.B.

Wolfe).

programs

have

on

other

academic

areas,

especially

language

and

emergent

literacy.

This

project

is

the

ﬁrst

to

investigate

the

effects

of

an

intensive

pre-kindergarten

mathematics

intervention

on

the

oral

language

and

letter

recognition

skills

of

preschool

children.

1.

Oral

language

Both

receptive

and

expressive

oral

language

skills

are

strongly

early

literacy

development

(e.g.,

Cooper,

Roth,

Speece,

&

Schatschneider,

2002),

later

academic

success

(e.g.,

Bishop

&

Edmundson,

1987;

Catts,

1993;

Pankratz,

Plante,

Vance,

&

Insalaco,

2007;

Paul,

Hernandez,

Taylor,

&

Johnson,

1996;

Snow,

Barnes,

Chandler,

Goodman,

&

Hemphill,

1991),

and

future

linguistic

skill

(e.g.,

Conti-Ramsden,

Botting,

Simkin,

&

Knox,

2001;

Pankratz

et

al.,

2007).

Speciﬁc

components

of

early

oral

language,

including

vocab-

ulary,

grammar,

semantics,

and

narrative

discourse

processes

(i.e.,

complexity

and

content

analysis),

have

been

shown

to

indepen-

dently

predict

later

academic

success

from

pre-kindergarten

(e.g.,

Dickinson,

McCabe,

Anastaspoulos,

Peisner-Feinberg,

&

Poe,

2003).

Speciﬁcally,

early

oral

language

competencies

are

good

predictors

of

later

literacy,

nearly

equivalent

to

the

established

predictors

of

alphabetic

knowledge

and

phonological

awareness

(Pearson

&

Hiebert,

2010).

The

NICHD

Early

Child

Care

Research

Network

(2005)

demon-

strated

that

these

core

aspects

of

oral

language

skill

contributed

signiﬁcantly

and

independently

to

later

reading

success.

Further,

0885-2006/$

–

see

front

matter ©

doi:10.1016/j.ecresq.2011.12.002

Author's personal copy

490 J.

Sarama

et

al.

/

Early

Childhood

Research

Quarterly

27 (2012) 489–

502

oral

language

was

affected

by

the

degree

of

linguistic

complexity

in

children’s

environments.

Because

oral

language

skills

demon-

strated

a

stronger

connection

to

later

academic

outcomes

for

children

from

low-resource

backgrounds

than

children

from

high-

resource

backgrounds,

rich

linguistic

experiences

at

early

ages

may

be

especially

important

for

children

at

risk

for

academic

failure.

2.

Letter

recognition

Similarly,

letter

recognition,

one

early

component

of

ortho-

graphic

development

and

pre-reading

skill,

is

a

strong

independent

predictor

of

later

reading

success

(e.g.,

decoding,

naming

speed,

phonological

awareness,

writing,

see

Denton

&

West,

2002;

McGill-

Franzen,

2010;

Molfese,

Modglin,

et

al.,

2006).

For

example,

letter

recognition

scores

at

Kindergarten

have

been

reported

as

strong

predictors

of

second

grade

and

fourth

grade

word

recognition

and

reading

comprehension

(Catts,

Fey,

Tamblin,

&

Zhang,

2002).

Letter

recognition

serves

as

a

base-level

step

in

the

process

of

‘cracking’

the

alphabetic

principle,

beginning

the

process

of

con-

necting

sound

with

symbol

and

growth

in

phonological

awareness

(e.g.,

Adams,

1990;

Treiman

&

Bourassa,

2000;

Vellutino,

Scanlon,

&

Tanzman,

1994).

Children

from

low-income

backgrounds

are

at

a

signiﬁcant

disadvantage

in

the

development

of

this

skill

(Bradley

&

Corwyn,

2002;

Denton

&

West,

2002).

For

example,

in

a

study

of

preschool

skill

gains

within

a

literacy

skills-based

curriculum

with

progress

monitoring

for

children

from

low-income

backgrounds,

53%

made

either

no

gains

or

gains

of

one

letter.

On

the

other

hand,

47%

gained

seven

or

more

letters

(Molfese,

Modglin,

et

al.,

2006).

The

importance

of

letter

recognition

to

later

literacy

develop-

ment

and

the

wide

variability

in

the

growth

of

letter

identiﬁcation

skill

suggest

the

need

for

highlighting

interventions

and

programs

that

facilitate

the

development

of

this

core

academic

skill

(Brown,

Molfese,

&

Molfese,

2008).

3.

Linking

language

and

literacy

with

mathematics

Different

bodies

of

research

report

conﬂicting

ﬁndings

con-

cerning

the

effects

of

mathematics

curricula

on

early

language

and

literacy.

The

impact

of

time-on-task

(or

time

on

instruc-

tion)

on

learning

provides

prima

facie

justiﬁcation

for

the

concern

of

subject-matter

conﬂict

(e.g.,

Bodovski

&

Farkas,

2007).

From

this

frequently-voiced

perspective

(see

Clements

&

Sarama,

2009;

Farran,

Lipsey,

Watson,

&

Hurley,

2007;

Lee

&

Ginsburg,

2007;

Sarama

&

Clements,

2009a),

the

introduction

of

a

mathematics

curriculum

could

decrease

time

devoted

to

language

and

liter-

acy

activities,

impeding

children’s

development

of

those

domains.

However,

this

assumes

that

mathematics

activities

have

little

or

no

positive

effects

on

language

and

literacy.

Evidence

from

both

educational

and

psychological

research,

however,

suggests

co-mutual

beneﬁcial

inﬂuences.

For

example,

similar

developmental

milestones

exist

in

the

learning

of

math-

ematics

and

language.

Children

generally

begin

learning

number

words

at

the

same

time

as

other

linguistic

labels.

By

the

age

of

two,

most

children

recognize

which

words

are

reserved

for

numbers

and

use

these

words

only

in

appropriate

contexts

(Fuson,

1988).

Across

alphabetic

languages,

there

is

a

developmental

pattern

of

recogniz-

ing

word

before

syllable,

syllable

before

rime-onset,

and

rime-onset

before

phoneme

(see

also

Anthony,

Lonigan,

Driscoll,

Phillips,

&

Burgess,

2003;

Ziegler

&

Goswami,

2005).

Similarly

in

mathematics

,

development

proceeds

from

conceptualizing

numbers

as

unbreak-

able

quantitative

categories

and

relationships

(size

of

bears

and

correspon-

dence

between

these

sizes

and

sizes

of

household

objects),

and

ordering

and

patterns

(a

sequential

plot

using

a

patterned

narrative

structure).

4.3.

Do

the

variables

of

gender

or

ethnic

group

moderate

the

relationship

between

treatment

group

and

language/literacy

outcomes?

Research

reviews

indicate

that

differences

between

girls

and

boys

in

early

mathematics

are

small

and

inconsistent

(Clements

&

Sarama,

2009)

and

curriculum

interventions

usually

report

no

interactions

with

gender

(Clements

&

Sarama,

2008a;

Preschool

Curriculum

Evaluation

Research

Consortium,

2008;

Sarama

et

al.,

2008).

However,

parents’

use

of

spatial

language

was

only

girls’,

not

boys’,

mental

transformation

skill

(McGuinness

&

Morley,

1991)

and

such

spatial

language

use

may

be

more

important

for

girls

(Cannon,

Levine,

&

Huttenlocher,

2007).

Thus,

it

is

reasonable

to

ascertain

whether

classroom-based

mathematics

experiences

have

different

effects

on

girls’

and

boys’

language

and

emergent

literacy.

This

question

is

exploratory;

however,

as

gender

appears

frequently

in

the

literature

mathematics

learning,

this

analysis

will

contribute

to

the

ﬁeld.

Similarly,

there

has

been

no

consistent

evidence

of

differential

effectiveness

in

most

preschool

mathematics

curriculum

interven-

tions

for

children

of

different

racial/ethnic

identities.

However,

the

TRIAD/Building

Blocks

intervention

was

found

to

be

differen-

tially

effective

for

one

ethnic/racial

comparison:

children

identiﬁed

as

African-American

learned

less

than

other

children

in

the

same

control

classrooms

while

children

identiﬁed

as

African-American

learned

more

than

other

children

in

the

same

Building

Blocks

classrooms

(Sarama

&

Clements,

2009b).

It

may

be

that

the

intervention

is

particularly

effec-

tive

in

ameliorating

the

negative

effects

of

low

expectations

for

learning

for

children

of

African-American

descent

(cf.

National

Mathematics

Advisory

Panel,

2008).

If

so,

it

is

important

to

examine

the

degree

to

which

these

resiliency

effects

could

transfer

to

other

academic

domains.

In

a

similar

vein,

effects

may

be

different

for

other

sub-

groups,

such

as

Hispanic

children

and,

at

the

school

level,

schools

with

different

percentages

of

children

with

limited

English

proﬁ-

ciency

(LEP)

and

of

those

receiving

free

or

reduced

school

lunch,

all

of

which

may

moderate

any

effect

of

the

treatment

(National

Research

Council,

2009).

4.4.

If

the

treatment

has

signiﬁcant

impacts,

are

there

signiﬁcant

indirect

effects

through

aspects

of

the

classroom

and

teaching

environment

on

the

relationship

between

children’s

assignment

to

treatment

group

and

their

mathematics

achievement?

The

quantity

of

mathematics

activities

and

the

quality

of

the

classroom’s

mathematics

environment,

the

total

number

of

Author's personal copy

492 J.

Sarama

et

al.

/

Early

Childhood

Research

Quarterly

27 (2012) 489–

502

computers

children

were

using

to

engage

with

the

intervention’s

software,

and

the

number

of

different

mathematics

activities,

signiﬁcantly

mediated

mathematics

learning

(Clements,

Sarama,

Spitler,

et

al.,

2011)

and

thus

may

mediate

any

relationship

between

experimental

condition

and

children’s

development

of

emergent

language

and

literacy

skills.

For

example,

teachers’

use

of

mathematics

vocabulary

and

“math

talk”

has

been

gains

in

child

mathematics

knowledge

(Klibanoff,

Levine,

Huttenlocher,

Vasilyeva,

&

Hedges,

2006).

The

Building

Blocks

curriculum

includes

a

print

curriculum

and

professional

development

com-

ponent

in

which

teachers

are

explicitly

trained

in

the

elicitation

of

explanations

for

solutions

to

mathematical

questions.

Focus

on

mathematizing

everyday

activities

in

the

classroom

and

the

use

of

explicit

techniques

to

elicit

mathematical

language

and

discussion

within

the

classroom

may

encourage

the

growth

of

a

classroom

climate

particularly

suited

for

the

development

of

mathematics

and

language

skills.

Aspects

of

such

a

climate

may

include

general

interactional

patterns,

but

also

the

number

of

mathematical

activities

(including

computer

activities,

which

have

been

shown

to

generate

more

academic

communication

and

language

than

other

classroom

contexts,

see

Clements

&

Sarama,

2003,

2008b).

5.

Method

5.1.

Participants

The

participants

used

in

these

analyses

were

from

the

ﬁrst

year

of

the

large-scale

research

project,

Scaling-up

TRIAD:

Teaching

Early

Mathematics

for

Understanding

with

Trajectories

and

Technologies.

The

two

participating

school

districts

were

targeted

because

they

traditionally

serve

children

from

low-resource

communities.

To

be

involved

in

the

study,

the

districts

were

required

to

agree

to

ran-

domly

assign

all

eligible

schools

(those

whose

preschool

teachers

had

not

worked

in

any

Blocks

project)

to

treat-

ment

groups.

District-level

adoption

was

used

because

the

intent

was

to

measure

impacts

of

the

curriculum

“at

scale;”

that

is,

to

determine

how

the

curriculum

might

function

in

practice

when

it

is

adopted

district-wide

(involving

all

teachers,

not

just

volun-

teers).

Using

a

table

of

random

numbers,

all

eligible

schools

within

each

district

were

publicly

(supervised

by

two

school

administra-

tors

and

three

staff

members)

assigned

to

one

of

three

treatment

groups:

Building

Blocks,

the

Building

Blocks

plus

follow

through,

or

the

control

group.

The

ﬁrst

two

treatment

groups

were

identical

for

the

purposes

of

the

present

study.

In

subsequent

years,

the

Building

Blocks

plus

follow-through

group

was

characterized

by

additional

training

for

kindergarten

teachers

but

this

did

not

occur

until

after

the

current

study’s

data

were

collected

and

analyzed.

There-

fore,

in

the

current

analysis,

only

two

groups

were

compared,

26

Building

Blocks

and

17

control

schools.

Up

to

15

children

from

each

class

whose

parents

provided

consent

were

randomly

selected

to

participate.

The

present

study

involved

two

subsets

of

the

full

original

sam-

ple.

The

ﬁrst

included

all

those

for

whom

we

have

letter

recognition

scores:

1037

children

(50%

female;

56%

African

American,

19%

His-

panic,

19%

White)

from

38

schools

(24

Building

Blocks,

15

control),

out

of

the

1305

children

in

the

original

randomly-assigned

42

schools

(26

Building

Blocks,

17

control)

across

both

districts.

The

second

subset

consists

of

1027

children

(52%

female;

54%

African

American,

21%

Hispanic,

19%

White),

who

were

assessed

by

the

research

team

on

a

measure

of

oral

language.

Of

the

total

1,305

children

participating

in

the

larger

study,

approximately

80%

were

represented

on

both

language

and

literacy

measures.

Table

1

summarizes

the

demographics

of

each

sample.

We

had

power

of

.80

to

detect

effects

of

d

=

.10

or

less

with

our

sample

of

42

clusters

(schools),

and

an

average

of

24

children

in

each

cluster

(school).

5.2.

Curricula

5.2.1.

Treatment

math

curriculum

Building

Blocks

(Clements

&

Sarama,

2007b)

is

a

National

Science

Foundation-funded

mathematics

curriculum

based

on

a

comprehensive

Curriculum

Research

Framework

(Clements,

2007).

The

curriculum

focuses

on

two

main

domains

of

mathemat-

ics,

number

and

geometry/spatial

skills;

woven

throughout

these

domains

are

subthemes,

such

as

sorting

and

sequencing,

as

well

as

mathematical

processes,

both

general,

such

as

communicating,

reasoning,

representing,

and

problem

solving

and

the

overarching

mathematizing,

and

speciﬁc,

such

as

number

or

shape

composition

and

patterning.

Together,

these

concepts,

skills,

and

processes

were

determined

to

be

critical

mathematical

building

blocks.

Building

Blocks’

instructional

approach

is

ﬁnding

the

mathe-

matics

in,

and

developing

mathematics

from,

children’s

activity

(Clements

&

Sarama,

2007b).

Children

are

guided

to

extend

and

mathematize

(i.e.,

explicate,

articulate,

and

describe)

their

every-

day

activities,

from

block

building

to

art

to

songs

to

puzzles,

in

mathematical

language.

Thus,

the

processes

of

communicating

and

reasoning,

and

mathematizing

are

continually

developed

through

discussions.

Teachers

ask

students

to

solve

problems

or

tasks,

and

then

ask

such

questions

as

“How

do

you

know?,”

“Why?,”

and

“Can

you

tell

how

you

ﬁgured

that

out?”

Activities

include

whole

group

(about

10

min

per

day),

small

group

(10–15

min

once

per

week

for

each

child,

working

in

groups

of

four

with

the

teacher),

and

cen-

ters

(including

a

computer

center,

5–10

min

twice

a

week

for

each

child).

The

curriculum

includes

30

weeks

of

instruction;

teach-

ers

completed

from

24

to

30

weeks.

More

detailed

descriptions

Table

1

Demographics

of

participants

with

oral

language

and

letter

recognition

scores.

Children

Teachers

Schools

Total

children

(males)

Agea(SD)

Ethnicity

Total

teachers

Total

schools

SESbELLc

African

American

Hispanic

White

Asian/

Paciﬁc

Native

American

Other

Students

with

oral

language

scores

Building

Blocks

726

(346)

64

(4.2)

55%

20%

19%

3%

2%

<1%

71

26

82.4%

11.8%

Control

301

(144)

64

(4.0)

51%

24%

17%

3%

2%

<1%

34

17

82.3%

16.2%

Total

1,027

(490)

63.9

(4.09)

54%

21%

18%

4%

2%

<1%

105

42

82.3%

13.5%

Students

with

letter

recognition

scores

Building

Blocks

714

(355)

59.3

(4.07)

58%

17%

21%

3%

1%

<1%

67

24

82.9%

10.9%

Control

323

(161)

59.4

(4.05)

52%

24%

16%

6%

1%

<1%

30

15

84.1%

14.5%

Total

1,037

(516)

59.3

(4.06)

56%

19%

4%

1%

<1%

97

38

83.4%

12.2%

aAge

in

months

at

time

of

assessment.

bMean

percent

free

or

reduced

lunch

in

schools.

cMean

percent

English

language

learning

(ELL)

in

schools.

Author's personal copy

J.

Sarama

et

al.

/

Early

Childhood

Research

Quarterly

27 (2012) 489–

502 493

of

Building

Blocks

are

available

(Clements

&

Sarama,

2004,

2007b,

2007c,

2011b).

CRT

evaluations

of

the

curriculum

have

yielded

effect

sizes

ranging

from

.50

to

2.10

(Clements

&

Sarama,

2007c,

2008a).

5.2.2.

District’s

preschool

literacy

curricula

The

ﬁrst

district

implemented

the

Houghton-Mifﬂin

pre-K

liter-

acy

curriculum,

Where

Bright

Futures

Begin.

The

curriculum

features

ten

thematic

segments

(e.g.,

Animals

Everywhere),

each

consisting

of

three

weeks

of

theme-related

instruction.

The

ﬂexible

lesson

struc-

ture

is

designed

to

develop

critical

early

learning

skills,

including

oral

language,

listening

comprehension,

vocabulary,

phonologi-

cal

awareness,

print

awareness,

and

alphabet

knowledge.

It

also

teaches

early

mathematics

skills

on

a

daily

basis,

with

multiple

top-

ics

taught

during

small

group

and

whole

group

times.

The

teacher

manual

includes

support

for

teaching

English

language

learners

(ELL)

and

for

implementing

formative

assessment

strategies

(see

www.hmhschool.com).

Professional

development

for

Where

Bright

Futures

Begin

was

provided

for

teachers

three

times

during

the

year,

each

time

with

an

emphasis

on

literacy.

The

second

district

implemented

a

comprehensive,

inte-

grated

curriculum,

Opening

the

World

of

Learning

(OWL),

which

was

designed

for

full

day

implementation,

with

components

added

to

language

and

literacy,

including

mathematics,

science,

social

studies,

art,

and

social–emotional

development.

OWL

mathematics

activities

were

presented

as

small-group

activities.

Components

included

suggested

vocabulary,

with

procedures

provided

for

extra

support

as

well

as

extension

activities

(see

www.pearsonlearning.com/microsites/owl/main.cfm).

Six

profes-

sional

development

sessions

on

OWL

were

provided

during

the

year.

5.2.3.

Fidelity

of

implementation

and

effects

on

mathematics

A

(Clements,

Sarama,

Spitler,

et

al.,

2011)

docu-

mented

that

teachers

implemented

the

curriculum

with

adequate

ﬁdelity.

On

a

5-point

Likert

scale

items,

with

−2

as

strongly

disagree

and

+2

as

strongly

agree,

the

mode

was

1,

and

the

mean

was

.77

in

fall

and

.86

in

spring.

Less

than

15%

of

teachers

had

an

average

below

.50

(Clements,

Sarama,

Spitler,

et

al.,

2011,

p.

141).

Children

in

the

Building

Blocks

group

learned

more

mathematics

than

the

children

in

the

control

group

(effect

size,

g

=

.72).

5.3.

Measures

5.3.1.

Letter

recognition

Schools

administered

assessments

of

letter

recognition

at

the

end

of

the

pre-K

year.

The

assessments

were

PALS-PreK

(Invernizzi,

Sullivan,

Swank,

&

Meier,

2004)

and

MCLASS:CIRCLE

(Landry,

2007).

Both

measures

have

undergone

signiﬁcant

development

and

piloting,

and

are

widely

used

as

early

childhood

emergent

literacy

screeners

(Invernizzi

et

al.,

2004;

Landry,

2007).

PALS-PreK

has

good

inter-rater

reliability

(.99

across

all

tasks),

split-half

reli-

ability

(Guttman

split-half

reliability

ranging

from

.71

to

.94

across

tasks),

and

internal

consistency

(Cronbach’s

alpha

ranging

from

.75

to

.93,

Invernizzi

et

al.,

2004).

The

MCLASS:CIRCLE

has

good

internal

consistency

(Cronbach’s

alpha

ranging

from

.90

to

.93),

high

stability

of

letter

recognition

over

time

(ICCs

ranging

between

.71

and

.76),

and

a

strong

relationship

between

letter

recognition

scores

and

print

awareness

on

the

Preschool

Comprehensive

Test

of

Phonological

and

Print

Processing

(PCTOPPP,

Wagner,

Torgesen,

&

Rashotte,

1999)

was

.76

(Swank

et

al.,

2006).

Both

assessments

measure

recognition

of

upper

and

lowercase

letters

by

showing

letters

of

the

alphabet

and

asking

them

to

name

each

letter.

The

important

methodological

difference

between

these

two

assess-

ments

was

that

the

assessments

were

timed

in

the

ﬁrst

site

and

untimed

in

the

second.

This

led

to

a

signiﬁcant

difference

in

mean

scores

between

the

two

sites;

scores

were

predictably

higher

in

the

second.

Therefore,

we

calculated

z-scores

separately

for

each

site

based

on

the

mean

and

standard

deviation

of

the

raw

letters

correct.

These

standard

scores

were

utilized

in

all

subsequent

analyses.

5.3.2.

Oral

language

We

chose

to

use

The

Renfrew

Bus

Story

–

North

American

Edi-

tion

(RBS;

Glasgow

&

Cowley,

1994),

a

standardized

measure

of

oral

language

using

narrative

retell,

to

evaluate

children’s

oral

language.

The

assessment

involves

telling

a

child

a

story,

and

then

asking

the

child

to

retell

the

story

using

the

pictures

in

the

wordless

storybook

as

prompts.

At

the

end

of

the

story,

assessors

asked

children

two

inferential

questions.

We

assessed

oral

language

in

the

context

of

a

story

retell

task

because

this

type

of

assessment

is

an

ecologically

valid

method

of

eliciting

language

from

children

as

most

are

familiar

with

the

requirements

and

procedures

of

storytelling;

most

having

read

sto-

ries

in

school

or

at

home

by

an

early

age

(Botting,

2002;

Curenton,

2004;

Fey,

Catts,

Proctor-Williams,

Tomblin,

&

Zhang,

2004).

The

resultant

scores

from

the

Bus

Story

assessment

give

us

indicators

of

aspects

of

a

child’s

oral

language,

such

as

sentence

length

or

complexity

of

utterances

that

are

present

in

the

child’s

story

retell.

has

demonstrated

strong

predictive

relation-

ships

with

literacy

and

language

skills

three

years

after

initial

assessment

(Pankratz

et

al.,

2007).

The

U.K.

version,

on

which

the

North

American

version

was

based,

has

extensive

validity

evidence,

including

research

demonstrating

its

ability

to

predict

adolescent

academic

performance

from

performance

on

the

RBS

at

four

years

of

age

(Stothard,

Snowling,

Bishop,

Chipchase,

&

Kaplan,

1998).

The

North

American

version

is

highly

correlated

with

the

United

Kingdom

version

on

the

information

(.98)

and

sentence

length

(.98)

subtests,

as

reported

in

the

North

American

version

man-

ual

(RBS,

Glasgow

&

Cowley,

1994).

Test–retest

reliability

reported

in

the

manual

ranged

from

.58

for

complexity

to

.79

for

sentence

length

(Glasgow

&

Cowley,

1994).

Subscales

of

the

RBS

were

found

(Pankratz

et

al.,

2007)

to

be

strongly

another

measure

of

oral

language,

the

Oral

and

Written

Language

Scales

(OWLS,

Carrow-Woolfolk,

1996).

The

RBS

sentence

length

score

was

signif-

icantly

the

OWLS

oral

composite

score

(r

=

.89,

p

<

.0001),

and

the

OWLS

total

score

(r

=

82,

p

=

0015),

and

the

information

score

was

signiﬁcantly

the

OWLS

oral

composite

score

(r

=

.79,

p

=

.0024,

Pankratz

et

al.,

2007).

The

stories

that

children

retell

during

the

RBS

assessment

are

transcribed

and

scored

on

a

series

of

dimensions

by

trained

coders

naïve

to

the

group

assignment

of

the

child.

The

dimensions

on

which

the

transcribed

stories

are

scored

consist

of

three

primary

subtests

rating

the

content

of

the

story

retells,

including

information,

complexity,

and

sentence

length.

Three

additional

subtests

were

two

inferential

reasoning

items

(emotive

and

practical

content,

respectively)

and

independence.

The

validity

of

the

transcribing

procedure

for

the

RBS

was

determined

in

the

present

study

by

evaluating

a

one-way

single

measure

intraclass

correlation

(ICC)

between

a

random

7

and

10%

sample

of

stories

that

were

transcribed

by

two

different

people

(LeBreton

&

Senter,

2008).

The

two

transcriptions

were

expert

scored

by

a

researcher,

and

the

ICCs

between

the

pairs

of

scores

on

the

subtests,

which

represented

the

difference

in

transcriptions

from

the

tapes

(audio

and

videotape),

were

.99

for

information,

.97

for

Sentence

length,

.94

for

complexity.

The

information

subtest

is

a

measure

of

how

many

of

the

32

key

concepts

from

the

original

story

the

children

used

in

their

story

retell,

with

some

key

concepts

being

worth

two

points.

The

total

raw

score,

with

a

maximum

possible

score

of

52,

is

then

converted

to

a

standardized

score.

To

score

well

on

this

mea-

sure,

children

must

remember

key

concepts

(memory),

know

the

meaning

of

the

words

representing

the

concepts

well

enough

to

Author's personal copy

494 J.

Sarama

et

al.

/

Early

Childhood

Research

Quarterly

27 (2012) 489–

502

use

them

appropriately

in

their

retell

(vocabulary),

and

have

a

sufﬁcient

understanding

of

story

structure

to

use

the

words

or

concepts

in

the

right

sequence

(book/story

knowledge).

Thus,

this

subtest

represents

proﬁciency

for

a

set

of

integrated

skills.

The

information

subtest

is

correlated

with

age

(Glasgow

&

Cowley,

1994),

and

scores

on

this

dimension

have

been

shown

to

predict

future

academic

skill

(Pankratz

et

al.,

2007).

For

this

study,

we

attempted

to

improve

the

precision

of

the

measurement

by

submitting

the

information

items

to

Rasch

analy-

sis.

This

was

the

only

subtest

appropriate

for

Rasch

analysis

for

two

reasons.

First,

the

other

subtests

include

a

single

item,

obviating

the

use

of

Rasch

analyses.

Second,

combining

subtests

would

not

be

desirable

as

each

provides

unique

information

about

components

of

oral

language.

The

Rasch

analysis

supported

a

unidimensional

construct,

although

some

of

the

partial

credit

items,

with

possible

scores

of

0,

1,

or

2,

required

recoding.

All

but

three

partial-credit

items

were

recoded

such

that

they

either

received

0

or

1.

In

these

cases,

all

responses

that

would

have

received

a

2

were

given

a

1,

and

all

0s

remained

0s.

The

ﬁnal

Rasch

model

for

the

information

scale,

with

recoded

partial

credit

variables,

had

an

item-reliability

of

.99

and

a

person

reliability

of

.79.

The

slightly

lower

but

accept-

able

(Bond

&

Fox,

2001)

person

reliability

may

have

been

due

to

the

overrepresentation

of

below-average

performing

children

and

the

limited

number

of

items.

The

two

methods

of

coding

the

infor-

mation

scores

(i.e.,

standard

score

and

Rasch

scores)

were

highly

correlated

(r

=

.88).

Inter-rater

reliability

was

calculated

using

ICC

to

determine

scoring

agreement

between

two

different

scorers

using

the

same

transcriptions

(random

10%

sample).

One-way

intraclass

correlation

with

single

measures

was

used

to

account

for

error

due

to

having

more

than

two

raters,

but

not

always

the

same

two

raters

rating

each

target.

Agreement

was

.97

for

information.

Complexity

is

measured

by

the

number

of

complex

utterances

children

use

in

their

story

retells.

Complex

utterances

are

deﬁned

as

those

that

include

a

subordinate

clause

or

a

relative

clause.

Complex

utterance

use

is

correlated

with

age

(Glasgow

&

Cowley,

1994).

As

with

information,

inter-rater

reliability

was

calculated

for

a

random

10%

sample,

and

was

.81

for

complexity.

Sentence

length

is

also

age

(Glasgow

&

Cowley,

1994).

This

particular

measure

of

sentence

length,

the

mean

of

the

ﬁve

longest

utterances,

although

correlated

with

indicators

of

language

development,

is

a

quicker

yet

less

reliable

adaptation

of

the

more

commonly

used

measure

of

oral

language

utterance

length,

mean

length

of

utterance

(MLU).

MLU

is

not

reasonable

for

use

with

the

RBS

because

reliable

measures

of

MLU

require

a

minimum

of

50–100

utterances

(Eisenberg,

Fersko,

&

Lundgren,

2001),

and

the

RBS

is

intended

to

be

a

brief

oral

language

screen-

ing

tool

that

often

elicits

fewer

than

50

utterances.

Using

the

same

random

10%

sample

of

transcripts

as

was

used

for

informa-

tion

and

complexity,

inter-rater

agreement

was

.94

for

sentence

length.

The

inferential

reasoning

scale

includes

children’s

answers

to

two

inferential

questions

at

the

end

of

the

assessment.

The

single,

original

question

from

the

manual

was

“Do

you

think

the

bus

was

happy

to

be

on

the

road

again?

Why

(or

Why

not)?”

This

question

requires

children

to

make

inferences

about

emotion.

A

second

question

that

requires

inferences

that

are

more

practical

in

nature

was

added

to

this

administration:

“How

do

you

think

the

driver

found

the

bus?”

There

are

no

explicit

answers

to

these

questions

in

the

story;

therefore,

the

child

must

be

able

to

think

inferentially

to

answer

the

questions.

It

should

be

noted

that

validation

on

this

scale

was

not

reported

in

the

manual,

and

the

scores

are

based

only

on

two

questions.

However,

as

described

below,

we

attempted

to

bolster

what

was

provided

in

the

manual

on

this

scale

by

increasing

the

detail

with

which

the

responses

were

analyzed.

This

exploratory

work

may

lay

the

groundwork

for

future

development

of

the

scale.

Children’s

raw

answers

to

the

inferential

reasoning

questions

on

the

RBS

were

recoded

based

on

a

coding

scheme

created

in

addition

to

the

one

outlined

in

the

manual.

The

manual

guidelines

only

provide

limited

suggestions

for

how

to

decide

if

an

answer

is

“acceptable”

or

“unacceptable.”

For

example,

an

answer

is

deemed

acceptable

if

it

is

a

“moral

explanation.”

We

therefore

created

a

second

scheme

to

include

a

series

of

scores

for

each

inferential

question.

We

devised

the

system

based

on

themes

that

emerged

from

the

data,

and

subsequently,

each

response

was

given

a

rating

comprised

of

a

summary

of

scores

on

the

following:

causal

plau-

sibility,

reference

to

story,

empathetic,

practical,

and

moral.

Causal

plausibility

captured

the

degree

of

understanding

of

causal

reason-

ing

in

the

child’s

retell.

An

example

of

an

answer

receiving

credit

on

this

scale

is

the

following

answer

to

the

second

inferential

ques-

tion:

“The

driver

found

it

because

he

found

his

tracks.”

This

child

found

a

plausible

reason

that

the

driver

might

have

found

the

bus,

when

no

reason

was

given

in

the

story.

Reference

to

story

repre-

sented

the

degree

to

which

the

child

made

reference

to

the

original

story

in

his

or

her

answer

to

the

question.

The

following

answer

obtained

credit

on

this

scale

because

the

answer

refers

explicitly

to

events

from

the

story:

“She

was

running

fast,

saw

bus

in

water,

jumped

over

gate

too.”

If

the

child

showed

evidence

of

empathizing

with

the

characters

in

the

story,

credit

was

given

for

the

empathetic

scale.

For

example,

a

response

that

received

credit

on

this

scale

is

one

child’s

response

to

the

ﬁrst

inferential

question:

“Cause

of

when

he

went

in

the

water

he

was

sad.”

Responses

that

showed

evidence

of

practical

reasoning

were

given

credit

on

the

practical

scale.

A

response

to

the

ﬁrst

inferential

question

given

credit

on

this

scale

is,

“Because

he

was

all

clean

and

didn’t

want

to

get

all

mucky

and

icky.”

The

child

was

given

credit

on

the

moral

scale

if

their

responses

included

references

to

moral

reasoning.

For

exam-

ple,

the

following

response

to

the

ﬁrst

inferential

question

that

got

credit

on

the

moral

scale

is,

“Because

now

he

knows

that

he

can’t

be

naughty

again.”

Scores

on

item

components

one

and

two

range

from

0

to

2

points,

where

“0”

means

the

answer

to

the

inferential

question

does

not

possess

the

given

property,

“1”

means

that

the

answer

possess

the

property

to

a

limited

extent,

and

“2”

signiﬁes

that

the

response

demonstrates

an

advanced

form

of

this

property.

For

example,

a

response

coded

with

a

“1”

on

reference

to

story

means

that

there

was

an

indirect

or

weak

connection

to

the

story

evident

in

the

child’s

response.

Components

three

to

ﬁve

were

either

scored

“0”

if

the

answer

did

not

have

that

property

or

“1”

if

the

answer

did

have

that

property.

The

total

raw

summary

scores

on

this

measure

ranged

between

0

and

5

for

The impacts of an early mathematics curriculum on oral language and literacy

Abstract and Figures