Content uploaded by Jimmy H.M. van Rijt
Author content
All content in this area was uploaded by Jimmy H.M. van Rijt on Aug 09, 2019
Content may be subject to copyright.
Linguistics
and
Education
52
(2019)
78–88
Contents
lists
available
at
ScienceDirect
Linguistics
and
Education
jo
ur
nal
ho
me
p
age:
www.elsevier.com/locate/linged
When
students
tackle
grammatical
problems:
Exploring
linguistic
reasoning
with
linguistic
metaconcepts
in
L1
grammar
education夽
Jimmy
H.M.
van
Rijta,b,∗,
Peter
J.F.
de
Swartb,
Astrid
Wijnandsc,d,
Peter-Arno
J.M.
Coppend
aDepartment
of
Teacher
Education,
Fontys
University
of
Applied
Sciences,
P.O.
Box
558,
6130
AN
Sittard,
The
Netherlands
bDepartment
of
Linguistics,
Center
for
Language
Studies,
Radboud
University
Nijmegen,
P.O.
Box
9103,
6500
HD
Nijmegen,
The
Netherlands
cDepartment
of
Teacher
Education,
Utrecht
University
of
Applied
Sciences,
Padualaan
97,
3584
CH
Utrecht,
The
Netherlands
dRadboud
Teachers’
Academy,
Radboud
University
Nijmegen,
P.O.
Box
9103,
6500
HD
Nijmegen,
The
Netherlands
a
r
t
i
c
l
e
i
n
f
o
Article
history:
Received
21
December
2018
Received
in
revised
form
17
June
2019
Accepted
18
June
2019
Keywords:
L1
Grammar
teaching
Linguistic
metaconcepts
Linguistic
reasoning
Grammatical
concepts
Syntax
Explicit
knowledge
a
b
s
t
r
a
c
t
When
teaching
grammar,
one
of
the
biggest
challenges
teachers
face
is
how
to
make
their
students
achieve
conceptual
understanding.
Some
scholars
have
argued
that
metaconcepts
from
theoretical
linguistics
should
be
used
to
pedagogically
and
conceptually
enrich
traditional
L1
grammar
teaching,
generating
more
opportunities
for
conceptual
understanding.
However,
no
empirical
evidence
exists
to
support
this
theoretical
position.
The
current
study
is
the
first
to
explore
the
role
of
linguistic
metaconcepts
in
the
grammatical
reasoning
of
university
students
of
Dutch
Language
and
Literature.
Its
goal
was
to
gain
a
better
understanding
of
the
characteristics
of
students’
grammatical
conceptual
knowledge
and
reasoning
and
to
investigate
whether
students’
reasoning
benefits
from
an
intervention
that
related
linguistic
metaconcepts
to
concepts
from
traditional
grammar.
Results
indicate,
among
other
things,
that
using
explicit
linguistic
metaconcepts
and
explicit
concepts
from
traditional
grammar
is
a
powerful
contributor
to
the
quality
of
students’
grammatical
reasoning.
Moreover,
the
intervention
significantly
improved
students’
use
of
linguistic
metaconcepts.
©
2019
The
Authors.
Published
by
Elsevier
Inc.
This
is
an
open
access
article
under
the
CC
BY-NC-ND
license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
1.
Introduction
Since
the
beginning
of
the
twenty-first
century
there
has
been
a
renewed
interest
in
grammar
teaching
in
L1
classrooms,
both
in
research
and
in
policy
making
(Hudson
&
Walmsley,
2005;
Locke,
2010).
This
interest
has
become
even
more
apparent
in
recent
years,
since
the
well-rehearsed
argument
emerging
in
the
1970s
that
grammar
education
has
no
impact
on
literacy
development
(e.g.,
Andrews,
2005;
Elley,
Barham,
Lamb,
&
Wylie,
1975;
Graham
&
Perin,
2007)
is
starting
to
crumble.
While
traditional
parsing
exer-
cises
generally
fail
to
improve
students’
writing,
there
is
a
growing
body
of
empirical
evidence
indicating
positive
effects
of
contextu-
alized
grammar
teaching
on
writing
development
(e.g.,
Fearn
and
Farnan,
2007;
Fontich,
2016;
Jones,
Myhill,
&
Bailey,
2013;
Myhill,
夽This
work
was
supported
by
the
Netherlands
Organisation
for
Scientific
Research
(NWO)
under
grant
number
023.009.034.
∗Corresponding
author
at:
Department
of
Teacher
Education,
Fontys
University
of
Applied
Sciences,
P.O.
Box
558,
6130
AN
Sittard,
The
Netherlands.
E-mail
addresses:
j.vanrijt@fontys.nl,
j.vanrijt@let.ru.nl
(J.H.M.
van
Rijt),
p.deswart@let.ru.nl
(P.J.F.
de
Swart),
astrid.wijnands@hu.nl,
astrid.wijnands@hu.nl
(A.
Wijnands),
p.a.coppen@let.ru.nl
(P.-A.J.M.
Coppen).
Jones,
&
Lines,
2018;
Myhill,
Jones,
Lines,
&
Watson,
2012;
Watson
&
Newman,
2017).
Most
grammar
teaching
in
L1
contexts
is
still
fairly
‘traditional’,
in
two
distinct
but
related
senses
(cf.
the
systematic
literature
review
of
Van
Rijt,
De
Swart,
&
Coppen,
2018).
First,
grammar
in
L1
contexts
is
mostly
traditional
in
terms
of
its
teaching
approaches
(e.g.
focusing
on
rules,
parsing
isolated
sentences,
labelling
parts
of
speech).
Second,
it
is
traditional
in
the
sense
that
it
uses
a
tra-
ditional
body
of
grammar
knowledge
(e.g.
structuralistic
parts
of
speech
terminology)
–
cf.
Van
Rijt
and
Coppen
(2017).
The
term
‘traditional’
henceforth
refers
to
both
pedagogical
and
linguistic
aspects
of
grammar
teaching
in
this
paper.
Elements
of
these
tradi-
tional
aspects
of
school
grammar
can
even
be
perceived
in
modern
pedagogical
approaches,
such
as
Halliday’s
Systemic
Functional
Grammar
(SFL)
(Halliday
&
Matthiessen,
2004;
see
also
Berry,
2016
and
Myhill,
2018).
For
example,
Jones
and
Chen
(2012)
and
Macken-
Horarik,
Love,
and
Horarik
(2018)
report
that
teachers
in
Australia
struggle
with
making
connections
between
traditional
grammat-
ical
terminology
and
rhetorical
choices
in
writing,
even
though
this
is
what
the
official
ACARA
curriculum,
which
leans
heavily
on
functional
grammar,
demands
of
them.
As
a
result,
teachers
resort
to
(more)
traditional
forms
of
grammar
teaching.
Much
of
the
L1
https://doi.org/10.1016/j.linged.2019.06.004
0898-5898/©
2019
The
Authors.
Published
by
Elsevier
Inc.
This
is
an
open
access
article
under
the
CC
BY-NC-ND
license
(http://creativecommons.org/licenses/by-nc-nd/4.
0/).
J.H.M.
van
Rijt
et
al.
/
Linguistics
and
Education
52
(2019)
78–88
79
grammar
teaching
across
the
globe
can
therefore
be
considered
traditional
in
either
a
pedagogical
or
a
linguistic
sense,
or
in
both
respects,
even
when
a
country’s
educational
ideology
promotes
something
different
(Fearn
and
Farnan,
2007;
Horn,
2003;
Lefstein,
2009;
Van
Rijt
et
al.,
2018;
Watson,
2015).
One
of
the
main
reasons
why
grammar
education
contends
with
a
traditional
image
relates
to
teacher
knowledge.
Research
shows
that
language
teachers
generally
lack
sufficient
metalinguistic
knowledge
(e.g.
Alderson
&
Hudson,
2013;
Sangster,
Anderson,
&
O’Hara,
2013;
Van
Rijt,
Wijnands,
&
Coppen,
2019)
and
experience
low
self-confidence,
even
anxiety,
in
the
linguistic
domain
(Giovanelli,
2015).
Teaching
grammar
based
on
real
insights
rather
than
rules
of
thumb
presents
teachers
with
severe
challenges,
both
when
teaching
grammar
in
isolation
and
when
teaching
grammar
effectively
in
the
context
of
writing
(cf.
Myhill,
Jones,
&
Watson,
2013).
Teaching
grammar
based
on
parsing
isolated
sentences
puts
less
cognitive
strain
on
teachers,
who
mostly
lack
the
knowledge
and
confidence
to
teach
grammar
insightfully.
Moreover,
research
into
teacher
beliefs
has
revealed
that
teachers
generally
tend
to
adopt
teaching
styles
that
match
the
content
and
pedagogies
which
they
have
experienced
themselves
as
learners
(e.g.
Phipps
&
Borg,
2009;
Watson,
2015).
This
way,
more
traditional
forms
of
grammar
teaching
remain
per-
sistent
in
education.
In
addition,
Van
Rijt
et
al.
(2018)
found
that
the
educational
literature
on
L1
grammar
teaching
is
not
up-to-date
with
insights
from
modern
linguistic
theory,
and
predominantly
addresses
traditional
grammatical
concepts
(e.g.,
subject,
noun,
and
verb)
rather
than
potentially
useful
(meta)concepts
from
modern
linguistics
(cf.
Section
1.1).
This
is
likely
a
result
of
current
practice
and
policy,
but
it
may
also
be
indicative
of
a
similar
lack
of
linguistic
knowledge
for
educational
researchers.
It
can
therefore
be
concluded
that
both
current
grammar
teach-
ing
and
research
into
L1
grammar
teaching
are
mainly
traditional.
Still,
traditional
grammar
education
is
subject
to
much
criticism
(Giovanelli,
2015;
Hudson,
2004),
the
main
points
of
critique
being
that
it
focuses
on
rules
of
thumb
and
lower-order
think-
ing
rather
than
on
real
conceptual
insights
(Berry,
2015;
Coppen,
2009;
Myhill,
2000;
Van
Rijt
et
al.,
2019)
and
that
it
is
chiefly
concerned
with
‘rules
and
compliance’
and
‘error
eradication
and
notational
rules’
(Myhill,
Jones,
&
Wilson,
2016;
Myhill
&
Newman,
2016).
Grammar
education
should
rather
be
about
talking
and
rea-
soning
about
language
on
an
informed
level.
Understanding
the
relevant
linguistic
concepts
and
metaconcepts
is
one
of
the
great-
est
challenges
for
grammar
teachers
(Fontich,
2016;
Hulshof,
2013;
Myhill,
2000;
Ribas,
Fontich,
&
Guasch,
2014).
This
holds
for
any
kind
of
grammar
education,
whether
its
goal
is
to
teach
cultural
knowledge,
to
develop
students’
literacies,
to
acquire
a
foreign
language,
or
to
stimulate
reasoning
skills
(see
Fontich
&
Camps,
2014
or
Van
Rijt
et
al.,
2018).
Such
an
understanding
is
even
more
important
for
students
who
study
languages
or
linguistics
at
uni-
versity,
where
they
are
expected
to
develop
an
academic
expertise
in
this
domain.
What
is
more,
a
fair
amount
of
these
students
will
become
teachers,
who
need
a
very
sound
grammatical
knowledge
in
their
language
teaching
(Alderson
&
Hudson,
2013;
Giovanelli,
2016;
Jones
&
Chen,
2012;
Myhill
et
al.,
2013;
Sangster
et
al.,
2013).
The
current
study
is
set
in
a
higher
education
context,
although
its
relevance
and
implications
extend
well
beyond
higher
educa-
tion.
Some
previous
research
has
focused
on
traditional
grammatical
concepts
in
the
context
of
writing
education
(e.g.,
Fontich,
2016;
Myhill
et
al.,
2012;
Myhill
&
Newman,
2016;
Watson
&
Newman,
2017).
From
these
studies,
it
has
become
clear
that
a
good
subject
knowledge
of
grammar
is
also
essential
for
teachers
in
effectively
making
connections
between
grammar
and
writing.
The
studies
cited
above
show
that
it
is
possible
to
teach
traditional
grammatical
concepts
without
adopting
traditional
pedagogical
means.
How-
ever,
making
students
reason
or
talk
about
traditional
grammar
in
writing
remains
difficult,
since
students
find
it
hard
to
conceptually
grasp
linguistic
metalanguage
(Watson
&
Newman,
2017).
In
line
with
this,
some
scholars
have
suggested
that
bridg-
ing
the
gap
between
linguistic
theory
and
L1
grammar
education
can
solve
the
problem
of
a
limited
conceptual
understanding
of
grammar
(e.g.,
Carter,
1982;
Hudson,
2004;
Mulder,
2010;
Van
Rijt
&
Coppen,
2017;
Van
Rijt
et
al.,
2018),
claiming
among
other
things
that
(meta)concepts
from
modern
linguistic
theory
can
be
used
to
strengthen
traditional
grammar
education.
For
example,
understanding
the
passive
voice
(a
notoriously
difficult
grammat-
ical
structure,
see
Myhill,
2003),
could
benefit
from
an
approach
that
focuses
on
the
so
called
‘mapping
problem’
in
the
framework
of
Lexical-Functional
Grammar
(cf.
Bresnan,
Asudeh,
Toivonen,
&
Wechsler,
2016),
introducing
modern
linguistic
metaconcepts
such
as
semantic
roles,
which
remain
undiscussed
in
traditional
grammar
(see
Van
Rijt
et
al.,
2018).
Another
example
is
how
the
metaconcept
of
valency
can
be
used
to
distinguish
between
(more
or
less)
oblig-
atory
and
non-obligatory
syntactic
elements
in
a
sentence
(i.e.,
the
difference
between
objects
and
adverbials)
–
see
Perini
(2015).
Although
theoretically
an
approach
uniting
(meta)concepts
from
modern
linguistic
theory
and
traditional
grammar
education
seems
promising,
no
empirical
evidence
exists
to
prove
its
effi-
cacy
for
linguistic
reasoning
skills
or
linguistic
understanding
(see
Hulshof,
2013).
The
current
exploratory
study
tries
to
fill
this
gap
by
investigating
the
effect
of
grammar
education
based
on
linguistic
(meta)concepts
on
students’
competence
in
grammatical
reason-
ing
and
linguistic
problem
solving.
More
specifically,
it
examines
whether
a
short
intervention
focusing
on
such
metaconcepts
in
the
context
of
traditional
grammar
has
positive
effects
on
the
quality
of
students’
grammatical
reasoning.
1.1.
Linguistic
concepts
and
metaconcepts
When
modern
linguistic
theory
is
turned
to
for
enriching
tradi-
tional
school
grammar,
the
question
arises
which
(meta)concepts
are
suitable
for
this
enrichment.
Different
linguistic
schools
(e.g.
generative
linguistics,
cognitive/construction
grammar,
SFL)
emphasize
different
aspects
of
sentence-level
linguistics,
holding
different
views
on
language
acquisition
and
structure.
To
avoid
adhering
to
any
one
particular
linguistic
school,
and
to
bene-
fit
from
the
full
width
of
modern
linguistic
theory,
Van
Rijt
and
Coppen
(2017)
have
conducted
a
Delphi
study
among
linguistic
experts
from
different
backgrounds.
The
experts
reached
a
gen-
eral
agreement
among
on
the
26
crucial
(meta)concepts
from
the
syntax-semantics
interface.
According
to
these
experts,
the
26
metaconcepts
were
very
suitable
for
enriching
school
grammar,
in
which
syntax
plays
a
large
role.1
Van
Rijt
and
Coppen
(2017)
and
Van
Rijt
et
al.
(2018)
argue
that
a
better
conceptual
understanding
of
grammar
can
be
achieved
by
making
these
(theory-neutral)
metaconcepts
the
target
of
gram-
mar
learning
and
instruction,
with
the
important
caveat
that
these
concepts
should
not
be
used
to
completely
replace
the
traditional
grammatical
terminology,
but
that
they
should
instead
be
related
to
concepts
from
traditional
grammar
education
(see
also
Mulder,
20102).
1The
metaconcepts
from
Van
Rijt
and
Coppen
(2017)
include
both
metaconcepts
from
modern
linguistic
theory
as
well
as
metaconcepts
that
can
also
be
found
in
traditional
grammar.
The
reason
for
this
is
that
even
though
modern
linguistics
has
generated
several
new
insights,
these
mostly
build
upon
and
expand
the
more
traditional
metaconcepts
(cf.
Van
Rijt
et
al.,
2018)
–
cf.
Allan
(2007).
2The
importance
of
not
simply
replacing
one
set
of
grammatical
concepts
for
another
cannot
be
overestimated
(Mulder,
2010).
It
would
be
unrealistic
to
assume
that
traditional
grammatical
terminology,
which
has
existed
since
classical
antiquity
(see
Seuren,
1998),
could
suddenly
be
replaced
by
a
more
modern
set
of
concepts,
since
this
will
lead
to
severe
clashes
with
the
existing
grammar
education
(see
Van
80
J.H.M.
van
Rijt
et
al.
/
Linguistics
and
Education
52
(2019)
78–88
Following
Kimball
(2007),
concepts
can
be
defined
as
‘abstract
realities
of
the
mind’
(see
also
Lipman,
2003,
p.
181),
whereas
metaconcepts
can
be
defined
as
‘[.
.
.]
the
mind’s
generalized
representation
of
one
or
more
concepts’.
In
other
words:
meta-
concepts
are
concepts
with
an
overarching
value,
which
can
be
used
for
conceptualization
or
rational
deduction,
mediating
stu-
dents’
understanding
of
underlying
subject-specific
concepts
(Van
Drie
&
Van
Boxtel,
2008;
Van
Rijt
et
al.,
2018).
In
grammar
edu-
cation,
for
instance,
‘valency’
could
be
considered
a
metaconcept,
which
can
be
used
to
enhance
understanding
of
other
grammatical
concepts,
such
as
direct
and
indirect
objects.
In
another
example,
the
metaconcept
of
modality
could
be
used
as
a
meaningful
over-
arching
metaconcept
to
discuss
traditional
concepts
such
as
modal
adverbs
and
modal
auxiliaries.
According
to
Gombert
(1992,
p.
191),
metalinguistic
under-
standing
encompasses
two
types
of
relevant
knowledge:
declar-
ative
knowledge,
which
is
the
knowledge
regarding
grammatical
content,
and
procedural
knowledge,
which
is
the
ability
to
effec-
tively
work
with
this
knowledge.
Declarative
and
procedural
knowledge
are
mutually
intertwined
(see
also
Moseley
et
al.,
2005),
and
thus
both
required
in
effective
grammar
education.
Ribas
et
al.
(2014)
make
a
similar
assumption
in
saying
that
‘(.
.
.)
there
are
close
ties
between
grammatical
concepts
and
studying
and
reflect-
ing
procedures,
and
that
the
latter
are
not
merely
an
unimportant,
superficial
part
of
the
way
language
is
perceived
and
language
knowledge
is
constructed’
(2014,
p.
15).
In
the
current
study,
it
is
assumed
that
the
declarative
knowl-
edge
of
grammar
should
entail
more
than
just
the
concepts
from
traditional
grammar,
but
also
(meta)concepts
from
modern
lin-
guistic
theory.
It
is
also
assumed
that
the
concomitant
procedural
knowledge
should
involve
more
than
just
the
rules
of
thumb
and
audit
questions
that
are
dominant
in
traditional
grammar
edu-
cation.
Rather,
working
with
modern
linguistic
(meta)concepts
requires
a
type
of
reasoning
that
is
more
common
for
linguistic
experts
(Fontich
&
García-Folgado,
2018,
p.
31;
Honda
&
O’Neil,
2007;
Kuiper
&
Nokes,
2014,
Ch.
1;
Tallerman,
2015).
When
lin-
guists
are
trying
to
grasp
a
syntactic
structure,
they
will
try
to
syntactically
manipulate
the
sentence
under
scrutiny,
for
instance
by
constructing
an
analogous
example,
by
topicalizing
constituents
or
by
verifying
whether
a
certain
element
can
be
omitted.
This
repertoire
of
linguistic
reasoning
components
is
crucially
impor-
tant
for
linguists
trying
to
describe
the
language
reality,
because
although
traditional
grammar
education
suggests
otherwise,
most
real-life
sentences
cannot
be
parsed
or
analyzed
unambiguously
(Coppen,
2009),
in
part
because
conceptual
categories
themselves
are
sometimes
‘fuzzy’
(Kuiper
&
Nokes,
2014).
The
present
study
is
focused
on
grammar
teaching
per
se,
independent
of
any
con-
textualization.
Therefore,
findings
are
also
of
importance
in
more
contextualized
approaches
to
grammar
(see
discussion
section).
1.2.
The
current
study
There
are
good
theoretical
grounds
to
assume
that
a
pedagogical
approach
to
grammar
teaching
targeting
linguistic
metaconcepts
and
linguistic
reasoning
is
crucial
for
a
deeper
understanding
of
the
subject
matter.
The
current
study
is
the
first
to
empirically
explore
the
role
of
both
linguistic
(meta)concepts
and
appurtenant
linguistic
reasoning
in
L1
grammar
education,
focusing
on
univer-
Rijt
et
al.,
2019).
Moreover,
according
to
Van
der
Aalsvoort
and
Kroon
(2015)
it
is
of
great
importance
that
teachers
do
not
feel
‘overruled’
by
academics,
who
sometimes
hold
different
views
on
which
content
should
be
considered
important.
Instead,
academics
and
teachers
should
join
forces
and
opt
for
a
state
of
cooperation,
in
which
the
academic
discipline
and
the
related
school
subject
can
mutually
benefit
from
one
another.
sity
students
of
Dutch
Language
and
Literature.
We
aimed
to
answer
the
following
research
questions:
(1)
What
are
the
characteristics
of
students’
grammatical
prob-
lem
solving
in
terms
of
their
grammatical
or
linguistic
(meta)concept
use
(declarative
knowledge)
and
their
linguistic
reasoning
strategies
(procedural
knowledge)?
(2)
What
characterizes
good
quality
grammatical
reasoning?
(3)
What
is
the
effect
of
a
short
intervention
aimed
at
fostering
lin-
guistic
metaconcepts
and
linguistic
manipulations
on
students’
grammatical
reasoning?
2.
Method
2.1.
Participants
24
first
year
students
of
Dutch
Language
and
Literature
from
a
large
university
in
the
Netherlands
(9
males)
voluntarily
partic-
ipated
in
the
study.
All
students
signed
a
consent
form
in
which
they
stated
that
their
data
could
be
used
for
scientific
research
anonymously.
In
addition,
the
department
of
Dutch
Language
and
Literature
approved
of
the
investigation.
In
the
pre-test,
23
students
participated
due
to
the
illness
of
one
student.
In
the
post-test,
22
students
took
part
(again
with
drop-outs
due
to
illness).
For
the
measurements
that
relate
to
the
effect
of
the
intervention,
only
the
data
from
students
that
were
present
at
both
the
pre-
and
post-test
could
be
taken
into
account.
In
all
other
measurements,
all
available
data
was
taken
into
account.
2.2.
The
intervention
In
total,
students
were
exposed
to
twelve
hours
of
contact
time
in
the
intervention,
consisting
of
a
mixture
of
lectures
and
seminars.
Prior
to
each
seminar,
an
assignment
had
to
be
submitted
which
consisted
of
reasoning
tasks
that
were
aimed
at
applying
the
declar-
ative
and
procedural
knowledge
described
in
the
present
paper.
The
intervention
the
students
participated
in
was
designed
in
such
a
way
that
metaconcepts
from
modern
linguistic
theory
would
be
the
focus
of
the
course,
making
appropriate
connections
between
them
and
traditional
parts
of
speech.
The
intervention
focused
on
four
somewhat
related
metaconcepts:
predication
(cf.
van
Eynde,
2015),
valency
(cf.
Perini,
2015),
complementation
(Perini,
2015)
and
modification
(cf.
Morzycki,
2015),
although
if
the
occasion
called
for
it,
other
metaconcepts
were
taken
into
account
as
well.
Sev-
eral
more
modern
concepts,
related
to
these
metaconcepts,
were
also
introduced,
involving
concepts
such
as
agent,
patient,
argument
and
adjunct.
These
concepts
can
all
be
used
well
to
enrich
under-
standing
of
traditional
parts
of
speech.
All
main
traditional
parts
of
speech
were
covered
in
the
intervention.
(See
e.g.,
Tallerman
(2015),
or
Van
Rijt
and
Coppen
(2017)
for
general
explanations
of
the
concepts
from
the
intervention.)
In
Appendix
1,
an
overview
of
the
intervention
is
given.
Finally,
the
second
author
of
this
paper,
who
acted
as
the
inter-
ventions’
instructor,
also
paid
attention
to
linguistic
reasoning
in
the
form
of
good
practices,
demonstrating
how
linguists
apply
and
combine
subject-specific
procedural
and
declarative
knowledge.
The
intervention
was
implemented
in
the
first
term
of
the
aca-
demic
year,
meaning
that
students
only
had
their
knowledge
from
secondary
school
Dutch
as
a
basis.
After
finishing
pre-university
secondary
education,
students
are
expected
to
master
Dutch
at
a
level
that
is
comparable
to
the
C1
level
of
the
Common
European
Framework
(but
see
endnote
3).
J.H.M.
van
Rijt
et
al.
/
Linguistics
and
Education
52
(2019)
78–88
81
2.3.
Pre-test
and
post-test
Prior
to
and
after
the
intervention,
the
participating
students
individually
tackled
a
set
of
grammatical
problems
consisting
of
sentences
that
could,
in
principle,
not
be
analyzed
unambiguously
in
traditional
grammar,
or
they
had
to
explain
why
one
sentence
could
be
considered
grammatical,
whereas
another
one
could
not
(e.g.,
Jan
regent
nat
(‘Jan
is
raining
wet’,
in
literal
translation)
versus
Jan
regent’*
(‘*Jan
rains’).
In
the
case
given,
only
het
(‘it’)
can
be
combined
with
the
verb
regenen
(‘to
rain’)
in
Dutch,
unless
a
com-
plement
is
added,
in
which
the
adjective
nat
(‘wet’)
is
predicated
of
Jan.
Nat
is
a
secondary
predicate
in
such
cases,
and
the
construc-
tion
as
a
whole
receives
a
resultative
interpretation,
meaning
that
it
rains
in
such
a
way
that
Jan
becomes
wet
(cf.
Broekhuis
&
Corver,
2015,
p.
239)
–
something
that
cannot
be
effectively
explained
in
traditional
school
grammar,
in
which
metaconcepts
such
as
pred-
ication
and
complementation
play
no
role.
See
Appendix
2
for
a
short
overview
of
the
grammatical
problems
that
students
had
to
tackle.
In
both
the
pre-
and
post-test,
students
were
invited
to
solve
four
grammatical
problems
as
extensively
as
possible,
two
of
which
were
designed
in
such
a
way
that
they
could
elicit
the
metacon-
cepts
from
the
intervention,
whereas
the
other
two
problems
were
filler
items
that
related
to
grammatical
phenomena
that
were
too
advanced
for
the
students
to
tackle,
since
they
dealt
with
topics
they
had
not
yet
been
educated
in
(e.g.,
binding).
The
filler
items
were
meant
to
measure
whether
any
increase
or
decrease
in
progress
could
not
simply
be
attributed
to
a
general
increase
in
cognitive
ability.
To
nullify
any
effects
related
to
the
difficulty
or
order
of
the
tasks,
the
group
was
randomly
divided
in
two
and
the
pre-
and
post-test
items
were
counterbalanced
(Shadish,
Cook,
&
Campbell,
2002,
p.
109).
The
post-test
was
conducted
five
weeks
after
the
pre-
test.
Students
submitted
their
data
online,
via
Qualtrics.
In
total,
this
led
to
180
student
reasonings
about
grammatical
problems,
90
of
which
related
to
target
items,
and
90
related
to
filler
items.
2.4.
Quality
of
students’
reasoning
To
assess
the
quality
of
the
students’
reasoning,
an
independent
panel
consisting
of
four
experienced
(full)
professors
of
Linguistics
was
asked
to
individually
rate
the
reasonings
on
a
Likert
scale,
rang-
ing
from
1
(‘very
poor’)
to
5
(‘excellent’).
These
professors
were
not
informed
of
the
aim
or
context
of
the
study,
nor
were
they
aware
that
an
intervention
had
taken
place.
To
rule
out
any
possible
order
effects,
the
order
of
student
analyses
was
completely
randomized
for
each
of
the
rating
professors.
To
assess
the
inter-rater
reliability,
a
two-way
mixed,
absolute,
average-measures
intra-class
correla-
tion
(ICC)
was
used
(McGraw
&
Wong,
1996).
The
resulting
ICC
can
be
considered
good
(ICC
=
.70),
(see
Cicchetti,
1994),
indicating
a
high
degree
of
agreement
among
raters.
2.5.
Qualitative
analysis
of
students’
reasoning
The
student
reasonings
were
analyzed
qualitatively
and
induc-
tively
(cf.
Cohen,
Manion,
&
Morrison,
2011;
Strauss
&
Corbin,
1990)
to
avoid
missing
any
relevant
data,
following
the
constant
compar-
ison
method.
The
first
and
fourth
author
of
the
current
paper
first
engaged
in
open
coding
via
Atlas.ti,
aiming
to
capture
any
possibly
relevant
data
regarding
grammatical
concepts
and
linguistic
rea-
soning.
The
analysis
was
guided
by
sensitizing
concepts
(cf.
Bowen,
2006)
from
traditional
grammar
and
the
metaconcepts
from
Van
Rijt
and
Coppen
(2017).
Once
the
open
coding
had
been
completed,
we
iteratively
and
systematically
re-examined
our
prior
coding
to
achieve
consistency.
Both
researchers
reached
absolute
agreement
on
the
coding,
solving
any
differences
in
opinion
through
discus-
sion.
In
the
next
stage,
axial
coding
clustered
the
data
into
thematic
groups.
For
the
linguistic
(meta)concepts,
we
distinguished
between
explicit
occurrences
and
implicit
occurrences.
If
a
student
would
describe
a
particular
concept
in
detail
without
labelling
it,
it
was
coded
as
an
implicit
mention,
allowing
us
to
measure
the
effects
of
explicit
metalinguistic
knowledge
versus
implicit
metalinguistic
knowledge.
A
typical
example
is
that
students
would
sometimes
refer
to
‘the
person
carrying
out
the
action’,
which
we
coded
as
an
implicit
reference
to
the
concept
of
agent.
In
the
present
study,
therefore,
it
is
assumed
that
declarative
knowledge
can
manifest
itself
both
implicitly
and
explicitly.
For
each
of
the
180
student
reasonings,
the
number
of
times
a
code
occurred
was
registered.
Finally,
for
each
reasoning
it
was
determined
whether
its
main
focus
was
on
meaning
(semantic)
or
on
form
(syntactic).
An
example
of
a
prototypical
reasoning
with
a
semantic
focus
is:
‘Jan
is
the
subject,
and
in
this
sentence
it
appears
as
if
though
Jan
was
raining.
This
is
anatomically
impossible.
It
is
however
possible
for
Jan
to
become
wet
due
to
the
rain.’
A
proto-
typical
example
with
syntactic
focus
is:
‘The
verb
to
rain
normally
does
not
go
accompanied
with
a
subject
complement,
since
to
rain
is
normally
never
a
linking
verb.
In
the
sentence
Jan
is
nat,
which
is
less
marked,
nat
does
form
a
logic
subject
complement.
Zijn
is
a
linking
verb
in
that
sentence.’
2.6.
Analysis
of
good
quality
grammatical
reasoning
To
analyze
which
variables
were
most
strongly
related
to
the
quality
of
students’
grammatical
reasoning,
multiple
regression
analyses
were
carried
out.
Because
the
grammatical
reasonings
are
nested
within
students,
a
multilevel
design
was
adopted
which
allowed
for
a
random
effect
of
students
on
the
intercept
and
with
a
fixed
slope.
This
design
was
used
because
it
was
assumed
that
there
may
be
differences
between
students,
but
that
the
effect
of
the
various
types
of
declarative
and
procedural
knowledge
is
sim-
ilar
for
all
student
reasonings.
The
ICC
indicated
that
11.67%
of
the
variance
in
grammatical
reasoning
ability
can
be
explained
by
dif-
ferences
between
students.
This
percentage
is
sufficient
to
adopt
a
multilevel
design
for
the
multiple
regression.
First,
we
examined
which
types
of
declarative
knowledge
could
best
predict
the
quality
of
students’
reasoning,
taking
into
account
both
implicit
and
explicit
concepts.
Second,
variables
related
to
procedural
knowledge
were
the
target
of
a
separate
multiple
regression
analysis.
Finally,
the
significant
predictors
of
these
two
models
were
combined
to
generate
the
best
possible
predicting
regression
model.
In
all
of
these
steps,
a
random
effect
of
student
on
intercept
was
taken
into
account.
2.7.
Analysis
of
the
effect
of
the
intervention
To
measure
whether
the
intervention
had
a
significant
impact
on
the
general
quality
of
students’
reasoning,
independent
sam-
ples
t-tests
were
undertaken.
Independent
samples
t-tests
were
also
used
to
determine
whether
the
intervention
could
evoke
sig-
nificantly
more
metaconcepts
in
the
students’
reasoning.
Given
the
fact
that
these
students
have
experienced
years
of
traditional
gram-
mar
education,
it
is
not
unthinkable
that
metaconcept
use
cannot
be
stimulated
in
a
short
intervention,
especially
given
that
the
inter-
vention
did
not
resemble
their
previous
experiences
in
explicit
grammar
instruction.
Additionally,
using
chi-squared
tests,
it
was
tested
whether
stu-
dents
focused
more
on
syntax
than
on
semantics
in
the
post-test
compared
to
the
pre-test.
82
J.H.M.
van
Rijt
et
al.
/
Linguistics
and
Education
52
(2019)
78–88
Table
1
Axial
codes
with
examples
from
the
data.
Example
of
open
code(s)
related
to
axial
code
Example
from
data
Linguistic
(meta)concepts
(declarative)
Traditional
grammatical
concepts Subject,
preposition,
etc. [.
.
.]
It
is
strange
that
‘Jan’is
the
subject
in
this
sentence
[.
.
.]
Modern
grammatical
concepts
Agent,
argument,
adjunct,
etc.
To
rain
is
an
intransitive
verb,
and
does
not
call
for
a
human
agent.
Metaconcepts
Valency,
predication,
modification,
etc.
Raining
has
a
valency
of
0.
Linguistic
reasoning
strategies
(procedural)
Rules
of
thumb
Audit
question,
rule
of
thumb
[.
.
.]
If
you
subsequently
ask
the
question
‘what
has
Jan
walked?’
the
answer
is
‘a
round’.
Inferences
Simple
deductions,
correlations
‘If
X
is
omissible,
it
is
an
adverbial’.
Linguistic
manipulations
Topicalization,
replacement,
constructed
examples,
etc.
‘If
you
would
use
the
verb
in
an
abstract
manner,
you
would
get
a
structure
like
‘something
tastes
of
something’.
3.
Results
3.1.
Characteristics
of
students’
analyses
The
overall
quality
of
students’
reasoning
varied
from
1
to
4.5
(M
=
2.44,
SD
=
0.75).
A
prototypical
example
of
a
poor
analysis
(mean
score
1.50)
is
the
following:
‘I
think
X
is
a
direct
object.
The
answer
to
the
question
‘who
or
what
walks
Jan’
is
namely
X.’
(Translation
by
the
authors.)
Such
an
analysis
is
characterized
by
the
application
of
an
audit
question
without
any
further
reflection
or
consideration.
Moreover,
the
analysis
reveals
a
general
lack
of
conceptual
understanding.
An
example
of
a
good
analysis
(mean
score
4.00):
‘The
verb
to
walk
is
monovalent.
In
sentence
1,
a
semantic
role
is
served
out
to
‘Jan’,
making
this
sentence
grammatically
correct.
In
sentence
2,
the
predicate
is
‘stuklopen’.
This
verb
is
divalent
(SOMEONE
WALKS
SOMETHING
BROKEN).
‘Jan’
again
receives
a
role.
The
other
role
goes
to
‘his
shoes’.
This
sentence
too
is
therefore
grammatically
correct.
In
sentence
3,
the
predicate
is
‘to
walk’,
just
as
in
sentence
1.
Here,
there
are
however
two
constituents
requiring
a
role:
‘Jan’
and
‘his
shoes’.
Subsequently,
this
sentence
is
ungrammatical.’
In
this
analysis,
an
explanation
is
given
based
on
linguistic
con-
cepts
(valency
patterns,
a
subcategorisation
frame
and
semantic
roles),
which
are
actively
used
in
the
reasoning.
Moreover,
the
stu-
dent
comes
to
a
clear
conclusion
based
on
the
application
of
these
concepts.
The
student
reasonings
counted
an
average
of
50.41
words
(SD
=
26.99,
range
=
14–143).
In
the
pre-test,
students
wrote
an
average
of
51.59
words
(SD
=
29.22),
whereas
in
the
post-test,
the
number
of
words
used
averaged
at
49.28
(SD
=
24.78).
There
was
no
significant
difference
between
these
measurements
in
terms
of
the
number
of
words
used
(t(178)
=
.57,
p
=
.57).
Qualitative
analysis
resulted
in
a
series
of
open
codes,
which
were
reduced
and
related
t