Content uploaded by Aleidine Moeller
Author content
All content in this area was uploaded by Aleidine Moeller on Jun 27, 2018
Content may be subject to copyright.
Goal Setting and Student
Achievement: A Longitudinal Study
ALEIDINE J. MOELLER
University of Nebraska
Department of Teaching,
Learning and Teacher Education
118 Henzlik Hall
Lincoln, NE 68588–0355
Email: amoeller2@unl.edu
JANINE M. THEILER
University of Nebraska
Department of Teaching,
Learning and Teacher Education
115 Henzlik Hall
Lincoln, NE 68588–0355
Email: jtheiler@windstream.net
CHAORONG WU
University of Nebraska
Department of Educational
Psychology
248 Teacher’s College Hall
Lincoln, NE 68588–0345
Email: wuchaorong@huskers.
unl.edu
The connection between goals and student motivation has been widely investigated in the
research literature, but the relationship of goal setting and student achievement at the class-
room level has remained largely unexplored. This article reports the findings of a 5-year
quasi-experimental study examining goal setting and student achievement in the high school
Spanish language classroom. The implementation of LinguaFolio, a portfolio that focuses on
student self-assessment, goal setting, and collection of evidence of language achievement, was
introduced into 23 high schools with a total of 1,273 students. By using a hierarchical linear
model, researchers were able to analyze the relationship between goal setting and student
achievement across time at both the individual student and teacher levels. A correlational anal-
ysis of the goal-setting process and language proficiency scores reveals a statistically significant
relationship between the goal-setting process and language achievement (p<.01).
MOVING TO A STANDARDS–BASED,
student-centered learning environment has
required a deeper understanding and inves-
tigation of the factors that influence student
achievement, such as motivation, self-regulation,
ability, effort, time management, self-assessment,
and persistence. Research that has examined
the connection of the learning environment,
goals, and student motivational outcomes has
contributed significantly to our understanding
and has set the stage for the next important
research step: determining how to facilitate the
writing of goals in the classroom and to examine
the relationship between goal setting and student
achievement (Ames, 1992b).
LinguaFolio, a standards-based, self-directed,
formative assessment tool designed to increase
learner autonomy through a carefully structured
goal-setting process, was used as an intervention to
The Modern Language Journal,96,ii,(2012)
DOI: 10.1111/j.1540-4781.2011.01231.x
0026-7902/11/153–169 $1.50/0
C
⃝2011 The Modern Language Journal
determine the relationship between goal setting
and student achievement. The purpose of this ar-
ticle is to report the findings of a 5-year study with
23 school districts that implemented LinguaFolio
in their Spanish language classrooms.
LITERATURE REVIEW
Goal Orientation
Broadly defined, goal setting is the process of es-
tablishing clear and usable targets, or objectives,
for learning. Goal theory proposes that there are
two general goal orientations students can adopt:
a task-focused orientation with an intrinsic focus
on learning and improving and an ability-focused
orientation with an extrinsic focus on external re-
wards (e.g., getting good grades and doing better
than other students). The former is commonly
referred to as learning, task involvement, or mas-
tery goals and the latter is labeled as performance
or ego-involving goals (Dweck, 1986; Dweck &
Leggett, 1988; Elliott & Dweck, 1988).
Extensive research has linked mastery and per-
formance achievement goals to very distinct ways
154 The Modern Language Journal 96 (2012)
of thinking about oneself and learning activities.
Amastery goal fosters a motivational pattern as-
sociated with a deeper level of engagement that
secures and maintains achievement behavior. This
deeper level of engagement promotes internaliza-
tion of the connection between effort and achieve-
ment (Weiner, 1979). A performance goal fosters
apatternofmotivationassociatedwithfailure
avoidance (Covington, 1984; Dweck, 1986; Dweck
& Leggett, 1988; Elliott & Dweck, 1988; Nicholls,
1984, 1989; Nicholls, Patashnick, & Nolen, 1985).
Students who use performance goals are focused
on how they will be judged and attribute results
to lack of ability.
The effort–achievement connection of mastery
goal orientation is supported by evidence (Ames
& Archer, 1988; Nicholls et al., 1985) that links
mastery goals to an attribution belief that effort
leads to success. With a mastery goal, individuals
are oriented toward developing new skills, trying
to understand their work, improving their level
of competence, or achieving a sense of mastery
based on self-referenced standards (Ames, 1992a;
Brophy, 1983; Meece, Blumenfeld, & Hoyle, 1988;
Nicholls, 1989). This goal construct is congru-
ent with Brophy’s description of a “motivation to
learn,” whereby individuals are focused on mas-
tering and understanding content and demon-
strating a willingness to engage in the process
of learning. Self-efficacy—the belief that one can
succeed at something—plays a significant role in
motivation. Self-efficacy is domain-specific and is
dependent on past experiences within a certain
context. If an individual succeeds at something,
he or she will remain motivated. If he or she fails,
efficacy may be low. Self-efficacy influences an in-
dividual’s choice of activities, level of effort, per-
sistence, and emotional reactions to success or
failure (Bandura, 1997; Zimmerman, 2000).
A mastery goal is associated with a wide range
of motivation-related variables that contribute to
positive achievement and that are necessary medi-
ators of self-regulated learning (Ames, 1992a). Ac-
cording to Jagacinski and Nicholls (1984, 1987),
when mastery goals are adopted, pride and satis-
faction are associated with successful effort, and
guilt is associated with inadequate effort (Wentzel,
1987, as cited in Wentzel, 1991). Mastery goals
have also been associated with a preference for
challenging work and risk-taking (Ames & Archer,
1988; Elliott & Dweck, 1988; Meece et al., 1988;
Stipek & Kowalski, 1989) and positive attitudes to-
ward learning (Ames & Archer, 1988; Meece et al.,
1988).
Conversely, performance goals focus on one’s
ability and sense of self-worth (Covington, 1984;
Dweck, 1986; Nicholls, 1984). Achievement is
measured by doing better than others and, more
importantly, the recognition that results from
such superior achievement. Learning is viewed
only as a way to achieve a desired goal (Nicholls,
1979, 1989). Performance-based goals emphasize
the connection between ability and outcome, and
a person’s self-worth is determined by a percep-
tion of an individual’s ability to perform (Coving-
ton & Berry, 1976; Covington & Omelich, 1984).
As a result, the expenditure of effort can threaten
self-concept of ability when trying hard does not
lead to success.
Goal Setting and the Autonomous Learner
Autonomy is “the ability to take responsibility
for one’s learning,” as defined by Benson (2001),
Dickinson (1987), and Holec (1981). It has been
established that autonomy is a long-term aim of
education (Candy, 1988; Pennycook, 1997) and a
key factor in successful language learning. The re-
cent paradigm shift in language education from
teacher- to student-centered learning further em-
phasizes the importance of self-regulated and au-
tonomous learning. Thus, it is important that
learners develop responsible attitudes and auton-
omy (Scharle & Szab´
o, 2000). Benson, Dickinson,
and Holec argued that autonomy is not innate but
develops through learner training; that is, learn-
ers need to be taught learning strategies and how
to use them. Thus, it is important to consider
processes or activities by which teachers might
overtly guide their learners toward increased
autonomy.
Goal setting in language learning is commonly
regarded as one of the strategies that encour-
ages learner autonomy (Locke, Shaw, Saari, &
Latham, 1981; Wentzel, 1991; Yang, 1998). A num-
ber of studies indicate that goal setting affects per-
formance and enhances achievement (Boekaerts,
2002; Edwins, 1995; Griffee & Templi, 1997; Mo-
riarity, Pavelonis, Pellouchoud, & Wilson, 2001;
Schunk, 2003). In particular, studies have shown
that appropriate goal setting, along with timely
and specific feedback, can lead to higher achieve-
ment, better performance, a high level of self-
efficacy, and self-regulation. In spite of this com-
pelling evidence in support of goal setting, 85%
of individuals responded “no” when asked “Were
you taught how to set goals in school?” (Bishop,
2003). The case for goal setting has clearly been
made and supported by research studies, yet this
important learning strategy has been largely ig-
nored in classrooms.
It is important to emphasize that simply set-
ting one’s own goals would not necessarily im-
prove achievement (Schunk, 2003). There are a
Aleidine J. Moeller, Janine M. Theiler, and Chaorong Wu 155
number of important factors that must be con-
sidered, and effective goal properties are among
them. Research offers various models describ-
ing quality goals. Some researchers identify dif-
ficulty, specificity, and proximity (Schunk, 2003;
West & Thorn, 2001) as key features of effective
goals, whereas others state that high-quality goals
should be SMART; that is, learning goals should
be specific, measurable, attainable, relevant, and
time bound (Doran, 1981; Miller & Cunningham,
1981). Studies have found that higher results are
achieved if goals are specific, measurable, and
challenging (D¨
ornyei, 2001; Locke et al., 1981;
Pintrich & Schunk, 1996), not unrealistic or out-
side the student’s capacity.
Researchers emphasize that for goal setting to
improve performance, students should be allowed
to participate in setting their own goals (Azevedo,
Ragan, Cromley, & Pritchett, 2002; Tubbs, 1986,
as cited in Griffee & Templi, 1997). Participatory
goal theory states that students who choose their
own goals perform at higher levels than students
who have goals set for them (Mento, Steel, & Kar-
ren, 1987, as cited in Griffee & Templi, 1997).
Research reveals that many educators often cre-
ate their own learning goals and rarely encour-
age students to adapt these goals to their per-
sonal needs and interests (Marzano, Pickering, &
Pollock, 2001). Many overestimate their students’
ability to set their own learning goals (Boekaerts,
2002). Whereas teachers commonly set specific
goals or teaching outcomes for a class, these goals
can be quite distinct from the goals that the stu-
dents themselves are pursuing during the same
class. According to D¨
ornyei (2001), these differ-
ences between teacher and student goals can lead
to a lack of connection between the teacher’s “of-
ficial class goal” (p. 59) and that of an individual
student. This disconnect can, in turn, result in a
lack of understanding on the part of the students
as to how and why they are involved in the learn-
ing process.
Students may find intrinsic value, attainment
value, or utility value when they participate in a
learning task (Cross & Steadman, 1996). When
students do not understand the goal of a task
or do not invest themselves in a task, there is a
lack of ownership in the learning. The value of
the learning task is diminished, thereby affecting
their motivation to engage in that task. Connect-
ing learning tasks with students’ goals increases
the value of the task and thus increases motiva-
tion. In the classroom, identifying goals increases
motivation by assigning value to learning tasks and
connecting learning tasks to students’ own objec-
tives. When students can attach personal value
to tasks that are assigned to them, tasks become
purposeful and students are more willing to meet
the costs of achievement. Researchers echo this
and stress that effective goals are not simply im-
personal “outcomes to shoot for” (D¨
ornyei, 2001,
p. 82) but rather standards by which students
can evaluate their own performance and which
mark their progress. Goals designed and eval-
uated by students themselves are more authen-
tic and meaningful to the students (Bellanca &
Fogarty, 1991, as cited in Moriarity et al., 2001).
Boekaerts (2002) indicated that an optimal strat-
egy is a combination of a learning goal set by a
student and approved by the teacher. In this case,
a learning goal becomes a joint agreement of both
sides to “invest efforts.” This agreed-upon learn-
ing goal has “a better chance of being accom-
plished” (Boekaerts, 2002, p. 18). Within this type
of educational environment, students are more in-
trinsically motivated to produce high-quality work
because they are not simply doing an assignment,
fulfilling a requirement, or preparing for a test—
they are taking a step toward reaching their own
aspirations. This type of instruction also creates
an environment conducive to motivating students
to engage in their own learning process. This is
a cycle in which “to be motivated, students must
consciously participate in the learning environ-
ment of the classroom ... on the other hand, to
motivate students, learning environments must of-
fer opportunities that will invite students’ efforts
and participation ... tasks must be engaging and
meaningful” (Turner, 1995, p. 413).
According to research conducted by Oxford
and Shearin (1994), “goal setting can have ex-
ceptional importance in stimulating L2 [second
language] learning motivation, and it is there-
fore shocking that so little time and energy
are spent in the L2 classroom on goal setting”
(p. 129). To effectively integrate goal setting into
the curriculum, educators must be familiar with
interventions that facilitate this process as well as
how to effectively implement such interventions.
Which classroom interventions lead to a mastery
goal orientation that may contribute to enhanced
language achievement? How do we implement
these interventions such that students focus on ef-
fort versus ability, develop intrinsic interest in lan-
guage learning, and make use of effective learning
strategies?
According Ames (1992b), instructional inter-
ventions must connect with all aspects of the in-
structional plan and design:
Comprehensive intervention requires attention
to salient classroom structures, identification of
156 The Modern Language Journal 96 (2012)
principles and strategies that can be mapped onto
these structures, and a generation of exemplary prac-
tices that can be integrated into all curriculum areas
and within all aspects of day-to-day classroom routine.
(p. 268)
As a result of changing classroom plans and de-
sign, teachers may need to adjust their goals for
learning and their belief systems (Good, Grouws,
Mason, Slavings, & Cramer, 1990; Marshall, 1988;
Nicholls, Cheung, Lauer, & Patashnick, 1989;
Paris & Newman, 1990). A classroom goal-setting
intervention should consist of explicitly teaching
and illustrating the connection between effort
and achievement. LinguaFolio, whose building
blocks consist of the very principles established
through research, therefore served as an appro-
priate, if not ideal, intervention to explore goal
theory and student achievement at the classroom
level. An examination of the evolution of the Eu-
ropean Language Portfolio will assist in more fully
understanding the origin, purpose, and adapta-
tion of LinguaFolio in the United States.
The European Language Portfolio
The unification of European nations led to in-
creased mobility of individuals among and be-
tween countries, creating the need for unified
benchmarks that identified language skills and
competencies required for employment and edu-
cational purposes. The European Language Port-
folio (ELP) was developed by the Council of Eu-
rope (2010) as a product of its comprehensive
frameworks for foreign language education. Its
purpose was to accompany, reinforce, and guide
any foreign language curriculum that aims to pro-
vide its learners with a communicative approach
to language education while facilitating and rein-
forcing lifelong learning skills essential to success
in any activity outside the classroom.
The ELP has two basic functions: a reporting
function and a pedagogical function (Little & Per-
clov´
a, 2001). The reporting function serves as a
means for students to literally display the skills
they have acquired throughout their learning ex-
perience. This function is especially important for
students who are applying for jobs or universi-
ties and want to attract potential employers or
schools by exhibiting their language competen-
cies. It is important to note, however, that the
portfolio is not intended to substitute for official
certificates or diplomas that are awarded as the
result of formal tests but rather to “supplement
them by presenting additional information about
the owner’s experience and concrete evidence of
his or her foreign language achievements” (Little
& Perclov´
a, 2001, p. 3).
The pedagogical function of the ELP com-
prehensively addresses the communicative skills
involved in language learning and how these
skills are being taught, used, and acquired in
the classroom. This function is defined by Little
and Perclov´
a (2001) in their guide for teachers
and teacher trainers as “a means of making the
language learning process more transparent to
learners, helping them to develop their capacity
for reflection and self-assessment, and thus en-
abling them gradually to assume more and more
responsibility for their own learning” (p. 3). In
this view, the ELP claims to promote and develop
learner autonomy and motivation with sufficient
guidance from the teacher. This is thought to be
achievable through continuous practice of reflec-
tion and self-assessment, metacognitive skills that
are regularly utilized when working with the port-
folio.
LinguaFolio
The National Council of State Supervisors of
Foreign Languages (NCSSFL) adopted LinguaFo-
lio as its official project for the 2005 Year of Lan-
guages. Building on the knowledge and insights
gleaned from the European case studies and ex-
periences with the ELP, NCSSFL created a version
of such a portfolio for American schools that was
named LinguaFolio.
LinguaFolio is a portfolio that focuses on build-
ing autonomous learners through student self-
assessment, goal setting, and collection of evi-
dence of language achievement. LinguaFolio pro-
vides students with strategies to improve achieve-
ment, transforms standards into classroom goals
in the form of “can do” statements, informs stu-
dents of short- and long-term goals, and involves
students directly in the assessment process. This
formative assessment tool consists of three com-
ponents:
1. Language Biography: Students record infor-
mation about current and past experiences with
language as well as their learning habits and strate-
gies.
2. Language Passport: Students assess their
own language skills in the form of “can do” state-
ments to identify their level of language profi-
ciency and to follow their growth in proficiency.
3. Dossier of Evidence: Students identify goals,
create an action plan that details the path to
goal attainment, and provide examples of their
work that serves as evidence of accomplishment
Aleidine J. Moeller, Janine M. Theiler, and Chaorong Wu 157
of learning goals. A final step involves student re-
flection to determine at what level the goal was
accomplished. (NCSSFL, 2010)
The LinguaFolio goal-setting process (the
Dossier of Evidence component) involves stu-
dents directly in the learning process as they keep
track of learning goals and track progress toward
these goals. At the beginning of each new the-
matic unit, textbook chapter, or learning unit,
students identify and record their learning goals
for the chapter and address one or more of the
four skills: reading, writing, listening, and speak-
ing. An action plan is created that delineates how
these goals will be achieved. The action plan typ-
ically takes the form of tasks that students will
perform to achieve their end goal. Students re-
visit their goals at the end of the chapter or unit,
select evidence that supports mastery of the goal,
and record responses to the following reflective
questions: “Did you meet your goals?” “How do
you know?” “How could you have better met your
goals?” “Are you satisfied with your performance
in this chapter? Why or why not?” “Based on the
evidence that you chose, what can you do now that
you could not do at the beginning of this chap-
ter?” Asking students to revisit goals they set at
the beginning of the chapter encourages them
to make SMART (specific, measurable, attain-
able, realistic, time-bound) goals SMARTER, by
adding evaluation and reflection segments aimed
at examining the quality of the work completed.
This personal reflection on the learning supplies
the important element of feedback as defined by
Locke and Latham (1990) as “knowledge of one’s
performance” (p. 173) or as “knowledge about
performance” (West & Thorn, 2001, p. 42). In this
case, students provide their own feedback (self-
assessment) as they monitor their own progress
(Marzano et al., 2001). Feedback “tells people
what is” and “goals tell them what is desirable”
(Locke & Latham, 1990, p. 197). Feedback pro-
vides information on progress made toward goal
accomplishment, and goals facilitate the ability
to evaluate this progress using a goal standard
(Locke & Latham, 1990). See the LinguaFolio im-
plementation cycle designed to guide classroom
application of these principles (Appendix A).
Acollaborationofstatesadaptedthe
NCSSFL LinguaFolio to interface with the
Standards for Foreign Language Learning,
the American Council on the Teaching of
Foreign Languages (ACTFL) Performance and
Proficiency Guidelines, and their individual
state foreign language frameworks. State-specific
versions of LinguaFolio were developed and
piloted in North Carolina, South Carolina,
Indiana, Kentucky, Virginia, and Nebraska.
Classroom-Based Research
A quantitative research study was conducted at
the classroom level to determine the educational
value related to the goal-setting process as facili-
tated through the integration of LinguaFolio in
the language classroom. The unique challenges
inherent in working in K–12 educational settings
for any length of time can discourage researchers
from studying young learners, or minors, as such
studies require very strict procedures to secure
institutional approval that include parental con-
sent. This examination of goal setting and stu-
dent achievement required institutional review
board permissions, district and school approval,
parental consent, student assent, and teacher con-
sent on an annual basis. Working in K–12 schools,
researchers were faced with concerns of ensuring
rigor in a natural and difficult-to-control environ-
ment laden with attrition due to teachers leaving
or transferring schools, replacement teachers not
invested in study participation, and student attri-
tion as they transitioned to more advanced (and
nonmandatory) levels of study. By choosing hier-
archical linear modeling—a statistical tool that al-
lowed researchers to use all of the data collected in
spite of attritional challenges—researchers were
able to overcome these significant obstacles.
PURPOSE AND RESEARCH QUESTIONS
The purpose of this study is to analyze the rela-
tionship between goal-setting ability and second-
language performance for high school students in
the Spanish language classroom. Students’ goal-
setting processes and language proficiency per-
formance were analyzed for 4 consecutive years
(with an additional year for a pilot that informed
the study). Researchers investigated trends in the
goal-setting process and the relationship between
goal setting and language production in reading,
writing, and speaking.
1. What is the relationship between goal setting
and performance for students of Spanish?
2. What are the general trends in goal setting
for Spanish students in levels 1, 2, 3, and 4?
METHODOLOGY
Participants
A purposive sampling of teachers was recruited
for the study that would allow the researchers to
158 The Modern Language Journal 96 (2012)
TABLE 1
Student Sample
Student Participants 2005 2006 2007 2008 2009
Tota l 106 3 9 4 527 48 4 3 3 2
First-Year Students 80 270 186 48 173
Second-Year Students 13 123 307 261 72
Third-Year Students 13 1 34 158 37
Fourth-Year Students NA NA NA 16 51
follow the same students and teachers for several
years in an attempt to reduce the external vari-
ables in the study. Researchers identified and re-
cruited teacher participants through a statewide
“Improving Teacher Quality”—a funded initia-
tive in which Spanish educators participated in a
10-day intensive immersion institute designed to
enhance language skills, pedagogical practices,
and technological integration into their class-
rooms. The participants also received extensive
training in goal setting and reflection, and they
were introduced to LinguaFolio, a classroom-
based, structured intervention designed to pro-
mote self-regulation among learners. Between
2005 and 2009, researchers recruited 21 teacher
participants and their 1,273 individual students.
These 21 teachers represented 23 Nebraska
schools, including 19 public and 4 parochial
schools of varying sizes.
In each year of the study, the sample grew in di-
versity and size. New teachers and students were
recruited in an effort to increase the sample size
while current participants continued with their
language learning experience. New recruits were
from all levels of language learning and not solely
from the first year of language study. This was
done in an effort to provide the greatest level
of breadth and depth of sample diversity. Addi-
tionally, a number of participants were lost due
to teacher, school, or student attrition. As a re-
sult of this longitudinal tiered recruitment and
attrition, the final data set consisted of data stem-
ming from students at all levels of Spanish, some
of whom were followed for multiple consecutive
years and some of whom participated for only a
snapshot of their Spanish learning experience. All
data were valuable for the purposes of this study
as explained in the section addressing hierarchi-
cal linear modeling (HLM) methodology. Table 1
outlines the sample composition for the duration
of the study.
Instruments and Procedures
All participating teachers and their students
engaged in the LinguaFolio goal-setting process
as an established component of their Spanish-
learning classroom curriculum. The LinguaFolio
goal-setting process required students to establish
personalized goals and action plans in accordance
with chapter objectives, to collect classroom-based
evidence throughout a chapter or unit, and to re-
flect on relative goal attainment upon completion
of a chapter or unit. This process was repeated
with each subsequent chapter (see Appendix A
for visual representation of the LinguaFolio cycle
of implementation).
The independent variable for this study (Lin-
guaFolio Goal-Setting Process) consists of three
components: setting goals, establishing an action
plan for goal attainment, and reflecting on rela-
tive attainment of goals. For each year of the study,
researchers collected and analyzed qualitative
data in the form of student-produced goals, action
plans, and reflections. In 2005, with the collection
of the first year of goal-setting data, researchers
identified a need for a rubric to score student-
produced goals and reflections. Researchers en-
tered 200 student-produced goals into a qualita-
tive database and analyzed those goals to deter-
mine the natural and authentic separation of ac-
tual student data. Working independently, 5 re-
searchers explained the similarities and differ-
ences by establishing common themes. The team
then met to compare and contrast its findings
and arrived at a final, comprehensive rubric de-
rived from actual student-produced data. The
rubric was peer-reviewed by 3 teacher partici-
pants, and their feedback was applied to finalize
the LinguaFolio Goal-Setting Process Rubric (see
Appendix B).
With each annual collection of goals (with a
mean of 1,000 goals per year), interrater reliabil-
ity was established at the 90% level through the
following process.
A random sample of 10 sets of LinguaFolio
goal-setting data (goal, action plan, reflection)
was reviewed by each of the 3 researchers in-
volved in the goal-rating process. They utilized
the LinguaFolio Goal-Setting Process Rubric to
independently rate each of the 10 student sample
sets. Each of the 3 researchers independently pro-
duced 30 scores (10 goal scores, 10 action plan
scores, 10 reflection scores), resulting in a total of
90 scores (30 scores ×3 researchers). After rat-
ing the 10 student goals independently, all scores
were compared during a meeting that allowed for
a discussion of similarities and differences. An-
other random sample consisting of 10 LinguaFo-
lio goal-setting data was reviewed, each rater re-
peating the previous process. This continued un-
til researchers arrived at 90% agreement among
Aleidine J. Moeller, Janine M. Theiler, and Chaorong Wu 159
their 90 scores. With the establishment of 90%
agreement, a final sample rating was conducted,
eliminating chance occurrence. The 3 researchers
then divided all student samples, each taking a
portion of the data set to rate. These scores were
utilized as raw data in analyzing the relationship
between student goals and achievement.
Dependent variables for this study include
teacher-independent scores produced annually
through an online proficiency assessment—the
Standardized Measure of Proficiency (STAMP) as-
sessment. STAMP, a statistically validated, realia-
based, textbook-independent, and computer
adaptive assessment, produces a comprehensive
score for proficiency in reading, writing, and
speaking (Avant Assessment, 2010).
For each year of the study, teachers guided stu-
dent participants through the goal-setting pro-
cess, and students stored all goals and reflec-
tions in folders or binders. At the culmination
of a Spanish course, students participated in the
STAMP assessment. At the conclusion of each aca-
demic year, researchers collected all LinguaFo-
lio student-produced goals and they analyzed the
goals using the LinguaFolio Goal-Setting Process
Rubric.
Hierarchical Linear Modeling
Hierarchical structuring is commonly found in
organizational settings such as educational sys-
tems (i.e., assignments within students within
classes within teachers within schools within dis-
tricts within regions, and so on). In longitudi-
nal studies conducted within educational systems,
repeated measurements are made on the same
experimental unit, or subject, over time. In the
case of this study, researchers desired to make
repeated measures representing Spanish learner
growth while accounting for the nested learning
structure. HLM was adopted, as HLM is able to
capture these “measurement occasions” within a
nested structure. In this study, these measurement
occasions (lower level or level 1) are nested within
students (higher level or level 2). These students
(level 2) are then nested within teachers (high-
est level, or level 3). Proc Glimmix in Statisti-
cal Analysis Software (SAS) facilitated the HLM
analyses.1
Researchers initially planned on utilizing the
HLM to analyze reading, writing, and speak-
ing scores in light of all components of the
goal-setting process variable (goal setting, action
plan, reflection). However, the high correlations
(Table 3) among these variables implied that
collinearity might occur if they were all included
TABLE 2
Goal Process Scores by Level of Spanish
Spanish Level Goal Action Plan Reflection
All Levels µ2.48 2.41 2.28
SD 0.828 0.929 0.794
n877 847 876
First Year µ2.39 2.19 2.18
SD 0.729 0.911 0.784
n346 320 347
Second Year µ2.42 2.39 2.28
SD 0.818 0.871 0.793
n376 372 374
Third Year µ2.98 2.97 2.49
SD 0.956 0.973 0.824
n131 131 131
Fourth Year µ2.10 2.67 2.58
SD 0.571 0.381 0.434
n24 24 24
as predictors. When examining the correlations
between the goal-setting process components and
the STAMP results, the goal-setting and action
plan variables presented stronger correlations
with STAMP proficiency scores. Because of these
stronger correlations and because this study in-
volved an analysis of the link between goal setting
and student achievement, researchers opted to
concentrate on goal and action plan writing as
predictors for the HLM analysis. Although this
eliminated the student reflection detail in the
goal-setting process and student achievement re-
lationship, it increased accuracy and power by de-
creasing the error risked with collinearity among
three factors. For each dependent variable, re-
searchers established the best-fit model through
aseriesofunconditionalandconditionalhierar-
chical linear models. For all of the models, re-
stricted maximum likelihood (REML) was used
with Satterthwaite approximation for degrees of
freedom.
RESULTS
Descriptive analyses were run for goal writing,
action plan writing, and reflection scores accord-
ing to levels of Spanish. The results of the descrip-
tive analysis are depicted in Table 2.
The descriptive analysis revealed a consistent
increase in goal setting, action plan, and reflec-
tion mean from level 1 through level 3 of Span-
ish. The analysis revealed a drop in goal-setting
and action plan means between years 3 and 4, but
the mean score for reflection continued to rise
160 The Modern Language Journal 96 (2012)
TABLE 3
Correlations for LinguaFolio and STAMP
STAMP STAMP STAMP LF LF LF
Writing Speaking Reading Goal Plan Reflection
LF Goal Pearson Correlation .376∗∗ .341∗∗ .263∗∗ 1.623
∗∗ .623∗∗
n836 801 845 877 845 875
LF Plan Pearson Correlation .419∗∗ .383∗∗ .211∗∗ .623∗∗ 1.468
∗∗
n807 777 817 845 847 845
LF Reflection Pearson Correlation .249 .221∗∗ .237 .623∗∗ .468∗∗ 1
n835 800 845 875 845 876
Note. LF =LinguaFolio; STAMP =Standardized Measure of Proficiency.
∗∗p<.01 (2-tailed).
between years 3 and 4. The decrease in mean
scores from the third year to the fourth year for
both goal setting (µ=2.98, 2.10) and action
plan (µ=2.97, 2.67), coupled with a decrease in
standard deviation for goal setting (SD =0.956,
0.973), action plan (SD =0.973, 0.381), and re-
flection (SD =0.825, 0.434), calls for a closer
analysis of the data. Results may not accurately
depict growth because conducting aggregate de-
scriptive analyses of means considers the scores
even of those students who may be writing goals
for the first time as third- or fourth-year Spanish
students. When considering the data represented
in Table 2, the third- and fourth-year declines may
result from analyzing the data in aggregate form
without considering sample attrition or retention.
With disaggregation of the data to represent solely
students participating in all 4 years of consecu-
tive levels of Spanish instruction, the sample size
decreases significantly (n=24). Conducting an
analysis with such a small sample risks produc-
ing questionable results. As such, the descriptive
analysis primarily served as a source of guidance
as researchers continued with their data analysis.
A correlational analysis of the goal-setting pro-
cess and STAMP assessment variables reveals a
statistically significant relationship between each
component of the goal-setting process and each
component of the STAMP assessment (p<.01).
The results of the correlational analysis are de-
picted in Table 3. Restricting the investigation
of relationships to a correlational analysis ne-
glects potential differences resulting from stu-
dent, teacher, and other predictors. HLM was
therefore adopted to further investigate the rela-
tionship between goal setting and student achieve-
ment. HLM allows for a deeper understanding of
the relationship between goal setting and achieve-
ment, with the emphasis no longer on snapshot
relationships but rather growth relationships that
account for variation due to student, teacher, and
other predictors.
Empty and Unconditional Models
To build a model in HLM, researchers be-
gan with a basic, or empty, model. The empty
model aimed to reveal the source of variance in
the absence of specific predictors. In this case,
goal-setting process predictors were absent in the
empty model. A three-level empty model (ran-
dom intercept only) was fitted for each depen-
dent variable (STAMP reading, writing, speaking
scores). The three levels in this model represent
measurement occasion (level 1), student (level
2), and teacher (level 3). The random effects at
each level and the interclass correlations (ICCs)
are depicted in Table 4.
As is evident from Table 4, some variance in
STAMP outcomes can be accounted for by stu-
dent and teacher differences. For example, 35.4%
of the variance for STAMP writing can be ac-
counted for by student difference (this includes
the teacher difference by definition) and 22.7%
can be attributed to teacher difference. Sixty-four
percent of the 35.4% student difference can be
attributed to teacher differences.
Level of Spanish, centered at the first year of
Spanish, was then included in the empty model.
Due to the limited levels of Spanish (four), re-
searchers chose to not pursue more complex
models such as quadratic or cubic models, opting
to remain with the linear model. It was found that
including the random slope at the student level
did not improve the model fit for STAMP writ-
ing and STAMP speaking (deviance difference =
2.78; !df =2, p>.05; deviance difference =
Aleidine J. Moeller, Janine M. Theiler, and Chaorong Wu 161
TABLE 4
Random Effects for Empty Models and Interclass Correlations
STAMP Writing STAMP Speaking STAMP Reading
Random Intercept
Variance at Level 2
.125 .105 .160
Random Intercept
Variance at Level 3
.223 .151 .108
Residual Variance .634 .592 .359
ICC Level 1 within Level 2
and Level 3
.354 .302 .427
Level 1 within Level 3 .227 .178 .172
Level 2 within Level 3 .64 .590 .403
Note. ICC =interclass correlation; STAMP =Standardized Measure of Proficiency.
27.7; !df =2; p>.05, respectively) but did im-
prove for STAMP reading (deviance difference =
77; !df =2, p<.05). Thus, the baseline model
for STAMP writing and STAMP speaking did not
include random slope, but the baseline model for
STAMP reading did include random slope at the
student level.
Conditional Models
Goal-setting and action plan variables were then
included in the baseline model. These two vari-
ables were at the measurement occasion level
(level 1), and three new predictors were estab-
lished for both goal-setting and action plan vari-
ables. The teacher-level predictor (level 3) was
represented by the mean scores of all students
for an individual teacher. For goal setting, this
was centered at 2, for action plan writing it was
centered at 2.4. The student-level predictor (level
2) was represented by the difference between the
mean scores for an individual student and the
mean score attributed to all students of the cor-
responding individual teacher. The difference of
scores associated with each Spanish level and the
means of an individual student served as the mea-
surement occasion level predictor (level 1). Thus,
six covariates were created and were included into
the baseline model. The results are shown in Table
5. The regression equations for the final model of
STAMP writing, speaking, and reading are pro-
vided in Appendix C.
Hierarchical linear modeling results are inter-
preted in the same manner for STAMP read-
ing, writing, and speaking (Table 5). Results for
STAMP writing and the goal-setting process will
be explained as a representative model of HLM
interpretation. The 1.54 intercept implies that
the STAMP writing score is 1.54 for a student in
his or her first year of Spanish study, with the
mean of the combined scores for all students
of a teacher (the “teacher mean score”) of 2
for goal setting and 2.4 for action plan writing,
with the individual student mean goal-setting and
action plan scores equal to the teacher’s mean
scores, and with the student goal-setting and ac-
tion plan scores considered to be average. The
level of Spanish of 0.683 implies that comple-
tion of additional years of Spanish relates to a
0.683 increase in STAMP writing score for each
year.
The teacher-level goal setting of 0.17 implies
that the STAMP writing score of a student in-
creases by 0.17 if the average goal-setting score
of the teacher is 1 unit higher (average score
of a teacher =average of all student scores of
that teacher). The student-level goal setting of
0.283 implies that a student’s STAMP writing score
increases by 0.283 if the student’s goal-setting
score increases by 1 unit. The measurement oc-
casion level goal setting of 0.005 implies that the
STAMP writing score increases by 0.005 if a stu-
dent’s goal-setting score is 1 unit higher than the
average.
The teacher-level action plan writing of 0.369
implies that the STAMP writing score of a stu-
dent increases by 0.369 if the average action plan
writing score of the teacher is 1 unit higher. The
student-level action plan writing of 0.162 implies
that a student’s STAMP writing score increases by
0.162 if the student’s action plan writing score
increases by 1 unit. The measurement occasion
level action plan writing of 0.214 implies that the
STAMP writing score increases by 0.214 if the ac-
tion plan writing score is 1 unit higher than aver-
age. This interpretation of the HLM results would
162 The Modern Language Journal 96 (2012)
TABLE 5
Parameter Estimates and Model Fit Statistics for Final Conditional Models
STAMP Writing STAMP Speaking STAMP Reading
Parameters Estimate SE p-Value Estima te SE p- Val ue Estimate SE p-Value
Fixed Effects
Intercept 1.54 0.153 <.0001 1.32 0.123 <.0001 1.30.074<.0001
Level of Spanish 0.683 0.035 <.0001 0.605 0.034 <.0001 0.373 0.033 <.0001
BWT Goal 0.17 0.283 .557 0.42 0.223 .078 0.256 0.127 .061
BWS Goal 0.283 0.052 <.0001 0.19 0.052 .0003 0.244 0.047 <.0001
WS Goal 0.005 0.085 .952 –0.003 0.008 .973 0.106 0.08 .188
BWT Plan 0.369 0.273 .198 0.151 0.214 .493 0.03 0.117 .799
BWS Plan 0.162 0.046 .0004 0.159 0.045 .0005 –0.043 0.041 .29
WS Plan 0.214 0.07 .0027 0.111 0.067 .102 –0.07 0.066 .291
Random Effects
Residual Variance 0.335 0.033 0.311 0.031 0.313 0.011
Intercept Variance
BWT
0.142 0.061 0.08 0.033 0.015 0.033
Intercept Variance
BWS
0.154 0.035 0.149 0.033 0.033 0.039
Intercept-Linear
Covariance BWS
0.062 0.015
Linear Variance BWS 0∗—a
Model Fit
AIC 1750.68 1626.88 1656.91
BIC 1753.18 1635.38 1660.24
No. of Parameters 11 11 11
Note. AIC =Akaike Information Criterion; BIC =Bayesian Information Criterion; BWS =between
student/student level (level 2); BWT =between teacher/teacher level (level 3); WS =within stu-
dent/measurement occasion level (level 1).
aNot estimable.
∗2.52−19.
apply in an equivalent manner, with the appropri-
ate respective numbers, for both STAMP speaking
and STAMP reading scores.
DISCUSSION AND CONCLUSION
Analysis of the data reveals a consistent increase
over time in the mean goal, action plan, and re-
flection scores of high school Spanish learners.
This trend held true for all levels except for the
progression from third- to fourth-year Spanish for
action plan writing and goal setting. With the dis-
aggregation of the data at the third and fourth
year, this consistency continued; however, a lim-
ited sample size is a limitation at these levels. The
greatest improvement in goal setting occurred be-
tween the second and third levels of Spanish. This
can be explained in part by the attrition of those
students who discontinued their Spanish studies,
as this sufficed to meet the language require-
ment for entry into college. This increase in score
could also be attributed to a myriad of reasons
that may include smaller class size, more oppor-
tunities to use the second language, and higher
motivation among those students who chose to
continue language studies beyond the minimum
language requirement. Given the indicated rela-
tionship between goal setting and achievement,
future research to further clarify development in
both areas, as well as the interrelationship, should
be considered. For example, a similar study with a
control group would provide increased clarity and
strengthen our understanding of the relationship
between goal setting and achievement. Although
causation certainly cannot be claimed with the sta-
tistical analyses conducted in this study, the con-
sistent growth in goal, action plan, and reflection
scores may serve as a rationale for future consid-
eration of the factors involved in increasing skill
proficiency (practice, educational level, maturity,
etc.).
One might hypothesize that these lesser mean
scores (goal-setting process) for the third and
fourth levels are due to a training effect on the
Aleidine J. Moeller, Janine M. Theiler, and Chaorong Wu 163
part of teachers (Sch¨
arer, 2004). Knowledge of
the goal-setting process and its implementation
has evolved during this study, and trainings for
teacher participants were adjusted accordingly.
Students who accumulated 4 consecutive years of
participation began the study with teachers who
received their training when the least was known
about the goal-setting process. Their students, in
turn, likely received a less detailed and structured
plan of guidance when compared to those who
began participating in the third or fourth year of
the study. Each iteration of the study brought new
understanding about goal setting, which was im-
mediately implemented in the training sessions
with participating teachers. An additional factor
contributing to this may be the sense of commu-
nity that was established among the participants
via Blackboard, face-to-face meetings, and addi-
tional training at regional and state conferences
and seminars. This provided a forum for teach-
ers to share insights, experiences, and successes
(Sch¨
arer, 2004).
Correlational analyses revealed a statistically sig-
nificant positive relationship between each com-
ponent of the goal-setting process and each com-
ponent of the STAMP assessment (p<.01). These
correlations reveal a positive relationship between
proficiency and the writing of goals, action plans,
and reflections—a learner more practiced and
skilled at goal setting relates positively to higher
language achievement in Spanish. Thus, a student
with a higher goal, action plan, or reflection score
will likely also be a student who achieves a higher
STAMP proficiency score in reading, writing, or
speaking. Conversely, a student who has lower goal
setting, action plan, or reflection scores will likely
be a student who achieves a lower STAMP profi-
ciency score. Although there is a strong positive
relationship between goal setting and language
achievement, causality can only be established
through experimental research using a control
group.
Although the correlation results might be con-
sidered a defense of “better goal writers equal
better users of language,” the HLM elucidates
the nature of the relationship of the growth that
occurs with both goal setting and proficiency.
The HLM analysis revealed a statistically signifi-
cant relationship at the student level when con-
sidering both goal writing and action plan writ-
ing in relation to STAMP proficiency (p<.001).
The relationship of this growth is independent
of level of language learning or achievement
level—growth in the ability to write goals or ac-
tion plans is related to growth in Spanish lan-
guage proficiency. This general finding elucidates
the need to focus on the goal-setting process for
the potential benefit of all learners, whether they
be high achievers, low achievers, beginning stu-
dents, or advanced students (at the secondary
level). Kohonen (2002) found that especially
lower secondary and elementary students are ac-
customed to relying on the teacher for planning
and guiding them through the learning process
and found little value in taking on the responsi-
bility themselves. Lacking the key metacognitive
skills that are stimulated by the ELP/LinguaFolio,
many students struggled to make the connec-
tion between reflection and self-assessment ex-
ercises and their language learning. This fact
emphasizes that the goal-setting process is espe-
cially important for these learners to achieve aca-
demic success and can serve as a powerful inter-
vention. LinguaFolio, as evidenced in this study,
can serve as an effective tool for promoting self-
regulation in learners through structured goal
setting.
The HLM teacher-level goal-setting score refers
to a combined mean goal score of all learners
for one particular teacher, and the HLM analy-
sis does not reveal a significant relationship for
teacher-level scores and student STAMP scores.
An increase in the teacher-level goal-setting score
does not relate to an increase in STAMP scores
of individual students (writing, p=.557; read-
ing, p=.06; speaking, p=.078). The action
plan scores at the teacher level do not reveal a
relationship with the STAMP scores of individual
students (reading, p=.799; writing, p=.198;
speaking, p=.493). This finding gains more
meaning when one recalls that the previously
mentioned HLM statistically indicated a relation-
ship between student growth in the goal-setting
process and proficiency (p<.001). The pres-
ence of student-level-related growth combined
with the lack of relationship between teacher
goal/action plan writing growth and student
STAMP growth emphasizes the student-specific
nature of the effects of goal writing and lan-
guage proficiency. In other words, according to
the HLM findings, growth in student proficiency
is related to growth in student goal writing inde-
pendent of the teacher. Factors that may influ-
ence these results include teacher-independent
variables that impact student achievement, such
as motivation, the goal-setting process itself,
and the meaningfulness of the curriculum. Re-
searchers have emphasized that simply setting
one’s own goals does not necessarily improve
achievement (Schunk, 2003), that a number of
important factors contribute to improved perfor-
mance, such as high-quality goals (D¨
ornyei, 2001;
164 The Modern Language Journal 96 (2012)
West & Thorn, 2001), setting one’s own goals
(Azevedo et al., 2002; Griffee & Templi, 1997),
teacher- and student-agreed-upon learner goals
(Boekaerts, 2002), and the learning environment
(Turner, 1995).
One might also attribute the lack of rela-
tionship between teacher-level scores and stu-
dent STAMP scores to the very prescribed goal-
setting procedures used uniformly by the par-
ticipating teachers (see Appendix A). Such a
prescribed set of procedures may have reduced
the variations between teacher means. Allowing
for variability in goal-setting procedures may re-
veal teacher effect for goal writing and student
achievement.
This study does not imply that teachers do not
influence the language acquisition process. When
turning to the empty model (Table 4), there is
indication of variability for student achievement
that is related to differences among teachers. The
empty model is the basic level of development
of the more complex HLM model, and as such, it
does not take into account multiple variables. The
focus in the empty model is growth in proficiency
independent of other variables. Because there
is a statistically indicated relationship between
teacher and student growth in proficiency in the
empty model, we can assume that the teacher con-
tribution to variability in student proficiency is re-
lated to factors independent of the goal-setting
process. This assumption stresses the fact that
the teacher relationship to student achievement
(STAMP scores) may involve other variables. Vari-
ables such as classroom climate, use of the second
language, grouping, learner-centered instruction,
teacher language proficiency, and teacher and
student personality have correlated with higher
achievement. Classroom observations, teacher in-
terviews, and lesson plan reviews would provide
valuable data regarding what is happening inside
those classrooms in which students consistently
outperform other classes of students.
This study provides insight into relationships
that exist between the goal-setting process and
student achievement in the Spanish language
classroom. Researchers found a significant rela-
tionship between a student’s ability to set goals
and language achievement in the Spanish lan-
guage classroom. A growth relationship was also
revealed, with growth in goal-setting ability sig-
nificantly relating to growth in proficiency. This
growth relationship proved to be significant at the
individual student level, independent of the class-
room teacher. Interestingly, whereas the teacher
did not account for variance in the growth re-
lationship, the teacher did account for general
variance in student proficiency according to the
HLM empty model. These combined results call
for future investigation into the nature of teacher
effect in the foreign language classroom, such
as the teacher’s role in the goal-setting process.
Qualitative studies are recommended that inves-
tigate both general classroom teacher effect as
well as the teacher effect component on stu-
dent goal-setting processes. How the teacher in-
troduces the goal-setting process, the degree of
peer and teacher feedback of the goals, the con-
sistent and regular review of goal setting during
the course of the semester, the degree of partici-
pation of the student in the identification of the
learning goals, the personalization of the learn-
ing goals, and the use of SMART goals to evaluate
the quality of student goals may play a significant
role in the degree of student achievement. Finally,
this study has introduced LinguaFolio as a poten-
tial intervention for the integration of the goal-
setting process into the language learning class-
room. Given the indicated relationship between
the goal-setting process and student achievement,
the need for such interventions is underscored.
Future investigations may further elucidate the
optimal manner in which students might navigate
this goal-setting process to increase motivation,
promote autonomous learning, and enhance aca-
demic achievement.
NOTE
HLM uses all available information within a
data set, and meaningful variance is not lost, as
would be the case with listwise elimination of cases
in techniques such as analysis of variance and re-
gression. HLM presents another data-related ad-
vantage within the context of this study when con-
sidering that it enables researchers to analyze rela-
tionships in growth between or among variables.
Due to the emphasis on growth, students may en-
ter at any level of learning, and their data will
be of value for this study. In this study, HLM re-
lates growth in goal-writing ability with growth in
proficiency, and the growth rate is considered to
be constant. Because of this underlying assump-
tion of constant growth rate, the comparison of
growth is unrelated to level of language learning.
Thus, students may enter the study at any level
and may be included in this longitudinal analysis
of growth relationships. For these reasons, HLM
is the strongest and most appropriate statistical
analysis procedure for this study, as researchers
desire to most accurately model the true growth-
related relationships between outcomes and pre-
dictors within the nested educational context of
Aleidine J. Moeller, Janine M. Theiler, and Chaorong Wu 165
Spanish learning experience within students
within teachers.
REFERENCES
Ames, C. (1992a). Achievement goals and classroom mo-
tivational climate. In J. Meece & D. Schunk (Eds.),
Students’ perceptions in the classroom (pp. 327–348).
Hillsdale, NJ: Erlbaum.
Ames, C. (1992b). Classrooms: Goals, structures and stu-
dent motivation. Journal of Educational Psychology.
84,261–271.
Ames, C., & Archer. J. (1988). Achievement goals in the
classroom: Students’ learning strategies and moti-
vation processes. Journal of Educational Psychology,
80,260–267.
Avant Assessment. (2010). About STAMP (Standards-
based Measurement of Proficiency) Web-based Test of
Foreign Language Assessment. Retrieved from
http://avantassessment.com/products/about_
stamp.html
Azevedo, R., Ragan, S., Cromley, J., & Pritchett, S.
(2002). Do different goal-setting conditions facilitate
students’ ability to regulate their learning of complex
science topics with RiverWeb? Retrieved from ERIC
database (ED482509).
Bandura, A. (1997). Self-efficacy: The exercise of con-
trol. New York: Freeman.
Bellanca, J., & Fogarty, R. (1991). Blueprints for thinking
in the cooperative classroom (2nd ed.). Palatine, IL:
IR/Skylight Publishing.
Benson, P. (2001). Teaching and researching autonomy in
language learning .London:Longman.
Bishop, J. (2003). Empowering students to succeed.
NCSC News.Retrievedfromhttp://www.ncsc.info
Boekaerts, M. (2002). Motivation to learn. Educational
Practice Series. Geneva: International Bureau of
Education.
Brophy, J. E. (1983). Fostering student learning and
motivation in the elementary school classroom. In
S. Paris, G. Olson, & H. Stevenson (Eds.), Learn-
ing and motivation in the classroom (pp. 283–305).
Hillsdale, NJ: Erlbaum.
Candy, P. (1988). On the attainment of subject-matter
autonomy. In D. Boud (Ed.), Developing student au-
tonomy in learning (2nd ed., pp. 59–76). London:
Kogan Page.
Council of Europe. (2010). The European Language Port-
folio. Retreived from www.coe.int/portfolio
Covington, M. V. (1984). The motive for self worth. In R.
Ames & C. Ames (Eds.), Research on motivation in
education: Student motivation (Vol. 1, pp. 77–113).
San Diego, CA: Academic Press.
Covington, M. V., & Berry, R. G. (1976). Self-worth
and school learning . New York: Holt, Rinehart, &
Winston.
Covington, M. V., & Omelich, C. L. (1984). Task-
oriented versus competitive learning structures:
Motivational and performance consequences.
Journal of Educational Psychology,76 ,1038–1050.
Cross, P. C., & Steadman, M. H. (1996). Classroom re-
search: Implementing the scholarship of teaching.San
Francisco: Jossey-Bass.
Dickinson, L. (1987). Self-instruction in language learn-
ing.Oxford:OxfordUniversityPress.
Doran, G. T. (1981). There’s a S.M.A.R.T. way to write
managements’ goals and objectives. Management
Review,70,35–36.
D¨
ornyei, Z. (2001). Motivational strategies in the lan-
guage classroom.Cambridge:CambridgeLanguage
Teaching Library.
Dweck, C. S. (1986). Motivational processes affecting
learning. American Psychologist,41,1040–1048.
Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive
approach to motivation and personality. Psycholog-
ical Review,95,256–273.
Edwins, S. D. (1995). Increasing reflective writing and
goal-setting skills on high ability sixth grade math-
ematics students. Retrieved from ERIC database
(ED392065).
Elliott, E. S., & Dweck, C. S. (1988). Goals: An approach
to motivation and achievement. Journal of Person-
ality and Social Psychology,54,5–12.
Good, T. L., Grouws, D. A., Mason, D. A., Slavings, R. L.,
&Cramer,K.(1990).Anobservationalstudyof
small-group mathematics instruction in elemen-
tary schools. American Educational Research Jour-
nal,27 ,755–782.
Griffee, D. T., & Templi, S. A. (1997). Goal setting affects
task performance. Retrieved from ERIC database
(ED413782).
Holec, H. (1981). Autonomy and foreign language learn-
ing. Oxford: Pergamon.
Jagacinski, C. M., & Nicholls, J. G. (1984). Conceptions
of ability and related affects in task involvement
and ego involvement. Journal of Educational Psy-
chology,76,909–919.
Jagacinski, C. M., & Nicholls, J. G. (1987). Competence
and affect in task involvement and ego involve-
ment: The impact of social comparison informa-
tion. Journal of Educational Psychology,79 ,107–
114.
Kohonen, V. (2002). The European language portfolio:
From portfolio assessment to portfolio-oriented
language learning. In V. Kohonen & P. Kaikko-
nen (Eds.), Quo vadis, foreign language education?
(pp. 77–95). Tampere, Finland: Department of
Teacher Education in Tampere University.
Little, D., & Perclov´
a, R. (2001). European Language
Portfolio: A guide for teachers and teacher trainers.
Retrieved from http://www.coe.int/T/DG4/
Portfolio/
Locke, E. A., & Latham, G. P. (1990). A theory of goal
setting and task performance. Englewood Cliffs, NJ:
Prentice Hall.
Locke, E. A., Shaw, K. N., Saari, L. M., &
Latham, G. P. (1981). Goal setting and task
performance: 1967–1980. Psychological Bulletin,
90,125–152.
166 The Modern Language Journal 96 (2012)
Marshall, H. H. (1988). In pursuit of learning-oriented
classrooms. Teaching and Teacher Education,4,85–
98.
Marzano, R. J., Pickering, D. J., & Pollock, J. E.
(2001). Classroom instruction that works: Research-
based strategies for increasing student achievement.
Alexandria, VA: Association for Supervision and
Curriculum Development.
Meece, J. L., Blumenfeld, P. C., & Hoyle, R. H. (1988).
Students’ goal orientations and cognitive engage-
ment in classroom activities. Journal of Educational
Psychology,80,514–523.
Mento, A. J., Steel, R. P., & Karren, R. J. (1987). A meta-
analytic study of the effects of goal setting on task
performance: 1966–1984. Organizational Behavior
and Human Decision Processes,39 ,52–83.
Miller, A. F., & Cunningham, J. A. (1981). How to avoid
costly job mismatches. Management Review,70,29–
31.
Moriarity, J., Pavelonis, K., Pellouchoud, D., & Wilson,
J. (2001). Increasing student motivation through the
use of instructional strategies. Retrieved from ERIC
database (ED455962).
NCSSFL (National Council of Supervisors for
Languages). (2010). LinguaFolio. Retrieved
from http://www.ncssfl.org/links/index.php?
linguafolio
Nicholls, J. G. (1979). Quality and equality in intellectual
development: The role of motivation in education.
American Psychologist,34,1071–1084.
Nicholls, J. G. (1984). Conceptions of ability and
achievement motivation. In R. Ames & C. Ames
(Eds.), Research on motivation in education (Vol. 1,
pp. 39–73). San Diego, CA: Academic Press.
Nicholls, J. G. (1989). The competitive ethos and demo-
cratic education. Cambridge, MA: Harvard Univer-
sity Press.
Nicholls, J. G., Cheung, P. C., Lauer, J., & Patashnick, M.
(1989). Individual differences in academic moti-
vation: Perceived ability, goals, beliefs, and values.
Learning and Individual Differences,1,63–84.
Nicholls, J. G., Patashnick, M., & Nolen, S. B. (1985).
Adolescents’ theories of education. Journal of Ed-
ucational Psychology,77 ,683–692.
Oxford, R., & Shearin, J. (1994). Language learning
motivation: Expanding the theoretical framework.
Modern Language Journal,78,12–28.
Paris, S. J., & Newman, R. S. (1990). Developmental
aspects of self-regulated learning. Educational Psy-
chologist,27 ,87–102.
Pennycook, A. (1997). English and capital: Some
thoughts. The Language Teacher,21,55–57.
Pintrich, P. R., & Schunk, D. H. (1996). Motivation in
education: Theory, research, and applications.Engle-
wood Cliffs, NJ: Prentice Hall.
Sch¨
arer, R. (2004). A European Language Portfolio from pi-
loting to implementation (2001–2004). Consolidated
report—final version.Strasbourg:LanguagePolicy
Division, Council of Europe.
Scharle, A., & Szab´
o, A. (2000). Learner autonomy: A
guide to developing learner responsibility. Cambridge:
Cambridge University Press.
Schunk, D. H. (2003). Self-efficacy for reading and writ-
ing: Influence of modeling, goal setting, and self-
evaluation. Reading and Writing Quarterly,19 ,159–
172.
Stipek, D. J., & Kowalski, P. S. (1989). Learned help-
lessness in task-orienting versus performance-
orienting testing conditions. Journal of Educational
Psychology,81,384–391.
Tubbs, M. (1986). Goal setting: A meta-analytic exami-
nation of the empirical evidence. Journal of Applied
Psychology,71,474–483.
Turner, J. (1995). Influence of classroom contexts on
children’s motivation for literacy. Reading Research
Quarterly,30,410–441.
Weiner, B. (1979). A theory of motivation for some class-
room experiences. Journal of Educational Psychol-
ogy,71,3–25.
Wentzel, K. R. (1987). Social, emotional, and cognitive fac-
tors related to adolescent classroom goals and academic
achievement in adolescence: A developmental systems
approach to the study of achievement motivation.(Un-
published doctoral dissertation). Stanford Univer-
sity, Stanford, CA.
Wentzel, K. R. (1991). Social competence at school: Re-
lationship of social responsibility and academic
achievement. Review of Educational Research,61,
1–24.
West, R. L., & Thorn, R. M. (2001). Goal setting, self-
efficacy, and memory performance in older and
younger adults. Experimental Aging Research,27 ,
41–65.
Yang, N. (1998). Exploring a new role for teachers: Pro-
moting learner autonomy. System,26,127–135.
Zimmerman, B. J. (2000). Attaining self-regulation: A
social-cognitive perspective. In M. Boekaers, P. R.
Pintrich, & M. Zeidner (Eds.), Handbook of self-
regulation: Theory, research, and applications (pp.
13–39). San Diego, CA: Academic Press.
Aleidine J. Moeller, Janine M. Theiler, and Chaorong Wu 167
APPENDIX A
LinguaFolio Cycle of Implementation
1. Review
Self-
Assessment of
Competency
Level Rubric
5. Review
goals. Choose
work to
represent goal
attainment.
6. Write
reflection
based on
evidence and
goals
2. Write
personal goals
based on
chapter
content.
3.Determine
tasks for
achieving
chapter goals.
4.Save goals,
tasks and all
completed
class work in
a folder
LinguaFolio Cycle of
Implementation
168 The Modern Language Journal 96 (2012)
APPENDIX B
LinguaFolio Goal-Setting Process Rubric
4/High 3/Mid-High 2/Mid-Low 1/Low
Goals Goals use authentic
language and are
tied to context.
Goals are growth
oriented, theme
based, measurable,
specific, realistic,
challenging,
personally relevant,
and time bound (“by
the end of this
chapter ...”).
Goals do not
necessarily use
authentic language.
Goals are somewhat
contextualized,
growth oriented,
and connected to a
theme. Goals are
measurable,
somewhat specific,
realistic, and
somewhat
challenging.
Goals do not use
authentic language
and/or are not
growth oriented, not
theme based, broad,
unfocused, vague or
too specific, too
challenging, or not
at all challenging.
Goals are not
authentic, there is
no focus on growth,
and they are too
broad, unrealistic,
and/or generic. The
student is unable to
articulate a goal.
Action Plan Breaks down goal
into a specific action
plan with manage-
able tasks. It is clear
how each goal will be
achieved.
Action plan present,
but not specific, or
additional steps
would be necessary
in order to make the
goal manageable.
Action plan present,
but specific steps for
success are not
articulated,
extremely vague
(i.e., “study,”
“listen”).
No action plan for
improving
achievement.
Evidence &
Reflection
Goals are reflected
upon and are
consistently revised
when deemed
inappropriate by the
student.
Goals are reflected
upon and are
sometimes revised
when deemed
inappropriate.
Goals are reflected
upon, but they are
not revised when
deemed
inappropriate.
Goals are not
reflected upon.
Each sample of work
in the dossier
includes a rationale
for why it was chosen
and how it relates to
the goals that were
set. The rationale is
very clearly stated,
labeled, and dated.
Most samples of
work in the dossier
include a rationale
for why they were
chosen and how they
relate to the goals
that were set. The
rationale is briefly
stated and may or
may not be labeled
and dated.
Few samples of work
in the dossier
include a rationale
for why they were
chosen and/or how
they relate to the
goals that were set.
The rationale, if
stated, is vague and
lacking labels and
dates.
No samples of work
in the dossier
include a rationale
for why they were
chosen and/or how
they relate to the
goals that were set.
Aleidine J. Moeller, Janine M. Theiler, and Chaorong Wu 169
APPENDIX C
Regression Equations for the Final Model of STAMP
STAMP Writing, Level 1 (within student):
(STAMPWriting)tij =ß0ij+ß1ij∗(LevelofSpanish)
+ß2ij∗(level 1goalsettingtij )
+ß3ij∗(level 1plantij )+etij
STAMP Writing, Level 2 (between student):
ß0ij =δ00j+δ01j ∗(level 2goalsettingij)
+δ02j∗(level 2planij )+U0ij
ß1ij=.683
ß2ij=.005
ß3ij=.214
STAMP Writing, Level 3 (between teacher):
δ00j=1.54+.17 ∗(level 3goalsettingj)
+.369 ∗(level 3planj)+V0j
δ01j=.283
δ02j=.162
Note:
1t: tth level of Spanish.
i: ith student.
j: jth teacher.
STAMP Speaking, Level 1 (within student):
(STAMPSpeaking)tij=ß0ij+ß1ij ∗(LevelofSpanish)
+ß2ij∗(level 1goalsettingtij )
+ß3ij∗(level 1plantij )+etij
STAMP Speaking, Level 2 (between student):
ß0ij=δ00j +δ01j∗(level 2goalsettingij )
+δ02j∗(level 2planij )+U0ij
ß1ij=.605
ß2ij=−.003
ß3ij=.111
STAMP Speaking, Level 3 (between teacher):
δ00j =1.32 +.42 ∗(level 3goalsettingj)
+.151 ∗(level 3planj)+Voj
δ00j =1.32 +.42 ∗(level 3goalsettingj)+.151
δ01j =.19
δ02j =.159
STAMP Reading, Level 1 (within student):
(STAMP Reading)tij =ß0ij +ß1ij ∗(LevelofSpanish)
+ß2ij ∗(level 1goalsettingtij)
+ß3ij ∗(level 1plantij) +etij
STAMP Reading, Level 2 (between student):
ß0ij =δ00j +δ01j∗(level 2goalsettingij )
+δ02j∗(level 2planij )+U0ij
ß1ij=δ10j +U1ij
ß2ij=.106
ß3ij=−.07
STAMP Reading, Level 3 (between teacher):
δ00j =1.3+.256 ∗(level 3goalsettingj)
+.03 ∗(level 3planj)+V0j
δ01j =.244
δ02j =−.043
δ10j =.244
