ArticlePDF Available

Assessment and academic writing: A look at the use of rubrics in the second language writing classroom

Authors:

Abstract

Abstract One of the most important aspects of the job of an English teacher is giving students the feedback and corrections they need to improve as second language learners. This is especially true for written English. In writing classes the process of providing feedback to students on their writing takes-up significant amounts of time and effort both inside and outside of the classroom. In order to streamline the feedback process teachers often make ...
Assessment and Academic Writing:
A Look at the Use of Rubrics
in the Second Language Writing Classroom
Gavin BROOKS*
Abstract
One of the most important aspects of the job of an English
teacher is giving students the feedback and corrections they need to
improve as second language learners. This is especially true for written
English. In writing classes the process of providing feedback to students
on their writing takes-up significant amounts of time and effort both inside
and outside of the classroom. In order to streamline the feedback process
teachers often make use of tools, such as rubrics, to help them provide
their students with feedback. Traditionally rubrics have been seen as tools
that have the potential of “increased consistency of scoring, the possibility
to facilitate valid judgment of complex competencies, and promotion of
learning.” (Jonsson & Svingby, 2007, p.130) However, recent studies in
the L1 writing classroom have shown that there are some significant
problems with the last of these items, using rubrics as a means of
promoting learning. This paper looks at some of the current research on
the use of rubrics in the classroom and attempts to construct a clearer
picture of both the benefits and drawbacks of the use of rubrics for both
grading and as a teaching tool in the L2 writing classroom. It is hoped
that in doing so this research will provide insight into the tools teachers in
Japan are using to respond to their students’ written work and act as a
starting point for further research into how to improve these tools.
─────────────────────────────────────────
*AssociateLectureofEnglish,SchoolofPolicyStudies,KwanseiGakuinUniversity
Kwansei Gakuin University
Humanities Review
Vol. 17, 2012
Nishinomiya, Japan
Article Contents
I. Introduction
II. Historical Overview of the Rubric
1. Rubrics in the L1 Classroom
2. Rubrics in the ESL/EFL Classroom
III. The Effectiveness of Rubrics in the Language Classroom
1. A case for using rubrics in the L2 writing classroom
2. A case against using rubrics in the L2 writing classroom
IV. Conclusion
1. Discussion
2. Final reflections
References
Appendix #1
I. Introduction
Written English may be one of the most important skills that students will learn
at university in terms of both their future academic and professional lives. While
many of the students studying English at universities in Japan will never be required
to use English outside of the classroom on a day to day basis they are often asked
to write compositions as part of the language tests that act as gatekeepers to their
future jobs or further studies. The feedback that students receive from their teachers
plays an important role in the students’ development as English language writers.
Because of this, one of the most important questions that we need to ask ourselves
as teachers of written English is: “What is the best way to respond to and guide
students’ writing in a way that enables them to improve in both their current and
future written assignments?” This question is essential, as a large part of the
language teacher’s job in the writing classroom is taken up with grading and
providing feedback to students. In fact, the job of evaluating students’ writing and
giving them feedback is so central to the English language writing class many
teachers do not even question how or why this process is necessary or how it should
be done: “Many of the decisions that both L1 and L2 writing teachers make in their
classes revolve around assessment of students’ writing . . . (and) because a culture
of assessment is built into the schooling enterprise, teachers rarely ask whether they
need to assess their students.” (Casanave, 2007, p.113) However, a closer
examination of how and why they are providing feedback to their students is an
essential part of professional development and a vital experience for teachers
interested in improving themselves as educators.
Gavin BROOKS
II. Historical Overview of Rubrics
1. Rubrics in First Language Education
Rubrics were first introduced into the L1 writing classroom as a means of
assessing student writing. In the traditional view of how writing should be assessed
there was an assumption that it was possible to come up with some type of
objective score that could be assigned to a students’ composition and that the
validity of this “‘true’ measure of student ability . . .can only be established through
technical and statistical rigor” (Huot, 1996, p.550). (See Figure #1) Because of this
there was a push for researchers and teachers to come up with a set of tools that
allow the reader to assign a valid score to the student’s writing. One of the most
commonly used of these tools is the rubric. A rubric is defined as “a scoring tool
for qualitative rating of authentic or complex student work. It includes criteria for
rating important dimensions of performance, as well as standards of attainment for
those criteria” (Jonsson & Svingby, 2007, p.131). Rubrics have long been a part of
the writing classroom in first language classrooms around the world.
Rubrics were first proposed as a tool to analyze writing in 1912 when Noyes
suggested the use of a rubric as a means of standardizing the evaluation of student
compositions: “Our present methods of measuring compositions are controlled too
much by personal opinion, which varies with the individual. What is wanted is a
clear-cut, concrete standard of measurement which will mean the same thing to all
people in all places and is not dependent upon the opinion of any individual”
(Noyes, 1912 as cited in Turley & Gallagher, 2008, p.88). Of these scales the most
famous is the Hillegas scale, which was developed in 1912 and “gave English
Traditional Writing Assessment
Procedures, Purposes and Assumptions
Procedure Purpose Assumptions
Scoring Guideline Recognize features of writing quality Writing quality can be defined and
determined
Rater Training Forster agreement on independent rater
scores
One set of features of student writing for
which raters should agree
Scores On Papers Fix degree of writing quality for
comparing writing ability and making
decisions on that ability
Student ability to write can be coded and
communicated numerically
Interrater Reliability Calculate the degree of agreement
between independent raters
Consistency and standardization to be
maintained across time and location
Validity Determine the assessment measures what
it purports to measure
An assessment’s value is limited to distinct
goals and properties in the instrument itself
Figure #1: The traditional view of writing assessment (Huot, 1966, p.551)
Assessment and Academic Writing
teachers the first reliable means of estimating objectively the quality of their pupils’
written production” (Hudelson, 1923, p.164).In1915Thorndikeimprovedupon
Hillegas rubric for grading student compositions by “substituting new specimens for
certain of the original samples and by including several examples in the steps at or
near the middle of the scale” (Hudelson, 1923, p.164).
One of the key benefits of these, and other rubrics, is that they are an attempt
to provide some type of inter-rater reliability. This is done as an attempt to get
around “one of the most vexing dilemmas in writing assessment . . . the
inconsistency with which different readers tend to evaluate the same piece of
writing” (Casanave, 2007, p.124). However, it is important to note that these rubrics
“were never designed to (improve student writing) directly, and any who attempt to
employ them for a such a purpose are certain to be disappointed” (Hudelson, 1923,
p.163). In fact the early rubrics developed by Noyes and his contemporaries were
designed not for the student, but for the administrator as a means “to provide a
standardized form of measurement that wouldallowadministratorsandinvestigators
to ‘measure and express the efficiency of a school system’1) so that comparisons and
rankings could be made between schools across the nation (Turley & Gallagher,
2008, p.88).
However, in the early 70’s, as the “process approach” method for teaching
composition became popular in classrooms around the United States, rubrics had to
evolve from an assessment tools into something that could be used to provide
students with feedback on how well their essay met a certain set of criteria and
some insight into what they can do to improve themselves as writers. (Ferris, 2009)
In the field of first language composition whether or not the rubric is an effective
tool in providing students with the feedback that they need to improve as writers is
a topic of debate in a variety of academic journals. Researchers have come out both
in support of (H. G. Andrade, 2000; H. L. Andrade, Wang, Du, & Akawi, 2009) or
against (Broad, 2000; Kohn, 2006; Wilson, 2007) the use of rubrics as a means of
providing students feedback about their written work. One only needs to look at the
sub-title of the 2008 paper by Turley and Gallagher, “Reframing the Great Rubric
Debate”, to see that there are strong feelings on both sides of this issue. One of the
reasons for this debate is that how writing is being taught in L1 classrooms has
changed significantly over the years and teachers are not being asked to simply
evaluate their students’ writing but to engage in a dialogue with their students in an
attempt to help them improve as writers:
─────────────────────────────────────────
1) Turley and Gallagher take this quotation from Hillegas, Milo B., (1913), A Scale for the
Measurement of Quality in English Composition by Young People. Teachers College, 13(4), 28
50.
Gavin BROOKS
a relatively recent shift in writing pedagogy has not translated into a shift in
writing assessment. Teachers are given much more sophisticated and
progressive guidance nowadays about how to teach writing but are still told to
pigeonhole the results, to quantify what can’t really be quantified. Thus, the
dilemma: Either our instruction and our assessment remain “out of synch” or
the instruction gets worse in order that students’ writing can be easily judged
with the help of rubrics. (Wilson as cited in Kohn, 2006, p.14)
2. Rubrics in the ESL/EFL Classroom
Many techniques have entered the second language writing classroom by way
of the first language composition classes because, as “Silva, Leki and Carson (1997:
399) point out, ‘second language writing is situated at the intersection of second
language studies and composition studies’ . . .(and) work that has focused
exclusively on Ll writing assessment contributes greatly to our understanding of
both the process and the product of L2 writing assessment” (Kroll, 1998, p.222).
The rubric is one of those things that has been borrowed by L2 teachers from their
colleagues teaching in the L1 writing classroom. Similar to how they were used in
first language classrooms, rubrics began in second language writing programs as a
means of providing teachers with a standardized way to evaluate their students’
writing. They were also used as a tool to facilitate the placement of students at the
appropriate level. In fact, today “most, if not all writing programs have entry and
exit criteria or grading rubrics to guide teachers at various levels of the program.”
(Ferris, 2009, p.121)
In the field of second language writing the ECP, or ESL Composition Profile
(see Appendix #1), is probably one of themostrecognizablerubricsand“(i)t,orits
offspring, will be familiar from workshop handouts or Xeroxes left behind in faculty
coffee rooms” (Haswell, 2005, p.107). This rubric was developed in 1981 using
research taken from the compositions of first language students. Three researchers
from Educational Testing Services (ETS) took research done in 1953 on the grades
and comments on the written assignments of first-year students studying at
Middlebury College, Cornell and the University of Pennsylvania to come up with a
rubric that was composed of five main traits. Each of these traits was then broken
up into a number of sub-traits that the researchers believed could then be used to
objectively grade English compositions written by second language speakers.
(Haswell, 2005) One benefit of the ESL Composition Profile (Jacob et al., 1981) is
that it has been established to have a high degree of both internal and external
validity, with scores given on the rubric being shown to be both consistent between
raters and as being “highly correlated with (student’s scores on) the TOEFL and
Michigan Test Battery” (Bacha, 2001, p.374) It is no accident that this rubric was
Assessment and Academic Writing
designed by researchers working for a testing organization as the ECP provides
these testing services with an invaluable tool that allows them to grade a large
number of student essays using multiple raters while still maintaining a high level of
inter-rater reliability. Because of this, other ESL/EFL testing companies are now
also using rubrics as a means of grading the writing component of their tests. While
the traits may vary from test to test (see Figure 2) the underlying rationale and
principles remain the same.
In most writing classrooms the teacher has no need for this type of inter-rater
reliability and is more likely to be interested in the pedagogical value of the rubric
and the benefits of students accessing the rubrics to improve the quality of their
writing. However, the debate about the pedagogical effectiveness of rubrics that is
being played out in first language classrooms and research journals is only just
Main Traits of Scoring Rubrics for Six Tests of ESL Writing
Test Trait
Test in English for Education Purposes
(Associated Examining Board)
Content
Organization
Cohesion
Vocabulary
Grammar
Punctuation
Spelling
Certificate in Communicative Skill in English
(Royal Society of Arts/ University of Cambridge
Local Examinations Syndicate)
Accuracy [of mechanics]
Appropriacy
Range [of expression]
Complexity [organization and cohesion]
Test of Written English
(Educational Testing Service)
Length
Organization
Evidence
Style
Grammar
Sentences
Michigan English Language Battery Topic development
Sentences
Organization/ coherence
Vocabulary
Mechantcs
Canadian Test of English for Scholars and Trainees Content
Organization
Language use
International English Language Testing System Regtster
Rhetorical organization
Style
Content
Figure 2: Traits measured by various rubrics usedinstandardizedESL/EFLTests(Haswell,2007,p.8)
Gavin BROOKS
reaching the field of second language writing. While rubrics are mentioned in both
texts and journals devoted to the study of second language writing (Bitchener,
Young, & Cameron, 2005; Ferris, 1995; Hyland, 2010) they are usually mentioned
in passing as one of a number of possible assessment tools with very little time
given to the analysis of their effectiveness as tools for rating and improving student
writing. For example, a look at the issues of the Journal of Second Language
Writing over a 4 year period, from 2008 to 2011, reveals only 9 original research
articles that even mention rubrics and, in all of these articles, rubrics are used
unquestioningly as a tool for evaluating students’ written work. In fact, a further
search of this journal reveals only 2 articles from 1992 to 2011 that actually
question the effectiveness of rubrics (Paulus, 1999; Weigle, 2007). One of these,
Weigle, simply mentions the current controversy that exists in first language writing
about the use of rubrics before dismissing the issue without providing any sources
or evidence for her position: “while holistic scales are faster and more efficient,
analytic scales tend to be somewhat more reliable than holistic scales, and certainly
provide more useful feedback to students, as scores on different aspects of writing
can tell students where their respective strengths and weaknesses are.” (2007,
p.203).
This view is changing and second language researchers such as Haswell (1998,
2005) are beginning to ask if rubrics are the best tools for language teachers to use
as a means of improving their students’ ability to write. However, as with the use of
rubrics in first language composition, this is not the type of question that allows
researchers to come down either in favor or against the use of rubrics in the
classroom. In their article “On the Uses of Rubrics”, Turley and Gallagher (2008)
point out that, “instead of declaring all rubrics ‘good’ or ‘bad,’ we need to examine
what they do, why, and in whose interest” (p. 92). They propose a 4 point heuristic
to analyze the value of rubrics (or any pedagogical tool):
1. What is the tool for?
2. In what context is it used?
3. Who decides?
4. What ideological agenda drives those decisions? (Turley & Gallagher, 2008,
p.87)
It is these 4 questions that provide a starting point for the evaluation of the
effectiveness of rubrics in English language writing classes at Japanese universities.
Assessment and Academic Writing
III. The Effectiveness of Rubrics in the Language Classroom
1. A case for using rubrics in the L2 writing classroom
While they might have their downside, rubrics can be useful tools in the
language classroom. Along with setting relevant tasks, setting a clear topic and
prompts, helping students to choose the appropriate rhetorical modes and giving
students adequate time to complete the writing task; setting an appropriate scoring
criteria and the need to attain valid and reliable scores are essential elements of a
successful English writing program (Jacobs et al., 1981). Rubrics can help teachers
to achieve this goal as they set clear criteria for both the students and the teacher
when it comes to grading written work.
Furthermore, rubrics also make it possible to evaluate components within
written assignment, such as rhetorical structures, grammatical accuracy and the
ability to stay on topic. This is especially important for second language writers as
the level of the various skills essential to writing can vary significantly from student
to student. In the L2 writing classroom we are much more likely to see “varying
levels of proficiency/skill in different aspects of the product (and) products can vary
widely across genres” (Kroll, 1998, p.224). Because second language writers are
more likely to show varying levels of performance on the different traits “if we do
not score for these traits and report the scores, much information is lost” (Hamp-
Lyons, 1995, p.760). Rubrics make it easier for the teacher to record a score for
each of these traits, or sub-traits, so that students are able to receive feedback on,
and improve, the areas that require attention.
Another advantage of rubrics in the second language classroom is that they
help the teacher or evaluator focus on more than just the sentence level structures
found in the written assignment. In a 1993 study that involved six graders
evaluating six samples of student writing Sweedler-Brown found that when a
holistic scale was used to grade two sets of essays, one set that had the grammatical
and spelling errors already corrected by the researcher and one set that had not been
corrected, those essays that had poor mechanics consistently got lower scores,
regardless of the proficiency of the rhetorical structures used in the essay. However,
she found that graders who used a rubric to evaluate the essays were shown to focus
more on “the high quality of the essays’ organization and paragraph development
(and) were not distorted by the different qualities of the sentence-level features in
the original and corrected essays” (Sweedler-Brown, 1993, p.11). The ability to
focus on and encourage students to improve their discourse and rhetorical skills is
essential if we want our students to become better writers of English as a second
language.
Gavin BROOKS
2. A case against using rubrics in the classroom
While there are many positive things to be said for using rubrics in the
classroom the use of rubrics in the second language writing class is not without
problems. Recently a number of researchers have begun to raise “significant
concerns about the consequences of writing assessment and the ways in which
assessment practices sometimes seem to be antithetical to teaching practices” (Kroll,
1998, p.222). For example, Haswell (2005) notes that one of the big issues with
rubrics are that they do not solve the problems involved with holistically grading an
essay that they were designed to address. Rather, a rubric with five traits would
simply be asking “the rater to perform theholistic(rating)fivetimes”(Haswell,
2005, p.107). Also, while the rubric may succeed in grading how well the writer has
met the criteria set out by the rubric, it does not do a good job of taking
individuality into account and rubrics will often penalize the use of creativity,
humor, or clever writing. These are often things that second language writing
teachers are trying to encourage in their students. With second language learners the
problem of tailoring their writing to meet the criteria laid out in the rubric is often
compounded by the fact that second language students are often not aware of the
different genres that the rubrics may have been developed to evaluate, genres that
first language speakers are exposed to from an early age. As such, second language
writers will often answer an essay question in a different way than a native speaker
would, and subsequently receive a lower score on a rubric that is designed for, or
by, L1 writers. However, as “(t)here is no single written standard that can be said to
represent the ‘ideal’ written product in English...wecannot easily establish
procedures for evaluating ESL writing intermsofadherencetosomemodelof
native-speaker writing.” (Kroll, 1990, p.141)
Another problem, for both L1 and L2 speakers, is that they may not know how
to use the rubrics to improve their writing. Because rubrics are usually designed as a
way of rating students those students are often not provided with adequate training
on how to use the rubric to improve their writing skills. This problem is further
compounded in the field of second language writing as many of the rubrics being
used are based on rubrics that were designed for first language speakers and are
often incomprehensible to L2 learners as they may contain information about traits
or metaskills that the L2 writer is unable to understand.
IV. Conclusion
1. Discussion
So what can language teachers do? Should we be incorporating rubrics into our
writing classes? Well, as I have already stated, this is not a simple yes/no answer.
Assessment and Academic Writing
Rubrics can provide both teachers and students with a valuable tool for improving
students’ second language writing. However, there are some steps we should be
taking to make sure that the rubrics we use are providing our students with the
support they need to develop as English language writers. To begin with, it is
important that the rubrics we use in the classroom are developed for the type of
assignment we are asking the students to perform. Many teachers take a one-size fits
all approach to grading rubrics, often using a modified version of the same rubric to
grade a wide variety of assignments or making use of one of the standardized
writing rubrics that can be found in language teaching books. This approach can
often lead to confusion on the part of both the rater and the students as the rubric
may not be designed to evaluate the traits that the teacher is hoping to see in his or
her students’ compositions. Teachers need to take into account “the purpose of the
essay task, whether for diagnosis, development or promotion . . . in deciding which
scale is chosen. Revisiting the value of these scales is necessary for teachers to
continue to be aware of their relevance” (Bacha, 2001, p.371). In fact, the most
effective rubrics are those that are “developed on-site for a specific purpose with a
specific group of writers and with the involvement of the readers who will make
judgments in [that] context” (Hamp Lyons 1991 c:248).” (Kroll, 1998, p.228)
However, “the downside of this sort of procedure is that for a thorough analytic
judgment, each writing assignment would need to be scored on a specifically created
assessment instrument” (Kroll, 1998, p.228) the creation of which can be a time
consuming process.
Another issue with the use of rubrics in the second language classroom is their
accessibility to the students. If the students are unable to comprehend the categories
and or sub-categories contained in the rubric they will not be able to use it in any
meaningful way to improve their writing abilities. Even when students are able to
understand the rubric they may not understand how it relates to their composition,
or be unaware of how to use the information provided by the rubric to improve their
writing. The solution to this problem goes beyond just teaching students how to
read the rubric. As they are stakeholders in the writing and assessment process the
most effective rubric is one that has been “created with (the) students and reflects
their values, goals, and language” (Turley & Gallagher, 2008, p.90). While this can
also take up time it is an essential part of the process of using rubrics in the
classroom as it is the only way to ensure that both the students and the teacher
understand the nature of the assessment and it ensures a solution that links together
the concerns of the various “stakeholders” in the assessment process.
2. Final Reflections
While the analysis of the use rubrics in the second language classroom is still
Gavin BROOKS
in its infancy I believe that it will become more important in the future. Similarly to
what is happening now in the field of first language writing, both teachers and
researchers working in the field of L2 composition will, in the near future, be forced
to look at the tools they are using to assess their students and decide if these tools
are doing the job for which they were designed. This should not a be viewed as a
negative trend as “rating process research can help us learn more about and improve
writing teachers’ everyday feedback practices” (Connor-Linton, 1995, p.765). Which
in turn will enable us to better help our students reach their full potential as second
language writers.
Assessment and Academic Writing
Appendix #1: Example of ESL Composition Rubric
(Haswell, 2005, p.108)
Gavin BROOKS
References
Andrade, H. G. (2000). Using rubrics to promote thinking and learning. Educational Leadership,
57(5), 1319.
Andrade, H. L., Wang, X., Du, Y., & Akawi, R. L. (2009). Rubric-Referenced Self-Assessment
and Self-Efficacy for Writing. The Journal of Educational Research, 102(4), 287302.
Bitchener, J., Young, S., & Cameron, D. (2005). The effect of different types of corrective
feedback on ESL student writing. Journal of Second Language Writing, 14(3), 191205.
Bacha, N. (2001). Writing evaluation: what can analytic versus holistic essay scoring tell us?
System, 29, 371383.
Broad, B. (2000). Pulling Your Hair out: “Crises of Standardization in Communal Writing
Assessment.” Research in the Teaching of English, 35(2), 213260.
Broad, B., & Boyd, M. (2005). Rhetorical Writing Assessment The Practice and Theory of
Complementarity. Journal of Writing Assessment, 2(1), 720.
Casanave, Christine Pearson. Controversies in Second Language Writing: Dilemmas and
Decisions in Research and Instruction. Ann Arbor, MI: University of Michigan Press,
2003.
Clary, R. M., Brzuszek, R. F., & Fulford, C. T. (2011). Measuring Creativity: A Case Study
Probing Rubric Effectiveness for Evaluation of Project-Based Learning Solutions. Creative
Education, 2(4), 333340.
Connor-Linton, J. (1995). Looking behind the curtain: What do L2 composition ratings really
mean? TESOL Quarterly, 29(4), 762765.
Ferris, D. (1995). Student reactions to teacher response in multiple-draft composition
classrooms. TESOL Quarterly, 29(1), 3353.
Ferris, D. (2009). Response To Student Writing: Implications for Second Language Students.
New York, NY: Routledge.
Hamp-Lyons, L. (1995). Rating Nonnative Writing: The Trouble with Holistic Scoring. TESOL
Quarterly, 29(4), 759762
Haswell, R. (1998). Rubrics, prototypes, and exemplars: Categorization theory and systems of
writing placement. Assessing Writing, 5(2), 231268.
Haswell, R. (2005). Researching Teacher Evaluation of Second Language Writing. In T. Silva &
P. K. Matsuda (Eds.), Second Language Writing Research: Perspectives on the Process of
Knowledge Construction (pp.105120). London: Lawrence Erlbaum Associates.
Hudelson, E. (1923). The development and comparative values of composition scales. The
English Journal, 12(3), 163168.
Huot, B. (1996). Toward a new theory of writing assessment. College Composition and
Communication, 47(4), 549566.
Hyland, K. (2010). Teaching and Researching Writing (2nd Edition). Harlow: Pearson ESL.
Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and
educational consequences. Educational Research Review, 2, 130144.
Kohn, A. (2006). The trouble with rubrics. English Journal, 95(4), 1215.
Kroll, B. (1990). What does time buy? ESL student performance on home versus class
compositions. In B. Kroll (Ed.), Second Language Writing: Research Insights for the
Assessment and Academic Writing
Classroom (pp.140154). Cambridge: Cambridge University Press.
Kroll, B. (1998). Assessing writing abilities. Annual Review of Applied Linguistics, 18, 219
242.
Paulus, T. (1999). The effect of peer and teacher feedback on student writing. Journal of Second
Language Writing, 8(3), 265289.
Sweedler-Brown, C. O. (1993). ESL essay evaluation: The influence of sentence-level and
rhetorical features. Journal of Second Language Writing, 2(1), 317.
Turley, E., & Gallagher, C. (2008). On the uses of rubrics: Reframing the great rubric debate.
English Journal, 97(4), 8792.
Weigle, S. (2007). Teaching writing teachers about assessment. Journal of Second Language
Writing, 16(3), 194209.
Wilson, M. (2007). Why I Won’t Be Using Rubrics to Respond to Students’ Writing. The
English Journal, 96(4), 6266.
Gavin BROOKS
... Paragraphs are the building blocks of academic reading and writing. However, EFL learners encounter different problems in effective paragraph reading (Brooks, 2013). Existing studies also show that Arab EFL students face problems in academic reading (Qrqez & Ab Rashid, 2017) and writing (Ahmed, 2019;Javid, Farooq, & Umer, 2013;Khadawardi, 2022;Qadeer & T'chiang, 2022). ...
... 154). Brooks (2013) affirmed that language instructors should go beyond sentence level to use rhetorical and discourse skills if they wish to enhance their students' writing skills. Hence, there is a need to thicken the investigation by providing more evidence on the efficiency of PGA. ...
Article
Full-text available
Writing and reading are two interconnected literacy skills. In English as a foreign language (EFL) mixed-ability classes, the process genre approach (PGA) has become necessary for EFL students to exchange skills and processes of reading with writing academic paragraphs. This study examines and correlates the achievement scores of academic paragraph reading and writing among EFL mixed-ability class students. Data were collected by using the achievement tests in reading and writing among mixed-ability classes of EFL students in the preparatory year of Najran University. Data were analyzed through correlation analysis and the paired sample t-test. The results showed that the mixed-ability classes of students scored high levels in paragraph understanding and moderate levels in paragraph writing, with high values of standard deviations that reflect mixed-ability classes. The results also showed that the mixed-ability levels scored low in identifying signal words and using transition words. The results also revealed a moderate positive correlation between paragraph understanding and writing. This result indicates that PGA has a potentially positive effect in mixed-ability classes, where PGA teaching materials and assessments are effective scaffolding tools for mixed-ability reading and writing classes.
... The raters were not informed of the teaching procedure or the learners involved in the study, and all the tasks were mixed together with no indication of when they had been written. The tasks were rated using the ESL composition profile by Jacob et al. (1981), which is a popular and reliable rating scale for writing tasks among language teachers and researchers (Brooks, 2012;Ghanbari, Barati & Moinzadeh, 2012). This scale assesses five components, including content (0-30), organization (0-20), vocabulary (0-20), language use (0-25), and mechanics (0-5). ...
Article
Full-text available
This paper investigated the impact of direct unfocused written corrective feedback (WCF) on EFL students' writing improvement, self-efficacy, and anxiety. To this aim, 52 Iranian male learners were selected as participants by using the Oxford Placement Test and randomly placed in an experimental and a control group. The participants completed a pre-test that included a writing task, the writing self-efficacy questionnaire (WSEQ), and the Second Language Writing Anxiety Inventory (SLWAI) to assess their writing skill, writing self-efficacy, and writing anxiety, respectively. Having attended 15 sessions of writing instruction in which only the experimental group received WCF, the participants again completed a writing task, the WSEQ, and the SLWAI in the posttest procedure. The results showed that the experimental group outperformed the control group in all three constructs, indicating that WCF has a positive impact on EFL students' writing performance, self-efficacy, and anxiety. Implications of the study are presented.
... Challenges in academic writing have been discussed widely and consistently in different contexts; in addition to small-scale surveys which target specific problems, there is the so-called "research synthesis", which tries to make sense of the vast body of research in second language writing (Silva, Brice, 2004;Leki, Cumming, Silva, 2008;Silva, Matsuda, 2010;Hinkel, 2011). The areas covered in these comprehensive reviews include challenges in discourse construction (Plakans, 2009), development and arrangement of ideas, cohesion, coherence (Fareed, Ashraf, Bilal, 2016), grammar and vocabulary (Hinkel, 2004), plagiarism (Pecorary, 2003), assessment (Brooks, 2012). Most of these problems result from differences between L1 and L2 in terms of organizing, structuring and utilizing discourse moves; constructing, placing or omitting thesis statements; different approach to rhetorical development, argumentation, persuasion, exposition/ narration; neglecting to account for counterarguments and to anticipate audience reactions; supporting arguments and claims by means of statements of personal opinions and beliefs; leaving argumentation unsupported; sequencing ideas and explanatory information differently; less fluent and less detailed/explanatory prose; shorter and less elaborated texts; relying more on personal opinions and including less fact-based evidence in argumentation and exposition; over-or under-estimating the amount of readers' background knowledge and the need for textual clarity, explicitness, and specificity; delaying or omitting thesis/main point statements; shortening conclusions/closings; employing different strategies for extracting/citing information from sources, as well as paraphrasing, quoting, and including source material in their own writing. ...
Article
Full-text available
Academic writing in courses taught in English is one of the areas which pose challenges for university students. Due to a lack of consistent practices in teaching writing in secondary education, school graduates encounter serious difficulties in their transition to academic discourse and writing in English required for some disciplines. The article discusses students’ attitudes and perceived challenges in writing, and reveals discrepancies between students and instructors’ perspectives. The discussion is mapped against the widely researched area of challenges in academic writing, and is based on analyzing quantitative and qualitative data, obtained from a small-scale survey with BA and MA students of English studies in a Bulgarian university, and focus-group interviews with their instructors. The findings of this interdisciplinary study provide insights into students’ challenges in academic writing. It also helps to raise students’ awareness of their own strengths and weaknesses, and identify areas for instructors’ intervention and remedial work.
... The use of rubrics as an assessment tool has been traced back to 1912, although becoming much more widely used from the 1970s on (Brooks 2012). Mertler (2000) describes rubrics as "rating scales… used with performance assessments. ...
Method
Full-text available
Rubrics are used widely in education and other contexts to measure performance. This is a brief note reviewing the literature and seeking to draw some conclusions as to what 'best practice' is for their development and application
... Each set of criteria changes a four-level subjective judgment scale into interval scores. This profile is one of the most commonly used and dependable profiles for ESL composition rating (Lee et al., 2008), and its traits were designed by writing researchers working for a testing organization and are probably one of the most recognizable rubrics in the field of second language writing (Brooks, 2012). Furthermore, Ghanbari et al. (2012) stated that Jacob's ESL Composition profile had gained popularity among second language teachers and researchers since its introduction in 1981. ...
Article
Full-text available
English as a foreign language (EFL) writing is one of the most important skills besides speaking for candidate teachers in the undergraduate English programs at higher institutions. The current study hypothesizes that pair-reviews or collaboration in the teaching and learning process of paragraph writing contributes to students’ successful language development. The subject of the study was a class (N=28) of third-semester students majoring in English language teaching at the Faculty of Islamic Education and Teacher Training of Universitas Islam Negeri Imam Bonjol in Padang, Indonesia. The students’ writing skills were developed through exposure to actual writing by introducing paragraph developments. Through the process of planning and writing, they were guided to the use of pair-reviews. After a training period, they were tested for the final exam, in which no review was allowed. The study found that students’ writing competence as the result of pair-reviews varied depending on the type of paragraphs being developed. The student’s average score was 73.5 in the illustration type, 78.28 in the comparison type, and 71.96 in the categorization type. In general, students’ average scores during this pair-review were 74.58. In the post-test, students’ average score was 79.3, indicating that pair-reviews significantly affect their writing competence. A conclusion was drawn regarding the influence of pair-reviews on their final writing product (post activity). The study further elaborates on the activities during pair-reviews, roles, and the practice of collaboration in Minangkabau culture and traditions in Padang.
... Rubrics as an influential assessment instrument in learning have been applied, along with standardising the assessment of students in higher education, for almost three decades around the world. Rubrics have been used as a tool to analyse writing since 1912, when Noyes first used rubrics to systematise the evaluation of student compositions (Brooks, 2012). Noyes first proposed uniform, definite, and fixed standards of measurement to avoid subjective judgments (Noyes, 1992). ...
Article
Full-text available
This article highlights the potential of rubrics as tools for effective assessment in education by encouraging students to pursue high-quality education, assessing students in ways other than traditional testing, and fostering outcome-based education. It highlights the importance of quantitative research for data gathering and analysis, as well as the use of statistical applications like SPSS for data analysis in educational research. The study examines various types of rubrics and their components to create a full rubric for classroom effectiveness. It also provides literature on appropriate rubric usage for students and teachers, as well as rubric researchers. The findings suggest that a nationwide database of discipline-specific rubrics is necessary for preserving and advancing standards in technical and higher education. A guidelines handbook should be developed before employing rubrics. Implementing rubrics at elementary and secondary levels is essential for students to advance to higher education, as they may be exposed to unprepared OBE approaches in cities with inadequate communication and technology skills. The Ministry of Higher Education should determine teaching obligations and responsibilities, and universities should implement rubrics immediately and emphasise them. The findings are relevant to both students and teachers, emphasising the importance of user perception, rubric design, and intended use in determining the usefulness of rubrics in raising student performance. By optimising variables based on trustworthy data and knowledge of the intended educational context, educators can maximise students' learning opportunities.
... A continuous remainder is given by the teachers to their students for the improvement of English, which is considered the second language for them. It is one of the crucial duties of English teachers from their job perspective (Brooks, 2013). It is done through feedback and corrections. ...
Article
Full-text available
Grading rubrics are valuable tools in education that clarify expectations, facilitate transparent grading, improve self-regulation, and increase learning outcomes. This paper explores the use and impact of grading rubrics from the perspective of prospective teachers, reviewing literature, research, and a survey. The survey was done with 400 prospective teachers from 4 universities in Lahore, and each of them filled out a questionnaire about their thoughts and experiences with rubrics. The findings of this survey are included in the paper as well. This work contributes to the existing body of research on the use of rubrics in education by giving a detailed and critical evaluation of the research that has already been conducted, as well as by presenting fresh empirical data from the viewpoint of prospective teachers, thus intending to stimulate more research and practice on grading rubrics in the field of education.
... p. 534). Turley and Gallagher (2008, p. 88) observed that though, the rubrics designed by Noyes and his contemporaries were structured as a homogenous form of measurement for the "administrators and investigators measuring and evaluating the nationwide school system efficiency" (Brooks, 2013). However, recent studies on rubrics' use and effectiveness reveal that the teacher-made rubrics, specifically, not only serve as a reflection of the teacher's expectation towards the learning of their learners but also assist the teachers in thinking primarily what to teach and what direction to set for their learners in the learning process (Rini, & Purnawarman, 2019, p. 362). ...
Article
Full-text available
This paper examines the receptiveness among prospective teachers for rubrics as effective feedback for learning and assessment. It also investigates the dispositions among prospective teachers toward the practice of rubrics in their future classrooms. It uses a qualitative research approach and collects data from 15 participants who were selected through a purposive sampling method from four universities. The data is analyzed using NVivo 12 Pro, and themes are derived and calculated based on the frequency of responses. The paper finds that using rubrics for grading has various benefits for both prospective teachers and their learners, such as improvement in grades, motivation, awareness, feedback, support, skillfulness, quality, time management, critical thinking, etc. The paper also finds that there are some challenges and gaps in the implementation and practice of rubrics in the Pakistani education system.
... The rubric as an assessment tool in pedagogical practice was introduced in a language classroom to analyse writing in 1912 (Brooks, 2013). It is defined as "a scoring tool for qualitative rating of authentic or complex student work." ...
Conference Paper
Full-text available
The paper describes the importance of a rubric as an assessment tool in art education. The assessment of a work of art, for example, a landscape or a portrait painting usually consists of a combination of objective information and a subjective point of view, which makes it difficult for educators to assess students’ learning outcomes. The use of rubrics is considered an innovative way for educators not only to measure the student’s comprehension and skills but also as a teaching method to increase learners’ engagement in order to bring the creation of art to the forefront of the learning process within school art education. The relevance of the subject of this paper is defined by the changes in the evaluation system during the ongoing education reform in Latvia. The rubric, as an assessment tool in Latvia, was first introduced in 2020 as a part of the reform of the school curriculum. Rubrics for creative art assignments observe the main stages of the learning process, including sketching, expression of original ideas, creative work, documentation of the creation process and self-evaluation. It can be used for both, summative and formative assessments of learning outcomes. The criteria are based on the Common European Framework of Reference for Visual Literacy which was developed by The European Network of Visual Literacy. Therefore, the research aim is to determine the optimal way to evaluate students’ work in art lessons within the framework of school art education. The research involved two stages of data collection. During the first stage (2021) there was conducted a survey of 60 Latvian secondary school teachers that identified several problematic issues concerning the evaluation criteria. In the second stage (2022) the in-depth analysis was performed to investigate the connection between the feedback from educational practice and recommended rubrics in new curricula.
... Ancak yabancı dil olarak Türkçe öğretimi alanında genel geçer kabul gören puanlama anahtarları mevcut değildir. Standart dereceli puanlama anahtarlarının güvenirliğinin artırılabilmesi için öğreticilerin/değerlendiricilerin de değerlendirme prosedürlerini dikkatli şekilde uygulamaları ve puanlama anahtarının kullanımına iyi derecede hâkim olmaları gerekir (Brooks, 2013). Tüm dil öğretim merkezlerince kabul görecek ortak ve standart, seviyelere göre ayrıştırılmış dereceli puanlama anahtarları geliştirildikten sonra ikinci basamak puanlama anahtarlarının kullanımını detayları ile ve örneklerle açıklayan birer kılavuz kitapçıklarının hazırlanmasıdır. ...
Article
Full-text available
Yabancı dil olarak Türkçe öğretimi sürecinde dört temel dil becerisinden yazma becerisini ölçme değerlendirmeye yönelik literatür incelendiğinde alan yazında ağırlıklı olarak yazmanın değerlendirilmesine odaklanıldığı, ölçmeye değerlendirme içinde kısaca yer verildiği görülür. Oysa istenilen şekilde doğru ve objektif değerlendirme çıktılarının elde edilmesi için gerekli en temel unsur sağlıklı bir ölçmenin yapılabilmesiyle mümkündür. Ölçme araçlarının öğrenici yeterliğini doğru bir şekilde ölçebilmesi için taşıması gereken bazı özellikler olmalı ve bazı kriterlere uygun olarak hazırlanması gerekmektedir. Yazma becerisi öğrenicinin dil düzeyinin anlaşılabilmesinde oldukça etkili bir beceridir. Çünkü öğrenici yazarken hedef dilde düşünebilme, yapı ve içerik bilgisi, dilin zenginliklerine hâkimiyet gibi birçok yönünü ortaya koyar. Ancak yazma becerisi okuduğunu/dinlediğini anlama becerilerinin aksine objektif şekilde puanlanabilmesi ve değerlendirilebilmesi zor bir beceridir. Okuma ve dinleme becerilerinde soruların belirli doğru cevapları vardır ve öğrenici yanıtları bu cevaplar üzerinden puanlanarak başarılı ya da başarısız olarak değerlendirilebilirler. Oysa yazma becerisinde tek ve kesin bir doğru cevap yoktur ve her öğrenicinin yazma kâğıdı müstakildir. Dolayısıyla değerlendirilmesi güç olan yazma becerisinin ölçme ve değerlendirme sürecinin sağlıklı şekilde gerçekleştirilebilmesi için iyi yapılandırılmış ve geliştirilmiş sorulara ve dereceli puanlama anahtarlarına ihtiyaç duyulmaktadır. Araştırmada literatür taraması yöntemi kullanılarak bir derleme çalışması yapılmıştır. Alan yazındaki çeşitli çalışmalar taranarak konuya ilişkin bilgiler bir araya getirilmiştir. Çalışmanın evrenini yabancı dil öğreticileri/değerlendiricileri oluşturmaktadır. Araştırma ile yazma becerisinde öğrenicinin hedeflenen yeterliklere sahip olup olmadığının belirlenebilmesi için hazırlanacak yazma sorularının taşıması gereken özellikler tespit edilmeye ve genel kriterler belirlenmeye çalışılmıştır.
Article
Full-text available
Instructional rubrics help teachers teach as well as evaluate student work. Further, creating rubrics with your students can be powerfully instructive.
Article
Instructors and administrators in the portfolio program at City University urgently desired to standardize their evaluations of students' writing - to make their judgments quick, easy, and homogeneous. Because they refused to compromise the rhetorical and pedagogical integrity of their decisions, however, participants in this study found that evaluative ambiguity and conflict stubbornly remained. Though extremely frustrating for the writing faculty involved, the evaluative crises they experienced set the stage for a radical reconceptualization of the process of standardization. In fact, their struggles delivered communal writing assessment to the doorstep of hermeneutic standardization, a paradigm that can accommodate both of writing assessment's historically antagonistic commitments: to fairness and consistency as well as to the diversity, complexity, and context-dependence that are characteristic of rhetorical experience. Using qualitative methods to analyze observation- and interview-based data as well as written documents, I explore how City University's writing instructors grappled with their crises of standardization. Participants experienced multiple breakdowns in the project of standardization, of which this article details the two most severe: crises of textual representation and crises of evaluative subjectivity. I conclude by examining conflicting interpretations - psychometric and hermeneutic - of City University's crises. In advocating the second interpretation, I argue that viewing City University's struggles through a hermeneutic lens can lead to revised understandings and practices that allow teachers of composition to honor more fully their theoretical, pedagogical, and ethical commitments when judging students' writing.
Article
Maja Wilson believes that efforts to standardize language through rubrics and generalized comments provide a disservice to students and undermine the power of the reading and writing experience. She advocates making use of our subjectivity as readers, conceding that her values cannot be standardized and often shift in response to interactions with students and their writing.
Article
This book provides an authoritative, readable and up-to-date guide to the major themes and developments in current writing theory, research and teaching. Written in a clear, accessible style, it covers theoretical and conceptual issues, addresses current questions and shows how research has fed into state-of-the-art teaching methods, practices, materials and software applications. Thoroughly updated and revised, this second edition also contains a new chapter on important issues in writing such as genre, context and identity.
Article
This volume synthesizes and critically analyzes the literature on response to the writing of second language students, and discusses the implications of the research for teaching practice in the areas of written and oral teacher commentary on student writing, error correction, and facilitation of peer response. The book features numerous examples of student texts and teacher commentary, as well as figures and appendices that summarize research findings and present sample lessons and other teaching materials. It is thus simultaneously comprehensive in its approach to the existing research and highly practical in showing current and future teachers how this material applies to their everyday endeavors of responding to student writing and teaching composition classes. Response to student writing--whether it takes the form of teachers' written feedback on content, error correction, teacher-student conferences, or peer response--is an extremely important component of teaching second language writing. Probably no single activity takes more teacher time and energy. Response to Student Writing is a valuable theoretical and practical resource for those involved in this crucial work, including L2 composition researchers, in-service and preservice teachers of ESOL/EFL writers, and teacher educators preparing graduate students for the teaching of writing. © 2003 by Lawrence Erlbaum Associates, Inc. All rights reserved.
Article
Research in L1 and L2 student writing has suggested that teacher response to student compositions is most effective when it is given on preliminary rather than final drafts of student essays (Freedman, 1987; Krashen, 1984). One area of research in L1 and L2 composition is the assessment of student reactions to the feedback they receive from their teachers (Cohen & Cavalcanti, 1990; Hedgcock & Lefkowitz, 1994; Leki, 1991; McCurdy, 1992). However, most previous studies of ESL student response to their teachers' written comments on their essays have been undertaken in single-draft, rather than multiple-draft, contexts.In this study, 155 students in two levels of a university ESL composition program responded to a survey very similar to the ones utilized by Cohen (1987) and McCurdy (1992) in single-draft settings. The results of the survey indicated that students pay more attention to teacher feedback provided on preliminary drafts (vs. final drafts) of their essays; that they utilize a variety of strategies to respond to their teachers' comments; that they appreciate receiving comments of encouragement; and that, overall, they find their teachers' feedback useful in helping them to improve their writing. Responses also showed that students had a variety of problems in understanding their teachers' comments, suggesting that teachers should be more intentional in explaining their responding behaviors to their students.