Content uploaded by David Carless
Author content
All content in this area was uploaded by David Carless on Jan 24, 2015
Content may be subject to copyright.
Exploring learning-oriented assessment processes
David Carless
Springer Science+Business Media Dordrecht 2014
Abstract This paper proposes a model of learning-oriented assessment to inform
assessment theory and practice. The model focuses on three interrelated processes: the
assessment tasks which students undertake; students’ development of self-evaluative
capacities; and student engagement with feedback. These three strands are explored
through the analysis of assessment practice in context. The research method involves in-
depth classroom observations of five recipients of awards for teaching excellence across
multiple disciplines; and semi-structured interviews with these teachers and a sample of
their students. Findings highlight assessment tasks promoting thinking and practicing in the
discipline; the use of critical reviews to develop student understandings of quality work;
and ‘same day feedback’ to promote timely dialogues with students. The coherence of the
model is discussed and some areas for further exploration are suggested.
Keywords Learning-oriented assessment Assessment task design
Student self-evaluation Engagement with feedback
Introduction
The theory and practice of student assessment in higher education has generated a lot of
debate over the last 20 years or so. It is probably fair to say that there have been a number
of positive developments during that time in promoting assessment for learning: more
varied assessment tasks than merely a diet of final examinations; greater transparency in
assessment criteria and strategies for promoting engagement with them; and a growing
awareness of the importance, and the challenges, of developing effective feedback pro-
cesses. There remain, however, various concerns about assessment and a recent paper in
this journal describes a lack of sophistication in assessment practice and limited incentives
D. Carless (&)
Faculty of Education, University of Hong Kong, Pokfulam, Hong Kong
e-mail: dcarless@hku.hk
123
High Educ
DOI 10.1007/s10734-014-9816-z
to innovate, especially in contexts where research is prioritized over teaching (Norton et al.
2013).
There is now widespread acceptance for approaches to assessment focused on pro-
moting and enhancing student learning (Sambell et al. 2013). A key aspect of such
approaches is the extent to which assessment tasks are conducive to stimulating appro-
priate student learning approaches. A fundamental challenge for teacher management of
assessment, however, is that it has to do ‘double duty’ (Boud 2000), serving varied and
sometimes potentially competing functions. Assessments have to encompass formative
assessment for learning and summative assessment for certification; they have to focus on
the immediate task and equipping students for lifelong learning; and they have to attend to
the learning process and the substantive content domain (Boud 2000). Many teachers in
higher education perceive that they lack individual autonomy and find themselves pulled in
different directions by assessment purposes other than facilitating student learning (James
2014). Effective assessment practice should focus on enhancing student learning processes,
but needs to be informed by the awareness that assessments do double duty.
In this paper, I draw on the analysis of assessment practices in context to develop three
interrelated contributions. The first is to present and discuss a model of learning-oriented
assessment. Second, I present data from case studies of five recipients of teaching awards
to provide some illustrations of how learning-oriented assessment processes are imple-
mented in undergraduate education (see Carless 2015, for a fuller analysis). Third, I probe
the implications for the model in light of the findings and suggest some areas for further
research. The overall aim of the paper is to explore investigate learning-oriented assess-
ment processes and discuss implications for theory and practice.
Framing the study through learning-oriented assessment
Learning-oriented assessment is defined as assessment where a primary focus is on the
potential to develop productive student learning processes. In particular, the ‘right kind’ of
summative assessment can be fruitful in stimulating appropriate student learning dispo-
sitions and behaviors. Summative assessment can be learning-oriented when, for example,
it encourages deep rather than surface approaches to learning and when it promotes a high
level of cognitive engagement consistently over the duration of a module. The processes of
working towards well-designed summative assessment can also afford opportunities for
formative assessment strategies, such as peer feedback, student self-evaluation and related
teacher feedback.
Learning-oriented assessment has hitherto attracted some modest attention in the lit-
erature (Carless 2007; Hernandez 2012), but has not yet been conceptualized or explored in
detail. I propose in Fig. 1, three simple but hopefully powerful interlocking principles of an
approach to assessment predicated on the development of student learning processes. The
aim of the model is to capture the core elements of a learning-oriented assessment
approach and indicate their inter-relationships. The three principles are developed from
synthesizing and reformulating key literature on assessment for learning in higher edu-
cation (e.g. Boud and Falchikov 2007; Gibbs 2006; Sadler 2010; Sambell et al. 2013).
The apex of the model is represented by the assessment tasks which students are
undertaking: key drivers of their efforts and learning approaches. Conceptualizing the
nature of assessment task design and implementation could take various forms. For the
purposes of this paper, I frame the discussion of learning-oriented assessment task design
through the lens of ways of thinking and practicing (WTP) in the discipline (McCune and
High Educ
123
Hounsell 2005) because it seems to me to be a particularly useful way of encapsulating
some of the key issues around the development of quality undergraduate student learning
in context.
From a study of undergraduate courses in the biosciences, McCune and Hounsell (2005)
suggest that high quality learning relates to students developing an evolving grasp of WTP
in the subject. They describe WTP as denoting the richness, depth and breadth of what
students might learn through engagement with a given subject area in a specific context, for
example, coming to terms with particular disciplinary forms of discourse, values or ways
of acting (McCune and Hounsell 2005). WTP could also encompass an evolving famil-
iarity with the conventions of scholarly communication within the discipline and the
relevant professional community (Anderson and Hounsell 2007). A major means of sup-
porting the development of WTP is by engaging with and assessing ‘real-life’ problems,
contextualized within specific disciplinary situations what is sometimes referred to as
authenticity in assessment.
Turning now to the other two strands of the model, learning-oriented assessment task
design is supported by the interconnected elements (illustrated at the bottom of Fig. 1by
inverted arrows) of evaluative expertise and engagement with feedback. Evaluative
expertise on the left of the figure represents the evolving ability of students to engage with
quality criteria, develop their self-evaluative capacities and make informed judgments
about their own work, and that of others. Evaluative expertise is critical for student
learning because to monitor and improve their learning, students need to know what quality
performance involves and entails (Sadler 1989). A crucial role of the teacher is to assist
students in developing this capability in discerning quality and making complex judgments
(Sadler 2010). Developing assessment for informing judgment involves exposure to
models and opportunities for practice (Boud and Falchikov 2007).
Feedback is both a core aspect of improvement and something which research evidence
indicates is difficult to manage effectively (Evans 2013). For students to engage effectively
with feedback (the right hand side of the figure), they need to develop a conception of
quality which begins to approach that of the teacher (Sadler 2010). They need this evolving
capacity in order to facilitate the decoding and uptake of feedback messages which can
often seem cryptic or opaque. The feedback strand of the learning-oriented assessment
model places emphasis on student engagement with feedback. It is what the students can
do with feedback, rather than how the teacher provides it, which is crucial (Boud and
Molloy 2013). There are, for example, challenges in relation to timing and modes of
feedback; student understanding and uptake of feedback; and student affective responses
(Evans 2013). Current thinking suggests that the key purpose of feedback is to support
Learning oriented
assessment tasks
Developing evaluative
expertise
Student engagement
with feedback
Fig. 1 A model of learning-
oriented assessment
High Educ
123
students in developing their self-regulative capacities (Nicol and Macfarlane-Dick 2006);
and feedback needs to be re-engineered to encourage dialogues of different forms: peer
feedback; internal feedback to the self; as well as external feedback from the teacher (Nicol
2010).
The two arrows leading from the top to the bottom of the figure suggest that the nature
of the assessment task or tasks impact on prospects for the development of evaluative
expertise and engagement with feedback. Task-type is one salient issue, for example, ‘on-
display assignments’, such as oral presentations or posters in which work is openly evident
to peers rather than just privately submitted to tutors (Hounsell 2003) provide opportunities
for student appreciation of quality and associated development of evaluative expertise. The
number and sequencing of tasks are another factor, for example, multi-stage assignments
tend to provide more opportunities for student engagement with feedback than a single
end-of-semester task.
To sum up, I have framed the discussion around three elements of a model of learning-
oriented assessment: task design prioritizing assessing WTP in situations which mirror
real-life uses of the discipline; the development of student evaluative expertise; and student
engagement with feedback. The unified nature of the model suggests that task design
should set up possibilities for the other two strands; and that there is interplay and potential
overlap between students’ development of evaluative expertise and their engagement with
feedback. The model does not seek to measure student learning outcomes; instead it
suggests three important precursors for the kind of learning processes which are likely to
stimulate student engagement.
Method
The research is framed around the following questions:
How is learning-oriented assessment implemented by selected award-winning teachers?
What are the teachers and students perceptions of the main learning-oriented assessment
issues?
Participants
The teacher participants from an international research intensive university had all received
internal awards for teaching excellence. Candidates for these awards are expected to
demonstrate excellence in: teaching and engagement with student learning; curriculum
design and innovation; and the scholarship of teaching (Prosser 2013). A proposition
underlying the choice of sample was that award-winning teachers might carry out learning
and assessment practices which engage students and cast light on learning-oriented
assessment processes. This was to be explored via in-depth qualitative data collection and
analysis of a small purposefully selected sample of teachers.
I began the research process by carrying out an exploratory case study of an award-
winning teacher from the Faculty of Business whose participation in a previous interview
study (Carless et al. 2011) had evidenced innovative practices relevant to the research
focus. This case formed the prototype for extending the study across additional disciplines
so as to allow contextualization of insights in different subjects. Based on an overview of
award-winning teachers in the university, I contacted potential participants and subsequent
High Educ
123
case studies focused on teachers of the following subjects: Architecture, Geology, History
and Law.
The teachers were involved in undergraduate and postgraduate teaching, and doctoral
supervision. After they agreed to participate, I negotiated with them a suitable under-
graduate class to observe bearing in mind the aims of the research. Student participants
were undergraduates enrolled in the selected courses taught by the award-winners. In the
process of observations, my co-researcher and I interacted with a range of students and
invited a sample of them to participate in semi-structured interviews.
Data collection
The study sought to understand how teachers implemented and students experienced
assessment in the modules under investigation. It did not aim to measure student
achievement, instead it sought to explore the processes in which students were involved
and their perceptions of issues arising. In line with this orientation, the principal means of
data collection were classroom observations and interviews. I was assisted by a co-
researcher, a recent doctoral graduate specializing in qualitative research methods similar
to those adopted in the study.
The main purposes of classroom observations were to develop an understanding of how
classroom processes unfolded, particularly in relation to aspects relevant to the model of
learning-oriented assessment. Classes were of 2 h duration in Geology, History and Law;
3 h in Business; and in Architecture studio activities were open-ended and did not follow a
set time schedule. Six to ten sessions per teacher were observed, totaling 39 sessions across
the five teachers or 92 h of classroom observations. Detailed field notes were collected to
describe classroom processes; develop provisional insights into issues relevant to the
research focus; and identify issues for follow-up through interviews.
I carried out two main formal semi-structured interviews with the teachers: one at the
outset of the study to understand their views on learning-oriented assessment issues and
how teaching and assessment were approached in the courses; and the other to explore
issues arising from the observations and the student data. Additional shorter interviews
and/or follow-up e-mails were also used to collect their feedback on my provisional
analysis.
Students from each of the classes were interviewed in order to gauge their perceptions
of the learning-oriented assessment processes in the modules under discussion. I carried
out some interviews myself whereas my co-researcher conducted the majority. Interviews
focused on the relevant learning-oriented assessment issues arising in a particular course.
Students were usually interviewed once for about half an hour, although longer interactions
were common and a number of students were interviewed twice when the pertinent issues
were significant over a sustained period of time. Overall, 90 interviews with 54 students
were carried out and these were evenly spread across the five disciplines. All teacher and
student interviews were recorded, transcribed verbatim and analyzed as per the procedures
described below.
Data analysis
The observational and interview data were assigned codes which represented my inter-
pretation of their main essence. The coded data were organized thematically around the
three main concepts established a priori by adopting a learning-oriented assessment
High Educ
123
perspective. Case reports were developed for each of the five cases, including the multiple
elements of classroom observational data, teacher and student perceptions.
For the purposes of this paper, data reduction and selection of examples from the case
reports are a central move. My aim is not to provide a comprehensive analysis of a case but
to present one example from three different cases which illustrate the relevant feature of
the model of learning-oriented assessment. Selection of quotations seeks to present a
balanced view of the evidence from the wider dataset.
The trustworthiness of interpretations was mainly developed through the following
strategies: triangulation between observational data, teacher interviews and student inter-
views; prolonged engagement with the participants in the classes under discussion; and
data collection and interpretation operating in tandem. I also carried out extensive member
checking whereby the informants commented on drafts of the case studies pertaining to
their teaching and I made modifications based on their clarifications. Peer reviews by both
assessment specialists and scholars belonging to the relevant disciplines were also carried
out. The involvement of two researchers was also particularly useful in that I presented
issues to my co-researcher and made revisions, or reflected further, based on her
comments.
Findings
The findings are divided into three main sections which address the corresponding com-
ponent of the learning-oriented assessment model. In each sub-section, I begin with a brief
general overview across cases and then discuss in detail a key feature from a particular
case.
Assessment task design and implementation
Task design across the cases involved different elements and emphases. The Architecture
case involved continuous assessment of designs for a village house which were collected
into a cumulative portfolio. In Geology, there was assessed laboratory work on rocks and
minerals; a group project on a chosen novel problem; and a final exam. In Business, there
was a written assignment based on a business case; oral presentations of product ideas; and
a participation grade involving both oral classroom and written online contributions. The
Law case involved traditional tests and exams as well as a portfolio-based reflective media
diary identifying and analyzing legal issues from local newspaper reports. The History case
is chosen for discussion in detail because it exemplifies prominent features of the design
and implementation of learning-oriented assessment tasks.
Making History is a first year foundation course taken by a diverse cohort of 110
students from a variety of disciplines, taught by Marty (all names are pseudonyms). The
main intended learning outcomes are for students to develop their abilities to: engage
critically with representations of the past; analyze and use evidence to construct historical
accounts; critically interpret interconnections between the past and the present; and reflect
upon the value of historical awareness.
There are multiple assessment elements for the course. The first task is a fieldwork
report of 1,000 words which involves a choice between a Museum visit (to a local Museum
of the student’s own choice) or a Scavenger Hunt (an internet-based simulation in which
students visit local landmarks examining historically-based clues and artifacts). The tea-
cher explains some of the thinking behind these tasks in the following quotation:
High Educ
123
It’s important to get them out of the classroom into the context so that they can see
ways in which people may look at the past …They go into the museum and are
asked to critique the space. They are provided with a list of questions as a framework
so that they are not simply passively accepting what there is to see. There may be
more than one version of a story and competing discourses; some of the students are
attuned to this, whereas others are not.
In Marty’s views, I see a focus on providing tasks which relate to what it is to be a
historian: involvement in critique, contestation, competing discourses and the meaning of
the past.
There is also continuous assessment: 30 % of the course grade involves assessed par-
ticipation (15 % for participation in the weekly tutorials); and 15 % for a weekly personal
response task entitled ‘one sentence response’. This latter task requires students to com-
plete during the weekly lecture, a short handwritten personal response of about 20–30
words to an issue which is the focus of the next class. Two examples of these one sentence
response tasks are:
•What are the essential qualities of a good museum?
•How might thinking historically be able to help us realize a better future?
Aspects of the teacher rationale for this exercise are addressed in the following
quotation:
I want to assess their learning experience during classroom time and provide an
incentive for attendance. I am a firm believer in the value of short written exercises. I
think it is a great way of honing their communication skills; after all we live in an age
of Twitter and students rarely have call to write long research pieces.
He explained that originally he had conceived it as partly a convenient way of taking the
class attendance (an institutional requirement). Later, he saw it more as a way of
encouraging student participation; giving students a voice in the class; and informing his
teaching by helping him understand students’ prior understandings and experiences.
The final assignment is an individual project counting 40 % (10 % for a draft and 30 %
for the final submission). This is a 3,000 word piece of writing, although it can alterna-
tively be presented in the form of a podcast, wiki or other use of technology. Students can
choose from a list of topics or propose one of their own. The teacher spoke about the
project in the following terms:
I try as far as possible to get them to generate the materials and then to analyze and
critique. It’s an attempt to give students an opportunity to showcase their abilities;
master the discourse of history; see the past is alive and that participants are re-
shaping it; and see themselves as historians.
I interpret this quotation as reiterating an intention to involve students in ways of
thinking and practicing as historians.
Space precludes a detailed discussion of student views on the different assessment
items. Instead, I highlight a representative selection of their overall views on the assess-
ment approach:
The assessment provides real flexibility. You have activities to do and you are graded
on them. Overall, I would say it is good.
High Educ
123
I quite like the assessment approach because I can take the initiative to learn. The
teacher encouraged us to raise our own opinions and search for evidence to support
them.
The assessments for this module motivate you to learn more. If you want to get an
outstanding overall result, you have to work hard on each task.
The assignments of this course require us to do some field research and independent
thinking. It was hard work but it gave me a sense of achievement.
The assessment is open and flexible and you can have your own ideas. What I have
gained is regeneration of ways of thinking, understanding issues from different
angles.
These comments bring out some of the student perceptions of features of the assess-
ment: flexibility; opportunities to take the initiative; motivation and independent thinking.
The final student comment is suggestive of the kind of learning outcomes which are being
developed: thinking skills and alternative ways of understanding.
In sum, the design of the assessment was that students were involved in multiple tasks
rather than a single end of semester assignment. The content involved the teacher pro-
viding tasks focused on participation in the discipline, mirroring historical ways of
thinking.
Developing student evaluative expertise
In different ways, all of the five teachers in the study communicated performance criteria to
students and endeavored to exemplify some of the elements of quality assignments. The
Geology teacher explicitly explained the qualities of a good group project in his intro-
duction to that particular task. The History teacher held an optional additional workshop to
try to illustrate strategies for tackling the project assignment, shared a good sample
assignment and added some related commentary. The Law teacher posted annotated
exemplars of good previous assignments on the course website which was much appre-
ciated by students. The Business teacher videoed student oral presentations, replayed short
extracts and engineered reflective discussions which illuminated the characteristics of good
business presentations.
The most extended treatment of the development of evaluative expertise arose in Sam’s
teaching of a Year 1 course: Introduction to Architectural Design taken by 65 students
divided into six tutor groups of around 11 students per group. Assessment for the course
was based on a portfolio of designs for a house in a nearby village, including presentations
and iterations of designs. An important aspect of the development of the assessment
portfolio was a series of critical reviews, called ‘crits’, where students present in front of
their peers a design project to a jury of teachers and receive comments on their work. There
were a number of crit sessions: two crits at an interval of about 1 month with two tutors
providing commentary; and a final review involving a jury of four tutors, including one
coming from outside the university. The crit is part of the signature pedagogy of design-
based subjects (Schrand and Eliason 2012) and mirrors WTP in the discipline as the need
to pitch designs is a core element of the architecture profession.
Sam commented on the crit as follows:
High Educ
123
All teaching is based on some kind of dialogue. The crit allows it to be personal, it
also allows for feedback to be reflective of the students’ intentions. In addition, you
are opening it up to an outsider view.
The crit is somewhat different to feedback on work in progress because of the public
nature of the dialogue. So the crit is both specific to an individual student and reflective in
relation to bringing together wider public ideas about Architecture. A challenge for the
audience of peers is in picking up messages that may either inform their own work or
stimulate their own understandings of the theory and practice of Architecture and its
discourses.
Sam highlighted three aspects of quality in architectural design: craft—the quality of the
craftsmanship in the designs; contribution, including originality and creativity; and concept
as embodied in the overall design process. I asked Sam how students come to develop a
sense of quality:
I think it’s a balance between rationale and intuition. In some sense it is rule-driven,
but you also have to know the limit of those rules. …I also want them to learn how
to strategize the process because the goal is not to produce a great design but to learn
how to structure a design process.
I took up a parallel line of inquiry by asking a student (pseudonym Laurence) what he
thought the tutors were looking for in terms of quality work:
I think they are looking for consistency throughout the body of work; depth of
thought; and craftsmanship, the quality of the drawings. It’s also about communi-
cation: the clarity of the intentions in your drawings. But having said that, what the
tutors are looking for is somewhat unclear to me and sometimes I get disillusioned …
there are an infinite number of permutations.
My interpretation was that Laurence was developing a good grasp of what the Archi-
tecture teachers were looking for, whilst exhibiting in the final sentence an awareness of
the challenges involved. In creative disciplines, such as design subjects, the individual
open-ended nature of problems can represent both a source of anxiety and a stimulus for
engagement.
I also had discussions with students soon after their crits. Gloria expressed the view that
it is important to interact with the tutors and try to justify the design because that is an
important part of becoming an architect. Persuasive presentation and the ability to ‘sell’
your design to a group of observers is a skill which students are trying to develop. She
related this to the idea of becoming your own critic:
I need to ask myself how I can make my design better. It is part of learning to be self-
critical. …We learn to be critical from listening to the teachers’ analysis and so we
can start to see things from another viewpoint.
In his crit, Yeung dialogued actively with the tutors and strived to justify his design.
Afterwards I asked him about this and he commented as follows:
It is difficult to convince the tutors but we should learn to make the judgments and
decisions for ourselves. I need to try to convince the tutors even though I know that I
may yield to their ideas eventually.
High Educ
123
Our observations of crits and interviews with participants led us to infer that learning to
be self-critical and making judgments for themselves were major elements of the processes
underpinning the critical reviews.
My prior expectation was that normally the development of evaluative expertise would
involve some kind of peer feedback so that students might involve themselves in critiquing
each other’s work. To my initial surprise, despite repeated observations I saw limited
interaction between the student presenter and their peers during the crits. Students told me
that this is because the crits already take a long time; and it is difficult for peers to provide
insightful comments when they are not deeply familiar with a particular design. From my
perspective, this initially seemed like a missed opportunity. Through further interviewing
and observation, however, I came to understand that peer feedback in a studio-based
discipline took a different form than that often seen in a more conventional classroom
setting. My interviews and observations indicated that peer feedback arose more informally
during work in the studio and was particularly salient in the final review discussed next.
The final review which lasted from first thing in the morning until late at night created a
lot of interest and excitement in the studio. Students were walking around examining
designs which were displayed on the walls and through models on the floor. Some students
were taking photos of the designs and engaging in discussion of aspects that caught their
eyes. Ching made a useful summary of this process:
After taking a look at others’ drawings and models, I can learn how to tackle some
technical problems and it can inspire me to think differently…Everyone’s drawings
and models are out there, it is faster and easier for you to see how a problem can be
dealt with than asking people individually.
The informal peer interaction and the visual ‘on-display’ element of the work were
particular features of the final review and I return to these issues in the Discussion section.
In sum, students in the Architecture case were involved in the iterative development of a
portfolio in which a central feature was regular presentations of their work for critical
review. These processes exposed them to dialogues with the teacher which seemed to help
them to hone their self-evaluative capacities and develop an evolving sense of quality in
architectural design. The public nature of the design studio and the on-display assignments
were also a site for spontaneous peer feedback.
Promoting student engagement with feedback
The five teachers all strived to provide relevant feedback experiences to students. The
interplay between feedback dialogues and the development of student evaluative expertise
in Architecture is evident from the previous section. In the Geology case, timely feedback
on a brief outline was provided at the outset of the group project so as to try to steer
students to find a suitable initial topic and path for their project. The History teacher
provided feedback on draft individual projects to inform the next stage of student work.
The Business teacher engineered in-class feedback dialogues both about the process of
learning and in relation to the particular assessment tasks.
In this section, I focus on feedback in the Tort Law class taught by Chris with a class
size of 180 students. The assessments for the course were a test (weighting 20 %); a
reflective media diary (20 %) focused on analyzing relevant legal issues occurring in the
local media; and a final examination (60 %). For the test and the examination, Chris
implemented a strategy of ‘same day feedback’ when there was immediate discussion of a
completed assessment task. This takes place on an optional basis in the classroom where
High Educ
123
the assessment has taken place, with students discussing possible answers to the legal
problem they have just attempted. This is also supplemented by same day online discus-
sion. Chris stated the rationale as follows:
Same day feedback is a forum for debate so that learning continues after the
assessment. It allows students to engage in a discussion of the assessment problem,
clear up misconceptions and reinforce good learning when their focus is greatest. The
immediacy of the feedback overcomes a major obstacle to learning, delayed or
ignored feedback, by providing feedback in a way and at a time when it can be
absorbed. A key principle is that the timing of feedback should be as close to the
point of submission as possible.
In this quotation, Chris shows awareness of the need to tackle the common failure of
students to engage with common forms of post-assignment feedback which are often
received several weeks after a task has been completed. The same-day discussion and
debate also brings elements of dialogue into the feedback process. Obviously, students do
not have an opportunity to revise their answer to the test or exam, but the feedback aims to
clarify issues immediately and feed-forward to future related tasks.
Chris also suggests a further rationale as follows:
The in-class or online discussion often uncovers aspects of the question not con-
sidered in the marking rubric. This is not surprising as legal problems are by nature
complex and open-ended. Students are expressly encouraged to critique the argu-
ments and offer alternative ones which, if adequately supported by cogent legal
analysis, can be included in a revised marking rubric.
This brings a collaborative element to the rubric and an additional incentive for student
engagement.
In our interviews, students expressed a variety of perspectives on the strategy of same
day feedback:
I like the immediate feedback, because the memory is fresh in my mind and I can
remember all the details.
I became aware of the depth of other students’ answers; it is always good to know
how other students think.
When some students have thought of some creative points that the tutors haven’t
considered, if you can make a good case you can help shape the marking scheme.
It feels a bit like a heart attack because you find out immediately about your answers.
I felt awful because I missed a lot of points in my answers, but I really appreciate this
kind of interactive learning.
I infer that students are generally positive about the immediacy of the teacher feedback,
the opportunity to learn from the thinking of their classmates and the potential to inform
the marking scheme. This last point seems particularly significant in relation to issues, such
as ownership of assessment and the motivational impact this implies. The only somewhat
negative finding is signaled by the last two student quotations above: the potential emo-
tional impact of perceiving you have done badly. This discouraged some students from
attending the feedback session because it came at the start of an examination week and
they were worried that immediate realization of missteps could have a negative impact on
High Educ
123
future performance in other examinations. Students, and sometimes their teachers, may shy
away from critical feedback for fear that it may disrupt emotional equilibrium.
In sum, this vignette exemplifies some of the possibilities in promoting student
engagement with feedback through timely dialogue around performance. A particular
feature of the dialogue is that it comes immediately after the assessment, so students
receive feedback whilst content is fresh in their memory. Perhaps more significantly,
students are involved in articulating their answers, receiving feedback from the tutor and
trying to justify their own particular stance with the possibility of contributing to shaping
the marking scheme. From this I infer that the main benefit of the immediate feedback
session is not just the timing but the way that it engages the students in reflective
discussion.
Discussion
This study explored a model of three interlocking learning-oriented assessment principles
in the practices of a sample of award-winning teachers. The integrated nature of the model
suggests that in any particular case one or more of its principles may emerge as more
prominent and this has been illustrated through three examples in the context of different
disciplines. In the History case, the design and implementation of learning-oriented
assessment tasks was particularly salient, whereas the other two elements were present but
less prominent. In the Architecture case, there was a strong interplay between the portfolio
assessment task, the development of evaluative expertise and engagement with feedback.
In the Law case, the chosen vignette focused on engagement with feedback through timely
discussion of an assessment task involving legal problems, whereas the development of
student evaluative expertise is less explicit or observable.
The assessment task design and implementation in History focused on a series of tasks
which involved students in WTP in the discipline through exploring how history is pre-
sented in a site, such as a museum; the relationship between the past and the present; and
how history relates to students’ own individual lives. Similar to the findings of Anderson
and Hounsell (2007) the design of teaching, learning and assessment activities was crafted
to draw students into performing historical ways of thinking and acting. A contrasting
finding was that Anderson and Hounsell (2007) found that university assessment regula-
tions or established departmental patterns of assessment limited teachers’ freedom to create
assignments that would be congruent with disciplinary practices, whereas this did not occur
in the current study. I suggest that a possible by-product of gaining a teaching award or a
characteristic of some award-winning teachers may be a confidence and determination to
overcome contextual or logistical barriers in a quest to do what they think will impact
positively on their students.
The processes encouraging the development of students’ evaluative expertise have been
highlighted through exploring critical reviews in Architecture. This case provides evidence
from a studio-based discipline to add weight to the conceptual insights of Sadler: activities
such as critical reviews provide students with opportunities to develop a sense of quality
that begins to approximate the connoisseurship of the expert. Through these processes,
students seemed to be learning to be self-critical and to self-evaluate their work pro-
actively. The Architecture case also illustrates Hounsell’s notion of on-display assign-
ments. The public nature of an assessment event, such as a crit, can bring transparency into
assessment by exemplifying the kind of criteria and standards which are being applied; and
provide potential for students to learn from peers’ presentations and related teacher
High Educ
123
feedback. Although there was little peer input during the formal teacher-led crit sessions,
peer interaction was particularly evident in the informal discussions around the displays
during the final review process. The extent to which useful peer feedback arises sponta-
neously or needs to be engineered by teachers within classroom time may vary across
disciplines and across classes. These notions of ‘spontaneous’ and ‘engineered’ peer
feedback may merit further consideration.
I suggested at the outset that an aspect of learning-oriented assessment is to alleviate the
double duty phenomenon by focusing summative assessment as well as formative
assessment on promoting student learning. The Architecture case provides some support
for that position in that the portfolio represented both a final summative product and
formative processes of iteration, feedback and self-evaluation underpinning its develop-
ment. This resonates with the position of Orr and Bloxham (2012) who assert that in Art
and Design students’ development work and final work are assessed in their totality,
challenging the notion of the separation of formative and summative assessment. This
scope for interweaving of teaching, learning and assessment is something teachers in other
contexts or disciplines might strive to emulate.
Student engagement with feedback was particularly a feature of the Architecture and
Law cases. From the Architecture case, I infer two useful principles of effective feedback
processes: they are dialogic rather than uni-directional; and within class rather than end-
loaded. Importantly, the crits acted both as a means of feedback and a forum for developing
student understandings of the nature of quality work. The same day feedback example in
Law focused on engaging students in prompt dialogues around quality legal analysis
immediately after an assessment, despite the constraints of a large class size and limited
time available. This Law vignette also provides a flavor of some key elements of feedback
processes (cf. Evans 2013), such as timeliness; interplay of teacher commentary and stu-
dent expression of views; and the affective impact of feedback.
Conclusion
I have discussed three principles of learning-oriented assessment in relation to a model
which represents what I see as the essence of assessment practice focused on developing
productive student learning processes. The model represents a coherent approach in that it
views assessment tasks, students’ evaluative expertise and engagement with feedback not
as isolated aspects but as parts of an integrated whole. This is particularly well illustrated
by the Architecture case in which there was coherence between the portfolio assessment
task; and the interplay between student self-evaluation and dialogic feedback stimulated by
the processes of critical reviews of students’ designs. Facilitating factors in that case were
the intensive teacher-student interactions and the small size of the tutorial group. The large
classes in History and Law did not, however, represent insurmountable barriers to varied
learning-oriented assessment practices, suggesting that the determination and commitment
of teachers can tackle some of the challenges in developing effective assessment processes.
The model itself and the interplay between its different elements invite further explo-
ration. I end with two related issues for further research. What are the main inter-disci-
plinary commonalities or differences in the principles underpinning the design and
implementation of learning-oriented assessment tasks? Under what circumstances does
feedback open up or close down opportunities for students to develop their own evaluative
expertise?
High Educ
123
Acknowledgments This research is funded by the Research Grants Council of Hong Kong: HKU
740812H.
References
Anderson, C., & Hounsell, D. (2007). Knowledge practices: ‘Doing the subject’ in undergraduate courses.
The Curriculum Journal, 18(4), 463–478.
Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in
Continuing Education, 22(2), 151–167.
Boud, D., & Falchikov, N. (2007). Developing assessment for informing judgment. In D. Boud & N.
Falchikov (Eds.), Rethinking assessment in higher education (pp. 181–197). London: Routledge.
Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design.
Assessment and Evaluation in Higher Education, 38(6), 698–712.
Carless, D. (2007). Learning-oriented assessment: Conceptual basis and practical implications. Innovations
in Education and Teaching International,44(1), 57–66.
Carless, D. (2015). Excellence in University Assessment: Learning from award-winning teaching. London:
Routledge.
Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in
Higher Education,36(4), 395–407.
Evans, C. (2013). Making sense of assessment feedback in higher education. Review of Educational
Research, 83(1), 70–120.
Gibbs, G. (2006). How assessment frames student learning. In C. Bryan & K. Clegg (Eds.), Innovative
assessment in higher education (pp. 23–36). London: Routledge.
Hernandez, R. (2012). Does continuous assessment in higher education support student learning? Higher
Education, 64, 489–502.
Hounsell, D. (2003). Student feedback, learning and development. In M. Slowey & D. Watson (Eds.),
Higher education and the lifecourse (pp. 67–78). Maidenhead: Society for Research into Higher
Education.
James, D. (2014). Investigating the curriculum through assessment practice in higher education: The value
of a ‘learning cultures’ approach. Higher Education, 67, 155–169.
McCune, V., & Hounsell, D. (2005). The development of students’ ways of thinking and practising in three
final-year biology courses. Higher Education, 49, 255–289.
Nicol, D. (2010). From monologue to dialogue: Improving written feedback processes in mass higher
education. Assessment and Evaluation in Higher Education, 35(5), 501–517.
Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and
seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218.
Norton, L., Norton, B., & Shannon, L. (2013). Revitalising assessment design: What is holding new lecturers
back? Higher Education, 66, 233–251.
Orr, S., & Bloxham, S. (2012). Making judgments about students making work: Lecturers’ assessment
practices in art and design. Arts and Humanities in Higher Education, 12(2–3), 234–253.
Prosser, M. (2013). The four-year degree in Hong Kong: An opportunity for quality enhancement. In R.
Land & G. Gordon (Eds.), Enhancing quality in higher education (pp. 201–212). London: Routledge.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science,
18, 119–144.
Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment and
Evaluation in Higher Education, 35(5), 535–550.
Sambell, K., McDowell, L., & Montgomery, C. (2013). Assessment for learning in higher education.
London: Routledge.
Schrand, T., & Eliason, J. (2012). Feedback practices and signature pedagogies: What can liberal arts learn
from the design critique? Teaching in Higher Education, 17(1), 51–62.
High Educ
123
