Deepening computer programming skills by using web-based peer assessment

Abstract

Peer assessment is a method of motivating students, involving students marking and providing feedback on other students' work. This paper reports on the design and implementation of a novel web-based peer assessment system for computer programming courses, and discusses its deployment on a large programming module. The results indicate that this peer assessment system has successfully helped students to develop their understanding of computer programming.

Full text

Page 1

152
DEEPENING COMPUTER PROGRAMMING SKILLS BY USING
WEB-BASED PEER ASSESSMENT

Jirarat Sitthiworachart
Department of Computer Science
University of Warwick
Coventry CV4 7AL, UK
jirarat@dcs.warwick.ac.uk

ABSTRACT
Peer assessment is a method of motivating
students, involving students marking and providing
feedback on other students’ work. This paper
reports on the design and implementation of a
novel web-based peer assessment system for
computer programming courses, and discusses its
deployment on a large programming module. The
results indicate that this peer assessment system
has successfully helped students to develop their
understanding of computer programming.
Mike Joy
Department of Computer Science
University of Warwick
Coventry CV4 7AL, UK
M.S.Joy@warwick.ac.uk
Keywords
Peer assessment, deep learning, web-based
assessment, computer programming.
1. INTRODUCTION
Assessment is a tool for learning, but traditional
assessment methods often encourage surface
learning, characterised by memorisation and
comprehension of information. Deep learning,
such as creating new ideas, and critical judgement
of a student’s work, can be encouraged by the use
of peer assessment [1,2,10]. When students
evaluate each others’ work they think more deeply,
see how others tackle problems, learn to criticise
constructively, and display important cognitive
skills such as critical thinking [3,4]. As part of a
study investigating the
assessment can promote deep learning in a
programming course, we have developed a novel
web-based peer assessment
advantages over ordinary peer assessment
because students can be more critical (due to
anonymity the system provided), the processing of
the marks is automated, and the lecturer can
easily monitor the marking. We describe the tool
Permission to make digital or hard copies of all or part of this
work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or
commercial advantage and that copies bear this notice and the
full citation on the first page. To copy otherwise, to republish, to
post on servers or to redistribute to lists, requires prior specific
permission.
extent that peer
tool. It has
and the peer assessment process it supports, and
report on its deployment on a large computer
programming course
2. WHAT IS PEER ASSESSMENT?
Falchikov [5] defines peer assessment as “the
process whereby groups rate their peers”.
Somervell [6] states that peer assessment
engages students in making judgements on the
other students’ work. In the peer assessment
process, students are involved both in the learning
and in the assessment process. Peer assessment
is primarily a tool for learning rather than for
summative assessment [7]. Dochy and McDowell
[8] remark that “peer assessment is not only a tool
to provide a peer with constructive feedback which
is understood by the peer. Above all, peer
assessment is a tool for the learner himself.”
Receiving many and frequent peer feedbacks can
prevent some errors and provide hints for making
progress in learning [9]. Thus the peer assessment
process provides many benefits to students,
including the following:
• encouragement of students’ deep learning
skills in programming by making judgements
and providing feedback on other student’s
work [2,10];
• students have opportunities to compare and
discuss about what constituted a good or bad
piece of work, which help them to improve
their programming style and think more deeply
about the quality of work [12];
• when marking, students realise mistakes that
they had made in their own answers - the
more marking students did, the better their
own results became[10];
• development of self-assessment and reflective
learning [2,6]; and
• deepening of students’ understanding of the
assessment process [7].
4th Annual LTSN-ICS Conference, NUI Galway
© 2003 LTSN Centre for Information and Computer Sciences

Page 2

153
3. PEER ASSESSMENT EXERCISE
The peer assessment experiment was performed
on 215 first year undergraduate students (189
male and 26 female) of whom 153 students’ first
language is English and 62 students who are not
native English speakers. The age range of the
students was 18-26, with most in the 18-20 band.
The UNIX programming module in the Computer
Science department at the University of Warwick
was chosen for this investigation. This module
aims to give students a basic understanding of the
UNIX operating system, and competence in
programming using a UNIX shell. Students learn
how to design and develop programs in the shell,
which is a programming language that allows
programs to be written in many styles. There are
three programming assignments in this module,
which students submit via the department’s
“BOSS” online submission system [11]. The
second of the three assignments was marked
using a peer assessment process. The purposes
of performing the experiment in peer assessment
were:
• to investigate the extent that peer assessment
in a programming course promotes deep
learning;
• to assess the accuracy
judgements during
exercise; and
• to provide evidence that peer assessment in
computer programming
pedagogical effect.
of students’
assessment a peer
has a positive
3.1 Process
This peer assessment exercise was divided into
three separate stages, as shown in Figure 1. Test I
and test II were provided in order to measure the
students’ evaluation skills, before and after the
peer assessment. Students analyse and evaluate
short example shell programs in test I and test II,
which are similar in content but cosmetically
different.
Stage I: Students do the assignment in their
own time. Then they submit the
assignment via the online submission
system. Ten automatic tests are then
run on the submitted programs.
Stage II: Students were divided into small
groups (three students per group),
each group consisting of students with
a range of abilities. Each student was
assigned three
assignments to mark during the first
half hour of a lab session. Then they
discussed their marking with the other
students in their group, who marked the
same assignments.
other students’
Stage III: In their own time, each student marked
the quality of three markers’ marking.
This additional stage aims to make
students take marking more seriously
during the previous stage.

Figure 1 Peer assessment process
3.2 Mark scheme
The marking scheme is illustrated in Figure 2,
using the following definitions.
Automatic test: The online submission system
tests a student’s assignment
against different inputs to check
whether it functions correctly.
Ten tests are used.
Marker: Student
assignments.
Feedback marker: Student feedback marker
reports on the quality of the
marking given by the three
markers.
Script: Assignment
submit
submission system.
In this peer assessment process, 50% of the
marks are awarded by the teacher (automatic
tests) and the remaining 50% are awarded by the
students (peer assessment):
• Automatic Test
• Peer Assessment
- Part I: mark assignment
- Part II: mark quality of marking
marker marks
that students
online via the
50%
30%
20%

Page 3

154
Peer marks are based on three markers; the
average of the three marks is calculated. If one of
markers does not appear to have marked work
seriously (that is, does not appear to have read the
program carefully and answer the marking criteria
properly), the mark he or she gives will not be
included in the average and the other marks will
be scaled. The marking of assignments by
students is possible since they are given guidance,
automatic test scores and results, a marking
scheme, and well-explained marking criteria.

Figure 2 Peer assessment mark scheme
4. DESIGN OF WEB-BASED PEER
ASSESSMENT

Figure 3 Architecture of the web-based peer
assessment system
The web-based peer assessment software uses
the standard combination of Apache web server,
the PHP4 programming language, and a MySQL
database running on a Linux platform. This
architecture is illustrated in Figure 3. Dynamic web
pages are written in PHP4 and static web pages
are written in HTML.
This web-based peer assessment provides
anonymity for all users. Students are allowed to
revise the marks they give until the marking
deadline is reached. They receive a username and
password by email before starting the peer
assessment exercise. After students login, the
menu page displays three steps for students to
follow (i.e. mark assignment, mark quality of
marking, and see mark). They can see the scripts
that they have been assigned to mark easily by
clicking on the script buttons (Figure 4). They can
view the automatic test results by clicking on the
link on each script page to open a popup window
displaying the results. A “Things to consider” link
is provided below containing marking guidance.

Figure 4 Assignment script on Mark web page
4.1 Step I: Mark assignment
In this visual inspection step, students mark and
provide feedback on other students’ assignments
by answering nine questions about:
• Readability (comments, indentation, variable
names);
• Correctness (correct output, appropriate error
handling, correct exit status); and

Page 4

155
• Style (easy to follow, well structured, use of
appropriate utilities).
These are answered for each script by selecting
simple multiple choices, i.e. No, Partial, and Yes.
The default answer is set as unmarked. Students
give a comment for each group of three questions.
An explanation of the marking criteria is provided
for each group of questions by clicking on the links
on the left.
4.2 Step II: Mark quality of marking
In this step, students mark the quality of marking
given by each of the three markers on a particular
script. They need to answer three questions about
whether the suggestions the markers gave in each
section (readability, correctness, and style) are
relevant, well explained and useful to students.
The marking given by the three markers is
displayed at the top of the page and the student
enters the feedback marks at the bottom.
4.3 Step III: See mark
In this final step, students can see their mark from
both the automatic test and the peer assessment.
A Marking calculations link at the bottom of the
page provides an explanation of how the overall
mark is calculated. If the students do not mark any
of three scripts, they may lose some marks. The
full mark and comments that the three peer
markers gave the student’s assignment are also
available. This also includes the full mark that they
were given based on the quality of their own
marking.
4.4 Monitor marking
The monitor marking web page reports the
students’ marks and any absent markers, and is
only available for tutors. The highlighted columns
show the standard deviation of the three markers
for both Step I and Step II in order to know how
spread out the marks are. If the standard deviation
is less than a preset value, it is acceptable, but if
the standard deviation is more than a given upper
limit, it means the marks from the three markers
have a very wide range, and the tutor may have to
reconsider the marks for that student. The tutor
can access each script by using the ‘Script ID’ box
at the top of the web page.
5. RESULTS
At the end of the process, each student was
required to fill in a detailed online questionnaire,
and the following preliminary observations suggest
that the exercise has been beneficial.
• 69% of students realise mistakes that they
made in their own answer when marking other
students’ work.
• 76% of students discuss with their groups
when marking and think this discussion helps
them understand more about the assignment.
A few students find starting a discussion
difficult.
• 58% of students feel comfortable when
assigning marks. A few students did not fully
understand the marking criteria.
• 65% of students are satisfied with their mark
from the peer assessment, and considered
that the peer feedback they received was
relevant and useful.
• 80% of students agree that seeing good and
bad programs help
programming, and marking helps them to think
more deeply about their own work.
For most of the questions in the questionnaire, the
responses when compared by gender were
broadly similar. However, male students appeared
to feel significantly more comfortable assigning
marks than did the female students. Almost 80% of
non-English students would like to recommend the
peer assessment to friends as a way of learning
more, but only 50% of English students would like
to do this, and this may form the basis for further
investigation.
them in learning
6. DISCUSSION





Figure 5 Deep learning approach

Peer assessment encourages students to develop
higher level cognitive skills in learning computer
programming as follows:
• Analyse aspects of
assessment starts students thinking about
what aspects of their programs were wrong
and what worked well when they compare
their work with other students’ work.
• Evaluate whole programs - making judgement
on other students’ work helps student to
evaluate their own programs. They can
recognise what constitutes good and poor
programs.
• Synthesize better programs - students analyse
and evaluate the different styles of solving
programming problems, which help them to
create better programs.
The students’ responses generally appear to
support our view that students can learn from each
programs - peer
Analyse
Evaluate
whole programs
better programs
aspects of programs
Synthesize
PEER
ASSESSMENT

Page 5

156
other through this process. Seeing different ways
of solving programming problems and marking
each other’s works
assessment and writing better programs:
“I got the chance to observe two scripts that
used different methods than my own solution to
satisfy the specification. In order to be confident
in my ability to mark these scripts fairly, I had to
spend a long time studying them and hence
acquiring an improved knowledge of how shell
scripts are composed.”
“Marking others’ work helps me criticise my
own work and remind me of my own problems.”
Most students seemed satisfied with their marks,
and consider that with adequate guidance the
marks from peers could be as reliable as the
marks from a tutor. However, some students think
the marks awarded by students are graded using
different standards to those awarded by tutors.
Markers could only base marks on how good the
script was compared to their own answers, and
some students therefore think no student is really
qualified to mark another student’s work, as they
are not trained for marking and are not experts:
“I would say peer assessment in a better way to
learn how to write
Nevertheless, some markers may not have the
skill of marking and understanding of script.”
“I think it is hard to mark a student when you’ve
never marked assignments before, especially
UNIX scripts as I never had any previous
experience with it. Additionally, the things I
consider good or bad may not be the same for
other people.”
It was difficult in the students’ view to avoid
friendship marking, resulting in over-marking (they
often felt favourable towards their friends) [12]:
“When marking I was inclined to be generous
because I expected that everyone else would
be to me.”
“Very hard to do when you are marking
someone who is technically in the same boat
as you.”
helps students’ self-
a good program.
7. CONCLUSIONS
We have described a peer assessment process,
together with supporting web-based software,
which we have used to test the effectiveness of
peer assessment in
languages. The process we have used is novel,
since students are engaged not only in marking
each other’s work, but also in evaluating the
quality of marking of their peers. Students use
deep (analysis, evaluation and synthesis) rather
than surface learning strategies in the peer
assessment process, contributes to effective
learning. Students also realise their own strengths
learning programming
and weaknesses when marking a good or poor
piece of work. Preliminary evaluation of the
exercise indicates that it has contributed positively
to the students’ learning experience.
8. REFERENCES

[1] AAHE Bulletin, “Deep Learning, Surface
Learning”, 45(8), 10-13, 1993.
[2] Topping, K., “Peer assessment between
students in colleges and universities”, Review
of Educational Research, 68, 249-276, 1998
[3] Sluijsmans, D., Dochy, F., and Moerkerke, G.,
“Creating a learning environment by using self-
peer- and co-assessment”,
Environments Research, 1, 293-319, 1999
[4] Tsai, C., Lin, S., and Yuan, S., “Developing
science activities through a net worked peer
assessment system”, Computers & Education,
38, 241-252, 2002
[5] Falchikov, N., Learning Together: Peer
tutoring in higher education, Routledge
Falmer, London, 2001
[6] Somervell, H., “Issues
enterprise and higher education: the case for
self-, peer and collaborative assessment”,
Assessment and
Education, 18, 221-233, 1993
[7] Brown, G., Bull, J., and Pendlebury, M.,
Assessing student
education, Routledge, London, 170-184, 1997
[8] Dochy, F., McDowell, L., “Assessments as a
tool for learning”, Studies in Educational
Evaluation, 23(4), 279-298, 1997
[9] Lin, S., Liu, E., and Yuan, S., “Web-based
peer assessment: feedback for students with
various thinking-styles”, Journal of Computer
Assisted Learning, 17, 420-432, 2001
[10] Bhalerao, A. and Ward, A., “Towards
electronically assisted peer assessment: a
case study”, Association
Technology journal (ALT-J), 9(1), 26-37, 2001
[11] Joy, M., Luck, M., “The BOSS System for On-
line Submission and Assessment”, CTI
Computing: Monitor, 10, 1998
[12] Brindley, C., Scoffield, S., “Peer Assessment
in Undergraduate Programmes”, Teaching in
Higher Education, 3(1), 79-89, 1998
Learning
in assessment,
Evaluation in Higher
learning in higher
for Learning