11. ASSESSING PROBLEM-BASED LEARNING: A CASE STUDY OF A PHYSICS PROBLEM-BASED LEARNING COURSE
- SourceAvailable from: mhc.mcgraw-hill.co.uk
- 2nd edition 01/2003; SRHE and Open University Press.
Article: An experience in learning.Nursing Research 11/1957; 6(2):77-8. · 1.56 Impact Factor
Handbook of Enquiry & Problem Based Learning. Barrett, T., Mac Labhrainn, I., Fallon, H. (Eds). Galway: CELT,
2005. Released under Creative Commons licence. Attribution Non-Commercial 2.0. Some rights reserved.
11. ASSESSING PROBLEM-BASED LEARNING: A CASE
STUDY OF A PHYSICS PROBLEM-BASED LEARNING COURSE
In 2001 the School of Physics in the Dublin Institute of Technology set up the Physics
Education Research Group (PERG) to carry out research to inform curriculum development,
teaching and assessment practices. The group has engaged in a number of research projects
aimed at obtaining a better understanding of how students learn and how educators can help
students learn and develop. In September 2001 the Physics Education Research Group engaged
in collaborative action research in order to design, implement and evaluate a Problem-based
Learning first year physics course (Bowe and Cowan, 2004).
Rationale for Change
Possibly the most significant and externally visible changes made in the move from the use of
predominantly traditional pedagogical approaches to Problem-based Learning were made to the
assessment strategy. The Problem-based Learning team was aware of the important and vital
role assessment plays in driving learning and viewed the assessment strategy as having a
supportive and formative role in helping students develop as learners and as individuals within
the groups. The team saw assessment as not just a way of checking students have learned but
an integral and integrated part of the learning process, as defined by Angelo (1995: 7):
Assessment is an ongoing process aimed at understanding and improving student
learning. It involves making our expectations explicit and public; setting
appropriate criteria and high standards for learning quality; systematically
gathering, analysing, and interpreting evidence to determine how well performance
matches those expectations and standards; and using the resulting information to
document, explain, and improve performance.
The team was also very aware of the role assessment plays in determining the “hidden
curriculum” (Margolis, 2001) and approaches to learning adopted by the students. Previously,
the assessment strategy was comprised primarily of summative tests during the academic year,
in both theory and laboratory practice, and an examination at the end of the year, where the
emphasis was on examining students’ knowledge and understanding of physics principles. It
was felt that this sort of assessment strategy encouraged surface and strategic approaches to
learning. As it was envisaged that the Problem-based Learning approach would encourage
students to adopt a deep approach to learning (Marton et al., 1984), it was important that the
assessment strategy also required students to take this approach. Therefore the assessment
strategy had to be better integrated into the learning process so that the students would see it as
something that is there to help them learn and develop. Hence, more continuous assessment
methods are used and extensive feedback is provided to each student. The assessment strategy
not only examines the products, such as reports, presentations and logbooks, but also examines
each individual’s contribution to the process in such a way as to support each student’s
development and learning.
The development of the new assessment strategy began by first determining the purpose of the
assessment strategy itself. Research in Problem-based Learning and group learning has already
established that students can find it difficult to adapt to these new pedagogical approaches
(Savin-Baden, 2000; Jacques, 2002). Students entering tertiary education may also be unsure
of how to participate in a learning group and may not know how to chair such a process.
Students may also feel working in groups is unfair in terms of workload and the distribution of
tasks. The new assessment strategy was developed to take all these factors into account and to
be an integrated part of the learning process and not just something extra the students must
complete. In addition to the aims of the original assessment strategy, this new assessment
strategy aimed to:
Examine conceptual understanding and problem-solving skills
Encourage and reward individual contribution to the group process
Support and evaluate the development of group, communication and presentation
Identify problems and areas of potential improvement
Previously, the assessment strategy would have been aligned with learning outcomes associated
solely with knowledge and understanding of physics concepts and principles, and laboratory
skills. In order to achieve the aims of the new assessment strategy it was necessary to introduce
learning outcomes associated with conceptual understanding, problem-solving, group processes
and key skills. On successful completion of this course the learner will be able to:
Identify and analyse potential solutions to the mechanics problem
Deal with omitted information and use a proper technique to solve open-ended
Work effectively in, and lead a peer group
Communicate effectively through oral presentations and written reports
Research a topic and retrieve information
Once the learning outcomes had been devised it was important to choose appropriate
assessment methods that would ensure the achievement of each learning outcome could be
determined. An alignment matrix (Biggs, 1999; Cowan, 2002) was used to ensure the learning
outcomes, assessment and the facilitation of learning matched each other in a pedagogically
sound manner (Bowe and Cowan, 2004). In terms of the assessment strategy, this meant
choosing appropriate assessment methods and devising suitable assessment criteria.
Once the purpose of the assessment strategy had been determined the next step was to decide
on what was to be assessed. Each problem would involve the students working together in
groups, with and without a tutor present, and then presenting their solution either orally or by a
written report. Some of the problems also involved laboratory work. Therefore it was decided
to assess using the following methods:
Individual contribution to the group process
Chairing of the group process
o Experiment proposal
End-of-year open-book examination
Devising assessment criteria and assessing “products” such as the oral presentations, written
reports, proposals and logbooks did not pose any significant problems as these are standard
assessment methods used in many courses. However, the fact that these “products” were
completed and submitted by groups and not individuals did raise a number of important issues.
It was felt that each individual’s contribution to the “product” should be assessed, particularly
as the students were in first year and it was felt there would be problems associated with some
individuals doing much more work than others. Also, there was a concern about awarding
marks to individuals who may not have contributed anything to the “product”.
A far greater challenge arose from the aspiration to assess individual contribution to the group
process. It was felt that the assessment strategy should reward those students who work hard in
the group process and endeavour to contribute constructively to the process. In this way the
strategy should also penalise those students who do not make an effort to contribute to the
group process. For the purpose of formative, developmental assessment, it was felt that the
feedback should be individualised to help each learner and this required assessing each
student’s contribution to both the process and the products. To overcome some of the problems
associated with assessing individual contribution to both the process and the product, it was
necessary to involve the students in the assessment process through the use of self and peer
Another reason for involving the students stemmed from the fact that in the Problem-based
Learning course each tutor roamed from group to group, and hence was not present at all times
while the group were working together. Therefore the tutor’s assessment alone may not be
reliable or reflect any individual’s true contribution. As the students themselves are in the
group throughout the process, they can observe contributions with and without a tutor present.
Along with the pragmatic reasons for involving the students there are also pedagogical benefits
as self-assessment helps students develop the ability to be realistic judges of their own
performance and to monitor their own learning. These skills are needed to become self-
directed and lifelong learners (Candy et al, 1994). Self-assessment therefore allows the student
to develop metacognitive skills - "the capacity to learn, the capacity to know how to learn,
[and] the capacity to know what he has learned" (Heron, 1988: 78). states that self assessment
involves two key elements:
The development of knowledge and an appreciation of the standards and criteria for
meeting those standards
The capacity to make judgements about whether or not the work meets those
Developing these self-assessment skills is worthwhile and beneficial but it is also a slow
process that must be facilitated by the tutor. It requires the students to engage in dialogue
regarding the assessment criteria and the quality and standard of their work and contributions.
However, in order for this dialogue to take place students need to able to understand the
concepts of quality held by those making the judgements (Sadler, 1989). This can be achieved
through practice making judgements, and having the responsibility to make these judgements,
against the criteria. Inherent in the process is the opportunity for the students to discuss,
debate, and negotiate these assessment criteria. The use of self-assessment in the Problem-
based Learning course requires students to reflect on their experiences in the group process and
make judgements on their contributions not only to the “product” but also to the process.
McGill and Brockbank (1998) state the fact that reflection, which draws on both the cognitive
and meta-cognitive levels is a great strength and an assessment system which addresses
reflective learning should do so in terms of both outcome and process.
Devising the criteria to assess the chairperson and each individual’s contribution to the group
process was very problematic and many different models were piloted. After four years, and
through progressive reflection and evaluation, a model has emerged that works very well from
both the students’ and tutors’ perspective. As the assessments are primarily developmental,
providing timely and effective feedback in the learning process is imperative. The assessment
of the process has to be clear and transparent to the students and relative easy for the tutors to
facilitate. The feedback had to be directly related to the criteria so the students can use the
same criteria to assess themselves. It was decided that the criteria the tutors would look for
when assessing the process (contribution and chair) would be a set of actions that demonstrate
the students are constructively contributing to the process. Table 1 shows the “criteria” used to
assess individual contribution to the group process, as a list of actions under five headings.
It should be noted that the rationale for each assessment method, along with the criteria, are
discussed with the students in the induction process as well as at various intervals throughout
the academic year. The level of contribution is assessed by the extent to which each student
displays the attributes and actions in Table 1. However it must be noted that a student it not
expected to complete the entire list of actions. In practice the tutors tend to look for extremes,
that is, actions that are being completed properly, as well as uncompleted actions, which have
negative effects on the group process.
Figure 1: Assessment of Contribution to the Group Process
Assessment of Contribution to the Group Process
Category and Description
Working towards Understanding
It is each member's responsibility to strive towards a
complete understanding of the physics involved in each
problem. It is not sufficient to just sit back and listen in
the hope of learning something later but you must be
actively engaged in the process and trying to understand
ensuring you understand the other
group members by asking questions
stating what you understand to be
summarising the groups' position
looking for mistakes in the process,
thinking and calculations
persuading and defending a position
asking each other questions to ensure
asking others to
asking for justification
Completing tasks on time
Reporting back to the group
Working towards Group Understanding
One of the aims of the process is that by the end of a
problem the group has achieved the same level of
It is each group member's responsibility to complete this
task to the best of their ability and report back to the
In many cases some of the group members will have a
greater prior knowledge of the subject matter. In this
situation it is their responsibility to help the other
students learn by explaining and teaching the physics
involved. In this way the students can learn from each
other and also by teaching the subject the students with
prior knowledge can identify any holes in their
Assisting Group Focus
It is each members responsibility to help keep the group
focused on the problem and to maintain a good group
explaining your understanding
supporting opinions of others
showing openness and acceptance of
helping to analyse and reconcile
commenting on group process, giving
recognition to others
The role of the chair is to manage the group as they work towards a solution. The chair is not
expected to contribute to the content of the solution but to ensure the group members work well
together. Table 2 shows the “criteria” used to assess the chair in the group process, as a list of
actions under four headings.
Figure 2: Assessment of the Chair
Assessment of Contribution to the Group Process
Category and Description
Time management / scheduling / structure
It is important that the time in class is used efficiently
and the group schedule tasks and structure the process to
enable them to complete the problem in the time
allowed. It is the responsibility of the chair to ensure the
group is aware of the time constraints and to schedule
and structure tasks.
Equal opportunity and participation
setting deadlines and milestones
monitoring the time
planning the session
delegation of tasks
It is the chair's responsibility to ensure every member of
the group is given an equal opportunity to contribute and
participate. Everyone's opinion is valid and must be
considered and discussed.
It is the chair's responsibility to ensure the group stays
focused and remains concentrated on the problem.
If the group feel they cannot continue without the tutor
intervening and they have tried all alternative avenues,
they can then ask the tutor focused questions relating to
the content. However the tutor can respond with more
questions in an attempt to get the group thinking about
the problem from different perspectives.
asking questions of all group members
ensuring everyone is included in the
ensuring everyone has the opportunity
asking group members questions to
ensure they are following the solution
periodic summarising of learning
organising the group to ask the tutor
relevant and pertinent questions
The students work in groups on approximately one problem per week. For each problem the
students are awarded individual marks comprised from the assessment of both the process and
product - report or presentation. Table 3 shows the continuous assessment methods used
throughout the academic year.
Figure 3: Continuous Assessment Strategy
Problems 1 - 12 Problems 13 –24
Group Learning Process
At the start of the academic year, when the students are first introduced to Problem-based
Learning, each student’s individual contribution to the group and learning process is assessed
by a tutor based on the criteria in Table 1. Each student is awarded a mark out of ten and given
constructive developmental feedback. In each group, one student acts as the chair, with the role
rotated after each problem is completed. The chair is also assessed, but based on different
criteria, as shown in Table 2, as they are only expected to manage the group process. This
assessment of individual contribution and the chair is primarily formative, aimed at helping the
students develop an awareness of their roles within the group and what it means to contribute
constructively to the group process. However these marks do contribute to each student’s
overall summative grade.
At the end of each problem, each group submits a written report that is assessed as a “single
product” and awarded a mark based on standard assessment criteria, which is then given to
each student within that group. For some of the problems, each group gives a twenty minute
presentation that is also assessed as a “single product” and awarded a mark, which is then given
to each student within that group.
The relative weighting between the process and product marks changes as the course
progresses. At the start of the course when the students are first introduced to Problem-based
Learning, the process mark (contribution or chair) has a weighting of 80%, with 20% for the
report or presentation. This is to encourage the students to concentrate on the group process
and develop their abilities to work in, and chair, a group. After a number of problems, and
when each student has chaired the group process, the process weighting is reduced
progressively until both the process and product are equally weighted.
After about twelve problems the students have become very aware of their roles and of the
expectations the tutors have of them as individuals and as group members. Collaborative
assessment is introduced about halfway through the academic year after the students have
participated in a workshop where the rationale and objectives of self-assessment are explained,
followed by a negotiation of the assessment criteria. From this point on, after each problem,
each student is required to self-assess their own contribution to the group process and award a
reasoned mark, provide a justification for that mark, explain where they lost marks and describe
what they will do differently in the next problem session. The students’ self-assessments are in
effect, reflective journals which the students return to after each problem. The tutors also
continue to award a mark with feedback, and the average of these two marks goes towards the
summative assessment. This same process is also used to assess the chair.
From about halfway through the academic year, upon completion of a problem, each group still
produces a report or gives a presentation that is assessed as a “single product” and awarded a
mark based on provided assessment criteria. However, in the reports and presentation each
student is required to describe their contribution and state what percentage of the mark they feel
they deserve. Complete justification must be provided with examples given of the level of
This continuous assessment and feedback process is designed to assist student learning and
promote deep learning. To augment this process, a WebCT on-line learning resource centre
was developed, which includes online tutorials, assignments, quizzes, individual students’
feedback pages, calendar, notice board and details of the laboratory project programme (Bowe,
2001). The feedback from both the formative and summative assessments is provided through
the WebCT site. The students are also required to complete regular online multiple-choice
quizzes as part of the overall continuous assessment.
There is also an end of year examination that is open book, which involves the testing of the
students’ abilities to problem-solve, as well as their understanding of the physics concepts.
The evaluation of the Problem-based Learning course (Bowe and Cowan, 2004) highlighted the
important and vital role the assessment strategy played in success of the course. The
assessment strategy was seen by the students as supportive and helpful in terms of their
development as members of a learning group. As the students developed their group and
communication skills the groups themselves worked better together thus improving and
enhancing the learning. It also highlighted that fact that students need time to adapt to group
learning and that this process can be supported through the use of an appropriate assessment
strategy. It was also evident from the evaluation that the rate of development of group skills
was greatly improved when the current assessment strategy was introduced. This development
was further improved when self and peer assessment was introduced.
I wish to thank the School of Physics staff, who were involved in the development and
implementation of the Problem-based Learning course; Dr. Siobhan Daly, Dr. Robert Howard
and Dr. Cathal Flynn, and also the School management for supporting the development of the
Problem-based Learning course.
Angelo, T. A. (1995) Reassessing (and Defining) Assessment, AAHE Bulletin, 48, 3, 7.
Biggs, J. (1999) Teaching for Quality Learning at University. Milton Keynes: Open University
Bowe, B. and Cowan, J. (2004) “Physics: A Lecture-based Course and a Problem-based
Course: A Comparative Evaluation of Problem-based Learning,” in M. Savin-Baden, M.
and K. Wilkie (eds.) Challenging Research into Problem-based Learning. Buckingham:
SRHE/Open University Press.
Bowe, B. (2001) “Developing and Using an Online Resource to Support Students Learning,”
paper presented to Dublin Institute of Technology e-learning innovations conference, 22nd
Boud, D. (1988) Developing Student Autonomy in Learning. 2nd ed. London: Kogan Page.
Brockbank, A. and McGill, I. (1998) Facilitating Reflective Learning in Higher Education.
Buckingham: Open University Press.
Candy, P. C, Crebert, G. and O'Leary, J. (1994). Developing Lifelong Learners Through
Undergraduate Education. Commissioned Report No. 28. Canberra: National Board of
Employment, Education and training.
Cowan, J. (2002) “What Makes a Successful Education Innovation?” Paper presented at
Teaching Physics at First Year Level Conference, Dublin Institute of Technology, 2nd May,
Jaques, D. (2000) Learning in Groups, 3rd ed. London : Kogan Page Margolis, E. (ed.) (2001)
The hidden curriculum in higher education. New York: Routledge.
Marton, F., Hounsell, D. and Entwistle, N. (1984) The Experience of Learning. Edinburgh,
Scottish Academic Press.
Savin-Baden, M. (2000) Problem-based Learning in Higher Education: Untold Stories.
Buckingham: SRHE/Open University Press
Sadler, R. (1989) “Formative Assessment and the Design of Instructional Systems.”
Instructional Science, 18, 119–44.