PBL in teaching computing: An overview of the last 15 years

Abstract

In computing courses, the teaching and learning approach normally emphasizes theoretical knowledge at the expense of practical knowledge. The major disadvantages of this approach are learners' lack of motivation during class and their quickly forgetting the knowledge they have acquired. With a view to overcoming these difficulties, Problem Based Learning (PBL), an institutional method of teaching, has been applied to teaching computing disciplines. Despite the growth of the practice of PBL in various disciplines of Computing, there is little evidence of its specific characteristics in this area, the effectiveness of different PBL methodological approaches, or of benefits and challenges encountered. In this context, this paper presents a systematic mapping study in order to identify studies which involve best practices when using the PBL method in Computing between 1997 and 2011, answering five research questions: “What are the main characteristics of PBL that support teaching in Computing?”; “What are the criteria for applying PBL effectively in this area?”; “How is the PBL methodology applied?”, “What are the advantages and benefits of applying PBL in Computing?” and, finally, “What are the main challenges about learning in PBL in Computing?”.

Full text

PBL in Teaching Computing:
An overview of the Last 15 Years
Armanda Maria C. Amorim Oliveira
CIn – Centro de Informática
UFPE – Universidade Federal de Pernambuco
Recife, Brasil
amcao@cin.ufpe.br
Simone C. dos Santos, Vinicius Cardoso Garcia
CIn – Centro de Informática
UFPE – Universidade Federal de Pernambuco
Recife, Brasil
scs@cin.ufpe.br, vcg@cin.ufpe.br
Abstract— In computing courses, the teaching and learning
approach normally emphasizes theoretical knowledge at
the expense of practical knowledge. The major
disadvantages of this approach are learners´ lack of
motivation during class and their quickly forgetting the
knowledge they have acquired. With a view to overcoming
these difficulties, Problem Based Learning (PBL), an
institutional method of teaching, has been applied to
teaching computing disciplines. Despite the growth of the
practice of PBL in various disciplines of Computing, there
is little evidence of its specific characteristics in this area,
the effectiveness of different PBL methodological
approaches, or of benefits and challenges encountered. In
this context, this paper presents a systematic mapping
study in order to identify studies which involve best
practices when using the PBL method in Computing
between 1997 and 2011, answering five research questions:
"What are the main characteristics of PBL that support
teaching in Computing?"; "What are the criteria for
applying PBL effectively in this area?"; "How is the PBL
methodology applied?", "What are the advantages and
benefits of applying PBL in Computing? "and, finally, "
What are the main challenges about learning in PBL in
Computing?".
Keywords-component; Problem Based Learning; Education;
Computing.
I.
I
NTRODUCTION
In the evolution of the Information Technology and
Communication (ICT) industry, professionals in the field of
Computer face constant changes which leads to there being a
need to adapt concepts, methods and techniques for specific
situations that may arise in the labor market. These issues
affect not only businesses, but also software engineers, who
besides having as a prerequisite vast knowledge of software
development tools, should also have a broad view of the
problem to be solved, ie, an understanding of business,
entrepreneurialism and interpersonal skills [1].
To succeed in the field of Computer Science, one needs to
have knowledge of and to master a variety of skills such as:
being at ease with mathematics, logic, problem solving,
algorithmic thinking and programming. Unfortunately, many
students struggle to develop these skills, especially when the
subject is related to object-oriented programming, discrete
mathematics, data structures and analysis of algorithms [2].
This happens because of the way in which teaching is
approached in undergraduate courses, since what can be seen is
a lack of practice after being given the theory [3]. Practice
along with theory enables students to make connections, and
thus to link prior knowledge to new knowledge learned during
the new learning process [4, 5].
As a consequence of this factor, what also needs to be
mentioned is that teachers get greatly frustrated and students
abandon their courses, as many give up working in the area due
to such difficulties [2]. When students do not give up, there is
poor professional training, as they do not have enough practice
for the labor market. [6]
With a view to finding solutions for these problems, the
instructional method of teaching, Problem Based Learning
(PBL), has been applied to the teaching of computing, aiming
to promote collaborative and motivating learning, based on
problem solving. According to Peterson [7], PBL meets three
important criteria that promote ideal learning: first, it provides
an environment where students are immersed in a practical
activity, secondly, they can receive guidance and support both
from other students and a responsible teacher/ tutor; and,
finally, learning is based on solving a real problem.
Despite the growth of the practice of PBL in various
disciplines of Computing, there is little evidence of its specific
characteristics in this area, of the effectiveness of different
methodological PBL approaches, or of the benefits and
challenges encountered. It was out of this context, the problem
tackled by this research arose which sought to understand the
studies that highlight best practices for using the PBL method
in disciplines related to the field of computing. To this end, the
method of systematic mapping was used and focused on
constructing a broad overview of how the PBL method was
applied in the everyday routines of these courses from 1997 to
2011.
II. P
ROBLEM
B
ASED
L
EARNING
I
N
T
EACHING
C
OMPUTING
S
CIENCE
Problem Based Learning (PBL) is an instructional method
of teaching that differs from the traditional model by virtue of
its using real-life practical problems to start the learning
process in order to encourage the development of problem-
solving skills. These problems based on reality are worked on
in groups, in order to initiate, direct, motivate and focus
learning, unlike traditional methods that put the problem at the
end of the presentation of content [8, 9].
This approach began in the medical field, in the 1970s, at
McMaster University's Medical School [10], the precursor

being the physician and educator Howard Barrows who
developed methods to instruct doctors to develop their own
capacities for reflection when not in medical school [11]. The
goal of this method is not only the solution of the problem, but
rather learning from the problem presented, ie reporting
solutions based on the problem and the process used to obtain
them. Barrows [12] describes six key characteristics of PBL:
• Learning is student centered;
• Learning takes place in small groups of students;
• There is the presence of a tutor who is seen as a
facilitator or guide;
• The problems are presented at the start of the process.
• The problems encountered are used as instruments to
attain the knowledge and skills to solve problems;
• New information is acquired through self-directed
learning.
In the process of teaching, teachers must understand
correctly the problem as being the basis in the learning method,
which differs from the traditional model of education, which is
teacher-centered. PBL directs the role of learning to the
students and the teacher's role changes to a single form of
teaching: that of guiding, as well as that of teaching. [6]
In Computing Science, the difficulty lies in having good
professionals in the area of information technology, viz.,
software engineers. PBL tries to overcome this difficulty by
offering students a way to acquire knowledge and develop the
skills and attitudes expected of a professional.
Software engineers face many problems of adapting
concepts, methods and techniques to the specific situations of
problems in the market, in general influenced by the variation
in scope, cost and deadlines of software projects. For the
professional training of software engineers, PBL approaches
can help by fostering the ability to work in teams, to solve
problems and also to encourage the development of one’s skills
and attitudes, such as self-directed learning skills, cooperation,
ethics and respect for people’s other points of view. [3]
Authors like Peng [6] discuss problems with reference to
textbooks on programming, and state that part of the books are
written with the main focus on the logical structure of
knowledge, starting with the introduction of an abstract and
unclear concept, for example, the concepts of "object",
"property", "event" or "method", and thus do not respect the
students’ cognitive rules.
Bearing in mind the benefits of the PBL method, in
different areas of knowledge, it is appropriate to make a more
in-depth study with the aim of supplying a broad overview of
research studies that that have been undertaken in the area of
Computing Science and of analyzing gaps that may yet need to
be explored in research on PBL
III.
S
YSTEMATIC
M
APPING
The study of Systematic Mapping (SM) is a method
designed to provide a broad overview of a particular area of
research that enables the results to be identified, quantified and
analyzed, thereby establishing evidence on a particular topic
[13, 14].
The Systematic Mapping conducted in this study was
divided into three stages, based on the guide by Kitchenham
[13], namely:
1. Planning Systematic Mapping: drawing up a protocol
containing all the information of the study;
2. Driving the Systematic Mapping: carrying out searches
and collecting data;
3. Presenting Systematic Mapping: analysis of results and
writing a report from the information stored.
The initial research studies were conducted using digital
libraries: IEEExplore Digital Library, ACM Digital Library,
Elsevier Scopus, Elsevier ScienceDirect. The selection of these
sources was based on the relevance and credibility of articles
indexed in the area of Computer Science.
Starting out from the objective of answering the central
question of the study: "How is learning based on PBL
characterized and what is its contribution and challenges for
teaching Computing Science?", five secondary issues were put
forward for answer:
1.What are the main characteristics of PBL that were found
and supporting education in Computer Science?
2. What are the criteria for the effective application of PBL
in Computer Science?
3. How is the PBL methodology applied in the teaching of
Computer Science?
4. What are the advantages and benefits of applying PBL in
Computer Science?
5. What are the main challenges about learning in PBL in
Computer Science?
For the construction of the key terms of the research, the
key-words of the central question were used. In addition to the
key-words, their synonyms were identified based on the area of
research, which facilitated the search: Learning (learner,
education, educational program, constructivism, constructivist,
instruction); Problem Based Learning (PBL, problem-based
learned); Contribution (advantages); Challenges (outcome,
results); Computer (software, computer engineering,
information technology, computer science).
The search string was constructed by combining key-
words and synonyms. Two operators were, OR (or) between
the synonyms and AND (e) between the key-words, as shown
in Table I
TABLE I. S
EARCH
S
TRING
.
Search String
(learning OR learner OR education OR methodology OR approaches OR
"educational program" OR constructivism OR constructivist OR instruction)
AND "problem based learning" OR PBL OR "problem based learned") AND
(contribution OR advantages) AND (challenges OR outcome OR results)
AND (computer OR software OR "computer engineering" OR "information
technology" OR "computer science")

Initially, using the search string resulted in 2,464 studies
being located. From that number, the process of selecting
primary studiesbbeagan and was conducted first of all on the
title and abstract. As a result, 2,412 studies wwere discarded
from this total.
For the classification of the studies, inclusion and
exclusion criteria wer designed, which are:
• Inclusion: the relevance that the study has in relation
to the proposed research question: Studies that
describe research related to the topic of the PBL
teaching methodology and Computer Science; studies
that relate experiences of the teaching methodology
being favourable for teaching on Computer Science
courses.
• Exclusion: Studies published in editorials, prefaces,
summary articles, interviews, news and reviews;
Studies with experimental results that are inconsistent
or with incomplete content; Studies that are not part
of the research area.
Based on reading the introduction and conclusion of the
Relevant Studies and the criteria for inclusion and exclusion, a
total of 52 Primary Studies was obtained. Of the 133 ostensibly
Relevant Studies, 80 of them were considered "Not Relevant"
and 12 were unavailable for download, as shown in Table II.
TABLE II. R
ESULTS OF THE SEARCH
.
During the process of analyzing the studies, they were
classified according to their year of ublication year. Thus a
count could be made of studies on PBL over the last 15 years.
Figure 1 shows a graph of the the evolution of the PBL
approach over the years.
Figure 1. Distribution of results published over the years shown above.
As can be seen, it is clear that PBL approaches in the area
of Computer Science have been growing since 2009, with a
high incidence in 2010.
IV. M
AIN
R
ESULTS
The studies found were associated with categories
corresponding to the secondary research questions outlined in
Section III. To prove each association, evidence was collected
from each "Primary Study", represented in the following
subsections by the notation "EP". The following subsections
present and comment the results for each category. Due to size
limitations of this article, only some evidence of the complete
work will be mentioned.
A. Characteristics of PBL
The purpose of this question was to map the characteristics
of the PBL method in the teaching of Computing Science. The
topics were defined as per the characteristics of PBL defined
by Woods [15] and Barrows [16]. Figure 2 shows the primary
studies qualified for each of these topics.
Figure 2. Results on Characterístics of PBL
In general, the authors are clear when they affirm the need
for a problem that starts the process, and that all knowledge
should be acquired through solving problems, thus enabling the
student to have greater contact with real-world problems, as the
evidence makes clear:
EP_11: “The problem-based learning method dictates that the
students’ projects must aim at solving a problem.
EP_03: “In PBL students learn by addressing ill-defined and
open-ended problems and reflecting on their experiences, thus
developing problem-solving strategies and building domain
knowledge in a self-directed manner.”
Another important feature is the need for a change in
attitude and role of teachers and students, in the latter, thereby
stimulating the capacity for self-initiative:
EP_08: “…teaching based on problem solving (in which
students have an active role in their learning) allow students to
understand difficult concepts better and retain the knowledge
acquired for a long period of time.”
EP_14: “The educator should use a problem to prompt
students’ experiences and interests, and then help them to be actively
involved with further learning.”
EP_27: “the author takes the method of problem-based learning
and suitably arranges the course design, which with a good result not
only improves the quality of teaching, but also trains students'
capacity for self-learning, active exploration and mutual
cooperation.”

Finally, the studies highlight the characteristic of
collaborative learning activities with emphasis on group
learning as a key point of PBL:
EP_08: “The design project is structured around a formalized
base group which exists for the duration of the subject. The base
groups are selected at random from the different degree programs
and, in general, consist of three members with no two members from
the same degree program.”
EP_11: “It is clearly an advantage of the PBL curriculum that
students have ample opportunities to practice teamwork skills.”
B. Effectiveness
The purpose of this question was to research in primary
studies which of them showed their effectiveness, ie good
results as to applying the PBL method. The studies were
divided into eleven (11) topics that diversify the effectiveness
of the PBL process, as given in Figure 3.
Figure 3. Results on the Effectiveness in PBL.
Most studies (35.7%) are summarized into one topic, where
the authors state that PBL is more effective when combined
with collaboration strategies, such as collaborative learning,
and constructing motivating applications, such as developing
games, competitions or applications related to market
problems, as highlighted in the evidence:
EP_01: “This article described o PBL mode under the guidance
of instruction system design, constructivism learning theory,
Brunner's discovery method and PBL pedagogy.”
EP_03: “Combining PBL with cooperative learning provides a
mechanism for students to maximize their own and other group
members’ learning by working in teams to accomplish a common task
or goal.”
EP_29: “In a game development project, students can develop
their own requirements for a domain in which they are experts.”
EP_42: “… develop a National RoboCode Competition both
authors saw potential benefits to enhance their PBL initiative and to
add an interesting challenge for their students.
EP_43: “The cases are written in close consultation with industry
practitioners to ensure that they resemble, as realistically as
possible, real-world business problems.”
EP_44: “I examined how student self-efficacy, as it relates to
being software development professionals, changed while involved in
PBL environment.”
Other criteria were also mentioned, such as the formation of
teams of group work, e-learning support tools, development of
critical vision by the students and activities outside the
classroom:
EP_34: “The team includes a group of people (about 5-6 people)
that work together to learn or work on a subject or project. They have
different backgrounds, such as academic disciplines, skills and
expertise, and working experiences.”
EP_n:“… in an internet-based learning environment, it is
considerably more efficient for students to exchange information and
opinions in comparison to the traditional classroom-oriented
learning environment, which indirectly encourages communication
among students.”
EP_18: “MALESAbrain encourages learners to judge or
critically evaluate the solutions posted by others before exploring
further knowledge-content.”
EP_28: “it is also necessary to leave homework, especially
experimental projects to students so as to consolidate their
knowledge learned in the classroom.”
C. PBL Methodology
The purpose of this question was to point how the PBL
methodology is applied in the teaching of Computer Science.
To answer this question, five elements considered essential in a
PBL methodology applied to the teaching of computing were
used, which, according to the authors Santos et al. [3] and [17],
are problem, environment, content, people and processes.
Figure 4 shows the topics and the number of primary studies
associated with these elements
Figure 4. Results on the PBL Methodology.
Among the criteria cited, the part of content was the most
discussed, 34.4% of the primary studies doing so, in which the
disciplines that adopted PBL methodology were cited: software
engineering, robotics, embedded systems, information systems
management, operating systems, digital systems, and the
development and quality of software. Some pieces of evidence
comment on the proposals of these disciplines:
EP_01:“The Software engineering course was taken as the
example to illustrate the implementation of the PBL model.”
EP_02:“Topics are introduced in a way such that students are
confident with what computer studies imply despite the use of the
robot. Robotics is only the “toy” for motivating beginner students on
the subject.”
EP_16: “In order to verify the effect of PBL in online teaching,
we have made an experiment on the course of "Object-Oriented
Programming With Java" over the last 3 years: we recorded
students’ performance, their feelings and the outcome of study; by
making comparison, we analyzed and summarized the strategy for
guidance, improved the strategies and teachers’ ability in guiding
students.”

The criteria associated with the processes of the PBL
method, for example, the steps of teaching and assessment
methods used were also highlighted in 27.66% of the primary
studies:
EP_03: “Student feedback is used extensively to evaluate the
performance of both the teaching staff and the subject. These student
evaluations consist of both formalized university-wide evaluations
(JCET) and informal evaluations conducted solely for this subject by
teaching staff associated with the subject.
Another important element was the environment,
mentioned in 21.3% of the studies, in which the most diverse
environments that support teaching based on PBL were
discussed:
EP_08: providing a richer learning environment, offering
students alternative ways of gaining knowledge and information,
enabling more accurate assessment, and individualizing practice,
feedback and reflection.”
D. Advantages and benefits of PBL
This question aimed to investigate the advantages and
benefits of using PBL methodology in Computer Science.
Various benefits were listed, divided into eight (8) different
topics. The most discussed topic, with 38.24%, was that of the
students’ improvement in developing their skills. Also from the
perspective of the students, other topics were highlighted such
as the ease in solving problems, a better critical view on
various subjects, motivated students mmotivated by the lesson
and more participatory, self-directed learning, planning study
time, being better prepared for the work environment work.
From the perspective of the teacher, what stands out is the
improvement in the practice of teaching. Fig.5 summarizes all
the topics, as well as primary studies associated with them.
Figure 5. Results on the Advantages and Benefits of PBL.
Among the pieces of evidence found, are those that stand
out which reinforce these advantages and benefits:
EP_40: “… the application of this methodology have enabled
students to receive all the theoretical concepts and, also, the students
have developed transversal skills such as writing, learning and oral
expressions in the same course, thereby developing a project
consisting of a semi-complex digital system.”
EP_07: “PBL skill is very important in learning. Students are
encouraged to use their present knowledge and skills to find the
answer in PBL. Assessment is ongoing and regular so as to provide
feedback that assists, extends and improves learning. The tutor is an
essential part of providing appropriate and constructive feedback
that is meaningful to students, supports and empowers their learning,
and contributes to their development.”
EP_44: “Twenty-seven students indicated that because of the
course they were ready to deal with the demands of actual software
development projects, even though some seemed surprised about
their new-found confidence.”
EP_24: “Student feedback indicated that the project was
preferred as a learning experience over traditional, lecture-based
methods. The product-based method maintained student engagement
and encouraged intentional learning tendencies.”
EP_16:“Class participation rate has increased by 73%, and the
rate of on-time handing in assignment by 36%, and the percentage of
significant questions increased prominently too.”
E. Challenges Identified
This question aimed to list the main challenges and
problems encountered during the process of teaching / learning
with PBL in Computer Science. Twelve (12) different topics
that address challenges and problems were identified. Figure 6
shows the 12 topics and the number of primary studies
associated with them.
Figure 6. Results on Challenges found in PBL.
According to the studies, the main challenges of using the
methodology are the lack of belief in and knowledge of the
methodology, deficiencies in the formation of basic
foundations of Computer Science and difficulty in applying the
methodology as highlighted by some of the evidence found:
EP_28: “It is difficult for a teacher to organize and control the
learning process when the number of students is large.”
EP_28: “Before taking this course, students generally have
learned object-oriented programming language(s) such as C, Java,
unfortunately they usually only know the language syntax but don’t
have practical project experiences.”
EP_35: “… in o PBL environment, these students have to climb a
stiff learning curve and overcome much resistance that might pose
challenges to students in the initial stage, and limit the potential
effects of PBL.”
EP_25: “… the traditional teaching method is more efficient in
knowledge of teaching and learning; in contrast, PBL may require
teachers and students to expend more time and effort. Under the
limited class hours, the teaching timetable may not be completed in

PBL. In this regard, we believe that some appropriate additions and
adjustments be made according to the actualnumber of class hours.”
EP_43: “Unfortunately, the adoption of a real-world project is
not always a feasible option. This is mostly due to difficulty in gaining
access to suitable real-world organizations and resource constraints
in managing the projects.”
EP_29: “What challenges educators the most is often not the lack
of teaching materials, but the constraints imposed upon a course.
These include (1) type of audience, (2) the limited amount of time
available for a prearranged set of course objectives, (3) a rigid
curriculum, and (4) the choice of teaching methods.”
EP_14: “The variety of student backgrounds had the potential to
cause problems when teaching the module.”
Another important point refers to the learning process,
since not all students learn equally. Also, some students take
on different responsibilities, thus making teachers’ evaluation
of this difficult. As a result, teachers very often do not know
how to assess students and hence the effectiveness of the
methodology, by ensuring that students have achieved the
desired goals, as the evidence stresses:
EP_28: “When students are organized as groups, their
capabilities may not be developed equally. Some play a dominant role
in the group, while others are there just to make up the number.”
EP_11: “Besides the fundamental challenge of creating a good
problem, educators are faced with the task of deciding how to
evaluate the technique’s effectiveness and how to assess whether
students have met the overall learning objectives for the course.
V. F
INAL
R
EMARKS
This paper has presented and discussed the results of a
systematic mapping, and has identified the main characteristics
of having used PBL in the field of Computer Science from
1997 to 2011. After the initial assessment of 2,464 articles, 52
primary studies were selected tthat were relevant for the
research. From five research questions, it was possible to find
evidence in a satisfactory way which showed that most
applications of the PBL methodology are geared towards its
applicability in the disciplines of Software Engineering and
Programming.
From all the evidence reviewed, it was moreover possible
to conclude that the PBL methodology has been applied in
Computer Science, and been widely accepted by teachers and
students and facilitated improving the education and training of
professionals by providing a broader view of the real problems
in the area. However, there is still a need to overcome several
challenges and, above all, to provide good results from the
application of the methodology with a view to disseminating
positive points of this method and to increase belief in it by
both educators as well as the students who have no knowledge
or understanding about PBL.
R
EFERENCES
[1] J. R. Savery; T. M. Duffy. “Problem Based Learning: An Instruction
Model and its Constructivist Framework”. 1995.
[2] K. Martin; D. Chinn. “Collaborative, Problem-Based Learning in
Computer Science” . 2005.
[3] S. C. Santos; M. C. M. Batista; A. P. C. Cavalcanti; J. Albuquerque; S.
R. L Meira. “Applying PBL in Software Engineering Education”.
CSEET 2009, Hyderabad, Índia, 2009.
[4] H. Arksey; L. O'Malley. “Scoping Studies: Towards a Methodological
Framewrok”. 2005.
[5] C. R. Coles. “Is problem-based learning the only way?” . 1991.
[6] W. Peng. “Practice and Experience in the Application of Problem-based
Learning in Computer Programming Course”. International Conference
on Educational and Information Technology. 2010.
[7] K. Petersen. “Systematic Mapping Studies in Software Engineering”.
2007
[8] R. G. Milter; J. E. Stinson. “Educating leaders for the newcompetitive
environment”. 1995.
[9] L. R. C. Ribeiro. “A aprendizagem baseada em problemas (PBL): Uma
implementação na educação em engenharia na voz dos atores”. 2005.
[10] D. Boud; G. Feletti. “The Challenge of Problem Based Learning”. 1997.
[11] R. Deslisle. “How to use problem-based learning in the classroom.”
Alexandria, ASCD, 1997.
[12] H. S. Barrows.”Problem Based Learning (PBL)”. 2001.
[13] B. Kitchenham. “Guidelines for performing Systematic Literature
Reviews in Software Engineering”. 2007.
[14] M. Peterson. “Skills to Enhance Problem Based Learning”. 1997.
[15] D.R. Woods. “Problem-based learning: How to gain the most from
PBL”. 1994.
[16] H. S. Barrows. “Problem Based Learning in Medicine and Beyond”.
1996.
[17] S. C. Santos and A. Pinto, “Assessing PBL with Software Factory and
Agile Processes”, CATE, Naples, Italy, 2012.