Content uploaded by Ruth Aluko
Author content
All content in this area was uploaded by Ruth Aluko on Apr 26, 2016
Content may be subject to copyright.
1
From evaluation to reflection-on-action: Lessons learnt from the impact of a
distance education programme
Abstract
Reflection-on-action is necessary to derive meaning from one’s experiences. This
paper revisits research data from an elongated study on the impact of a distance
education programme on the professional practice of graduates. The study focused
on 300 graduates and 128 principals, selected through multi-stage and purposive
sampling. The researcher used a mixed-methods research design with specific focus
on Kirkpatrick’s, and Baldwin and Ford’s training evaluation models. The
researcher’s curiosity was triggered by the need to understand possible reasons for
the participants’ views, as these are contrary to the norm. This account indicates a
clear institutional policy on quality assurance, practices guided by the policy, an
ongoing monitoring of the distance education students’ profiles, improved
programme design, student support structures, programme design and research
focused on programmes as possible reasons. The author argues that higher
education practitioners, irrespective of delivery mode, could benefit from the valuable
lessons learnt from the exercise.
Keywords: Distance education, evaluation, impact, professional development,
quality, reflection, reflection-on-action
2
Introduction
Although higher education is involved in quality assurance at various levels, a main
concern related to quality systems is the lack of a programme evaluation dimension
(Mizikaci 2010). Evaluation means to understand the value of something in order to
do things better (Rubin 1995). Furthermore, evaluation is a tool for decision-making
and for assessing and promoting the quality of a programme (Wilson-Strydom 2004).
Evaluation in higher education usually focuses on the aims of a programme and
investigates to what extent the intentions of the programme initiators are being
realised (Hall and Hall 2004).
Scholars are divided on whether evaluation differs from research (Wilson-Strydom
2004). However, Patton (1990) stresses that when one systematically and
empirically examines and judges accomplishments and effectiveness through careful
data collection and thoughtful analysis, one is engaged in evaluation research.
Although evaluation is a valuable process, it is more important to learn from such an
exercise. One of the tools for such learning is reflection, which is also known as
reflective practice. Reflective practice requires a professional to step back from an
experience to make sense of it, understand what it means, learn from it and apply
that learning to future situations (Britton 2010).
Although it has been proved that distance education provides wider access, the
mode is not without its challenges in terms of its effectiveness, impact on students’
professional development and efficiency in delivery (Harreveld 2010).
3
Feedback from a longitudinal study (Aluko 2009; Aluko 2012; Aluko and Shonubi
[n.d.]) on the impact of a distance education programme: the Advanced Certificate in
Education: Education Management (ACE: EM), showed that graduate-participants
were of the opinion that they benefitted immensely from the programme. The
researcher was interested in investigating possible reasons for their positive views.
This reflection was never part of the longitudinal study, but the data from the study
triggered the researcher’s curiosity, especially considering the fact that one of the
main concerns of distance education programmes among stakeholders is the quality
of such programmes (Belawati and Zuhairi 2007).This led to the following questions:
Why did graduate-participants judge the programme as beneficial to their personal
and professional life with confirmation from their principals? What has made it
possible for the programme to meet its goals? What possible lessons could be
learnt? The author argues that such lessons could be applied to the management of
all educational programmes in higher education, irrespective of their delivery modes.
From programme evaluation to reflection: Conceptualising the process of
reflection
By three methods we may learn wisdom: first, by reflection, which is noblest;
second, by imitation, which is easiest; and third, by experience, which is the
bitterest. (Confucius)
Programme evaluation is an important activity that directly relates to quality (Wilson-
Strydom 2004). Institutions need to attend to the twin issues of quality assurance
and quality control (Mugridge 1999). Thus, evaluation is not an end in itself, but
should rather be used as a means to maintain or improve the quality of products and
4
processes (Rathore and Schuemer 1998). It is incumbent upon education providers
to ensure that an institutional climate that encourages continual review and
evaluation, supported by policies and procedures, is in place (Mugridge 1999). There
is a need for a culture of quality in higher education institutions by which all
stakeholders are empowered to be responsible (Aluko 2007). In the process of
evaluation, one of the learning tools available to an evaluator is reflection.
The definition of the term “reflection” is slippery due to its elusive boundaries (Jay and
Johnson 2002). Nonetheless, Dewey (1933, 9), provides one of its earliest definitions
as the “active, persistent and careful consideration of any belief or supposed form of
knowledge in the light of the grounds that support it and the further conclusions to
which it tends”. Reflection is thinking with a purpose. For this reason, all evaluation is
a form of reflection (Hall and Hall 2004). Reflection-in-action helps practitioners to
think about what they are doing while engaged in the process (Schön 2011), while
reflection-on-action concerns reflecting on past actions (Hall and Hall 2004).
Scholars are of the opinion that reflection-on-action encourages the scope and depth
of reflection because of the lack of opportunity to stand aside from the dynamics of
an activity while it is in process (Boud, Keogh and Walker 1985 and Schön 2011).
These should be regarded as complementary since it is difficult to separate action
from thought (Moon 1999). This paper, however, focuses on reflection-on-action.
Evidence available from literature shows that a trigger for reflection is often a state of
discrepancy between reality and expectation (Pammer, Knipfer, Krogstie, Wessel,
Prilla and Lindstaedt 2012). With regard to this paper, findings from the longitudinal
study that triggered the researcher’s reflection showed a discrepancy between the
participants’ impression (that the programme has added value to their lives and their
5
workplace) and the impression considered to be the norm (that distance education
programmes are not of good quality).
Background to this paper
As indicated previously, this paper emanated from a longitudinal study on the impact
of a distance education teaching programme on the professional practice of
graduates. The University of Pretoria is a contact teaching and research institution,
which began to offer distance education programmes through its Faculty of
Education in 2002. Its target population was teachers, especially those in rural areas,
where a university presence was not strongly felt. Students enrolled in the
programme were mostly female, over 40 years of age, with little or no access to
modern technology. In 2006, one of the requirements the Council on Higher
Education (CHE) demanded of the University was to evaluate the impact of its ACE:
EM programme (University of Pretoria 2006). Accordingly, in 2007, the researcher
conducted a pilot study on behalf of the institution, followed by a longitudinal study
between 2009 and 2012. The programme has recently been discontinued due to the
South African government’s decision to encourage teachers to focus on teaching
subjects with the aim of improving their classroom practice (Council on Higher
Education 2010).
For the longitudinal study, the researcher adopted a mixed-methods research
design, using both surveys and interviews to capitalise on the strengths of both
qualitative and quantitative methods (Punch 2005). Prospective participants were
identified through multi-stage and purposive sampling techniques (Punch 2005), and
were drawn from the major provinces from which students enrolled for the
6
programme (Gauteng, Eastern Cape, KwaZulu-Natal, Limpopo and Mpumalanga). A
total of 300 graduates and 128 principals participated in the study. Interviewees were
identified for the interviews from the surveys returned by participants.
In order to ensure the content validity of the instruments, the researcher focused on
the stated programme outcomes at national and institutional levels, combined with
the graduates’ aspirations (Aluko 2009). The training evaluation models of
Kirkpatrick (1996), and Baldwin and Ford (1988) were adapted to further probe the
contextual factors that determine the extent of learning transfer (Aluko and Shonubi
[n.d.]). The researcher received an ethical clearance certificate from the Faculty’s
Ethics Committee, and sought permission from the different provincial departments
of education.
The Atlas.ti 5.0 (Computer-assisted Qualitative Data Analysis software) was used to
analyse the interview transcripts. Relevant quotations and codes were identified
based on the concepts and themes frequently mentioned by interviewees (Hardy and
Bryman 2004). The quantitative aspects of the questionnaires for the longitudinal
study were analysed by the University of Pretoria’s Department of Statistics. It
arrived at the cumulative frequencies and cumulative percentages. The data was
revisited as a result of the researcher’s curiosity to investigate possible reasons for
participants’ positive comments about the programme. This was triggered by these
views’ contrast to the seemingly persistent negative view regarding the quality of
distance education programmes by some stakeholders.
7
The cyclic process of reflection and its application to the paper
Scholars have expatiated on Dewey’s work by attempting to identify the process of
reflection (Lee 2005; Jay and Johnson 2002; Rodgers 2002; Boud et al. 1985; and
Pammer et al. 2012.). However, Jay and Johnson (2002) summarise the process of
reflection as involving several common processes: describing the situation, surfacing
and questioning initial understandings and assumptions, and approaching the
subject with persistency, an attitude of open-mindedness, responsibility, and whole-
heartedness. According to the scholars, one evaluates insights gained from this
process with reference to: additional perspectives, one’s own values, experiences
and beliefs, and the larger context within which the questions are raised. The
process of reflective practice thus brings together the skills of self-understanding,
critical thinking, analysis and experience-based learning (Barefoot Collective 2009).
After careful consideration of the scholars cited above, the researcher identified the
following as the process of reflection to be focused on. These are briefly explained
and applied in the sections that follow:
The content of reflection
Returning to the experience
Attending to feelings
Re-evaluation of experience
Generating possible explanations
The outcome of reflection
Intelligent action/experimentation
8
The content of reflection
The content of reflection stage can also be termed the experience stage. In the
workplace learning environment, this is work-related and therefore becomes the
content of reflection (Pammer et al. 2012). What gives an experience value is the
meaning that one perceives in it and then constructs from the experience (Dewey
1938). Experiences alone are not enough; the ability to perceive and then weave
meaning among the threads of experience is critical (Rodgers 2002). With reference
to this paper, the researcher’s experience was the study conducted regarding the
impact of the ACE: EM programme on the professional practice of graduates.
Returning to experience
An experience exists in time and is therefore linked to the past and the future. It is
triggered by curiosity, without which there is little energy for the hard work of
reflection (Rodgers 2002). One can only think reflectively when one is willing to
endure suspense and undergo the trouble of searching (Dewey 1933). Thus, the
researcher had to return to the research data to highlight possible reasons for the
participants’ perceptions of the quality of the programme, and possible reasons for
the programme’s positive impact on the participants’ professional and personal lives.
Attending to feelings
Building on Dewey’s work, Boud et al. (1985) argue that reflection is not just a
process of thinking, but one that essentially involves feelings and emotions. This is
because there is a puzzling, troubling or interesting phenomenon with which the
individual is trying to deal. While trying to make sense of it, individuals also reflect on
the understandings that have been implicit in their actions. These understandings
must be surfaced, criticised, restructured, and embodied into further action (Schön
9
1983). Emotion is an important facet of learning because it affects motivation and the
ability to understand what is being experienced (Hinett 2003). Although not classified
as one of the basic emotions, curiosity, which led to the development of this paper,
can induce an emotion (Wikipedia 2014). As this emotion represents a thirst for
knowledge, curiosity is a major driving force behind scientific research and other
disciplines of human study (Wikipedia 2014).
Re-evaluation of experience
Dewey (1933) refers to this phase as “intellectualisation”; a process of
intellectualising what at first is merely an emotional quality of the whole situation. It is
not a casual process, but a disciplined one that demands of the individual to
continually ground his or her thinking in evidence and not overlook important data
that may not fit his or her evolving ideas (Rodgers 2002). With regard to this phase,
the researcher’s reflection was guided by the data that emanated from the
longitudinal study because of its contrast to the generally accepted perception that
distance education programmes are of inferior quality. This led to a new and better
understanding of the experience and allowed for the derivation of implications,
conclusions or lessons learned, as described by Pammer et al. (2012).
Generating possible explanations
The stage above leads to a generalisation and abstraction of the concrete
experience (Pammer et al. 2012). According to the scholars, this is a core part of the
reflective process that differentiates it from repetitive thought and rumination. This
stage also demands that one refines the suggestions as probabilities, which
encourages one consult resources beyond oneself (Rodgers 2002). In order to
generate possible explanations for the findings, the researcher had to critically look
10
into what probabilities literature offers and into systems the University has put in
place to explain how the programme has managed to attain its outcomes.
The outcome of reflection
Reflection implies that something is believed in (or disbelieved) – this resulting, not
of its own account, but of the account of something that stands as witness, evidence,
proof, voucher or warrant. This functions as grounds of belief (Dewey 1933, 10). The
outcome of the researcher’s reflection is presented and discussed in the section that
follows.
Literature supports outcomes as being necessary requirements for the success of
any educational programme (Aluko and Hendrikz 2012; Pulsipher 2009; Walsh 2009;
Nonyongo and Ngengebule 2008; Welch and Reed 2005; Lomas 2004;
Commonwealth of Learning 2004). Rodgers (2002) advises that the one who reflects
should seek meaning, and create from this a theory to live by. This theory is
expected to guide practice until it encounters a situation where the theory no longer
serves (Rodgers 2002).
Intelligent action/experimentation
This is the level at which theory is tested (Dewey 1933). This experimentation, which
involves interactions between the self, others and one’s environment, in turn serves
as the next experience from which learning can continue. Dewey called this
phenomenon continuity. Even though the lessons identified in this paper are part of
the day-to-day affairs of the University, it is necessary to share these lessons with
education practitioners. Hopefully, these will help to ascertain whether the meaning
ascribed to the experiences fits, makes sense, and can be relied on in future
11
experiences (Rodgers 2002). Nevertheless, as advised by the same author, the
researcher does not rule out the fact that more questions, problems and ideas will
arise in the process of continually monitoring the quality of the programmes run by
the University.
Outcomes of the researcher’s reflection and its implication for management
practices in higher education
Reflection has the potential to lead to a better understanding of one’s own work
practice and work-related experiences, and can guide future behaviour (Pammer
et al. 2012). This reflection-on-action account shows that the programme has
benefitted from the following aspects. These are elucidated in the sections that
follow:
A clear institutional policy on quality assurance
Practices guided by the policy
An ongoing monitoring of the distance education students’ profiles
Student support structures
Programme design
Research focused on programmes
A clear institutional policy on quality assurance
Management provides the framework for the policies, procedures, practices and
leadership of an organisation (Mizikaci 2010). As indicated previously, quality is a
major concern in distance education. From the onset, the University decided that, if
its distance education programmes could not conform to the same quality standards
12
as its contact programmes, it would rather not run them at all. A clear institutional
policy on quality assurance becomes important in higher education. The absence of
this becomes more pronounced in distance education in light of persistent negative
perceptions of the quality of its programmes, not just among outsiders, but also
among academics (Du Plessis and Van der Merwe 2005; Perraton 2007; Porto and
Berge 2008). This means that providers need to focus on ensuring quality if they
want their programmes to be successful.
At the University of Pretoria, quality is seen as a priority. It is an iterative process and
is considered to be part of a philosophy that pervades the University’s managerial,
teaching and administrative styles, irrespective of the mode of delivery (University of
Pretoria 2009). In order to enhance quality, it is of the utmost importance that
institutions create awareness among relevant stakeholders and involve them in the
process. This is very advantageous as it serves, for instance, as a way of giving
students a voice, and at the same time helps the institution to increase accountability
(Lomas 2004).
McNeese State University (2006) defines an institutional policy as a policy with a
wide scope that has the potential for campus-wide impact. One of the areas in which
higher education institutions need to take a stance regards quality. Quality became a
key occupation of higher education policies in the 1980s (Walsh 2009). In its survey
of 11 Western countries, the Centre for Higher Education Policy Studies (CHEPS)
(2004) observed that the concern for the quality of higher education is an ongoing
issue in many countries, but that it seemed to have moved away from being a top
priority.
13
One of the challenging areas with regard to the quality of distance education is that
of completion rates. Scholars have, however, warned that this should not be taken
out of context as there are particular reasons for it being more prevalent in this mode
of delivery, and must therefore be interpreted in line with other factors (Aluko 2007;
Tucker 2001). However, it appears from the lessons learnt from this programme, that
it is possible for institutions to have a firm grip on student retention rates if more
attention is paid to the quality of distance education programmes. Table 1 supports
this statement.
Table 1: Non-completion rates of distance education students at UP
(All programmes)
Year
Students who discontinued
their studies
Percentage of total
enrolment
2006
419
4%
2007
650
4%
2008
584
3%
2009
762
4%
2010
430
2%
2011
650
3%
2012
689
4%
Source: University of Pretoria (2010–2012)
It is evident from Table 1 that the highest non-completion rate of distance education
students at the University is 4%. Unfortunately, this is not a common phenomenon in
distance education programmes due to the peculiar characteristics of this mode of
delivery (Singh 2000; McKenzie and Schweitzer 2001; Antony and Gnanam 2004).
14
Practices guided by the policy
Something that is relevant to the ideas stated above is the importance of a policy
guiding actual practices and not just those existing on paper. Welch and Reed
(2005) have stressed that a formal commitment by the government is crucial to
quality in the distance education delivery mode. The South African government has
identified this mode as being paramount to redressing the imbalances of the past
(South Africa 1996). It has, through a research team, developed a distance
education quality standard framework for South Africa, with the aim of correcting
observed anomalies (Council on Higher Education 2004c). However, it was not until
recently (May 2012) that it has come up with a draft policy document on distance
education that is yet to be finalised.
Relevant and specific policies regarding distance education becomes important in
light of the fact that the criteria of the CHE’s Higher Education Quality Committee
(HEQC) for programme accreditation, which all higher education programmes are
expected to meet, irrespective of the mode of delivery, do not sufficiently meet the
needs of distance education (Welch and Reed 2005). However, the National
Association of Distance Education Organisations of South Africa (NADEOSA), in
conjunction with providers of distance education in the country, have developed
quality criteria that guide the practice of this mode of delivery (Welch and Reed
2005).
The Unit for Distance Education at the University of Pretoria developed its policy
document on distance education from these quality criteria. This policy guides its
practices (University of Pretoria 2009). Other documents that have guided its policy
15
included the Commonwealth of Learning’s Planning and implementing open and
distance learning systems (Commonwealth of Learning 2004) and the CHE’s Criteria
for programme accreditation and Criteria for institutional audits (Council on Higher
Education 2004a; Council on Higher Education 2004b). Nevertheless, the researcher
believes that having a policy in place is not as important as putting what has been
determined into practice. Since the essence of policy is to ensure and improve
quality, Welch and Reed (2005) have warned that quality is not fixed and static, but
develops with changes in education thinking and practices, as well as with advances
in technology.
An ongoing monitoring of the distance education students’ profiles
University policy-makers need quality data to make data-driven decisions regarding
programme improvement to accomplish their stated mission, among other things
(Pulsipher 2009). Some of the data kept by the Unit for Distance Education at the
University of Pretoria includes age, access to technology, work and geographical
profiles of students, and graduation and retention rates. Managers can make better
and more informed decisions when they have access to quality data (Pulsipher
2009).
For instance, keeping track of students’ profiles has helped the Unit for Distance
Education decide on the form of delivery to adopt for the programme and what
venues will be made available for contact sessions. Table 2 reflects the changing
technology profile of distance education students.
16
Table 2: Profile of students who enrolled for the first time between
2004 and 2012
Year
2004
2006
2007
2008
2009
2010
2012
Number of students
3 187
5 087
5 643
8 011
6 102
5 675
3 354
Cellphone use
98%
99%
99%
99%
99%
99%
100%
Internet use
0%
2%
1%
3%
8%
13%
25%
Source: University of Pretoria (2010–2012)
In Table 2, it is clear that the cellphone profile of students has remained constant
over the years (2004 to 2010) with a slight increase in 2012 (100%). The internet
profile increased from 0% in 2004 to 25% in 2012. These monitored changes
(University of Pretoria 2010–2012) have informed the Unit’s choice of supporting
students through mobile phones, while other options are being explored as a
reaction to the increase in access to the internet.
Student support structures
Student support structures have been defined in the broadest terms as
“all activities beyond the production and delivery of course materials that assist in the
progress of students in their studies”, which can include both academic and non-
academic services (Simpson 2000, 6). In distance education teaching models,
greater emphasis is placed on the quality of student support services, because most
students who enrol for distance education programmes from traditional learning
backgrounds are ill-equipped to handle the unique demands of studying at a
distance (Lowe 2005).
17
The University is well aware that studying at a distance while working full-time is
difficult, thus, it developed extensive academic support structures to help students to
succeed in their studies (University of Pretoria 2009). These include long and short
contact sessions, tutorial letters, assignments, short message service (SMS) and an
academic enquiry service. Unfortunately, most of the students enrolled in the
University’s distance education programmes up to now reside in rural areas with little
or no internet connection. There is a need for higher education institutions to provide
support structures that are relevant to their students’ contexts (Aluko and Hendrikz
2012). Attesting to the importance of these structures, Nonyongo and Ngengebule
(2008) stress that effective learner support is likely to contribute to the quality of
distance education programmes.
Programme design
The University of Pretoria redesigned the ACE: EM programme in 2007 with the goal
of promoting access to distance education, providing quality distance education and
providing effective student support (Fresen and Hendrikz 2009). According to the
scholars, when one reconceptualises a learning programme, one has to consider the
purpose of the programme, its structure, the articulation between modules in the
programme, the learning activities, the support material and the assessment
strategy. All these were taken into consideration when redesigning the ACE: EM
programme, while the redesign methodology now guides similar projects (Fresen
and Hendrikz 2009). To attest to the quality of the design of the ACE: EM
programme, one of its modules (Organisational Management 2) won the Biennial
Quality Courseware Awards in 2010.
18
It has been established that most open and distance student learning occurs
independently of the teacher’s presence. Students focus primarily on engagement
with the material they receive (Evans 1997; Welch and Reed 2005). Although
modern technology has taken over the delivery of distance education programmes,
most countries in Africa are still trapped in the first-generation mode of delivery.
Therefore, the onus is on providers of distance education to improve on the quality of
their programme design.
Research focused on programmes
Research focused on programmes situated within the delivery mode help institutions
to obtain detailed information about the programme activities and the effectiveness
of the programme from the viewpoint of various stakeholders (Mizikaci 2006). The
University’s Unit for Distance Education has a dedicated research office that
conducts both operational and academic research into its distance education
practices. The major aim of research in the Unit is to inform practice, while outcomes
are presented at conferences and are written up as research papers.
Research in distance education has moved beyond comparison between the mode
and its conventional counterpart. Attention is increasingly placed on how research
can be used to inform practice, which the researcher believes should stem from the
internal research structures put in place at the institutional level. Scholars have
advised that research in distance education should focus more on students’ success
and should be learning-centred. This would likely sensitise faculties to the individual
student and prepare them to facilitate distance education (Diaz 2000; Lockee, Moore
and Burton 2001).
19
According to Lee, Driscoll and Nelson (2004), understanding trends and issues in
terms of topics and methods is paramount in advancements in research into distance
education. One of the most important areas in need of research in distance
education is the evaluation of programmes; and even when this is carried out,
scholars have been encouraged to use not only the quantitative approach, but rather
to use the mixed-methods approach (Mizikaci 2006; Fahy, Spencer and Halinski
2007; Aluko 2009). Such studies must also be grounded in sound theoretical and
conceptual frameworks (Saba 2000), which should be used as a guide for action
(Magagula 2002).
Conclusion
Although higher education is involved in quality assurance at various levels, a main
concern related to quality systems is the lack of a programme evaluation dimension
(Mizikaci 2010). Programme evaluation is an important activity that relates directly to
quality. The researcher’s initial intent was to evaluate the impact of one of the
distance education programmes at the University of Pretoria (ACE: EM) through a
longitudinal study (Aluko 2009; Aluko and Shonubi [n.d.]). Graduate-participants in
the study indicated that the programme has added value to their professional
practice, to which their principals agreed. However, the data triggered the
researcher’s curiosity, after considering that one of the main concerns of distance
education programmes among institutions and stakeholders is the quality of such
programmes (Belawati and Zuhairi 2007).
It emerged that the programme benefited from a clear institutional policy on quality,
ongoing monitoring of distance education students’ profiles to enable managers to
20
make relevant decisions, good programme design to enhance the quality of the
programme, and effective student support structures. Other benefits included
dedicated research and a policy on distance education that informs distance
education practices at the institutional level.
A particular advantage of distance education is that it makes teacher preparation and
professional development programmes accessible to people located in remote, rural
areas that do not have convenient access to higher education institutions (United
Nations Educational, Scientific and Cultural Organisation 2002). However, quality
and quality management are organisational obligations that are vested in
management’s commitment towards an understanding of quality (Aluko, Fraser and
Hendrikz 2008). Literature suggests that the singular reason for mixed reactions to
distance education mainly relates to quality (Du Plessis and Van der Merwe 2005).
A clear institutional policy is the starting point to ensure quality in higher education.
This helps institutions to be accountable to relevant stakeholders. In return, it helps
boost student attrition rates – a topic that has long plagued the distance mode of
delivery. One hopes that such a policy will not just exist on paper, but will determine
providers’ practices. There is a need to establish structures that will regularly help to
monitor students’ profiles. Outcomes of such activities will help management make
well-informed decisions. An example of this is the support structures that should be
extensive and relevant to students’ particular contexts. In addition, well-articulated
programme design often goes a long way in attesting to the quality of programmes.
Lastly, focused research on distance education practices should aim to make an
impact on practices at such institutions. Scholars have stressed that all the valuable
21
lessons that emanate from a reflection like this have probable effects on the overall
quality of education programmes, which lead to the improved performance of
students (Killen 2002; Chickering and Ehrmann 2003; Bornman 2004; Magagula and
Ngwenya 2004; Bates 2005).
In support of the above, scholars (Welch and Reed 2005; Walsh 2009) have
stressed the urgent need for providers of the distance mode of delivery to review
quality assurance mechanisms for higher education at the national and institutional
level, in order to discuss new challenges facing the changing distance education
environment, and to build quality assurance capacity to enhance quality standards in
a globalised higher education market. One of the ways in which institutions can do
this is by evaluating the impact of their programmes.
Although it is not a dedicated distance education institution, the University of Pretoria
is still in the business of distance education. The researcher is aware that the
practices that govern the administration of distance education at the University follow
guidelines that also reflect the underlying philosophy towards education adopted by
this institution. These invariably guide all programmes run by the University,
irrespective of their mode of delivery. From these valuable lessons, one could safely
assume that there is a need for evaluators to practice reflection-on-action more
often. Scholars have stressed that this encourages the scope and depth of reflection
(Schön 2011). Although curiosity comes naturally to children, a childlike wonder
about the world is something that adults must often cultivate (Rodgers 2012).
22
References
Aluko, F.R. “A comparative study of distance education programmes assessed in
terms of access, delivery and output at the University of Pretoria.”
Unpublished doctoral thesis, Department of Curriculum Studies, Pretoria:
University of Pretoria, 2007.
_____. “The impact of an Advanced Certificate in Education (ACE) programme on
the professional practice of graduates.” The International Review of Research
in Open and Distance Learning 10(4) (2009). [cited 2 April 2010]. Available
from www.irrodl.org.
Aluko, F.R., W.J. Fraser and J. Hendrikz. “Some interfaces in conventional and
distance education programmes in a postmodern context.” South African
Journal of Higher Education 22(3) (2008) : 484–497.
Aluko, F.R. “Evaluating the success of distance education programmes: Challenges
for the NextGen leadership.” A paper presented at the International
Symposium on Educational Reform Conference, USA, June 14–15, 2012.
Aluko, F.R. and J. Hendrikz. “Supporting distance education students: The pilot
study of a tutorial model and its impact on students’ performance.” Progressio
(34)2 (2012) : in print.
Aluko, F.R. and O.K. Shonubi. “Going beyond Kirkpatrick’s training evaluation model:
The role of workplace factors in distance learning transfer.” A paper re-
submitted for review to the Africa Education Review Journal [n.d.] : in print.
Antony, S. and A. Gnanam. “Quality assurance in distance education:
The challenges to be addressed.” Higher Education 47 (2004) : 143–160.
Baldwin, T.T. and J.K. Ford. “Transfer of training: A review and directions for future
research.” Personnel Psychology 41 (1988) : 63–105.
Barefoot Collective. [Online] [cited May 2009]. Available from www.barefootguide.
org/Book/Barefoot_Guide_to_Organizations_Whole_Book.pdf .
Bates, A. Technology, E-learning and distance education. 2nd ed. USA: Routledge,
2005.
Belawati T. and A. Zuhairi. “The practice of a quality assurance system in open and
distance learning: A case study at Universitas Terbuka Indonesia (The
Indonesia Open University).” The International Review of Research in Open
and Distance Learning 8(1) (2007). Available from http://www.irrodl.org/index.
php/irrodl/article/view/340/774.
Bornman, G. “Programme review guidelines for quality assurance in higher
education: A South African perspective.” International Journal of Sustainability
in Higher Education 5(4) (2004) : 372–383.
Boud, D., R. Keogh and D. Walker. “Promoting reflection in learning: A model.” In
Reflection: Turning experience into learning, edited by D. Boud, R. Keogh,
and D. Walker, 18–40. New York: Nichols. 1985.
Britton, B. “Self-reflection”. In Capacity development in practice, edited by. J. Ubels,
N.-A. Acquaye-Baddoo and A. Fowler. London, Washington DC: Earthscan,
2010.
Centre for Higher Education Policy Studies. Higher education policy: Issues and
trends, 2004. [cited 5 January 2013]. Available from http://doc.utwente.nl/
48217/1/ engreport04hepolicyissuesandtrends.pdf.
Chickering, A. and S. Ehrmann. “Implementing the seven principles: Technology as
a lever.” Teaching, Learning and Technology (2003). [cited 23 January 2007].
Available from http://www.tltgroup.org/programs/seven.html.
23
Commonwealth of Learning. Planning and implementing open and distance learning
systems: A handbook for decision makers. Vancouver: Commonwealth of
Learning, 2004. [cited 7 April 2011]. Available from www.col.org.
Council on Higher Education. Criteria for programme accreditation, 2004a. [cited
8 April 2010]. Available from http://www.che.ac.za/documents/d000084/.
_____. Criteria for institutional audits, Higher Education Quality Committee, June
2004. Pretoria: Council on Higher Education, 2004b.
_____. Enhancing the contribution of distance higher education in South Africa,
2004c. [cited 12 January 2007]. Available from http://www.che.ac.za/
documents/d000070/index.php.
_____. Higher Education Monitor: The state of the Advanced Certificate in Education
(ACE) programmes in South Africa, 2010. [cited 2 April 2012]. Available from
http://www.che.ac.za/documents/d000213/Higher_Education_Monitor_11.pdf .
Dewey, J. How we think: A restatement of the relation of reflective thinking to the
educative process. Boston: D.C. Heath, 1933.
Dewey, J. Experience and education. New York: Collier Books: Macmillan, 1938.
Diaz, D. Carving a new path for distance education research, 2000. [cited 29 July
2010]. Available from http://ts.mivu.org.
Du Plessis, P. and W. Van der Merwe. “Mergers, different modes of delivery and
assuring quality for learners.” 2005. Available from http://www.fotim.ac.za/
fotim_conferences/papers/duplessisandvdmerwe.doc.
Evans, T. Understanding learners in open and distance education. London: Kogan
Page, 1997.
Fahy, P.J, B. Spencer and T. Halinski. “The self-reported impact of graduate
program completion on the careers and plans of graduates.” Quarterly Review
of Distance Education 9(1) (2007) : 51–71.
Fresen, J.W. and J. Hendrikz. “Designing to promote access, quality and student
support in an advanced certificate programme for rural teachers in South
Africa.” International Review of Research in Open and Distance Learning,
10(4) 2009. [cited 12 April 2010]. Available from http://www.irrodl.org/
index.php/irrodl/issue/view/37.
Hall I. and D. Hall. Evaluation and social research: introducing small-scale practice.
United Kingdom: Palgrave Macmillan, 2004.
Hardy, M. and A. Bryman. Handbook of data analysis. Thousand Oaks: Sage
Publication, 2004.
Harreveld, R.E. “A capability approach to open and distance learning for in-service
teacher education.” In Teacher education through open and distance learning,
edited by P.A. Danaher and A. Umar. Canada: Commonwealth of Learning,
2010.
Hinnet, K. “Improving learning through reflection – part one.” The Higher Academy,
2003. [cited 6 March 2014]. Available from http://www.heacademy.ac.uk/
assetdocuments/resources/database/id485_improving_learning_part_one.pdf.
Jay J.K. and K.L. Johnson. “Capturing complexity: a typology of reflective practice for
teacher education.” Teaching and Teacher Education 18 (2002) : 73–85.
Killen, R. Outcomes-based education: Principles and possibilities. Unpublished
manuscript, University of Newcastle, Faculty of Education, 2002. [cited
19 December 2006]. Available from http://www.schools.nt.edu.au/curricbr/
cf/outcomefocus/Killen_paper.pdf.
Kirkpatrick, D.L. Evaluating training programmes: The four levels. San Francisco:
Berrett-Koehler Publishers, 1996.
24
Lee, H.J. “Understanding and assessing pre-service teachers' reflective thinking.”
Teaching and Teacher Education 21 (2005) : 699–715.
Lee, Y., M.P. Driscoll and D.W. Nelson. “The past, present, and future of research in
distance education: Results of a content analysis.” American Journal of
Distance Education 18(4) (2004) : 225–241.
Lockee, B., M. Moore and J. Burton. Old concerns with new distance education
research. 2001. [cited 7 January 2013]. Available from www.educause.edu/
ir/library/pdf/EQMO120.pdf.
Lomas, L. “Embedding quality: the challenges for higher education.” Quality
Assurance in Education 12(4) (2004) : 57–165.
Lowe, S.D. “Responding to learner needs in distance education: Providing academic
and relational support (PARS).” In Making distance education work:
Understanding learning and learners at a distance, edited by S.J. Levin, 73–87.
Okemos, Michigan, USA: Learners Association.net, 2005.
Magagula, C. Evaluation report of the course for distance education policy-makers in
Southern Africa. Vancouver: Commonwealth of Learning, 2002. [cited
7 January 2013]. Available from http://dspace.col.org/handle/123456789/120 .
Magagula, C. and A. Ngwenya. A comparative analysis of the academic
performance of distance and on-campus learners. 2004. [cited 29 January
2004]. Available from
http://tojde.anadolu.edu.tr/tojde16/articles/magagula.htm.
McKenzie, K. and R. Schweitzer. “Who succeeds at university? Factors preceding
academic performance in first-year Australian university students.” Higher
Education and Development 20(1) (2001) :21–33.
McNeese State University. Institutional policies definition and approval process.
2006. [cited 7 January 2013]. Available from http://www.mcneese.edu/
policy/institutional-policies-definition-and-approval-p.
Mizikaci, F. “A systems approach to programme evaluation model in quality in higher
education.” Quality in Education 14(1) (2006) : 37–53.
Mizikaci, F. Total quality management in higher education: An evaluation model for
practitioners. Germany: LAP Lambert Academic Publishing, 2010.
Moon, J.A. Reflection in learning and professional development. London: Kogan
Page Limited, 1999.
Mugridge, I. “Quality assurance in open and distance learning.” Indian Journal of
Open Learning 8(3) (1999) : 213–220. [cited 7 December 2012]. Available from
http://journal.ignouonline.ac.in/iojp/index.php/IJOL.
Nonyongo, E.P. and A.T. Ngengebule, eds. Learner support services: Case studies
of DEASA member institutions. Pretoria: University of South Africa, 2008.
Pammer V., K. Knipfer, B. Krogstie, D. Wessel, M. Prilla and S. Lindstaedt.
Reflective learning at work – a position and discussion paper. 2012. Available
from http://ceur-ws.org/Vol-790/paper4.pdf .
Patton, M. Qualitative evaluation and research methods. 2nd ed. London: Sage
Publications, 1990
Perraton, H. Open and distance learning in the developing world. 2nd ed. London:
Routledge, 2007.
Porto, S.C.S. and Z.L. Berge. “Distance education and corporate training in Brazil:
Regulations and interrelationships.” International Review of Research in Open
and Distance Learning 9(2) (2008). [cited 7 July 2008]. Available from
http://www.irrodl.org/index.php/irrodl/article/view/478/1052 .
25
Pulsipher, S. Preparing quality teachers: Tracking a cohort of students from pre-
admission to graduation, 2009. [cited 7 January 2013]. Available from
http://education.byu.edu/cites/documents/Student%20ProfileSusan%20Pulsip
her.pdf.
Punch, K. Introduction to social research: Quantitative and qualitative approaches
2nd ed. London: Sage, 2005.
Rathore, H. and R. Schuemer, eds. Evaluation concepts and practice in selected
distance education institutions. ZIFF Paper 108, Hagen: Central Institute for
Distance Education Research, FernUniversitat, 1998. [cited 17 January 2008].
Available from http://deposit.fernuni-hagen.de/1908/1/ZP_108.pdf .
Rodgers, C. “Defining reflection: Another look at John Dewey and reflective thinking.”
Teachers College Record 104(4) (2002) : 842–866.
Rubin, F. A basic guide to evaluation for development workers. Oxford: Oxfam
Publications, 1995.
Saba, F. “Research in distance education: A status report.” The International Review
of Research in Open and Distance Learning 12(6) (2000). [cited 17 January
2008]. Available from http://www.irrodl.org/index.php/irrodl/article/
viewArticle/4/337.%20.
Schön, D.A. The reflective practitioner: How professionals think in action. London:
Temple Smith, 1983.
Schön, D.A. The reflective practitioner: how professionals think in action. England:
Ashgate Publishing Limited, 2011.
Simpson, O. Supporting students in open and distance learning. London: Kogan
Page, 2000.
Singh, M. “The HEQC and the challenges of quality assurance in South African
higher education.” Paper presented at the Conference on Quality Assurance
and Distance Learning, Pretoria: University of South Africa, January 29 to
February 3, 2000.
South Africa. Teacher education offered at a distance in South Africa. Pretoria:
Department of Education, 1996.
Tucker, S. “Distance education: Better, worse, or as good as traditional education?”
Online Journal of Distance Learning Administration IV(IV) (Winter 2001). [cited
10 October 2005]. Available from http://www.westga.edu/~distance/ojdla/
winter44/tucker44.html .
United Nations Educational, Scientific and Cultural Organisation. Teacher education
guidelines: Using open and distance learning – technology, curriculum, cost,
evaluation. Paris: UNESCO, 2002.
University of Pretoria. National review: ACE (Education Management) UP response.
Pretoria: University of Pretoria, 2006.
_____. Distance education policy. Pretoria: University of Pretoria, 2009.
_____. Data from the Unit for Distance Education. Pretoria: University of Pretoria,
2010–2012.
Walsh, P. Global trends in higher education, adult and distance learning, 2009. [cited
7 January 2013]. Available from http://www.icde.org/filestore/Resources/
Reports/FINALICDEENVIRNOMENTALSCAN05.02.pdf.
Welch, T. and Y. Reed eds. Designing and delivering distance education: Quality
criteria and case studies from South Africa. Johannesburg: Nadeosa, 2005.
Wikipedia. “Curiosity.” 2014. [cited 10 March 2014]. Available from
http://en.wikipedia.org/wiki/Curiosity.
