Content uploaded by Duan Van der Westhuizen
Author content
All content in this area was uploaded by Duan Van der Westhuizen on Nov 20, 2016
Content may be subject to copyright.
Guidelines for
Online Assessment
for Educators
Duan vd Westhuizen
Guidelines for Online
Assessment for Educators
Duan vd Westhuizen
The Commonwealth of Learning (COL) is an intergovernmental organisation created by Commonwealth
Heads of Government to promote the development and sharing of open learning and distance education
knowledge, resources and technologies.
Commonwealth of Learning, 2016
© 2016 by Commonwealth of Learning. Guidelines for Online Assessment for Educators is made available
under a Creative Commons Attribution-ShareAlike 4.0 Licence (international): https://creativecommons.
org/licences/by-sa/4.0/.
For the avoidance of doubt, by applying this licence the Commonwealth of Learning does not waive any
privileges or immunities from claims that they may be entitled to assert, nor does the Commonwealth of
Learning submit itself to the jurisdiction, courts, legal processes or laws of any jurisdiction.
Guidelines for Online Assessment for Educators
Author: Duan vd Westhuizen
ISBN 978-1-894975-87-2
Views expressed in this publication are those of the author and do not necessarily reflect the views of
COL. All products and services mentioned are owned by their respective copyright holders, and mere
presentation in the publication does not mean endorsement by COL.
Published by:
COMMONWEALTH OF LEARNING
4710 Kingsway, Suite 2500
Burnaby, British Columbia
Canada V5H 4M2
Telephone: +1 604 775 8200
Fax: +1 604 775 8210
Web: www.col.org
Email: info@col.org
Cover: “Rhodes Park School Pupils in the School Computer Lab during a LC preparatory session - picture taken by GTP
Zambia Team 2_resize,” by IICD, is licensed under CC BY 2.0 https://www.flickr.com/photos/iicd/4445785692/
i
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
ACKNOWLEDGEMENTS
The Commonwealth of Learning acknowledges the advice and contribution of Professor Asha
Kanwar and Dr Jessica Aguti in the preparation of these guidelines. The publication was prepared
by Professor Duan vd Westhuizen at the University of Johannesburg and reviewed by Dr Janet
Welch at the University of Alberta. COL also appreciates the input of Dr Sanjaya Mishra and
various deans of education: Dr Mmabaledi Seeletso, Dr Fatuma Chege, Professor Ibrahim Olatunde
Salawu, Dr Michael Wilfred Ng’umbi, Professor Daniella Coetzee and Dr Joyce Asiimwe Ayikoru.
We thank Dr Dania Sheldon for her editing, Mr Terry Sunderland for designing and typesetting
the publication, Ms Ania Grygorczuk for her cover artwork, and Ms May Li for co-ordinating
the project.
ii GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
TABLE OF CONTENTS
Acknowledgements i
List of Figures iv
List of Tables v
List of Abbreviations and Acronyms v
1.0 INTRODUCTION 2
2.0 EDUCATION IN THE DEVELOPING WORLD 4
2.1 Introduction 4
2.2 MDG2: Universal Primary Education 4
2.3 SDG4: Quality of Education 5
2.4 The Role of ICT in Open and Distance Learning in the Developing World 6
3.0 CLARIFYING THE CONCEPT “ONLINE ASSESSMENT” 8
3.1 Introduction 8
3.2 Benefits of Online Assessment 8
3.3 Techniques of Online Assessment 8
3.3.1 Multiple-choice tests 9
3.3.2 True-or-false items 10
3.3.3 Essays 10
3.3.4 Short-answer tests 12
3.3.5 Online games 12
3.3.6 Student journaling, blogging and wiki building 13
3.3.7 Online, digital or ePortfolios 14
3.4 Further Dimensions of Online Assessment 14
3.4.1 Learning analytics 14
3.4.2 Self- and peer-assessment tasks 15
4.0 WHAT INFORMS GOOD ONLINE ASSESSMENT PRACTICE 17
4.1 Balance between Formative and Summative Assessment Tasks 17
4.2 Authentic Learning and Assessment 17
iii
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
4.3 Development of 21st-century Skills 18
4.4 Bloom’s Revised Taxonomy 19
4.5 Consider the Dierences between Print Medium and Online Medium 20
5.0 PRINCIPLES FOR ONLINE ASSESSMENT 21
5.1 Introduction 21
5.2 Prerequisites for Implementing Online Assessment 21
5.3 Best-practice Principles for Online Assessment 22
6.0 LEARNING MANAGEMENT SYSTEMS AND ONLINE ASSESSMENT 26
6.1 Introduction 26
6.2 Moodle™ 26
6.2.1 The Assignment module 27
6.2.2 The Quizzes module 28
6.3 Google Classroom 30
7.0 ONLINE LEARNING RESOURCES 32
7.1 Sources of Pre-made Assessment Options 32
7.1.1 The OER Commons 32
7.1.2 Microsoft tools 33
7.1.3 TED Education 33
7.2 Polling and Survey Tools 34
7.2.1 TodaysMeet 34
7.2.2 Socrative 35
7.3 Social Media 36
7.4 Productivity Suites 37
7.4.1 Google Apps for Education 37
7.4.2 Microsoft Oce 38
7.5 General Tools and Websites for Online Assessment 39
8.0 CONCLUSION 41
9.0 REFERENCES 42
iv GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
10.0 APPENDICES 46
Appendix A: Definitions and Types of Assessment 46
Appendix B: Assessment Design within Curriculum Design 49
Appendix C: Traditional Authentic Learning and Assessment Explained 53
Appendix D: Bloom’s Taxonomy 55
Appendix E: General Principles for Traditional Assessment 56
List of Figures
Figure 1. Screen capture of the MoodleTM Rubric Editor 11
Figure 2. Screen capture from the Center for Online Education website 12
Figure 3. Participating level related to a user’s experience in MoodleTM 15
Figure 4. 21st-century learning 18
Figure 5. Screen capture of a MoodleTM assignment submission page 27
Figure 6. Screen capture of various types of MoodleTM quiz tools 29
Figure 7. Screen capture of the Google Classroom assignment tool interface 31
Figure 8. Screen capture of the OER Commons landing page 32
Figure 9. Screen capture of the Microsoft Education landing page 33
Figure 10. Screen capture of a sample TodaysMeet interaction 34
Figure 11. Screen capture of creating a quiz in Socrative 35
Figure 12. Screen capture of the Socrative reporting grid 36
Figure 13. Screen capture showing the use of Excel to assess student teachers
during practical sessions 39
v
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
List of Tables
Table 1. Gamification Tools 13
Table 2. Bloom’s Taxonomy and Online Assessment Tools 19
Table 3. Online Assessment Principles 23
Table 4. Options for Open-source Deployment 26
Table 5. Online Assessment Tools 40
Table 6. Other Useful Sites for Online Assessments 40
List of Abbreviations and Acronyms
COL Commonwealth of Learning
COLTEI COL Teacher Education Initiative
CPD Continuous professional development
EFA Education for All
GCE Global Campaign for Education
HEI Higher education institution
ICT Information and communication technologies
LMS Learning management system
MDG Millennium Development Goal
ODL Open and distance learning
OER Open educational resources
OWG Open working group
SDG Sustainable Development Goal
SSA Sub-Saharan Africa
TE Teacher education
TEI Teacher education institution
TES Teacher education standards
TESSA Teacher Education in Sub-Saharan Africa
TTI Teacher training initiative
2 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
1.0 INTRODUCTION
In the face of massive educational needs in the developing world, and considering the role that
information and communications technologies (ICT) and online learning can play in addressing
those needs, these guidelines aim to spark discussions and help teachers/educators plan for
integrating online assessment into their teaching and learning. Throughout this paper, the terms
teacher and educator are used interchangeably.
Assessment is a critical component of the education process. In the summative mode, assessment
results are considered reflective of the knowledge, skills and competencies of students, and these
results influence their employability. Students therefore consider summative assessments “high
stakes.” Whilst summative assessment is often referred to as assessment of learning, assessment in
the formative mode shifts the focus to assessment for learning; both are critical and should therefore
be well planned and executed. Educators use formative assessment tasks to gauge and shape student
learning through eective feedback, and these should also be used to modify teaching practices (see
Appendix A). It is therefore imperative that educators be well versed in the theories and techniques
of assessment as well as the impact that online technologies have on practice.
Education institutions in the developing world are increasingly using online learning for the
training of various professionals, so it has become important to improve educators’ capacity to use
online technologies appropriately for both teaching and assessment. Educators in these institutions
include teachers, teacher trainers, lecturers in universities and tutors in colleges. All these dierent
persons carry out various forms of teaching or training and are inevitably required to assess their
students.
These guidelines are intended to support educators in critically examining the role of online
assessment in their teaching, and to provide dierent strategies and tools that can be used for online
assessment, thereby empowering educators to make informed decisions about how they use ICT for
assessment. These guidelines should therefore:
• expand educators’ theoretical grounding of assessment;
• shift educators’ core beliefs that assessment should measure the extent to which learners can
remember and reproduce knowledge;
• help educators to set authentic assessment tasks that yield products with value in real life;
• develop an understanding of how the aordances of online tools can maximise learning;
• help educators critically examine the role of online assessment in teaching and learning;
• provide examples of dierent strategies and tools that can be used for online assessment;
and
• empower teachers and educators to make informed decisions on online assessment.
3
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
There are many dimensions by which education provision and learning can be classified. Some
important dimensions are, for example, chronology (whether teaching and learning are synchronous
or asynchronous), pedagogy (whether they are instructor-led or learner-led) and openness. It
appears that as technology continues to evolve, practitioners and researchers have not agreed on
common definitions (Moore, Dickon-Dean, & Galeyen, 2011); terms such as blended learning,
distance learning, open learning, online learning and eLearning are used interchangeably. However,
it is important to distinguish between these terms, as distinct dierences exist and these dierences
may have profound implications for the design of assessment approaches in distance, open and
traditional programmes.
This work starts with a brief overview of education in developing countries, to contextualise the
use of the guidelines. Although this document is intended to be a practical tool, it is necessary to
include some theoretical analysis of the concept of online assessment. This is given in Sections 3 and
4, together with the identification and description of relevant online assessment techniques. Section
5 presents ten principles for online assessment; these were the core focus of this work. Finally, some
tools that may be useful for teachers and educators are identified and briefly described.Section 6
references learning management systems and online assessment, whilst Section 7 covers other online
assessment resources.
4 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
2.0 EDUCATION IN THE DEVELOPING WORLD
2.1 Introduction
In this section, education in the developing world is briefly contextualised against globalised eorts
(as represented by the Millennium Development Goals, MDGs, and the Sustainable Development
Goals, SDGs) to improve the economic and social lot of people living in the developing world.
It is broadly accepted that the quality of teaching and teacher education will be instrumental in
improving living standards in countries that are plagued by poverty. In a modern global economy,
it is critical to have a labour force with appropriate skills and knowledge, and the ability to learn and
re-learn throughout life.
Teachers and all other educators today must be able to teach in the transmission mode, but
they must also be able to partner as co-learners with students, help students learn how to learn,
reconstruct their own knowledge in dierent forms, be innovative and creative, and be ICT-savvy
learners. Section 5 of these guidelines includes ten principles that will distil this point. To appreciate
the importance of transforming assessment in schools, colleges and universities, it is vital to first
examine the extent to which the developing world has met the challenges of the MDGs, and
further, how they will meet SDG4: Quality Education for All.
How did the developing world fare in terms of achieving the MDGs, and specifically in terms of
education? The next sections give brief overviews of the performance of the developing world in
terms of MDG2: Universal Primary Education (UPE) and SDG4: Quality of Education. The timid
advances made by the developing world in terms of MDG2 highlight the fact that more teachers are
needed for educating the world’s children. SDG4, on the other hand, emphasises the importance of
achieving quality education and strongly advocates for the recruitment and development of quality
teachers, who should in turn be trained in quality teacher education programmes, by quality teacher
educators.
2.2 MDG2: Universal Primary Education
According to the Global Monitoring Report (UNESCO, 2015, p. i), “we are not there yet,” and
the results after 15 years of monitoring the MDGs are “sobering.” In terms of Goal 2 (UPE),
there have been some gains: more children are enrolling for primary education than before. The
net enrolment ratios improved significantly, rising by as much as 20 per cent in 17 countries. Many
more countries are oering free basic education (15 countries in sub-Saharan Africa have dropped
school fees since 2000), and fewer children are repeating school years. School meal schemes
have reached 386 million children in 169 countries. The improvement of infrastructure has also
impacted the success of education. For example, in Mozambique, the number of primary and high
schools increased three-fold between 1992 and 2010. Significant improvement in the enrolment of
girls is also evident in the developing world (UNESCO, 2015, p. 94).
However, most countries, especially the poorest, still have a long way to go to achieve 100 per cent
rates of primary school completion. Further, in 32 countries as much as 20 per cent of children who
5
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
have enrolled in primary schools will not stay through to the last grade. By 2012, at least 58 million
children who were supposed to be in school were not. Gender, ethnicity, location and poverty
remain obstacles to education. In addition, the challenges involved in obtaining data in some
countries makes it dicult to present an accurate picture of the status of education there.
2.3 SDG4: Quality of Education
SDG4, adopted by 193 countries in 2015, commits to “[e]nsure inclusive and equitable quality
education and promote life-long learning opportunities for all.” In June 2016, the UNESCO
Institute for Statistics (UIS) published “The Indicators to Monitor the SDG4-Education 2030
Agenda.”
The People’s Action for Learning (PAL) Network conducts large-scale, citizen-led investigations
that have made “robust contributions to understanding the learning crisis” in the developing world
(PAL Network, 2012). In 2016, it is too early to gauge what progress has been made in attaining
the goals of SDG4. Yet it is significant that several developing countries, such as India, Pakistan
and some African countries (e.g., Mali, Tanzania, Kenya, Uganda and Senegal) had already begun
programmes of self-assessment from 2005 to 2012, to “influence educational policy and practice
from the ground up” (PAL Network, 2012). This particular page, and others on that website,
elaborate on this innovative approach to monitor progress in SDG4.
According to Olugbenga Adedeji and Olaniyan (2011), many developing countries face the
challenge of poorly skilled teacher educators and teachers. In some cases, teacher educators are
not trained for teacher education and they lack the pedagogical skills needed to be eective.
Consequently, they cannot model or teach appropriate pedagogies to student teachers. In addition,
insucient and inadequate teaching and learning materials, poor teacher educator support,
and inadequate continuous professional development (CPD) programmes further hamper the
improvement of teacher education in these countries (Commonwealth of Learning, 2015).
Teachers’ professional capacities and ability to eectively utilise ICT in the classroom for teaching,
learning and assessment ultimately depend on their motivation, knowledge and skills. That fact, in
turn, raises the bar for transforming teacher training. Teachers will teach as they have been taught,
and improved teacher practices in the school classroom are therefore dependent on improved
teacher education practices. Addressing improved teaching practices often involves a cultural or
pedagogical change for teachers and teacher educators, which will not happen in the short term.
Several endeavours are underway to positively influence education in Africa and other developing
countries, of which this work represents a small part of COL’s eort to improve the quality of
education. Alongside COL’s eorts have been a number of others, including the One Laptop per
Child (OLPC) project that sought to “provide each child with a rugged, low-cost, low-power,
connected laptop” at a cost of less than USD 100 each. The success of this project is a matter of
debate. The OLPC was conceived before the advent of the tablet computer, and the organisation
is now championing the concept of a USD 75 tablet computer, with a flexible LCD display. Other
organisations, such as UNESCO, developed an “ICT Competency Framework for Teachers,”
6 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
aimed at “helping countries to develop comprehensive national teacher ICT competency policies
and standards” (UNESCO, 2011, p. 1). The UNESCO framework is a clear acknowledgement of
the fact that the quality of teachers, and the quality of the teacher training institutions that train
those teachers, will significantly impact the shaping of a skilled workforce in developing countries,
which in turn will advance economic and social growth in those countries. The Open University
(UK) initiated a project called Teacher Education in Sub-Saharan Africa (TESSA). TESSA’s aim
is to develop contextualised ICT-based learning materials for developing countries. The Global
Campaign for Education and TESSA initiatives have taken a prominent role in endeavours to
improve teacher education. Yet despite the eorts of these organisations, the MDG targets were not
attained by 2015.
The Global Monitoring Report (UNESCO, 2015) specifically highlighted and explored one aspect
of this situation. This report showed that pupil–teacher ratios declined in 146 countries, and that
the quality of education improved through the increased use of assessments that met national
standards. Conversely, the report identified the “quality and availability of teacher training as a
serious challenge”; for example, less than 75 per cent of primary school teachers were trained
according to national standards. Further, the report specifically “underscores the importance of
investing in teachers, instructional materials, curriculum development, eective pedagogy, teaching
in mother tongue languages and using appropriate technology” (p. 188).
A number of important factors arise from the Global Monitoring Report that have specific bearing
on the development of the present Guidelines for Online Assessment. First, it is considered essential
that significant investments be made in teachers and teacher education. Secondly, there is a strong
emphasis on increasing the use of assessments that are nationally standardised. This gives impetus
to the broadening and deepening of knowledge and skills in assessment, and all that it encompasses
for teachers and teacher educators. Thirdly, there is a call to adopt eective pedagogies that are not
teacher-dominated and that promote critical thinking, group work and reflection. Finally, there is
recognition that technology to support learning remains a challenge for improving learning and
narrowing gaps in achievement for learners in isolated areas, or settings where learners are under-
served.
2.4 The Role of ICT in Open and Distance Learning in the Developing
World
The ubiquity of ICT in all aspects of work and personal life prompted UNESCO to develop
curricula for school and teacher education more than a decade ago, when it was acknowledged that
[i]nformation and communication technology (ICT) has become, within
a very short time, one of the basic building blocks of modern society. Many
countries now regard understanding ICT and mastering the basic skills
and concepts of ICT as part of the core of education, alongside reading,
writing and numeracy. (Anderson & van Weert, 2002, p. 3)
7
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Wright, Dhanarajan and Reju (2009) explored a number of challenges distance educators face
in developing and emerging countries. They identified limited educational resources, as well as
a lack of infrastructure and the cost of Internet bandwidth and associated equipment as further
impediments to ICT adoption for learning. According to them, greater emphasis needs to be placed
on quality assurance systems to counter negative perceptions of distance education, so as to be
responsive to the needs of students and sta. The success of open and distance education using ICT
in developing and emerging nations is dependent on the extent to which these issues are addressed.
In the developing world, the uptake of ICT has been hampered by what is commonly known as
the “digital divide.”For example, Dela Pena-Bandalaria (2007) described how using ICT in open
and distance learning (ODL) has presented significant challenges to providing quality academic
processes in the Philippines.Similarly, Mnyanyi and Mbwette (2009) noted the lack of ICT skills
amongst the general populace in Tanzania. Wright (2014) identified five key barriers to the uptake
of ICT in ODL:
1. The availability of reliable and aordable electricity.
2. The expansion of Internet connectivity.
3. The professional development of teachers and teacher educators.
4. Raising awareness of the value of teachers.
5. Implementing sustainable ICT projects.
According to the editorial in The eLearning Africa Report 2015, technology is driving change in
Africa and fuelling the economic growth of African economies (Elletson & Burgess, 2015). There
is now an urgent need for radical change; Africa is at a “tipping point.” The authors acknowledged
that education is the key to unlocking Africa’s future, and they identified the building of teacher
capacity as particularly fundamental. Burgess quoted the results of a survey in which 74 per cent
of teachers stated that they did not have sucient support in their country to increase their digital
literacy. Teachers mostly wanted to improve ICT training for teachers and students, and they
desired capacity-building programmes in ICT to be made compulsory.
8 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
3.0 CLARIFYING THE CONCEPT “ONLINE ASSESSMENT”
3.1 Introduction
Learning is a complex process comprising many dimensions and variables. Assessment, which is an
integral part of learning, is equally complex and involves many aspects and dimensions that need
to be considered before it can be comprehended fully (Arend, 2007). The general principles that
apply to the setting of face-to-face assessment tasks naturally also apply to the setting of online
assessment tasks. It is therefore expected here that the reader is already familiar with the theoretical
underpinnings of the general principles of assessment.
3.2 Benefits of Online Assessment
The most widely used traditional assessment tools are multiple-choice tests, true/false tests, short
answers and essays. In many institutions, assessment tasks are completed by using traditional pen/
pencil and paper, whilst in the online world, traditional tools are enhanced and new assessment
techniques have been created. There are now many online tools that can be used to carry out the
same kinds of assessments. Online assessment tools have several benefits over traditional pen-and-
paper tests. Some of these are:
• Test items can be randomised when the assessment is taken, so no student will have test
items appear in the same order as the student who is taking the same test on the next
workstation.
• In the case of multiple-choice questions, distractors can be randomised.
• Test items can be tagged by level of diculty.
• “Pools” of test items can be used from which the tool can randomly assign dierent test
items to dierent students.
• Some types of test items can be scored by the tool that is used, relieving the teacher or
teacher educator from that burden.
• Online tools can give immediate feedback to students.
• Online tools can perform item analyses on the test items, which will help the assessor
identify poor questions.
3.3 Techniques of Online Assessment
Listed below are short descriptions of the most common digital techniques used by educators
to assess learning, for either formative or summative assessment. In the online assessment world,
multiple content formats can be used. Whilst traditional pen and paper represent two-dimensional
questions, online tests can incorporate digital features such as audio, video, animation and
innovative item types. The opportunities this presents for dierent types of learning are staggering.
9
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
The principles of universal design for learning (UDL) in traditional assessment can also be applied
in the online assessment world. The website for the National Center on Universal Design for
Learning (udlcenter.org) gives a thorough overview of the principles of UDL.
A number of the learning management systems (LMSs) that institutions are using have a variety
of capabilities for employing dierent assessment techniques. For example, Moodle™ provides
opportunities for dierent kinds of assignments, such as essay questions, quizzes and more (see
Section 6.2).
3.3.1 Multiple-choice tests
Multiple-choice tests are a common form of online assessment. It is important to create multiple-
choice tests properly, because in online assessment, the flaws of poorly written items are potentially
more visible to the world when they are placed online. Students may also have to cope with
technical challenges, which exacerbates the impact of poorly
written questions.
A great advantage of online multiple-choice tests is that they
are easy to administer. In addition, item analysis enables the
test-setter to eliminate poorly written items. Online multiple-
choice questions are instantly marked by the software that
is used to deliver them. As the questions do not measure
writing ability, students are not disadvantaged if they have
poor writing skills in another language. The questions can,
however, measure reading ability. It is a fallacy that multiple-
choice tests are “objective.” Scoring them may be objective,
but humans create questions and formulate the response
options, which certainly involve subjective decisions.
The disadvantages of traditional and online multiple-choice tests are that they are subject to
guessing and often are time-consuming to create. In the online world, however, the reuse of
previous items becomes easier, and accordingly, the process becomes less time-consuming.
A multitude of tools exist for creating multiple-choice question items. Most LMSs contain quiz
tools. Further, a host of free and commercial tools are available online. At the time of publication,
some popular tools are:
• Withoutbook: http://www.withoutbook.com
• ProProfs: http://www.proprofs.com
• That Quizz: https://www.thatquiz.org
• PollEverywhere: https://www.polleverywhere.com
In Sections 6 and 7, we identify other tools, as well as websites where such tools can be found.
Best Practice
Well-written multiple-choice tests
can assess dierent types of content
and measure achievement at multiple
levels of learning objectives. Questions
can range from simple recall and
comprehension to more complex
levels, such as analysing a scenario,
applying principles, discriminating
between concepts, interpreting
content, judging relevance and
selecting the best solutions.
10 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
3.3.2 True-or-false items
Traditional true-or-false question items require students to indicate which of two potential
responses is true. A student therefore has a 50 per cent chance of being correct by guessing. With
an online true-or-false question, guessing can be reduced by requiring an explanation. This does
impact the ease of marking, but it is an ecient way to digitally collect additional information. The
use of negative marking for the incorrect answer can be created in some software systems, which
reduces the calculation involved when applying the same approach in a traditional exam.
As is the case for multiple-choice questions, a multitude of tools are available for generating true-or-
false question items, and these are identified in Sections 6 and 7.
3.3.3 Essays
Essays are flexible and can assess higher-order learning skills. However, they are time-consuming for
educators to score. If essays are submitted online, it may become easier to mark them using online
rubrics, or by having an online marking scheme with prepared comments or other anticipated
responses. These comments can be dragged onto the electronic essay, or new comments can
be made on the essay. However, subjectivity may be an issue during marking. One of the most
significant advantages of completing essay questions online is the ability to read from text as
opposed to hand-written responses.
Figure 1 presents a screen capture of grading an essay using an online rubric. The student would
have written the essay and uploaded it to the LMS (in this example, Moodle™). The educator
would then be able to use the “Manual Grading” method to mark the essay. This tool allows the
educator to insert custom comments. An educator can write a number of anticipated responses
and simply copy and paste these as custom comments. Alternatively, the educator can make use
of the “Rubric Editor,” which is available from the “advanced grading method management”
screen following the “advanced grading” link. The editor allows for naming the rubric form and
for adding a description, new criteria and levels of performance. Criterion descriptions should
be entered, and for each level at which performance can be set, a level definition and the marks
associated with the level should be specified.
11
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Figure 1. Screen capture of the MoodleTM Rubric Editor
(source: https://docs.moodle.org/31/en/File:gradingfrom-rubric-editor.png)
Several commercial and free tools for checking grammar are available online. Online Proofreader
(https://www.paperrater.com/free_paper_grader) is a tool that allows students (or educators)
to merely paste text into a window and view detailed statistics about errors in areas such as word
choice, grammar, spelling and referencing techniques. This gives students the opportunity to “pre-
check” their own work prior to submission. These tools
also give educators a quick way to score the technical and
language dimensions of an essay.
Other online software is available that allows for multiple
types of checking. A tool such as eMarking Assistant
(http://emarkingassistant.com/emarking-assistant/) allows
for comments in multiple formats and basic plagiarism
detection. Many of these products have free trials or low-
cost licences for educators.
Best Practice
Another popular type of tool is a
plagiarism checker. Using a product
such as TurnitIn™ will compare student
work with other works and provide a
similarity report. Best practice for many
institutions is to ask students to submit
the similarity report when they submit
their work. This gives the student an
opportunity to address any similar
content themselves prior to submission.
12 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
3.3.4 Short-answer tests
These test items require the student to fill in a word or phrase
in response to a direct question, or to enter a word or phrase
that was left out of a statement. The advantage of having
students take this type of examination online is that the
answers can be scored immediately by comparing the student
response to a pre-populated answer. The disadvantage is
that often students will not type the exact answer and will be
marked wrong. Spelling mistakes, for example, can cause an
incorrect grade, whereas the teacher would have accepted the answer.
As is the case for multiple-choice items and true/false items, a multitude of tools are available for
generating short-answer question items, as will be discussed in Sections 6 and 7.
3.3.5 Online games
Online games oer exciting assessment opportunities. They can provide a safe, creative environment
in which students can learn to experiment, collaborate and solve problems. They can be used in
almost all educational disciplines for a variety of assessment types. In 2016, the Center for Online
Education produced a list of 50 websites with game resources for teachers, as shown in Figure 2.
At the time of writing these guidelines, the list is available through http://www.onlinecolleges.
net/50-great-sites-for-serious-educational-games/.
Figure 2. Screen capture from the Center for Online Education website
Recently, the term “gamification” entered the educational discourse.
Gamification is the use of gaming principles in the field of education
in order to get students involved, engaged, and excited about learning.
Gamification introduces concepts like badges, levels, achievements, and
game points to the classroom. Students are rewarded with these concepts
when they succeed, but are not penalized when they don’t. By introducing
a system of rewards without harsh penalties, students are not afraid to step
outside of their comfort zone and fail. By removing their fear of failure we
subconsciously encourage them to learn. (eLearning Industry, 2016)
Best Practice
For assessment where the students
write their response to a question,
it is best practice not to release the
grades to students immediately. The
teacher should check each incorrect
answer against the supplied answer for
accuracy and override any mistakes.
13
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Table 1 gives examples of some of the gamification online tools.
Table 1. Gamification Tools
Title Website Description
Room Recess www.roomrecess.com Free educational games for elementary students.
Youtopia http://www.youtopia.com/info/ Uses points, badges and leaderboards. Applicable across K through 12.
Open Badges https://openbadges.org/ A project by Mozilla, this free software can create, issue and verify digital
badges.
Minecraft https://education.minecraft.net/ This free system is an open game for anyone in the world promoting
creativity and problem solving. Mostly used in higher grades, but has
been used in grades as low as two and three.
Second Life http://secondlife.com/ An immersive which allows for opportunities to engage in teaching,
collaboration, community building and assessment.
3.3.6 Student journaling, blogging and wiki building
Many LMSs provide functionalities for student journaling, blogging or wiki building. There are also
tools freely available on the Internet.
Outside of your LMS, journaling is a particular useful tool for encouraging student reflection,
and teacher educators may elect to assess the journal entries by using a rubric. The Centre for
the Enhancement of Teaching and Learning provides detailed information on the assessment of
journals, at http://ar.cetl.hku.hk/am_rj.htm (Chan, 2009).
Specific guidelines for journal entries may be given (structured
journal), or students can create free-form journals. Journals
are typically private, visible only to teaching sta and the
individual student. Journals can be onerous to mark when
large numbers of students are involved.
Building a blog or wiki is an important strategy for encouraging individual or collaborative student
writing. Think of a blog as a website organised by posts according to the date the posts were made.
Think of a wiki as a website that organises by content (such as Wikipedia does). Depending on your
desired outcome, either or both can be chosen in a learning environment.
Blogging is similar to journaling, except that there are more features for providing access to the
information. One or more students can build private, semi-private or public blogs. Blogs can be
assessed in the same manner as journals. Blogs are a particularly useful tool for teacher educators
and teacher students when they are developing a professional online identity (POI). Many teacher
education programmes are now including the development of a POI for each student teacher as an
outcome of the programme.
Wikis are also tools that can be created by one or more students and can be constructed as private,
semi-private or public. Wikis are particularly useful for collaborative group projects and are
conducive to social constructivist learning. Wikis are non-linear, evolving, complex, networked
online resources with multiple authors, and they can be used to support student collaboration
and co-production. At the time of writing this document, Internet Techies (2015) has listed ten
free wiki software platforms that educators may consider for developing collaborative knowledge
making: http://www.clickonf5.org/7599/10-free-opensource-wiki-software-engine.
Best Practice
To maintain student engagement
with the journal, it is important to
give regular and timely feedback.
14 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
3.3.7 Online, digital or ePortfolios
Typically, a portfolio is a collection of student work that is organised, reflected upon and presented
to show content comprehension and learning growth over a period of time. Reflections on the
individual artefacts as well as the overall portfolio are critical components of this assessment tool.
Portfolios enable deeper learning for students, and educators gain a better understanding of their
students’ knowledge and skills.
Online portfolios can be constructed using a variety of ICT tools –– for example, Evernote, or
the open-source tool Mahara (https://mahara.org/), the latter of which integrates with Moodle.
These tools allow students to write documents and upload photos, audio and video. All content
can be tagged and, if necessary, shared to other media tools. Evernote is versatile in that it exists
in both computer and mobile app formats, and content can be synchronised across multiple
computers and mobile devices. Basic websites, blogs and wikis can also be used as online portfolio
tools. The most significant advantage of creating a digital portfolio is the ability to include a variety
of content. ePortfolios, for example, can contain movies, audio, presentations, text, hyperlinks and
animations.
The use of online ePortfolio tools is strongly aligned with the upper level of Bloom’s revised
taxonomy; this sets outcomes at the “Create” level, where students are expected to be creating,
composing, constructing, designing, generating, inventing and producing.
3.4 Further Dimensions of Online Assessment
The use of online assessment tools exploits the aordances of online tools in several other ways that
are directly linked to assessment practices. Two of these are learning analytics and the facilitation of
self- and peer-assessment.
3.4.1 Learning analytics
Many educational organisations use LMSs such as BlackBoard™ (a propriety, paid-for tool) or
open-source systems such as Moodle™ or Canvas™. Embedding the student learning experience
in an LMS allows the teacher to monitor data such as the amount of time spent online and
which pages were accessed. This enables educators to award a “participation mark” as part of the
assessment scheme in a programme. Whilst this is not an exact science, many educators use it as
one tool with which to measure student engagement.
This introduces the concept of “learning analytics.” The Society for Learning Analytics and
Research (SoLAR) defines this as “the measurement, collection, analysis and reporting of data
about learners and their contexts, for purposes of understanding and optimizing learning and the
environments in which it occurs” (Khalil & Ebner, 2016). Learning analytics are particularly useful
for identifying students who are at risk of failing a programme. Although this is strictly speaking
not a method of assessment per se, it remains a useful tool for assessing learner engagement.
15
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Analytics can be gathered for any website. This means that without an LMS, it is still possible to
monitor trac and activity in course materials. Organisations that go this route really need to have
the technical expertise available to set up the sites to collect relevant data. Google Analytics is a
useful tool for this.
Mattingly, Rice and Berge (2012) explained that most LMSs contain built-in tools to perform
learning analytics. In Moodle, for example, a number of “plug-ins” can be added to help educators
see lists of activities, statistics, course participation, etc. Figure 2 is a screen clip of the participation
of a user in a Moodle course.
Figure 3. Participating level related to a user’s experience in MoodleTM
(source: https://docs.moodle.org/31/en/Logs).
At Purdue University, educators began employing the Purdue Early Warning System, using
learning analytics, to identify “at-risk” students, combining the LMS tools with Google Analytics
and institutional management information systems. Data were exported to Excel or to statistical
software tools to identify trends in performance, spot outliers and anticipate who may be at-risk
students.
3.4.2 Self- and peer-assessment tasks
The primary aims of self- and peer-assessment are to:
• increase student responsibility and autonomy;
• achieve a more advanced and deeper understanding of the subject matter, skills and
processes;
• elevate the role and status of students as assessors;
• encourage a deeper approach to learning;
• involving students in critical reflection; and
• develop in students a better understanding of their own subjectivity and judgement.
16 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Many LMSs provide functionality for self- and peer-assessment. Students may individually or in
groups submit assessment tasks. The LMS will then randomly allocate submitted assignments
to a predetermined number of students. Such an allocation should be accompanied by a rubric.
For example, each student may be allocated two other submissions to assess as well as their own.
The system will then average the allocated marks. The peer-assessment process may be done
anonymously or not. One interesting student observation is that if the grading is anonymous, they
cannot seek further details from the reviewer. Continuing the conversation between students is an
important part of formative assessment.
There are other collaborative tools outside an LMS environment that can facilitate self- and
peer-assessment in real time. Products such as Google Groups and Google Hangouts, Microsoft
OneDrive/SharePoint, blogs or Edmodo are some examples.
“Nirvana School 6th IT,” by Johan Bichel Lindegaard, is licensed under CC BY 2.0 https://www.flickr.com/photos/accidentdesigns/5410455318/
17
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
4.0 WHAT INFORMS GOOD ONLINE
ASSESSMENT PRACTICE
It is impossible to separate assessment from curriculum. Assessment is important because it has
a strong impact on learning and is an indicator of the quality of a programme, as well as of its
teaching and learning activities. The theoretical perspectives presented in Appendix B (Assessment
Design with Curriculum Design) assume an alignment between pedagogy and assessment. For
example, deciding on an assessment strategy is an essential part of the curriculum planning process.
The assessment strategy adopted must provide evidence that the programme purpose is being met.
In addition, much of the curriculum is informed by assessment and is designed to focus on those
aspects of the curriculum most closely associated with the assessment requirements. What follows
now are areas specific to good online assessment practices.
4.1 Balance between Formative and Summative Assessment Tasks
Several experts have described the uses and limitations of summative and formative assessments
(Perie, Marion, Gong, & Wurtzel, 2007). Summative assessments, which assess “learning,” fail
to provide educators with timely information on how to adapt their teaching or what content to
re-teach. Although summative assessments may be useful for ranking and comparing educational
schools (if the same summative assessment is used), they may yield disaggregated data that were
produced out of context. Whilst these may be helpful for adjusting future teaching regimes, current
students do not benefit from such analyses. Where summative assessments are delivered online, the
foundation of a summative assessment can be altered. Having the summative assessment online
can allow it to be delivered at any time, in any place and in any space. This added functionality may
change how educators approach summative assessment.
Formative assessments, in contrast, are administered frequently
by teachers during a learning unit to assess student learning as it
happens. Used eectively, formative assessment provides information
that helps the teacher adjust teaching and learning activities to
improve learning. Formative online assessments oer the same
advantages as traditional tools but can also take on new forms.
Examples include the use of multimedia and the ease with which multiple submissions are possible.
4.2 Authentic Learning and Assessment
Authentic assessment is an increasingly popular approach, as it allows for a more holistic assessment
of student abilities. Appendix C (Traditional Authentic Learning and Assessment Explained)
describes authentic assessment in more detail. Herrington, Reeves and Oliver (2014) have
explained that “authentic learning is a pedagogical approach especially appealing to educational
technologists.” The given learning tasks use knowledge situated in realistic contexts. These realistic
tasks can potentially cognitively challenge student teachers to solve problems and to think in the
same ways that experienced teachers do. In addition, the complex tasks implicit in the approach
Best Practice
Oer learners practice
opportunities with sucient
online self-tests.
18 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
require the creation of real products and innovations, are immediately usable in a school classroom
and are therefore more worthy than de-contextualised assessment tasks.
4.3 Development of 21st-century Skills
Jerald (2009) described how the increasing permeation of ICT into all spheres of society in
the 21st century has necessitated a rethink of the traditional curriculum. “Schools must provide
students with a broader set of ‘21st century skills’ to thrive in a rapidly evolving, technology-
saturated world” (p. 1). To be successful in both their careers and their personal lives, students
must better understand how to apply what they learn in school subjects to deal with real-world
challenges. Teachers cannot simply require the reproduction of information during assessment
tasks, as doing so will not suciently prepare students for the kinds of tasks they will face in the
real world. Students need to develop a broader set of competencies: the ability to think critically
about information, solve novel problems, communicate and collaborate, create new products and
processes, and adapt to changes. The Partnership for 21st Century Learning (www.p21.org) has
classified these skills (see Figure 4).
LEARNING AND INNOVATION SKILLS
4Cs
Critical Thinking • Communication
Collaboration • Creativity
LIFE
AND
CAREER
SKILLS
• Flexibility and
adaptability
• Initiative and
self-direction
• Social and
cross-cultural skills
• Productivity and
accountability
• Leadership and
responsibility
“ Develop social and
emotional competencies
for life and career”
KEY SUBJECTS
English, reading, language
arts, world languages,
mathematics, economics,
science, geography, history,
government and civics
21STCENTURY THEMES
Global awareness,
including literacy in
finance, economics,
business, entrepreneurship,
civics, health and the
environment
INFORMATION,
MEDIA AND
TECHNOLOGY SKILLS
• Information literacy
• Media literacy
• ICT literacy
“ To be able to be eective
in the 21st century, workers
must be able to create,
evaluate and eectively
utilise information, media
and technology.”
21STCENTURY SUPPORT SYSTEMS
21st-century Standards • Assessment of 21st-century Skills
21st-century Curriculum and Instruction • 21st-century Professional Development
21st-century Learning Environments
Figure 4. 21st-century learning (adapted from Partnership for 21st Century Learning, 2016)
19
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
The question therefore arises: How can teacher educators address the needs of student teachers
to prepare them to meet the challenges of 21st-century schools? According to Rotherham and
Willingham (2009), “new assessments that can accurately measure richer learning and more complex
tasks” are needed, and “education faces enormous challenges in developing the ability to deliver these
assessments at scale” (p. 17). A comprehensive rethink of assessment strategies and tasks that include
the measurement of 21st-century skills is necessary. The P21 initiative recommends that:
• assessment systems be based on multiple measures of students’ abilities, including 21st-century
skills;
• assessment of 21st-century skills be listed as an integral part of the academic assessments in
math, reading and science;
• reporting requirements be expanded to include information on whether the student is
achieving 21st-century skills; and
• therefore, in addition to setting assessment tasks that only measure knowledge or skills about
the content of programmes, assessment criteria should deliberately evaluate the extent to
which some 21st-century skills have been attained.
4.4 Bloom’s Revised Taxonomy
In 2001, Bloom’s Taxonomy was revised to align with 21st-century learning environments. See
Appendix D for more detail on Bloom’s Revised Taxonomy.
When online assessment tasks are developed by teacher educators, care should be taken to ensure
that the range of cognitive skills are assessed, using some of the keywords listed in Appendix D. Of
particular importance is that the ultimate level at which student outcomes can be set is the “Create”
level (see Appendix D). At this level, student learning outcomes are geared towards creating new
meanings or structures. Key words associated with this level are “combine,” “compose,” “plan,”
“generate” and “design.”
A number of tools are appropriate for the evaluation of 21st-century learning skills. The website of
Cathy Schrock is particularly useful for educators (http://www.schrockguide.net/bloomin-apps.
html). Schrock aligns each of the cognitive levels of Bloom’s taxonomy with tools for dierent kinds
of mobile devices –– i.e., for IOS devices, Android devices, Windows devices and Google Apps tools.
Table 2. Bloom’s Taxonomy and Online Assessment Tools
Bloom’s Cognitive Level Online Tool
Remember YouTube, Wordle, Diigo, Oce applications, Google Notes, Listmaster, MindMeister, PowToon, TED-Ed
Understand Skitch (iPad), Sketchbook Express, Blogspot, Twitter, Mindnode
Apply Tape a Talk, Cartoon Drawer, Adobe Connect, Picasso, Wiki, Smartsheet, Weebly
Analyse Spreadsheet tools, Evernote, MindMash, Wufoo
Evaluate Skype conferencing, Pro’s and Con’s, Socrative, Snopes,
Create Presentation tools (Prezi), YouTube, Screenr, Google Sites, Wix, WordPress
20 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
The list presented in Table 2 is representative of all possible tools. New tools become available daily,
and depending on the assessment task, most can be used for setting assessment tasks that cover the
range of cognitive levels as espoused by Bloom’s revised taxonomy. See Sections 6 and 7 for a list of
other possible assessment tools.
4.5 Consider the Dierences between Print Medium and Online
Medium
The advent of modern computer monitors, and the clarity that they provide, can suggest that humans
read text presented in the digital realm in the same way as they would read printed text. However,
reading digital and printed text are not the same processes.
During a typical reading situation on web pages, the reader is generally unable to see the physical
amount of text available. Yet at the same time, the reader is able to access multiple sources, using
several tabs, in dierent designs and layouts. That eciency of access is impossible in the print
environment.
Whilst the major cognitive processes involved in reading from print and digital media are the same, an
additional challenge exists in the digital medium. Navigation is required, and other technical obstacles
may interrupt the reading process (for example logging into a website or downloading a plug-in).
Knowledge of navigation tools (e.g., hyperlinks, tabs, menus, the “back” button) are prerequisites for
being digitally literate. In contrast, navigation through printed resources is relatively easy, linear and
familiar.
Online access and retrieval tasks require readers to search for information in a more abstract space than
in printed books or documents, without seeing the full text that may be available. In the online world,
search tools and menus on web pages give clues to the reader as to what the content may be about. In
the print medium, tables of contents and indices fulfil a similar function.
When students are expected to integrate and interpret tasks that require them to compare information
from dierent locations (thus using multiple texts, and perhaps diverse text formats), and because the
texts are usually not visible simultaneously (unless students have multiple screens), they must rely on
their short-term memory to perform these tasks.
The credibility of printed resources is often not in question. In contrast, when reading online, the
student must also assess the credibility of the content, given that online publishing is so simple.
It is therefore prudent for teacher educators to consider these
dierences between the mediums. When students are assessed
in the online environment, they are simultaneously being
assessed for their digital skills, and any deficiency in these skills
may very well be an impediment to performance. In order to
ensure that web pages are well designed, the Essential Guide to
Visual Design (Articulate Global Inc., 2016) may be useful for
teachers who want to design appealing web pages that support learning via page design.
Best Practice
Good visual design regarding colour,
contrast, repetition, alignment and
balance are key determinants to
support comprehension.
21
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
5.0 PRINCIPLES FOR ONLINE ASSESSMENT
5.1 Introduction
In Section 2, we established a context for education in the developing world by referencing the
Global Monitoring Report and the extent to which countries in the developing world had achieved
the MDGs. In particular, we noted that quality education requires quality teachers and teacher
educators. The case was made that teachers and teacher educators may benefit by learning more
about assessment, as improved assessment practices support quality education. Finally, a case was
further made that in the developing-world setting, ICT can contribute to quality education. Given
the constraints and challenges in education that much of the developing world faces, we identified a
set of theoretical constructs deemed appropriate to inform online assessment. General principles for
traditional assessment guidelines can be found in Appendix E.
It needs to be understood that online assessment principles cannot be enacted separately from
curricula. Any curriculum should espouse pedagogical approaches that are congruent with these
principles. The principles presented here and in Appendix E assume a pedagogy that is not driven
by so-called transmission models of teaching. Therefore, “teaching for assessment” may, in this case,
be a good thing, as the principles oered here represent a pedagogy that is aligned with the needs
of modern societies. Such societies require their graduates to be digitally fluent, creative, responsive
to fast-paced and ever-changing contexts, and “globally flexible.” These principles mark a move
away from content replication, transmission models of teaching and rote learning.
It is acknowledged that not all online assessment tasks can always be congruent with the online
principles presented here and the general principles in Appendix E. Therefore, all assessment tasks
in a programme should be considered as a whole when programmes are evaluated, to determine the
extent to which they reflect these principles.
5.2 Prerequisites for Implementing Online Assessment
It is prudent at this point to identify the conditions under which online assessment in teacher
education can be viable. Two dimensions of primary importance are institutional readiness and
teacher educator readiness.
In terms of institutional readiness, it will be impossible to utilise online assessments if institutional
policies, resources and practices are not supportive of such endeavours. Therefore, the successful
implementation of online assessment depends on:
• supportive institutional policies for ICT integration and ICT use for assessment;
• access for students and sta to devices and technologies that are appropriate for the setting
within which the users are situated;
• reliable access to the Internet for using the plethora of web-based tools that are available for
online assessment;
22 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
• sustainability plans and strategies for ICT integration, and access to devices and the
Internet;
• capacity building for sta in ICT integration and online assessment, by means of
professional development activities;
• encouragement of academic sta to use ICT tools, by including the use of such tools as
performance indicators during performance assessment;
• sucient technical support for academic sta and teachers using online assessment tools;
• ICT integration in the curricula of programmes, and the modelling, coaching and
preparation of teacher educators in the use of dierent online assessment tools;
• policies that provide for the earning of final marks in courses not only from summative
assessment tasks, but also from formative assessment tasks.
Although institutions can have excellent policies, supportive structures and top-notch
infrastructures, without teacher-educator or teacher “buy-in,” it is unlikely that entrenching online
assessment and reaping its benefits will succeed. Therefore, institutional leaders should find ways
to “turn heads” around ICT integration and online assessments. Whereas prescriptive policies may
go some way in moving teacher educators towards using online assessment and teaching student
teachers to use online assessment tools, the full realisation of eective online assessment will
only ensue when: (i) educators are fully convinced of its benefits, (ii) they have a well-developed
pedagogical philosophy that is neither teacher-centred nor content-centred, (iii) they have a deep
knowledge of sound assessment practices and (iv) they have the necessary ICT know-how to use the
ever-increasing pool of online assessment tools that are available.
5.3 Best-practice Principles for Online Assessment
Assessment best-practice principles that were developed for traditional assessment also apply, as
already discussed in section 4. What follows are ten online principles that apply to digital forms of
assessment.
23
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Table 3. Online Assessment Principles
Online
Principle Description Example
1Longitudinal Reflection. Online formative assessment
tasks are used diagnostically to improve teaching practices
and learning tasks. Teacher educators should use the digital
outcomes of assessment tasks as an opportunity to collect
data and reflect on teaching practices.
Using a journaling tool or ePortfolio to
reflect learning over a longer span of time.
2Higher-quality Feedback. Online assessment tasks will best
support learning when they are accompanied by feedback
that is timely, suciently detailed and constructive. Feedback
on formative assessment tasks has been shown to have a
significant influence on learning success.
The use of pre-formulated responses to
essay assessment tasks makes for more
ecient marking. In addition, educators
can give generic feedback to an entire
group in an online learning environment,
as well as individual feedback to students.
3Readymade Tools. The performance criteria for online
assessment tasks should be made explicit by the use of
rubrics or assessment standards.
The use of online rubrics is particularly
useful here. Rubrics should be made
available to students beforehand to guide
their learning.
4Technology-enabled Authentic Learning. Online
assessment tasks should be characterised by having real-life
value. These are preferable to polished products resulting
from problems that are ill-defined, require understanding
of the complexity of problems that often cut across several
disciplines, and for which multiple solutions exist.
Students can be required to use a multitude
of online tools to present their solutions to
authentic problems –– e.g., a spreadsheet
for data analysis, a presentation tool
for presenting solutions to appropriate
audiences, and a word processor to write
detailed reports.
5Enhanced Collaboration. In the range of assessment tasks
in a programme, include online assessment tasks that require
discussion and collaboration amongst students. Social
constructivist approaches to learning have always been
regarded as beneficial to learning.
Several online tools facilitate collaboration
–– e.g., discussion tools in LMSs, or
other collaborative tools, such as Google
Hangouts or Skype meetings.
6Exploit a Variety of Techniques. A variety of online
assessment techniques and tools are used in programmes.
Variation within assessment will allow for more accurate
measurement.
Section 3.3 details the variety of assessment
techniques that are possible using online
tools.
24 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Online
Principle Description Example
7Address Diversity. Well thought out assessment strategies
should take into account student diversity so as not to
advantage or disadvantage any group of students. Types of
diversity that may exist amongst students include:
• socio-economic status;
• access to devices;
• access to and stability of Internet connection;
• racial, ethnic, cultural or tribal dierences;
• learners with special educational needs (both cognitive and
physical);
• second-language English speakers;
• stable electricity supply;
• cultural variables: the customs of dierent cultural groupings
may be vastly dierent, even oppositional, and these
disparities can encompass the use of imagery, colloquial
expressions, forms of greeting, metaphors and humour.
Educators should consider issues of diversity
when selecting online assessment tasks.
For example, in some areas, web-based
assessments may not be possible but mobile
options may very well be. A text-based
collaborative tool such as WhatsApp may be
very useful.
The website “Can software be racially
biased?” at http://www.cheatsheet.com/
gear-style/can-software-be-racially-biased.
html/?a=viewall is a handy reference for
educators who wish to check whether their
online resources may be racially biased.
Project Implicit (https://implicit.harvard.edu/
implicit/selectatest.html) provides a handy
series of tests whereby educators can test for
personal biases that they may unknowingly
have and that may impact their setting of
assessment tasks.
Several assistive devices are available
for students with physical or learning
impairments. The webpage “Assistive
Technology Basics” (https://www.
understood.org/en/school-learning/assistive-
technology/assistive-technologies-basics)
provides more information about this.
Assistive technologies exist for reading,
writing, text-to-speech and devices for the
physically less abled.
Technology tools to assist English second-
language speakers should have two
functions: listening and speaking. The
webpage “Technology in the Foreign
Language Classroom” (http://study.
com/academy/lesson/technology-in-the-
foreign-language-classroom.html) contains
handy information for this dimension of
accommodating diversity in assessment.
25
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Online
Principle Description Example
8Prepare IT and People. Sucient systems should be in place
for supporting online assessment tasks. This is an institutional
imperative to procure sucient supportive mechanisms. It
may include the development of digital skills programmes for
sta and students, supplying sta and students with devices
and Internet access, and providing professional development
opportunities for sta members to learn about ICT integration
and online assessment. See section 5.2. for details on
prerequisites.
See section 5.2.
9Monitor and Remediate. The results of online assessment
tasks should be used to identify “students at risk.” Most LMSs
have “evaluation” tools that allow teacher educators to identify
students at risk. These tools examine the time students spent in
online modules, or the number of tasks they have completed,
or the marks that they have obtained for assessment tasks, to
identify “at-risk” students. Data mining and learning analytics
are useful tools to identify students at risk of failing; they allow
teacher educators to proactively intervene and take corrective
measures.
10 Web Design Best Practices. Webpages that contain online
assessment activities should follow best-practice web design
principles. More information can be found on the World Wide
Web Consortium’s (W3C) website, www.w3.org.
26 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
6.0 LEARNING MANAGEMENT SYSTEMS AND
ONLINE ASSESSMENT
6.1 Introduction
In this section, a number of LMSs are identified. Each has built-in functionalities to assist teachers
and teacher educators in developing high-quality assessment aligned with best practice. There
are literally hundreds of LMSs available today. Some are open source, such as Moodle™, Canvas,
Sakai and aTutor. In addition, Software as a Service (SaaS) or
cloud-based LMSs are available via a “pay-per-usage” financial
model –– e.g., Google Classroom, Joomla and WizIQ. There
are also proprietary, licensed and purchased systems, such as
BlackBoard, EasyCampus and Go1.
6.2 Moodle™
Moodle™ fits in a class of LMSs that are considered open source. Moodle™ appears to be a popular
LMS and is in use in a number of developing countries, so it is useful to discuss its assessment
functionalities. Appendix F explores Sakai, which is another common open-source LMS.
One of the prevailing questions about using open-source software is when the “freemium” payment
model applies. It is often the case that the basic elements of the tool are free, but extra options need
to be paid for, or the number of student accounts are limited in free versions.
Also, considerable expertise is needed to set up, update and fine-tune open-sources systems. There
are private companies that may be contracted to set up such systems, or that even have dedicated
servers for those systems and are willing to customise, but this may be costly. At least one qualified
IT specialist needs to be employed at the institution, regardless of the system, and a separate server
and hardware must be purchased. Some Internet hosting companies may oer hosting services for
open-source systems. Table 4 summarises the options for deploying an open-source solution.
Table 4. Options for Open-source Deployment
Options for Open-source Deployment
Open-source Platform
(free) + Maintenance ($)
Development on Open-
source Platform ($)
Hosting ($) Support ($)
Internal Resources Internal cost for
maintenance
Can be done by internal
programmers
Can host on internal
equipment
Can hire people
internally to do the
support
External Resources External cost for
maintenance
Can be contracted out to
programmers
Can subscribe to a
hosting company
Can contract to an
external company
Best Practice
LMSs can be used to support online or
face-to-face course determinants that
facilitate comprehension.
27
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
For assessment in Moodle™, the institution can choose to use any one of a number of modules.
The Assignment, Quiz, Survey, Workshop and Choice modules are the most used. Other modules,
such as the Forum and the Journal, can also be used for assessment.
6.2.1 The Assignment module
Assignments are assessment tasks with a due date and associated with a specific grade. Students use
this tool to upload files representative of the assessment task. Data about each upload are recorded.
Teachers can view uploaded files and mark the submitted work online. Alternatively, a teacher can
choose to download all of the assignments for oine marking. Moodle™ can be set to send the
student a notification of the mark allocated. Some common
features enabled in the Assignment module are:
• automated due dates
• maximum grade indicated
• late assignments permitted (teacher discretion)
• assignment linked to master gradebook
• commenting feature (with email notification)
• resubmissions and regrading allowable
• student work date-stamped
Figure 5 is an example of what a student would see when submitting an assignment using the
Moodle™ Assignment tool. Such assignments can be accompanied by online rubrics made known
to students before or after submission.
Best Practice
Facebook allows the uploading of
images, sound clips and video clips,
which can easily be used to spark
discussion and deepen learning.
Figure 5. Screen capture of a Moodle Assignment submission page
(source: http://distance.uvic.ca/onlinehelp/tutorials/moodle/assign.htm)
28 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
6.2.2 The Quizzes module
The use of frequent quizzes during class time has proven positive for student achievement,
attendance and confidence. It maintains student study eort and promotes course engagement. It
enables practice to help students evaluate their learning and focus their study eorts accordingly.
In addition, quizzes can give students prompt evaluative feedback. In Moodle™, a number of quiz
types are available:
• Calculated questions. These are handy for mathematics or science education. They allow
educators to create individual numerical questions using “wildcards.”
• For a question such as, “Calculate the area of a rectangle,” the tool will randomly
assign values for {base} and {height}, and no student will encounter the same
values to calculate.
• Essay questions. Essay questions must be manually reviewed and graded by the educator,
who can also elect to print out essays; several online marking tools exist to assist with on-
screen marking.
• Matching questions. Matching questions present a list of items or statements which must
be correctly matched against another list of statements –– for example, “Match the capital
city with the country.” One list could contain “Uganda, Malawi, Botswana, South Africa”
and “Kigali, Blantyre, Lilongwe, Gaborone, Cape Town, Pretoria.” Each match is equally
weighted to contribute towards the mark for the total question. Additional “distractors”
can be given.
• Embedded answers (close test / gap fill). This type of question draws on external tools
such as Hot and consists of text (in Moodle™ format) that has various answers embedded
within it, including multiple-choice, short answers and numerical answers.
• Multiple choice. These questions allow only one answer in response to a question or
statement by providing radio buttons next to the answers. An option exists whereby more
than one option oered can be correct.
• Short answer. Students must type in a word or phrase in response to a question. Answers
may or may not be case sensitive. Answers can be a single word or a phrase. Answers must
exactly match a list of acceptable answers. The use of “wild cards” expands the options of
acceptable answers.
• Numerical. Numerical answers can be set to accept an error. This allows a fixed range of
answers to be assessed as a correct answer. For example, if the answer is 45, an accepted
error of 5 can be set, so any number between 40 and 50 will be accepted as correct.
• True/False. In this type of question, a student is given only two choices to select from:
true or false. The question content can include an image or html code or require an
explanation.
29
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
In the majority of these question types, feedback can be enabled, and the appropriate feedback
message is shown to the student after the answer, whether a correct or incorrect response was
selected. This is particularly useful in formative assessment tasks.
Figure 6 shows a screenshot from the Moodle™ Quiz environment.
Figure 6. Screen capture of various types of Moodle™ quiz tools (source: Simmons College, 2016)
This quiz module allows users to design and set quiz tests that are kept in a categorised database
and can be reused within courses and/or between courses. Students can be allowed multiple
attempts at quizzes. Each attempt is marked by the system, and the teacher educator can choose
whether feedback is given and whether the correct answers are shown. The other features of the
Quiz module are:
• Teachers can define a database of questions for reuse in dierent quizzes.
• Questions can:
• be stored in categories for easy access, and these categories can be “published” to
make them accessible from any course;
• be automatically graded and can be re-graded if questions are modified;
• be shued (randomised) to reduce cheating, and answers can also be shued;
• allow HTML editing and the addition of images;
• be imported from external text files;
30 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
• have embedded descriptive text and graphics;
• be edited without opening a quiz (categories can as well);
• import images in GIFT format.
• Quizzes can:
• have a limited time window outside of which they are not available;
• be attempted multiple times and show feedback and/or correct answers (at the
teacher’s discretion);
• have cumulative attempts, if desired, and finished over several sessions;
• have a detailed quiz report for statistical analysis.
• Short-answer questions can use wildcards providing answer alternatives.
• Images work in questions from external published categories.
• Overview reports show more information than earlier versions of this module and allow
deletion of attempts.
Both teacher–student assessments and peer-to-peer assessments are supported.
6.3 Google Classroom
Google Classroom is considered a cloud-hosted introductory LMS. It has been designed to work
with the power of Google Apps for Education (see Section 7.4.1) and oers several free services to
academic institutions. As it is a “cloud” solution, the institution does not have to cover the cost of
infrastructure for hosting the application. This may present privacy and security problems for some
institutions. For more information, go to https://www.google.com/edu/.
According to the Google website, Google Classroom is “mission control, designed with teachers
and students to connect the class, track progress and achieve more together.’’ Google Classroom
is included in the Google Apps for Education environment and oers basic LMS functionalities.
It helps teacher educators and teachers create and organise assignments quickly, provide feedback
eciently and communicate easily with their classes, and it promotes a paperless learning
environment. In addition, many other online third-party apps seamlessly integrate with Google
Classroom.
Figure 7 is a screenshot of a Google Classroom interface for viewing submitted assignments.
31
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Figure 7. Screen capture of the Google Classroom assignment tool interface
(source: Classthink, 2014)
32 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
7.0 ONLINE LEARNING RESOURCES
Learning management systems oer several options for online assessment. However, a plethora
of free or low-cost options for assessment are available from the Internet. Some of the tools listed
in Table 5 in section 7.5 are completely free to use, whilst others may have some functionalities
disabled in the free version but are still very useful. It is important to consider the bandwidth
consumption of the tools in contexts where bandwidth may be problematic. The descriptions below
are not intended to be manuals or advertisements for the tools; rather, they introduce a variety of
resource types that can be used with online assessment.
7.1 Sources of Pre-made Assessment Options
7.1.1 The OER Commons
The OER Commons (2016) claims to be
busy around the clock adding new educational resources to OER Commons.
We work with the finest producers of instructional content in the world
and gather their best work together especially for you. Then we ensure the
resources are carefully described and fully indexed because we want you to
be able to find exactly what you need, when you need it.
The OER commons web environment (www.oercommons.org), as shown in Figure 9, contains
several resources that are useful for online assessment. For example, a search in the Educational
Assessment area (conducted May 2016) yielded 1,563 assessment resources, with extensive
coverage across several disciplines.
Figure 8. Screen capture of the OER Commons landing page(source: https://www.oercommons.org)
33
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
7.1.2 Microsoft tools
Microsoft, like many other companies, is making significant investments in education and has
released several online assessment tools. The website found at https://education.microsoft.com/
and shown in Figure 9 facilitates access to a wealth of Microsoft resources and tools that educators
and teachers may find useful for learning about online assessment. The Microsoft Innovative
Educator (MIE) Expert programme was created to recognise global educator visionaries using
technology to pave the way for their peers for better learning and student outcomes. The MIE
experts work closely with Microsoft to lead innovation in education, advocating and sharing their
thoughts on the eective use of technology in education with peers and policy makers. As shown in
Figure 9, they give Microsoft insights on new products and tools for education, and they exchange
best practices as they work together to promote innovation in teaching and learning.
Figure 9. Screen capture of the Microsoft Education landing page (Microsoft, 2015)
7.1.3 TED Education
The use of video is finding increasing acceptance as a teaching strategy in education, and
the research shows significant learning gains. TED videos are of high quality and oer many
opportunities for learning. TED-Ed is a service oered by TED for enhancing teaching using sound
pedagogical principles.
TED-Ed lessons are premised on a four-step process: watch, think (quiz format), dig deeper and
discuss. These education pages enable educators to integrate ICT into their teaching using these
four steps. Educators can also set assignments for students to create similar lessons and assessments.
Students can be asked to manufacture their own videos, including scripting, shooting, editing and
annotating them. They can do this in teams, with dierent roles being assigned to dierent students
within the team.
34 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
The TEDx Program may be a useful way to frame community engagement eorts. TEDx is
designed to help communities, organisations and individuals to spark conversation and connection
through local TED-like experiences. At TEDx events, a screening of TED Talks videos — or a
combination of live presenters and TED Talks videos — sparks deep conversation and connections
at the local level. TEDx events are planned and coordinated independently, under a free licence
granted by TED (https://www.ted.com/about/programs-initiatives/tedx-program).
7.2 Polling and Survey Tools
7.2.1 TodaysMeet
TodaysMeet is a backchannel chat platform for classroom teachers and students to take in-class
formative assessment and also a tool for enhancing interaction during classes. A backchannel enables
conversation to take place alongside the primary activity, presentation or discussion. It takes “great
care to respect the needs and privacy of students while giving educators the tools for success”
(TodaysMeet, 2016). Educators can create a “room” in seconds, even without having an account,
students can join quickly with no registration, and powerful conversations can be started that
augment activities in the traditional classroom.
TodaysMeet harnesses the backchannel technology and turns it into a platform that can enable
new activities and discussions, and extend in-class conversations to outside the classroom, giving all
students a voice, even anonymously if so desired. Engagement during class time is thereby greatly
enhanced. Students can learn from each other and share their insights, and deepened learning
ensues. TodaysMeet enables instant formative assessment opportunities for teacher educators,
and they can respond to students in real time or outside class hours. Figure 10 is a screenshot of
TodaysMeet entries made during the Johannesburg workshop, illustrating how the tool can be used
to garner information (in this case, from participants in a workshop) to modify teaching approaches
or to re-emphasise content.
Figure 10. Screen capture of a sample TodaysMeet interaction
35
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Teacher educators may download a transcript of conversations, which in itself may become a
resource for teaching and learning.
7.2.2 Socrative
Socrative is a web-based student response system that educators can use to engage students during
classroom sessions. Socrative (2016) claims that it
empowers you to engage and assess your students as learning happens.
Through the use of real-time questioning, result aggregation, and
visualization, you have instant insight into levels of understanding so
you can use class time to better collaborate and grow as a community of
learners.
Figure 11 is a screenshot showing how simple the tool is to use and what question types are
available.
Figure 11. Screen capture of creating a quiz in Socrative
One of the most powerful features of Socrative is the reporting function. In the screenshot in
Figure 12, the results of a true-or-false questionnaire taken during a workshop are shown. In this
example, five statements about authentic learning were made, and participants were requested to
decide whether each statement was true or false (answers given here are for illustrative purposes
only and do not reflect the real knowledge of the participants).
As can be seen in the report grid (Figure 12), two important pieces of information became apparent
to the teacher educator:
(i) For three of the questions (3, 4 and 5), more than half of the participants got the answer
incorrect. This means either the question was formulated poorly, or there may have been
conceptual misunderstandings regarding those questions.
(ii) Student Mmaba may be in real trouble and need intervention, as Mmaba answered only one
question correctly.
36 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Figure 12. Screen capture of the Socrative reporting grid
Socrative has several other powerful features –– for example, a voting feature, exit quizzes, and
games, which may greatly enhance the teaching, learning and assessment experience. Teacher
educators may “start” an assessment activity and have it open for several days, weeks or months,
collecting data over an extended period of time. For more information, visit www.socrative.com.
7.3 Social media
Social media is an umbrella term used for a number of web
services. These services allow “account holders” to create
profiles, communicate socially, share thoughts, photographs
and video clips, and play games against “friends.” It is about
sharing, connecting and networking with others. The most
well-known social media services are likely to be Facebook,
Twitter, YouTube and Instagram. An example of a professional
social media site is LinkedIn. Whereas these services are not necessarily geared towards educational
practices, the sheer volume of subscribers to these services –– in other words, their popularity
–– means it is very likely that teachers, teacher educators and students will have an account.
Facebook had 1.6 billion users at the end of 2015, Instagram 400 million, and Twitter 320
million. Therefore, these are handy tools for instantly connecting students, teachers and teacher
educators. The webpage http://www.onlinecollege.org/2012/05/21/100-ways-you-should-
be-using-facebook-in-your-classroom-updated/ lists “99 Ways You Should Be Using Facebook in
Your Classroom.” Facebook is continually updating its services, and the creative educator can find
good ways to use these. A Facebook page, Facebook in Education, can be found at www.facebook.
com/education. Facebook oers services such as Facebook Mobile, which allows learners access to
most of Facebook’s functionalities on tablets and smartphones, Facebook Messenger, which is an
instant message service, and Facebook Paper, which is an app for the IOS environment that serves
as the equivalent of a newspaper. Educators can therefore easily create information repositories on
Facebook, elicit conversations and publish important “news” items.
Best Practice
Facebook allows the uploading of
images, sound clips and video clips,
which can easily be used to spark
discussion and deepen learning.
37
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Many educators are reluctant to use Facebook for teaching
purposes, as they feel that Facebook is a media platform
on which they share personal information, and they are
uncomfortable about students having access to information
about them that they may consider private. However,
Facebook privacy settings can circumvent these obstacles.
Many educators also use Twitter for assessment purposes. The
most common way is to tweet feedback on assessment tasks or
pose questions for their followers. In such cases, a “hashtag”
typically is used. Lively online conversations can be had on Twitter, eliciting a wide variety of
responses from students. The limit of 140 characters per tweet makes students choose their words
carefully.
For more information about using social media for assessment, access the Edutopia website at
http://www.edutopia.org/blog/frictionless-formative-assessment-social-media-paige-alfonzo to
access a blog on “Frictionless Formative Assessment with Social Media.”
7.4 Productivity Suites
7.4.1 Google Apps for Education
Academic institutions may apply to become Google Apps for Education (GAFE) users. Google
Apps for Education is a suite of online applications that Google oers to schools and educational
institutions –– absolutely free. Included apps are Google Mail, Calendar, Drive, Docs and Sites, and
access to a host of other online tools supported by Google. Applications are cloud-based and can
be accessed from any device with an Internet connection. The institutional domain (web address)
is linked to the Google services, and a Google Dashboard is used to administer all teacher educator
and student accounts and services. Google Drive gives access to Google Docs, which is a suite of
applications including a word processor, spreadsheet tool, presentation tool and form builder. The
advantage of the Google applications is that all documents live in the “cloud,” so documents are
easily shareable with other users for collaborative work. A host of other tools are available –– for
example, Google Hangouts, Google Play Books and Google Sites, the last being a free website
builder.
As with Moodle™, several add-ons have been developed for the Google environment. For example,
JoeZoo Express is a free Google Docs add-on that can change the way students’ work is marked in
Google Documents. The tool enables educators to give feedback on students’ Google Documents by
simply highlighting text, then selecting feedback statements from a predetermined menu of options.
Google further maintains a community website for the sharing of ideas. Excellent ideas from other
teachers can be found at https://edutrainingcenter.withgoogle.com/resources_ideas. The website
“10 Google Docs Hacks Every Teacher Should Know” (http://dailygenius.com/10-google-docs-
hacks-every-teacher-should-know/) also contains many good ideas on how to use Google Tools for
online learning and assessment tasks.
Best Practice
Facebook oers many ways to protect
the privacy of their users. Educators
can, with the right settings, use
Facebook for teaching, learning and
assessment, without letting students
have access to their entire profiles.
38 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Google oers certification training courses for teachers and teacher educators. “Fundamentals
Training” is the starting point for certification. Details can be found at https://edutrainingcenter.
withgoogle.com/fundamentals/preview. In addition to Fundamentals Training, Google oers
certification programmes in workflow operations, learning analytics, managing learning resources,
designing interactive criteria, teaching beyond school walls, research skills and “giving students a
voice.”
7.4.2 Microsoft Oce
Microsoft Oce tools such as Word and Excel have many features that educators can use to
develop assessment tasks. Word has functionalities to create forms; the rest of the document gets
locked, so the integrity of the document’s layout remains intact. Excel has even more powerful
features that can be used to create online assessments. Using data validation, students can be
prevented from entering certain kinds of data into cells. It is also relatively simple to create drop-
down items in Excel, and using formulae and functions,
the correctness of students’ answers can be checked.
Excel is also an excellent tool for teacher educators when
student teachers are in schools for practical learning.
Most institutions make use of some form of assessment
instrument for assessing student lessons. Teacher educators
often find themselves writing the same thing over and over
as they visit dierent students. In Figure 13, a screenshot
presents an instrument developed for school observations.
The tool is backed up by a host of “stock comments” that populate the appropriate sections on
the instrument. The instrument also facilitates the entry of other comments. The instrument frees
the observer from frantically writing notes, so more detailed feedback can be given during the
debriefing session. Naturally, the feedback form is also much neater than the hand-scribbles that
often characterise feedback notes.
Best Practice
Using conditional formatting, cell colours
can change depending on whether a
correct or incorrect answer has been
entered into a cell –– for example, red if
the answer was incorrect and green if the
answer was correct.
39
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Figure 13. Screen capture showing the use of Excel to assess student teachers during practical
sessions
In PowerPoint, the add-on tool Poll Everywhere (https://store.oce.com/en-001/app.
aspx?assetid=WA104218073&ui=en-US&rs=en-001&ad=US&appredirect=false) can be used to
interact with students during lessons. For more powerful add-ons that will enhance the use of
PowerPoint, see the website http://www.skilledup.com/articles/30-best-add-ins-and-apps-for-
microsoft-powerpoint.
7.5 General Tools and Websites for Online Assessment
The list of available tools for online assessment presented here is not exhaustive, and it would
be nearly impossible to discuss them all here. For the benefit of the reader, Table 5 lists links to
other online assessment tools that are freely available to educators. “Freely” is a term that must be
understood correctly, as described in the opening paragraph of this section. One of the prevailing
questions in using open-source software is when the “freemium” payment model applies. This
means that the basic elements of the tool are free, but extra options are paid for. MoodleTM is likely
to be the most popular LMS in higher education today, because it is oered as a “free” product.
However, considerable expertise is needed to set up and fine-tune the system. Private companies
may be contracted to set MoodleTM up and customise it, but this may be costly. At least one
qualified IT specialist is required to be employed at the institution, and a separate server and
separate hardware must be purchased, although some Internet hosting companies may oer hosting
services for MoodleTM. It has been estimated that the extra expenses associated with MoodleTM
deployment may amount to the equivalent of USD 10,000 a year –– excluding the IT sta salaries
that are needed for the maintenance and deployment of the tool at the institutional level.
40 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Table 5. Online Assessment Tools
Tool Link Purpose
Tagul
Wordle
https://tagul.com
www.wordle.net
Developing word clouds. Useful for detecting patterns and
important concepts in large amounts of text.
Vocabgrabber http://vocabgrabber.com
This tool targets the key vocabulary from a text intelligently
and demonstrates how those words are used in context. It
develops English ability at all levels of proficiency.
Padlet
Online notice board https://padlet.com A virtual “wall” useful for planning events, collecting ideas
and developing concepts.
Kahoot
Quiz generator for groups https://getkahoot.com Group-oriented quizzes.
VUE
MindMeister
https://vue.tufts.edu
www.mindmeister.com Mind-map generators.
Google Sites https://sites.google.com Free website builder with a Google account.
Google Forms https://drive.google.com
Google Forms is part of Google Drive, useful for creating
online tests. Google Forms now comes with a functionality
to create self-marking quizzes.
Evernote https://evernote.com Excellent tool for online portfolios
Three Ring https://threering.com
A tool for online portfolios, but unlike Evernote, where each
student uses a personal account and has to give the teacher
educator access to the portfolio, Three Ring is educator-
driven and controlled.
As stated before, it is virtually impossible to provide an up-to-date list of online assessment tools in
the dynamic, ever-changing online world. However, simple Google searches will yield several.In
addition, numerous organisations, individual blogs and other websites (see Table 6 ) list some
websites and tools that may be useful for educators.
Table 6. Other Useful Sites for Online Assessments
Other Useful Sites
26 Free tools http://www.teachthought.com/pedagogy/assessment/26-
teacher-tools-to-create-online-assessments
A list of free online
assessment tools
Webtools http://evscicats.com/blog/web-tools-for-teachers-assessment A blog containing a list of
online assessment tools
Edutopia Formative tool http://www.edutopia.org/blog/5-fast-formative-assessment-
tools-vicki-davis
Five fantastic, fast, formative
tools
Rubric Makers http://www.teach-nology.com/web_tools/rubrics A list of free rubric generators
Teach. Learn. Grow. https://www.nwea.org/blog/2014/33-digital-tools-advancing-
formative-assessment-classroom
33 tools for educators
Edudemic http://www.edudemic.com Connecting education and
technology
Free Tech for Teachers http://www.freetech4teachers.com Technology blog
41
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
8.0 CONCLUSION
These guidelines have explored the concept of online assessment for teachers and educators.
Specific attention has been given to the developing world, where many may be considered novice
users of online technologies for teaching, learning and assessment. The guidelines take into account
the contexts in which these technologies may be employed, and the principles presented are
suitably responsive to theoretical constructs that should inform modern-day teaching and learning
and assessment, but also to the very contexts that may constrain the implementation of online
technologies. The examples given at the end of the work are just that: examples. It will remain the
task of resourceful and creative educators to apply what is oered in this publication appropriately
in their contexts.
42 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
9.0 REFERENCES
Abdi, A. A., Shultz, L., & Pillay, T. (2015). Decolonizing global citizenship: An introduction.
In A. A. Abdi, L. Shultz, & T. Pillay (Eds.), Decolonizing global citizenship education (pp. 1–10).
Rotterdam, The Netherlands: Sense Publishers.
Anshul, B. (2009). Development of an online course system and open access online repository. Retrieved
from http://ethesis.nitrkl.ac.in/222/1/Anshul_thesis.pdf
Archdiocese of Canberra and Goulburn Catholic Education Oce & Association of Independent
Schools of the ACT Incorporated. (2011, June). Teachers’ guide to assessment. Canberra, Australia:
Authors. Retrieved from http://www.det.act.gov.au/__data/assets/pdf_file/0011/297182/
Teachers_Guide_to_Assessment_Web.pdf
Arend, B. (2007). Course assessment practices and student learning strategies in online courses.
Journal for Asynchronous Learning Networks, 11(4), 3–17.
Articulate Global Inc. (2016, May 1). Essential guide to visual design. Retrieved from https://
community.articulate.com/e-books/essential-guide-to-visual-design
Bilbao-Osorio, B., Dutta, S., & Lanvin, B. (Eds.). (2013). The global information technology report
2013. Growth and jobs in a hyperconnected world. Geneva, Switzerland: SRO-Kundig.
Brown, S. (1988). Criterion referenced assessment: What role for research? British Journal of
Educational Psychology, Monograph series no. 3, 1–14.
Chan, C. (2009). Types of assessment methods: Reflective journal. Retrieved from http://ar.cetl.hku.
hk/am_rj.htm
COL. (2015, April 15). What is the Commonwealth of Learning? Retrieved from https://www.col.
org/about/what-commonwealth-learning
Cormode, G., & Krishnamurthy, B. (2008). Key dierences between Web 1.0 and Web 2.0. First
Monday, 13(6). Retrieved from http://firstmonday.org/article/view/2125/1972
Council for Higher Education. (2014). Distance higher education programmes in a digital era: Good
practice guide. Retrieved from http://www.che.ac.za/sites/default/files/publications/CHE%20
-%20Distance%20Higher%20Education.pdf
D’Antonio, S. (2009). Open educational resources: Reviewing initiatives and issues. The Journal of
Open, Distance and e-Learning, 24(1), 3–10. doi:10.1080/02680510802625443
Dela Pena-Bandalaria, M. (2007, March). Impact of ICTs on open and distance learning in a
developing country setting: The Philippine experience. The International Review of Research in
Open and Distributed Learning, 8(1). Retrieved from http://www.irrodl.org/index.php/irrodl/
article/viewArticle/334
43
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Department of Education. (1997, July 24). Education white paper 3: A programme for the
transformation of higher education. Retrieved from http://www.che.ac.za/sites/default/files/
publications/White_Paper3.pdf
Elletson, H., & Burgess, A. (2015). The eLearning Africa Report 2015. Berlin, Germany: ICWE
GmbH. Retrieved from www.elearning-africa.com/report
Friedlander, J., & Serban, A. M. (2004, Summer). Meeting the challenges of assessing student
learning outcomes. New Directions for Community Colleges, 2004(126), 101–109. Retrieved from
http://www.wou.edu/cai/files/2016/02/2assessing_student_learning_outcomes_article.pdf
Global Campaign for Education & Education International. (2012). Closing the trained teacher
gap. Rosebank, South Africa: Global Campaign for Education. Retrieved from http://www.
campaignforeducation.org/docs/reports/ECNAT%20Report_RGB.pdf
Herrington, J., Reeves, T., & Oliver, R. (2014). Authentic learning environments. In J. M. Spector
(Ed.), Handbook of research on educational communications and technology (pp. 401–412). New
York, NY: Springer Science+Business Media.
Institute of Distance Education, University of Swaziland. (2015, November). Educational
technology for eective teaching: A handbook for educators. Kwaluseni, Swaziland: Author. Retrieved
from http://www.oerafrica.org/resource/educational-technology-eective-teaching-handbook-
educators
International Assembly for Collegiate Business Education. (2016, June 13). Bloom’s taxonomy of
educational objectives. Retrieved from http://www.iacbe.org/oa-blooms-taxonomy.asp
Jerald, C. D. (2009). Defining a 21st century education. Alexandria, VA: Center for Public
Education. Retrieved from http://www.centerforpubliceducation.org/Libraries/Document-
Library/Prototypes/21st-Century
Johannessen, Ø. (2009). In search of the sustainable knowledge base: Multi-channel and multi-
method? In F. Scheuermann & F. Pedró (Eds.), Assessing the eects of ICT in education: Indicators,
criteria and benchmarks for international comparisons (pp. 13–20). doi:10.2788/27419
Khalil, M., & Ebner, M. (2016). What is learning analytics about? A survey of dierent methods
used in 2013–2015. In Proceedings of the Smart Learning Conference, Dubai, UAE, 7–9 March,
2016 (pp. 294–304). Dubai, United Arab Emirates: HBMSU Publishing House.
Lange, P., & Lancaster, H. (2014, July). 2014 Africa – mobile broadband market. Retrieved from
http://www.budde.com.au/Research/Africa-Mobile-Broadband-Market.html?r=51
Mattingly, K. D., Rice, M. C., & Berge, Z. L. (2012). Learning analytics as a tool for closing the
assessment loop in higher education. Knowledge Management & E-Learning: An International
Journal, 4(3), 236–247.
Mbembe, A. (2015, May). Decolonizing knowledge and the question of the archive. Retrieved from
http://wiser.wits.ac.za/system/files/Achille%20Mbembe%20-%20Decolonizing%20Knowledge%20
44 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
and%20the%20Question%20of%20the%20Archive.pdf
Mnyanyi, C. B., & Mbwette, T. S. (2009). Open and distance learning in developing countries:
The past, the present and the future. Retrieved from https://www.ou.nl/Docs/Campagnes/
ICDE2009/Papers/Final_paper_280Mnyanyi.pdf
Moore, J. L., Dickon-Dean, C., & Galeyen, C. (2011). e-Learning, online learning, and distance
learning environments: Are they the same? Internet and Higher Education, 14, 129–135.
National Institute of Education Singapore. (2015, April 15). TE21: A teacher education model for
the 21st century. Retrieved from http://www.nie.edu.sg/about-nie/teacher-education-21
Norton, L. (2003). Assessing student learning. In H. Fry, S. Ketteridge, & S. Marshall (Eds.), A
handbook for teaching and learning in higher education (2nd ed., pp. 132–149). New York, NY:
Routledge.
OECD. (2015). Students, computers and learning: Making the connection. Pisa, Italy: OECD
Publishing.
Opoku-Mensah, A. (2015). The state of eLearning readiness in Africa. In H. Elletson & A. Burgess
(Eds.), eLearning Report Africa 2015 (pp. 6–9). Berlin, Germany: ICWE GmbH.
Pedder, D., & James, M. (2012). Professional learning as a condition for assessment for learning.
In J. Gardner (Ed.), Assessment and learning (pp. 33–48). London, UK: Sage.
People’s Action for Learning Network. (2015, July). Citizen-led basic learning assessments
for children: An innovative approach. Retrieved from http://palnetwork.org/wp-content/
uploads/2015/07/001-PAL-Network-Comparative-Statistics-2012-ENGLISH.pdf
People’s Action for Learning Network. (2016). PAL Network position statement on SDG4. Retrieved
from http://palnetwork.org/pal-network-position-statement-on-sdg4/
Perie, M., & Marion, S. G. (2008, August 12). Interim assessment practices and avenues for state
involvement. TILSA SCASS Interim Assessment Subcommittee. Retrieved from http://www.ccsso.
org/Documents/2008/Interim_Assessment_Practices_and_2008.pdf
Rastogi, S. (2015, November 18). Educational technology for eective teaching: A handbook
for educators. Kwaluseni, Swaziland: Institute of Distance Education, University of Swaziland.
Retrieved from http://www.oerafrica.org/resource/educational-technology-eective-teaching-
handbook-educators
Rotherham, A. J., & Willingham, D. (2009, September). 21st century skills: The challenges ahead.
Educational Leadership, 67(1), 16–21.
UNESCO. (n.d.). Distance vs open learning. Retrieved from http://portal.unesco.org/education/
en/ev.php-URL_ID=18650&URL_DO=DO_TOPIC&URL_SECTION=201.html
UNESCO. (2002). Information and communication technology in education: A curriculum for
45
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
schools and programme of teacher development. Paris, France: UNESCO Publishing.
UNESCO. (2011). ICT in education. Retrieved from http://www.unesco.org/new/en/unesco/
themes/icts/teacher-education/unesco-ict-competency-framework-for-teachers/
UNESCO. (2015). Education for All 2000–2015: Achievements and challenges. Paris, France:
Author.
UNESCO. (2015, April 15). The teacher training initiative for sub-Saharan Africa. Retrieved
from http://www.unesco.org/new/en/education/themes/education-building-blocks/teacher-
education/ttissa/
University of Conneticut. (n.d.). What is assessment? Retrieved from http://assessment.uconn.edu/
what-is-assessment/
Wright, C. (2014, April 16). 5 key barriers to educational technology adoption in the developing world.
Retrieved from http://edutechdebate.org/2014-ict4edu-trends/5-key-barriers-to-educational-
technology-adoption-in-the-developing-world/
Wright, C. R., Dhanarajan, G., & Reju, S. A. (2009). Recurring issues encountered by distance
educators in developing and emerging nations. The International Review of Research in Open and
Distributed Learning, 10(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/
view/608/1180
46 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
10.0 APPENDICES
Appendix A: Definitions and Types of Assessment
Defining traditional assessment
Assessment originates from its Latin root assidere, which means “to sit beside another.” The term
attempts to be responsive to the questions:
• What do we want the students to learn?
• Why does the learning matter?
• What do we want the students to do/produce?
• How well do we expect them to do it?
According to Pedder and James (2012), “when teachers and students use assessment information
for supporting improvements in learning, they are fulfilling the educational purposes of assessment”
(p. 38). Assessments provide educators with both objective and subjective data in order to ascertain
student progress and skill mastery. Besides a score, which gives quantitative data about how much
learning has taken place, information about student misconceptions can be determined by analysing
assessment results. Information from assessments helps teachers determine which instructional
approaches are best for certain students, what their students may already know about a given topic,
and what topics need to be re-taught.
The purposes of assessment can thus be identified as:
• Promoting student learning by providing the student with feedback during formative
assessment tasks, to help improve their performance.
• Measurement: evaluating student knowledge, understanding, abilities or skills.
• Standardisation: providing a mark or grade that enables a student’s performance to be
established. The mark or grade may also be used to make progress decisions.
• Certification: enabling the public (including employers) and higher education providers
to know that an individual has attained an appropriate level of achievement that reflects
the academic standards set by the awarding institution and agreed norms. This may
include demonstrating fitness to practise or meeting other professional requirements set by
professional boards (Norton, 2003).
Types of traditional assessment
Several types of assessment are identified in the literature:
a) Formative assessment
This type of assessment is administered during the learning process and seeks to determine how
students are progressing whilst attempting to achieve a learning goal. It is typically considered
47
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
assessment for learning. It can be used at the beginning of a learning unit and during the teaching
of that unit. Teachers use it to check student understanding and may use it to make decisions about
re-teaching particular sections of work. Formative assessment can also be considered assessment
as learning. Good formative assessments are evident when educators expect students to use the
results of assessments, peruse and reflect on feedback, then apply the feedback to monitor their
own progress and learning. Educators should use information from the formative assessment task
to decide how to modify or amend further teaching and learning. Students should also learn from
feedback and practise to improve. This monitoring of one’s own progress eectively becomes
“assessment as learning” and contributes to their attainment of learning outcomes.
b) Diagnostic assessment
This type of assessment is typically given at the beginning of a unit of learning and is a
diagnostic test that attempts to determine what students already know about a topic, what their
misconceptions are regarding a unit of learning, or where gaps in knowledge exist. However,
formative assessment tasks can also be used diagnostically during teaching and learning.
c) Summative assessment
This type of assessment is often evaluative in nature and given at the end of a learning unit or
academic period. Summative assessments give a benchmark of a student’s mastery of a topic after
a period of teaching. They are assessments of learning. The culmination of students’ learning
achievements through tasks are recorded as the extent to which they demonstrate their mastery and
knowledge of the content. Summative assessment data provide teachers with information about
how eective teaching strategies have been, the time needed for instruction and how to improve
teaching for future students.
d) Norm-referenced tests
These tests measure students against a national “norm” or average in order to rank students against
each other. An example of norm-referenced tests are the “Common Core Standards” that are used
in the United States.
e) Criterion-referenced tests
These tests measure student performance against a standard or specific goal. Criterion-referencing
attempts to provide information about previously determined standards and describes the
knowledge and skills which are prevalent at a given level of attainment. Brown defined criterion-
referenced assessment as “[a]n evaluative description of the qualities which are to be assessed (e.g.,
an account of what pupils know and can do) without reference to the performance of others”
(Brown, 1988, p. 4).
f) Continuous assessment
This is not a type of assessment per se but rather a way in which assessments are used by educators
to calculate the final score of the performance of students. In typical educational contexts, the final
result for a student is either the score achieved in a large assessment task, such as an examination, or
48 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
the average of scores for summative assessment tasks during the semester and the final assessment.
When the scores of all (or most) assessment tasks during the year accumulate to constitute the final
score, without there necessarily being a final examination weighted to count the most, continuous
assessment is used in a programme.
49
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Appendix B: Assessment Design within Curriculum Design
Deciding on the assessment strategy is an essential part of the curriculum planning process. It must
provide evidence that the programme purpose is being met. In addition, much of the curriculum
is informed by assessment, which should be designed to focus on those aspects of the curriculum
most closely associated with the assessment requirements. The framework (Figure D1), proposed by
the Council for Higher Education (CHE), in South Africa, illustrates a macro design model which
positions assessment within curriculum design.
WHAT ARE THE SOURCES THAT CONTRIBUTE TO THE CURRICULUM?
ACCREDITATION BODIES • NATIONAL POLICIES • TARGET QUALIFICATIONS • EXISTING RESEARCH
These are analysed and brought together to illuminate student needs and learning environments,
and national and regional demands. In turn, they inform the following:
Approach to teaching and
learning (pedagogy)
What are appropriate
assessment strategies?
Formative and summative
assessments
Feedback
Recording and reporting
What content and what skills
are emphasised?
What are appropriate
teaching and learning
methods?
Independent study methods
Group activities
Work-integrated learning
Student support
Purpose of programme
and outcomes
MEDIATED
by
Learning
materials
Learning
technologies
Assessment
tools
Internal Moderation Processes
External Moderation Processes
Figure D1. The curriculum and assessment design model (adapted from CHE, 2014)
50 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
In 2002, the Assessment Reform Group (Cambridge University, School of Education) produced ten
principles that are foundational to assessment for learning. Assessment should:
1. Be regarded as a key professional skill for teachers
2. Be part of eective planning of teaching and learning
3. Promote commitment to learning goals and a shared understanding of the criteria by which
they are assessed
4. Be recognised as central to classroom practice
5. Focus on how students learn
6. Provide constructive guidance about how to improve
7. Develop learners’ capacity for self-assessment so that they can become reflective and self-
managing
8. Take account of the importance of learner motivation
9. Be sensitive and constructive, because any assessment has an emotional impact
10. Recognise the full range of achievement of all students
Assessment design is directly related to the pedagogical approaches that are appropriate for
particular disciplines. Therefore, assessment design influences learning processes. Whilst assessment
as feedback focuses on practices to improve student learning, it is often neglected in favour of
summative assessments that rely on outdated information-transmission models of teaching, which
means that the assessments do not address the needs of students in modern complex and globalised
societies. Many current assessment systems do not allow students to improve their own learning,
because the assessments are “considered to be an endpoint instead of a beginning or a step forward”
(Birenbaum et al., 2005, p. 3). This means that summative assessment (testing what has been
learned) tends to drive the teaching (teaching for the test).
Conversely, assessment for learning places more emphasis on the formative, is integrated into the
curriculum and is contextual, embedded and flexible. This typically is recognised as a deep approach
to learning. Through active, constructivist engagement with knowledge, a surface approach to
learning is avoided. The emphasis on reproduction by an incrementally developed knowledge is
negated. Lecturers typically develop a module with the content in mind first, and when they set
assessment tasks, they attempt to cover the subject area, and they place less emphasis on facilitating
students’ learning. A fundamental distinction exists between approaches to teaching that are, in
Prosser and Trigwell’s (1999) terminology, “conceptual change/student focused” and “information-
transmission/teacher-focused.” Therefore, it is recommended that the assessment tasks be designed
before the content of the module, resulting in an integrated assessment system which serves the
purposes of (i) assessment for learning and (ii) assessment of learning. This is often described as
“constructive alignment,” meaning that there is consistency between the three related components
of curriculum design:
51
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
• What do students need to learn (i.e., what learning outcomes will they achieve)?
• What teaching methods will be used to enable them to achieve these learning outcomes?
• What assessment tasks and criteria will be used to show that students have achieved the
learning outcomes intended?
Aligning assessment tasks with learning outcomes
There is no point in setting assessment tasks if they are not aligned to the curriculum outcomes (as
we explored in section 4). Much has been written about the importance of linking the assessment of
outcomes to the improvement of student learning and development. Yet, there is limited evidence
of the role that assessment results have played in the selection of teaching methods. This remains a
challenge in teacher education.
Further, there appears to be a lack of knowledge about assessment processes, tools and models,
especially in the context for which this document has been written. Apparently, few teacher educators
have been formally trained in developing outcomes that are aligned with the curriculum or in using
appropriate methods to create assessments and instruments that yield a meaningful and sustainable
overall assessment strategy. The fact that there seems to be little consensus amongst sta about
what they are trying to achieve at the programme and course levels, or about their various personal
pedagogic models, leads to incongruent outcomes and methods of measuring those outcomes.
Often, the outcomes are stated broadly, without specifying the particular skills or competencies that
students should acquire. Therefore, it remains dicult to implement and sustain a comprehensive
student assessment regime in teacher education settings (Friedlander & Serban, 2004).
Below are some salient points regarding the alignment of outcomes and assessment tasks:
• Avoid over-emphasising content in assessment tasks, as doing so can neglect the cognitive
demands of an outcome; for example, if an outcome may require students to learn to apply
the principles of a concept, do not assess just the content of the answers.
• Ensure a match between the pedagogical approach when teaching and the assessment task.
If the teaching approach requires listening, that will not encourage the development of
cognitive skills that may require higher-order thinking. The teaching approach should model
higher-order thinking skills in alignment with what the assessment task requires.
• It is not always necessary to assess each and every learning outcome. It may be useful
to prioritise learning outcomes in terms of the absolute minimum outcomes expected,
compared with the essential learning outcomes, and focus on those priorities.
• Ensure that the assessment task really corresponds with the learning outcome(s). A mismatch
between the intended outcomes of a learning unit and an assessment task can easily occur.
An outcome may state that students need to apply the principles of a concept to a particular
scenario, yet the assessment task may be a discussion of the principles.
• The inclusion of formative assessment tasks gives students an opportunity to practise the
learning that is required. Good feedback is required for formative assessment tasks.
52 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
It is extremely important that good feedback be given on formative assessment tasks. Hattie (2015)
ranked the influences of a number of curriculum variables on student performance. In his earlier
work, feedback was identified as the number one influence on student learning. As he expanded his
work, other variables were isolated, but feedback remained in the top quintile as a determinant of
learning success.
53
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Appendix C: Traditional Authentic Learning and Assessment Explained
Authentic assessment has been reported to deepen knowledge and understanding and to improve
problem-solving and social skills, as assessment tasks are real-world related or an approximation of
a real-world scenario. Authentic assessments set meaningful and engaging tasks, in a rich context,
where the learner applies knowledge and skills and performs the task in a new situation. It is a
pedagogical approach that situates learning tasks in the context of how knowledge will be used in
the future professional lives of students. Today, innovative educators see authentic assessment as
a means to develop students’ knowledge in ways that transfer to real-world practice and realistic
learning tasks. Using ICT in assessment provides the ideal conditions for authentic assessment in
online courses. The aordances of modern ICT as cognitive tools and delivery platforms facilitate
authentic learning experiences. Herrington, Reeves and Oliver (2014) identified the following
characteristics of authentic assessment tasks:
• An authentic context reflects the way the knowledge will be used in real life. Online learning
tasks cannot simply provide suitable examples from real-world situations to illustrate content
that is being taught. Students should engage with real-life problems that will engage their
emotional commitment as well as their cognitive interest.
• Authentic assessment is driven by ill-defined tasks that have real-world relevance. They are
complex and need to be completed over a longer period of time; they are not a series of
shorter, disconnected activities. In such tasks, students are not given a recipe to follow––
they have to define the sub-tasks themselves.
• Authentic assessment tasks may span several disciplines or subject areas. Some of the outcomes
of such tasks may even fall outside the domain-specific outcomes. Students may work
collaboratively in teams and assume dierent roles, or develop dierent sets of expertise.
• Authentic assessment tasks can be considered as learning tasks, and learning and assessment
thereby become seamless. Levels of performance are clearly articulated, and students are given
the opportunity to demonstrate their knowledge and skills.
• Authentic assessment tasks lead to the creation of polished products. The outcome of the
assessment is a product that is valuable in its own right and is usable by students in practice.
• Authentic assessment tasks require that students access a variety of levels of expertise. The
Internet, as a global learning village, gives students access to resources and experts,
and tools such as blogs and other social media platforms give students a voice and an
opportunity to interact.
• Authentic assessment tasks require that problems be viewed from multiple perspectives.
• Authentic assessment tasks require reflection. This reflection is critical in the learning process
and should be contextualised in real-world situations.
54 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
• Authentic assessment tasks require students to articulate their understanding using
appropriate tools. Students are obliged to use appropriate ICT tools –– for example,
presentation tools to explain solutions, spreadsheet tools to perform calculations, or word
processing tools to write proposals.
• Authentic assessment tasks are accompanied by coaching and scaolding support systems. The
students are primarily supported by the teacher educator, but they may also receive support
from the professionals in an authentic setting –– for example, teachers in practice.
Callison and Lamb (2004) identified seven signs of authentic learning: (1) student-centred, (2)
access to multiple resources beyond the institution, (3) students working as scientific apprentices,
(4) students collecting original data, (5) commitment to learning beyond the task, (6) authentic
assessment of process, product and performance, and (7) team collaboration.
Herrington, Reeves and Oliver (2014) concluded that “authentic learning as a pedagogical
approach is especially appealing to educational technologists.” Knowledge situated in realistic
contexts is less likely to remain “inert” or theoretical. Realistic tasks can potentially cognitively
challenge student teachers to solve problems and think in the same ways that experienced teachers
do. In addition, the complex tasks implicit in the approach require the creation of real products
and innovations, are immediately usable in a school classroom, and are therefore more worthy than
decontextualised assessment tasks.
55
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Appendix D: Bloom’s Taxonomy
Educators should be familiar with Bloom’s 1956 original taxonomy of thinking skills. The
taxonomy was a means of expressing dierent kinds of thinking and was adapted as a tool for lesson
planning and for writing about the objectives or outcomes of learning. It continues to be one of the
most universally applied models in teacher education and provides a way to organise thinking skills
into six levels, from the most basic to the higher-order levels of thinking, each level building on the
previous level, from the simplest to the most abstract. The taxonomy was revised in 2001 to update
it for the 21st century, and major changes in terminology and structure were made. In the revised
framework, “action words” or verbs, instead of nouns, are used to label the six cognitive levels,
and three of the cognitive levels are renamed. The “synthesis” level from the original taxonomy
disappears and is replaced with “evaluation,” which was at the apex of the taxonomy. In the revised
taxonomy, “creating” is the uppermost level.
Table G1 contains the six levels in the cognitive domain and the action verbs that are associated
with each level of the taxonomy (International Assembly for Collegiate Business Education, 2016).
Table G1. Bloom’s Revised Taxonomy
Cognitive Level Sample Action Verbs
Remembering Define
Duplicate
Identify
List
Name
Recall
Recognise
Reproduce
Retrieve
Tell
Understanding Calculate
Categorise
Clarify
Classify
Compare
Conclude
Contrast
Describe
Discuss
Exemplify
Expand
Explain
Identify
Illustrate
Infer
Interpret
Locate
Match
Outline
Paraphrase
Predict
Report
Restate
Summarise
Translate
Applying Carry out
Classify
Demonstrate
Execute
Illustrate
Implement
Practice
Solve
Use
Utilise
Analysing Appraise
Attribute
Compare
Contrast
Deconstruct
Detect
Dierentiate
Discriminate
Distinguish
Examine
Formulate
Infer
Integrate
Organise
Parse
Relate
Select
Sequence
Structure
Test
Evaluating Appraise
Check
Coordinate
Critique
Defend
Detect
Dispute
Judge
Monitor
Prioritise
Rate
Reconstruct
Select
Support
Verify
Creating Change
Combine
Compile
Compose
Construct
Create
Design
Formulate
Generate
Hypothesise
Improve
Invent
Plan
Predict
Produce
Source: International Assembly for Collegiate Business Education (2016)
56 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
Appendix E: General Principles for Traditional Assessment
General Principle 1: Assessment tasks should be closely aligned with learning unit
outcomes.
Enacting this general principle may mean re-examining learning unit outcomes. This is
especially necessary if the outcomes mostly address content-replication outcomes. Assessment
practices should be congruent with the pedagogical approach that is adopted during course
delivery.
For further information about this general principle, see Appendix B: Assessment Design within
Curriculum Design.
General Principle 2: Assessment tasks should have outcomes that develop cognitive skills
across the range of categories as espoused by Bloom’s revised taxonomy of cognitive
objectives.
There should be a deliberate eort to set assessment tasks that require students to create
learning artefacts. This will enhance creativity and improve digital fluency.
For further information about this general principle, see section 4.3, Development of 21st-
century Skills.
General Principle 3: There should be a carefully considered balance between formative
and summative assessment tasks.
The assessment tasks of a programme should include a sucient number of formative
assessment tasks that promote learning. Formative assessment tasks can allow multiple
submissions of the task that exhibits learning. Formative assessment tasks can be scored and
can comprise as much as 50 per cent of the grades that can be earned in programmes. The final
grades in programmes should not be made up by summative assessments only.
For further information about this general principle, see Appendix C: Traditional Authentic
Learning and Assessment Explained.
General Principle 4: Summative assessments should cover the curriculum appropriately.
Although it is not necessary to cover each and every outcome that was set for a learning unit, it
remains important to prioritise outcomes that are more essential than others.
For further information about this general principle, see Appendix C: Traditional Authentic
Learning and Assessment Explained.
General Principle 5: Assessment tasks should deliberately set out to develop 21st-century
learning skills.
Marking schemes or rubrics that are designed for assessment tasks should explicitly account for
the demonstration of these skills and reward students who do so.
57
GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
General Principle 6: Assessment tasks should be free from any form of bias.
Assessment tasks should be gender responsive and free from racial, ethnic or religious bias.
This general principle is informed by COL requirements in all their work, and in line with
contemporary thinking about the biases that exist in this world.
General Principle 7: Assessment tasks during training should have relevance in work-
integrated learning.
Although closely aligned with Online Principle 4, this is important enough to be considered
a separate general principle. Assessment tasks should be relevant for either work-integrated
learning or professional development.
General Principle 8: Assessment tasks in education should account for Indigenous
knowledge systems.
This general principle emerged as a result of a workshop discussion during a time when HE
institutions in South Africa had made international headlines due to the (often violent) actions
of student protestors voicing their grievances. A central theme in the students’ protests related
to a call for “decolonising knowledge.” At present, debates are taking place about defining
this concept precisely. Although the protests took place in an African context, the call for
decolonising knowledge is heard across Commonwealth countries. Abdi, Shultz and Pillay
(2015) stated that educational
endeavours should also take into account the specificities of the locations
of research and diverse experiential realties that inform the social
and cultural platforms that should contextualize the rationale for the
observational and analytical categories that are selected to undertake the
concerned research. (p. 1)
For Mbembe (2015), “Human history, by definition, is history beyond whiteness. Human
history is about the future. Whiteness is about entrapment” (n.p.).
In these guidelines, we are loath to define general principles that seem to be politically
inspired. However, it is of the utmost importance to heed the calls of many students in the
developing world, who wish for an acknowledgement of Indigenous knowledge systems (IKS)
in HE curricula and assessment. IKS refers to local knowledge that is unique to a given culture
or society and as such should also be considered valuable. IKS often stand in direct contrast
with the knowledge systems advocated by universities, research institutions and private firms
in the industrialised world. These latter systems often form the basis for policy making in
agricultural practices, healthcare administration, food preparation and consumption, education
systems and natural resource management, often not taking account of the impact of those
decisions on rural communities in the rural world (Warren, 1991).
This general principle raises again the inseparability of assessment from curriculum. Good
assessment practices can only be realised within good curricula.
58 GUIDELINES FOR ONLINE ASSESSMENT FOR EDUCATORS
General Principle 9: Assessment tasks should be externally quality assured.
It is international best practice that curricula and exit assessments be benchmarked against
those in other programmes, and it is recognised that such moderating may very well improve
the assessment tasks that are given to assess student learning. Benchmarking can play a role in
developing an open knowledge base for assessment. Ultimately, international collaboration is
necessary for assessment because of the complex issues involved, the diversity of stakeholders and
the need for agreed frameworks for international comparisons (Johannessen, 2009).
Institutional policies will prescribe the proportion of assessment tasks that need to be
moderated or benchmarked. However, it is recommended that institutional policies provide for
benchmarking all the assessment tasks that are included in a course or a module. Moderators
should therefore receive not only summative assessment tasks and scripts to moderate, but the
entire gamut of learning resources, syllabi, assessment tasks and possible course evaluations when
they have to evaluate the quality of online assessments.
4710 Kingsway, Suite 2500
Burnaby, BC V5H 4M2
Canada
Tel: +1.604.775.8200
Fax: +1.604.775.8210
E-mail: info@col.org
Web: www.col.org Printed on Sugar SheetTM
, 100% Forest Free
SUGAR
SHEET
100% Forest Fr e e
TM
