Importance of Online Assessment in the E-learning Process

Abstract

In this article, we present some general aspects of the on-line assessment activity. For the purpose of this paper, we want to focus in the assessment activity that take place in the e-learning process and discuss the importance of this action for each participant of the process. After that, we outline the desirable characteristics of an assessment tool under an adaptive learning environment, following with the description of the main attributes of an adaptive assessment tool for an adaptive learning environment actually in development.

Full text

Session F3B
0-7803-9141-1/05/$20.00 © 2005 IEEE July 7 – 9, 2005, Juan Dolio, Dominican Republic
ITHET 6
th
Annual International Conference
F3B-1
Importance of Online Assessment in the
E-learning Process
Héctor Barbosa, Francisco Garcia
Department of Computer Science, University of Salamanca
Plaza de la Merced, s/n
37008, Salamanca, SPAIN
{barbosah, fgarcia} @usal.es
Abstract - In this article, we present some general aspects
of the On-line Assessment activity. For the purpose of this
paper, we want to focus in the assessment activity that take
place in the e-learning process and discuss the importance
of this action for each participant of the process. After
that, we outline the desirable characteristics of an
assessment tool under an adaptive learning environment,
following with the description of the main attributes of an
Adaptive Assessment Tool for an Adaptive Learning
Environment actually in development.
Index Terms – Adaptive Assessment Tool, Adaptive learning
environments, Assessment in e-learning, Learning Technology
Standard specifications.
I
NTRODUCTION
In recent years, instructional and educational institutions have
been incorporating information and communication
technologies in learning and teaching processes in order to
increase the quality, efficiency, and dissemination of
education. As long as those projects cover the needs of
individuals in a particularly way, the success and
transcendence of such developments could be incremented by
performing adaptability to each user so the learning
experience can be enhanced.
To be sure that all of these efforts don’t become groups of
isolated isles, most of these projects look to be compliant to
some accepted standards, so they can be applicable,
compatible and interchangeable between them. One accepted
standard is the IMS, a global learning consortium that
develops and promotes the adoption of open technical
specifications for interoperable learning technologies that
become the standards for delivering learning products
worldwide. Among the inherent importance of these works,
we want to emphasise in the role of the assessment activity
inside the e-learning process. We want to concentrate in this
task, and see how it can help to improve the e-learning process
to all the participants: students, teachers, and content
designers.
The rest of this paper is structured as follow: In the next
section, we depict how we see the importance of the
assessment activity in the e-learning process dividing the
information for each main element in the e-learning process.
In section “Desirable Characteristics for an Assessment Tool”,
we put in relief the general and desirable characteristics of an
assessment tool for an adaptive learning system; also, we
explain some desirable components of quality and efficiency
for the assessment activity. In the Proposal section, we
describe a proposal for an Adaptive Assessment Tool that will
be part of an adaptive learning system [5], now in
development in the Department of Computer Science of the
University of Salamanca (Spain). Finally, in the last section
we give our conclusions and further work.
I
MPORTANCE OF THE ASSESSMENT ACTIVITY
Conceptualizing the learning process to its basic elements, we
can identify at last the following elements:
1. The educational material to be taught by the teacher in a
classroom.
2. The teaching and learning activities that take place in a
classroom.
3. The assessment activity to measure the student learning
and,
4. The report of the score results given by the teachers to the
students. This conception is well suitable for the traditional
educative process.
FIGURE I
ASSESSMENT
IN THE E-LEARNING PROCESS
However, this is a more complex process in which there
are several factors that should be taken into account like
student learning styles, the technical implications, the adaptive
educative content, the learning and knowledge management,

Session F3B
0-7803-9141-1/05/$20.00 © 2005 IEEE July 7 – 9, 2005, Juan Dolio, Dominican Republic
ITHET 6
th
Annual International Conference
F3B-2
feedback, motivation, etc. Traditionally, assessment activity
has been seen like task aside of the e-learning process and
there is a danger in focusing research on assessment
specifically [4], as this tends to isolate the assessment process
from teaching and learning in general.
In figure I, we characterize the importance of assessment
in the learning flow. As we can note, the tests and evaluations
not only are an integral part of the learning process, but also is
an element that complete and close a circular activity,
contributing as a feedback source for: the users (giving the
scores and feedback), for the instructors (by giving support
and feedback) and for the instructional designer (to update the
contents of the learning system) as well. This circular
conceptualization of the learning process allow us to see the
significance of the assessment because it helps to the
adaptation of the system by setting a new user knowledge
level, evaluating and determining new learning profiles, assign
the user grades and, in consequence, performing user content
re adaptation. This is how we see the importance of the
assessment task for the adaptation process.
As Kendle and Northcote assert [4], the evaluation
should be one of the first considerations of design when you
prepare an online course, integrating it in the program and not
considered by apart.
I. Importance for the Educative Content and the Adaptation
Process.
The results of the test made by the students could allow an
adecuation of the web site that reflects the new knowledge
topics or the new syllabus that will be taken. According to the
Australian Flexible Learning Framework [1], assessment,
especially when is included within a real learning task or
exercises, could be an essential part of the learning
experience, giving to the entire Web site the characteristic to
adapt itself to the needs of the users. This could be an
interesting feature of an educational Web site because the
improvement of the online teaching experience by giving to
the student:
• Convenient Feedback: the web site shows the results of
the assessment and gives the feedback to the user to
improve their scores.
• Once the lesson taken and the corresponding evaluation
was carried out, the educative content could be readapted
to the new knowledge level of the user by imparting new
lessons, based upon the results of the assessment.
• From the last feature, we could establish user-tailored
content information to give the student the convenient,
adapted and meaningful information, improving the
learning experience.
• Gives flexibility, enables group work when the test is
designed to be made for several individuals, have the
potential to be interactive, learners can assess assessments
remotely and could be ease to use [4].
II. Importance for the User.
For the student, the assessment activity informs progress and
guide learning; also, it is essential for the accreditation process
and measures the success of the student. Assessment tasks can
be seen as the active components of study, also assignments
provide learners with opportunities to discover whether they
understand and not, if they are able to perform competently
and demonstrate what they have learnt in their studies.
Furthermore, the feedback and grades that assessors
communicate to students serve to both teach and motivate [7].
In addition, we have to consider another and emerging
approach for online assessment that refers to group
collaboration, sharing and learning where students are able to
see solutions to authentic problems from many other students.
It is necessary to mention that a fast, instantly and effective
feedback is very important to the learners, also to have access
to multiple attempts, take charge of their own learning and
track their own progress. Removing the need for fixed
delivery dates and locations normally required in traditional
tests is seen as a benefit for the students [4].
Conclusively, we would like to mention an assessment
categorization:
•
Formative assessment: this is an assessment that helps to
give a convenient feedback and motivation to the student
and do not have scores. Also brings convenient feedback
to designers of materials.
• Summative assessment: this is a scored assessment and
gives a result to measure outcomes and success of the
learner.
• Norm assessment: use the achievements of a group to set
the standards for specific grades and is used in most
universities.
• Criterion assessment: establish the standard (criterion)
and mark students against it.
• There is another kind of assessment called alternative
assessment. Here, the integration of the assessment
activity with learning processes and real-life performance
as opposed to display of “inert knowledge”. Know as
authentic assessment it is very much based on the
constructivist approach that enables students to
demonstrate knowledge by working on authentic task,
putting students un control of their own learning [1] and
helping the students to develop the necessary skill for
autonomous (and lifelong) learning.
III. Importance for the Teachers and Assessors.
Most of the online learning systems and learner management
systems are automated and linked between them, This give to
the instructors/assessors the way to perceive the benefits to
include a documented and consistent assessment process
where the technology allows ease of monitoring the learner
process and the provision of immediate feedback. All those
features improve the quality of assessment process [4].
Another factor of importance is that the teachers could
have time saving since the system could have the
characteristic to automate the marking. Other critical factor
when online assessment is applied is to minimize the
academic dishonesty. To avoid this, a widely used method is
to have an online database of assessment as a part of a test
bank that allows generating random tests from randomised

Session F3B
0-7803-9141-1/05/$20.00 © 2005 IEEE July 7 – 9, 2005, Juan Dolio, Dominican Republic
ITHET 6
th
Annual International Conference
F3B-3
items. Online assessment allows improving the assessment
process, support flexibility, saves time in the long term and
allows quick feedback to learners, creates consistent and
standardised assessment, the progress can be monitored, the
assessment activity can be documented, it is financially
attractive and learners can be assessed equally [4].
From our point of view, the assessment activity could be
considered as an integrator step that help the entire process to
get self adecuation to the user needs, giving feedback to both
the student and the instructors, also, the assessment action
could activate other activities that could improve the entire
instructional process.
Such activities could be the recording of the lessons
taught and evaluated to set a new user knowledge level and
the new user profile by taking in account the learning style.
D
ESIRABLE CHARACTERISTICS FOR AN ASSESSMENT TOOL
Nowadays, it is necessary to produce educative Internet-based
systems that permit the dissemination of the education,
covering the needs of diverse learning group profiles. To
obtain this, it is desirable that such systems perform automatic
task to adapt itself to each user, disconnecting the content
from its presentation by using a semantic approach rather than
a syntactical one, defining a meaningful web.
In consequence, learning systems must be flexible and
efficient, and one way to accomplish that is to be an open and
standardized system. We want to focus on the following
standards by giving their general characteristics and their
support features for a learning system:
The LTS (Learning Technology Standard) specification is
a group of agreements about the characteristics that a learning
element should have. The use of standards ensures
instructional technologies to work with other systems
(interoperability), follow-up information about learners and
contents (manageability) generate learning objects that are
usable in other contexts (reusability) and avoid obsolescence
(durability) [5]. Among the learning technologies standards
we must to mention the IMS Specifications that are part of the
LTS specification, developed in 1997 as a project of the
National Learning Infrastructure Initiative at Educause. Its
mission is to promote distributed learning environments. For
the IMS, many areas require interoperability when learning is
distributed, thus it details a set of specifications that build a
framework to interchange educational elements [2]. This
framework covers among others, the following aspects:
• IMS Learning Resources Metadata Specification: describe
learning resources for searching and discovering, with
foundation in the IEEE LOM.
• IMS Question and Test Interoperability (IMS QTI), for
share test items and other assessment tools. It defines a
data model for the presentation of questions, test, and the
correspondent results reports [6].
Those specifications allow the creation of many types of
educational designs following a consistent notation to develop
a system that could have a homogeneous implementation in
several courses or learning contexts, giving the systems extra
characteristics like compatibility and exchangeability. In
addition, those specifications define an abstract data model
using XML, supporting the deployment of item banks and
well documented content format.
One aim is to make those systems to work in adaptive
learning systems and, given the fact that the assessment
activity is an important and integral part of the e-learning
process; it is desirable that this valuation activity could be
adaptable as well. If we want the assessment to be
interoperable, compatible, and shareable, we must have to
develop a standardized tool.
The implementation of the adaptability in the assessment
activity by giving the instructor:
• Tools to design the assessment with several types of
resources (text, graphics, video, animations) to improve
the student´ understanding in the examination
• Design different types of assessment
• Define content structures and groups of students
• Manage the assessments and the questions linked to each
assessment
• Define the grades for each question and assessment
• Define and manage the schedule for the assessment that
will be taken.
The Australian Quality Training Framework defines some
desirable components of ‘quality’ in assessment: validity,
reliability, flexibility, and fairness [4]. In addition, this
framework highlights requirements such as assessment
information to candidates; standards of performance required
in the workplace; sufficiency of evidence; provision of
feedback to candidates; equitable assessment processes and
language, literacy, and numeracy considerations in
assessment.
In addition, Kendle and Northcote [4] have criteria to
guide the design and development of effective qualitative
online assessment tasks:
• Variety: including both quantitative and qualitative
methods.
• Authenticity: using open-ended tasks that simulate
workplace tasks, as well as appropriate quantitative tasks.
• Collaboration: allowing for interaction between learners
and others, and using appropriate communication
technologies.
•
Feedback: ensuring appropriate feedback mechanisms are
possible using peer feedback and peer tutoring.
• Online resources: making full use of available
quantitative packages as well as other internet resources.
• Learner responsibility: providing options and
opportunities for accountability within assessment task.
To obtain the desired results of the assessment activities,
Hayes provides a list of expectations for online assessment
and states that assessment should [4]:
• Be clearly related to the aims and objectives of the subject
• Occur at integrated moments along the learning
continuum
• Embody students utilising authentic, real life skills and
processes

Session F3B
0-7803-9141-1/05/$20.00 © 2005 IEEE July 7 – 9, 2005, Juan Dolio, Dominican Republic
ITHET 6
th
Annual International Conference
F3B-4
• Include provision of proactive skills support if required,
and
• Allow for students to make choices and be informed of
their learning progress
P
ROPOSAL
Now, we want to outline an adaptive assessment tool (AAT)
that could be part of an adaptive learning environment (ALE).
We want to emphasize that this AAT is a proposal to the
HyCo model, which define some modules integrated to deliver
adaptive units of learning based in open standards and
conforming an open system, using some technologies, which
are described next.
I. The HyCo Model
For our proposal, it is important to describe the HyCo
(Hypertext Composer) model because we want to define and
construct an assessment tool that will be an integral element of
this system.
HyCo is an authoring tool to compose semantic learning
objects for web-based E-learning systems, by creating
hypermedia educational resources and to access to that content
(figure II). This authoring tool uses learning technology
standards or specifications to save these semantic objects that
are delivered in web e-learning environments as encapsulated
packages in order to ensure their reusability, interoperability,
durability, and accessibility. These learning objects are close
to the semantic web field because they combine hypermedia
and semantic capabilities. The aim of the HyCo is to use these
semantic learning objects in order to define learning domains
for the ALE. In addition to provide an e-learning environment
where teachers have tools to create didactic materials and
students carry out their knowledge acquisition through the
most suitable adaptive learning technique giving the student’s
characteristics, the learning activities provided and the
learning objects’ features [5].
This system use open standards technologies such Java,
XML and the IMS specifications to ensure multiplatform
adaptability making their elements reusable, durable and
accessible. To acquire those characteristics, it is necessary to
separate the educative content and its presentation, so the
HyCo platform save the contents in XML files, allowing the
introduction of LTS (Learning Technology Standard),
particularly the IMS specification.
The fact of separating the content and the presentation
forces to offer to the authors a way to generate an independent
result of the authoring tool. In this way, HyCo has an output
gallery that supports HTML, PDF, TXT, RTF and PS output
formats.
II. Used Technologies
First, we would like to describe the necessary technologies
and concepts used in this tool:
• ALE: Adaptive Learning System. This system conform its
learning components to build the most suitable adaptive
learning experience.
• Learning Designs: This is a sequence of educative
activities. This could attach several learning objects and
definitions such objectives, prerequisites, and activities to
complete or fulfil the lesson. These designs are defined
using the IMS LD (IMS Learning Domain) specification.
• Metadata: By using metadata objects, we give to the AAT
the characteristics of interoperation, reusability and
interchange among other systems.
• SLO´s: The Semantic Learning Objects, compliant with
IMS Metadata, stored as XML files in IMS metadata SLO
repositories. By using SLO´s, we could differentiate
between the educative content and the learning processes.
• XML: Extensive Markup Language, to ensure an
interchangeable tool using meta-data, building a well-
documented and deployable tool.
• IMS specification: Define a data model for the
representation of educative objects for learning systems.
Among the IMS specifications, we could find the IMS
QTI (IMS Question and Test Interoperability), for the
representation of questions and tests and the
correspondent results reports.
FIGURE II
H
YCO MODEL [5]
III. Proposal model for an Adaptive Assessment Tool (AAT)
In the figure III, we outline the main components and the
processes to develop an AAT, based on the HyCo model
(figure II) taking into account the most important modules to
configure our AAT model. The idea is to show the expected
flow to develop an adaptive unit of learning according to the
products made by the HyCo platform, but in this case, we will
talk about an adaptive assessment as the adaptive unit of
learning.
• Learning Objects: The teacher or the instruction
designer makes a series of test questions that will
integrate an educative resource. In this process, all the

Session F3B
0-7803-9141-1/05/$20.00 © 2005 IEEE July 7 – 9, 2005, Juan Dolio, Dominican Republic
ITHET 6
th
Annual International Conference
F3B-5
IMS metadata that can be inferred from the original
resource are defined, becoming into a SLO. After that, a
XML file is generated, specifically for each SLO,
designed exclusively for assessing purposes. This file is
FIGURE III
ASSESSMENT
ACTIVITY INTO THE HYCO MODEL
stored in an IMS Metadata SLO repository, allowing us to
have learning objects in form of assessment questions that
comply with the IMS QTI specification and that can be
attached to learning activities of learning designs, being
compatible with specific learning style tests, defined
below.
• Learning Style Tests: On the other side, we have to take
into account the learning styles of the students. Felder and
Silverman Learning Style Model and the Kolb’s
Experiential Learning Theory [5] are two well know
examples. The former proposes ten dimensions: two
related to the way students receive the information
(sensorial, intuition), and the other eight related to the
way the information is processed (visual-verbal;
inductive-deductive; active-reflexive; sequential-global).
Kolb’s approach also takes into account the way the
information is perceived (theorist and activist
dimensions), and the way the information is processed
(reflectors and pragmatist dimensions). However, the idea
is not to prescribe any learning style, but provide authors
with a flexible structure where different learning style
approaches can be described and used to characterize the
learning styles of learners and activities. Once the
learning style approach is selected, a learning style test
could be defined. Each of these tests could assembly a
series of assessment items into a specific test for a
specific learning style definition.
• Test Definition: When the learning objects (assessment
items) and the learning style tests are defined, a test
definition could be made in concordance with a learning
style definition. We want to recall that, as it is described
in the HyCo model, there are four kind of test: learning
style test, current knowledge and initial and final
knowledge. The formers are included in a unit of study.
• Delivery: To complete the release of the AAT we must
encapsulate it into a framework that integrate other
elements to complete the adaptation for the student:
o Learning Design: This process integrate
objectives and prerequisites for the test.
o Learning style: When we identify the student
learning style, we can select a definition test for
a specific learning style, in concordance with the
learning style of the student.
o Adaptation Rules: categorize the student
stereotypes, defining adaptive statements and
techniques.
• Interaction Model: Delivers an adaptive unit of learning,
including mainly the learning material and the test this
module tracks the behaviour of the student during the
interaction with the system when the learning activities
are visited and the results of the test.
• Updating the Student Model: Integrates the results of
the interaction model, storing the learning style and the
acquired knowledge for each the student.
C
ONCLUSIONS AND FURTHER WORK
Online assessment is an important step inside the e-learning
process because gives convenient feedback to all participants
in the process, helping to improve the learning and teaching
experience.
In this paper, we wanted to emphasize the role of the
assessment inside the e-learning process and defining the
factors of importance to the main elements that participate in
this process: the educative content and adaptation process, the
users or students and the teachers and assessors. We think that
the assessment activity takes place in a specific point of the
process as we show it in the figure I, and we conceptualized
the activity as the link that closes the chain of the e-learning
process.

Session F3B
0-7803-9141-1/05/$20.00 © 2005 IEEE July 7 – 9, 2005, Juan Dolio, Dominican Republic
ITHET 6
th
Annual International Conference
F3B-6
According to the new developments in the area of e-
learning we can see that most of them look to be compliant
with accepted standards like the LTS. This gives the
convenience to those developments to be interoperable and
adaptable to different platforms. In concordance, referring to
the assessment activity we can think that it must be
interoperable as well, because it is one element of the e-
learning process and plays an important role inside this
experience. When we talk about assessment we could define
some components of quality, especially for the users; some of
those are validity, reliability, flexibility, and fairness [6].
Adaptability is another key factor in assessment. Given
the fact that assessment is an important element of the e-
learning process and that this process looks to be
interoperable, then we can think that the assessment tool could
be used with different educative content administrators with
different conceptualizations and ways to design and apply a
test for their students. To face this situation it is necessary to
develop an assessment tool that give several ways to design an
test with different types of resources, different kind of
assessments, group of students, kind of questions, managing
schedules, etc.
Under this conceptualization, we want to create an
Adaptive Assessment tool (AAT) that could take into account
the specific characteristic of the HyCo system and be
intrinsically part of it.
We want to adopt the model developed for the HyCo
platform to integrate the assessment module into it, to do that
we will:
• Develop an application where the professor or
instructional designer could integrate the test questions,
following the same two-phases that in the HyCo process,
to integrate an XML file as output associated with a
repository of learning objects in metadata format,
following the IMS specifications.
• On the other side, we ensure that the assessment tool take
into account the pedagogical aspects to conform an
adaptive systems by considering the learning styles of
each student integrating this style with a set of questions
to create a Learning Test Definition.
• After that, following the integration with the HyCo
system, the test is attached with other learning activities
in the Learning Domain Model.
• The adaptation model is the component that integrates all
the definitions made with the Learning Domain Model:
the learning design, the test, learning style and adaptive
rules, generating an IMS LD file containing a deliverable
learning design to the next module, the Interaction Model.
• The Interaction Model delivers and adaptive unit of
learning to the student – an IMS CP file–, tracks the
behaviour of the student like the learning activities visited
and the result of the test made by him/her. At the end, this
module updates the student model.
Among the IMS specifications, namely the IMS QTI,
devoted entirely to the area of the assessment, which we will
analyze and evaluate to the development of the AAT for the
HyCo (Hypertext Composer), developed in the Institute of
Educational Sciences in the University of Salamanca, Spain.
A
CKNOWLEDGMENT
We want to thank to the group AWEG – Adaptive Web
Engineering Group – of the University of Salamanca for their
contributions and ideas for the development of this work.
Héctor Barbosa thanks the National System Council of
Technological Education (COSNET – Mexico) and the
National System of Technological Institutes (SNIT – Mexico)
for its financial support.
R
EFERENCES
[1] Backroad Connections Pty Ltd. “Assessment and Online Teaching”,
Australian Flexible Learning Framework Quick Guides series,
Australian National Training Authority, Version 1.00, October 15 2003,
pp.3
[2] Berlanga A., Garcia, F., “Towards Semantic Metadata for Learning
Elements”. In proceedings of the fifth. International Conference on
Information Technology Based Higher Education and Training ITHET
2004. Y. Akpinar (Ed.) (2004) pp. 572-577.
[3] Berlanga, A., Garcia, F., “An Open Model to Define Adaptive
Educational Hypermedia Systems Based on Learning Technology
Specifications”. In proceeding of the 3
rd
. International Workshop on
Web Semantics, WebS 2004. Held in conjunction with 15
th
. International
Database and Expert Systems Applications, DEXA 2004. IEEE
Computer Society Press. (2004) pp. 198-202. ISBN 0-7695-2195-9.
[4] Booth, R., Berwyn, C. “The development of quality online assessment in
vocational education and training”. Australian Flexible Learning
Framework, Vol.1, 2003, pp. 17
[5] Garcia, F., Berlanga, A., Moreno Ma. N., Garcia, J., Carabias, J. “HyCo
an Authoring Tool to Create Semantic Learning Objects for Web-based
E-Learning Systems”. In Web Engineering. Fourth International
Conference, ICWE 2004, proceedings. Published in N. Koch, P.
Fraternali, M. Wirsing (Eds.). Series: Lecture Notes in Computer
Science. Springer Verlag. Vol. LNCS 3140, pp. 344-348. ISBN 3-540-
22511-0.
[6] IMS QTI Item overview, Version 2.0 Public Draft, 07 June 2004.
http://www.imsglobal.org/question/qti_item_v2p0pd/indes.html
[7] Thorpe, M., “Assessment and ‘third generation’ distance education”, In
Exploring assessment in flexible delivery of vocational education and
training program, Australian National Training Authority, 2004, pp. 20.