Conference PaperPDF Available

Automated e-learning quality evaluation

Authors:

Abstract and Figures

The quality assurance and evaluation of e-learning is of high priority for any up-to-date higher educational institution. The paper is devoted on automation of related processes on the basis of a modern approach for integration of heterogeneous software systems. It presents the context, substance and objectives of a study related to the automated data retrieval in e-learning quality evaluation. It describes two specific experiments on such integration for e-learning quality evaluation in terms of students' satisfaction and in terms of compliance with appropriate quality standards. INTRODUCTION Quality(evaluation) being a key instrument of quality assurance and quality enhancement in higher education (HE) is one of the most typical components of the Bologna process. Since e-learning technology is becoming an integral part of contemporary learning activities in all learning modes offered by higher educational institutions, hence the evaluation of e-learning quality is assumed as a top priority. The monitoring, control and evaluation of the e-learning quality are key elements of more and more institutional and national quality systems (i.e. internal and external evaluation) of HE in Europe (e.g. Norway, Sweden, Great Britain, Bulgaria). The majority of the academic community understands the importance of this issue – something demonstrated by the large number of publications on the topic. Various experiments have been conducted to evaluate the quality of e-learning in the terms of its various components (like learning outcomes, learning process, academic staff, learning materials and activities, infrastructure, students satisfactions, etc.), based on different relevant quality models and standards (e.g. Context-Inputs-Process-Product of Stufflebeam, Reaction-Learning-Behaviour-Results of Kirkpatrick, etc.). nteresting examples in this field are the suggested models and approaches for quality evaluation of e-learning: as a whole – in terms of planning, development, process and product [8]; of the e-learning environments from the students' perspective [5], of the electronic educational resources [7], of the learners' satisfaction (including learners with special educational needs) [6] etc. Another extremely discussed aspect of e-learning quality assurance and evaluation is the need of automation of its related processes. The fact that the automated approach is typical for this form of education logically implies the conclusion that it is appropriate to use automated tools in evaluating its quality. This is the only possible manner to use effectively and in full degree all the data that have been collected and stored in electronic format during the organization and conduction of the e-learning. The paper is dedicated namely on this important aspect of the subject. It presents the context, essence and the purposes of a study on a modern approach for integration of heterogeneous software systems and its application for automated data retrieval in e-learning quality evaluation. Two specific experiments are described, carried out during this study, with regard to such integration for e-learning quality evaluation in terms of students' satisfaction and in terms of compliance with appropriate quality standards.
No caption available
… 
No caption available
… 
No caption available
… 
Content may be subject to copyright.
International Conference on e-Learning’15
150
Automated e-learning quality evaluation
Rositsa Doneva, Silvia Gaftandzhieva
Abstract: The quality assurance and evaluation of e-learning is of high priority for any up-to-date
higher educational institution. The paper is devoted on automation of related processes on the basis of a
modern approach for integration of heterogeneous software systems. It presents the context, substance and
objectives of a study related to the automated data retrieval in e-learning quality evaluation. It describes two
specific experiments on such integration for e-learning quality evaluation in terms of students’ satisfaction
and in terms of compliance with appropriate quality standards.
Key words: System Integration, Service Oriented Integration, e-Learning Quality Assurance,
Automated e-Learning Quality Evaluation.
INTRODUCTION
Quality(evaluation) being a key instrument of quality assurance and quality
enhancement in higher education (HE) is one of the most typical components of the
Bologna process.
Since e-learning technology is becoming an integral part of contemporary learning
activities in all learning modes offered by higher educational institutions, hence the
evaluation of e-learning quality is assumed as a top priority.
The monitoring, control and evaluation of the e-learning quality are key elements of
more and more institutional and national quality systems (i.e. internal and external
evaluation) of HE in Europe (e.g. Norway, Sweden, Great Britain, Bulgaria). The majority
of the academic community understands the importance of this issue – something
demonstrated by the large number of publications on the topic. Various experiments have
been conducted to evaluate the quality of e-learning in the terms of its various components
(like learning outcomes, learning process, academic staff, learning materials and activities,
infrastructure, students satisfactions, etc.), based on different relevant quality models and
standards (e.g. Context-Inputs-Process-Product of Stufflebeam, Reaction-Learning-
Behaviour-Results of Kirkpatrick, etc.).
nteresting examples in this field are the suggested models and approaches for
quality evaluation of e-learning: as a whole – in terms of planning, development, process
and product [8]; of the e-learning environments from the students’ perspective [5], of the
electronic educational resources [7], of the learners’ satisfaction (including learners with
special educational needs) [6] etc.
Another extremely discussed aspect of e-learning quality assurance and evaluation is
the need of automation of its related processes. The fact that the automated approach is
typical for this form of education logically implies the conclusion that it is appropriate to use
automated tools in evaluating its quality. This is the only possible manner to use effectively
and in full degree all the data that have been collected and stored in electronic format
during the organization and conduction of the e-learning.
The paper is dedicated namely on this important aspect of the subject. It presents the
context, essence and the purposes of a study on a modern approach for integration of
heterogeneous software systems and its application for automated data retrieval in e-
learning quality evaluation. Two specific experiments are described, carried out during this
study, with regard to such integration for e-learning quality evaluation in terms of students’
satisfaction and in terms of compliance with appropriate quality standards.
AUTOMATED DATA RETRIEVAL IN E-LEARNING QUALITY EVALUATION
(CONTEXT, ESSENCE AND PURPOSE OF THE STUDY)
Regardless of the needs for evaluating the e-learning quality, no matter whether for
internal or external evaluation, or for HE quality assurance and improvement, it is based
on the respective criteria and regulatory procedures. Usually such criteria systems (quality
International Conference on e-Learning’15 151
models) are very detailed and include a large number of evaluation criteria. For example,
the Swedish ELQ model for HE e-learning quality, consists of ten quality aspects, in
relation with which a total of 33 distinct criteria and 12 sub-criteria are evaluated while in
Bulgaria, the criteria system for evaluation of distance study programmes includes 46
quality indicators.
The quality evaluation, applying such standards, requires the collection, analysis and
interpretation of a huge amount of data in terms of learning materials used; infrastructure;
e-learning environment; tools and intensity of communication, cooperation and
interactivity; the application of a student assessment system; flexibility and adaptability of
the learning process; student and faculty support; team qualification and experience, etc.
On the one hand, the above is an argument for the automation of these activities. But
on the other hand, it poses a number of problems – it requires extraction and processing of
data from different information systems, which are often based on a different server,
operating system, communication platform, database etc. These problems are essentially
related to the need of integration of heterogeneous information systems.
Similar problems completely affect not only quality assurance, but also all aspects of
the automated information servicing of a typical university and of all various activities
necessary for the implementation of educational, management and administrative
functions of the institution as such.
The paper presents some of the results of a study, which aims to apply the
advantages of the modern approach for integration of heterogeneous software
applications, called Service Oriented Integration, in the field of HE.
The idea (fig. 1) is to apply the typical software architectural model Enterprise Service
Bus (ESB), based on Service-Oriented Architecture (SOA), to the integration of university
information systems. In accordance with the model, each university software system is
represented once by the relevant Web Services and is integrated into the so-called
“Integration Service Bus”. It allows the combination of these lower level services into
higher level business services by the Business Service Bus, so that they adequately meet
different new requirements, initiatives or changes in the information servicing of the
university.
In order to improve visibility, without any loss of generality, the scheme shown in fig.
1 reflects only the part of the studied architectural model which is of interest to the HE
aspect under discussion in this paper – e-learning quality evaluation. The scheme
illustrates:
Software applications: Learning Management System (LMS), Centralised system
for generating queries and reports and decision making (called University
Business Intelligence System – UBIS), other university systems and external
information systems;
Web Services: LMS Services, UBIS Services and other services;
Business services: Internal Evaluation of a Field of Study (FS), Internal Evaluation
of a University Unit, Internal Evaluation of the University and External Evaluation.
So far in the frame of this study a set of the web services necessary to represent the
different university systems to the Integration Service Bus has been specified (for some
examples see fig. 1). Various experiments have also been conducted on the software
implementation of web services, as well as on their usage for integration of different
university systems.
Two of these experiments, related to data retrieval from the LMS, aiming to automate
the quality evaluation of the conducted e-learning are presented in the following sections
of this paper. The results of such evaluation could be used within the framework of
different procedures for internal and external evaluation of a university and its educational
activity. The precise software systems, used in the experiments, are as follows:
one of the most widespread open source LMS – Moodle and
International Conference on e-Learning’15
152
specially developed UBIS, called UBIS-Jaspersoft, based on the popular software
Jaspersoft BI Suite, which allows the development of Business Intelligence
solutions for organizations of various type, including higher educational
institutions.
Figure 1. Integration of university information systems
EXPERIMENT: STUDENTS SATISFACTION
The first experiment is related to the integration of UBIS-Jaspersoft and LMS Moodle.
It aims to extract, analyse and interpret data about students’ satisfaction by the conducted
e-learning.
Figure 2. Electronic survey
For studying the students’ opinion the typical for each similar quality management
system approach is used, namely by conducting a survey. It is accomplished in an
electronic form. In order to fulfil this, the Moodle module Feedback is used as a tool for
creation of templates with questions that can be further used multiple times. The already
created survey template is included as part of the learning activities in each electronic
course (e-course) in order to be fulfilled by all the participating students after completion of
the training. The electronic questionnaire (see Fig. 2) contains around 50 questions for
quality evaluation of the conducted e-course in terms of its different characteristics, such
as course documentation and educational goals; learning materials and activities; team for
provision; communication; assessment methods; feedback; etc.
International Conference on e-Learning’15 153
Once the questionnaires are completed, the data is stored in the system’s database
and can be used for analysis of the results through the LMS Moodle module Feedback.
The problem that has to be solved within the scope of the experiment is that the built-in
module Feedback offers opportunities for generation of summarised and statistical reports
about the poll result only within a specific course and its participants.
In order for the obtained data from the survey to be used in accordance with the
purpose stated above (at internal and external evaluation in HE), it is necessary to provide
automated software tools for the synthesis and analysis of the results at a more general
level, e.g. for all e-courses in a field of study or for all e-courses in a scientific field (SF).
In solving this problem, the experiment is carried out in 4 (four) steps using the
system UBIS-Jaspersoft and the capabilities of its basic software Jaspersoft BI Suite for
creating reports and analyses by retrieving data from different sources, for storing and
organizing reports in a repository and for presenting them in the preferred by the user
form. Jaspersoft offers powerful tools for integration with various user software
applications through shared web services.
In Step 1 UBIS-Jaspersoft is integrated with Moodle database, which is set as a data
source for retrieving of the data from the students’ surveys and creation of reports,
reflecting student satisfaction. It should be noted that besides to relational databases (such
as the Moodle database) JasperSoft can be connected to just about any data source,
including JDBC, XML, CSV, Hibernate, POJO etc.
In Step 2 templates of analytical reports are developed, which can be later used to
generate the real reports containing summarised results from the conducted survey or
from other similar surveys, related to e-learning quality evaluation.
For templates description JasperSoft supports:
wide range of tools for visual design and good report layouts, representing data in
the form of tables, charts, or crosstabs-based reports;
a number of scripting languages (Java, Groovy, JavaScript, etc.) for construction
of expressions in order to declare report variables, to perform various calculations,
grouping data into the report, to specify report text field content, or to customize
further the appearance of report objects;
extensive set of query languages, including SQL, HQL, EJBQL, etc. for retrieving
data from different data sources.
In the present case 6 (six) templates are developed. They can be used in automated
surveys for the quality of e-courses in a concrete FS or SF in order to obtain
summarised information as follows:
the number of e-courses, in which conduction of the survey is planned (with added
questionnaires) by each FS offered in the university training;
a list of e-courses by the corresponding FSs, in which conduction of the survey is
planned;
the number of e-courses by FSs, where the survey is already conducted (with
completed questionnaires);
a list of e-courses by FSs, where the survey is conducted along with the
corresponding number of participants in the survey (number of completed
questionnaires);
summarised results of the survey by FSs;
summarised results of the survey by the evaluated characteristics of e-courses
and by SFs.
The templates include a presentation of the summarized results both in the forms of
tables and charts. Fig. 3 shows a screenshot from the template dedicated to generate the
summarised results of the survey by SFs during its design.
The templates include a presentation of the summarized results both in the forms of
tables and charts. Jaspersoft creates a source code in XML format for each designed
International Conference on e-Learning’15
154
template. The implemented templates are compiled in a special internal format and are
stored in the Jaspersoft repository, which is realized in Step 3. By this manner they can be
both used by the level of the very same UBIS-Jaspersoft system and by other external
application for the generation of the relevant reports, that are filled with data from the given
data source (Moodle database). The completed reports can be exported to a specified
document format (PDF, XLS, XLSX, XML, HTML, XHTML, CSV, DOC, etc.).
Figure 3. Template designing
Step 4. realizes the ultimate goal of the experiment to integrate LMS Moodle (as
external application) with UBIS-Jaspersoft through the shared UBIS-Jaspersoft web
services, represented in the integration service bus (fig. 1).
A new module is developed as a supplement to the LMS Moodle (as a Moodle report
on a system level), which adds to the system the already implemented reporting
functionality (Step1-3), using the corresponding web services. This allows the generation
of reports based on the above-mentioned templates by the LMS level.
This reporting functionality can be used in a similar way to expand the functionality of
other systems and for the development of higher level business services.
EXPERIMENT: CONFORMITY OF E-LEARNING QUALITY WITH STANDARDS
The purpose of the second experiment is to provide opportunities for LMS Moodle to
provide data to other specialized tools for automation of internal or external evaluation
procedures (other software systems or high level business services). That is why, the
extracted data must allow conformity verification of training conducted in LMS Moodle with
appropriate quality standards.
In accordance with the selected software architectural model, 4 (four) experimental
web services are developed to support corresponding Moodle integration. The services
and their functions are selected in such a way as to provide data (extracted from the
Moodle database), applicable in evaluation of some of the most common indicators of the
known e-learning quality standards. Table 1 describes web services, the results returned
by the functions implemented by them (from conceptual viewpoint) and respectively in the
evaluation of which quality indicator, they can be used.
International Conference on e-Learning’15 15
5
Table 1. Web-services for evaluation of e-learning quality indicators
Web
Service
Web Service Functions Results Quality Indicato
r
1. A list of team member names providing e-learning, the relevant e-courses and
their role (administrator, teacher, tutor, evaluator, educational content
designer, author, quality manager, etc.) in each of the courses
Team for provision of design,
implementation and
maintenance of e-learning
2. A list of various learning activities and resources used in the training (lesson,
page, book, quiz, assignment, chat, choice, database, forum, glossary)
and their number for each e-course
Provision of e-learning with
virtual learning activities and
resources
3. Teachers’ and students’ activity in usage of tools for synchronous communication
for each e-course
Teachers’ and students’ activity in usage of tools for asynchronous
communication for each e-course
Usage intensity of communication tools (synchronous and asynchronous) by
teachers and students
System for control and
stimulation of students’ and
teachers’ activity
4. Participation duration in e-learning (real use of activities and resources) by the e-
students (incl. in different courses)
Sustainability of the ensured
e-learning infrastructure
Web services are developed according to the documentation for the creation of web
services in Moodle [3, 4]. It is envisaged the functions of each service to provide
alternative access for client applications based on different protocols for client-server
communication (XML-RPC, SOAP, REST) and also to support the exchange of data via
the most popular for this purpose protocol (XML, JSON, AMF). The implemented web
services were tested through specially developed client applications.
CONCLUSION
The study and experiments, exposed in the current paper prove the feasibility of the
Service Oriented Integration approach for HE needs. It provides a common base for
integration of the full range of heterogeneous university information systems (providing
student admission, training, teaching, researches, management, educational marketing,
quality control, etc.) in order to improve the management, performance, reliability and
especially openness of a university to offer new services in response to specific needs.
The results described in the paper are achieved as part of a project for the
implementation of the studied architectural model carried out at the Plovdiv University [1].
The final stage of realizing an overall modern solution for automated quality evaluation of
e-learning is yet to take place. The building of the Integration Service Bus will be
completed by presenting through web services the student administration system (which
will provide data about students, curricula, etc.), academic staff development system
(providing data for teacher competences) etc. Based on this, it will be possible for different
processes related to assurance and evaluation of the quality of education to be
automatized (presented on the Business Service Bus level).
REFERENCES
[1] Gaftandzhieva S., Doneva R., Integrating Moodle with university information
systems, Proceedings of National Conference „Education and research in the Information
Society“ (G. Totkov and I.Koichev), Association for the Development of the Information
Society, Sofia, ISSN 1314-0752, 39-48.
[2] Kawachi Paul, Quality Assurance for OER: Current State of the Art and the TIPS
Framework, eLearning Papers, Issue 40. Assessment, certification, and quality assurance
in open learning, ISSN 1887-1542, pp. 3-13, January 2015.
[3] Moodle Docs, Adding a web service to a plugin, http://docs.moodle.org/
dev/Creating_a_web_service_and_a_web_service_function, last access 7.04.2015.
[4] Moodle Docs, Web services API, http://docs.moodle.org/dev/Web_services_API,
last access 7.04.2015.
International Conference on e-Learning’15
156
[5] Noaman A., A. Ragab, A. Fayoumi, A. Khedra, A. Madbouly, HEQAM: A Deve-
loped Higher Education Quality Assessment Model, Proceedings of the 2013 Federated
Conference on Computer Science and Information Systems, pp. 739–746, 2013.
[6] Rabai Latifa Ben Arfa, Neila Rjaibi, Assessing Quality in E-learning including
learner with Special Needs, Proceedings of The Fourth National Symposium on
Informatics, Technologies for Special Needs, April 23-25, 2013.
[7] Wetzler Ph., St. Bethard, K. Butcher, J. Martin, T. Sumner , Automatically
Assessing Resource Quality for Educational Digital Libraries, Proceedings of the 3rd
Workshop on Information credibility on the web table of contents, Madrid, Spain, 2009.
[8] Zhang W., Y L Cheng, Quality Assurance in E-Learning: PDPP Evaluation Model
and its Application, The International Review of Research in Open and Distance Learning,
Research Articles, Vol. 13, No3, pp. 66-82, June 2012.
ABOUT THE AUTHORS
Prof. Rositsa Doneva, PhD, Plovdiv University “Paisii Hilendarski”, Department of
Electronics, Communications and Information Technology, Е-mail: rosi@uni-plovdiv.bg.
Silvia Gaftandzhieva, PhD student, Plovdiv University “Paisii Hilendarski”,
Department of Computer Science, Е-mail: sissiy88@uni-plovdiv.bg.
The paper has been reviewed.
... електронни регистри на българските студенти и преподаватели. Забелязва се и определен прогрес в автоматизираното оценяване на качеството (ОК) на обекти на електронното обучение -от среди за дистанционно обучение на висшите училища(ВУ) [1, 2], до е-курсове, дигитални дейности и ресурси [3,4,5]. ...
... Хилендарски" бяха проведени изследвания за възможностите за автоматизирано извличане на данни за качеството на обекти от сферата на ВО и анализирани възможностите за автоматизирано извличане на данни за оценяване на измерители от критериалните системи на НАОА (специалности в дистанционна форма на обучение, институция, професионални направления и др. [3,4,6]). В резултат на проведените изследвания беше проектирана и експериментирана софтуерна система КОМПАС 2.0 [1]. ...
... В резултат на проведените изследвания беше проектирана и експериментирана софтуерна система КОМПАС 2.0 [1]. С помощта на КОМПАС 2.0 е изготвен доклад-самооценка на средата за дистанционно обучение на ПУ (2015 г.), а редица приложения (документи и справки) към него са генерирани чрез автоматизирано справки в система за е-обучение [3,4,6]. ...
Conference Paper
Full-text available
Conceptual and Computer Modeling of Processes with Accumulation and Synthesis of Data in Higher Education: A significant part of the contemporary university information systems lack the tools for modelling, modification , virtualization and accompanying of processes. The paper introduces a software prototype, based on process virtualization tools in different spheres of higher education. A major role among them is played by tools for modelling and dynamic management of processes in corporate information infrastructures. Processes are modelled as workflows of activities. They are accompanied by visualization methods of related information resources stored in digital repositories (available for personal use or collaboration).
... за специалност, професионално направление или област на висше образование, е създаден специализиран модул ST_Surveys в Moodle. Модулът разширява функционалността на представено в [7,8] софтуерно средство за автоматизиран анализ на резултати от анкетни проучвания за удовлетвореността на студенти от провеждани е-курсове. ...
... Специализираният модул ST_Surveys разширява функционалността на Moodle на базата на следните софтуерни решения [7,8]: ...
Conference Paper
Full-text available
The paper presents a model for quality evaluation of test items (TI) and tests used in various forms of e-learning in higher education. The model is based on surveys among students and experts concerning the design, development, implementation and evaluation of TI and tests in e-learning. The software module has been developed and experimented in order to processing the surveys. It allows extracting and summarizing of data from other university data sources, not only from the system where the e-learning has been conducted.
... In connection with the external evaluation by accrediting institutions, several studies have been conducted for the automated extraction and analysis of data for distance learning quality evaluation, including data on the used learning materials, infrastructure, e-learning environment, means of communication and collaboration, student evaluation system, flexibility and adaptability of the learning process, student support, team qualification, etc. Doneva and Gaftandzhieva [90] explore the possibilities for automated data extraction for evaluating the criteria from the criteria system of NAOA for evaluating distance learning programmes were analyzed. This analysis aims to determine which data can be extracted from university systems (e-learning environment, learning process management system, academic staff development system, etc.) to support distance learning quality assessment. ...
... A special case is using the Subsystem B (B-Si) capabilities to define accumulative functions (templates), depending on the specific procedure for QE and IT infrastructure of the institution. Clearly, the automated QE of DL requires collection, analysis and interpretation of a huge amount of data in terms of the attitude of students and experts to learning courses, software tools used, etc., from different information systems [23]. Therefore, the choice of templates appropriate for QE of DL is made on the basis of the analysis results. ...
Article
Full-text available
The paper aims to propose, investigate and test suitable means for automation of the processes for dynamic quality evaluation of objects in a given subject area. On the basis of the theoretical study a number of conceptual and computational models are proposed. In consequence, the architecture of a software system for dynamic quality evaluation is defined and a corresponding software prototype is built over an existing university information infrastructure. General models are applied for dynamic quality evaluation of different objects in the field of higher education.
... На базата на предложените модели след направен анализ на възможностите за автоматизирано извличане на данни от университетската информационна инфраструктура [4] са създадени концептуални и компютърни модели на методики, съответстващи на критериалните системи на Националната агенция за оценяване и акредитация-НАОА [5] за акредитация на специалност с дистанционна форма на обучение, професионални направления и специалности от регулираните професии и на докторски програми. Разработени са и софтуерни средства за динамично извличане, натрупване и обобщаване на данни, свързани с оценяване на е-курсове [12,14] и на проекти с представяния (или управление), които са уеб базирани [13]. ...
Conference Paper
Full-text available
DYNAMIC QUALITY EVALUATION IN HIGHER EDUCATION (WITH APPLICATIONS IN E-LEARNING) Abstract. The paper presents a general model of process for quality evaluation (QE). Conceptual and computational models of methodology for QE are defined on its basis as well as the functional specifications of a software system for dynamic QE (SDQE). The SDQE provides possibility to generate reports (for the needs of QE procedures) based on given time schedule and data mining from different information sources. Thus, the SDQE can be used by any quality control system to prepare the necessary reports and documents automatically. Corresponding software prototype was built on an existing university information infrastructure. Experiments are performed for automation of procedures of the National Evaluation and Accreditation Agency, for e-learning quality management, etc. The results of the study can be multiplied for QE in other subject domains.
... FETCH-DEP is developed on the base of the proposed architecture (see Figure 1) in 4 (four) steps [5,6]: ...
Article
Full-text available
The paper presents an approach towards project quality, aiming to reach a higher maturity level in the quality management of the finishing this year FETCH project (Future Education and Training in Computing: How to learn anytime anywhere) funded by the European Commission Lifelong Learning Programme. It describes the concept of this approach and the developed corresponding software tool for dynamic evaluation of the project progress.
Article
Full-text available
This paper presents a study on known approaches for quality assurance of educational test and test items. On its basis a comprehensive approach to the quality assurance of online educational testing is proposed to address the needs of all stakeholders (authors of online tests, teachers, students, experts, quality managers, etc.). According to the proposed approach is developed an original software application Test Quality Evaluation (TQE) for the automation of the stakeholders' activities for quality assurance of educational tests throughout the whole lifecycle. The application retrieves and provides analysis of data from online tests conducted and specially designed surveys for quality evaluation of educational tests by students and experts. It allows tracking and evaluating the quality of educational tests in real time and provides the related quantitative data in different levels of generalization-in the level of a separate educational test, of educational tests of an entire course, or educational tests of a subject area. The software application has been put under real-time testing for quality evaluation of educational tests, included in e-learning courses from different subject areas that prove its applicability.
Article
E-learning website selection is a deliberate decision with a significant influence on the educational sector. E-learning website selection decision is based on multiple attributes/criteria. In the present research, the various attributes having a huge influence in the evaluation process of E-learning websites that are termed collectively as ranking criteria were identified and extracted from the available literature at the first step. Now, the relative importance or the significance of each ranking criteria was determined by collecting data through the interviews with experts and applying some aggregate operations on that data. Simply, the priority weight of each ranking criteria was determined at the second step. The third step is concerned with the evaluation work of the alternatives against the identified ranking criteria by considering the priority weights determined at the second step. In the present study, E-learning websites selection problem is resolved using matrix method which is capable to solve such types of multiple-attribute decision making (MADM) problems. Further, the results obtained from the proposed methodology are compared with the existing approaches namely Weighted Distance Based Approximation and Technique for Order Preference by Similarity to Ideal Solution to validate the applicability and utility of the proposed method.
Conference Paper
Full-text available
Purpose: The paper proposes an alternative to the traditional way of conducting surveys within internal university systems for quality assurance using mobile technologies in order to increase the students' activity. Design/methodology/approach: An analysis of the needs of internal university quality systems in conducting surveys as well as an overview of existing software tools for conducting mobile surveys have been made. After specifying the functional and non-functional requirements the mobile application for conducting surveys for the purposes of internal systems for quality assurance of higher education has been developed. Experiment for its application in specific surveys are conducted. Findings: The developed mobile application allows conducting surveys within university systems for quality assurance and tools for authorized group of users that allow monitoring of the students' activity in surveys and automated analysis of the results. Research implications: A solution relating to automation of the process of interviewing and summarizing the data in conducting surveys that are an integral part of the institutional quality assurance systems of universities is proposed. Thus the study supports the development of these systems in the direction of building a coherent European Higher Education Area. Practical implications: The results of the study would certainly influence positively for improvement of practices for quality assurance in higher education institutions (see. Originality/value) Originality/value: The developed and probated at Plovdiv University mobile application for conducting surveys is probably the first of its kind in the country. Its means for automated monitoring of the students' activeness as survey participants and for subsequent analysis of the survey results allow members of university quality committees to generate summary reports. But more over they could monitor ongoing surveys and analyse intermediate data at any time. The results of the presented research promise to be useful for the other educational institutions as well.
Thesis
Full-text available
MODEL AND SYSTEM FOR DYNAMIC QUALITY EVALUATION IN HIGHER EDUCATION The main objective of the PhD thesis is to propose, investigate and test suitable means for automation of the processes for dynamical quality evaluation of objects in a given subject area, especially in higher education. The study examined the general theory, existing organizations, models, standards and systems for quality evaluation in higher education. On the basis of the theoretical study a number of conceptual and computational models are proposed, as follows: • general model of a process for quality evaluation; • model of a system for dynamic quality evaluation; • model of a methodology for quality evaluation. In consequence the architecture of a software system for dynamic quality evaluation is defined and a corresponding software prototype is built over an existing university information infrastructure. General models are applied for dynamic quality evaluation of different objects in the field of higher education, as: • objects that are evaluated according to the criteria system of the National Evaluation and Accreditation Agency (using the software prototype for dynamic quality evaluation); • e-learning courses (standalone software application is developed); • web based presented collaboration projects (standalone software application is developed). Applications are experimented in real situations at Plovdiv University and prove the adequacy of the models. The results obtained in the thesis could be replicated for quality evaluation in other subject areas.
Conference Paper
Full-text available
High educational institutions use different information systems to automate management, operations and processes, decision making and educational services provided. Student information systems, е-Learning systems, Library systems, etc. are such examples. Same data usually is entered manually into multiple systems, which could lead to errors. In addition, part of information in a certain system has to be sent to other institutions. This imposes the need of examining the methods used for integration of heterogenic software systems that is the subject of the present work. An appropriate solution and corresponding experiments for integration of Moodle with information systems of the universities, based on service-oriented architecture, are proposed.
Conference Paper
Full-text available
With the rise of community-generated web content, the need for automatic assessment of resource quality has grown, par- ticularly in the realm of educational digital libraries. We demonstrate how developing a concrete denition of qual- ity for web-based resources can make machine learning ap- proaches to automating quality assessment tractable. Using data from several previous studies of quality, we gathered a set of key dimensions and indicators of quality that were commonly identied by educators. We then performed a mixed-method study of digital library quality experts, show- ing that our characterization of quality captured the subjec- tive processes used by the experts when assessing resource quality. Using key indicators of quality selected from a sta- tistical analysis of our expert study data, we developed a set of annotation guidelines and annotated a corpus of 1000 dig- ital resources for the presence or absence of our key quality indicators. Agreement among annotators was high, and ini- tial machine learning models trained from this corpus were able to identify some indicators of quality with as much as a 18% improvement over the baseline.
Conference Paper
This paper presents a developed higher education quality assessment model (HEQAM) at King Abdulaziz University (KAU). This is because of; there is no universal unified quality standard model that can be used to assess the quality criteria of higher education. Besides, there are shortcomings in the coverage of some current educational quality standards. A Developed questionnaire to examine the quality criteria at KAU is investigated. The analytically hierarchy process is used to identify the priority and weights of the criteria and their alternatives. The model is constructed of three levels including eight main objectives and 53 alternatives. It included e-services criteria which is one of the recent university components, in addition to new sub-criteria for enhancing the model. It produces important recommendations to KAU higher authorities for achieving demanded quality services. Also, it helps KAU to achieve one of its strategic objectives to be a paperless virtual university.
Quality Assurance for OER: Current State of the Art and the TIPS Framework, eLearning Papers, Issue 40. Assessment, certification, and quality assurance in open learning
  • Kawachi Paul
Kawachi Paul, Quality Assurance for OER: Current State of the Art and the TIPS Framework, eLearning Papers, Issue 40. Assessment, certification, and quality assurance in open learning, ISSN 1887-1542, pp. 3-13, January 2015.
Adding a web service to a plugin
  • Moodle Docs
Moodle Docs, Adding a web service to a plugin, http://docs.moodle.org/ dev/Creating_a_web_service_and_a_web_service_function, last access 7.04.2015.
Assessing Quality in E-learning including learner with Special Needs
  • Neila Rabai Latifa Ben Arfa
  • Rjaibi
Rabai Latifa Ben Arfa, Neila Rjaibi, Assessing Quality in E-learning including learner with Special Needs, Proceedings of The Fourth National Symposium on Informatics, Technologies for Special Needs, April 23-25, 2013.
Quality Assurance in E-Learning: PDPP Evaluation Model and its Application The International Review of Research in Open and Distance Learning
  • W Zhang
  • Cheng
Zhang W., Y L Cheng, Quality Assurance in E-Learning: PDPP Evaluation Model and its Application, The International Review of Research in Open and Distance Learning, Research Articles, Vol. 13, No3, pp. 66-82, June 2012. ABOUT THE AUTHORS Prof. Rositsa Doneva, PhD, Plovdiv University " Paisii Hilendarski ", Department of Electronics, Communications and Information Technology, Е-mail: rosi@uni-plovdiv.bg. Silvia Gaftandzhieva, PhD student, Plovdiv University " Paisii Hilendarski ", Department of Computer Science, Е-mail: sissiy88@uni-plovdiv.bg. The paper has been reviewed.
Totkov and I.Koichev), Association for the Development of the Information Society
  • S Gaftandzhieva
  • R Doneva
Gaftandzhieva S., Doneva R., Integrating Moodle with university information systems, Proceedings of National Conference "Education and research in the Information Society" (G. Totkov and I.Koichev), Association for the Development of the Information Society, Sofia, ISSN 1314-0752, 39-48.
The International Review of Research in Open and Distance Learning
  • W Zhang
  • Cheng
Zhang W., Y L Cheng, Quality Assurance in E-Learning: PDPP Evaluation Model and its Application, The International Review of Research in Open and Distance Learning, Research Articles, Vol. 13, No3, pp. 66-82, June 2012.