Content uploaded by Živana Komlenov Mudrinski
Author content
All content in this area was uploaded by Živana Komlenov Mudrinski on Jul 31, 2014
Content may be subject to copyright.
Informatica 37 (2013) 221–230 221
Usability and Privacy Aspects of Moodle: Students' and Teachers'
Perspective
Mirjana Ivanović, Zoran Putnik, Živana Komlenov
Faculty of Science, University of Novi Sad, Novi Sad, Serbia
Tatjana Welzer, Marko Hölbl, Tina Schweighofer
University of Maribor, Faculty of Electrical Engineering and Computer Science, Maribor, Slovenia
Keywords: learning management system, privacy, security, experiences, usability, localization
Received: August 3, 2012
Choosing complex sets of tools, usually called learning management systems (LMSs), for creating
perfect blends of traditional classroom activities and the most appropriate e-learning course
components has become a common practice. Our institutions have opted for open source LMS Moodle.
After years of its application in everyday teaching practice we were inspired to analyse the effectiveness
of this platform. In this paper, results of the surveys compiled in order to reflect the student and teacher
experiences with Moodle are presented. Main focus is providing insights into opinions, expectations and
possible reluctance regarding usability and privacy when using its functionalities
Povzetek: V prispevku so predstavljene dolgoletne izkušnje in analize sistema Moodle.
1 Introduction
Contemporary standards in education require usage of
different tools in order to supplement teaching and
learning processes, as well as efficient assessment. A
learning management system is often the foundation of a
reliable e-learning platform and complies with standards
and best practices recommended by respectable
educational and corporate stakeholders (Georgouli,
Skalkidis, & Guerreiro, 2008).
At our departments at the Faculty of Science,
University of Novi Sad in Serbia and the Faculty of
Electrical Engineering and Computer Science, University
of Maribor in Slovenia such a solution is used for the
design and delivering of courses that are supporting
classroom training (Budimac, Putnik, Ivanović, Bothe, &
Schützler, 2011).
Several years ago we decided to use and possibly
extend an existing e-learning platform for our eCourses
instead of developing a new one from scratch. After
testing several systems we drew conclusions on the
available tools. The system we chose had to be one of the
established general purpose LMS solutions, preferably an
open source one (Ahmed, 2005). Such a platform, apart
from its flexibility and considerable cost savings, would
offer possibilities for extensibility and customization
according to one’s specific needs.
The evaluation of open source LMSs was conducted
according to a set of minimum criteria, which included
active community, stable development status, good
documentation, didactic objective and focus on the
presentation of content and communication
functionalities.
Our final choice was Moodle (Rice, 2008), for its
fine basic features, great extensibility and even some
potential adaptability features which were further
developed in Novi Sad (Komlenov, Budimac, &
Ivanović, 2008). A number of comparative studies and
research papers (Al-Ajlan & Zedan, 2008; Di Domenico,
Panizzi, Sterbini, & Temperini, 2005; Graf & List, 2005;
Munoz & Van Duzer, 2005; Stewart et al., 2007)
corroborated our choice.
Moreover, this solution has been accepted by the
University of Maribor as the official LMS, and has also
been introduced at a significant number of faculties in
Novi Sad in the last couple of years, which certainly
makes joint studies and reuse of teaching material among
our universities more feasible (Bothe, Budimac,
Cortazar, Ivanović, & Zedan, 2009).
Moodle is a modular and extensible platform which
offers features to support different educational styles. It
chiefly follows the established usability conventions
(Melton, 2006): it has a simple interface, uses a minimal
number of words, features roll-overs providing extra
information, etc. Still, usability and privacy concerns
must be addressed in detail when using such a solution.
In this paper the results of two surveys are presented
in order to reflect the experiences of students and
teachers with Moodle, regarding mainly those issues, and
consequently the impact of using this LMS in everyday
teaching practice on the academic achievements of
students. The study was conducted as a part of a bilateral
project between our institutions. Participation in the
study was voluntary and anonymous for both students
and teachers.
The results of our study should be of interest to
university administrators, faculty members, and students
who plan to offer, teach, or take courses implemented in
Moodle. Also it can help many universities that are still
deciding the extent of their offerings of online or blended
222 Informatica 37 (2013) 221–230 M. Ivanović et al.
courses and the most appropriate platforms to use in
structuring their offerings.
The rest of the paper is organized as follows. In
Section 2 research endeavours somewhat similar to ours
are observed, since their results and methodologies
applied induced our investigation. However, it is focused
on slightly different aspects of the platform in question.
Section 3 discusses the survey outline and methods used
in the conducted research. The discussion of the results
from both students’ and teachers’ perspectives is
presented in Section 4. Conclusions are drawn in Section
5 to foster future research and innovations in online
teaching practice.
2 Related work
A significant number of published papers report on
students’ and/or teachers’ perceptions of e-learning and
the usability of the employed e-learning tools. This
certainly includes Moodle (Kakasevski, Mihajlov,
Arsenovski, & Chungurski, 2008; Kennedy, 2005;
Kirner, Custódio, & Kirner, 2008; Liyanagunawardena,
2008; Melton, 2006), as one of the LMSs most frequently
used at universities worldwide. Their focus groups were,
however, usually students participating in one selected
study program or even more often a single course, quite
rarely complemented with their teachers.
However, some of the studies provided valuable
conclusions and provoked further research. While some
of them focused on technology-based components of
such platforms, others studied the human factor of those
systems considering student and instructor satisfaction,
the importance of participant interaction in online
environments, etc. There were even attempts to develop
comprehensive assessment models, incorporating
concepts from both information systems and education
disciplines (Ozkan & Koseler, 2009).
It was, for instance, found that most information
technology majors perceive learning to be more fun and
of better quality within a technology-enhanced online
learning environment (Parker, 2003). Furthermore,
students who take online courses perceive a higher level
of quality in their educational endeavours (Hannay &
Newvine, 2006). However, lack of interaction, presence,
or both may result in students’ different observations on
how well they may or may not have performed in an
online class (Picciano, 2002; Song, Singleton, Hill, &
Koh, 2004).
There seems to be a strong positive correlation
between the degree of social presence and perceived
learning as well as perceived quality of the instructor
(Richardson & Swan, 2003). Not surprisingly, it was also
revealed that participants of elective online courses tend
to rate the modules positively while those in the
obligatory courses often rate them more negatively
(Smart & Cappel, 2006).
Students that experienced at least one well designed
course enriched with resources, timely feedback and
interactions with teachers generally report positive
experiences (Weaver, Spratt, & Nair, 2008). The
instructor’s support in learning in fact strongly
contributes to learning achievements and course
satisfaction.
Besides the instructor’s expertise and support, only a
few other variables proved to be important for students’
perceptions of learning achievements and course
satisfaction (Paechter, Maier, & Macher, 2010): the
structure and coherence of the teaching material and the
course, the stimulation of learning motivation, and the
facilitation of collaborative learning.
Teachers may also exhibit differing opinions about
online learning and its effectiveness for the student
(Bisoux, 2007). It is not rare for teachers to still perceive
online learning as having numerous shortcomings,
including (Totaro, Tanner, Noser, Fitzgerald, & Birch,
2005): the lack of instructor-student/student-student
interaction; no structured classroom environment;
students tending to teach themselves the course material;
the difficulty of teaching quantitative courses online; the
challenges of administering exams online, etc.
The open source learning management system
Moodle is widely adopted at many universities and other
organizations thanks to its tightly integrated set of tools
designed from a social constructivist perspective. The
advantages it offers over other (commercially) available
LMSs were often analysed during the last couple of
years.
The benefits of Moodle over rather popular
proprietary LMSs like Blackboard (Kennedy, 2005) can
be seen in Moodle’s outstanding facilities developed to
support communication in various ways, but also in
providing better structure for all sorts of courses, i.e.
more functional and likeable course organization.
Additionally, Moodle’s registration system and
assignment submission module (Melton, 2006) and other
standard modules (Kakasevski et al., 2008) were also
assessed to some degree in terms of usability.
Nevertheless, surveys conducted in parallel at more
than one university, with comparable groups of students
of similar background, as well as their teachers (Tanner,
Noser, & Totaro, 2009), are quite rare, especially those
that address not only basic but also some important
advanced features of the chosen platform.
Accenting privacy issues is also very important since
the urge to protect security and privacy of data has lately
become significant and extensively studied subject (Eibl,
2009; Klobučar, Jenabi, Kaibel, & Karapidis, 2007;
Weippl & Tjoa, 2005).
Therefore we decided on conducting such a twofold
survey at our institutions to provide ourselves and our
colleagues from other universities, but also other
interested parties, with possibly useful students’ and
teachers’ insights in the current usability and privacy
aspects of Moodle.
3 Survey outline and methods
The survey was twofold – one part aimed at students and
the other at teachers. It was composed of a majority of
closed questions, and some specific ones offering the
possibility for answering more freely. Most open
questions were bond to the closed ones in two ways:
Usability and Privacy Aspects of Moodle… Informatica 37 (2013) 221–230 223
additional description after choosing an answer (e.g.,
“Do you distribute you teaching material
periodically or at the beginning of the semester?
Periodically. Period: ____”);
additional description after answering (e.g., "Do you
use the blog functionality within Moodle? YES NO
– If NO, please indicate why: ____").
We distributed the survey electronically, using
Moodle’s Feedback module. The response of both
students and teachers was surprisingly fast – we needed
only about 10 days to collect all answers in the survey for
students and about 3 weeks to conduct the survey for
teachers.
However, there were some differences between the
aims of the survey used to collect teachers’ experiences
and opinions and the one prepared for students. The goal
of the featured survey for students was to provide
insights into their opinions, expectations and reservations
regarding the usability of Moodle, the quality of teaching
material available, usage of assessment means,
communication and collaboration tools, as well as their
privacy concerns.
The survey intended for teaching staff was compiled
of differently formulated questions. Yet the goal was
similar – to provide insights into their experiences,
opinions, expectations and cautiousness regarding the
effects of using Moodle in their teaching practice.
Teachers were required to assess the usability of various
Moodle’s modules and comment on the ways they
employ them in the courses they maintain and teach.
4 Results and discussion
Our institutions, the Faculty of Science, University of
Novi Sad in Serbia and the Faculty of Electrical
Engineering and Computer Science, University of
Maribor in Slovenia, have been implementing e-learning
using Moodle for several years now.
The Faculty of Electrical Engineering and Computer
Science, University of Maribor employed Moodle for the
first time as an obligatory teaching tool in the year 2007
when the execution of the teaching process according the
Bologna declaration started. Nevertheless, Moodle had
also been used to some extent before that.
The Faculty of Science, University of Novi Sad
started using Moodle in 2004. Until now the majority of
courses have been implemented in this LMS, especially
those taught within Computer Science study programs.
After Moodle was used for several years, we decided
to analyse its many specific aspects. A joint project
between our institutions gave us the possibility of
investigating possible differences in two different
study/work environments.
4.1 Students’ perspective
The survey was conducted at both institutions, with
comparable numbers of students (136 in Slovenia, 130 in
Serbia). However, the distribution of participants
according to their year of studies was different (Table 1),
because the existing undergraduate studies in Slovenia
have recently been transformed into a three-year
program, which also involved obligatory use of Moodle,
so only students from the first two years were available
at the time the survey was conducted.
Slovenian results were collected at study programs
Computer Science and Informatics (65.44%), and
Communication Technologies and Media
Communications (34.56%), while Serbian survey
participants were all students of Computer Science.
Table 1 depicts the distribution of survey participants
according to their year of studies at both universities.
Year of study Novi Sad Maribor
1 43.85% 36.76%
2 16.92% 63.24%
3 17.69%
4 10.00%
5 8.46%
PhD 3.08%
Table 1: Survey participants according to their year of
studies.
Gender-wise, there was more than 2/3 of male, and
1/3 of female students (Table 2).
Gender Novi Sad Maribor
male 67.69% 77.21%
female 32.31% 22.79%
Table 2: Survey participants according to their gender.
4.1.1 Overall quality of the existing teaching
material
The majority of students assessed the quality of the
teaching material currently available at our Moodle sites
(mainly static content plus some electronic lessons for
self-study purposes, enriched by assessment and
communication facilities) as very good or good (Figure
1). Interestingly, there were 10 times more students that
graded the available material as excellent in Novi Sad,
and also none of the survey participants there assessed
the available resources as very bad.
It might be possible that students from an EU
university have greater expectations than the students in
a developing country, but this result certainly shows that
it is possible to develop and conduct courses of high
quality even without any special funding or much
institutional support.
What we were especially curious about were
students’ suggestions on how to improve the teaching
material quality. They included the following:
introducing additional exercises with different difficulty
levels or examples of previous exams, more tests and
assignments for students’ self-evaluation, lessons with
adaptive elements, video content, links to additional
literature, etc.
224 Informatica 37 (2013) 221–230 M. Ivanović et al.
All this suggests that students actually value course
creators’ efforts to involve more complex and interactive
activities and resources in their courses, which in some
cases might require usage of additional, either third-party
or own, modules implemented for Moodle.
29,23%
53,08%
15,38%
2,31%
0,00%
44,12%
2,21%
49,26%
1,47%
2,94%
0,00%
10,00%
20,00%
30,00%
40,00%
50,00%
60,00%
excellent very good good bad very bad
Novi Sad Maribor
Figure 1: Overall quality of teaching material.
4.1.2 Graded tests
Quality of Moodle features, primarily Quiz module and
similar, that enable students to take graded tests was also
investigated in this research. 72.31% (Novi Sad) /
42.65% (Maribor) of survey participants had already
been examined in such a way. They found it to be very
convenient and they especially valued the increased
speed of the grading process. The main problems that
arouse while solving online tests identified by our
students are the time limits and the possibility of
hardware failure, which lowers their concentration.
Only 26.15% (Novi Sad) / 35.29% (Maribor) of
students think that tests done using computers offer more
possibilities to cheat compared to the usual settings when
paper tests are used. Nevertheless, the teaching staff
keeps constant efforts to reduce this ever existing
assessment problem to the minimal possible level by
administering tests in controlled environments like
supervised computer rooms, limiting the access to certain
IP addresses, etc. This practice is understood and
supported by 81.54% (Novi Sad) / 44.12% (Maribor) of
students. Thus, apart from the fact that online testing is
much more employed in grading students in Novi Sad
than in Maribor, it is interesting to notice that Serbian
students are less resistant to all kinds of cheating
restrictions.
4.1.3 Collaborative assignments
Regarding teamwork experiences, 27.94% of students in
Maribor and 52.31% of students in Novi Sad (almost
twice as much) had already done some collaborative
assignments using Moodle’s modules suitable for such
efforts (Wiki, Workshop, etc.). They generally found
these activities both challenging and valuable as learning
experiences, and responded very well to the team-
building practice promoted through them.
The fact that students are willing to work in small
teams in order to solve various assignments, together
with their satisfaction with what Moodle’s modules
intended to foster collaborative activities offer, is
backing the already proven hypothesis that students who
use opportunities in self-regulated and collaborative
learning experience higher learning achievements
(Paechter et al., 2010).
4.1.4 Usage of communication tools
Moodle has communication capabilities leaning towards
Web 2.0 functionalities, like blog, forums, wiki, or chat.
However, students are not very eager to use those
features in their studies (Table 3).
Most of them say they still prefer personal
communication with professors and teaching assistants or
use email communication instead, which to some extent
fits global trends noticed in other studies.
Tools Novi Sad Maribor
forums and
instant
messages
18.46% 39.71%
blogs 23.08% 11.03%
chat 26.15% 30.88%
Table 3: Frequent usage of communication tools.
4.1.5 Expressing opinions
Considering online surveys like this one, great majority
of students, 91.54% (Novi Sad) / 80.15% (Maribor) of
them, had no problem with filling the surveys out if they
were to be completed anonymously. In general they
value every opportunity to state their opinion on matters
that directly influence the quality of the courses they
attend. The rest of the surveyed students expressed their
lack of belief in the possibility to be completely
anonymous while filling out online surveys in
environments like Moodle which systematically keep
records of all user actions.
However, it is important to notice that 26.92% (Novi
Sad) / 62.5% (Maribor) of the students are afraid of the
consequences if they express a negative opinion or
criticize a teacher within a Moodle course, for instance
using a discussion forum. Yet only 6.92% (Novi Sad) /
16.47% (Maribor) of those students claim that their fear
is based on some previous negative experience.
Additionally, students would assess their teachers and
courses they attend more freely if they would be assured
anonymity.
4.1.6 Privacy concerns
Most of the survey participants (93.85% in Novi Sad and
91.18% in Maribor) are satisfied with privacy in Moodle.
Others that are not so satisfied gave their reasons for that.
As the most frequent cause they stated that other students
who participate in the same course are able to track their
online status and participation in various activities (for
example submissions of assignment solutions). So the
majority of students stated their wish for privacy,
Usability and Privacy Aspects of Moodle… Informatica 37 (2013) 221–230 225
signalling that at least Moodle’s grade book should be
more frequently and thoroughly used by their teachers.
On the other hand, some Serbian students in fact
wished to be able to see all student grades thinking of
that possibility as of a way to improve the transparency
of grading. These dilemmas support previous evaluations
of student perceptions of various e-learning components
that showed that the students’ strongest preference was to
submit assignments and have the ability to check their
grades online (Buzzetto-More, 2008).
A specific view of privacy in Moodle was
investigated in detail, namely who should be able to
access data from other users’ profiles (Figure 2).
Considering the question used to explore which pieces of
information from user accounts should be hidden,
expected answers such as email addresses, phone
numbers, student ID numbers, etc. were received.
Some students also mentioned hiding first/last access
times and activity logs of course participants. Most of
these problems, now that we are aware of their existence
and impacts on students’ confidence, can be easily solved
by changing certain system administration settings and
introducing small modifications in course access
privileges for users in the student role.
3,85%
31,54%
49,23%
15,38%
1,47%
17,65%
53,68%
27,21%
0,00%
10,00%
20,00%
30,00%
40,00%
50,00%
60,00%
Anyone Registered
users
Course
participants
Nobody
Novi Sad Maribor
Figure 2: Accessing data from user accounts/system logs.
4.1.7 Technical problems and localization
A number of complaints appeared regarding the stability
of the platform – 23.85% (Novi Sad) / 16.18% (Maribor)
of students reported some technical issues. They had
been usually in fact facing hardware and software
limitations of the employed servers. The inconveniences
were identified as: connection problems, slow response
in case of many users connected to Moodle, difficulties
when opening or downloading specific types of files in
certain browsers, etc.
In Maribor, practically all of the survey participants
believe that the Slovenian localization of Moodle is
rather good. Similarly, Serbian language packs are well
maintained according to 96.92% of the questioned
students.
Students generally consider the localization to be
rather important, which corresponds with the findings of
other studies claiming that the use of native language in
Moodle makes the accomplishment of students’ tasks
easier (Melton, 2006). Still, a lot of them habitually
prefer using the interface in English.
4.2 Teachers’ perspective
The other part of the survey was conducted with
comparable numbers of teachers and teaching assistants,
25 in Maribor and 18 in Novi Sad, all working with
students that participated in the first survey.
4.2.1 Design and implementation of learning
resources provided online
Preparation of online learning resources is becoming one
of the regular activities of our teaching staff, although it
is not strictly required by the management at our
faculties. Nevertheless, it requires extra effort and a
certain amount of time (Table 4).
However, we expected the teachers to complain even
more about the time management problems. Relatively
mild feedback could, unfortunately, be credited to the
fact that a lot of teachers simply are not motivated to, or
do not have enough time, energy, or possibly even skills
to produce more than small quantities of very simple
online resources (totally opposite from what their
students expect them to do).
Answer Novi Sad Maribor
more than for
traditional
resources
38.89% 28%
less than for
traditional
resources
33.33% 48%
the same as for
traditional
resources
27.78% 24%
Table 4: Time needed for the preparation of teaching
material.
Although teachers from both institutions see the
benefits of organizing e-learning efforts by using systems
like Moodle, in Novi Sad 33.33% of them still prefer
having their own home pages for (at least some of) their
courses, saying that it is easier to maintain such pages,
that it makes their work more flexible and independent,
or that they are simply not significantly motivated to
change their habits.
Interestingly, none of the survey participants in
Maribor prefers such an option. In fact, 76% of them, as
well as 44.44% of teachers in Novi Sad think that using
Moodle is much better than maintaining separate course
pages.
They point out that the LMS solves administrative
issues, keeps all resources in one place making them
easily accessible to students, and provides better
structure of courses and more features to implement
various online activities.
Finally, 22.22% (Novi Sad) / 24% (Maribor) of
teachers prefer neither Moodle nor their own course
226 Informatica 37 (2013) 221–230 M. Ivanović et al.
pages. They use the LMS in some of their courses, but
still employ other mechanisms for specific course
activities.
Separate tools often have a simpler and more
likeable GUI or provide specific development
instruments for certain course segments – lessons created
and followed in a flexible way, more readable forums,
better implemented chat and instant messaging options,
freely structured surveys, complex wiki editing and
tracking, special types of quizzes, etc.
Regarding the existing standard modules in Moodle,
Lesson module seems to be one of the most precious
ones equally in Novi Sad and Maribor (Figure 3).
0,00% 0,00%
12,50%
37,50%
50,00%
10,00%
25,00%
50,00%
0,00%
15,00%
0
0,1
0,2
0,3
0,4
0,5
0,6
very low low acceptable high very high
Novi Sad Maribor
Figure 3: Usability of Moodle’s Lesson module.
Teachers that participated in the survey, when using
Moodle, apart from providing downloadable resources
(lecture slides, assignments used for lab exercises, etc.)
or links to external references, often present the teaching
material shaped as more or less complex eLessons, built
using Lesson module. This module was even extended in
order to support creation of (semi-)adaptive eLessons
(Komlenov et al., 2008).
Some of the questioned teachers also use modules
like Glossary to explain key terms related to the topics
they teach, or to provide their students with different
kinds of tips or generally offer them easily accessible
reference points. Glossary module received relatively
good grades as well, especially in Novi Sad (Figure 4).
0,00% 0,00%
28,57%
57,14%
14,29%
5,56% 5,56%
61,11%
22,22%
5,56%
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
very low low acceptable high very high
Novi Sad Maribor
Figure 4: Usability of Moodle’s Glossary module.
4.2.2 Graded tests
Only 16.67% (Novi Sad) / 32% (Maribor) of survey
participants use online tests to officially assess their
students. The rest of them do not use this possibility at all
or they just provide tests for students’ self-assessment
that are always available and can be solved numerous
times, but teachers that offer such tests do not analyse the
results of their students.
Generally our colleagues still prefer paper tests due
to possible organizational problems that can appear when
online testing is practiced, issues concerning security and
cheating, or they simply do not find online testing serious
enough for grading the topics they teach. For some
specific subjects there are also no suitable types of
questions within the available tools.
Teachers that use online tests for official assessment
have rather positive experiences with them. They
especially value the implemented grading mechanisms
that save them a lot of time so they can invest some more
hours in preparation of bigger pools of questions that can
be exploited in the following years as well. To prevent
cheating they restrict solving tests to:
certain amounts of time (all such teachers in both
Novi Sad and Maribor),
specific computer labs (all teachers in Novi Sad and
75% of teachers in Maribor),
only particular IP addresses (all teachers in Novi Sad
and 25% of teachers in Maribor).
Generally not many survey participants think that
students have more opportunities to cheat when solving
tests in Moodle than when doing paper tests (Table 5).
Interestingly, although about the same percentage of
students (26.15%) and teachers (27.78%) believe so in
Novi Sad, Slovenian teachers should take some more
measures of precautions, since only 4% of them believe
that it is easier for students to cheat when solving
electronic tests instead of paper ones, while 35.29% of
their students support that claim.
Assessment Novi Sad Maribor
more than when
doing paper
tests
27.78% 4%
less than when
doing paper
tests
27.78% 57%
the same as
when doing
paper tests
44.44% 39%
Table 5: Opportunities for cheating within tests solved in
Moodle.
Quality of Moodle testing features, mainly Quiz
module, was assessed as well (Figure 5). Some teachers
that had not previously used this functionality chose not
to grade it, so the assessment would not be influenced by
their lack of experience with the options it offers.
Usability and Privacy Aspects of Moodle… Informatica 37 (2013) 221–230 227
4.2.3 Individual and collaborative assignments
The practice to distribute individual assignments to
students using Moodle, and afterwards to collect their
solutions, is rather common at both institutions.
0,00%
14,29%
28,57%
42,86%
5,00%
20,00%
35,00%
14,29%
10,00%
30,00%
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
0,45
very low low acceptable high very high
Novi Sad Maribor
Figure 5: Usability of Moodle’s Quiz module.
For this purpose teachers usually apply a variety of
options provided in the Assignment module. And they
are generally very satisfied with its quality (Figure 6).
50,00%
37,50%
9,09%
63,64%
0,00% 0,00%
12,50%
0,00%
0,00%
27,27%
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
very low low acceptable high very high
Novi Sad Maribor
Figure 6: Usability of Moodle’s Assignment module.
On the other hand, collaborative activities using
appropriate Moodle’s modules like Wiki have so far been
introduced in only a couple of courses at both
institutions.
Hence we received only 5 responses regarding the
quality of functionalities of the Wiki module in Novi
Sad. In Maribor, however, 20 survey participants
assessed this module (Figure 7).
20,00% 20,00% 20,00%
0,00%
70,00%
40,00%
5,00%
10,00%
5,00%
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
very low low acceptable high very high
Novi Sad Maribor
Figure 7: Usability of Moodle’s Wiki module.
All in all, it received a lot of negative comments.
Nobody addressed its usability as very high. Although
this module satisfies the basic needs of students in their
efforts to solve various team assignments, it is obvious
that, despite its recent restructuring, teachers still think
that it is not as functional as separate wiki systems.
4.2.4 Usage of communication tools
Moodle’s communication tools are leading mechanisms
of informing students about organizational and other
issues within our courses according to 72.22% (Novi
Sad) / 84% (Maribor) of teachers. Other common
communications means are regular message boards (used
by 11.11% of teachers in Novi Sad and 16% of teachers
in Maribor), electronic message boards (used by 11.11%
of teachers in Novi Sad and 68% of teachers in Maribor),
personal/course pages, etc.
Teachers were therefore asked to assess the quality
of Moodle’s communication tools (Figures 8, 9 and 10),
particularly having in mind their fitness to the teaching
methods they practice and needs/habits of their students.
Discussion forums (Figure 8) seem to be well
implemented in Moodle, while Chat module (Figure 9)
received significantly lower grades.
0,00% 0,00%
18,75%
56,25%
4,35%
13,04%
34,78%
25,00%
21,74%
26,09%
0
0,1
0,2
0,3
0,4
0,5
0,6
very low low acceptable high very high
Novi Sad Maribor
Figure 8: Usability of Moodle’s Forum module.
Chat module is in fact implemented with very basic
functionalities, so it certainly cannot be an adequate
replacement for one of the separate chat products leading
on the current software market. One would then expect
students to use chat in Moodle much more rarely than
discussion forums, but such a conclusion would be quite
wrong. Students have obviously found proper uses for
chat as well, even with limited functionality and Spartan
design of this LMS component.
228 Informatica 37 (2013) 221–230 M. Ivanović et al.
66,67%
23,53%
29,41%
16,67%
16,67%
0,00%
0,00%
5,88%
5,88%
35,29%
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
very low low acceptable high very high
Novi Sad Maribor
Figure 9: Usability of Moodle’s Chat module.
Instant messaging system integrated in Moodle is
more commonly applied by our teachers in their
communication with students and colleagues, alike
among students themselves, possibly because of its
possibilities to serve as both synchronous and
asynchronous means of communication, but also because
of its more user-friendly implementation.
7,14%
50,00%
9,52%
14,29%
0,00%
42,86%
0,00%
33,33%
42,86%
0,00%
0
0,1
0,2
0,3
0,4
0,5
0,6
very low low acceptable high very high
Novi Sad Maribor
Figure 10: Usability of Moodle’s instant messaging
functionalities.
4.2.5 Expressing opinions
Majority of teachers, 72.22% (Novi Sad) / 71%
(Maribor) of them, do not have a problem with
answering this type of surveys. Actually they believe that
conducting online surveys is a rather uncomplicated task
if Moodle’s Feedback module is used. Its usability was
assessed as pretty high (Figure 11), thus it does not
surprise that this once third-party module became one of
the standard Moodle modules.
50,00%
9,52%
23,81%
0,00%
0,00%
33,33%
16,67%
0,00%
33,33%
33,33%
0
0,1
0,2
0,3
0,4
0,5
0,6
very low low acceptable high very high
Novi Sad Maribor
Figure 11: Usability of Moodle’s Feedback module.
When it comes to dealing with opinions of students
concerning course organization, quality of teaching
material, grading issues and other course matters, our
teachers find it too challenging.
In fact, only 16.67% (Novi Sad) / 20% (Maribor) of
them already received critics within Moodle (usually in
discussion forums). In such cases they elaborated their
decisions online or in face-to-face meetings with students
and/or improved the material in question.
4.2.6 Privacy concerns
All teachers that took part in the survey are generally
satisfied with the level of privacy Moodle provides for
their students. For example, they are content with the fact
that students can only check the data regarding their own
marks using the integrated grade book, with the
possibility for groups of students to be defined both as
separate and visible to each other, etc.
Teacher that took part in the survey have no privacy
concerns regarding their own personal data, probably
since they publish just some bits of information they
really wish to share with their students. They are also
protected to a certain extent by the role they have within
the system.
4.2.7 Technical problems and localization
On the subject of technical problems, 33.33% (Novi Sad)
/ 37.5% (Maribor) of teachers said that they had
encountered some difficulties while using Moodle.
Primarily they were connected to responsiveness of the
system while updating content, time required to clear
cash/reload material, slow GUI rendering, lack of mass
show/hide/move resources, etc.
Some of the issues are obviously the responsibility of
the employed server, not Moodle itself, but there is also a
certain amount of difficulties caused by Moodle’s
interface and specific implementation of some of its
features that novice teachers have to get familiar with. Of
course, some of the problems existed only in previous
versions of the platform, not in the latest one.
Regarding localization, 33.33% of teachers in Novi
Sad consider it properly done, while others have no
opinion on the quality of Serbian language packs since
they have never used them. In Maribor, however, 94% of
survey participants are satisfied with the quality of
Slovenian translation.
While only the teachers in Novi Sad still have the
habit to use Moodle’s interface in English, all of them, as
well as 83% of teachers in Maribor, think that usage of
course content in foreign languages is beneficial for their
students. That practice promotes mobility of students and
internationalization of studies in general, opens more
possibilities for students to attain double/joint degrees,
and is also valuable for their later professional life.
5 Conclusions
In this paper we presented the analysis of a survey
conducted among Serbian and Slovenian students and
teachers investigating usability and privacy aspects of
Usability and Privacy Aspects of Moodle… Informatica 37 (2013) 221–230 229
Moodle. Comparison of the results from each group
showed that a number of differences in perception exist,
possibly due to the heterogeneous points of view and
motivations for online learning between teachers and
students.
Still, while not always being able to formulate
precisely their problems and dilemmas, both students and
teaching staff are generally aware of the benefits of e-
learning strategies and are very willing to present ideas
for potential changes in the application of certain features
of the system, as well as initiatives for upgrades of
teaching material and techniques.
Students in both Maribor and Novi Sad are generally
satisfied with frequently used Moodle’s features and
currently available teaching material. Teachers find most
of the available Moodle modules to be rather functional,
but they also commented on the poor functionalities of
some of them.
From the teachers’ perspective, the major obstacle to
even greater application of various online activities in
their practice presents a relatively low percentage of
students who use instructive and communicative features
of Moodle. Forums, chats, blogs, wikis, and other
elements characterizing Web 2.0 are fairly unexploited.
Online activities can be a good supplement to traditional
methods of teaching and learning, but students have to be
willing to participate in them and use the offered tools in
a proper way.
Mechanisms that we currently employ using
Moodle’s modules make it easier for teachers to produce
clear and easy readable, high quality teaching material
and improve communication with their students.
Problems that teachers are facing in the application of
Moodle’s features are mainly connected with the lack of
time to learn how to use them and to prepare all the
wished resources and activities.
A number of teachers that participated in the survey
believe that their efforts would be much more successful
if professional instructional designers were hired to help
them in the preparation and maintenance of their courses.
Regarding possible privacy issues, the majority of
students are satisfied with the privacy level offered by
Moodle, though they gave specific remarks and
expressed their general opinion that access to their
private data should be limited. Teachers, on the other
hand, seem to have no privacy concerns whatsoever.
We are aware of the fact that participants of our
surveys were highly computer-skilled individuals due to
their professional orientation, thus some of the
assessments might have been somewhat different if they
were made by students and teachers in different fields of
study.
Other possible limitations of this investigation could
be those that we did not take into consideration neither
the possibility that some students might have experienced
Moodle only within elective courses, which could have
added to their general enthusiasm, nor the students final
achievements and grades earned in courses supported by
resources and activities developed in Moodle. A wider
study with similar goals but varied groups of participants
of diverse profiles could additionally prove the
correctness of our conclusions.
References
[1] Ahmed, O. (2005). Migrating from Proprietary to
Open Source Learning Content Management
Systems, Department of Systems and Computer
Engineering, Carleton University, Ottawa.
[2] Al-Ajlan, A., & Zedan, H. (2008). Why Moodle, In
L. O’Conner (Ed.),
The 12
th
IEEE International
Workshop on Future Trends of Distributed
Computing Systems (FTDCS '08)
(pp. 58–64), Los
Alamitos: IEEE Press.
[3] Bisoux, T. (2007). The Evolution of E-Learning,
BizEd, January/February 2007, 22–29.
[4]
Bothe, K., Budimac, Z., Cortazar, R., Ivanović, M.,
& Zedan, H. (2009). Development of a Modern
Curriculum in Software Engineering at Master
Level across Countries.
Computer Science and
Information Systems
, 6 (1), 1–21.
[5]
Budimac, Z., Putnik, Z., Ivanović, M., Bothe, K., &
Schützler, K. (2011). On the Assessment and Self-
Assessment in a Students Teamwork Based Course
on Software Engineering.
Computer Applications in
Engineering Education
, 19 (1), 1-9.
[6] Buzzetto-More, N. A. (2008). Student Perceptions
of Various E-Learning Components.
Interdisciplinary Journal of E-Learning and
Learning Objects
, 4, 113–135.
[7] Di Domenico, F., Panizzi, E., Sterbini, A., &
Temperini, M. (2005). Analysis of commercial and
experimental e-learning systems.
Quality,
Interoperability and Standards in e-learning Team
,
TISIP Research Foundation, Trondheim.
[8] Eibl, C. J. (2009). Privacy and Confidentiality in E-
Learning Systems. In M. Perry et al. (Eds.):
The 4
th
International Conference on Internet and Web
Applications and Services (ICIW 2009)
(pp. 638–
642), Los Alamitos: IEEE Press.
[9] Georgouli, K., Skalkidis, I., & Guerreiro, P. (2008).
A Framework for Adopting LMS to Introduce e-
Learning in a Traditional Course.
Educational
Technology & Society
11 (20), 227–240.
[10] Graf, S., & List, B. (2005). An Evaluation of Open
Source E-Learning Platforms Stressing Adaptation
Issues. In P. Goodyear et al. (Eds.),
The 5
th
IEEE
International Conference on Advanced Learning
Technologies (ICALT’05),
(pp. 163–165), Los
Alamitos: IEEE Press.
[11] Hannay, M., & Newvine, T. (2006). Perceptions of
Distance-Learning: A Comparison of Online and
Traditional Learning.
MERLOT Journal of Online
Learning and Teaching
, 2 (1), 1–11.
[12] Kakasevski, G., Mihajlov, M., Arsenovski, S., &
Chungurski, S. (2008). Evaluating Usability in
Learning Management System Moodle. In V. Luzar
- Stiffler, V. Hljuz Dobric, Z. Bekic (Eds.),
The 30
th
International Conference on Information
230 Informatica 37 (2013) 221–230 M. Ivanović et al.
Technology Interfaces (ITI 2008), (pp. 613–618),
Los Alamitos: IEEE Press.
[13] Kennedy, D.M. (2005). Challenges in Evaluating
Hong Kong Students’ Perceptions of Moodle. In
Goss, Halima (Eds.),
The Australasian Society for
Computers in Learning in Tertiary Education
Conference (ASCILITE 2005)
, (pp. 327–336).
[14] Kirner, T.G., Custódio, C. de A., & Kirner, C.
(2008). Usability Evaluation Of The Moodle
System From The Teachers' Perspective. In M.B.
Nunes, M. McPherson (Eds.),
The International
Conference e-Learning (IADIS 2008)
, (pp. 371–
378).
[15]
Klobučar, T., Jenabi, M., Kaibel, A., & Karapidis,
A. (2007). Security and Privacy Issues in
Technology- Enhanced Learning. In Cunningham
P., Cunningham M. (Eds.):
Expanding the
Knowledge Economy: Issues, Applications, Case
Studies
(pp. 1233–1240). Amsterdam: IOS Press.
[16]
Komlenov, Z., & Ivanović, M. (2008). Introducing
adaptivity to e-lessons to enhance student learning.
In R. Williams, D. Remenyi (Eds.),
The 7
th
European Conference on e-Learning (pp. 571–580).
[17] Liyanagunawardena, T.R. (2008). Measuring
Student Perception and Actual Usage of Online
Learning Management System.
Communications of
the IBIMA
4 (21), 165–168.
[18] Melton, J. (2006). The LMS Moodle: A Usability
Evaluation.
Languages Issues 11/12(1), 1–24.
[19] Munoz, K., & Van Duzer, J. (2005).
Blackboard vs.
Moodle: A Comparison of Satisfaction with Online
Teaching and Learning Tools
, Humboldt State
University.
[20] Ozkan, S., Koseler, R. (2009) Multi-dimensional
students’ evaluation of e-learning systems in the
higher education context: An empirical
investigation.
Computers & Education 53 (4),
1285–1296.
[21] Paechter, M., Maier, B., Macher, D. (2010)
Students’ expectations of, and experiences in e-
learning: Their relation to learning achievements
and course satisfaction.
Computers & Education, 54
(1), 222–229.
[22] Parker, M. (2003). Technology-enhanced e-
Learning: Perceptions of First Year Information
Systems Students at the Cape Technikon. In J. Eloff
et al. (Eds.),
The South African Institute of
Computer Scientists and Information Technologists
(SAICSIT 2003)
(pp. 316–319).
[23] Picciano, A. (2002). Beyond Student Perceptions:
Issues of Interaction, Presence, and Performance in
an Online Course.
Journal of Asynchronous
Learning Networks
. 6 (1), 21–40.
[24] Rice, H.W. (2008).
Moodle 1.9 E-Learning Course
Development – A complete guide to successful
learning using Moodle
, Birmingham: Packt
Publishing.
[25] Richardson, J.C., & Swan, K. (2003). Examining
Social Presence in Online Courses in Relation to
Students' Perceived Learning and Satisfaction.
Journal of Asynchronous Learning Networks 7(1),
68–88.
[26] Smart, K.L., & Cappel, J.J. (2006). Students’
Perceptions of Online Learning: A Comparative
Study.
Journal of Information Technology
Education
5, 201–219.
[27] Song, L., Singleton, E., Hill, J., & Koh, M. (2004).
Improving Online Learning: Student Perceptions
and Challenging Characteristics.
Internet and
Higher Education
7, 59–70.
[28] Stewart, B., Briton, D., Gismondi, M., Heller, B.,
Kennepohl, D., McGreal, R., & Nelson, C. (2007).
Choosing MooDLe: An evaluation of Learning
Management Systems at Athabasca University.
International Journal of Distance Education
Technologies
5 (3), 1–7.
[29] Tanner, J.R., Noser, T.C., Totaro, M.W. (2009).
Business Faculty and Undergraduate Students’
Perceptions of Online Learning: A Comparative
Study.
Journal of Information Systems Education
20 (1), 29–40.
[30] Totaro, M.W., Tanner, J.R., Noser, T.C., Fitzgerald,
J.F., Birch, R (2005). Faculty Perceptions of
Distance Education Courses: A Survey.
Journal of
College Teaching & Learning
2 (7), 13–20.
[31] Weaver, D., Spratt, C., & Nair, C.S. (2008).
Academic and student use of a learning
management system: Implications for quality.
Australian Journal of Educational Technology 24
(21), 30–41.
[32] Weippl, E.R., & Tjoa, A.M. (2005). Privacy in e-
learning: anonymity, pseudonyms and authenticated
usage.
Interactive Technology and Smart Education
2 (4), 247–256.
