Current Trends and Challenges of Developing and Evaluating Learning Management Systems

Abstract

Many Universities recognize the necessity of using Learning Management Systems (LMSs) to increase learners' motivation, encourage interaction, provide feedback and provide support during the learning process. There are many proprietary and open source LMSs that can be found as alternative products. With the ever-growing number of LMSs, the task of developing and evaluating LMS becomes even more important. This paper discusses the factors that affect the use of LMS, the developing issues that have an impact on LMS and the evaluation processes that should be taken to select the suitable LMS. Also, it presents challenges that face LMS success, efficiency, assessment, evaluation, selection and usability. Last it shows the current trends to answer a question of how the LMS of the future will look like.

Full text

Current Trends and Challenges of Developing and
Evaluating Learning Management Systems
Nouran M. Radwan1*, M. Badr Senousy2, Alaa El Din M. Riad1
Faculty of Computer & Information Sciences, Mansoura University, Egypt
* Corresponding author. email: radwannouran@yahoo.com
Manuscript submitted August 15, 2014; accepted October 17, 2014.
doi: 10.7763/ijeeee.2014.v4.351
Abstract: Many Universities recognize the necessity of using Learning Management Systems (LMSs) to
increase learners' motivation, encourage interaction, provide feedback and provide support during the
learning process. There are many proprietary and open source LMSs that can be found as alternative
products. With the ever-growing number of LMSs, the task of developing and evaluating LMS becomes even
more important. This paper discusses the factors that affect the use of LMS, the developing issues that have
an impact on LMS and the evaluation processes that should be taken to select the suitable LMS. Also, it
presents challenges that face LMS success, efficiency, assessment, evaluation, selection and usability. Last it
shows the current trends to answer a question of how the LMS of the future will look like.
Key words: Learning management system, e-learning, LMS success, LMS challenges, LMS trends.
1. Introduction
Learning Management Systems (LMSs) are web based applications that are being used today in
e-learning. With the improvement of e-learning concept, Learning Management Systems are gaining interest
as a delivering and managing teaching or training learners. There has been a sudden increase in using the
applications of the learning management systems in higher education. [1]. LMSs contain features that assist
in the designing, sharing, delivery, management and evaluation of learning resources to all learners [2].
Effective Elearning comes from spreading more educational opportunities and help learners to develop
their skills [3]. In higher education institutions, LMSs provide prosperous learning environment. The
investments to LMS continue to increase; there are hundreds of LMS products available in marketplace [4],
[5]. Therefore, there is a need to help organizations with tools necessary for developing and evaluating
these systems [6].
Then [7]-[10] show that many universities are conscious about using LMS as a tool to help in
disseminating materials to the learners. LMS called course management system or learning content
management system is software designed to assist instructors and learners with course management. LMS
is considered as a useful tool to manage educational resources. It is necessary to have a LMS that can be
adapted easily to changing requirements.
The rest of this paper is organized as follows: Section 2 presents the factors that affect the LMS success. In
Section 3, the developing of LMS is showed. Then Section 4 is about the evaluation process of LMS. While
Section 5 presents a framework that contains challenges and work directions; last Section 6 is a view of the
current trends in LMS.
International Journal of e-Education, e-Business, e-Management and e-Learning
361 Volume 4, Number 5, October 2014
Sadat Academy for Management Sciences, Egypt
1
2

2. LMS Success
Management information system is a set of systems that helps managers to take better decisions [11].
LMS is regarded as a management information system [12]. LMS is an information system (IS) that supports
teaching and learning activities and the management and communication associated with them [13].
Development of management Information systems is not easy as it needs to visualize the complete
information system with functionality [9]. Implementation of specific LMS functionalities depends on
specific requirements. As LMS solution is successful for a university it does not mean that will be successful
for other universities or organizations. The 99 LMS functionalities are listed in alphabetical order and not in
an importance order as seen in Appendix [14], where MIS functionalities are checked.
DeLone and McLean’s [15] IS success model includes three components: the creation of a system, the use
of the system, and the consequences of the use of the system. Holsapple and Lee-Post adapted DeLone and
McLean’s model for use in the learning management system to be: system design, system usage, and system
outcome. Many studies have used the D&M IS success model in the learning field, but many researchers
express their need to propose an IS success model for e- learning purposes and especially for LMS. In [16]
system quality, service quality, content quality, learner perspective, instructor attitudes, and supportive
issues had a considerable effect on the learners’ perceived satisfaction. [13], [17] Showed that learner and
instructor involvement improve effectiveness of learning process. Results of [18] reveals six factors
including learners’ characteristics, instructors’ characteristics, extrinsic motivation, service quality, system
quality, and information quality that influence the acceptance of e-learning systems in developing countries.
Perceived usefulness, perceived ease of use, user satisfaction, learner characteristics, instructor, LMS
characteristics and organization characteristics have influence on LMS success [19]. System quality is very
important factor in relation to the service quality, information quality and learning community [20] [21].
The following lines identify the critical factors that affect LMS integrated from previous studies that affect
LMS success:
1) Personal factors / PF (learner and instructor):
 User characteristics (UC).
 User satisfaction (US).
 Perceived usefulness (PU).
2) System factors / SF (LMS characteristics):
 System quality (infrastructure).
 Service quality (organization).
 Information quality (course).
3) Organizational factors/OrgF:
 Management support.
 Training.
4) Supportive factors/SupF :
 Ethical and legal issues together with privacy.
 Plagiarism and copyright concepts.
 Cost.
Table 1 is an integration of different validated e-learning success models from previous studies to
illustrate the success factors of LMS symbolized by x, where no model has a complete set of factors.
LMS characteristics play an important role in evaluating LMS [20]. LMS Characteristics as shown in Table
2 which are system quality, service quality and information quality [22]. System quality (SQ) in a LMS,
measures the essential features including system performance and user interface. Examples of system
quality measures in the LMS are response time, usability, availability, reliability, completeness, and security.
International Journal of e-Education, e-Business, e-Management and e-Learning
362
Volume 4, Number 5, October 2014

Service quality (SrQ) is concerned with the support given by the service provider of LMS, whether the
service is delivered by the university organization or external providers. It has become an important
element in research related to information systems [23]. Information Quality (IQ) is concerned quality
measures derived from user perspectives [24] is a term to describe the quality of the content of information
systems. The criteria for measuring information quality are multidimensional such as speed of access to
information, accuracy and clarity [25]. Table 2 illustrates the quality attributes (criteria) for each LMS
characteristics (SQ, SrQ, and IQ)
TABLE 1. Reference Models and Success Factors
Success factors of LMS
Reference,
Year
PF
SF
OrgF
SupF
UC
US
PU
[16], 2009
x
x
x
x
[13], 2010
x
x
x
x
[17], 2010
x
x
x
x
[18], 2012
x
x
x
[19], 2012
x
x
x
x
x
[20], 2014
x
x
x
x
[21], 2014
x
x
x
x
TABLE 2. LMS Characteristics and Quality Attributes [23]-[25]
LMS Characteristics
Quality attributes
SQ
SrQ
IQ
Accessibility
x
x
Accuracy
x
x
Assurance
x
Availability
x
x
Completeness
x
x
x
Consistency
x
Currency
x
Effectiveness
x
Efficiency
x
Empathy
x
Flexibility
x
Format
x
Functionality
x
Interactivity
x
Legibility
x
Relevancy
x
Reliability
x
x
x
Responsiveness
x
x
Sufficiency
x
x
Tangibility
x
Timeliness
x
Understandability
x
Usability
x
3. Developing of LMS
LMS provides the university a set of tools that help in managing course catalogues, record data from
learners and provide reports to management. Most LMS includes features such as discussion forums, chats,
automated testing, assessing tools and student tracking [26].The aim of developing this learning
management tool is to provide the users with an attractive, user-friendly, secure, and a comprehensive
interactive interface with easy-to-use facility [27].
LMS systems support different features which can be analyzed from different aspects which are:
pedagogical aspect, learner environment, instructor tools, course and curriculum design, administrator
tools and technical specification [28]. LMS functional requirements include Learners should be able to keep
International Journal of e-Education, e-Business, e-Management and e-Learning
363
Volume 4, Number 5, October 2014

track of their learning progress while performing the activities of a learning design; instructors should
upload, discuss and review assignments; the system should support the creation and delivery of
self-assessments; allow the implementation of various pedagogical approaches and course designs for the
different target audiences [29].
Currently there is a need of Low cost and adaptable interacting LMS to provide e-learning courses. A
study was performed to present the developing, implementation and evaluation of LMS tool. The tool
supports the most common features of an e-Learning such as view courses, view texts and videos, manage
quizzes, display presentations and editing processes. Where each user account has its own personalized
page and appearance and the system generates different kinds of interactive submenu pages that are
readily accessible. The study findings is that the keys to deploy a successful LMS are understanding where
system is being deployed, who uses it, how it needs to integrate with existing and future systems, and what
specific educational tasks should be automated [30].
Fig. 1. Modified six steps to successful LMS implementation.
The LMS implementation process involves six major steps as shown in Fig. 1. Planning specifies the
requirements organization. The plan should include all the tasks needed to implement the LMS from the
vendor’s point of view. Configuration includes understanding of data and operations and understanding of
the system’s data fields, functionality, and capabilities. While integration means that it may integrate with a
International Journal of e-Education, e-Business, e-Management and e-Learning
364
Volume 4, Number 5, October 2014

number of systems such as Systems containing user accounts and profiles. Course and data migration
happens when changing from LMS software to another. It is needed to move the data and courses from your
legacy system to the new LMS. It is a complex task that needs a specific sequence, and addressing to any
incompatibilities between the way the data and courses were stored in the legacy system versus the new
LMS.
The last major step before going live with LMS is to conduct user acceptance testing. Testing the LMS
ensures a fully working system, configuration, courses, and data are available in the system as you expect
them to be. Once all tests have been completed, end-to-end, and all bugs have been fixed, then go live with
LMS [31].
The users of LMS are learners who use the system for the educational process, Instructors who use the
LMS to supervise, assist and evaluate the learners and administrator who take the support of all the users to
control the functioning of the system.
It is noticed that current LMSs are lacking in some functionalities such as receiving feedback from the
users and processing accounting information. There could be more studies needed in different aspects like
extra modules for indicating the best content of similar subjects, transmission any information from the
participating universities, and checking the quality of the content. These can be taken as future work [1].
The next Figure presents a generic architecture of LMS that consists of four layers: presentation layer,
service layer, application layer and database layer.
Fig. 2. A generic LMS logical architecture.
4. Evaluation of LMS
Evaluation of Software is a type of assessment that seeks to determine if software is the best suitable for
the requirement includes functional requirements, non-functional requirements and user requirements of a
given customer. Based on a prepared list of criteria along with some practical experimentation, it is possible
to determine if the software would be helpful to the customer or not [32].
Evaluation is a decision making process that is needed to select the most suitable LMS option from a set
of alternatives due to organization requirements. One of the approaches to decision making is multi-criteria
decision making [33]. It solves problems and help in taking decisions involving multiple criteria. Taking a
decision could correspond to choose the best alternative from a set of alternatives or to choose a small set
International Journal of e-Education, e-Business, e-Management and e-Learning
365
Volume 4, Number 5, October 2014

of good alternatives by analyzing the different criteria [34].
The evaluation process of LMSs is in general costly, time consuming, and needs an effort. A study was
developed to present this evaluation process steps. Some of these steps are used to insert the entries
needed into the algorithm to get a result that refers to the most suitable LMS satisfying the specified
requirements [35]. Another study [33] used the Evaluation Cycle Management as an evaluation system
which consists of two phases of evaluation methods: Criteria evaluation and Usability evaluation. In which,
the results gained from the criteria evaluation model is being verified on user usability testing. In case user
usability testing shows unacceptance, it returns to the criteria evaluation once again to edit the criteria
value and get the needed results.
Fig. 3. Evaluation process [33], [35].
From these previous studies the altered evaluation process steps can be integrated and illustrated as
follows in Fig. 3. Where first step is determining the cost of LMS needed to select the set of available
software. Functional requirements are defined to select the most appropriate software set that meet these
requirements. Nonfunctional requirements are determined and weighted by decision maker, an expert or
group of experts according to the specified criteria. Ranking the nonfunctional requirements will help in the
selection of most appropriate software. Then testing user requirements will be performed to take a decision
and select the software or going back and weight again the nonfunctional requirements.
International Journal of e-Education, e-Business, e-Management and e-Learning
366
Volume 4, Number 5, October 2014

5. Challenges and Work Directions
Assessing the success of LMS and the quality of LMS systems is one of the most significant issues in
e-elearning field. Several studies have conducted to assess the quality of LMS systems and evaluate LMS as it
has become an important issue. Latest research showed that there is a need to measure the efficiency of
LMS and to explore other factors for measuring the success of e-learning systems in general and LMS in
particular. A framework that contains challenges and work directions of this area is proposed as shown in
Fig. 4.
Fig. 4. Challenges and work directions of LMS.
5.1. LMS Success
A study was conducted to indicate the factors that influence the successes of e-learning systems. The
study uses fuzzy TOPSIS technique as a new method to evaluate e-learning systems. The results show that
system quality, information quality, service quality have the most positive impact on learners understanding
of e-learning. Also the findings that the system quality is very important factor in relation to the service
quality, information quality and learning community [20] Future work may discuss other factors for
measuring the success of LMS from different perspectives such as learner's, instructor's and organization.
Further studies are needed to explore the critical factors affecting organizations’ implementation and
deployment of LMS especially in developing countries.
5.2. LMS Efficiency
A research was performed to focus on the system quality concept and explores usability, accessibility,
reliability, and stability dimensions to evaluate the effect of these dimensions on e-learning system
efficiency. The results show that usability factor was found as important dimension that affects the system
quality and also the system quality is the main factor that increase or decrease the efficiency of LMS. The
future studies deals with further evaluation to other system quality dimensions such as objectivity,
completeness, and consistency, examine the relationship between system interface and information quality to
achieve the LMS efficiency [36].
5.3. LMS Quality Assessment Model
A work was carried to contribute on developing website quality assessment model for website’s quality
evaluation depending four quality parameters such as accuracy, feasibility, utility and propriety. The work
analysed the system quality of an existing e-learning site and obtained the feedback of the users where the
preliminary assessment phase and the criterion based analysis of each parameter is made. Then the
evaluation process has been accomplished with two phases of evaluation: preliminary assessment phase
and evaluation results analysis phase. With the suggestions obtained from the assessment process, a new
improved e-learning environment is developed to satisfy the users with their best quality of content and
design [37]. Future exploration is still necessary that assess various e-learning platforms including LMS.
International Journal of e-Education, e-Business, e-Management and e-Learning
367
Volume 4, Number 5, October 2014

Designing a website quality assessment model including the quality parameters that affect the LMS to act as a
tool for organizations to achieve a base standard of consistent quality that is essential for user satisfaction is a
challenge.
5.4. LMS Evaluation and Selection
With the ever-growing number of LMSs, the task of selecting the most suitable becomes even more
important. Multi criteria decision making methods help in taking decisions involving multiple criteria.
Taking a decision could correspond to choose the best alternative from a set of alternatives or to choose a
small set of good alternatives by analyzing the different criteria [34]. A future work is necessary to develop an
evaluation model that considers quality standards from software engineering perspectives and takes into
consideration the quality aspects of the e-learning system [38]. In future studies, decision making process of
LMS evaluation can be supported by the fuzzy set theory [39].
5.5. Relationships between Usability and Learner Characteristic's
A study was executed to investigate the validity and reliability of a Greek version of system usability scale
was investigated in the context of LMSs perceived usability evaluation. The results indicate that the degree
of contribution of various interaction characteristics to the system usability scale score requires further
examination. Results from the study related to how learners achieve their goals, what criteria they use to
evaluate information, how they adapt to any given learning environment, which navigation strategies they
follow, and how different structuring of the learning material affects their learning effectiveness. Future
work deals with exploring the factors that influence the system usability of LMSs and developing educational
usability evaluation practices. Further research also includes investigating relationships between usability and
various learner characteristic's such as gender, age, ICT competence [36], [40].
6. Current Trends of LMS
There are many Learning Management System trends and many others are still to come since eLearning
technology is rapidly changing. Then a question is emerging which is how the LMS of the future will look
like. The following part presents some key trends that may find an outstanding place in the progress of the
LMS.
Fig. 5. Current trends of LMS.
6.1. Personal Learning Environment
A current trend of LMS is that Personal Learning Environment (PLE) features will become part of LMS.
PLE refers to a set of tools, social software and systems that provide control to the students to direct their
own learning and achieve educational goals. Learner under the PLE can create his profile, customizes
LMS content, and participates with other learners. Further research deals with the necessity to LMSs to
International Journal of e-Education, e-Business, e-Management and e-Learning
368
Volume 4, Number 5, October 2014

include constructivist instructional design methods and pedagogy, particularly emphasizing active and shared
learning and personalized attention to all students [41], [42].
6.2. Cloud Based LMS
Web-based services such as cloud computing, mobile internet, and modern format for distributing web
content are some trends that may affect e-learning environment. Cloud based LMS is a trend where are
testing out cloud based LMS to focus on education excluding most of barriers associated with cost and
support. Cloud based training means the courses are available anytime and anywhere there is an Internet
connection. The future LMS will increasingly be run on cloud for its agile, flexible and economic
characteristics. Cloud-based LMS are able to take advantage of the convenience and flexible aspects of the
technology. Learners can be better served by Cloud- based LMS that could contain social bookmarking tools,
document sharing applications, social networking applications, timeline tools, and media options[41]-[43].
The organizations face a challenge in implementing Cloud-based LMS such as costs, a lack of resources,
and resistance by users to the implementation of systems. However, Cloud-based LMS can reduce costs due
to lower requirements of hardware and software, and less need for on-site maintenance. The limitations of
cloud-based LMS are high speed Internet connection is needed for the efficiency of the presented e-learning
services, and issues surrounding the security of a cloud remain unclear. As the speed and stability of the
Internet are continuing to improve, it seems likely that the popularity of cloud computing for e-learning will
increase [44].
6.3. Talent LMS
Talent management identifies the current skills of learners and the gaps in skills. A LMS is a tool to
present courses. The talent management and LMS have traditionally worked independently of each other.
The integration of talent management and LMS can fill the skill gaps of the learner and improve the
learner’s job-related skills. Another new trend is the integration of talent management into the LMS where
learning management systems can recommend new training courses due to learner’s needs and skills [45].
6.4. Gamification with LMS
The “Gamification” of courses is a movement that is gaining traction in educational research communities
and universities. Games are designed for user learning, entertaining and satisfaction. Gamification with LMS
to is needed to gain learners interest and may fulfill learning goal. An interesting way to enhance LMS is an
attempt to engage and motivate learners with games. It is observed that the games have good acceptance
among the learners. Future work deals with quantifying the learning degree and the satisfaction degree of
learner if they are using games in the learning process [46], [47].
6.5. Mobile LMS
There is a trend to add mobile learning functionality to LMS. This allows learners to continue learning
process by accessing their learning materials using mobile devices. Mobile devices in future need to support
Mobile content and LMS access, personal notification systems and response systems. For further research,
there is a need for analyzing the usage of video lessons with the mobile devices to arrive out the results of
learning effectiveness, performance and the system quality [5], [48].
7. Conclusion
With the fundamental changes in e-learning technology, there is a need to take into considerations the
current trends and challenges of developing and evaluating Learning Management Systems (LMS) to benefit
learners and organizations. This paper presents the factors that have an impact on LMS success which are
International Journal of e-Education, e-Business, e-Management and e-Learning
369
Volume 4, Number 5, October 2014

personal factors, system factors, organizational factors and supportive factors. It shows the importance of
these factors from previous studies. Then it presents the developing process of LMS, and current LMSs
challenges in functionality. And with the ever-growing number of LMSs, the paper presents the evaluation
process for evaluating and selecting the suitable LMS satisfying the organization requirements.
The next section is about the challenges and work directions that discuss the LMS success, LMS efficiency,
LMS Quality Assessment Model, LMS evaluation and selection and LMS usability. LMS success is affected by
LMS characteristics which are system quality, information quality, and service quality. While the system
quality is the main dimension that increase or decrease the efficiency of LMS and the usability factor is a
main factor that affects the system quality. Also there is a need to contribute in developing LMS quality
assessment model including the quality to achieve a base standard of consistent quality essential for user
satisfaction. LMS evaluation and selection is another challenge with the ever-growing number of LMSs.
Therefore, an evaluation model for LMS selection is needed for decision making process that considers
quality standards from software engineering perspectives and the quality aspects of the e-learning system
where Multi criteria decision making methods can help. Last further examination is for studying the factors
that influence the system usability of LMS.
The last section presents the current trends of LMS to answer question like how the LMS will look like in
the future. Some key trends are shown which are Personal Learning Environment (PLE), Cloud based LMS,
Talent LMS, Gamification with LMS, Mobile LMS. PLE of LMS allows learners with the ability to create his
profile, customizes LMS content, and participates with other learners. Cloud-based LMS served learners
better as it provide them with social bookmarking tools, document sharing applications, social networking
applications, timeline tools, and media options. Cloud based LMS needs to be studied taking into
considerations costs, lack of resources, and resistance by users. Another trend is Talent LMS that integrates
the talent management and LMS that could recommend and present training courses to learners due to
learner's needs and skills. The Gamification can enhance learning process as it could engage and motivate
learners with games. The satisfaction degree of learner if they are using games is a future study. Adding
mobile learning functionality to LMS is a trend while learners can access their learning materials using
mobile devices.
This couldn’t cover all the LMS issues. It has just shed light on some views and future work. Further
studies are needed to deal with analysis of the presented challenges to overcome it and trends to know the
return benefits.
Appendix
The 99 LMS functionalities listed in alphabetical where MIS functionalities are checked [14].
LMS Functionalities
MIS
1. Administration

2. Administrative Reporting

3. Assessment Tools Built in
4. Authentication & Security
5. Authoring - 3D Simulation
6. Authoring - Courses
7. Authoring - Gamification
8. Authoring - mLearning
9. Authoring - PowerPoint
Conversion
10. Authoring - Serious Game
11. Authoring - Storyboarding
12. Blended/Hybrid Learning
International Journal of e-Education, e-Business, e-Management and e-Learning
370
Volume 4, Number 5, October 2014

13. Career Tracking
14. Certification Management
15. Certification Tracking
16. Classroom Management
17. Collaboration Management
18. Competency Management
19. Compliance - AICC
20. Compliance - 3rd Party
Authoring Tools
21. Compliance - 3rd Party
Teleconferencing Tools
22. Compliance - SCORM
23. Compliance - Tin Can API
24. Compliance Management
25. Conferencing
26. Content Library
27. Content Management
28. Course Catalog
29. Course Interactivity
30. Coursework Grading
31. Custom Learning Vocabulary
32. Custom User Interface
33. Customizable Branding
34. Customizable Fields
35. Customizable Functionality
36. Customizable Reporting
37. Data Import/Export

38. Data Management

39. Development Tracking
40. Document Management

41. eCommerce
42. eLearning Management
43. Event Management
44. Exam Engine
45. Goal Setting / Tracking
46. Individual Development Plans
47. Installation (Hosted, Local, Saas,
Cloud)
48. Instructor Led Classes
49. Instructor Scheduling
50. Job Hierarchies
51. LCMS
52. Legacy System Integration
53. Licensing (Free, Trial, Open
Source, Paid)
54. Live Video Presentations
55. Locations Served
56. Maintance (Backups, etc)
57. Mobile Access
58. Multi-Currency
59. Multi-Language
60. Multi-Lingual Courseware
61. Multi-Lingual User Interface
62. Multimedia Environment
International Journal of e-Education, e-Business, e-Management and e-Learning
371
Volume 4, Number 5, October 2014

63. Multi-Organization Structures
64. Multiple Delivery Formats
65. Notifications - Email
66. Notifications - SMS
67. Offiline Learning
68. Online Learning
69. Performance Assessment
70. Perpetual Licence
71. Platform
72. Podcast Management
73. Registrar Enrollment
74. Registration Management 
75. Reporting 
76. Resource Management
77. Self-Enrollment
78. Self-Paced
79. Self-Registration
80. Single Sign On
81. Skills Assessment
82. Skills Tracking
83. Social Learning
84. Software Development Kit
85. Student Management
86. Student Portal
87. Student Self Service
88. Student Tracking
89. Survey Management
90. Term License
91. Test Building
92. Test Scoring
93. Testing
94. Training Metrics
95. Training Tracks
96. User Access Controls
97. Users Size Served
98. Virtual Classes
99. Waiting Listing
References
[1] Sharma, S., & Vatta, S. (2013). Role of learning management systems in education. International Journal
of Advanced Research in Computer Science & Software Engineering, 3(6), 997-1003.
[2] Asiri, M., Mahmud, R., Bakar, K., & Ayub, M. A. (2012). Factors influencing the use of learning
Science and Education, 2(2), 126-137.
[3] Trivedi, R., Mohd, N., & Sharma, R. (2013). Proposed framework for open source based elearning
implementation in Uttarakhand. International Journal of Engineering Research & Technology, 2(11),
2278-0181.
[4] Dias, S., & Diniz, J. (2013). FuzzyQoI model: A fuzzy logic-based modelling of users’ quality of
interaction with a learning management system under blended learning. Computers & Education, 69,
38–59.
[5] McIntosh, D. (2014). Vendors of learning management and elearning products. Trimeritus eLearning
International Journal of e-Education, e-Business, e-Management and e-Learning
372 Volume 4, Number 5, October 2014
management system in Saudi Arabian higher education: A theoretical framework. Canadian Center of

International Journal of e-Education, e-Business, e-Management and e-Learning
373 Volume 4, Number 5, October 2014
Solutions Inc., Retrieved August, 2014, from http://www.trimeritus.com/vendors.pdf.
[6] Mtebe, J. (2014). A Model for assessing learning management system success in higher education in
Sub-Saharan Countries. The Electronic Journal of Information Systems in Developing Countries, 61(7),
1-17.
[7] Awang, N., & Darus, Y. (2011). Evaluation of an open source learning management system: Claroline.
Procedia-Social and Behavioral Sciences, 67, 416-426.
[8] Chung, C., Pasquini, L., & Koh, C. (2013). Web-based learning management system considerations for
higher education. Learning and Performance Quarterly, 1(4), 24-37.
[9] Fawareh, H., (2013). Elearning management systems general framework. World Academy of Science,
Engineering and Technology, 7(9), 1221-1225.
[10] Aggarwal, A., Adlakha, V. , & Ross, T. (2012). A Hybrid approach for selecting a course management
system: A case study. Journal of Information Technology Education: Innovations in Practice, 11, 283-300.
[11] Thaku, D. (2014). What is MIS? Define the function and characteristics of mis? Retrieved August, 2014,
from http://ecomputernotes.com/mis/ what-is-mis/functionandcharacteristicsofmis.
[12] Lee, J. (2010). The effects of learning management system quality and learner’s characteristics
regarding scholastic performance. Proceedings of e-Learning Week 2010 International Conference, COEX
Seoul, South Korea (pp.15-33).
[13] Klobas, J., & McGill, T. (2010). The role of involvement in learning management system success. Journal
of Computing in Higher Education, 22, 114–134.
[14] Pappas, C. (2013). Learning management systems comparison checklist of features. from
http://elearningindustry.com/learning-management-systems-comparison-checklist-of-features.
[15] Kim, K., & Trimi, S. (2012). The impact of CMS quality on the outcomes of e-learning systems in higher
education: an empirical study. Decision Sciences Journal of Innovative Education, 10(4), 575-587.
[16] Ozkan, S., & Koseler, R. (2009). Multi-dimensional students’ evaluation of e-learning systems in the
higher education context: An empirical investigation. Computers & Education, 53, 1285–1296.
[17] Ahmed, S. (2010). Hybrid e-learning acceptance model: Learner perceptions. Decision Sciences Journal
of Innovative Education, 8(2),313-346.
[18] Bhuasiri, W., Xaymoungkhoun, O., Hangjung Z., Rho, J., & Ciganek, A. (2012). Critical success factors for
e-learning in developing countries: A comparative analysis between ICT experts and faculty. Computers
& Education, 58,843–855.
[19] Al-Busaidi, K. Learners’ perspective on critical factors to lms success in blended learning: an empirical
investigation. Communications of the Association for Information Systems, 30(2), 11-34.
[20] Fard, K., Tafreshi, F., Nilashi, M., & Jalalyazdi, M. (2014). Assessing the critical factors for e-learning
systems using fuzzy TOPSIS and fuzzy logic.International Journal of Computers & Technology, 12(6),
3546-3561.
[21] Lwoga, E. (2014). Critical success factors for adoption of web-based learning management systems in
Tanzania. International Journal of Education and Development using Information and Communication
Technology, 10(1).
[22] Henry, B. (2012). The effective application of LMS for sustainable knowledge management skills &
abilities.Science and Technology, 2(6), 198-202.
[23] Almarashdeh, I., Sahari, N., Zin, M. A., & Alsmadi, M. (2010). The success of learning management
system among distance learners in Malaysian universities. Journal of Theoretical and Applied
Information Technology, 21(2),80-91.
[24] Hilmi, M., Pawanchik. S., & Mustapha, Y. (2012). Perceptions on service quality and ease-of-use:
evidence from Malaysian distance learners. Malaysian Journal of Distance Education, 14(1), 99–110.

International Journal of e-Education, e-Business, e-Management and e-Learning
374 Volume 4, Number 5, October 2014
[25] Alla, O. M., & Faryadi, Q. (2013). The effect of information quality in e-learning system.International
Journal of Applied Science and Technology, 3(6), 24-33.
[26] Fariha, Z., & Zuriyati, A. (2014). Comparing moodle and efront software for learning management
system. Australian Journal of Basic and Applied Sciences, 8(4), 158-162.
[27] Bhuiyan, T., & Kundu, R. (2014). Developing and Evaluating desktop-based learning management
system. Journal of Modern Science and Technology, 2(1),82-99.
[28] Srđević,B., Pipan, M., Srđević, Z., & Arh, T. (2012). AHP supported evaluation of LMS quality.
Proceedings of International Workshop on the Interplay between User Experience (UX) Evaluation and
System Development (I-UxSED 2012) (pp. 52-57).
[29] Vogten, H., & Koper, R. (2014). Towards a new generation of learning management systems.
Proceedings of 6th International Conference on Computer Supported Education (CSEDU).
[30] Bhuiyan, T., & Kundu, R. (2014). Developing and evaluating a desktop-based learning management
system. Journal of Modern Science and Technology, 2(1), 82-99.
[31] Foreman, S. (2013). The six proven steps for successful LMS implementation. Learning Solution
Magazine. Retrieved 2014, from
http://www.learningsolutionsmag.com/articles/1217/the-six-proven-steps-for-successful-lms-imple
mentation-part-two.
[32] Wisegeek, (2014). What is software evaluation? Retrieved 2014, from
http://www.wisegeek.com/what-is-software-evaluation.htm.
[33] Pipan, M., Arh, T., & Blažič, B. (2010). The evaluation cycle management-method applied to the
evaluation of learning management systems. Integrating Usability Engineering for Designing the Web
Experience: Methodologies and Principles (pp. 58-80).
[34] Aruldoss, M., Lakshmi, T., & Venkatesan, V. (2013). A Survey on multi criteria decision making methods
and its applications. American Journal of Information Systems, 1(1),31-43.
[35] Cavus, N. (2011). Selecting a learning management system (LMS) in developing countries: instructors'
evaluation. Interactive Learning Environments.
[36] Mustafa, M., (2013). The impact of system quality in e-learning system. Journal of Computer Science and
Information Technology, 1(2), 14-23.
[37] Jayakumar, R., & Banbehari, M., (2014). Website quality assessment model (WQAM) for developing
efficient e-learning framework- a novel approach. International Journal of Engineering and Technology,
5(5), 3770-3780.
[38] Valdez-Silva, E., Reyes, P., Alvarez, M., Rojas, J., & Dominguez, V. (2013). Expert system for evaluating
learning management systems based on traceability. In: K. Elleithy & T. Sobh (Eds.), Innovations and
Advances in Computer, Information, Systems Sciences, and Engineering (pp. 1103-1113). New York:
Springer.
[39] Ozbek, A. (2013). Performance evaluation of learning management system. NWSA-Education Sciences,
8(2), 156-178.
[40] Katsanos, C., Tselios, N., & Xenos, M. (2012). Perceived usability evaluation of learning management
systems: a first step towards standardization of the system usability scale. Proceedings of Greek
Panhellenic Conference on Informatics 2012 (pp. 302-307).
[41] Czerkawski, B., & Gonzales, D. (2014). Emerging priorities and trends in distance education. Major
Trends, Issues, and Challenges with Learning Management Systems.
[42] Abazi-Bexheti, L., Apostolova-Trpkovska, M., & Kadriu, A. (2014). Learning management systems:
Trends and alternatives. International Convention on Information and Communication Technology,
Electronics and Microelectronics, Opatija, Adriatic Coast, Croatia, 891-895.

M. Badr Senousy is a professor of computer and information systems at Sadat Academy
for Management Sciences, Cairo, Egypt. He has received a PhD in computer science in
1985 at George Washington University, USA.
Alaa El Din M. Riad is a professor of computer and information systems at Faculty of
Computers and Information Sciences, Mansoura, Egypt. He has received a PhD in electric
al engineering in 1992 from Mansoura University, Egypt, and an MS in electrical
engineering in 1988 from Mansoura University, Egypt. He has supervised many master
and doctorate studies.
International Journal of e-Education, e-Business, e-Management and e-Learning
375 Volume 4, Number 5, October 2014
[43] Papula, J. (2013). The benefits of an innovative learning management system, the case from higher
education in Slovak Republic. Proceedings of ICERI 2013 Conference: International Conference of
Education, Research and Innovation (pp. 2671-267).
[44] Karim, K., & Goodwin, R. (2013). Using cloud computing in e-learning systems. International Journal of
Advanced Research in Computer Science & Technology, 1(1),65-69.
[45] Bhatia, S. (2014). Learning management system trends, in the new millennium, the corporate learning
management system has developed into a business-critical technology platform. Retrieved 2014, from
http://www.trainingmag.com/learning-management-system-trends.
[46] Souza-Concilio, I., & Pacheco, B. A. (2013). Games and learning management systems: A discussion
about motivational design and emotional engagement. SBC –Proceedings of SBGames –São Paulo –SP –
Brazil (pp. 38-45).
[47] Holman, C., Aguilar, S., & Fishman, B. (2013). GradeCraft: what can we learn from a game-inspired
learning management system? Proceedings of the Third International Conference on Learning Analytics
and Knowledge (pp. 260-264).
[48] Specht, M., & Klemke, R. (2013). Enhancing learning with technology. Proceedings of the Third
International Conference on e-Learning (eLearning-2013), Belgrade, Serbia (pp. 26-27).
Nouran Radwan is a PhD student at Computer and Information System Department,
Faculty of Computer and Information, Mansoura University, Egypt. She obtained her B.S.
in information systems from Sadat Academy at 2005 and she got M.SC degree in
information system from the Arab Academy for Science and Technology in 2011. Now
she is an assistant lecturer in Computer and Information Systems Department at Sadat
Academy for Management Sciences, Cairo, Egypt. Her research interest is the e-learning,
learning management systems, and education technology.