Content uploaded by Fawwad Hassan Jaskani
Author content
All content in this area was uploaded by Fawwad Hassan Jaskani on Aug 30, 2022
Content may be subject to copyright.
International Journal of Software Engineering (IJSE), Singapore
Volume 2: Issue 3: Page 34-50
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
K-Nearest Neighbour-Based Machine Learning
Usability Evaluation Method for E-Learning Mobile
Systems
Muhammad Asghar*
Department of Computer Science
Islamia University of Bahawalpur,
Pakistan
m.asgharazeem@gmail.com
Fawwad Hassan Jaskani
Department of Computer Engineering
Islamia University of Bahawalpur,
Pakistan
favadhassanjaskani@gmail.com
Abstract—Mobile e-learning applications' usability is
mostly dependent on features such as learning, cognition and
memory, and their efficiency. User perception and usability
features are grouped according to user perception, and there is
no tool or theory to analyze or access these features. In this
study, we used a combination of quantitative and qualitative
research methods. Surveys in quantitative analysis are used to
gather information on mobile app user needs and the
perceptions of students. Taking into account the user's
perspective (student), we determined that efficiency is the most
important feature, and we used a Genetic Algorithm to score
usability. The weight of each usability feature was taken into
account while assigning a grade. Predicted usability attributes
based on Machine Learning Model predictions reveal a
prediction accuracy of 80.46 percent for KNN. E-learning
programmes can benefit from the use of this method to assess
the most critical usability requirements.
Keywords—KNN, Machine Learning, E-Learning
I. INTRODUCTION
The fundamental difficulty facing mobile learning users is
the usability of mobile learning applications, which give
users with a more or less optimal mobile learning
experience. Learning, efficiency, memorability, accuracy
and user satisfaction are the core five components that
constitute usability in research [1]. The usability element
relies mostly on the concept of literature-based ease of use.
It's easy to use if a user can complete jobs simply, which
means he concentrates on enhancing task efficiency and is
associated with removing problems from the user interface.
It also includes user pleasure in enhancing, appealing and
aesthetically acceptable mobile learning applications.
Distance training has unique advantages. It gives a winning
technique to meet specific needs, such as over-crowded
educational facilities, and it can help students and professors
live far away from schools and universities every time and
everywhere. Their "everywhere" time. If there is already a
learning material it can be a beneficial resource for certain
students, including students with disabilities. E-learning is
the latest remote learning method by sharing information
and processes for online training. Users are given detailed
data and services including cultural events, technological
experiences, facilities and physical and cognitive skills. E-
learning facilities for learners across the country are highly
important to reduce the digital divide and gap in social and
cultural application. Maximum user compatibility and
access should be a primary target for producers of e-learning
apps and a prerequisite that users can use these apps
financially. Many aspects contribute to the usability of a
mobile app, which affects a user's overall efficiency and
efficiency. Three main results should be achieved using an
interface: (a) Usability of the mobile app enables user
acquaintance with the UI · Users should easily reach their
goal by using the app.
Error-free programmes must be available. The other aspects
are quiet if your app is not working correctly.
Usability issues are further enhanced by user interface
mistakes, which need trouble-fixing instead of going on in
every step or click.
The system does not support formats, and data transmission
is problematic because mobility users suffer compatibility
problems while installing third-party solutions. Because of a
lack of responsive interface when utilizing various devices,
users experience usability challenges. Even when a new
interface is viewed on mobile learning applications, it
becomes frustrating for users [1]–[4]. Problems discovered
in study are solely linked to usability patterns in M-learning
applications for the qualitative point of view. These
concerns originating from algorithm tactics to overcome
further usability concerns are also highlighted in this study.
The proposed technique covers a vast range by applying
traditional usability assessment methodologies with GA and
generative machine learning models. This study also helps
to identify and prioritize usability concerns for M-learning
apps and helps to create a holistic approach to usability
concerns for mobile learning applications. The study will
focus on the following objectives:
(a)To identify the usability assessment problems in mobile
learning applications being used in Pakistan educational
institutes.
(b)To propose a new and improved model of usability
assessment for m-learning applications based on K-Nearest
Neighbors (KNN).
(c)To develop a prototype tool of the proposed model.
(d)To evaluate the tool for its effectiveness and efficiency.
II. LITERATURE REVIEW
This section presents some state of art research studies,
using machine learning for usability features clustering and
classification. Oztekin et al [1] provide information about
machine learning-based usability evaluation methods for
eLearning. They introduce a new machine learning-based
usability assessment approach for eLearning systems. The
proposed approach will provide usability experts with
invaluable advice on improved measures.
Jovic et al [2] purposed a new technique for implementing
machine learning-based methods in E-learning. They look at
how machine learning and its methods can be applied to
various e-learning platforms and learning styles. E-learning
frameworks can be built to be more effective for students
and teachers using machine learning. Students will benefit
from more personalized learning material, which will
improve their motivation and learning experience. In
contrast, teachers will benefit from automated activities,
saving time organizing, visualizing, and preparing learning
content. Instructors can easily classify "at-risk" students by
evaluating the collected and analyzed data, and machine
learning algorithms can modify course content to help each
student overcome their weaknesses. Bayesian Networks
(BNs), Decision Trees (DT), Artificial Neural Networks
(ANN), Deep Learning (DL), Association Rule Mining
(ARM), and Clustering methods are among the machine
learning models examined in this paper (CM).
Nazeeh et al [3] provide knowledge level assessment in e-
learning using machine learning. Their study proposes a
new framework for a complex learning environment that
follows current e-Learning trends. The experimental results
show that a support vector machine model outperforms
others in determining information levels, with 98.6% of
instances correctly categorized and a mean absolute error of
0.0069.
Moubayed et al [4] provide information about challenges
and research opportunities in E-learning using machine
learning. The area of e-learning is examined in terms of
meanings and characteristics in this article. Furthermore, the
various difficulties that the various participants in this phase
face are addressed. In addition, some of the works that have
been suggested in the literature to address these issues are
discussed. Then there's a quick rundown of some of the
most common ML and DA techniques. Finally, some of the
research opportunities that use such techniques are
suggested in order to provide insight into areas that merit
further study and exploration.
Khanal et al [5] purposed new recommendation systems for
e-learning using machine learning. The four strands of
content-based, collaborative filtering, knowledge-based, and
hybrid systems are used to provide an overview of
recommendation systems in the e-learning context. They
created a taxonomy that includes all of the elements needed
to create an efficient recommendation framework. Machine
learning methods, algorithms, datasets, measurement,
valuation, and performance were discovered to be essential
components. By offering a much-needed summary of the
current state of research and remaining challenges, this
paper provides a major contribution to the field.
Aher et al [6] purposed a new technique for a
recommendation of courses in E-Learning using machine
learning algorithms. Course Recommendation System
makes use of data mining techniques like clustering and the
association rule algorithm. Based on the preferences of other
students for a specific set of courses, the system
recommends a course to the student. Weka, an open-source
data mining application, was used to compare the findings.
Deng et al [7] purposed a new technique for student
assessment in E-learning using machine learning. They
provide a system for measuring students' attention focused
on machine learning methods. The system uses a Gabor
wavelet to extract eye state features and trains the model
using support vector machines (SVM) to complete
automatic classification on students' eye states. Experiments
on thousands of facial images show that the proposed
method achieves a high level of accuracy, which is useful in
real-world applications.
Lu et al [8] provide information about e factors affecting
acceptance of E-Learning using machine learning. Six
hundred and seventy-nine data samples were obtained from
303 students at Vietnam's Academy of Journalism and
Communication (AJC). The prediction results are up to
81.52 percent accurate, which helps students select the best
learning method for them.
Huang et al [9] give data about the evaluation of E-learning.
The proposed learning boundary improvement systems can,
for instance, measure understudies' powerful web-based
learning time, separate the segment of a message in the
conversation segment that is firmly identified with the
learning themes, and identify literary theft in understudies'
schoolwork. The numeric boundaries are then taken care of
into a Support Vector Machine (SVM) classifier to foresee
every student's achievement to see whether they match the
understudies' examination propensities. The test results
show that after the contributions to the classifier are
''decontaminated" by the learning boundary improvement
instruments, the forecast rate for the SVM classifier can be
expanded up to 35.7 percent by and large. This remarkable
accomplishment shows that the proposed calculations do
produce compelling learning boundaries for broadly utilized
e-learning stages.
Lykourentzou et al [10] purposed another method for
Dropout expectation in e-learning utilizing machine
learning. A dropout forecast approach for e-learning courses
is proposed in this paper, which depends on three regular
machine learning strategies and definite understudy
information. Feed forward neural organizations help vector
machines, and probabilistic outfit summed up fluffy
ARTMAP are the machine learning strategies utilized. Since
one procedure may neglect to effectively distinguish certain
e-learning understudies while another succeeds, three choice
plans were additionally assessed, which joined the
aftereffects of the three machine learning methods in an
unexpected way. The technique's discoveries were
discovered to be fundamentally better compared to those
distributed in the connected writing as far as generally
speaking exactness, affectability, and accuracy.
Mahboob et al [11] purposed another procedure for r
understudy appraisal in E-Learning utilizing machine
learning. In this investigation, classifiers like Decision
Trees-J48, Naive Bays, and Random Forest are utilized to
improve the nature of understudy information by
eliminating boisterous information first and having better
prognostic precision accordingly. The paper's center has
been limited to undergrad programs. The discoveries of the
test incorporate a bunch of rules for understudies with low
evaluations. Execution looking at was additionally conveyed
to guarantee that the discoveries were correct, precise, and
solid.
Purwoningsih et al [12] purposed another method for online
students' practices location utilizing machine learning. The
point of this paper is to recognize understudy action in e-
learning as an ally of learning setting examination. LMS
information and a social segment profile from the Student
Information System were utilized to decide understudy
action designs (SIS) following E-Learning, three potential
classes of understudy conduct patterns arose.
Borakati [13] purposed another method for clinical E-
learning course utilizing machine learning. This exploration
depended on a multi-focus associate investigation that took
a gander at gastrointestinal recuperation after elective
medical procedure. Partners were approached to finish a
progression of e-learning modules on significant parts of the
examination, just as an input survey. The AFINN brought
about a mean conclusion score of + 1.54/5.
Singh et al [14] purposed another procedure for courseware
evaluation utilizing machine learning. The advantages of
utilizing Machine Learning (ML) as an e-arranging
technique to support learning and courseware creation are
featured in this paper. Understudies' course appraisal
overviews are generally viewed as amazingly exact and in
any event modestly tenable by scientists. Low reaction rates,
reprisal, evaluations, and examinations with past educators,
then again, would all be able to influence the outcome's
unwavering quality. In this paper, machine learning
calculations were utilized to keenly examine communication
log information from the LMS to make a prescient guide
that permits understudies' online association conduct to be
planned to their course result. The different learning
apparatuses and practices, just as their adequacy all through
the course, are then assessed and approved utilizing
different ML calculations.
Tan et al [15] give data about Student Dropout in E-
Learning program using machine learning technique.
Individual qualities and scholastic achievement were picked
as information attributions in this investigation. Fake Neural
Networks (ANN), Decision Trees (DT), and Bayesian
Networks were utilized to assemble forecast models (BNs).
In the model preparing and evaluation strategies, an
expansive example of 62,375 understudies was utilized. The
consequences of each model were plotted in a disarray
framework and the paces of exactness, accuracy, review,
and F-measure were determined. The outcomes showed that
every one of the three machine learning approaches were
fruitful at foreseeing understudy dropout, yet DT performed
better. At last, a few suggestions for future examination
were made.
Hamdi [16] provide information about machine Learning
based UML for ELearning Settings. Thus, we suggest
describing an early attempt to bridge the gap between web-
based learning and experience-learning agents. The ultimate
aim is to create a fully-automated multi-agent environment
capable of assisting in the elaboration and distribution of
highly-personalized educational material efficiently for
everyone, anywhere at any time, taking into account the
personal profile and complex actions of each learner during
the eLearning process. Authors depend on the framework of
software engineering to explain techniques from early
concepts to fully-developed systems. For now, and as far as
this paper is concerned, the attempt is to focus on the
relationship between two core areas, Unified Modelling
Language (UML) and agents. Tangible outcomes remain the
integration of machine learning agents based on approaches
such as Decision Tree Learning, AdaBoost, and Ensemble
Learning. Fuzzy agents are a special case of soft computing
approaches used for profile customization.
Munwar et al [17] provide information about Machine
Learning Techniques in eLearning Paradigm. E-learning is
an important tool and methodology to train and teach people
these days. Via eLearning and distance learning paradigm,
several world ranking degree awarding universities have
begun to deliver various courses for high school grade
education to graduate level and even postgraduate level.
This research paper focuses extensively on supervised and
unsupervised learning methods and techniques that allow
the e-learning system to simplify learner response and
student inquiries. The primary drawback of e-learning
model is not timely responses to learner's questions, which
decreases student's learning curve. The main demand in the
e-learning world is to deal with the classification of machine
learning, but not completely automated with this model.
Training data set is used to train the system, then test data
set to verify this method. This paper analyses the various
techniques of machine learning and proposes a solution
using these techniques.
Krendzelak et al [18] provide information about Machine
Learning and its applications in e-Learning systems.
Exploiting the ever-increasing online learning frameworks
assumes a paramount part of self-adjustment, particularly
because of working people. All things considered, learning
systems do not mostly adjust to the profiles of learners.
Apprentices need to spend a lot of time before reaching the
learning goal perfect with their insight base. This paper
explores machine learning and its e-learning frameworks
implementations. Machine learning is a kind of artificial
intelligence (AI) that gives machines the ability to learn
without being specifically customized. Machine learning
focuses on advancing machine projects that can evolve and
change with new information.
Hall et al [19] give data about s Improving Human
Arithmetic Learning utilizing Machine Learning. They
anticipate which future inquiries a customer would answer
inaccurately dependent on an assortment of math questions
and replies? Information from PC produced specialists and
human clients was utilized to survey the exactness and
appropriateness of four machine learning models. Our
models accomplish correctness’s of around 79% to 96
percent on virtual specialists, with choice trees playing out
the most noteworthy. Our models accomplish exactness’s in
the scope of 63% to 69 percent on human information, with
the choice tree beating different methodologies by and by.
We intend to coordinate these mistake expectation models
into future E-learning frameworks focused on human
number juggling learning.
Yan et al [20] provide information about online learning
analysis based on machine learning. The aim of this paper is
to look at the relationship between students' online learning
habits and their course grades. Teachers should understand
the learning impact of each course in a typical learning
environment. The data created by the learners' interaction
with the operating platform during the learning process is
known as online behavior data.
Hussain et al [21] give data about distinguishing helpful
meetings in an e-learning framework utilizing machine
learning. In view of understudies' responsibility, inspiration,
intricacy, and dedication during the course, we utilized
machine learning (ML) calculations and relapse
investigation to order advantageous meetings. The
discoveries showed that the meeting scores are firmly
identified with the information understudy qualities
(commitment, trouble, responsibility, and steadfastness).
Moreover, the discoveries show that profound learning and
irregular woods models are viable machine learning
calculations for foreseeing helpful meetings.
Siddiq et al [22] provide information about Machine
Learning in E-Learning. Many things are possible with e-
learning, particularly for learners who want to learn
whenever and wherever they want online. Predicting grades
is the first step in personalization. The second step is to
compose the recommendation. We addressed the
complexities of e-learning in this project.
William et al [23] proposed another strategy for
development of online schooling model with the
combination of machine learning. The occasions of the year
2020 have shown that society is as yet delicate. A pandemic
like Coronavirus infection 2019 has shown this. The manner
in which society directs all things has changed. This
incorporates schooling, which has marked its future on the
utilization of ICTs.
Vijai et al [24] Analyzed ELearning platform reviews using
SVM. Users are taking more eLearning courses than ever
before, particularly in the aftermath of a pandemic. The
opinions of registered users are used to share the
satisfactions and criticisms of the courses. The text reviews
received from the feedback or comments sections of most
online course webpages are found to be unstructured and
inconclusive.
Krendzelak et al [25] gives data about the use of machine
learning in E-learning. Machine learning and its executions
in E-Learning frameworks are explored in this paper.
Machine learning is a type of man-made brainpower (AI)
that permits machines to learn without being unequivocally
modified. Machine learning centers around the advancement
of machine projects that can develop and improve as new
information is presented. In the light of above research,
there is need of an intelligent usability features predictive
system.
III. METHODOLOGY
This section describes the main methodology of this
research. First we have collected information form students
and on the basis of their recommendation we have scored
the usability features by using Genetic Algorithm, details
are as follows:
As we have used supervised and unsupervised classifiers
for our data validation, so in this step we train our
unsupervised learning classifier with unlabeled data. And
send the results to the normalized database for clustering.
First we have used Genetic algorithm in this approach and
then applied support vector machine classification model for
predicting the best features.
A. Data Collection
We have collected the data from the students of the Islamia
University of Bahawalpur. The questions of the
questionnaires are twelve. The responses are collected from
online Google forums. We have created 19 Questions in a
questionnaire. These responses are then extracted into an
excel sheet into a CV file. There are a total of 40000
students in the Islamia University of Bahawalpur. The
dataset contains 200 students (samples of 500) of different
departments of Islamia University of Bahawalpur. The
excel sheet responses will be further used for data analytics.
There are two types of questions, i.e. Open-ended
(universal) and Closed-ended (individual) questions.
There are a total of 40000 students in the Islamia University
of Bahawalpur. The dataset contains 200 students (samples
of 500) of different departments of Islamia University of
Bahawalpur. The excel sheet responses are used for data
analysis. There are two types of questions, i.e. Open-ended
(universal) and Closed-ended (individual) questions in the
questionnaire. Figure 3.2 shows the proportion of different
departments from which people responded i.e. Computer
Science and IT department from Islamia University of
Bahawalpur and S.E College Bahawalpur:
Figure 1 Graph explains the proportion of different
departments from which people responded
Table below indicates the qualitative and quantitative
analysis as well as the number of usability assessment
methods (UEMs) applied to each usability attribute. The
responses of the students is then coded in an excel sheet. In
this table, it can be seen the number of occurrences e.g.
Survey, Controlled Observations, Eye tracking, Thinking
aloud and interview, are usability evaluation methods which
are applied to particular usability attributes i.e. Efficiency,
Satisfaction, Effectiveness, Learnability, Memorability,
Cognitive Load, Errors, Simplicity and Ease of Use.
Table 1 Number of occurrences of usability evaluation
methods (UEMs) applied to particular usability attributes.
Attribute
Surve
y
Controlled
Observation
Eye
Tracking
Thinking
Aloud
Inter
view
Efficiency
3
8
0
0
0
Satisfaction
1
0
0
0
0
Effectiveness
5
6
0
0
0
Learnability
3
5
0
0
0
Memorabilit
y
4
3
1
1
0
Cognitive
Load
0
1
1
1
0
Errors
1
4
9
0
0
Simplicity
4
2
9
0
1
Ease of Use
2
0
9
0
0
After asking questions from different students, we have
collected the usability information in descriptive and visual
forms. Extracting usability features from quantitative data is
a challenging task because gathering information from the
respondents was not easy. The quantitative analysis of
questionnaires provides the knowledge and capability for a
greater understanding of choice of decisions. After asking
questions from different students we have collected the
usability information in descriptive and visual forms.
Following Table 3.3 shows the number of polarity of each
response from students response polarities are 1 for Highest
polarity i.e. Strongly Agree and 5 for lowest polarity i.e.
Strongly Disagree:
Table 2 Response Polarity
Features
1.
Strongly
Agree
2.
Agree
3.
Neutral
4.
Disagree
5.Strongly
Disagree
Efficiency
48
22
21
8
7
Effectiveness
38
26
18
6
9
Ease of Use
39
28
18
8
7
Learnability
41
27
21
4
5
Memorability
29
10
9
7
13
Cognition
12
9
7
5
13
Consistency
9
7
6
4
14
Figure below shows the score of each feature in bar graph,
with the score of all recommended features, i.e. Efficiency
with 0.553, effectiveness with 0.434, Ease of use 0.350,
Learnability 0.210, Memorability with 0.250, and Cognition
with 0.125 and consistency with 0.125.
Figure 2 Score vs features bar graph
B. Framework
The proposed approach shown in figure below has three
phases as shown in Figure 3:
In first phase we will apply some quantitative analysis
on the data and then pass to next phase where we select
the best features that has been scored by the students as
best weighted features.
After features selection the data passed to machine
learning models where the prediction of the best
features occurred by training of the GA based SVM. In
this phase we have used different machine learning
models that are performing on the basis of some
evaluation parameters.
On the basis of these parameters we will evaluate the
model of proposed framework. We also show a way of
extracting a classified list of leading features to boost
this
Figure 3 Architecture of Designed System
C. Inclusion of Multi-Objective Optimization
An optimization approach should be used for the right and
optimum solution or benefit. The dilemma of optimization
comprises of one or more goals or of seeking minimal or
optimum benefit. MOO is called; the problems that have
more than one goal.
Such questions can be seen in everyday life as in the fields
of architecture, mathematics, economy, social sciences,
agriculture, automobile, aviation and more. In different daily
challenges, the priorities under discussion are confounding.
And refining a given approach for a single purpose will
contribute to unintended outcomes for the other goals.
A logical and practical approach to a multi-objective
dilemma is to pursue any solution that achieves its goals at
the required level without any additional solution.
According to our work, two GA and NSGA-II algorithms
are built for problems with multiple objectives.
GA and NSGA-II are metaheuristic and ideally suited to
various kinds of issues addressed in the next segment. In the
other side, conventional GA and NSGA-II are updated to
include numerous objective challenges by providing
answers to the diversity and the usage of exercise functions.
D. Multi-Objective Genetic Algorithm
Multi-target GA is a population-based solution. It is useful
for addressing multi-target challenges and interface
difficulties. A GA may be adapted to find a variety of
solutions that are not affected by each other in one round.
The capacity of GA to scan simultaneously multiple areas of
solution space and can identify numerous solutions for
difficult multimodal, discontinuous and non-convex
solutions.
GA crossover operator may accomplish decent solutions
systems with diverse goals in order to create new solutions
that are not affected by unknown areas of the Pareto front.
In comparison, the operator would not use any of the multi-
objective GA to scale and prioritize goals. GA was thus the
most popular heuristic approach used in our work for
optimizing features. Figure below shows the GA based
KNN architecture used in this research:
Figure 4 GA Based KNN Architecture for Usability
Features Classification
E. GA Based KNN Classification
In the KNN classification, classifier operates in a function
field dependent on the mean of the adjacent training
samples. KNN is one of the trustworthy classification
algorithms and is part of the supervised classification.
Clustering is one of the most common principles in the field
of unattended analysis. Our theory is that identical data
points belong to similar classes or clusters, based on their
distance from the local centers. K-means is a commonly
used algorithm for clustering. It produces 'k' similar data
point clusters. Data incidents outside such classes can be
identified as exceptions.
K-Nearest Neighbors (KNN) is one of the simplest
algorithms used for regression and classification problems in
machine learning. KNN algorithms use data to identify new
data points based on measurements of similarity (e.g.
distance function). A majority vote classifies its neighbors.
In this study we have used following parameters:
Table 3 KNN Classification Parameters
Parameters
Values
Description
N_neighbors
Any integer
value
Number of k
neighbors on the
basis of which
closest match
classified in the
usability.
IV. RESULTS
In classification K-NN, classifier operates in feature space
dependent on adjacent training samples. We checked KNN
model on our data and followed the findings seen in figures
below, we tested on test sixth 0.2, 0.3, 0.4 and modified the
distance i.e. Manhattan, Euclidean and Hamming distance
respectively. We have modified n (neighbor’s values).
Following are the prediction results of KNN model at the
learning rate of 70%.
The process could be discussed in a separate section if the
experiment is complex. IUB students have been employed
as our population and raw data from questionnaires, mobile
use, observation and testing will be collected, which implies
its unlabeled data. We turn our data into a processing data
frame. We add labels and processing features. We add
counts of words and weights, etc. This functionality is
introduced to the CSV data file.
We collected data from students at Bahawalpur Islamic
University. The questionnaires are 12. The questions are 12.
The answers are gathered from Google forums online. In a
questionnaire, we developed 19 questions. These answers
are subsequently extracted to a CV file using an Excel sheet.
The Islamic University in Bahawalpur has a total of 40,000
students. The dataset includes 200 (500 samples) students
from several Islamic University departments in Bahawalpur.
Excel sheet answers are used to analyses the data. The
questionnaire has two sorts of questions, i.e. open questions
(universal) and closed questions (individual). Section 3
identifies qualitative and quantitative analysis and the
number of methods for the evaluation of usability (UEMs)
used for each usability feature. The kids' responses are then
coded in an excellent sheet. In this table, the number of
occurrences e.g. surveys, controlled observations, eye
tracking, thought aloud and interview, are usability
assessment methods that can be applied to specific usability
attributes such as efficiency, satisfaction, efficiency,
learning capacity, memory load, error, simplicity and ease
of use. This section is divided into two parts, GA based
feature selection and classification of usability features
using Hypertuned KNN Model
A. GA Based Feature Seleciton
According to GA based feature selection, Ease of use
feature has the highest scoring of 8, with benchmark 8.5,
GA Rank of -0.5. While effectiveness has scored 8 while
benchmark is 7, learnability scored was at 7 and benchmark
at 6.6. Table below shows the score, benchmark and rank of
each feature based on Genetic Algorithm Analysis.
Table 4 Feature Rank based on GA analysis
Features
Scoring
Benchmark
GA
Selection
Classification
Usability
Analysis
Ease of Use
8
8.5
-0.5
93%
Good
Effectiveness
8
7
+1
92%
Very
Good
Learnability
7
8
+0.3
89%
Average
Memorability
6
6.6
+0.2
95%
Very
Good
Cognition
5
7
-0.2
93%
Fair
Accuracy
5
6
-0.1
92%
Good
Training
4
5
+2
92%
Very
Good
Testing
3
2.4
-0.4
93%
Fair
Figure below shows the score of each feature as analyzed by
GA.
Figure 5 Features Ranking by GA
Figure 6 Feature Selection Rate by GA
Figure above shows the feature selection by GA.
B. KNN Based Classification
Feature selected by GA has been splitted into testing and
training set. Further on which KNN is trained and evaluated.
Below results shows the prediction of best usability feature
by different hyper tuned model of KNN.
i. Neighbors
KNN has been tested at different number of neighbors. At
n=3 KNN has shown the best accuracy of 88.5% in
predicting the correct feature as recommended by students.
Figure below shows the results of KNN using different
numbers of N_neighbors:
Figure 7 Results of KNN Model with respect to Different
Numbers of Neighbors
ii. Distance
Three types of distances based classification has been
carried out, namely: Minkowski, Euclidean and Hamming
distance. KNN has shown the highest accuracy of 90.45%
with Minkowski. Figure below shows the comparison of
different distances based classification of KNN Model:
Figure 8 Hypertuned Distances based Classification
Based on Hypertuned models, classification of KNN Model
has been conducted again to predict the highly
recommended feature. Following figure shows the list of
highly recommended features as classified by KNN Model
at distance = Minkowski and N = 3.
Figure 9 Highly Recommended Features by KNN Model
Ease of Use has recommended by KNN model as highly
recommended usability feature at the accuracy of 92%.
V. CONCLUSIONS
This article aims to highlight usability difficulties in
education apps in Pakistani educational institutions. It aims
to make it easier to adapt digital technologies in education.
This work is focused on quantitative and quality analysis
and we will take interviews and questionnaires in this study
to determine the mental perception of users within the
framework of the designer. This paper identifies the main
aspects for the usability of these applications and will also
offer the design heuristics for the design of such
applications. Considering the impression of the user
(student), Efficiency has been analyzed as the highest
usability attribute to satisfy the student. After this we
employed a Genetic Algorithm (GA) to evaluate the
usability features. Each usability feature is measured by the
weight of each usability feature. We have determined the
rank of main functions utilized in the Genetic Algorithm
questionnaire and benchmark. After that, GA has been used
to choose features, GA have obtained the best features and
we have also projected the usability characteristics of
students based on selected features based on the best
features. GA based KNN model with default parameters has
considered the best algorithmic approach in predicting the
best characteristics with 92% accuracy. Furthermore this
study will be improved by adding feature selection and
extraction algorithms as well as by using Support Vector
Machine with different kernels.
REFERENCES
[1] A. Oztekin, D. Delen, A. Turkyilmaz, and S. Zaim, “A machine
learning-based usability evaluation method for eLearning
systems,” Decis. Support Syst., vol. 56, no. 1, pp. 63–73, 2013,
doi: 10.1016/j.dss.2013.05.003.
[2] J. Jović, M. Milić, S. Cvetanović, and K. Chandra,
“Implementation of machine learning based methods in elearning
systems,”
10th Int. Conf. eLearning (eLearning-2019), 26-27 Sept., no.
September, pp. 39–44, 2019.
[3] N. Ghatasheh, “Knowledge Level Assessment in e-Learning
Systems Using Machine Learning and User Activity Analysis,”
Int. J. Adv. Comput. Sci. Appl., vol. 6, no. 4, 2015, doi:
10.14569/ijacsa.2015.060415.
[4] A. Moubayed, M. Injadat, A. B. Nassif, H. Lutfiyya, and A.
Shami, “E-Learning: Challenges and Research Opportunities
Using
Machine Learning Data Analytics,” IEEE Access, vol. 6, pp.
39117–39138, 2018, doi: 10.1109/ACCESS.2018.2851790.
[5] S. S. Khanal, P. W. C. Prasad, A. Alsadoon, and A. Maag, “A
systematic review: machine learning based recommendation
systems for e-learning,” Educ. Inf. Technol., vol. 25, no. 4, pp.
2635–2664, 2020, doi: 10.1007/s10639-019-10063-9.
[6] S. B. Aher and L. M. R. J. Lobo, “Combination of machine
learning algorithms for recommendation of courses in E-Learning
System based on historical data,” Knowledge-Based Syst., vol.
51, pp. 1–14, 2013, doi: 10.1016/j.knosys.2013.04.015.
[7] Q. Deng and Z. Wu, “Students’ Attention Assessment in
eLearning based on Machine Learning,” IOP Conf. Ser. Earth
Environ. Sci., vol. 199, no. 3, 2018, doi: 10.1088/1755-
1315/199/3/032042.
[8] D. N. Lu, H. Q. Le, and T. H. Vu, “The factors affecting
acceptance of e-learning: A machine learning algorithm
approach,” Educ.
Sci., vol. 10, no. 10, pp. 1–13, 2020, doi:
10.3390/educsci10100270.
[9] C. J. Huang, S. S. Chu, and C. T. Guan, “Implementation and
performance evaluation of parameter improvement mechanisms
for intelligent e-learning systems,” Comput. Educ., vol. 49, no. 3,
pp. 597–614, 2007, doi: 10.1016/j.compedu.2005.11.008.
[10] I. Lykourentzou, I. Giannoukos, V. Nikolopoulos, G. Mpardis,
and V. Loumos, “Dropout prediction in e-learning courses
through the combination of machine learning techniques,”
Comput. Educ., vol. 53, no. 3, pp. 950–965, 2009, doi:
10.1016/j.compedu.2009.05.010.
[11] T. Mahboob, S. Irfan, and A. Karamat, “A machine learning
approach for student assessment in E-learning using Quinlan’s
C4.5, Naive Bayes and Random Forest algorithms,” Proc. 2016
19th Int. Multi-Topic Conf. INMIC 2016, 2017, doi:
10.1109/INMIC.2016.7840094.
[12] T. Purwoningsih, H. B. Santoso, and Z. A. Hasibuan, “Online
Learners’ Behaviors Detection Using Exploratory Data Analysis
and Machine Learning Approach,” Proc. 2019 4th Int. Conf.
Informatics Comput. ICIC 2019, no. Icic, pp. 1–8, 2019, doi:
10.1109/ICIC47613.2019.8985918.
[13] A. Borakati, “Evaluation of an international medical E-learning
course with natural language processing and machine learning,”
BMC Med. Educ., vol. 21, no. 1, pp. 1–10, 2021, doi:
10.1186/s12909-021-02609-8.
[14] S. Singh and S. P. Lal, “Educational courseware evaluation using
machine learning techniques,” 2013 IEEE Conf. e-Learning, e-
Management e-Services, IC3e 2013, pp. 73–78, 2013, doi:
10.1109/IC3e.2013.6735969.
[15] M. Tan and P. Shao, “Prediction of student dropout in E-learning
program through the use of machine learning method,” Int. J.
Emerg. Technol. Learn., vol. 10, no. 1, pp. 11–17, 2015, doi:
10.3991/ijet.v10i1.4189.
[16] A. Hamdi-cherif, “Machine Elearning – Learning Agents and
UML for Elearning Settings,” vol. 2, no. 1, 2008.
[17] M. M. Iqbal, M. Farhan, Y. Saleem, and M. Aslam, “Automated
Web-Bot Implementation using Machine Learning Techniques in
eLearning Paradigm,” 11th Mex. Int. Conf. Artif. Intell., no.
January, 2012.
[18] M. Krendzelak, “Machine learning and its applications in e-
learning systems,” ICETA 2014 - 12th IEEE Int. Conf. Emerg.
eLearning Technol. Appl. Proc., pp. 267–269, 2015, doi:
10.1109/ICETA.2014.7107596.
[19] T. Hall and H. Kamper, “Towards improving human
arithmetic learning using machine learning,” 2020 Int.
SAUPEC/RobMech/PRASA Conf. SAUPEC/RobMech/PRASA
2020, 2020, doi:
10.1109/SAUPEC/RobMech/PRASA48453.2020.9041140.
[20] N. Yan and O. T.-S. Au, “Online learning behavior analysis
based on machine learning,” Asian Assoc. Open Univ. J., vol. 14,
no. 2, pp. 97–106, 2019, doi: 10.1108/aaouj-08-2019-0029.
[21] M. Hussain, W. Zhu, W. Zhang, J. Ni, Z. U. Khan, and S.
Hussain, “Identifying Beneficial Sessions in an e-learning System
Using
Machine Learning Techniques,” 2018 IEEE Conf. Big Data Anal.
ICBDA 2018, pp. 123–128, 2019, doi:
10.1109/ICBDAA.2018.8629697.
[22] S. Siddiq, “Machine Learning in E-Learning Computer Science
4490Z Final Report Machine Learning in E-Learning Department
of Computer Science London , Ontario , Canada Author :
Samirah Siddiq Email : ssiddi62@uwo.ca Supervisor : Dr . Hanan
Lutfiyya,” no. May, pp. 0–39, 2017, doi:
10.13140/RG.2.2.36213.37605.
[23] W. Villegas-Ch, M. Román-Cañizares, and X. Palacios-Pacheco,
“Improvement of an online education model with the integration
of machine learning and data analysis in an LMS,” Appl. Sci.,
vol. 10, no. 15, 2020, doi: 10.3390/APP10155371.
[24]
[25] R. Muralidharan, T. Kanagasabapathy, and R. P. Vijai Ganesh,
“Analyzing ELearning platform reviews using Sentimental
Evaluation with SVM Classifier,” J. Phys. Conf. Ser., vol. 1767,
no. 1, 2021, doi: 10.1088/1742-6596/1767/1/012012.
[26] M. Krendzelak, “Machine learning and its applications in e-
learning systems,” ICETA 2014 - 12th IEEE Int. Conf. Emerg.
eLearning Technol. Appl. Proc., no. August, pp. 267–269, 2015,
doi: 10.1109/ICETA.2014.7107596.
