Content uploaded by Marjan Gusev
Author content
All content in this area was uploaded by Marjan Gusev on May 24, 2017
Content may be subject to copyright.
Content uploaded by Marjan Gusev
Author content
All content in this area was uploaded by Marjan Gusev on Jul 09, 2015
Content may be subject to copyright.
Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
FACTA UNIVERSITATIS(NI ˇ
S)
SER.: ELEC.ENERG. vol. 16 March 2005, 253-262
Improved Learning Methodology System
Marjan Guˇ
sev
Abstract: Education is one of the systems that have been always upgrading. During
past years the teaching process has been always modified and upgraded in order to
enable more efficient learning. The use of new ICT technologies enables innovative
ideas to make the learning process more efficient. The students gain with better skills
and obtained knowledge by these innovations.
We established an e-learning system that supports the education process. This
system offers not just content available on-line by using web technologies, but also a
wide variety of simulators, animations and films as multimedia approach to the sys-
tem. In addition to this system we use a system of interactive quizzes and question-
naires as part of the self testing tool that help students understand basic concepts and
acquire skills. The system of e-testing helps the professor to assess the knowledge and
grade the students. The on-line learning tool is the interactive system that supports the
homework assignment and grading tool and helps both the students to learn and the
professors to check the knowledge obtained.
In this paper we report the results from using these innovations as improvedlearn-
ing methodology,and how it affects the level and degree of obtained knowledge and
skills, students’ interest and average score in total knowledge.
We present indicators to measure the results gained by introducing new method-
ology. These indicators concern quantity and quality measures, obtained by analyzing
the pass rate and average scores in the class.
Keywords: E-learning, E-testing, on-line learning, knowledgeassessment.
1 Introduction
The instructivism principle is used usually in university and high educational sys-
tems as conventional method of teaching. At our institution the lecturing is mainly
realized with huge classes that have more then 150 students [1]. There are a lot of
Manuscript received February 2, 2004
The author is with Faculty of Natural Sciences and Mathematics, Arhimedova 5, 1000 Skopje,
University of Ss. Cyril and Methodius, Skopje, Macedonia, E-mail: anastas,marjan@ii.edu.mk
253
254
M. Guˇsev
problems associated with big classes, like no personal contact with the students, no
possibility for student’s active participation during lectures, no possibility for inter-
active communication, no efficient way to setup different assignments and projects
and check students knowledge. Solution to these problems is usually solved by
several teaching assistants, which involves different criteria and similar problems.
2 Methodology
A possible solution to analyzed problems is offered by the usage of challenges of
the new emerging ICT technologies. A new e-learning platform is established as
complementary process to teaching. We use not only as support to learning, but
also as support to the teaching process.
The system is efficiently used in the realization of homework assignments and
projects. The new e-testing system enabled a lot of benefits to realize assessment
of obtained knowledge each week during the course and enabled conditions for
continuous grading and assessment.
The innovations in usage of new ICT technology required a lot of e-Learning
content to be available on web. Afterwards it also required development of new
tools that support management and administration of users, privileges, access rights,
but also a development of new system for realization of quizzes and question-
naires, and interactive communication with students for homework assignment and
knowledge assessment. To realize the new technology challenges we had to change
the conventional grading and evaluation system and establish new paradigms [2].
These systems are efficiently used for evaluation of students’ participation during
the complete semester and objective evaluation of obtained knowledge and skills.
These innovations were implemented for several courses and in this article
we measure the performance of the following: Computer Architecture, Advanced
Computer Architecture, Microprocessors and microcontrollers and Internet pro-
gramming [3], [4].
3 Evaluation System
The evaluation systemwas build to enable the following aims: to measure students’
participation and assess obtained knowledge and skills. It should enable a new
approach in the learning process and motivate the students (1) for studying through
the whole year, (2) for active participation and (3) for finalizing the course at the
end of the semester. All these problems were the main reason for bad scores and
delayed finalization of the courses.
The continuous grading system helps the professor to check the student’s ability
Improved Learning Methodology System
255
of absorbing learning objectives. The feedback part of this system is the student’s
awareness of his/her weaknesses and knowledge gaps (if any). This system helps
the student learn the required course background on time and gain the required
skills. It also tracks the student activities and makes easier finishing the obligatory
homework and seminar projects on time.
The evaluation system is based 1/3 on coursework and 2/3 on midterm and final
exams. The detailed structure is presented on Table 1. Completed homework and
the seminar project are obligatory for the course. The homework assignments and
projects must be completed in a given time deadline. Late homework submission
may be accepted after the deadline only once gaining only half of the proposed
credits. Therefore this system motivates the students to do their homework on
time enabling the teaching staff to coordinate appropriately for evaluation of the
homework.
Table 1. Evaluation system
Assignment Credits %
Homework 50 16.7%
Project 1 20 6.7%
Project 2 30 10.0%
Midterm Exam 100 33.3%
Final Exam 100 33.3%
Total credits 300 100.0%
Table 2. Grading system.
Credits % Grade
271 - 300 90 % 10
241 - 270 80% - 90% 9
211 - 240 70% - 80% 8
181 - 210 60% - 70% 7
151 - 180 50% - 60% 6
0 - 150 50% FAILED
Instead of realization of ordinary homework assignments, we use the on-line
learning tool described in [5]. It enables the students to evaluate their knowledge
using Internet access either at the faculty or in home environment. The seminar
projects present practical skills obtained by the knowledge gained in classes. The
seminar projects are evaluated in the same way as the homework.
Each student can achieve a maximum of 300 credits. The grading system is pre-
sented on Table 2. The student can pass the course only if he achieves more than
50% the credits. Midterm and final exams (first and second colloquiums) test the
students’ knowledge, by two different ways as two sets of questions which include:
(1) theoretical part, performed on the online learning system and (2) practical ex-
ercises, as written part of the exam. At least 30% of the midterm and final exams
are required for the student to pass. In order to pass with the lowest grade the stu-
dent must complete the homework and pass both colloquiums, i.e. midterm and
final tests. The student can get the highest grade only if he/she completes both the
homework and the projects and achieves outstanding results on both colloquiums.
Those that fail have a chance to take an exam at the end of the semester provided
that they have submitted the homework assignments and projects.
The automated electronic testing system includes concepts of random question
256
M. Guˇsev
generation and random position of available option answers. This type of computer-
based testing has property to mislead the student of learning position numbers of
possible answers for a given question. Thus, instead of memorization of the ques-
tions and line numbers of the offered answer options, the students were forced
to learn the relations between questions and answers, i.e. the concepts and other
knowledge skills.
The assessment measures specific knowledge skill in given time constraint.
Negative grading for wrong answers is used as a system of punishment. This is
a discipline measure to prevent from cheating and guessing possible answer where
the student can win a positive assessment based on lottery or good luck. That’s why
the student is motivated to click only on answer options he/she is very sure that they
represent the right answer instead of guessing. He/she usually avoids clicking if he
is not sure about the answer since he will score negative points. This is another
motivation for students to learn real concepts and relations.
4 Achieved Results for Computer Architecture Course
The basic Computer Architecture course is placed in the second semester of In-
formatics studies at University of Cyril and Methodius, Faculty of Science and
Mathematics, Institute of Informatics [1] in Skopje, Macedonia. During the last
several years the course has been reorganized to meet the challenges and needs of
modern teaching and grading. The course information and statistics are accessible
over the Internet on [3].
In this course we implemented on-line accessible e-Learning course material,
covering all lectures and tutorials. All lecture slides, textbook and supporting ma-
terials, like animations and films were set on web. In addition to this, we created
a lot of simulators that helped students understand concepts. These simulators had
either web interface and run on our engines or were small programs easy for down-
load and usage on personal computers. For first time we invented VHDL and used
simulators in logic design.
Two projects were assigned, the first requiring programming tasks with im-
plementation of computer arithmetic and the second with VHDL design of simple
logic circuits. We were very glad to realize a great students’ interest in the second
project introduced with simple logic design and VHDL. The students were happy
and found a lot of fun realizing the project.
The homework assignments were realized by the on-line testing tool. A set of
1850 questions formed the question database.
Table 3 presents the pass rate of the students in June sessions. In academic
2001/2002 year we used conventional teaching system. There were 180 full time
Improved Learning Methodology System
257
students graded by the traditional system with written and oral exams. Only 43
students completed the exam successfully in the first exam session in June 2002.
The next academic year the results improved to84 students finished in June session
out of 195 that enrolled. In the third analyzed academic 2003/2004 year there were
169 enrolled students and 80 of them passed. This situation is presented on Figure
1.
Table 3. Pass rate for Computer Architecture course in june sessions
academic year enrolled passed %
2001/2002 180 43 23.9%
2002/2003 195 84 43,1%
2003/2004 169 80 47,3%
The quantity of students that passed the exam was increased from 23.9% to
43.1% and reached 47.3% in final year as shown on Figure 2. The rest of the
students took the exam in some later examination sessions. These numbers show
great improvement analyzing that it the pass rate reaches the rate of students that
actually continues studying.
Fig. 1. Exam Statistics in June Sessions for
Computer Architecture Fig. 2. Comparison of pass rate for Computer
Architecture course
In previous years the students used to pass the exam in later sessions and not
in the first (normal) June exam session. By inventing the new learning method
the students were motivated to pass the exam with continuous learning and use
the benefits of the new grading system. This is not due to lowering the quality
of the course materials, but due to the new methodology and motivation to keep
tense on the students to finish their obligations on time. Actually this is the greatest
achievement by invention of the new methodology in learning.
Not only the quantity indicators were improved, but also the quality indicators
show this tendency. The course statistics are presented in 4. The course average
was 63% for June sessions in academic year 2002/2003 and improved to 71.3% one
258
M. Guˇsev
year later. These averages are higher then averages obtained by first year students’
grades.
One very obvious fact is that there was a high interest to complete the home-
work and the seminar projects in time, since the students completed approximately
67.8% of the homework assignments and 94% ofthe seminar projects. About 29%
of the students did not finish the obligatory homework and were not allowed for
exam. Additional 16.9% of the students did not finish at least one project. Aver-
ages obtained on midterm and final exams are 54.2% for academic year 2002/2003
and 68.1% in academic year 2003/2004.
Averages on homework assignments and projects show slight negligible changes.
However the highest improvement is in averages of midterm and final exams.
Table 4. Average credits gained per assignment for Computer Architecture course.
Type of assignment 2002/2003 2003/2004
Homework (maximum 5 10) 33,9 34,9
Project 1 (maximum 20) 18,2 15,7
Project 2 (maximum 30) 28,7 27,0
Midterm and final (maximum 2 100) 108,3 136,2
Course average (maximum 300) 189,1 213,8
Course average (%) 63,0% 71,3%
5 Achieved Results for Advanced Computer Architecture Course
Advanced Computer architecture course started in academic 2002/2003 year. The
course statistics are presented in [4] along with definition of projects and curricula.
Similar issues about advanced computer systems existed in the old curriculum
in the course with the title “Microprocessors and microcomputers”. It covered ILP
concepts and topics but lacked VLIW organization and compiler techniques. All
the students had to make only one project given at the end of the course, without
time limit. However, there were no deadlines, so it took the students almost a year
to complete the project, i.e. they finished the project before taking the exam. This
was the reason to set a real deadline for the project. This resulted in extremely
low interest in passing the course through active participation and colloquiums, i.e.
midterm and final tests.
The statistics about pass rate is shown on Table 5 and Figure 3. We can
observe that after introducing the new methodology the number of students that
have successfully passed in June sessions is increasing and approaching 100%.
One very obvious fact is that there was an extremely high interest to complete
homework assignments and projects on time. The students complain to the timing
Improved Learning Methodology System
259
Table 5. Pass rate for Advanced Computer Systems course in june sessions.
academic year enrolled passed %
2001/2002 42 24 57.1%
2002/2003 51 39 76.5%
2003/2004 66 63 95.5%
requirements, they had more time to complete the first project then the second,
which resulted in approx 90% for the first project and 84% for the second project.
The quality indicators of colloquia, i.e. midterm and final exam tests show a good
performance of 75.5%. All relevant data is shown on 6. Course average of 81%
is more then typical achievement of students, representing a good knowledge and
obtained skills for the course.
Fig. 3. Exam statistics in june sessions for
advanced computer systems course.
Fig. 4. Comparison of pass rate for advanced com-
puter systems course.
Only 29% and 37% of the students passed the course through colloquiums in
the academic years 2001/2002 and 2002/2003 correspondingly. However, since
there was no deadline on the seminar project it took some of the students almost a
year after the end of the semester to complete the course. The other reason for such
late submission of the seminar project is that it had little influence on the grading
(only half grade up or down). There was no grading scale of the seminar projects
given to the students in advance.
Six months after the end of the semester in the academic year 2002/2003, only
10 students had finished their seminar work. Their average course grade was 8.4.
Under similar circumstances in the academic 2001/2002 year only 12 students fin-
ished the course with average grade of 8.4. This comparison is presented on Figure
2. There is no significant difference in the average grade of the courses: (1) 8.7 for
“Advanced Computer Systems” and (2) 8.4 for the old “Microprocessors and mi-
crocomputers” course. However, there is significant improvement of the throughput
260
M. Guˇsev
Table 6. Average credits gained per assignment for advanced computer systems course.
Type of assignment 2002/2003 2003/2004
Homework (maximum 5 10) 46,7 49,2
Project 1 (maximum 20) 18,8 17,5
Project 2 (maximum 30) 24,4 25,2
Midterm and final (maximum 2 100) 150,2 151,0
Course average (maximum 300) 240,1 242,9
Course average (%) 80,0 81,4
of students that finish the course by the end of the semester. This is obvious if we
compare the results as presented on Figure 4.
6 Achieved Results for Internet programming Course
The Internet Programming course is placed in the fifth semester of Informatics
studies at University of Cyril and Methodius, Faculty of Science and Mathematics,
Institute of Informatics [1] in Skopje, Macedonia. This course started in 2002/2003
and it was offered to two generations, so the number of students that attended the
course was a lot more then the following 2003/2004 year.
In this course we implemented on-line accessible e-Learning course material,
covering all lectures and tutorials. All lecture slides, textbook and supporting ma-
terials, like animations and films were set on web. In addition to this, we set a lot
of exercise that helped students practice skills.
The homework assignments were realized by the on-line testing tool. A set of
1403 questions formed the question database.
Table 7 presents the pass rate of the students in June sessions. The pass rate
was increased and in both cases s very high, meaning that the students were highly
motivated to finish the course on time and that the system offered them a goof
learning and motivation platform. This is also presented on Figure 5.
Table 7. Pass rate for internet programming course in june sessions.
academic year enrolled passed %
2002/2003 142 129 90,8%
2003/2004 46 46 100,0%
The quantity of students that passed the exam was increased from 23.9% to
43.1% and reached 47.3% in final year. The rest of the students took the exam in
some later examination sessions. These numbers show great improvement analyz-
ing that it the pass rate reaches the rate of students that actually continues studying.
Improved Learning Methodology System
261
Fig. 5. Exam statistics in june sessions for Internet programming course.
Not only the quantity indicators were improved, but also the quality indicators
show this tendency. The course statistics are presented in Table 8. The course
average was 82,9% for June sessions in academic year 2002/2003 and improved to
88,6% one year later. These averages are higher then averages obtained by usual
students’ grades.
Table 8. Average credits gained per assignment for internet programming course.
Type of assignment 2002/2003 2003/2004
Homework (maximum 5 10) 45.0 48.8
Project 1 (maximum 20) 19,0 18,8
Project 2 (maximum 30) 25,7 27,9
Midterm and final (maximum 2 100) 159,0 170,3
Course average (maximum 300) 248,7 265,8
Course average (%) 82,9% 88,6%
One very obvious fact is that there was a high interest to complete the home-
work and the seminar projects in time, since all the students completed assign-
ments.
7 Conclusion
Better education is not gained only by improving the curricula only. Usually better
quality means improvement of teaching process and establishing new environments
for better education. This means usage of new ICT technologies, but also a great
move towards establishing content for learning materials, creation of simulators
that improve skills and creation of knowledge concepts. The grading system must
262
M. Guˇsev
also be reviewed and upgraded in order to facilitate new methods and continuous
learning.
We used quantity criteria based on pass rate to measure and compare conven-
tional system and new methodology. The quality indicator is the average obtained
in the first exam (June) session for the course. By analyzing these criteria we ob-
tained higher averages of grades and higher pass rates of the course.
The students accepted the new way of teaching and grading better than ex-
pected, which resulted in 38% more students passing an advanced computer sys-
tems course with more requirements than before and with comparable average
mark.
In the advanced computer systems course we introduced a lot of innovation in
homework assignments and projects realized with ILP simulators as tool to obtain
knowledge and specially to make further research on ILP processor behavior [6].
These tools helped them not just understand main concepts and topics, but deeply
get into computer architecture and analyze reasons for deadlocks and stalls due to
data dependencies in conditions of high parallelism on instruction level.
The methodology described in this paper aims to motivate the students to study
through the whole year, with active class participation. These innovations moti-
vated new curricula in current courses. Introducing indicators for development we
can observer that students benefit with more knowledge and skills obtained.
References
[1] Institute of Informatics. Accessed on 20.01.2005. [Online]. Available:
http://www.ii.edu.mk
[2] Tempus CD JEP 16160-2001. electronically. Accessed on 20.01.2. [Online].
Available: http://stella.ii.edu.mk
[3] J. Markovski and M. Guˇsev, “Parallel and pipelining compiler techniques for ILP
course,” in Proc. of the Workshops on Computer Science Education.Niˇs, Sebia and
Montenegro: Faculty of Electonic engineering, 2004, pp. 55–58.
[4] Lj. Antovski, J. Markovski, and M. Guˇsev, “Designing simple logic circuits,” in Pro-
ceedings of the Workshops on Computer Science Education.Niˇs, Serbia and Mon-
tenegro: Faculty of Electonic engineering, 2004, pp. 59–64.
[5] M. Guˇsev and G. Armenski, “E-learning realized by e-testing,” in CiiT 2001, Bitola,
Macedonia, Dec. 20–23, 2001, pp. 181–188.
[6] A. Miˇsev and M. Guˇsev, “Visual simulators for ILP dynamic OOO processor,” in Proc.
of the Workshop on Computer Architecture Education, ISCA 2004, Munich, 2004, pp.
87–92.
