Content uploaded by Lorenzo Salas-Morera
Author content
All content in this area was uploaded by Lorenzo Salas-Morera
Content may be subject to copyright.
Journal of Technology and Science Education
Journal of Technology and Science Education. Vol. 2(1), 2012, pp 39
On-line ISSN 2013-6374 - Print-ISSN 2014-5349 - DL: B-2000-2012 - http://dx.doi.org/10.3926/jotse.30
ANALYSIS OF ONLINE QUIZZES AS A TEACHING AND ASSESSMENT TOOL
Lorenzo Salas-Morera, Antonio Arauzo-Azofra, Laura García-Hernández
University of Córdoba, Escuela Politécnica Superior, Área de Proyectos de Ingeniería
Córdoba/Spain
lsalas@uco.es, arauzo@uco.es, ir1gahel@uco.es
Received November 2011
Accepted February 2012
Abstract
This article deals with the integrated use of online quizzes as a teaching and assessment tool in the general
program of the subject Proyectos in the third course of Ingeniero Técnico en Informática de Gestión over five
consecutive years. The research undertaken aimed to test quizzes effectiveness on student performance when
used, not only as an isolated assessment tool, but also when integrated into a combined strategy, which
support the overall programming of the subject. The results obtained during the five years of experimentation
using online quizzes shows that such quizzes have a proven positive influence on students' academic
performance. Furthermore, surveys conducted at the end of each course revealed the high value students
accord to use of online quizzes in course instruction
Keywords - Engineering education, educational technology, blended learning, online quizzes, e-assessment.
----------
1 INTRODUCTION
The actual implementation of the European Higher Education Area in Spain has led to deep changes in official
regulations and university course catalogues, and to the inclusion of interactive teaching and learning methods.
With the application of these methods, student responsibility in the learning process, and the need to acquire
both content knowledge as well as specific and generic professional-related skills, has increased notably
(Secretaría General de Universidades, 2009).
This paper presents the results of five consecutive years applying a combination of teaching strategies —
discussion forums, group tutoring, collaborative learning, peer review and online quizzes (Salas-Morera,
Cubero-Atienza, Redel-Macías, Arauzo-Azofra & García-Hernández, 2012) — and focuses on analyzing the
effectiveness of online quizzes. In general, the following secondary objectives must be achieved in order to
involve students in these teaching-learning methods (Salas-Morera, Berral-Yerón, Serrano-Gómez & Martínez-
Jiménez, 2009; Lan & Yan, 2009): a) students must be motivated to work regularly and to follow the correct
sequence of activities in the course programme; b) it is essential to monitor working time of students and
check that it is compatible with the number of credits assigned to the subject, c) the teacher should have
sufficient time to properly attend to all student requirements regarding forum, correcting papers, tutoring,
etc.; d) the students must receive quick feedback on each of the learning activities that they do, and e) the
students should interact proactively with each other and with the teacher.
In general, e-learning techniques involves a combination of teaching strategies (Lahwal, Amaimin & Al-Ajlan,
2009), such as assessment or self-assessment quizzes, discussion forums, and tutoring, that provide tools for
further and collaborative learning (Nevgi, Virtanen & Niemi, 2006). In this context, the teacher’s role is critical
(De Laat,& Lally, 2003), as the teacher must now move from being a mere transmitter of information (as in
traditional educational systems) to becoming a content facilitator, technical consultant, and social animator —a
more complex and laborious role (Jianhua & Yinjian, 2008). E-assessment has two key advantages: it
substantially reduces teacher work time on assessments, and allows for rapid teacher feedback on student
Journal of Technology and Science Education - http://dx.doi.org/10.3926/jotse.30
Vol. 2(1), 2012, pp 40
learning outcomes (Whitelock, 2009). Therefore, depending on the objectives to be met, e-assessment is
expected to be highly efficient.
Furthermore, the use of online evaluation by means of a series of online quizzes (each one with little influence
on the final evaluation) obtains better student academic results than using only standard exams as th e learning
assessment tool. Nevertheless, a set of precautions must be taken (Angus & Watson, 2009): a) online
assessment methods should be effective substitutes for traditional ones; b) students should be at least
indifferent to the use of these methods compared to traditional methods; and c) the online methods should
achieve better results than traditional ones. Thus, it is highly beneficial to the learning process to use periodic
quizzes as it compels students to work consistently and in accordance with other scheduled class activities
(Savander-Ranne, Lundén & Kolari, 2008).
With regard to implementing methodological changes required to adapt to the European Higher Education
Area, this paper proposes the following objectives:
Analyze the effectiveness of using online quizzes as a teaching and assessment tool
Test the influence of quizzes on student academic performance
Analyze workload that quizzes suppose for students, and quiz compatibility with the other planned
activities
Solicit student opinion regarding utility of questionnaires
2 METHODS
According to the ECTS system (Secretaría General de Universidades, 2009), the average student should be able
to pass all subjects scheduled in an academic year with a total of between 1500 and 1600 working hours (in
Spain) —equivalent to 25-30 hours per credit in a standard course of 60 credits. For this to be possible, it is
crucial to establish a teaching methodology that integrates three key elements: a) is designed so that students
acquire the knowledge and skills needed; b) student workload is consistent with subject objectives and credit
allocation; and c) there is adequate information regarding methods, assessment criteria, and a detailed
schedule of tasks.
Therefore, when scheduling activities, the first issue is to make sure students can complete the required tasks
in the total working hours assigned to the subject. To achieve this goal, the students completed a series of
weekly surveys to determine the appropriateness of the allocation of hours for each activity and its consistency
with the overall objectives of the subject. Specifically, the subject Proyectos is worth 4.5 ECTS credits, and thus
requires activities totalling between 112 and 135 hours of student work time. Proyectos deals with project
methodology, organization, and management. The subject aims to strengthen student abilities in: organizing
and planning; analysis and synthesis; applying knowledge in practice; troubleshooting; decision-making; oral
and written communication in a professional context; adapting to new situations; and managing technical
information. It is expected that upon completing this subject, students will be able to organize the process for
carrying out engineering projects, from managing technical documentation to implementing of solutions.
The Moodle platform was used as a tool to support teaching, specifically in the area of discussion forums and
periodic quizzes. The general structure of the subject is as follows: classroom lectures with the whole group;
practical classes in computer labs in groups of 25 students; participation of students in online forum; forum
activity feedback through group tutoring; periodic online quizzes online, and feedback on quizz results through
group tutorials. Overall student assessment is broken down in the following manner: Final examination in
theory and problems (40%); participation in forum (10%); computer classroom activities (12.5%); group
tutorials (12.5%); and quizzes (25%). The following sections of this article will focus exclusively on the
methodology used in online quizzes.
The main purpose of online quizzes in Moodle is to encourage students to study regularly, and at a pace that is
consistent with course requirements. The course is divided into three thematic blocks - TBI, TBII, and TBIII,
respectively - and a total of six quizzes, evenly spread out during the semester. The topics covered in each quiz
is as follows: 1) first half of TBI, 2) complete TBI block, 3) first half of TBII, 4) complete blocks TBI and TBII, 5)
first half of TBIII and 6) complete blocks TBI , TBII y TBIII. Online quiz scheduling is designed to encourage
students to work at the pace set by the teacher and in accordance with the other subject activities. The
expectation is that by working at the pace set by the teacher, and in line with course objectives, students will
display a more productive attitude in other subject activities. Each quiz is open for 4 days, and the total
Journal of Technology and Science Education - http://dx.doi.org/10.3926/jotse.30
Vol. 2(1), 2012, pp 41
estimated time for completion, including review of concepts, is 2 hours. There are several sets of questions for
each quiz, which are shared by all teachers in various subjects in the area of engineering projects. These sets of
questions vary in degree of difficulty, and are assigned randomly to students in different questionnaires. Table
1 shows some examples of questions related to project planning (there is at least one correct answer).
Question
Answers
Partial
credit
Answers
%
Correct
Activities
can be
Those that the project manager choose
-0.50
6/25
50.76%
The smallest units of work in which the project can be broken
down logically and it is possible to allocate resources and run-
time
0.17
25/25
The initial interviews of project definition
0.17
16/25
All administrative tasks
0.17
17/25
The wording of the document and binding
0.17
15/25
The final installation in the system of the client company
0.17
18/25
The meeting with the client company personnel to explain the
operation of the application
0.17
16/25
Those that are agreed between the project team and client
-0.50
5/25
Critical
paths
and
floats…
They have the highest average time
0.50
12/23
39.43%
All critical paths are the same length
-0.33
18/23
Delays in critical paths generate the delay of the entire project
0.50
20/23
In activities with positive independent float, in no case is
important to use all the free float
-0.33
3/23
Table 1. Some of the questions used in quizzes and statistical data provided by Moodle
After students have completed the two questionnaires for each block, the teacher gets from Moodle the
statistics of students᾽ answers, analyzes the results (see last three columns of Table 1), and identifies the main
weaknesses. This report contains the partial credit assigned to each answer, the number of students that chose
each option, and the overall mark obtained. A 2-hour tutorial session is then scheduled with the entire group.
During this session, concepts that produced the poorest quiz results are discussed with students in order to
reinforce student understanding of them. Given the incremental nature of quizz content, students are required
to periodically review concepts throughout the semester, which significantly improves students’ mastery of
subject.
In addition to weekly surveys regarding student workload for each of the planned activities (which serve as
feedback for activity scheduling for following academic year), other surveys regarding qualitative and
quantitative aspect of the subject are completed by students at the end of each course. Moreover, we analysed
the possible influence of questionnaires on the general student academic performance. The following section
lists and discusses the results obtained.
3 RESULTS
3.1 Students’ workload
After every week of the semester, a survey is programmed in Moodle about the time the students spent
working in each one of the activities of the subject: preparation and completion of questionnaires, forum
activity, studying, preparation of papers and reports, and tutoring assistance. Table 2 shows the average results
obtained over five years compared to the working time initially allocated. It shows how, in the students'
opinion, the overall amount of work performed substantially exceeds the assigned according to the course
credits (4.5 ECTS x 25-30 hours/credit = 112.5 to 135). In the new organization, 6 ECTS credits have been
allotted for the subject (150 hours of student work), which should remedy this situation.
Regarding the distribution of work over the semester (Figure 1), students spent a fairly uniform time in
preparation for papers and quizzes during the whole semester, while personal study substantially increased
when approaching the final exam. Although the forum is an assessable activity, it was the only activity in which
Journal of Technology and Science Education - http://dx.doi.org/10.3926/jotse.30
Vol. 2(1), 2012, pp 42
students worked less time than programmed over the years of study. In fact, although not in significant
number, some students completely renounced participating in the forum, despite the fact that debates are
meant to enrich the content of classes and tutoring sessions to foster more successful group work. Thus it is
clear that students, at least in part, are not prepared for collaborative learning that requires more active
participation than in traditional classes.
Weekly
average
S.D.
Total
Initially
allotted
Preparation and completion of quizzes
1.06
0.47
15.83
12
Forum
0.53
0.12
8.02
10
Studying
3.26
1.26
48.91
35
Papers
0.96
0.55
14.44
10
Individual tutoring
0.03
0.06
0.48
1
Classroom activities
4.00
0.00
60.00
60
TOTAL
9.85
2.02
147.68
128
Table 2. Average weekly workload of students in the five-years study.
3.2 Quizzes’ results
Periodic quizzes via Moodle are intended to capture students' attention regarding the most relevant concepts
of the subject and to link those concepts to other subject activities in order to increase participation and
effectiveness of group tutorials and practical classes. As shown in Table 2, the average time spent on preparing
and completing quizzes has been somewhat higher than projected in the teaching guide, which gives an idea of
the importance that students give to this activity, which accounts for 25% of the final marks. Table 3 shows the
overall results of the quizzes. The table shows irregular but consistently high participation and relatively high
marks.
Fig. 1. Average distribution of student work along the 15 weeks of the semester.
A χ2 test was taken to ascertain quizzes’ possible positive influence on students’ final marks. The hypothesis (of
the statistical test) was that students with marks of 6.5 or higher out of a maximum mark of 10, passed the
theoretical part of the subject, and that there is a statistically significant relationship between the two
variables. [χ2 (1, N = 336) = 3.19, p = 0.07].
Journal of Technology and Science Education - http://dx.doi.org/10.3926/jotse.30
Vol. 2(1), 2012, pp 43
3.3 Students’ opinions
Comprehensive surveys about the utility students attach to quizzes, or to participation in the forum activity,
have been carried out at the end of each year under study to provide feedback for teaching guides to be used
the following academic year. In response to the survey question regarding quizzes’ usefulness in keeping up
with subject content, students gave quizzes a rating between 4.06 and 4.20 out of 5. This, together with high
levels of participation and high average grades, shows that quizzes has been very helpful.
4 CONCLUSIONS
The integration of quizzes with other instructional activities in a teaching strategy has been very favourable.
Students highly appreciated the role of the quizzes that, with moderate effort, helped keep students up with
the subject, strengthened their involvement in other activities, and had a very positive impact on academic
outcomes.
On the other hand, it has been proven that there is a statistically significant relationship between quizzes and
other assessable activities, such as the theoretical part of the subject. Thus, although there are other activities
in the subject, such as the forum, tutorials and group practices, overall academic success of those who made an
average effort (more than 6.5 rating) on quizzes was very high.
Given the results of using quizzes, and their relationship to the mark for the theoretical section of the subject, it
is reasonable to consider eliminating the theoretical section and substituting it with quizzes. However, there is
no certainty regarding the usefulness of quizzes for evaluating certain competencies that must be reinforced,
and could perhaps be better assessed in an overall exam. On the other hand, quizzes have the added
advantage of guiding students through the learning process with moderate and continual effort. This
advantage would probably be lost if the quizzes had greater weight in the final mark. Likewise, increasing the
weight of quizzes in the final mark would have a significant repercussion on student study hours, which are
already very high in comparison with the number of study hours anticipated in the initial work plan.
2006-2007
2007-2008
86
1
2
3
4
5
6
64
1
2
3
4
5
6
N
80
81
77
75
74
69
N
56
62
50
53
55
59
MÁX
10
8.5
9.6
8.8
9.3
10
MÁX
8.3
10
9.6
9.3
9.3
9
MÍN
1
2.1
3
2.8
2.3
4
MÍN
2.2
2.6
1.7
3.3
3.8
2
MEDIAN
7.5
5.7
7.6
6
8.6
8
MEDIAN
6.3
6
7.2
6.1
8.6
7
MEAN
7.4
5.6
7.3
6.1
7.6
7.8
MEAN
6.1
5.9
6.8
6.1
7.9
7.1
PASSES
69
51
73
55
59
68
PASSES
45
45
42
41
48
56
S.D.
2.0
1.4
1.5
1.5
2.2
1.2
S.D.
1.5
1.6
1.7
1.4
1.8
1.6
2008-2009
2009-2010
78
1
2
3
4
5
6
70
1
2
3
4
5
6
N
62
60
59
63
63
63
N
65
64
59
54
56
53
MÁX
8.4
9.3
9.7
9.3
9.7
9
MÁX
8.4
9.6
8.8
9.8
9.6
9.9
MÍN
1
2.4
2.8
1.5
3.9
3
MÍN
2.2
0.8
2.7
0.1
4
4
MEDIAN
5.7
6.3
8
8
7.6
7
MEDIAN
5.7
5.9
6.5
4.7
8.2
8
MEAN
5.6
6.0
7.5
7.0
7.4
7.0
MEAN
5.5
5.7
6.5
4.8
7.8
7.6
PASSES
46
41
52
47
58
61
PASSES
39
41
51
21
50
48
S.D.
1.5
1.8
1.7
2.3
1.4
1.2
S.D.
1.4
2.1
1.3
2.4
1.5
1.6
2010-2011
38
1
2
3
4
5
6
N
31
31
29
33
23
29
MÁX
9.8
8.7
9.2
8.7
9.4
9.6
MÍN
3
4.1
4.0
4.7
6.6
4.5
MEDIAN
7
6.1
6.7
7.1
8.1
7.2
MEAN
6.9
6.1
6.6
7.0
8.1
7.1
PASSES
26
25
26
31
23
28
S.D.
1.8
1.1
1.3
0.9
0.8
1.2
Table 3. Quizzes results for the five-years study.
Journal of Technology and Science Education - http://dx.doi.org/10.3926/jotse.30
Vol. 2(1), 2012, pp 44
REFERENCES
Angus, S. D., & Watson, J. (2009). Does regular online testing enhance student learning in the numerical
sciences? Robust evidence from a large data set. British Journal of Educational Technology, 40(2), 255–272.
http://dx.doi.org/10.1111/j.1467-8535.2008.00916.x
De Laat, M., & Lally, V. (2003). Complexity, theory and praxis: Researching collaborative learning and tutoring
processes in a networked learning community. Instructional Science, 31, 7–39.
http://dx.doi.org/10.1023/A:1022596100142
Jianhua, Z., & Yinjian, J. (2008). A case study on collaborative e-tutoring. Proceedings of the 2008 IEEE
International Symposium on IT in Medicine and Education, 439-444.
Lahwal, F., Amaimin, M., & Al-Ajlan, A. (2009). Perception cultural impacts: Principles for trainer’s skills for e-
learning. Proceedings of the NCM 2009 - 5th International Joint Conference on INC, IMS, and IDC. Article
number 5331619, 986-993.
Lan, Y. F., & Yan, C. L. (2009). A practical approach to encourage students’ participation in asynchronous online
discussions based on expectancy theory. Proceedings of the International Conference on Virtual Environments,
Human-Computer Interfaces and Measurements Systems. Article number 5068907, 271-276.
Nevgi, A., Virtanen, P., & Niemi, H. (2006). Supporting students to develop collaborative learning skills in
technology-based environments. British Journal of Educational Technology, 37(6), 937-947.
http://dx.doi.org/10.1111/j.1467-8535.2006.00671.x
Salas-Morera, L., Berral- Yerón, J., Serrano-Gómez, I., & Martínez-Jiménez, P. (2009). An assessment of the ects
in software engineering: A teaching experience. Proceedings of the IEEE Transactions on Education, 52 (1), 177-
184.
Salas-Morera, L., Cubero-Atienza, A. J., Redel-Macías, M. D., Arauzo-Azofra, A., & García-Hernández, L. (2012).
Effective use of e-learning for improving students’ skills. In R. Babo, & A. Azevedo (Eds.), Higher Education
Institutions and Learning Management Systems: Adoption and Standardization (Chapter 14 pp. 292-314).
Information Science Reference (IGI Global).
Savander-Ranne, C., Lundén, O., & Kolari, S. (2008). An alternative teaching method for electrical engineering
courses. Proceedings of the IEEE Transactions on Education, 51(4), 423-431.
http://dx.doi.org/10.1109/TE.2007.912500
Secretaría General de Universidades. (2009). Guía del usuario del ECTS. Retrieved from
http://ec.europa.eu/education/lifelong-learning-policy/doc/ects/guide_es.pdf
Whitelock, D. (2009). Editorial: E-assessment: Developing new dialogues for the digital age. British Journal of
Educational Technology, 40(2), 199–202. http://dx.doi.org/10.1111/j.1467-8535.2008.00932.x
Citation: Salas-Morera, L., Arauzo-Azofra, A., & García-Hernández, L. (2012). Analysis of online quizzes as a
teaching and assessment tool. Journal of Technology and Science Education (JOTSE), 2(1), 39-45.
http://www.dx.doi.org/10.3926/jotse.30
On-line ISSN: 2013-6374 - Print ISSN: 2014-5349 - DL: B-2000-2012
Journal of Technology and Science Education - http://dx.doi.org/10.3926/jotse.30
Vol. 2(1), 2012, pp 45
AUTHOR BIOGRAPHY
Lorenzo Salas-Morera
He is a professor of University of Córdoba (Spain). His primary areas of research are intelligent user adaptive
systems for automated facility layout design, projects management and educational technology. Since 2002, he
has led several projects for the adaptation of teaching methodologies to the European Higher Education Area
and, in particular, to the ECTS. He received the University of Córdoba Social Council award of Educational
Innovation in 2008.
Antonio Arauzo-Azofra
He is currently assistant professor at the University of Córdoba (Spain). His primary areas of research are
feature selection applied in classification problems for intelligent data mining in information systems and
intelligent user adaptive systems for automated facility layout design. He is in active collaboration on several
projects for the improvement of teaching methodologies adapted to the European Higher Education Area.
Laura García-Hernández
She is currently assistant professor at the University of Córdoba (Spain). Her research interests are in the fields
of Facility Layout Design, Interactive Algorithms, Evolutionary Computation, Project Management and
Educational Technology. She is in active collaboration on projects for the adaptation of teaching methodologies
to the European Higher Education Area.
Published by OmniaScience (www.omniascience.com)
Journal of Technology and Science Education, 2012 (www.jotse.org)
Article's contents are provided on a Attribution-Non Commercial 3.0 Creative commons license. Readers are allowed to copy, distribute
and communicate article's contents, provided the author's and Intangible Capital journal's names are included. It must not be used for
commercial purposes. To see the complete licence contents, please visit http://creativecommons.org/licenses/by-nc/3.0/es/
