Content uploaded by Adi Katz
Author content
All content in this area was uploaded by Adi Katz on Jun 26, 2021
Content may be subject to copyright.
Research Article https://doi.org/10.12973/eu-jer.10.3.1137
European Journal of Educational Research
Volume 10, Issue 3, 1137 - 1156.
ISSN: 2165-8714
https://www.eu-jer.com/
Learning Supported by Technology: Effectiveness with Educational
Software
Sharon Tzur
Ariel University, ISRAEL
Adi Katz
Shamoon College of Engineering, ISRAEL
Nitza Davidovich*
Ariel University, ISRAEL
Received: December 16, 2020 ▪ Revised: March 3, 2021 ▪ Accepted: May 4, 2021
Abstract: This study addresses the effectiveness of learning via educational software. Recent decades have seen the integration of
technologies, which are changing teaching and transforming teachers into mediating, facilitating, and guiding figures by means of
digital learning methods that serve as a major tool in schools, colleges, and universities. The current study focuses on instruction
provided within the Israeli Air Force and examined the effectiveness of instruction provided via educational software in terms of
learning products: Bloom’s revised taxonomy, Te’eni’s affective-cognitive model of organizational communication and the STEM
model. We randomly divided the learners into three groups who studied the same topic: one group studied with the educational
software only, the second with the educational software together with an instructor, and the third with an instructor who used a
presentation. The learners took a test and four months later they took another test to examine the effectiveness of the instruction
over time. The research results show that the recall levels and performance levels on the tests were almost identical in all groups,
but in the categories of understanding and applying the addition of an instructor strongly contributed to achievements: Those who
received instruction via educational software achieved the best results in the understanding category, while those who studied with
an instructor who used a presentation achieved the best results on the test with regard to application of the studied material. The
findings of this study can illuminate the effectiveness of using educational software in learning processes in all educational systems.
Keywords: Learning products, educational software, technological training, cognitive categories.
To cite this article: Tzur, S., Katz, A., & Davidovich, N. (2021). Learning supported by technology: Effectiveness with educational
software. European Journal of Educational Research, 10(3), 1137-1156. https://doi.org/10.12973/eu-jer.10.3.1139
Introduction
In recent decades, gradually more technologies are emerging that are changing the manner of teaching in class and
transforming teachers into mediators (Nachmias & Mioduser, 2001; Waldman, 2007). Digital teaching tools are
gradually being integrated as major work tools to a large extent in the various educational systems and are being
utilized in schools, colleges, and universities, as well as in military instruction (Nachmias & Mioduser, 2001). The online
learning environment has unique aspects and emphases that relate to students’ pedagogical and personal conduct. This
learning is important particularly for the Z-generation, born beginning from 2000 directly into technological progress,
as a major part of their lives is conducted on social networks (such as Twitter, Instagram, WhatsApp, and Snapchat).
Hence, using an online environment is completely natural for this generation that is almost unfamiliar with a reality
that does not include technologically generated conduct. Therefore, it is only natural that learning activities as well take
place in learners’ natural environment, i.e., in an online environment (Rotem & Peled, 2008).
In the current study, we focus on examining the effectiveness of technology enhanced instruction, in terms of learning
outcomes (students’ achievements), the ability to preserve knowledge over time, and learners’ preceptions.
Literature Review
There are two main types of technology enhanced instruction: technology-based training and computer-assisted
instruction. Technology-based training mainly utilizes the internet, with an emphasis on social networks. This
technique is fast, readily available, and relatively cheap, since the training resources are on the web and accessible to
students everywhere and anytime. Hence, this method generates relatively high commitment by learners to the process
and is manifested in a high level of learning and performance (Cotton, 2008; Kruse & Keil, 2000). Computer-assisted
*
Corresponding author:
Nitza Davidovich, Ariel University, Faculty of Social Sciences and Humanities, Israel. d.nitza@ariel.ac.il
© 2021 The Author(s). Open Access - This article is under the CC BY license (https://creativecommons.org/licenses/by/4.0/).
1140 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
instruction is instruction based on the computer rather than on internet resources. This type of instruction is more
limited as it requires learners to physically sit at a computer and watch, for instance videoclips that are on the
computer; educational software is one of these methods (Schmeeckle, 2003).
The topic of learning via technology can be divided into several concepts: “Computerized learning” means using
computerized means for learning purposes. “Digital learning” are all pedagogical means that utilize information and
communication technology to enhance learning. “Technology-enhanced learning” is learning that takes place by means
of a computerized tool connected to internet or intranet media. This learning is usually carried out with a separation in
place and time between the learner and the teacher and it enables learning flexibility and the ability to reinforce
learning processes by new opportunities that technology offers learners (Beetham & Sharpe, 2013).
Learning in an online environment has many benefits. It offers learners varied interactive means of learning through
which they can advance their learning process independently (Ben-Zadok et al., 2009). This independence includes,
among other things, decisions concerning the extent and order in which the contents are consumed, the time and pace
of learning, and so on. In such an environment, learners develop the abilities of critical thinking, control, and self-
monitoring. They are able to identify the value of freedom imparted by self-learning, to become more independent in
their learning process, and the learning can become an important goal for life (Rappel, 2017). One important area in
this context is individually adapted learning, which can become more precise thanks to technology. The various ways in
which learners choose their learning process indicates the difference between learners and thus the significance of
individually adapted education for maximally realizing learners’ abilities and for reaching better achievements (Hwang
et al., 2018).
Various studies have shown the ability of technology-enhanced learning systems to improve learning processes and
learners’ achievements (Joubert, 2013; Laborde & Straesser, 2010). The main purpose of online learning and practice
environments is to help and advance learners’ understanding of a certain topic or achievement of a certain educational
goal, in a learner-centered experience (Cicognani, 2000). The online learning environment is an important way of
assimilating innovative elements in learning processes, with an emphasis on inquiry and discovery processes that are
manifested in much practice (Ross et al., 2016).
Educational software is also called courseware or WBT – web-based training, meaning digital courses/instruction.
Educational software is a collection of digital pages packaged as a learning unit, which include texts, audio, video,
pictures, as well as interactions that allow learners to answer questions or perform assignments.
The rapid development of computer technology in the resent years also included a flourish of wide range of educational
software to support teaching and learning, and the amount of these software continually grows. Educational software
development draws on a variety of instructional approaches and different types of software can be used to address a
range of educational goals. Different orientations to teaching and learning have led the development of a wide spectrum
of educational software packages (Stoddart & Niederhauser, 1993). Educational software have the opportunity to
integrate multimedia and interaction both for students and teachers (Coomans & Lacerda, 2015). On the one hand,
educational software can play an important role in enhancing the education, but on the other hand, not all educational
software is suitable and productive for teaching and learning (Lê, & Lê, 2007). Teachers need to decide whether an
intended educational software can improve their teaching and how; learners need to know how the use of a specific
software might impact the learning experience. It has been found that teachers' beliefs shape the implementation of
education software in the context of school reform initiatives. They tend to select and integrate educational technology
in ways that are consistent with their personal pedagogical perspectives about instructional practice (Niederhauser &
Stoddart, 2001). Lê, & Lê (2007) categorize four different perspectives of software in the context of education: software
as tools, as instructors, as facilitators of learning, and as virtual classes. As a tool, educational software is used for
performing certain functions such as calculating, drawing and editing. As an instructor, a software’s primary role is to
teach learners to develop new knowledge and skills. Educational software as facilitators of learning follow the
constructivist model and plays less focus on instruction and more on the active role of learners in the learning process,
putting emphasize on the learning experiences and the kind of communicative interaction between learners and
teachers. Virtual classes are educational software packages designed for a targeted group of learners such as a web-
based academic course and a multimedia-based training program.
Many researchers pointed to the importance of measuring and evaluating the quality of educational software. There are
different methods of evaluating educational software, and different researchers hold different approaches to doing so.
Many authors have created evaluation checklists, each emphasizing different aspects. Coomans and Lacerda (2015)
combine pedagogical and technical aspects, taking into consideration both cognitive and usability aspects, respectively.
Squires and Preece (1999) propose an approach that adapts the idea of usability heuristics by taking account of a socio-
constructivist learning perspective. Draper (1996) and Kennedy and McNaught (1997) have indicated that a formative,
iterative design process of educational software, which involves real representative students, produces more useable
and effective products. They argue that it is important to evaluate the software while it is being developed, by testing it
on learners and modifying accordingly. Babiker and Elmagzoub (2015) concluded that educator must create their own
applications if they really want to make use of them as an effective tool in education.
European Journal of Educational Research 1141
Studies on the effectiveness of technology enhanced learning
The purpose of the current study is to examine the effectiveness of technology enhanced instruction. The literature
includes diverse studies that examined this topic. For instance, Schmeeckle (2003) investigated technology enhanced
learning in order to provide a comprehensive evaluation of the instruction’s effectiveness as well as the differences in
motivation to learn when using the two types of instruction – technology enhanced and frontal. His study, which
compared instruction combining educational software and multimedia, and traditional instruction in class, found no
significant difference between technology enhanced and frontal instruction with regard to learning effectiveness.
Weiser et al. (2016) conducted a comparison between frontal learning in class and learning in two online environments
– one-way video (only the instructor sees the students) and two-way video (the instructor and the students see each
other). Students’ level of participation was found to predict their success in many cases and therefore students’
participation in learning is critical. This participation is usually obtained by frontal instruction, where the instructor
arouses interest, asks questions, and brings practical examples from daily life. Weiser found no significant advantage of
frontal learning and active participation is mainly affected by the lecturer’s manner of instruction. Rappel (2017) found
that online learning based on social aspects of learning facilitates collaborative learning affected by learning peers.
Another study compared between a traditional learning method and technology enhanced learning and found that the
most effective way of learning is a combination of the traditional learning method with technology enhanced learning
that reinforces the achievements of traditional learning (Ma et al., 2015).
Examining the effectiveness of technology enhanced learning in the current study
The current study focuses on learning via educational software, with the purpose of exploring the effectiveness of this
tool versus frontal instruction. Three conceptual frameworks guided our exploration of the effectiveness of educational
software. With regard to the learning achievements, we followed Bloom’s revised taxonomy, and the STEM (science,
mathematics, technology, engineering) theory. In terms of the ability to support communicational strategies, we used
the affective-cognitive model of organizational communication (Te’eni, 2001). In addition, we explored the students’
ability to remember (retrieve knowledge from long-term memory over time) in the medium term (up to 4 months).
Bloom’s (1956, as cited in Melamed, 2013) taxonomy is a tool for categorizing learning tasks, constructed of six
hierarchical cognitive levels ranging from simple to complex, such that achieving a high cognitive level requires
obtaining knowledge on the previous levels. The taxonomy is beneficial for constructing and evaluating curricula,
preparing lesson plans, and planning tests to examine achievements. We present the cognitive process dimensions of
Bloom’s revised taxanomy (Krathwohl, 2002):
Remember – A basic level that includes the ability to retrieve relevant knowledge from long-term memory.
Understand – A higher level in which a basic ability to determine the meaning of instructional messages, including
oral, written, and graphic communication.
Apply - Where the learner is also required to provide solutions to new situations, and carry out or use a procedure
in a given situation.
Analyze – The ability to break material into its constituent parts and detect how the parts relate to one another and
to an overall structure or purpose.
Evaluate – The ability to conduct a quantitative or qualitative judgment of some product and to justify the criteria
used to reach this judgment. The product can be, for instance, ideas, actions, or processes.
Create - The ability to put elements together to form a novel, coherent whole or make an original product.
Bloom’s taxonomy can also be used to evaluate how technology affects the learning products of technology enhanced
learning processes. In the current study, we use Bloom’s taxonomy to examine the learning products of technology
enhanced instruction versus frontal instruction. In the present study we have not covered all Bloom’s categories, for the
practical reason of the study schedule, and chose to focus on the first three categories – remember, understand, and
apply.
Learning products in the STEM (science, mathematics, technology, engineering) subjects can be analyzed, specifically
exploring what learners can do after learning versus what they could before. The analysis is performed by defining
general teaching goals, in terms of what the learner will know and will be capable of doing by the end of the lesson. The
analysis allows definition of individual teaching goals that must be easy to evaluate and that focus on the final outcome
and executive terms. These goals include the ability to implement knowledge, plan assignments, analyze and interpret
data, plan a system with consideration of constraints, formulate and solve problems, and more (Davidovich & Shiler,
2016).
Te’eni’s affective-cognitive model of organizational communication (2001) describes the many different levels included
in interpersonal communication. The model stresses behavior that is motivated by a communication goal and relates to
the complexities characteristic of communication: cognitive, emotional, and dynamic complexity. In order for the
communication process to be efficient and beneficial, it is necessary to adapt the communication process to its level of
1142 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
complexity. For this purpose, communicators use different communication strategies according to the purpose of the
communication and according to the media that they use to communicate. Communication strategies are relevant to the
process of learning where content is conveyed by messages from a frontal teacher or through digital technology. On the
other side of the media, messages can come from the learner (such as questions about the content being taught or
requesting additional explanation due to misunderstanding). The following is a list of communication strategies.
Contextualization: It is important to add the context to the message conveyed in order to generate understanding
by recipients. Senders who use this strategy add different layers of information and interpretation to the core
message (main message), which explain among other things: how an action can be performed; how it can be
divided into parts; what is the motivation for action; what alternative interpretations exist, why action is needed,
and so on. Contextualization is essential for understanding, as well as for improving functioning and solving
problems. In the learning domain, it is very important to understand the message learned. Adding an illustration
or visual demonstration of the material conveyed is an example of the contextualization strategy.
Affectivity: In an act of communication it is important to include in the message components that describe feelings
and sensations. Affectivity can be used to motivate, to preserve desirable feelings and inclinations. This quality
should be utilized when dealing with emotional complexity. When dealing with learning it is important to create in
learners' positive feelings of self-satisfaction, self-efficacy, pleasure, interest, and motivation.
Control through testing and control through planning: Control is a matter of supervising and adjusting the
communication process in order to ensure efficient communication. There is a need for clear discourse and for
managing dominancy and control, and there is a continuous need to coordinate between control and process.
When communication takes place face to face, the feedback received from the recipient is immediate, in contrast to
learning via educational software, where there is no immediate possibility of receiving feedback. Feedback allows
the message sender to understand how the message was interpreted, both in cognitive terms (was the message
understood?) and in affective terms (did the message generate the correct feelings in order to fit the purpose of
the communication?). Accordingly, the message sender will take action. In a face-to-face encounter of an instructor
and learners, the instructor can understand immediately whether the message conveyed was understood or if
additional explanations are necessary.
Perspective taking: As part of a communication process, a possible query is whether the perspective and view of
the message recipient are part of the communication’s purpose or do they remain outside the range of
communication. This strategy, where the sender relates actively to (imagines) the perspective of the message
recipient, is essential in order for communication to be comprehensible. This strategy includes personal speech,
asking questions about matters and approaches related to the recipient, and supporting them. In a learning
situation, it is important that the instructor or teacher take into consideration, for instance, students’ background
prior to the learning, their level of motivation to learn, and so on.
Attention focusing: In this strategy the message conveyor tries to affect the processing of the message by the
recipient by stressing certain elements to focus unique attention. This strategy includes use of different techniques
to affect the processing of information, such as changing the font size and shape of a textual message or using vocal
intonation in a spoken message.
Methodology
Research Goal
The purpose of the study was to examine the effectiveness of technology enhanced instruction versus frontal
instruction in the short range (immediately after the instruction) and in the medium range (up to four months). The
timeframe of four months was set since the Air Force schedules instruction programs on the same topic at four-month
intervals. Effectiveness was determined by tests that examine the first three cognitive process dimensions of Bloom’s
revised taxonomy and based on questionnaires examining the effectiveness of the instruction and the motivation of the
trainees according to the STEM model (Davidovich & Shiler, 2016) and the affective-cognitive model of organizational
communication (Te’eni, 2001).
We tested the differences between the groups via a Mann-Whitney U-test, a non-parametric method appropriate for the
perceptions variables that we measured by Likert scales which are ordinal-level (Jamieson, 2004; Kuzon et al., 1996)
and for relatively small sample sizes (Norman, 2010). In addition to testing the reliability of the questionnaires was
performed by Cronbach’s alpha test after no reverse questions were found, the Friedman test was proposed to test the
differences between the test results and those tested at different times (paired variables).
Sample and research context – the Israeli Air Force
The study was conducted in the Air Force and its aim was to examine the effectiveness of technology enhanced
instruction over time and in three different cognitive process dimensions. The Israeli Air Force has been using
educational software for about 20 years through a special department in charge of producing such software for the
technical system in the corps, i.e., the Department for Research and Development of Instruction. The incentive for
continued utilization of educational software is to spare personnel and to provide technicians with detailed
illustrations of the aircraft systems. Educational software in the Air Force is characterized by high-level technological
European Journal of Educational Research 1143
architecture but does not generate communication, emotions, or the ability to control the instruction process. In
addition, the software is not updated regularly although the technological systems contain endless changes and
developments. Educational software saves on instruction hours and instructors as technicians can study on their own
whenever convenient for them. In this study we examined the efficacy of technology enhanced instruction via
educational software versus frontal instruction by professional experienced instructors.
Thousands of technicians serve in the Air Force in a wide variety of functions responsible for maintaining the aircraft
and the different systems. Periodic instruction is scheduled by the unit commander in charge of specific systems, in
coordination with the vocational branch in Air Force headquarters. Instruction sessions cover about 40 hours over six
months and are divided as follows:
Mandatory frontal instruction on generic and general subjects – literature, tools, foreign objects, and safety.
Practical instruction concerning the aircraft or specific systems, provided by an instructor with no aides
(presentations/educational software).
Frontal instruction on professional subjects – instruction sessions in the classroom given by an instructor, usually
via a presentation.
Instruction via educational software – technology enhanced instruction, where the technicians learn from
educational software on a computer at their convenience, and then take a multi-choice test unaccompanied or
mediated by an instructor.
The study consisted of 84 Air Force technicians. The group of technicians was in the 18-22 age range. The technicians
were a quality group who completed an aircraft technician course.
Measures of effectiveness and learning products
In this study we examined the effectiveness of technology enhanced instruction by two different theories that deal with
learning products and output. One is Bloom’s revised taxonomy for examining the effectiveness of learning in different
categories: remember, understand, and apply. The second is teaching goals as measured by the STEM model. In
addition, we examined the effectiveness of learning through the lens of the affective-cognitive model of organizational
communication (Te’eni, 2001) in order to analyze the effect of technology enhanced instruction via educational
software on additional perceptions by the learners.
Learning effectiveness in terms of Bloom’s taxonomy and STEM model
As aforementioned, we used Bloom’s taxonomy to examine the learning products of technology enhanced instruction
versus frontal instruction by the three categories – remember, undrestand, and apply. To test the learning products, a
STEM model was used, which examines what learners can do after learning versus what they could have done before it
at the end of the lesson.
These include the ability to implement knowledge, plan assignments, analyze and interpret data, plan a system with
consideration of constraints, formulate and solve problems, and more (Davidovich & Shiler, 2016).
Learning effectiveness in terms of the affective-cognitive model of organizational communication
Te’eni’s affective-cognitive model claims that in order for a communication process to be efficient, it is necessary to
adapt the communication process to its level of complexity. Communicators use different communication strategies
according to the purpose of the communication and according to the media that they use to communicate. In the
process of learning, different media support the use of communication strategies differently. In our case, the learning
content is conveyed by a frontal teacher or through learning software. We examined the perceived ability of the
learning media to support the Te’eni’s communication strategies: contextualiztion, affectivity, control testing, control
planning, perspective taking, and attention focusing.
Research hypotheses
On the remember dimension, there will be no change in the effectiveness of learning between instruction via
educational software and instruction provided by an instructor (either with educational software or with a
presentation) according to Bloom’s revised taxonomy, as noted by Schmeeckle (2003) who found no essential
differences in the effectiveness of learning between frontal and technology enhanced learning.
Frontal instruction combined with educational software will be more effective on the dimensions of understand and
apply according to Bloom’s revised taxonomy, since these topics of the instruction also contain ambiguous and non-
routine messages (Daft & Lengel, 1983). In addition, according to the Media Naturalness Theory, face-to-face
communication in frontal instruction is preferable to technology enhanced instruction (Weiser et al., 2016). Studies
have found that the most effective learning is a combination of the traditional learning method and technology
enhanced learning (Ma et al., 2015).
1144 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
Frontal instruction will be more effective according to the STEM model (Davidovich & Shiler, 2016), since face-to-
face communication with instructors leads to a higher level of media wealth and of the learning experience (Chang
et al., 2017). In addition, the most effective learning is a combination of the traditional learning method and
technology enhanced learning (Ma et al., 2015).
Frontal instruction will be more effective according to the affective-cognitive model of organizational
communication (Te’eni, 2001), since it is characterized by two-way communication and by a greater degree of
participation (Weiser et al., 2016). In addition, according to Te’eni’s model one of the main conditions for the
existence of effective communication is real action in which the message recipient can share the knowledge of the
message sender, and this is best facilitated by frontal instruction. According to Te’eni (2001), emotions are a central
component of motivation for learning, and frontal instruction with an instructor facilitates expressing emotions to
raise motivation and to support the learning process, any miscomprehension that might arise, and more. The
control of frontal learning is better as it is possible to receive immediate feedback during the instruction (Te’eni,
2001). Frontal instruction also enables better perspective taking, namely, allows adapting the instruction to the
learner’s perspective and views. This strategy can be enhanced by frequent checking of learners’ comprehension,
asking questions, and speaking to them directly (Te’eni, 2001).
The research procedure and the experimental manipulation
The research participants were, as stated, 84 Air Force technicians. Since our main goal was to compare the
effectiveness of different learning conditions, we designed an experiment in which we were able to detect causal
relationships. We applyed a between-subjects (or between-groups) study design: different participants test each
condition, so that each participant was only exposed to a single learning treatment. In other words, our independent
variable was the learning condition, which we directly manipulated. The research’s dependent variables that we
expected to vary as a result of the independent-variable manipulation are various learning effectiveness measures.
Experimantal manipulation: We divided the participants randomly into three groups of 28 technician each. Each group
of technician learned the topic of safety in F16 aircraft in one of the three methods:
Educational software – unguided learning in the computer room.
Educational software with guided learning (frontally, by an instructor).
Frontal guided learning (by an instructor, with the assistance of a presentation).
The experimental manipulation related to the learning method only. The same educational software was used by
groups 1-2, and the same instructor guided groups 2-3. The experiment was held for the three groups simultaneously in
order to prevent information leaks between the groups. The instruction was carried out at the technician’ military base
by a regular instructor at the base in the existing classrooms with which the technicians were familiar. This enhanced
the internal validity as we reduced the chance that any external variable in the respondents’ environment, to which
they were unaccustomed, could have disrupted the experimental process.
Dependent variables: Learning effectiveness was measured identically in the three groups by tests and questionnaires
that examined the first three stages of Bloom’s revised taxonomy, based on questionnaires that examined the
effectiveness of the instruction and the motivation of the trainees by the STEM model (Davidovich & Shiler, 2016) and
by the ability to apply communication strategies described in the affective-cognitive model of organizational
communication (Te’eni, 2001).
Tools for measuring learning effectiveness
Bloom’s revised taxonomy – The three groups were given identical tests at the end of the instruction. Each test
contained 30 questions on the following dimensions according to Bloom’s revised taxonomy: 10 questions on the
remember dimension (such as: What is the pressure in the system?), 10 questions on the understand dimension
(such as: What is the purpose of the system?), and 10 questions on the apply dimension (such as: What would be the
reaction if an event were to occur?). Approximately four months later the technicians were given another test on
these three dimensions.
Examining learning products according to the STEM model – The three groups were given an identical
questionnaire at the end of the instruction, comprised of one question for each of the 11 learning products defined
according to the model, on a Likert scale of 6 points (1 – low, 6 – high). The questionnaire included questions about
the learning experience and the technicians' feelings about the effectiveness of the learning, the extent to which the
training helped the technicians impart knowledge, ability to interpret data, troubleshooting and a broad
understanding of the system. Reliability of the questionnaire was measured using the Alpha Cronbach statistic, and
was found to be satisfactory: α=0.80.
Examining strategies according to the affective-cognitive model of organizational communication (Te’eni, 2001) –
The three groups were given an identical questionnaire at the end of the instruction, with several questions for each
strategy, on a Likert scale of 6 points (1 – low, 6 – high). Statements 1-4 measured learners’ perception of the
learning method’s contextualization capacity. Statements 5-7 measured the method’s perceived ability to insert
European Journal of Educational Research 1145
emotions in the learning process. Statements 8-11 measured the teacher’s perceived ability to control the learning
process and to perform adjustments accordingly and when necessary. Statements 12-16 measured the perceived
ability of the learning method to take the learner’s perspective. Statements 17-19 examined the perceived ability of
the learning method to focus the learners’ attention on that which is significant at any given moment.
Cronbach’s alpha statistic for the first four statements was α=0.56, α=0.85 for statements
5-7, α=0.87 for statements 8-11, α=0.83 for statements 12-16, and α=0.46 for statements 17-19. Reliability of the
whole questionnaire was α=0.93. Figure 1 presents the between-subjects experimental design, the three learning
effectiveness measures, and the time each measure was applied.
Figure 1 – Experimental design
Findings
The purpose of the study was to examine the effectiveness of technology enhanced instruction versus frontal
instruction in the short term (immediately after the instruction) and in the medium term (up to four months). The
effectiveness was examined by tests on three dimensions according to the first phases of Bloom’s revised taxonomy and
based on two questionnaires that examined the instruction’s effectiveness and the motivation of the trainees according
to the STEM model (Davidovich & Shiler, 2016) and the affective-cognitive model of organizational communication
(Te’eni, 2001). We shall present the results of the tests and of the questionnaires.
Results of phase 1 tests according to Bloom’s taxonomy
Figure 2 describes the general distribution of scores for all examinees on all cognitive process dimensions.
Figure 2 – General distribution of scores on the cognitive process dimension test
It is evident from Figure 2 that the scores on the test questions using the education software alone are low compared to
teaching mediated educational software with guided learning and frontal guided learning. Figure 3 describes the
distribution of scores by the three dimensions in Bloom’s taxonomy.
6.63 7.34 7.42
0
2
4
6
8
10
Educational software alone Educational software mediated
by instructor
Instructor who teaches
frontally with a presentation
General distribution of scores
0.47 =
σ
0.40 = σ
0.44 = σ
1146 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
Figure 3 – Summary of test results by the dimensions in Bloom’s revised taxonomy
It is evident from Figure 3 that the scores on the test questions concerning the remember dimension are almost
identical in the three methods. Concerning the understand dimension there is a clear advantage to including an
instructor with the educational software. In contrast, on the apply dimension level there seems to be an advantage to
frontal learning with an instructor who uses a presentation.
In order to examine the significance of the differences between the three groups, a Mann-Whitney test was conducted,
appropriate for relatively small samples where the total number of observations is less than 30. The level of
significance for examining the various differences was 5%. Table 1 presents the results of the analysis. It is evident that
use of educational software with an instructor gave a better result (mean=7.34) compared to educational software
alone (mean=6.63), U=75.50, p<.001. Moreover, an instructor teaching with a presentation was found to generate
better results (mean=7.42) than educational software alone (mean=6.63), U=64.00, p<.001. No significant difference
was found between educational software with an instructor (mean=7.34) and an instructor who uses a presentation
(mean=7.42), U=371.00, p=.357.
Table 1 – Comparison of test scores by the Mann Whitney Test
Educational
software alone
[1]
N=27
Educational software
mediated by
instructor [2]
N=28
Instructor who teaches
frontally with a
presentation [3]
N=28
U
2-3
U
1-3
U
1-2
Effect
size
1-3
Effect
size
1-2
Mean
(SD)
Mean
ranking
Mean
(SD)
Mean
ranking
Mean
(SD)
Mean
ranking
6.63
(0.4)
16.8
7.34
(0.44)
38.80
7.42
(0.47)
39.21
371.0
(P =
0.065)
64.0
(P =
0.00)
75.50
(P = 0.00)
0.43
0.47
Results for statements in the questionnaire according to the affective-cognitive model of organizational communication
(Te’eni, 2001)
Figure 4 describes the distribution of scores on questionnaires constructed by Te’eni’s model, which describes the
many different layers of interpersonal communication and emphasizes the need to use communication strategies in
order to achieve the goal of the communication.
7.3
6.52 6.07
7.32 7.75
6.96
7.36 7.18 7.71
0
1
2
3
4
5
6
7
8
9
Summary of tests by Remember dimensions
Educational software alone
Educational software mediated by instructor
Instructor who teaches frontally with a presentation
0.42 = σ
0.47 = σ
0.45 = σ
0.43 = σ
0.45 = σ
0.47 = σ
0.45 = σ
0.48 = σ
0.49 = σ
European Journal of Educational Research 1147
Figure 4 – Scores on questionnaires that examined the ability to use communication strategies according to Te’eni
It is evident from Figure 4 that the ability to use efficient and beneficial strategies is low when using educational
software alone, and that the overall score when an instructor is present, whether mediating when studying with
educational software or teaching frontally with a presentation, is almost identical and significantly different than when
studying with educational software alone.
Results of the questionnaires according to the STEM model
As stated, learning products must be clear and easy to evaluate and not vague. It is also necessary to measure and focus
on the final outcome in terms of performance. The STEM model examines learning products by 11 criteria (Davidovich
& Shiler, 2016) measured by questionnaires. Figure 5 shows the scores for the questionnaires constructed according to
the STEM model.
Figure 5 – Means of questionnaire scores according to the STEM model
It is evident from Figure 5 that the scores for the questionnaires of the educational software group were low, and that
the scores in the group of the instructor teaching with a presentation were the highest.
3.06
4.4 4.37
0
1
2
3
4
5
Educational software alone Educational software mediated
by instructor
Instructor who teaches frontally
with a presentation
Scores on communication strategy questionnaires according to Te'eni
2.95
3.47 3.79
0
1
2
3
4
5
Educational software alone Educational software mediated by
instructor
Instructor who teaches frontally
with a presentation
Mean scores for STEM questionnaires
0.77 =σ
0.36 = σ
0.35 =σ
0.54 =σ
0.35 =σ
0.24 =σ
1148 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
Test results Stage II
Figure 6 presents the distribution of the scores by Bloom’s three dimensions:
Figure 6 - Summary of test results by Bloom’s taxonomy dimensions in Stage II
It is evident from Figure 7 that the scores on questions in the remember dimension were almost identical in the three
groups. In the understand dimension the advantage of combining an instructor with the educational software was still
maintained. In contrast, in the apply dimension the advantage found for the group who studied with an instructor
teaching frontally with a presentation was maintained.
Table 2 presents a comparison of the differences in the test scores in Stage II. It is evident that educational software
mediated by an instructor produced better results (mean= 6.49) than software alone (mean=5.25), U=74.50, p<.001.
Moreover, the group that learned with an instructor teaching frontally with a presentation produced better results
(mean=6.35) than that which learned with educational software only (mean=5.25), U=66.00, p<.001. No significant
difference was found between the group of educational software mediated by an instructor (mean=6.49) and the group
with the instructor teaching with a presentation (mean=6.35), U=375.00, p=.354.
Table 2 – Comparison of test scores in terms of the dimensions in Bloom’s taxonomy Stage II according to the Mann
Whitney Test
Educational
software alone [1]
N=27
Educational
software mediated
by instructor [2]
N=28
Instructor who teaches
frontally with a
presentation [3]
N=28
U
2-3
U
1-3
U
1-2
Effect
size
1-3
Effect
size
1-2
Mean
(SD)
Mean
ranking
Mean
(SD)
Mean
ranking
Mean
(SD)
Mean
ranking
5.25
(0.51)
19.20
6.49
(0.42)
34.3
6.35
(0.44)
33.5
375.0
(p = 0.065)
66.0
(p = 0.00)
74.50
(p = 0.00)
0.65
0.78
5.55
5.15 5.05
6.34 6.78 6.34 6.28 6.23 6.55
0
2
4
6
8
10
Remember Understand Apply
Summary of tests by knowledge levels Stage II
Educational software alone
Educational software mediated by instructor
Instructor who teaches frontally with a presentation
0.39 =σ
0.41 =σ
0.27 =σ
0.47 =σ
0.44 =σ
0.71 =σ
0.43 =σ
0.46 =σ
0.58 =σ
European Journal of Educational Research 1149
Figure 7 presents a comparison between the general test scores in Stage I vs. Stage II.
Figure 7 – Comparison of test scores in terms of the dimensions in Bloom’s taxonomy, between Stage I and Stage II
Evidently, the ability to remember (retrieve knowledge from long-term memory) was lower in the group of using
educational software alone (1.38) compared to the group with an instructor who mediated the software (0.85) and to
the group with an instructor teaching frontally with a presentation (1.07).
Figure 8 presents a comparison between the test scores in Stage I and Stage II on the remember dimension.
Figure 8 – Comparison of test scores in the remember dimension between Stage I and Stage II
It is evident that the scores for the remember dimension dropped more among the group that had learned from
educational software alone (1.75) than among the group with an instructor who mediated the software (0.98) and
among the group with an instructor teaching frontally with a presentation (1.08).
Changes in the three groups scores between stages were significant. The Friedman statistic (used as a non-parametric
alternative to the parametric related samples t-test) was 25 (p=0.00) for the educational software alone group, 15.2
(p=0.00) for the educational software mediated by instructor group, and 14.2 (p=0.00) for the frontal instruction group.
6.63 7.34 7.42
5.25
6.49 6.35
0
2
4
6
8
10
Educational software alone Educational software mediated by
instructor
Instructor who teaches frontally
with a presentation
Comparison of test scores between Stage I and Stage II
Stage I Stage II
0.51 =σ
0.44 =σ
0.44 =σ 0.47 =σ
0.42 =σ
0.40=σ
7.3 7.32 7.36
5.55
6.34 6.28
0
2
4
6
8
10
Educational software alone Educational software mediated
by instructor
Instructor who teaches frontally
with a presentation
Comparison of test scores between Stage I and Stage II - Remember
Stage I Stage II
0.39 =σ
0.52 =σ
0.36 =σ
0.49 =σ
0.32 =σ
0.39 =σ
1150 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
Figure 9 presents a comparison between the test scores in Stage I and Stage II on the understand dimension.
Figure 9 – Comparison of test scores in the understand dimension between Stage I and Stage II
It is evident that scores for the understand dimension dropped more among the group that had learned from
educational software alone (1.37) than among the group with an instructor who mediated the software (0.97) and
among the group with an instructor teaching frontally with a presentation (0.95).
Changes in the three groups scores between stages were also significant for this dimension. Friedman statistic was 19
(p=0.00) for the educational software alone group, 18 (p=0.00) for the educational software mediated by instructor
group, and 11 (p=0.00) for the frontal instruction group.
Figure 10 presents a comparison between the test scores in Stage I and Stage II on the apply dimension.
Figure 10 – Comparison of test results on the apply dimension between Stage I and Stage II
It is evident that scores on the apply dimension dropped more among the group that had learned from educational
software alone (1.02) and among the group with an instructor teaching frontally with a presentation (1.16) than among
the group with an instructor who mediated the software (0.62).
Changes in between stages were significant for this dimension. Friedman statistic was 18 (p=0.00) for the educational
software alone group, 13.2 (p=0.00) for the educational software mediated by instructor group, and 22 (p=0.00) for the
frontal instruction group. Summarized in the table are the Friedman test for all groups.
6.52
7.75 7.18
5.15
6.78 6.23
0
2
4
6
8
10
Educational software alone Educational software mediated
by instructor
Instructor who teaches frontally
with a presentation
Comparison of test scores between Stage I and Stage II - Understand
Stage I Stage II
0.39 =σ
0.49 =σ
0.52=σ 0.48 =σ 0.49 =σ
0.52=σ
6.07
6.96 7.71
5.05
6.34 6.55
0
2
4
6
8
10
Educational software alone Educational software mediated
by instructor
Instructor who teaches frontally
with a presentation
Comparison of test scores between Stage I and Stage II - Apply
Stage I Stage II
0.39 =σ
0.49=σ
0.52=σ
0.42 = σ 0.39 =σ
0.47 =σ
European Journal of Educational Research 1151
Table 3- Summary of Friedman test results for all groups (Friedman test for differences in remember between stages 1 & 2)
Friedman
statistic
p
Friedman
statistic
p
Friedman
statistic
p
Education software alone
25.0
0.00
19.0
0.00
18.0
0.00
Education software mediated by instructor
15.2
0.00
18.0
0.00
13.2
0.00
Instructor teaching frontally with presentation
14.2
0.00
11.0
0.00
22.0
0.00
Discussion
Discussion of the results in Stage I
We now discuss the results regarding the test results in Stage I by Bloom’s taxonomy.
On the remember dimension, which according to Bloom is the basic dimension and includes the ability to recall
information items such as facts and terms learned, almost identical results are evident in the different learning
methods, i.e., it seems that educational software is as effective in imparting knowledge to trainees, with no need for
mediation by an instructor.
On the understand dimension, which according to Bloom is a higher dimension that includes demonstrating a basic
ability to interpret and translate the knowledge into meaning, the achievements in the group that learned with
educational software only were significantly lower. In contrast, when combining an instructor with educational
software, understanding was greatly improved. A possible explanation of the data is that the instructor’s mediation
enables deeper comprehension of the material and the addition of further layers that aim beyond remembering the
data (which cannot be mediated). According to the Media Naturalness Theory, a higher level of naturalness is facilitated
in frontal (face-to-face) instruction, and since such instruction requires of learners less cognitive effort the quality of
the learning is better (Weiser et al., 2016). In addition, according to the Media Richness Theory, richer media suits
complex and ambiguous messages on the comprehension and application dimensions and therefore it is not possible to
make do with educational software alone in order to establish knowledge on these dimensions rather it is necessary to
reinforce the learning via face-to-face communication with an instructor (Daft & Lengel, 1983). In addition, the
presence of an instructor enables two-way communication that has a higher level of trainee participation. This is
because the instructor creates involvement and interest by asking questions. This communication, which in many cases
predicts trainees’ success, enables better comprehension of the studied material (Weiser et al., 2016).
According to Te’eni’s (2001) affective-cognitive model of organizational communication, contextualization is essential
in order to generate understanding. Different layers that are relevant for the topic studied, such as explanations,
demonstrations, illustrations, and various emphases, can be brought by an instructor in face-to-face communication. In
this type of communication, the instructor can take the learners’ perspective and according to their difficulty's
expansions can be made through the different context layers in a manner that focuses on the difficulty at any given
moment. For instance, an instructor can explain why a certain action is necessary, bring possible alternatives for
various actions, and thus reach deeper understanding, improved functioning, and higher problem solution abilities.
On the apply dimension, where according to Bloom the learner is also required to provide solutions to new situations,
the scores on the educational software alone are once again as low as in the educational software mediated by an
instructor. The highest scores are for the condition of an instructor who teaches frontally with a presentation. A
possible explanation of the findings is that when the software is combined with an instructor, the instructor evolves
from lecturer to facilitator, i.e., not only mediates knowledge but also guides learners to use the study material
efficiently, while revealing and explaining more complex tasks that also appeal to the trainees’ application ability (Sun
et al., 2008). In addition, on the apply dimension as well, two-way communication between learners and instructor
enables a higher level of participation by trainees. Within the instruction, the instructor can bring varied practical
examples from daily life that help apply that studied to situations and problems in the field (Weiser et al., 2016).
In summary, if the trainees are required to display the remember dimension educational software may suffice, as an
effective and convenient tool. But if more thorough understanding is required, than a combination of an instructor and
educational software will provide excellent results. In contrast, if the learners are required to reach the apply
dimension, it is preferable to have an instructor who teaches frontally with a presentation, where it is possible to
provide specific illustrations of the use and application of the theoretical material in the field.
We now discuss the results of questionnaire items that measure the ability to use communicational strategies according
to the affective-cognitive model of organizational communication.
In the overall summary of the results obtained on the questionnaires based on the affective-cognitive model of
organizational communication (Te’eni, 2001), educational software achieved the lowest scores, educational software
1152 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
mediated by an instructor achieved considerably higher scores, and nearly identical scores with a slight preference for
a mediating instructor were found for an instructor teaching frontally with a presentation.
For the ability to apply the contextualization strategy, educational software achieved a low score. When the software
was combined with a mediating instructor the score improved greatly, particularly because the trainees gave a high
score to this method’s ability to provide explanations and to facilitate comprehension of the studied material. In the
context of understanding, combining educational software with an instructor was the best. An instructor who teaches
frontally with a presentation achieved lower scores than the method that combines educational software with an
instructor. It seems that learners perceive the combination of the two – educational software and instructor – as
enabling a wider range of details and explanations. Educational software is perceived as providing information and the
instructor too is perceived as providing information. We did not expect to see a difference in the level of information
between the group with an instructor teaching with a presentation and the group learning from educational software.
Then again, it is only logical that each method separately (software or instructor only) was perceived as providing a
certain level of information that is lower than that of the two methods combined, so it appears that the perceived level
of information is significantly higher. In the software group, in addition, compared to the two other groups, there was
less of a feeling that comprehension of the material was consistent with what the software wished to convey.
When using educational software, it is necessary to add information in order to enrich the learning experience and to
clarify messages that might be interpreted wrongly, and one of the best ways of doing this is by an instructor who
mediates between the software and the student and generates rich communication that facilitates easier use of the
contextualization strategy (Chang et al., 2017).
For the ability to apply the affectivity/emotion strategy, educational software received a very low score, showing that it
did not manage to express the emotional aspect and to give room to learners’ feelings. It also did not arouse a sense of
success in one’s studies and motivation to study. This is probably one of the reasons that the test results ultimately
achieved a low overall total, with an emphasis on understanding and applying, compared to the other options that
combined mediation by an instructor who generated interest and motivation to learn. Use of educational software
alone differs significantly than the two other methods with regard to the perceived place of emotions in the learning
process. A significant difference was also found in the perceived ability to arouse motivation, between software
mediated by an instructor and an instructor using a presentation. It appears that in frontal interactions by an instructor
using a presentation, rather than educational software, the instructor has more opportunity to generate motivation,
since the communication produced in frontal teaching gives more place to emotions and interpersonal contexts than in
educational software mediated by an instructor. We hypothesize that an instructor who mediates educational software
communicates in this situation primarily in order to support comprehension of the software and is thus more focused
on it and on its contents and less inclined towards motivating discussions with the class.
With regard to control and planning, scores on the educational software were very low and lowest on the other
questions in the questionnaire as well. The instructor-mediated software method and the frontal instructor method
received similar very high scores. The educational software did not let trainees give affective feedback and check and
understand the material throughout the learning, and therefore did not provide a sense of control over the learning
process. Frontal learning with an instructor gave the highest sense of control during the learning process, significantly
higher than instructor-mediated software as well. Students who had an instructor who taught with a presentation
received a higher sense of control than those who had an instructor who mediates educational software. Apparently,
when learning from educational software in the instructor-mediated method the student feels less control than in a
situation of learning with an instructor in class. Learners apparently feel that the educational software dictates learning
even if an instructor is involved in its mediation. The software seems to produce a sense of consistent progress by a
predetermined order, while frontal learning may be freer and less structured. According to Te’eni’s theory, the
feedback received by the learners in face-to-face communication is immediate, in contrast to the delayed feedback
received at times when using educational software. The software is static and does not perform adjustments, since the
learner’s feedback is almost never taken into account. Most educational software is not built to change dynamically
according to its users’ understanding. The combination of educational software mediated by an instructor is a way of
achieving control: The instructor is the human element that makes it possible to compensate for the static nature of the
software and for the fact that it does not take into account learners’ feedback on understanding.
For the ability to apply the perspective taking strategy as well, scores for the educational software were lowest,
significantly lower than the two other methods. The perception is that educational software does not allow one to take
the learner’s perspective. The software does not take into account the learner’s degree of comprehension throughout
the learning process or his or her previous knowledge. The software might present to the learner information that is
trivial for him because he has previous knowledge of the field, or alternately it cannot present information that is
critical for his understanding as he lacks some basic background in the studied material. When combining an
instructor’s mediation, the situation is better but still less good than an instructor who teaches frontally with a
presentation. A possible explanation is that an instructor with a presentation includes trainees more in the learning
process (Weiser et al., 2016). An instructor in frontal learning can give expression to the learners’ different
perspectives, take their degree of understanding into account, and proceed at a pace that is suitable for their
European Journal of Educational Research 1153
understanding. He can change displays (for instance, switch from table to figure) and add examples dynamically
according to the questions and comments of the students. In questions dealing with the ability to adapt the instruction
to the technicians’ level of understanding, instruction with a presentation received the highest scores.
For the ability to adapt and focus attention, educational software received the lowest scores, and the combination of
instructor-mediated educational software was found to be the best. The educational software is less good at focusing
trainees’ attention, because it facilitates relatively passive learning. The learner normally faces a visual and auditory
technological display (for instance, a visual view of the pilot’s cockpit and a simulation of moving switches and
performing different operations in the pilot’s cockpit). The score improves greatly in the situation of an instructor
teaching frontally with a presentation and it is highest when an instructor accompanies learning with educational
software. In frontal learning the instructor can clarify to the student's which topics or details he sees as most critical,
can use vocal intonation to create a sense of importance and interest, and so on. All this is possible also in a situation of
an instructor mediating educational software, in which case he can also focus the student’s attention on the important
parts of the software, recommend skipping less important parts, indicate significant areas on the screen, etc.
We now discuss the results of questionnaires in the STEM model.
In the STEM model, the educational software received low scores relative to software mediated by an instructor and an
instructor teaching with a presentation. On the topic of assistance and imparting knowledge, all methods received
almost identical scores with software having a slight advantage. This topic was also manifested in test scores. In the
ability to analyze tasks and the ability to analyze and interpret data, the software received low scores also when
mediated by an instructor, with a frontal instructor teaching with a presentation being the best option, and this seems
to have helped the trainees successfully answer questions that measure the ability to apply knowledge. In planning
ability and integration in a team as well, lower scores were evident compared to an instructor with a presentation, as
evident also from the affective-cognitive model of organizational communication. Mediation helped the trainees to
properly answer the questions that measured understanding. The educational software received a particularly low
score for changes in the system. The software is not updated on a regular basis and therefore the instructor cannot
include in it various updates. This is its main weakness. Interestingly, also on the question of modern methods and tools
that the software presumes to represent, relatively low scores were obtained and the mediating instructor did not
result in any considerable improvement. The conclusion is that aside from technological performance (“pyrotechnics”),
it is important for the trainees to receive current knowledge and they are very critical of errors.
Discussion of the results in Stage II
The purpose of Stage II was to examine the drop-in test results for each type of instruction over time and to assess the
effectiveness of the instruction not only in the short term rather also in the medium term after four months.
As expected, in all tests a drop was evident in the results four months later. The drop in the general level was highest
when only educational software had been used. The second highest drop was in the group who had studied with an
instructor teaching frontally with a presentation, and the lowest was for the combination of an instructor who mediatd
educational software. When examining the drop by the dimensions in Bloom’s revised model, for the dimensions of
remember and understand the greatest drop was when only educational software was used. A possible explanation of
the data is that the instructor’s mediation focuses on the desire that trainees will understand the material rather than
only remember data, which can be learned without mediation. The instructor forms two-way communication that
facilitates a higher level of trainee participation, generates interest, and asks questions. Emphasizing understanding
and not only remembering enables less loss of the study material learned (Weiser et al., 2016). On the apply dimension,
we were surprised to see that the greatest loss was when the instruction was performed by an instructor teaching with
a presentation.
In the current study, on most parameters learning with educational software was found to be inferior to the other
learning methods. Educational software is usually a static tool with a low capacity for topical development. In a
dynamic and changing world, almost in any field of learning it is necessary to update educational software more often
in order to prevent loss of the material studied and to overcome its lack of relevance. In addition, educational software
that changes with time generates interest among the learners. Studying with the same software does not create interest
and might even create a sense of contempt by those compelled to use it repeatedly against their will. Changing
educational software has the potential of increasing trainees’ trust in the process of studying with it, creating interest
and inquisitiveness, and thus forming higher motivation for self-study. At present, smart systems are utilized in
industry (Industry 4.0), in the urban field in the form of smart cities, in the home field in the form of smart homes; there
is smart transportation, individually adapted smart medical care, and more. There is no reason that the pedagogical
field should not progress to smart, individually adapted learning. Smart educational software can utilize elements of
machine learning with artificial intelligence capabilities that study each of the users, adapting their displays, exercises,
level of difficulty, and learning pace to the learner’s needs and abilities. Different learners have diverse backgrounds
and previous experience and most educational software does not enable individual adaptation. As in many domains,
“one size does not fit all”. For instance, production has been revolutionized and now enables personal adaptation of
1154 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
products to the client’s needs. There is no reason that the existing advanced technological abilities cannot be harnessed
in the domain of educational software as well. For example, it is possible to create exercise and assignment databases
that suit the learner’s goal (for instance, the type of knowledge required according to his or her role in the
organization) and abilities, which are dynamic as well. For instance, a student can begin on a basic level and in time the
system will know that the same student is now more experienced and thus will adjust the difficulty level of the
assignments accordingly, “spirally”. In addition, it is possible to adapt educational software to the professional
background of the users and to replace examples and illustrations according to the relevant content world of each
learner. In a world with such dominant social media, it is also possible to think of combining social elements in
educational software, such as by presenting in the software other users’ difficulties, insights, and solutions. Another
direction in the social context is including game-oriented elements (gamification) in learning, which produce fun and
raise motivation to learn (De-Marcos et al., 2014), as in competition with other users, such as having a table that shows
leading students in the class regarding use of the software, accumulating virtual prizes, and so on. Digital game-based
learning, uses tools that incorporate educational content into computers to produce significant changes in learning
outcomes with the aim of engaging learners in an entertaining way. Digital game-based learning is an innovative
instructional method that can achieve diverse educational outcomes including academic skills, efficiency behaviors, and
emotional regulation (Pauline-Graf & Mandel, 2019).
Conclusion
The current study explored the effectiveness of technology enhanced learning in the short and medium term by using
several tools: Bloom’s taxonomy, the STEM model (Davidovich & Shiler, 2016), and the affective-cognitive model of
organizational communication (Te’eni, 2001). The study found that educational software is an inferior method when
used as the single learning tool. The inclusion of a human instructor who mediates the educational software to the
students is very valuable on most parameters measured. In addition, frontal learning with an instructor who uses a
presentation was found to have many advantages. The instructor can contribute significantly to instruction with
educational software both on the level of learning achievements evident from the test results and with regard to the
subjective sense of the learning experience and the different perceptions of learning as evident from the questionnaire
results. Therefore, the Air Force and other organizations must adapt the instruction to their specific goals. When it is
necessary to convey retrieval of knowledge from long term memory, the software as a tool is not only beneficial but
rather also efficient and economical, but when the instruction is more complex and there is need to arouse motivation
and to allow participation of the trainees and connection to the instruction goal, an instructor should be included in the
learning process. Educational software is a static tool with a low capacity for topical development, and in a dynamic and
changing world it is necessary to update the software often or to consider other tools such as utilizing a skills
classroom. In this classroom the software is not a standalone tool; after learning with the software trainees apply what
they learned in practice in a sterile classroom with the necessary physical means and accompanied by an instructor. In
this way, theoretical instruction (educational software) is combined with practical instruction (skills classroom).
Recommendations
In addition, in the current study we focused only on the first three dimensions of Bloom's revised taxonomy –
remember, understand and apply. It will be very interesting to replicate our study for examining the effects of different
learning conditions on the next three and highest Bloom’s taxonomy dimensions. Such studies would provide important
insights on the influence of learning conditions on learners’ ability to analyze, evaluate and create. Analyzing and
evaluating are information based thinking skills, and creativity is a problem solving skill. Instructional design principles
for the 21st century need to put emphasize on these high level thinking skills (Sahin, 2009).
The study stresses the added value of a human instructor within technology enhanced instruction. The practical
implications of the study for decision-makers in organizations are to adapt instruction for relevant populations and
occupations based on the instruction goals with regard to different cognitive process levels. Organizations must invest
recourses in training programs for instructors, so the latter will be able to accompany learners effectively.
Organizations and instructors need to understand the importance of the availablity of instructors to answer questions
and guide, while learners use educational software. Training programs for these instructors should include an
emphasis on the use of communicational strategies such as contextualization, and perspective taking.
In addition, the study raises the question of the effectiveness of static educational software, which is the most common
at present, and poses considerable questions concerning its relevance. For educational software to be beneficial for
learning processes and perceived positively by their users, they must be evolving, dynamic, updated, and adapted to the
learners. It is time for those assimilating educational software to demand that developers apply elements of smart
technology in order to generate individual adaptation to learners, with regard to their needs, abilities, study pace, and
so on. The time has definitely come to transition to smart educational software.
We recommend that researchers examine the effectiveness of static versus interactive educational software in future
studies. Learners' abilities to remember, understand and apply can be compared in a group that use static software
European Journal of Educational Research 1155
versus a group that use an interactive software that encourages active learning (in which the ability to proceed to
following learning stages is possible only after performing actions such as answering questions and solving problems).
Limitations
While the study was conducted using data from a specific context - Israeli Air Force technicians, it is possible to
generalize the findings, to a nearly complete extent, to other learning contexts such as higher education institutes, job
training of various kinds, and also other countries. Of course, the findings’ applicability may be higher in organizational
contexts where the job type characteristics, the organizational atmosphere and the culture are similar to those found in
the training of Israeli Air Force technicians.
Authorship Contribution Statement
Tzur: Conceptualization, design, analysis, writing. Katz: editing/reviewing, supervision. Davidovich: editing/reviewing,
supervision, final approval.
References
Babiker, M., & Elmagzoub, A. (2015). For effective use of multimedia in education, teachers must develop their own
educational multimedia applications. Turkish Online Journal of Educational Technology, 14(4), 62-68.
Beetham, H., & Sharpe, R. (Eds.). (2013). Rethinking pedagogy for a digital age: Designing for 21st century learning.
Routledge.
Ben-Zadok, G., Herskovits, A., & Nachmias, R. (2009). Examining online learning processes based on log files analysis: A
case study. In A. Mendez-Vilas (Ed.), Research, reflections and innovations in integrating ICT in education (pp. 55-
59). Formatex.
Chang, C. C., Liang, C., Chou, P. N., & Lin, G. Y. (2017). Is game-based learning better in flow experience and various types
of cognitive load than non-game-based learning? Perspective from multimedia and media richness. Computers in
Human Behavior, 71, 218-227. https://doi.org/10.1016/j.chb.2017.01.031
Cicognani, A. (2000). Concept mapping as a collaborative tool for enhanced online learning. Educational Technology &
Society, 3(3), 150-158.
Coomans, S., & Lacerda, G. S. (2015). Petese, a pedagogical ergonomic tool for educational software evaluation. Procedia
Manufacturing, 3, 5881-5888. https://doi.org/10.1016/j.promfg.2015.07.895
Cotton, K. (2008). Computer‐assisted instruction. In Encyclopedia of Special Education (pp. 514-520). John Wiley &
Sons. https://doi.org/10.1002/9780470373699.speced0481
Daft, R. L., & Lengel, R. H. (1983). Information richness: A new approach to manage information processing and
organizational design. Department of Management Texas A&M University.
https://apps.dtic.mil/sti/pdfs/ADA128980.pdf
Davidovich, N., & Shiler, Z. (2016). Skill-based teaching for undergraduate STEM majors. American Journal of
Engineering Education 7(1), 29-36. https://doi.org/10.19030/ajee.v7i1.9683
De-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014). An empirical study comparing gamification and
social networking on e-learning. Computers & Education, 75, 82-91.
https://doi.org/10.1016/j.compedu.2014.01.012
Draper, S. (1996). Observing, measuring, or evaluating courseware: A conceptual introduction. CiteSeerX.
https://cutt.ly/PmqBOh6
Hwang, G. J., Spikol, D., & Li, K. C. (2018). Guest editorial: Trends and research issues of learning analytics and
educational big data. Journal of Educational Technology & Society, 21(2), 134-136.
Jamieson, S. (2004). Likert Scales: How to (ab)Use Them. Medical Education, 38(12), 1217-1218.
https://doi.org/10.1111/j.1365-2929.2004.02012.x
Joubert, M. (2013). Using digital technologies in mathematics teaching: Developing an understanding of the landscape
using three "grand challenge" themes. Educational Studies in Mathematics, 82(2) , 341-359.
https://doi.org/10.1007/s10649-012-9430-x
Kennedy, D. M., & McNaught, C. (1997). Design elements for interactive multimedia. Australasian Journal of Educational
Technology, 13(1), 1-22. https://doi.org/10.14742/ajet.1916
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into Practice, 41(4), 212-218.
https://doi.org/10.1207/s15430421tip4104_2
1156 TZUR, KATZ & DAVIDOVICH / Learning Supported by Technology
Kruse, K., & Keil, J. (2000). Technology-based training: The art and science of design, development, and delivery. Jossey-
Bass.
Kuzon, W. M., Urbanchek, M. G., & McCabe, S. (1996). The seven deadly sins of statistical analysis. Annals of Plastic
Surgery, 37(1), 265–272.
Laborde, C., & Straesser, R. (2010). Place and use of new technology in the teaching of mathematics: ICMI activities in
the past 25 years. Mathematics Education, 42(1), 121-133. https://doi.org/10.1007/s11858-009-0219-z
Lê, Q., & Lê, T. (2007). Evaluation of educational software: Theory into practice. Technology and teaching, 11,115-124.
Ma, J., Han, X., Yang, J., & Cheng, J. (2015). Examining the necessary condition for engagement in an online learning
environment based on learning analytics approach: The role of the instructor. The Internet and Higher
Education, 24, 26-34. https://doi.org/10.1016/j.iheduc.2014.09.005
Melamed, U. (2013). Implementing information skills as a route to significant learning. Mofet Institute. [in Hebrew]
Nachmias, R., & Mioduser, D. (2001). Integrating the Internet in education. Eureka, 14, 6-16. Melamed, U. (2013).
Implementing information skills as a route to significant learning. [in Hebrew]
Niederhauser, D. S., & Stoddart, T. (2001). Teachers’ instructional perspectives and use of educational
software. Teaching and teacher education, 17(1), 15-31. https://doi.org/10.1016/S0742-051X(00)00036-6
Norman, G. (2010). Likert Scales, Levels of Measurement and the “Laws” of Statistics. Advances in Health Sciences
Education, 15(5), 625-632. https://doi.org/10.1007/s10459-010-9222-y
Pauline-Graf, D., & Mandel, S. E. (2019). Defining Preliminary Research for Digital Game-Based Learning Evaluation:
Best Practices. International Journal of Educational Methodology, 5(4), 623-635.
Rappel, L. (2017). Self-firection in on-line learning. Journal of Educational System, 1(4), 6–14.
Ross, M., Perkins, H., & Bodey, K. (2016). Academic motivation and information literacy self-efficacy: The importance of
a simple desire to know. Library & Information Science Research, 38(1), 2-9.
https://doi.org/10.1016/j.lisr.2016.01.002
Rotem, A., & Peled, Y. (2008). School turns on-line. Mofet Institute. [in Hebrew]
Sahin, M. C. (2009). Instructional design principles for 21st century learning skills. Procedia-Social and Behavioral
Sciences, 1(1), 1464-1468. https://doi.org/10.1016/j.sbspro.2009.01.25
Schmeeckle, M. (2003). Online training: An evaluation of the effectiveness and efficiency of training law enforcement
personnel over the Internet. Journal of Science Education and Technology, 12(3), 205-260.
https://doi.org/10.1023/A:1025028806189
Squires, D., & Preece, J. (1999). Predicting quality in educational software. Interacting with computers, 11(5), 467-483.
https://doi.org/10.1016/S0953-5438(98)00063-0
Stoddart, T., & Niederhauser, D. (1993). Technology and educational change. Computers in the Schools, 9(2-3), 5-23.
https://doi.org/10.1300/J025v09n02_02
Sun, P. C., Tsai, R. J., Finger, G., Chen, Y. Y., & Yeh, D. (2008). What drives a successful e-learning? An empirical
investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183-1202.
https://doi.org/10.1016/j.compedu.2006.11.007
Te’eni, D. (2001). A cognitive-affective model of organizational communication for designing IT. Bar-Ilan University.
Waldman, N. (2007). Only the enthusiasts hold the baton – Is that so? Teacher-teachers eager to improve teaching with
the help of the computer. In college, 19 , 403-415. [in Hebrew]
Weiser, O., Blau, I., & Eshet-Alkalai, Y. (2016). Synchronous face-to-face learning and in one- and two-way
videoconferencing: The role of pedagogy media, personality factors, and level of media naturalness. Open University.
[in Hebrew]
