Is Technology Mediated Learning Really Improving Performance Of Students?

Abstract

This paper examines the role of information technology in learning environments. In particular, it goes through the analysis of the impact of the use of information technology in high school students’ performance. It describes and analyzes the initiative carried on by Impara Digitale Study Center: 370 students from different high schools experienced a full school year with a new teaching model, using tablets and computers instead of text books, according to a cooperative model, where each learner adopted his own device. The experience is paramount and it opens a stream of questions for further and more extensive diffusion, i.e. institutionalizing the adoption of personal devices in learning environments. We explored the different theories that can help with answers and we designed our research by using the widely adopted TAM Model, where grades are used as a measure of learning effectiveness. We also measured learning effectiveness in a control sample, using in the same schools same teachers and more traditional learning approaches. Our conclusions show that the new method improves students’ performance only if teachers, who play a pivotal role in their technology acceptance, properly support them.

Full text

1
Is Technology Mediated Learning Really Improving
Performance Of Students?
Leonardo Caporarello1, Massimo Magni 1 Ferdinando Pennarola1,*
1 Università L. Bocconi
Abstract
This paper examines the role of information technology in learning environments. In particular, it goes through the
analysis of the impact of the use of information technology in high school students’ performance. It describes and
analyzes the initiative carried on by Impara Digitale Study Center: 370 students from different high schools experienced a
full school year with a new teaching model, using tablets and computers instead of text books, according to a cooperative
model, where each learner adopted his own device. The experience is paramount and it opens a stream of questions for
further and more extensive diffusion, i.e. institutionalizing the adoption of personal devices in learning environments. We
explored the different theories that can help with answers and we designed our research by using the widely adopted
TAM Model, where grades are used as a measure of learning effectiveness. We also measured learning effectiveness in a
control sample, using in the same schools same teachers and more traditional learning approaches. Our conclusions show
that the new method improves students’ performance only if teachers, who play a pivotal role in their technology
acceptance, properly support them.
Keywords: TAM, Technology Mediated Learning, Learning Effectiveness
Received on 01 May 2016, accepted on 10 August 2016, published on 02 December 2016
Copyright © 2016 Leonardo Caporarello et al., licensed to EAI. This is an open access article distributed under the
terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/3.0/), which permits
unlimited use, distribution and reproduction in any medium so long as the original work is properly cited.
doi: 10.4108/eai.2-12-2016.151719
*Corresponding author. Email: ferdinando.pennarola@unibocconi.it
1. Learning through technology: a long
debated issue in information systems
research,but still a challenge for
schools and educators
Digital revolution has an impact on every aspect of life. One
of the most discussed issues regarding the influence of
technology in everyday activities is education. Today’s, the
young generation lives in a connected world surrounded by
digital technologies, and many observers predict a growing
distance between school and out-of-school experiences for
students, unless schools update their instructional tools and
methods. This posed a great number of questions in
the literature, stimulating a new flow of study regarding the
use of ICT in educational environments (Rienties et al.
2016).
The main problem of technology in school is related to the
fact that the closed classroom represents a physically
outdated teaching model which does not match the
interconnected virtual world in which students live in: they
are learning collaboratively through a vast array of informal
learning spaces both on and off school, but, when it comes to
daily school life, they are still packed into outdated
traditional models. These learning spaces need to adapt to
meet the emerging needs of a wide range of pedagogies.
Meanwhile, the “consumerization” of IT, with mass market
devices that can be also used for work related reasons, is
pouring into the kids’ pockets powerful tools that are also
contributing in life changes. In this research we explored to
what extent the adoption of tablet technologies, originally
designed for a mass consumption market, can be also
powerful tools for learning (Bourgonjon et al. 2013).
Schools, in many circumstances, are acting as the linking
point to introduce students into the work worlds; they can
profit from widely adopted technologies, by leveraging on
the diffused practice of Bringing Your Own Device
(BYOD).
EAI Endorsed Transactions
on e-learning Research Article
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
1Authors are listed in alphabetical order. Authors are grateful to Impara Digitale, the non profit association that authored the experimental teaching and
learning environment described in the paper. (*) Corresponding Author

Leonardo Caporarello, Massimo Magni and Ferdinando Pennarola
2
What happens when students bring their tablets to school?
Could the teaching / learning environment be revolutionized
thanks to this consumer oriented technology? Is it possible to
fully integrate such devices into the teaching and learning
process? What can be the results of this experience in terms
of learning improvements? The key word here is integration:
bringing a tablet to school is just the beginning of a new
learning journey.
While there have been past initiatives on ICT in education,
they were limited to the introduction of digital devices and
isolated competences within the learning sector (Cheung and
Vogel, 2013). Not enough attention was paid to integration
and support actions. Devices were placed in separated
classrooms and competences were isolated in a minority of
professors in the scientific areas. Past research has explored
the issue in many ways. Therefore, our research is
particularly worthwhile because it helps to shed further light
on the role of technology for learning purposes in a more
extensive setting. Indeed, we look at digital education in a
wider setting, thus involving students not in just one specific
activity, but throughout their entire learning experience
across all the different topics. Thus, by relying on traditional
theoretical framework we outline how a comprehensive
approach toward technology-supported learning may affect
students’ behaviors.
Researchers conducted several studies in order to understand
whether technology improves the learning experience in
some way, and most importantly, if it enhances students’
performance too. Bernard et al. (2004) perform a meta-
analysis of representative prior studies and argue that the use
of information and communication technologies cannot
guarantee greater learning effectiveness or satisfaction than
classroom-based, face-to-face learning. Information System
analysts also caution that the capabilities offered by
multimedia only provide an opportunity to generate benefits
rather and guaranting them (Lim, Benbasat, and Ward, 2000).
Anyhow, schools began to introduce digital tools in
education about twenty years ago, buying computers and
starting computing courses. From that moment, schools felt
the urge to keep the pace with technology innovation trying
to encourage students to have a more interactive relationship
with study material. United States have been the precursor of
this trend over the years, followed in a non-uniform way by
other industrialized countries. The primary aim of integration
of technology into schools was to improve teaching and
learning in different subjects and also with an
aim of increasing motivation for both students and teachers
(Bourgonjon et al. 2013).
Arguments to sustain this purpose were that ICT can have
several advantages, like creating more dynamic interaction
between students and teachers, increasing collaboration and
team work in problem
2. When real adoption matters: the
development of the Technology
Acceptance research stream
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
solving activities, stimulating creativity and helping students
to control and monitor their own learning. Further, students
will have to be able to use ICT as adults in working
environments, whatever they’ll be, so the introduction of
technology in schools would allow them to develop skills
that will be useful for them in their future academic and
professional lives (Bourgonjon et al. 2013). Another aspect
to be considered is that technology characteristics can
enhance or inhibit efficient delivery of instructional material
(Alavi and Leidner, 2001) and thus may play a crucial role in
influencing the learning process (Kozma, 1991). Nicholson,
Nicholson and Valacich (2008) analyzed two key
characteristics: vividness and interactivity. In their study
they proved that a more vivid and interactive presentation is
likely to increase both students’ satisfaction and interest.
They went further in their analysis, investigating the
relationship between technology characteristics and task
complexity. This represents a major factor to be taken into
consideration, since complexity influences the effectiveness
of technology characteristics for the learner.
Wood (1986) states that it is more complex when there are
more information cues to process, more acts to execute or
increased interdependence between the cues and acts. Then,
more complete tasks require the learner to generate a more
elaborate model (White and Frederiksen, 1990), thus there is
an increase in cognitive load, and this can result in lower
performance and learning (Bannert, 2002). This means that
students that have to face more complexity need to reach
higher levels of attention and engagement in order to
succeed and to obtain better performance. A direct
consequence of this fact is that students will have higher
performance in tasks that are more complex when vividness
and interactivity are high, and the same applies for the
students’ perceived mental effort (Nicholson, Nicholson and
Valacich, 2008). Schools have to analyze all of these
elements when making the choice between face-to-face
lessons and Technology Mediated Learning, and they also
have to deeply assess the kind of technological
support they want to invest in.
Maybe, the founding fathers of the Technology acceptance
literature stream of research can be considered Fishbein and
Ajzen, who in 1975, proposed their “Theory of Reasoned
Action”, known as TRA, drawn from social psychology,
and it became one of the most influential theories of human
behavior. They suggested that a person’s actual behavior
could be determined by considering his or her prior
intention along with the beliefs that the person would have
for the given behavior. So, they gave two definitions:

Is Technology Mediated Learning Really Improving Performance Of Students
3
• Attitude Toward Behavior: “an individual’s positive or
negative feelings (evaluative affect) about performing the
target behavior” (Fishbein and Ajzen 1975, p.216)
• Subjective Norm: “the person’s perception that most people
who are important to him think he should or should not
perform the behavior in question” (Fishbein and Ajzen 1975,
p.302).
The authors referred to the intention that a person has, prior
to an actual behavior, as the behavioral intention of that
person, and defined it as a measure of one’s intention to
perform a behavior. The Attitude toward behavior is the sum
of the products of all of the main beliefs (bi) about the
consequences of performing that behavior and the evaluation
of those consequences (ei): A = Σ bi ei
They also considered the Subjective norm as the sum of the
product of an individual’s normative beliefs (nbi) and his/her
motivation to comply to them (mci): SN = Σ nbi mci
Thus, Behavioral Intention is calculated as the sum of A and
SN: B = A + SN
Starting from Fishbein and Ajzen’s TRA, Fred Davis
proposed his first version of TAM in 1985. The concept
behind his model was that System Use is a response
determined by User motivation, which in turn is directly
explained by an external stimulus that consists in the actual
system’s features and capabilities (Davis, 1985). Starting
from this basis Davis refined the model, obtaining the first
version of the Technology Acceptance Model. The first TAM
was based on three factors: Perceived ease of use, Perceived
usefulness and Attitude toward using the system. Davis
referred to Perceived usefulness saying that “people tend to
use or not use an application to the extent they believe it will
help them perform their job better” (Davis, 1989 – p.2), while
the importance of Ease of use was to be found in the evidence
of the Effort being a finite resource that an individual could
allocate to various activities for which he/she is responsible
(Radner and Rotschild, 1975).
The impact of perceived usefulness on system utilization was
suggested by Schulz and Slevin (1975), and expanded by
Robey (1979). The latter theorized that: “A system that does
not help people perform their jobs is not likely to be received
favorably in spite of careful implementation efforts” (Robey,
1979 – p. 537). These studies proved that perceived
usefulness provided a reliable prediction for use. At the same
time, support for the relevance of perceived ease of use could
be found in the meta-analysis of Tornatzky and Klein’s
(1982) research on innovation adoption, where they studied
the connection between the characteristics of a system and its
adoption, finding that the complexity of an innovation was
the factor that had the most consistent significant relationship
with the adoption.
Bandura (1982) then showed the effect of both perceived
ease of use and perceived usefulness in predicting behavior,
suggesting that the latter would be best predicted by both,
self-efficacy and outcome judgments. These two factors were
defined by Davis (1985) and put together in the first TAM. In
1991, Ajzen expanded the TRA – Theory of Reasoned Action
(1975). It was made necessary, in the author’s word, by the
original model’s limitations in dealing with behaviors over
which people have incomplete control. Ajzen suggested that
the stronger is the intention to engage in a behavior, the more
likely should be its performance, but this holds true only if
the individual can actually decide to perform or not the
behavior in question. The fundamental assumption that lies
under this model is that people’s intentions capture the
motivational factors that influence behaviors. But intention is
not sufficient to explain the performance of the behavior,
because there is a pivotal element to be assessed: people’s
actual control over the behavior itself. This consists, in fact,
in the set of non-motivational factors (as availability of
requisites, opportunities and resources) that influence the
performance (or non performance) of a behavior. But more
important then the actual control over the behavior is the
person’s perception of the control over the behavior: this
element is the main element of differentiation from the
Theory of Reasoned Action. Perceived behavioral control
refers to people’s perception of the ease or difficulty of
performing the behavior, and in this case Ajzen intended this
concept in a way similar to the Bandura’s (1982) concept of
self-efficacy which “is concerned with judgments of how
well one can execute courses of action required to deal with
prospective situations” (Bandura, 1982 – p.122).
In 1991 Thompson, Higgins and Howell proposed the Model
of PC Utilization, based on Triandis’ (1977) theory of human
behavior. Triandis believed that much of the work in
psychology was becoming fragmented, lacking a theoretical
framework for guiding future research, so he developed a
comprehensive model synthesizing relations among attitudes,
values, and other acquired behavioral dispositions to action.
This model does not suggest a causal relationship between
the cognitive component of attitudes and the affective
component, instead they are seen as independent (even if
related) factors that determine behavior indirectly through
intentions.
The affective component of attitudes has a like/dislike
connotation:
- Perceived consequences are related to the evidence that an
act is perceived as having potential consequences, which
carry a value and a probability of happening;
- Social factors are made of the person’s internalization of
norms, roles and values. These elements affect the
individual’s opinions on behaviors that are appropriate,
desirable or morally correctly;
- Facilitating conditions include all of the conditions that are
objective factors that make an act easy or difficult to be
made;
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation

4
Thompson et al. (1994) argue that this model is too
complex and it would be hard to employ it in its entirety.
Starting from this point they developed the Theory of
Planned Behavior, including six factors that were
hypothesized to have an effect on the use of PCs:
- Social factors (norms) influencing PC use
- Affect toward PC use
- Complexity of PC use
- Job Fit with PC use
- Long-term consequences of PC use
- Facilitating conditions for PC use
The result of this research, was a scheme that was different
from the Triandis model, but was not a complete framework
yet, due to the fact that previous experience was not fitted
into the model.
Contributions continue in 1994, Thompson Higgins and
Howell introduced the concept of Experience, seen as a
“reinforcement” (in the words of the authors): objective
consequences are interpreted by the individual, and this leads
to reinforcement that affects the perceived consequences in
two ways, because on one hand it changes the perceived odds
that a behavior will have particular repercussions, and it
varies the value of these repercussions. This way of defining
the concept of experience allows including a feedback loop in
the model.
Innovation Diffusion Theory played an important role in this
development. This theory has its basic foundations in the
acknowledgment of the fact that potential users’ perception
of the information technology innovation influences its
adoption, and is based on the Rogers’ identification of five
characteristics of an innovation (Rogers, 1983) which affect
the rate of diffusion of it. The main problem was that the
existing tools used at the time to tap these factors were not
reliable. Thus, the main constructs used to build this model
were the various Perceived Characteristics of using an
innovation. Rogers focused on five key characteristics:
- Relative advantage: “the degree to which an innovation is
perceived as being better than its precursor”;
- Compatibility: “the degree to which an innovation is
perceived as being consistent with the existing values, needs
and past experiences of potential adopters”;
- Complexity: “the degree to which an innovation is
perceived as being difficult to use”;
- Observability: “the degree to which the results of an
innovation are observable to others”;
- Trialability: “the degree to which an innovation may be
experimented with before adoption”.
These influence the phase of Perception that is the antecedent
of Decision.
The conclusion of this model is that innovation that are
perceived by potential users as having greater relative
advantage, compatibility, trialability, observability, and less
complexity, will be adopted more rapidly than other
innovations. It is important to remember that the premise of
this model was the work of Rogers (1962) whose conclusion
was that with successive groups of consumers adopting the
new technology its market share would eventually reach the
saturation level in the future.
Social Cognitive Theory also contributed in this domain. One
of the most important works in psychology was Bandura’s
(1986) Social Cognitive Theory, a widely accepted and
empirically validated model of individual behavior. Campeau
and Higgins (1991,1995) applied an extended version of it to
the context of computer utilization in order to study
performance. Their model included five constructs:
- Behavior Modeling: several studies showed that observing
someone else performing the target behavior increases the
subjects’ perception of their ability to do it successfully
(Bandura et al. 1977, Brown and Inouye 1978, Schunk 1981,
Bandura 1982), thus this model hypothesizes that people who
received behavioral modeling training will develop higher
perceptions of self-efficacy. Furthermore, modeling has been
demonstrated (Bandura 1971) to influence outcome
expectations as well; in fact, modeled behavior that is
rewarded is usually adopted by the observers, and it can also
directly influence performance.
- Self-efficacy: SCT argues that self-efficacy perception
affects a person’s outcome expectation (Bandura 1978) and it
is also a determinant of the subject’s actual ability to
perform the behavior.
- Outcome Expectations: it is inferred directly from SCT that
expectations about the consequences of behavior are a strong
drive guiding people’s actions. This holds true because
individuals are more likely to undertake actions that they
consider to be resulting in valued outcomes than those that
they do not see as having desirable consequences.
- Prior Experience: Wood and Bandura (1989) demonstrated
that prior success is expected to increase self-efficacy, while
prior adversity decreases self-efficacy. Also it can contribute
to the formation of outcome expectations, as noted by
Bandura (1986) “response outcomes influences behavior
antecedently by creating expectations of similar outcomes on
future occasions” (p. 229). It also has been found to be a
significant predictor of current performance.
Finally, Venkatesh and Davis proposed a second version of
the Technology Acceptance Model, named TAM2, in 2000.
They conducted a study in order to extend the original TAM
including additional key determinants of the model’s
Perceived usefulness and Intention to use constructs, and also
to understand how these determinants are influenced by
increasing in user experience over time with the system.
It is important to notice that in this model, the subjective
norm construct has a direct effect on intention to use: the
rational for such an effect is that individuals may choose to
perform a behavior (use a system in the case of technology),
even if they are not favorable toward it or its consequences, if
they feel that people important to them think that they should
(Fishbein and Ajzen 1975; Ajzen 1991). The introduction of
voluntariness is another factor on novelty in respect to the
original model. This choice was made on the base of a study
conducted by Hartwick and Barki (1994) where the authors
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
Leonardo Caporarello, Massimo Magni and Ferdinando Pennarola

5
The first real initiative of this kind was launched in the early
1990s, the “Program for the Development of Educational
Technologies”, that offered support to all schools to create
computer labs and to invest in the professional development
of all teachers. Along with these national initiatives, local
authorities and sometimes single schools have led their own
policies in the field of ICT for education, since in Italy,
school building and maintaining are under the responsibility
of local governments. Moreover, schools are granted
significant administrative autonomy, and can raise funds
from the private sector in order to improve their
infrastructure. This structure of governance implies that by
2007, some schools had already been equipped with ICT
infrastructures beyond the standard computer labs. The
National Plan for Digital Schools consists in one large-scale
intervention and three pilot projects:
- LIM Plan: Interactive whiteboards
- Cl@asse 2.0
- Scuol@ 2.0
- Digital Publishing
Only voluntary schools participate and, for the most intensive
interventions, schools have to elaborate and submit a project
explaining the intended objectives of ICT introduction. The
main objective of the plan is to introduce ICT as part of the
daily tools of classroom activities, and at the same time it
aims at innovating teaching practices in Italian schools.
Impara Digitale is an association born in 2010 to promote
the development of an innovative teaching methodology,
which permits Italian schools to benefit from the introduction
of new technologies. The main purpose of the association is
the modeling of a teaching methodology for a school
embedded in the cloud-computing environment, through the
use of personal mobile technologies. Impara Digitale‘s main
activities are research, experimentation, sharing and diffusion
of findings, inside of a stable national network.
Schools can choose to become part of the association on a
voluntary basis. Those that adhere, receive in exchange a
number of services all centered around a “cloud learning”
model: sharing teaching pedagogies and learning resources
on the cloud to improve learning and to make it more
closer with the digital life experiences students experiment
everyday.
In 2010 a 2-year pilot experiment started in selected
classes of one Italian high school. The experiment rapidly
spread over the country and a network of 14 participating
institutions was gathered at the beginning of the school year
2012/13.
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
4. The initiative and the research settings
A report prepared for the U.S. Department of Education
(2011), “International Experiences with Technology in
Education”, shows that most countries are investing in ICT.
Given the low penetration of ICT in education compared to
most other OECD countries, in 2007 Italian Government
started the current national policy for large scale
introduction of ICT in all schools, namely the “National
Plan for Digital Schools”, in order to reduce the digital
divide of the school environment. The current policy marks
a clear discontinuity with previous national efforts, because
it aims at introducing the use of ICT directly in the everyday
classroom, rather than in separated computer labs, and it
transcends the disciplinary boundaries: it seeks ICT
adoption in all subject fields (Abdullah & Ward, 2016).
3. Enhancing learning experience through
technology adoption: the Italian case
found that subjective norm had an influence on intention
only in mandatory settings, but not in voluntary ones. The
result in TAM2 is that voluntariness is hypothesized to
moderate the effect of subjective norm on intention to use.
Also, passing through the concept of Internalization of
Social Influence, Venkatesh and Davis (2000) prove that
subjective norm will have a positive direct effect on
Perceived Usefulness and on Image. Also, Image will have a
positive and direct effect on Perceived Usefulness. For the
first time Experience is explicitly included into Technology
Acceptance Model, because studies had found that after
implementation, when more about the system was known by
the users through direct experience, the normative influence
subsided (Harwick and Barki 1994; Agarwal and Prasad
1997; supported by Ram and Jung 1991). So it could be
concluded that the direct effect of Subjective norm on
intention for mandatory systems would weaken with
increased experience, while the effect on perceived
usefulness would decrease in both mandatory and voluntary
settings. In their work, Venkatesh and Davis (2000) define
Job Relevance as “an individual’s perception regarding the
degree to which the target system is applicable to his or her
job” (p. 6), and they proved that this construct would have a
positive effect on perceived usefulness, as well as output
quality and result demonstrability. The rest of the TAM
hypothesis about perceived ease of use and perceived
usefulness remained intact. In 2003 the Unified Theory of
Acceptance and Use of Technology was proposed. In 2003,
Venkatesh, Morris, Davis and Davis presented a new model
that had the purpose of unifying the existing models
regarding technology acceptance so to obtain a unique
and powerful tool to assess this topic. The result was the
creation of UTAUT, a model found to outperform the eight
individual models that it concentrate in itself with a R2 of
69%. In order to have a better result Venkatesh, Thong and
Xu (2012) proposed a second version of the Unified Theory
of Acceptance and Use of Technology, the UTAUT2, that
also includes Hedonic Motivation, Price Value and
Experience and Habit.
Is Technology Mediated Learning Really Improving Performance Of Students

6
Each school proposed one or more of its high classes
(average size of 25 students): students were asked to buy
their own tablet – as substitute of textbooks – and bring it to
school every day. Teachers were trained to restructure
their teaching syllabus in order to leverage digital resources,
by accessing to (but not only) a centralized database of
certified public available sources on all subjects taught (i.e.
mathematics,
Italian literature, history, physics, chemistry, biology, music,
etc.). A constructivist learning approach was used to design
the whole learning calendar: students were asked to learn and
interact in teams and individually, supported by their
teachers. It is important to remark that in Italy the single class
is a strong organizational unit. In fact, the student group stays
the same not only throughout the day, but also over the whole
school cycle (5 year term for high school grade). Similarly,
the group of teachers follows the class throughout its entire
cycle. Regular tests were held along the school year – as with
traditional classes (text based learning) in the respective
institutions – and each student received grades and
feedbacks. Each school appointed a control sample, i.e. one
or more class unit with traditional teaching and learning
methods, using the same faculty body of the experimental
class. This allowed for a close comparison that controls for
teachers’ method and grades policy. While the resources and
tools are different, the studied contents are the same. After
the data cleaning, our valid dataset has an experimental
sample of 370 students of 21 classes in 9 different high
schools.
Each student participating to the study was profiled
anonymously (his/her identity was hidden with a numeric
code) and filled out an entry (beginning of the school year)
and an exit questionnaire (end of the school year).
Questionnaires were built around the TAM described earlier.
13 constructs were identified and every survey question is
linked to a construct’s measurement (2). The Questionnaire
counts a total of 62 questions, grouped into thirteen
constructs, expression of the variables of the TAM, plus a
social mapping section made by two additional questions.
Every school’s registrar provided the whole grade record (all
the subjects learned) for each student participating to the
study.
The main question of this work is whether new technologies
have a significant positive impact on students’ performance
or not. In order to find an answer, database construction has
been a fundamental step of the research project. A series of
datasets was necessary in order to analyze different aspects.
Schools sent two tables of grades for every student involved,
one for each quarter of the school year, containing every
single vote that the student received for each subject.
In the questionnaire, students were asked to answer questions
in a scale from 1 to 5 where 1 = “I definitely disagree”, 5 = “I
totally agree”.
The first operation was to transform negative answers in
positive ones in order to be able to compare all of them, for
example: “I do not plan to use much this technology during
the rest of the quarter”, was part of the construct “Intention to
use”, but it was posed in a negative form, so when the student
answered 5 = “I totally agree”, it meant exactly the opposite
in terms of intention to use. The single answers were grouped
into constructs, and a mean was calculated for every student
and construct, in a way that allows relating performance and
TAM variables. The following are our research hypothesis:
Hypothesis 1. Students perceive the technology is useful, but
they do not sense a comparative advantage in relationship to
books, unless they have an effective teachers encouragement,
that help them use the technology as a real tool for their
studies.
1a. Perceived usefulness has a positive significant effect on
students’ performance in term of total grades average.
1b. Perceived advantage (meaning comparative advantage of
the use of technology versus the
use of books) has a negative significant effect on students’
performance.
1c. Perceived teachers encouragement has a positive
significant effect on student’s performance.
Hypothesis 2. Classmates’ encouragement has a positive but
marginally significant effect on students’ performance.
Hypothesis 3. Perceived advantage and Satisfaction have a
positive significant effect on perceived usefulness.
Hypothesis 4. Top students do not perceive the technology
as useful, and they do not sense a comparative advantage in
relation to books. Teachers are still the main factor
influencing students’ performance.
4a. Perceived Usefulness has no significant effect on high
performing students
4b. High performing students do not perceive a comparative
advantage of technology in relation to
books.
4c. Teachers’ encouragement has a positive significant effect
on high performing students’ performance.
Hypothesis 5. Low performing students perceive the
technology is useful, but not better than books in comparative
terms. Teachers’ encouragement has the most significant
effect on performance, and previous experience has a positive
but marginally significant effect on performance.
5a. Perceived usefulness has a positive significant effect on
performance of bad students.
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
2 They are: 1) Perceived Usefulness of technology, 2) Perceived Ease of Use, 3) Attitude: Satisfaction, 4) Attitude: Preference, 5) Intention to use, 6)
Perceived Advantage of technology, 7) Perceived Teachers’ encouragement, 8) Perceived Classmates’ encouragement, 10) Awareness of true technology
potential, 11) Internet Access, 12) Technical Support, 13) Previous Experience with internet and computers, 14) Self Efficacy in the use of Internet
Leonardo Caporarello, Massimo Magni and Ferdinando Pennarola

7
5b. Perceived Advantage of technology has a negative
significant effect on performance of bad students.
5c. Teachers’ encouragement has a positive significant effect
on bad students’ performance.
5d. Previous experience in the use of technology has a
positive but marginally significant effect on bad students’
performance.
Hypothesis 6. Students that perceive a higher teachers
encouragement show a higher positive and more significant
effect of perceived usefulness on their performance, than
students that perceive a lower teachers encouragement.
Hypothesis 7. Intention to use has a positive significant
effect on students’ performance.
Also the second hypothesis is confirmed by this
regression: 2. Classmates’ encouragement has a positive
(coeff. = 0.160) but marginally significant (p-value = 0.075)
effect on students’ performance. Since Perceived Usefulness
appears to be a fundamental variable, we ran a regression
using it as dependent variable, with Ease of Use, Perceived
Advantage of technology, Satisfaction and Preference as
independent variables. Results are below:
Table 2
To test the first hypothesis we run a regression where
performance was the dependent variable, and constructs were
the independent ones. In particular we used all of the
constructs except: Intention to use3, Awareness of true
technology potential and Self-Efficacy in using the computer
and the Internet.
Results are shown in the table below:
Table 1
3The reason for exclusion of Intention to Use from the set of independent variables was that in TAM literature this construct takes the place of
dependent variable. Because of this, the impact of Intention to Use on performance is tested separately from other constructs.
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
5.Results
Thus, it can be inferred that each part of the first hypothesis
is confirmed: 1a. Perceived Usefulness has a positive (coeff.
= 0.261) significant (p-value = 0.019) effect on students’
performance.
1b. Perceived Advantage of Technology has a negative
(coeff. = -0.545) significant (p-value = 0.000) effect on
students’ performance.
1c. Teachers Encouragement has a positive (coeff. = 0.462)
significant (p-value = 0.000) effect on
students’ performance.
This regression confirms the third hypothesis:
3. Perceived Advantage has a positive (coeff. = 0.437) and
significant (p-value = 0.000) effect on
Perceived Usefulness, as well as Satisfaction (coeff. = 0.561,
p-value = 0.000).
In order to make a deeper analysis we divided the sample in
two groups using their annual grades
average:
- Top students are those whose average is equal or greater
than 7 on a scale from 1 to 10;
- Low performing students are those whose average is minor
than 7 on a scale from 1 to 10.
At this point we run two different regressions using these
two sub-samples. Results of the first one
are shown in the table below:
Table 3
Is Technology Mediated Learning Really Improving Performance Of Students

8
Only top students make the first sub-sample. These results
prove the fourth hypothesis to be true in each of its part:
4a. Perceived Usefulness has no significant effect (p-value =
0.182) on top students’ performance.
4b. Top Students do not perceive a comparative advantage of
technology in relationship with books (Advantage coeff. =
-0.490, p-value = 0.000).
4c. Teachers’ encouragement has a positive (coeff. = 0.247)
significant (p-value = 0.029) effect on top students’
performance.
We ran the same regression on the second sub-sample,
composed by low performing students:
Table 4
Results show that the fifth hypothesis holds true in each of
its part:
5a. Perceived usefulness has a positive (coeff. = 0.182)
significant (p-value = 0.047) effect on performance of bad
students.
5b. Perceived Advantage of technology has a negative (coeff.
= -0.182) significant (p-value = 0.042) effect on performance
of bad students.
5c. Teachers’ encouragement has a positive (coeff. = 0.218)
significant (p-value = 0.007) effect on bad students’
performance.
5d. Previous experience in the use of technology has a
positive (coeff. = 0.127) but marginally significant (p-value =
0.072) effect on bad students’ performance.
Hypothesis 5b means that low performing students perceive
the technology to be useful, but they do not feel a real
advantage of technology compared to books. The key role of
teachers’ encouragement is clearly confirmed.
Since teachers’ encouragement has been proven to be a key
variable in almost every analysis conducted until now, the
original sample was divided into two sub-sample on the base
of teachers’ encouragement perception:
- Students perceiving a high teachers encouragement, who
expressed a judgment equal or greater than 3.5 on a scale
from 1 to 5;
- Students perceiving a low teachers encouragement, who
expressed a judgment minor than 3.5 on a scale from 1 to 5.
We ran two separate regression, one for each sub sample.
Results appear in the tables below:
Table 5
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
Results show that Hypothesis 6. is confirmed: the first sub
sample show a higher impact of Perceived Usefulness (0.414
> 0.238) with a higher significance (p-value 0.068 < 0.079).
Also it’s worth to be pointed out that even if the second
group gives a grade of 3.5 to teachers’ encouragement, it still
remains a key determinant of students’ performance. In order
to go further with this analysis on teachers encouragement
we also made a One-Way ANOVA, using Perceived teachers
encouragement as discriminant to divide the sample in two
groups, and taking all of the constructs using until now as
dependent variables of the Analysis of Variance:
Table 6
Group 1 represents students who perceive a teachers
encouragement greater than 3.5, while group 2 is made by
students who perceive teachers encouragement to be minor
than 3.5. It can be easily noticed that there is a significant
difference in the judgments expressed on each of the key
variables of the TAM between the first and the second group
of students: those who perceived teachers’ encouragement to
be higher gave a higher grade to every other variable.
Leonardo Caporarello, Massimo Magni and Ferdinando Pennarola

9
This result proves the fundamental role played by
teachers’ encouragement on the perception that students
have of technology and, thus on their attitude toward it.
Finally, to test the impact of Intention to Use on Students’
performance, we ran a final regression, using Annual Grades
Average as dependent variable, and all of the constructs as
independent variables, including those that were eliminated
from the analysis at the beginning. The impact of
Intention to Use on the dependent variable resulted to be
positive and significant:
Table 7
This result proves the seventh hypothesis to hold true:
7. Intention to Use has a positive (coeff. = 0.208) and
significant (p-value = 0.048) effect on students’ performance.
This paper analyzed the impact of technology on high
school students’ performance. The attempt of it was to
understand whether technology is improving them or not, and
it can be concluded that one year of experimentation is not
sufficient to give a final answer to this question. Once the
technology absorption will be fully completed, another
investigation should test the same hypothesis and check the
validity of the model. It is evident that both teachers and
students have to adapt to the new teaching methodology that
technology requires them to use in order to be effective.
This work gives a contribution to the understanding and
application of technology-mediated learning. Consistent with
previous studies, we demonstrated that we couldn’t expect
technology per se to revolutionize teaching and learning.
Indeed, the effectiveness of technology is tied to the
organizational environment in which it is implemented and
on the characteristics of the users. Part of the students’
responses is to be ascribed to teachers: Italian school system
is very static, and resistant to change and reforms, and
teachers play a big role in this environment. It is not new or
unusual that teachers are the first actors of resistance toward
change in educational environments, and schools should take
advantage of the champions of change among the teaching
committee to involve everyone in the change (Bourgonjon et
al. 2013). Without this involvement students will never have
the support and the encouragement they need in the use of
technology at school, and this will lead to poor performance
improvement. Also, students should receive a better training
in the use of technology, because even if they are considered
“digital natives” truth is that they are used to employ
technology only in social and informal contexts, and the
result of this kind of usage is that they do not learn how to
fully take advantage of technology in their learning
processes.
The results of this research shows that teachers should act on
two sides: on one hand they should try engaging more the
students to understand the usefulness of technology in class,
and at the same time they should try listening more to
students and cooperating with them to meet their expectations
on how school should be, on their idea of school. Digital
natives are bored by the current system, where there is little
exchange of ideas between students and teachers, and where
the lesson is passive for them: traditional lectures do not
make sense anymore for a generation made of people that can
get all of the notions they are interested in from the internet
(Rienties et al. 2016). They have to be involved and truly
engaged and curious about the topics that schools want them
to learn about. Even if there is no evidence of significant
improvement in students’ performance with technology, we
all can see that the traditional method is not working anymore
for new students, and it has to be changed. Technology sure
can be helpful in this direction, but only if the actors of the
school environment are willing to embrace the change and be
an active part of it.
Furthermore, we cannot expect students to be enthusiastic of
change in schools, if they do not believe it can truly happen
on a bigger scale, and not only in isolated initiatives like the
one presented in this work.
Future research could focus on teachers’ training, their
teaching methods and the choice of resources. They are all
factors potentially determining the perceived
“Encouragement” as described in this paper. Finally, as the
consumerization of IT spreads mainly among young
generations, an interesting research opportunity would be to
explore the introduction of digital learning earlier at school.
Aware of the potential of such initiatives, some primary
schools are already experimenting education mediated by
tablet. This trajectory could be beneficial for the educational
institution on the advantages in terms of decision making and
learning processes that are tied to the introduction of systems
that support individuals in the sharing, managing and
exchange information.
Finally, our model should be also replicated in other cultures.
Indeed, previous studies outlined that acceptance of
technology and individuals’ behaviors in a technology
mediated environment may vary across cultures (e.g. Farrell,
2015).
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
5.Conclusions and further research
directions
References
[1] Abdullah, F., & Ward, R. (2016). Developing a General
Extended Technology Acceptance Model for E-Learning
(GETAMEL) by analyzing commonly used external factors.
Computers in Human Behavior, 56, 238-256.
[2] Agarwal, R. and Prasad, J. (1997). The Role of Innovation
Characteristics and Perceived Voluntariness in the
Acceptance of Information Technologies. Decision
Sciences ,28(3), pp. 557- 582.
Is Technology Mediated Learning Really Improving Performance Of Students

10
[3] Ajzen, I. (1991). The theory of Planned Behavior.
Organizational Behavior and Human Decision Processes,
50(2), pp. 179-211.
[4] Alavi, M and Leidner D.E. (2001). Technology-mediated
learning – a call for greater depth and breadth of research.
Information Systems Research, 12(1), pp. 1-10.
[5] Anderson, R.D. (1995). Curriculum reform: Dilemmas
and promise. Phil Delta Kappan, 77, pp. 33- 36.
[6] Anderson, R.D. and Helms, J.V. (2001). The ideal of
standards and the reality of schools: needed research. Journal
of Research in Science Teaching, 38(1), pp. 3-16.
[7] Avvisati, F., Hennessy, S., Kozma, R.B. and Vincent
Lancrin, S. (2013), Review of the Italian Strategy for Digital
Schools, OECD Education Working Papers No. 90, OECD
Publishing.
[8] Bandura, A. (1978). Reflections on Self-Efficacy.
Advances in Behavioral Research and Therapy, 1, pp.
237-269.
[9] Bandura, A. (1982). Self-Efficacy Mechanism in Human
Agency. American Psychologist, 37(2), pp. 122-147.
[10] Bandura, A. (1986). Social Foundations of Thought and
Action: A social cognitive theory, Prentice Hall, Englewood
Cliffs, NJ.
[11] Bandura, A., Adams, N.E. and Beyer, J. (1977).
Cognitive Processes Mediating Behavioral Change. Journal
of Personality and Social Psychology, 35(3), pp. 125-139.
59
[12] Bannert, M. (2002). Managing Cognitive Load: Recent
Trends in Cognitive Load Theory. Learning and Instruction,
12, pp. 139-146.
[13] Bernard, R.M., Abrami P.C., Lou, Y. and Borokhovski,
E. (2004). A methodological morass? How we can improve
quantitative research in distance education. Distance
Education, 25, pp. 175-198.
[14] Bernard, R.M., Abrami, P.C., Lou, Y., Borokhovski, E.,
Wade, A. and Wozney, L. (2004). How does distance
education compare with classroom instruction? A meta
analysis of empirical literature. Review of Educational
Research, 74(3), pp. 379-439.
[15] Bourgonjon, J., De Grove, F., De Smet, C., Van Looy,
J., Soetaert, R., & Valcke, M. (2013). Acceptance of game-
based learning by secondary school teachers. Computers &
Education, 67, 21-35.
[16] Brown, I. Jr., and Inouye, D.K. (1978). Learned
Helplessness Through Modeling: the Role of Perceived
Similarity in Competence. Journal of Personality and Social
Psychology, 36(8), pp. 900- 908.
[17] Campeau, D.R. and Higgins, C.A. (1991). A Social
Cognitive Theory Perspective on Individual Reactions to
Computing Technology. Proceedings of the 12th
International Conference on Information Systems, New York,
NY, pp. 187-198.
[18] Campeau, D.R. and Higgins, C.A. (1995). Application
of Social Cognitive Theory to Training for Computer Skills.
Information Systems Research, 6(2), pp. 118-143.
[19] Campeau, D.R., Higgins, C.A. (1995). Computer Self
Efficacy: Development of a Measure and Initial Test. MIS
Quarterly, 19(2), pp.189-211.
[20] Campeau, D.R., Higgins, C.A. and Huff, S. (1999).
Social Cognitive Theory and Individual Reactions to
Computing Technology: a Longitudinal study. MIS
Quarterly, 23(2), pp. 145-158.
[21] Cheung, R., & Vogel, D. (2013). Predicting user
acceptance of collaborative technologies: An extension of the
technology acceptance model for e-learning. Computers &
Education, 63, 160-175.
[22] Cuban, L. (2001). Oversold and Underused: Computers
in the Classroom. Cambridge, MA: Harvard University Press.
60
[23] Davis, F. (1993). User acceptance of information
technology: system characteristics, user perceptions and
behavioral impacts. Int. J. Man-Machine Studies, 38, pp.
475-487.
[24] Davis, F.D. (1985). A Technology Acceptance Model
for Empirically testing new end user Information Systems:
Theory and Results. Doctoral Dissertation, MIT Sloan School
of Management, Cambridge, MA.
[25] Davis, F.D. (1989). Perceived Usefulness, Perceived
Ease of Use, and User Acceptance of Information
Technology. MIS Quarterly, 13(3), pp. 319-340.
[26] Farrell, W. (2015). A Framework to Support Global
Corporate M-Learning: Learner Initiative and Technology
Acceptance across Cultures. International Association for
Development of the Information Society.
[27] Fishbein, M. and Ajzen, I. (1975). Belief, Attitude,
Intention and Behavior: an Introduction to Theory and
Research. Addison-Wesley, Reading, MA.
[28] Hartwick, J. and Barki, H. (1994). Explaining the Role
of User Participation in Information System Use.
Management Science, 40(4), pp.40-465.
[29] Karahanna, E. and Straub, D.W. (1999). The
Psychological Origins of Perceived Usefulness and Ease of
Use. Information and Management, 35(4), pp. 237-250.
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
Leonardo Caporarello, Massimo Magni and Ferdinando Pennarola

11
EAI Endorsed Transactions on
e-Learning
06 - 11 2016 | Volume 3 | Issue 12 | e5
EAI
European Alliance
for Innovation
[30] Karahanna, E., Straub, D.W. and Chervany, N.L. (1999).
Information Technology Adoption across time: a Cross-
Sectional Comparison of Pre-Adoption and Post-Adoption
Beliefs. MIS Quarterly, 23(2), pp. 183-213.
[31] Keys, C.W. and Bryan, L.A. (2001). Co-constructing
inquiry-based science with teachers: Essential research for
lasting reform. Journal of Research in Science Teaching,
38(6), pp. 631-645.
[32] Lim, K.H., Benbasat, I. and Ward, L.M. (2000). The role
of multimedia in changing first impression bias. Information
Systems Research, 11(2), pp. 115-136. 61
[33] Moore, G.C., and Benbasat, I. (1991). Development of an
Instrument to Measure the Perceptions of Adopting an
Information Technology Innovation. Information Systems
Research, 2(3), pp. 192-222.
[34] Nicholson, J.A., Nicholson, D.B. and Valacich, J.S.
(2008). Examining the Effects of Technology Attributes on
Learning: a Contingency Perspective. Journal of Information
Technology Education, 7, pp. 185-204.
[35] OECD (2010), The Nature of Learning, OECD
Publishing.
[36] Papert, S. (1987). Computer criticism vs. Technocentric
thinking. Educational Researcher, 16(1), pp. 22-30.
[37] Radner, R. and Rothschild, M. (1975). On the allocation
of effort. Journal of Economic Theory, 10, pp. 358-376.
[38] Rienties, B., Giesbers, B., Lygo-Baker, S., Ma, H. W. S.,
& Rees, R. (2016). Why some teachers easily learn to use a
new virtual learning environment: a technology acceptance
perspective. Interactive Learning Environments, 24(3),
539-552.
[39] Robey, D. (1979). User attitudes and management
information system use. Academy of Management Journal,
22, pp. 527-538
[40] Rogers, E.M. (1962). Diffusion of Innovation, Free
press, Glencoe.
[41] Rogers, E.M. (1983). Diffusion of Innovation. Third
Edn., Free Press, New York, NY.
[42] Schultz, R.L. and Slevin, D.P. (1975). Implementation
and organizational validity: an empirical investigation.
Implementing Operations Research / Management Science,
R.L. Schultz and D.P. Slevin (eds.), American Elsevier, New
York, NY, pp.153-182.
[43] Schunk, D.H. (1981). Modeling and Attributional effects
on children’s achievement: a Self-Efficacy
[44] Analysis. Journal of Educational Psychology, 73,
pp.93-105.
[45] Taylor, S. and Todd, P.A. (1995a). Assessing IT Usage:
the Role of Prior Experience. MIS Quarterly, 19(2), pp.
561-570. 62
[46] Taylor, S. and Todd, P.A. (1995b). Understanding
Information Technology Usage: a test of competing models.
Information Systems Research, 6(4), pp. 144-176.
[47] Thompson, R.L., Higgins, C.A., and Howell, J.M. (1991).
Influence of Experience on Personal Computer Utilization:
Testing a Conceptual Model. MIS Quarterly, 15(1), pp.
124-143.
[48] Tornatzky, L.G. and Klein, K.J. (1982). Innovation
Characteristics and Innovation Adoption- Implementation: a
meta-analysis of findings. IEE Transaction of Engineering
Management, EM 29(1), pp. 28-45.
[49] Triandis, H.C. (1977). Interpersonal Behavior, Brooke
Cole, Monterey, CA. U.S. Department of Education, Office of
Educational Technology (2011), International Experiences
with Technology in Education: Final Report, Washington, DC.
[50] Venkatesh, V. and Davis, F.D. (2000). A theoretical
Extension of the Technology Acceptance Model: four
Longitudinal field Studies. Management Science, 46(2), pp.
186-204.
[51] Venkatesh, V., Morris, M.G., Davis, G.B. and Davis, F.D.
(2003). User Acceptance of Information Technology: Toward
a Unified View. MIS Quarterly, 27(3), pp. 425-478.
[52] Venkatesh, V., Thong, Y.L. and Xu X. (2012). Consumer
Acceptance and Use of Information Technology: Extending
the Unified Theory of Acceptance and Use of Technology.
MIS Quarterly, 36(1), pp. 157-178.
[53] White, B.Y. and Frederiksen, J.R. (1990). Casual Model
Progressions as a Foundation for Intelligent Learning
Environments. Artificial Intelligence, 42, pp. 99-157.
[54] White, B.Y. and Frederiksen, J.R. (1998). Inquiry,
Modeling and Metacognition: Making science accessible to all
students. Cognition and Instruction, 16, pp. 3-118. 63
[55] Wood, R. (1986). Task Complexity: Definition of the
construct. Organizational Behavior and Human Decision
Processes, 37(1), pp. 60-82.
[56] Wood, R. and Bandura, A. (1989). Social Cognitive
Theory of Organizational Management. Academy of
Management Review, 14(3), pp.361-384.
Is Technology Mediated Learning Really Improving Performance Of Students