Content uploaded by Carlo Giovannella
Author content
All content in this area was uploaded by Carlo Giovannella on Jul 22, 2020
Content may be subject to copyright.
Measuring the effect of the Covid-19 pandemic on the
Italian Learning Ecosystems at the steady state: a school
teachers’ perspective
Carlo Giovannella1,2, Marcello Passarelli1,3, Donatella Persico1,3
1ASLERD (https://en.wikipedia.org/wiki/ASLERD)
2Dept. SPFS, University of Rome Tor Vergata, Rome, Italy
CNR – Institute of Educational Technologies, Genoa, Italy
Abstract. To the best of our knowledge, this is the first study conducted on the
Italian school system to capture the teachers’ perspective, experiences and
perceptions about education two months after the beginning of the COVID-19
pandemic lockdown, when the on-line educational processes were fully in
place and reached their steady state. The paper reports a descriptive analysis
integrated by a network analysis and a search for causal relationships among
the variables that have been investigated. Respondents reported adequate
reactions of the institutions and individual teachers prevented the collapse of
the educational system in spite of loss of contact with 6-10% of the students
and a significant increase of the workload that caused individual time
management challenges. Although the main teaching strategies adopted by
teachers reproduced standard classroom dynamics, the possibility to operate in
the comfort zone generated a positive feeling about using technologies, a
perception of increased digital skills ownership and a change in the mental
setting about educational processes. In turn, this generated an increase in the
perceived sustainability of on-line education and, in about a third of the
teachers, the wish to adopt a blended configuration for future teaching
activities. Almost all participants recognized the relevance of a digital
pedagogy and the need to include it in the training curricula to prepare future
teachers.
Keywords: COVID-19 pandemic, on-line learning, emergency remote
education (ERE), learning ecosystems’ reaction, school teachers, future
perspectives, descriptive analysis, causal discovery
1. Introduction
In the first half of 2020, almost all educational ecosystems (schools, universities,
private centres) around the globe were forced to cancel face-to-face (f2f) classes [1]
as a non-pharmaceutical intervention to contain the spread of the COVID-19
pandemic. In these circumstances, shifting courses from f2f to online was a policy
response mandated by the compelling need to keep teachers, staff, students and
society as safe as possible in the face of a public health emergency whose spread was
unexpected, exceptionally fast and poorly understood. In April 2020 the lockdown
was almost complete and affected more than 90% of the worldwide students, almost
200 countries and more than 1,5 billions of students [2]. Mass school closure was
simply a quick fix adopted in “less-than-ideal circumstances” [3]. The pressing haste
with which many educational institutions moved to online education may have
prevented them from harnessing its strengths and facing its limitations. This is
probably going to be a theme of investigation for many scholars and for several years
to come. The public debate about the way schools (as well as universities) reacted to
the emergency has proliferated in the grey-literature, media and on social networks
(e.g., [4, 15, 16]. “The temptation to compare online learning to face-to-face
instruction in these circumstances” [3] has brought about negative considerations
along the lines that online learning is no substitute for the “real thing.” At the same
time, it has also generated (perhaps over-) optimistic expectations that after this “great
online learning experiment” [5] our educational institutions and their teaching staff
will be readier than ever to move to online or blended learning once and for all. In
any case, such a dramatic emergency can be regarded as a catalyst for change and an
opportunity to reflect on the nature of the educational ecosystems (places, processes,
contents, competences, etc.) [6-8]. Among the positive outcomes, authors [16, 18]
refer to increased teacher awareness of the technology affordances for learning,
including better chances for providing personalised feedback, more intensive sharing
of challenges and solutions adopted and consequent participation in professional
communities, not to mention all the pre-COVID research results in support of online
learning in general [19]. The pandemic has also shed light on some undesirable effects
of ICT uptake on a global scale to counter those of the physical distancing. Primarily,
inequalities in access to education due to social, economical, personal and family
conditions, not merely to lack of digital connectivity: “Social, educational, health and
digital inequalities have never been clearer” [9]. In addition, virtual contact is often
seen only as a surrogate of social contact between students, especially for young
children [15,16]. The need for parental support has increased, particularly for students
with poor self-regulated learning skills, which in turn reinforces the inequalities [18].
Online fatigue and emotional wellbeing of all the actors concerned are also mentioned
by many. The whole experience, and the evidence emerging from it, should thus lead
to informed decisions concerning appropriate educational policies [6-8, 9]. The aim
is not only improving readiness and effectiveness in coping with possible future
emergencies in a sustainable manner [18] but also preparing teachers and students
alike to harness the potential of smart learning environments.
As we write, i.e. July 2020, scientific literature and research evidence about the
impact of the Emergency on schools are still limited, and for understandable reasons
they provide a rather scattered picture of the situation. Surveys addressing teachers
have been realized in India [14], Vietnam [10] and Massachusetts [20], with different
aims. In particular, the second study reports on a survey conducted among
Vietnamese teachers in conditions very similar to the present work: about two months
after the national school suspension during a 6 days period. Besides the use of
surveys, other research methods have been used to collect data about the challenges
faced by teachers during the emergency: for example, reference [18] analysed Twitter
Hashtags to understand how teachers built collective knowledge, seeked emotional
support, and designed their teaching by interacting with others in professional
communities. In this regard, the emergency situation may have improved teachers’
behaviour, as usually content sharing and community building are neglected by
teachers [39,40]. Last but not least, a few studies concerning similar situations in the
past (such as the 2003 SARS epidemic and the 2009 schools closure for influenza
pandemic) provide evidence of the non uniqueness of this situation [11-12, 21].
It will only be in the coming months and years that scholarship will be able to
fully analyse the educational, policy and societal implications of this emergency.
Nonetheless, we believe that this study is timely in analysing the outcomes of a survey
aiming to provide a broad picture concerning a range of variables related to school
settings, operational conditions, educational activities carried out during the
pandemic, teachers’ perceptions about the impact of this unprecedented experience
on their work and their mental setting with respect to technologies and their future
commitment with on-line learning.
The study is intended to provide an early contribution to the understanding of what
has been school education during the pandemic, an historical documentation, a point
of reference for future similar studies and, hopefully, and a first step towards a
collective reflection on possible developments of our educational system in view of
fostering policy decision making.
2. Experimental setting
2.1 Questionnaire
The survey was carried out through a questionnaire comprising 3 sections, for a total
of 80 questions. The first section includes 6 socio-biographical background questions
(gender, age, school level, school curriculum and teaching subject, geographical
location); the second includes 43 questions (20 of which are open questions or
requests of explicative comments) and is aimed at investigating the perceived
response of the learning ecosystem to the pandemic and the operating conditions at
what we consider the “steady state” of lockdown measures (i.e. after about two
months from the beginning of the schools lockdown); the third section includes 31
questions (17 of which are open questions or requests of explicative comments)
investigating the changes in teachers’ opinions about technologies and on-line
learning and their expectations for the future.
In this paper we focus on the analysis of the answers given to multiple choice or
quantitative questions that allow us to provide a snapshot of the situation and
investigate the teachers’ perception about the capability of the learning ecosystems to
react, the operational conditions and the type of educational activities carried out
(variables listed in Table 1). We also investigate which of these variables could have
modified their perception about technologies and expectations for the future
(variables listed in Table 2).
To the best of our knowledge, this is the first nation-wide study to investigate the
effect of the pandemic on teachers’ perception about on-line learning. Our study was
built on the ground of previous experiences – descriptive investigations - conducted
by one of the authors with a sample of university students [8] and a couple of high
schools [13]. Thus, the need for a bespoke questionnaire and of a research method
intended to shed light on the network of relationships that connect the variables listed
in Tables 1 and 2.
2.1 Participants
Participants have been contacted by email or by announcing the survey on social
media. Facebook turned out to be the more effective dissemination channel. We
announced the call for participation in more than 30 teachers groups, with a total of
about 60000 (non-unique) subscribers. Since we had the goal of collecting a snapshot
of the Italian situation after about two months since the beginning of the schools
lockdown (March 5th, 2020), the survey was only open from May 13th, 2020 to May
24th, 2020. Before closing the survey we checked that the sample would be
representative of Italian teachers. The survey was completed by 336 teachers (306
females, 29 males, 1 non-binary) employed in primary (142), lower secondary (84)
or upper secondary (110) schools. Considering macroregional area, the survey was
completed by 142 North Italy teachers, 97 Central Italy teachers, and 113 South Italy
and Italian islands teachers.
We detected a slight gender unbalance (91% females in the sample, compared to
83% in the target population, p < .001), and a slight unbalance in geographical
distribution (38%, 29%, 34% for North, Centre, and South Italy, respectively,
compared to 40%, 22%, and 38% for the population [23]; p = .009)., However, the
sample seemed representative of the population age (49.10 vs 48.90, p = .684) [22]
and school level (p = .118). School level and teacher gender were associated (χ2(2) =
18.89, p < .001), as there were significantly more males employed in upper secondary
schools (18.2% of upper secondary school teachers, vs. 4.7% for lower secondary and
3.5% for primary school). Geographical zone was associated with neither gender
(χ2(2) = 1.06, p = .590) nor school level (χ2(4) = 3.46, p = .484).
Fig. 1. Percentage of non-missing responses to multiple choices and quantitative questions as
function of the question ID.
0
20
40
60
80
100
1 6 11 16 21 26 31 36
Percentage of
answers (%)
As additional control we have measured the fatiguing effect induced by the length
of the questionnaire and it turned out to be very low, with a missing rate - in the case
of multiple choice and numerical questions - of less than 5% even towards the end of
the survey (see Fig. 1A).
3. Results
In order to explore teachers’ feelings and opinions as well as the complex network of
relationships that connect the variables investigated by sections II and III of the
questionnaire, we pursued multiple strategies. First, we carried out descriptive and
univariate analyses (section 3.1), exploring the observed distributions of the variables
being considered (Tables 1 and 2) of this section. Then we fitted multiple linear
regression models to explore which variables would better predict key outcomes
(section 3.2.1). Subsequently, to obtain a birds’ eye view of the variables’ relations,
we employed the paradigm of network analysis for visualizing the partialized
correlations between variables (section 3.2.2), and infer the direction of causality for
some of these associations (section 3.2.3).
3.1 Descriptive and univariate analyses
Technological context. More than 92% of the teachers report having needed less than
two weeks to adapt /get used to on-line education and this confirms that, with the
exclusion of the 8% that still didn’t feel comfortable with it, for all the rest the
operational conditions photographed by the survey should be considered “steady
state” ones. This conclusion is in line with results of a survey [27] conducted two
months after the lockdown with the principals of the Irish primary schools compared
with those of a survey conducted two weeks after the lookdown.
More than 86% of the Italian teachers used a laptop to connect and carry on their
didactic activities. This is not surprising, since the lockdown has strongly reduced the
personal mobility and, thus, the usefulness of smartphones (see also ref. [8]) that,
nevertheless, were still used - possibly in parallel - by 40% of the teachers. About
12% of the respondents used the smartphone also to connect to the internet. Almost
35% of teachers used a tablet and about 22% a desktop computer. Less than half of
the teachers, 44%, had a wide or an ultrawide band access to the internet, 36%
accessed using an ADSL connection while the rest relied on smartphones, satellite
connections or other technologies. 12% lamented a lack or limited availability of
devices adequate to carry on the on-line activities, while more than 36% complained
about insufficient bandwidth and 8% about limited traffic allowed by their internet
providers. According to the teachers, students also experienced similar issues, which
prevented their partial or full participation in educational activities. 10% of the
teachers declared to have lost contact with 20% of the students or more, 20% to have
lost the contacts with a proportion of students between 5% and 20%, while 45% lost
the contact with less than 5% of the students and about 25% with none. The proportion
Table 1. Survey Section II: teachers’ perception about the capability of the learning ecosystems
to react, the operational conditions and the features of the educational activities carried out.
Variable
Average
t-test
Difference between
school levels
School Readiness to
swap to on-line
education (SR)
M = 6.23
[5.98, 6.48]
t(335) = 5.83, p <
.001, Cohen’s d = .32
F(2, 333) = 3.45, p =
.032, R2 = .01; higher
for upper secondary
Technological
Adequacy of On-line
Environments (TAOE)
M = 6.36
[6.10, 6.62],
t(334) = 6.47, p <
.001, Cohen’s d = .35
F(2, 332) = 1.62, p =
.200, R2 < .01
Digital Safety of
technological
environments (DS)
M = 6.52
[6.26, 6.78]
t(330) = 7.58, p <
.001, Cohen’s d = .42
F(2, 328) = 2.24, p =
.108, R2 < .01
Teachers’
Technological
Readiness (TTR)
M = 5.93
[5.72, 6.14]
t(332) = 4.06, p <
.001, Cohen’s d = .22
F(2, 330) = 2.79, p =
.063, R2 = .01
Teachers’ Pedagogical
Readiness (TPR)
M = 5.85
[5.65, 6.05]
t(333) = 3.41, p <
.001, Cohen’s d = .19
F(2, 331) = 2.89, p =
.057, R2 = .01
Workload Increase (WI)
%, tested against the
baseline of 0
M = .65 [.63,
.68]
t(335) = 45.2, p <
.001, Cohen’s d =
2.47
F(2, 333) = 5.35, p =
.005, R2 = .03, lower
for primary schools
Teachers’ Time
Management Capacity
(TTMC) (scale -5, +5)
M = -.43 [-.74,
-.12]
t(335) = -2.75, p =
.006, Cohen’s d = .15
F(2, 333) = 1.01, p =
.364, R2 < .01
Students’ Time
Management Capacity
(STMC) (scale -5, +5)
M = -.67 [-.95,
-.40]
t(331) = -4.82, p <
.001, Cohen’s d = .26
F(2, 329) = 5.00, p =
.007, R2 = .02, lower
for primary schools,
higher for upper
secondary
Educational Activity:
Lecture-Discussion
(EALD) (scale -5, +5)
M = .37 [.13,
.60]
t(335) = 3.11, p =
.002, Cohen’s d = .17
F(2, 333) = 4.37, p =
.013, R2 = .02, higher
for primary schools
Educational Activity:
Transmissive-
Interactive (EATI)
(scale -5, +5)
M = 1.06 [.81,
1.31]
t(334) = 8.43, p <
.001, Cohen’s d = .46
F(2, 332) = 2.05, p =
.130, R2 < .01
Educational Activity:
Asynchronous-
Synchronous (EAAS)
(scale -5, +5)
M = .85 [.58,
1.12]
t(334) = 6.22, p =
.002, Cohen’s d = .34
F(2, 332) = 7.61, p <
.001, R2 = .04, higher
for upper secondary
Educational Activity:
Individual-
Collaborative (EAIC)
(scale -5, +5)
M = -.36 [-.67,
-.05]
t(334) = -2.26, p =
.024, Cohen’s d = .12
F(2, 332) = .23, p =
.796, R2 < .01
Reproducibility of
Classroom Dynamics
(RCD)
M = 5.32
[5.08, 5.57]
t(331) = 5.32, p =
.151, Cohen’s d = .08
F(2, 329) = 6.14, p =
.002, R2 = .03, higher
for upper secondary
is not dependent on school level (Ꭓ2(10) = 12.70, p = .241). From these data, one can
estimate an average dispersion to be ranging between 6% and 10% that corresponds,
nationwide, to 400K-670K students.
We may reasonably expect that such “infrastructural” criticalities may have
affected the quality of education and caused, in some cases, a significant divide.
Although documented in less details, similar and heavier difficulties have been
detected also in [10] and especially in [14].
Table 2. Survey Section III: teachers’ perception about technologies and their expectations for
the future
Variable
Average
t-test
Difference between
school levels
Sustainability of On-
line Education (SOE)
M = 5.17
[4.93, 5.42]
t(329) = -2.63, p =
.009, Cohen’s d = .14
F(2, 327) = .41, p =
.664, R2 < .01
Change in the Idea of
Educational Experience
(CIEE)
M = 5.18
[4.89, 5.47]
t(319) = -2.17, p =
.030, Cohen’s d = .12
F(2, 317) = 1.39, p =
.250, R2 < .01
Improvement in the
Feeling towards
Technologies (IFT)*
M = 6.30
[6.01, 6.59]
t(329) = 5.45, p <
.001, Cohen’s d = .30
F(2, 327) = 5.06, p =
.007, R2 = .02, higher
for primary school,
lower for upper
secondary
Improvement in
Technological Skills
(ITS)
M = 6.88
[6.63, 7.12]
t(328) = 10.85, p <
.001, Cohen’s d = .60
F(2, 326) = 7.19, p <
.001, R2 = .03, higher
for primary school,
lower for upper
secondary
Intention to Work in
On-line Learning
(IWOL)
M = 5.14
[4.83, 5.46]
t(324) = -2.24, p =
.026, Cohen’s d = .12
F(2, 322) = .45, p =
.639, R2 < .01
Relevance of Teacher
Education in Digital
Pedagogy (REDP)
M = 8.04
[7.81, 8.27]
t(322) = 21.79, p <
.001, Cohen’s d = 1.21
F(2, 320) = 1.21, p =
.300, R2 < .01
Extent to which
schools should Rely on
On-line Learning
(SROL)
M = 5.22
[4.96, 5.48]
t(323) = -2.08, p =
.038, Cohen’s d = .12
F(2, 321) = .23, p =
.798, R2 < .01
Degree of School e-
Maturity (SeM)
M = 6.36
[6.13, 6.59]
t(324) = 7.33, p <
.001, Cohen’s d = .41
F(2, 322) = 1.98, p =
.140, R2 < .01
Readiness of the learning ecosystems. To investigate a) the capability of the
learning ecosystems to react to the epidemic and b) the details of the operational
conditions that have been put in place we employed univariate analyses. For Likert-
type response scales, we carried out one-sample t-tests against the midpoint of the
scale (5.5 for 10-point scales). Results are reported in Tables 1 and 2.
As shown in Table 1, we observed relatively high levels of perceived: a) readiness
(SR) of the schools to swap from f2f to on-line didactics (SP); b) technological
adequacy of the on-line environments (TAOE); c) teachers’ technological readiness
(TTR); teachers’ pedagogical readiness (TPR). Similar results have been observed
also in [10] and partially in [14].
Fig. 2. Time spent per day on-line by teachers to support and deliver distance learning
Fig. 3. Overall teachers’ workload per day to support and deliver distance learning
Workload and time organization. Beside such a positive impression, the shift to on-
line education determined a substantial perceived increase of the workload (estimated
around 65% more than the usual), as illustrated by fig. 2. and fig. 3. The increase of
2%
5%
19%
44%
30%
On-line activities per day
less than 30'
30' - 1h
1h - 2h
2h - 4h
more than 4h
1%
1%
3%
13%
23%
30%
29%
Workload related to on-line activities per day
less than 30'
30' - 1h
1h - 2h
2h - 4h
4h - 6h
6h - 8h
more than 8h
the workload generated by on-line education during the pandemic aligns well with
previous results [10, 14], although in [10] it seems smaller than in Italy.
In the teachers’ opinion these operational conditions induce a lower self-reported
capacity to manage their own time with respect to the pre-COVID outbreak
conditions, see (TTMC) in Table 1. The same effect holds true when teachers evaluate
their students’ time management capability, see (STMC) in Table 1. It is worthwhile
to note that in the case study conducted recently on two Rome’s high-schools the
teachers’ opinion on (TTMC) resulted to be more or less confirmed, while - in
disagreement with the teachers’ opinion - both students and parents think that
(STMC) improved.
Teaching activities. We asked teachers to rate the teaching activities they carried on
during the lockdown along four axes: lessons vs. discussions (EALD in Table 1),
transmission vs. interaction (EATI in Table 1), asynchronous vs. synchronous (EAAS
in Table 1), and individual vs. collaborative (EAIC in Table 1) - all scales ranging
from -5 to +5. As shown in Table 1, teachers deemed their didactic activities to be
more discussion-based (M = .37 [.13, .60], interactive (M = 1.06 [.81, 1.31]),
synchronous (M = .85 [.58, 1.12]), and directed to individuals (M = -.36 [-.67, -.05]).
The above results can be well justified by the attempt to reproduce classroom
dynamics. In fact, looking at fig. 4, one realizes that about 88% of the teachers
delivered synchronous video-lectures, 82% assigned homework to be realized mainly
individually and 53% organized synchronous homework correction. Only 27% of
teachers organized synchronous team working and less than 20% collaborative
asynchronous activities, possibly because collaborative work is deemed to increase
the workload. This despite the amount of research studies devoted to the potential of
online collaborative learning based on asynchronous communication, and the efforts
devoted to train teachers to design this kind of activities [30]. It appears that teachers,
due to the lack of time, tried. More in general, only 12% attempted to organize more
innovative activities and this may indicate either a limited technological or
pedagogical preparedness or the intention to minimize the effort and time needed to
design new activities suitable for the new setting and go beyond very traditional and
often transmissive activities. Similarly, if we consider assessment modes (fig. 5), we
can see that individual assignments, on-line tests, and synchronous oral interviews
are overwhelmingly represented. Collaborative and group assignments were used by
less than 20% of respondents. Coherently, technologies (fig. 6) have been largely
employed to produce (76%) and share (87%) contents, assign homework (82%),
deliver transmissive lectures – e.g. videos – (56%), organize synchronous classroom
exercises (62%). The level of personalization of the teaching activities was fairly high
(52%) – student side - while the use of technologies to diversify the didactic
approaches – didactic and pedagogical side - was lower than what one can expect
(41%). Additionally, only one third of the teachers employed the technological
environment to plan (39%) and manage (33%) the educational processes, and this
may indicate a tendency towards a spontaneous/uncoordinated organization and
delivery of the didactic activities. Another use of the technological environment that
seems to be rarely employed is fostering socialization (19%). However, this datum
should not lead to the conclusion that socialisation between students did not take place
altogether, as research evidence concerning social media use alongside formal
learning processes abounds in the literature, leading us to believe that social media
might have served this purpose [28,29].
Fig. 4. Percentage of teachers that adopted the listed typologies of didactic activities
Fig. 5. Percentage of teachers that adopted the listed typologies of assessment methodologies
As for the difficulties faced by the teachers during this on-line experience (fig. 7),
the main one (reported by 40% of respondents) concerns the expressive modalities,
D_SC, that are felt as very limited with respect to a f2f interaction, D_LE.
020 40 60 80 100
production and delivery video lectures
production and delivery audio lectures
synchronous video lectures
assign video to watch
assign materials to read
assign exercise to do
assign on-line test
synchronous working groups
collaborative asyncronous activities
synchronous exercise correction
others activities
010 20 30 40 50 60 70 80
on-line oral interview
on-line test
individal home assignment
individal home assignment with oral
presentation
group home assignment
group home assignment with oral
presentation
others assessment mehodologies
Fig. 6 Teachers’ purposes in using technologies (%).
Fig. 7. Difficulties faced by teachers (%).
The second in the ranking, as already mentioned above, is the limited bandwidth
of the internet connectivity, D_LC (36%). Apparently, the lack of technological skills
is not felt as a relevant problem by 90% of the teachers, at least when implementing
the educational strategies described in the previous section, but 28% met some
020 40 60 80 100
Content production
Content sharing
Trasmissive lectures
Interactive lectures
Assign homeworks
Organize synchronous exercitations
Synch & asynch communication
Diversify the didactic methodologies
Didactic personalization
Collaboration and team working
Foster socialization
Plan didactic activities
Manage didactic activities
Student assessment
Student self-assessment
Others
0 5 10 15 20 25 30 35 40 45
Lack of an adequate device
Limited bandwidth
Limited internet traffic
Inadequate home environments
Limited technological skills
Get habit to the tech environments
Lack of a blackboard
Multiple technological environments
Too complex procedure
Lack of technical assistence
GDPR
Didactic materials finding
Communication with students
Limited expressive modalities
Diffculty to concentrate
Discomfort to use videocamera
Others
difficulties to get used to the novel technological environments, D_HT, and 14%
reported difficulties due to using multiple environments (including tools and apps),
D_MT and the lack of technical assistance. Similar percentages have been found in
[14]. It is interesting to note that in Italy 17% missed having a blackboard, D_MB.
Another notable aspect is the difficulty that 17% of them met due to inadequate
home environments, D_IHE, that may also generate a lack in concentration (12%),
while 13% felt a certain discomfort in using the video-camera. The percentage that
met family problems (17%) turned out to be much lower than in [14], where it was
53%. Another aspect that is interesting to highlight is the limited problems caused by
the GDPR (16%). In normal conditions and in the case of a strict respect of the GDPR
the delivery of most of the activities carried on during the pandemic wouldn’t be
possible, especially in consideration of the fact that almost all students involved were
minors (in Italy, younger than 18 years old). This indicates that most of the barriers
generated by GDPR are not considered very relevant and people could easily agree
to bypass them, at least in emergency conditions. Very different from the situation in
[14] where 74% of the teachers showed concern about privacy issues.
A look into the future. Since at the time of the data collection the operational
conditions could be considered “steady state” ones, we tried also to stimulate a first
reflection on a possible future. Interestingly, we observed (Table 2) a relevant
improvement in the feeling with Technologies (IFT)* that may be related to the
improvement in the technological skills (ITS) and a general agreement on the need to
train present and future teachers to digital pedagogy (REDP). Similar opinions have
been detected also in [10 and 14]. On the other hand, we also observed a rather
multifaceted opinion on the intention to work with on-line learning (IWOL), on how
much the school should rely on on-line learning activities (SROL), on the change in
ideas about educational processes (CIEE) and, overall, on the perceived sustainability
of the on-line learning (SOE). The discrepancy in opinions among different groups of
teachers is evident in the wide range and high dispersion of responses to these
variables.
Despite the above contrasting opinions, all in all, the on-line experience forced by the
pandemic outbreak seems to have induced a quite positive opinion on the e-maturity
of the schools (SeM in Table 2), that is a complex construct composed not only by
the quality and adequacy of the technological settings and by the available digital
competencies but also, among the other variables, by the effectiveness in the
management of the digital environments and learning processes thereof and by the
vision about the development of the digital setting [26].
Finally, the preference for future teaching modality is largely in favour of the f2f
one (66%) but a consistent number of teachers (32%) would prefer, and feel ready, to
continue in blended configuration. This latter is a quite high percentage with respect
to what we suspect could have been observed during the pre-COVID time. Although
we cannot make a direct comparison with pre-emergency data, in 2019 the number
of Italian teachers registered in the eTwinning community is 70000, i.e. less than 10%
of all Italian Teachers, and this corresponds more or less, to the percentage of those
that are commonly considered innovative teachers [24,25]). As expected, the
preference for the blended configuration, with respect to the average reported in fig.
8, decreases among the teachers of the primary schools and increases among those of
the secondary and high schools.
Fig. 8. The teachers’ future preferred teaching modality.
The scenario and the data described above, however, do not allow us to identify
clearly the possible relationships among the investigated variables (Tables 1 and 2),
neither their possible causal dependence. In the next sections we will try to shed light
on this aspect and complete the answer to our research question.
3.2. Prediction, correlation and causality
Linear regression models. Our exploration of variables associations started with
standard linear regression models, with the objective of identifying the variables that
would better predict what we consider to be key outcomes. The variables we tried to
predict are the intention to be involved in on-line education in the future (IWOL), the
belief that technology-based education is sustainable (SOE), and the change in the
idea of educational experience (CIEE). The main predictors tentatively considered
were age, location (North / Centre / South Italy), school level, perceived school
technological readiness, the eight difficulties reported most often (i.e. difficulty in
adapting to new tools and environments, using too many new tools, having an
inadequate working environment, having limited connectivity/bandwidth, limited
expression modalities, difficulty in communicating, missing having a blackboard, and
difficulties with GDPR), the four axes of proposed activities, self-reported change in
time management capacity, and estimated change in time management capacity
among students.
66%
32%
2%
Future preferred teaching modality
f2f
blended
on-line
Regarding intention to be involved in distance education in the future (Adjusted
R2 = .25), the main predictors seem to be change in time management capacity, TTMC
(b = .20, t(299) = 3.74, p < .001) and change in students’ time management capacity,
STMC (b = .24, t(299) = 3.82, p < .001), followed by perceived school technological
readiness, TAOE (b = .22, t(299) = 3.34, p < .001), difficulty in getting used to the
new tools and environments (b = -.90, t(299) = -2.49, p = .013), and teaching in upper
secondary schools (b = -.89, t(299) = -2.47, p = .014).
Regarding sustainability of technology-based education (Adjusted R2 = .23), this
belief seems to mainly be predicted by self-reported change in time management
capacity (b = .13, t(304) = 3.11, p = .002), students’ change in time management
capacity (b = .19, t(304) = 3.83, p < .001), perceived school technological readiness
(b = .21, t(304) = 4.10, p < .001), difficulty in getting used to the new tools and
environments D_HT (b = -.57, t(304) = -2.00, p = .046), and having reported about
an inadequate home working environment, D_IHE (b = -.74, t(304) = -2.22, p = .027).
Lastly, change in pedagogical ideas (Adjusted R2 = .10) is only predicted by self-
reported change in time management capacity (b = .12, t(296) = 2.30, p = .022),
change in students’ time management capacity (b = .13, t(296) = 2.05, p = .042), and
teacher age (b = -.04, t(296) = -2.43, p = .016).
Partialized correlations. While multiple linear regression can help us understand
which variables seem to be the best predictors of specific outcomes, the complexity
of the topic being examined would warrant a more comprehensive approach, since
many of the variables being considered are strongly associated and may interact in
complex ways.
Network analysis offers useful tools for visualizing complex webs of variable
relationships, among which is the plotting of least absolute shrinkage and selection
operator (LASSO) regularized partial correlation networks [31].
Partial correlations measure the degree of association between two variables after
controlling for all other variables being considered; as such, they are a useful measure
of direct association. Using partial correlations instead of the more common 0-order
correlations [32] helps rule out spurious correlations that would (incorrectly) appear
to be meaningful while examining 0-order correlation matrices.
Using LASSO regularization further aids in the interpretability of the network by
only visualizing relatively strong associations and setting to 0 all weaker associations.
This simplification reduces statistical background noise, guiding the interpretation of
results towards more meaningful associations. In fig. 9, we reported the LASSO-
regularized partialized network of the main variables considered in the study (tuning
parameter for the LASSO was set at .5).
In the graph, wider lines represent stronger associations. Positive partialized
correlations are in blue, negative partialized correlations are in red. Visual
examination of the graph shows that, for example, activity axes form an almost
isolated cluster: they are related to each other, but they are very weakly related to few
other variables. Among difficulties, the only one that seems to have strong
associations with other variables is the inadequate home environment (D_IHE),
which seems to reduce (slightly) perceived sustainability of on-line learning (SOE,
.08) and increase the capacity to reproduce classroom dynamics (RDC, .07).
Readiness of schools (SR) and teachers (TTR and TPR), as well as e-maturity (SeM),
form a strong correlated cluster but seem to be related also to the perceived adequacy
of school technology (TAOE), which, in turn, is related to perceived sustainability of
on-line learning (SOE, .31).
Fig. 9. LASSO-regularized partialized network of the main variables considered in this study
In accordance to regression models, time management capacity – of both teachers
(TTMC) and students (STMC) – seem to be related to beliefs about the future use of
on-line learning in the schools (SROL; .27 and .36, respectively). However, after
accounting for all other variables in the dataset, their relation to future intentions
(IWOL; both .31) and perceived sustainability (SOE; .28 and .33, respectively) appear
to be weaker than what was suggested by multiple linear regression. Instead, the main
variables being associated with outcome variables are previously held beliefs about
the relevance of educational technology training for teachers (REDP) and the
improved feeling towards educational technologies (IFT). It is important to note that
these apparent inconsistencies between linear regression are, to an extent, to be
expected: network analysis considers all variables at once, resulting in less spurious
links with outcome variables due to taking into account correlations and mediations
involving linear regression predictors. In addition, LASSO regularization favours
sparse, parsimonious networks by culling weaker effects from the graph. Therefore,
while linear regression offers a useful approximation of which variables predict
specific outcome variables, network analysis results should be considered more
comprehensive.
The relationships evidenced by this graph will be further examined in the next
subsection.
Causal discovery. One of the main draws of network analysis is the possibility of
inferring causal relationships from observational data. This is based on Pearl’s
concept of d-separation [33] by which we mean a set of criteria that can determine
whether two (sets of) variables are independent given a set of other variables. The key
part of the procedure is finding, in the graph, three variables – X, Y, and Z – such
that: (1) Y is connected to both X and Z, (2) X is not connected to Z (when considering
0-order correlations), and (3) X and Z are not independent when conditioning for Y.
If there is such a set of variables, it is possible to orient towards Y both the edge
connecting Y and X and the edge connecting Y and Z. This is because X and Z would
be independent (when conditioning on Y) only if they are common causes of Y. Were
there to be a chain (either X -> Y -> Z or X <- Y <- Z), X would be independent from
Z when conditioning on Y; and the same holds true for the only other possible
configuration, X <- Y -> Z. Directing those edges puts new constraints in place, which
can be used to further infer the direction of edges in the graph.
An easy implementation of this iterative procedure is the PC algorithm, which
identifies the causal structure reported in fig. 10 (using α = .01 and an order-
independent and non-conservative version of the algorithm (see [34], for details).
It should be noted that a main drawback of this procedure is that it relies on strict
assumptions, which are rarely met in real-world data. For example, accurate causal
discovery would require that there are no hidden variables (and especially hidden
common causes) in the network. As such, results from the PC algorithm should be
interpreted tentatively, and not be regarded as factual results. However, in a purely
exploratory analysis such as this one, they can aid and guide interpretation of results.
From the graph, we can observe that some variables are, indeed, where we would
expect them; for example, the intention to be involved with distance education
(IWOL) or the preference for blended learning in the future (FBL) are both at the end
of the causal chain, like the relevance of educational pedagogy training (REDP).
a)
b)
Fig. 10. Causal structure of the main variables considered in this study.
This aligns with our theoretical understanding, for which intention to use is a result
of several processes and conditions, rather than a cause. Schools’ (perceived)
readiness (SR) and adequacy of technology (TAOE), on the other hand, are towards
the start of the causal structure, and they are indeed preconditions that are unlikely to
be effects, for example, of the capacity of teachers to reproduce class dynamics
(RCD).
The e-maturity of schools (SeM) seems to be the only variable to have a direct
effect on the capacity of teachers to reproduce classroom dynamics (RCD); this, in
turn, appears to have a cascading effect on perceived sustainability (SOE) of on-line
education and the intention to employ distance education in the future (IWOL). This
can be understood in the following way: the e-maturity has been interpreted, instead
as a global indicator [26] that supports the smartness of the learning ecosystem [35],
as a complex variable which includes both perceived schools’ and teachers’ readiness
and thus is used as predictor of the teachers’ expectations during this emergency
period: reproducibility of the classroom dynamics.
It is interesting to underline how an improved feeling with technologies (IFT) is
related with the perception of an increase in digital/technical skills (ITS), with the
change of opinion about the educational experience (CIEE) and with the conviction
that in the future the school should relies on, at least partially, on-line learning
(SROL). Unexpected is the causal relation between SROL and SOE. One would have
expected an influence of the perceived sustainability of on-line learning (SOE) on
school reliance on it (SROL) and not the contrary. This could be explained by a
possible preconception: I believe that the school should use, at least partially the on-
line learning and thus this latter becomes sustainable. Another interesting insight is
how time management capacity of teachers seems to be influenced by (perceived)
time management capacity of students. It is possible that (lack of) time management
on the part of students disrupts the schedule of teachers.
The graph in Fig. 11b offers some insights on potential relationships between
variables. For example, experiencing difficulty due to the limited capacity of
expression (D_LE) during distance education seems to actually be an effect of
difficulty in communicating with students (D_SC) and having an inadequate home
environment (D_IHE). As such, we could predict that making the home environment
of teachers more suitable for working would have a positive effect on their
expressivity using distance education tools, even if the tools themselves are
unchanged. D_LE seems to be caused, as could be expected, by the prevalent
interactive characteristic of the didactic activities (EATI). EATI together with the
prevalent synchronous nature (EAAS) and the prevalence of discussion (EALD)
contribute to the delivery of didactic activities intended more for an individual use
rather than for collaborative work.
4. Conclusions and future work
The present paper provides a snapshot of the learning ecosystems' reaction -
Institutions and teachers - to the pandemic seen by the teachers perspective as well as
of the on-line educational processes that have been delivered at a steady state
operational conditions. It also explores the directed network of relationships among
the set of variables that we have considered in this survey. This sheds light on how
setting and operational condition can modify teachers’ mental setting and
expectations about technologies and on-line learning, but it also establishes a
benchmark for future surveys and research aimed at investigating similar phenomena
in the same or similar contexts or, more in general, for future studies on the adoption
of on-line learning.
Specifically, this study demonstrates the reasonable e-maturity and robustness of
both the Italian school system and the technological infrastructure, which did not
collapse thanks to the pedagogical and technological promptness and professionalism
of teachers that have been capable to overcome many personal difficulties (as the
increase of workload, bad connectivity and, sometimes, the inadequacy of the home
setting) to assure educational continuity. It should be noted, though, that the
technological infrastructures leveraged the availability of free accessible and easy to
use video conferencing cloud applications and of easy to use and modular
collaborative working cloud environments that once bent toward the e-learning (e.g.
google classroom) assured, at least, content sharing and co-production and basic
assessment procedures (see figs 4-6).
All of this wouldn’t have been possible just a few years ago [11,12]. Despite this,
teachers also reported a potential risk of digital divide for 6%-10% of the student
population, which deserves great attention and adequate counteractions in terms of
both policy making [9] and research on inclusive education. The teaching strategies
adopted by most teachers in this emergency are, in fact, very far from ideal solutions
to maximise inclusiveness. For example, a more intensive use of asynchronous
communication tools would have probably attenuated the exclusion effects pointed
out by our data [36].
Technology adequacy and teachers’ readiness are two main components of the e-
maturity of a learning ecosystem, whose value influences the perception of
sustainability of on-line learning and the intention, in a third of the teachers, to use it
in the future (blended configuration). It should be noted, though, that these factors are
related to “the capability to reproduce classroom dynamics”, which has a dual
interpretation. On one hand, the use of technology to reproduce “traditional” learning
dynamics is seen, by researchers, as a reductionist approach to technology enhanced
learning [37]. On the other hand, for teachers, it might have provided a kind of
“comfort zone” from which they can then depart for more creative experiments.
Maybe beside the concern about the local production of the personal protection
devices we should, finally, start caring about the realization of an interoperable,
sustainable, cloud based, open and easy to access and to use, modular learning
environment capable to satisfy teachers and students basic needs and to be used as
driver for the gradual introduction of more pedagogically advanced practices (more
than 90% of the learning processes either f2f, blended for on-line are still transmissive
ones).
The present work should be considered as a starting point for further analysis,
research and surveys that could be oriented in several different directions. As the first
step, we intend to analyse more in depth the textual answers and comments provided
by participants to the present survey, either to confirm the scenario that has emerged
up to now from the quantitative analysis or to evidence all relevant details and
possible contradictions that may have been hidden behind it. Other interesting
directions that will be explored in the short period will concern the comparison,
always on a national scale, between the perspective of school teachers and that of
university teachers (primary and secondary education vs tertiary education) and
between school teachers and students' parents (schools vs. families). The comparison
among all main actors of a learning ecosystem - students, parents and teachers
perspective - on the other hand, will be carried on only thanks to local case studies
since these will allow to compare individuals that belong for sure to the same context.
As for the case of this paper, all future analysis will be conducted with the aim to
catch from one side an instant picture of an extraordinary happening represented by
educational processes delivered during a pandemic, and from the other to extract
lessons to be learnt for the future of the technology enhanced learning, its integration
in the educational processes, for the further development of a digital pedagogy and
an adequate digital education literacy. These latter should go in parallel with a further
and adequate development of the infrastructure to guarantee everyone and in all
countries an individual and high quality access to the internet. A future high quality
education for all, see SDG 4 [38], needs to consider the digital dimension, the
avoidance of the digital divide and the sustainability of the digital infrastructures, all
aspects that have not been sufficiently emphasised in the description of the UN 2030
Sustainable Development Goals. Because of this, on a medium-long term it would be
also very important to promote comparative studies on the data that are being
collected all over the world while we write. Finally, we deem very important to follow
up the evolution of the perception of participants in this and other surveys and of the
operative settings to evidence persistent effects that may have been induced by the
pandemic, despite any political decisions and social pressure.
Further research could also investigate the impact on teacher competence and
school readiness of the large amount of resources invested at European, national and
regional level to develop competences and tools for technology-enhanced learning.
In terms of competences, our respondents were - on average - better off than expected
by many [16, 20, 3], although our data do not allow us to ascribe the merit to specific
national or international initiative. In terms of infrastructures, the cloud applications
more widely adopted during the emergency were mostly commercial ones, even if
early exploration of the educational value of their functionalities may be tracked down
to the VII Framework Program. It would thus be interesting to look for correlations
between teachers competence and digital skills and their previous involvement or
exposure to research results concerning Technology Enhanced Learning.
References
1. Wikipedia:
https://en.wikipedia.org/wiki/Impact_of_the_2019%E2%80%9320_coronavirus_pandemi
c_on_education. Accessed 2020/07/15
2. UNESCO, https://en.unesco.org/themes/education-emergencies/coronavirus-school-
closures (2020). Accessed 2020/07/15
3. Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between
emergency remote teaching and online learning. EDUCAUSE Review.
https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-
teaching-and-online-learning.
4. Thelwall, M., & Levitt, J. M. (2020). Retweeting Covid-19 disability issues: Risks, support
and outrage. El profesional de la información, 29(2), e290216.
5. Zimmerman, J. 2020. Coronavirus and the Great Online-Learning Experiment. The
Chronicle of Higher Education, March 10.
https://www.chronicle.com/article/Coronavirusthe
6. Giovannella, C. 2020. La scuola dopo il Covid-19: tutte le opportunità da non sprecare,
https://www.agendadigitale.eu/scuola-digitale/la-scuola-dopo-il-covid-19-tutte-le-
opportunita-da-non-sprecare/ 2020/04/03
7. Giovannella, C. 2020. Didattica a distanza: un primo bilancio per ripensare la scuola del
futuro. https://www.agendadigitale.eu/scuola-digitale/didattica-a-distanza-un-primo-
bilancio-per-ripensare-la-scuola-del-futuro/ 2020/05/25
8. Giovannella, C. 2020. Effect induced by the Covid-19 pandemic on students’ perception
about technologies and distance learning. In "Pedagogical Approaches, Ludic and Co-
Design Strategies & Tools Supporting Smart Learning Ecosystems and Smart Education,
Springer Verlag, in print
9. Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices:
digital technologies and distance education during the coronavirus emergency. Learning,
Media and Technology, 45(2), 107-114.
10. Cam-Tu Vu, Anh-Duc Hoang, Van-Quan Than, Manh-Tuan Nguyen, Viet-Hung Dinh,
Quynh Anh Le, Thu-Trang Thi Le, Pham Hung Hiep, Yen-Chi Nguyen, Dataset of
Vietnamese teachers’ perspectives and perceived support during the COVID-19 pandemic,
Data in brief 31, 105788 Elsevier, 2020 https://doi.org/10.1016/j.dib.2020.105788
11. Fox, R.: SARS epidemic : Teachers ’ experiences using ICTs. Education. 319–327 (2003)
12. Fox, R.: ICT Use During SARS: Teachers’ Experiences. Journal of Technology and
Teacher Education, 15(2), (2007)
13. Giovannella C., unpublished
14. Verma G, Priyamvada, COVID-19 and Teaching: Perception of School Teachers on Usage
of Online Teaching Tools, Mukt Shabd Journal, June 2020, in print
15. Bates, T. (2020). What have we learned from Covid-19 about the limitations of online
learning – and the implications for the fall?? https://www.tonybates.ca/2020/07/04/what-
have-we-learned-from-covid-19-about-the-limitations-of-online-learning/
16. Anderson, J. (2020). The coronavirus pandemic is reshaping education.
https://qz.com/1826369/how-coronavirus-is-changing-education/
17. Tran, T., Hoang, A. D., Nguyen, Y. C., Nguyen, L. C., Ta, N. T., Pham, Q. H., ... & Nguyen,
T. T. (2020). Toward Sustainable Learning during School Suspension: Socioeconomic,
Occupational Aspirations, and Learning Behavior of Vietnamese Students during COVID-
19. Sustainability, 12(10), 4195.
18. Trust, T., Carpenter, J. P., Krutka, D. G., & Kimmons, R. (2020). # RemoteTeaching &#
RemoteLearning: Educator Tweeting During the COVID-19 Pandemic. Journal of
Technology and Teacher Education, 28(2), 151-159.
19. Anderson, T. (2008). The theory and practice of online learning. Athabasca University
Press.
20. Trust, T., Whalen, J (2020b). Should Teachers be Trained in Emergency Remote Teaching?
Lessons Learned from the COVID-19 Pandemic. Journal of Technology and Teacher
Education, 28(2), 189-199.
21. Cauchemez, S., Ferguson, N. M., Wachtel, C., Tegnell, A., Saour, G., Duncan, B., & Nicoll,
A. (2009). Closure of schools during an influenza pandemic. The Lancet infectious
diseases, 9(8), 473-481.
22. http://www.piolatorre.it/rubrica/read-art.asp?id=2609; https://www.truenumbers.it/eta-
degli-insegnanti/
23.https://www.orizzontescuola.it/avvio-anno-scolastico-suddivisione-dei-docenti-regione-e-
livello-istruzione/
24. https://www.tecnicadellascuola.it/130-istituti-italiani-sono-tra-i-piu-innovatori-in-ue
25. https://www.etwinning.net/en/pub/index.htm
26. Sergis S., Sampson D. G., From Teachers’ To Schools’ ICT Competence Profiles, in
Theorizing Why in Digital Learning: Opening Frontiers for Inquiry and Innovation with
Technology, Springer, 307-327 (2014)
27. Dempsey M., Burke J., Covid-19 Practice in Primary Schools in Ireland Report: A Two-
months Follow-up,
28. Greenhow, C. & Chapman, A., 2020. Social distancing meet social media: digital tools for
connecting students, teachers, and citizens in an emergency. Information and Learning
Sciences. DOI 10.1108/ILS-04-2020-0134
29. Manca, S., Ranieri, M. (2016). Is Facebook still a suitable technology-enhanced learning
environment? An updated critical review of the literature from 2012 to 2015. Journal of
Computer Assisted Learning 32 (6), 503-528.
30. Pozzi, F., Ceregini, A., & Persico, D. (2016). Designing networked learning with 4Ts. In
S. Cranmer, N. B. Dohn, M. de Laat, T. Ryberg & J. A. Sime (Eds.), Proceedings of the
10th International Conference on Networked Learning 2016 (pp.210-217). Retrieved from
http://www.networkedlearningconference.org.uk/abstracts/pdf/P15.pdf
31. Epskamp, S., Borsboom, D., & Fried, E. I. (2016). Estimating psychological networks and
their accuracy: a tutorial paper. arXiv preprint, arXiv:1604.08462.
32. Baba, K., Shibata, R., & Sibuya, M. (2004). Partial correlation and conditional correlation
as measures of conditional independence. Australian & New Zealand Journal of Statistics,
46(4), 657-664.
33. Hayduk, L., Cummings, G., Stratkotter, R., Nimmo, M., Grygoryev, K., Dosman, D., ... &
Boadu, K. (2003). Pearl's D-separation: One more step into causal thinking. Structural
Equation Modeling, 10(2), 289-311.
34. M. Kalisch, M. Maechler, D. Colombo, M.H. Maathuis and P. Buehlmann (2012). Causal
Inference Using Graphical Models with the R Package pcalg. Journal of Statistical
Software 47(11) 1–26, http://www.jstatsoft.org/v47/i11/.
35. Giovannella C., "Smartness" as complex emergent property of a process. The case of
learning eco-systems, ICWOAL 2014, IEEE publisher, 1-5 (2014)
36. César, M., & Santos, N. (2006). From exclusion to inclusion: Collaborative work
contributions to more inclusive learning settings. European Journal of Psychology of
Education, 21(3), 333-346. https://doi.org/10.1007/BF03173420
37. Meyer, K. A. (2011). Is online learning a disruptive innovation?. Planning for Higher
Education, 39(4), 44.
38. UNESCO. https://en.unesco.org/education2030-sdg4/targets. Accessed 20 July 2020
39. Persico, D., Passarelli, M., Manganello, F., Gewerc Barujel, A., & Rodríguez Groba, A.
(2020). The participatory dimension of teachers’ self-regulated professional learning about
learning design: beliefs versus behaviours. Professional Development in Education, 1-13.
40. Persico, D., Manganello, F., & Passarelli, M., Participatory approaches to Learning Design,
Interaction Design & Architectures (IxD&A, N.44, 2020 (in print)
