ArticlePDF Available

Differential effi cacy of the resources used in B-learning environments


Abstract and Figures

Background: Learning is increasingly frequent in B-Learning spaces. It is therefore necessary to study the characteristics that guarantee deeper and more successful learning in these learning environments. Method: We work with sample of 233 university students using the Moodle 3.1 platform in the third year of their degrees in Health Sciences. The effectiveness of four types of B-Learning on Learning Results (LR), behaviors on the platform, and student satisfaction are all studied. Prior knowledge is also used as a covariable. Results: It was found that the B-Learning environment in which the students obtained better general Outcomes Learning Results (LR) and a higher degree of satisfaction was the one that included the use of infographics and virtual laboratories based on Self-Regulated Learning (SRL). Conclusions: The design of B-Learning environments together with the use of SRL, is a factor that enhances effective learning and increases student satisfaction, especially if they include infographics and virtual laboratories. In addition, the use of these resources implements better overall LR on a larger number of students. Likewise, it promotes more homogeneous groups in the general LO. Future investigations will be aimed at verifying these results in other knowledge branches.
Content may be subject to copyright.
María Consuelo Sáiz Manzanares, César Ignacio García-Osorio, José Francisco Díez-Pastor
Over recent decades, the use of Blended Lear ning (B-Learni ng)
environments is increasingly frequent in both state-regulated
and non-regulated education. These environments are managed
with Learning Management System (LMS). Both imply an
important challenge in the research of learning processes. Among
others, we may highlight the studies of Azevedo (2014) on Self-
Regulated Learning (SRL) that are based on the theoretical work
of Zimmerman & Moylan (2009) applied to hypermedia learning
environments. In particular, Taub & Azevedo (2019) found
signi cant differences among student users of the LMS in the
application of cognitive and metacognitive skills, taking account
of the level of previously acquired knowledge. The students with
high levels used more complex cognitive and metacognitive skills
for its resolution. These results are essential for the design of
personalized systems of intelligent tutoring.
Thus, the design of an LMS can be diverse and can include
different resources that are to do with strengthening the quality of
learning (Margulieux, McCracken, & Catrambone, 2016). These
resources mean that multi-channel information can be gathered,
which will facilitate the study of the traceability of cognitive,
affective and metacognitive skills (Azevedo et al., 2013). These
learning spaces also make the analysis of SRL processes possible
for both individuals and collectives. The advantage of these spaces
is that they provide the teacher will a lot of information through
different registries (eye-tracking, thinking aloud, note taking and
drawing, log- les, and facial recognition, among others). This eld
of investigation is known as Advanced Learning Technologies
(ALTs). Learning spaces in the 21st century therefore have to
include technological resources that encourage the development of
SRL, metacognitive instruction, and monitoring. One of the most
effective tools is MetaTutoring (Azevedo et al., 2013) that can be
implemented through d ifferent hypermedia resourc es (quizzes with
automated feedback, vir tual laboratories, infographs, etc.). They all
Copyright © 2019 Psicothema
Differential ef cacy of the resources used in B-learning environments
María Consuelo Sáiz Manzanares, César Ignacio García-Osorio, José Francisco Díez-Pastor
Universidad de Burgos
Abstract Resumen
Background: Learning is increasingly frequent in B-Learning spaces. It
is therefore necessary to study the characteristics that guarantee deeper
and more successful learning in these learning environments. Method:
We work with sample of 233 university students using the Moodle
3.1 platform in the third year of their degrees in Health Sciences. The
effectiveness of four types of B-Learning on Learning Results (LR),
behaviors on the platform, and student satisfaction are all studied. Prior
knowledge is also used as a covariable. Result s: It was found that the
B-Learning environment in which the students obtained better general
Outcomes Learning Results (LR) and a higher degree of satisfaction was
the one that included the use of infographics and virtual laboratories
based on Self-Regulated Learning (SRL). Conclusions: The design of
B-Learning environments together with the use of SRL, is a factor that
enhances effective learning and increases student satisfaction, especially
if they include infographics and virtual laboratories. In addition, the use
of these resources implements better overall LR on a larger number of
students. Likewise, it promotes more homogeneous groups in the general
LO. Future investigations will be aimed at verifying these results in other
knowledge branches.
Keywords: Blended learning B-learning, virtual labs, infographics, self-
regulated learning (SRL), learning outcomes.
E cacia diferencial de los recursos empleados en entornos B-learning.
Antecedentes: es cada vez más frecuente que el aprendizaje se realice
en espacios B-Learning. Por ello, es preciso estudiar cuáles son las
características que garantizan en estos entornos aprendizajes más
profundos y exitosos. Método: se trabajó en la plataforma Moodle 3.1
con una muestra 233 estudiantes universitarios de tercero de grado en la
rama de Ciencias de la Salud. Se estudió la efectividad de cuatro tipos de
B-Learning sobre los Resultados de Aprendizaje (RA), las conductas de
aprendizaje y la satisfacción de los estudia ntes. Asimismo, se utilizó como
covariable los conocimientos previos. Resultados: se halló que el entorno
B-Learning en el que los estudiantes obtuvieron mejores RA generales
y mayor grado de satisfacción fue el que incluía el uso de infografías y
de laboratorios virtuales basados en aprendizaje autorregulado (SRL).
Conclusiones: el diseño de entornos B-Learning, junto con la utilización
de SRL, es un factor que potencia aprendizajes e caces e incrementa
la satisfaccn de los estudiantes, especialmente si incluyen infografías
y laboratorios virtuales. Además, el uso de estos recursos implementa
mejores RA generales en un mayor número de estudiantes. Futuras
investigaciones irán dirigidas a comprobar estos resultados en otras ramas
de conocimiento.
Palabras clave: blended learning (B-learning), laboratorios virtuales,
infografías, aprendizaje autorregulado, resultados de aprendizaje.
Psicothema 2019, Vol. 31, No. 2, 170-178
doi: 10.7334/psicothema2018.330
Received: December 2, 2018 • Accepted: March 26, 2019
Corresponding author: María Consuelo Sáiz Manzanares
Facultad de Ciencias de la Salud
Universidad de Burgos
09001 Burgos (Spain)
Differential efficacy of the resources used in B-learning environments
have the purpose of providing feedback to the learner for problem-
solving tasks that help monitoring: focalization of the object of the
task, activation of previous knowledge, searching for strategies to
resolve the problem, evaluation throughout the problem-solving
process, changes if needed to the problem-solving strategy, and
nal evaluation (Taub & Azevedo, 2019). Those resources respect
the learning pace of the student, facilitating personalization (Sáiz,
García-Osorio, Díez-Pastor, & Martín, 2019).
From among all the possible registers, it is worth highlighting
the behavioral patterns in the learning process. Among them,
information can be obtained on: frequency of access of the
student to the platform and the use of resources (Cerezo, Sánchez-
Santillán, Paule-Ruiz, & Núñez, 2016).
Likewise, recent studies (Sáiz, Marticorena, García-Osorio,
& Díez-Pastor, 2017) have pointed to a greater effectiveness of
B-L earning spaces i n which 80% of the teach ing is done in virt ual
environments and 20% is done Face to Face (F2F). Along these
lines, other researchers have indicated that the effectiveness of
these spaces will increase, if they include hypermedia resources
(videos, quizzes, forums…) (Cerezo et al., 2016) and active
methodologies such as Project-Based Learning. (PBL) (Bannert,
Reimann, & Sonnenberg, 2014). With regard to the hypermedia
resources, among the most novel is the use of infographs. These
graphs display informat ion and can include: words, phrases, images,
and videos. If they include images and interactive resources they
can be especially effective. It seems that they facilitate dynamic
conceptual understanding of the subject matter among students
(Al-Dairy & Al-Rabaani, 2017). In addition, they increase the
motivation and the creativity of the students (Damman, Vonk,
van den Haak, van Hooijdonk, & Timmermans, 2018; Papic &
Sušilović, 2018). In summary, the infograph is considered a low-
cost resource that i mproves conceptual understand ing and retention
(Kiernan, Oppezzo, Resnicow, & Alexander, 2018; Nogueira-
Frazão & Martínez-Solana, 2019) and problem-solving (Santos,
Pereira-Neto, & Neves, 2019). In addition, this resource facilitates
the development of SRL and re ection on the contents under study
(Balkac & Ergun, 2018). It has likewise been demonstrated that
the infograph increases its effectiveness, if it contains questions
for SRL (Haşlaman, 2018; Santos et al., 2019). Along these lines,
this resource is administered to achieve personalized learning
among the students (Gutiérrez, Barriga, Ramírez-Corona, López-
Malo, & Palou, 2016).
Another novel hypermedia resource in B-Learning spaces, is
the use of virtual laboratories. Alves et al. (2016) and Viegas et al.
(2018) af rmed that their implementation can improve teaching
in a virtual as well as a presential mode. The virtual laboratory
has especially been applied as a resource in the disciplines of
Engineering and the Health Sciences. The advantages of its use
are the personalization of teaching and cost reductions (Viegas et
al., 2018). This tool strengthens re exive learning based on the use
of metacognitive skills and SRL (Achuthan, Francis, & Diwakar,
2017). In addition, the virtual laboratory has shown itself to be
more effective that the presential laboratory (Koretsky, Kelly, &
Gummer, 2011). The reasons are centered on the improvement of
conceptual understanding with this tool, increased SRL, and the
use of metacognitive skills. It all increases student motivation,
satisfaction, and performance (Viegas et al., 2018). However, the
weak point of this resou rce is excessive individuality in t he learning
process. Authors such as Gustavsson et al. (2009) therefore advise
combining its use with collaborative actions such as PBL.
In summary, the use of infographs and virtual laboratories
in B-Learning spaces appears to increase the personalization of
learning and to strengthen SRL in real time (Krum, 2014; Kunze,
& Rutherford, 2018; Harley, Taub, Azevedo, & Bouchet, 2018). In
addition, personalization increases learning results and student
interaction with the platform (Sáiz, Marticorena, Díez-Pastor, &
García-Osorio, 2019).
Equally, the emergence of this type of resource, with
increasingly greater celerity, makes it necessary to evaluate its
effectiveness in the learning process, in the light of different
quality standards: metacognitive design, technological knowledge
and content (Vongkulluksn, Xie, & Bowman, 2018). In turn, each
one of them contains a series of sub-standards that are described
in Table 1.
Xie, Di Tosto, Chen, & Vongkulluksn (2018), in a meta-analysis
on the quality of teaching in different B-Learning environments
found high correlations between the evaluation of teaching
materials and metacognitive structure in their design. Moreover,
they found lower correlations between the valuation of the mater ials
and the technological structure, for their implementation on the
platform. These results point to the importance of using friendly
interfaces (Sáiz, Cuesta, Alegre, & Peñacoba, 2017). Furthermore,
different researchers (Cerezo, Bernardo, Esteban, Sánchez,
& Tuero, 2017) pointed out that the use of SRL in university
environments increases student satisfaction with the teaching.
Moreover, as previously mentioned, the LMS includes tools that
register the interactions with the different agents involved in the
teaching. Nevertheless, the majority of these environments include
no tools for the analysis of these data (Xie, Kim, Cheng, & Luthy,
2017) making the inclusion of plugins necessary (Luna, Castro, &
Romero, 2016).
In view of the studies referred to above, which place emphasis
on the rapid incorporation of technological resources at the service
of learning in B-Learning spaces, especially in the teaching of
H ig he r Ed uc a ti on , i t a pp e ar s re le va nt t o e va lu at e th ei r ef fe c ti ve ne ss
in real educational environments (Wiley, Bliss, & McEwen, 2014).
In particular, the effectiveness of different B-Learning designs
is examined in this work that implement different learning
Table 1
Standards of quality in the evaluation of hypermedia resources adapted from
Xie, Di Tosto, Chen, & Vongkulluksn (2018) p. 95
Quality standards of
materials Sub-standards
Metacognitive design
Adjustment of proposed activities
Well-developed monitoring
Feedback throughout the process
Inquiry-based activities
Depth of learning is well developed
Well-worked content
Clarity of objectives
Conceptual progression
Differentiation between concepts
Adaptation of the level worked in the presentation of
the concepts
The development of critical thought
Technological structure
Channels of communication
María Consuelo Sáiz Manzanares, César Ignacio García-Osorio, José Francisco Díez-Pastor
resources, prepared with different technological tools (infographs,
virtual laboratories, videos, quizzes, process-oriented automatic
feedback, and PBL) applied in different combinations.
The research questions were as follows:
RQ1 Are there signi cant differences in student learning
results in different B-Learning environments?
RQ2 Are there signi cant differences in student learning
behaviors in different B-Learning environments?
RQ3 Are there signi cant differences in student
satisfaction in the different B-Learning environments?
The study had a sample of 233 students from the third year of
degree courses in Health Sciences (see Table 2) in two subjects:
subject 1 (Group 1 and Group 3) and subject 2 (Group 2 and Group
4). Tea chin g was i mparte d by the sam e tea che r, an exp ert i n vir tu al
teaching, to all four groups.
The criteria for the inclusion of the student in the study was
systematic participation in the course work with an acceptable
presential and/or virtual participation of 85% to 100%. The
exclusion criteria were non-systematic participation on the course
(less than 80%). The assignation of the groups was done using
convenience sampling in accordance with the possibilities of
intervention. In Table 3, the characteristics of each group can be
seen, as described in the section on procedure. Likewise, the logs
in each of the groups were: Group 1 = 4064, Group 2 = 4982,
Group 3 = 10111, and Group 4 = 13433.
Learning Strategies Scale (ACRAr) by Román and Poggioli
(2013). In this study the Scale of Metacognitive Skills was
applied. It has an inter-judge reliability of α = .90, a content
validity of r = .88, and a construct validity of r = .88. In this
study, the value found was α = .70.
Questionnaire: on previous knowledge prepared ad hoc
with 8 questions measured on a Likert-type scale of 1 to 5
on the key concepts of the subjects. In subject 1, (Groups 1
and 3) achieved a reliability index of α = .89 and in subject
2 (Groups 2 and 4) of α = .91.
Learning Management System (LMS). In this study a
Moodle v.3.1 -based LMS was used. Different hypermedia
resources were included (videos, quizzes, infographs,
and simulation laboratory) in different combinations as a
function of the groups under study, see section on procedure.
Likewise, logs were analyzed on the following indicators
of behavior on the platform: 1. Access to complementary
information; 2. Access to guidelines to carry out PBL; 3.
Access to information on theoretical content; 4. Access to
feedback from the teacher; 5. Mean number of visits per
Teaching support videos: prepared ad hoc and published
in the Institutional Repository of University of Burgos
(Sáiz, 2018a, 2018b). In Group 2 and in Group 3, two
videos were used on the contents of the study units in
which self-regulation strategies were used, by the teacher,
to guide conceptual comprehension, and inter-relations.
Likewise, after the presentation of each concept, a
comprehension quizz was administered with feedback on
the responses.
The two videos used in Group 4 treated the content of the
study unit, which included self-regulation strategies. In this
case the comprehension (quizz) questions were done after
watching the video and included process-oriented feedback
(the reason for the correct response was indicated) on the
responses from the students.
Table 2
Descriptive statistics in the different groups under study with respect to the
independent variables assigned to gender and age
Group N PK
Men Women
age SDage NM
age SDage
1 58 2.68 9 24.67 4.12 49 23.82 5.10
2 63 2.00 7 22.00 1.73 56 24.02 4.98
3 55 2.40 6 22.00 1.55 49 22.53 2.34
4 57 2.39 8 24.63 7.50 49 22.33 1.96
Note: PK = Previous knowledge; Group 1 = B-Learning [20 (BL)-80% (F2F)] methodology;
Group 2 = B-Learning [80 (BL)-20% (F2F)] methodology + videos + quizzes + product-
oriented feedback; Group 3 = B-Learning [80 (BL)-20% (F2F)] methodology + videos
+ quizzes + process-oriented feedback + infographs + virtual laboratory; Group 4 =
B-Learning [80 (BL)-20% (F2F)] methodology + videos + quizzes + process-oriented
feedback; Mage
= Mean Age; SDage = Standard Deviation Age
Table 3
Description of the groups under study
Resources Group 1 Group 2 Group 3 Group 4
Use of the virtual platform (LMS) 20% 80% 80% 80%
Use of videos with product-oriented quizzes on the virtual platform (LMS) No Yes No No
Use of videos on the virtual platform (LMS) No No Yes Yes
Use of quizzes on paper Yes N o No No
Use of process-oriented comprehension quizzes on the virtual platform (LMS) No No No Yes
Use of a virtual laboratory through the virtual platform (LMS) No No Yes No
Use of infographs on the virtual platform (LMS) No No Yes No
Use of quizzes for the evaluation of conceptual content with product-oriented feedback on the virtual platform (LMS) No Yes No No
Use of quizzes for the evaluation of conceptual content with process-oriented feedback on the virtual platform (LMS) No No Yes Yes
Note: PBL methodology was applied in all groups
Differential efficacy of the resources used in B-learning environments
Quizzes: in the four groups under study, evaluation
questionnaires on the contents of the quizz format were
used. The multiple-choice test-type questions offered four
possible responses and only one true option. The quizzes
were administered at the end of each thematic unit. In Table
4, the different options may be seen that were applicable to
the groups in the study (see Figure 1 and Figure 2).
Infographics: prepared with free Piktochart software,
a resource used only in Group 3. The structure of the
infograph consisted in the presentation of key thematic
concepts through images. Then, self-regulation questions,
which guided the responses through key images, were
included on the concepts that had been seen. Subsequently,
the question was asked “are you clear about those concepts”.
If the response was negative, the question was asked “which
are you not clear about?” The infographs are published
under open access and held in the Repository of University
of Burgos (Sáiz, 2018c, 2018d, 2018e).
Table 4
Options for the administration of the evaluation questionnai res in the groups under study
Group Group 1 Group 2 Group 3 Group 4
Form of administration On paper
On the LMS
Type of feedback to the response
Product-oriented (right-wrong) X
Product- oriented (right-wrong) (see example in Figure 1) X
Process-oriented (correct-incorrect and
orientation on the reason why) (see example in Figure 2) X X
Response time of the results One week
Immediate at the end of the questionnaire
Figure 1. Quizz with product-oriented feedback administered to groups 1 and 2
Figure 2. Quizz with process-oriented feedback administered to groups 3 and 4
María Consuelo Sáiz Manzanares, César Ignacio García-Osorio, José Francisco Díez-Pastor
Virtual labor atories: use was made of two virtual lab oratories
designed from a simulation structure that provided the
student with help in the form of a guide to prepare the project
(PBL). Those laboratories were structured around SRL
and the teacher guided the different practices by applying
modeling techniques and self-instructions for solving the
tasks. The steps for solving the task were recorded on video.
The student could access this resource over the platform,
whenever needed.
Learning results: the learning results from different tests
were considered: Preparation of the PBL, Defense of the
PBL, Quizzes, and Learning Results, Totals.
Scale of satisfaction with teaching activity. This instrument,
based on the work of Marsh (1987), called Student Evaluation
of Educational Quality (SEEQ) was adapted by Bol, Sáiz,
& Pérez (2013). Their scale contains ve clusters for the
analysis of teaching quality: materials, continuous evaluation,
motivation of the teacher, course workload, and the degree of
general satisfaction. It has a general index of reliability of α
= .92. Likewise, the one found for this study was α = .86.
A multi-group design was used with pre-treatment measures
(equaling the metacognitive skills of the group), and subsequently
in the learning results, learning behavior, and student satisfaction,
considering the previous knowledge as a co-variable, in order to
control the perceived dif culty.
In Group 1, a B-Learning (20% in el LMS and 80% F2F)
teaching methodology was used, without hypermedia resources
and the quizzes were done on paper, see section on instruments.
In Group 2, a B-Learning (80% on LMS and 20% F2F)
teaching methodology was applied. In addition, the group included
hypermedia resources:
a. Videos on the contents of the subject matter that
included quizzes, the responses to which received
product-oriented feedback.
b. Quizzes for the evaluation of the contents, which were
done on the LMS with process-oriented feedback: see
instruments section.
In Group 3, a B-Learning teaching (80% LMS and 20% F2F)
methodology was administered and the following hypermedia
resources were included:
a. Videos on the contents of the subject matter that
included quizzes after treating each concept, similar
to those applied to Group 2.
b. Quizzes for the evaluation of the contents, which were
done on the LMS with process-oriented feedback; see
instruments section.
c. A virtual ad hoc laboratory; see instruments section.
This material was made available to the students from
the third week of the semester.
d. Infographs: previously described.
In Group 4, a B-Learning teaching (80% LMS and 20% F2F)
methodology was administered and the following hypermedia
resources were used:
1. Vid eos prepared ad hoc, without questions and quizzes in
the video. However, the student after watching the video
could complete a comprehension quizz-type questionnaire
that could be repeated as many times as desired.
2. Quizzes for the evaluation of the contents, which were done on
the LMS with process-oriented feedback, described earlier.
Data analysis
Before starting the study, the homogeneity of the groups
under the variable ‘use of metacognitive skills’ was analyzed.
A xed-effects ANOVA (type of group) was applied to test the
variable metacognitive skills measured on the ACRA Scale of
Metacognitive Skills ACRA, F(3, 233) = 1.251, p = .292, η2 = 0.02.
It was also studied whether differences existed at the level of
previous knowledge of the students, for which purpose a xed-
effects ANOVA (type of group) was completed. In this case,
signi cant differences were found between the groups, F(3, 229) =
14.13, p = .00, η2 = 0.16, due to which this variable was considered
a co-variable. The research questions were tested with ANCOVAS.
All these analyses were done on the statistical software package
SPSS v.24, applying a con dence interval of 95%.
A xed-effects A NCOVA (type of B-L earning) was applied to test
RQ1 on the different learning results (PBL preparation, PBL defense,
Quizzes, and Learning Results (LR) totals). Signi cant differences
were found in all the LR and the effect values were measured,
following the classi cation of Cohen (Kelley & Preacher, 2012),
in the LR for the quizzes (η2 = 0.45), in the LR on the preparation
of the PBL (η2 = 0.31), and in the LR Totals (η2 = 0.20). Likewise,
the differences in the Bonferroni test were found for all the groups
except in the LR, in the defense of PBL, and between Groups 3 and
4 in the LR on the quizzes (see Table 5). However, the covariable
previous knowledge had effects of the LR Quizzes, F(1, 228 ) = 67.24, p
= .000, η2 = 0.47 and not on the other results [LR preparation of PBL
F(1, 228) = 0.38, p = .54, η2 = 0.002); LR defense of PBL, F(1, 228) = 0.19,
p = .661, η2 = 0.001; LR Total, F(1, 228) = 2.38, p = .124, η2 = 0.01)].
With respect to RQ2, differences were found between all the
groups, in all learning behaviors on the platform, except in access to
information on the oretical content. The differences in th e Bonferroni
test results for groups 4, 3, 2, and 1 were found in hierarchical
order from greater to lesser. With regard to access to guidelines to
conduct PBL, the differences were found in Group 3 on the others in
favor of this one (see Table 6). In relation to the covariable previous
knowledge, it no had no effect on any of the learning behaviors in
this analysis (Access to complementary information F(1, 228) = 0.60,
p = .44, η2 = 0.003; Access to the Guidelines for PBL F(1, 228) = 0.81,
p = .37, η2 = 0.004; Access to information on theoretical content F(1,
228) = 1.01, p = .32, η2 = 0.004; Access to teacher feedback, F(1, 228)
= 0.46, p = .50, η2 = 0.002; Mean number of visits per day, F(1, 228) =
1.61, p = .21, η2 = 0.01.
Subsequently, a xed-effects ANCOVA (type of B-Learning)
was applied to test RQ3, nding signi cant differences between
all the groups and under all the indicators of satisfaction with
teachers, except for the motivation of the teacher towards the
subject in which differences were only found between Groups 1
and 4. The highest measurements under satisfaction were found in
Group 3, a group in which infographs and virtual library resources
Differential efficacy of the resources used in B-learning environments
were administered (see Table 7). In this case, the co-variable
previous knowledge had no effects on any of the variables, Subject
matter F(1, 18 4) = 0.13, p = .72, η2 = 0.001; Continuous evaluation
F(1, 184) = 0.39, p = .53, η2 = 0.002; Teacher motivation F(1, 184) =
0.02, p = .88, η2 = 0.000; Workload F(1, 184) = 3.78, p = .05, η2 =
0.02; General satisfaction, F(1, 184) = 0.18, p = .67, η2 = 0.001.
Table 5
Single factor xed-effects ANOVA (Type of group) and value of the effect and test of mean differences between the Bonferroni groups for the variable learning results
Group 1
n = 58
Group 2
n = 63
Group 3
n = 55
Group 4
n = 57
Single-factor ANOVA (type of
B-Learning) Bonferroni
by group
M SD M SD M SD M SD df F p η2
LR preparation of PBL
(Maximum score 2.50) 2.22 .18 2.19 .42 2.40 .37 2.72 .24 3,228 33.60 .000* 0.31
-.18* 1,3
-50* 1,4
-.21* 2,3
-.53* 2,4
-.32* 3,4
LR defense of the PBL
(Maximum score 2.50) 1.73 .29 1.76 .18 1.73 .18 1.64 .15 3,228 3.54 .015* 0.04 .12* 2,4
LR Quizzes
(Maximum score 3) 2.15 .42 2.59 .21 2.71 .14 2.72 .15 3,228 62.66 .000* 0.45
-.44* 1,2
-.56* 1,3
-.57* 1,4
-.12* 2,3
-.13* 2,4
LR Total
(Maximum score 10) 8.60 .72 9.05 .43 9.09 .49 8.15 1.15 3,228 19.95 .000* 0.21
-.45* 1,2
-.49* 1,3
.45* 1,4
.90* 2,4
.94* 3,4
* p <.05 Note. LR = Learning Results; M = Mean; SD = Standard Deviation; df = degrees of freedom; η2 = eta squared. Group 1 = Methodology B-Learning [20 (BL)-80% (F2F)]; Group 2 =
Methodology B-Learning [80 (BL)-20% (F2F)] + videos+ quizzes + product-oriented feedback; Group 3 = Methodology B-Learning [80 (BL)-20% (F2F)] + videos + quizzes + process-oriented
feedback + infographs + virtual laboratory; Group 4 = B-Learning [80 (BL)-20% (F2F)] methodology + videos + quizzes+ process-oriented feedback
Table 6
Single factor xed-effects ANOVA (Type of group) and effect value and test of i ntergroup different of means of Bonfer roni in the variables learning behavior on the
Group 1
n = 58
Group 2
n = 63
Group 3
n = 55
Group 4
n = 57
Single-factor ANOVA (type of
B-Learning) Bonferroni
by group
M SD M SD M SD M SD df F p η2
Access to complementary 5.07 4.60 5.87 5.49 11.38 6.34 36.46 23.82 3,229 78.00 .00* 0.51
-31.39* 1,4
-30.58 2,4
-25.07 3,4
Access to orientations to be
done on the PBL 3.60 3.00 3.94 3.18 9.27 7.51 5.61 7.10 3,229 12.47 .00* 0.14
-5.67* 1,3
-5.34* 2,3
3.66* 3,4
Access to information on
theoretical content 13.81 7.49 13.64 7.35 14.56 10.83 14.63 8.12 3,229 .21 .89 0.003
Access to teacher feedback 18.43 21.14 25.44 33.18 90.47 28.40 114.28 46.13 3,229 117.37 .00* 0.61
-72.04* 1,3
-95.85* 1,4
-65.03* 2,3
-88.84* 2,4
-23.81* 3,4
Mean visits per day 1.08 .58 3.04 1.02 8.20 3.08 7.90 3.25 3,229 139.55 .00* 0.65
-1.96 1,2
-7.12 1,3
-1.96 2,1
-5.16 2,3
-4.86 2,4
* p <.01 Note. M = Mean; SD = Standard Deviation; df = degrees of freedom; η2 = eta squared. Note. Group 1 = B-Learning [20 (BL)-80% (F2F)] methodology; Group 2 = B-Learning [80 (BL)-
20% (F2F)] methodology + videos+ quizzes + product-oriented feedback; Group 3 = B-Learning [80 (BL)-20% (F2F)] methodology + videos + quizzes + process-oriented feedback + infographs
+ virtual laboratory; Group 4 = B-Learning [80 (BL)-20% (F2F)] methodology + videos + quizzes+ process-oriented feedback
María Consuelo Sáiz Manzanares, César Ignacio García-Osorio, José Francisco Díez-Pastor
It appears that the covariable previous knowledge in uenced
the learning results outcomes, particularly in the areas related with
the completion of individual tasks, but not for collaborative tasks.
Likewise, no incidents were observed in accessing the different
LMS resources, nor in satisfaction with the teaching activity. The
rst results coincided with those found by Taub & Azevedo (2019).
This conclusion is relevant to the design of activities in the LMS.
For example, collaborative tasks could be proposed between the
students in such a way that those with a higher level of previous
knowledge support the work of students with lower levels. In
addition, the Meta-Tutoring from the implementation of different
hyper-media resources appeared to compensate the effect of the
variable previous knowledge on the LR.
It was also found that the differences in the LR were not
homogeneous between the different evaluation procedures. For
example, better LR results were found in the preparation of the
PBL in Group 4, in which a B-Learning (80%/20%) methodology
was implemented with hypermedia resources of video displays
and post-video comprehension quizzes. However, for the Defense
of PBL, the best results were found in Group 2, in which a
B-Learning (80%-20%) methodology was applied, which included
videos with quizzes and process-oriented feedback. The patterns
of access were greater in groups 3 and 4, which implemented the
B-Learning (80%-20%) methodology with more sophisticated
hypermedia resources. To do so, it appears that the B-Learning
(80%-20%) design with more complex hypermedia resources
increased the activity of the students on the LMS. In addition, if
this environment included infographs and virtual laboratories,
there was a general increase in LR (Alves et al., 2016; Cerezo et
al., 2016; Gustavsson et al., 2009; Margulieux et al., 2016; Sáiz
et al., 2017; Viegas et al., 2018). One possible explanation is that
the combination of these resources strengthens the conceptual
comprehension of the student (Al-Dairy & Al-Rabaani, 2017)
and the personalization of learning (Kunze & Rutherford, 2018;
Harley et al., 2018), which facilitate the SRL and foreseeably the
use of metacognitive skills. All of which nally increase student
motivation (Zimmerman & Moylan, 2009).
Moreover, greater satisfaction was found with the teaching
activity in Group 3 which included t he use of infographs and virt ual
laboratories; a result that is agreement with those of Achuthan et
al. (2017), Haşlaman (2018), and Santos et al. (2019).
In summary, it appears that the design of the B-Learning spaces
in uences the LR and behavioral-learning patterns in the students.
However, there are other variables, such as previous knowledge
that could be exerting some in uence (Xi et al., 2018), especially
on the LR.
Nevertheless, the generalization of the results of this study
must be treated with prudence, attending to the characteristics of
the sample: number, origin, sampling, history of student learning,
speci city of quali cation and subject matter. Future investigations
will be directed towards establishing the ef cacy of B-Learning
(80%-20%) environments in other areas of knowledge, in which
different hypermedia resources are included.
This study was completed with grants from the Vice-Rectorate
of Teaching, Teaching Research and Research Staff of the
University of Burgos for teaching innovation 2019 awarded to the
B-learning Teaching Innovation Group in Health Sciences and
the Vice-Rectorate of Research and Knowledge Transfer 2019
awarded to support research groups recognized for DATAHES.
The suggestions of the reviewers have also served to improve the
quality of this document.
Table 7
Indicators of Student Satisfaction with t he quality of teaching [Bol et al. (2013) adapted from Marsh (1987)]
Group 1
n = 31
Response rate
Group 2
n = 51
Response rate
Group 3
n = 53
Response rate
Group 4
n = 54
Single-factor ANOVA (type of
B-Learning) Bonferroni
by group
M SD M SD M SD M SD df F p η2
Materials 4.11 .74 4.29 .64 4.56 .53 3.79 .89 3,185 10.82 .00* 0.15
-.44* 1,3
.50 2,4
.77 3,4
Continuous evaluation 3.79 .93 4.23 .75 4.55 .64 3.46 .97 3,185 17.25 .00* 0.22
-.76* 1,3
-.76* 2,4
1.08* 3,4
Motivation of teacher 4.37 .75 4.40 .68 4.64 .40 3.75 .93 3,185 14.87 .00* 0.19
.62* 1,4
.65* 2,4
Workload 4.23 .80 3.84 .67 4.02 .82 3.65 .93 3,185 3.81 .01* 0.06 -.58* 1,4
General satisfaction 4.13 .67 4.31 .55 4.58 .42 3.69 .82 3,185 18.90 .00* 0.24
-.46* 1,3
.44* 1,4
.62* 2,4
.89* 3,4
* p <.01 Note. M = Mean; SD = Standard Deviation; df = degrees of freedom; η2 = eta squared
Differential efficacy of the resources used in B-learning environments
Al-Dairy, H. M., & Al-Rabaani, A. H. (2017, December). An analytical
study of research orientations for infographics applications in
education. Paper presented at the 6th Internat ional Conference on
Information an d Commu nication Technology and Accessibility
(ICTA), Muscat, Oman. doi: 10.1109/ICTA. 2017.8336059
Achuthan, K., Francis, S. P., & Diwakar, S. (2017). Education and
Information Technologies, 22(6), 2825 -2855. doi: 10.10 07/s10639-017-
Alves, G., Fidalgo, A., Marqués, M. A., Viegas, C., Felgueiras, M., Costa,
R., … & Dobboletta , E. (2016, October). Spreading remote labs usage:
A system – a community – a federation. Paper presented at the 2nd
international conference of the Portuguese society for engineering
education (CISPEE2016), Vila Real, Portugal. doi: 10.1109/
Azevedo, R. (2014). Multimedia learning of metacognitive strategies. In
R. Mayer (Ed.), The Cambridge handbook of multimedia learning
(2nd ed., pp. 647-672). Cambridge: Cambridge University Press.
Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet,
F., & Landis, R. (2013). Using trace data to examine the complex roles
of cognitive, metacognitive, and emotional self-regulatory processes
during learning with multi-agent systems. In R. Azevedo & V. Aleven
(Eds.), International handbook of metacognition and learning
technologies (pp. 427-449). Amsterdam: Springer.
Balkac, M., & Ergun, E. (2018). Role of Infographics in Healthcare.
Chinese Medical Journal, 131(20), 2514-2517. doi: 10.4103/0366-
Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining
techniques for analysing patterns and strategies in students’ self-
regulated learning, Metacognition and L earning, 9(2), 161-185. doi:
10.1007/s11409- 013-9107-6
Bol, A., Sáiz, M. C., & Pérez, M. (2013). Validación de una encuesta sobre
la actividad doc ente en Educación Superior [Validat ion of test teach ing
activity in Higher Education]. Aula Abierta, 41(2), 45-54.
Cerezo, R., Bernardo, A., Esteban, M., Sánchez, M., & Tuero, E. (2017).
Programas para la promoción de la autorregulación en Educación
Superior: un estudio de la satisfacción diferencial entre metodología
presencial y virtual. European Journal of Education and Psychology,
8(1), 30-36. doi: 10.1016/j.ejeps.2015.10.0 04
Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M.P., & Núñez, J.C.
(2016). Students’ LMS interaction patterns and their relationship
with achievement: A case study in higher education. Computers &
Education, 96, 42-54. doi: 10.1016/j.compedu.2016.02.006
Damman, O. C., Vonk, S. I., Van den Haak, M. J., Van Hooijdonk, C.
M. J., & Timmermans, D.R.M. (2018). The effects of infographics
and several quantitative versus qualitative formats for cardiovascular
disease risk, including heart age, on people’s risk understanding.
Patient Education and Counseling, 101(8), 1410-1418. doi: 10.1016/j.
Gustavsson, I., Nilsson, K., Zackrisson, J., García-Zubia, J., Hernández-
Jayo, U., & Nafalski, A. (2009). On objectives of instructional
laboratories, individual assessment, and use of collaborative remote
laboratories. IEEE Transactions on Learning Technologies, 2(4), 263-
274. doi: 10.1109/TLT.2009.42
Gutiérrez, J. V., Díaz-Barriga, F., Ramírez-Corona, N., López-Malo.,
A., & Palou, E. (2016). Personal learning environments: Analysis of
learning processes, re ection, and identity in an academic context.
Paper presented at the ASEE Annu al Conference and Exposition,
Conference Proceedings 2016, New Orleans, LA. doi: 0000-0001-
Harley, J. M., Taub, M., Azevedo, R., & Bouchet, F. (2018). “Let’s Set
Up Some Subgoals”: Understanding Human-Pedagogical Agent
Collaborations and Their Implications for Learning and Prompt and
Feedback Compliance. IEEE Trans act ion s on Lea rni ng Tech nol ogies,
11(1), 54-66. doi: 10.1109/TLT.2017.2756629
Haşlaman, T. (2018). Öğrencilerin Özdüzenleyici Öğrenen Olmalarının
İnfogra k Aracılığıyla Destek lenmesi: “Nasıl Daha İyi Öğrenebilirim?
[Supporting Students as Self-Regulated Learners with Use of
Infographics: “How can I Learn Better?”] İlköğret im Online, 17, 277-
292. doi: 10.17051/ilkonline.2018.413771
Kelley, K., & Preacher, K. J. (2012). On Effect Size. Psychological
Methods, 17(2), 137-152. doi:10.1037/a0 028086
Kiernan, M., Oppezzo, M.A., Resnicow, K., & Alexander, G. L (2018).
Effects of a methodological infographic on research participants’
knowledge, transparency, and trust. Health Psychology, 37(8), 782-
786. doi: 10.1037/hea0000631
Koretsky, M., Kelly, C., & Gummer, E. (2011). Student Perceptions of
Learning in the Laboratory: Comparison of Industrially Situated
Virtual Laboratories to Capstone Physical Laboratories. Journal
of Engineering Educat ion, 100(3), 540-573. doi: 10.1002/j.2168-
Krum, R. (2014). Cool Infographics: Effective Communication with D ata
Visualization and Design. Hoboken: Wiley
Kunze, A., & Rutherford, T. (2018). Blood from a stone: Where teachers
report nding time for computer-based instruction. Computers &
Education, 127, 165-177. doi: 10.1016/j.compedu.2018.08.022
Luna, J. M., Castro, C., & Romero, C. (2017). MDM Tool: A Data Mining
framework integrated into Moodle. Computer Applications in
Engineering Education, 25, 90-102. doi: 10.1002/cae.21782
Margulieux, L. E., W., McCracken, M., & Catrambone, R. (2016).
A taxonomy to de ne courses that mix face-to-face and online
learning. Educational Research Review, 19, 104-118. doi: 10.1016/j.
edurev.2 016.07.001
Marsh, H. W. (1987). Students’ evaluations of university teaching:
Research ndings, methodological issues, and directions for future
research. International Journal of Educational Research, 11(3), 253-
288. doi: 10.1016/0883-0355(87)90001-2
Nogueira-Frazão, A. G., & Martínez-Solana, Y. (2019). Digital
Infographics: The Key to Information Paradigm. In M. Túñez-López,
V.A. Martínez-Fernández, X. López-García, X. Rúas-Araújo, &
F. Campos-Freire (Eds.), Communication: Innovation & Quality.
Studies in Systems, Decision and Control (Vol 154, pp. 293-312).
Cham: Springer. doi: 10.1007/978-3-319-91860-0_17
Papic, A., & Sušilović, S. (2018). Students’ preferences regarding the
transfer of information and knowledge through infographics tools.
Paper presented at 41st International Convention on Information
and Communica tion Technology, Electronics and Microelectronics
(MIPRO), Opatija, Croatia, 0574-0579. doi: 10.23919/
Sáiz, M.C. (2018a). Laboratorio virtual 1 [Virtual lab 1]. Burgos:
Repositorio Institucional de la Universidad de Burgos. Retrieved from
Sáiz, M.C. (2018b). Laboratorio virtual 2 [Virtual lab 2]. Burgos:
Repositorio Institucional de la Universidad de Burgos. Retrieved from
Sáiz, M.C. (2018c). Infografía 1 [Infograph 1]. Burgos: Repositorio
Institucional de la Universidad de Burgos. Retrieved from http://riubu.
Sáiz, M.C. (2018d). Infografía 2 [Infograph 2]. Burgos: Repositorio
Institucional de la Universidad de Burgos. Retrieved from http://riubu.
Sáiz, M.C. (2018e). Infografía 3 [Infograph 3]. Burgos: Repositorio
Institucional de la Universidad de Burgos. Retrieved from http://riubu.
Sáiz, M. C., Cuesta, I. I., Alegre, J. M., & Peñacoba, L. (2017). Effects
of Different Types of Rubric-Based Feedback on Learning Outcomes.
Frontiers in Education, 2(34). doi:10.3389/feduc.2017.00034
Sáiz, M.C., García-Osorio, C. I., Díez-Pastor, J. F., & Martín, L.J. (2019).
Will personalized e-Learning increase deep learning in higher
education? Information D iscovery and Delivery, 47(1), 53-63. doi:
10.1108/IDD -08-2018- 0039
Sáiz, M. C., Marticorena, R., García-Osorio, C. I., & Díez-Pastor, J. F.
(2017). How do B-learning and learning patterns in uence learning
outcomes? Frontiers in Psychology, 8(745), 1-13. doi: 10.3389/
Sáiz, M.C., Marticorena, R., García-Osorio, C. I., & Díez-Pastor, J.
F. (2019). Does the Use of Learning Management Systems With
Hypermedia Mean Improved Student Learning Outcomes? Frontiers
in Psychology, 10(88), 1-14. doi: 10.3389/fpsyg.2019.00088
María Consuelo Sáiz Manzanares, César Ignacio García-Osorio, José Francisco Díez-Pastor
Santos, C., Pereira-Neto, J. M., & Neves, M. (2019). The In uence of
Infographics in Accessing Information: Multidimensionality in Visual
Representation and Con guration of Different Media. In F. Rebelo &
M. Soares (Eds.), Advances in Ergonomics in Design. AHFE 2018.
Advances in Intelligent Systems and Computing (Vol. 777, pp. 497-
508). Cham: Springer. doi: 10.1007/978-3-319-94706-8_53
SPSS (2016). Statistical Package for the Social Sciences (SPSS) (Version
24). Madrid: IBM.
Taub, M., & Azevedo, R. (2019). How Does Prior Knowledge In uence
Eye Fixations and Sequences of Cognitive and Metacognitive SRL
Processes during Learning with an Intelligent Tutoring System?
International Journal of Arti ci al in Education, 29, 1-28. doi: 10.1007/
Viegas, C., Pavani, A., Lima, N., Marqués, A., Pozzo, I., Dobboletta, E.,
Atencia, V., Barreto, D., Calliari, F., Fidalgo, A., Lima, D., Temporão,
G., & Alves, G. (2018). Impact of a remote lab on teaching practices
and student learning. Computers & Education, 12 6, 201-216. doi:
Vongkulluksn, V. W., Xie, K., & Bowman, M. A. (2018). The role of value
on teachers’ internalization of external barriers and externalization of
personal beliefs for classroom technology integration. Computers &
Education, 118, 70-81. doi: 10.1016/j.compedu.2017.11.009
Wiley, D., Bliss, T. J., & McEwen, M. (2014). Open educational resources:
A review of the literature. In J. Michael Spector, M. David Merrill,
Jan Elen & M. J. Bishop (Eds.), Handbook of research on educational
communications and technology (pp. 781-789). New York, NY:
Springer New York. doi: 10.1007/978-1-4614-3185-5_63
Xie, K., Di Tosto, G., Chen, S-Bo., & Vongkulluksn, V.W. (2018). A
sy ste mat ic r evi ew o f de sig n an d t ech nol ogy com po nen ts o f ed uca tio na l
digital resources. Computers & Education, 127, 90-106. doi: 10.1016/j.
Xie, K., Kim, M. K., Cheng, S-L., & Luthy, N. C. (2017). Teacher
professional development through digital content evaluation.
Textbooks in the digital world. Educational Technology Research and
Development, 65(4), 1067-1103. doi: 10.1007/s11423-017-9519-0
Zimmerman, B.J., & Moylan, A. (2009). Self-regulation: Where
metacognition and motivation intersect. In D. J. Hacker, J. Dunlosky &
A. C. Graesser (Eds.), The educational psychology series. Handbook
of metacognition in education (pp. 299-315). New York, NY, US:
Routledge/Taylor & Francis Group.
... This will help in providing students with personalised help [7,8], which increases their motivation [9]. In particular, teachers, through the use of different didactic resources (e.g., virtual labs, infographics, flipped learning chatbot experiences, serious games, multimedia resources [10], online project-based learning (OPBL) methodology) [11,12] and intelligent tutoring systems (ITS) [13], can promote individual and group work [14]. In addition, the joint use of ALT and LMS allows the recording of interactions during the teaching-learning process. ...
... However, the presence of a large number of resources that apply ALT does not ensure better academic results or greater satisfaction. The resources that have been shown to be most effective are those that contain automatic process-oriented feedback [11,27]. Their use seems to be directly related to better learning outcomes and higher student satisfaction [11,27]. ...
... The resources that have been shown to be most effective are those that contain automatic process-oriented feedback [11,27]. Their use seems to be directly related to better learning outcomes and higher student satisfaction [11,27]. As a result, researching the development of automatic feedback using ALT resources is both a challenge and an area requiring continuous focus in this field. ...
Full-text available
: The use of advanced learning technologies (ALT) techniques in learning management systems (LMS) allows teachers to enhance self-regulated learning and to carry out the personalized monitoring of their students throughout the teaching–learning process. However, the application of educational data mining (EDM) techniques, such as supervised and unsupervised machine learning, is required to interpret the results of the tracking logs in LMS. The objectives of this work were (1) to determine which of the ALT resources would be the best predictor and the best classifier of learning outcomes, behaviours in LMS, and student satisfaction with teaching; (2) to determine whether the groupings found in the clusters coincide with the students’ group of origin. We worked with a sample of third-year students completing Health Sciences degrees. The results indicate that the combination of ALT resources used predict 31% of learning outcomes, behaviours in the LMS, and student satisfaction. In addition, student access to automatic feedback was the best classifier. Finally, the degree of relationship between the source group and the found cluster was medium (C = 0.61). It is necessary to include ALT resources and the greater automation of EDM techniques in the LMS to facilitate their use by teacher
... Moreover, the joint use of LMS and ALT enables interactions to be recorded (Azevedo and Gašević, 2019;Hosain et al., 2019;Noroozi et al., 2019). The use thereof accounts for over 72% of variance in student learning outcomes (Sáiz-Manzanares et al., 2019a). One possible reason is that the use of ALT boosts SRL learning and the use of metacognitive strategies (planning, evaluation and design of task solving; Hull et al., 2015) as well as student motivation (Zimmerman, 2005), all of which enhances personalised learning (Enembreck and Barthès, 2005;Sáiz-Manzanares et al., 2019b;Martín-Antón et al., 2020), learner autonomy (Remesal et al., 2017;Zorrilla-Pantaleón et al., 2021) as well as self-awareness and self-reflection (Taub et al., 2017;Nurmi et al., 2020). ...
... Nevertheless, research is required into the design of such environments, since the mere use of virtual platforms by no means ensures effective learning (Yamada and Hirakawa, 2016;Park and Jo, 2017;Sáiz-Manzanares et al., 2017). Carefully designed methodological aspects (objectives, conceptual and procedural content, assessment criteria) as well as technological aspects (Sáiz-Manzanares et al., 2019a) must be applied if these environments are to foster the development of metacognitive strategies and self-regulation. Moreover, virtual learning platforms must embrace student follow-up systems so that teachers can track the behaviour of each of their students throughout the learning process (Jommanop and Mekruksavanich, 2019;Troussas et al., 2021;Krouska et al., 2021b;Sáiz-Manzanares et al., 2021b). ...
Full-text available
Higher education in the 21st century faces the challenge of changing the way in which knowledge is conveyed and how teachers and students interact in the teaching-learning process. The current pandemic caused by SARS-CoV-2 has hastened the need to face up to this challenge and has furthered the need to approach the issue from the perspective of digitalisation. To achieve this, it is necessary to design training programmes geared towards teaching staff and which address both the use of technology and instructional design aimed at promoting the development of self-regulated learning (SRL) and automatic feedback systems. In this study, work was carried out with 23 teachers (8 inexperienced and 15 experienced teachers) in a training programme conducted through Moodle. The aims were: (1) to test whether there were any significant differences between the behaviour patterns of new teachers compared to experienced teachers, (2) to determine whether clusters of behaviour patterns corresponded to the type of teacher and (3) to ascertain whether the level of teacher satisfaction with the training activity in digital teaching will depend on the type of teacher. A quantitative as well as a qualitative design was applied. Differences were found in the behaviour patterns in the training activities for the development of rubrics and use of learning analytics systems in virtual learning environments. It was also found that the type of teacher did not correspond exactly to the behaviour cluster in the learning platform. In addition, no significant differences were found in the level of satisfaction between the two kinds of teacher. The main contribution this study makes is to provide a detailed description of the training stage as well as the materials required for its repetition. Further analytical studies are required on teacher perception of training programmes in digital teaching in order to provide personalised training proposals that lead to an effective use of teaching in digital environments.
... In the last decade, the application of data mining techniques has been included to analyze data [58][59][60][61]. These techniques can be divided into supervised and unsupervised learning techniques [55]. ...
... Regarding the second techniques, there is no supervisor and only data entry is available; the objectives in this case are to find regularities, irregularities, relations, similarities, and associations at the entry [48]. However, in data analysis, the application of more traditional techniques such as those offered by descriptive statistics, the analysis of correspondence, the analysis of variance, the analysis of regression, etc., should not be forgotten [61]. ...
Full-text available
In the last few years, research in the field of sustainability has experienced a significant increase in interest between sustainability and other areas (inclusive education, active methodologies, and society). Moreover, the use of mixed research methods (quantitative and qualitative) along with the application of data mining techniques, enables the analysis of information and the connection between the different studies. The objectives of this paper were: (1) To establish the results of the research related to the concepts of sustainability, inclusive education, and disability. (2) To study the key concepts that are detected in the articles selected with respect to the concepts of sustainability, inclusive education, disability, and their relations. In order to do so, two studies were carried out (quantitative and qualitative). In the first study, K-means and heat map clustering techniques were applied. In the second study, the technique of text mining was applied. One hundred and thirty-three scientific papers were studied, of which 54 fulfilled all the inclusion criteria. Three clusters were found in the first study; cluster 1 included the categories: inclusive society, educational innovation, and active methodologies. Cluster 2 included active methodologies and society and economy and cluster 3 included inclusive society and society and economy. In the second study, the highest Krippendorff's Alpha coefficient were found in articles that linked sustainability with social transformation stemming from a change in education by means of the use of active teaching methods and technological resources. The research moves towards the development of competencies in sustainability at all stages of the educational system, and in all areas of knowledge.
... Recent research also shows that the use of hypermedia resources facilitates the development of deeper and higher quality learning (De Kock, 2016), as systematic planning through interactive platforms, where both audio and video elements are additionally combined, facilitates the development of metacognitive strategies during task solving. These platforms set the objective, the planning in the development of the resolution, the evaluation of the resolution steps and the evaluation of the final result from a respect for the student's learning pace (Sáiz-Manzanares et al., 2019a). Working from the OPBL methodology combined with the feedback resources that LMSs include further increases motivation and autonomy in student learning (Chen et al., 2020). ...
Full-text available
Monitoring students in Learning Management Systems (LMS) throughout the teaching–learning process has been shown to be a very effective technique for detecting students at risk. Likewise, the teaching style in the LMS conditions, the type of student behaviours on the platform and the learning outcomes. The main objective of this study was to test the effectiveness of three teaching modalities (all using Online Project-based Learning -OPBL- and Flipped Classroom experiences and differing in the use of virtual laboratories and Intelligent Personal Assistant -IPA-) on Moodle behaviour and student performance taking into account the covariate "collaborative group". Both quantitative and qualitative research methods were used. With regard to the quantitative analysis, differences were found in student behaviour in Moodle and in learning outcomes, with respect to teaching modalities that included virtual laboratories. Similarly, the qualitative study also analysed the behaviour patterns found in each collaborative group in the three teaching modalities studied. The results indicate that the collaborative group homogenises the learning outcomes, but not the behaviour pattern of each member. Future research will address the analysis of collaborative behaviour in LMSs according to different variables (motivation and metacognitive strategies in students, number of members, interactions between students and teacher in the LMS, etc.).
... It has been proven that with these techniques, the prediction percentage is in the range of 79-83% [11]. Additionally, it has been found that the use of teaching methodology in virtual environments based on self-regulated learning (SRL) [12] together with project-based learning (PBL) [13,14] and the use of continuous assessment methodology [15] are predictors of the achievement of effective learning (60.4%) [16] and decrease the dropout rate [15,17]. In summary, these studies indicate that one of the ways to decrease the percentage of students who drop out of university is to have systems in the LMS that facilitate the detection of the student at risk throughout the teaching-learning process [18,19]. ...
Full-text available
In this study, we used a module for monitoring and detecting students at risk of dropping out. We worked with a sample of 49 third-year students in a Health Science degree during a lockdown caused by COVID-19. Three follow-ups were carried out over a semester: an initial one, an intermediate one and a final one with the UBUMonitor tool. This tool is a desktop application executed on the client, implemented with Java, and with a graphic interface developed in JavaFX. The application connects to the selected Moodle server, through the web services and the REST API provided by the server. UBUMonitor includes, among others, modules for log visualisation, risk of dropping out, and clustering. The visualisation techniques of boxplots and heat maps and the cluster analysis module (k-means ++, fuzzy k-means and Density-based spatial clustering of applications with noise (DBSCAN) were used to monitor the students. A teaching methodology based on project-based learning (PBL), self-regulated learning (SRL) and continuous assessment was also used. The results indicate that the use of this methodology together with early detection and personalised intervention in the initial follow-up of students achieved a drop-out rate of less than 7% and an overall level of student satisfaction with the teaching and learning process of 4.56 out of 5.
... Likewise, it was confirmed that the teaching style appears to condition student behavior on the LMS, which supports what has been found in other investigations [1,2,[8][9][10][11][12]. This aspect sheds light on the frequency of use of resources and activities, suggesting that it depends on the instructional design that the teacher implements. ...
Full-text available
Teaching in Higher Education is with increasing frequency completed within a Learning Management System (LMS) environment in the Blended Learning modality. The use of learning objects (activities and resources) offered by LMS means that both teachers and students require training. In addition, gender differences relating to the number of students in STEM (Science, Technology, Engineering, and Mathematics) and Non-STEM courses might have some influence on the use of those learning objects. The study involves 13 teachers (6 experts in e-Learning and 7 non-experts) on 13 academic courses (4 STEM and 9 Non-STEM) and a detailed examination of the logs of 626 students downloaded from the Moodle platform. Our objectives are: (1) To confirm whether significant differences may be found in relation to the use of learning objects (resources and activities) on Moodle, depending on the expertise of the teacher (expert vs. non-expert in e-Learning); (2) To confirm whether there are significant differences between students regarding their use of learning objects, depending on the expertise of the teacher (expert vs. non-expert in e-Learning); (3) To confirm whether there are significant differences for the use of learning objects among students as a function of gender. Differences were found in the use of Moodle learning objects (resources and activities) for teachers and for students depending on the expertise of the teacher. Likewise, differences were found for the use of some learning objects as a function of gender and the degrees that the students were following. Increased technological training for both teachers and students is proposed, especially on Non-STEM qualifications, in order to mitigate the effects of the technological gap and its collateral relation with the gender gap and the digital divide.
... Previous studies in different disciplines have examined the implementation of blended learning and its potential benefits for learning (see Bergmann & Sams, 2012;Brent et al., 2002;Davies et al., 2013;Lage et al., 2000;Sáiz et al., 2020aSáiz et al., , 2020bSáiz et al., 2019aSáiz et al., , 2019b. Stone (2012) flipped two courses, genetic diseases and general biology. ...
Full-text available
Objectives: The objective of the study was to understand the views, experiences, and challenges that preservice English teachers perceived in a flipped classroom, which was implemented in a language teacher education program following our institution’s desire to promote blended learning. Method: Two focus groups were conducted with the students (19). We analyzed the data using thematic analysis. Results: The results show that the students perceived flipped practices as innovative and beneficial for their learning. However, their responses also reveal that the autonomy required from them and taking more responsibility for their own learning were particularly challenging because of their lack of familiarity with the approach and their past learning experiences rooted in traditional teaching and learning. Implication for Theory and/or Practice: Our argument is that prior to implementing blended learning approaches, language teacher education and higher education should examine students’ readiness for such approaches and provide them with support for carrying out those practices.
Full-text available
Situación actual del estado de la cuestión (estado del arte) y del propio tema o asignatura para la que se plantea el proyecto La actualización en la formación de profesionales dentro del ámbito de la atención temprana (atención a niños y niñas con edades de 0-6 años y a sus familias) es una necesidad que se ha visto incrementada en los últimos años debido al avance imparable de lo que se ha denominado Advance Learning Technologies (ALT) (Azevedo, 2014) y de las técnicas de Inteligencia Artificial (AI) (Saiz-Manzanares y Velasco-Saiz, 2020). Concretamente, la Agenda 2030 establece como uno de sus objetivos prioritarios conseguir en las personas una educación de calidad a lo largo de la vida. De igual modo, la Unión Europea ha establecido como una de sus metas prioritarias lograr una digitalización en el ámbito de la docencia universitaria. En este estudio esta necesidad se ha concretado respecto de la formación de profesionales de la sanidad y de la educación. Específicamente, atendiendo a la definición de Atención Temprana según lo explicitado en el libro blanco, esta hace referencia al "conjunto de intervenciones dirigidas a la población infantil de 0-6 años, a la familia y al entorno, que tienen por objetivo dar respuesta lo más pronto posible a las necesidades transitorias o permanentes que presentan los niños con trastornos en su desarrollo o que tienen el riesgo de padecerlos. Estas intervenciones, que deben considerar la globalidad del niño, han de ser planificadas por un equipo de profesionales de orientación interdisciplinar o transdisciplinar (Federación Estatal de Asociaciones de Profesionales de Atención Temprana, 2003, p. 12). En esta definición se hace hincapié en la necesidad de una detección precoz de posibles dificultades en el desarrollo y de una intervención interdisciplinar y/o transdisciplinar sobre la globalidad de la persona. Dicha definición situada en el marco de la sociedad del siglo XXI se debe dirigir de una forma unívoca hacia la inserción de medios y recursos incluidos en ALT y en AI. Por ello, específicamente tanto los procesos de evaluación como los de intervención en los entornos de educación y atención en la primera y en la segunda
Full-text available
En este trabajo se propone un método basado en fórmulas para evaluar preguntas de tipo calculada simple y calculada de opción múltiple. El método realiza la conversión de fórmulas de notación infija a postfija, para su posterior evaluación y resolución. Las preguntas incluyen incógnitas que toman valores aleatorios basados en rangos o en conjuntos específicos de valores. Esto contribuye a la asignación de preguntas con valores independientes en la evaluación de cada estudiante, por consecuencia, las respuestas serán diferentes. El método es validado mediante la evaluación de un cuestionario (examen) que forma parte de un sistema, el cual tiene como finalidad gestionar el conocimiento de estudiantes en el área disciplinar de matemáticas de nivel universitario.
Full-text available
Healthcare workers face life and death situations and due to this, their work can be physically and emotionally exhausting. Fatigue, low self-esteem, depersonalization and professional isolation are their main characteristics. In addition to these factors, the SARS-CoV-2 health alert also contribute to this.
Full-text available
Learning management systems (LMSs) that incorporate hypermedia Smart Tutoring Systems and personalized student feedback can increase self-regulated learning (SRL), motivation, and effective learning. These systems are studied with the following aims: (1) to verify whether the use of LMS with hypermedia Smart Tutoring Systems improves student learning outcomes; (2) to verify whether the learning outcomes will be grouped into performance clusters (Satisfactory, Good, and Excellent); and (3) to verify whether those clusters will group together the different learning outcomes assessed in four different evaluation procedures. Use of the LMS with hypermedia Smart Tutoring Systems was studied among students of Health Sciences, all of whom had similar test results in the use of metacognitive skills. It explained 38% of the variance in student learning outcomes in the evaluation procedures. Likewise, three clusters that grouped the learning outcomes in relation to the variable ‘Use of an LMS with hypermedia Smart Tutoring Systems vs. No use’ explained 60.4% of the variance. Each cluster grouped the learning outcomes in the different evaluation procedures. In conclusion, LMS with hypermedia Smart Tutoring Systems in Moodle increased the effectiveness of student learning outcomes, above all in the individual quiz-type tests. It also facilitated personalized learning and respect for the individual pace of student-learning. Hence, modules for the analysis of supervised, unsupervised and multivariate learning should be incorporated into the Moodle platform to provide teaching tools that will undoubtedly contribute to improvements in student learning outcomes. HIGHLIGHTS-Learning management systems (LMS) that incorporate hypermedia Smart Tutoring Systems and personalized student feedback can increase self-regulated learning (SRL). -Learning management systems with hypermedia Smart Tutoring Systems increased the effectiveness of student learning outcome. -The use of an LMS with hypermedia Smart Tutoring Systems vs. No use’ explained 60.4% of the variance in student learning outcome.
Full-text available
In an era of accountability, implementation of computer-based instruction (CBI) may be attractive to personalize learning and raise test scores. Although prior work has examined how CBI may serve these purposes, we demonstrate that it also has an unintentional impact on how teachers allocate time in the classroom. The purpose of this study was to investigate the potential collateral damage of implementing CBI in elementary classrooms. Using the implementation of the supplemental CBI, Spatial Temporal (ST) Math as a case study, we analyze how 468 teachers report spending instructional minutes in each subject per week, and which subjects are reported as being limited to accommodate the program. Results reveal teachers vary in time spent across subjects and from where they draw time for ST Math. Most time for ST Math comes from a reduction of non-math core subject time with some variation according to grade. We provide initial evidence of teacher autonomy in subject time allocation within the context of CBI implementation; further study is needed to illuminate drivers of teacher's decisions.
Purpose – Recent research in higher education has pointed out that personalized e-learning through the use of learning management systems, such as Moodle, improves the academic results of students and facilitates the detection of at-risk students. Design/methodology/approach – A sample of 124 students following the Degree in Health Sciences at the University of Burgos participated in this study. The objectives were as follows: to verify whether the use of a Moodle-based personalized e-learning system will predict the learning outcomes of students and the use of effective learning behaviour patterns and to study whether it will increase student satisfaction with teaching practice. Findings – The use of a Moodle-based personalized e-learning system that included problem-based learning (PBL) methodology predicted the learning outcomes by 42.3 per cent, especially with regard to the results of the quizzes. In addition, it predicted effective behavioural patterns by 74.2 per cent. Increased student satisfaction levels were also identified through the conceptual feedback provided by the teacher, arguably because it facilitated a deeper understanding of the subject matter. Research limitations/implications – The results of this work should be treated with caution, because of the sample size and the specificity of the branch of knowledge of the students, as well as the design type. Future studies will be directed at increasing the size of the sample and the diversity of the qualifications. Originality/value – Learning methodology in the twenty-first century has to be guided towards carefully structured work from the pedagogic point of view in the learning management systems allowing for process-oriented feedback and PBL both included in personalized e-learning systems.
With the rise of the Internet and the proliferation of online content, the design and evaluation of educational digital resources (EDRs) are pressing and challenging issues. They warrant an investigation of what exactly are the features that increase the quality of EDRs. In a previous professional development program, we trained and supported teachers in evaluating and selecting EDRs with the support of a scientifically validated rubric. In this present study, through quantitative, qualitative, and text-mining methods, we analyzed the review data of 1200 resources produced that professional development program in order to provide a big picture of the quality of currently available products, and to identify the features that characterize quality digital resources. Our findings suggest the need for digital repositories to reflect or make visible how resources fit particular instructional design models.
Objective: Given participants' research literacy is essential for clinical trial participation, evidence-based strategies are needed that improve literacy and easily accessed online. We tested whether an infographic letter-that illustrated how dropouts can distort study conclusions-improved participant knowledge about the impact of dropouts relative to a control letter. Method: In three distinct online samples purposely recruited to assess reproducibility, young ethnically diverse adults were randomized to read an infographic letter or control letter in a hypothetical scenario. Secondary outcomes included participants' perceived transparency of the research organization, perceived value of retention, and perceived trust of the organization. We purposely included two discriminant items, perceived value for the trial outcome and keeping commitments in general, both hypothesized not to change. Results: Across samples, ∼20% more infographic participants correctly answered how dropouts affected study conclusions than control participants. For example (Experiment 3), nearly 90% of infographic participants correctly answered versus only two thirds of controls (88.7% vs. 66.7%, absolute percentage difference 22.0%, p < .0001). Infographic participants had substantially higher transparency, perceived value for retention, and trust (Cohen's ds = 0.4-1.0, ps < .0001), yet importantly did not value the study outcome or report keeping commitments more than control participants (Cohen's ds = 0.0-0.1, ps > .10). Conclusions: Promisingly, this transparent, visually powerful methodological infographic improved knowledge and trust. Future trials could embed and experimentally test whether such low-cost online infographics improve not only research literacy, but also trial retention, especially among populations with less initial trust about research. (PsycINFO Database Record
The issues of memory continuity and informative credibility present themselves to Media Online as one of the main everyday obstacles. The search for structures and mechanisms for informative condensation and qualitative progression is the challenge that arises for this second stage of the current Technological Paradigm of Information and Communication. In a process where, due to the demands of a Net Generation, already digital native, visuality is in command, Digital Literacy is a crucial and undeniable necessity and, to Journalism, the Digital Infographic presents itself, as the structure that embraces all the Media competences, conferring to information and time, cohesion, longevity and efficiency.
Remote Laboratories have become part of current teaching and learning, particularly in engineering. Their potential to aid students beyond their hands-on lab classes has been a matter of discussion in literature. Teachers and researchers are aware that the thorough analysis of both strengths and shortcomings of remote labs in didactical implementations may not only lead to the improvement of these resources but also of the pedagogical implications in engineering classes. The present study was carried out in a Higher Education Institution in Brazil in two different courses during three consecutive semesters where a remote lab (VISIR) addressing electric and electronic topics was implemented, yielding 471 students’ academic results and opinions. These students’ results (while using VISIR) cross-analysed with the course characteristics, reveal some factors teachers may tackle to foster student learning and motivation. The conclusions point to the need for VISIR interface modernization and showed it is more useful in basic courses than in more advanced ones, when dealing with classic lab experiments. Results also show that teachers’ involvement plus their ability to brief students on VISIR's usefulness have a significant influence not only on students’ performance but also on their perception of learning and satisfaction with the tool. In the analysed cases, the students with more learning needs seemed to be the ones who could benefit more from VISIR.
Infographics are powerful tools for presenting information and information design has distinguished itself as a specific discipline that has efficient communication of information as main task. With new online revolution, concept of infographics has evolved by combining several ways of presentation in many formats and there are different denominations; simultaneously, information designers should consider some principles and differences when designing for internet or printed material. This article presents a theoretical reflection and practical examples that pretend to reflect these matters, trying to create a relationship between infographic typologies, objectives and how information is presented, considering final users.