ArticlePDF Available

Constructionist Learning Tool for Acquiring Skills in Understanding Standardised Engineering Drawings of Mechanical Assemblies in Mobile Devices

Authors:

Abstract and Figures

The purpose of Graphic Design is to transfer information about design into reality and concerns the analysis, design and representation of mechanical components and assemblies. For the correct rendering of mechanical components, this discipline requires a command of, and the ability to, manage techniques and systems for graphical representation and standardisation; 3D models in a virtual environment enable engineering students to develop graphical skills and spatial awareness. The objective of the present study is the development of an application for smart devices (mobile phones and tablets), based on the constructionist theory of learning, which will enable first year engineering degree students to acquire the technical drawing knowledge and skills necessary to render mechanical assemblies. The mobile application tested and designed in this work is called ARPAID. It is a learning tool aimed at teaching students about the representation of mechanical assemblies as part of an engineering Graphic Design course. Teaching material and a process for evaluation have been designed. A detailed description is given of a classroom activity accompanied by a tabulation and analysis of the results obtained. This mobile application, when used in a Graphic Design course, promotes a more rapid understanding of spatial relationships and problems, fosters students’ learning and motivation, and develops higher order skills. Results from before and after the use of the application will be presented and do indeed show significant improvements in student performance.
Content may be subject to copyright.
sustainability
Article
Constructionist Learning Tool for Acquiring Skills in
Understanding Standardised Engineering Drawings of
Mechanical Assemblies in Mobile Devices
Fernando J. Fraile-Fernández 1, * , Rebeca Martínez-García1and Manuel Castejón-Limas 2


Citation: Fraile-Fernández, F.J.;
Martínez-García, R.; Castejón-Limas,
M. Constructionist Learning Tool for
Acquiring Skills in Understanding
Standardised Engineering Drawings
of Mechanical Assemblies in Mobile
Devices. Sustainability 2021,13, 3305.
https://doi.org/10.3390/su13063305
Academic Editor: Daniel Burgos
Received: 13 February 2021
Accepted: 12 March 2021
Published: 17 March 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
1Department of Mining Technology, Topography and Structures, University of León, 24071 León, Spain;
rmartg@unileon.es
2Department of Mechanical, Informatics and Aerospace Engineering, University of León, 24071 León, Spain;
manuel.castejon@unileon.es
*Correspondence: fjfraf@unileon.es; Tel.: +34-987-291-000 (ext. 5352); Fax: +34-987-291-787
Abstract:
The purpose of Graphic Design is to transfer information about design into reality and
concerns the analysis, design and representation of mechanical components and assemblies. For the
correct rendering of mechanical components, this discipline requires a command of, and the ability
to, manage techniques and systems for graphical representation and standardisation; 3D models in a
virtual environment enable engineering students to develop graphical skills and spatial awareness.
The objective of the present study is the development of an application for smart devices (mobile
phones and tablets), based on the constructionist theory of learning, which will enable first year
engineering degree students to acquire the technical drawing knowledge and skills necessary to
render mechanical assemblies. The mobile application tested and designed in this work is called
ARPAID. It is a learning tool aimed at teaching students about the representation of mechanical
assemblies as part of an engineering Graphic Design course. Teaching material and a process for
evaluation have been designed. A detailed description is given of a classroom activity accompanied
by a tabulation and analysis of the results obtained. This mobile application, when used in a
Graphic Design course, promotes a more rapid understanding of spatial relationships and problems,
fosters students’ learning and motivation, and develops higher order skills. Results from before and
after the use of the application will be presented and do indeed show significant improvements in
student performance.
Keywords:
constructionism; virtual learning environment; computer aided drawing; technical engi-
neering drawing; mechanical assemblies; unity3d; mobile application; technology-enhanced learning
1. Introduction
Over the last few years many Engineering Schools have updated their plans of study.
These new plans have begun, little by little, to incorporate new technologies, reflecting
how the university education system is adapting to the world of industry 4.0.
The world of today is a network of instantaneous communication systems where
information can be accessed or sent on the spot. The society of the 21st century is interactive,
information is portable, and connectivity almost total [
1
]. One of the principal academic
strategies set out for education in the EU’s document, Horizon 2020 [
2
], is the incorporation
of Virtual Learning Environments and Augmented Reality (AR) into higher education.
The subject of Graphic Design is transversal and technological in nature. It embraces
the techniques of graphic communication to express ideas and concepts and it could be
defined as a technology that coordinates intellectual and instrumental skills [
3
]. It is an
area of knowledge that focuses on the analysis, design, and representation of mechanical
pieces, mechanisms and assemblies, components, installations, plots, etc. To ensure the
correct rendering of mechanical assemblies, this discipline requires a command of, and
ability to, manage techniques and systems for standardised graphical representation.
Sustainability 2021,13, 3305. https://doi.org/10.3390/su13063305 https://www.mdpi.com/journal/sustainability
Sustainability 2021,13, 3305 2 of 31
Virtual Learning Environments and Augmented reality (AR) are technologies with
far reaching potential in many disciplines, including education. AR is a hybrid reality that
combines virtual and physical information in real time; that is, through the use of technol-
ogy, virtual information is added to physical data about a real object or environment to
create an artificial reality [
4
]. Alan B. Craig (2013), in “Understanding Augmented Reality” [
5
],
defines AR as “a medium in which information is added to the physical world in registration with
the world”. In his article entitled “A Survey of Augmented Reality” [
6
], Ronald T. Azuma
(1997) defines three key characteristics of AR: the combination of the real and the virtual,
real-time interaction, and 3D registration.
There is an ever-growing body of literature focusing on teaching and learning experi-
ences with VR and AR. The majority of the applications have been within the compulsory
schooling system (ESO: the Spanish Secondary Education system), and to a lesser extent
in the university and higher education systems [
4
,
7
11
]. AR has been trailed principally
in science subjects including: Biology [
12
14
], Mathematics [
15
,
16
], Physics [
7
,
17
], Chem-
istry [
18
], and Earth sciences [
19
,
20
], with some studies taking place in the Humanities
and Arts [
21
23
]; the least explored field is that of teacher education [
24
]. All these studies
agree that the use of AR in educational environments leads to a series of educational
benefits amongst which we find: increased motivation and interest from students about the
subject material; the development of creative skills; positive learning; increased attention,
commitment and satisfaction; improved understanding of subject knowledge; improved
educational outcomes; increased recall and self-sufficiency. The principal limitations de-
scribed include: access to technology; technical problems with the App used; the small
number of students participating in studies; problems with App usability; cognitive over-
load; poor development of higher order skills; limitations on the ability of educators to
adapt the App; short duration of studies; emphasis on qualitative data.
Many research studies have demonstrated that technological applications increase
academic achievement and student motivation; these effects are more pronounced for
applications involving the greatest levels of student interaction with the environment.
According to Bower et al. [
24
] the use of AR in education provides particular benefits for the
acquisition of knowledge and learning content that may be abstract, difficult to understand
or difficult to observe. It is an innovative technology that enables the visualisation of data
and 3D modelling, and for this reason it is an ideal tool for the development of graphical
skills and spatial awareness in engineering students.
In many Engineering Schools across our country greatest emphasis is placed on the
acquisition of deep theoretical learning with the development of practical skills essential for
future professional development being relegated to second place. This tends to demotivate
students from learning subject content. In master and theory classes we observe high levels
of student passivity, a lack of participation and a lack of active engagement behaviours.
The problem lies in the educational model (which is sometimes far too teacher centred)
and the teaching body, which does not practice methods that promote active engagement.
Student motivation and enabling students to see, at every opportunity, the potential, real
world uses for the material they are working on has been one of the fundamental driving
forces behind this investigation into teaching innovation.
The question we would like to pose is the following: is there a realistic way of solving
the difficulties encountered in presential learning environments that is efficient, cheap,
and accessible for all our students while at the same time provides the opportunity of
improving understanding, spatial awareness, attention span, assessment, and tuition? Our
answer is yes. We believe that the use of mobile applications based on Virtual Environments
and Augmented Reality (AR) in the classroom, within a constructionist paradigm of learning,
can provide us with the means to achieve, or perhaps it would be better to say “catalyse”, the
achievement, to a greater or lesser degree, of the objectives laid out in our initial question.
In order to reverse the current situation, we need to modify the educational model and
put in place methods that are active and constructionist. In this way, by allowing students to
interact and manipulate the models provided, we hope to enable them to construct their
Sustainability 2021,13, 3305 3 of 31
own knowledge base in a way that is deeper and more enduring and also allows them to
acquire more active classroom behaviours.
It is safe to say that practically all students in our faculty habitually use either a
mobile phone or a tablet, and that these devices can all capture images of the world with a
high-resolution camera. Consider then, the influence on learning and the acquisition of
skills if we could develop an application that could superimpose onto this real image other
information relevant to a classroom learning activity, and could do this synchronously, via
a certain chain of events or “triggers”; an application with which students can interact,
respond to requests for information or ask questions.
Through this educational proposal we hope to deliver a student experience that is
both innovative and, at the same time, strengthens their knowledge of Graphic Design and
develops their spatial awareness. Our method is based on presenting students with real-life
problems, the understanding of which is facilitated through the virtual environment and AR
such that they are enabled to use what they learn as a tool for solving problems, generating
ideas, and developing improvements in the field of engineering. We hope to increase
academic achievement, motivate and encourage student participation by combining the
use of mobile phones or tablets, with AR, a cutting-edge technology that is also very
attractive due to its links with video gaming. Using our application, students will be able to
create, share, and use learning content throughout a given classroom activity; at the same
time, this technology will facilitate the dissemination of resources at critical moments of the
learning process. In addition, we will evaluate the app through an investigation comparing
the achievements of an experimental group (using the app) compared to a control group
(not using the app) to determine both qualitatively and where possible, quantitatively,
whether our app improves learning outcomes.
2. Experimental Design: Methods and Tools
2.1. Educational Paradigms of Learning
A paradigm is a particular way of seeing the world, of interpreting reality, based on
a specific philosophy. According to the author, Martin-Trujillo L., “a paradigm is a specific
framework through which we view the world, understand it, interpret it, and intervene in it” [25].
The principal theories upon which we have based the design of our application are
the following:
Constructivism embraces a set of psychological and pedagogical theories based on
the idea that all educational input should have an impact on Human Development and
that this is the fundamental objective of education. Learning takes place within the in-
dividual and arises out of the relationships and exchanges individuals have with their
environment [25,26].
It is based on a model of active, constructive learning. Students
should construct their own understanding of reality linked to their prior knowledge. Con-
structivism is organised around three basic concepts: the student is actively responsible
for their learning process; knowledge is the result of a constructive process at a social
level; and the function of the teacher is to enable the student’s process of constructing
learning through direction and guidance. This theory maintains that instructive learning
must provide macro and micro-level support to the student so they can construct their own
knowledge, and be involved in meaningful learning [27].
One of the foundations of constructivism is cognitive development, as proposed by Pi-
aget. Cognitive development is a set of transformations that take place throughout life and
through which a person acquires new knowledge and improves their skills in perception,
thinking and understanding. These skills are used to solve the practical problems of life.
Piaget studied how children interpret the world at different ages and how they acquire
knowledge as they develop [
28
]. The child has their own logic and ways of knowing, and
these follow predictable patterns of development as the child interacts with the world
and reaches maturity. Children actively construct knowledge about their environment
using what they already know and by interpreting new happenings and objects. Piaget
divided cognitive development into four stages: sensorimotor, preoperational, concrete
Sustainability 2021,13, 3305 4 of 31
operational, and formal operational. It is assumed that the child’s thinking is qualitatively
different at each stage of development. According to Piaget, cognitive development does
not simply consist of qualitative changes in experience and skills but rather in radical
transformations in how knowledge is organised. The theory states that the acquisition of
knowledge and the development of thought are achieved through a principle of adaptation
whereby thinking is moulded by reality through two basic processes: assimilation and
accommodation. Assimilation consists in incorporating those elements exterior to the
individual to the structures of knowledge they already possess. Accommodation is the
process that completes assimilation, that is, once an experience has been incorporated into
an individual’s cognitive structures, a readjustment or re-accommodation is required to
integrate the new knowledge with pre-learned knowledge.
Another constructivist theory is the sociocultural theory proposed by Lev Vygotsky
which gives greater relevance to the social environment as a facilitator of development
and learning. In contrast to Piaget, Vygotsky does not talk about assimilation, but rather
appropriation [
29
]. Vygotsky’s theory is characterised by three fundamental factors and the
interactions between them: interpersonal factors, historic-cultural factors, and individual
factors. He stated that it is not possible to understand a child’s development without
understanding the culture in which they grew up. He thought that the thinking patterns
of an individual were not dependent on innate characteristics but rather that they were
a product of cultural institutions and social activities. A child is born with basic mental
skills, amongst which are perception, attention, and memory, and thanks to interactions
with peers and more competent adults, these innate functions develop into superior mental
functions. Vygotsky considers five fundamental concepts: mental functions; psychological
skills; the zone of proximal development; thinking tools; and mediation [30].
Meaningful learning is a concept developed by David Ausubel and refers to a type of
learning in which the student makes associations between new learning and previously
learned knowledge, readjusting and reconstructing both sets of information in the process.
Meaningful learning is a mechanism that allows both the acquisition and ordering of large
numbers of ideas and information in any field of knowledge [31].
Discovery learning proposed by Jerome Bruner is based around the idea that what is
going to be learned should not be given in its final form, but rather it must be reconstructed
by the student before it can be incorporated into their existing cognitive structures. Dis-
covery learning encourages mental development and there is nothing so personal as a
discovery made by oneself, personally. The theory maintains that a student should acquire
knowledge for themselves, guided by the teacher, but that their experience should be led
by the discoveries they themselves make. Discovery is a form of inductive reasoning since
students pass from specific examples to the formation of rules, concepts, and general prin-
cipals. This type of learning is also known as problem-based learning,inquiry-based learning,
or experiential learning [32].
Seymour Papert’s notion of constructionism coincides with constructivism in its con-
ceptualisation of learning as “the creation of structures of knowledge”, independent of the
circumstances of learning. However, his constructionism transcends this concept and
establishes a set of circumstances in which learning becomes more meaningful, specifically
the context of learning by doing. Papert maintained that “learning is hugely facilitated when
the learner is consciously engaged in the construction of a meaningful social object or entity which
can take many forms from a sandcastle, to a poem, to a computer program” [33].
In his book, “The children’s machine”, Papert talks of the important role these real-world
social objects play as catalysts for those constructed inside the mind. As a result, Papert
identifies two types of construction, one external and the other internal to the individual.
Moreover, a successive cycle of learning is set in train: as the real-world object is built,
knowledge is constructed in the mind and this knowledge gives the potential for the
construction of more complex real-world constructions, which in turn generate further
knowledge, and so on [34].
Sustainability 2021,13, 3305 5 of 31
According to Papert, better learning is not derived through finding better methods of
teaching, but through offering the learner better opportunities to construct. Fundamental
considerations here are the importance of providing the student with adequate materi-
als, the environment or surroundings, and indeed the social context in which learning
takes place.
For a technology to be effective, it is necessary to provide novice learners with an easy
way to get started (what Papert terms “low floors”) but also, over time, offer them a way of
working on projects that are ever more sophisticated (“high ceilings”) [35].
A constructionist learning tool, therefore, allows students the opportunity to exper-
iment, explore and express themselves freely, without a pre-established script, a fixed
itinerary or timetable. By also providing the student with other ways to actively inter-
act, i.e., through a screen, we are increasing the number of ways they can express their
ideas and build their projects, In this way we are contributing to their development as
Creative Thinkers or, in the words of Mitchel Resnick, creator of the graphical programming
language, Scratch, and disciple of Papert, so called X-students [35].
2.2. Problem Definition and Objectives
Over the years that we have been teaching Graphic Design we have observed that
many students have difficulties understanding the sections on Systems and Assemblies in
regard to understanding their function, the position of pieces and the relative displacement
between components. Understanding these issues is fundamental for correct, standardised
representation of mechanical assemblies. Errors in understanding the mechanism leads,
inevitably, to errors in their graphical representation.
The use of mobile applications has been studied by many authors and it has been
shown to produce good outcomes in similar contexts. The characteristics of this technology
promote and enrich the teaching and learning process; however, like any other technology
it is necessary to have a precise understanding of the steps that should be followed for its
successful incorporation into classroom practice. There are numerous investigations and
research studies that endorse the didactic use of this kind of technology and analyse the
factors that influence its use and uptake [3640].
Our investigation hopes to achieve the following objectives:
Taking a constructionist approach to learning, we hope to provide students with an
innovative classroom experience through the use of the ARPAID mobile app and,
at the same time, reinforce their knowledge of Graphic Design and improve their
spatial awareness.
Adapt learning and the acquisition of skills to the strategic use of ICTs, particularly
Augmented Reality and Applications for Mobile Devices. Our method is based on
presenting students with real-life problems in a virtual environment, so enabling them
to understand and apply what they have learned as a tool to resolve further problems,
generate ideas, and propose improvements to the engineering community.
Increase academic achievement, both in traditional examinations and in individual
and group tasks.
Motivate and encourage student participation through the combined use of mobile
phones and tablets and AR, a cutting-edge technology that is attractive due to its
associations with gaming. In this way we will enable students to construct their own
knowledge through the use of subject content provided via the app throughout the
whole classroom learning experience.
Enable the increased availability of resources to students at critical moments of the
learning process.
Assess results, through an investigation, evaluating qualitatively and where possible,
quantitatively, the potential improvement in learning of an experimental group (using
the app) and a control group (not using the app).
These objectives are focused on facilitating students’ meaningful learning. The au-
thors [
41
] demonstrate that the teaching of engineering should not be based solely on
Sustainability 2021,13, 3305 6 of 31
theoretical concepts and indeed, that interaction with multimedia applications enables
students to acquire knowledge more quickly. As cognitive theory tell us, it is necessary
for students to think, understand, touch, interact, and reason through concepts for them-
selves [
42
]. As a result, we have designed an application that not only offers theoretical
concepts but also shows students the real objects we work with. The application is built
such that these objects can be touched, rotated, displaced, etc, and ultimately, internalised
and understood. This process simplifies spatial understanding of mechanical assemblies
so that students can interpret them correctly and be able to produce cross-section and
assembly drawings of these systems. A further advantage is portability. Thanks to mobile
phones and tablets, students can access learning content without the need for a computer.
The accomplishment of these objectives will provide us with a tool that will be of great
help to students in aiding them to understand concepts and resolve theoretical and spatial
problems. We will attempt to evaluate these objectives through the assessment tools set out.
2.3. Investigative Method
Hypothesis:
Use of the mobile application ARPAID in the classroom by students when working
on individual projects will help promote meaningful learning of subject material related to technical
drawing of mechanical assemblies.
ARPAID is proposed as a means of acquiring knowledge and enabling interaction with
two of the assemblies students must resolve and draw as part of the practical assessment
task in this subject. During the app design process, it was decided that, besides being able
to manipulate mechanical assemblies it would be convenient for the app to have a facility
for self-evaluation and to provide a set of activities and exercises focused on the deepening
of learning, even some form of gamification: a serious or applied game.
A fundamental characteristic of ARPAID is the combination of synchronous usage in
presential classroom sessions with the teacher, with asynchronous usage during individual
tasks undertaken individually by students. Synchronous sessions will involve extensive
use of the ARPAID’s AR capabilities, as students are provided with material and activities
as part of their class-based lesson. A connection to a remote server holding a database will,
in addition, allow for individualisation of tasks required of users.
Moreover, it was deemed valuable to enable an iterative process of assessing ARPAID:
evaluating its usability and potential for improving the user interface and the app’s function-
alities. This assessment was envisaged as a two-pronged process. On one hand, user attitudes
were evaluated through interviews and satisfaction questionnaires and on the other, through
the discreet and systematic collection of data concerning the variables and parameters of
app usage which were stored and tabulated in a database for subsequent analysis.
Lastly, and thinking ahead to future initiatives, it was deemed necessary to open
up the possibility for the app to handle a greater number of mechanical assemblies and
practical activities such that it could have a higher degree of flexibility. This needs to be
achieved without increasing the file storage requirements on the mobile device or reducing
its performance, thus, we intend to use so called asset bundles comprising material located
on a remote server that can be downloaded to a device as necessary, to be used by the
mobile application.
For all these reasons and given the projected complexity of our investigation, we
considered it convenient to break it down into three sequential stages. At the time of
writing this article, the first stage is complete and the process of preparing for the second
is underway.
Stage I: Implementation of the ARPAID App
During the first stage, work will focus on the implementation of ARPAID with the basic
applications that comprise its core functionality. Here, the aim is to provide students with a
robust application, with essential functions fully developed, that is easy to use, intuitive and
Sustainability 2021,13, 3305 7 of 31
aesthetically attractive. At this stage, the objective is to verify, through a statistical analysis
of data obtained, whether ARPAID use has a positive impact on subject learning.
Stage II: Assessment of usability, user interface and app functioning. Incorporation of
new functionalities, especially in AR features.
In the case that we confirm our hypothesis that the mobile app has a beneficial
impact on students’ skills acquisition, the second stage of project development will be
initiated. During this stage, the level of user satisfaction will be assessed with regards
to learning content and practical tasks provided and we will also gather information
concerning student opinions and feelings about using ARPAID. This process of assessing
user experience is essential and needs to be systematised where possible, complemented
by a stealth assessment [
43
], to obtain indicators for improvement and feedback to help
optimise the app. Additionally, new features will be incorporated to complement essential
functions. The following augmented reality features are worth noting: recognition of real
mechanisms with superimposed virtual labelling (brand numbers, designation of parts,
functional dimensions, indication of tolerances and fits, etc.); customized tests for each
student during the class with questions depending on the analysis of the use of the app, and
collaborative multi-user activities in the same location (co-located experience in a shared
space). Once this is achieved, we will again investigate ARPAID’s impact on learning.
Stage III: Externalisation with Asset Bundles
In this final stage of the project a new version of ARPAID will be created which will
make use of UNITY 3D’s asset packages to allow access to a large archive of assembly
models, problems, quizzes, tasks, and activities. To conclude the project, it is proposed that
the analysis of ARPAID’s effect on learning should be iterated once again.
A summary of the proposed research project, with a stage-by-stage breakdown of
essential activities, can be seen on the diagram shown in Figure 1.
Sustainability 2021, 13, x FOR PEER REVIEW 7 of 32
For all these reasons and given the projected complexity of our investigation, we con-
sidered it convenient to break it down into three sequential stages. At the time of writing
this article, the first stage is complete and the process of preparing for the second is un-
derway.
Stage I: Implementation of the ARPAID App
During the first stage, work will focus on the implementation of ARPAID with the
basic applications that comprise its core functionality. Here, the aim is to provide students
with a robust application, with essential functions fully developed, that is easy to use,
intuitive and aesthetically attractive. At this stage, the objective is to verify, through a sta-
tistical analysis of data obtained, whether ARPAID use has a positive impact on subject
learning.
Stage II: Assessment of usability, user interface and app functioning. Incorporation
of new functionalities, especially in AR features.
In the case that we confirm our hypothesis that the mobile app has a beneficial impact
on students’ skills acquisition, the second stage of project development will be initiated.
During this stage, the level of user satisfaction will be assessed with regards to learning
content and practical tasks provided and we will also gather information concerning stu-
dent opinions and feelings about using ARPAID. This process of assessing user experience
is essential and needs to be systematised where possible, complemented by a stealth as-
sessment [43], to obtain indicators for improvement and feedback to help optimise the app.
Additionally, new features will be incorporated to complement essential functions. The
following augmented reality features are worth noting: recognition of real mechanisms
with superimposed virtual labelling (brand numbers, designation of parts, functional di-
mensions, indication of tolerances and fits, etc.); customized tests for each student during
the class with questions depending on the analysis of the use of the app, and collaborative
multi-user activities in the same location (co-located experience in a shared space). Once
this is achieved, we will again investigate ARPAID’s impact on learning.
Stage III: Externalisation with Asset Bundles
In this final stage of the project a new version of ARPAID will be created which will
make use of UNITY 3D’s asset packages to allow access to a large archive of assembly
models, problems, quizzes, tasks, and activities. To conclude the project, it is proposed
that the analysis of ARPAID’s effect on learning should be iterated once again.
A summary of the proposed research project, with a stage-by-stage breakdown of
essential activities, can be seen on the diagram shown in Figure 1.
Figure 1. Diagram showing the three stages of the investigation.
Figure 2 shows a flow diagram of the first stage of the investigation. As can be seen,
to verify our hypothesis, the student corpus was divided into two groups, one a control
and the other an experimental group. A series of instruments (quizzes and practical tasks),
Figure 1. Diagram showing the three stages of the investigation.
Figure 2shows a flow diagram of the first stage of the investigation. As can be seen,
to verify our hypothesis, the student corpus was divided into two groups, one a control
and the other an experimental group. A series of instruments (quizzes and practical tasks),
were developed to be undertaken by students either before or after the experiment and
which provided the data used in our statistical analysis.
Sustainability 2021,13, 3305 8 of 31
Sustainability 2021, 13, x FOR PEER REVIEW 8 of 32
were developed to be undertaken by students either before or after the experiment and
which provided the data used in our statistical analysis.
Figure 2. Flow diagram of experimental procedures.
2.4. Mobile Application. Development in UNITY3D
The ARPAID app has been developed for use on the two leading mobile operating
systems, IOS and Android. It was made available to be downloaded for free on Google
Play Store. Due to Apple’s terms and conditions of use, it was only possible to produce a
test version of the app for a period of 90 days, using TestFlight. However, our students
using iPhones or iPads were able to use the app without any limitations for the duration
of this investigation.
In the successive stages of this project’s development, the ARPAID app will make
wide ranging use of the possibilities afforded by AR on mobile devices, mainly from the
second stage. A list of design elements used in our mobile application is shown in Appen-
dix A. ARPAID was created using the UNITY3D graphic engine for videogame develop-
ment [44]. The fundamental reasons we decided to use UNITY3D are as follows:
Multiplatform design: executable programs generated in UNITY3D are supported on
a wide range of devices.
Capability to place three-dimensional models in a three-dimensional virtual space
and tailor their properties using C# programming scripts.
Facility for creating complex and dynamic user interfaces (UI).
Outstanding graphics quality capable of creating realistic scenes, with high quality
textures and complex lighting effects, achieved using the surface shader in the render
pipeline.
Possibility of linking and exchanging information with relational databases, on re-
mote servers via the TCP/IP (MySQL-Apache-PHP) protocol.
Figure 2. Flow diagram of experimental procedures.
2.4. Mobile Application. Development in UNITY3D
The ARPAID app has been developed for use on the two leading mobile operating
systems, IOS and Android. It was made available to be downloaded for free on Google
Play Store. Due to Apple’s terms and conditions of use, it was only possible to produce
a test version of the app for a period of 90 days, using TestFlight. However, our students
using iPhones or iPads were able to use the app without any limitations for the duration of
this investigation.
In the successive stages of this project’s development, the ARPAID app will make wide
ranging use of the possibilities afforded by AR on mobile devices, mainly from the second
stage. A list of design elements used in our mobile application is shown in Appendix A.
ARPAID was created using the UNITY3D graphic engine for videogame development [
44
].
The fundamental reasons we decided to use UNITY3D are as follows:
Multiplatform design: executable programs generated in UNITY3D are supported on
a wide range of devices.
Capability to place three-dimensional models in a three-dimensional virtual space
and tailor their properties using C# programming scripts.
Facility for creating complex and dynamic user interfaces (UI).
Outstanding graphics quality capable of creating realistic scenes, with high qual-
ity textures and complex lighting effects, achieved using the surface shader in the
render pipeline.
Possibility of linking and exchanging information with relational databases, on remote
servers via the TCP/IP (MySQL-Apache-PHP) protocol.
Sustainability 2021,13, 3305 9 of 31
The integration of multiplatform Augmented Reality with AR Foundation [
45
]. AR
Foundation functions as an intermediary enabling the use of AR on different plat-
forms and devices independently of the various manufacturers’ proprietary solutions
(HoloLens, Magic Leap, AR Kit or AR Core).
The initial version of ARPAID essentially comprises five UNITY3D scenes.
2.4.1. User Authentication and Registration Scene
This scene (Figure 3) allows the authentication of students so that they can login to
use ARPAID, or their registration if they are accessing ARPAID for the first time. It also
saves all relevant information, including the date and time that the session was initiated,
onto a database hosted on an external server.
Sustainability 2021, 13, x FOR PEER REVIEW 9 of 32
The integration of multiplatform Augmented Reality with AR Foundation [45]. AR
Foundation functions as an intermediary enabling the use of AR on different plat-
forms and devices independently of the various manufacturers’ proprietary solu-
tions (HoloLens, Magic Leap, AR Kit or AR Core).
The initial version of ARPAID essentially comprises five UNITY3D scenes.
2.4.1. User Authentication and Registration Scene
This scene (Figure 3) allows the authentication of students so that they can login to
use ARPAID, or their registration if they are accessing ARPAID for the first time. It also
saves all relevant information, including the date and time that the session was initiated,
onto a database hosted on an external server.
Figure 3. Login and registration page.
2.4.2. Main Menu Scene
Once authenticated, students proceed to a screen informing them of ARPAID’s two
modes of use:
Synchronous Mode: Classroom use, for direct tracking, scanning of QR codes or im-
ages projected onto the whiteboard.
Asynchronous Mode: To enable ARPAID use outside the classroom so that students
can complete individual tasks, and for teachers to provide notes or files containing rele-
vant tracking codes for the subject being studied.
2.4.3. QR Code Scanning Scene (AR Tracking)
Via the main menu, students can access a scene that, through scanning and recogni-
tion of the relevant QR code (Figure 4), will upload a scene where they can manipulate a
particular mechanical system. The decision was made to use QR codes, because in class-
room situations, these monochrome codes were those best recognised by AR packages
available on mobile devices.
Figure 3. Login and registration page.
2.4.2. Main Menu Scene
Once authenticated, students proceed to a screen informing them of ARPAID’s two
modes of use:
Synchronous Mode: Classroom use, for direct tracking, scanning of QR codes or
images projected onto the whiteboard.
Asynchronous Mode: To enable ARPAID use outside the classroom so that students
can complete individual tasks, and for teachers to provide notes or files containing relevant
tracking codes for the subject being studied.
2.4.3. QR Code Scanning Scene (AR Tracking)
Via the main menu, students can access a scene that, through scanning and recognition
of the relevant QR code (Figure 4), will upload a scene where they can manipulate a partic-
ular mechanical system. The decision was made to use QR codes, because in classroom
situations, these monochrome codes were those best recognised by AR packages available
on mobile devices.
Sustainability 2021,13, 3305 10 of 31
Sustainability 2021, 13, x FOR PEER REVIEW 10 of 32
Figure 4. Specification sheet for a hoist ring assembly with the QR code for scanning.
2.4.4. Assembly Manipulation Scene
Once the tracking image has been recognised, the Assembly Manipulation scene is
loaded. In this way, a copy of the mechanical assembly associated with the QR code is
presented in virtual space. This is shown in Figure 5.
Figure 5. Assembly Manipulation scene.
Besides the model of the mechanical assembly, this screen contains four zones, or
panels, in which various different icons appear. These panels constitute the scene’s user
interface (UI), and they are superimposed, with some degree of transparency over the view
of the virtual space provided by the main camera.
Figure 4. Specification sheet for a hoist ring assembly with the QR code for scanning.
2.4.4. Assembly Manipulation Scene
Once the tracking image has been recognised, the Assembly Manipulation scene is
loaded. In this way, a copy of the mechanical assembly associated with the QR code is
presented in virtual space. This is shown in Figure 5.
Sustainability 2021, 13, x FOR PEER REVIEW 10 of 32
Figure 4. Specification sheet for a hoist ring assembly with the QR code for scanning.
2.4.4. Assembly Manipulation Scene
Once the tracking image has been recognised, the Assembly Manipulation scene is
loaded. In this way, a copy of the mechanical assembly associated with the QR code is
presented in virtual space. This is shown in Figure 5.
Figure 5. Assembly Manipulation scene.
Besides the model of the mechanical assembly, this screen contains four zones, or
panels, in which various different icons appear. These panels constitute the scene’s user
interface (UI), and they are superimposed, with some degree of transparency over the view
of the virtual space provided by the main camera.
Figure 5. Assembly Manipulation scene.
Besides the model of the mechanical assembly, this screen contains four zones, or
panels, in which various different icons appear. These panels constitute the scene’s user
Sustainability 2021,13, 3305 11 of 31
interface (UI), and they are superimposed, with some degree of transparency over the view
of the virtual space provided by the main camera.
The ARPAID user interface was tested and analysed methodically to achieve maxi-
mum convenience, speed, and ease of use.
The user can change the observation viewpoint through specific touchscreen gestures.
The ability to visualise the assembly from any direction facilitates spatial understanding
and allows students to ascertain the relative positions of one component in the assembly
with respect to the others. Using a single finger, touching and swiping across the screen
produces a rotation with respect to the X and Y axes (in UNITY3D the Y-axis is vertical).
Rotation about the Z-axis is achieved via the horizontal component of finger motion
but only when this occurs along a small band on the lower edge of the screen. With
regards linear displacement, Z-displacement (depth) is equivalent to zoom and is achieved
by touching two fingers on the screen. If the fingertips are moved apart, the camera
approaches the object (positive zoom), while if they are moved together the camera recedes
from the object (negative zoom). Panorama motion, that is, displacement along the X or Y
axes, is achieved by moving sliders that can be found in the Visualisation Panel located in
the bottom left-hand corner of the screen.
The relative displacement of assembly components, taking into account certain restric-
tions with regards to positioning and degrees of freedom for each one, is fundamental to
internalising and understanding how the assembly works. In this ARPAID scene, the user
can move each one of the component elements, by tapping a finger on the component and
dragging it across the screen. To aid interpretation of the system, the movements of each
element are restricted to assembly displacements.
If a component requires any other sort of motion necessary for assembly or disassem-
bly of the system, a slider bar is added on the upper edge of the Visualisation Panel.
ARPAID uses two ways to limit the movement of components to within the geometric
restrictions and positions of the real-world system: coding restrictions and collisions.
Preference is given to coding restrictions such that limitations on the relative movements of
components is defined by exact calculation of the system’s kinematics and are programmed
using conditional strings. However, assembly displacements of components can leave
them in random positions so complicating calculations. In these situations, the second
option comes into play, that of defining the range of movement through the detection of
mesh collisions between the gameobjects comprising the system.
The Visualisation Panel complements the options to change the observation viewpoint
and relative displacement of assembly components which we have just described. This last
panel is located at the bottom left-hand corner of the screen.
Special attention has been given to the design specifications of the Visualisation Panel
such that it should be compact, easy to use, with self-explanatory icons. The central icon
alters the viewpoint to give an isometric view. Around this central icon are six others
used to access the six orthographic representations of the assembly being studied. In the
bottom right corner of the panel is an icon that displaces the assembly’s components to
show the model in exploded view. The icon in the bottom left-hand corner of the panel
blocks all camera motion, so enabling a higher precision of component motion. The Reset
icon, in the upper right-hand corner of the panel, reinitialises the viewpoint and positions
of assembly components.
Lastly, the icon in the upper left of the panel reloads the QR scanning scene. Figure 6
shows a detail of the panel.
The Detail Part Panel is located in the upper right-hand corner of the screen. On
this panel is a list of all the components in the assembly on which several actions can be
performed. Being able to hide, reveal, or visualise with a degree of transparency individual
components of an assembly is a way of deepening knowledge through the manipulation
of—in this case, virtual objects and entities [
33
]. The possibility of showing components
with some transparency, as seen in Figure 7, provides a novel view of a fully assembled
working system containing internal parts that would not be possible in real life.
Sustainability 2021,13, 3305 12 of 31
Sustainability 2021, 13, x FOR PEER REVIEW 12 of 32
Figure 6. Visualisation panel.
The Detail Part Panel is located in the upper right-hand corner of the screen. On this
panel is a list of all the components in the assembly on which several actions can be per-
formed. Being able to hide, reveal, or visualise with a degree of transparency individual
components of an assembly is a way of deepening knowledge through the manipulation
of—in this case, virtual objects and entities [33]. The possibility of showing components
with some transparency, as seen in Figure 7, provides a novel view of a fully assembled
working system containing internal parts that would not be possible in real life.
Figure 7. View of a component with transparency.
Each component listed in the Detail Part Panel (Figure 8) has an associated button
that, if pressed, loads the Detail Part Definition and Manipulation scene.
Figure 8. Detail of buttons and toggles on the Detail Part Panel.
Finally, in the top left-hand corner of the screen, an Information Panel has been
added. Here, students can access all the relevant, necessary information to facilitate un-
derstanding of whichever assembly they are working on. In addition, they can find course
Figure 6. Visualisation panel.
Figure 7. View of a component with transparency.
Each component listed in the Detail Part Panel (Figure 8) has an associated button
that, if pressed, loads the Detail Part Definition and Manipulation scene.
Sustainability 2021, 13, x FOR PEER REVIEW 12 of 32
Figure 6. Visualisation panel.
The Detail Part Panel is located in the upper right-hand corner of the screen. On this
panel is a list of all the components in the assembly on which several actions can be per-
formed. Being able to hide, reveal, or visualise with a degree of transparency individual
components of an assembly is a way of deepening knowledge through the manipulation
of—in this case, virtual objects and entities [33]. The possibility of showing components
with some transparency, as seen in Figure 7, provides a novel view of a fully assembled
working system containing internal parts that would not be possible in real life.
Figure 7. View of a component with transparency.
Each component listed in the Detail Part Panel (Figure 8) has an associated button
that, if pressed, loads the Detail Part Definition and Manipulation scene.
Figure 8. Detail of buttons and toggles on the Detail Part Panel.
Finally, in the top left-hand corner of the screen, an Information Panel has been
added. Here, students can access all the relevant, necessary information to facilitate un-
derstanding of whichever assembly they are working on. In addition, they can find course
Figure 8. Detail of buttons and toggles on the Detail Part Panel.
Finally, in the top left-hand corner of the screen, an Information Panel has been added.
Here, students can access all the relevant, necessary information to facilitate understanding
of whichever assembly they are working on. In addition, they can find course activities
that have been set and assessment and self-assessment quizzes so that their depth of
learning can be evaluated. The Information Panel, shown in Figure 9, comprises three
sections, each of which contains different categories of information. The first icon accesses
information to help users navigate ARPAID. The second icon links to documentation
concerning the assembly being studied: the types of components it contains; the number
of each component it contains; what every component is made of; fits and tolerances;
roughness and surface condition.
Sustainability 2021,13, 3305 13 of 31
Sustainability 2021, 13, x FOR PEER REVIEW 13 of 32
activities that have been set and assessment and self-assessment quizzes so that their
depth of learning can be evaluated. The Information Panel, shown in Figure 9, comprises
three sections, each of which contains different categories of information. The first icon
accesses information to help users navigate ARPAID. The second icon links to documen-
tation concerning the assembly being studied: the types of components it contains; the
number of each component it contains; what every component is made of; fits and toler-
ances; roughness and surface condition.
Figure 9. Information panel.
The third section of the Information Panel contains a series of activities and self-eval-
uation quizzes designed to determine the level of learning they have attained. The inclu-
sion of a full framework for self-assessment and of stealth assessment [46] is expected to be
part of the second stage of project development.
2.4.5. Detail Part Definition and Manipulation Scene
In this scene the focus is enabling user interaction with individual components of the
assembly, outside the context of the mechanical system. Manipulating the assembly and
how certain components move in relation to others is fundamental, however, it is no less
important to have a detailed understanding of the shapes and dimensions of individual
component elements in order to understand how each one interacts with other assembly
components. This process enables the student to see which of a component’s functional
dimensions are intrinsic to its correct performance within the system.
The design of the user experience (UX) [47] and user interface (UI) [48] is similar to the
Assembly Manipulation Scene in terms of its organisation and the rules it follows. How-
ever, the icons are a different colour so that students can easily identify which scene they
are in.
A further panel has been added, in the bottom right-hand corner of the screen, to
provide students with all the details of the dimensions and shapes of every part in the
assembly. This is the Definition Panel (Figure 10) and enables viewing of an isometric
projection of the component, including dimensions, that can be used later to elaborate the
necessary drawings for a standardised representation of the component. The image size
can be altered using a slider bar.
Figure 9. Information panel.
The third section of the Information Panel contains a series of activities and self-
evaluation quizzes designed to determine the level of learning they have attained. The
inclusion of a full framework for self-assessment and of stealth assessment [
46
] is expected
to be part of the second stage of project development.
2.4.5. Detail Part Definition and Manipulation Scene
In this scene the focus is enabling user interaction with individual components of the
assembly, outside the context of the mechanical system. Manipulating the assembly and
how certain components move in relation to others is fundamental, however, it is no less
important to have a detailed understanding of the shapes and dimensions of individual
component elements in order to understand how each one interacts with other assembly
components. This process enables the student to see which of a component’s functional
dimensions are intrinsic to its correct performance within the system.
The design of the user experience (UX) [
47
] and user interface (UI) [
48
] is similar to
the Assembly Manipulation Scene in terms of its organisation and the rules it follows.
However, the icons are a different colour so that students can easily identify which scene
they are in.
A further panel has been added, in the bottom right-hand corner of the screen, to
provide students with all the details of the dimensions and shapes of every part in the
assembly. This is the Definition Panel (Figure 10) and enables viewing of an isometric
projection of the component, including dimensions, that can be used later to elaborate the
necessary drawings for a standardised representation of the component. The image size
can be altered using a slider bar.
Sustainability 2021, 13, x FOR PEER REVIEW 14 of 32
Figure 10. Screen for defining and manipulating detail parts.
Figure 11 shows a diagram of the structure and flow of the app, based on the scenes
created in the UNITY3D Editor.
Figure 11. Diagram of the app in the UNITY3D development environment.
The composition of the scenes in the UNITY3D editor requires the use of assets gen-
erated in external applications. These assets include 3D meshes, textures, and UI icons.
Appendix B describes the process followed for their creation.
To summarise the implementation of the mobile app ARPAID, Figure 12 shows a
flow diagram of the process with particular attention to software requirements and file
format compatibility considerations.
Figure 10. Screen for defining and manipulating detail parts.
Sustainability 2021,13, 3305 14 of 31
Figure 11 shows a diagram of the structure and flow of the app, based on the scenes
created in the UNITY3D Editor.
Sustainability 2021, 13, x FOR PEER REVIEW 14 of 32
Figure 10. Screen for defining and manipulating detail parts.
Figure 11 shows a diagram of the structure and flow of the app, based on the scenes
created in the UNITY3D Editor.
Figure 11. Diagram of the app in the UNITY3D development environment.
The composition of the scenes in the UNITY3D editor requires the use of assets gen-
erated in external applications. These assets include 3D meshes, textures, and UI icons.
Appendix B describes the process followed for their creation.
To summarise the implementation of the mobile app ARPAID, Figure 12 shows a
flow diagram of the process with particular attention to software requirements and file
format compatibility considerations.
Figure 11. Diagram of the app in the UNITY3D development environment.
The composition of the scenes in the UNITY3D editor requires the use of assets
generated in external applications. These assets include 3D meshes, textures, and UI icons.
Appendix Bdescribes the process followed for their creation.
To summarise the implementation of the mobile app ARPAID, Figure 12 shows a flow
diagram of the process with particular attention to software requirements and file format
compatibility considerations.
Sustainability 2021, 13, x FOR PEER REVIEW 15 of 32
Figure 12. Flow diagram for implementing ARPAID, software requirements and file compatibility.
2.5. Mechanical Assemblies Modelled for Use in ARPAID
For the first stage of the investigation two complete assemblies were modelled. These
assemblies form part of the practical work undertaken for this course:
Magnetic Bit-holder assembly (Figures 7 and 13a)
Double Articulated Hoist Ring assembly (Figures 4,5 and 13b)
(a) (b)
Figure 13. Examples of Post-Test exercises: (a) Magnetic bit-holder; (b) Hoist ring assembly.
At present, work is going ahead to model a further set of mechanical assemblies to
build up a library of learning resources for our students.
Figure 12. Flow diagram for implementing ARPAID, software requirements and file compatibility.
2.5. Mechanical Assemblies Modelled for Use in ARPAID
For the first stage of the investigation two complete assemblies were modelled. These
assemblies form part of the practical work undertaken for this course:
Magnetic Bit-holder assembly (Figures 7and 13a)
Double Articulated Hoist Ring assembly (Figures 4,5and 13b)
At present, work is going ahead to model a further set of mechanical assemblies to
build up a library of learning resources for our students.
Sustainability 2021,13, 3305 15 of 31
Sustainability 2021, 13, x FOR PEER REVIEW 15 of 32
Figure 12. Flow diagram for implementing ARPAID, software requirements and file compatibility.
2.5. Mechanical Assemblies Modelled for Use in ARPAID
For the first stage of the investigation two complete assemblies were modelled. These
assemblies form part of the practical work undertaken for this course:
Magnetic Bit-holder assembly (Figures 7 and 13a)
Double Articulated Hoist Ring assembly (Figures 4,5 and 13b)
(a) (b)
Figure 13. Examples of Post-Test exercises: (a) Magnetic bit-holder; (b) Hoist ring assembly.
At present, work is going ahead to model a further set of mechanical assemblies to
build up a library of learning resources for our students.
Figure 13. Examples of Post-Test exercises: (a) Magnetic bit-holder; (b) Hoist ring assembly.
2.6. Study Sample and Participants
The investigation took place throughout the academic year 2019–2020 in the Graphic
Design II module common to the first-year degree courses in Electronic Engineering,
Electrical Engineering, and Aerospace Engineering at the University of León, Spain (ULE).
Our sample comprised 153 students in total.
Due to the way that teaching is organised in the School of Engineering at ULE, the
investigation has had to use already existing groups of students, that is, natural groups.
Since we were not able to randomly assign individuals to different groups, this could mean
that, initially, there are noticeable differences between the groups; in other words, they
will not be equivalent. As a consequence the investigation is quasi-experimental [
49
], and its
design involves Pre-Test and Post-Test assessment of two student groups: an experimental
group which will use ARPAID, and a control group which will not.
With the aim of evening out potential differences between the control and experimental
groups it was decided that the experimental group should contain students from two-
degree courses. Thus, the experimental group comprised students in the Aerospace and
Electrical Engineering degree programs while the control group comprised students in the
Electronic Engineering degree program.
Both groups had the same teacher for this subject (the first author of this article),
used the same materials, notes, and book of practical exercises; furthermore, the tests and
examination exercises undertaken by all students were of the same level of difficulty. The
only difference between the two groups was the experimental group’s use of ARPAID
for two weeks. To evaluate the effects of the app on student learning, both groups took
a quiz at the beginning of the investigation (Pre-Test) and one at the end (Post-Test). The
groups also completed four assessed exercises, one of which was a formal examination and
counted towards their final course mark.
Sustainability 2021,13, 3305 16 of 31
The sample is composed of 153 students: 67 studying the Aerospace Engineering
degree (43.79%); 16 studying the Electrical Engineering degree (10.46%); and 70 studying
the Electronic Engineering degree (45.75%). Thus, the experimental group comprises 83
students (54.25%) while the control comprises 70 students (45.75%). Of the 153 students, 30
are female (19.61%) and the remaining 123 are male (80.39%). The age range of the sample
is from 19 to 45 years of age.
2.7. Resources Used
The application we developed can be used on mobile devices such as the latest
generation of smartphones and tablets. The minimum requirements for the AR used are
IOS version 13, and Android 7.0.
The following instruments were used to determine whether ARPAID had a positive
effect on learning Technical Drawing of Mechanical Assemblies:
Pre-Test Knowledge Quiz
Post-Test Knowledge Quiz
Pre-Test Practical Exercise: Separator Assembly
Post-Test Practical Exercise 1: Hoist ring Assembly
Post-Test Practical Exercise 2: Bit-holder Assembly
Post-Test Examination Exercise
2.7.1. Pre-Test Knowledge Quiz
In the absence of the possibility of assigning students randomly to either the experi-
mental or control group, the Pre-Test Quiz was used to determine the level of equivalence
between groups prior to the investigation.
The questions on the quiz were of a general nature, relevant to the subject of me-
chanical assemblies, and was completed in the initial, explanatory classroom session. The
quiz was aimed at measuring students’ level of subject knowledge in the initial stages of
instruction. Through the test, we hoped to determine whether or not there were significant
differences between the levels of knowledge in each group and in the case that large differ-
ences were discovered between control and experimental groups, these differences could
be quantified.
The Pre-Test Quiz contained 20 multiple-choice questions in which the student had to
select the correct answer, or answers, from four to six possibilities provided. It was put
together by the teaching team carrying out the investigation (the quiz can be found in
Supplementary Materials).
2.7.2. Post-Test Knowledge Quiz
The teaching taking place within this investigation was not the only source of knowl-
edge and learning in this subject; rather, it acted to complement other approaches. As a
result, it would be expected that, as the course progresses, we might see a certain amount
of levelling out in student achievement between the groups. If the ARPAID experiment
has a significant influence on learning, however, we should see the maintenance of initial
differences—albeit reduced.
As a result, after working with ARPAID, students were given a Post-Test Quiz to
evaluate their progress. It was decided that the Post-Test should follow the same for-
mat as the Pre-Test, except that the order of the questions and possible answers was
rearranged randomly.
2.7.3. Pre-Test Practical Exercise
The Pre-Test Quiz completed by students attempts to highlight, in a general way, any
differences in knowledge level between the two groups in the investigation. The Pre-Test
Practical Exercise is a more specific measure of these differences since it comprises a task in
which the student must understand the blueprints for a particular mechanical assembly,
in this case, the Separator assembly shown in Figure 14a. This practical task takes place
Sustainability 2021,13, 3305 17 of 31
after the explanation of concepts and tackling of problems in the classroom. It is scheduled
after explanatory sessions, directly before the initiation of the ARPAID experiment. It
is hypothesised that the biggest equivalences between experimental and control groups
should be seen at this point in the project and in the case where ARPAID has a significant
effect on learning the levels of the two groups should then diverge.
Sustainability 2021, 13, x FOR PEER REVIEW 18 of 32
(a) (b)
Figure 14. Examples of assessed exercises: (a) Separator assembly; (b) Connection block assembly.
The Pre-Test and the first Post-Test assemblies must be drawn in pencil, with tradi-
tional tools, while the second Post-Test assembly is completed using AutoCAD software.
Bearing in mind that before attempting the Post-Test drawing exercises the experi-
mental group will have had access to ARPAID and, as a result, will have been able to work
with the 3D models; if it is true that ARPAID genuinely has a positive impact and pro-
motes meaningful learning, the grades achieved by the experimental group should im-
prove relative to those achieved by the control group.
It must be said that the mechanical assemblies that students are asked to draw are
the same as the models used by the experimental group when working in ARPAID.
2.7.5. Post-Test Examination Exercise
In the assessment regime of the Graphic Design course, the section devoted to me-
chanical assemblies accounts for 35% of the final grade and final assessment involves a
formal examination where students must draw blueprints for the assembly shown in Fig-
ure 14b.
Furthermore, the examination exercise is totally new, i.e., not related to any of the
assemblies students will have worked on in ARPAID. Thus, it is a fairly objective measure
of whether ARPAID’s impact on meaningful learning can be generalised and not limited
solely to assemblies used in the experiment (Bit-holder and Hoist ring) but to other mech-
anisms. Positive results here would mean that ARPAID reinforces meaningful learning
and the acquisition of skills related to the theory of graphical representation of mechanical
assemblies.
2.8. Timeline for the Investigation
The authors of the present investigation have planned and devised a timeline for the
completion of Stage I which includes the following steps (Figure 15):
Step 1: Literature review to delimit the terms of the investigation and discover the
state of the art in terms of theory and technology.
Figure 14. Examples of assessed exercises: (a) Separator assembly; (b) Connection block assembly.
2.7.4. Post-Test Practical Exercise
Besides the assembly used for the Pre-Test Practical Exercises, students must complete
drawings of two further mechanical assemblies. These three obligatory tasks are designed
to be of increasing levels of difficulty, each one having additional elements, containing
new standard mechanical components and concepts related to tolerances and fits, and
surface roughness. The Separator assembly used for the Pre-Test (Figure 14a), and the two
Post-Tests exercises, the Bit-holder (Figure 13a) and the Hoist ring assembly (Figure 13b),
represent this pedagogic methodology of gradually increasing complexity.
The Pre-Test and the first Post-Test assemblies must be drawn in pencil, with tradi-
tional tools, while the second Post-Test assembly is completed using AutoCAD software.
Bearing in mind that before attempting the Post-Test drawing exercises the exper-
imental group will have had access to ARPAID and, as a result, will have been able to
work with the 3D models; if it is true that ARPAID genuinely has a positive impact and
promotes meaningful learning, the grades achieved by the experimental group should
improve relative to those achieved by the control group.
It must be said that the mechanical assemblies that students are asked to draw are the
same as the models used by the experimental group when working in ARPAID.
Sustainability 2021,13, 3305 18 of 31
2.7.5. Post-Test Examination Exercise
In the assessment regime of the Graphic Design course, the section devoted to mechan-
ical assemblies accounts for 35% of the final grade and final assessment involves a formal
examination where students must draw blueprints for the assembly shown in Figure 14b.
Furthermore, the examination exercise is totally new, i.e., not related to any of the
assemblies students will have worked on in ARPAID. Thus, it is a fairly objective mea-
sure of whether ARPAID’s impact on meaningful learning can be generalised and not
limited solely to assemblies used in the experiment (Bit-holder and Hoist ring) but to
other mechanisms. Positive results here would mean that ARPAID reinforces meaningful
learning and the acquisition of skills related to the theory of graphical representation of
mechanical assemblies.
2.8. Timeline for the Investigation
The authors of the present investigation have planned and devised a timeline for the
completion of Stage I which includes the following steps (Figure 15):
Step 1: Literature review to delimit the terms of the investigation and discover the
state of the art in terms of theory and technology.
Step 2: Designing the experiment, defining the hypothesis, objectives and methodolo-
gies that will be used.
Step 3: Initial ARPAID design, establishing its structure, methodology and components.
Step 4: Preparation of research assessment instruments.
Step 5: Development and implementation of an ARPAID prototype.
Step 6: Analysis of the ARPAID prototype, unit tests, iteration.
Step 7: Working with the teacher in the technical drawing class.
Step 8: ARPAID implementation on the two different operating systems to be used
(IOS and Android).
Step 9: Assignment of students to either the experimental or control group.
Step 10: Completion of the Pre-Test Quiz by all students.
Step 11: Completion of the Pre-Test Practical Exercise by all students.
Step 12: Students in the experimental group download ARPAID to their mobile
devices. Students are given two weeks to use the application and work with the
material it contains.
Step 13: Completion of the Post-Test Quiz by all students.
Step 14: Completion of Post-Test practical drawing activities 1 and 2 by all students.
Step 15: Completion of Post-Test Examination Exercise.
Step 16: Analysis and statistical processing of the data.
Step 17: Examination of the investigation’s results and drawing conclusions.
Step 18: Proposals made for improvements and planning for Stage II of the investigation.
Sustainability 2021, 13, x FOR PEER REVIEW 19 of 32
Step 2: Designing the experiment, defining the hypothesis, objectives and methodol-
ogies that will be used.
Step 3: Initial ARPAID design, establishing its structure, methodology and compo-
nents.
Step 4: Preparation of research assessment instruments.
Step 5: Development and implementation of an ARPAID prototype.
Step 6: Analysis of the ARPAID prototype, unit tests, iteration.
Step 7: Working with the teacher in the technical drawing class.
Step 8: ARPAID implementation on the two different operating systems to be used
(IOS and Android).
Step 9: Assignment of students to either the experimental or control group.
Step 10: Completion of the Pre-Test Quiz by all students.
Step 11: Completion of the Pre-Test Practical Exercise by all students.
Step 12: Students in the experimental group download ARPAID to their mobile de-
vices. Students are given two weeks to use the application and work with the material
it contains.
Step 13: Completion of the Post-Test Quiz by all students.
Step 14: Completion of Post-Test practical drawing activities 1 and 2 by all students.
Step 15: Completion of Post-Test Examination Exercise.
Step 16: Analysis and statistical processing of the data.
Step 17: Examination of the investigation’s results and drawing conclusions.
Step 18: Proposals made for improvements and planning for Stage II of the investi-
gation.
Figure 15. Timeline for the completion of Stage I.
3. Results and Discussion
3.1. Evaluation of Learning Outcomes
After the tabulation, processing and analysis of the data obtained during the investi-
gation a detailed exploration of the results will be done.
3.1.1. Pre-Test Knowledge Quiz
Descriptive statistical measures applied to the results of the Pre-Test Quiz confirmed
the initial hypothesis that there would be a lack of equivalence between the experimental
and control groups. This is largely due to the fact that the mean grade of students at intake
to the Aerospace Engineering degree is significantly higher than that of student groups
starting other degree courses. Even when normalising this group to the group with the
lower mean intake grade, the lack of equivalence is maintained.
The mean grade obtained in the Pre-Test Quiz was 8.73 (out of 20) for the control
group while the experimental group scored 3.3 points higher gaining mean grade of 12.03.
Figure 15. Timeline for the completion of Stage I.
Sustainability 2021,13, 3305 19 of 31
3. Results and Discussion
3.1. Evaluation of Learning Outcomes
After the tabulation, processing and analysis of the data obtained during the investi-
gation a detailed exploration of the results will be done.
3.1.1. Pre-Test Knowledge Quiz
Descriptive statistical measures applied to the results of the Pre-Test Quiz confirmed
the initial hypothesis that there would be a lack of equivalence between the experimental
and control groups. This is largely due to the fact that the mean grade of students at intake
to the Aerospace Engineering degree is significantly higher than that of student groups
starting other degree courses. Even when normalising this group to the group with the
lower mean intake grade, the lack of equivalence is maintained.
The mean grade obtained in the Pre-Test Quiz was 8.73 (out of 20) for the control
group while the experimental group scored 3.3 points higher gaining mean grade of 12.03.
Analysis shows that the standard deviation of test scores for the control group was
smaller than for the experimental group—2.95 and 3.75, respectively. The greater variation
seen in the experimental group can be explained by its heterogenous nature, containing
students with extremely good results and others with much lower academic achievement
(see Figure 16).
Sustainability 2021, 13, x FOR PEER REVIEW 20 of 32
Analysis shows that the standard deviation of test scores for the control group was
smaller than for the experimental group—2.95 and 3.75, respectively. The greater varia-
tion seen in the experimental group can be explained by its heterogenous nature, contain-
ing students with extremely good results and others with much lower academic achieve-
ment (see Figure 16).
Figure 16. Pre-Test Quiz—Descriptive statistics.
3.1.2. Pre-Test Practical Exercise
The next hypothesis to be tested was that the presential classroom sessions, both the-
ory lessons and those involving practical tasks, would have a levelling out effect on the
two student groups. Indeed, despite initial differences, in these circumstances learning
can be optimal and enable students to reach the same high standards, independent of their
initial level of learning.
Analysis of data obtained from students’ performances in the Pre-Test Practical Ex-
ercise, undertaken by the whole student sample, verified the levelling effect of classroom
learning (see Figure 17).
Figure 17. Pre-Test Practical Exercise—Descriptive statistics.
In this exercise, students in the control group obtained a mean score of 5.09 (out of
10) which was similar to that obtained by the experimental group, 5.76. Furthermore,
learning appeared to be uniform between and within groups as shown by the fact that the
standard deviation of scores was almost the same for both (1.42 for the control and 1.47
for the experimental group).
Figure 16. Pre-Test Quiz—Descriptive statistics.
3.1.2. Pre-Test Practical Exercise
The next hypothesis to be tested was that the presential classroom sessions, both
theory lessons and those involving practical tasks, would have a levelling out effect on the
two student groups. Indeed, despite initial differences, in these circumstances learning can
be optimal and enable students to reach the same high standards, independent of their
initial level of learning.
Analysis of data obtained from students’ performances in the Pre-Test Practical Exer-
cise, undertaken by the whole student sample, verified the levelling effect of classroom
learning (see Figure 17).
In this exercise, students in the control group obtained a mean score of 5.09 (out of 10)
which was similar to that obtained by the experimental group, 5.76. Furthermore, learning
appeared to be uniform between and within groups as shown by the fact that the standard
deviation of scores was almost the same for both (1.42 for the control and 1.47 for the
experimental group).
Sustainability 2021,13, 3305 20 of 31
Sustainability 2021, 13, x FOR PEER REVIEW 20 of 32
Analysis shows that the standard deviation of test scores for the control group was
smaller than for the experimental group—2.95 and 3.75, respectively. The greater varia-
tion seen in the experimental group can be explained by its heterogenous nature, contain-
ing students with extremely good results and others with much lower academic achieve-
ment (see Figure 16).
Figure 16. Pre-Test Quiz—Descriptive statistics.
3.1.2. Pre-Test Practical Exercise
The next hypothesis to be tested was that the presential classroom sessions, both the-
ory lessons and those involving practical tasks, would have a levelling out effect on the
two student groups. Indeed, despite initial differences, in these circumstances learning
can be optimal and enable students to reach the same high standards, independent of their
initial level of learning.
Analysis of data obtained from students’ performances in the Pre-Test Practical Ex-
ercise, undertaken by the whole student sample, verified the levelling effect of classroom
learning (see Figure 17).
Figure 17. Pre-Test Practical Exercise—Descriptive statistics.
In this exercise, students in the control group obtained a mean score of 5.09 (out of
10) which was similar to that obtained by the experimental group, 5.76. Furthermore,
learning appeared to be uniform between and within groups as shown by the fact that the
standard deviation of scores was almost the same for both (1.42 for the control and 1.47
for the experimental group).
Figure 17. Pre-Test Practical Exercise—Descriptive statistics.
3.1.3. Post-Test Quiz
It is necessary to reiterate that the Pre-Test Quiz was used to determine whether there
were significant differences between the experimental and control groups prior to the
experiment taking place. This and the Post-Test Quiz were not designed specifically to
evaluate the impact of ARPAID on the students’ acquisition of technical drawing skills
since the questions in the Quiz are generic rather than focussed on evaluating the skills
that ARPAID was designed to develop and strengthen.
The Post-Test Quiz reflects the situation immediately after the experiment (see
Figure 18).
Sustainability 2021, 13, x FOR PEER REVIEW 21 of 32
3.1.3. Post-Test Quiz
It is necessary to reiterate that the Pre-Test Quiz was used to determine whether there
were significant differences between the experimental and control groups prior to the ex-
periment taking place. This and the Post-Test Quiz were not designed specifically to eval-
uate the impact of ARPAID on the students’ acquisition of technical drawing skills since
the questions in the Quiz are generic rather than focussed on evaluating the skills that
ARPAID was designed to develop and strengthen.
The Post-Test Quiz reflects the situation immediately after the experiment (see Figure 18).
Figure 18. Post-Test Quiz—Descriptive statistics.
Although this instrument cannot be used to evaluate the impact of ARPAID, it can
suggest trends. If the experimental group shows a greater improvement compared to the
control, this will be, fundamentally, due to the effect of ARPAID, since this is the only
factor that varies between the groups.
Comparing the results of the quizzes Pre- and Post-Test, it is clear that there is a sig-
nificant increase in achievement in both groups. This indicates that the personal learning
work completed by students added to the classroom-based sessions complemented by
tutorials has been instrumental in enabling students to construct knowledge and deepen
their understanding of the subject material. The improvement in mean score was 3.18
points (out of 20) for the control group while that seen in the experimental group was 1.48.
This is not a surprising result since, as we mentioned, classroom teaching and self-study
are factors that level out learning outcomes. However, differences between the variation
of scores within groups could be indicative of ARPAID’s influence, acting as an additional
factor promoting uniformity of learning. The standard deviation of scores in the experi-
mental group was more or less constant pre and Post-Test: 3.75 and 3.74, respectively,
indicating that learning was fairly uniform within this group. However, within the control
group, the standard deviation of scores increased from 2.96 Pre-Test to 4.01 Post-Test. This
implies that some students had improved their results considerably while others achieved
a much smaller increase in their scores.
With respect to the Pre-Test Practical Exercise, after statistical analysis and adjusting
values to the same scale (20), there was an improvement in achievement for both groups:
1.73 points in the control group and 1.99 points in the experimental group. Although it is
possible that this improvement could be put down to hard work on the part of students
in both groups through self-study and practice, the greater achievement of the experi-
mental group suggests that using ARPAID had a positive influence in this improvement.
Figure 19 shows the evolution over time of mean scores measured at key points in
the investigation: the initial state, prior to any intervention (Pre-Test Q); the state post
classroom teaching intervention (Pre-Test PrEx); and finally, the state post ARPAID inter-
vention experiment (Post-Test Q). As can be seen, before any teaching took place, the dif-
Figure 18. Post-Test Quiz—Descriptive statistics.
Although this instrument cannot be used to evaluate the impact of ARPAID, it can
suggest trends. If the experimental group shows a greater improvement compared to the
control, this will be, fundamentally, due to the effect of ARPAID, since this is the only factor
that varies between the groups.
Comparing the results of the quizzes Pre- and Post-Test, it is clear that there is a
significant increase in achievement in both groups. This indicates that the personal learning
work completed by students added to the classroom-based sessions complemented by
tutorials has been instrumental in enabling students to construct knowledge and deepen
their understanding of the subject material. The improvement in mean score was 3.18 points
Sustainability 2021,13, 3305 21 of 31
(out of 20) for the control group while that seen in the experimental group was 1.48. This is
not a surprising result since, as we mentioned, classroom teaching and self-study are factors
that level out learning outcomes. However, differences between the variation of scores
within groups could be indicative of ARPAID’s influence, acting as an additional factor
promoting uniformity of learning. The standard deviation of scores in the experimental
group was more or less constant pre and Post-Test: 3.75 and 3.74, respectively, indicating
that learning was fairly uniform within this group. However, within the control group, the
standard deviation of scores increased from 2.96 Pre-Test to 4.01 Post-Test. This implies
that some students had improved their results considerably while others achieved a much
smaller increase in their scores.
With respect to the Pre-Test Practical Exercise, after statistical analysis and adjusting
values to the same scale (20), there was an improvement in achievement for both groups:
1.73 points in the control group and 1.99 points in the experimental group. Although it is
possible that this improvement could be put down to hard work on the part of students in
both groups through self-study and practice, the greater achievement of the experimental
group suggests that using ARPAID had a positive influence in this improvement.
Figure 19 shows the evolution over time of mean scores measured at key points
in the investigation: the initial state, prior to any intervention (Pre-Test Q); the state
post classroom teaching intervention (Pre-Test PrEx); and finally, the state post ARPAID
intervention experiment (Post-Test Q). As can be seen, before any teaching took place, the
difference in level between the two groups was 3.3 points. After the classroom teaching
sessions, when students had to complete the Pre-Test Practical Exercise, this difference had
reduced to 1.34 points (59.39%). This was the state of affairs before the ARPAID experiment
took place, after which the students took the Post-Test Quiz. Here, the difference between
groups increased again to 1.67 points, suggesting that ARPAID did indeed contribute
to learning.
Sustainability 2021, 13, x FOR PEER REVIEW 22 of 32
ference in level between the two groups was 3.3 points. After the classroom teaching ses-
sions, when students had to complete the Pre-Test Practical Exercise, this difference had
reduced to 1.34 points (59.39%). This was the state of affairs before the ARPAID experi-
ment took place, after which the students took the Post-Test Quiz. Here, the difference
between groups increased again to 1.67 points, suggesting that ARPAID did indeed con-
tribute to learning.
Figure 19. Differences in achievement of the student groups at various key stages of the investiga-
tion.
3.1.4. Post-Test Practical Exercises and the Examination Exercise
These three exercises constitute the basic instruments used to verify whether use of
ARPAID has a positive impact on the experimental group and if it has contributed to
meaningful, deep learning about the standardised drawing of mechanical assemblies. The
exercises constitute a direct application of the skills acquired during the learning process.
Student grades in these exercises are therefore an objective measure of their degree of
learning and the level of skill attained by the student.
In order to analyse the data, three Supervised Learning Classifiers have been applied
[50] to the scores obtained by students in these three exercises: Linear Discriminant Anal-
ysis (LDA); Quadratic Discriminant Analysis (QDA) and a Support Vector Classifier
(SVC). The aim was that the classifiers should, based on a student’s score, be able to cor-
rectly place that student into a class: Class 1 (student used ARPAID) or Class 0 (student
did not use ARPAID). After initial training of the classifier with the available data, class
assignation was completed using the scores obtained by students in the three assessed
exercises.
Each of the analyses has been represented graphically, in a Confusion Matrix
Heatmap, from which we extracted indicators (or metrics) that enabled an evaluation of
classification quality: the precision and the sensitivity (also known as the true positive rate
or recall). Figure 20 shows the Confusion Matrix for QDA (a) and the SVC (b). Results
obtained using LDA classifier are not shown as these were inferior to the classifications
obtained using the other methods.
Figure 19.
Differences in achievement of the student groups at various key stages of the investigation.
3.1.4. Post-Test Practical Exercises and the Examination Exercise
These three exercises constitute the basic instruments used to verify whether use of
ARPAID has a positive impact on the experimental group and if it has contributed to
meaningful, deep learning about the standardised drawing of mechanical assemblies. The
exercises constitute a direct application of the skills acquired during the learning process.
Sustainability 2021,13, 3305 22 of 31
Student grades in these exercises are therefore an objective measure of their degree of
learning and the level of skill attained by the student.
In order to analyse the data, three Supervised Learning Classifiers have been ap-
plied [
50
] to the scores obtained by students in these three exercises: Linear Discriminant
Analysis (LDA); Quadratic Discriminant Analysis (QDA) and a Support Vector Classi-
fier (SVC). The aim was that the classifiers should, based on a student’s score, be able
to correctly place that student into a class: Class 1 (student used ARPAID) or Class 0
(student did not use ARPAID). After initial training of the classifier with the available
data, class assignation was completed using the scores obtained by students in the three
assessed exercises.
Each of the analyses has been represented graphically, in a Confusion Matrix Heatmap,
from which we extracted indicators (or metrics) that enabled an evaluation of classification
quality: the precision and the sensitivity (also known as the true positive rate or recall).
Figure 20 shows the Confusion Matrix for QDA (a) and the SVC (b). Results obtained using
LDA classifier are not shown as these were inferior to the classifications obtained using the
other methods.
Sustainability 2021, 13, x FOR PEER REVIEW 23 of 32
(a) (b)
Figure 20. Confusion Matrices for (a) Quadratic Discriminant Analysis and (b) the Support-Vector Classifier applied to
the marks obtained by students in the three assessed exercises (normalised data).
Looking at the results for QDA, we see that this classification process asserts a 90%
true positive rate, that is to say, when a trio of results from the three exercises is predicted
to have come from a student who used ARPAID, the probability that this is a correct pre-
diction is 90%. The probability of false negatives is 10%. The probability of identifying
true negatives is slightly low at 68.75% and that of false positives is 31.25%.
The best results were obtained using the SVC. Here we see that the classifier has a
true positive rate of 91.4%, while at the same time, the probability of a false negative is
8.6%. The probability of correctly identifying true negatives is down to 69% and that of
false positives is 31%.
The classification quality metrics, precision and sensitivity were very promising for
both types of analysis shown above. The precision—that is, the ratio of true positives to
the sum of these and the false positives, is 0.86 for both QDA and SVC. The sensitivity,
the number of true positives with respect to the total number of positives found overall,
is slightly better: 0.9 for QDA and 0.91 for SVC. Figure 21 shows the values of all the most
commonly used classification quality metrics.
Figure 21. Classification quality metrics.
It can be seen therefore that our data sample contains a structure that justifies its
classification in the manner suggested; this is shown in general and particularly by the
Figure 20.
Confusion Matrices for (
a
) Quadratic Discriminant Analysis and (
b
) the Support-Vector Classifier applied to the
marks obtained by students in the three assessed exercises (normalised data).
Looking at the results for QDA, we see that this classification process asserts a 90%
true positive rate, that is to say, when a trio of results from the three exercises is predicted
to have come from a student who used ARPAID, the probability that this is a correct
prediction is 90%. The probability of false negatives is 10%. The probability of identifying
true negatives is slightly low at 68.75% and that of false positives is 31.25%.
The best results were obtained using the SVC. Here we see that the classifier has a
true positive rate of 91.4%, while at the same time, the probability of a false negative is
8.6%. The probability of correctly identifying true negatives is down to 69% and that of
false positives is 31%.
The classification quality metrics, precision and sensitivity were very promising for
both types of analysis shown above. The precision—that is, the ratio of true positives to
the sum of these and the false positives, is 0.86 for both QDA and SVC. The sensitivity, the
number of true positives with respect to the total number of positives found overall, is
slightly better: 0.9 for QDA and 0.91 for SVC. Figure 21 shows the values of all the most
commonly used classification quality metrics.
Sustainability 2021,13, 3305 23 of 31
Sustainability 2021, 13, x FOR PEER REVIEW 23 of 32
(a) (b)
Figure 20. Confusion Matrices for (a) Quadratic Discriminant Analysis and (b) the Support-Vector Classifier applied to
the marks obtained by students in the three assessed exercises (normalised data).
Looking at the results for QDA, we see that this classification process asserts a 90%
true positive rate, that is to say, when a trio of results from the three exercises is predicted
to have come from a student who used ARPAID, the probability that this is a correct pre-
diction is 90%. The probability of false negatives is 10%. The probability of identifying
true negatives is slightly low at 68.75% and that of false positives is 31.25%.
The best results were obtained using the SVC. Here we see that the classifier has a
true positive rate of 91.4%, while at the same time, the probability of a false negative is
8.6%. The probability of correctly identifying true negatives is down to 69% and that of
false positives is 31%.
The classification quality metrics, precision and sensitivity were very promising for
both types of analysis shown above. The precision—that is, the ratio of true positives to
the sum of these and the false positives, is 0.86 for both QDA and SVC. The sensitivity,
the number of true positives with respect to the total number of positives found overall,
is slightly better: 0.9 for QDA and 0.91 for SVC. Figure 21 shows the values of all the most
commonly used classification quality metrics.
Figure 21. Classification quality metrics.
It can be seen therefore that our data sample contains a structure that justifies its
classification in the manner suggested; this is shown in general and particularly by the
Figure 21. Classification quality metrics.
It can be seen therefore that our data sample contains a structure that justifies its
classification in the manner suggested; this is shown in general and particularly by the
results of SVC, which demonstrates this most clearly. That is to say, the results obtained
support the existence of two sub-populations or classes in the sample: one composed of
those students who used ARPAID and another composed of students who did not. These
two clearly delineated classes can be separated according to the scores obtained in the three
assessed exercises.
Thus, the use of ARPAID does appear to have an effect on the grades
achieved by students, meaning we have reached the objectives of the investigation.
Figure 22 shows histograms of the two classes. It can be seen that, despite some
overlap, the classes are well separated and that with the best student achievement and
smallest variation is class 1, that is, the class representing individuals who used ARPAID.
Sustainability 2021, 13, x FOR PEER REVIEW 24 of 32
results of SVC, which demonstrates this most clearly. That is to say, the results obtained
support the existence of two sub-populations or classes in the sample: one composed of
those students who used ARPAID and another composed of students who did not. These
two clearly delineated classes can be separated according to the scores obtained in the
three assessed exercises. Thus, the use of ARPAID does appear to have an effect on the
grades achieved by students, meaning we have reached the objectives of the investiga-
tion.
Figure 22 shows histograms of the two classes. It can be seen that, despite some over-
lap, the classes are well separated and that with the best student achievement and smallest
variation is class 1, that is, the class representing individuals who used ARPAID.
Figure 22. Histograms showing class separation: Class 1, the Experimental Group; Class 0, the
Control Group.
Regarding the percentage weight of each one of the assessed exercises as a discrimi-
nator for classification, it would seem that the exercise that most distinguishes students
who used ARPAID is the Examination Exercise which had a weighting of 60%. The second
most influential was Post-Test Practical Exercise 2 (the Hoist ring) with a weighting of
38%, and lastly, Post-Test Practical Exercise 1 (the Bit-holder) with a weighting of 2%. Ac-
cording to these data, the Bit-holder exercise is irrelevant to the classification process in-
dicating that students who did use ARPAID and those who didn’t performed equally well
in that assessment. The explanation for this result could be related to the fact that this
assembly is the only one that students must draw using the computer assisted design
package, AutoCAD.
Many of the students in the control group did not complete all the exercises set, which
seems to indicate some difficulties and a lack of confidence when attempting this course
work. In contrast, this issue was not seen amongst students in the experimental group.
Once it has been confirmed that use of ARPAID does indeed have an impact on learn-
ing skills for technical engineering drawing of mechanical assemblies, it is necessary to
establish the nature of this impact: is it negative or positive? The answer is of course ob-
vious looking at the mean scores obtained by students in the three assessed exercises used
in the investigation. As shown in Figure 23, these mean scores are always highest in the
experimental group.
Figure 22.
Histograms showing class separation: Class 1, the Experimental Group; Class 0, the
Control Group.
Regarding the percentage weight of each one of the assessed exercises as a discrimi-
nator for classification, it would seem that the exercise that most distinguishes students
who used ARPAID is the Examination Exercise which had a weighting of 60%. The second
most influential was Post-Test Practical Exercise 2 (the Hoist ring) with a weighting of
38%, and lastly, Post-Test Practical Exercise 1 (the Bit-holder) with a weighting of 2%.
According to these data, the Bit-holder exercise is irrelevant to the classification process
Sustainability 2021,13, 3305 24 of 31
indicating that students who did use ARPAID and those who didn’t performed equally
well in that assessment. The explanation for this result could be related to the fact that
this assembly is the only one that students must draw using the computer assisted design
package, AutoCAD.
Many of the students in the control group did not complete all the exercises set, which
seems to indicate some difficulties and a lack of confidence when attempting this course
work. In contrast, this issue was not seen amongst students in the experimental group.
Once it has been confirmed that use of ARPAID does indeed have an impact on
learning skills for technical engineering drawing of mechanical assemblies, it is necessary
to establish the nature of this impact: is it negative or positive? The answer is of course
obvious looking at the mean scores obtained by students in the three assessed exercises
used in the investigation. As shown in Figure 23, these mean scores are always highest in
the experimental group.
Sustainability 2021, 13, x FOR PEER REVIEW 25 of 32
Figure 23. Descriptive statistics for the Post-Test Practical and Examination exercises.
The fact that the Examination Exercise had a 60% weight in the classification suggests
that use of ARPAID had a positive impact, not only on understanding of the mechanisms
modelled but also on students’ ability to generalise learning to any kind of assembly. That
is, the results suggest that use of ARPAID has a positive impact on meaningful learning
in this subject area and on the acquisition of skills for the graphical representation of me-
chanical assemblies. At a later stage, following this initial experiment and further ARPAID
development, it will be necessary to study this issue.
4. Conclusions
This article describes the evaluation of a mobile application called ARPAID. This ap-
plication can be used to teach the graphical representation of mechanical assemblies in
engineering. It is a new tool and an interesting complement to other learning processes. It
is an accessible and portable tool to help students understand material in the assembly
drawing module of the Graphic Design course at university level.
Results show that ARPAID causes a significant improvement in the achievement of
students in the experimental group using the application.
The increase in achievement recorded for the experimental group was not only ob-
served for exercises involving assemblies modelled in ARPAID but rather, it was seen for
a range of assemblies given to students. This fact allows us to suppose that this improve-
ment in performance can be extrapolated or generalised to any exercise.
ARPAID has been given a favourable welcome by the students. They have reported
that it has been extremely useful in studying course material and has enabled them to
understand essential concepts. Furthermore, the fact they have been provided with a
novel tool that is intuitive to use and takes a very non-traditional approach, has contrib-
uted to an increased motivation amongst students for this part of the course. It must be
emphasised that, during the experimental phase of the investigation, the average number
of logins per student to the ARPAID app was 7.38, with a maximum of 28 uses by a single
student. This shows how well accepted ARPAID has been amongst the student body, and
in some cases, students have continued to use it several months after the end of the aca-
demic year.
The results obtained allow us to conclude that use of ARPAID is a method that com-
plements traditional methods and that it motivates and helps students in their learning
process.
The promising initial results of the project have provided all those involved in this
work with an incentive to go ahead with the further stages of its development with re-
newed energy and enthusiasm.
Figure 23. Descriptive statistics for the Post-Test Practical and Examination exercises.
The fact that the Examination Exercise had a 60% weight in the classification suggests
that use of ARPAID had a positive impact, not only on understanding of the mechanisms
modelled but also on students’ ability to generalise learning to any kind of assembly. That
is, the results suggest that use of ARPAID has a positive impact on meaningful learning
in this subject area and on the acquisition of skills for the graphical representation of
mechanical assemblies. At a later stage, following this initial experiment and further
ARPAID development, it will be necessary to study this issue.
4. Conclusions
This article describes the evaluation of a mobile application called ARPAID. This
application can be used to teach the graphical representation of mechanical assemblies in
engineering. It is a new tool and an interesting complement to other learning processes.
It is an accessible and portable tool to help students understand material in the assembly
drawing module of the Graphic Design course at university level.
Results show that ARPAID causes a significant improvement in the achievement of
students in the experimental group using the application.
The increase in achievement recorded for the experimental group was not only ob-
served for exercises involving assemblies modelled in ARPAID but rather, it was seen for a
range of assemblies given to students. This fact allows us to suppose that this improvement
in performance can be extrapolated or generalised to any exercise.
ARPAID has been given a favourable welcome by the students. They have reported
that it has been extremely useful in studying course material and has enabled them to
Sustainability 2021,13, 3305 25 of 31
understand essential concepts. Furthermore, the fact they have been provided with a novel
tool that is intuitive to use and takes a very non-traditional approach, has contributed to an
increased motivation amongst students for this part of the course. It must be emphasised
that, during the experimental phase of the investigation, the average number of logins per
student to the ARPAID app was 7.38, with a maximum of 28 uses by a single student. This
shows how well accepted ARPAID has been amongst the student body, and in some cases,
students have continued to use it several months after the end of the academic year.
The results obtained allow us to conclude that use of ARPAID is a method that
complements traditional methods and that it motivates and helps students in their learn-
ing process.
The promising initial results of the project have provided all those involved in this
work with an incentive to go ahead with the further stages of its development with renewed
energy and enthusiasm.
5. Study Limitations
An important issue in this investigation has been the impossibility of testing out
ARPAID’s synchronous capabilities during class-based sessions because of the quarantine
imposed due to Covid-19. Indeed, the explanatory sessions and classroom tasks planned
for the second fortnight in March and April 2020 had to be cancelled and replaced with
online videoconferencing sessions.
A further difficulty that emerged during implementation of ARPAID was its deploy-
ment on devices running the IOS operating system. Due to a conflict concerning privacy
and security needs for data collected during use of ARPAID, the directors of the Apple
Development Program prevented its upload to the official App Store at the point when the
investigation was about to take place. Fortunately, it was possible to test a fully functional
beta version of ARPAID using the TestFlight app.
The technological requirements for the extensive use of AR on mobile devices could
be seen as a disadvantage, given that not all students will have access to top of the line
mobile phones. As a result, it is necessary to find a compromise between the available
characteristics of ARPAID and mobile device capabilities such that technology does not
become a discriminatory barrier to the acquisition of learning.
6. Future Lines of Investigation
As described in the previous sections of this article, our investigation was designed to
be developed in three stages. To date only the first of these stages has been completed. The
lines of research that we intend to pursue from this point are the following:
Stage II: Evaluation of ARPAID’s usability, its user interface and general function-
ing. The incorporation of new features, particularly in the field of AR. Inclusion of
mechanisms for stealth assessment.
Stage III: Externalisation of ARPAID using asset bundles and the creation of a remote
database of resources.
A promising line of research, and one which is already in the process of being im-
plemented, involves automating, as far as possible, the uploading of new mechanical
assemblies to ARPAID, regardless of the complexity of said assemblies (both in terms of
the number of components and in the hierarchical structure of sub-assemblies). In this way,
we hope to reduce the level of human intervention to a minimum.
Another possible avenue of work is studying the social and motivational aspects of
ARPAID usage. We hope to discover which elements promote a good learning environment
and to evaluate the effect of features related to gamification along the line of serious gaming.
Supplementary Materials:
The following are available online at https://www.mdpi.com/2071
-1050/13/6/3305/s1, Figure S1: Pre-Test and Post-Test Quizzes, Document S1: Data Analysis,
Spreadsheet S1: Investigation Data, Video S1: DemoARPAID.mp4.
Sustainability 2021,13, 3305 26 of 31
Author Contributions:
Conceptualization, F.J.F.-F.; Investigation, F.J.F.-F. and R.M.-G.; Writing—
Original Draft Preparation, F.J.F.-F. and R.M.-G. Writing—Review & Editing, F.J.F.-F., R.M.-G. and
M.C.-L.; Supervision, F.J.F.-F., R.M.-G. and. M.C.-L.; Project Administration, F.J.F.-F.; Funding Acqui-
sition, F.J.F.-F. All authors have read and agreed to the published version of the manuscript.
Funding:
This work has been financially supported by University of León through the Plan de Apoyo
a la Innovación Docente de la ULE, funding round 2019-20.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Acknowledgments: Not applicable.
Conflicts of Interest: The authors state they have no conflict of interest.
Appendix A. Design Elements of Augmented Reality
In their article concerning the use of AR applications in higher education, Wohlgenannt
et al. [
51
] define the common design elements of these applications. Bearing in mind the
revision proposed by Radianti et al. [
52
] and taking this as our foundation, a list of design
elements used in our mobile application is shown in Table A1.
Table A1. Definition of design elements inour AR application.
Design Element Definition Aplication
Realistic Surroundings The graphic quality of the virtual environment is high quality and
has been designed to replicate a specific real-world environment. 4
Passive Observation Students can look around the virtual environment. 4
Moving Around Student can explore the virtual environment by themselves by
teleporting or flying around. NOT APLICABLE
Basic Interation with Objects Students can select virtual objects and interact with them in
different ways. 4
Assembling Objects Students can select virtual objects and put them together and can
even create new objects by assembling several individual objects. 4
Interaction with Other Users
Students can interact with other students and teachers. The
interaction can take place through an avatar or via communication
tools such as instant messaging or voice chat.
Stage I x
Stage II 4
Stage III 4
Role Management
The AR application offers different functionalities for different roles.
Stage I x
Stages II and III 4
Screen Sharing
The AR application allows students and teachers to stream
applications and files from their local desktop and share them on
virtual screens.
x
User-generated content Students can create new content such as 3D models, and upload
this new content to the virtual environment. x
Instructions
Students have access to tutorials or to instructions about how to use
the AR application, and how to perform learning tasks. 4
Immediate Feedback Students receive immediate textual, audio or haptic feedback. Stage I x
Stages II and III 4
Knowledge Test Students can test their learning progress through knowledge tests,
quizzes or challenges.
Stage I x
Stages II and III 4
Virtual Rewards Students can receive virtual rewards for succesful completion of
learning tasks.
Stage I x
Stages II and III 4
Making Meaningful Choices Students learn in the virtual environment by participating in a
scenario (role-play) that can end in several ways.
Stage I x
Stages II and III 4
Sustainability 2021,13, 3305 27 of 31
Appendix B. External Software for Assets Creation
3D Modelling with CATIA [53]
UNITY3D is a development engine that creates 3D simulations using a set of files and
documents termed assets that can be uploaded to a scene as gameobjects. There are many
different types of asset, but undoubtedly amongst the most important, are the 3D objects or
meshes. Every mesh gameobject has a component called a mesh renderer, which is responsible
for its 3D representation on the screen. Gameobjects also have a component termed transform
that enables the positioning of a mesh within the virtual space and which allows the user
to change its position, rotation, and scale.
UNITY3D does not have native tools for three-dimensional modelling of complex
surfaces. As a consequence, the meshes comprising the components in the assemblies we
used in ARPAID had to be modelled using external software and later imported into the
UNITY3D environment.
ARPAID has been designed with certain quality specifications such that the student
will find the working environment easy to use and as immersive as possible in order to
enjoy a satisfactory User Experience (UX) [
54
]. The goal is that the user has a genuine feeling
of working with a real object thus, the manipulation of said object should be easy and
intuitive. In this way our attention can be focussed on the experiment and not in how to
resolve various issues with the platform.
In order to achieve this goal, the assemblies used within ARPAID must be represented
with a high degree accuracy with respect to geometry and dimension. For this reason, they
have been modelled in one of the most widely respected software packages for Computer
Assisted Design (CAD): CATIA produced by Dassault Systèmes.
Each assembly is formed by making a series of models of its constituent parts, po-
sitioned in space according to set geometrical restrictions to give a hierarchical structure
of solids that is stored in a single product file. As can be appreciated from Figure A1, the
tree-diagram for the product files in CATIA is similar to the hierarchical organisation of
prefabs and gameobjects in UNITY3D, being based on so called parent-child relationships.
Sustainability 2021, 13, x FOR PEER REVIEW 28 of 32
different types of asset, but undoubtedly amongst the most important, are the 3D objects
or meshes. Every mesh gameobject has a component called a mesh renderer, which is respon-
sible for its 3D representation on the screen. Gameobjects also have a component termed
transform that enables the positioning of a mesh within the virtual space and which allows
the user to change its position, rotation, and scale.
UNITY3D does not have native tools for three-dimensional modelling of complex
surfaces. As a consequence, the meshes comprising the components in the assemblies we
used in ARPAID had to be modelled using external software and later imported into the
UNITY3D environment.
ARPAID has been designed with certain quality specifications such that the student
will find the working environment easy to use and as immersive as possible in order to
enjoy a satisfactory User Experience (UX) [54]. The goal is that the user has a genuine feeling
of working with a real object thus, the manipulation of said object should be easy and
intuitive. In this way our attention can be focussed on the experiment and not in how to
resolve various issues with the platform.
In order to achieve this goal, the assemblies used within ARPAID must be repre-
sented with a high degree accuracy with respect to geometry and dimension. For this rea-
son, they have been modelled in one of the most widely respected software packages for
Computer Assisted Design (CAD): CATIA produced by Dassault Systèmes.
Each assembly is formed by making a series of models of its constituent parts, posi-
tioned in space according to set geometrical restrictions to give a hierarchical structure of
solids that is stored in a single product file. As can be appreciated from Figure A1, the tree-
diagram for the product files in CATIA is similar to the hierarchical organisation of prefabs
and gameobjects in UNITY3D, being based on so called parent-child relationships.
Figure A1. Conservation of model hierarchy between CATIA and UNITY3D.
3DStudio Max as a Format Converter [55]
Unfortunately, the models created with CATIA cannot be included directly into
UNITY3D scenes. CATIA generates objects in solid form using a CGM (Convergence Ge-
ometric Modeler) kernel, that is specifically adapted for engineering applications and
CAM/CAE. In contrast, UNITY3D uses the mesh object format, far better suited to appli-
cations where real-time updates and calculations are needed. The available export func-
tions in CATIA do not support meshes, thus it was necessary to transform formats using
an intermediate piece of software. We chose 3DStudio Max for several reasons:
Allows the direct import of native CATIA products (assemblies, *. catProduct).
Allows export of files in the two mesh formats supported by UNITY3D (FBX and
OBJ).
Maintains the hierarchical structure of assembly components related as parent/child.
Enables the modification of the Point or Centre of Rotation (Pivot) of components in
an assembly to adapt these to the rotational restrictions of the assembly modelled.
Figure A1. Conservation of model hierarchy between CATIA and UNITY3D.
3DStudio Max as a Format Converter [55]
Unfortunately, the models created with CATIA cannot be included directly into
UNITY3D scenes. CATIA generates objects in solid form using a CGM (Convergence
Geometric Modeler) kernel, that is specifically adapted for engineering applications and
CAM/CAE. In contrast, UNITY3D uses the mesh object format, far better suited to applica-
tions where real-time updates and calculations are needed. The available export functions
in CATIA do not support meshes, thus it was necessary to transform formats using an
intermediate piece of software. We chose 3DStudio Max for several reasons:
Sustainability 2021,13, 3305 28 of 31
Allows the direct import of native CATIA products (assemblies, *. catProduct).
Allows export of files in the two mesh formats supported by UNITY3D (FBX and OBJ).
Maintains the hierarchical structure of assembly components related as parent/child.
Enables the modification of the Point or Centre of Rotation (Pivot) of components in
an assembly to adapt these to the rotational restrictions of the assembly modelled.
Enables individualised assignment of different materials to the various surfaces in a single
assembly. This task is extremely difficult in the UNITY3D development environment.
Creation of Textures with Gimp [56]
Besides ensuring geometric and dimensional precision, in order to keep the experience
as true to life as possible, it is essential that the appearance of the three dimensional model is
as close as possible to that of the real object it is simulating. This includes paying particular
attention to the types of materials assigned to the object’s surfaces. In UNITY3D, the most
important material parameters depend on the type of shader selected. The shader is linked
to the render pipeline, and enables UV mapping, to simulate roughness, and the adding
of texture to surfaces. Textures give a realistic appearance to objects in the UNITY3D
environment and are provided as bitmaps, either from photographs or synthesised by
graphic design programs.
In developing ARPAID, a program called GIMP was used for the creation and handling
of tileable textures (Figure A2), that repeat to form a mosaic to fill an entire surface. A good
texture is one that faithfully imitates the real appearance of a surface and where the joins
between repetitions are unnoticeable.
Sustainability 2021, 13, x FOR PEER REVIEW 29 of 32
Enables individualised assignment of different materials to the various surfaces in a
single assembly. This task is extremely difficult in the UNITY3D development envi-
ronment.
Creation of Textures with Gimp [56]
Besides ensuring geometric and dimensional precision, in order to keep the experi-
ence as true to life as possible, it is essential that the appearance of the three dimensional
model is as close as possible to that of the real object it is simulating. This includes paying
particular attention to the types of materials assigned to the object’s surfaces. In UNITY3D,
the most important material parameters depend on the type of shader selected. The shader
is linked to the render pipeline, and enables UV mapping, to simulate roughness, and the
adding of texture to surfaces. Textures give a realistic appearance to objects in the
UNITY3D environment and are provided as bitmaps, either from photographs or synthe-
sised by graphic design programs.
In developing ARPAID, a program called GIMP was used for the creation and han-
dling of tileable textures (Figure A2), that repeat to form a mosaic to fill an entire surface.
A good texture is one that faithfully imitates the real appearance of a surface and where
the joins between repetitions are unnoticeable.
(a) (b)
Figure A2. (a) Tileable texture for a knurled surface; (b) UV Normal map of surface.
Icons in Illustrator [57]
A hugely important factor for a satisfactory ARPAID User Experience (UX) is the
design of the Graphic User Interface (GUI). The use of icons to transmit information notice-
ably improves the usability of the app, but only where the iconography employed has
been well thought out and designed bearing in mind current conventions in digital semi-
otics (see Figure A3). The main objectives of our GUI design were, fundamentally: clarity;
simplicity of form; uniformity; colour discrimination; and ease and intuitiveness of inter-
pretation [58].
Figure A2. (a) Tileable texture for a knurled surface; (b) UV Normal map of surface.
Icons in Illustrator [57]
A hugely important factor for a satisfactory ARPAID User Experience (UX) is the de-
sign of the Graphic User Interface (GUI). The use of icons to transmit information noticeably
improves the usability of the app, but only where the iconography employed has been well
thought out and designed bearing in mind current conventions in digital semiotics (see
Figure A3). The main objectives of our GUI design were, fundamentally: clarity; simplicity
of form; uniformity; colour discrimination; and ease and intuitiveness of interpretation [
58
].
AutoCAD: Blueprints for Models [59]
Since we are concerned with mechanical assemblies in the context of teaching technical
drawing, a great deal of documentation received by students is in the form of standardised
blueprints, with components represented both in perspective and dihedral views. Thus,
while three-dimensional modelling was completed in CATIA, standardised blueprints were
created in AutoCAD due to its impressive capabilities as a technical drawing package and
its versatility when creating drawings in different formats.
Sustainability 2021,13, 3305 29 of 31
Sustainability 2021, 13, x FOR PEER REVIEW 30 of 32
Figure A3. Design template for App icons as displayed in Illustrator.
AutoCAD: Blueprints for Models [59]
Since we are concerned with mechanical assemblies in the context of teaching tech-
nical drawing, a great deal of documentation received by students is in the form of stand-
ardised blueprints, with components represented both in perspective and dihedral views.
Thus, while three-dimensional modelling was completed in CATIA, standardised blue-
prints were created in AutoCAD due to its impressive capabilities as a technical drawing
package and its versatility when creating drawings in different formats.
References
1. Del-Cerro-Velázquez, F.; Morales-Méndez, G. Realidad Aumentada como herramienta de mejora de la inteligencia espacial en
estudiantes de educación secundaria. Rev. Educ. Distancia 2017, 54, doi:10.6018/red/54/5.
2. Horizon 2020 Projects. Horizon 2020. Available online: https://ec.europa.eu/programmes/horizon2020/en/h2020-sections-pro-
jects (accessed on 21 October 2020).
3. Suárez, J.P.; González, P.M.; Martín, G.; García, M. Expresión Gráfica: Pasado, Presente y Futuro en el Diseño en la Ingeniería.
Available online: https://docplayer.es/53548935-Expresion-grafica-pasado-presente-y-futuro-en-el-diseno-en-la-inge-
nieria.html (accessed on 21 October 2020).
4. Cabero-Almenara, J.; García Jiménez, F.; Barroso Osuna, J. La producción de objetos de aprendizaje en “Realidad Aumentada”:
La experiencia del SAV de la Universidad de Sevilla. IJERI Int. J. Educ. Res. Innov. 2016, 6, 110–123.
5. Craig, A.B. Understanding Augmented Reality: Concepts and Applications; Morgan Kaufmann: Waltham, MA, USA, 2013.
6. Azuma, R.T. A survey of Augmented Reality. Presence Teleoperators Virtual Environ. 1997, 6, doi:10.1162/pres.1997.6.4.355.
7. Akçayir, M.; Akçayir, G.; Pektaş, H.M.; Ocak, M.A. Augmented reality in science laboratories: The effects of augmented reality
on university students’ laboratory skills and attitudes toward science laboratories. Comput. Hum. Behav. 2016, 57, 334–342,
doi:10.1016/j.chb.2015.12.054.
8. Reuter, R.; Hauser, F.; Muckelbauer, D.; Stark, T.; Antoni, E.; Mottok, J.; Wolff, C. Using Augmented Reality in Software Engi-
neering Education? First Insights to a Comparative Study of 2D and AR UML Modeling. In Proceedings of the 52nd Hawaii
International Conference on System Sciences, Grand Wailea, HI, USA, 8–11 January 2019; Volume 6, pp. 7798–7807,
doi:10.24251/hicss.2019.938.
9. Chen, H.; Feng, K.; Mo, C.; Cheng, S.; Guo, Z.; Huang, Y. Application of augmented reality in engineering graphics education.
In Proceedings of the ITME 2011—IEEE International Symposium on IT in Medicine and Education, Guandzhou, China, 9–11
December 2011; Volume 2, pp. 362–365, doi:10.1109/ITiME.2011.6132125.
10. Berkemeier, L.; Zobel, B.; Werning, S.; Ickerott, I.; Thomas, O. Engineering of Augmented Reality-Based Information Systems:
Design and Implementation for Intralogistics Services. Bus. Inf. Syst. Eng. 2019, 61, 67–89, doi:10.1007/s12599-019-00575-6.
11. Salar, R.; Arici, F.; Caliklar, S.; Yilmaz, R.M. A Model for Augmented Reality Immersion Experiences of University Students
Studying in Science Education. J. Sci. Educ. Technol. 2020, 29, 257–271, doi:10.1007/s10956-019-09810-x.
12. Fuchsová, M.; Adamková, M.; Lapšanská, M.P. Uses of Augmented Reality in Biology Education. In Augmented Reality in Edu-
cational Settings; Brill|Sense: Leiden, The Nethelands, 2019; pp .168–194, doi:10.1163/9789004408845_008.
Figure A3. Design template for App icons as displayed in Illustrator.
References
1.
Del-Cerro-Velázquez, F.; Morales-Méndez, G. Realidad Aumentada como herramienta de mejora de la inteligencia espacial en
estudiantes de educación secundaria. Rev. Educ. Distancia 2017,54, 1–14. [CrossRef]
2.
Horizon 2020 Projects. 2020. Available online: https://ec.europa.eu/programmes/horizon2020/en/h2020-sections-projects
(accessed on 21 October 2020).
3.
Suárez, J.P.; González, P.M.; Martín, G.; García, M. Expresión Gráfica: Pasado, Presente y Futuro en el Diseño en la Ingeniería.
Available online: https://docplayer.es/53548935-Expresion-grafica-pasado-presente-y-futuro-en-el-diseno-en-la-ingenieria.
html (accessed on 21 October 2020).
4.
Cabero-Almenara, J.; García Jiménez, F.; Barroso Osuna, J. La producción de objetos de aprendizaje en “Realidad Aumentada”:
La experiencia del SAV de la Universidad de Sevilla. IJERI Int. J. Educ. Res. Innov. 2016,6, 110–123.
5. Craig, A.B. Understanding Augmented Reality: Concepts and Applications; Morgan Kaufmann: Waltham, MA, USA, 2013.
6. Azuma, R.T. A survey of Augmented Reality. Presence Teleoper. Virtual Environ. 1997,6. [CrossRef]
7.
Akçayir, M.; Akçayir, G.; Pekta¸s, H.M.; Ocak, M.A. Augmented reality in science laboratories: The effects of augmented reality
on university students’ laboratory skills and attitudes toward science laboratories. Comput. Hum. Behav.
2016
,57, 334–342.
[CrossRef]
8.
Reuter, R.; Hauser, F.; Muckelbauer, D.; Stark, T.; Antoni, E.; Mottok, J.; Wolff, C. Using Augmented Reality in Software
Engineering Education? First Insights to a Comparative Study of 2D and AR UML Modeling. In Proceedings of the 52nd Hawaii
International Conference on System Sciences, Grand Wailea, HI, USA, 8–11 January 2019; Volume 6, pp. 7798–7807. [CrossRef]
9.
Chen, H.; Feng, K.; Mo, C.; Cheng, S.; Guo, Z.; Huang, Y. Application of augmented reality in engineering graphics education.
In Proceedings of the ITME 2011—IEEE International Symposium on IT in Medicine and Education, Guandzhou, China, 9–11
December 2011; Volume 2, pp. 362–365. [CrossRef]
10.
Berkemeier, L.; Zobel, B.; Werning, S.; Ickerott, I.; Thomas, O. Engineering of Augmented Reality-Based Information Systems:
Design and Implementation for Intralogistics Services. Bus. Inf. Syst. Eng. 2019,61, 67–89. [CrossRef]
11.
Salar, R.; Arici, F.; Caliklar, S.; Yilmaz, R.M. A Model for Augmented Reality Immersion Experiences of University Students
Studying in Science Education. J. Sci. Educ. Technol. 2020,29, 257–271. [CrossRef]
12.
Fuchsová, M.; Adamková, M.; Lapšanská, M.P. Uses of Augmented Reality in Biology Education. In Augmented Reality in
Educational Settings; Brill|Sense: Leiden, The Nethelands, 2019; pp. 168–194. [CrossRef]
13.
Ma, M.; Fallavollita, P.; Seelbach, I.; Von Der Heide, A.M.; Euler, E.; Waschke, J.; Navab, N. Personalized augmented reality for
anatomy education. Clin. Anat. 2016,29, 446–453. [CrossRef] [PubMed]
14.
Carlson, K.J.; Gagnon, D.J. Augmented Reality Integrated Simulation Education in Health Care. Clin. Simul. Nurs.
2016
,
12, 123–127. [CrossRef]
15.
Cascales-Martínez, A.; Martínez-Segura, M.J.; Pérez-López, D.; Contero, M. Using an augmented reality enhanced tabletop
system to promote learning of mathematics: A case study with students with special educational needs. Eurasia J. Math. Sci.
Technol. Educ. 2017,13, 355–380. [CrossRef]
Sustainability 2021,13, 3305 30 of 31
16.
Lin, H.C.K.; Chen, M.C.; Chang, C.K. Assessing the effectiveness of learning solid geometry by using an augmented reality-
assisted learning system. Interact. Learn. Environ. 2015,23, 799–810. [CrossRef]
17.
Salmi, H.; Thuneberg, H.; Vainikainen, M.-P. Making the invisible observable by Augmented Reality in informal science education
context. Int. J. Sci. Educ. Part B 2017,7, 253–268. [CrossRef]
18.
Shen, L.C.; Wu, T.T.; Hsu, W.C. The Application of Augmented Reality to the Education of Chemistry—Take the Course of Nature
Science in Junior High School as an Example. In Innovative Technologies and Learning; Springer: Berlin/Heidelberg, Germany, 2019;
Volume 11937 LNCS, pp. 41–48. [CrossRef]
19.
Bursztyn, N.; Shelton, B.; Walker, A.; Pederson, J. Increasing undergraduate interest to learn geoscience with GPS-based
augmented reality field trips on students’ own smartphones. GSA Today 2017,27, 4–10. [CrossRef]
20.
Chen, C.-P.; Wang, C.-H. Employing augmented-reality-embedded instruction to disperse the imparities of individual differences
in earth science learning. J. Sci. Educ. Technol. 2015,24, 835–847. [CrossRef]
21.
Squires, D.R. Instructional Designs and Educational Technologies within Augmented Reality Transmedia Storytelling: IDET ARTS;
Springer: Cham, Switzerland, 2019; pp. 121–128. [CrossRef]
22.
Buhl, M. Students and Teachers as Developers of Visual Learning Designs with Augmented Reality for Visual Arts Education. In
Proceedings of the 16th European Conference on E-Learning (ECEL 2017), Porto, Portugal, 26–27 October 2017; Mesquita, A.,
Peres, P., Eds.; Academic Conferences Ltd.: Reading, UK; pp. 94–100.
23.
Huang, Y.; Li, H.; Fong, R. Using Augmented Reality in early art education: A case study in Hong Kong kindergarten. Early Child
Dev. Care 2016,186, 879–894. [CrossRef]
24.
Bower, M.; Howe, C.; McCredie, N.; Robinson, A.; Grover, D. Augmented Reality in education—Cases, places and potentials.
Educ. Media Int. 2014,51, 1–15. [CrossRef]
25. Trujillo Flórez, L.M. Teorías Pedagógicas Contemporáneas, 1st ed.; Fondo Editorial Areandino: Bogotá, Columbia, 2017.
26. Ackermann, E. Piaget’s constructivism, Papert’s constructionism: What’s the difference. Futur. Learn. Gr. Publ. 2001,5, 438.
27. Fosnot, C.T.; Perry, R.S. Constructivism: A Psychological Theory of Learning. In Constructivism: Theory, Perspectives and Practice;
Fosnot, C.T., Ed.; Teacher College Press: New York, NY, USA; London, UK, 2005.
28.
Piaget, J. The Role of Action in the Development of Thinking. In Knowledge and Development: Advances in Research and Theory;
Overton, W.F., Gallagher, J.M., Eds.; Springer: Boston, MA, USA, 1977; Volume 1, pp. 17–42. [CrossRef]
29.
Mevarech, Z.R.; Kramarski, B. Vygotsky and Papert: Social-cognitive interactions within Logo environments. Br. J. Educ. Psychol.
1993,63, 96–109. [CrossRef] [PubMed]
30.
Vygotsky, L.S. Mind and Society: The Development of Higher Psychological Processes; Cole, M., John-Steiner, V., Scribner, S., Souberman,
E., Eds.; Harvard University Press: Cambridge, MA, USA, 1978.
31.
Ausubel, D.P. The Nature of Meaning and Meaningful Learning. In The Acquisition and Retention of Knowledge: A Cognitive View;
Springer: Dordrecht, The Netherlands, 2000; pp. 67–100. [CrossRef]
32.
Bruner, J.S. The Act of Discovery. In Search of Pedagogy; Routledge Journals: New York, NY, USA; Taylor & Francis: New York, NY,
USA, 2006; pp. 57–66.
33.
Papert, S. Situating Constructionism. In Constructionism; Harel, I., Papert, S., Eds.; Ablex Publishing Corporation: Norwood, NJ,
USA, 1991.
34. Papert, S. La Máquina de los Niños. Replantearse la Educación en la Era de los Ordenadores, 1st ed.; Paidós: Barcelona, Spain, 1995.
35.
Resnick, M. Lifelong Kindergarten: Cultivating Creativity Through Projects, Passion, Peers and Play, 1st ed.; The MIT Press: Cambridge,
MA, USA, 2017.
36.
Chen, K.-W.; Hsu, F.-C.; Hsieh, Y.-Z.; Chou, C.-H. To Design an Interactive Learning System for Child by Integrating Blocks with
Kinect. In Proceedings of the EDUCON2014—IEEE Global Engineering Education Conference, Istanbul, Turkey, 3–5 April 2014;
pp. 20–22.
37.
Klari´c, Š.; Hadžiahmetovi´c, H.; Novoselovi´c, D.; Havrlišan, S. Implementation and comparative analysis of mobile phone
application for learning and teaching in mechanical engineering education. Teh. Vjesn. 2019,26, 1176–1181. [CrossRef]
38.
Kannapiran, S.; Kob, C.G.C.; Rus, R.C.; Shah, A.; Dewi, N.R. Development of mobile application upon mechanical engineering
students’ learning styles. J. Phys. Conf. Ser. 2020,1567. [CrossRef]
39.
Al-Khanjari, Z.; Al-Kindi, Z.; Al-Kindi, E.; Kraiem, N. Developing educational mobile application architecture using SOA. Int. J.
Multimed. Ubiquitous Eng. 2015,10, 247–254. [CrossRef]
40.
Ghayyur, S.A.K.; Awan, D.; Khiyal, M.S.H. A Case of Engineering Quality for Mobile Healthcare Applications Using Augmented
Personal Software Process Improvement. Mob. Inf. Syst. 2016,2016. [CrossRef]
41.
Clark, R.C.; Nguyen, F.; Sweller, J. Efficiency in Learning: Evidence-Based Guidelines to Manage Cognitive Load; Pfeifer: San Francisco,
CA, USA, 2006.
42. Meier, R. Professional Android 4 Application Development; Wiley: Indianapolis, IN, USA, 2012.
43.
Shute, V.; Ventura, M. Stealth Assessment. In Stealth Assessment: Measuring and Supporting Learning in Video Games; Shute, V.,
Ventura, M., Eds.; The MIT Press: Cambridge, MA, USA, 2013; pp. 31–65. [CrossRef]
44.
Plataforma de Desarrollo en Tiempo Real de Unity. Motor de VR y AR en 3D y 2D. Available online: https://unity.com/es
(accessed on 21 October 2020).
45.
Marco de Trabajo AR Foundation de Unity. Software de Realidad Aumentada Para Desarrollo Multiplataforma. Available online:
https://unity.com/es/unity/features/arfoundation (accessed on 21 October 2020).
Sustainability 2021,13, 3305 31 of 31
46.
Shute, V.J.; Wang, L.; Greiff, S.; Zhao, W.; Moore, G. Measuring problem solving skills via stealth assessment in an engaging video
game. Comput. Human Behav. 2016,63, 106–117. [CrossRef]
47.
Li, X.; You, Y. Kano model analysis required in APP interactive design based on mobile user experience. Int. J. Multimed.
Ubiquitous Eng. 2016,11, 247–258. [CrossRef]
48.
Alomari, H.W.; Ramasamy, V.; Kiper, J.D.; Potvin, G. A User Interface (UI) and User eXperience (UX) evaluation framework for
cyberlearning environments in computer science and software engineering education. Heliyon 2020,6, e03917. [CrossRef]
49.
Pérez Juste, R.; Galán González, A.; Quintanal Díaz, J. Métodos y Diseños de Investigación en Educación; Universidad Nacional de
Educación a Distancia: Madrid, Spain, 2012.
50. Faul, A.C. A Concise Introduction to Machine Learning; Taylor And Francis: Boca Ratón, FL, USA, 2020.
51.
Wohlgenannt, I.; Fromm, J.; Stieglitz, S.; Radianti, J.; Majchrzak, T.A. Virtual Reality in Higher Education: Preliminary Results from
a Design-Science-Research Project. In Proceedings of the 28th International Conference on Information Systems Development,
Toulon, France, 28–30 August 2019.
52.
Radianti, J.; Majchrzak, T.A.; Fromm, J.; Wohlgenannt, I. A systematic review of immersive virtual reality applications for higher
education: Design elements, lessons learned, and research agenda. Comput. Educ. 2020,147, 103778. [CrossRef]
53.
Design Engineering. CATIA—Dassault Systèmes. Available online: https://www.3ds.com/products-services/catia/ (accessed
on 21 October 2020).
54.
Zhang, J.; Kamioka, E.; Tan, P.X. Emotions detection of user experience (Ux) for mobile augmented reality (mar) applications. Int.
J. Adv. Trends Comput. Sci. Eng. 2019,8, 63–67. [CrossRef]
55.
Autodesk. 3ds Max. 3D Modeling, Animation & Rendering Software. Available online: https://www.autodesk.com/products/
3ds-max/overview (accessed on 21 October 2020).
56. GIMP—GNU Image Manipulation Program. Available online: https://www.gimp.org/ (accessed on 21 October 2020).
57.
Adobe Illustrator. Software de Gráficos Vectoriales. Available online: https://www.adobe.com/es/products/illustrator.html
(accessed on 21 October 2020).
58.
Hussain, A.; Mkpojiogu, E.O.C.; Ishak, N.; Mokhtar, N. A study on the perceived mobile experience of myeg users. Int. J. Interact.
Mob. Technol. 2019,13, 4–23. [CrossRef]
59.
Autodesk. AutoCAD for Mac & Windows. 2D/3D CAD Software. Available online: https://www.autodesk.com/products/
autocad/overview (accessed on 21 October 2020).
... Temuan studi ini telah meyakinkan bahwa selain mendukung interaktivitas pembelajaran, simulasi virtual berdampak pada peningkatan performa penalaran sehingga ini dapat dijadikan sebagai alat kognitif dalam konteks pembelajaran yang lebih luas. Visualisasi konsep atau teori yang abstrak dapat memotivasi mahasiswa dalam belajar dan keterampilan berpikir tingkat tinggi mereka dapat berkembang (Fraile-Fernández et al., 2021). Walaupun dalam konteks studi saat ini kami tidak mengobservasi secara ekplisit terkait motivasi belajar mahasiswa dengan pengaplikasian simulasi virtual ini, tapi nyatanya penerimaan atau respon mahasiswa sangat baik yang ditandai dengan interaktivitas yang terbangun dalam pembelajaran. ...
Article
Full-text available
Kesulitan tutor pada semua jenis pembelajaran (tatap muka dan online) adalah ketika mereka mengajarkan konsep abstrak pada perkuliahan fisika modern, terutama untuk meningkatkan keterampilan penalaran mahasiswa. Kami melihat peluang pada kemajuan teknologi digital dapat membantu mengatasi masalah ini. Studi saat ini bertujuan untuk menganalisis kinerja keterampilan penalaran mahasiswa pada mata kuliah fisika modern menggunakan simulasi virtual PhET yang terintegrasi di dalam platform LMS. Penelitian ini menggunakan desain ekperimen, dimana dua kelompok sampel disiapkan (kelompok eksperimen dan kontrol), dan diintervensi dengan dua proses belajar. Kelompok eksperimen dibelajarkan dengan simulasi virtual PhET yang terintegrasi di dalam platform LMS, sementara kelompok kontrol dengan pembelajaran tatap muka dengan metode ekspositori. Kedua kelompok sampel adalah mahasiswa yang menempuh matakuliah fisika modern di FKIP Universitas Mataram. Instrumen yang telah valid dipekerjakan untuk mengukur kinerja keterampilan penalaran mahasiswa sebagai pretest dan posttest. Hasil analisis pretest dan posttest menunjukkan bahwa kinerja keterampilan penalaran mahasiswa pada perkuliahan fisika modern meningkat dengan intervensi pembelajaran menggunakan simulasi virtual PhET terintegrasi dengan platform LMS. Hasil analisis statistik juga menunjukkan keunggulan intervensi pembelajaran menggunakan simulasi virtual PhET terintegrasi dengan platform LMS jika dibandingkan dengan metode ekpositori. The Experience of Teaching Modern Physics Using PhET Virtual Simulations: An Analysis of Student Reasoning Skill Performance Abstract The difficulty for tutors in all types of learning (face-to-face and online) is when they teach abstract concepts in modern physics courses, especially to improve students' reasoning skills. We see an opportunity in advances in digital technology that can help overcome this problem. The current study aims to analyze the performance of students' reasoning skills in modern physics courses using PhET virtual simulations that are integrated into the LMS platform. This study used an experimental design, in which two sample groups were prepared (experimental and control groups), and intervened with two learning processes. The experimental group was taught by PhET virtual simulation integrated into the LMS platform, while the control group was taught by face-to-face learning using the expository method. The two sample groups are students taking modern physics courses at FKIP, University of Mataram. Instruments that have been declared valid are employed to measure the performance of students' reasoning skills as a pretest and posttest. The results of the pretest and posttest analysis show that the performance of students' reasoning skills in modern physics courses increases with learning interventions using PhET virtual simulations integrated with the LMS platform. The results of the statistical analysis shows the advantages of learning interventions using PhET virtual simulations integrated with the LMS platform when compared to the expository method.
... The findings of this study have confirmed that in addition to supporting learning interactivity, virtual simulations have an impact on improving the reasoning performance of STEM students so that it can be used as a cognitive tool in a wider learning context. Visualization of abstract concepts or theories can motivate STEM students in learning and their higher order thinking skills can develop [35]. Although in the context of the current study we did not explicitly observe the learning motivation of STEM students with the application of this virtual simulation, in fact the acceptance or response of STEM students was very good which was marked by the interactivity that was built in learning. ...
Article
Full-text available
The difficulty of tutors in all types of learning (face-to-face and online) is when they teach abstract concepts in modern physics courses, especially to improve students' reasoning skills. We see an opportunity that advances in digital technology can help overcome this problem. This study aims to improve the reasoning performance of STEM students in modern physics courses using virtual simulation integrated with the LMS platform. Experimental design was prepared with one control group (face-to-face learning with expository method). The sample was 54 STEM students at the University of Mataram which was divided into the experimental group (n = 27) and the control group (n = 27). Reasoning skills were measured using an essay test instrument, and the results were analyzed descriptively (analysis of increasing reasoning skills scores) and statistically (analysis of differences in reasoning skills scores between sample groups). The results of this study have clearly shown that the reasoning performance of STEM students in modern physics courses can be improved by learning using virtual simulation on the LMS platform. Descriptive and statistical analysis of the reasoning performance of STEM students shows the advantages of learning using virtual simulation when compared to face-to-face learning that relies on expository methods. We recommend using virtual simulation on the LMS platform to teach abstract concepts that are not limited to modern physics but in science learning in general.
... The teachers made a great effort to finalize the programming of a mobile application, based on Augmented Reality, and adapt it to the new pandemic circumstances so that the students could work on the competences on mechanisms included in the syllabus of the subject. The use of the app was an important aid and has been highly valued by the students, as well as having a positive influence on learning [32]. This academic year, 2022-2023, is the third year that the app is being used, and the experience has been awarded the Prize for Innovation in Teaching 2022, awarded unanimously by the Social Council of the University of León. ...
Article
Full-text available
University education in times of COVID-19 was forced to seek alternative teaching/learning methods to the traditional ones, having to abruptly migrate to the online modality, changes that have repercussions on student satisfaction. That is why this study aims to compare the level of student satisfaction in face-to-face and “forced” online modalities under COVID-19. A quantitative, cross-sectional methodology was applied to two groups of students: Under a face-to-face modality (n = 116) and under an online modality (n = 120), to which a questionnaire was applied under a Likert scale, with four dimensions: Course design structure, content, resources, and instructor. Non-parametric statistics, specifically the Mann–Whitney U-test, were used to compare the groups. The results showed that there are significant differences in the level of satisfaction of students in the face-to-face and online “forced” modalities (p = 0.01984 < 0.05), and the dimensions of the level of satisfaction that presented significant differences were course design structure (p = 0.04523 < 0.05) and content (p = 0.00841 < 0.05). The research shows that students in the face-to-face modality express a higher level of satisfaction, which is reflected in the dimension design structure of the course, specifically in its workload indicator, as well as in the dimension content, in its indicators, overlapping with other courses and materials.
Article
Full-text available
A virtual tour of the onshore wind farm near Gaomei Wetland, Taichung City, Taiwan, was produced by producing panoramic images of the site by stitching images captured with a full-frame digital single-lens reflex camera and a multi-row panorama instrument, which automatically and precisely divided each scene into several images. Subsequently, the image stitching quality was improved by calculating the root mean square error (RMSE) of tie point matching and adjusting the tie points. Errors due to eccentricity attributed to the camera’s relative position to the rotational axis of the multi-row panorama instrument were examined and solved; the effect of the overlap rate on image stitching quality was also investigated. According to the study results, the overlap rate between the original images was inversely proportional to the RMSE and directly proportional to the time required for photography and image processing. The stitching quality was improved by resolving eccentricity and by increasing the number of tie points. The RMSEs of the panoramas of all stations were all less than 5 pixels. Subsequently, multimedia materials providing information on wind turbine attributes were combined with the panorama platform to establish a virtual reality tour platform. The content of the platform could be accessed with a smartphone and viewed with a virtual reality device and could promote both tourist attractions and wind energy.
Article
Full-text available
Abstract: Increasing student motivation and engagement in classroom (and during the study in general) is the aim of every lecturer. Never stopping development of new digital tools and media present a new challenge in the educational process. The goal of this research is to increase the knowledge and understanding of the influence of Bring Your Own Device (BYOD) approach (and use of the mobile devices in classrooms in general) on: teachers’ practice and students’ classroom activities, students’ attitude about bringing the mobile phones in the class and mobile phone applications in education processes. This research focuses on undergraduate and postgraduate mechanical engineering students. Personal reflection of the lecturers and online survey for students was used as a tool to investigate participants’ attitude towards mobile applications as a method of promotion of active learning in engineering education. Keywords: Bring Your Own Device (BYOD); engineering education; mobile phones; reflection
Article
Full-text available
The use of technology in this era of globalisation is growing from day to seconds. Mobile learning or also known as m-learning is commonly heard lately as we are in the 21st century of education. Mobile applications developed are to enhance m-learning among students. Henceforth, making students engaged to m-learning is only possible when we had identified their learning styles. Current paper presents the development of mobile application for Polytechnic Mechanical Engineering Students (PolyMES) based on the previous analysis of learning styles among premier polytechnic mechanical engineering students in Malaysia. The aim of this study is achieved whereby the design for PolyMES is developed according to these mechanical engineering students’ learning style. The mobile application developed requires the understanding of students’ learning style to enhance m-learning. PolyMES is designed to be user friendly application that enables students to study the subject at their own space. They can read the simple notes with related video and images. In addition, students are able to test themselves with the quiz and exercises given and also view the list of formulas and symbols with sample calculation as a practice. The features available in PolyMES designed to suit students’ different types of learning styles. As known every individual has their own learning styles of retaining new information and skills. Hereby, PolyMES design is developed to cater students using smart phones at various learning style and might be useful for further implementations of m-learning.
Article
Full-text available
Despite the widespread availability and increasing use of cyberlearning environments, there remains a need for more research about their usefulness in undergraduate education, particularly in STEM education. The process of evaluating the usefulness of a cyberlearning environment is an essential measure of its success and is useful in assisting the design process and ensuring user satisfaction. Unfortunately, there are relatively few empirical studies that provide a comprehensive test of the usefulness of cyberlearning in education. Additionally, there is a lack of standards upon whose usefulness evaluators agree. In this research, we present multiple user studies that can be used to assess the usefulness of a cyberlearning environment used in Computer Science and Software Engineering courses through testing its usability and measuring its utility using user interface and user experience evaluations. Based on these assessments, we propose an evaluation framework to evaluate cyberlearning environments. To help illustrate the framework utility and usability evaluations, we explain them through an example SEP-CyLE (Software Engineering and Programming Cyberlearning Environment). The evaluation techniques used are cognitive walkthroughs with a think-aloud protocol and a heuristic evaluation survey. We further use a network-based analysis to find the statistically significant correlated responses in the heuristic evaluation survey with regard to the students’ perceptions of using SEP-CyLE. Our goal is to improve cyberlearning practice and to emphasize the need for evaluating cyberlearning environments with respect to its designated tasks and its users using UI/UX evaluations. Our experiments demonstrated participants were able to utilize SEP-CyLE efficiently to accomplish the tasks we posed to them and to enhance their software development concepts, specifically, software testing. We discovered areas of improvement in the visibility and navigation of SEP-CyLE's current design. We provide our recommendations for improving SEP-CyLE and provide guidance and possible directions for future research on designing cyberlearning environments for computer education.
Article
Full-text available
The aim of this study is to investigate the augmented reality (AR) immersion experiences of university students studying in science education. The relationship between interest, usability, emotional investment, focus of attention, presence and flow was examined for university students studying in the science education department who used AR technology. The research design adopted is a correlational method, an established experimental research method. The sample consisted of 180-university students studying in the science education department (32 males, 148 females) of the Faculty of Education. The data obtained were analyzed according to the structural equation model. A model was developed which is capable of explaining 71% of the variance in university students’ flow experiences. According to our model, focus of attention and presence of university students have an influence on their flow experiences. The study also showed that emotional investment and presence of university students influences their focus of attention. In addition, usability and emotional investment of the students influences their interest.
Article
Full-text available
Researchers have explored the benefits and applications of virtual reality (VR) in different scenarios. VR possesses much potential and its application in education has seen much research interest lately. However, little systematic work currently exists on how researchers have applied immersive VR for higher education purposes that considers the usage of both high-end and budget head-mounted displays (HMDs). Hence, we propose using systematic mapping to identify design elements of existing research dedicated to the application of VR in higher education. The reviewed articles were acquired by extracting key information from documents indexed in four scientific digital libraries, which were filtered systematically using exclusion, inclusion, semi-automatic, and manual methods. Our review emphasizes three key points: the current domain structure in terms of the learning contents, the VR design elements, and the learning theories, as a foundation for successful VR-based learning. The mapping was conducted between application domains and learning contents and between design elements and learning contents. Our analysis has uncovered several gaps in the application of VR in the higher education sphere—for instance, learning theories were not often considered in VR application development to assist and guide toward learning outcomes. Furthermore, the evaluation of educational VR applications has primarily focused on usability of the VR apps instead of learning outcomes and immersive VR has mostly been a part of experimental and development work rather than being applied regularly in actual teaching. Nevertheless, VR seems to be a promising sphere as this study identifies 18 application domains, indicating a better reception of this technology in many disciplines. The identified gaps point toward unexplored regions of VR design for education, which could motivate future work in the field.
Article
Full-text available
Mobility is the trend right now. It is transforming the user experience from the confines of the desk to the convenience of anytime-anywhere. MyEG Services Berhad ("MYEG") is a concessionaire for Malaysian Electronic Government ("E-Government") MSC Flagship Application. MYEG builds, operates and owns the electronic channel that delivers services from various Government agencies to Malaysia citizens and businesses. To make their services up-to-date and in trend, MyEG app was developed and can also be used to check summons, pay summons, renew road-tax and renew auto insurance. To make sure that this application is efficient, fulfilling the customer needs and satisfaction , a usability evaluation was conducted. The evaluation was conducted in "Jabatan Teknologi Maklumat & Komunikasi, Politeknik Seberang Perai", with 15 participants consisting of both lecturers and students. The think-aloud protocol was used while conducting the evaluation. The result of the evaluation revealed that overall the app is efficient, successful in fulfilling the users' requirement and needs and promotes users mobile experience.
Conference Paper
Although there has been much speculation about the potential of Augmented Reality (AR) in teaching for learning material, there is a significant lack of empirical proof about its effectiveness and implementation in higher education. We describe a software to integrate AR using the Microsoft Hololens into UML (Unified Modeling Language) teaching. Its user interface is laid out to overcome problems of existing software. We discuss the design of the tool and report a first evaluation study. The study is based upon effectiveness as a metric for students performance and components of motivation. The study was designed as control group experiment with two groups. The experimental group had to solve tasks with the help of the AR modeling tool and the control group used a classic PC software. We identified tendencies that participants of the experimental group showed more motivation than the control group. Both groups performed equally well.
Chapter
In the modern society featuring developed information technology, the integration of technology into various aspects of life has been normal. Studies regarding the integration of technology into educational scenarios have also flourished. This study will integrate the augmented reality (AR) technology into the teaching of natural science for the eighth grade of middle school. This technology assists learners in learning about chemical units, such as “mole number”, “atomic weight” and “molecular weight”, to understand the composition and characteristics of matter. The concepts of this unit are relatively abstract, so the characteristics of AR combined with learners’ life experience can present the world of microscopic particles, which are invisible to the naked eye and complex and abstract, enabling learners to establish correct rules and imagine the microscopic particle world properly and paving a smooth way for future chemistry learning.
Chapter
A mixed methodology study measuring the usage rates of Augmented Reality (AR) information overlay mapping time-on-task in informal learning environments at a local art museum. This study investigates whether AR systems provide a uniquely beneficial learning context due to AR’s native function to overlay information onto physical spaces at an art museum and the impact on participant perceived self-efficacy and overall engagement within the AR-enhanced environment. Participants also took part in an open-ended survey within the application. The quantitative data collected suggest that participants were highly engaged and utilized the application at the art museum extensively (N = 143); the qualitative results indicated that AR participants were exceedingly motivated and perceived an enhanced sense of engagement with the exhibit sites.