E-assessment in the TRADILEX project.

Abstract

TRADILEX is a project which aims at studying how Didactic Audiovisual Translation fosters language learning. It has been implemented in a virtual course through a series of Lesson Plans
which contain AVT tasks, and two language skills tests. Assessing the tasks and the language skills in a virtual and asynchronous environment was a paramount challenge. There was a need to have
all the data centralised in a concrete platform because making use of different applications was confusing for both, the teachers, and the students in the course. Therefore, two key elements were taken into account: the importance of feedback and the importance of scaffolding. That is why the team chose Moodle because it facilitates both, the task of providing students with instant feedback and the inclusion of scaffolding in the form of Didactic-Mediated Dialogue in the virtual course through HTML5 code. The resources of the virtual platform allowed us to create tasks with e-rubrics which facilitate
assessment and foster communication through specific applications. Besides, teachers have their own virtual space with video tutorials about the FAQs and fora to share their thoughts on the process of assessment. E-assessment is thus a keydimension of the TRADILEX project.

Full text

Designing
Online Assessment
Solutions that are rigorous,
trusted, flexible and scalable
Klaus-Dieter Rossade, José Janssen,
Carlton Wood & George Ubachs

Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
EADTU Special interest Group on Online Assessment
European Association of Distance Teaching Universities (EADTU) | Netherlands
George Ubachs (Chair, author)
Stefan Meuleman (Coordination, layout)
Piet Henderikx
Theodora Boskou
Open University (OUUK) | United Kingdom
Klaus-Dieter Rossade (Author)
Carlton Wood (Author)
Open Universiteit (OUNL) | Netherlands
José Janssen (Author)
FernUniversität in Hagen | Germany
Andreas Kempka
Stefan Stürmer
Open University | Israel
Michal Ben Shaul
University of Ljubljana | Slovenia
Rasa Urbas
Marko Papic
CALED | Ecuador
Mary Elizabeth Morocho Quezada
Universidade Aberta | Portugal
Diogo Casanova
Lúcia da Graça Cruz Domingues Amante
Universitat Oberta de Catalunya (UOC) | Spain
Ana Guerrero
Nadja Gmelch
Loles González
UniDistance | Switzerland
Damien Carron
Henrietta Carbonel
Tampere University of Applied Sciences (TAMK)| Finland
Kirsi Saarinen

Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
Thomas More University College | Belgium
Quinten Verdonck
Artevelde University of Applied Sciences | Belgium
Soetkin Lootens
Olaf Spittaels
Eef Vervalle
Hellenic Open University | Greece
Efthimios Zervas
Dimitris Kalles
Universidad Nacional de Educación a Distancia (UNED)| Spain
Miguel Santamaria
Ángeles Sánchez-Elvira Paniagua
José L. Aznarte
National Center for Distance Education | Czech Republic
Jan Beseda
Jyväskylä University | Finland
Pessi Lyyra
Tomi Waselius
Johannes Kepler Universität (JKU) | Austria
Thomas Fischer
Ursula Niederländer
Open University (OUC) | Cyprus
Erato Ioanna Sarri
Michalis Epiphaniou
Further contributions
Universidad Nacional de Educación a Distancia (UNED) | Spain
Elena Martín-Monje, M. Dolores Castrillo, Paz Díez-Arcón, Llanos Tobarra Abad,
Jesús González Boticario, M Ángeles Escobar Álvarez, Ana Ibáñez Moreno, Noa
Talaván and Antonio Tinedo-Rodríguez
Universidade Aberda | Portugal
L. Amante, M. C. Pinto and I. Oliveira
Open Universiteit (OUNL) | Netherlands
Ekaterina Muravyeva

Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
Universidad Técnica Particular de Loja (UTPL) | Ecuador
Luis Moncada Mora, Duval Quezada Chávez and María José Rubio Gómez
Czech Association of Distance Teaching Universities (ČADUV) | Czech Republic
Lucie Rohlíková
Published by
European Association of Distance Teaching Universities | The Netherlands
Correspondence
European Association of Distance Teaching Universities (EADTU) att George Ubachs,
Managing Director Parkweg 27, 6212 XN Maastricht, The Netherlands Tel: +31 43 311
87 12 | E-mail: secretariat@eadtu.eu www.eadtu.eu | empower.eadtu.eu
Suggested citation
Rossade, K. D., Janssen, J., Wood, C., & Ubachs, G. (2022). Designing Online
Assessment - Solutions that are Rigorous, Trusted, Flexible and Scalable. Maastricht,
The Netherlands: EADTU.
License used
This work is licensed under a Creative Commons Attribution ShareAlike 4.0
International License: https://creativecommons.org/licenses/by/4.0/ This license lets
others distribute, remix, tweak, and build upon this work (even commercially) as long
as credit is provided for the original creation. This is among the most accommodating
of CC licenses offered, and recommended for maximum dissemination and use of
licensed materials.
Disclaimer
The work of EADTU is supported by the European Commission, DG EAC, under the
Erasmus+ Programme. The European Commission supports for the production of this
publication and does not constitute an endorsement of the contents which reflects the
views only of the authors, and the Commission cannot be held responsible for any use
which may be made of the information contained therein.

Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
Contents
Introduction
7
Chapter 1: Assessment Design
13
Chapter 2: Trust, privacy & ethics
17
Chapter 3: Operational processes, technology and support
24
Final remarks
31
References
34
Annex I: Good Practices
36
1: Assessment approaches at the UAb: the present and the future
37
Diogo Casa Nova
Universidade Aberta | Portugal
2: El modelo de ‘evaluación en línea controlada’ de la Universidad Técnica Particular de Loja
41
Luis Moncada Mora, Duval Quezada Chávez and María José Rubio Gómez
Universidad Técnica Particular De Loja (UTPL)| Ecuador
3: Examples of Digital Tools and Support at Tampere University of Applied Sciences
46
Kirsi Saarinen
Tampere University of Applied Sciences (TAMK) | Finland
4: Continuous Assessment Method on Engineering Physics Courses at TAMK
48
Kirsi Saarinen
Tampere University of Applied Sciences (TAMK) | Finland
5: Online Proctoring
Ekaterina Muravyeva, José Janssen
50
Open Universiteit (OUNL) | Netherlands
6: Moodle Test Activity Pilot project
56
L. Amante; M. C. Pinto; I. Oliveira; D. Casa Nova
Universidade Aberta | Portugal
7: Data-driven learning to improve writing skills in foreign languages
59
Elena Martín-Monje, M. Dolores Castrillo
Universidad Nacional de Educación a Distancia (UNED) | Spain

Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
8: Pros and cons of online learning and assessment methods in higher education
62
Pessi Lyyra & Tomi Waselius
University of Jyväskyla (JYU) | Finland
9: Using MS TEAMS for online peer assessment: an innovative educational proposal
66
Paz Díez-Arcón, Elena Martín-Monje & M. Dolores Castrillo
Universidad Nacional de Educación a Distancia (UNED) | Spain
10: Open book assessment
69
Lucie Rohlíková
University of West Bohemia | Czech Republic
11: Agile, collaborative academic decision-making at scale and speed
72
Klaus-Dieter Rossade
The Open University (OUUK)| United Kingdom
12: Avoiding e-proctoring, ensuring trust: UNED's AvEx
75
José L. Aznarte, Ángeles Sánchez-Elvira, Miguel Santamaría, Llanos Tobarra Abad, Jesús
González Boticario
Universidad Nacional de Educación a Distancia (UNED) | Spain
13: Improving engagement and student performance through e-continuous assessment
78
Miguel Santamaría Lancho
Universidad Nacional de Educación a Distancia (UNED) | Spain
14: Continuous assessment in the subject of English II for Tourism: development of a
methodological proposal for the UNED 60/40 course
83
M Ángeles Escobar Álvarez & Ana Ibáñez Moreno
Universidad Nacional de Educación a Distancia (UNED) | Spain
15: E-assessment in the TRADILEX project
87
Noa Talaván, Antonio Tinedo-Rodríguez
Universidad Nacional de Educación a Distancia (UNED) | Spain
Annex II: Assessment and recognition of MOOC courses and MOOC pathways
91

Assessment
Design 01

Annex I: Good Practices
Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
GP15
E-assessment in the TRADILEX project
Noa Talaván, Antonio Tinedo-Rodríguez
Universidad Nacional de Educación a Distancia (UNED) | Spain
Introduction
Assessing in virtual environments is a challenge and a need for the 21st century education.
Assessment goes beyond accountability because, according to Camilloni, Celman, Litwin, & Palou de
Maté (1998), it is an essential component of a planning for it provides both, students, and teachers,
with relevant information on the process of learning and on the process of teaching. E-assessment
has the advantage of providing students with immediate feedback and it can be implemented in
different ways such as a web-based system with assessment tools (Alruwais, Wills, & Wald, 2018,
p.34).
Assessment is crucial for the TRADILEX project because after doing each activity in the course which
aims at fostering language learning through Audio-visual Translation (AVT) tasks, students receive
their feedback within a maximum span of time of seven days. Fernández Abascal et al. (2010)
highlighted the importance of extrinsic motivation, which is a type of operant conditioning, when it
comes to achieving a goal or completing a task, because you can alter the level of motivation by
provoking certain stimuli. Burgers, Eden, van Engelenburg, & Buningh, (2015) highlighted how
positive feedback is a booster for motivation.
When designing the TRADILEX project, the washback effect of the tests and the activities included on
it was something crucial because as Harris & McCan (1994) affirm, the washback effect, which is the
influence of a test in the course, plays an essential role since it is linked to motivation. According to
Harries & McCan (1994, p. 27), in the field of language learning, a good test is that test which
TRADILEX is a project which aims at studying how Didactic Audiovisual Translation fosters
language learning. It has been implemented in a virtual course through a series of Lesson Plans
which contain AVT tasks, and two language skills tests. Assessing the tasks and the language skills
in a virtual and asynchronous environment was a paramount challenge. There was a need to have
all the data centralised in a concrete platform because making use of different applications was
confusing for both, the teachers, and the students in the course. Therefore, two key elements
were taken into account: the importance of feedback and the importance of scaffolding. That is
why the team chose Moodle because it facilitates both, the task of providing students with
instant feedback and the inclusion of scaffolding in the form of Didactic-Mediated Dialogue in the
virtual course through HTML5 code. The resources of the virtual platform allowed us to create
tasks with e-rubrics which facilitate assessment and foster communication through specific
applications. Besides, teachers have their own virtual space with video tutorials about the FAQs
and fora to share their thoughts on the process of assessment. E-assessment is thus a key
dimension of the TRADILEX project.
87

Annex I: Good Practices
Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
contains authentic and real-life examples of tasks learners will have to perform in the future. That is
why the initial and the final tests for B1 and B2 used in TRADILEX were thoroughly designed by
members of the team. The names of these tests were ITIS (Initial Test of Integrated Skills) and FITIS
(Final Test of Integrated Skills) (Couto-Cantero, Sabaté-Carrové, & Gómez-Pérez, 2021). Even though
the tests were originally designed for online environments, its implementation was a challenge that
has been improved progressively according to the perceptions of both teachers and students in pre-
piloting experiences.
Furthermore, the e-assessment dimension of the TRADILEX project goes beyond testing the
language skills at the beginning and at the end of the project. García-Aretio (2014) focused on the
importance of formative assessment in the design of e-learning courses, and it impregnates the
TRADILEX project because every single task is assessed through an ad hoc interactive rubric.
Technological challenges
A project such as TRADILEX, which is fully developed in online environments, necessarily faces
technological challenges such as:
• Choosing an appropriate virtual platform.
• Didactic-Mediated Dialogue (DMD).
• Its asynchronous nature.
• Dropout rate.
Choosing an appropriate virtual platform that centralizes the data is crucial when it comes to
designing the e-assessment dimension of a project which is fully implemented online. Making use of
multiple platforms could be discouraging for both teachers and students. In this case, the research
team took the decision of making use of Moodle. The main advantages of Moodle are the following
ones:
• It is a free and open-source Learning Management System (LMS).
• It includes apps for assessment and for learning: fora, chats, blogs, collaborative glossaries,
testing tools, etc. Besides, more apps by developers may be installed to meet specific needs
of the project.
• It is based on a constructivist approach of learning in which social communication is at the
very core.
• It is constantly updated.
Following García-Aretio (2014), DMD plays an essential role when it comes to designing virtual
courses because it somehow replaces the role of the teacher by providing students with specific and
clear instructions on how to follow the course.
88

Annex I: Good Practices
Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
Figure 1. DMD in the virtual course
In the TRADILEX project, DMD has taken a prominent role and it is available in English and in Spanish.
It has been embedded in the course through HTML and there is also a PDF guide available for
students. Besides, each lesson plan has its own guidelines to facilitate the process.
The main objective of this thorough design is to avoid dropout rate. Besides, taking the
asynchronous nature of the course into account, different fora were created for specific purposes so
that students and teachers could easily communicate with each other.
Figure 2. Fora in the virtual course
Good Practices
The good practices consist of the development of a system which aims at facilitating the
communication between teachers and students through a series of e-rubrics for tasks and for tests
that allows students to know their marks and their feedback when the teacher has just marked them
as it is displayed in Figure 3.
89

Annex I: Good Practices
Designing online assessment - solutions that are rigorous, trusted, flexible and scalable
Figure 3. One of the rubrics used in TRADILEX
The inclusion of communication tools as shown in Figure 2 and the thorough design of the DMD as
displayed in Figure 3 are also good practices of the project. Students and teachers involved in
TRADILEX show their satisfaction and express their positive feedback on the process of virtualisation
and centralisation of e-assessment. In fact, the very nature of Moodle fosters ubiquity so teachers
can assess anytime anywhere, by providing them with the flexibility they need. Besides, a specific
space has been created for teachers so that they can share their thoughts on the process. This space
contains video tutorials with the FAQs to facilitate the assessment task.
References
Alruwais, N., Wills, G., & Wald, M. (2018). Advantages and challenges of using e-assessment. International
Journal of Information and Education Technology, 8(1), 34-37.
Burgers, C., Eden, A., van Engelenburg, M. D., & Buningh, S. (2015). How feedback boosts motivation and play
in a brain-training game. Computers in Human Behavior, 48, 94-103.
Camilloni, A., Celman, S., Litwin, E., & Palou de Maté, M. d. (1998). La evaluación de los aprendizajes en el
debate didáctico contemporáneo. Paidós.
Couto-Cantero, P., Sabaté-Carrové, M., & Gómez-Pérez, M. d. (2021). Preliminary design of an Initial Test of
Integrated Skills within TRADILEX: an ongoing project on the validity of audiovisual translation tools in teaching
English. Research in Education and Learning Innovation Archives, 73-88.
Fernández Abascal, E. G., García Rodríguez, B., Jiménez Sánchez, M. P., Martín Díaz, M. D., & Domínguez
Sánchez, F. J. (2010). Psicología de la motivación. Ramón Areces.
García-Aretio, L. (2014). Bases, mediaciones y futuro de la educación a distancia en la sociedad digital. Síntesis.
Harris, M., & McCan, P. (1994). Handbooks for the English Classroom: Assessment. McMillan.
90