Content uploaded by Martin Ebner
Author content
All content in this area was uploaded by Martin Ebner on Feb 22, 2022
Content may be subject to copyright.
Citation: Edelsbrunner, S.; Steiner, K.;
Schön, S.; Ebner, M.; Leitner, P.
Promoting Digital Skills for Austrian
Employees through a MOOC: Results
and Lessons Learned from Design
and Implementation. Educ. Sci. 2022,
12, 89. https://doi.org/10.3390/
educsci12020089
Academic Editor: Ching Sing Chai
Received: 30 November 2021
Accepted: 25 January 2022
Published: 27 January 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
education
sciences
Article
Promoting Digital Skills for Austrian Employees
through a MOOC: Results and Lessons Learned from
Design and Implementation
Sarah Edelsbrunner 1, Karin Steiner 2, Sandra Schön 1, Martin Ebner 1, * and Philipp Leitner 1
1Educational Technology, Graz University of Technology, 8010 Graz, Austria;
sarah.edelsbrunner@tugraz.at (S.E.); sandra.schoen@tugraz.at (S.S.); philipp.leitner@tugraz.at (P.L.)
2ABIF, 1140 Vienna, Austria; Karin.steiner@abif.at
*Correspondence: martin.ebner@tugraz.at
Abstract:
Digital skills are now essential, not only in information and communications technology
(ICT) jobs, but for employees across all sectors. The aim of this article is to detail how employees’
digital skills can be fostered through a Massive Open Online Course (MOOC), how such an offer
is used and what the effects of such a measure are. Using an approach oriented at action research
and design-based research activities, the authors describe the basics of their finding on existing
European competence frameworks for digital skills and European projects that used MOOCs, the
development and design of the MOOC, the evaluation on the basis of learning analytics insights and
a questionnaire, as well as a reflection. The MOOC was offered as Open Educational Resources (OER)
on the Austrian MOOC platform iMOOX.at from March to April 2021, with 2083 participants, of
whom 381 fully completed the course (at end of June 2021) and 489 filled out the final questionnaire.
Keywords: digital skills; employees; lifelong learning; MOOC; open educational resources (OER)
1. Introduction: The Need for Austrian Employees to Acquire Digital Skills
Information and communications technology (ICT) skills are required in more and
more workplaces, even in jobs not traditionally associated with ICT skills or investment.
Digital competences, or digital skills, have become a new kind of basic competence for the
21st century, much like writing, reading and math [
1
]. The skills and jobs survey (ESJS)
from the European Centre for the Development of Vocational Training (CEDEFOP), for
example, has shown that fundamental ICT skills are required for seven out of ten adult
employees in the EU. At the same time, according to the survey, “about one in three of
these employees is at risk of digital skill gaps” [
2
], p. 52. A survey on behalf of the Austrian
Labor Market Service (AMS) confirms these findings for Austria and adds that digitization
of the workplace will also create new job profiles that do not yet exist, or for which there
is not yet any specific training. Employees must therefore learn general skills that will
allow them to flexibly address future challenges in the workplace and embrace future
developments quickly [
3
]. However, current offers for employees are often only comprised
of “training sessions” in using specific programs or systems, and neglect more general
digital competence that can be transferred to other settings [
1
]. In this context, the project
“Digital Skills for 500 private employees in Austria”, a cooperation of ABIF (an independent
social science research and consulting institute with a focus on applied research), GPA-
djp (The Union of Private Sector Employees, Printing, Journalism, and Paper) and Graz
University of Technology (TU Graz) aimed to provide training in basic, general digital
skills for Austrian employees in the private sector using an eight-week Massive Open
Online Course (MOOC). The purpose of this contribution and research is to explore and
systematically describe how a MOOC for digital skills for employees can be planned and
Educ. Sci. 2022,12, 89. https://doi.org/10.3390/educsci12020089 https://www.mdpi.com/journal/education
Educ. Sci. 2022,12, 89 2 of 17
implemented to promote digital skills among employees in the Austrian private sector and
beyond, how it may be used and what its effects on participants are.
This contribution will answer the following two research questions:
1.
How can a MOOC be planned and implemented to support the development of digital
skills among employees in the Austrian private sector?
2. Which of our experiences should similar future projects consider?
To answer the question, we used a research and action approach that intertwines
research activities with the development and implementation of an activity—in our case, a
MOOC. This is related to an action research design, where the people directly responsible
for a pedagogical intervention are—at least partly—involved in the research process [
4
].
In general, we followed an “action–reflection cycle”, a recursive process of planning,
implementation (acting), observation and reflection [
5
]. The approach is also influenced by
the discussion of design-based research [
6
], in which a sound, research-based development
of an educational intervention is accompanied by an evaluation and then critically reviewed.
Figure 1details the two interconnected activities of research and MOOC development
and their phases from April 2020 to October 2021.
Educ. Sci. 2022, 12, 89 2 of 17
be planned and implemented to promote digital skills among employees in the Austrian
private sector and beyond, how it may be used and what its effects on participants are.
This contribution will answer the following two research questions:
1. How can a MOOC be planned and implemented to support the development of dig-
ital skills among employees in the Austrian private sector?
2. Which of our experiences should similar future projects consider?
To answer the question, we used a research and action approach that intertwines
research activities with the development and implementation of an activity—in our case,
a MOOC. This is related to an action research design, where the people directly responsi-
ble for a pedagogical intervention are—at least partly—involved in the research process
[4]. In general, we followed an “action–reflection cycle”, a recursive process of planning,
implementation (acting), observation and reflection [5]. The approach is also influenced
by the discussion of design-based research [6], in which a sound, research-based develop-
ment of an educational intervention is accompanied by an evaluation and then critically
reviewed.
Figure 1 details the two interconnected activities of research and MOOC develop-
ment and their phases from April 2020 to October 2021.
Figure 1. Overview of MOOC development and research activities. Note: The representation is not
to scale.
From a research perspective, we first searched the scientific literature and project re-
ports for existing implementations and experiences with digital skills development in Eu-
rope, focusing on implementations with MOOCs. This is not a systematic, worldwide lit-
erature search, which we could not find, but a targeted analysis of relevant EU publica-
tions.
For the description of the planning and development phase of the MOOC, we used
the notes and records of the project team for this article. These were prepared in such a
way that the most important planning and development steps, as well as the design of the
MOOC itself, are as comprehensible as possible. For the evaluation of the MOOC itself,
we collected data and made different evaluations using a learning analytics system (“Cre-
ator’s Dashboard”) that was developed at TU Graz and is directly integrated into the
MOOC platform iMooX.at. It collects, analyzes and visualizes learners’ data, for example
about enrolments, activity and interactions. In addition, protocols were written during the
live sessions of the MOOC. A third form of data collection on the MOOC was done
through a written online survey at the end of the course of all participants who wanted to
download a certificate of participation. The 32-item questionnaire consists of general ques-
tions on the background of the participants (single-choice) and a second part where par-
ticipants are asked to rate their MOOC experience on a five-item Likert scale. Several ques-
tions offer an additional free-text field for explanations or comments. The original ques-
tionnaire is publicly available (see Supplementary Materials). The answers to the ques-
tionnaires are analyzed using descriptive methods.
Finally, the procedure also includes a (critical) reflection on the implementation. For
this purpose, the research team reviewed and discussed the results of the evaluation. In
this way, a collection of possible adjustments and changes was made for a renewed im-
plementation of such a project.
Figure 1.
Overview of MOOC development and research activities. Note: The representation is not
to scale.
From a research perspective, we first searched the scientific literature and project reports
for existing implementations and experiences with digital skills development in Europe,
focusing on implementations with MOOCs. This is not a systematic, worldwide literature
search, which we could not find, but a targeted analysis of relevant EU publications.
For the description of the planning and development phase of the MOOC, we used
the notes and records of the project team for this article. These were prepared in such a
way that the most important planning and development steps, as well as the design of the
MOOC itself, are as comprehensible as possible. For the evaluation of the MOOC itself, we
collected data and made different evaluations using a learning analytics system (“Creator’s
Dashboard”) that was developed at TU Graz and is directly integrated into the MOOC
platform iMooX.at. It collects, analyzes and visualizes learners’ data, for example about
enrolments, activity and interactions. In addition, protocols were written during the live
sessions of the MOOC. A third form of data collection on the MOOC was done through a
written online survey at the end of the course of all participants who wanted to download
a certificate of participation. The 32-item questionnaire consists of general questions on the
background of the participants (single-choice) and a second part where participants are
asked to rate their MOOC experience on a five-item Likert scale. Several questions offer
an additional free-text field for explanations or comments. The original questionnaire is
publicly available (see Supplementary Materials). The answers to the questionnaires are
analyzed using descriptive methods.
Finally, the procedure also includes a (critical) reflection on the implementation. For
this purpose, the research team reviewed and discussed the results of the evaluation.
In this way, a collection of possible adjustments and changes was made for a renewed
implementation of such a project.
In the following section, we first describe the related work on MOOCs in adult ed-
ucation, existing concepts and experiences for the promotion of digital skills throughout
Europe; then, following the steps of our action research design, we describe the design and
Educ. Sci. 2022,12, 89 3 of 17
planning phase of the MOOC, its implementation, the results of the evaluation and, finally,
a reflection on the project.
2. Related Work
Our project is not the first of its kind. There have been several practical and scientific
research projects on MOOCs in adult education and, more specifically, for furthering digital
skills. In the following sections, we describe the related work.
2.1. MOOCs as Further Education for Employees
Massive Open Online Courses (MOOCs) are online courses for many people (“mas-
sive”), i.e., more than 150 participants [
7
]. The term “open” is used because the first
MOOCs were offered by university members, but were accessible online without formal
restrictions, such as a university entrance qualification, and usually free of charge [
7
,
8
].
MOOCs were never meant for students only; rather, they were always intended for other
target groups, such as employees, seniors, employees or “lifelong learners” in general.
The model of so-called “xMOOCs”, which are presentation-oriented online courses for
many individuals [
9
], has become widely established. In this type of MOOC, learning
videos and material for self-regulated learning are offered in course form and exchange
between learners is supported by forums. Unsurprisingly, this learning setting has been
of interest for employees from the beginning. In German-speaking Europe, among the
first MOOCs for employees were, for example, a MOOC about open educational resources
(COER13.de) [
10
], the vhsMOOC 2013 for adult educators [
11
] or the Magenta MOOC for
employees of the Deutsche Telekom company [12]. Schulmeister [13] anticipated MOOCs
as a method for professional development. Nevertheless, a survey in Germany in 2018
(n= 1003
) showed that only very few individuals (10%) know what a MOOC is, and only
1% have already used them for private or professional purposes [
14
]. In the past few years,
more research was carried out in the field of MOOCs, for example on various teaching
scenarios [
15
]. To more effectively address the needs of adults, we developed a design
approach of “inverse blended learning” that tries to enrich the online course setting with
“analog” materials, such as a printed handbook or learner meet-ups [
16
]. To sum up, we
have determined that MOOCs are a helpful measure to reach and support the learning of
adults and employees, but it is far from being well established.
2.2. Concepts of Digitals Skills for European Citizens and Current Approaches for Their Development
In Europe, the needs of citizens with regards to digital skills have been addressed for
several years now. In order to define the specific competences that are needed, competence
frameworks have been developed to define such learning goals as a basis for interventions
and training. The current developments concerning a digital competence framework, in-
cluding the Austrian adaptation, and existing approaches to foster them amongst European
citizens in member states are presented in the following section.
2.2.1. Existing Competence Frameworks (DigComp 2.1 and DigComp AT)
The Digital Competence Framework for Citizens (for short, DigComp) was first de-
veloped and published in 2013 by the European Commission, and is acknowledged as the
reference tool for a common understanding of digital competence for citizens. It provides
an overall, complete and shared understanding for multiple stakeholders of what digital
competences are [17].
There have been several follow-up publications to the original framework [
17
], trans-
lating the DigComp for use in the workplace and in the labor market, such as ‘DigComp
into Action’ [
18
], a user guide for anyone seeking to promote digital competences in the
workplace; ‘Developing digital competence for employability’ [
19
], a report of a consulta-
tion workshop with stakeholders; and the ‘DigComp at Work Implementation Guide’ [
20
],
which offers specific guidelines for the development of training offers in digital competences.
Educ. Sci. 2022,12, 89 4 of 17
These follow-up publications encourage the use of the DigComp framework for the
development and strategic planning of digital competence initiatives, both at European
and at the member state level [
19
]. Training and assessment based on DigComp can be
shared and expanded more easily to other EU countries thanks to a shared framework and
vocabulary [18].
The “DigComp 2.1 Framework” [17] consists of five parts, which are:
•
Competence areas identified to be part of digital competence; 5 areas articulated into a
total of 21 competences;
•Competence descriptors and titles that are pertinent to each area in general terms,
•
Proficiency levels for each competence, ranging from “foundation” to “highly specialized”;
•Knowledge, skills and attitudes applicable to each competence;
•Examples of DigComp use.
The ‘DigComp at Work Implementation Guide’ [
20
] suggests using DigComp in
the respective language version and reusing existing resources if they are accessible and
have a similar purpose. Such a transfer was done for Austria: the Austrian national
framework, DigComp 2.2 AT [
21
], is, in large part, a translation of the European DigComp
2.1 framework into German. However, some parts have been adapted or added to. The
most important addition is an additional competence area numbered “zero”. Competence
area zero describes first access to the field of digital competence and the requirements for
access and the start of participation in the field of digital competence. “Zero” was chosen
so that the other competence areas would retain the same numbers and therefore would
be directly comparable with the European counterpart. Further additions to the Austrian
version of DigComp 2.2 AT are “Buying and selling” (2.4) and “Protecting yourself from
fraud and infringement of consumer rights” (4.4). Competence area 5, called “Problem
solving” in the European framework, was expanded to “Problem solving and further
education” to emphasize the importance of lifelong learning [21].
2.2.2. Supporting the Development of Digital Skills through MOOCs: Existing Experiences
To locate the Digital Skills MOOC within a wider European perspective, an overview
of DigComp 2.1-based MOOCs for employees from other member states is given in this
section. The ‘DigComp into Action’ guide [
18
] already provides an extensive overview of
projects based on DigComp 2.1 for various target groups and in various settings, so the
following examples in Table 1are extracted from this report. The examples were chosen
based on the criteria that the target group of this offer is (among others) adult employees
and a MOOC is used. Table 1gives an overview of the member states’ approaches for
the development of digital skills of adult employees using MOOCs and shows whether
these MOOCs are available as OER, are complemented by job profile descriptions and their
respective digital competences and whether the offers are provided as a blended MOOC
with additional face-to-face units.
Table 1.
Overview of other member states’ approaches for the furthering of digital skills of adult
employees. Source: own selection and visualization based on [
18
]. An “X” indicates that this criterion
is fulfilled, a “-“ means it is not.
Project Name Main Target Group OER Digital Competence
Profile for Employees
Supplemented by
Face-to-Face Offer
MU.SA Museum professionals X X
(for employees) X
INTEF Teachers in Spain
Open access (but no open licenses)
X
(for teachers) -
Elene4Work Young adults entering
the labor market
Some of the recommended
MOOCs are OER, others not - -
Educ. Sci. 2022,12, 89 5 of 17
MU.SA is a training project for museum professionals, developed in Italy, Portugal and
Greece. At first, the job roles for different museum professionals and their necessary digital
skills were defined. The first stage of the training consists of a general non-job-specific
training phase using digital OER in the form of a MOOC titled “Essential digital skills for
museum professionals”, focusing on transversal digital skills in 22 modules. The materials
offered consisted of interactive images and forums for discussion and socializing [
22
].
The second stage of the training consists of a specialization training course in a blended
learning format, including self-study phases and face-to-face lectures. The third stage
involves workplace learning in museums and cultural organizations affiliated with the
project. An additional goal of MU.SA was to establish communities of practice to ensure
that the results are carried on beyond the end of the project [
18
]. An analysis showed
that the MOOC also attracted participants from other countries than the project members,
mainly other EU member states [22].
In Spain, the INTEF MOOC was developed to train teachers in one or more competence
areas. After offering three pilot MOOCs in 2014, the project evolved into a catalog of “Nano
Open Online Learning Experiences”. The contents are based on the 2017 Spanish Common
Digital Competence Framework for Teachers, which is an adaptation of DigComp to the
teaching profession. The MOOCs take between 1 and 20 h of work to finish. If a teacher
completes several courses, they can receive a digital micro-credential.
One program featuring MOOCs, but not with adult professionals as a target group,
is eLene4work. It was an Erasmus+ project from 2015–2017. The program is directed
towards young adults entering the labor market. Part of the project includes MOOCs for
enhancing participants’ digital and soft skills. The participants have the option to take a
self-assessment test to find out which soft skills they might want to develop further. They
can then select and participate in some of over 200 MOOCs selected and classified by soft
skill. The participants are also given advice on how to make the most of their MOOC
experience, such as how to develop soft skills by using the learning activities embedded in
social and connectivist-type MOOCs [
18
]. They are encouraged to keep a record of their
learning in a personal journal in case they are part of the eLene4work field study.
There are other projects geared towards employees’ digital competencies described in
the publication, but numerous tools only focus on a self-assessment of one’s digital compe-
tences, often with subsequent links to suitable training offers. In Austria, the Fit4Internet
project (part of the wider Austrian digitization strategy for all citizens) provides a self-
assessment tool for digital competence based on DigComp 2.2 AT (Fit4Internet). It aims
to assess the digital competence of all Austrian citizens, comprising, but not limited to,
private sector employees. Fit4Internet also provides an online catalog of training offers
(MOOCs, but also face-to-face training) categorized according to the six competence areas
and eight competence levels defined by the Digital Competence Framework for Austria.
However, the training offers listed are not all openly licensed and free of charge. Some of
the offers are online courses and others are face-to-face courses.
With this is mind, the DigiSkills project coordinators decided against developing and
providing another assessment tool and instead opted for a MOOC.
3. Results
3.1. Planning Phase: Design Considerations and Marketing
3.1.1. Motives
A MOOC was chosen for the DigiSkills training in order to easily reach the large target
group of Austrian private sector employees. MOOC participants can further train their
digital skills flexibly and independently of time and place, which is suitable for people
working part or full time. It is assumed that most employees have at least a smartphone,
tablet or computer, so they will be able to follow a MOOC. There are no financial obstacles
to participation, as the MOOC is free of charge.
Educ. Sci. 2022,12, 89 6 of 17
3.1.2. Target Group
The main target group of the MOOC are Austrian private sector employees, but since
the MOOC is offered as an OER on iMooX.at, another possible target group are German-
speaking adults beyond the private sector interested in furthering their digital skills, which
is a potentially quite large and heterogeneous group. Since the MOOC is potentially open
to everyone interested in furthering their digital competences, the DigiSkills project did not
include extensive descriptions of job profiles and their respective digital skill requirements,
as was included in other member states’ projects. When planning a MOOC for private
sector employees, it must be considered that they are not typical MOOC participants.
Studies have shown that participants in MOOCs still tend to be individuals with prior
experience in higher education and do not automatically tend to attract people who are
distanced from (online) education [23].
3.1.3. Goals of the MOOC
The goal of the MOOC is, first and foremost, to promote digital skills among employees
in the Austrian private sector by allowing them to take part in the freely available online
course, especially amongst employees who do not usually work using a computer and who
display no or few digital skills. On the level of labor market stakeholders, the project is also
meant to empower the works council as an interest group by allowing them to contribute
to the creation and the content of this program. Lastly, the goal is to future-proof this self-
study MOOC so it can be used by employees free of charge on the iMooX platform, even
after the end of the guided phase, as a way of furthering their own education independent
of time and place. It is also planned to repeat the MOOC every two years as a guided
MOOC by the project partners.
A quantitative goal of the project is to gather at least 500 participants who would
finish the MOOC and obtain a participant certificate [
1
]. To reach this quantitative goal,
it must be considered that the completion rate for MOOCs is often much lower than the
number of people registered. Some participants may only register to try out online learning.
For others, their learning outcomes can be met without completing a whole MOOC and
obtaining a certificate, so they could drop out without certificates as well.
3.1.4. The MOOC Platform
In Austria, there is a MOOC platform that was chosen for the implementation. The
Austrian national MOOC platform, iMooX.at, is hosted by the University of Technology
of Graz (TU Graz) and is committed to offering open educational resources (OER). All
iMooX.at courses are offered free of charge and only a valid e-mail address is required
to sign in [
24
]. Courses can be followed at everyone’s own pace and via any device
with Internet access, which is especially beneficial to the target group of the DigiSkills
project—employees working various shifts and not necessarily living close to the nearest
training center.
3.1.5. Co-Design with Stakeholders
During the planning phase of the MOOC, six workshops with members of the works
council were organized to find out which specific challenges and changes arise due to
digitalization at the workplace and which specific content would be necessary in a training
offer for employees. The invitation was sent to 13,000 members of the works council in
different economic sectors in Austria [
1
]. However, the size of the workshops had to be
reduced due to the measures against COVID-19. These workshops were a first step to
establishing collaboration and continuous dialogue between the different labor market
stakeholders collaborating on this MOOC.
Three workshops with works councils were held online via Zoom at the beginning of
2020, during the first lockdown in Austria. The first workshop focused on the competence
requirement in modules 0 and 1 of the DigComp model. The second workshop was
dedicated to modules 2 and 3, and the third workshop to modules 4 and 5. In September
Educ. Sci. 2022,12, 89 7 of 17
2020, a fourth workshop was held face-to-face, where the competence requirements in
all modules were the topic (see Figure 2). The participants in the workshops were works
councilors recruited by the GPA. Between 15 and 20 participants took part in each workshop.
The first three workshops took place without the specific MOOC instructional design
background, and it proved difficult to take the results into account, as the competences
required by the representatives of the different sectors were too diverse, yet specific.
Educ. Sci. 2022, 12, 89 7 of 17
employees working various shifts and not necessarily living close to the nearest training
center.
3.1.5. Co-Design with Stakeholders
During the planning phase of the MOOC, six workshops with members of the works
council were organized to find out which specific challenges and changes arise due to
digitalization at the workplace and which specific content would be necessary in a train-
ing offer for employees. The invitation was sent to 13,000 members of the works council
in different economic sectors in Austria [1]. However, the size of the workshops had to be
reduced due to the measures against COVID-19. These workshops were a first step to
establishing collaboration and continuous dialogue between the different labor market
stakeholders collaborating on this MOOC.
Three workshops with works councils were held online via Zoom at the beginning
of 2020, during the first lockdown in Austria. The first workshop focused on the compe-
tence requirement in modules 0 and 1 of the DigComp model. The second workshop was
dedicated to modules 2 and 3, and the third workshop to modules 4 and 5. In September
2020, a fourth workshop was held face-to-face, where the competence requirements in all
modules were the topic (see Figure 2). The participants in the workshops were works
councilors recruited by the GPA. Between 15 and 20 participants took part in each work-
shop. The first three workshops took place without the specific MOOC instructional de-
sign background, and it proved difficult to take the results into account, as the compe-
tences required by the representatives of the different sectors were too diverse, yet spe-
cific.
Figure 2. Stakeholder workshop. The setting, commented module plan and feedback for selected
content (from left to right). Source: ABIF.
Ultimately, topics had to be found that would appeal to all adult professionals in
Austria. Therefore, in the summer of 2020, a rough MOOC concept and the basic content
of all modules were defined in an ABIF-internal workshop. The participants of the last
workshop were then confronted with these ideas, where they could rate the topics, or add
new topics or examples from everyday work life.
3.1.6. MOOC Design and Overview
This MOOC was designed to run as a “Blended MOOC” and later as a self-study
MOOC without online live sessions. Figure 3 gives an impression of the online course.
The online phase of the MOOC is divided into eight units, with a new unit being made
available each week:
• 101 of the Internet;
• Safe surfing and downloading;
• Digital collaboration;
• Digital tax returns, signatures, and official services;
• Digital job applications: MS Word and Europass (the European online CV);
• Social Media: posts, privacy and copyright;
• How to spot fakes and false information online;
Figure 2.
Stakeholder workshop. The setting, commented module plan and feedback for selected
content (from left to right). Source: ABIF.
Ultimately, topics had to be found that would appeal to all adult professionals in
Austria. Therefore, in the summer of 2020, a rough MOOC concept and the basic content
of all modules were defined in an ABIF-internal workshop. The participants of the last
workshop were then confronted with these ideas, where they could rate the topics, or add
new topics or examples from everyday work life.
3.1.6. MOOC Design and Overview
This MOOC was designed to run as a “Blended MOOC” and later as a self-study
MOOC without online live sessions. Figure 3gives an impression of the online course.
The online phase of the MOOC is divided into eight units, with a new unit being made
available each week:
•101 of the Internet;
•Safe surfing and downloading;
•Digital collaboration;
•Digital tax returns, signatures, and official services;
•Digital job applications: MS Word and Europass (the European online CV);
•Social Media: posts, privacy and copyright;
•How to spot fakes and false information online;
•Problem solving and continuing to learn.
Each unit consists of one or more video(s) and documents for further reading.
The forum was used for reflection and to share one’s own experiences. The participants
were also able to ask questions about the various MOOC topics and modules. The videos
gave an overview of the topics and provided guidance, for example, on how to get a
digital signature or how to look for a job online. In this sense, the entire MOOC was very
low-threshold, which was also its goal. The participants were also able to choose which
modules they wanted to complete and were given badges for each completed module, even
if they did not want to complete the whole MOOC.
As with other MOOCs on the platform, there are some support options to guide the
learners: participants can mark completed activities to record their own progress within a
unit. Additionally, at the end of each unit, there is a single- or multiple-choice quiz. The
participants have five attempts to take each quiz and need to answer at least 75% of the
questions correctly to pass the quiz.
Educ. Sci. 2022,12, 89 8 of 17
Educ. Sci. 2022, 12, 89 8 of 17
• Problem solving and continuing to learn.
Each unit consists of one or more video(s) and documents for further reading.
The forum was used for reflection and to share one’s own experiences. The partici-
pants were also able to ask questions about the various MOOC topics and modules. The
videos gave an overview of the topics and provided guidance, for example, on how to get
a digital signature or how to look for a job online. In this sense, the entire MOOC was very
low-threshold, which was also its goal. The participants were also able to choose which
modules they wanted to complete and were given badges for each completed module,
even if they did not want to complete the whole MOOC.
As with other MOOCs on the platform, there are some support options to guide the
learners: participants can mark completed activities to record their own progress within a
unit. Additionally, at the end of each unit, there is a single- or multiple-choice quiz. The
participants have five attempts to take each quiz and need to answer at least 75% of the
questions correctly to pass the quiz.
Figure 3. Screenshot of lesson 3 of DigiSkills showing the video, documents and the quiz. Source:
screenshot of the course. URL: https://imoox.at/mooc/course/diska (accessed on 30 July 2021).
Furthermore, the participants were invited to engage with the MOOC topics in an
offline setting in the form of a specially designed workbook that could be obtained from
stakeholders or the organizers before the start of the MOOC. The printed booklet is de-
signed to help participants who do not yet possess a great deal of digital skills. It contains
the MOOC content as texts and exercises (gap fill, reflection questions) with solutions for
self-assessment and to repeat the content. The idea for the workbook was based on the
booklet accompanying the MOOC “Gratis Online Lernen” and the idea of the didactical
concept of “inverse blended learning” [15].
The self-study phase is complemented by three non-compulsory 1.5 h live sessions
via Zoom at the beginning, in the middle and at the end of the MOOC as in a “Blended
MOOC” [16]. These live events are supposed to foster interest and a connection between
the participants and allow them to ask questions and give feedback.
For each completed quiz, the participants receive a digital badge. If the participants
have completed all of the quizzes and filled out the compulsory questionnaire at the end,
they can download a PDF certificate attesting “Basic Digital Skills” [1]. For professionals,
Figure 3.
Screenshot of lesson 3 of DigiSkills showing the video, documents and the quiz. Source:
screenshot of the course. URL: https://imoox.at/mooc/course/diska (accessed on 30 July 2021).
Furthermore, the participants were invited to engage with the MOOC topics in an
offline setting in the form of a specially designed workbook that could be obtained from
stakeholders or the organizers before the start of the MOOC. The printed booklet is designed
to help participants who do not yet possess a great deal of digital skills. It contains the
MOOC content as texts and exercises (gap fill, reflection questions) with solutions for
self-assessment and to repeat the content. The idea for the workbook was based on the
booklet accompanying the MOOC “Gratis Online Lernen” and the idea of the didactical
concept of “inverse blended learning” [15].
The self-study phase is complemented by three non-compulsory 1.5 h live sessions
via Zoom at the beginning, in the middle and at the end of the MOOC as in a “Blended
MOOC” [
16
]. These live events are supposed to foster interest and a connection between
the participants and allow them to ask questions and give feedback.
For each completed quiz, the participants receive a digital badge. If the participants
have completed all of the quizzes and filled out the compulsory questionnaire at the end,
they can download a PDF certificate attesting “Basic Digital Skills” [
1
]. For professionals,
such a certificate may be important to use as evidence of further education to provide to
their employers or during job applications [25].
3.1.7. Marketing Issues and Activities
Apart from stakeholders learning about the MOOC during the workshops, the course
was promoted via mailing lists to members of GPA-djp and directly to members of the
works council. ABIF promoted the MOOC in a mailing list with 3000 recipients and
AMS (the Austrian Labor Market Service) promoted it in their research network with
8000 members. In addition, iMooX.at promoted the MOOC on its social media channels.
The above-mentioned workbooks were also handed out to potential participants at the
stakeholder institutions as part of the marketing activities.
Educ. Sci. 2022,12, 89 9 of 17
3.2. Implementation Phase
Between 1 March and 25 April 2021, the MOOC was offered on iMooX.at as a blended
MOOC [
16
], which means that the asynchronous online course was enriched with face-to-
face activities in regular intervals.
As of May 2021, after the end of the guided MOOC phase, the online course remains
open and accessible for everyone free of charge and as an OER, but as a self-study MOOC
without live sessions. The data gathered via the creator’s dashboard as well as the ques-
tionnaire results presented below represent all MOOC activities from 1 March 2021 to
30 June
2021—in other words, the whole guided phase and the beginning of the following
self-study phase.
3.2.1. MOOC Participants and Activities
According to the data gathered via the creator’s dashboard, during this time frame till
end of June 2021:
•2083 participants signed up for the DigiSkills MOOC on iMooX.at;
•4765 badges were issued (one is available per unit);
•381 participants fully completed the MOOC;
•369 certificates were issued.
Looking at an activity overview provided by the Creator’s Dashboard in Figure 4, most
participants enrolled onto the course at the beginning of the course, and those enrolments
then slowed down with a number of spikes in later weeks. The first interaction graph
shows that most people who enrolled onto the course also interacted, i.e., accessed activities
in the course, directly at the start of the course or in the first few weeks after the course
start date.
Educ. Sci. 2022, 12, 89 9 of 17
such a certificate may be important to use as evidence of further education to provide to
their employers or during job applications [25].
3.1.7. Marketing Issues and Activities
Apart from stakeholders learning about the MOOC during the workshops, the course
was promoted via mailing lists to members of GPA-djp and directly to members of the
works council. ABIF promoted the MOOC in a mailing list with 3000 recipients and AMS
(the Austrian Labor Market Service) promoted it in their research network with 8000
members. In addition, iMooX.at promoted the MOOC on its social media channels. The
above-mentioned workbooks were also handed out to potential participants at the stake-
holder institutions as part of the marketing activities.
3.2. Implementation Phase
Between 1 March and 25 April 2021, the MOOC was offered on iMooX.at as a blended
MOOC [16], which means that the asynchronous online course was enriched with face-to-
face activities in regular intervals.
As of May 2021, after the end of the guided MOOC phase, the online course remains
open and accessible for everyone free of charge and as an OER, but as a self-study MOOC
without live sessions. The data gathered via the creator’s dashboard as well as the ques-
tionnaire results presented below represent all MOOC activities from 1 March 2021 to 30
June 2021—in other words, the whole guided phase and the beginning of the following
self-study phase.
3.2.1. MOOC Participants and Activities
According to the data gathered via the creator’s dashboard, during this time frame
till end of June 2021:
• 2083 participants signed up for the DigiSkills MOOC on iMooX.at;
• 4765 badges were issued (one is available per unit);
• 381 participants fully completed the MOOC;
• 369 certificates were issued.
Looking at an activity overview provided by the Creator’s Dashboard in Figure 4,
most participants enrolled onto the course at the beginning of the course, and those enrol-
ments then slowed down with a number of spikes in later weeks. The first interaction
graph shows that most people who enrolled onto the course also interacted, i.e., accessed
activities in the course, directly at the start of the course or in the first few weeks after the
course start date.
Figure 4. Overview of enrolments, first interactions with the course, completions and visits over
time. Source: TU Graz, iMooX creator’s dashboard. Comment: the diagram does not display enrol-
ments before the course start date.
Figure 4.
Overview of enrolments, first interactions with the course, completions and visits over time.
Source: TU Graz, iMooX creator’s dashboard. Comment: the diagram does not display enrolments
before the course start date.
Most participants completed the course towards the end of April 2021, which coincided
with the end of the guided MOOC phase. There were no graduations prior to this date since
the units were released on a weekly basis and it is necessary to complete all of the quizzes
to complete the MOOC. The visits graph provides information about when people accessed
the course. Most visits to the course were taken during the guided MOOC phase, with
spikes each week that coincide with the weekdays when the new weekly unit
was released
.
3.2.2. Quiz Activities
The quiz of unit 1 was completed by 769 participants by 30 June. The quiz of unit
8 was only completed by 416 participants. As Figure 5shows, the number of completed
quizzes slightly decreased each week, which might hint at participants dropping out or
completing the course slower than others.
Educ. Sci. 2022,12, 89 10 of 17
Educ. Sci. 2022, 12, 89 10 of 17
Most participants completed the course towards the end of April 2021, which coin-
cided with the end of the guided MOOC phase. There were no graduations prior to this
date since the units were released on a weekly basis and it is necessary to complete all of
the quizzes to complete the MOOC. The visits graph provides information about when
people accessed the course. Most visits to the course were taken during the guided MOOC
phase, with spikes each week that coincide with the weekdays when the new weekly unit was
released.
3.2.2. Quiz Activities
The quiz of unit 1 was completed by 769 participants by 30 June. The quiz of unit 8
was only completed by 416 participants. As Figure 5 shows, the number of completed
quizzes slightly decreased each week, which might hint at participants dropping out or
completing the course slower than others.
Figure 5. Number of completed quizzes by unit. Source: TU Graz, own visualization based on data
from iMooX Creator’s Dashboard (n = 2083 registered participants at the end of June 2021).
3.2.3. Time Spent in the MOOC
The Creator’s Dashboard also gives insight into where participants spend the most
time in a MOOC. It can be seen from Figure 6 that the participants spent most of their time
on “assignments”, i.e., quizzes and activities. A total of 21% of the time was spent watch-
ing the educational videos provided. “Navigation” made up 28% of the time spent in the
MOOC. This category comprises all activity within the MOOC that cannot be narrowed
down to specific exercises, such as reading texts and descriptions provided in the MOOC.
Figure 6. Time division: percentage of time, that learners spent on different kinds of activities.
Source: TU Graz, Creator’s Dashboard, 30 June 2021.
Figure 5.
Number of completed quizzes by unit. Source: TU Graz, own visualization based on data
from iMooX Creator’s Dashboard (n= 2083 registered participants at the end of June 2021).
3.2.3. Time Spent in the MOOC
The Creator’s Dashboard also gives insight into where participants spend the most
time in a MOOC. It can be seen from Figure 6that the participants spent most of their
time on “assignments”, i.e., quizzes and activities. A total of 21% of the time was spent
watching the educational videos provided. “Navigation” made up 28% of the time spent in
the MOOC. This category comprises all activity within the MOOC that cannot be narrowed
down to specific exercises, such as reading texts and descriptions provided in the MOOC.
Educ. Sci. 2022, 12, 89 10 of 17
Most participants completed the course towards the end of April 2021, which coin-
cided with the end of the guided MOOC phase. There were no graduations prior to this
date since the units were released on a weekly basis and it is necessary to complete all of
the quizzes to complete the MOOC. The visits graph provides information about when
people accessed the course. Most visits to the course were taken during the guided MOOC
phase, with spikes each week that coincide with the weekdays when the new weekly unit was
released.
3.2.2. Quiz Activities
The quiz of unit 1 was completed by 769 participants by 30 June. The quiz of unit 8
was only completed by 416 participants. As Figure 5 shows, the number of completed
quizzes slightly decreased each week, which might hint at participants dropping out or
completing the course slower than others.
Figure 5. Number of completed quizzes by unit. Source: TU Graz, own visualization based on data
from iMooX Creator’s Dashboard (n = 2083 registered participants at the end of June 2021).
3.2.3. Time Spent in the MOOC
The Creator’s Dashboard also gives insight into where participants spend the most
time in a MOOC. It can be seen from Figure 6 that the participants spent most of their time
on “assignments”, i.e., quizzes and activities. A total of 21% of the time was spent watch-
ing the educational videos provided. “Navigation” made up 28% of the time spent in the
MOOC. This category comprises all activity within the MOOC that cannot be narrowed
down to specific exercises, such as reading texts and descriptions provided in the MOOC.
Figure 6. Time division: percentage of time, that learners spent on different kinds of activities.
Source: TU Graz, Creator’s Dashboard, 30 June 2021.
Figure 6.
Time division: percentage of time, that learners spent on different kinds of activities. Source:
TU Graz, Creator’s Dashboard, 30 June 2021.
3.3. Evaluation
3.3.1. Questionnaire and Participants
A total of 489 participants filled out the questionnaire, which is 23% of all registered
participants (n= 2083). This is higher than the number of participants who finished the
course (n= 381).
The participants in the survey were 29% male and 71% female. The largest share of
participants was in Austria (76%), however there was also a significant share of participants
from Germany (20%) and another 2% from Switzerland. With regards to the educational
background of the participants as shown in Figure 7, approximately one third (32%) held
Educ. Sci. 2022,12, 89 11 of 17
a master’s degree or equivalent; 23% completed A-levels or equivalent; and another 21%
completed a vocational school or apprenticeship. Finally, 13% of participants completed
elementary school or middle school education and 2% had not completed any schooling
(so far).
Educ. Sci. 2022, 12, 89 11 of 17
3.3. Evaluation
3.3.1. Questionnaire and Participants
A total of 489 participants filled out the questionnaire, which is 23% of all registered
participants (n = 2083). This is higher than the number of participants who finished the
course (n = 381).
The participants in the survey were 29% male and 71% female. The largest share of
participants was in Austria (76%), however there was also a significant share of partici-
pants from Germany (20%) and another 2% from Switzerland. With regards to the educa-
tional background of the participants as shown in Figure 7, approximately one third (32%)
held a master’s degree or equivalent; 23% completed A-levels or equivalent; and another
21% completed a vocational school or apprenticeship. Finally, 13% of participants com-
pleted elementary school or middle school education and 2% had not completed any
schooling (so far).
Figure 7. Educational background of questionnaire participants, by percentage (n = 489).
Most participants worked full-time (185) or part-time (134) at the time of the survey.
The third largest group consisted of people currently looking for work (98). Thirty-three
participants were in training, 32 were self-employed and 32 were students. Twenty-one
participants were marginally employed, 15 were still in school and 18 answered that they
were retired. Five participants were on maternal/paternal leave and three were on disa-
bility. The survey asked the participants about which area they were currently employed
in or in which area their last employment was. Most of the participants (31%) were or had
been employed in the educational sector. The second largest sector was office/com-
merce/management (26%). The third largest group of participants was or had been em-
ployed in the social and health sector (17%). Another 6% were or had been working in
trades or technology. Lastly, 20% of the surveyed participants chose “other”. The sectors
most frequently mentioned in the corresponding free-text field were retail, catering and
the insurance sector.
3.3.2. Reasons for Attending the MOOC
The survey participants were asked to rate different statements on a five-point Likert
scale (“strongly agree” through “strongly disagree”). To give more concise results,
“strongly agree” and “agree” is pictured as one category in the graphs. The same goes for
“disagree” and “strongly disagree”.
As can be seen in Figure 8, 85% of the respondents stated that they were interested
in the content and topic of the course, and 24% took on the course because of who was
teaching it. The names and roles of the teachers were not explicitly used to promote the
course, which might explain why almost half of the surveyed disagreed with the state-
ment.
Figure 7. Educational background of questionnaire participants, by percentage (n= 489).
Most participants worked full-time (185) or part-time (134) at the time of the survey. The
third largest group consisted of people currently looking for work (98).
Thirty-three participants
were in training, 32 were self-employed and 32 were students. Twenty-one participants
were marginally employed, 15 were still in school and 18 answered that they were retired.
Five participants were on maternal/paternal leave and three were on disability. The survey
asked the participants about which area they were currently employed in or in which area
their last employment was. Most of the participants (31%) were or had been employed in
the educational sector. The second largest sector was office/commerce/management (26%).
The third largest group of participants was or had been employed in the social and health
sector (17%). Another 6% were or had been working in trades or technology. Lastly, 20%
of the surveyed participants chose “other”. The sectors most frequently mentioned in the
corresponding free-text field were retail, catering and the insurance sector.
3.3.2. Reasons for Attending the MOOC
The survey participants were asked to rate different statements on a five-point Likert
scale (“strongly agree” through “strongly disagree”). To give more concise results, “strongly
agree” and “agree” is pictured as one category in the graphs. The same goes for “disagree”
and “strongly disagree”.
As can be seen in Figure 8, 85% of the respondents stated that they were interested
in the content and topic of the course, and 24% took on the course because of who was
teaching it. The names and roles of the teachers were not explicitly used to promote the
course, which might explain why almost half of the surveyed disagreed with the statement.
A total of 45% of the participants agreed that the MOOC content complemented
their current training or education; 29% disagreed or strongly disagreed. Over half (55%)
answered that the content complemented their current job, while 20% disagreed with
the statement. Most participants agreed with the statement that they were following the
MOOC to gain experience with online education (65%), with 27% answering that the
MOOC was part of their training or education. A much larger portion of those surveyed,
however, stated that the MOOC was simply additional training or further education (66%).
In addition, 26% of the participants mentioned reorientation in the job market as a reason
for attending the MOOC.
Educ. Sci. 2022,12, 89 12 of 17
Educ. Sci. 2022, 12, 89 12 of 17
Figure 8. Interest in topic and interest in teacher as reasons for attending the MOOC, by percentage
(n = 489).
A total of 45% of the participants agreed that the MOOC content complemented their
current training or education; 29% disagreed or strongly disagreed. Over half (55%) an-
swered that the content complemented their current job, while 20% disagreed with the
statement. Most participants agreed with the statement that they were following the
MOOC to gain experience with online education (65%), with 27% answering that the
MOOC was part of their training or education. A much larger portion of those surveyed,
however, stated that the MOOC was simply additional training or further education
(66%). In addition, 26% of the participants mentioned reorientation in the job market as a
reason for attending the MOOC.
The participants were asked to rate which benefits an online education in the form of
a MOOC brings them. As can be seen in Figure 9, 66% agreed or strongly agreed that a
MOOC is an especially useful format for someone currently in employment; 31% agreed
or strongly agreed that an online course is ideal for them because of their place of resi-
dence; 26% agreed or strongly agreed that online education is useful because of care re-
sponsibilities. A much smaller but noteworthy share (10%) agreed or strongly agreed that
online courses are ideal for them because of a disability.
Figure 9. Further reasons for attending an online course as opposed to a face-to-face course, by per-
centage (n = 489).
Figure 8.
Interest in topic and interest in teacher as reasons for attending the MOOC, by percentage
(n= 489).
The participants were asked to rate which benefits an online education in the form
of a MOOC brings them. As can be seen in Figure 9, 66% agreed or strongly agreed
that a MOOC is an especially useful format for someone currently in employment; 31%
agreed or strongly agreed that an online course is ideal for them because of their place of
residence; 26% agreed or strongly agreed that online education is useful because of care
responsibilities. A much smaller but noteworthy share (10%) agreed or strongly agreed
that online courses are ideal for them because of a disability.
Educ. Sci. 2022, 12, 89 12 of 17
Figure 8. Interest in topic and interest in teacher as reasons for attending the MOOC, by percentage
(n = 489).
A total of 45% of the participants agreed that the MOOC content complemented their
current training or education; 29% disagreed or strongly disagreed. Over half (55%) an-
swered that the content complemented their current job, while 20% disagreed with the
statement. Most participants agreed with the statement that they were following the
MOOC to gain experience with online education (65%), with 27% answering that the
MOOC was part of their training or education. A much larger portion of those surveyed,
however, stated that the MOOC was simply additional training or further education
(66%). In addition, 26% of the participants mentioned reorientation in the job market as a
reason for attending the MOOC.
The participants were asked to rate which benefits an online education in the form of
a MOOC brings them. As can be seen in Figure 9, 66% agreed or strongly agreed that a
MOOC is an especially useful format for someone currently in employment; 31% agreed
or strongly agreed that an online course is ideal for them because of their place of resi-
dence; 26% agreed or strongly agreed that online education is useful because of care re-
sponsibilities. A much smaller but noteworthy share (10%) agreed or strongly agreed that
online courses are ideal for them because of a disability.
Figure 9. Further reasons for attending an online course as opposed to a face-to-face course, by per-
centage (n = 489).
Figure 9.
Further reasons for attending an online course as opposed to a face-to-face course, by
percentage (n= 489).
3.3.3. Passing on the Knowledge
In the next section, the participants were asked to rate how likely they were to pass on
the knowledge obtained in the MOOC to other people, which is the principal idea of OER.
Over a third of the respondents (34%) agreed or strongly agreed that they were planning to
pass on the knowledge to coworkers or employees. Another third of those surveyed (32%)
reported wanting to pass on the knowledge to clients. An almost equally large share (36%)
planned to pass on the new knowledge to their family or circle of friends.
Educ. Sci. 2022,12, 89 13 of 17
3.3.4. Rating the MOOC Contents and Instructional Design Aspect
The participants were asked to rate the content and the methodological and instruc-
tional aspect of the course, again using a five-point scale in accordance with the Austrian
grading system used in an educational context, where 1 is the best grade (“very good”)
and 5 is the worst (“insufficient”). The content of the MOOC received an average grade of
1.68 (~0.76 standard deviation); the methodological aspect was given an average of 1.57
(
~0.71 standard
deviation). An open-ended question asked the participants to mention
aspects that they particularly liked. The aspects that were mentioned most often were
the videos, which were seen as enjoyable and easy to follow; the simple explanations and
practical examples; and the combination of different activities, including the booklet. It also
becomes evident that the flexibility and possibility to learn anytime and anywhere was seen
as a positive aspect of this MOOC. Another open-ended question invited the participants
to share aspects they were not satisfied with. The most frequent criticism regarded the
sign-up for the course, which posed a challenge for some participants. Several participants
also seemed to have trouble navigating the platform and were confused by the possibility
of ticking items to mark their own learning process. Some participants complained about
the fact that all of the examples were focused on the Windows operating system. The
participants were asked about what they felt should be changed, should the course be
revised and relaunched in the future. A total of 186 participants stated that there is no
need for revision. The most frequently given answer concerned the speed of the videos,
which was seen as too fast, especially when screencasts or examples were given. Some
participants found the content too simplistic or too easy to follow and would have liked
more detailed explanations or links for further reading. Some participants would have
enjoyed more diverse quiz formats in addition to the multiple-choice quizzes. Another
request was to have transcripts and subtitles for the videos and possibly offer the course in
other languages in a future relaunch.
3.4. Reflection
3.4.1. Reaching the Target Group
In an openly licensed and publicly available MOOC, the participants will be hetero-
geneous. However, it is necessary to keep a specific target group in mind when planning
the content and methodology. Information about this “ideal” target group was provided
by the stakeholders in preliminary workshops. A closer look at the demographics of the
participants can give us an answer to the question of how such an offer is used. The first
thing that stands out here is the large proportion of women on the course (71%). This could
be related to a high share of women in the field of office administration [
26
] and might be
partly related to the known higher proportion of females working as adult educators [
27
].
A significant share of the participants reported working full-time or part-time in the private
sector, so it can be assumed that the MOOC reached its main target group of private sector
employees. Additionally, the fact that a large share of the participants reported having a
master’s degree and working in the educational sector leads to the assumption that the
MOOC was extensively used by trainers and coaches in adult education, either for their
own furthering of digital skills or along with their group, which is in the spirit of OER.
Even if the target group was not reached directly, they should still benefit indirectly from
the MOOC.
3.4.2. MOOC Contents and Instructional Design Considerations
When planning and implementing a MOOC to promote Austrian employees’ digital
skills, DigComp 2.2 AT offers a sound basis for creating and structuring the content.
Differently from an ongoing face-to-face course, the content and methods of a MOOC must
be agreed on well before the start of the MOOC, as there is little flexibility to change the
basic structure or methodology of the course once it is active. The MOOC was taken by
participants with varied educational backgrounds and from many different sectors, so
it can be assumed that participants’ previous knowledge varies significantly. This was
Educ. Sci. 2022,12, 89 14 of 17
reflected in the open-ended feedback form: some respondents stated that the content was
too simple, while for others, simply signing up for the course was challenging. To answer
the question of how the offer was used and what its effect are, it can be said that only a
fraction of the enrolled users took part in the (non-compulsory) live sessions. However,
the option was appreciated by the participants since it provides the opportunity to ask
additional questions and get to know other participants, which, in turn, might create a
sense of community.
3.4.3. Potential Adaptations for Future DigiSkills MOOCs
One way of dealing with the different levels of prior knowledge and interest is to
focus the course more strongly and to differentiate in the course. The prerequisites of the
course could be clarified, for example, with a self-assessment test or a survey: only those
people who have minimal prerequisites (e-mail address) but can still learn essential things
will be recommended for the course. One solution to this problem might be a preliminary
self-assessment or entry exam to determine the level of the participants and then presenting
them with relevant videos or suggesting an entry point into the course based on the
results. Another possibility within the course is to describe more differentiating offers, e.g.,
advanced courses or different tasks depending on personal interests. Such differentiations
are always connected with the problem that they can make the course more complicated.
4. Discussion
We would now like to critically discuss the results described in the article and compare
them with other projects and implementations.
First, the analysis of the digital skills frameworks and European projects has shown
that there are some, but not many, training measures using a MOOC for employees on a
large scale in Europe; i.e., this project was not a novel innovation, but is still unique. During
the planning of the MOOC, the need for corresponding educational offers became very
clear, e.g., in the workshops with stakeholders, not least since the offer of a MOOC was
seen as thoroughly modern, but also caused uncertainty as to whether it was the right
instrument, because it already requires initial digital skills. We know this discussion from
other MOOCs aimed at beginners in digital learning [
15
]. In a way, COVID-19 was helpful
here: due to the changes involved—restrictions on social contact and a ban on meetings in
spring 2021—a MOOC that can be completed fully online was just the right thing to offer.
The activities of the participants during the MOOC, the completion rate and also
the presented interaction courses—higher activities at the beginning of the course, with a
downturn in the course—follow the usual patterns that we know from other MOOCs and
do not show any conspicuous features. This is particularly pleasing, since this MOOC is a
voluntary offer and participation is not integrated into a degree course, for example, and
the participants do not yet have much experience with digital technologies. The positive
feedback from the participants, taken from the evaluation, shows that with the MOOC,
we have developed and implemented a very good basic measure to promote digital skills
among employees in Austria.
In the reflection section, we have elaborated on some further reflections and insights
that should be critically monitored in future implementations. It turned out that quite a
few adult educators finished the course, which are not the main target group. This could
also lead, for example, to the creation of targeted educational offers for this target group.
The Mu.Sa project has also expanded its offer with specialization modules, so this need for
differentiating offers is also evident in other projects.
As always, in researching MOOCs, we cannot make assumptions about participants
who did not enroll, are not active in the MOOC, or those who did not fill out the question-
naire or take part in the live sessions. While the Learning Analytics Creator’s Dashboard
does give us information about the number of inactive participants, we can only cautiously
assume their motives and reasons for not participating or not finishing the MOOC, which
might include such factors as lack of motivation, time constraints or technical issues. We
Educ. Sci. 2022,12, 89 15 of 17
assume that not all of the dropouts are necessarily an issue that needs to be solved, as some
might indicate, for example, that the participants never wanted to complete the course or
only sought specific information that was important for them. At the same time, we are
aware of the fact that a MOOC is a very good, but not sufficient, measure to promote digital
skills among employees.
5. Conclusions and Outlook
The Digital Skills MOOC, based on DigComp 2.2 AT, was designed to upskill 500 Austrian
private sector employees and complements the European strategy for meeting the needs
created by the digitalization of the labor market. This paper answered the questions of
how the MOOC was developed and implemented, how it was used and what the potential
effects of such an educational intervention are, using a four-step action research design
as the background for the description, analysis and reflection. In summary, our research
shows very satisfactory results and we see a MOOC as a powerful measure to support the
development of digital skills among employees.
The MOOC itself will be updated within a planned second round based on the findings
and reflections presented in this paper and will be relaunched regularly as a guided MOOC
after another promotion phase. As all the content in the MOOC is openly licensed with a
Creative Commons license, the course as a whole, or parts of it, may be reused by trade
unions or adult education centers if they would like to offer seminars or courses based on
this content. In these future training, the online content may very well be enriched with
face-to-face classes instead of online live sessions [
1
]. To improve the MOOC experience for
learners and course creators, the Creator’s Dashboard that provides learning analytics will
be integrated further and new features will be added to reveal how particular activities
engage different types of learners. Learning analytics will also identify distractions and
help automate mundane organizational tasks, so that participants can focus on learning.
Concerning future research and development, we see the practical need to adapt
examples and practice examples concerning the digitals skills framework to a special target
group and their settings and conditions. Thus, for future projects for slightly different
target groups or other countries, it is recommended to follow the national frameworks.
You can orientate yourself on our developments and topics, but it is highly likely that your
target group uses other applications—for example, to book a train or for digital signatures.
Here, the projects must also plan and carry out longer development work, ideally with the
target group and stakeholders.
Supplementary Materials:
The following supporting information can be downloaded at: https:
//doi.org/10.5281/zenodo.5734179, File S1: Fragebogen zum MOOC DigiSkills für alle–Machen Sie
sich fit für die digitale Welt (Questionnaire to the MOOC DigiSkills for all–Get ready for the digital
world; available in German). The MOOC described is currently available on the platform imoox.at
(accessed on 29 November 2021).
Author Contributions:
Conceptualization, S.S.; methodology, K.S.; software, P.L.; formal analysis,
S.E.; investigation S.E., K.S.; data curation, K.S., S.E.; writing—original draft preparation, S.E.;
writing—K.S., S.S., M.E.; visualization, P.L., S.S.; supervision, M.E.; project administration, K.S.;
funding acquisition, K.S. All authors have read and agreed to the published version of the manuscript.
Funding:
This research was carried out in part within the framework of the following projects and
the corresponding (co-)funding: “Digital Skills for 500 Private Employees in Austria-Development,
Implementation and Evaluation of a Massive Open Online Course (MOOC)” with funding from
the Digitalisation Fund of the Vienna Chamber of Labor (AK Wien) as well as “iMooX-The MOOC
Platform as a Service for all Austrian Universities” (2020–2023) with funding from the Austrian
Federal Ministry of Education, Science and Research (BMBWF).
Institutional Review Board Statement:
The study did not require ethical approval, as the used data
was anonymized and used in accordance to the privacy declaration of the iMooX platform and
therefore to all legal requirements.
Informed Consent Statement:
Informed consent was obtained from all subjects involved in
the study
.
Educ. Sci. 2022,12, 89 16 of 17
Data Availability Statement:
The data presented in this study are available on request from the
corresponding author. The data are not publicly available due to privacy issues. The questionnaire
(in German) is available at Zenodo: 10.5281/zenodo.5734179.
Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design
of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or
in the decision to publish the results.
References
1.
Ebner, M.; Kreiml, T.; Nemeth, M.; Taschwer, M. Digital Skills für 500 Privatangestellte in Österreich–Entwicklung, Durchführung und
Evaluierung eines Massive Open Online Course (MOOC); ABIF: Vienna, Austria, 2019.
2.
European Centre for the Development of Vocational Training (CEDEFOP). Insights into Skill Shortages and Skill Mismatch
Learning from Cedefop’s European Skills and Jobs Survey. Publications Office of the European Union: Luxembourg, 2018.
Available online: http://data.europa.eu/doi/10.2801/645011 (accessed on 29 November 2021).
3.
Bliem, W.; Löffler, R.; Van den Nest, E. Industrie 4.0 und Digitalisierung–Bedrohung oder Chance? Qualitative Interviews zu den
Veränderungen in der Arbeits-und Berufswelt; Arbeitsmarktservice Österreich, Abt. Arbeitsmarktforschung und Berufsinformation:
Vienna, Austria, 2018.
4.
Kemmis, S.; McTaggart, R.; Lewin, K.I.; Nixon, R. The Action Research Planner. Doing Critical Participatory Action Research; Springer:
Singapore, 2014. [CrossRef]
5. Robson, M. Action Research: Principles and Practice. Action Learn. Res. Pract. 2016,13, 283–298. [CrossRef]
6.
Collins, A.; Joseph, D.; Bielaczyc, K. Design Research: Theoretical and Methodological Issues. J. Learn. Sci
2004
,13, 15–42.
[CrossRef]
7.
McAuley, A.; Stewart, B.; Siemens, G.; Cormier, D. Massive Open Online Courses Digital Ways of Knowing and Learning, the MOOC
Model for Digital Practice. Available online: www.elearnspace.org/Articles/MOOC_Final.pdf (accessed on 29 November 2021).
8.
Ebner, M.; Lorenz, A.; Lackner, E.; Kopp, M.; Kumar, S.; Schön, S.; Wittke, A. How OER enhance MOOCs–A Perspective
from German-speaking Europe. In Open Education: From OERs to MOOCs; Jemni, M., Kinshuk, Khribi, M.K., Eds.; Springer
International: Berlin/Heidelberg, Germany, 2016; pp. 205–220. [CrossRef]
9.
Wedekind, J. MOOCs–eine Herausforderung für die Hochschulen? In Hochschuldidaktik im Zeichen von Heterogenität und Vielfalt;
Reinmann, G., Ebner, M., Schön, S., Eds.; BoD: Norderstedt, Germany, 2013.
10.
Arnold, P.; Kumar, S.; Schön, S.; Ebner, M.; Thillosen, A. A MOOC on Open Educational Resources as an Open Educational
Resource: COER13. In The MOOC Case Book: Case Studies in MOOC Design, Development and Implementation; Corbeil, J.R., Corbeil,
M.E., Khan, B.H., Eds.; Linus Learning: New York, NY, USA, 2015; pp. 247–258.
11.
Klotmann, E.; Köck, C.; Lindner, M.; Oberländer, N.; Sucker, J.; Winkler, B. (Eds.) Der vhsMOOC 2013. Wecke den Riesen auf;
W. Bertelsmann: Bielefeld, Germany, 2014.
12. Deutsche Telekom Magenta MOOC. Available online: http://www.telekom.com/magenta-mooc (accessed on 29 November 2021).
13. Schulmeister, R. The Position of xMOOCs in Educational Systems. Eleed 2014,10. Available online: http://eleed.campussource.
de/archive/10/4074 (accessed on 29 November 2021).
14. Bertelsmann Stiftung. Monitor Digitale Bildung; Bertelsmann: Gütersloh, Germany, 2018. [CrossRef]
15.
Ebner, M.; Schön, S.; Braun, C. More Than a MOOC—Seven Learning and Teaching Scenarios to Use MOOCs in Higher Education
and Beyond. In Emerging Technologies and Pedagogies in the Curriculum. Bridging Human and Machine: Future Education with
Intelligence; Yu, S., Ally, M., Tsinakos, A., Eds.; Springer: Singapore, 2020; pp. 75–87. [CrossRef]
16.
Ebner, M.; Schön, S. Inverse Blended Learning—A didactical concept for MOOCs and its positive effects on dropout-rates. In The
Impact of MOOCs on Distance Education in Malaysia and Beyond; Ally, M., Amin Embi, M., Norman, H., Eds.; Routledge: London,
UK, 2019.
17.
Carretero, S.; Vuorikari, R.; Punie, Y. DigComp 2.1: The Digital Competence Framework for Citizens with Eight Proficiency Levels and
Examples of Use; Publications Office of the European Union: Luxembourg, 2018. [CrossRef]
18.
Kluzer, S.; Pujol Priego, L. DigComp into Action-Get inspired, make it happen. In A User Guide to the European Digital Competence
Framework; Carretero Gomez, S., Punie, Y., Vuorikari, R., Cabrera Giraldez, M., Okeeffe, W., Eds.; Publications Office of the
European Union: Luxembourg, 2018. [CrossRef]
19.
Centeno, C.; Vuorikari, R.; Punie, Y.; O
´
Keeffe, W.; Kluzer, S.; Vitorica, A.; Lejarzegi, R.; Martínez de Soria, I.; Bartolomé, J.
Developing Digital Competence for Employability: Engaging and Supporting Stakeholders with the Use of DigComp; Publications Office of
the European Union: Luxembourg, 2019. [CrossRef]
20.
Centeno, C. DigComp at Work Implementation Guide; Okeeffe, W., Ed.; Publications Office of the European Union: Luxembourg,
2020. [CrossRef]
21.
Bundesministerium für Digitalisierung und Wirtschaftsstandort. Digitales Kompetenzmodell für Österreich. DigComp 2.2 AT;
Bundesministerium für Digitalisierung und Wirtschaftsstandort: Vienna, Austria, 2021.
22.
Menino Homem, P. Insights into piloting a community builder MOOC to help museum professionals facing 21st century
challenges. In The Future of Museum Professionals in the Digital Era. The Success Story of Mu.SA; Kameas, A., Polymeropoulou, P.,
Eds.; Hellenic Open University Press: Athens, Greece, 2020; pp. 69–72.
Educ. Sci. 2022,12, 89 17 of 17
23.
Neuböck, K.; Kopp, M.; Ebner, M. What do we know about typical MOOC participants? First insights from the field. In
Proceedings of the eMOOCs 2015 Conference, Mons, Belgium, 18–20 May 2015; pp. 183–190.
24.
Ebner, M. iMooX-a MOOC platform for all (universities). In Proceedings of the 2021 7th International Conference on Electrical,
Electronics and Information Engineering (ICEEIE), Malang, Indonesia, 2 October 2021.
25.
Kopp, M.; Ebner, M. La certificación de los MOOC. Ventajas, desafíos y experiencias prácticas. Rev. Esp. Pedagog
2017
,75, 83–100.
[CrossRef]
26.
Gresch, N.; Sauer, B. Debates on Women’s Representation in Austria or: The Development of the Pitfalls of a Conservative Gender Regime;
European University Institute: San Domenico di Fiesole, Italy, 2015.
27.
Gutknecht-Gmeiner, M. Die statistische Erfassung des Erwachsenenbildungs- und Weiterbildungspersonals in Österreich.
MAGAZIN Erwachsenenbildung.At. Das Fachmedium Für Forsch. Prax. Und Diskurs 2008,4, 1–22.
