IMPACT ASSESSMENT OF A MOOC PLATFORM: CONSIDERATIONS, DEVELOPMENT, AND RESULTS

Abstract

In 2020, the project 'iMooX - The MOOC Platform as a Service for all Austrian Universities' was launched. It is co-financed by the Austrian Ministry of Education, Science and Research. After half of the funding period, the project management wants to assess and share results and outcomes but also address (potential) additional “impacts” of the MOOC platform. Building upon work on OER impact assessment, this contribution describes in detail how the specific iMooX.at approach of impact measurement was developed. Literature review, stakeholder analysis, and problem-based interviews were the base for developing a questionnaire addressing the defined key stakeholder “MOOC creators”. The article also presents the survey results in English for the first time but focuses more on the development, strengths, and weaknesses of the selected methods. The article is seen as a contribution to the further development of impact assessment for MOOC platforms.

Full text

1
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
IMPACT ASSESSMENT OF A MOOC PLATFORM:
CONSIDERATIONS, DEVELOPMENT, AND RESULTS
June 2023, Potsdam
IMPACT
Martin Ebner, Sarah Edelsbrunner,
Katharina Hohla-Sejkora, Bettina Mair,
Sandra Schön
(Graz University of Technology)
Silvia Lipp, Iris Steinkellner, Ivana Stojcevic
(University of Graz)
Charlotte Zwiauer
(University of Vienna)

2
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
License
These slides may be used under the terms of the
CC licence CC BY 4.0 International
(https://creativecommons.org/licenses/by/4.0/ ).
Please use the links and references as sources for
individual illustrations, if applicable.

3
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Full paper Will be published in the EMOOCs 2023 Proceedings:
Martin Ebner, Sarah Edelsbrunner, Katharina Hohla-
Sejkora, Bettina Mair, Sandra Schön, Silvia Lipp, Iris
Steinkellner, Ivana Stojcevic, Charlotte Zwiauer (2023).
Impact assessment of a MOOC platform: Considerations,
development, and results.

4
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Research Interest
How to assess
MOOC platform
impact

5
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Agenda
•Introduction: impact assessment
•The MOOC platform iMooX
•Our approach to measure impact
•Results or impact assessment

6
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Impact Assessment (of OER)
Potential key questions and relevant indicators from
investments on the core of OER.
Source:Ebner, M., Orr,
D. & Schön, S., 2022.
„OER Impact
Assessment: A
framework for higher
education institutions
and beyond.
Approaches to assess
the impact of Open
Educational
Resources“, In:Open
Education Studies, vol.
4, no.1,2022,pp.296-
309.
https://doi.org/10.1515
/edu-2022-0018

7
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
iMooX.at – the Austrian MOOC platform
7
TU Graz Universität Wien

8
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
iMooX.at – the Austrian MOOC platform
Expansion to the
national MOOC platform
(2020-2023) supported by
2013 founded
(TU Graz | University of Graz);
now hosted by TU Graz
All materials CC licensed
100% GPNR conform
Member of
Member of

9
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Impact Assessment of iMooX
Potential key questions and relevant indicators from
investments on iMooX to the open issue of ‘impact’.

10
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Overview of Impact Assessment
development of iMooX
1. Identification of relevant stakeholders of the
iMooX.at platform or MOOCs where impacts
might occur
2. Description of activities, results, and
outcomes of the iMooX.at project (number of
MOOCs and participants)
3. 5 problem-centered interviews were
conducted with selected Austrian MOOC
creators at universities as base for
questionnaire
4. Online survey amongst MOOC creators
(N=17)

11
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
1. Identification of stakeholders
Identified stakeholders in
MOOC development
and implementation at
iMooX partner
universities.

12
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
2. Problem-based interviews for development of
questionnaire

13
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
2. Problem-based interviews for development of
questionnaire

14
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
2. Problem-based interviews for development of
questionnaire

15
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
3. Structure and Question
respectivly results of the survey
(-> see next slides)

16
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Results: Survey

17
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Results: Survey

18
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Results: Description of activities, results and
outcome of the iMooX project (national platform)

19
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Results: Description of activities, results and
outcome (national platform)
A total of 54,917 registered users at iMooX (09/22)

20
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Discussion and open questions
•Pros and cons to ask MOOC creators
•Alternative: Asking users

21
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Acknowledgements
.
„iMooX –the MOOC platform as a service
for all Austrian universities” (2020-2023),
co-funded by the Federal Ministry Republic
of Austria Education Science and Research.
Partners: TU Graz, University of Vienna
Parts of the presented development and research was carried out
Within the project

22
WE CARE ABOUT EDUCATION
TU Graz, www.tugraz.at
Contact
Dr. Martin Ebner (head of Educational
Technology team and iMooX)
Graz University of Technology
martin.ebner@tugraz.at