Content uploaded by Farrah Dina Yusop
Author content
All content in this area was uploaded by Farrah Dina Yusop on Jun 07, 2021
Content may be subject to copyright.
Citation:
Suria Baba & Yusop, F.D. (2020). Use of Delphi technique to inform MOOC design and
development (p. 62-67). In Farrah Dina Yusop, Rosemaliza Kamalludeen, Zahiruddin
Fitri Abu Hassan & Mohd Shukri Nordin (Eds). MOOCs in Malaysia Towards
Globalised Online Learning. Kuala Lumpur: UPNM Press. ISBN: 9978-967-2414-20-9.
Use of Delphi technique to inform MOOC design and development
Suria Baba, Universiti Malaysia Kelantan, suria@umk.edu.my
Farrah Dina Yusop, Universiti Malaya, farah@um.edu.my
Abstract
This chapter highlights the application of Delphi technique that led into the design and
development of a MOOC on media literacy competencies.
Introduction
Media literacy has been one of the issues discussed especially in the third world countries as
well as in ASEAN region. Because each country has its own media culture, a Delphi study was
seen as the most relevant method of exploring the unique and cultural aspect of media within
a specific media society. The modified Delphi study aimed to identify relevant stakeholders’
perspectives about (1) media system and culture in the country, and the role of media in
bringing about social enhancement, and (2) the important media literacy competencies
presumed relevant and important for inclusions in a curriculum on media literacy for the young
people in the tertiary education delivered via a Massive Online Open Course (MOOC)
platform. This study was part of a work package under the Erasmus+ Media Literacy Project
entitled “International Media Study: Media Literacy (MEDLIT) as Media Competence
Program for Social Change” involving nine universities from Austria, Germany, Malaysia,
Thailand, The Netherlands and Vietnam.
This chapter highlights the application of Delphi study that led into the design and development
of the MEDLIT MOOC.
Delphi study as a method of data collection
Delphi is one of widely used techniques in educational research setting especially in forming
specific guidelines and standards, in predicting future trends, and in developing curricula or
learning experiences for students (Green, 2014). It is a group communication process involving
a panel of experts who judge, weigh and rank a set of ideas on a specific issue (Wallner, Alpen,
& Adolf, 2017).
In this Delphi study, research teams from two local public universities in Malaysia carried out
face-to-face structured interviews with a group of pre-identified experts. All members strictly
followed a set of interview protocols established to ensure standardization in the data collection
process and to ensure the quality of the data gathered.
The experts were asked ten open-ended questions intended to seek their opinions on the role of
media in bringing about social change and in empowering people, about media culture, aspect
of media use that constraints or limit participation in social change, and expectation on media
literacy program in the country. Experts were also asked to rank the importance of 39
knowledge items they considered as relevant for a media literacy education for university
students. For the purpose of standardization, media literacy in this study refers to the ability
and capability to access, analyze, and evaluate media content responsibly.
The data collection was carried out over a period of six weeks. All in all, a total of 76 experts
participated in the Delphi study. They are representatives of experts from (1) institutions of
higher learning, (2) news organizations, (3) government agencies, (4) non-government
agencies, (5) professional associations, (6) telecommunication and advertising companies, and
(7) others which include politicians, motivators, human resource managers, and business
people. A list of potential respondents for the study was created prior to the actual data
collection. A letter of invitation requesting participation was emailed to the potential
respondents and was followed up with calls.
64% of the experts are male while 36% are female (36%). All experts are between 24 to 70
years old with a mean of 50.7 years, indicating the respondents were experienced individuals
in their respective fields. Majority of the experts were from educational institutions and non-
governmental organizations, followed by experts from government agencies, news
organizations, and professional organizations. Only small percentage of experts was from
industry, politicians and opinion leaders including motivators.
The duration of face-to-face interviews vary between 40 to 70 minutes in which 25 to 50
minutes were allocated for the experts to respond to the open-ended questions. During the
interviews, the respondents were first explained on the purpose and significance of the study.
A majority of the respondents provided verbal responses to the open-ended question and self-
administered written responses to the closed-ended questions. All verbal responses were
recorded and later transcribed.
Ensuring validity and reliability
Qualitative research emphasizes on description, meaning, emic perspectives, and participants
as the research expert, naturalistic, emerging designs, process-based theory, and researchers as
the main instruments (Creswell (2013). Data collection techniques vary and may include
observations, interviews, and documentation to obtain the data (LeCompte & Schensul, 1999;
Marshall & Rossman, 2006)
Application of various methods has prompted researchers to obtain and ensure the validity and
reliability regarding trustworthiness in conducting qualitative research. After completion of
data collection, researcher will then review all the data and compose it into classes or topics
over the information sources. Furthermore, this process involves complex reasoning such as
inductive and deductive, and views of context or setting of whether participants or research
sites meet the study requirements (LeCompte & Schensul, 1999); (Marshall & Rossman, 2006).
Researchers run back and forth among themes and databases until they have accomplished an
across-the-board set of themes (Creswell, 2013). Inductive thinking should be encouraged that
requires constant checking of facts against the data. In addition, researchers will continue to
research so as to form a pattern and data retrieval are interconnected through an iterative
process. This situation happened during this study and pattern arise when all respondents gave
their views on the matters and been triangulated by the 12 researchers. Rigorousness occur in
this situated environment to ensure trustworthiness.
Qualitative researchers act as the key instrument in information collection, gathering
participants’ viewpoints, and various subjective perspectives, and presenting a holistic and
complex picture, involving an emerging and evolving design rather than a tightly prefigured
design (Cresswell, 2013). Considering these characteristics, the researcher needs to articulate
trustworthiness in the process of conducting a qualitative research towards attaining validity
and reliability of data. The four analytical elements, namely credibility, transferability,
dependability and confirmability are involved in determining trustworthiness in conducting a
qualitative research.
One of the validity elements conducted throughout this study is ensuring rigour through
triangulation technique that involves utilizing numerous information sources as a part of an
examination to deliver comprehension to get better insights of the phenomenon studied.
Triangulation also been identified as a method for verifying outcomes and one way to examine
validity. This strategy accepts that a shortcoming in one technique will be cross-checked by
another technique, and that it is constantly conceivable to comprehend different documents.
Implementing the triangulation method will be part of the strategy in enhancing confirmation,
also guarantee that the data is rich, vigorous, far-reaching and thick. Utilizing numerous
techniques can lead towards more profound comprehension. It is been practiced by all
researchers that by triangulating the data between and within the researchers it helps in ensuring
trustworthiness and data is highly rigour.
To address issue pertaining validity and reliability which are the vital part in ensuring its been
priorities since 12 researchers involve with different stakeholders of 76 respondents. To ensure
this been taken account the procedure are based from principles suggested by Merriam (2009)
Creswell (2013). These principles are been contextualize in the conducting this Delphi Study
adopting qualitative research.
One can ensure the data gathered are sufficient and reliable after rigorous cross checking. One
of the techniques is member checking which is considered as one of the credible assurance
approaches to analyze qualitative data. It covers criteria such as examining the information,
classifications, elucidations, and conclusions with respondents upon all data gathered.
Triangulation part of to ensure confirmability that involves using multiple data sources,
methods, and theories to provide validating evidence in an examination to create
understanding. According to Creswell (2013), researchers triangulate information and deliver
validity to their outcomes when they are allocate verification to document a code in different
sources of data. In this study triangulation occurs within the researchers and between
respondents. In such triangulation that in context with this study helps to ensure rigour towards
the building up trustworthiness.
From research findings to MOOC development
In sum, experts agree that media has an influence in bringing social change. Media is expected
to function as disseminator of accurate and unbiased information in their reports, empower the
society by cultivating critical thinking, raise awareness and cultivate good values among
public.
Experts also emphasize on the importance of educational institutions to provide impactful
media literacy program that gives priority to promoting journalism ethics.
Data from the findings have helped researchers to create five (5) course learning objectives
(CLO) for the MEDLIT MOOC development. This CLO been translated into relevant activities
and assessment in MOOC that will enhance the idea of media literacy as a media competence
for social development. The CLOs are:
● ability to identify media role, media use and media impact in the society;
● ability to explain the concept of media reception and its relation to audience
participation;
● ability to describe how media content/text is created (media representation vs reality)
and evaluate contents trustworthiness;
● ability to explain the meaning of "freedom of expression" and how/why media are
regulated; and
● ability to describe ethic and cultural values in responsible communication and media
use.
The development team then designed and developed the relevant learning activities and
resources for the MOOC based on these CLOs.
Conclusion
This chapter describes the use of Delphi study to inform the design and development of a
MOOC course titled Media Literacy (MEDLIT) as Media Competence Program for Social
Change. Data gathered from 76 respondents were analyzed based thematic analysis technique
with the help of a qualitative analysis software. The course learning outcomes (CLOs) derived
from the analyses then were used as the main guide to inform the MOOC development.
References
Creswell, J. W. (2013). Qualitative Inquiry & Research Design: Choosing among Five
Approaches. SAGE Publication, Third Edition.
Green, R. A. (2014). The Delphi technique in educational research. Sage Open, 4(2).
doi:10.1177/2158244014529773
LeCompte, M. D., & Schensul, J. J. (1999). Designing and Conducting Ethnographic Research.
Lanham MD: AltaMira.
Marshall, C., & Rossman, G. B. (2006). Designing Qualitative Research. SAGE Publication,
4th Edition.
Wallner, C., Alpen, S., & Adolf, M. (2017). Fostering media literacy competencies for
navigating digital media cultures: Findings of a comparative study in Southeast Asia.
Paper presented at the International Association for Media and Communication
Research (IAMCR), Cartagena, Colombia.
