Content uploaded by Imelda Gozali
Author content
All content in this area was uploaded by Imelda Gozali on Jun 15, 2023
Content may be subject to copyright.
Computer Assisted Language Learning Electronic Journal CALL-EJ), 24(1), 2023, 280-304
Examining Language Teachers’ Change in TPACK-HOTS Self-
Perception during the COVID-19 Pandemic in Indonesia
Imelda Gozali (imelda.gozali@ukwms.ac.id) *Corresponding Author
English Language Education Study Program, Teacher Education Faculty, Widya
Mandala Surabaya Catholic University, Indonesia
Anita Lie (anita@ukwms.ac.id)
English Language Education Study Program, Teacher Education Faculty, Widya
Mandala Surabaya Catholic University, Indonesia
Siti Mina Tamah (mina@ukwms.ac.id)
English Language Education Study Program, Teacher Education Faculty, Widya
Mandala Surabaya Catholic University, Indonesia
Abstract
In light of the ongoing concern of the Indonesian Ministry of Education to continuously
upgrade teachers’ competence in the use of technology and impart critical thinking as part
of 21st-century skills, this study sought to examine language teachers’ self-perception of
their TPACK and Higher-Order Thinking Skills (HOTS) competence during the COVID-
19 pandemic. To this end, an explanatory sequential mixed-method study was designed
to gather data from 137 languages (English and Bahasa Indonesia) teachers from various
parts of Indonesia, employing a questionnaire and interview for data collection. The
quantitative and qualitative data analysis revealed that the teachers perceived a rise in all
TPACK subdomains, with the highest increment in TK and the smallest for CK. The
teachers also rated themselves higher in HOTS and opined that TPACK supported HOTS-
based lesson implementation. The implications for teacher training and pedagogy are also
discussed.
Keywords: Higher Order Thinking Skill (HOTS), Indonesia, language
teachers, pandemic, Technological Pedagogical Content Knowledge (TPACK)
Introduction
The COVID-19 pandemic has undoubtedly impacted the lives of people all over the
planet, including the extensive country-archipelago that is Indonesia. Apart from the
disastrous consequence of the emergency situation on the nation’s health system, the
Ministry of Education also had to act quickly to switch to a distance learning mode, which
was enacted differently depending on the technological readiness of each region and the
teachers’ prior technological knowledge (Lie et al., 2020). Now, more than two years
after President Jokowi announced the first COVID-19 case on March 2, 2020, the
pandemic has been mainly brought under control due to aggressive vaccination drives
281
and different stages of mobility restriction. It seems thus opportune at this juncture to
look back at how the pandemic has changed the educational landscape in Indonesia,
especially in terms of teachers’ competence in integrating technology into their teaching,
which has been brought to the fore during the Emergency Remote Learning situation.
When it comes to assessing teachers’ technological competence concerning their
pedagogical and knowledge of the subject matter, the TPACK (Technological
Pedagogical and Content Knowledge) framework (Koehler & Mishra, 2009) might be the
first that comes to mind. Indeed, Indonesia’s national teacher certification program,
locally termed the Program Profesi Guru, included TPACK mastery as one of the criteria
in the rubric for the in-service teachers’ micro-teaching evaluation. Besides its utilization
as an assessment instrument, TPACK surveys have been extensively used by scholars in
studies related to teachers’ self-perception of their TPACK skills (Castéra et al., 2020;
Efwinda & Mannan, 2021; Novita et al., 2022; Roussinous & Jimoyiannis, 2019). In light
of the pandemic, it can be intuited that teachers’ TPACK self-perception might have
undergone a substantial change. Does technology incorporation mean better teaching? Is
managing online classes better or worse than the ‘offline’ ones? Did the work-from-home
mode mean that teachers could enhance their content knowledge through the various
online professional development (PD) programs? A TPACK-based survey,
complemented with qualitative, extended response data, might be able to shed light on
those questions.
However, technological integration alone is insufficient to prepare students, and
their teachers, for the Society 5.0 that looms ahead. Together with ‘technological skill
and digital literacy’, the Organization for Economic Cooperation and Development
(OECD) proposed ‘critical thinking’ as one of the components that make up the 21st-
century skills (Ananiadou & Claro, 2009). Inculcating critical thinking skills in
Indonesian students has also been the thrust of the educational policy in the country, as
spelled out by the prevailing 2013 Curriculum. The concept of ‘critical thinking’ is further
operationalized into Higher Order Thinking Skills (henceforth, HOTS), which was first
postulated by Bloom et al. (1956, as cited in Krathwohl, 2002) and subsequently modified
by Anderson and Krathwohl (2001). The Indonesian Ministry of Education and Culture
mandated that teachers ought to design and implement HOTS-based assessments
(Suhardi, 2018). Hence, teachers are faced with a herculean task: apart from incorporating
technology into the classroom in the upcoming blended, hybrid, or limited face-to-face
lessons, they still need to ensure that they are conversant with the HOTS concept and
apply it in their teaching and assessment. This challenge raises more interesting questions:
is there any relationship between HOTS and technology-integration skills? Does the
pandemic have any bearing on the perceived HOTS skills of the teachers?
This study then attempts to provide some insights into the answers to the
aforementioned questions. Besides, given the paucity of research relating TPACK and
HOTS among language teachers, this study also intends to fill the contextual gap. Hence,
this study specifically addresses the following research questions:
1. Is there any difference between language teachers’ self-perceived TPACK
competence at the beginning of the pandemic compared to 19 months after?
2. Is there any difference between language teachers’ self-perceived HOTS level at
the beginning of the pandemic compared to 19 months after?
282
3. Is there any perceived relationship between TPACK and HOTS competence of
the language teachers?
Literature Review
Coined first as TPCK in 2006, TPACK is a further development by Mishra and
Koehler of Shulman’s (1986) PCK construct (Mishra & Koehler, 2006). It represents the
central intersection of the three domains of Technology, Pedagogy, and Content
Knowledge that a teacher should have. Seen in this way, the TPACK framework consists
of seven subdomains; Technological Knowledge (TK), Pedagogical Knowledge (PK),
Content Knowledge (CK), as well as the intersection between two or three domains,
namely Technological-Pedagogical Knowledge (TPK), Technological-Content
Knowledge (TCK), Pedagogical-Content Knowledge (PCK), and finally the
Technological Pedagogical Content Knowledge (TPACK). A diagrammatic
representation of the TPACK framework, as well as a brief, tabulated explanation of the
meaning of each subdomain, is given in Figure 1 and Table 1 respectively.
Figure 1
The TPACK Framework (Mishra & Koehler, 2006)
283
Table 1
Brief Explanation of each TPACK Domain (Mishra & Koehler, 2006)
No.
Domain
1
Pedagogical Knowledge
(PK)
Knowledge about process and practices or
methods of teaching and learning and how it
encompasses educational purposes, values, and
aims (e.g. student learning, classroom
management, lesson plan development, and
implementation).
2
Content Knowledge
(CK)
Knowledge about the actual subject matter that is
to be taught (e.g., central facts, concepts, theories,
procedures).
3
Technological
Knowledge (TK)
Knowledge about standard technologies and how
to operate them (e.g., from books and chalkboards
to the internet and digital video).
4
Pedagogical Content
Knowledge (PCK)
Knowledge about pedagogy that is applicable to
the specific teaching content (e.g., knowing what
teaching approaches fit the content, knowing how
elements of content can be arranged for better
teaching).
5
Technological
Pedagogical Knowledge
(TPK)
Knowledge of how teaching may be changed as
the result of using particular technologies (e.g.,
knowing that a range of tools exists, the ability to
select based on fitness, and knowledge of
affordances of these tools for pedagogical
practice).
6
Technological Content
Knowledge (TCK)
Knowledge about how technology and content are
reciprocally related (e.g., knowing how subject
matter can be changed by the application of
technology).
7
Technological
Pedagogical Content
Knowledge (TPCK)
Knowledge for good teaching with technology
which requires understanding how technologies
can support teaching subject matter (e.g., knowing
how technologies can help overcome problems in
the process of teaching and learning, and how they
can be used for constructive content and
pedagogy).
The TPACK framework is typically used to assess teachers and their self-
perception of their skill at teaching with technological integration, with extensive
research having been done in this respect (Castéra et al., 2020; Efwinda & Mannan, 2021;
Roussinos & Jimoyiannis, 2019). The TPACK instrument, in the form of a survey
operationalizing each of the TPACK domains, has been developed and tested
(Bostancıoğlu & Handley, 2018; Chai et al., 2010; Schmidt et al., 2009), as well as
adapted by a plethora of other studies (Scott, 2021). In the field of language teaching,
studies on teachers’ TPACK are likewise burgeoning (Aisyah et al., 2021; Cheng, 2017;
Ersanli, 2016; Habibi et al., 2019; Raygan & Moradkhani, 2020; Tseng et al., 2020).
284
Raygan and Moradkhani (2020) underscored the strongly mediating effect of EFL
teachers’ attitude and TPACK in integrating technology, with supportive school climate
playing only secondary role. In their critical review of research on TPACK in language
teaching, Tseng et al. (2020) revealed that language teachers in the studies reviewed have
not demonstrated an optimal level of self-confidence in digital teaching and the
deployment of technology for student-centered learning, focusing its use mainly for
motivation, exercises, and course presentation.
During the COVID-19 outbreak, the challenges teachers faced during the
emergency remote teaching, specifically when jump-started with hardly any prior
experience with the online mode, have been the subject of numerous research (Arcueno
et al., 2021; Kholik et al., 2020; van der Spoel et al., 2020; Wen & Kim Hua, 2020; and
many more). In particular, teachers’ aptitude for incorporating technology before and
during the pandemic as measured by TPACK attracted the attention of several scholars
(Chen & Hsu, 2021; Mourlam et al., 2021; Nelson, 2020). Mourlam et al. (2021), for
example, carried out a survey study involving 167 teachers in the USA to investigate their
self-perception of TPACK competence before and during the pandemic. The survey was
administered in June 2020 through email list from public school websites and social
media. The result showed that the teachers perceived a decrease in five out of the seven
TPACK domains, with only TK and TCK domains remaining relatively similar before
and during the pandemic. They concluded that the instructional context the teachers found
themselves in (the sudden shift to online teaching) had caused the teachers to brush up on
their knowledge of technological tools, which explained their confidence in their TK and
TCK, while at the same time making them sense the inadequacy of their pedagogical skill
in the new, online environment. Similarly, Chen and Hsu (2021) conducted a two-phase
survey study before (in 2017) and during the pandemic (in 2020), making use of the
TLPACK (Technology, Learner, Pedagogy, Academic discipline content, and Content
Knowledge) survey engaging 500 teachers in Taiwan. TLPACK is a further elaboration
of TPACK, with the inclusion of Learners and Academic discipline content domains (Hsu
& Chen, 2019 in Chen & Hsu, 2021). Adopting a quota sampling method, they
disseminated more than 500 survey forms in 2017 and 2020, and the returned forms were
randomly stratified to obtain a balanced representative of each teaching level. They
discovered that, overall, the teachers’ perception of their TLPACK was lower during the
pandemic, particularly knowledge about the learners, which exhibited the most
significant decrease. They concluded that, just like Mourlam et al. (2021), teachers might
lose some of their confidence in their overall teaching competence due to the emergency
online learning situation (Chen & Hsu, 2021). Yet another “pre vs. in” pandemic study
was conducted by Nelson (2020), who administered a TPACK survey to preservice
teachers before the pandemic (2019) and clinical teachers in the Spring of 2020. For the
pre- to post-survey, the result showed a statistically significant increase for the clinical
teachers in the TPK domain after partaking in the Emergency Remote Learning. In
addition, both preservice and clinical teachers rated themselves low in TK self-efficacy,
which might be attributed to the insufficient technology integration in the program’s
curriculum (Nelson, 2020). Overall, it can be seen from the above-cited studies that the
effect of the pandemic on teachers’ TPACK self-efficacy has not been conclusive,
specifically on the technological and pedagogical dimensions.
In the context of Indonesia, where the present study is situated, research on the
effect of the pandemic on teachers’ TPACK competence has also been carried out. In
285
terms of language teachers, the pandemic has had varying impacts on the teachers’
competence in integrating technology, mediated by their TPACK readiness. For example,
a vocational high school English teacher in the study of Aisyah et al. (2021) has been
teaching using the Telegram Bot application for a few years, and hence the advent of the
pandemic did not constitute a major difficulty for him to switch to the full online lessons.
Consequently, this teacher demonstrated a highly-developed TPACK competence and is
a model of a successful implementation of teacher-made teaching applications. On the
other hand, another study involving 43 EFL teachers demonstrated that the teachers are
only at the median level of readiness in learning and applying technology in the classroom
due to, among others, their low self-efficacy in using technology (Surayya et al., 2021).
Teachers who were perceived to have a high level of TPACK nevertheless still fail to
leverage on technological affordances to enact sociocultural practices in their English
language teaching (Novita et al., 2022). Overall, the aforementioned studies had not
provided conclusive evidence of the impact of the pandemic on language teachers’ self-
perceived TPACK competence. In other words, to the best of our knowledge, no study
has been conducted in Indonesia pertaining to teachers’ TPACK self-rating for the onset
and during the pandemic.
Higher-Order Thinking Skills (HOTS), in turn, originated from the six cognitive
categories of Benjamin Bloom, which comprise Knowledge, Comprehension,
Application, Analysis, Synthesis, and Evaluation (Bloom et al., 1956, as cited in
Krathwohl, 2002). HOTS is constituted by the last three categories. Subsequently,
Anderson and Krathwohl proposed a revision to the concept and terminology of the
taxonomy into Remember, Understand, Apply, Analyze, Evaluate, and Create
(Krathwohl, 2002). In Indonesia, where teachers are encouraged to incorporate HOTS
into their teaching approaches and activities, it is customary to denote each verb in
Anderson and Krathwohl’s taxonomy with the letter “C” (for Cognitive) and the number
indicating their levels. Therefore, HOTS is known to be the ability to Analyze (C4),
Evaluate (C5), and Create (C6) in the corresponding subject matter.
Arising from the directive from the Indonesian Ministry of Education and Culture
to incorporate HOTS into the school curriculum, a plethora of studies have been
conducted in this respect to depict the impacts, successes, and challenges of the endeavor
(Singh & Marappan, 2020). In the field of English language learning, scholars have
examined the inclusion of HOTS in teachers’ questioning strategy (Gozali et al. , 2021;
Yulia & Budiharti, 2019), in English textbooks (Tyas et al., 2020), in cultivating reading
skills (Indriyana & Kuswandono, 2019), and in formative assessment (Rachmawati &
Purwati, 2021). Despite the sporadic success illustrated by some of the aforementioned
studies, implementing HOTS in a language classroom still meets with numerous
challenges stemming from teachers’ inadequate understanding of HOTS, the lack of
infrastructural support from the institution, and unclear government policy (Singh &
Marappan, 2020). These manifold challenges are exacerbated by the emergency remote
learning enforced by the pandemic. Teachers were unsure of how to deliver HOTS-based
lessons online (Umam et al., 2022; Widyastuti, 2022), or were compelled to adjust the
syllabus requirement with regards to HOTS incorporation (Faradella, 2022).
Notwithstanding this rather bleak outlook, the online learning involuntarily imposed by
the pandemic could also constitute an opportune moment to investigate the relationship
between HOTS integration and teachers’ technological aptitude in a language class.
However, the literature in this respect is still rather scarce.
286
Outside of the language teaching milieu, several studies have attempted to
elucidate the interplay between TPACK and HOTS. Researchers developed TPACK-
based teaching media in online physics lessons (Bakri et al., 2021; Ilmi et al., 2020;
Sinuraya et al., 2021) and science classes (Nisfah & Purwaningsih, 2022) to improve
students’ HOTS capabilities. At the same time, reinforcing teachers’ TPACK and HOTS
competence is seen to be a priority in teachers’ professional development (Mansyur et al.,
2021). Indeed, by incorporating technology in the classroom, teachers will be able to
promote students’ problem-solving and critical thinking skills, which can be equated to
HOTS (Pasani, 2018). Hence, the dearth of research examining the intertwining of
TPACK and HOTS in language teachers is rather regrettable, since technological mastery
in support of critical thinking skills should not be the purview of science and maths
teachers alone. Therefore, the present study aims to shed light on the relationship between
TPACK and HOTS in language teachers, while taking advantage of the emergency
remote learning during the pandemic in which the teachers were largely obliged to use
technology for teaching.
Method
Design
This study was designed primarily as mixed-method research, particularly as an
explanatory sequential mixed-method type, which made use of qualitative data to provide
further insight into the quantitative responses (Cresswell & Clark, 2017). Numerical data
was obtained from a survey, while the written interview provided the qualitative part of
the input. The mixed-method design was deemed the most apt for this study due to, in the
first place, the ease of collecting quantitative data from validated and modified TPACK
questionnaires. However, the third research question of this study, which seeks to delve
into the relationship between TPACK and HOTS, necessitates qualitative data, hence the
choice of mixed-method design. Lastly, since this study aims to first determine if a
perceptual difference exists and then to explain the difference, the explanatory sequential
mixed method was selected among the different mixed method designs. A simplified
diagram of the design is shown in Figure 2.
Figure 2
The Diagram of the Research Design (Explanatory Sequential Mixed-Method)
Participants
The institution where the authors are affiliated is involved in a nationwide, in-
Quan data
collection and
analysis
Qual data
collection and
analysis
Explained
by
287
service teachers certification program locally termed Program Profesi Guru or PPG.
The alumni of this PPG program were then invited to take part in this research. During
the pandemic, the PPG program was conducted fully online, and so the alumni were
likewise contacted through either personal messages or WhatsApp groups, which
were kept active to maintain contacts and disseminate news to the alumni. Out of the
hundreds of alumni spread throughout several WhatsApp groups, 137 language
teachers (English and Bahasa Indonesia) indicated their willingness to participate.
They work in various schools in Indonesia, hailing from 56 cities and 13 provinces of
the country. The profile of the participants is shown in Table 2.
Table 2
The Profile of the Participants
N
%
Teaching subjects
English
102
74.5
Bahasa Indonesia
35
25.5
Gender
Male
44
32.1
Female
93
67.9
School level
Senior high school
57
41.6
Junior high school
70
51.1
Elementary school
10
7.3
Teaching experience
(years of service)
Senior (>15 years)
31
22.6
Practitioner (10-15 years)
46
33.6
Apprentice (5-9 years)
45
32.8
Novice (<4 years)
15
10.9
Instruments
Quantitative part
For the quantitative part of the study, this research made use of a survey that has
been adapted from Rolando et al. (2021) and Zaeni et al. (2021) and has also been
subsequently shown through Structural Equation Modeling (SEM) and Confirmatory
Factor Analysis (CFA) to be valid and reliable (Tamah et al., 2022). Briefly, the
questionnaire consists of 26 self-reported statements spread across the seven TPACK
subdomains, with HOTS-related verbs (e.g., “analyze, evaluate, create,) in the question
formulation. An example of a statement under the TK (Technological Knowledge)
subdomain is “I can solve my technical problems using technology,” with the “solve”
verb being considered as at the Analyze level (C4) of the revised Bloom’s Taxonomy.
The respondents provided their answers by choosing from a 4-point Likert scale, ranging
from “Strongly Disagree” to “Strongly Agree.” The entire survey statements are
reproduced here as Appendix A.
To answer Research Question 1 (the difference between teachers’ TPACK
perception at the beginning of the pandemic and at present), all the statements were posed
in a pair-wise manner, in a ‘then and now’ fashion. In other words, for the ‘then’
statements, the participants were asked to provide the answers by thinking retrospectively
to the beginning of the pandemic, approximately in March 2020 in the case of Indonesia.
The ‘now’ context means October 2021, when the data collection was conducted. The use
288
of retrospective perception in this questionnaire is in line with Mourlam et al. (2021) who
studied the teachers’ self-perception of TPACK before and after the pandemic. Indeed,
retrospective perception has been a popular methodological choice among scholars to
gauge the impact of the pandemic on various aspects of life, such as the use of technology
(Guazzini et al., 2022), teenagers’ lifestyle and mental health (Ali et al., 2021), and
people’s psychological vulnerability (Silveira et al., 2022). Overall, the part of the
questionnaire corresponding to RQ 1 is termed Part 1.
For RQ 2 (the difference between self-perceived HOTS level at the beginning of
the pandemic and at present), Part 2 was added to the questionnaire. It simply asked the
participants to rate themselves, on a scale of 1-10, on their HOTS level both at the
beginning of the pandemic and now (October 2021).
Lastly, to answer RQ 3 (the perceived relationship between TPACK and HOTS),
the participants were asked to choose from a 4-point Likert scale to express their
agreement to the following statement: “By implementing TPACK, my HOTS competence
increases.”
Qualitative part
The demographic data collected through the survey included an option for the
participants to indicate whether or not they were willing to be interviewed after filling out
the questionnaire. Out of the 137, a total of 111 participants indicated their consent to be
interviewed. Among the 111 participants, eight were selected for the interview through
purposive sampling, namely depending on their responses to Part 1 of the survey, in line
with the explanatory sequential mixed-method design. In addition, they were also selected
based on stratified random sampling in order to achieve a balanced representation of
participants in terms of gender, teaching level, and teaching experience. The interview
took the form of a written structured interview or WSI (Whetzel et al., 2003), carried out
through WhatsApp chats. The WSI was chosen due to its ability to garner responses from
several participants simultaneously, scattered in different locations, in a relatively short
time. This is deemed necessary owing to the hectic schedule of the participants at the time
of data collection (the academic year was in full swing) and the need to seize the moment
while online classes were still in force. Although WSI suffers from the absence of non-
verbal cues and the negotiation of meaning, its reliability rests on the premise that there
is a uniform means of interview administration, and the minimization of interviewers’
bias for not having direct communication with the participants (Whetzel et al., 2003).
Besides, the interview transcript has high validity since it comes directly from the
participants themselves. Due to the nature of the research design, in which the qualitative
data is devised to provide explanatory information to the quantitative ones, the interview
questions were only drafted upon the completion of the statistical data. The complete
interview questions are presented in the Results section.
Data collection
After formulating the content, translating into Bahasa Indonesia, trying out, and
testing the validity and reliability (Tamah et al., 2022), the survey was distributed in
Google Form format through various WhatsApp groups and social media. The first part
of the Google Form contained the identity of the researchers, the research purpose, a
289
statement of confidentiality, and a consent form. The participants indicated their consent
to partake in the research by proceeding to fill out the questionnaire. Reimbursement of
internet data was given to those who completed the survey.
As has been previously explained, the interview was carried out to shed further
insights into the quantitative data analysis result. Hence, after discovering the pattern
shown by the quantitative data, eight participants were selected based on their responses,
as detailed in the Results section. They were first contacted through WhatsApp according
to the cell phone numbers indicated in the Google Form and were asked once again for
their willingness and consent to take part in the interview. The questions were given in
writing through WhatsApp chat, and they responded in a similar manner. The profile of
the interviewees is shown in Table 3.
Table 3
The Profile of the Interviewees
No
Name
(Pseudonyms)
Gender
Teaching level
Teaching
subject
Teaching
experience
1
Esther
F
Junior high school
English
Apprentice
2
Andy
M
Senior high school
English
Apprentice
3
Diana
F
Senior high school
English
Senior
4
Elsie
F
Senior high school
English
Apprentice
5
Danny
M
Junior high school
English
Apprentice
6
Ivan
M
Elementary school
English
Practitioner
7
Sally
F
Junior high school
English
Practitioner
8
Mary
F
Senior high school
English
Apprentice
Data analysis
For the descriptive statistics, Microsoft Excel was used to analyze the data, while
IBM SPSS 22 was employed for the inferential statistics.
For RQ 1, the scores for each TPACK subdomain were first aggregated to produce
the average value for the ‘then’ and ‘now’. Then, descriptive statistics were used to find
the means for each of the composite values of the subdomains, both in the ‘then’ and
‘now’ contexts. The highest and lowest values of the means were then identified. Other
data obtained from the descriptive statistics include the skewness values, which revealed
that the data in this Part 1 were only moderately skewed (the values ranging from -0.812
to 0.387). This, together with the fact that n>30, warrants the assumption that the data is
normally distributed. Hence, the independent, two-sample t-test was used to check for
significant differences between the ‘then’ and ‘now’ values for each TPACK subdomain.
Cohen’s d was also calculated with the help of Microsoft Excel to provide alternatives to
the result of the t-test.
Likewise, the data in Part 2 on the different perceptions of HOTS’ level at the
beginning and ‘now’ were quantified and analyzed statistically. Here, it was also assumed
that the data was normally distributed based on the skewness values (from -0.277 to
1.000) and the number of participants (n>30). The independent, two-sample t-test was
thus performed to determine whether there are any significant differences between the
self-rated HOTS level of the participants at the beginning of the pandemic and 20 months
after. Similarly, Cohen’s d was also computed. Part 3 of the data was only analyzed using
290
descriptive statistics.
The qualitative information from the interview was first translated into English,
tabulated, and analyzed for themes and patterns using simple code formatting (Miles et
al., 2014). The emerging themes were then summarized and displayed.
Findings
Research question 1
The first research question is the quest to discover if there is any difference
between teachers’ self-perceived TPACK competence, encompassing all the individual
TPACK subdomains, at the beginning of the pandemic and at the point when the data was
collected (October 2021). To this end, the survey result summary showing the descriptive
statistics of the responses is given in Table 4.
Table 4
The Summary of the Descriptive Statistics of the TPACK-HOTS Survey Result at the
Beginning of the Pandemic (March 2020) and Now (October 2021)
No
Domains
Mar-20
Oct-21
M1
SD1
M2
SD2
M2 - M1
1
TK
2.562
0.605
3.119
0.481
0.557
2
PK
2.860
0.643
3.197
0.558
0.337
3
CK
2.856
0.554
3.056
0.554
0.200
4
TCK
2.776
0.539
3.148
0.468
0.372
5
PCK
2.839
0.511
3.097
0.473
0.258
6
TPK
2.838
0.521
3.170
0.433
0.332
7
TPACK
2.759
0.559
3.222
0.470
0.463
Firstly, assuming a mid-point value of 2.5 on the 4-point Likert scale, it can be
seen that the participants rated themselves as above average in all the TPACK domains,
as all the mean values were above 2.5. Next, at the beginning of the pandemic, namely
March 2020, the highest mean was shown for the PK (Pedagogical Knowledge) domain
(M=2.860) and the lowest for TK (Technological Knowledge) (M=2.562). On the other
hand, circa October 2021, the highest value of the means was obtained for the TPACK
domain (M=3.222) and the lowest for CK (Content Knowledge) (M=3.056).
Looking at the column showing the difference between the two means (M2-M1),
positive figures indicate an increase in all domains. In other words, the participants
perceived that they have, on average, improved in all TPACK aspects during the
pandemic. The biggest increase is indicated for the TK (Technological Knowledge)
domain (M2-M1=0.557) and the lowest for CK (Content Knowledge) (M2-M1=0.200).
The summary of the t-test and Cohen’s d computation result to check the
significant difference is given in Table 5.
291
Table 5
The t-test and Cohen’s d Result for the Difference between the Beginning of the
Pandemic (March 2020) and Now (October 2021)
No
Domains
n
M
SD
Sig (2-tailed)
Cohen's d
1
TK
Mar-20
137
2.562
0.605
0.000
1.020
Oct-21
137
3.119
0.481
2
PK
Mar-20
137
2.860
0.643
0.000
0.560
Oct-21
137
3.197
0.558
3
CK
Mar-20
137
2.856
0.554
0.000
0.360
Oct-21
137
3.056
0.554
4
TCK
Mar-20
137
2.776
0.539
0.000
0.737
Oct-21
137
3.148
0.468
5
PCK
Mar-20
137
2.839
0.511
0.000
0.523
Oct-21
137
3.148
0.473
6
TPK
Mar-20
137
2.838
0.521
0.000
0.693
Oct-21
137
3.097
0.433
7
TPACK
Mar-20
137
2.759
0.559
0.000
0.896
Oct-21
137
3.222
0.470
The inferential statistics revealed that there existed a significant difference in all
the TPACK subdomains (p<0.05), from March 2020 to October 2021. Besides, the effect
sizes also fell in the range of ‘high’ for TK and TPACK domains (both > 0.8) and
‘medium’ for the remaining subdomains (all > 0.5) following Cohen’s standard (Cohen,
1988).
Upon obtaining the findings as revealed by the quantitative data as previously
mentioned, the interview questions were formulated to gain a deeper insight into the
reason behind the numerical results. Hence, the first question posed to the interviewees
is as follows: “(1) In the survey, you have indicated a substantial increase in your
Technological Knowledge (TK), since the beginning of the distance learning up to now.
Could you explain why?” The replies from the respondents were analyzed for common
or interesting themes. Invariably, almost all interviewees implied that an increment in
technological know-how is inevitable in the online learning situation due to the need to
engage and motivate learners to study online, or as fruits of institutionalized training.
Excerpts from two of the interviewees illustrate this point (the responses are quoted as
they were written):
“With distant learning, teachers have to use technology, like it or not. Students
lack motivation to study from home, so teachers have to learn to use interesting
teaching media.” (Sally, practitioner, English teacher in a junior high school)
“During the pandemic, I was invited to attend the Program Profesi Guru
(Teacher Certification Program), which was held fully online. From there, I
learned about online learning media that I can apply to my students.” (Danny,
apprentice, English teacher in a junior high school)
The second question asked to the participants was, “(2) In the survey, you have
indicated that there is not much increase in the Content Knowledge (CK) during the
pandemic. Could you explain why?” Here, one respondent attributed it to the need to
292
focus on improving technological skills, the status quo condition in the syllabus mandated
by the government, and another to the lack of interaction during online learning, as shown
by the excerpts:
“During the pandemic, teachers have to adapt more in terms of the teaching
medium and delivery rather than the object, so the object or Content
Knowledge, in this case English, is still the same.” (Andy, apprentice, English
teacher in a senior high school)
“The syllabus mandated by the government during distant learning didn’t
change, but the teachers had to adjust the method of delivery,” (Mary,
apprentice, English teacher in a senior high school).
“Face-to-face meetings make it easier for me to improve my Content
Knowledge (CK). Students’ responses, in terms of content and motivation, to
the material that I delivered, tend to be static [in online classes] because of one-
way communication. Critical questions or students’ difficulties in
understanding the material [in offline classes] spur me to improve in my CK.”
(Esther, apprentice, English teacher in a junior high school)
Thirdly, the interviewees were asked for the reason behind their high self-rating
for the TPACK subdomain 20 months into the Emergency Remote Learning situation
with the following question: “(3) In the survey, you indicated high agreement for
statements related to TPACK subdomain. Could you explain why?” Here, the participants
credited their high confidence in their TPACK rating either to schools’ support, self-
motivation, or the increase in the enthusiasm and engagement of their students. Extracts
from their replies are given:
“[My school] implemented various platforms at the beginning of distance
learning, such as Quipper, Quizziz, and Google Classroom. At first, it wasn’t
easy. However, slowly the teachers honed their skills and now they are used to
it.” (Andy, apprentice, English teacher in a senior high school).
“Due to the demand to be innovative and creative in using technology for
teaching, it made me learn on my own so I can master various applications
such as making teaching videos and using gamification-based interactive
media.” (Esther, apprentice, English teacher in a junior high school).
“I crafted my lesson in Google Jamboard in such a way that the students forgot
that they are learning school materials. I can see the enthusiasm of the students
in actively participating in online classes.” (Danny, apprentice, English teacher
in a junior high school).
Research question 2
RQ2 aimed to determine if a difference existed between teachers’ self-perceived
HOTS levels at the beginning of the pandemic (March 2020) and at the moment of data
collection (October 2021). In the survey, the participants were asked to rate themselves
on a scale of 1-10, both in the ‘then’ and ‘now’ time frame.
Similar to RQ1, the descriptive statistics and the significant difference test were
computed, the result of which is presented in Tables 6 and 7, respectively.
293
24%
74%
2%0%
"Implementing TPACK increases my HOTS
competence"
Strongly Agree
Agree
Disagree
Strongly disagree
Table 6
The Descriptive Statistics of Self-Rated HOTS Level at the Beginning of the Pandemic
(March 2020) and Now (October 2021)
Mar-20
Oct-21
M1
SD1
M2
SD2
HOTS rating
6.270
1.704
7.533
1.485
Table 7
The t-value and the Cohen’s d for the Difference between the Self-Rated HOTS Level at
the Beginning of the Pandemic (March 2020) and Now (October 2021)
n
M
SD
Sig (2-tailed)
Cohen's d
HOTS rating
Mar 2020
137
6.270
1.704
0.00
0.790
Oct 2021
137
7.533
1.485
The preceding tables show that there is an increment in the teachers’ self-rating
of HOTS capability in the 20 months into the pandemic, as evidenced by the positive rise
in the means from March 2020 to October 2021 shown in Table 6. It is further confirmed
by Table 7 that this rise is statistically significant (p < 0.05). The effect size as presented
by Cohen's d also falls under the ‘medium’ category (> 0.5) by Cohen’s standard.
Research question 3
The third and the last RQ sought to discover the perceived relationship, if any,
between TPACK and HOTS competence according to the language teachers. Here, the
participants were asked to indicate their agreement in the form of a 4-point Likert scale
to a statement that says, “By implementing TPACK, my HOTS competence increases.”
A summary of their responses to this statement is displayed in Figure 3.
Figure 3
The Summary of the Participants’ Responses to the TPACK-HOTS Relationship
Statement
294
The pie chart shows that an overwhelming majority of the participants (74%)
chose Agree, 24% selected Strongly Agree, an almost negligible number chose Disagree,
while none answered Strongly Disagree.
The eight participants interviewed were asked for the reason behind their strong
agreement to the relationship between TPACK and HOTS in teachers. Almost all of the
teachers responded to the effect that teaching with technology facilitates them in making
HOTS-based tasks, exercises, and assessments. The following excerpts illustrate this
point:
“In my opinion, TPACK is related to HOTS, or to be exact, it facilitates the
implementation of HOTS-based lessons. For example, in HOTS-based English
lessons, students are expected to be able to analyze, compare, and categorize
verbs according to the tenses. I used Jamboard which I designed in such a way
that students can do it in a collaborative manner.” (Ivan, practitioner, English
teacher in an elementary school).
As a corollary, teachers’ TPACK competence is also seen to promote HOTS in
the students, as affirmed by an interviewee:
“The more creative the teachers are in presenting lessons that engage students,
the higher the students’ motivation will be to explore their thinking skills, in
accordance with the use of technology in the 21st century.” (Elsie, apprentice,
English teacher in a senior high school).
Discussion
Employing a validated TPACK-HOTS questionnaire, this study set out to find out
first whether there is a change in the language teachers’ perception of their TPACK
competence from the beginning of the pandemic in Indonesia (circa March 2020) to the
moment when the data was collected (October 2021). From the 137 survey forms
analyzed, the quantitative data analysis showed a statistically significant increase in all
the TPACK domains, particularly in those of TK and TPACK. This result differs from
those reported by Chen and Hsu (2021), Mourlam (2021), and Nelson (2020), whose
results presented a decline in some domains. One of the possible reasons might simply be
the time frame when the data was collected. The above-cited studies collected data in
2020, when the pandemic was still in full swing, and a relatively shorter time (about a
few months) has elapsed between the beginning of the outbreak and the data collection.
This study, on the other hand, was conducted around 20 months after the start of the
pandemic in Indonesia. Therefore, as attested by one of the interviewees, at this juncture,
the teachers have “honed their [technological] skill and are now used to [technology].”
Hence, they perceived that, overall, their TPACK competence had somewhat improved
during the pandemic. The outcome of this study is also in line with that of Huang et al.
(2022), who provided statistical evidence on the mediating role of TPACK on teachers’
increased self-efficacy during the pandemic.
Another possible reason for the significant increase of all TPACK domains in this
study is the profile of the participants being surveyed. Most of the teachers who responded
to the call to participate in the research were alumni of the Pendidikan Profesi Guru
(“PPG”) or Teacher Certification Program course held by the university where the
295
researchers are affiliated with. Improving in-service teachers’ TPACK competence has
been one of the focuses of the PPG training (Apriliaswati, 2020; Hartarti et al., 2019).
During the pandemic, this short-term (approximately four months) course was conducted
entirely online, with the in-service teachers being trained and assessed on how they
employ technology in their teaching. It is then of little wonder that many of them
perceived an increase in their TPACK competence.
A potentially interesting finding is the fact that the teachers did not indicate a
marked improvement in their Content Knowledge (CK), which in this case is English
language skills, during the pandemic. One would expect that the flourishing of free, online
webinars in recent times should have facilitated the acquisition of CK for teachers, but it
is not the case. The reasons put forward by the interviewees were quite revealing. On the
one hand, teachers did not seem to put too much premium on increasing their CK during
the Emergency Remote Learning and focused their attention on technological know-how
instead. While this might be understandable and sensible, one wonders if the emphasis
on TK is done at the expense of other types of knowledge. On the other hand, it was also
interesting that for some teachers, CK is seen to be a function of students’ critical inquiry,
as well as the government's mandate on the emergency curriculum. This way of acting
appears to imply a lack of internal motivation in the teachers to improve their CK.
Although research on CK per se attracted lesser attention on the part of scholars
these days, it can be imagined that knowledge of subject matter for teachers is
fundamental. Some studies documented a positive relationship between teachers’ subject
knowledge and students’ achievement (Metzler & Woessmann, 2012; Shepherd, 2013).
Teachers’ CK has also been shown to have a direct impact on their ability to teach the
relevant points and to make clarifications (Jadama, 2014). During the pandemic, some
EFL students even admitted that the teachers’ content knowledge matters more to them
than the teachers’ technological competence (Gozali & Cahyono, 2022). In fact,
Renandya et al. (2018) had voiced their concerns regarding the English proficiency of
Indonesian EFL teachers, which directly impacted the effectiveness of their lesson
delivery. Hence, considering the importance of continuously developing the CK
competence in the teachers, further research can be done to verify whether the pandemic
has had a detrimental effect on teachers’ CK in overall professional development.
Secondly, this study was also aimed at finding whether the language teachers
perceived a change in their level of HOTS from the beginning of the pandemic to twenty
months later. The quantitative data revealed a statistically significant increase in the
participants’ self-perception of HOTS levels. It is interesting to contrast this finding with
several studies that documented the challenges faced by teachers in implementing HOTS-
based lessons or assessments to students during the pandemic (Lie et al., 2020; Ritonga
et al., 2021; Sirait et al., 2021). The difficulties in implementing HOTS range from the
lack of support from schools, students’ mixed ability, or even teachers’ own (deficient)
understanding of HOTS.
So, why did the teachers perceive a rise in their HOTS level during the pandemic?
The answer lies in the data analyzed under Research Question 3. When asked whether
there is a relationship between TPACK and HOTS, a significant majority (74%) voted
for “Agree.” The qualitative data reveal that teachers perceived their improvement of
TPACK skills to be in tandem with that of HOTS. In other words, TPACK is seen to
empower teachers by facilitating the affordances of technologically-based HOTS media,
materials, and teaching approaches. The perception of the language teachers is thus
296
aligned with the previously cited studies outside of the language teaching field evidencing
the success of incorporating technology in the teaching and learning process, with the aim
of improving the students’ HOTS (Bakri et al., 2021; Ilmi et al., 2020; Pasani, 2018;
Zainuddin et al., 2021). In sum, despite the possible challenges facing the teachers in their
effort to inculcate HOTS in their students during the pandemic, they still perceived that
their HOTS competence has improved, leveraged by their increased confidence in
employing technology in the classroom. Therefore, even when classes revert to fully
“offline”or face-to-face when the pandemic is over, the increased confidence of the
teachers in technological competence as shown in this study constitutes a powerful
arsenal not just for embedding technology in the classroom, but also to promote critical
thinking, as embodied by HOTS, in the students. We then believe that this study, albeit
conducted within the context of the COVID-19 pandemic, offers meaningful insight into
the post-pandemic education milieu in demonstrating both the gain in teachers’ self-rated
competence in technological integration and its implication on the teachers’ perception
of their ability to enact HOTS-based lessons.
Conclusion
In the quest to find whether there are differences in the language teachers’ self-
rated level of TPACK and HOTS throughout the 20 months of the pandemic, as well as
the perceived relationship between TPACK and HOTS, this study has unraveled three
outcomes. Firstly, the teachers rated themselves higher on average in all of the TPACK
domains, from the beginning of the pandemic (March 2020) until the data collection
moment (October 2021). The largest increment, as measured by the means difference
between “then” and “now,” was noted in the TK subdomain, and the highest value for the
means for the “now” time frame was in the TPACK subdomain. Overall, this quantitative
data and the subsequent theme that emerged from interview data analysis point to the
conclusion that the teachers in this study felt somewhat confident of their technological
capability in teaching after 20 months of Emergency Remote Learning had elapsed.
However, the relatively smaller rise in the mean difference of the CK subdomain may
imply that teachers focused on technology at the expense of their subject matter
knowledge. Secondly, this study also revealed that teachers also perceived their HOTS
competence level to increase in a statistically-significant way during the pandemic. Lastly,
the vast majority of the teachers concurred that implementing TPACK results in an
increase in HOTS capability, which indirectly is the reason for their high self-rating for
HOTS after 20 months of Emergency Remote Learning.
This study thus would like to recommend that, taking advantage of the teachers’
familiarity with technology for pedagogy at the present moment, a more holistic
professional development for teachers should be designed, in which this technological
know-how is exploited to promote teachers’ subject matter mastery. As mentioned earlier,
more systematic research can be conducted to investigate the impact of the pandemic on
teachers’ level of Content Knowledge. Lastly, in view of the possibility of having a
blended or hybrid learning mode when the pandemic is better managed, teachers should
not rest on their laurels and continue to make use of their acquired technological
knowledge to impart HOTS-infused media and assessment in the new setting.
Health protocol and time constraints have impeded the researchers from
297
conducting face-to-face interviews, and even a teleconference, which would have yielded
richer qualitative data. The pooling of the participants could have been made more
comprehensive so as to reach a more diverse type of participants. Despite the limitations,
it is hoped that this study has made a contribution to the burgeoning literature on TPACK,
specifically on TPACK-HOTS, which is highly applicable to teachers’ professional
development in Indonesia.
Acknowledgement
A part of this paper has been assisted by the researchers’ colleague who
contributed a part of the quantitative data analysis. The research has been funded by
Widya Mandala Surabaya University Research Institute. The authors are therefore
grateful to their colleague, Ir. Lusia Permata Sari Hartanti, S.T., M. Eng., IPM. and also
Dr. Hartono Pranjoto, who chairs the Research Institute.
References
Aisyah, R. N., & Munir, A. (2021). Technological Pedagogical Content Knowledge
(TPACK) in action: Unraveling Indonesian English as a foreign language
teachers’ TPACK by implementing Telegram. Computer Assisted Language
Learning Electronic Journal (CALL-EJ), 22(3), 17-32. http://callej.org/journal/
22-3/Aisyah-Setiawan-Munir2021.pdf
Ali, A., Siddiqui, A. A., Arshad, M. S., Iqbal, F., & Arif, T. B. (2022, June). Effects
of COVID-19 pandemic and lockdown on lifestyle and mental health of
students: A retrospective study from Karachi, Pakistan. In Annales Médico-
psychologiques, revue psychiatrique (Vol. 180, No. 6, pp. S29-S37). Elsevier
Masson. https://doi.org/10.1016/j.amp.2021.02.004
Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new
millennium learners in OECD countries.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching,
and assessing. Longman New York
Apriliaswati, R. (2020, August). Dispositions of in-service English teachers towards
developing pedagogical competence by technology literacy. In UICELL
Conference Proceeding (pp. 290-301). https://journal.uhamka.ac.id/uicell/
article/view/5412
Arcueno, G., Arga, H., Manalili, T. A., & Garcia, J. A. (2021, March). TPACK and
ERT: Understanding teacher decisions and challenges with integrating
technology in planning lessons and instructions. In DLSU Research Congress
Bakri, F., Kusuma, R., & Permana, A. H. (2021, October). TPACK and augmented
reality in kinematics practicum module: Forming HOTS Physics education
students. In Journal of Physics: Conference Series (Vol. 2019, No. 1, p.
012041). IOP Publishing. https://iopscience.iop.org/article/10.1088/1742-
6596/2019/1/012041/pdf
Bostancıoğlu, A., & Handley, Z. (2018). Developing and validating a questionnaire
298
for evaluating the EFL ‘Total PACKage’: Technological Pedagogical Content
Knowledge (TPACK) for English as a Foreign Language (EFL). Computer
Assisted Language Learning, 31(5-6), 572-598. https://doi.org/10.1080/
09588221.2017.1422524
Castéra, J., Marre, C. C., Yok, M. C. K., Sherab, K., Impedovo, M. A., Sarapuu,
T., ... & Armand, H. (2020). Self-reported TPACK of teacher educators across
six countries in Asia and Europe. Education and Information Technologies,
25(4), 3003-3019. https://doi.org/10.1007/s10639-020-10106-6
Chai, C. S., Koh, J. H. L., & Tsai, C.-C. (2010). Facilitating preservice teachers'
development of Technological, Pedagogical, and Content Knowledge
(TPACK). Educational Technology & Society, 13 (4), 63–73.
http://www.jstor.org/stable/jeductechsoci.13.4.63
Chen, Y.-J., Hsu, R.L.-W. (2021). Understanding the difference of teachers’
TLPACK before and during the COVID-19 pandemic: Evidence from two
groups of teachers. Sustainability, 13(16), 8827. https://doi.org/10.3390/
su13168827
Cheng, K. H. (2017). A survey of native language teachers’ technological
pedagogical and content knowledge (TPACK) in Taiwan. Computer Assisted
Language Learning, 30(7), 692-708. https://doi.org/10.1080/09588221.2017.
1349805
Cohen, J. (1988). Statistical power analysis for the behavioral sciences, 2nd ed.
Lawrence Erlbaum.
Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods
research. Sage publications.
Efwinda, S., & Mannan, M. N. (2021, March). Technological Pedagogical and
Content Knowledge (TPACK) of prospective Physics teachers in distance
learning: Self-perception and video observation. In Journal of Physics:
Conference Series (Vol. 1806, No. 1, p. 012040). IOP Publishing.
Ersanli, C. Y. (2016). Improving Technological Pedagogical Content Knowledge
(TPACK) of pre-service English language teachers. International Education
Studies, 9(5), 18-27. https://doi.org/10.5539/ies.v9n5p18
Faradella, N. E. (2022). The implementation of Higher-Order Thinking Skills
(HOTS) in teaching English process during online learning. Erudita: Journal
of English Language Teaching, 2(1). https://doi.org/10.28918/erudita.v2i1.
5350
Gozali, I., & Cahyono, B. Y. (2022) Studentsʼ perspective on the importance of EFL
teachersʼ TPACK (Technological Pedagogical Content Knowledge) and XK
(Contextual Knowledge) for learning English during the pandemic. PASAA, 64,
244‒277. https://www.culi.chula.ac.th/publicationsonline/files/article/
MFbAYqOYcfWed23413.%20Gozali_Jan%2030.pdf
Gozali, I., Lie, A., Tamah, S. M., & Jemadi, F. (2021). HOTS questioning ability
and HOTS perception of language teachers in Indonesia. Indonesian Journal of
Applied Linguistics, 11(1), 60-71. https://doi.org/10.17509/ijal.v11i1.34583
Guazzini, A., Pesce, A., Gino, F., Duradoni, M. (2022). How the COVID-19
pandemic changed adolescents’ use of technologies, sense of community, and
loneliness: A retrospective perception analysis. Behavioral Sciences, 12(7), 228.
https://doi.org/10.3390/bs12070228
299
Habibi, A., Yusop, F. D., & Razak, R. A. (2020). The role of TPACK in affecting
pre-service language teachers’ ICT integration during teaching practices:
Indonesian context. Education and Information Technologies, 25(3), 1929-
1949. https://doi.org/10.1007/s10639-019-10040-2
Hartati, T., Heryanto, D., Annisa, N., Nuriyanti, R., Saputra, A. H., & Sutedi, R.
(2019). Higher Order Thinking Skill and literacy base Technological
Pedagogical Content Knowledge (TPACK) in order to improve students
learning quality of in job primary school teacher’s professional educational
program. In International Journal of Science and Applied Science: Conference
Series (Vol. 3, No. 1, pp. 136-141). https://doi.org/10.20961/ijsascs.v3i1.32552
Hsu, L., & Chen, Y. J. (2019). Examining teachers’ technological pedagogical and
content knowledge in the era of cloud pedagogy. South African Journal of
Education, 39. https://doi.org/10.15700/saje.v39ns2a1572
Huang, S., Yin, H., Jin, Y., & Wang, W. (2022). More knowledge, more satisfaction
with online teaching? Examining the mediation of teacher efficacy and
moderation of engagement during COVID-19. Sustainability, 14(8), 4405.
https://doi.org/10.3390/su14084405
Ilmi, A. M., & Sunarno, W. (2020). Development of TPACK based-physics learning
media to improve HOTS and scientific attitude. In Journal of Physics:
Conference Series (Vol. 1440, No. 1, p. 012049). IOP Publishing.
Indriyana, B. S., & Kuswandono, P. (2019). Developing students' Higher Order
Thinking Skills (HOTS) in reading: English teachers' strategies in selected
junior high schools. Journal of English Teaching, 5(3), 204-216.
https://doi.org/10.33541/jet.v5i3.1313
Jadama, L. M. (2014). Impact of subject matter knowledge of a teacher in teaching
and learning process. Middle Eastern & African Journal of Educational
Research, 7(1), 20-29. https://d1wqtxts1xzle7.cloudfront.net/33405606/
majerissue7-libre.pdf
Kholik, A., Yektyastuti, R., Mawardini, A., Hamamy, F., Sudjani, D. H., Ramdhani,
M. R., & Gunadi, G. (2020). Analysis of Technological Pedagogical Content
Knowledge (TPACK) on Indonesian certified teacher. PalArch's Journal of
Archaeology of Egypt/Egyptology, 17(6), 8634-8642. https://archives.palarch.
nl/index.php/jae/article/view/2269
Koehler, M., & Mishra, P. (2009). What is technological pedagogical content
knowledge (TPACK)? Contemporary issues in technology and teacher
education, 9(1), 60-70. https://citejournal.org/volume-9/issue-1-09/general/
what-is-technological-pedagogicalcontent-knowledge/
Krathwohl, D. R. (2002). A revision of bloom’s taxonomy: An overview. Theory
into Practice, 41(4), 212-218. https://doi.org/10.1207/s15430421tip4104_2
Lie, A., Tamah, S. M., Gozali, I., Triwidayati, K. R., & Utami, T. S. D. (2020).
Secondary school language teachers’ online learning engagement during the
COVID-19 pandemic in Indonesia. Journal of Information Technology
Education: Research, 19, 803-832. https://doi.org/10.28945/4626
Mansyur, A., Siregar, T. M., & Oktora, M. (2021). Analysis of teacher professional
development to improve application of TPACK, HOTS, and 21st Century life
skills stimulation concepts. ICE-TPD.
Metzler, J., & Woessmann, L. (2012). The impact of teacher subject knowledge on
300
student achievement: Evidence from within-teacher within-student variation.
Journal of Development Economics, 99(2), 486-496. https://doi.org/10.2139/
ssrn.1634795
Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis: A
methods sourcebook. Sage.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge:
A framework for teacher knowledge. Teachers College Record, 108(6), 1017-
1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x
Mourlam, D., Chesnut, S., Strouse, G., DeCino, D., Los, R., & Newland, L. (2021,
March). Did they forget? Understanding teacher TPACK during the COVID-
19 pandemic. In Society for Information Technology & Teacher Education
International Conference (pp. 1616-1622). Association for the Advancement
of Computing in Education (AACE).
Nelson, R. (2020). Examining preservice teacher technology development during
the COVID-19 pandemic [Doctoral dissertation, The University of Texas at San
Antonio]. https://www.proquest.com/openview/bca82523e76b51345352
e5da98143c17/1?pq-origsite=gscholar&cbl=44156
Nisfah, N., & Purwaningsih, E. (2022). Optimization of collaborative inquiry
integrated into TPACK to improve higher-order thinking skills. Jurnal
Kependidikan, 6(1). doi:https://doi.org/10.21831/jk.v6i1.36440
Novita, D., Purwati, O., Anam, S. (2022). In-service EFL teachers’ sociocultural-
based TPACK beliefs and practices: Voice of teachers and students. Computer
Assisted Language Learning Electronic Journal (CALL-EJ), 23(1), 278-293.
http://callej.org/journal/23-1/Novita-Purwati-Anam2022.pdf
Pasani, C. F. (2020). TPACK untuk mengembangkan HOTS dan berbagai literasi.
In Senpika I. Banjarmasin.
Rachmawati, D. L., & Purwati, O. (2021). EFL teachers’ attitudes and competence
in developing HOTS-based formative assessment. JEES (Journal of English
Educators Society), 6(2). https://doi.org/10.21070/jees.v6i2.1060
Raygan, A., & Moradkhani, S. (2022). Factors influencing technology integration in
an EFL context: investigating EFL teachers’ attitudes, TPACK level, and
educational climate. Computer Assisted Language Learning, 35(8), 1789-1810.
https://doi.org/10.1080/09588221.2020.1839106
Renandya, W. A., Hamied, F. A., & Joko, N. (2018). English language proficiency
in Indonesia : Issues and prospects. The Journal of AsiaTEFL, 15(3), 618–629.
https://doi.org/10.18823/asiatefl.2018.15.3.4.618
Ritonga, A. W., Wargadinata, W., Hasan, N., & Ahmad, B. M. B. (2021). Teacher’s
challenges in implementing HOTS in learning Arabic during Covid-19
pandemic. Izdihar: Journal of Arabic Language Teaching, Linguistics, and
Literature, 4(1), 1-14. https://doi.org/10.22219/jiz.v4i1.15606
Rolando, L. G. R., Salvador, D. F., Vasconcellos, R. F. R. R., & Luz, M. R. M. P.
D. (2021). TPACK for meaningful learning survey: "Paths" for professional
development of biology teachers in Brazil. TOJET: The Turkish Online Journal
of Educational Technology, 20(2), 168-181. https://eric.ed.gov/?id=EJ1304887
Roussinos, D., & Jimoyiannis, A. (2019). Examining primary education teachers’
perceptions of TPACK and the related educational context factors. Journal of
Research on Technology in Education, 51(4), 377-397. https://doi.org/10.1080/
301
15391523.2019.1666323
Schmidt, D. A., Baran, E., Thompson, A. D., Koehler, M. J., Mishra, P., & Shin, T.
(2009). Survey of preservice teachers' knowledge of teaching and technology.
Récupéré le, 2, 1-8. https://news.cehd.umn.edu/wp-content/uploads/2009/06/
TPCK_survey.pdf
Scott, K. C. (2021). A review of faculty self-assessment TPACK instruments
(January 2006–March 2020). International Journal of Information and
Communication Technology Education (IJICTE), 17(2), 118-137.
https://www.igi-global.com/article/a-review-of-faculty-self-assessment-tpack-
instruments-january-2006--march-2020/268777
Shepherd, D. L. (2013, June). The impact of teacher subject knowledge on learner
performance in South Africa: A within-pupil across-subject approach. In
International Workshop on Applied Economics of Education, Cantanzaro (pp.
1-32). https://doi.org/10.2139/ssrn.1634795
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching.
Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189x
015002004
Silveira, S., Hecht, M., Matthaeus, H., Adli, M., Voelkle, M. C., & Singer, T. (2022).
Coping with the COVID-19 pandemic: Perceived changes in psychological
vulnerability, resilience and social cohesion before, during and after lockdown.
International Journal of Environmental Research and Public Health, 19(6),
3290. https://doi.org/10.3390/ijerph19063290
Singh, C. K. S., & Marappan, P. (2020). A review of research on the importance of
higher order thinking skills (HOTS) in teaching english language. Journal of
Critical Reviews, 7(8), 740-747. https://doi.org/10.31838/jcr.07.08.161
Sinuraya, J., Motlan, M., & Mihardi, S. (2021, November). Need analysis of TPACK
oriented of ICARE based student worksheet for improving students' ability to
complete HOTS questions. In Proceedings of the 3rd International Conference
on Innovation in Education, Science and Culture, ICIESC 2021, 31 August
2021, Medan, North Sumatera Province, Indonesia. http://dx.doi.org/10.4108/
eai.31-8-2021.2313804
Sirait, S., Murniarti, E., & Sihotang, H. (2021). Implementation of HOTS-based
learning and problem based learning during the pandemic of COVID-19 in
SMA Budi Mulia Jakarta. Advances in Social Sciences Research Journal, 8(2),
296-305. https://doi.org/10.14738/assrj.82.9727
Suhardi, D. (2018). Peraturan Sekretaris Jenderal Kementerian Pendidikan dan
Kebudayaan tentang Pedoman Pelatihan Kurikulum 2013 bagi Guru dan
Tenaga Kependidikan Tahun 2018. The Indonesian Ministry of Education and
Culture.
https://repositori.kemdikbud.go.id/6283/1/Pedoman%20Umum%20PKB%202
018%20%28datadikdasmen.pdf
Surayya, S. A., Asrobi, M., & Farizi, Z. (2021). Scrutiny on Indonesian EFL teachers’
willingness to understand and implement the TPACK in VLE. Journal of
Education Technology, 5(4), 645-653. https://doi.org/10.23887/jet.v5i4.40986
Tamah, S. M., Lie, A., Gozali, I., & Hartanti, L. P. S. (2022). HOTS-oriented
TPACK survey validation. Pedagogika, 148(4), 184-206. https://doi.org/10.
15823/p.2022.148.10
302
Tseng, J. J., Chai, C. S., Tan, L., & Park, M. (2020). A critical review of research on
technological pedagogical and content knowledge (TPACK) in language
teaching. Computer Assisted Language Learning, 1-24. https://doi.org/
10.1080/09588221.2020.1868531
Tyas, M. A., Nurkamto, J., & Marmanto, S. (2020). Cultivating students' Higher-
Order Thinking Skills in EFL classes: The role of the teacher and the textbook.
International Online Journal of Education and Teaching, 7(1), 267-276.
https://eric.ed.gov/?id=EJ1244250
Umam, A., Rahmawanti, M. R., & Jaelani, A. (2022). EFL teachers’ problems in
HOTS implementation during COVID pandemic. Journal of English Education
and Teaching, 6(4), 489-497. https://doi.org/10.33369/jeet.6.4.489-497
van der Spoel, I., Noroozi, O., Schuurink, E., & van Ginkel, S. (2020). Teachers’
online teaching expectations and experiences during the Covid19-pandemic in
the Netherlands. European Journal of Teacher Education, 43(4), 623-638.
https://doi.org/10.1080/02619768.2020.1821185
Wen, K. Y. K., & Kim Hua, T. (2020). ESL teachers' intention in adopting online
educational technologies during COVID-19 pandemic. Journal of Education
and E-Learning Research, 7(4), 387-394. https://doi.org/10.20448/journal.509.
2020.74.387.394
Whetzel, D. L., Baranowski, L. E., Petro, J. M., Curtin, P. J., & Fisher, J. L. (2003).
A written structured interview by any other name is still a selection instrument.
Applied HRM Research, 8(1), 1-16. http://applyhrm.asp.radford.edu/2003/
MS%208_1_%20Whetzel.pdf
Widyastuti, N. W. S. (2022). The implementation of HOTS-based English learning
in junior high school during online classes. Journal of Educational Study, 2(3),
215-225. https://doi.org/10.36663/joes.v2i3.381
Yulia, Y., & Budiharti, F. R. (2019). HOTS in teacher classroom interaction: A case
study. EduLite: Journal of English Education, Literature and Culture, 4(2),
132-141. https://doi.org/10.30659/e.4.2.132-141
Zaeni, A., Rahayu, W, & Makmuri (2021). Pengembangan instrumen self-
assessment technological pedagogical content knowledge (TPACK) calon guru
matematika berbasis HOTS. Teorema: Teori dan Riset Matematika, 5(2), 59-
68. https://jurnal.unigal.ac.id/index.php/teorema/article/view/4960.
https://doi.org/10.25157/teorema.v6i1.4960
Zainuddin, M., Kharis, M., Waluyo, B., & Nahdiyah, U. (2021). Integrating TPACK
based HOTS-Textbooks: A case study to attest teaching style in primary school.
Review of International Geographical Education Online, 11(5), 3662-3670.
https://doi.org/10.48047/rigeo.11.03
Appendix A
The TPACK-HOTS Integrated Questionnaire
A
Technological Knowledge
1
I can learn and use new technology easily.
(C3 Apply)
2
I can solve my technical problems using technology.
(C5 Evaluate)
303
3
I can be creative when using technology.
(C6 Create)
B
Pedagogical Knowledge
1
I can choose learning strategies according to the needs
and conditions of students
(C5 Evaluate)
2
I can manage/organize the steps of the learning
method to make it easier for students to understand the
material.
(C6 Create)
3
I can manage/organize the class so that students don’t
get bored in learning.
(C6 Create)
4
I can choose the correct form of assessment according
to the characteristics of the material.
(C5 Evaluate)
5
I can choose the form of assignments that help
students to think critically.
(C5 Evaluate)
C
Content Knowledge
1
I judge that my knowledge of teaching English is
sufficient.
(C5 Evaluate)
2
I can solve problems related to English.
(C5 Evaluate)
3
I can develop a deeper understanding of English.
(C6 Create)
D
Technological – Content Knowledge
1
I can search and use technology created specifically
for English
(C3 Apply)
2
I consider that my knowledge of technology for
English research is sufficient.
(C5 Evaluate)
3
With technology, I can collaborate with colleagues to
deepen my knowledge of English
(C6 Create)
E
Pedagogical – Content Knowledge
1
Without technology, I can explain various theories
and problems in English science.
(C4 Analyze)
2
Without technology, I can choose a suitable learning
method for learning English.
(C5 Evaluate)
3
Without technology, I can arrange the stages of
material to support the understanding of English.
(C6 Create)
F
Technological – Pedagogical Knowledge: I can use technology to
304
1
Make students apply their knowledge in the real
world.
(C3 in
students)
2
Help students find information on their own.
(C5 in
students)
3
Make students design forms of information
representation in various ways (text, graphics, videos,
comics, etc.).
(C6 in
students)
4
Make students collaborate with each other in using
technology.
(C6 in
students)
G
Technological – Pedagogical – Content Knowledge
1
I can combine technology with the methods used to
teach English content.
(C6 Create)
2
I can choose technology in my classroom to improve
what I teach, how I teach, and what students learn.
(C5 Evaluate)
3
I can create independent learning activities with
technology for learning English.
(C6 Create)
4
I can evaluate English learning combined with
technology based on indicators.
(C5 Evaluate)
5
I can help my colleagues integrate technology,
pedagogy, and content in my school.
(C5 Evaluate)