Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
Pilot Implementation of a
Computer-based Training Course on
Newborn Hearing Screening and Teleaudiology
among Primary Healthcare Providers
in Low Resource Settings
Talitha Karisse L. Yarza, MClinAud,1 Abegail Jayne P. Amoranto, MSc,2,3 Monica B. Sunga,2
Abby Dariel F. Santos, RN,2 Mark Lenon O. Tulisana, RN, PGDip,2 Teresa Luisa G. Cruz, MD, MHPEd,1,3
Philip B. Fullante, MD,1,3 James P. Marcin, MD, MPH,4 Luis G. Sison, PhD,5
Charloe M. Chiong, MD, PhD1,3 and Pora Grace F. Marcelo, MD, MPH2,3
1Philippine National Ear Institute, National Institutes of Health, University of the Philippines Manila
2National Telehealth Center, National Institutes of Health, University of the Philippines Manila
3College of Medicine, University of the Philippines Manila
4Department of Pediatrics, University of California Davis School of Medicine
5Electrical and Electronics Engineering Institute, College of Engineering, University of the Philippines Diliman
ABSTRACT
Objecve. Newborn hearing screening (NHS) in the Philippines has been mandated by law since 2009. However,
lack of awareness and knowledge about NHS remains a challenge, especially among healthcare providers. This paper
describes the pilot implementaon of a computer-based training (CBT) course on NHS and teleaudiology among
primary healthcare providers (PHCPs) in rural Philippines.
Methods. A four-module web-based training course on newborn hearing screening and teleaudiology in an online
learning management system (LMS) was eld-tested among PHCPs from eight rural communies in the Philippines.
Parcipants were given four weeks to complete the course.
Results. Forty-two PHCPs parcipated in the CBT. Thirty-four (81%) completed the whole course (mean arion
rate of 4.8% per module) at a mean duraon of 10.2 days. Baseline data shows that parcipants had no NHS training,
although the majority (83%) had informaon and communicaons technology (ICT) training. Comparison of pre-
and post-test mean scores showed a 24.0% (p<0.001)
signicant increase in the post-test in all four modules.
Passing rates (i.e., score ≥70%) from pre- to post-test
increased by 54.6% (range: 38-80% increase). Usability
of the CBT was rated high with a mean score of 4.32 out
of 5 (range: 4.13 to 4.47), covering all eight parameters.
Parcipants expressed general sasfacon and a
posive atude on CBT to improve knowledge on NHS
and teleaudiology.
Conclusion. Even in low resource sengs where gaps
in ICT infrastructure exist, eLearning can be used as an
alternave approach to increase awareness and support
training of healthcare providers on newborn hearing
screening.
Keywords: newborn hearing screening, computer-based
training, eLearning
eISSN 2094-9278 (Online)
Published: September 28, 2023
hps://doi.org/10.47895/amp.v57i9.5050
Corresponding author: Talitha Karisse L. Yarza, MClinAud
Philippine Naonal Ear Instute
Naonal Instutes of Health
University of the Philippines Manila
2nd Floor, The Ear Unit,
Philippine General Hospital
Ta Avenue, Ermita, Manila 1000, Philippines
Email: tlyarza@up.edu.ph
ORCiD: hps://orcid.org/0000-0001-9506-0361
VOL. 57 NO. 9 2023 103
ORIGINAL ARTICLE
INTRODUCTION
Newborn Hearing Screening (NHS) has become a
standard of care in many parts of the world.1,2 e Joint
Commission on Infant Hearing (JCIH) recognizes that
the NHS is the rst step in identifying childhood hearing
loss and recommends a 95% screening rate to implement
a universal and national program.3 In the Philippines, the
enactment into law of the "Universal Newborn Hearing
Screening and Intervention Act" in 2009 established a
program for the early diagnosis, intervention, and prevention
of hearing loss that mandated all newborns to have access
to hearing screening. is is led by the Department of
Health (DOH) and its technical arm, the Newborn Hearing
Screening Reference Center (NHSRC) of the National
Institutes of Health, University of the Philippines Manila
(UPM). Implementing the law involves a continuing
process of building and instituting basic infrastructures,
setting operational and technical standards, and quality
assurance programs. e provision also includes certication
of personnel and accreditation of newborn hearing centers
(NHCs) categorized from A to D. is classication is based
on an NHC's ability to provide a range of hearing services for
children – from screening, conrmatory diagnostic testing,
rehabilitation and/or surgical intervention.4
However, despite the law's enactment, less than 10% of
newborns are screened for hearing loss.5 Challenges include
the lack of succeeding conrmatory diagnostic and inter-
vention services,6-8 the complex geography of the Philippine
archipelago that render dicult physical access to NHC,
and the high proportion of babies born outside hospitals
where specialists and equipment are not available.9 NHS
services can be prohibitive to the poor in this context: the
high cost of screening devices means the existing NHC
recover investments thru patient service fees. ere is a
lack of accredited screening facilities and personnel. Many
screening NHCs cater to babies but are not clearly organized
into an NHS care referral system.10 ese concerns hinder
the full implementation of the National Newborn Hearing
Screening and Intervention Program.
ese challenges reect the situation in most developing
countries characterized by the shortage of healthcare
providers and nancial resources.11,12 e lack of proper
training and poor performance of screeners render 80%
of hearing screening programs ineective.13-17 e World
Health Organization (WHO)1 highlights working within the
framework of primary health care, especially in low resource
settings, in addressing the global burden of disabling hearing
loss. e WHO emphasized the importance of capacity-
building and quality assurance.
e role of health care professionals is crucial in the health
care system.11 eir capacities remain inarguably essential to
provide healthcare services eectively. Continuing profes-
sional development (CPD) must be in place as a mechanism
to support the continuous delivery of essential health services.
CPD for professionals serving rural and remote commu-
nities poses a challenge. Consequently, eLearning has been
applied in health education to increase access to training.18-20
Never has this been more relevant than now, in the setting
of a pandemic wherein distance, online, and computer-based
learning have become the primary format of education.
is is especially true in low resource countries where face-
to-face encounters are still limited. 21,22
eLearning comprises a new educational paradigm and
new learning methods that utilize digital media and tools
to promote learning.23 Alternatively called computer-based
training (CBT), this is designed to be remotely accessible
and delivered in synchronous (or real-time) or asynchronous
mode. e latter aords learners the opportunities for learning
at a time and place convenient for them.
ere are contradicting ndings comparing outcomes
of traditional classroom teaching and the use of CBT.24-26
Yet, these are but media of instruction. Eectiveness relies
on the eort placed throughout the education process: prior
to, the actual training, and learner assessment and feedback,
as well as the training program evaluation. Schmidt and
Brown27 pointed out that the comparison of CBT to face-to-
face training does not intend to nd signicant dierences
between the two media but rather an armation that CBT
can be an eective alternative or complement, such as in
blended learning.28 When one method is no better than
the other in the context of learning outcomes, CBT is
justiable, especially in conditions of distance learning.
e design and development phases facilitate and
contribute to better learning outcomes.29-31 Additionally,
Sinclair and colleagues32 refocus the debate on learning that
promotes behavioral change and improvements in patient
outcomes, emphasizing the importance of standardized
assessment beyond the knowledge gained and satisfaction
with instruction. Similarly, the measure (of CBT) should
eventually include organizational impact.33
The Hearing for Life Project
Increasing the Rates of Newborn Hearing Screening with
Novel Technologies and Telehealth, or the Hearing for Life
(HeLe) research program is led by the University of the
Philippines (UP), in collaboration with the University of
California, under the Commission on Higher Education
(CHED) - Philippine California Advanced Research
Institute (PCARI). e HeLe is proposed to increase hearing
screening rates through several convergent components. e
HeLe is a strategic approach towards democratizing access to
hearing screening for all Filipino newborns. is is through
building capacities of primary care government Rural Health
Units (RHUs) to become Category A NHC, more accessible
to communities. Primary healthcare providers (PHCPs)
serving these RHUs, will be equipped with competencies
in the NHS. e HeLe research program also includes the
development and eventual deployment to these Category A
NHCs of low-cost, accurate, and clinically validated hearing
VOL. 57 NO. 9 2023104
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
screening devices with telehealth capabilities.34 at is,
NHS data will automatically be relayed to the HeLe NHS
Registry; and in the case of newborns screened to warrant
conrmatory testing to ascertain hearing loss, these patients
will be tele-referred through the HeLe telehealth system.
ese Category A NHCs will be linked with Category
B to D NHCs, organized as the NHS and intervention
service delivery network (SDN) for infants with hearing
impairment within the locale. ese are steps towards a more
sustainable National Newborn Hearing Screening Program.
e capacity building component of the HeLe capitalizes
on the use of information and communications technology
(ICT) to train clinicians to facilitate NHS service delivery,
and ethically and securely manage health information of
patients. It builds on the existing training programs and
experience of the Newborn Hearing Screening Reference
Center (NHSRC) and the National Telehealth Center
(NTHC), of the National Institutes of Health, UP Manila.
e HeLe Capacity Building Program for the PHCPs was
designed and developed using a blended-learning approach
combining: (1) computer-based training, (2) face-to-face
training, and (3) on-site coaching. It aims to build the abilities
of PHCPs, to value, practice NHS by integrating the use of
novel technologies, and appropriately refer patients identied
to have potential congenital hearing problems to the closest
Category B NHC for conrmatory diagnosis, and eventual
intervention. e design of the HeLe Capacity Building
Program is discussed in another paper.
Aside from the known benets of CBT, this was also a
means to minimize the number of days PHCPs need to be
pulled-out from their clinics to attend and learn the HeLe, its
innovations in support of the Universal NHS and Intervention
Program. is is the rst known computer-based training,
embedded in a blending learning strategy in the Philippines
intended for PHCPs serving government rural health units.
The HeLe Computer-Based Training Course
e Hearing for Life CBT Course, a three-day face-
to-face training and on-site coaching comprise the blended
learning strategy of the HeLe Capacity Building.
e Hearing for Life CBT Course aims to provide
the learners with theoretical and procedural knowledge on
NHS and teleaudiology. It was developed as a web-based,
learner-paced (asynchronous) course delivered via Learning
Management System (LMS). It was made accessible to
targeted learners using common web browsers. e CBT is
composed of a pre-test, and four Modules that target specic
learning objectives, described in Table 1. Each Module
contains learning units that embody didactics, learning
exercises, and an assessment section that the learners must
complete before moving to the next Unit or Module. For the
learner, the post-test serves as a summative assessment for the
HeLe CBT. e study participants were given four weeks to
complete the course.
e HeLe Computer-Based Training Course was
deployed in two batches, as prerequisite to the face-to-face
training (FTF) learning event, which, in turn, was held four
weeks after the start of the CBT implementation period.
e CBT course administrator provided instructions
via email, as well as technical support in accessing the
course. During the period of implementation, for Batch
1, Modules 1 to 3 were made available altogether. On the
other hand, Module 4 was released three weeks after the
start of the course. is was due to the delays in HeLe device
development aecting the original timeline. e participants
were advised of the schedule of release. For Batch 2, Modules
1 to 4 were released altogether.
A forum in the HeLe social media platform was created
for announcements, i.e., additional resources. Questions
from participants were also addressed through this; the
HeLe social media page was designed to encourage group
interactions/discussions.
is paper presents the results of the pilot implementation
of the developed CBT among primary healthcare providers
(PHCPs) serving in selected rural health units.
METHODS
Research Design, Seng, and Recruitment of
Parcipants
is is a descriptive study utilizing mixed methods to
describe the experience of primary healthcare providers
(PHCPs) from rural health units (RHUs) and assess the
learning outcomes resulting from the use of the developed
HeLe CBT Modules.
e PHCPs were recruited from eight RHUs: four
from the Western Visayas Region, three from the Romblon
islands, and one from the province of Bulacan. Each RHU
covers an average of 31 barangays (SD = 16.6; ranges from
12 to 52 barangays per RHU), caters to a mean population
of 47,732 (SD = 28,287) individuals, and has an average
of 316 livebirths per year (SD = 235.9). Four out of the
eight RHUs provide services to geographically isolated and
disadvantaged areas (GIDAs).
Purposive sampling
Pilot sites were identied in line with the overall
objectives of the HeLe research program. e selection was
based on the following criteria:
1. A previous eHealth project implementation site,
considered as a successful pilot site;
2. With an Electronic Medical Record (EMR);
3. With internet connectivity; and
4. Proximal to certied NHS conrmatory and intervention
centers, which are part of the (HeLe) NHS Service
Delivery Network (SDN).
VOL. 57 NO. 9 2023 105
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
ese inclusion criteria resulted from discussions by
the research proponents, who are experts in the eld of
NHS, public health, eHealth and medical device research
implementation, and health professions education in the
Philippines. Likewise, these sites were selected based on
a review of relevant studies on the implementation of
telehealth projects, and analysis of the existing healthcare
system, and the potential for an SDN for hearing screening
and audiology. A nal list of HeLe sites was prepared. e
HeLe research proponents engaged the RHU physicians as
the facility HeLe program managers, who, in turn, assigned
their PHCPs as the designated hearing screeners of these
selected RHU pilot sites. Each RHU was initially asked to
identify three to ve PHCPs to undergo training on NHS
and teleaudiology. e minimum number of trainees per
RHU was set to include at least one physician (usually the
Municipal Health Ocer), and one to two public health
nurses and/or midwives. An average of four PHCPs was
recommended for training by their respective MHOs per
RHU. We calculated the minimum sample size at 30 PHCPs
from eight RHUs, assuming a 5% margin of error. Training is
acknowledged as a preparatory step to implement the NHS
service. All study participants consented to participation.
Data Collecon
A pre-test and post-test were administered for each
HeLe Course Module to assess learners' progress in their
knowledge of the basic concepts of NHS and eHealth.
A Usability questionnaire was adapted from Zaharias'
Usability Questionnaire.35,36 e questionnaire was used to
characterize and qualify the user's perceived usability of the
CBT. at is, the degree to which the PHCPs can use the HeLe
eLearning software to achieve the learning objectives with
eectiveness, eciency, and satisfaction.37 e questionnaire
contains 36 items grouped into eight parameters: (a) content
(of the CBT), (b) learning and support, (c) visual design, (d)
navigation, (e) accessibility, (f) interactivity, (g) self-assessment
& learnability, and (h) motivation to learn. Parameters b to
g relate to the web design and instructional aspects of the
CBT; these are the technology-related components of
the CBT. Parameter h relates to the aective dimension of
learning.36 e learners were asked to rate their agreement
Table 1. HeLe CBT Modules’ Learning Objecves and Units
Modules Learning Objecves Units Duraon
(in minutes)
1 Newborn Hearing Screening Course
1. Describe RA 9709, theorecal basis, purpose, and its history
2. Dene the dierent hearing screening and diagnosc modalies
in determining hearing loss in infants
3. List the available intervenons for hearing loss in infants in the
Philippines
4. Dene the Newborn Hearing Screening Registry and reporng
methods
5. Describe the Newborn Hearing Screening Program including the
signicance of early idencaon and intervenon of hearing loss
1. Introducon to the Universal
Newborn Hearing Screening and
Intervenon Act of 2009
2. Hearing Screening and Diagnosc
Modalies in Determining Hearing
Loss in Infants
3. Available Intervenons for Hearing
Loss in the Philippines
4. Reporng and Registry
16
25
10
18
2 How to Do Newborn Hearing Screening
1. Enumerate and describe the steps in conducng the newborn
hearing screening test using OAE or AABR
2. Describe the preparaon prior to conducng the test, and
3. Recall scripts in communicang a Pass or a Refer result
1. Screening (NHS) Using Otoacousc
Emissions (OAE)
2. Newborn Hearing Screening
(NHS) Using Automated Auditory
Brainstem Response Test (AABR)
18
17
3eHealth and ICT Tools for NHS Implementaon
1. Describe eHealth and the ethical and legal guidelines governing
its pracce
2. Describe the Community Health Informaon Tracking System
(CHITS) and its benets.
3. Dene telemedicine and its dierent types and uses
4. Demonstrate how to use CHITS to nd or register paents, put
them on queue, and manage their folders
1. The Ethico-Legal Aspects of eHealth
2. An Introducon to the Community
Health Informaon Tracking System
(CHITS)
3. Geng Started with CHITS
4. Telemedicine and the Naonal
Telehealth System (NTS)
29
18
15
15
4 Newborn Hearing Screening Using the HeLe Device and Teleaudiology
1. Describe the Hearing for Life Project
2. Explain the steps on how to do newborn hearing screening using
the HeLe device
3. Demonstrate how to use the CHITS Newborn Hearing Screening
Module, and
4. Demonstrate how to refer a paent through the NTS
Teleaudiology Module
1. The Hearing for Life Project
2. Using the HeLe AABR Device
3. Using the CHITS Newborn Hearing
Screening Module
4. Using the Naonal Telehealth
System (NTS) Teleaudiology Module
6
5
9
7
Total: 208 minutes (3 hrs., 28 mins.)
VOL. 57 NO. 9 2023106
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
to statements using a 5-point Likert rating scale, where 1
corresponds to strongly disagree and 5 for strongly agree.
A focus group discussion (FGD) was conducted to elicit
other qualitative feedback from the learners. e FGD was
conducted at the end of the face-to-face training of the HeLe
Capacity Building Program; the latter was implemented
four weeks after the start of the CBT course. e FGD
facilitator was guided by a semi-structured questionnaire
developed by the research team. Figure 1 shows the data
collection process relative to the training implementation
period.
Data Analysis
Data from all participants of Batch 1 (28 learners) and
Batch 2 (14 learners) were combined for analysis. e data
on the pre- and post-test scores and the duration of access to
the CBT were extracted from the LMS utilizing descriptive
statistics to present data. Item analysis was done to identify
knowledge gaps. Participants' ratings on the Usability
questionnaire were summarized and were presented using
mean scores. User Satisfaction was described using a qualitative
content analysis38,39 of the recordings and transcripts of the
FGD. Identied themes were categorized as Enablers and
Obstacles/Barriers to CBT use. e participants' statements
were quoted, enclosed in quotation marks, and italicized.
ese have been translated from Filipino to English. Words
or phrases in brackets were insertions to complete the intent
of the speaker, and ellipses signied deletions.
Ethical Consideraons
is study was approved by the University of the
Philippines Manila Review Ethics Board prior to implemen-
tation. A Research Agreement for the HeLe Research
Program was formally entered into by the pilot sites and the
PNEI, and individual informed consents from participating
PHCPs were obtained.
RESULTS
Characteriscs of Parcipants
e participants include 42 PHCPs from eight identied
RHUs; ages of the learners range from 22 to 64 years
(M=41.3, SD=12.4) and majority are female. Among them
are physicians, midwives, nurses, medical technologists, and
other support sta (e.g., IT designate, nurse aide, and sta).
e majority (83%) reported previous training in ICT for
health, while none had any experience with NHS training
(Table 2).
Compleon of the HeLe CBT Modules
Out of the 42 participants, the majority or 34 (81%)
learners completed the whole CBT course within the
prescribed four-week period. A total of eight participants
were unable to complete the course, with mean attrition
rate of 4.76% per module. Overall, the mean duration to
complete the CBT was 10.2 days (range: 3-21 days) or 245.3
Figure 1. Data collecon process.
Table 2. Demographics of the Primary Healthcare Providers
Demographics n (%)
Age (years)
Mean
SD
22-64
41.3
12.4
Sex
Male
Female
11 (26%)
31 (74%)
Occupaon
Physician
Nurse
Midwife
Medical Technologist
Others
9 (21.4%)
11 (26.2%)
15 (35.7%)
2 (4.8%)
7 (16.7%)
Previous training
Newborn Hearing Screening (NHS)
Informaon Communicaon Technology (ICT)
0 (0%)
35 (83%)
VOL. 57 NO. 9 2023 107
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
Table 3. Comparison of Average Pre-Test and Post-Test Score
per Module
Mean Pre-
test Score
(%)
Mean Post-
test Score
(%)
Mean %
Change (Pre-
to Post-test)
p value
Module 1 61.7 77.6 16.1 <0.001
Module 2 64.7 93.4 28.7 <0.001
Module 3 73.7 93.6 20.0 <0.001
Module 4 59.8 75.4 31.2 <0.001
Total 65.0 85.0 24.0 <0.001
hours for all participants. e duration of access extracted
from the LMS was computed from when the learner rst
started the course to the day it was completed and does not
exclusively account for the login hours.
Extracting the data for the two separate schedules of
CBT release, Batch 1 (28 learners) completed the course
for the mean duration of 13.6 days (range: 3-21 days)
or 327.9 hours, where 10 (45%) participants started the
course on week 1 but had to wait for the delayed modules
resulting to the increased in the duration of access. Despite
accessing the course early, the recorded time of completion
for Batch 1 signicantly increased. Batch 2 had a mean
duration of 3.9 days (range: 1-7 days) or 94 hours; 2 (20%)
participants accomplished the course within 24 hours. Apart
from the availability of all modules at the time of release
for Batch 2, other reasons for shorter time completion were
not explored.
Pre-Test and Post-Test Results
Participants undertook the HeLe pre-test prior to the
CBT and the post-test upon completing the four modules.
Passing rate of participants (i.e., those who had a score ≥70%)
from pre- to post-test increased overall by 54.6% (range:
38-80% increase for the four modules). Table 3 compares
pre- to post-test results showing a 24.0% (p<0.001) mean
signicant increase in the post-test in all four modules.
Figure 2 shows the proportion of participants who met
the 70% passing mark in the pre-and post-test. Mean pre-
test scores revealed failure in three of the four modules, with
an average of 65% across all modules. e range of mean
scores is 59.8% (Module 4) to 73.7% (Module 3). More
than half of the participants failed the pre-test on modules
1, 2, and 4. e learners only passed Module 3, in which the
majority of the content is already familiar to them and for
which they received prior training (eHealth and ICT tools).
Results are not unexpected, given that the participants had
no prior training on the NHS.
Pre-test item analysis showed knowledge gaps in
the following areas: (a) scope of the universal NHS law, (b)
categories of NHCs, (c) stop criteria for NHS, (d) early hearing
diagnostic and intervention services, (e) referral protocol for
newborns with 'refer' NHS results (for Module 1), and
(f ) testing procedures (for Module 2). Low scores were also
noted in items regarding (g) basic bioethical principles, (h)
components of Data Privacy Act, and (i) practice of telemedicine
(for Module 3). Module 4 introduces a novel technology –
the HeLe system – which was developed for the project; the
learners scored least in this module in the pre-test (mean
score of 59.8%).
After taking the CBT course, a signicant increase is
shown in the post-test for all four modules – as was expected.
e least improvement is observed for Module 1, at 16.1%.
Given the 71.05% passers for the post-test, the remainder of
about 29% of learners were still unable to meet the passing
score. Post-item analysis revealed knowledge gaps remain
for categories of NHCs, with only 50% of the participants
answered the related question correctly and 13.6% for stop
criteria and referral protocol.
Overall mean post-test score was 85%; thus, the mean
post-test score is considered as passing the HeLe CBT.
Figure 2. Proporon of learners
with passing scores for
pre-test and post-test.
VOL. 57 NO. 9 2023108
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
Figure 3. HeLe CBT usability: average scores
per parameter.
(a) content, (b) learning and support,
(c) visual design, (d) navigaon, (e)
accessibility, (f) interacvity, (g) self-
assessment and learnability, and (h)
movaon to learn; x = mean score
e range of mean scores for the four modules is 75.4%
(in Module 4) to 93.6% (in Module 3), again considered as
passing for each module. e trend is preserved as that of the
pre-test module mean scores. at is, performance is lowest
to highest in Modules 4, 1, 2, and 3, respectively, in both
pre-test and post-test.
Usability of CBT Course
e participant's overall perceived usability of the CBT
was rated high (mean score of 4.32 out of 5), covering all
eight parameters. e range of scores was from 4.13 to 4.47
out of 5. Figure 3 presents the average scores per parameter
rated by the learners.
Motivation to learn was the CBT Usability parameter
rated highest (4.47 out of 5). Participants agreed that the CBT
met the (learners') needs, and the frequent presentations of
activities increased their success. e CBT is also perceived
to provide opportunities to use new skills, giving learners
positive feelings about their accomplishments. Content also
received high ratings (4.4 out of 5). Participant ratings
show favorable agreement with clear learning objectives,
appropriate alignment and sequencing of course materials,
and consistent vocabulary and terminologies suitable for
the learners.
Accessibility received the lowest rating (4.13 out of 5).
Related questions include, "e pages and other components of
the course download quickly / open quickly," and "e course is
free from technical problems (i.e., hyperlink errors, programming
errors, etc.)." Interactivity was also rated relatively low (4.23
out of 5), which relates to the lack of (additional) resources
and variety of engaging or immersive and animated learning
exercises.
FGD: User Sasfacon
Of the 42 PHCPs-learners, 26 (62%) participated in a
series of FGD that sought qualitative feedback on the CBT
and the learners' satisfaction (or non-satisfaction) with its use.
During the FGD, the participants were asked to rate how
satised they were with the CBT to improve their knowledge.
Using a scale of 1 to 10, the respondents gave scores of 9,
8, and 7, rated by 19% (5 learners), 62% (16), and 7% (4),
respectively. is reects the learners' general satisfaction
and positive attitude towards the CBT use.
Content analysis of the responses revealed several
themes, categorized as Enablers and Obstacles/Barriers to
CBT use. e themes and the responses of the learners are
presented in Table 4.
Enablers of CBT Use
Use of interacve learning components
e inclusion of interactive learning approaches in
the CBT encourages further use and translates to a better
learning experience.
"One good thing about the modules is that they
include the demonstration part. [Such as] how to insert
[the ear probe] and type [or enter data into the EMR].
So maybe maintain that. I learn much better when I
see how [it is done]. Because if it's only instructions,
like, do this and that, [it will be hard]… it is better
[when there is a video] demonstration, [like,] where to
type in CHITS (Community Health Information and
Tracking System EMR)" – Participant 5
VOL. 57 NO. 9 2023 109
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
Flexible nature of the CBT that ts the users'
environment
e exible nature of the CBT allowed the learners to set
their own time and pace to complete the course.
"[I could] do it faster, [taking each module and
units of the course] should be continuous. As a health
ocer, we have constraints [while in the RHU], so we
do it [the CBT] at home. en sometimes patients still
come to our house to consult… so I have to [stop the
course and attend to the patient]. en, the continuity of
your thought and your interest is lost…. [For instance,]
I started with Module 1, Unit 1, then [proceeded to]
Unit 2, [and] 3. en I had to stop because of a patient.
[en it was only] after three days [that I had time
to work on the CBT]. I had to restart again from
Module 1, Unit 1 [and so on]. en I decided to stay up
until 11:00 PM and nish [the modules] … It doesn't
matter, as long as I pass, [and] take [the unit exams] …
then I passed [the exam] – Participant 8
e availability on-demand of the CBT also allowed the
participants to access the course at their preferred location.
Of the 26 FGD participants, 38% (10 learners) reported
accessing the modules at home, 42% (11) in the workplace or
RHU, and 12% (3) in commercial establishments.
Learner engagement, sense of accomplishment
e HeLe CBT was able to engage the learners and elicit
a sense of accomplishment.
"In the CBT, the topic becomes more exciting
because when you nish one page, you can proceed to
the next, then you can take the [unit] exam… It's more
exciting because you will know, I'm already done. I can
proceed to the next. It's like (comment from another
participant: "accomplishment"), yes, … [a sense of]
accomplishment, I am done with module 1, yehey (or
hooray)." – Participant 7
Table 4. FGD Themes and Learners' Responses
Enablers of CBT Use
Use of Interacve Learning Components
"One good thing about the modules is that they include the demonstraon part. [Such as] how to insert [the ear probe] and type [or enter data into
the EMR]. So maybe maintain that. I learn much beer when I see how [it is done]. Because if it's only instrucons, like, do this and that, [it will be
hard]. it is beer [when there is a video] demonstraon, [like,] where to type in CHITS (Community Health Informaon and Tracking System EMR)"
– Parcipant 5
Flexible Nature of the CBT that Fits the Users' Environment
"[I could] do it faster, [taking each module and units of the course] should be connuous. As a health ocer, we have constraints [while in the RHU], so
we do it [the CBT] at home. Then somemes paents sll come to our house to consult… so I have to [stop the course and aend to the paent]. Then,
the connuity of your thought and your interest is lost. For my experience, for module 1, since it's long, right? (Facilitator 2: it's more than 1 hour, yes)
[For instance,] I started with Module 1, Unit 1, then [proceeded to] Unit 2, [and] 3. Then I had to stop because of a paent. [Then it was only] aer three
days [that I had me to work on the CBT]. I had to restart again from Module 1, Unit 1 [and so on]. Then I decided to stay up unl 11:00 PM and nish
everything, [from] module 1 unit 3, [and then] unit 4, up to module 4. More than that, I almost slept at 12 midnight. I was already sleepy, It doesn't
maer, as long as I pass, [and] take [the unit exams] …then I passed [the exam] – Parcipant 8
Learner Engagement, Sense of Accomplishment
"In the CBT, the topic becomes more excing because when you nish one page, you can proceed to the next, then you can take the [unit] exam. That's
the most excing part. Because when your answers are wrong, you have to take it again, [thus,] you cannot proceed to the next page. We won't [be able
to] nish [the module] unless we do not pass the [unit] exams (or exercises). It's more excing because you will know, I'm already done. I can proceed
to the next. It's like (comment from another parcipant: "accomplishment"), yes, … [a sense of] accomplishment, I am done with module 1, yehey
(or hooray)." – Parcipant 7
Obstacles and Barriers to CBT Use
Compeng Core Work of Healthcare Provision vs. Connuing Professional Development In Situ and User Interface Issues
"… [Module 1] takes more than an hour to nish…it becomes dragging because the audio is a bit slow. It really takes up your me. [Module 1] is like
more than an hour or even two hours to nish the course. Module 2 was good [because of the beer pacing of the audio] ...Because when you're at
home, of course, you are red… And since [the pacing of the audio is] slow so it becomes dragging, you really get red halfway through [taking the
modules]. But your goal is to try to nish at least one module per day." – Parcipant 1
Unavailability, Delayed Release of Modules
"The concept of the CBT is great. However, in terms of the availability of all modules, I hoped the pre-test, post-test, and everything [were] uploaded
mely so that there won't be any gaps [when we take the course]." – Respondent 2
Poor Internet Connecvity
"… if your internet [connecvity is not good], ... you'll think your answers were incorrect [for the Unit exercises] … but it was just because of
[the unstable] internet." – Parcipant 3
"For us, one limitaon is that the CBT is highly internet-based… in remote areas like ours, a lile uctuaon [in the internet connecon], somemes
[cause] you have to reset from the start." – Parcipant 5
VOL. 57 NO. 9 2023110
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
Obstacles and Barriers to CBT Use
Compeng core work of healthcare provision vs.
connuing professional development in situ
Learners cite that the "overwhelming workload of the
PHCPs" is the biggest barrier to the HeLe CBT use and
completion. is is so, such that more than half of the FGD
participants completed the CBT outside the workplace and
completed this outside of oce hours.
User interface issues
Participants emphasized that appropriate conguration
of the "audio and visual" elements is essential. When there
are some "glitches" - specically typographical errors, over-
lapping slides and poor audio or voiceover, pacing, and unres-
ponsive "next" buttons) – these detract from the smooth use
of the CBT. Slow pacing also renders the CBT "dragging."
"… [Module 1] takes more than an hour to nish…
it becomes dragging because the audio is a bit slow…
Module 2 was good [because of the better pacing of the
audio] ...Because when you're at home, of course, you are
tired… so it becomes dragging... But your goal is to try
to nish at least one module per day." – Participant 1
Some participants experienced problems with the Unit
exercises where answers were incorrectly scored. Participants
promptly raised these concerns via email, and the course
administrator applied corrective measures.
Unavailability, delayed release of Modules
e delays in uploading and loading content or getting
o-track the schedule disrupt the continuity of learning.
Poor internet connecvity
Furthermore, problems in internet connectivity aect
the quality of the eLearning experience.
"… if your internet [connectivity is not good],...
you'll think your answers were incorrect [for the Unit
exercises] … but it was just because of [the unstable]
internet." – Participant 3
"For us, one limitation is that the CBT is highly
internet-based… in remote areas like ours, a little
uctuation [in the internet connection], sometimes
[cause] you have to reset from the start." – Participant 5
Similarly, the usability parameter accessibility was rated
lowest, which relates to the problems in loading of content
and technological glitches (i.e., delays in moving to the
next page of the Unit) which are often due to poor internet
connectivity.
DISCUSSION
e primary care health professionals from the Rural
Health Units selected for HeLe typies that of the majority
serving about 2500 local government-supported primary care
centers in the Philippines. e workforce reects the national
picture: it is feminized (majority are females), and represents
the range of young graduates to near-retirees. e learners
are health providers, mainly general care physicians, nurses,
and midwives, who are adept with the delivery of an array of
health promotion services (such as maternal antenatal care,
immunization, family planning) to primary care wellness
services (birthing and postpartum care, breastfeeding, and
nutrition promotion) and disease management (such as
care for acute respiratory infections, tuberculosis, non-
communicable diseases, and injuries). ese eight RHUs
provide periodic reports to the Department of Health (DOH)
on rendered services of public health import. Democratizing
access to NHS means necessarily expanding from the current
hospital-based NHS and targeting these community RHUs
and primary care facilities where newborns are delivered, and
their families live.
Population screening should be ethically done in an
environment where intervention is available and accessible
to the community. is is the spirit of the law dening the
Universal Newborn Hearing and Intervention Program.
Purposive sampling was implemented in this study to
meet the overall objectives of the HeLe research program. e
42 participants were selected based on the recommendation of
each MHOs, allowing for the participation of more screeners
per RHU but not exceeding to the point of disrupting the
usual clinical work when required to attend trainings. Also,
accommodating a certain number allowed by the resources
of the project. Notwithstanding social problems besetting
rural communities, the HeLe innovations depend on having
an IT infrastructure in place. Specically, internet access is a
requirement in order to eld test the web-based tele-referral
system and the electronic submission of NHS results to the
HeLe NHS Registry. ese primary care centers would have
the necessary IT equipment (for telehealth and the CHITS-
EMR), systems of which would also be used for the HeLe.
ese sites were also selected because they have had good/
successful eHealth implementations for telemedicine and
EMR. As such, the HeLe learners are already familiar with,
and the majority (if not all) use IT for health. Furthermore,
these RHUs have demonstrated leadership in eHealth, and
commitment to innovation testing – critical elements for
successful pilot project implementations.40
Arrangements have been made for these RHUs –
envisioned as future Category A NHC – to be formally
linked to a diagnostic hearing center (Category B NHC).
Research studies on screening are compelled to provide, at
the minimum, conrmatory testing to those who will be
found positive.41 For the HeLe, it was necessary to properly
VOL. 57 NO. 9 2023 111
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
organize the SDN for the NHS Program in the locale of
participating RHU.
e HeLe CBT is designed to be available on-demand
to support the learner's pace amidst their circumstances.19,42
e majority (34 out of 42, or 81%) completed the course
within the allotted four-week period.With the release of the
course on two separate schedules and the availability of all
modules, the learners from Batch 2 were able to complete the
course in less than four days; a dierence of about ten days
compared to Batch 1, where there was a delay in the release
of the last module. Overall, this is instructive and arms
that this simple structure (of allotting a specic duration)
in computer-based education is practicable also for PHCP
serving RHU. at is, setting the time to access the course
highlights exibility and relies on the learners' self-directed
learning to manage one's time and accomplish the task.43
Completion of the HeLe CBT qualied participation in
the subsequent in-person training program that centered on
skills building on the NHS.
e simple majority of the learners accomplished the
CBT at home or in an internet cafe. is preference to focus
solely on either clinical work or continuing professional
development (CPD) is not unexpected. Yet while patient
care naturally takes precedence, many (11 of the 26)
completed the CBT in the workplace. is is encouraging:
the IT infrastructure was in place (even if the internet was
intermittent), and the health workers were able to maximize
oce hours (conducting clinical work and continuing
professional education side-by-side).
ese 11 PHCPs epitomize "success" and what is
possible for the future regarding CBT not only for the NHS
but for continuing professional development, in general.
Perhaps what can be considered is that specic schedules for
each PHCP, within ocial clinic hours, can be reserved for
CBT-based CPD.
It is encouraging that passing rates from pre-test to post-
test increased overall by 54.6% and that the mean scores for
all four Modules were above passing. Post-test item analysis
showed knowledge gaps remain for specic areas. Imbibing
knowledge on the Categories of NHCs appears to be a challenge
to the learners, with only half of the participants correctly
identifying these despite the lower-order cognitive processing
required for this concept (Remember and Understand). On
item analysis, however, this may be due to the choice of test
format, i.e., the use of a multiple-true-false (MTF) question.
According to Brassil and Couch,44 this type of question
format is often confusing and leads learners to random
guessing rather than informed reasoning.
Similarly, the stop criteria and referral protocol were
still confusing to some learners, missing to answer the
related question correctly. ese require mid-level cognitive
processing (application and analysis).45,46 Rozul, Yarza, and
Ombao22,47 emphasized the importance of the mastery of
these key concepts as a competency of screeners, adding
weight on assigned points for these items in the online
adaptation of the NHS certication course. In practical
applications, it is best measured as a procedural skill.48 For
the CBT course, remediation can be done by including more
interactive learning exercises. Gaupp, Körner, and Fabry49
recommend including case-based simulations to encourage
analysis and decision-making. erefore, in light of the HeLe
blended learning approach, the CBT will promote better
knowledge transfer as it transitions to face-to-face training.50-52
erefore, further review and analysis of the HeLe CBT
course content and assessment tools are warranted for these
future enhancements. It is worthwhile to process with the
learners the course content, including the assessment and test
questions, primarily where the lowest scores were garnered.
Specic learners' perspectives can be obtained to improve the
succeeding course oerings.
Monitoring the attainment of educational objectives
through computer-based instruction of these crucial
competencies must be done diligently in the subsequent
oerings of this training module. If these competencies,
despite revisions in the curriculum, consistently cannot be
satisfactorily obtained by the learners through this CBT,
these competencies will have to be met through a dierent
training course format.
Learners rate the HeLe CBT to be highly usable, with
a range of scores for all eight parameters between 4.13 to
4.47 out of 5, the highest score. at is, in terms of content,
the technology components of the CBT, and the ability of
the CBT to encourage continuing learning (or at least,
the completion of the HeLe CBT course).36 e latter
(motivation to learn) is seen to be a consequence of positive
user experience and satisfaction with the CBT.53,54
e intermittent internet connectivity in the locale
remains a challenge beyond the control of the HeLe research
proponents; this continuing social development issue is raised
to the national government. e course may be improved
with regard to accessibility through more reliable internet
providers for the participants and the various new applications
in systems and designs that have developed since the initial
formulation of the course. An oine version of the CBT
can be made available, yet maintain the web-based mode for
learner assessment.
Technical components such as web design and
instructional parameters are seen to facilitate better learning
experiences and increase one's motivation to learn. Corollary,
issues with the user interface, availability of modules, and
internet access become barriers in completing the course.
Internet connectivity must be addressed by national
government investment. is is elevated by the HeLe
proponents to the CHED as part of the overall campaign
of the UP for the social development of the country's rural
communities.
However, the HeLe research team can and should
improve user interface concerns raised by the learners:
typographical errors, overlapping slides and poor audio or
voiceover, pacing, and unresponsive "next" buttons. e user
VOL. 57 NO. 9 2023112
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
interface (UI) is the point at which the PHCPs "interact with
a computer, website or application" such as the HeLe CBT.
e UI intends to "appeal to the human senses (sight, touch,
auditory and more). An eective UI is to make the user's
experience easy and intuitive, requiring minimum eort on
the user's part to receive the maximum desired outcome."55
e HeLe UI should encourage, rather than discourage,
users to complete the CBT.
Other operational and pedagogical concerns were with
Unit exercises where answers were incorrectly scored, and
the delayed upload of subsequent Modules. ese must be
corrected immediately. In general, a better and more stringent
preparation stage for CBT development must be invested
in to ensure that these 'glitches' that the PHCP identied
during this pilot run must not even be encountered by future
PHCP-learners.
In general, user satisfaction describes how the course is
perceived by the learners. Learner satisfaction stems from the
comparison between the learner's expectations before taking
the course, and his/her perceptions upon completion of the
course. ough expectations were not explored during the
pre-course period, the post-course FGD generated relevant
information.
It is important to note the high motivation to learn of
the participants was gleaned from the Usability Survey. is
was also qualied during the FGD with their expressions of a
sense of accomplishment, eagerness to nish the course, and
recognizing that the CBT provides them opportunities to
use new skills in new situations.53,54
e participants' feedback highlights the importance
of a sound instructional design and well-thought-out
operations in implementing eLearning programs.26,29-31,45
is relates to the high rating for the content parameter in the
Usability Survey, where alignment and sequencing of course
materials were highly appreciated. e progression of content
has contributed to the 'excitement' in moving through the
pages of the CBT.
User satisfaction is one of the most important factors
that aect the success of the course and should be monitored
regularly. Ideally, pre-course learner expectations must be
solicited to be able to compare with post-course perceptions.
e timing and schedule of the sessions for learner satis-
faction should be set and made known to generate a greater
percentage of responders. It may be a component of the course
that is considered a requirement for completion of the course.
Limitaons
e pilot implementation of the CBT is one component
of the HeLe Capacity Building Program that combines
blended learning strategies in training the PHCPs on NHS
and teleaudiology. e CBT is designed to target factual and
conceptual knowledge as a preparatory course to face-to-
face training. It would be interesting to show how learning
is integrated into the work setting. But for the present study,
measures of behavioral outcomes will not be solely due to
the CBT but rather the result of blending various teaching
strategies utilized in the HeLe training program. is is
described in a separate paper.
Furthermore, feedback from subject matter experts and
usability experts using standardized evaluation tools would
have provided a more comprehensive description of the
usability of the HeLe CBT. And exploring relationships
by quantifying associations among the technical aspects
of the usability components and motivation to learn as an
intrinsic construct.
CONCLUSION AND RECOMMENDATIONS
e current study demonstrated the feasibility of using
CBT to train health care providers in NHS and teleaudiology
in rural communities. is mode of education is seen to
fast-track training and accreditation of NHS Category A
screeners and their Rural Health Units as Category A NHC.
e current ICT infrastructure in the Philippines remains
a challenge that limited the online version of the CBT in
reaching more remote areas, especially the GIDAs. An oine
version of the CBT can be made available, yet maintain the
web-based mode for learner assessment. is may be explored
in future iterations of the study or for similar technology-
based health projects.
Utilizing eLearning requires intentional planning of
the instructional design and teaching strategies to facilitate
meaningful learning experiences. Its application has expanded
from the initial goal of reaching remote areas, but now also to
provide for remote learning for all in the setting of a pandemic
where face-to-face education is limited. us, the results of
the review and analysis of CBT usability and user satisfaction
can be a basis for future revisions and improvement in the
instructional design of similar endeavors. Furthermore,
this investigation showed the CBT as an eective teaching
strategy for distance learning and, more so, a viable option in
low resource settings where gaps in ICT infrastructure exist
competing with demands of clinical and public health work.
Finally, eLearning can be used as an alternative approach
to increasing awareness and support training of healthcare
providers on newborn hearing screening. is is one step
closer to ensuring a better and more equitable future for
children with hearing loss.
Acknowledgments
e implementation of the HeLe Project was endorsed
by the Department of Health. e HeLe CBT Program was
hosted in the UPM Virtual Learning Environment with
support from the NTTCP-HP & UPM-IMS. Finally, our
sincerest gratitude to all the healthcare providers from the
participating RHUs.
Statement of Authorship
All authors certied fulllment of ICMJE authorship
criteria.
VOL. 57 NO. 9 2023 113
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
Author Disclosure
All authors declared no conicts of interest.
Funding Source
is study is an output of the Hearing for Life Project and
funded by the Commission on Higher Education – Philippine
California Advanced Research Institutes (CHED-PCARI).
REFERENCES
1. World Health Organization. Newborn and infant hearing screening:
current issues and guiding principles for action. WHO Libr Cat
Data. 2010.
2. Quintos MRTR, Isleta PFD, Chiong CM, Abes GT. Newborn hearing
screening using the evoked otoacoustic emission: e Philippine
General Hospital experience. Southeast Asian J Trop Med Public
Health. 2003;34 Suppl 3:231-3.
3. American Academy of Pediatrics, Joint Committee on Infant
Hearing. Year 2007 position statement: principles and guidelines for
early hearing detection and intervention programs. Pediatrics 2007
Oct;120(4):898–921. doi: 10.1542/peds.2007-2333.
4. Chiong CM, Abes GT, Reyes-Quintos MRT, Ricalde RR, Llanes
EGDV, Acuin JM, et al., Universal newborn hearing screening and
intervention act of 2009: Manual of operations of RA 9709. Manila:
National Hearing Screening Reference Center. 2016.
5. Newborn Hearing Screening Reference Center. 2017 NHSRC Annual
Report. 2017.
6. Santos-Cortez RLP, Chiong CM. Cost-analysis of Universal
Newborn Hearing Screening in the Philippines. Acta Med Philipp.
2013;47(4):52–7. doi: 10.47895/amp.v47i4.1267
7. Olusanya BO, Wirz SL, Luxon LM. Community-based infant hearing
screening for early detection of permanent hearing loss in Lagos,
Nigeria: a cross-sectional study. Bull World Health Organ. 2008
Dec;86(12):956–63. doi: 10.2471/blt.07.050005.
8. Rivera AS, Lam HY, Chiong CM, Reyes-Quintos MRT, Ricalde
RR. e cost-eectiveness and budget impact of a community-based
Universal Newborn Hearing Screening Program in the Philippines.
Acta Med Philipp. 2017;51(1):28-35. doi: 10.47895/amp.v51i1.640
9. Olusanya BO. Screening for neonatal deafness in resource-poor
countries: challenges and solutions. Research and Reports in
Neonatology. 2015;5: 51–64.
10. Newborn Hearing Screening Reference Center. List of Accredited
NHS Centers 2016. Newborn Hearing Screening Reference Center.
[cited 2017 Jan]. Available from: https://nhsrc.ph/
11. Chotchoungchatchai S, Marshall AI, Witthayapipopsakul W,
Panichkriangkrai W, Patcharanarumol W, Tangcharoensathien
V. Primary health care and sustainable development goals. Bull
World Health Organ. 2020 Nov;98(11):792-800. doi: 10.2471/
BLT.19.245613.
12. Paliadelis, PS, Parmenter G, Parker V, Giles M, Higgins I. e
challenges confronting clinicians in rural acute care settings: a
participatory research project. Rural Remote Health. 2012;12:2017.
13. United Nations. Transforming our world: the 2030 Agenda for
Sustainable Development. In: Seventieth United Nations General
Assembly, New York. 25 Sep 2015 [cited 2017 Jan]. Available from:
https://undocs.org/en/A/RES/70/1
14. Ricalde RR, Chiong CM, Labra PJP. Current assessment of newborn
hearing screening protocols. Curr Opin Otolaryngol Head Neck
Surg. 2017 Oct;25(5):370-7. doi: 10.1097/MOO.0000000000000389.
15. Moeller MP, White KR, Shisler L. Primary care physicians' knowledge,
attitudes, and practices related to newborn hearing screening.
Pediatrics. 2006 Oct;118(4):1357-70. doi: 10.1542/peds.2006-1008.
16. Eisenman WD, Shisler L, Foust T, Buhrmannn J, Winston R, White
K. Updating Hearing Screening Practices in Early Childhood Settings.
Infants Young Child. 2008;21(3):186-93.
17. Kamal N. Newborn hearing screening: Opportunities and challenges.
Egypt J Ear Nose roat aAllied Sci. 2013 Jul;14(2):55-8. doi:
10.1016/j.ejenta.2013.01.002
18. Dolea C, Stormont L, Shaw D, Zurn P, Braichet JM. Increasing access
to health workers in remote and rural areas through improved retention.
In: First expert meeting to develop evidence-based recommendations
to increase access to health workers in remote and rural areas through
improved retention. Geneva: World Health Organization. 2009.
19. Rakototiana LB, Rajabo, Gottot S. Internet or dvd for distance learning
to isolated rural health professionals, what is the best approach?
BMC Med Educ. 2017 Sep;17(1):152. doi: 10.1186/s12909-017-
0991-3.
20. Estrella MM, Sisson SD, Roth J, Choi MJ. Ecacy of an internet-
based tool for improving physician knowledge of chronic kidney
disease: an observational study. BMC Nephrol. 2012 Sep;13:126.
doi: 10.1186/1471-2369-13-126.
21. Tria JZ. e COVID-19 pandemic through the lens of education
in the Philippines: the new normal. International Journal of
Pedagogical Development and Lifelong Learning. 2020;1(1):ep2001.
doi:10.30935/ijpdll/8311.
22. Rozul CDA, Yarza TKL, Catangay-Ombao JV, Cruz TLG.
Development and pilot implementation of the online certication of
universal newborn hearing screening personnel. Acta Med Philipp.
2022;56(11):53-65. doi:10.47895/amp.vi0.3361.
23. Sangra A, Vlachopoulos D, Cabrera N. Building an inclusive denition
of e–learning: an approach to the conceptual framework. Int Rev
Res Open Distance Learn. 2012 Apr;13(2):145-59. doi: 10.19173/
irrodl.v13i2.1161
24. Means B, Toyama Y, Murphy R, Bakia M, Jones K, Department of
Education, Oce of Planning, Evaluation and Policy Development,
SRI International. Evaluation of Evidence-Based Practices in Online
Learning: A Meta-Analysis and Review of Online Learning Studies.
2009. IES Cited: ED544210.
25. Lowes S. Online teaching and classroom change: the trans-classroom
teacher in the age of the internet. Innov J Online Educ. 2008
May;4(3):Article 1.
26. Tavrow P, Kekitiinwa Rukyalekere A, Maganda A, Ndeezi G, Sebina-
Zziwa A, Knebel E. A comparison of computer-based and standard
training in the Integrated Management of Childhood Illness in
Uganda. Operations Research Results 2(5). Bethesda, MD: Published
for the U.S. Agency for International Development (USAID) by the
Quality Assurance (QA) Project. 2002.
27. Schmidt, K. Brown D. A model to integrate online teaching and
learning tools into the classroom. J Technol Stud. 2004;30(2):86–92.
doi: 10.21061/jots.v30i2.a.4
28. Zayapragassarazan Z, Kumar S. Blended learning in medical
education. NTTC Bull. 2012;19(2):4-5.
29. Shaer K, Small JE. Blended learning in medical education: use of
an integrated approach with web-based small group modules and
didactic instruction for teaching radiologic anatomy. Acad Radiol.
2004 Sep;11(9):1059–70. doi: 10.1016/j.acra.2004.05.018.
30. Ruiz JG, Mintzer MJ, Leipzig RM. e impact of E-learning
in medical education. Acad Med. 2006 Mar;81(3):207–12. doi:
10.1097/00001888-200603000-00002.
31. Pereira JA, Pleguezuelos E, Merí A, Molina-Ros A, Molina-
Tomás MC, Masdeu C. Eectiveness of using blended learning
strategies for teaching and learning human anatomy. Med Educ. 2007
Feb;41(2):189–95. doi: 10.1111/j.1365-2929.2006.02672.x.
32. Sinclair P, Kable A, Levett-Jones T. e eectiveness of internet-
based e-learning on clinician behavior and patient outcomes: a
systematic review protocol. JBI Database of Systematic Rev Implement
Rep. 2015 Jan;13(1):52-64. doi: 10.11124/jbisrir-2015-1919.
33. Kirkpatrick DL. Evaluating training programs: e four levels.
San Francisco, CA: Berrett-Koehler; 1994.
34. Jemena F. Hearing for Life Project posts achievements. UP Manila
Newsletter No. 370. September-October 2018 [cited 2018 Dec].
Available from: https://www.upm.edu.ph/sites/default/files/
newsletters/
VOL. 57 NO. 9 2023114
A Computer-based Training Course on Newborn Hearing Screening and Teleaudiology
35. Zaharias P. Developing a usability evaluation method for e-learning
applications: From functional usability to motivation to learn. Int
J Hum Comput Int. 2003;25(1):1-11.
36. Zaharias P, Poulymenakou A. Developing a usability evaluation method
for e-learning applications: beyond functional usability. Int J Hum
Comput Int. 2009 Jan;25(1):75-98. doi: 10.1080/10447310802546716
37. Ergonomic Requirements for Oce Work with Visual Display
Terminals, ISO 9241-11, ISO, Geneva, 1998.
38. Erlingsson C, Brysiewicz P. A hands-on guide to doing content
analysis. Afr JEmerg Med. 2017 Sep;7(3):93-9. doi: 10.1016/j.aem.
2017.08.001.
39. Bengtsson M. How to plan and perform a qualitative study using
content analysis. NursingPlus Open. 2016;2:8–14. doi:10.1016/j.npls.
2016.01.001.
40. Macabasag RLA, Magtubo KMP, Marcelo PGF. Implementation of
telemedicine services in lower-middle income countries: lessons for
the Philippines.Journal of the International Society for Telemedicine
and eHealth. 20016;4(e24):-1-11.
41. Olusanya BO, Luxon LM, Wirz SL. Ethical issues in screening for
hearing impairment in newborns in developing countries. J Med Ethics.
2006 Oct;32(10):588-91. doi: 10.1136/jme.2005.014720. Erratum in:
J Med Ethics. 2007 Mar;33(3):186.
42. Farmer T, Koehler A. Design judgments in the creation of eLearning
modules. J Form Des Learn. 2022;6(1):1-12. doi: 10.1007/s41686-
022-00063-3.
43. Roddy C, Amiet DL, Chung J, Holt C, Shaw L, McKenzie S, et
al. Applying best practice online learning, teaching, and support
to intensive online environments: an integrative review. Front. Educ.
2017 Nov;2. doi: 10.3389/feduc.2017.00059.
44. Brassil CE, Couch BA. Multiple-true-false questions reveal more
thoroughly the complexity of student thinking than multiple-choice
questions: a Bayesian item response model comparison. I J STEM
Ed. 2019;6(16). doi:10.1186/s40594-019-0169-0.
45. Krathwohl DR. A revision of Bloom's taxonomy: An overview.
eory Pract. 2002;41(4):212-8.
46. Anderson LW, Krathwohl DR, Airasian PW, Cruikshank KA, Mayer
RE, Pintrich PR, Wittrock MC. A taxonomy for learning, teaching,
and assessing: A revision of Bloom's taxonomy of educational
objectives, abridged edition. White Plains, NY: Longman; 2001.
47. Rozul CDA, Yarza TKL, Catangay-Ombao JV. Outcomes ofonline
adaptation of the universal newborn hearing screening certication.
Acta Med Philipp.. 2022;56(12):36-41. doi:100.47895/amp.vi0.3364.
48. de Bruyn E, Mostert E, van Schoor A. Computer-based testing - the
ideal tool to assess on the dierent levels of Bloom's taxonomy. 2011
14th International Conference on Interactive Collaborative Learning,
Piestany, Slovakia. 2011;444-449. doi: 10.1109/ICL.2011.6059623.
49. Gaupp R, Körner M, Fabry G. Eects of a case-based interactive
e-learning course on knowledge and attitudes about patient safety:
a quasi-experimental study with third-year medical students. BMC
Med Educ. 2016 Jul;16:172. doi:10.1186/s12909-016-0691-4.
50. Stewart A, Inglis G, Jardine L, Koorts P, Davies MW. A randomised
controlled trial of blended learning to improve the newborn examination
skills of medical students. Arch Dis Child Fetal Neonatal Ed. 2013
Mar;98(2):F141-4. doi: 10.1136/archdischild-2011-301252.
51. Su WM, Osisek PJ, Starnes B. Applying the revised Bloom's
Taxonomy to a medical-surgical nursing lesson. Nurse Educ. 2004
May-Jun;29(3):116-20. doi: 10.1097/00006223-200405000-00010.
52. Näsström G. Interpretation of standards with Bloom's revised
taxonomy: a comparison of teachers and assessment experts. Intnal J Res
Method Educ. 2009;32(1):39-51. doi: 10.1080/17437270902749262
53. Keller JM. Motivational design of instruction. In: Reigeluth CM
(ed.), Instructional Design eories and Models: An overview of their
current status. Hillsdale, NJ: Erlbaum; 1983.
54. Hartnett M. e Importance of Motivation in Online Learning.
In: Motivation in Online Education. Singapore: Springer; 2016.
doi:10.1007/978-981-10-0700-2_2.
55. Indeed editorial team. What is a User Interface? (Denition, Types and
Examples) [Internet]. 2021 Sept 18 [cited 2021 Nov 13] In: Indeed.
com. Texas: Indeed, Inc. c2021. Available from: https://www. indeed.
com/career-advice/career-development/user-interface
VOL. 57 NO. 9 2023 115
Pilot Implementaon of an eHealth-enabled Service Delivery Model for NHS
