ThesisPDF Available

Effective Pedagogical Practices in Online English Language Teacher Education

Authors:

Abstract and Figures

Internet technology has made possible for students to be able to have access to continuous learning. Currently, online education has gained credibility and academic leaders’ belief about its value has increased in the US (2014 Survey of Online Learning). Studies are no longer solely focused on comparing face-to-face to online learning, but on learning about how to improve the online experience. In the Language Teacher Education field, online learning options are growing and Language Teaching post-secondary programs, certificates, and professional development courses are readily available. Consequently, it is essential for providers to demonstrate quality of instruction and for students to reflect on their needs as online learners to receive a quality educational experience. Challenges now are related to the new affordances allowed by the online environment in relation to the teaching and learning process and how these technology advances might enhance it. This study explores students and instructors’ perceptions of effective pedagogical practices in Online English Language Teaching Education (OLTE) courses. Instructors (N=18) and former students (N=125) from non-credit certificate, professional development and post-secondary credit OLTE courses were asked to complete an online survey that included items related to their perceptions regarding current pedagogical practices. Additionally, semi-structured interviews of 20 OLTE students, eight instructors, and two program coordinators were conducted to expand and support survey findings. The Community of Inquiry survey (Arbaugh et al., 2008) was the main tool used to evaluate students’ satisfaction and its weight as validated instrument served to measure pedagogical practices considered effective for OLTE courses in particular. These OLTE practices labeled as Language Teacher Educator presence were first ranked by experts and subsequently included in the longer survey. A main component of these practices is the consideration of context as recommended by language teacher educators’ authors Freeman, (2009), Graves (2009), Johnson (2009) and Murray (2013). Results of this study indicate positive perceptions about current pedagogical practices. However, the sense of collaboration between the groups surveyed presents a statistically significant difference. Analysis of surveys and interviews showed that instructional strategies can be improved. Issues for further research are related to collaborative tasks, the inclusion of oral participation, the perception of low instructors’ presence in discussions, and a lack of variety of instructional activities.
Content may be subject to copyright.
Effective Pedagogical Practices in Online English Language
Teacher Education
Item type text; Electronic Dissertation
Authors Rodriguez, Migdalia Elizabeth
Publisher The University of Arizona.
Rights Copyright © is held by the author. Digital access to this
material is made possible by the University Libraries,
University of Arizona. Further transmission, reproduction
or presentation (such as public display or performance) of
protected items is prohibited except with permission of the
author.
Downloaded 27-Jun-2016 18:08:48
Link to item http://hdl.handle.net/10150/613241
1
EFFECTIVE PEDAGOGICAL PRACTICES IN ONLINE ENGLISH
LANGUAGE TEACHER EDUCATION
by
Migdalia Elizabeth Rodríguez
__________________________
Copyright © Migdalia Elizabeth Rodríguez 2016
A Dissertation Submitted to the Faculty of the
GRADUATE INTERDISCIPLINARY DOCTORAL PROGRAM IN
SECOND LANGUAGE ACQUISITION AND TEACHING
In Partial Fulfillment of the Requirements
For the Degree of
DOCTOR OF PHILOSOPHY
In the Graduate College
THE UNIVERSITY OF ARIZONA
2016
2
THE UNIVERSITY OF ARIZONA
GRADUATE COLLEGE
As members of the Dissertation Committee, we certify that we have read the dissertation
prepared by Migdalia E Rodriguez, titled Effective Pedagogical Practices in Online English
Language Teacher Education and recommend that it be accepted as fulfilling the
dissertation requirement for the Degree of Doctor of Philosophy.
_______________________________________________________________________
Date: Jan 11, 2016.
Suzanne Panferov
_______________________________________________________________________
Date: Jan 11, 2016.
Robert Ariew
_______________________________________________________________________
Date: Jan 11, 2016.
Betul Czerkawski
Final approval and acceptance of this dissertation is contingent upon the candidate’s
submission of the final copies of the dissertation to the Graduate College.
I hereby certify that I have read this dissertation prepared under my direction and
recommend that it be accepted as fulfilling the dissertation requirement.
________________________________________________ Date: Jan 11, 2016.
Dissertation Director: Suzanne Panferov
3
STATEMENT BY AUTHOR
This dissertation has been submitted in partial fulfillment of the requirements for an
advanced degree at the University of Arizona and is deposited in the University Library to
be made available to borrowers under rules of the Library.
Brief quotations from this dissertation are allowable without special permission,
provided that an accurate acknowledgement of the source is made. Requests for permission
for extended quotation from or reproduction of this manuscript in whole or in part may be
granted by the head of the major department or the Dean of the Graduate College when in
his or her judgment the proposed use of the material is in the interests of scholarship. In all
other instances, however, permission must be obtained from the author
SIGNED: Migdalia Elizabeth Rodríguez
4
ACKNOWLEDGMENTS
My most sincere gratitude to all the ones who in way or another contributed to this project.
First, to the dear Chair of my committee Dr. Suzanne Panferov, who despite her numerous
responsibilities always found time and energy to guide me. Her continued support,
professional and emotional, and accurate feedback were paramount from the conception of
the idea and its interesting evolution. Thank you Dr P for helping me accomplish this task!!
I am greatly indebted to Dr. Robert Ariew, whose caring and generous character is certainly
an example to be followed. As Director of SLAT Dr. Ariew made me feel welcomed in the
program; his kind guidance and substantial chats as professor, advisor and committee
member proved invaluable in this learning experience. Many thanks to Dr. Betul
Czerkawski who also deserves my recognition. Her timely advice was indeed beneficial in
the favorable changes in this project.
I am also grateful to Dr. José Luis Ramírez from UniSon for his time and generous sharing
of his expertise. My deep appreciation to Dr. Jaime Olea also from UniSon, who was very
kind and considerate to share his knowledge about statistics patiently; his punctual help,
care and guidance encouraged me at all times.
I am extremely thankful to Jose Manuel Villafuerte, Daisy Giles, Angel Steadman and all
the participants who enthusiastically helped in this project. Without your valuable help, this
project would not have been possible.
I would like to express my gratitude to my institution, the University of Sonora, specifically
the Department of Foreign Languages and the administrative personnel in the office of
Academic Development for their various forms of support during my graduate studies.
I would like to acknowledge my amazing big family who were and are always encouraging
me to do my best. Thank you my lovely sisters Paty, Nena, Martha and sisters-in law
Vilma, Mari, Aurora, brothers and brothers-in-law and mother and father in-law for being
always supportive and understanding.
Special thanks to my kindhearted friends Nora, Paloma, Diana, Rosa, Nayely, Gaby,
Tanya, Asma, Amani, Ying, and my sister in Tucson, Shelley, for their sensible advice,
encouragement and emotional support. I greatly appreciate your friendship, help and
kindness.
Most importantly, I thank my loving husband and awesome children whose energy,
concern, unconditional love, support and help always gave me the strength to pursue my
goals. This is a family project !
5
DEDICATION
For my mom, Magdalena Rosales, whose example of hard work and determination showed
me the way to follow my dreams.
For my dad, Aurelio Rodríguez, who was concerned because I wouldn’t finish and was
getting old: I (finally) did it !
For my dearest angels: Valeria, Teclo and Valentina. Your unconditional support kept me
going! I’m so blessed to have you in my life. I’m so proud of you, and I LOVE you more ;)
For Teclo, my friend and dear husband: through all the good times and bad.
6
Contenido
List of Figures ....................................................................................................................... 10
List of Tables ........................................................................................................................ 11
Abstract ................................................................................................................................. 13
Chapter 1 Introduction .......................................................................................................... 14
Background of the Study .................................................................................................. 15
Statement of the Problem ................................................................................................. 20
Purpose of the Study ......................................................................................................... 23
Research Questions .......................................................................................................... 23
Significance of the Study .................................................................................................. 27
Conceptual Framework..................................................................................................... 28
Overview of the Research Method ................................................................................... 29
Limitations and Delimitations .......................................................................................... 32
Chapter 2 Literature Review................................................................................................. 34
Best practices in Online Teaching .................................................................................... 35
The OLC’s Scorecard for the Administration of Online Education Programs............. 37
The Quality Matters Higher Education Rubric............................................................. 41
The Community of Inquiry Framework ....................................................................... 42
CoI teaching, social and cognitive presences plus learning presence?......................... 45
Self-regulated learning ................................................................................................. 46
CoI relationship with OLC pillars and QM standards .................................................. 47
Best Practices in Online Teacher Education..................................................................... 48
Standards of Teacher Education Programs and Measures for Program Evaluation ..... 48
Online Teacher Professional Development ...................................................................... 49
English Language Teacher Education .............................................................................. 51
Standards in TESOL Teacher Education Programs ..................................................... 51
Online English Language Teacher Education Practices ................................................... 56
Challenges of Online English LTE ............................................................................... 56
OLTE and CoI Research............................................................................................... 58
Summary of the Literature Review .................................................................................. 62
Chapter 3 Methodology ........................................................................................................ 64
7
Research Design ............................................................................................................... 66
Population and Sampling Procedures ............................................................................... 67
Instrumentation ................................................................................................................. 69
Surveys ......................................................................................................................... 69
Interviews ..................................................................................................................... 79
Data Collection Procedures .............................................................................................. 82
Data Analysis Procedures ................................................................................................. 84
Ethical Considerations ...................................................................................................... 85
Chapter 4 Results .................................................................................................................. 87
Description of Participants ............................................................................................... 88
Demographic and General Information ........................................................................ 88
Interviews ......................................................................................................................... 88
OLTE Program Coordinators who answered interview questions ............................... 88
OLTE instructors who answered interview questions .................................................. 89
OLTE students who answered interview questions ...................................................... 90
Survey ............................................................................................................................... 91
OLTE instructors who completed the survey ............................................................... 91
OLTE students who completed the survey ................................................................... 92
Summary of Participants General Characteristics ............................................................ 97
Instructors ..................................................................................................................... 97
Students ........................................................................................................................ 98
Research Questions Results .............................................................................................. 99
Types and Context of OLTE courses ............................................................................... 99
Community of Inquiry and Online Language Teacher Education ................................. 103
Importance of the Establishment of a Community in an OLTE course ..................... 127
Language Teacher Educator Presence in OLTE courses............................................ 130
Language Teacher Educator Presence relationship to CoI elements .......................... 137
Self-regulated Learning and the Community of Inquiry ............................................ 142
Quality Assurance / Quality of Instruction in OLTE courses ........................................ 145
Instructional Activities in OLTE Courses .................................................................. 146
Most Engaging Instructional Activities in OLTE Courses ......................................... 148
8
Effective teaching practices ........................................................................................ 153
Quality Assurance practices and Perception of these Practices ................................. 162
Comparison of experts, instructors and students opinions about QA practices in OLTE
courses ........................................................................................................................ 165
Summary of Analysis of Results .................................................................................... 169
Chapter 5 Conclusion ........................................................................................................ 170
Summary of Findings ..................................................................................................... 171
Types and contexts of OLTE courses ......................................................................... 171
Relationship between the CoI and OLTE ................................................................... 172
Quality of instruction in OLTE courses ..................................................................... 176
Implications .................................................................................................................... 178
Types of OLTE courses .............................................................................................. 178
Community of Inquiry ................................................................................................ 178
Online teaching ........................................................................................................... 179
Limitations ...................................................................................................................... 180
Future Research .............................................................................................................. 180
Conclusion ...................................................................................................................... 181
Appendix A ........................................................................................................................ 183
Appendix B ......................................................................................................................... 194
Appendix C ......................................................................................................................... 206
Appendix D ........................................................................................................................ 209
Appendix E ......................................................................................................................... 211
Appendix F ......................................................................................................................... 213
Appendix G ........................................................................................................................ 216
Appendix H ........................................................................................................................ 217
Appendix I .......................................................................................................................... 218
Appendix J .......................................................................................................................... 221
Appendix K ........................................................................................................................ 222
Appendix L ......................................................................................................................... 224
Appendix M ........................................................................................................................ 228
Appendix N ........................................................................................................................ 229
9
Appendix O ........................................................................................................................ 230
References .......................................................................................................................... 231
10
List of Figures
Figure 1.1. Conceptual Framework………………………………………………………...29
Figure 2.1. Common quality indicators identified by Shelton (2011) ordered by most
common to least common……………………………………………………………….…39
Figure 2.2. Community of Inquiry (Garrison et al., 2000)………………………………....43
Figure 2.3. Practical inquiry model (Garrison et al., 2001)………………………………..44
Figure 2.4. The CoI revised by Shea and Bidjerano (2010)……………………………….46
Figure 2.5. Freeman’s Dimensions of the Scope of SLTE (2009)………………………....55
Figure 4.1. Scree Plot…………………………………………………………………..…109
11
List of Tables
Table 1. Statements to rank...................................................................................................72
Table 2. Summary of Ranking Results……………………………………………………75
Table 3. Summary of Coordinators information………………………………………..….89
Table 4. Instructors’ demographic information………………………………………..…..91
Table 5. Students’ Gender, Age and Nationality…………………………………………..92
Table 6. Students’ Years of Professional Experience……….…………………...……..….93
Table 7. Degrees Previously Earned……………………………………………………….94
Table 8. Percentage of participants who had training to study online from the institution
offering the OLTE course……………………………………………………………….…95
Table 9. Reasons to choose the OLTE course……………………………………...………96
Table 10. Learning Outcomes in an OLTE program are _____ than those in a Face-to-Face
program…………………………………………………………………………………….97
Table 11. Reliability coefficients of each factor and of total CoI………………………...107
Table 12. KMO and Bartlett Test……………………………………………………...….108
Table 13. Eigenvalues for Principal Component Analysis……………………………….109
Table 14. Pattern Matrix……………………………………………………………….…113
Table 15. Factor Correlation Matrix……………………………………………………...114
Table 16. Student Groups………………………………………………………………....116
Table 17. Highest Five Mean Ranks per Groups………………………………………....117
Table 18. Ranks with clear differences…………………………………………….…..…118
Table 19. CoI Presences Correlations………………………………………………….…120
Table 20. Language Teacher Educator Presence- Instructors’ perceptions…………...….131
12
Table 21. Language Teacher Educator Presence – Students’ perceptions…………….….132
Table 22. Mean Scores of Importance of LTE Desirable Practices from the Respondents
Perception…………………………………………………………………………………135
Table 23. Pattern Matrix…………………………………………………………...……..139
Table 24. LTEP items highest Spearman’s correlations with CoI items………………....140
Table 25. Spearman’s rho correlation - CoI and Self-regulated Learning aspects…...….144
Table 26. Spearman’s rho correlation - Cognitive presence outcomes and Self-regulated
Learning aspects……………………………………………………………..……………144
Table 27. Instructional Activities included in courses reported by students……………...147
Table 28. Instructional Activities included in courses reported by instructors…………...148
Table 29. Most Engaging Instructional Activities- Students and Instructors…………….149
Table 30. Importance of Language Teacher Educator Desirable Practices according to
Students…………………………………………………………………………...………166
13
Abstract
Internet technology has made possible for students to be able to have access to
continuous learning. Currently, online education has gained credibility and academic
leaders’ belief about its value has increased in the US (2014 Survey of Online Learning).
Studies are no longer solely focused on comparing face-to-face to online learning, but on
learning about how to improve the online experience. In the Language Teacher Education
field, online learning options are growing and Language Teaching post-secondary
programs, certificates, and professional development courses are readily available.
Consequently, it is essential for providers to demonstrate quality of instruction and for
students to reflect on their needs as online learners to receive a quality educational
experience. Challenges now are related to the new affordances allowed by the online
environment in relation to the teaching and learning process and how these technology
advances might enhance it.
This study explores students and instructors’ perceptions of effective pedagogical
practices in Online English Language Teaching Education (OLTE) courses. Instructors
(N=18) and former students (N=125) from non-credit certificate, professional development
and post-secondary credit OLTE courses were asked to complete an online survey that
included items related to their perceptions regarding current pedagogical practices.
Additionally, semi-structured interviews of 20 OLTE students, eight instructors, and two
program coordinators were conducted to expand and support survey findings. The
Community of Inquiry survey (Arbaugh et al., 2008) was the main tool used to evaluate
students’ satisfaction and its weight as validated instrument served to measure pedagogical
practices considered effective for OLTE courses in particular. These OLTE practices
labeled as Language Teacher Educator presence were first ranked by experts and
subsequently included in the longer survey. A main component of these practices is the
consideration of context as recommended by language teacher educators’ authors Freeman,
(2009), Graves (2009), Johnson (2009) and Murray (2013). Results of this study indicate
positive perceptions about current pedagogical practices. However, the sense of
collaboration between the groups surveyed presents a statistically significant difference.
Analysis of surveys and interviews showed that instructional strategies can be improved.
Issues for further research are related to collaborative tasks, the inclusion of oral
participation, the perception of low instructors’ presence in discussions, and a lack of
variety of instructional activities.
14
Chapter 1 Introduction
Current availability of technology and its continuous improvements in its
affordances have contributed to the enormous growth of online education in a short period
of time. Nowadays most higher education institutions (HEIs) offer courses that are
supported by a learning management system (LMS) (i.e. Desire2Learn, Blackboard,
Moodle). Depending on the time allotted for face-to-face (F2F) delivery, and the support
provided by Internet technology tools, these courses have different denominations that
range from fullyF2F, Web-facilitated, blended or hybrid to completely online. In Web-
facilitated courses, Internet technologies are used mainly as support to the F2F meetings. In
blended or hybrid courses, Internet technology tools are used as another form of delivery
along with the traditional F2F delivery. In nearly or fully online courses, Internet
technology tools are the ones used to facilitate the teaching-learning process. According to
Allen and Seaman (2011) annual survey for the Online Learning Consortium (formerly the
Sloan Consortium or Sloan-C), an organization dedicated to help to improve the quality of
online education, there was a growth from 9.6% (16.6 million) of online enrollment of the
total enrollment of degree granting postsecondary institutions in 2002 to 31.3% (19.6
million) in 2010 in the United States. The 2014 Survey of Online Learning conducted by
the Babson Survey Research Group and co-sponsored by the Online Learning Consortium
(OLC), Pearson and Tyton Partners reported that 70.8% of chief academic leaders believe
that online education is a critical component of their institution’s long-term strategy.
Online Language Teacher Education (OLTE) programs and certificates began in the
mid-1990s (Nunan, 2012). Currently, 68 distance learning options were found (August
2015) in a general web search of Teachers of English to Speakers of Other Languages
15
(TESOL) programs in a webpage announced as a directory of graduate schools in the
United States,(www.mastersintesol.net/programs/). It is important to point out that other
undergraduate, graduate language teacher education, certificate and post-secondary
programs are available and that some of them are not named “TESOL”. Some examples are
MA in Applied Linguistics, MA in English Language and Linguistics, MA in English as
Second Language (ESL) /English as a Foreign Language (EFL) Pedagogy. Thus, the
number of OLTE programs is higher and from what was found in the OLC survey,
expected to continue growing.
According to England (2012), students select online education because they want to
experience learning in a well-respected institution, the degree is unavailable in their city or
country, or their workload does not allow them to attend a face-to-face (F2F) program. In a
similar vein, Dede, Ketelhut, Whitehouse, Breit, and McCloskey (2009) state that online
teacher professional development (oTPD) programs are convenient because teachers’
education has to be in continuous improvement and the online factor allows them to have
asynchronous or synchronous interactions that can fit in their demanding schedule. For this
reason there is a “need to be sure that time, effort, and scarce resources are expended only
on quality programs that teach with and about best practices” (p.8).
Background of the Study
With the many options of online programs available as it was aforementioned, it is
essential for institutions to demonstrate that their programs offer quality and there is also a
need from students to ensure that the program they are selecting will produce satisfactory
results. Regarding the recognition of open and distance learning (ODL) as a good option to
16
provide instruction, Latchem and Jung (2012) state that “it is essential to evidence the
quality of the providers, teaching and learning and outcomes, demonstrate the benefits and
advantages of the new approaches to those committed to traditional means of education,
and work for continuous improvement” (p.21).
Quality Assurance (QA) and Quality Control (QC) are two concepts that need to be
distinguished. QA measures are the ones taken proactively to ensure quality in the process.
QC, on the other hand, is corrective and it is performed by an accreditation agency; the
focus is in the final product. According to Chalmers and Johnston (2012, p.1) “Quality
Assurance (QA) has long been applied in the corporate sector to ensure stakeholder
satisfaction, compare standards with those of other organizations and raise standards in the
face of competition.” In relation to higher education, they further state that QA “is most
typically judged in terms of fitness of purpose or value for money (judged by governments
and, with the introduction of fees, students)” (p.2). Fitness of purpose refers to how well
the outcomes either determined by the service provider or the outcomes expected by the
customers (in this case, students), are fulfilled. Student satisfaction is then also related to
the quality of programs. Additionally, students from an institution proven to provide quality
services are probably better prepared to perform in their field since they were exposed to a
higher and valued learning impact.
In relation to best practices, in the United States, two review processes designed to
assure the quality of online education are the ones provided by Quality Matters (QM) and
the Online Learning Consortium (formerly Sloan-C). QM is a faculty-centered peer review
benchmark for online course design. It is currently widely used in the US with more than
900 subscribers (QM Higher Education Program, 2015). QM is faculty driven and
17
collaborative; that is, certified faculty review online course design of other faculty, who are
subscribed QM members as well, with the use of a continuing and professional education
rubric (CPE rubric) which is updated every two years. This subscription gives the
institutions access to the complete CPE rubric to improve their online course design and
make the teaching learning more effective. The rubric is based on eight standards: course
overview introduction, learning objectives (competencies), assessment and measurement,
instructional materials, course activities and learner interaction, course technology,
learner support and accessibility and usability (Standards from The QM Higher Education
Rubric, Fifth Edition 6/15).
According to Moore (2011, p.92) “the Sloan-C effective practices collection enables
educators to share practices that help make quality online education more affordable,
accessible and effective”. The Online Learning Consortium (formerly The Sloan-C) quality
framework consists of five pillars or principles for achieving online course delivery quality.
The general goals of the principles are the following: Learning Effectiveness: online
learning outcomes meet or exceed institutional, industry, and/or community standards;
Scale: there is a continuous improvement of services while reducing cost for institutions
and students; Access: all learners who wish to learn online can have access to it; Faculty
Satisfaction: faculty are satisfied with their participation and receive support in course
preparation and delivery; Student Satisfaction: students are successful in learning online
and are pleased with their experience due to their interaction with instructors and peers.
QC or reviews are performed by accreditation or certification bodies that certify that
the programs meet the required standards. In the United States, the system of accreditation
bodies for higher education is complex. QA and accreditation agencies are independent of
18
the federal and state government. Regarding distance and online education, the Distance
Education and Training Council (DETC) is a national accrediting body for higher education
online schools and programs (See DETC Accreditation Handbook for a checklist of
accreditation standards). It is recognized by the U.S. Secretary of Education and the
Council for Higher Education Accreditation (CHEA). However, there are other eight
regional higher education accreditation boards that have the faculty to accredit online
programs as well. This complex system makes it difficult for students to evaluate a
prospective online program to study. Prospective students usually rely on the information
presented by the providers’ websites about their accreditation and QA measurements;
unfortunately, according to Murray (2013, p.14), on many websites this information is
comprised of “mere statements of quality that are impossible for potential customers to
verify.”
The United Kingdom uses a different approach to assure quality. The Quality
Assurance Agency (QAA) for higher education is responsible for assuring that academic
standards are met by auditing and reviewing programs on a regular basis. This agency
identifies best practices and publishes guidelines for high quality learning experiences.
Institutions themselves perform QA practices guided by the QAA. Interestingly, the QAA
does not consider online learning separately from F2F education, but has focused on the
broad principles of academic quality (Kirkpatrick, 2012). The European Foundation for
Quality in e-Learning (EFQUEL) was established to improve best practices in Europe and
beyond. It offers the European University Quality in e-Learning (UNIQUe) label which is
a certification system. EFQUEL membership is also open to non-European higher
education institutions (Ehlers, 2012). Universities from 31 countries such as Saudi-Arabia
19
and Malaysia are among its institutional members (http://efquel.org/membership/members-
list/).
Regarding LTE programs, some attempts have been made to establish what would
be ideal or best practices. In the English Language Teacher Education field, TESOL
International Association as a member of the Commission for the Accreditation of Educator
Preparation (CAEP/formerly NCATE) since 1999 has delimited some regulations. As
CAEP requires, TESOL International Association has established standards for programs in
P-12 ESL Teacher Education; these were approved in 2001, and revised again in 2005;
their latest revision was in 2010 (Standards for the recognition of initial TESOL programs
in P–12 ESL Teacher Education, 2010). CAEP requires programs in P-12 ESL Teacher
Education to follow the TESOL standards. TESOL International Association has
established standards for the ESL/EFL teachers of adults as well. However, currently there
is a lack of rubrics or standards for the Online LTE (OLTE) or TESOL programs.
In the UK, the Accreditation Council for TESOL Distance Education Courses
(ACTDEC) and the Online TEFL & TESOL Standards Agency (OTTSA ACTDEC and
OTTSA are two accreditation agencies specialized in OLTE. ACTDEC is a non-profit
organization governed by officers and Council “made up of co-opted members prominent
in the ELT profession and members elected by accredited providers” and fees are required
to pay the accreditation panel; thus, a cost of membership is required. OTTSA charges the
assessment of courses individually and if a visit is required, the applicant pays for the visit
and the moderator’s travel costs; their website only offers information about the key
personal or principal moderator. ACTDEC has currently nine accredited members
20
(ACTDEC, 2015) one of them from the US, and OTTSA has accredited 15 courses from a
total of four institutions (OTTSA, 2010).
In other countries where English is not the language of the majority, English is
taught as a foreign language and TESOL or LTE undergraduate and graduate programs are
offered online and F2F as well. In Mexico for instance, the Board for the Accreditation of
Education Programs in Humanities, COAPHUEM (in Spanish), is the agency in charge of
accrediting higher education humanities programs. This board evaluates and assures quality
of instruction in traditional settings; however, they have observed the potential of the use of
new technologies for the fully delivery of the courses and are just recently reviewing online
academic programs.
Statement of the Problem
Several authors (England and Hall, 2012; Hall and Knox, 2012; Healey, 2012) have
observed that the online environment requires different skills and strategies from instructors
to reach student satisfaction and positive learning outcomes compared to the ones used in
F2F classrooms. Thus, this different environment requires its own QA tool. There is the
belief that online courses are planned and delivered in the same manner as F2F courses and
that online instructors merely transfer the content used in F2F courses (i.e. Power Point
presentations, worksheets) to a LMS to mimic traditional instruction. Rudestam and
Schoenholtz-Read (2010) emphasize that replicating F2F traditional instructional practices
is not the “most effective use of the online environment” (p.11). They point out that an
effective use would be to provide opportunities for more collaborative and active learning.
Chen (2012) and Shin (2008) findings in TESOL teacher education confirm that common
21
effective components of online courses and programs include a student-centered interactive
and collaborative environment.
The assumed goal in online education is the creation of courses with high
instructional quality, practical applications to engage students, and the development of
standards for effective online courses. One of the objectives of QA is learning value,
perhaps the most important for faculty and students. Consequently, as Swan (2003, p.13)
asserts “learning effectiveness must be the first measure by which online education is
judged.” This is usually related to the creation of a compelling environment that promotes
collaboration and inquiry.
An engaged learner-centered approach to online learning implies the creation of an
optimum teaching-learning environment. Interestingly, aspects such as community,
collaboration, learning facilitation and teaching presence are salient in the review of
literature about best practices in online teaching (Hampel and Stickler, 2005; Jones and
Youngs, 2006; Pawan, Paulus, Yalcin& Chang, 2003; Palloff & Pratt , 2011; Reinders,
2012). For this reason, the main areas in relation to the development of an optimal online
environment identified by Palloff and Pratt (2011, p.7) will be studied and analyzed: the
establishment of presence, the creation and maintenance of a learning community, and the
effective development and facilitation of online courses.
The creation of the sense of presence in an educational environment is strongly
related to the creation of a learning community (Garrison, Anderson & Archer, 2000;
Palloff and Pratt, 2001). Literature review about this topic relies firstly on Constructivists
and Social Learning Theories (Bandura, 1971; Vygotsky, 1978) which in general conceive
22
learning as a constructive process in which people co-construct knowledge by reflecting on
their social context and how their collaboration with others helps them understand reality.
Lave and Wenger (1991, p.29) affirm that new participants in the learning community learn
and develop skills required to be fully participants and later become a more engaged
member who contributes to the practice. This social participation in Communities of
Practice (Wenger, 1998) exists when people collaborate by sharing ideas and interests and
they develop skills through interacting repeatedly with each other.
Garrison, Anderson and Archer (2000), propose their Community of Inquiry (CoI)
model that considers that participants in the community should be challenged and engaged
in inquiry to appropriate new knowledge. The CoI includes three essential elements in the
interaction and collaboration in the online environment: cognitive, teaching and social
presence. This model provides a framework for learning online. Cognitive presence refers
to the possibility of constructing knowledge through interaction. Teaching presence is
about the instructor’s ability to design and facilitate learning processes to establish and
maintain a learning community. Social presence can be understood in the sense that
teachers and learners can have an emotional projection; there is teacher immediacy or the
perception that there is a reduction of distance between instructors and students. Positive
interaction with an instructor who is enthusiast about the material and promotes
participation can be perceived as teacher immediacy (Allen, Witt & Wheeless, 2006). In
addition, according to Zhang and Oetzel (2006), immediacy research from colleges in
different countries suggests that teacher immediacy is positively correlated with learning.
It is assumed that students’ self-regulation strategies play an important role in
students’ success in an online learning experience (Zimmerman, 2000); thus, these
23
strategies have to be explored and correlated with their perceptions of learning outcomes.
Similarly, characteristics of successful pedagogical practices perceived by instructors
should be identified in order to inform and foster them for a productive online experience.
Purpose of the Study
The objective of this research is to identify effective usually referred to as best
pedagogical practices considered by QA and QC agencies to explore the challenges and
issues in defining and measuring quality of online TESOL and OLTE programs. Findings
from a literature review on this topic will be compared with results from a mixed-methods
research about perceptions of the current pedagogical practices in OLTE programs and their
long-term learning impact.
This research will focus its attention on the perceptions of pedagogical practices
from the main participants in OLTE instruction: program coordinators, instructors, and
students. Instructors and program coordinators will be asked how they assure quality of
instruction and long-term learning impact in OLTE courses; students’ perceptions of
learning effectiveness, and satisfaction will be explored as well. The effective pedagogical
practices information will be correlated with students’ perceived learning outcomes and
satisfaction to inform about aspects related to the judgment of learning effectiveness in
OLTE environments.
Research Questions
The overarching question for this research is the following:
24
What are the most effective pedagogical practices in Online Language Teacher
Education (OLTE) courses as identified by experts, instructors and students?
This question was investigated based on the exploration of pedagogical practices in
OLTE courses. In order to answer the main research question more specific queries were
explored. These are about the contexts of OLTE courses, how the Community of Inquiry
framework relates to OLTE courses, and the perception and pertinence of quality assurance
measures.
1. What are the general characteristics of the OLTE courses currently offered to in-service
language teachers and future language teachers?
2. How are the Community of Inquiry (CoI) elements incorporated into OLTE courses?
a) What is the relationship between the CoI elements (i.e. teaching, social, and
cognitive presence) and an OLTE program?
b) How do OLTE instructors and students perceive the establishment of a
Community as a critical element for a successful OLTE course?
c) How are the effective pedagogical practices integrated in to the OLTE courses?
d) Is there a relationship between the Language Teacher Educator Presence items
proposed and the CoI elements?
e) What is the relationship between the Self-regulation items and the CoI elements
and the proposed Language Teacher Educator Presence?
3. How is quality of instruction assured in OLTE courses?
25
a. What instructional activities are included in OLTE courses and are these
activities considered engaging?
b. What tasks and teaching practices do learners and instructors find most effective
and why?
c. How do instructors and students perceive quality assurance actions in OLTE
courses?
d. What are the strategies used by faculty and program coordinators to assure
quality of instruction and long-term learning impact in OLTE programs?
Definition of Terms and Language
Online education has been changed and it is expected that the terms connected to it
will continue to be updated as technology advances. For example, some definitions that
used to consider text-based communication only have been changed to include other types
of communication such as audio and video. The following definitions are the ones that
carry the meaning of how the terms are understood for this research.
Computer mediated communication (CMC): “umbrella term that encompasses various
forms of human communication through networked computers, which can be synchronous
or asynchronous and involve one-to-one, one-to-many, or many-to-many exchanges of text,
audio, and/or video messages.” (Lee & Oh, 2015).
Traditional/ Face-to-Face (F2F) instruction: in this type of instruction teaching and learning
takes place at the same time usually in a traditional classroom setting; the pace is directed
by the instructor.
26
Blended/ Hybrid instruction: traditional and online instructions are combined to an extent
that it is not just a web facilitated course where some materials such as syllabus and
lectures are in the course Learning Management System, but actual online interaction takes
place.
Fully Online instruction: it refers to the time and space for instruction is flexible; thus, it is
mainly self-paced: both instructors and students decide when to participate in this
instruction with some obvious considerations related to the general planning of the course.
Synchronous and asynchronous learning tools might be used.
Asynchronous Learning Networks(ALN):it is a web-based interaction that requires frequent
online collaboration with others; each person working at their own pace and preferred time
which means that the members of the class are not present that he same time. (Hiltz &
Goldman, 2005)
Synchronous e-learning: communication in real time through the use of chat rooms or
videoconferences. According to Hrastinski (2008) it “has the potential to support e-learners
in the development of learning communities.”
Online Discussion Forums: These forums allow course participants to communicate by
posting messages. This communication is usually text based and asynchronous.
Learning Management Systems (LMS): it is software that allows for the organization and
distribution of materials. Some examples are Moodle, Blackboard, and D2L. Even though
its components can be different, some of the ones shared in these LMS include monitoring
of users activities, course calendars, discussion forums, quizzes and grading.
27
Significance of the Study
The importance of this research relies in the fact that currently research related to
OLTE in general is scarce (England, 2012). Due to its significance, this topic was the
research priority of the International Research Foundation for English Language Education
(TIRF) for 2012-2013 (TIRF, 2014). It is important to point out that other research studies
have analyzed language teacher training to teach language online (Hampel & Stickler,
2005; Compton, 2009; Comas Quinn, 2011). However, the interest of this research is not
about training teachers to teach language online; the students in these courses or programs
might or might not teach online and might be pursuing a certificate, master’s degree or
PhD, an additional qualification or ongoing professional development. The focus of this
research is on current Online Language Teacher Education programs best pedagogical
practices considered by the main participants in this endeavor: coordinators, faculty and
students. This can be approached by informing about the meaningful experiences in the
teaching learning process in an online environment which translates into the improvement
of OLTE students teaching practices (either F2F or online).
Having some guidelines to follow as online instructor is a strong need in current
OLTE courses to assess and improve instruction in this environment. Through quality
assurance of instruction, the online experience will be enhanced. This study aims to explore
perceptions of effective pedagogical practices from OLTE students and instructors. That is,
it aims to describe the students’ expectations of the online presence of instructors of
language teacher education courses and instructors’ perceptions about their teaching
presence. A secondary goal is to propose the Language Teacher Educator Presence
construct as a first approach to help support the quality assurance endeavor in OLTE
28
courses by means of a short self- evaluation for instructors. Additionally, since online
delivery is widespread and it is more expected these days to have some type of learning
experience in online environments, the possibility that online students who are current
language teachers or future language teachers teach a hybrid (blended) or even a fully
online course is higher nowadays. Hence, besides informing instructors how they can
enhance their practice, students’ perceptions of effective pedagogical practices might
transfer to their own future teaching practices with an online environment component.
Conceptual Framework
The motivation for this research built after being part of the teaching learning
exchange in OLTE course environments with very different experiences. While most of
these experiences were mainly positive, some opportunities to improve for others were
identified. Some authors have pointed out that the technological aspect is usually the one
considered in studies related to online teaching (Kern, Ware & Warshauer, 2004; Rudestam
& Schoenholtz-Read, 2010). Due to this, there is an acknowledgement of the need to focus
more on helping instructors develop the pedagogical aspect as well (Comas Quinn, 2011;
Jones & Youngs, 2006) ; consequently, despite the fact that resources used can be
innovative, course design along with instructor’s dedication is going to be essential for an
effective learning scenario whether this is online or traditional.
The conceptual framework that follows (See figure 1.1) should be thought of as the
initial representation of how this complex task of learning about effective pedagogical
practices in OLTE was conceived. Although this framework was used to inform the
literature review, it only served as a first step to learn and build on other important concepts
29
such as instructional design and instructors’ conceptualization of their role(s) in this
environment.
Figure 1.1 Conceptual Framework
Overview of the Research Method
A mixed-method approach that includes subjective and objective measures seems to
be an optimal approach when dealing with opinions and perceptions (Plano Clark,
Creswell, Green & Shope, 2011). Q methodology (Brown, 1993; Van Exel& De Graf,
2005) allows for study of opinions by a small population. In this research it was used to
learn about beliefs regarding effective pedagogical practices from instructors and students
considered experts in online language teacher education. This methodology is described in
detail in chapter 3. Results from this study informed the selection of items for a longer
Effective Pedagogical
Practices in OLTE
(meaningful learning
that promotes
improvement)
Participants’ perspectives
(Coordinators, instructors and
)
Quality Assurance Practices in
Online Education
TESOL
guidelines(content
knowledge, strategies
and methods)
Online learning theories
and frameworks
(Community of Inquiry)
30
survey completed by OLTE instructors and students from a range of OLTE courses. A
semi-structured interview was conducted to uncover more details about the reasons why
some pedagogical practices were considered effective.
Overview of Dissertation
Chapter 1 Introduction
This chapter provides an overview of the study. It includes the reasons behind the
interest and significance of the study and its purpose. The research questions and the
connections between the concepts considered necessary for approaching the main query are
introduced. The definitions of key terms are relevant to understand the evolution of some
concepts and how they are interpreted for this study. The limitations and delimitations of
the study and its organization are also included.
Chapter 2 Literature Review
Chapter 2 incorporates quality assurance practices in online education in general
and language teacher education in particular, language teacher educator base knowledge
and common pedagogical practices, to build them into the main theoretical background in
this research which is the Community of Inquiry (CoI) framework. That is, characteristics
of quality assurance practices in online education related specifically to pedagogy are
highlighted to connect them to the ones recommended for teacher education and
specifically language teacher education. The overlapping of some of these characteristics is
then discussed in relation to the CoI framework. Previous studies related to OLTE courses
and the Community of Inquiry are also reviewed to establish the possibility to compare
results or build on them.
31
Chapter 3 Methodology
This chapter describes the methodology followed to conduct this research. A mixed-
methods approach was adopted and surveys and semi-structured interviewed were
designed. In order to include a section in the main survey, a pilot study about online
language teacher educator practices was conducted; the details of this previous work is
included in this chapter along with the decision of how to measure, compare and correlate
its results. Additionally, a detailed description of the instruments, data collection, data
analysis and ethical procedures is presented.
Chapter 4 Results
Chapter 4 includes the details about the types and contexts of OLTE courses, the
participants in this research and their perceptions regarding effective pedagogical practices
in OLTE courses. Results from a post-course completion surveys and semi-structured
interviews are described and analyzed. Data gathered allowed for descriptive and
inferential statistical tests considered pertinent to explore the community of inquiry and the
relationship between its presences in OLTE courses. Issues related to the reliability and
validity of the quantitative data are presented in this chapter along with the qualitative data
that supports answers to the research questions. Quantitative and qualitative data analysis is
connected to literature and previous studies to learn about some characteristics and quality
assurance measures in OLTE courses, and challenges that the online environment pose.
Chapter 5 Conclusion
32
Chapter 5 recapitulates key findings and presents implications of this research.
Suggestions for future research directions and other final recommendations are also
outlined.
Limitations and Delimitations
Due to the low number of students from specific OLTE courses willing to
participate, it was not viable to conduct inferential statistical tests for specific courses
separately. However, descriptive statistics were run. In order to have a more representative
sample, the inclusion of students from different types of OLTE courses was contemplated
after noticing that response rate was considerably low. Consequently, generalizations
should be handled carefully.
Decisions about how to group and classify the courses had to be made as well.
Some certificate programs have core courses and electives and students have a time set to
finish the courses to be granted the language teaching certificate; other programs have
different levels of certificates such as basic and advanced. Thus, trying to have the
participants OLTE courses considered for this research in one clearly defined group was
not possible. Chapter 4 deals with the details and considerations that were taken when
labeling the groups and classifying the courses. In order to overcome this complexity, a
complete table of results with the main group classification is included and statistical tests
were run to detect significant differences in the groups.
The aim of this study is not to generalize but to learn about common characteristics
of OLTE courses and understand what its participants consider of value to have an
33
engaging environment that fosters meaningful learning resulting in quality of instruction,
which is the ultimate goal of instructors and students.
It is assumed that other OLTE characteristics such as technological affordances and
support, either technical or related to administrative issues, are important in an online
teaching learning exchange. However, the focus in this research is in the actual instruction
and how it influences learning outcomes.
34
Chapter 2 Literature Review
Established and emerging technologies have the potential to enhance teaching
practices. According to Rudestam and Schoenholtz-Read (2010) blended or hybrid courses
have the opportunity to represent the best of both worlds since the combination of F2F and
online formats allows for capitalizing on the advantages of each approach” (p.4).
Currently, a great majority of F2F courses in HEIs include the online component in
different degrees such as the use of Internet technologies to support traditional instruction
or to combine both forms of instruction. Additionally, online delivery is now widespread.
Fully online courses and programs have now become a common option in HEIs due to the
advantages these represent for students who want to learn but do not have the possibility to
attend regular classes because of time or economic constrains (i.e. the need to move to
attend a specific institution). Moreover, this access to anytime, anywhere learning is also
attractive since it opens the possibility to study with instructors who are experts in the field.
This research about effective pedagogical practices in OLTE has as main objective
to explore perceptions of quality of instruction practices from the main participants in this
teaching learning process: students, instructors, and program coordinators. Effective
pedagogical practices are defined by the level of agreement these participants share about
practices that lead to desired learning outcomes. Quality teaching under a social
constructivist approach involves a learner-centered focus with the aim to promote critical
thinking, autonomy, collaboration among peers and instructor, and problem solving, among
others characteristics (Huang, 2002). Due to the nature of fully online courses, the
requirements for different strategies to reach the learning goals call for different teaching
35
approaches. The building of learning communities is commonly advised as an aid to overall
learning in traditional and online settings (Palloff & Pratt, 2007, 2005; Rovai, 2002; Shea,
Li, Swan & Pickett, 2005). For this reason, this research is mainly based on the study of the
characteristics of those communities and the actions that help in the perception of a
successful experience from students and instructors points of view. According to Pallof and
Pratt (2003, p. 17) an online community is different from an online learning community
since the former refers to “a listserv or online group where people meet to share a common
interest” while in the latter people engage in “collaborative learning and reflective practice
involved in transformative learning.”
As it was mentioned, the teaching learning process in an online environment
requires different instructional strategies than those used in traditional classroom settings.
Furthermore, it is likely that it also requires different learning strategies. As part of a study
about perceptions and expectations of OLTE courses it seems to be pertinent to include a
review of literature regarding best practices in online teaching in general, online teacher
education, English language teacher education, and online English language teacher
(OLTE) education practices.
Best practices in Online Teaching
The concepts of best practices and quality assurance practices in online education
are most of the time used interchangeably due to the assumption that quality assurance
practices are in fact best practices. The Online Learning Consortium (OLC)formerly known
as Sloan-C and Quality Matters (QM) are two organizations or agencies in the United
States currently considered as leading in regards to guiding towards best practices in online
36
learning. Both organizations offer a set of guidelines to help in the measure of actions that
lead to quality assurance practices. These guidelines follow principles or standards that
consider online delivery quality as a whole.
According to online learning researchers, a common indicator of successful online
programs is the creation and maintenance of a learning community (Arnold & Ducate,
2006; Garrison, Anderson, & Archer, 2001; Garrison & Cleveland-Innes, 2005; Palloff &
Pratt, 2011).According to Palloff and Pratt (2011) to foster this learning community,
online instructors should establish their interest to reach the course learning goals by
sharing their knowledge and help in the co-construction of meaning; through their
interactions they should be able to express a sense of who they are. That is, they should
establish a sense of presence, a vital element in an online environment that helps establish a
learning community (Palloff & Pratt, 2011). Related to instructor presence, Zhang and
Oetzel (2006) research from colleges in different countries suggests that teacher immediacy
is positively correlated with learning.
The Community of Inquiry framework developed by Garrison et al. (2000) has been
used extensively in research about effective online learning experiences (Swan & Ice,
2010). It considers this sense of presence through teaching, social and cognitive
components for a “successful higher educational experience” in a computer mediated
environment (Garrison, at al., 2000, p. 87).
Even though Chickering & Gamson (1987) seven principles for good practice in
undergraduate education were developed considering a traditional classroom setting, they
are often cited in papers related to best practices in online environments (Bangert,2004;
Graham, Cagiltay, Lim, Craner,& Duffy, 2001; Pollard, Minor & Swanson, 2014) Their
principles encourage instructor-to-student and student-to-student interaction not only in the
37
form of plain communication but in actual cooperation either by discussing about topics,
and providing feedback. They also recommend considering keeping students motivated by
including challenging tasks that allow the sharing of talents through the selection of
important topics for students.
Another aspect considered related to successful learning outcomes is students’ self-
regulation learning skills. These skills refer to the autonomy that has to be developed when
learning in an online environment. Since most activities are asynchronous, the learners are
not limited to a place or time; consequently, they have to decide how they are going to
approach the tasks. Barnard, Lan, To, Osland, and Lai (2009) developed an Online Self-
regulated Learning Questionnaire (OSLQ) to measure self-learning practices in order to be
able to establish a relationship between these practices and successful learning outcomes.
Tools presented as scorecards, surveys and questionnaires will be explored in more
detail in the following sections. Additionally, a review of previous research that has
considered a connection between them is presented.
The OLC’s Scorecard for the Administration of Online Education Programs
The OLC Five Pillars of Quality Online Education aim to “provide the support for
successful online learning.” The pillars were established in the mid 90’s to assess
Asynchronous Learning Networks (ALNs) (Lorenzo & Moore, 2002) and are currently
used to evaluate blended learning. As previously mentioned and briefly explained in
chapter 1, these pillars are the following: Learning Effectiveness; Scale; Access; Faculty
Satisfaction; and Student Satisfaction. The complete framework is available in the Online
Consortium website (http://onlinelearningconsortium.org/5-pillars/).
38
The Learning Effectiveness and the Student satisfaction pillars are the ones most
related to perceptions of effective pedagogical practices, the main objective of this research.
Thus, these pillars should be explored in more depth. The goal of the Learning
Effectiveness pillar is to demonstrate that “online learning outcomes meet or exceed
institutional, industry, and/or community standards.” This goal is measured with surveys
and interviews to faculty and learners that inquiry about learning gains where student
learning is at least equivalent to learning in traditional programs. The Student Satisfaction
pillar as its name implies refers to the level of fulfillment that the learning experience
brings to the student. Actors involved in this experience are the substantive interaction the
online learner has with instructors and e-mates, along with the matching of expectations in
relation to learning outcomes, services and orientation. This goal is evaluated through
surveys and interviews to current and former students, instructors’ perceptions and
outcomes measures. These measures translate to satisfaction levels that should be at least
equivalent to the ones expected in traditional delivery. Currently, the OLC recommends
the use of A Quality Scorecard for the Administration of Online Programs, a tool that
includes the criteria for the provision of quality in online education. Kaye Shelton (2010) is
the main developer of this quality scorecard to help in program evaluation. The author
reviewed in detail 13 paradigms for evaluating the quality of online education programs.
Respected educational entities such as the OLC’s Five Pillars of Quality (2002) and the
Council for Higher Education Accreditation (CHEA) study were included in this review
along with assessment models and recommendations from experts in the topic. It is
important to notice that even though the review included 13 articles and studies, the aspects
that each one of them considered for the evaluation of quality varied significantly.
39
Shelton’s review resulted in a thorough summary of common themes. For instance,
while institutional commitment, support and leadership was included in ten of the 13
paradigms reviewed, technology, evaluation and assessment were in six of the 13. In
regards to the latter result, she argues that technology “should be considered a critical
component to quality and success” (Shelton, 2011) since it is through technology that
online learning is possible and also technology infrastructure has to be reliable; thus, it
should be included in all the paradigms. Another interesting aspect that directly relates to
the objective of this research is that she mentions that faculty and student satisfaction were
included in only two of the 13 paradigms reviewed. Figure 2.1 presents Shelton’s result
from the comparison.
Figure 2.1. Common quality indicators identified by Shelton (2011) ordered by most
common to least common. Taken fromA Review of Paradigms for Evaluating the Quality
of Online Education Programs” by K. Shelton, 2011, Online Journal of Distance Learning
Administration, Volume IV, Number I. Copyright 2015 by Kaye Shelton.
40
Shelton (2011) concludes that due to the lack of a tool that included a consistent
method to evaluate the quality of education programs, she crafted this “Quality Scorecard
for the Administration of Online Education Programs.” This scorecard was developed
considering common quality indicators from previously researched models and
benchmarks, and the feedback from 43 experts in online education programs in higher
education through a Delphi study. In this type of study experts are asked their opinion in
more than one round in order to reach to consensus about a topic. An updated version as a
result of collaboration with other experts is available online to OLC members and defines
75 elements of quality adding up a total of 210 possible points comprised in nine aspects:
institutional support, technology support, course development and instructional design,
course structure, teaching and learning, faculty support, student support, and evaluation
and assessment (A Quality Scorecard, 2015). For each of the elements, the reviewers have
to decide on a score ranging from 0 (deficient) to 3(exemplary). Recently, the scorecard has
been translated and adopted by some Mexican institutions and some others in Latin
America.
Aspects considered in the course development and instructional design section of
the scorecard relate to underlying assumptions of learning theories recommended such as
constructive pedagogy (i.e. the recommendation of student-centered instruction, faculty and
student engagement, measurable learning outcomes). The teaching and learning section
incorporates items that confirm the inclusion of student to student and instructor to student
interaction; instructor feedback, and instructor use of strategies to create engaging tasks.
Additionally, the term (teaching) “presence” is included as instructors’ desirable
characteristic.
41
The Quality Matters Higher Education Rubric
Quality Matters is an organization committed to quality assurance in online
education. The QM organization members assist each other regarding the quality of course
design. Through peer review, faculty help each other to improve course design. Faculty
who are members of the organization are trained to use the QM rubric to give constructive
feedback to fellow faculty who are interested in improving their online courses.
It is important to point out that the QM rubric focuses on course design; through
task review, it helps to improve student engagement and learning outcomes among other
benefits after its implementation. However, it does not review course content, nor faculty
training and readiness or student engagement and readiness. The rubric is available to
subscribers at the QM website (https://www.qualitymatters.org/rubric). It includes 43
specific standards to evaluate course design comprised in eight general standards: Course
Overview and Introduction, Learning Objectives (Competencies), Assessment and
Measurement, Instructional Materials, Course Activities and Learner Interaction, Course
Technology, Learner Support and Accessibility and Usability. These general standards have
different number of specific standards (i.e. Course Overview and Introduction includes
eight specific standards and Learning Objectives (Competencies)includes five), and each of
the specific standards are given different scores that range from 1 to 3 and that add up to a
total of 102 possible points. The fact that a general standard has more specific standards
does not necessarily mean it adds a higher score; items in Course Overview and
Introduction add up to 16 points and items in Learning Objectives (Competencies)add up to
15 points. (QM Higher Education Rubric, Fifth Edition 6/15).
42
Specific standards that are important to notice for the purpose of this research are
the ones included in the Course Activities and Learner Interaction standard. Items in this
section ask about instructor’s feedback on assignments, and the support of active learning
through interaction. The other specific standards to note are the ones included in Learning
Objectives (Competencies). These are related to consistency of learning objectives with
course activities and learning outcomes.
The OLC scorecard helps to evaluate online programs as a whole while the QM
rubric considers course instructional design. It can be noticed that although some aspects in
both instruments seem to overlap, the OLC scorecard is more general and some of the items
in the QM rubric could be included in a general aspect of the scorecard. For example, some
Learning Objectives from the QM rubric are considered in the Course Development and
Instructional Design from the Sloan-C Scorecard.
The Community of Inquiry Framework
The Community of Inquiry framework considers three main elements: Teaching,
Social and Cognitive presence. These elements and its interrelations are considered
essential to obtain positive learning outcomes as it can be observed in Fig. 2.2.
43
Figure 2.2. Community of Inquiry (Garrison et al., 2000)
Teaching presence is commonly related as the responsibility of the instructors. This
presence includes the instructional design and organization of the course where topics,
goals, instructions and time frames are set. It also considers the instructor job as facilitator
and guide by engaging learners in the exploration of new knowledge, identifying doubts
and clarify them, and help develop a sense of community. Instructor’s direct instruction is
also part of this presence and it is related to helping learners keep focused on topics and
provision of timely and helpful feedback.
Social presence refers to the actual communication between participants and the
ability to keep a healthy environment. Social presence includes elements of affective
expression, open communication and group cohesion. Affective expression concerns to the
ability to establish social interaction, have emotional projection and develop a sense of
belonging in the course. Being able to communicate ideas and provide feedback to others
comfortably and respectfully is part of the open communication in this presence. The idea
44
of feeling part of a group and being acknowledged by its members is related to cohesive
activities.
Cognitive presence refers to the actual learning of subject matter and the ability to
reflect, think critically and find an application to what is learned. According to Garrison et
al. (2000) this presence can be better understood through the practical inquiry model (See
Fig 2.3.) which is a model of critical thinking. This model is based on Dewey’s practical
inquiry (1933) and it includes four levels of abstraction in critical thinking and cognitive
presence: triggering event, exploration, integration and resolution. The triggering event
category is related to the activities or content that triggered or motivated interest.
Exploration refers to the information used to explore content and problems to resolve as
part of the course activities. The construction of solutions through the reflection of the
information learned is part of the integration category. The resolution is the goal of the
model since the learner is able to apply the knowledge to practical situations that might or
might not be related to the course. Thus, it is expected that students reach the resolution
level to really appropriate the new knowledge.
Figure 2.3. Practical inquiry model (Garrison et al., 2001)
45
When participants are considered members of a Community of Inquiry it is assumed
that they take advantage of their common interests to critically analyze information.
The interrelationship of the presences was conceived as supportive; that is, the development
or fostering of one supports the others. A main principle when developing the framework
was that “the element of teaching presence is a means to an end to support and enhance
social and cognitive presence for the purpose of realizing educational outcomes.” (Garrison
at al., 2000, p.90)
The CoI framework has been used extensively to explore the presences individually
(i.e. Ice, Curtis, Phillips, &Wells, 2007; Shea, Li & Pickett, 2006). Before the proposal by
Arbaugh et al. (2008) of a CoI survey (https://coi.athabascau.ca/coi-model/coi-survey/) to
measure the three presences, only some studies aimed at considering the three components
(Arbaugh, 2007). The survey has been validated through principal component analysis
(Díaz, Swan, Ice & Kupczynski, 2010; Swan et al., 2008) with students from different
disciplines (Arbaugh et al., 2008,). It has been translated to other languages and has been
validated as well (Alaulamie, 2014; Moreira, Ferreira & Almeida, 2013; Yu & Richardson,
2015).
CoI teaching, social and cognitive presences plus learning presence?
Shea and Bidjerano (2010) studies to validate the CoI model have found that while
the CoI survey items can be trusted, there are some factors that could enhance the model.
They argue that a way to improve the model is by including the learners’ role. Thus, they
propose the inclusion of a “learning presence” (See Figure 2.4) that considers the role of
online learners’ self-directed learning in relation to self-regulated learning and self-efficacy
aspects. Their research is supported by findings about students’ positive self-efficacy
46
beliefs and their relation to higher academic achievement (Zimmerman, 2000; Zimmerman,
2008).
Figure 2.4. The CoI revised by Shea and Bidjerano (2010)
Self-regulated learning
As online learning requires different instructional skills, it also requires that learners
develop skills and behaviors to succeed in this environment. One main characteristic that is
advisable to foster is that of autonomy and the abilities that come with it such as time
management and a proactive approach to learning. Barnard at al. (2009) state that self-
regulatory learning skills are even more important in the online environment than in the
traditional one because these skills help students with task completion. They tested an
instrument in the form of a survey to measure self-regulated learning, the Online Self-
regulated Learning Questionnaire (OSLQ), with the objective of learning more about how
learners develop these skills and how these are facilitated by the online courses. This
questionnaire includes 24 items related to environment structuring, goal setting, time
management, help seeking, tasks strategies, and self-evaluation. Results indicated that there
47
is evidence concerning the validity and reliability of the questionnaire. Barnard-Brak,
Paton and Lan (2010) further state that the development of these skills should be
considered in the instructional design of online courses since they have found that first-time
online immersion is not automatically associated with self-regulatory skills.
CoI relationship with OLC pillars and QM standards
As it was mentioned, the Community of Inquiry survey is based on students’
perceptions of their learning processes in the online environment. Hence, it can be related
to students’ satisfaction and the learning effectiveness pillars of quality in online education
from the Online Learning Consortium. An example that shows this relationship between the
use of the CoI survey and quality improvement is the research presented by Richardson,
Ice, Boston, Powell and Gibson (2011). In this research, awarded by the OLC, the CoI
survey was used as the end of course survey and the collected data was used to evaluate
overall program efficacy and retention.
A recent research project conducted by Swan, Day, Bogle and Matthews (2014)
employed the Community of Inquiry and Quality Matters instruments to improve fully
online courses in a Teacher Leadership program. They revised course design using the QM
rubric and made some changes based on CoI survey answers and implemented the changes
in the following semesters. They found that using both instruments actually complement
each other since they measure different aspects and they can result in increased learning
outcomes. They recommend the use of both instruments to enhance instruction: the QM
framework to evaluate course design and the CoI framework to improve the learning
48
process. The QM reviews helped align course objectives to assessment and activities while
the CoI survey helped to know about the areas to improve in the learning process.
Best Practices in Online Teacher Education
The expression “best practice” is used to refer to the work that is aligned with
current research, that follows established standards, and that has reached good reputation.
Currently, best practices in the teaching field relate to practices which take into
consideration the socio cultural aspect involved in the teaching learning process and that
provide a student-centered approach that requires a more active, autonomous, hands-on
participation and that tries to emulate a more authentic therefore practical experience
(Brinthaupt, Fisher, Gardner, Raffo, & Woodard,2011; Freeman, 2009; Kupetz &
Ziegenmeyer, 2006; Nunan, 2002).These standards directly inform teacher education since
teachers should be able to reach the desirable outcomes of the teaching learning process.
Darling-Hammond (2012) asserts that research has reported that teachers do improve their
teaching practices after they participate in a certification processes.
Standards of Teacher Education Programs and Measures for Program Evaluation
In the US, the Association of Teacher Educators (ATE) has developed standards
that are aimed for teacher educators to impact the profession. These standards are related to
Teaching, Cultural Competence, Scholarship, Professional Development, Program
Development, Collaboration, Public Advocacy, Teacher Education Profession and Vision.
Generally speaking, these standards advocate for modeling desired practices, engaging in
inquiry and reflection to expand knowledge base and make the teacher education programs
49
more relevant (Standards for Teacher Educators, 2008). Interestingly, these standards have
not been used to accredit or evaluate programs but as referents for teacher educators self-
evaluation and definition of their roles.
Online Teacher Professional Development
Hill, Beisiegel and Jacob (2013) study about professional development research
presents examples of recent research comparing learning outcomes of online and face-to-
face professional development. Regarding this comparison, they concluded that no
significant difference in relation to learning outcomes was present in the reviewed studies
and that the only difference was that participants reported to be more satisfied with in-
person delivery mode.
In relation to online teacher professional development (oTPD) programs
specifically, Dede et al. (2009) list teachers’ self-reports as one of the recurring measures of
effectiveness of the outcomes and impacts. They emphasize the need to develop other
more objective measures and rely on both, qualitative and quantitative tools to measure
teacher change in order to have more valid and reliable results. An important
recommendation they make is to analyze outcomes over time since usually it is not possible
for teachers to change their practices immediately.
Ketelhut, McCloskey, Dede, Breit, and Whitehouse (2006), discuss about the
dichotomy between the program evaluation and empirical research. They sustain that
“program evaluations tend to ask questions about effectiveness, while empirical research
asks questions about impact” (p. 251). That is, program evaluations are interested in the
50
scalability, sustainability and cost-benefit of the program, while empirical research
objective is to know “What kinds of measures best capture the depth and durability of
teacher learning and change? What types of teacher change most influence student
achievement?” (p. 253). In relation to this, Dede et al. (2009) recommend establishing
evaluation parameters based on sound research. They suggest, among other research issues,
addressing the “impact of professional development on teacher change, particularly
improvements that transform practice; (and) effects of teacher change on student learning”
(p. 16).
Regarding the establishment of an online TPD community, Liu (2012, p. 703) states
that “technology is only a tool for community building” and that creating and sustaining an
online community require more than technology. She adds that this is a complex process
since learners have to take responsibility in their learning and in their colleagues learning as
well, plus in order to have motivated participants, their interests have to be considered. She
concludes that more research about the characteristics of effective online TPD communities
is necessary.
Some authors have also pointed out that the technological aspect is usually the one
considered in studies related to online teaching (Kern et al., 2004; Rudestam &
Schoenholtz-Read, 2010). Regarding this, other researchers have acknowledged the need to
focus more on helping instructors develop the pedagogical aspect as well (Comas Quinn,
2011; Dawley, 2007; Jones & Youngs, 2006; Pallof & Pratt, 2001). In relation to this
concern, Downing and Dymet’s (2013) study revealed that the pedagogical needs perceived
by online teacher educators were “how to construct effective learning outcomes, how to run
51
tutorials, how to ensure participation by all students, how to engage students, how to assess
students, and how to provide feedback to students” (p.102).
English Language Teacher Education
Standards in TESOL Teacher Education Programs
The definition of the knowledge base in LTE directly inform teacher education
standards or best practices since teachers should be able to reach the desirable outcomes of
the teaching learning process. In the English LTE field, the TESOL International
Association is a member of the Commission for the Accreditation of Educator Preparation
(CAEP/formerly NCATE) since 1999. As the CAEP requires, the TESOL International
Association has established standards for programs in P-12 ESL Teacher Education; these
were approved in 2001, and revised again in 2005; their latest revision was in 2010
(Standards for the recognition of initial TESOL programs in P–12 ESL Teacher Education,
2010). The CAEP requires programs in P-12 ESL Teacher Education to follow the TESOL
standards. The standards are comprised in five dimensions that include pedagogical
considerations. A very general description of the domains is presented next.
The Language standard includes aspects related to the instructor’s knowledge of its
components: phonology, morphology, syntax, semantics and pragmatics, discourse
varieties, genres, registers and styles. Practitioners should know about the role of culture
and should recognize and value students’ identities and that all the cultural background they
bring with them should be valued, respected, and used for their benefit. In order to create an
52
optimal environment for learning, teachers should design student centered instruction that
provides a supportive learning experience by fostering collaboration and scaffolding
between peers, and by acknowledging learner differences and prior knowledge. Technology
is also considered in this dimension; it has to be integrated as part of the resources used to
facilitate language and content learning. The principles considered in the assessment
domain are validity, reliability, practicality, fairness and authenticity. Assessment should be
formative, to help the teacher notice learning problems or differences in development.
Assessment has to be summative as well to test general proficiency and also to identify the
areas that need more development. Since these are standards for P-12 instruction, this
domain also includes the teachers understanding of the use of assessment instrument to
place, reclassify, or have students exit the ELL support program. The professionalism
domain refers to the teachers’ knowledge of policies, former and current teaching practices
and advances in the ESL field informed by research. Instructors should also have a
philosophy of education and should know about professional growth opportunities (TESOL
P-12 Teacher Education Program Standards, 2010).
Currently, there are no established standards for the training of EFL/ESL teachers to
teach in Higher Education (HE). The existing standards are for the ESL/EFL teachers of
adults by the TESOL International Association. The framework for these standards is
similar to the one for the P-12 ESL Teacher Education programs; nevertheless, it is not
presented through the teacher education perspective but through the practices instructors
should pursue in their teaching. These standards are contained in 8 domains as follows:
planning, instructing, assessing, identity and context, language proficiency, learning,
53
content, commitment and professionalism (Standards for ESL/EFL Teachers of Adults
Framework, 2008).
Katz and Snow (2009) make an account of different kinds of standards intended for
different audiences and educational levels, for example, ESL standards in the US and
Australia (which might vary from state to state), EFL standards in China, Egypt, and
standards for other languages such as the ones provided by the American Council on the
Teaching of Foreign Language (ACTFL) and the Council of Europe’s Common European
Framework of Reference (CEFR). They mention that the importance of establishing
standards is that they act as a frame of reference for language teaching and learning. They
also state that there are some shared characteristics even when standards are described for
different contexts (p. 73) because standards include current research findings, provide a
frame of reference, and reflect the performance of teachers, students, and teacher education.
In relation to the use of standards as a measurement of effective practices and the
variety of contexts, Mahboob and Tilakaratna (2012) affirm that the notion of a principles-
based approach (PBA) is less problematic. They argue that contexts are very diverse and
standards are “based on assumptions related to the distribution of resources, access to
knowledge, and appropriate infrastructure” (p. 1). Other assumptions pertain to the
development of standards by considering Western theories as global and leaving behind
local contexts. Consequently, there is a need to consult local experts to make ELT practices
“locally and contextually relevant” (p. 9).
As it can be observed, information about best or effective practices and standards
are mainly related to the content knowledge ESL and EFL teachers should have and their
54
desirable practices. As Hall and Knox (2009) point out, many areas of Language Teacher
Education are under-researched and one of them is the learning experiences of language
teachers in LTED programs which is the objective of this research. Authors such as
Crandall (2000), Farrel (2015), Freeman (2009), Freeman and Johnson (1998), Graves
(2009), and Johnson (2009) have highlighted the need for more research concerning this
issue.
Language Teacher Education (LTE) is shaped on research on general education and
what successful language teachers do in their practice. Freeman (2009) makes an account of
the development of LTE from the behaviorist approaches to the current broadened scope
that includes socio-cultural aspects such as identity, socialization and situations of practice.
He proposes the elements of substance, engagement, and influence or outcome. Broadly,
substance refers to what the teachers are expected to learn and what they assume as socio-
professional identity as teachers; engagement relates to how they are expected to learn it
and the outcome is the measure used to determine the observed results. He adds that this
dimension “also help(s) to anticipate the major new directions that are now happening
within the field” (p. 11). Freeman asserts that although classroom learning is not directly
the cause of teaching it is in fact influenced by teaching. Figure 2.5 represents Freeman’s
dimensions of the scope of second language teacher education (SLTE). As it can be
observed in sector C, social participation such as team projects, feedback groups, leads to a
more positive engagement, and helps towards the development of professional identity (p.
17)
55
Figure 2.5. Freeman’s Dimensions of the Scope of SLTE (2009)
In a similar vein, Johnson (2009) calls for a reconceptualization of LTE. She states
that first, the knowledge base of LTE has to be defined since it is what informs the content
(“what L2 teachers need to know”), the pedagogies (“how L2 teachers should teach”) and
the forms of delivery (“how L2 teachers learn to teach”) (p. 21). Second, the knowledge
teachers bring to the learning should be recognized; that is, teachers should share problems
of their praxis which will inform knowledge base as well. Third, the definition of language
and SLA has to be broadened; language should be conceptualized as social practice and
how L2 learners are able or not to participate in these practices. Last, she favors alternative
professional development and not just the formal offered in seminars or workshops, but
also a development that permits “self-directed, collaborative, inquiry-based learning that is
relevant to teacher’s classrooms” (p. 25).
56
Graves (2009) exploration to the curriculum of SLTE highlights engaging in
practice as essential to teacher-learners. She states that opportunities for observation,
planning, teaching, reflection and analysis should be considered to be able to connect
information to learning theories. She adds that the teaching contexts make teaching a
situated activity that has to consider the complexity of the cultural, political and social
systems. The understanding of these contexts will help them participate in communities of
practice and develop ways to improve practices. Graves (2009, p.120) also calls for
congruence in instructional practices stating that “how teacher-learners are taught must be
congruent with how they learn.” She concludes that teacher educators should also consider
how teacher-learners would like to be taught. Nguyen (2013)findings from a study about
the comparison of two SLTE curricula support Graves (2009) assertions that contextual
factors are determinant in relation to decisions of what to teach. These differences indicate
that in order to design meaningful instructional tasks the teacher-learner contexts should be
considered.
Online English Language Teacher Education Practices
Challenges of Online English LTE
As Graves (2009), Mahboob and Tilakaratna (2012), Nguyen (2013) emphasize,
ESL/EFL teachers work in a variety of contexts. Moreover, since online environments
allow for the possibility to teach students from different countries, experiences from
different contexts may be explored. Thus, Online TESOL Teacher Education (TE) has to be
considered globally. In relation to this, England and Hall (2012) point out that “there are no
57
globally accepted minimum standards for TESOL qualifications, the programs that
underpin them and the institutions that confer the awards” (p.189). In addition, candidates
might study an online program from one country and work in another and have difficulties
to practice the principles learned in the program due to context constraints (i.e. learning
about communicative approaches and trying to apply principles in a long dominated focus
on grammar context).
Other challenges in Online TESOL TE are related to administrative issues such as
instructor workload, the protection of course content, fair use (Opp-Beckman, 2012), the
marginalization of online programs in the institution, and the new roles of the distance
teachers (Hall & Knox, 2012). Besides administrative issues, Healey (2012) presents a list
of logistical aspects to consider when planning an online TESOL program. Some of the
mentioned aspects are related to the technology used to facilitate the courses, the course
delivery in relation to technical requirements and support, and the challenge of online
assessment to ensure that students taking the course are the ones registered.
Along with those challenges, come more possible advantages of online
environments. England and Hall (2012) mention that currently collaborative degrees are
growing. Working in a course development collaboration project among institutions to
create joint degrees in TESOL teacher education is more possible now. This consortium
prospect would require transparent quality assurance processes that can help institutions
reach to a consensus of aspects that define quality.
In her study about OLTE in general, Murray (2013) examined the characteristics of
the OLTE courses being offered (i.e. credit, non-credit) and the issues in relation to SLTE
58
in the online environment. She found that OLTE courses offered are greatly varied and
short professional development courses can be found along with fully online post-
secondary programs. Among the issues mentioned in this report, she mentions building a
community that responds to needs from different contexts to “create new knowledge
through discourse” (p. 42); instruction that promotes interaction, provides feedback and
situated learning taking advantage of technological affordances; and support for online
instructors through professional development.
Addressing gaps in OLTE will require awareness of the differences that online
learning and teaching encompass. As England (2012) states “the skills and knowledge of
effective online teacher educators will be somewhat different from those required of
traditional face-to-face TESOL faculty.” Certainly, more research about pedagogical issues
in online TESOL courses is necessary.
OLTE and CoI Research
The Community of Inquiry framework has been used in research related to language
teaching and language teacher education as well. As it was previously mentioned, the
cognitive presence Practical inquiry model consists of four stages of abstraction: triggering
event, exploration, integration and resolution. The students are given a triggering event that
will motivate them to explore the topic. In the integration stage students are able to
construct knowledge by combining information found and what was discussed in forums.
The resolution stage is the ultimate goal and students are able to apply that knowledge to
use in real situations. This model has been used mainly with data from discussion forums as
a tool to explore if online learners reach to the desired higher-order thinking. Researchers
59
have observed that students stayed in the first two stages: triggering event and exploration
(Garrison et al., 2000) and other studies have confirmed that students require clear
guidelines to move forward to the next phases (Akyol & Garrison, 2008; Garrison &
Arbaugh, 2007). The studies consulted for this research are presented chronologically as
follows.
In the OLTE field, Pawan, Paulus, Yalcin and Chang (2003) used the Practical
inquiry model to examine qualitatively online discussions from graduate-level LTE
courses. They found that there was a lack of interactions that they could code in the
resolution level from the Practical Inquiry model of Cognitive Presence. They also found
that there was sharing among students rather than inquiring, and a rather low instructor
participation in the discussions. They analyzed the messages in order to classify them as
belonging to a phase in the model and found that other sources of information were needed
to find more about the discussions dynamics in online forums such as students’ pressure to
participate and the time that faculty and students have to invest reading and replying or
giving feedback. They concluded that perhaps students were not aware of the purpose of
the discussions and recommend that instructors structure discussion, model participation,
facilitate and push discussion towards the integration and resolution phase (higher-order
thinking), and assign specific discussion roles. An interesting point they make is that
perhaps the integration and resolution stages are in fact reached but in other type of tasks
such as research papers.
Another research related to language teachers’ interaction in discussion forums is
the one by Arnold and Ducate (2006). In their study, they analyzed discussions from two
graduate level courses each from a different university for incoming teaching assistants in
60
the French, German and Spanish departments. The course was web-facilitated; that is,
instruction was mainly face-to-face with online discussions. Instructors posted the
triggering questions and assessment guidelines; students were in charge of the discussion
and were responsible for the outcomes. Each discussion lasted five days and a total of 27
discussions were analyzed. They classified these messages considering the CoI framework
teaching, social and cognitive presences because they wanted to examine critical thinking
and social engagement in asynchronous CMC. Additionally, students were asked to
complete a survey about their perceptions of this CMC experience at the end of the
semester. Date from survey revealed that 65% of students preferred having the discussion
with students from a different university. They also found that students engaged more in
social presence behaviors; nevertheless, they were able to reach to higher-order thinking:
integration and resolution phases in the cognitive presence construct. They concluded that
the triggering questions, the guidelines (rubrics and calendarization of posts)were essential
to guide students towards those outcomes; a task design to reach to a resolution level will
encourage those results. Also, even though the opportunity to feel free to discuss allowed
for more social activity, students presented more open communication and group cohesion
than emotional expressions. They attributed this behavior to the guidelines and rubrics.
Shin (2008) used the CoI framework to redesign her online courses and was able to
compare how beneficial these changes were. The courses compared were one from fall
2004 and two from fall 2005. Students’ interactions were coded based on the practical
inquiry model. The main changes were in the handling of discussions and teaching
presence. Some of the changes in the 2005 courses were the inclusion of guidelines such as
word limit per message and starter-wrapper technique (Hara, Bonk & Angeli, 2000) roles
61
for students; these roles consisted of a student being responsible for starting the discussion
and another student for summarizing it with specific guidelines and the instructor modeling
the first discussion. Students had also access to a Q & A forum to ask questions about the
course (i.e. where to find assignments) and a Cybercafé space to provide a space to discuss
issues that were not directly related to the course discussions. Highly important teaching
presence practices that Shin added to her OLTE courses were modeling and the use of
culturally inclusive pedagogy to help to establish social bonds (Shin & Bickel, 2008). She
provided examples of posts to guide students about course expectations and used culturally
inclusive pedagogy by sharing about her cultural background and relating it to language
teaching and eliciting teacher-learners to follow her example. She asserts that this practice
allowed for the building of a community of inquiry that sustained through Facebook even
after the course finished. She concluded with some recommendations for OLTE instructors:
The use of problem-solving approach in the discussion forum to have participants
share ideas that can help them reach to an application (resolution phase in the
Practical Inquiry model).
The use of the starter-wrapper technique so that students were in charge of the
discussion and being present as instructor to guide them when needed. She added
that this task has to consider the level of access to internet.
Be flexible as students might have problems with internet access
The use of culturally inclusive pedagogy to be sensitive to cultural differences.
Chen (2012) also adopted the Community of Inquiry framework as a tool to
examine practices at the University of Southern California online TESOL teacher education
program. Her research was about general perceptions of OLTE courses from instructors and
62
students in the master’s program in relation to aspects such as instructional design,
navigation, content, technical assistance, among others. She utilized two questionnaires in
which she comprised the CoI sections from each of the presences. For example, social
presence in the CoI survey consists of a total of nine items included in three subscales
(Affective Expression, Open communication, Group Cohesion) with three items in each. In
Chen’s questionnaire, the three affective expression items were comprised into one
statement, resulting in a total of 10 statements comprising the 34 items from the CoI
survey. A total of 33 participants, 27 students and six instructors completed the
questionnaires. Results indicated that online courses were found to be more academically
demanding by students. Even so, the majority of participants was satisfied with the program
and did perceive teaching, social and cognitive presence in their courses. Open
communication was the strongest element perceived from the social presence construct and
the cognitive presence elements were perceived as the weakest. An important finding was
that regarding the use of discussions forums, students answered that these were
“extensively used” and most of them perceived that instructors had difficulty in “keeping
course participants engaged in productive dialogue” (p. 120).
Summary of the Literature Review
Research solely dedicated to pedagogical practices in OLTE is limited;
consequently, related themes and studies have to be explored. Quality assurance and quality
control measures in online learning and in SLTE along with the guidelines or standards that
are considered are a starting point since one would assume that what is required from
teachers to do once they are practicing teachers should have a direct relationship to how
they are taught. That is, if teachers are asked to promote the construction of knowledge, it is
63
expected that they learn by constructing knowledge as well. Common pedagogical practices
found in the material reviewed relate and advise instructors to:
Build community through the meaningful interaction of course participants
(student-to-student, instructor-to-student)
Help students think critically and apply knowledge
Provide timely feedback and be present
Model participation and facilitate instruction not just monitor
Allow time for reflection to reach higher-order thinking
Help students develop self-regulation skills
More specific to OLTE courses:
Help students link conceptual knowledge to their practice
Have students become aware of the value of points of view from e-mates (i.e.
provide opportunities to share work, give feedback)
Consider context or situated learning in order to motivate students and help them
make meaningful connections
Reflect on their practices to identify strengths and weaknesses
64
Chapter 3 Methodology
The primary purpose of this research was to identify effective pedagogical practices
considered by the main participants in OLTE instruction: students, faculty and program
coordinators. This research focused its attention on students’ perceptions of learning
effectiveness, and satisfaction in OLTE courses and the strategies used by faculty and
program coordinators to assure quality of instruction and long-term learning impact in
OLTE courses and programs. As it was previously mentioned, students’ satisfaction is
closely connected to quality of instruction since it is perceived as fitness of purpose. Thus,
literature related to aspects considered by quality assurance (QA) and quality controlled
(QC) agencies was reviewed in order to explore the challenges and issues in defining and
measuring quality of online OLTE programs and specifically the aspects related to
pedagogical practices.
This chapter describes the methodology that was used for this dissertation research. It
includes the research design, population and sampling procedures, and a description of the
instruments used and their validity and reliability issues. Finally, data collection
procedures, data analysis procedures and ethical issues are presented.
The overarching question for this research is the following:
What are the most effective pedagogical practices in Online Language Teacher
Education (OLTE) courses as identified by experts, instructors and students?
In order to answer this question, some considerations were made. First, it was necessary
to explore the contexts in which these courses are commonly offered. Second, a validated
and reliable framework related to pedagogical practices was used to support findings.
65
Finally, information regarding quality assurance was pertinent to include due to its
relationship to student satisfaction. Thus, in order to answer the main question, the
following research questions were proposed:
Context of OLTE courses
1. What are the general characteristics of the OLTE courses currently offered to in-service
language teachers and future language teachers?
2. How are the Community of Inquiry (CoI) elements incorporated into OLTE courses?
a) What is the relationship between the CoI elements (i.e. teaching, social, and
cognitive presence) in an OLTE program?
b) How do OLTE instructors and students perceive the establishment of a
Community as a critical element for a successful OLTE course?
c) How are the effective pedagogical practices integrated in to the OLTE courses?
d) Is there a relationship between the Language Teacher Educator Presence items
proposed and the CoI elements?
e) What is the relationship between the Self-regulation items and the CoI elements
and the proposed Language Teacher Educator Presence?
3. How is quality of instruction assured in OLTE courses?
a. What instructional activities are included in OLTE courses and are these
activities considered engaging?
b. What tasks and teaching practices do learners and instructors find most effective
and why?
66
c. How do instructors and students perceive quality assurance actions in OLTE
courses?
d. What are the strategies used by faculty and program coordinators to assure
quality of instruction and long-term learning impact in OLTE programs?
Research Design
Quantitative and qualitative methods are used in this research in order to have access to
a better view of a complex construction such as perception of meaningful learning.
According to Creswell and Plano Clark (2011) using a single approach lacks detail and
explanation of a complex phenomenon. A mixed-methods approach allows for the
representation of multiple perspectives, a more complete understanding of the phenomenon,
and more support for analysis (Plano Clark et al., 2011).
Besides corroborating information to achieve validity, the goal of this research is to
provide richer details and insights about OLTE courses; the “what has happened …and the
why it has happened” (Tashakkori and Teddlie, 2010). It follows a pragmatic approach with
an embedded design using qualitative data to support quantitative data. This nested
procedure allows for the integration of all data analysis to reach to a deeper interpretation.
As Creswell (2003, p.12) states “pragmatism opens the door to multiple methods, different
worldviews, and different assumptions, as well as different forms of data collection and
analysis.”
In order to gather this type of data, a sequential mixed methods (Teddlie & Yu,
2007) approach was followed; that is, a survey and follow-up interview questions were
designed. The survey gathers quantitative data that allows for a statistical analysis which
67
helps in the correlation between the Community of Inquiry teaching, social and cognitive
constructs related to students learning and satisfaction. Furthermore, these constructs can be
correlated with other distinctive characteristics such as age, gender, teaching experience as
well. Qualitative data is gathered through follow-up online and face-to-face interviews to
explore and learn more about students, instructors and coordinators perceptions about the
teaching-learning process in an OLTE course. The interview questions were open to topics
emerging from the quantitative analysis results. For this reason, they were scheduled after a
first general analysis. Other available information pertaining the courses content (i.e.
syllabus, assignments, rubrics and grades), were gathered whenever possible to triangulate
results. Coordinators, faculty and students were asked if they would consent to share this
information as well.
Population and Sampling Procedures
The main target of data collection was on students who had been enrolled in an
online course as part of their language teaching training or professional development, and
instructors who had taught or were teaching an OLTE course. Following ethical
considerations, OLTE program coordinators and or sponsoring agencies were the first ones
to be contacted in order to reach to faculty and students. They were asked to forward the
invitation to participate in the study to their instructors and the students.
OLTE courses currently offered range from non-credit professional development to
courses to improve teaching skills, Language Teaching certificate courses, to credit degree-
seeking courses (i.e. part of a Bachelor’s, Master’s or PhD program) or specialization
courses such as the ones part of the English for Specific Purposes (ESP) Diploma. The
objective was to have a general picture of the students’ perceptions in regards to their
68
online environment experience: what they consider important to feel that the online course
was successful and how this learning experience can be improved. It is fundamental to
learn about instructors’ online pedagogical practices to compare this with students’
expectations and inform if these practices are in line with what it is expected and how these
expectations are being met. Additionally, it is essential to know about quality assurance
practices from institutional perspectives since these permeate to faculty practices, hence,
students learning. For this reason, program coordinators were contacted and asked to
participate themselves as well and not only be a link to instructors and students.
Communication with institutions offering TESOL online courses was established to
reach the population for this research. The institutions contacted offer on a regular basis
TESOL/TEFL/TESL certificates, BA and MA TESOL programs. An agency that grants
professional development scholarships to Mexican English language teachers mainly
through the online environment was also approached. It is important to point out that if the
study were limited to only a specific group of students, the risk of not having a reasonable
number of participants would be very high. For instance, the first online certificate program
contacted, forwarded the invitation to 13 students from one of their recently finished
certificate courses; when asked to participate, only three students completed the survey.
From another institution offering certificate courses as well, only one instructor answered
the survey. For this reason, the decision to open the invitation to students and instructors
from different types of OLTE courses was made in order to increase response rate
probabilities; also, research with participants from different courses and different
institutions has been conducted in previous studies about students learning perceptions
(Arbaugh et al., 2008).
69
Instrumentation
The objective was to craft a friendly short survey (around 20 minutes long) that could
provide information to have a general picture about the effective pedagogical practices
considered by OLTE students. Additionally, it was assumed that a small number of items
with multiple choice questions would make this instrument less burdensome and therefore
easier to administer. To be able to form a clearer picture and present a deeper analysis,
participants were asked to be contacted for a further online interview via voice-over-
Internet Protocol (VoIP) services such as Skype or Google +.A description of these
instruments follows.
Surveys
In order to learn about pedagogical practices in Language Teacher Education courses
proposed by experts, a literature review about best practices for online education was
conducted. This review included current best practices, standards, frameworks and or
guidelines proposed by accrediting agencies, QA organizations, and higher education
institutions, and base knowledge for language teachers. Common features considered
relevant by these bodies were summarized and categorized. Most of them include technical
aspects and issues related to instructional design; consequently, the task of separating them
was challenging. However, these aspects have a relationship with the pedagogical practices
and it was important to review them as well.
The complete OLTE surveys for students and instructors can be found in Appendix A
and B respectively. They both include eight main sections: Course general information,
Language Teacher Educator Presence, Language Teacher Educator Desirable Practices,
Self-regulated Learning, the CoI 34 survey items (Teaching, Social and Cognitive
70
Presence), Instructional Activities, Engaging Instructional Activities and Demographic and
other General Information. A detailed description of the surveys follows.
OLTE Survey for Students
Course General Information
This information allows for the classification of the type of online course considered
by students when answering the survey: the name of the course, the type of course
(professional development, certificate course, degree-seeking course), and the enrollment
date.
Language Teacher Educator Presence
Previous research studies related to perceptions of participants in an online learning
exchange have tried to study subjectivity by using the Q-method (Rodriguez, 2013; Wright,
2013) to measure participants’ viewpoints. The Q-method originally developed by William
Stephenson, is explained by Brown (1980) and later Van Exel & De Graf (2005),as a
systemic study of subjectivity. A brief explanation is that in this method the investigators
research relevant aspects related to the topic of their interest. They build a sample of
statements about a topic which may be obtained by a literature review, interviews, and even
researcher own beliefs; this is called the Q-set. Brown (1980) mentions that Q-sets might
contain from 40 to 50 statements. Participants are asked to rank the statements considering
their opinion according to preference or judgment from most important to least important,
for example. Consequently, they are forced to provide a judgment since they have to sort
statements. The resulting viewpoints from the sorting represent common ways of thinking.
An important point is that mentioned by McKeown and Thomas (2013), “participant
selection is governed by theoretical (persons are chosen because of their special relevance
71
to the goals of the study) or pragmatic (anyone will suffice) considerations (p. 31, emphasis
in original). In relation to this, Rodriguez (2013, p. 37) states that “Q-method examines
relationships between people” and this allows for the Q-method to involve a small number
of participants. After the participants’ first selection, the researcher is able to refine and
reduce the Q-set through statistical analysis such as correlations and factor analysis
(Brown, 1980).
For this research, an adapted Q methodology was used considering previous
modification in other studies (Wallis, Burns, & Capdevila,2009). As a first step, a review of
literature about the base knowledge language teachers should have and that should be
included in LTE courses and programs was conducted. This information included
suggestions about ways future language teachers should be taught; that is, their proposed
pedagogical practices. Following the main idea of the Q-method, statements were
constructed resulting in 11 items with recommendations from LTE authors such as Freeman
(2009) , Graves (2009) and Johnson (2009) with particular attention to the consideration of
context in instruction and the encouragement of reflective practice; participants had to rank
these statements in order of importance. The survey was the same except for some
adaptations in the wording to fit the statements for students and for instructors to respond.
It was decided to include 11 statements (table 1) taking in consideration that the survey had
to be short (from 5 to 10 minutes) to be able to fit under experts busy schedules thus,
making it more attractive for experts to agree to complete it (see Appendix C for the
complete ranking survey).
72
Table 1
Statements to rank
Please drag and drop the following statements to rank them from what in your opinion should be
the most important to the least important practices in an Online Language Teacher Education
course
As a student in a fully online course related to Language Teaching & Language Learning,
I think I should
have an instructor who presents language learning case studies from different contexts for
us as students to reflect, discuss and learn.
receive meaningful feedback from my peers about my reflections, papers or portfolio.
be able to reflect on and share my language teaching practices.
be asked about my opinion and evaluation of the course right after I finished it.
be able to establish a link between language learning theory and practice.
have an instructor who fosters an environment for us as student-teachers to develop a
language teaching pluralistic and multicultural view.
be able to share and discuss practical language teaching related needs pertaining to my
context (i.e. the instructor facilitated this exchange) and feel comfortable doing it.
be able to observe long term learning from the content and methodology of the course.
be contacted by the institution or the course instructor after a time the course finished to
know about my learning from this course and ways to improve it.
receive meaningful feedback from my instructor about my reflections, papers or portfolio.
be able to discuss practical language teaching related needs pertaining to my peers’
contexts (i.e. the instructor facilitates this sharing).
Following the drafting of the Q-set, the survey had to be piloted with the help of OLTE
instructors and students’ experts. OLTE experienced recognized professors who work in
graduate programs and who have written articles in journals or book chapters about their
experiences in this type of programs are the expert instructors informing this research. To
be considered OLTE expert students, participants had to be in a graduate LTE program, had
to be former OLTE course students and had to have professional language
teaching/language teacher education experience. An invitation was emailed to these
selected groups explaining them briefly how the results from their opinions were going to
be used in a longer OLTE survey (see Appendix D for invitation email).
73
These expert instructors and expert students were asked only one time to rank the items.
Twelve expert instructors answered the survey and one of those answers was discarded
because it was not fully completed. Eighteen expert students answered and 7 surveys were
discarded for the same reason. Thus, a total of 11 instructors and 11 students ranked the
same 11 items. Items and complete results from their ranking can be found in Appendix E.
Expert instructors who answered have more than eight years of professional language
teaching/language teacher education and eight of them have taught more than six different
online courses in OLTE programs. Eight of the 11 expert students have more than 8 years
of professional language teaching/language teacher education experience; the three other
students have from 4 to 6 years, from 2 to 4 years and less than 2 years of experience
respectively. Eight expert students have been enrolled in one to three courses and three
have been OLTE students in more than six courses.
Interestingly, instructors and students ranking was highly similar. The first four items,
item 7, and the last two items ranked by students and instructors were the same items;
however, their order was different. Table 2 includes the results from this ranking. To the
left there is the students’ ranking and to the right, there is the instructors’ ranking. The
column in the middle has the corresponding students’ ranking number; in this way, it is
easier to compare how both groups ranked the items. For instance, item ranked as number 4
for students was ranked as number 1 by instructors; number 2 for students was ranked
number 2 by instructors as well, and so forth.
It can be observed that items ranked in place 5 and 6 by students were ranked 8 and 9
by instructors respectively. Students’ item ranked as 5 is related to feedback from the
instructor; however, instructors ranked this item as number 8, less important. Students’ item
ranked as 6 is about long-term learning and for instructors that item was ranked as number
74
9. Peer to peer feedback was ranked number 6 by instructors and for students it was not as
important; they ranked it as number 9. Thus, from these results, it is clear that while
students consider feedback from instructor more important than feedback from peers,
instructors think that student to student feedback is more important than their own feedback
to students.
This section of the OLTE survey is called Language Teacher Educator Presence.
Considering the results from the pilot ranking survey, it was decided to include the first
four ranked items by instructors and students since these items were the same, and two
more items: the one ranked as number 5 by students and the one ranked as number 5 by
instructors, even though they were different. That is, the one about feedback and the one
about reflecting and discussing case studies from different contexts, having a total of six
items for the section. In this section participants had to provide information in relation to
these instructors’ practices to reach the course learning goals.
Language Teacher Educator Desirable Practices
The Q-method includes a filtering step in which instructors and students are asked again
to rank the items that were positioned in the first places. In this case, this step was not
followed due to two main reasons: expert instructors and students were asked to rank the
items one time and it was a difficult task to ask them to do it a second time due to their
busy schedules as it was mentioned. Also, there were only 11 items and the similarities
from the analysis of the ranking results allowed for deciding to include the first 5 from both
surveys in the research survey to know how these actions were perceived by the rest of the
participants. The rest of the items were included in the longer survey in a section called
Language Teacher Educator Desirable Practices, and students and instructors were asked to
75
evaluate how important these concepts were to be able to compare them with the results
from the piloting.
Table 2
Summary of Ranking Results
SR
Item SR
IR
Item
1
Be able to establish a link between
language learning theory and practice. 4 1
Establish rapport with my students so
they are able to share and discuss
practical language teaching related
needs pertaining to their context and
feel comfortable doing it (instructor-
student interaction).
2
Be able to discuss practical language
teaching related needs pertaining to my
peers’ contexts (i.e. the instructor facilitates
this sharing). 2 2
Facilitate the discussion of practical
language teaching related needs
pertaining to my students' contexts
with their online classmates (peer-peer
interaction).
3
Be able to reflect on and share my language
teaching practices. 1 3
Help my students establish a link
between language learning theory and
practice.
4
Be able to share and discuss practical
language teaching related needs pertaining
to my context (i.e. the instructor facilitated
this exchange) and feel comfortable doing
it. 3 4
Foster my students' reflection and
sharing of their language teaching
practices.
5
Receive meaningful feedback from my
instructor about my reflections, papers
or portfolio. 8 5
Present language learning case studies
from different contexts for my students
to reflect, discuss and learn.
6
Be able to observe long term learning from
the content and methodology of the course.
9 6
Foster meaningful peer to peer
feedback about students' reflections,
papers or portfolio.
7
have an instructor who fosters an
environment for us as student-teachers to
develop a language teaching pluralistic and
multicultural view. 7 7
Foster an environment for student-
teachers to develop language teaching
pluralistic and multicultural views.
8
have an instructor who presents language
learning case studies from different
contexts for us as students to reflect, discuss
and learn. 5 8
Provide meaningful feedback to my
students about their reflections,
papers or portfolio.
9
Receive meaningful feedback from my
peers about my reflections, papers or
portfolio. 6
9
Be able to find ways of knowing about
my students' long-term learning from
the content and methodology of the
course.
10
Be asked about my opinion and evaluation
of the course right after I finished it. 10 10
Ask about my students' opinion and
evaluation of the course right after we
finished it.
76
11
be contacted by the institution or the course
instructor after a time the course finished to
know about my learning from this course
and ways to improve it. 11 11
(either the institution I work(ed) for or
myself) contact students after a time
the course finished to know about their
long-term learning outcomes from the
course(s).
SR= Students' Ranking IR= Instructors'
Ranking
Self-regulated Learning
Participants were also asked about their self-regulated learning strategies and time
management practices to be able to find possible relationships between learner’s role and
effective pedagogical practices assuming that if learners dedication contributes to their
learning, their academic performance would be better and that in turn it might result in
higher satisfaction. The items included were selected from Barnard et al. (2009) Online
Self-regulated Learning Questionnaire (OSLQ). Again, in order to have a friendly
instrument, only some items from this questionnaire were included. A total of six items
from the 24 items in the short form of the Barnard et al. (2009) OSLQ were selected; they
are related to personal and behavioral factors such as goal setting, task strategy, time
management and help seeking since researchers found they “may have more influence on
the development of self-regulatory skills of online learners than environment” (Barnard-
Brak et al., 2010, p.68).
Community of Inquiry
Since the Community of Inquiry (CoI) favors the construction of knowledge and one of
the main goals of students is meaningful learning, its instrument, the CoI survey, was
regarded as a main element in this research survey to learn about students’ perceptions of
OLTE courses. Additionally, this survey has been validated and has proven reliable results.
77
It has been used with students from different online programs such as graduate-level
courses in either Education or Business (Arbaugh, 2008; Arbaugh et al. 2008), and in
general, from applied and pure disciplines (Arbaugh, Bangert, & Cleveland-Innes, 2010).
Although in the language teacher education field the CoI framework has been used and
proven to be one that can improve instruction (Shin, 2008), the CoI survey per se has not
been used to learn about OLTE programs in a way that its results can be analyzed and
compared to the ones obtained by courses from other disciplines. That is, the same validity
and reliability procedures can be replicated to analyze differences and similarities. Its
relevance for this study relies in the fact that research about how to establish a learning
community and the impact that instructors have on this will help in the development of
“high-quality online teaching and learning environments” (Shea et al.,2005, p.73).
The CoI survey was developed by Arbaugh et al. (2008) and has been validated with
different student populations (Swan et al., 2008; Arbaugh et al., 2008). It is composed of 34
items: 13 items to measure teaching presence, nine items for social presence and 12 items
for cognitive presence. All items were included in the OLTE survey for students in order to
run statistical analysis to compare previous results with those of the OLTE community.
Instructional Activities and Engaging Instructional Activities
In these two sections students were asked to select the type of instructional activities
included in their courses and if these activities were engaging. The drafting of the items for
the Instructional Activities section in the OLTE survey for students was made considering
common activities found in online courses in general such as asynchronous discussion
forum participation, problem solving tasks, quizzes and synchronous experiences and ones
more specific to language teacher education such as video recording and portfolio
78
assignments. The same activities were included in a following section identified as
Engaging Instruction al Activities to know which ones students perceived as most engaging
and for students to indicate the ones that were not part of their course. An open question
about other activities not listed and that they thought were engaging was included in this
section.
Demographics and General Information
This section includes a total of 16 questions including basic demographic questions, and
some open-ended questions. The first questions gathered general demographic and
background information: age range, gender , nationality, decision to choose online course,
educational level, teaching experience, online experience as student, training to study
online, opinion about online instruction and the feelings about the amount of information
covered . The survey ends with an optional open-ended question asking to provide
suggestions to improve it.
The demographic questions for this research were presented at the end of the survey
as recommended by Albert, Tullis, and Tedesco (2010) in order to have the items that
required most of the participant attention first since they were going to judge and express
their perceptions, and to have a more accurate representation of their perceptions about
online learning.
OLTE Survey for Instructors
This survey consisted of the same sections as the student survey and was almost
identical to the survey for students in order to allow for more direct comparisons. The
79
changes were in the wording to set the items from the perspective of the instructors. The
first three sections: course information, language teacher educator presence and language
teacher desirable practices have the same items. The following section, Self-regulated
Learning in the survey for students, is called Time Management in the survey for
instructors, and three statements from Barnard et al. (2009) Online Self-regulated Learning
Questionnaire (OSLQ) were selected to know about instructors practices related to this
aspect.
The next section, the CoI survey, was adapted to be able to use it to have instructors
reflect on their teaching practices. The following example shows how the first item in the
Teaching Practice, Design and Organization section in the CoI survey for students was
modified to suit the survey for instructors:
The instructor clearly communicated important course topics.(CoI - students)
As the instructor, I clearly communicated important course topics.(CoI -
modified item for instructors)
The next three sections are the same as the ones in the survey for students. That is,
instructors were asked about the instructional activities included in their courses and if they
perceived those activities as engaging. However, in the demographics and general
information a question about how instructors made the most of online discussions was
added.
Interviews
Semi-structured Interview of Students
The interview questions were used to gain a deeper understanding of the pedagogical
practices that students considered important to include in an online course. The interview
80
guiding questions inquired about course evaluation, pedagogical practices, community
building, discussion forums, and demographics (See Appendix F for semi-structured
interview of students).
Evaluation questions
The first questions were aimed at knowing about the types of evaluations used by
instructors and institutions to know about their performance. Students were also asked
when this evaluation was carried out, and how they evaluated the course; that is, the aspects
they considered important to have a successful learning experience. These questions lead to
knowing more about the types of tasks related to perceived meaningful learning.
Instructor pedagogical practices and Community building
Since pedagogical practices are mainly perceived as direct instruction from the
instructor, inquiries about instructors’ impact on the online course, appropriate level of
interaction, and actions performed by the instructors that students found interesting or not
productive were the objective of this section.
Four questions were related to community building. In this section students could share
their opinions about the need to build community and their suggestions about how to do it.
Students were also asked to explore their experience about dealing with multicultural views
in the building of this community in an online environment.
Discussion forums
Participation in discussion forums was the instructional activity present in all courses.
It seemed relevant to know more about the way discussions are handled since they are a
basic element in OLTE courses. Students were asked to share their common practices and
81
their opinion about this activity, and if they considered it overwhelming or bearable, how it
was handled, and how they thought it could be improved.
Semi-structured Interviews of Instructors
The guiding questions to interview instructors were aimed at learning about their
institution requirements; their pedagogical practices in general and how they evaluated
these practices or how they knew their practices were effective; the challenges faced as
online instructors; and how they could improve their course and their instruction (See
Appendix G for Semi-structured Interview of Instructors).
Instructors duties as required by the institution
These questions asked about the academic demands of their job as online instructors
such as course design involvement, the possibility to change course content or assessment,
and if there is a requirement to spend a specific number of hours online as it is when they
teach in a traditional classroom.
Instructors pedagogical practices
Questions included were related to the way instructors constantly evaluated they
course. Their opinion about building a community of learning and the activities they
promoted to foster a successful community were also inquired. A question related to
instructors’ participation in the course in general and particularly in discussions would
provide information on their beliefs about their role in the online environment to be able to
compare them with students’ expectations. Instructors were also asked about their strategies
82
to assure quality of instructions and how they evaluated their course apart from the
institution evaluation.
Instructors challenges and ways to improve instruction
It is important to consider a space for instructor to express the difficulties or
challenges they have faced and how they have overcome them. These challenges shape the
instruction since they provide enriching experience. Additionally, these challenges and its
solutions can be compared to the ones presented in face-to-face courses. By asking
instructors to reflect on the challenges faced, they might share their ideas about the ways
they think they can improve their course and overall instruction.
Semi-structured Interviews of Coordinators
Coordinators were contacted in order to know mainly about their practices to assure
quality of instruction. They were also asked some general information regarding their own
training to teach online, and the instructors of their program training to teach online. The
characteristics of effective OLTE instructors and their pedagogical practices were other
basic inquiries in these interviews. Additionally, questions about course design practices
and instructors duties were included (See Appendix H for Semi-structured Interviews of
Coordinators). These questions aimed to open the main topic of how they handled the
evaluation of their OLTE courses.
Data Collection Procedures
A data base of OLTE programs with information regarding program coordinators
names and emails was built. This information is available online using a web search tool.
83
Coordinators from the different institutions were asked to forward an email inviting their
instructors and former students to answer a survey. Other academic acquaintances were
contacted and asked to share the email with their colleagues as well. In this invitation,
instructors and students were told that no personal information other than demographics
would be asked in the survey and that it was anonymous. The email had information about
the research permission given through the Institution al review Board (IRB), the purpose of
the study and its benefits. A link in the email invitation would take possible participants to
the survey. The online survey software used to set up this survey and collect data was
Qualtrics. The survey included an informed consent, researcher contact information,
purpose of the study, risks, and benefits. Participants were advised to leave the survey at
any time if they felt discomfort answering the questions.
Due to the low response rate at the beginning, an offer to participate in a raffle of some
gift cards was included in the email inviting instructors and students to answer the survey.
Then a link in the survey where participants could provide name and email to participate in
the gift cards raffle was set (see Appendix I for Invitation email). Also at the end of the
OLTE survey there was a link to another short survey where participants could write their
name and email to be contacted for a further interview. Qualtrics allows embedding or
linking short surveys in a way that it seems to be only one survey to be able to handle data
in separate sets. Participants were told that the information they would provide in those
specific sections was going to be saved in a separate data set. The intention in this research
to separate data was to be able to keep the main survey anonymous in order to obtain more
insightful and perhaps honest data, to only know the names of the people interested in the
gift card and to have another set with the participants who would be contacted for an
interview. This new offer and help from OLTE coordinators did increase response rate;
84
personal messages to students directly from coordinators made the difference since students
probably felt that by sharing their experience, their opinion would be considered or that
they could contribute to the improvement of the courses.
Participants who gave their name and email to be contacted for further interview were
approached to set a meeting via Skype or Google docs. Some OLTE instructors, students
and coordinators whom the researcher knew that had answered the survey were asked if
they would agree to a face-to-face or online interview. All participants interviewed were
sent emails to set time and date for the interviews. One of the instructors interviewed
provided other information such as rubrics and course syllabus, and one of the coordinators
allowed the researcher to have access to the overall course design including rubrics,
syllabus and discussion questions in forums.
Data Analysis Procedures
The Statistical Package for the Social Science (SPSS) v. 22 was used to perform
statistical analysis. Data from the survey was downloaded and cleaned up to be able to
perform descriptive and inferential statistical tests. Descriptive statistics such as frequency
distributions, means and standard deviation, were run in order to analyze general
information and form a general idea of the sample. Inferential statistical tests such as
correlations and principal component analysis were used to determine the relationship
between the CoI elements and other constructs in the survey.
Quantitative analysis was conducted in order to find correlations between the CoI
constructs: teaching, social and cognitive presence and the proposed construct: language
teacher educator presence. A principal component analysis was run as well to determine
85
how factors loaded and if the results from this research could be compared to previous
studies with different populations.
Information from interviews was analyzed and grouped as similarities and
differences in opinions were identified. The qualitative analysis provided a deeper
understanding of what students and teachers find important to have in an online course in
order to consider it successful.
Ethical Considerations
Data collection was started after the Institutional Review Board (IRB) approved the
research. The study posed no risk to participants. In order to be as objective as possible in a
complex issue that involves subjective evaluation such as perceptions, the following
considerations were made: the researcher was not an online instructor or student at the time
the study was conducted; she is not a former instructor of the courses in the research; data
was collected through a post-course learning experience anonymous survey and interviews
to participants who stated in the survey they were willing to be interviewed. That is,
students would not feel their grade was compromised and it was hoped that by assuring
participants that their information was anonymous, they would express themselves more
freely while sharing their experience.
Since answers from the survey were anonymous, the researcher kept only emails
and names participants interested in the gift cards drawing and the ones who agreed to a
further interview. The survey software allowed having separate files; thus information was
not connected. Interviews do not include identifiable information and reports about them
86
will consider participants’ demographic information in its description when necessary for
clarification purposes.
Summary
This research considers perceptions of main participants in the OLTE scenario:
coordinators, instructors and students. Its main goal is to learn about pedagogical practices
considered effective by and for language teachers in an online environment. Electronic
surveys and online and face-to-face interviews are the instruments considered to gather
pertinent data. Previous studies and literature have been considered to build and contribute
new findings.
87
Chapter 4 Results
The purpose of this mixed-method study was to examine online instruction aimed at
language teachers and future language teachers in order to identify pedagogical practices in
OLTE courses that support a quality learning experience; hence, that are considered
effective by the main characters involved in this exchange namely coordinators, instructors
and students. These participants’ view points were asked to share their former experiences,
expectations, and overall perceptions in relation to the pedagogical practices in this type of
environment. Their views will inform course designers and instructors to enhance online
instruction. The research design for this study is non-experimental and correlational in
nature. A mixed-methods approach allowed for data gathering from surveys and interviews.
Data was collected through a post-course learning experience anonymous survey and
interviews of participants who stated in the survey that they were willing to be interviewed.
This chapter includes information concerning the reliability and validity of the
quantitative data gathered in the survey. Basic descriptive and inferential statistical tests
were run and their pertinence in this study is explained. Qualitative data analysis from the
interviews is presented to support findings as well. Thus, this chapter presents the analysis
of results to outline findings concerning the general characteristics of the participants, types
and contexts of OLTE courses, the relationship between the presences in the community of
inquiry framework, and quality of instruction measures in these courses. First, a broad
description of the participants in this study (coordinators, instructors, and students) is
provided. Answers to the research questions are presented following these descriptions.
88
Description of Participants
Demographic and General Information
As it was mentioned, information was gathered from different groups of OLTE
course participants. A total of 18 instructors and 125 students answered a survey and from
those groups, eight instructors and 20 students were interviewed either online or face-to-
face using a semi-structured interview questionnaire. Coordinators were contacted directly
and asked for an interview as well; two of them agreed to participate. Participants’
demographic and general information from interviews and surveys is presented as follows.
Interviews
OLTE Program Coordinators who answered interview questions
Two of the contacted OLTE program coordinators answered the invitation and
agreed to participate in an interview. A summary of their demographic and general
information is on table 3.
89
Table 3
Summary of Coordinators information
Coordinator 1 Coordinator 2
Type of program OLTE Certificate program Online BA in ELT program
Level of Education MA in English as a Second
Language
PhD in Education
Gender Female Female
Nationality American Mexican
Years of professional OTLE /
LTE experience
5 years OLTE experience 26 years of teaching
experience
Courses taught online 5 courses. She pilots them
when there are changes.
None as coordinator
One in Teaching English
Training to be an online
coordinator
No No
OLTE instructors who answered interview questions
Eight female instructors shared their experience in a semi-structured interview. Five
were American, two were English and one was Mexican. The age range of six of the
instructors is from 30 to 40 years old, plus one being in the range of 25 to 30 years old and
another one over 40 years old.
All of them had completed a master’s degree program. Five of them teach in an
OLTE certificate program, two in an OLTE undergraduate program and one in a
professional development course that is a certificate in online tutoring aimed to train
language teachers. The instructors from the OLTE certificate courses are colleagues
teaching in the same OLTE program. They answered that they did not have studies related
90
to online teaching; however, they mentioned that their training to teach online consisted on
being observers in an online course previous to the teaching of their course and that they
had regular meetings with their colleagues who are online instructors as well. Four of them
had had the experience of being online students themselves. The OLTE undergraduate
instructors and the professional development course instructor had studied their masters
program online and two of them are currently online students in a doctorate program. Six
instructors had more than two years of experience training language teachers online and
two instructors had been working as online instructors for more than six months but less
than a year.
OLTE students who answered interview questions
A total of 20 students were interviewed. Seven students were male and 13 were
females. Sixteen students were Mexican and four were American. Concerning their age
range, five students were from 25 to 30 and other five from 31 to 40 years old, 10 were
over 40 years old.
Six of them were enrolled in a degree-seeking course (three in an undergraduate
program and three in a master’s degree program); eight were former students from a
certificate course and the other six students had been recently enrolled in a professional
development course related to language teaching methodologies, materials, or the fostering
of critical thinking strategies. Eighteen were practicing language teachers; one is an
undergraduate instructor who plans to work as a language teacher and another one would
like to begin an English language teaching career now that she has retired from her job.
Regarding language teaching experience, 80% (16 students) had more than 8 years of
91
experience as language teachers, two had from 6 to 8 years and the other two had less than
4 years of experience.
Survey
OLTE instructors who completed the survey
OLTE instructors who completed the survey were 18. Their information regarding
gender, age, nationality, educational level and teaching experience and online experience is
shown in table 4.
Table 4
Instructors’ demographic information
Characteristic Instructors (N=18)
Gender
Female
Male
(n%)
17 (94.4%)
1 (5.6%)
Age
25-30
31-40
Over 40
1 (5.6%)
6 (33.3%)
11(61.1%)
Nationality
USA
UK
Mexico
Colombia
Thailand
Poland
6 (33.3%)
4 (22.2%)
4 (22.2%)
2 (11.1%)
1 (5.6%)
1 (5.6%)
Professional Experience
4-6
6-8
8+
2 (11.1%)
3 (16.7%)
13 (72.2%)
Online Teaching Experience
Less than 6 months
6 months – 2 years
2 – 4 years
4 – 6 years
More than 6 years
5 (27.8%)
3 (16.7%)
2 (11.1%)
6 (33.3%)
2 (11.1%)
92
Training to Teach Online
Yes
No
16 (88.9%)
2 (11.1%)
Teaching a … course
Professional Development
Certificate in LT
BA in LT
MA in LT
4 (22.2%)
5 (27.8%)
5 (27.8%)
4 (22.2%)
OLTE students who completed the survey
Gender, Age and Nationality
The sample was formed by 125students of online language teacher education
courses: 81 (65%) female and 44 (35%) male. Their age varied between 25 to over 40
years old, but it is mostly from 31 to over 40 (88%). They were mainly from Mexico
(78%), and from the USA (12%); the remaining 9.6% is represented by one student each
from the following countries: Canada, Cuba, Chile, Ecuador, New Zealand, South Africa,
Thailand, Turkey, Uruguay, and three students from the UK.
Table 5
Students’ Gender, Age and Nationality
Characteristic Students (N=125)
Gender
Female
Male
(n%)
81 (64.8%)
44 (35.2%)
Age
25-30
31-40
Over 40
15 (12%)
34 (27.2%)
76 (60.8%)
93
Nationality
Mexico
USA
Canada
Chile
Cuba
Ecuador
New Zealand
South Africa
Thailand
Turkey
UK
Uruguay
98 (78.4%)
15 (12%)
1 (0.8%)
1 (0.8%)
1 (0.8%)
1 (0.8%)
1 (0.8%)
1 (0.8%)
1 (0.8%)
1 (0.8%)
3 (2.4%)
1 (0.8%)
Educational Level and Teaching Experience
The majority of the participants, 96%, were in-service language teachers with more
than 2 years of experience (See Table 6). The degrees previously earned by participants are
presented in table 7; other degrees mentioned were MA in Education, MA in
Administration, MA in Linguistics, and a Mechanical Engineer teaching content in English.
Table 6
Students’ Years of Professional Experience
Frequency
Percentage
Valid 0-2 5
4.0
2-4 5
4.0
4-6 7
5.6
6-8 16
12.8
more than 8 92
73.6
Total 125
100.0
94
Table 7
Degrees Previously Earned
Degree %
Certificate in Language Teaching (or related to LT) 45
BA in Language Teaching (or related to LT) 37
MA in Language Teaching (or related to LT) 30
PhD in Language Teaching (or related to LT) 6
Other 33
Online Experience
Participants were students who had been enrolled during the years of 2008 through
2015 in fully-online professional development courses (66%), TESOL or TEFL certificate
courses (18%), or degree-seeking (BA, MA and PhD) courses (16%). The majority of the
students, 76%, were considering a recent OLTE course (2013-2015) to answer the survey.
The courses considered were related to EFL/ESL methodology, assessment, technology,
research resources, English for Specific Purposes, and listening and speaking skills. Most
of the students referred to certificate courses and professional development that are
provided by American institutions, other considered courses from an undergraduate
program in English Language Teaching in Mexico and graduate programs from American
and British institutions. (See Appendix J for more detailed information).
Most participants (74%) had been OLTE students for at least six months at the time
they answered the survey; 33% were in the 6 months to 2 year group, 26% in the 2 to 4
years group, 6% in the 4 to 6 years group and 10% in the more than 6 years group. Results
related to the training to study online from the institution offering the OLTE course can be
found on table 8; participants who said they would have liked to receive training stated that
they would have found it helpful to be able to navigate more efficiently (spend less time
95
clicking to learn where things were and use platform better); participants who did not think
training was necessary mainly answered that they had some previous online learning
experience, and that courses were user-friendly.
Table 8
Percentage of participants who had training to study online from the institution offering the
OLTE course
Yes 33% Online training mini-course 17%
Online information to navigate the course 25%
There is on-going support 14%
No 67% I would have liked to have some type of training
Training is not necessary
44%
56%
Table 9 presents participants’ motivation to enroll in the OLTE course. Students
enrolled in OLTE courses opt to online learning for a variety of reasons; however, the
flexibility that the online environment offers is their main reason. When asked about other
reasons not listed, they mentioned the need for professional development, to update their
practices, the opportunity to study a topic of their interest, the opportunity to be a sponsored
student and to enhance their listening and speaking skills.
96
Table 9
Reasons to choose the OLTE course
N
Flexibility
: I can have access any time and from
any place
104
Cost: No need to move, lower tuition 59
Program's Quality 84
Effectiveness 51
Program's Reputation 51
University/College's Reputation 65
Program's Faculty 36
Other. Please specify:
Professional development
Course was sponsored
Interesting, original, applicability
23
Students Opinion about Online instruction and Online Language Teacher Education
Participant’s general opinion about online instruction was mainly positive (73%,
N=125): 14% think that online instruction is better than face-to-face and 59% think it is the
same. However, 27% still think that online instruction is not as good as face-to-face. When
they were asked to be more specific and compare learning outcomes in an OLTE program
to those of face-to-face programs, their answers were mainly positive as well (see table 10).
Finally, when participants were asked if they felt overwhelmed by the amount of
information in the OLTE course, 38% answered they did and 62% responded they did not
feel overwhelmed.
97
Table 10
Learning Outcomes in an OLTE program are _____ than those in a Face-to-Face program
Frequency
Percent
Weaker 27
21.6
No Change 53
42.4
Stronger 33
26.4
Much Stronger 12
9.6
Total 125
100.0
Summary of Participants General Characteristics
Participants in this research were OLTE program coordinators, instructors and
students. Instructors and students were invited to complete an online survey and upon
completion of the survey were asked to participate in an interview. Survey respondents
were 125 students and 18 instructors; 20 students and eight instructors were interviewed.
The majority of the participants in general (N=145: 2 coordinators, 18 instructors and 125
students) were females (70%) over 40 years old (61%). Two program coordinators were
interviewed only, that is, they were not asked to complete a survey, one is in charge of a
Certificate in Teaching EFL program and the other of a BA in ELT program.
Instructors
Instructors were mainly from the US, UK and Mexico. Most instructors’
professional experience was of more than eight years (72%) and their online teaching
experience was divided between less than six months to 2 years (44%) and from 2 years to
more than 6 years (56%). Eighty-nine percent of instructors have had some type of training
98
to teach online. The instructors who answered the survey teach in Professional
Development courses (22%), Certificate in Language Teaching (28%) and courses from a
BA or MA in Language Teaching (50%). Most instructors’ (83%) general opinions about
online learning are positive.
Most instructors interviewed were in the age range of 30 to 40 years old; they all
had completed a master’s degree and none had a degree in online teaching. Five instructors
teach in an OLTE certificate program. Concerning online language teacher training
experience, six instructors had more than 2 years experience.
Students
Students who answered the survey were mainly from Mexico (78%) and the US
(12%).Most of them had more than six years of language teaching experience (86%) and
had been OLTE students for at least 6 months (74%). Students who had training to study
online were only 33%. The opinion of the 67% who did not have training was divided since
56% said it was not necessary and 44% responded they would have liked some type of
training. Students were mainly former online learners in Professional Development courses
(66%). The reason why students opted for an online course was mainly the flexibility of
studying anytime anywhere (83%), the program’s quality (67%) and the university or
college reputation (52%). Most of the students (N=125) general opinion about online
education was positive (73%); only 22% think the outcomes in online courses are weaker
than those of traditional programs.
Half of the students interviewed were over 40 years old. Students interviewed had
been enrolled in one of the three types of courses categorized and they had a nearly even
99
participation: six former students of professional development courses, eight students of a
certificate course and six students of an undergraduate or graduate program. Eighty percent
of the students interviewed had more than 8 years of language teaching experience.
Research Questions Results
The overarching question for this research is the following:
What are the most effective pedagogical practices in Online English Language Teacher
Education (OLTE) courses as identified by experts, instructors and students?
In order to answer this question, first the types of courses and the contexts in which
they are commonly offered were investigated. Second, the Community of Inquiry, a
validated and reliable framework related to pedagogical practices in the online environment
was used to explore the relationships between its presences in OLTE courses in order to
find about distinguishing features. Finally, information regarding the measures taken in
relation to the assurance of quality of instruction was relevant to incorporate due to its
relationship to student perception of effective practices that lead to satisfactory learning
outcomes. Thus, in order to answer the main question, answers to the following research
questions were summarized and analyzed.
Types and Context of OLTE courses
1. What are the general characteristics of the OLTE courses offered to in-
service language teachers and future language teachers?
Online courses offered to language teachers are greatly varied. However, one way
they can be classified for this research is as courses for professional development (related to
100
teaching methodologies, technology, skills development); courses to receive a certificate in
langue teaching for general or specific purposes and for other specific audiences as well
such as the teaching of adults, young adults or young learners; and courses which are part
of Bachelor’s, Master’s and Doctorate programs (degree-seeking).
Due to the wide range of OLTE courses and programs, these courses duration varies
as well. Online courses for degree seeking students such as those that are part of an
undergraduate and graduate program usually are 16 weeks or one-semester long. However,
professional development courses and certificate programs can be advertised as 40, 50, 60
hour courses, 6 weeks, 2 month, intensive and so forth. Particularly in this regard, the
selection of a certificate program can be a difficult task. First and foremost, the credentials
and types of jobs that require them can be disorienting for the prospective student doing an
online search. For some programs, language teaching experience is not necessary. Others
require students to have some previous experience in language teaching and to do some
core courses and then some electives to obtain their certificate. Additionally, some
certificate courses are divided and students can take the basic and advance course, part 1
and part 2, or foundation and specialty courses. Some include a practicum component and
this might be on site; that is, students would have to attend regularly face-to-face meetings,
observe other teachers and teach in the institution where the online courses are offered.
When there is the possibility of doing the practicum online, students have to find a teacher
who would be their tutor and let them observe their classes and teach their class; these
instructors have to evaluate the student and advise them. If the students are already
language teachers, then they can present evidence of their work on their own class.
101
It is important to mention that selecting a quality course among these types of
courses is a laborious enterprise. One of the students interviewed, a Mexican English
teacher with more than 10 years of experience teaching English mentioned that he had a
language teaching certificate from an institution in Canada and that at the Mexican
university where he currently works told him he had to do another certificate because they
would not consider that as a valid certificate; he said that even though he has a master’s
degree in teaching, he was not able to advance in his institution because of a lack of
credentials related to English language teaching specifically. Consequently, he opted for a
certificate. In the end, he had to complete two; the certificate recommended by the
university where he works is from an American university.
Selecting a certificate program is a difficult task since there is a lack of information
regarding the programs that have been evaluated and certified by a higher body or agency;
and when they do have that information on their website, it is not clear if those bodies or
agencies should be the ones certifying those online courses as well. Other former OLTE
students from Mexico mentioned that the institution where they work sponsored their
certificate and teachers who do have a BA in English language teaching degree also
enrolled because it would give them an extra credential from an American institution and
that they assumed it had a higher value.
In summary, as it was expected, survey results confirmed that the types and contexts
of the OLTE courses are very varied. They can range from a professional development
course, to language teaching certificate courses to courses for credit in an undergraduate or
graduate program. Awareness of this variety of course types and contexts is important
because students have to be careful when selecting an online course due to the wide range
102
of courses available. The offering of online certificate courses presents a particular
challenge to the interested teacher or future teacher. The information about these courses
varies greatly and can even be confusing in relation to the length of time it takes to earn a
certificate and the type of certificate that is obtained.
There is the possibility of falling into marketing techniques as well. Online courses
or programs might mention the quality of their courses through their site. This quality
might be offered through the “latest methodology” by highlighting that it is student-
centered for example, or their affiliations to well-known language teacher organizations
(i.e. TESOL). However, there is usually a lack of information regarding credentials and
certifications. That is, it is not common to find that the courses have been certified by
Quality Control agencies, and if this information is included, it is not explained that those
agencies are the ones in charge of certifying their online courses or programs. A Quality
Matters credential with a brief explanation of its value and links to further information
about this agency would be a strong addition to the information on the websites of these
courses and programs.
Seventy-five percent of the OLTE courses considered by students when completing
the survey were from 2013-2015; that is, students were basing their answers in a recent
online learning experience. Most participants in this research were former sponsored
students; either they received a scholarship from the university where they work or from a
recognized institution that offers scholarships to Mexican English language teachers.
Therefore, it is assumed that those institutions consider these OLTE courses as courses with
curriculum value. However, some participants were also not sponsored students who
103
selected online courses or programs mainly because they were a conveniently available
option to professional development, credentials or to further their educational level.
The national context from where the online courses are offered seems to uphold the
concept that online courses from Inner Circle countries (Kachru, 1986) were deemed to
have higher levels of endorsement. Comments from other Mexican teachers who already
had an English language teaching degree from a Mexican institution also reflected their
positive perception of having a certificate from an American or British institution. These
students completed an online certificate in Teaching EFL or a master’s degree and shared
how they thought American or British institutions have high quality programs. This
endorsement gave them the certainty that the courses were of high quality. This opinion is
in line with previous studies about the perception of international students about the US
higher education system; in this study 87% of Mexican students perceived the US to have a
high quality higher education system (What International Students Think about U.S. Higher
Education, 2015). When these OLTE research participants were asked about the reason of
their perception, some mentioned how it was also important that these courses were offered
as part of a university continuing education program, or from the TESOL organization
online courses.
Community of Inquiry and Online Language Teacher Education
Effective pedagogical practices are definitely directly related to student success and
satisfaction. How these are cultivated in online contexts is critical. One way of exploring
these concepts is through a CoI framework. The CoI survey is considered a reliable
resource that tells about the establishment of a learning community through engaging
104
critical thinking tasks (Garrison et al.,2000). The CoI framework considers teaching, social
and cognitive presence as essential in online instruction. The teaching presence includes
aspects related to the design and organization of the course, facilitation and direct
instruction. Social presence considers affective expression, the possibility of having open
communication and developing group cohesion. Cognitive presence refers to the inclusion
of engaging activities that foster critical thinking that lead to the interest to explore, co-
construct meaning and the ability to apply the acquired knowledge in tasks outside from the
ones included in the online course.
The CoI survey has been validated with data from online courses from different
disciplines such as Business Administration, Education, and Instructional Design among
others (Ke, 2010; Kozan & Richardson, 2013; Moreira et al., 2013). It has also been
translated to other languages to explore the reliability and validity in the version of the
language in question with positive results (Alaulamie, 2014; Yu &Richardson, 2015). Even
though the framework has been used in previous OLTE studies (Chen, 2012; Shin,
2008),the survey itself has not been validated in OLTE courses. This attempt to validate the
survey considered the recommended steps followed in previous studies. The objective in
this study was to learn about the relationships of the CoI presences in OLTE courses and
how the building of community is perceived by OLTE students; the research question
proposed is the following:
2. How are the Community of Inquiry (CoI) elements incorporated into
OLTE courses?
a. What is the relationship between the CoI elements: teaching social,
and cognitive presence in an OLTE program?
105
CoI elements: cognitive, social, and teaching presence are considered as significant
predictors of students’ satisfaction (Swan& Ice, 2010); thus, related to provide a quality
learning experience. Spearman rho and partial correlations were run to determine the
interdependence between CoI presences and to what extent social and teaching presence
predict cognitive presence. The CoI section of the students’ OLTE survey was analyzed
considering statistical tests run in earlier CoI studies in order to compare and contrast
results from this discipline and those previously reported, and to establish the validity and
reliability of the results from this research. In general, scores obtained in the social,
teaching and cognitive presence reflect that students did perceive a sense of belonging to a
community of inquiry. Social presence presented a stronger relation to cognitive presence
than teaching presence and further research about how teaching presence could be
improved to obtain more positive learning outcomes is necessary.
Perceived Community of Inquiry from OLTE Survey for Students
Ordinal responses to items used to assess the degree to which teaching, social, and
cognitive presence were perceived in the courses were scored using a 6-point Likert-type
scale (1 =strongly disagree; 2 = disagree; 3 = somewhat disagree; 4 = somewhat agree; 5 =
agree; 6 = strongly agree). Arbaugh et al. (2008) used a 4-point Likert-type scale when they
tested and validated the CoI survey with a multi-institutional sample. Subsequent studies
have used a 5-point Likert scale (Kozan & Richardson, 2014;Swan et al.,2008). According
to Chomeya (2010), a 6-point scale tends to force respondents to choose a direction. Since
the survey included other multiple choice items besides the ones from the CoI framework
as well, it was concluded that in order to construct a more uniform and friendly instrument,
a six-point scale was to be used for the complete survey.
106
The Shapiro-Wilk test is a test of normality appropriate for small sample sizes. It
tests the null hypothesis that the samples come from a normal distribution. It was run for
this data and it showed that none of the 34 analyzed variables presented a normal
distribution. All variables showed a negative skewness meaning that students’ answers
were mainly positive and that they perceived they belonged to a community of inquiry.
Descriptive statistics (i.e. mean, minimum, maximum, standard deviation) help in
the evaluation of suitability of the 34CoI item measurement. Appendix K shows the
descriptive statistics of the items. Results show that the data mean responses for all the 34
CoI survey items ranged from 5.70 for item 4 (The instructor clearly communicated
important due dates / time frames for learning activities) as the highest to 4.38 for item 16
(Online or web-based communication is an excellent medium for social interaction) as the
lowest. These same items had the highest and lowest standard deviations: Online or web-
based communication is an excellent medium for social interaction (SD= 1.287), and The
instructor clearly communicated important due dates / time frames for learning
activities(SD=.599).
When considering responses collectively, teaching presence items yielded a mean
score of 5.42 (SD=.80), social presence items ratings yielded a mean score of 4.89 (SD =
1.04)and cognitive presence items yielded a mean score of 5.32 (SD =.82). The global
median was of 5.24 and the global variance of .78. In general, considering these results, one
can assume that students perceived a sense of belonging to a community of inquiry.
Scores Reliability
Cronbach’s Alpha reliability coefficient is commonly used to evaluate how reliable
the scores of the instrument are. It calculates how well items measure a scale. According to
107
George and Mallery (2003) and Blunch (2008) a scale with a Cronbach’s Alpha of .9 has an
excellent internal consistency and a .8 scale has a good internal consistency. In this study,
the items from the CoI framework correlate well with each other and reveal excellent
internal consistency (See Table 11).
Table 11
Reliability coefficients of each factor and of total CoI
Factors Alpha Number of
Items
Teaching Presence
.95 13
Social Presence
.90 9
Cognitive Presence
.93 12
Total CoI
.96 34
Scores Validity
Previous studies mention the use of the Principal Component Analysis (PCA) to test
the construct validity of the CoI instrument (Arbaugh et al., 2008; Diaz, Swan, Ice &
Kupczynski, 2010; Kozan & Richardson, 2013; Shea & Bidjerano, 2009; Swan et al. 2008;
Yu & Richardson, 2015). A PCA is used when variables are highly correlated. This
analysis helps in the reduction of observed variables to principal components. Thus, it
allows the reduction of data dimensionality. The objective is to explain variance in the least
number of possible components. This analysis transforms correlated original variables in
another group of variables not correlated called factors or principal components which are
108
related to the original ones through a linear transformation (Meyers, Gamst, & Guarino,
2013; Ho, 2013; Suhr, 2009). In order to run a PCA, sample adequacy has to be considered.
In this study N=125 and the CoI items are 34; that is, there is a ratio of 3.7 participants per
item. According to Kline (1994) if the factor structure is clear, samples of 100 are adequate.
He adds that if ratio is considered, the necessary minimum is 2:1. Therefore, an analysis of
Principal Components using SPSS version 22 was used to further explore the34 items from
the CoI framework in this study to be able to draw comparisons with previous studies.
When items are interdependent, oblique rotation should be used. Loadings lower than .35
were suppressed in the statistical options in order to have a clear visualization of significant
values. In order to mirror previous analysis, and since Teaching, Social and Cognitive
Presence items are interdependent, Direct Obliminal with δ=0 specified to three factors
was run for the best correspondence to the framework.
The Principal Component Analysis output in SPSS includes the Kaiser-Meyer-Olkin
(KMO) test and Bartlett’s test. KMO is a measure of sampling adequacy. A result closer to
1 and higher than .80 suggests that it is appropriate to run a factor analysis and that it
should yield reliable factors (Field, 2009). Bartlett’s test of sphericity assesses that the
variables are unrelated and not appropriate for structure detection. A significance level of
less than 0.05 indicates that this analysis might be useful. Table 12 shows that data from
this study yielded a KMO of .906; the Bartlett’s test significance level is lower than 0.05.
Thus, these results indicate that conducting factor analysis on this data is appropriate.
109
Table 12
KMO and Bartlett Test
Kaiser-Meyer-Olkin Measure of Sampling
Adequacy
.906
Barlett’s Test of
Sphericity
Aprox. Chi-Square 3931.754
Df 561
Sig. .000
Figure 4.1 presents the Scree Plot were Eigenvalues are plotted against the number
of factors to determine where a significant drop presents within factor numbers
(Netemeyer, Bearden & Sharma, 2003). As it was mentioned, assuming the best
correspondence to the framework, a three-factor solution was specified.
Figure 4.1 Scree Plot
110
Previous studies have shown that PCA yields more than three components
(Alaulamie, 2014; Arbaugh et al., 2008; Diaz et al., 2010; Moreira et al., 2013). This
demonstrates that the three factors of the community of inquiry can overlap. In this study,
there were six factors with Eigen values greater than one. However, when analyzing the
resulting factors, it was observed that these were similar to the ones found in the mentioned
studies. Table 13 and Fig 4.1 show how the first component loaded the most variance
(46%) and that there was an evident decline in the following ones; even though the last
three components presented factors greater than one, the percentage of its variance was
lower than 5. The first three factors loaded 62% of the variance; in Arbaugh et al. (2008)
study to validate the CoI survey, the cumulative percentage for three factors was 61%. The
scree plot shows the total variance associated with each factor and that six factors loaded
Eigen values greater than one. It also shows a marked decrease after factor three.
Table 13
Eigenvalues for Principal Component Analysis
Component
Total
% of Variance
Cumulative %
1 15.69
46.15
46.15
2 3.41
10.02
56.18
3 1.87
5.51
61.69
4 1.58
4.65
66.34
5 1.47
4.34
70.67
6 1.27
3.74
74.41
111
Table 14 shows the Pattern Matrix generated with the 34 CoI items factor loadings,
with the three factors specified. Results are highlighted for interpretability. Values of less
than .35 were excluded from the analysis to have a clearer picture of where the factors
loaded. When running this analysis with three factors it is expected that all the teaching
presence items, all the social presence items, and all the cognitive presence items to load in
a separate component respectively.
Overall results from other studies have been varied. For some these expectations are
met and teaching, social and cognitive presence constructs correspond to a different
component (Arbaugh et al., 2008; Swan et al., 2008); however, other studies have presented
results in which items load in a different component as it is the case in Moreira et al.(2013)
which reflects how the three CoI constructs overlap.
As it can be observed, Teaching Presence items related to the design and
organization of the course are clustered together on factor 3. The rest of the Teaching
Presence items loaded on factor 1. All Social Presence items loaded in factor 2. Six of the
12 Cognitive Presence items loaded on Factor 3. From the remaining six Cognitive
Presence items, five items loaded on factor 2, Social Presence, and one in factor 1,
Teaching Presence.
Interestingly, the Design and Organization Teaching Presence items had shown this
behavior in previous studies by Arbaugh (2007) and Shea at al. (2005); that is, they have
reported that these four items load together in a different component from the rest of the
teaching presence items. These authors have concluded that perhaps students seemed to
perceive those items belonging to a separate factor due to the time when these activities
happen during the course, which is usually before the course begins. In a study by Moreira
112
et al. (2013) where the CoI survey was answered by 510 higher education Portuguese
students in blended online courses, findings also show an overlap of presences. In that case,
Social Presence and Cognitive Presence loaded mostly in factor 1.
In this OLTE results, component 2 was the one with most items. Another salient
aspect from the PCA run in this data set is the fact that with the exception of the previously
mentioned four design and organization items, the rest of the teaching presence items all
loaded in component 1 and all social presence items loaded in component 2.
113
Extraction Method: Principal Component Analysis.
Rotation method: Oblimin with Kaiser Normalization.
Rotation converged in 17 iterations.
Table 14
Pattern Matrix
Component
1 2 3
TPDO1clearly communicated important course topics. .462
.626
TPDO2clearly communicated important course goals.
.732
TPDO3provided clear instructions on how to participate in course learning activities. .663
TPDO4clearly communicated important due dates/time frames for learning activities. .664
TPF1was helpful in identifying areas of agreement and disagreement on course topics that helped me
to learn. .702
TPF2was helpful in guiding the class towards understanding course topics in a way that helped me
clarify my thinking. .741
TPF3helped to keep course participants engaged and participating in productive dialogue. .832
TPF4helped keep the course participants on task in a way that helped me to learn. .838
TPF5encouraged course participants to explore new concepts in this course. .925
TPF6reinforced the development of a sense of community among course participants. .783
TPDI1 helped to focus discussion on relevant issues in a way that helped me to learn. .758
TPDI2provided feedback that helped me understand my strengths and weaknesses. .767
TPDI3 provided feedback in a timely fashion. .705
SPAE1Getting to know other course participants gave me a sense of belonging in the course. .504
SPAE2I was able to form distinct impressions of some course participants. .531
SPAE3Online or web-based communication is an excellent medium for social interaction. .638
SPOC1I felt comfortable conversing through the online medium. .740
SPOC2I felt comfortable participating in the course discussions. .789
SPOC3I felt comfortable interacting with other course participants. .897
SPGC1 I felt comfortable disagreeing with other course participants while still maintaining a sense
of trust. .767
SPGC2I felt that my point of view was acknowledged by other course participants. .742
SPGC3Online discussions help me to develop a sense of collaboration. .630
CPTE1 Problems posed increased my interest in course issues. .572
CPTE2 Course activities piqued my curiosity.
.425
CPTE3 I felt motivated to explore content related questions. .445
CPE1 I utilized a variety of information sources to explore problems posed in this course. .421
CPE2 Brainstorming and finding relevant information helped me resolve content related questions.
.375
CPE3 Online discussions were valuable in helping me appreciate different perspectives.
.402
CPI1 Combining new information helped me answer questions raised in course activities .515
CPI2 Learning activities helped me construct explanations/solutions. .426
.466
CPI3 Reflection on course content and discussions helped me understand fundamental concepts in
this class.
.463
CPR1 I can describe ways to test and apply the knowledge created in this course.
.609
CPR2 I have developed solutions to course problems that can be applied in practice. .424
.643
CPR3 I can apply the knowledge created in this course to my work or other non-class related
activities.
.584
114
However, the 12 items from the cognitive presence loaded mainly in component 2
and 3: the three Triggering Event items loaded in component 2 (mainly social presence
related component), the three Resolution items loaded in component 3, the cognitive
presence related component; two items from the Integration section loaded in component 3
as well. Thus, in general, the component with more items was component 2, the
component mostly related to social presence.
Table 15 shows how much the factors are correlated. Factors 1 and 3 are the most
highly correlated and Factor 2 is mostly correlated with Factor 1.
Table 15
Factor Correlation Matrix
Factor 1 2 3
1 1.000
.393
.473
2 .393
1.000
.379
3 .473
.379
1.000
In a study about CoI students’ perceptions involving different groups of students as
it is the case of this study considering former students from different types of courses,
Moreira et al. (2013) ran a Kruskal-Wallis H test to analyze if the groups’ results differed
considerably about how they perceived themselves as members of a CoI. The non-
parametric Kruskal-Wallis H test is used to compare more than two independent samples of
different samples sizes. Participants should only belong to one of the groups to be able to
run this test. It helps determine if there are statistically significant differences. This
statistical procedure ranks the scores; thus, the lowest score gets to be ranked number 1and
115
when two scores are the same they get the average of their rank; for example, if the lowest
score is 11, it would be ranked as number 1, then if there are two scores 12, they would be
2.5, and so forth. Ranks are added and comparisons can be made. When the p value in the
test statistics is less than .05, the null hypothesis of no difference between the groups can be
rejected.
Table 16 presents the number of students who evaluated a course from a degree-
seeking program such as a BA, Masters Degree or PhD, a course from a Certificate
program, or a Professional Development course. These were the groups considered to run
the Kruskal-Wallis H test in order to know if their perceptions differed considerably. It is
important to mention that degree-seeking students have a longer interaction with e-mates
since their program might take more than two years to complete and they might even have
the same professor in different courses. The Certificate program courses group considered
50-hour or 60-hour courses aimed to earn a Certificate in English Language Teaching
which might give it some additional weight regarding curriculum value. The Professional
Development courses group included courses related to EFL/ESL assessment, EFL /ESL
methodology, and technology, among others; these courses usually last 8 to 10 weeks.
Table 16
Student Groups
Frequency
Percent
Valid BA, MA,
PhD 20
16.0
CERT 23
18.4
PD 82
65.6
Total 125
100.0
116
The Kruskal-Wallis H test showed that there was a statistically significant
difference in the Social Presence Group Cohesion item Online discussions help me to
develop a sense of collaboration between the groups with a mean rank of 57.18 for the BA,
MA & PhD group, 46.70 for the Certificate group and 68.99 for the Professional
Development group. This was the only item that presented a significant difference. A
complete table of mean ranks can be seen in Appendix L. A summary of the highest five
mean ranks per groups is shown in table 17.
Interestingly, the BA, MA & PhD group ranked highest three items related to
teaching presence and one to facilitation and a cognitive presence integration item. The
Certificate group scored higher two social presence items and both of them are related to
the affective expression. The teaching presence items from this group are related to the
design and organization of the course. Only one cognitive presence is in this group and it is
from the resolution subsection. The students from the Professional Development courses
ranked two cognitive presence integration items, one teaching presence facilitation items
and two social presence items related to open communication and group cohesion.
Table 17
Highest Five Mean Ranks per Groups
BA, MA & PhD Mean
Rank
TPDO4
(The instructor) clearly communicated important due dates/time frames for
learning activities.
68.8
TPDO3
(The instructor) provided clear instructions on how to participate in course
learning activities.
68.4
TPF2
(The instructor) was helpful in guiding the class towards understanding course
topics in a way that helped me clarify my thinking.
67.1
CPI1
Combining new information helped me answer questions raised in course
activities
66.0
TPDI3
(The instructor) provided feedback in a timely fashion.
65.9
117
Certificate
TPDO1
(The instructor) clearly communicated important course topics.
64.0
SPAE2
I was able to form distinct impressions of some course participants.
62.9
CPR1
I can describe ways to test and apply the knowledge created in this course.
62.9
TPDO2
(The instructor) clearly communicated important course goals.
62.4
SPAE1
Getting to know other course participants gave me a sense of belonging in the
course.
61.9
Professional Development
SPGC3
Online discussions help me to develop a sense of collaboration.
69.0
SPOC3
I felt comfortable interacting with other course participants.
68.1
CPI2
Learning activities helped me construct explanations/solutions.
67.8
CPI3
Reflection on course content and discussions helped me understand
fundamental concepts in this class.
67.5
TPF6
(The instructor) reinforced the development of a sense of community among
course participants.
67.5
TPDO=Teaching Presence Design & Organization, TPF=Teacher Presence Facilitation, TPDI=Teaching
Presence Direct Instruction, SPAE=Social Presence Affective Expression, SPOP=Social Presence Open
Communication, SPGC=Social Presence Group Cohesion, CPI=Cognitive Presence Integration, CPR=
Cognitive Presence Resolution
Even though 33 of 34 items did not present a significant difference, some items
presented a mean rank variance higher than 50; Table 18 presents the results that allow to
observe a considerable difference between groups. From this selection, teaching presence
(facilitation and direct instruction) items were ranked lower by the certificate courses
students. Social presence items were scored lower by the BA, MA, & PhD students mainly.
Cognitive presence items were ranked lower by both BA, MA, & PhD students and
Certificate courses students. The significance of these results lies in the fact that students
from the courses that belong to a degree-seeking program expressed a low social presence
perception despite the fact that they have a longer interaction with their e-mates. Students
in certificate programs perceived a low teaching presence and ranked as the lowest the
sense of collaboration (SPGC3) considered as essential for this type of learning
environment.
118
Table 18
Ranks with clear differences
CoI item Program Clasification N
Mean
Rank
TPF2
was helpful in guiding the class
towards understanding course
topics in a way that helped me
clarify my thinking.
BA, MA, PhD
20
67.1
CERT 23
53.0
PD 82
64.8
TPF5
encouraged course participants to
explore new concepts in this
course.
BA, MA, PhD
20
65
CERT 23
52.2
PD 82
65.5
TPDI3
provided feedback in a timely
fashion.
BA, MA, PhD
20
65.9
CERT 23
51.1
PD 82
65.6
SPAE3
Online or web-based
communication is an excellent
medium for social interaction.
BA, MA, PhD
20
52.8
CERT 23
57.4
PD 82
67.1
SPOC3
I felt comfortable interacting with
other course participants.
BA, MA, PhD
20
49.8
CERT 23
56.5
PD 82
68.1
SPGC1
I felt comfortable disagreeing with
other course participants while still
maintaining a sense of trust.
BA, MA, PhD
20
51.0
CERT 23
59.5
PD 82
66.9
SPGC3
Online discussions help me to
develop a sense of collaboration.
BA, MA, PhD
20
57.2
CERT 23
46.7
PD 82
69
CPTE1
Problems posed increased my
interest in course issues.
BA, MA, PhD
20
52
CERT 23
57.7
PD 82
67.2
CPTE2
Course activities piqued my
curiosity.
BA, MA, PhD
20
51.5
CERT 23
58.2
PD 82
67.2
CPE2
Brainstorming and finding relevant
information helped me resolve
content related questions.
BA, MA, PhD
20
63.3
CERT 23
49.1
PD 82
66.8
119
CPE3
Online discussions were valuable
in helping me appreciate different
perspectives.
BA, MA, PhD
20
62.4
CERT 23
51.6
PD 82
66.3
CPI2
Learning activities helped me
construct explanations/solutions.
BA, MA, PhD
20
55.0
CERT 23
52.9
PD 82
67.8
CPI3
Reflection on course content and
discussions helped me understand
fundamental concepts in this class.
BA, MA, PhD
20
59.9
CERT 23
49.6
PD 82
67.5
Bivariate and Partial Correlations among CoI Presences
In order to identify the relationship between Teaching Presence, Social Presence,
and Cognitive Presence, bivariate and partial correlation analyses were run (Table 19).
Bivariate correlations measure the strength of association between two variables. The
resulting value is called correlation coefficient; a positive correlation coefficient indicates
positive relationship. Additionally, the closest the value is to 1, the higher the correlation,
which suggests that increases in a presence are highly related to increases in the correlated
presence. Partial correlations also measure the association between two variables; however,
in this case, the effect of controlling variables is removed. In this case, the relationship
between two presences is analyzed and then it is compared with the same relationship once
the effect other presence is removed to observe significance. The non-parametric
Spearman’s rank-order (rho) correlation was used since as it was observed from the
Shapiro-Wilk test of normality, the analyzed variables presented skewness.
120
Table 19
CoI Presences Correlations
TP SP CP
Spearman's
rho
TP
Correlation
Coefficient -- .50** .70**
SP
Correlation
Coefficient
**
-- .76**
CP
Correlation
Coefficient
**
**
--
**. Correlation is significant at the 0.01 level (2-tailed).
Relationship between Cognitive Presence and Social Presence
The Spearman's rho correlation was run to determine the relationship between CoI
presences. In general, there was a strong, positive correlation between the three presences.
In particular, Cognitive and Social Presence, show the highest statistically significant
correlation (r
s
(123)= .76, p<.000). The shared variance between these two presences was
58% meaning that a strong relationship between cognitive presence and social presence was
perceived by participants and that increases in one presence are associated with increases in
the other presence.
The partial correlation analysis controlling for teaching presence for this set of
variables yielded a large and positive partial correlation as well between cognitive presence
and social presence, pr(123)=.61, p<.000. This indicates a shared variance of 37%; that is,
when the effect of teaching presence is controlled for, it suggests only a limited effect on
the relationship between cognitive presence and social presence.
121
Relationship between Cognitive Presence and Teaching Presence
As shown in table 19, cognitive presence and teaching presence were also strongly
correlated, r
s
(123)= .70, p<.000, with a 49% of shared variance. As perceived by
participants in this research, cognitive presence increases as teaching presence increases
and vice versa. In order to explore this relationship further, a partial correlation was run
controlling for social presence. It was found to be positive and moderate pr(123)=.57,
p<.000, revealing a 32% of shared variance. Thus, this result suggests that social presence
has a limited effect on the relationship between cognitive presence and teaching presence.
Relationship between Social Presence and Teaching Presence
Bivariate correlation analysis of social presence and teaching presence yielded a
moderate positive correlation rs(123)= .50, p<.000. This result indicates a 24% shared
variance between teaching and social presence; that is, increases in social presence have a
limited effect over increases in teaching presence. The partial correlation analysis
controlling for cognitive presence conducted revealed a negative non-significant partial
correlation pr (123) = -.015, p<.87. The shared variance was of 0.02% suggesting that when
cognitive presence is controlled for, the social presence- teaching presence association is
extremely weak; that is, cognitive presence is central for teaching presence to relate to
social presence and vice versa.
Summary of the relationship between CoI presences in OLTE courses
Considering the CoI items with the highest mean (The instructor clearly
communicated important due dates / time frames for learning activities) and the lowest
mean (Online or web-based communication is an excellent medium for social interaction),
122
it can be a