ArticlePDF Available

Academic performance, course completion rates, and student perception of the quality and frequency of interaction in a virtual high school

Authors:

Abstract and Figures

This study examined the relationship between students’ perceptions of teacher–student interaction and academic performance at an asynchronous, self-paced, statewide virtual high school. Academic performance was measured by grade awarded and course completion. There were 2269 students who responded to an 18-item survey designed to measure student perceptions on the quality and frequency of teacher–student interaction. Quality of interaction was subdivided into three constructs representing feedback, procedural, and social interaction. A confirmatory factor analysis helped to establish the fit of the statistical model for teacher–student interaction. Hierarchical logistical regression indicates that an increase in the quality and frequency of interaction resulted in an increased likelihood of course completion but had minimal influence on grade awarded. The estimated effect for quality and frequency composite items on completion was .83 and .56 respectively. Low practical significance of student–teacher interaction on grade awarded may be the result of mastery-based teaching approaches that skew grades for the completers toward the high end.
Content may be subject to copyright.
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
Running Head: LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
!
This is a prepublication draft of an article to appear in:
Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic
performance, course completion rates, and student perception of the quality
and frequency of interaction in a virtual high school. Distance Education, 34(1),
64–83. doi:10.1080/01587919.2013.770430
Academic performance, course completion rates, and student perception of the
quality and frequency of interaction in a virtual high school
Abigail Hawkins
Senior Manager, Learning Design and Development - Investools with TD Ameritrade
Charles R. Graham
Instructional Psychology & Technology, Brigham Young University, Provo, UT, USA
Richard R. Sudweeks
Instructional Psychology & Technology, Brigham Young University, Provo, UT, USA
Michael K. Barbour
Instructional Technology, Wayne State University, Detroit, MI, USA
!
!
Corresponding Author:
Charles R. Graham
301 MCKB BYU
Provo, UT 84602
charles.graham@byu.edu
801-422-4110
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
1!
Abstract
This study examined the relationship between students’ perceptions of teacher-student
interaction and academic performance at an asynchronous, self-paced, statewide virtual
high school. Academic performance was measured by grade awarded and course
completion. There were 2,269 students who responded to an 18-item survey designed to
measure student perceptions on the quality and frequency of teacher-student interaction.
Quality of interaction was subdivided into three constructs representing feedback,
procedural, and social interaction. A confirmatory factor analysis helped to establish the fit
of the statistical model for teacher-student interaction. Hierarchical logistical regression
indicated that an increase in the quality and frequency of interaction resulted in an
increased likelihood of course completion but had minimal influence on grade awarded.
The estimated effect for quality and frequency composite items on completion was .83 and
.56 respectively. Low practical significance of student-teacher interaction on graded
awarded may be the result of mastery-based teaching approaches that skew grades for the
completers towards the high end. The results of this study are discussed, as well as
implications for practitioners and researchers.
Keywords: virtual schooling, online k-12 learning, completion rates, interaction
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
2!
Within the last sixteen years virtual schooling has spread across 48 states and the District
of Columbia (Watson, Murin, Vashaw, Gemin, & Rapp, 2010). In 2000, there were an estimated
40,000 to 50,000 K-12 students enrolled in online programs (Clark, 2000). More recent estimates
put the figure at 1,030,000 K-12 students (Picciano & Seaman, 2009), a twenty-fold increase in
less than a decade. The student population among virtual schools is becoming increasingly
diverse. Historically, virtual school students were described as highly motivated,
honors/advanced, independent learners who were more likely to attend four-year college than
their face-to-face counterparts (Barbour, 2009). Today, a broader range of students is choosing
virtual schooling for the purpose of credit recovery or to fulfill a graduation requirement
(Watson, Gemin, & Ryan, 2008; Watson et al., 2010). The virtual school phenomenon is even
extending down into the elementary grades with more than 26 states offering at least some full-
or supplemental-online learning opportunities for grades K-5 (Watson et al., 2010).
Attrition from online courses, particularly at the K-12 level, has been a significant
challenge (Simpson, 2004; Smith, Clark, & Blomeyer, 2005; Zucker & Kozma, 2003). Though
reasons for student persistence in an online environment are complex and multifaceted (Willging
& Johnson, 2004), interaction may be a key factor to student success. In contrast to a sizable
amount of interaction research in post-secondary, there has been a paucity of empirical research
into online learning at the K-12 level (Barbour & Reeves, 2009). In a review of literature, Rice
(2006) found “very little research examining relationships between K-12 interaction that directly
related to student performance, satisfaction, and retention in distance education contexts” (p.
439). Clearly, more research is needed to examine the relationship between student success and
interaction in virtual schooling contexts to identify teacher best practices to improve student
retention and academic performance.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
3!
The purpose of this study was to examine the relationship between students’ perceptions
of teacher-student interaction and academic performance in a supplemental, asynchronous, self-
paced, statewide virtual high school. We begin this article with an exploration of the factors
related to academic success in K-12 and post-secondary online learning environments. We then
describe the analysis of student perceptions of teacher-student interaction and academic
performance using Pearson’s Product Moment Correlation Coefficient and Hierarchical Linear
Modeling. Finally, we conclude by discussing two changes institutions can make to improve
teacher-student interaction, along with two avenues of potential research.
Literature Review
Historically, distance education has faced higher dropout and failure rates compared to
traditional classrooms (Roblyer, Davis, Mills, Marshall, & Pape, 2008). Furthermore, it is
believed that attrition rates are higher for virtual school settings compared to post-secondary
online learning programs (Smith et al., 2005). While no official attrition statistics exists for
virtual schools by state or school type, individual evaluations of some K-12 online learning
programs indicate that attrition ranges broadly from 10 % up to 70 % (Roblyer & Davis, 2008).
For example in its first year of operation, Illinois Virtual High School had only a 53 %
completion rate (Clark, Lewis, Oyer, & Schreiber, 2002), while the Alberta Distance Learning
Center asynchronous courses had only a 47 % completion rate (Elluminate Inc., 2006). Problems
of attrition may be further masked by variations in how virtual schools calculate a successful
completion and the length of trial periods when students are not considered to be “officially”
enrolled (Hawkins & Barbour, 2010)
Factors Influencing Student Performance in K-12 Online Learning
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
4!
Several studies have examined the relationship between learner characteristics and
student performance. Roblyer and Marshall (2003) identified learner characteristics predictive of
high school student online course completion using the Educational Success Prediction
Instrument (ESPRI), which predicted passing students with 100 % confidence and failing with 95
% confidence with 135 students across 13 virtual high schools. Additionally, she found no
differences on personal characteristics (e.g., age, grade level, or gender). Successful students
scored higher in self-efficacy, individual initiative, organizational skills, access to technology,
and spent less time working outside of school. Roblyer’s (2008) replication study, using a larger
sample (n=4,100), also found successful students scored higher on technology access, self-
efficacy, and organization. Additionally, past performance (i.e., GPA) was a strong predictor of
success. However, samples in both studies were selective (e.g., the majority were Caucasian,
drawn from rural/suburban populations, had a historically high pass rate, and had time built into
their schedule for their online coursework).
Teacher support in the online learning environment has also been tied to online course
completion. However, the role of the teacher in the virtual school environment has expanded
from what we typically understand the traditional classroom teacher’s role to be. Three critical
roles have emerged: virtual school designer, teacher, and site facilitator (Davis, 2007,
November). The role of virtual school site facilitator (i.e., the onsite local mentor and advocate),
though not as thoroughly researched as the other two roles, has been tied to program and student
success (Davis & Roblyer, 2005). In their evaluation of a statewide online program, evaluators
found “facilitators that are directly working with students day-by-day are key to the success of
the program” (Roblyer, Freeman, Stabler, & Schneidmiller, 2007, p. 11). Though rarely
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
5!
documented, Mulcahy (2002) reported school-based teachers and principals often voluntarily
provided technical and supervisory support, along with significant academic tutoring.
Factors Influencing Student Performance in Post-Secondary Online Learning!
In contrast to the limited research at the K-12 level, there is a more substantial body of
literature focused on online learning in adult populations. Student success online was related to
several factors. While demographic characteristics (i.e., gender [Bernard, Brauer, Abrami, &
Surkes, 2004; Levy, 2007; Willging & Johnson, 2004], ethnicity [Dupin-Bryant, 2004],
occupation [Willging & Johnson, 2004], and age [Levy, 2007; Willging & Johnson, 2004]) were
not predictive of completion rates; prior academic success (Bernard et al., 2004; Dupin-Bryant,
2004; Wang & Newlin, 2000; Wojeiechowski & Palmer, 2005) and success in past online
courses (Dupin-Bryant, 2004) were predictive. Additionally, affective traits (e.g., student’s locus
of control [Willging & Johnson, 2004], motivation level [Jamison, 2003], and independent
learning styles [Diaz, 2000]) were predictive of success online. Factors external to the learner
such as pre-course orientation sessions (Wojeiechowski & Palmer, 2005), study skills (Osborn,
2001), and strong computer skills (Dupin-Bryant, 2004) have also been associated with student
success. Additionally, studies have identified interaction as a key factor for online student
success (Thurmond & Wambach, 2004, January; Wallace, 2003).
Interaction and Student Performance in Post-Secondary Online Learning
Scholars have identified five types of interaction in online distance education: learner-
instructor, learner-learner, learner-content, learner-interface, and vicarious interaction (Hillman,
Willis, & Gunawardena, 1994; Moore, 1989; Sutton, 2001). Post-secondary interaction research
has primarily focused the quality and/or frequency of interaction in relation to three outcome
variables: satisfaction, perceived learning and academic achievement (Swan, 2001). High quality
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
6!
and levels of interaction have been associated with increased learner satisfaction (Jung, Choi,
Lim, & Leem, 2002; Picciano, 2002; Russo & Benson, 2005; Shea, Fredericksen, Pickett, Pelz,
& Swan, 2001; Swan, 2001), perceived learning (Picciano, 2002; Rovai & Barnum, 2003; Stein,
Wanstreet, Calvin, Overtoom, & Wheaton, 2005), and academic achievement (Jung et al., 2002;
Picciano, 2002). Isolation and disconnection had the largest influence on students’ decision to
drop out and disengage (Bocchi, Eastman, & Swift, 2004; Willging & Johnson, 2004).
Different taxonomies have emerged to examine teacher-student interaction. Based on
distance education theories, Heinemann (2005) identified three major types of teacher
interactions: intellectual, organizational, and social. While there is substantial research into
interaction in post-secondary online learning (as evidenced above), researchers caution against
generalizing these findings to adolescents who often lack the ability to regulate their own
learning (Barbour & Reeves, 2009; Cavanaugh, Gillan, Kromrey, Hess, & Blomeyer, 2004; Rice,
2006). Typical of research in K-12 online learning, only a handful of studies have examined
these three types of interaction.
Interaction and Student Performance in K-12 Online Learning
Research on teacher-student intellectual/instructional interactions has emphasized the
importance of feedback on student satisfaction and persistence. Weiner’s (2003) qualitative study
of a cyber charter school found limited teacher-student interaction a major concern. Students
reported the lack of timely feedback was frustrating, impeded learning, and led to feeling
“ignored, lonely, or lost” (p. 49). Furthermore, researchers who identified virtual schooling best
practices also emphasized the importance of prompt feedback on student learning, progress, and
connectedness (DiPietro, Ferdig, Black, & Preston, 2008; Ferdig, Cavanaugh, Dipietro, Black, &
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
7!
Dawson, 2009). However, these studies were based on student/teacher perceptions or standards
analysis and did not tie the impact of interaction on actual student performance.
In relation to procedural interactions, Roblyer (2006), who identified best practices from
three successful virtual schools, found policies and practices that required teachers to track
student progress and proactively reach out to inactive students via email, telephone calls, and
monthly student and parental consultations. While these practices appear promising within these
three specific virtual schools, these practices were also not based on systematic research showing
improved academic performance.
Social interactions in the form of self-disclosure, humor, and encouragement were also
important to virtual high school student motivation and progress (DiPietro et al., 2008; Mulcahy,
Dibbon, & Norberg, 2008; Nippard, 2005; Roblyer, 2006; Weiner, 2003). Mulcahy et al. (2008)
found struggling students missed social interactions and felt distant from their online teachers,
preferring to seek help from their face-to-face teachers. Additionally, Nippard and Murphy
(2007) found a disconnect between the mediums that teachers and students used to create social
exchanges, likely making the positive effects difficult to achieve. However, both studies were
conducted with largely rural samples, which typically display a stronger sense of community
(Kannapel & DeYoung, 1999). While these studies provide insight into the nature of interactions
in K-12 virtual schooling, few connected these forms of interactions to student persistence or
academic performance.
Methodology
The purpose of this study was to examine the relationship between teacher-student
interaction and academic performance. This led to the following research question:
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
8!
What is the relationship between students’ perceptions of the quality and frequency of
teacher-student interaction and online course completion and academic performance?
Quality measures of teacher-student interaction were further sub-divided into the three categories
of feedback, procedural, and social interactions based on Heinemann (2005). We hypothesized
that both quality and frequency factors would be positively correlated with course completion
and academic performance. We also hypothesized that of the three different types of interaction
instructional interactions would likely have the greatest positive correlation with completion and
performance outcomes. We used correlation and hierarchical linear modeling research methods
from survey data to address this research question.
Context
Utah’s Electronic High School (EHS), a primarily supplemental, statewide, self-paced
asynchronous virtual school, was the research setting. With 46,089 student enrollments from
February 1, 2008 to January 31, 2009, the school operated on an open-entry/exit enrollment
model where students proceeded at their own pace with little, if any, student-to-student
interaction. Students had to submit a graded assignment every 30 days, remain active in the
course, and complete the course within a six-month time period. If they violated either of these
policies, they were automatically dropped from the course but could immediately re-enroll at any
time at no cost to the student or penalty on their academic record.
EHS teachers developed the primarily text-based curriculum using Utah’s State Core
Curriculum Standards. At the time of the study, there were 66 unique classes taught by 1 full-
and 76 part-time teachers. This is quite common for supplemental K-12 online learning
programs, as few employ full-time personnel. According to the 2008 audit, approximately 64 %
of teachers were also employed as teachers somewhere other than EHS (Center for Educational
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
9!
Leadership and Technology, 2008). Of these teachers, 90% were also employed full time
elsewhere. All respondents were certified to teach in their content area. Nearly sixty percent of
teachers taught only one class at EHS, while 28 % taught two classes, and 13 % taught three or
more classes. The average student-to-teacher ratio was 233:1, but ranged from 2 students to
1,726 students per section. While this may seem like a high student-to-teacher ratio, it is not
uncommon for online K-12 teachers to have student-to-teacher ratios two to four times that of a
traditional classroom teacher (and open enrollment/exit programs – like EHS – are also even
higher than the normal K-12 online learning range).
Overall, 34 % of students enrolled from February 1, 2008 to January 31, 2009 completed
their courses. Completion rates by individual disciplines ranged widely from 10.27 % to 62 %,
with a large degree of variance. Table 1 indicates the mean completion rates by discipline for
students enrolled from February 1, 2008 to January 31, 2009.
[Insert Table 1 about here]
The high completion rates for the courses in financial literacy, health and physical education,
driver’s education may be due to the fact that they were required for graduation.
Participants
Participants were invited from a pool of 67,759 students who were enrolled from
February 1, 2008 through September 29, 2009 and met the following criteria:
! Submitted a request to sit for their proctored final exam,
! Earned a grade,
! Did not earn a grade, but had enrolled in the class for more than six months as of May 1
st
,
2009, and
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
10!
! Did not earn a grade, yet enrolled in class for three months but less than six months or
completed at least 50 % of the class work.
A total of 2,269 surveys with full or partial data were received. This represented a 3.34 %
response rate. While the response rate is quite low, we did not have control over who had access
to the surveys. There is some evidence to indicate that as much as 40% of enrollments at EHS
represent students who have not started work on a course (Hawkins & Graham, 2010). he
students who have not started or have done very little work on a course are likely under-
represented in our survey because they have little motivation to respond to a survey about a
course into which they have not invested much energy.
Over two-thirds of respondents were females, and 84.4 % were Caucasian. Ninety-five
percent of respondents identified English as their native language. Overall, it was a fairly
homogenous group of respondents with a significant number of Caucasian and female
respondents. Other virtual schooling studies have also found an overrepresentation of dominant
cultures within the sample (Black, Thompson, Askenazi, Ferdig, & Kisker, 2010). Since EHS
does not collect demographic data on incoming students, it was not possible to compare the
respondent sample to the larger population.
Based on the survey data, 75.1 % of respondents (n=1705) reported having successfully
completed the course (i.e., a grade was awarded), while 24.9 % of respondents (n=564) indicated
they did not complete the course. The sample was skewed towards successful students, as
evidenced by the fact that overall completion rates for 2008-2009 were only 34 %. However, this
is typical of K-12 online learning in general, and supplemental programs in particular; which
often experiences a bimodal distribution of students skewed towards the higher achieving
students (Barbour, 2011). Interestingly, the grade distribution from A through D- for participants
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
11!
was reflective of the larger population (although this does not speak to students overall ability or
grade point average – as it is common for students to score 10% or more less in their online
courses as compared to their face-to-face performance [Barbour & Hill, 2011]). Possible reasons
for the response bias include motivational factors (i.e., if students didn’t complete the course, one
would not expect them to take time to complete the survey), how the survey was administered
(i.e., an email sent out to all students as well as an integrated step in the course completion
process just prior to signing up for a proctored exam), and respondents’ original motive for
enrolling at EHS (i.e., original credit or accelerated graduation). In addition, the nature of
volunteer subjects may also account for the higher completion rates, as volunteers generally
show higher levels of achievement, motivation and intellect than those who do not participant in
non-mandatory assessments (Rosenthal & Rosnow, 1975).
Instrument Development
Our choice to use a survey to collect data regarding student-teacher interaction was
rooted in the fact that it was the only practical way to access the EHS population. Additionally, it
should be noted that the nature of any K-12 online learning environment in the United States
involves multiple methods of interaction (Cavanaugh, Barbour, Brown, Diamond, Lowes,
Powell, Rose, Scheick, Scribner, & Van der Molan, 2009). While the learning management
system might capture the interaction that occurs within that environment (Dickson, 2005),
interaction also occurs through personal e-mail, telephone calls, text messaging, Skype and other
VOIP, social networks, and in person (Barbour & Hill, 2011; Barbour & Plough, 2009, 2012).
Unless researchers had teachers and students log each and every interaction, as it occurred, a
survey presented the best method to collect data related to frequency of interaction.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
12!
We first sought to find an existing survey instrument that could measure quality and
frequency of interaction in a virtual school setting. We reviewed 15 quantitative surveys where at
least one scale measured teacher involvement, interaction, or support and found that none were a
good fit for the study context (i.e., self-paced, asynchronous online secondary courses with little
or no interaction with peers). Based on the literature search a conceptual model of interaction
introduced by Heinemann (2005) was then chosen to frame the study. Quality interaction items
were designed to measure the three interaction types Heinemann identified:
1. Intellectual/Instructional interactions: exchanges related to academic feedback.
2. Organizational/Procedural interactions: exchanges related to class logistics,
procedures, and processes.
3. Social interactions: exchanges related to support, encouragement, and connectedness.
Thirteen items were developed to measure students’ perceptions of the three quality constructs of
student-teacher interactions and five items to measure students’ perceptions of the frequency of
interactions. The instrument was piloted on 10 youth, with feedback used to improve the survey
prior to its broader administration.
To estimate the construct validity of the instrument, we used confirmatory factor analysis
(CFA). The purpose of CFA is to determine whether items purported to measure a particular
construct shared a significant portion of common variance and load to the intended construct
(Hair, Black, Babin, Anderson, & Tatham, 2006). CFA can be used to assess the degree of
convergent and discriminant validity. Convergent validity refers to the extent to which the items
purported to assess a certain construct (i.e., frequency, social, feedback, etc.) share common
variance and load to the same construct. In contrast, discriminant validity refers to the extent to
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
13!
which constructs are discrete factors with little overlap. Constructs that have weak correlations
between each other indicate a high level of divergent validity.
CFA was performed using Mplus version 5.21. Preliminary CFA analyses were done on
four proposed single factor interaction scales (a) frequency, (b) feedback, (c) procedural, and (d)
social factor scales. Secondary analysis was conducted on the 13 quality items, where CFA was
used to determine whether a first-order factor structure (i.e., interaction quality composite score)
or a three-separate factor structure (i.e., instructional, procedural, social interaction composite
scores) best fit the data. Reliability was estimated using Cronbach’s alpha coefficient.
Data Collection
The study items were incorporated into an existing EHS class evaluation survey and
administered in two ways. In February 2010, the principal emailed an invitation and survey link
to 46,089 participants enrolled in courses from February 1, 2008 to January 31, 2009, with a
reminders sent to non-respondents four weeks later, and the survey was closed following another
four-week period. The survey was emailed or made available to all enrolled students regardless
of how much, if any, coursework they had completed. The second method of administration was
its integration into the course completion process as a part of the class evaluation that students
completed after finishing their coursework, but prior to taking the proctored exam. A total of
21,670 students had access to this route of completing the survey via its integration into the
course.
The survey administration was part of the EHS course evaluation and they did not
provide us with demographic data on all 67,759 students who had access to the survey only with
data on survey respondents. Student responses were mapped back to EHS’ student performance
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
14!
database, linking responses to actual course grade, date of enrollment, date of completion (if
applicable), course name, and instructor. Data were then de-identified.
Data Analysis
Three procedures were used to examine the relationship between quality and frequency
interaction variables with academic performance, as measured by grade awarded and course
completion status: Pearson’s product-moment correlation coefficient, hierarchical linear
modeling (HLM), and hierarchical logistical regression. HLM was used as a secondary analysis
used to examine the dependent variable grade awarded. Since the second dependent variable,
completion, was dichotomous, we conducted hierarchical logistical regression using Mplus. Both
regression statistics were conducted to estimate the degree of dependence among students with
the same teacher and the consequences of violating the assumption of independence.
Results
We begin this section with a discussion of the confirmatory factor analysis to determine
whether items measured intended constructs and shared a significant portion of common
variance. This is followed by a presentation of the descriptive statistics to provide the reader with
an understanding of the participants and response counts. This is concluded by a discussion of
the results of the Pearson’s product-moment correlation coefficient and hierarchical linear
modeling, which was designed to tell us the strength of the relationship between students’
perceptions of the quality and frequency of interaction and academic performance.
Confirmatory Factor Analysis
Model fit was determined using Chi-Square Test for Difference Testing, Comparative Fit
Index (CFI), Tucker-Lewis Index (TLI), and Root Mean Square Error of Approximation
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
15!
(RMSEA). To determine reasonably good fit, we were guided by CFI and TLI scores close to .95
or greater and RMSEA scores close to .08 or below (Brown, 2006). Table 2 indicates the fit
statistics on the preliminary analysis of the four proposed first-order factor scales.
[Insert Table 2 about here]
For all elements of the model, preliminary analysis found the model fit indices were within the
acceptable range for CFI and TLI and not RMSEA, indicating the items loaded to the intended
factors. Secondary analysis using CFA on the quality items was used to determine whether
quality items loaded to a single quality construct or loaded best to three separate constructs:
feedback, procedural, and social. Table 3 depicts the fit statistics for quality items loading to a
single-order factor structure, second-order factor structure, and three separate factor structure
(i.e., feedback, procedural, and social).
[Insert Table 3 about here]
Again, the fit statistics were within the acceptable range for CFI and TLI as outlined by Brown
(2006). It should be noted that the fit statistics were the same for the second-order and three-
separate order factor structures. Thus, to determine definitively which model for quality factor
structures was the best fit, we used the chi-square test (CHI) for difference testing comparing the
first-order and three-separate factor structure models. This resulted in CHI values of 471.202, df
= 2, and p <.00001, indicating that the three-separate factor model fit the data significantly better
than the first-order factor. Figure 1 depicts the path diagram for the proposed three-separate
factor structure model along with the correlations between constructs and items.
[Insert Figure 1 about here]
The high factor loadings indicated that items loaded to the intended constructs and had high
convergent validity. However, the high correlations between feedback, procedural, and social
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
16!
constructs indicated low discriminant validity. Practically speaking, participants responded
similarly on all three constructs implying the constructs these items measured were
distinguishable only theoretically and not statistically.
The high correlations between each construct and the fact that they pointed to a common
theoretical construct (i.e., quality), led us to test a second-order factor structure with feedback,
procedural, and social factors linked to an overarching theoretical construct we refer to as
interaction quality (see Figure 2).
[Insert Figure 2 about here]
Once more, this model demonstrated high correlations between the first order factor (i.e.,
quality) and second order factors (i.e., feedback, procedural, social). These correlations indicated
that the second order factors were part of a larger, theoretical construct we refer to as interaction
quality.
Reliability
Cronbach’s alpha for the interaction quality and quantity composites were .94 and .85
respectively. The reliability estimates for the three quality factors were .86 for feedback, .91 for
procedural, and .92 for social. These were strong reliability levels as they fell well above the
acceptable minimum value of alpha at .70 (Hair et al., 2006).
Descriptive Statistics
The mean composite scores for interaction quality and frequency items were 3.2 (on a 4-
point scale) and 2.43 (on a 5-point scale) respectively. Table 4 provides the mean scores for the
factors analyzed in the survey.
[Insert Table 4 about here]
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
17!
Scores for quality of interaction were higher than frequency of interaction, which may have been
a reflection of EHS’ independent study model.
There was a fairly normal distribution of scores for the quality composite factor. Quality
items were on four-point scale where 1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly
agree. The distribution of scores for the frequency composite of items collapsed into a five-point
scale where 1 = hardly ever, 2 = once a month, 3 = twice a month, 4 = once a week, and 5 =
twice a week. Unlike that of the quality variable, the scores were negatively skewed. Frequencies
of interaction varied based on the type of interaction. Table 5 indicates student reported
frequencies of interaction for feedback, procedural, and social interactions. While students
reported low frequencies of interaction in all three areas, students indicated that they interacted
more frequently over procedural issues than social or instructional matters.
[Insert Table 5 about here]
Initial Data Analysis: Pearson’s Product-Moment Correlation Coefficient
Mean quality and frequency composite scores were correlated by academic performance
as measured by grade awarded and course completion. Letter grades were re-coded on a scale of
1 to 13 with a 1 as an A+, 2 as an A, 3 as an A-, etc. Table 6 indicates the correlation
coefficients, N, statistical significance, and the measure of the strength of the association (r
2
).
[Insert Table 6 about here]
Correlations between perceived quality and frequency factors by grade awarded and course
completion were very weak to negligible at both the composite and factor levels. Quality had
higher correlations than Frequency. However, quality factors (by grade awarded and by
completion) only explained 2.1 % and 7.2 % of the variance respectively. Again, this was a very
weak correlation.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
18!
Further analysis of correlations at the item level showed items correlated to completion
had higher coefficient values than items correlated to grade awarded, though all items were .3 or
lower. We thought that there might be differences in how lower-performing students (i.e., credit
recovery) versus higher performing students (i.e., original credit) would perceive teacher-student
interactions. However, correlations comparing these student types again found very weak to
negligible relationships between academic performance and quality/frequency variables. In
summary, correlations between the variables were almost non-existent. These findings were
unexpected and led us to either (1) question their hypothesis that quality and quantity of
interactions would correlate with student performance in this context or (2) consider that perhaps
respondents were not independent of one another, a primary assumption for the Pearson’s
product-moment correlation coefficient.
Secondary Data Analysis: Hierarchical Linear Modeling and Hierarchical Logistical
Regression
The unexpected results from the correlational analysis led us to conduct hierarchical
linear modeling to identify the relationship between variables, believing that there may be
dependencies within groups of students who had the same teacher. The primary question in
conducting HLM and hierarchical logistical regression was to determine if students with the
same teacher were responding similarly in terms of the nature and frequency of interaction.
Table 7 displays the mean, standard deviation, and intraclass correlation for the dependent
variables: grade awarded and completion.
[Insert Table 7 about here]
The mean for grade awarded was 10.04 which translated to letter grade was an A-. Completion
mean is reported as a proportion with 74 % of respondents successfully completing the course.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
19!
Intraclass correlations were used to measure the degree of interdependence among observations
(i.e. student respondents) with the same teacher. The high intraclass correlation values were
evidence that there were dependencies in student responses grouped around the teacher
(Raudenbush & Bryk, 2001). These dependencies would explain the weak to negligible
correlation values from the initial analysis using Pearson’s product-moment correlation
coefficient, which masked the actual relationship between interaction type and academic
performance.
To examine interaction and grade awarded, we used HLM. Table 8 displays the estimates
of fixed effects. In this analysis grade awarded is the dependent variable and quality and
frequency of interaction are independent variables.
[Insert Table 8 about here]
In all instances the perceived quality and frequency of interaction associated with the
grade awarded were significant at p=.05 level. The estimate effect column in the table indicates
that for every 1 unit increase in the perception of the four point quality of interaction scale, there
was a .27 unit increase in the 12-point grade (e.g., B- to B is one unit difference. Note EHS does
not award failing grades, thus a 12-point grade scale). In other words, a four unit difference in
perception of interaction quality (i.e., the full scale) only accounted for a one unit grade
difference. The practical significance of this finding is minimal.
To examine interaction and completion, we used hierarchical logistical regression, as the
dependent variable was dichotomous in nature. Table 9 displays the estimates of fixed effects. In
this analysis completion is the dependent variable and quality and frequency of interaction are
independent variables.
[Insert Table 9 about here]
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
20!
The perceived quality and frequency of interaction associated with the completion were
significant at p=.05 level. For completers and non-completers there were significant differences
in scores on the quality composite construct. The scores on quality composite by completion had
an estimate effect at 0.83. Thus for every one unit increase on the four-point interaction scale for
quality items the log odds of completing the course increased by .83. Additionally, for every one
unit increase on the five-point frequency scale, the log odds of completing the course increased
.56. Thus a one unit difference in perception of quality of interaction is nearly the difference
between a non-completer and completer. Additionally a two unit difference on the frequency
scale differentiates non-completers from completers.
Discussion
There were three main findings resulting from this study: (1) methodological insights, (2)
key differences between completers and non-completers on quality and frequency of interaction,
and (3) little practical significance on differences between high and low performing students’
perceptions on quality and frequency of interaction. There were methodological understandings
that arose from this study that may be just as important as the findings related to virtual
schooling. Like many other studies measuring the relationship between interaction and academic
performance or perceived satisfaction (e.g., Fredericksen, Pickett, Shea, Pelz, & Swan, 2000;
Restauri, 2006; Swan, 2001), we used a correlation statistic to measure this relationship and
found a weak relationship. Since respondents who shared a common teacher may have answered
similarly, the assumption of independence was violated—a major premise of correlation
coefficients. Consequently, we conducted HLM and hierarchical logistical regression, resulting
in more meaningful insights on student perceptions of interaction in relation to academic
performance. Future research that draws participants from across multiple courses and instructors
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
21!
should account for the violation of response independence and nest students by teacher using
hierarchical linear modeling. Another methodological insight future researchers should heed that
resulted from the CFA was significantly high correlations between feedback, procedural, and
social factors, which meant that students generally responded similarly to the procedure,
feedback, and social interaction questions. Students split by grade responded differently on the
three constructs. However, since there were relatively few C and D students in the population
one could still get the interaction effect at the end without significantly changing the correlations
when examining the entire population.
Second, there were unique findings between completer and non-completer responses.
First, the results of this study indicate that the perceived quality and quantity of interaction
mattered to student completion rates. Higher quality interaction and more frequent interaction
scores increased the log odds of completion significantly. In other words, students who
completed the course perceived greater interaction and quality of interaction than non
completers. This finding supports the higher education research on the importance of interaction
to remain engaged (i.e., Bocchi, Eastman, & Swift, 2004; Willging & Johnson, 2004) and gives
statistical backing to the handful of qualitative studies pointing to the importance of interaction
in virtual schooling environments (i.e., DiPietro et al., 2008; Ferdig et al, 2009; Mulcahy,
Dibbon, & Norberg, 2008; Nippard, 2005; Roblyer, 2006; Weiner, 2003).
Finally, in contrast to interaction impacting completion, there was no practically
significant effect of interaction on the grade awarded. There were several possible reasons for
this outcome. First the results may be due to the fact that there was little variation in grades
awarded among respondents. The mean grade was an A-, with 76 % of respondents receiving an
A compared to only 0.6 % receiving a D and 4.1 % receiving a C. Thus, it would be difficult to
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
22!
detect differences in perceptions on such a small number of respondents. Another possible reason
for the absence of an effect was that students who completed the course were satisfied solely
with completion irrespective of the grade awarded, because, in many cases completion with any
non-failing grade was all that was required for graduation. For the bulk of students completion
may have been more of an important issue than the grade awarded. Teacher interaction impacted
completion but not necessarily grade awarded at a significant level. Finally, it was common
practice for teachers at EHS to allow students to resubmit work for an improved grade. One
could speculate that this increased interaction moved some students from non-completion to
passing the course and they perceived high interaction as a result of multiple resubmissions.
There were several methodological limitations to this study. First, the survey was
administered via email in March 2010 to 46,089 participants who were enrolled from February 1,
2008 to January 31, 2009 (i.e., the remaining 21,670 students were given the opportunity to
complete the survey as a part of the end of course procedures). Researchers indicate that recall
bias increases when there is a delay between the time of the experience and recall of said
experience (Stynes & White, 2006). The gap in time from when students experienced the course
and when they recalled their interaction experiences was problematic. Second, the low response
rate (i.e. 3.34 %) indicated a likelihood of non-response bias or the likelihood that respondents
and non-respondents differ significantly. There is evidence to suggest that as much as 40% of
enrolled EHS students have not started work on a course. It is likely that students who enrolled
but never started work on a course or who have done very little work on a course are
significantly under represented in the results.
According to Rosenthal and Rosnow’s (1975) review of literature on studies relying on
volunteers as subjects, the authors found that volunteers were “likely to show higher levels of
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
23!
achievement than their less achievement-motivated colleagues” (p. 40). While they found 15
studies showing no relationship between volunteering and intelligence, they found 20 studies that
indicated volunteers were significantly more intelligent than non-respondents. Essentially, those
who volunteered to complete a non-mandatory survey may not be representative of the larger
population. There is evidence of non-response bias in the fact that 75 % of those who responded
to the survey for this study had completed their online course compared to only 31 % of the
general population at EHS during the time of the study. Due to the fact that participants were
minors, dispersed geographically, and the difficulty of obtaining IRB approval through the State
and Brigham Young University, beyond a follow-up survey, we did not build in other means to
reach out to the non-respondents.
Conclusions and Implications
The quality and frequency of interaction had a significant impact on student completion
but not on grade awarded. Increased levels of the quality and frequency of interaction resulted in
increased student completion. However, there was no difference on grade awarded, a result likely
due to the limited variation in grades awarded, attitude towards completion, and resubmission
practices at EHS.
Based on the results of this study, along with the limitations of EHS’ instructional model,
there are two implications for practitioners in this and similar environments. First, interaction
matters in terms of both the quality and frequency of interaction. Students who completed the
course perceived their interactions with teachers more positively than their non-completer.
Teachers should continue to maintain a high quality and frequency of interaction with students,
particularly those at risk of dropping out. One way to achieve this is to place a greater emphasis
on interaction at the beginning of a course—a time when all students are most likely to be
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
24!
engaged in the course. Teachers should interact with students on multiple occasions in the first
day or so of enrollment because this may be when they are most primed to engage in the course.
These interactions would include: (a) an introduction to the learning management system
navigation, along with how to participate in the course and interact with the content (i.e.,
procedural); (b) general information about the course content and learning goals (i.e.,
instructional); and (c) an introduction to the teacher as a person and warm welcome to the class
(i.e., social). Second, teachers should take proactive measures to reach out to students regardless
of their progress in the course. The increased interaction may be enough to move students from
the non-completion status to completion status.
In addition to these implications for practice, there are also two areas of future inquiry.
First, perceptions of behaviors are often very different from actual behaviors (Fishman, Marx,
Best, & Tal, 2003). Mining data from learning management systems, which contain all teacher-
student interactions (i.e., teacher-student emails, student question postings, assignment
submission and feedback, etc.), would be a simple way to determine the relationship between
frequency of interaction and academic performance (Schneider, Krajcik, & Blumenfeld, 2005).
Furthermore, content analysis of such interactions with a clear coding system could also reveal
the quality of the interaction in a less subjective manner. Second, the fact that the overall student
perceptions of the quality and frequency of interaction were minimal may be due to the nature of
the specific context where the study occurred (i.e., a self-paced, asynchronous, open-entry
virtual school with large class sizes, failing grades are not awarded). Due to the wide variation in
virtual school types, similar studies with other virtual schools may yield different, and
potentially more positive, results for both grade awarded and completion.
References
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
25!
Barbour, M. K. (2007). What are they doing and how are they doing it: Rural student
experiences in virtual schooling. (Unpublished doctoral dissertation). University of Georgia,
Athens , GA. Retrieved from http://www.michaelbarbour.com/research/pubs/phd_2007.pdf
Barbour, M. K. (2009). Today's student and virtual schooling: The reality, the challenges, the
promise. Journal of Distance Learning, 13(1), 5-25.
Barbour, M. K. (2011). The promise and the reality: Exploring virtual schooling in rural
jurisdictions. Education in Rural Australia, 21(1), 1-20.
Barbour, M. K., & Hill, J. R. (2011). What are they doing and how are they doing it? Rural
student experiences in virtual schooling. Journal of Distance Education, 25(1). Retrieved
from http://www.jofde.ca/index.php/jde/article/view/725
Barbour, M. K. & Plough, C. (2009). Social networking in cyberschooling: Helping to make
online learning less isolating. Tech Trends, 53(4), 56-60.
Barbour, M. K. & Plough, C. (2012). Putting the social into online learning: Social networking in
a cyber school. International Review of Research in Open and Distance Learning, 13(3), 1-
18. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1154/2148
Barbour, M. K., & Reeves, T. C. (2009). The reality of virtual schools: A review of the literature.
Computers & Education, 52(2), 402-416.
Bernard, R. M., Brauer, A., Abrami, P. C., & Surkes, M. (2004). The development of a
questionnaire for predicting online learning achievement. Distance Education, 25(1), 31-47.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
26!
Black, E. W., Thompson, L. A., Askenazi, N., Ferdig, R., & Kisker, T. (2010). An assessment of
the health of virtual school participants and an innovative outreach program for chronically
ill children. Paper presented at the annual Virtual School Symposium, Phoenix, AZ.
Bocchi, J., Eastman, J. K., & Swift, C. O. (2004). Retaining the online learner: Profile of
students in an online MBA program and implications for teaching them. Journal of Education
for Business, 79(4), 245-253.
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford
Press.
Cavanaugh, C., Barbour, M. K., Brown, R., Diamond, D., Lowes, S., Powell, A., Rose, R.,
Scheick, A., Scribner, D., & Van der Molan, J. (2009). Research committee issues brief:
Examining communication and interaction in online teaching. Vienna, VA: International
Council for K-12 Online Learning. Retrieved from
http://www.inacol.org/research/docs/NACOL_QualityTeaching-lr.pdf
Cavanaugh, C., Gillan, K. J., Kromrey, J., Hess, M., & Blomeyer, R. (2004). The effects of
distance education on K-12 student outcomes: A meta-analysis. Naperville, IL: Learning
Point Associates.
Center for Educational Leadership and Technology. (2008). Utah's electronic high school audit
report. Marlborough, MA: Center for Educational Leadership and Technology.
Chyung, S. Y. (2001). Systematic and systemic approaches to reducing attrition rates in online
higher education. The American Journal of Distance Education, 15(3), 36-49.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
27!
Clark, T. (2000). Virtual high schools: State of the states - A study of virtual high school
planning and preparation in the United States. Malcomb, IL: Western Illinois University.
Clark, T., Lewis, E., Oyer, E., & Schreiber, J. (2002). Illinois Virtual High School evaluation,
2001-2002. Carbondale, IL: TA consulting and Southern Illinois University. Retrieved from
http://www.imsa.edu/programs/ivhs/pdfs/IVHS_FinalRpt.pdf
Davis, N. E. (2007, November). Teacher education for virtual schools. Unpublished manuscript.
Davis, N. E., & Roblyer, (2005). Preparing teachers for the "schools that technology built":
Evaluation of a program to train teachers for virtual schooling. Journal of Research on
Technology in Education, 37(4), 399-409.
Dennen, V. P., Darabi, A. A., & Smith, L. J. (2007). Instructor-learner interaction in online
courses: The relative perceived importance of particular instructor actions on performance
and satisfaction. Distance Education, 28(1), 65-79.
Diaz, D. P. (2000). Comparison of student characteristics, and evaluation of student success, in
an online health education course. (Unpublished doctoral dissertation). Nova Southeastern
University, Fort Lauderdale, FL.
Dickson, W. P. (2005). Toward a deeper understanding of student performance in virtual high
school courses: Using quantitative analyses and data visualization to inform decision making
No. 2008). Naperville, IL: Learning Point Associates. Retrieved from
http://www.mivu.org/upload_1/NCREL1.pdf.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
28!
DiPietro, M., Ferdig, R. E., Black, E. W., & Preston, M. (2008). Best practices in teaching K-12
online: Lessons learned from Michigan Virtual School Teachers. Journal of Interactive
Online Learning, 7(1), 10-38.
Dupin-Bryant, P. A. (2004). Pre-entry variables related to retention in online distance education.
American Journal of Distance Education, 18(4), 199.
Elluminate Inc. (2006). The impact of synchronous online learning in academic institutions:
Customer experiences from K-12 and higher education. Calgary, AB: Author.
Ferdig, R. E., Cavanaugh, C., Dipietro, M., Black, E. W., & Dawson, K. (2009). Virtual
schooling standards and best practices for teacher education. Journal of Technology and
Teacher Education, 17(4), 479-503.
Fishman, B., Marx, R., Best, S., & Tal, R. (2003). Linking teacher and student learning to
improve professional development in systemic reform. Teaching and Teacher Education,
19(6), 643-658.
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate
data analysis. New Jersey: Pearson Prentice Hall.
Hawkins, A., & Graham, C. R. (2010, April). Correlations between teacher-student interaction
and student academic performance in a large virtual high school (Vol. 2010, pp. 2934-
2940). Presented at the Society for Information Technology & Teacher Education
International Conference, San Diego, CA.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
29!
Hawkins, A., & Barbour, M. K. (2010). U.S. virtual school trial period and course completion
policy study. American Journal of Distance Education, 24(1), 5-20.
Hawkins, A., Barbour, M. K., & Graham, C. R. (2011). Strictly business: Teacher perceptions of
interaction in virtual schooling. Journal of Distance Education, 25(2).
Hawkins, A., Barbour, M. K., & Graham, C. R. (2012). “Everybody is their own island”:
Teacher disconnection in a virtual school. International review of research in open and
distance learning, 13(2), 122-143.
Heinemann, M. H. (2005). Teacher-student interaction and learning in online theological
education. part II: Additional theoretical frameworks. Christian Higher Education, 4(4), 277-
297.
Hillman, D. C., Willis, D. J., & Gunawardena, C. N. (1994). Learner-interface interaction in
distance education: An extension of contemporary models and strategies for practitioners. The
American Journal of Distance Education, 8(2), 30-42.
Jamison, T. (2003). Ebb from the web: Using motivational systems theory to predict student
completion of asynchronous web-based distance education courses. (Unpublished doctoral
dissertation). George Mason University, Fairfax, Virginia.
Jung, I., Choi, S., Lim, C., & Leem, J. (2002). Effects of different types of interaction on
learning achievement, satisfaction and participation in web-based instruction. Innovations in
Education & Teaching International, 39(2), 153-162.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
30!
Kannapel, P. J., & DeYoung, A. J. (1999). The rural school problem in 1999: A review and
critique of the literature. Journal of Research in Rural Education, 15(2), 67-79.
Levy, Y. (2007). Comparing dropouts and persistence in e-learning courses. Computers and
Education, 48(2), 185-204.
Moore, M. (1989). Editorial: Three types of interaction. The American Journal of Distance
Education, 3(2), 1-6.
Mulcahy, D. M. (2002). Re-conceptualizing distance education implications for the rural schools
of Newfoundland and Labrador. The Morning Watch, 30(1-2)
Mulcahy, D. M., Dibbon, D., & Norberg, C. (2008). An investigation into the nature of education
in a rural and remote region of Newfoundland and Labrador: The Straits
. St. John’s, NL: The Harris Centre, Memorial University of Newfoundland.
Nippard, E. (2005). Social presence in the web-based synchronous secondary classroom.
(Unpublished master's thesis). Memorial University of Newfoundland, Newfoundland.
Nippard, E., & Murphy, E. (2007). Social presence in the web-based synchronous secondary
classroom. Canadian Journal of Learning and Technology, 33(1). Retrieved from
http://www.cjlt.ca.erl.lib.byu.edu/index.php/cjlt/article/view/24/22
Osborn, V. (2001). Identifying at-risk students in videoconferencing and web-based distance
education. The American Journal of Distance Education, 15(1), 41-54.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
31!
Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence, and
performance in an online course. Journal of Asynchronous Learning Networks, 6(1), 21-40.
Picciano, A. G., & Seaman, J. (2009). K-12 online learning: A 2008 follow-up of the survey of
U.S. school district administrators. Retrieved from http://www.sloan-
c.org/publications/survey/k-12online2008
Raudenbush, S. W., & Bryk, A. S. (2001). Hierarchical linear models: Applications and data
analysis methods (2nd ed.). Thousand Oaks, CA: Sage Publications.
Restauri, S. L. (2006). Faculty-student interaction components in online education: What are the
effects on student satisfaction and academic outcomes? (Unpublished doctoral dissertation).
Capella University.
Rice, K. L. (2006). A comprehensive look at distance education in the K-12 context. Journal of
Research on Technology in Education, 38(4), 425-448.
Roblyer, M. D., & Davis, L. (2008). Predicting success of virtual school students: Putting
research-based models to practice. Retrieved from
http://www.westga.edu.erl.lib.byu.edu/~distance/ojdla/winter114/roblyer114.html
Roblyer, M. D., Davis, L., Mills, S. C., Marshall, J., & Pape, L. (2008). Toward practical
procedures for predicting and promoting success in virtual school students. American Journal
of Distance Education, 22(2), 90-109.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
32!
Roblyer, M. D., Freeman, J., Stabler, M., & Schneidmiller, J. (2007). External evaluation of the
Alabama ACCESS initiative: Phase 3 report. Eugene, OR: International Society for
Technology in Education. Retrieved from http://accessdl.state.al.us/2006Evaluation.pdf
Roblyer, M. D., (2006). Virtually successful: Defeating the dropout problem through online
school programs. Phi Delta Kappan, 88(1), 31-36.
Roblyer, M. D., & Marshall, J. C. (2003). Predicting success of virtual high school students:
Preliminary results from an educational success prediction instrument. Journal of Research
on Technology in Education, 35(2), 241.
Rosenthal, R., & Rosnow, R. L. (1975). The volunteer subject. New York, NY: John Wiley and
Sons.
Rovai, A. P., & Barnum, K. T. (2003). On-line course effectiveness: An analysis of student
interactions and perceptions of learning. Journal of Distance Education, 18(1), 57-73.
Russo, T. C., & Benson, S. (2005). Learning with invisible others: Perceptions of online
presence and their relationship to cognitive and affective learning. Educational Technology &
Society, 8(1), 54-62.
Schneider, R. M., Krajcik, J., & Blumenfeld, P. (2005). Enacting reform-based science materials:
The range of teacher enactments in reform classrooms. Journal of Research in Science
Education, 42(3), 283-312.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
33!
Shea, P., Fredericksen, E., Pickett, A., Pelz, W., & Swan, K. (2001). Measures of learning
effectiveness in the SUNY learning network. Online education volume 2: Learning
effectiveness, faculty satisfaction, and cost effectiveness (pp. 31-54). Needham, MA: Sloan
Center for Online Education.
Simpson, O. (2004). The impact on retention of interventions to support distance learning
students. Open Learning, 19(1), 79-95.
Smith, R., Clark, T., & Blomeyer, R. (2005). A synthesis of new research in K-12 online
learning. Naperville, IL: Learning Point Associates. Retrieved from
http://www.ncrel.org/tech/synthesis/synthesis.pdf
Stein, D. S., Wanstreet, C. E., Calvin, J., Overtoom, C., & Wheaton, J. E. (2005). Bridging the
transactional distance gap in online learning environments. American Journal of Distance
Education, 19(2), 105-118.
Sutton, L. A. (2001). The principle of vicarious interaction in computer-mediated
Communications. International Journal of Educational Telecommunications, 7(3), 223-242.
Swan, K. (2001). Virtual interaction: Design factors affecting student satisfaction and perceived
learning in asynchronous online courses. Distance Education, 22(2), 306-331.
Stynes, D. J., & White, E. M. (2006). Reflections on measuring recrecation and travel spending.
Journal of Travel Research, 45, 8-16.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
34!
Thurmond, V., & Wambach, K. (2004, January). Understanding interactions in distance
education: A review of literature. International Journal of Instructional Technology and
Distance Learning, Retrieved from http://itdl.org/journal/Jan_04/article02.htm.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychologist processes.
Cambridge, MA: Harvard University Press.
Wallace, R. M. (2003). Online learning in higher education: A review of research on interactions
among teachers and students. Education, Communication & Information, 3(2), 241.
Wang, A. Y., & Newlin, M. H. (2000). Characteristics of students who enroll and succeed in
psychology web-based classes. Journal of Educational Psychology, 92(1), 137-143.
Watson, J., & Gemin, B. (2008). Using online learning for at-risk students and credit recovery
North American Council for Online Learning. Retrieved from
http://www.inacol.org/.../NACOL_CreditRecovery_PromisingPractices.pdf
Watson, J., Gemin, B., & Ryan, J. (2008). Keeping pace with K-12 online learning: A review of
state-level policy and practice. Vienna, VA: North American Council for Online Learning.
Retrieved from http://www.kpk12.com/downloads/KeepingPace_2008.pdf
Watson, J., Murin, A., Vashaw, L., Gemin, B., & Rapp, C. (2010). Keeping pace with K-12
online learning: An annual review of state-level policy and practice. Vienna, VA: North
American Council for Online Learning. Retrieved from http://www.kpk12.com/wp-
content/uploads/KeepingPaceK12_2010.pdf
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
35!
Weiner, C. (2003). Key ingredients to online learning: Adolescent students study in cyberspace -
the nature of the study. International Journal of E-Learning, July-September, 44-50.
Willging, P., & Johnson, S. (2004). Factors that influence students' decisions to dropout of online
courses. Journal of Asynchronous Learning Networks, 8(4), 23-31.
Wojeiechowski, A., & Palmer, L. B. (2005). Individual student characteristics: Can any be
predictors of success in online classes? Online Journal of Distance Learning Administration,
8(2), November 8, 2010. Retrieved from
http://www.westga.edu.erl.lib.byu.edu/~distance/ojdla/summer82/wojciechowski82.htm
Zucker, A. A., & Kozma, R. B. (2003). The virtual high school: Teaching generation V. New
York: Teachers College Press.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
36!
Appendix A
1
EHS MISSION
Our mission is to educate, remediate, accelerate, and graduate Utah's
diverse learners with caring, qualified teachers using current
technology to provide rigorous curricula, timely access to quality
online instruction, and prompt, professional feedback to student work.
Instructions: How are we doing in meeting our mission? Your feedback does not impact your
credit/grade for this class.
You were added to [Course Name / Section] with [Teacher] on [Date].
[Student Name & ID Number]
Please describe the kind of credit you wanted when you signed up for this class ([Course Name /
Section]):
credit recovery - make up a failed class
original credit - first time to earn this credit
LEARNER SATISFACTION
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
Overall, I would rate the quality of this class as
outstanding
TIMELY ACCESS
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
When I had difficulty understanding the class policies
and procedures (e.g., turning in assignment, knowing
what my current grade was, which assignments I
needed to re-do, etc.), I could get help from my
teacher. [Procedural]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1
Items used to measure procedure, instructional or social interaction, along with the frequency of
interaction, are noted after each item. These notations that appear in italics within the [ ] brackets
did not appear in the version of the instrument completed by students.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
37!
My teacher helped me resolve technical issues quickly.
[Procedural]
Navigating the class materials was easy.
CARING TEACHERS
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
When I had difficulty understanding the class material,
I could get help from my teacher. [Instructional]
My teacher encouraged me to keep going with the
class. [Social]
I felt like my teacher wanted me to succeed in this
class. [Social]
By the time I left the class, I felt like my teacher knew
me. [Social]
By the time I left the class, I felt like I knew my
teacher. [Social]
I felt comfortable interacting with my teacher. [Social]
QUALIFIED TEACHERS
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
My teacher clearly communicated what I needed to do
to successfully complete the class. [Procedural]
My teacher clearly communicated what I was expected
to do on class assignments. [Procedural]
My teacher demonstrated the skills I was expected to
learn in this class.
Overall I would rate the quality of the teaching as
outstanding.
RIGOROUS CURRICULA
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
Compared to what I knew before I took this class, I
learned a lot.
The class challenged me to think in new ways.
I had opportunities in this class to explore how I could
use this information in my everyday life.
I put a great deal of time into this class.
[Course Name / Section]
10 or
8-9
6-7
4-5
2-3
0-1
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
38!
more
hours
hours
hours
hours
hours
hours
How many hours a week did you typically
spend working on this class?
CURRENT TECHNOLOGY
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
The curriculum included a good mix of written, audio,
visual, and interactive content.
EDUCATE, REMEDIATE, ACCELERATE
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
I would recommend this class from EHS to a friend.
PROMPT, PROFESSIONAL FEEDBACK
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
My teacher gave me useful feedback on my
assignments. [Instructional]
My teacher gave me prompt feedback on my
assignments. [Instructional]
My teacher frequently initiated contact with me about
the class. [Frequency]
I frequently initiated contact with my teacher about the
class. [Frequency]
[Course Name / Section]
1
day
2-3
days
4-6
days
1 week-
2 weeks
More than
2 weeks
My teacher would typically respond to my
questions in ... [Frequency]
[Course Name / Section]
Usually
twice a
week
Usually
once a
month
Usually
twice a
month
Usually
once a
month
Hardly
ever or
never
How often during the class did you and your
instructor interact on issues related to class
logistics (e.g., grades, class requirements,
etc.)? [Frequency]
How often during the class did you and your
instructor interact on issues related to the
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
39!
subject matter (e.g., clarifying, explaining,
expanding the class material, etc.)?
[Frequency]
How often during the class did you and your
instructor interact on issues related to social
matters (e.g., encouragement, motivation,
personal interest, etc.)? [Frequency]
Reason for taking this EHS class:
What was the main reason why you decided to take this class at EHS?
I wanted to make space in my schedule for other classes.
I thought this class would be faster to complete than a face-to-face (live) class.
I needed to make up for a class I failed or did not finish.
I wanted to graduate early (and I am not behind on my credits).
I wanted to earn my adult high school diploma.
I wanted to work through the class at my own pace.
I thought online classes are easier than face-to-face (live) classes.
Special needs (travel for competitions, family reasons, filming, health etc.) so I can't be
In school regularly.
The class was not offered at my school.
It sounded interesting. I just wanted to.
Other reason (please list):
ADDITIONAL FEEDBACK
What did your instructor do that helped you to
keep going with this class?
What did your instructor do that made it hard for
you to keep going with this class?
[Course Name / Section]
Strongly
Agree
Agree
Disagree
Strongly
Disagree
The transition to the new EHS system in July 2009
was smooth for me for this class.
The new EHS system (launched in July 2009)
improved the overall quality of my experience at EHS.
Yes
No
Would you be willing to have a follow-up conversation with someone to better
understand your experience at EHS with this class?
If Yes, please supply your phone number:
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
40!
To help us better serve students in this class in the future, please give your main reason for not
completing this class within 6 months.
(Submit Feedback to EHS)
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
41!
Table 1
Mean Percent Completion Rates by Discipline (n=46089)
Discipline
N
Percent Completion
SD
Computer science education
448
10.27
30.39
Fine arts
1393
14.86
35.58
World languages
1756
18.05
38.47
Language arts
7149
18.73
39.02
Science
3031
19.80
39.85
Electives/Career/Technology
5337
26.72
44.25
Social studies
7082
26.77
44.28
Mathematics
2034
26.84
44.33
Driver's education
2429
42.57
49.46
Health & physical education
8776
47.76
49.95
Financial literacy
6654
62.41
48.44
Total / Average
46089
34.18
47.43
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
42!
Table 2
Fit Statistics on Preliminary Analysis of First-Order Factor Scales
Model Fit Indices
Model
# of Items
CFI
TLI
RMSEA
Frequency First-Order Factor Structure
5
1.00
1.00
0.07
Feedback First-Order Factor Structure
4
0.99
0.99
0.19
Procedural First-Order Factor Structure
4
1.00
1.00
0.11
Social First-Order Factor Structure
5
0.99
0.98
0.27
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
43!
Table 4
Fit Statistics on Secondary Analysis of Models for Quality Items (n=13)
Model Fit Indices
Quality Factor Models
CFI
TLI
RMSEA
First-Order Factor Structure
0.95
0.98
0.20
Second-Order Factor Structure
0.96
0.99
0.18
Three-Separate Factor Structure (i.e.,
feedback, procedural, social)
0.96
0.99
0.18
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
44!
Table 4
Factor Composite Means, Standard Deviations, and Variances
Factors
N
Mean
SD
Variance
Quality Composite
2202
3.20
0.59
0.35
Feedback
2123
3.38
0.66
0.43
Procedural
2117
3.34
0.61
0.37
Social
2122
2.96
0.66
0.43
Frequency Composite
2182
2.43
0.91
0.83
Note: The number of students (N) varies as not all of the respondents answered all of the
questions.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
45!
Table 5
Reported Frequency of Student-Teacher Interaction by Type
Frequency of Interaction
Interaction Type
Hardly
Ever
Once a
month
Twice a
month
Once a
week
Twice a
week
Total
Respondents
Feedback/Instructional
43.4%
17.3%
14.4%
18.1%
6.7%
2,136
Procedural
37.3%
19.4%
15.5%
20.2%
7.7%
2,147
Social
61.7%
11.6%
8.3%
12.8%
5.6%
2,133
Note: The number of students (n) varies as not all of the respondents answered all of the
questions.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
46!
Table 6
Pearson’s Product-Moment Correlation Coefficient for Study Variables
Construct Composite
Pearson’s Coefficient (r)
N
p
r
2
Quality x Grade Awarded
.15
1,683
<.0001*
.02
Feedback x Grade Awarded
.14
2,123
<.0001*
.02
Procedural x Grade Awarded
.17
2,117
<.0001*
.03
Social x Grade Awarded
.10
2,122
<.0001*
.01
Quality x Completion
.27
2,186
<.0001*
.07
Feedback x Completion
.23
2,123
<.0001*
.05
Procedural x Completion
.30
2,117
<.0001*
.09
Social x Completion
.21
2,122
<.0001*
.04
Frequency x Grade Awarded
-.04
1,676
.142*
.00
Frequency x Completion
.10
2,167
<.0001*
.01
Note: *correlations significant at the .05 level. The number of students (N) varies as not all of the
respondents answered all of the questions and the number of completer/non completer versus
those who received a grade were not equivalent in size.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
47!
Table 7
Dependent Variable Mean, Standard Deviation, and Intraclass Correlations
Dependent Variables
Mean
SD
Intraclass Correlation*
Grade Awarded
10.04
1.62
0.21
Completion
0.74
0.44
0.27
Note: *Computed from the random intercept null model.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
48!
Table 8
Grade Awarded Estimates of Fixed Effects
Construct
Estimate
S,E.
Est./S.E.
p
Quality Composite
.27
.07
3.84
<.0001*
Feedback
.23
.07
3.43
<.0001*
Procedural
.36
.07
5.08
<.0001*
Social
.14
.06
2.21
.027*
Frequency Composite
-.09
.04
-2.27
.023*
Note: *correlations significant at the .05 level
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
49!
Table 9
Completion Estimates of Fixed Effects
Construct
Estimate
S,E.
Est./S.E.
p
Quality Composite
.84
.082
10.25
<.0001*
Feedback
.58
.051
11.48
<.0001*
Procedural
.62
.052
12.02
<.0001*
Social
.63
.058
10.98
<.0001*
Frequency Composite
.57
.061
9.31
<.0001*
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
50!
Figure 1.Three-Separate Factor Structure Model for Quality Items. This figure illustrates the
path diagram for the three separate factor structures and construct correlations.
LEARNER INTERACTION IN A VIRTUAL HIGH SCHOOL
Publication Info: Hawkins, A., Graham, C. R., Sudweeks, R. R., & Barbour, M. K. (2013). Academic performance,
course completion rates, and student perception of the quality and frequency of interaction in a virtual high
school. Distance Education, 34(1), 6483. doi:10.1080/01587919.2013.770430
51!
Figure 2. Second-Order Factor Structure Model for Quality Items. This figure illustrates the path
diagram for a second-order factor structure and construct correlations.
!
... The chatting (interaction) occurs on the provided online learning platform. Evidence exists suggesting that when educators encourage learners to interact/participate by providing such opportunities, the rate at which learners succeed in online courses increases exponentially (Croxton, 2014;Hawkins et al., 2013;Joksimović et al., 2015). Research by Goggins and Xing (2016) concurred with the previous assertion as their findings reveal that an increase in the number of interactions significantly predicted achievement of learning outcomes. ...
Conference Paper
Full-text available
After the outbreak of the COVID-19 pandemic, the majority of historically disadvantaged institutions (HDIs) in South Africa approved and adopted educational technologies to continue teaching and learning activities. Although this was a move in the right direction, very little is known in research as to whether HDI educators are effective in the use of educational technologies. Quantitative data was collected from students who were asked to evaluate the effectiveness of educators on four dimensions of online teaching and learning, namely virtual interaction, unit content migration, online course alignment, and web-based module structure. This study is descriptive by design. Descriptive statistics were used to make meaning of the data. The results reveal that HDI educators are effective with regard to four dimensions of online teaching and learning, namely virtual interaction, unit content migration, online course alignment and web-based module structure. Although literature points out that prior to the COVID-19 outbreak, HDI educators had little experience in the use of online educational technologies, the study's findings reveal that HDI educators are capable of making use of educational technologies for the purposes of online teaching and learning. To ensure that online teaching and learning is enhanced within HDIs, this study recommends that stakeholders in the higher education sector should work together to reduce the digital divide gap as it hinders students from harnessing the full potential of e-learning.
... Η εξ αποστάσεως εκπαίδευση και οι παιδαγωγικοί παράγοντες της τηλεδιάσκεψης Στην διαδικασία της μάθησης εμπλέκεται πλήθος παιδαγωγικών παραγόντων, όπως τα χαρακτηριστικά των διδασκομένων, η ίδια η φύση του εκπαιδευτικού αντικειμένου, οι στόχοι και η μεθοδολογία της διδασκαλίας που θα επιλέξει ο εκπαιδευτής (Hawkins, Graham, Sudweeks & Barbour, 2013;Joksimović, Gašević, Kovanović, Riecke & Hatala, 2015). Εάν ο εκπαιδευτής έχει καταφέρει για τους εκπαιδευόμενους να ενσωματώσουν την προσωπική τους εμπειρία και να αναπαραχθούν νέες γνώσεις τότε θα υπάρχει ένα ακόμα υψηλότερο ποσοστό ικανοποίησης των συμμετεχόντων (Αρμακόλας, Παναγιωτακόπουλος, Φραγκούλης (2019). ...
... Archambault and Larson (2015) then coded these attributes and found two overlapping categories; a new set of communication skills different from those utilized in face-to-face classrooms and a new set of organizational skills, also different from those required to teach face-to-face. This aligns with what Hawkins et al. (2013) discovered in that while the characteristics of good face-to-face teaching are similar to online education there are new and unique skills that are required for the teacher to thrive in an online instructional environment. ...
Thesis
Full-text available
Online literacy learning is still a relatively new field, however, with the advent of the COVID-19 pandemic in 2020, online learning became the primary mode of instruction for millions of students in public, charter, and private schools. This descriptive content analysis seeks to identify trends within the field of online literacy instruction from 2000 to 2021, contextual occurrences and some of the similarities and differences in the literature intended for academic audiences and the literature intended for practitioners. The articles for this content analysis were gathered from the Education Information Resource Center (ERIC) database and the Teacher Resource Center (TRC) database. 59 articles were identified as pertaining to the sample and were analyzed. Results found that within the 59 articles, those intended for academics outweighed those intended for practitioners and that the highest concentration of articles came from the end of the proposed timespan. Some differences between the articles intended for academics and practitioners were the ways the various articles addressed and were coded for student autonomy, transactional distance, and literacy instructional focus areas. The results of the content analysis revealed that there is lack of theoretical consistency within the research being produced for both academics and practitioners and there is a distinct lack of the transactional distance and systems theories, both of which underpin and are vital to the success of online literacy learning.
... Information and communication technology in teaching and learning has greatly influenced traditional education and created a new world in the field of learning. Changing the traditional patterns of education to spontaneous and selfcentered learning, change, increasing the quality of learning, reducing the role of learners and teachers, the possibility of lifelong learning education costs, and minimizing time and space constraints are prominent features of technology learning [3,4]. In the teaching method of lecture, it is assumed that students learn by ear, while research on the brain shows that this is not the way students learn. ...
Article
Full-text available
Objective: The main purpose of this study was to investigate the role of online education on the academic performance and achievement motivation of cancer students. Methods: The method of this research is the quasi-experimental or quasi-experimental method according to the nature of the subject and objectives of the research. Because in this study, it is not possible to completely control and manipulate variables. The statistical population of the present study was all children aged 6 to 12 years with cancer who were referred to Mahak Children's Hospital in 2021. The statistical sample was 60 people who were selected by available sampling and randomly divided into experimental (n=30) and control (n=30) groups. Then a training program was implemented for students using the WhatsApp social network. To collect data, two questionnaires were used; To assess students' academic performance, students' academic performance transcripts were used and Hermans' achievement motivation questionnaire was used to assess achievement motivation. Results: Based on the findings, online education has significantly increased academic performance in the experimental group. Online training has also significantly increased the motivation to progress in the experimental group. Conclusions: Therefore, the use of various virtual and online education can have a significant effect on the academic performance and motivation of cancer students. Introduction The most important mission of a country's education system is to create a suitable environment for the growth and excellence of intellectual and knowledge-based capital in the information society. With the rapid movement of the world in information technology and digital media, the role of information and communication technology in education is becoming more and more important. For all social groups to be able to participate effectively in such a society, they must learn continuous learning, creativity, and innovation, as well as active and constructive social participation. Achieving this requires a redefinition of the role and function of schools as the main educational institutions in society. The country's educational system needs a school that provides the possibility of continuous learning by using information and communication technology so that this technology is not considered as a tool but in the form of an empowering infrastructure for education and teaching. [1,2]. Information and communication technology in teaching and learning has greatly influenced traditional education and created a new world in the field of learning. Changing the traditional patterns of education to spontaneous and self-centered learning, change, increasing the quality of learning, reducing the role of learners and teachers, the possibility of lifelong learning education costs, and minimizing time and space constraints are prominent features of technology learning [3, 4]. In the teaching method of lecture, it is assumed that students learn by ear, while research on the brain shows that this is not the way students learn. Only about 20 percent of students learn aurally and another 80 percent visually and practically. However, the growth of digital and multimedia media is a result of the increasing development of these technologies. There are many types of digital media, of which computer games are the most interactive [5, 6].
... It was also found that there was a greater likelihood that students who reported having more frequent and higher quality interactions with faculty members would complete their courses [23]. Another study found that in online classrooms, the lack of teacher-student interaction may have been the main reason for learners' poor performance [24]. Together with the results of other studies on online forums, this indicates the importance of student-teacher interaction in online learning environments. ...
Article
Full-text available
Previous research has shown that social capital (teachers’ and peers’ interaction) is a challenge for rural students in China’s elite universities due to underlying issues of online learning self-efficacy (OLSE) and the quality of interaction. To understand how interaction quality is influenced, the present study drew on the achievement emotion theory to explore the mediating role of OLSE between social interactions (teacher–student, student–student) and achievement emotions (enjoyment, hopelessness, shame). Data were collected using an online questionnaire with a sample of rural students studying at elite universities (n = 479) in China. The results analyzed through Structural Equation Modeling confirmed the mediation model in which self-efficacy is a mediator in the relationships between social interactions and three types of achievement emotion as participants learned online during the coronavirus disease 2019 (COVID-19) lockdown.
... I argue that the assumption that motivation and engagement are purely linked to perceived student achievement through extrinsic factors (Hamdi, 2018;Cerasoli et al, 2014) is not an accurate depiction, but these dimensions should be addressed through a holistic approach that can possibly be replicated in other asynchronous and synchronous virtual schools. The literature that does exist (Kim et al, 2015;Hawkins and Graham, 2013;Toppin and Toppin, 2015) shows viewpoints that rest upon the students' perception of freedom and autonomy that a virtual school presents, but uses grades and performance as a measure of achievement while neglecting to view progression from students that were not deemed as high performers. ...
Thesis
Full-text available
Motivated by a long standing interest in improving my own pedagogical practice, in facilitating deeper learner engagement and bringing tools that could assist learners in their process of collaboration, this action research project looked into the areas of learner engagement, collaborative learning and motivation through the use of online tools in a synchronous remote classroom. Nine participants enrolled in lower-secondary grades 7th, 8th and 9th in a remote classroom spent five weeks carrying out various activities using four collaborative tools in the classroom (Google Docs/Slides, Kialo, Easelly and Padlet). The tasks were designed to aid in student collaboration, developing greater autonomy in the classroom and determining which tool(s) were more effective in facilitating that. Using the action research cycle, each week’s lesson plans were carefully implemented in context and guided by students’ participation as well as the researcher’s observations. The participants answered one questionnaire at the beginning and end of the study, and participated in in-class oral reflections that revolved around the tasks and tools explored thus far. The questionnaire was designed to set the tone of the study; it gave them time to reflect on their role as students in an online, remote environment. It also had a set of open-ended questions that allowed them to describe in detail how they felt about remote learning, collaboration and online activities. The same questionnaire was given to them at the end of the study to see whether there were changes. The in-class oral reflections concerned their perception on the different tools used and were aimed at gaining a greater understanding of their needs; this information was then recorded in the teacher observation journals and used in deciding the tasks, tools and activities to be done in subsequent lessons . The data collected helped identify specific online collaborative tools that increased engagement and motivation, as well as those that did not.
... As another underlying component of the online learning climate, student-content interaction is argued to positively affect satisfaction in online instruction (Zhang & Lin, 2020). The contribution of learners' interaction with the content to enhancing their satisfaction has been widely acknowledged in the literature (Hawkins et al., 2013). ...
Article
Full-text available
The ever-increasing emergence of online courses has affected students' learning outcomes as well as their participation in and satisfaction with the courses. As a result, exploring the factors which influence students' online course satisfaction might be exigent. As an attempt to fill this lacuna, the purpose of this study was to test a model of online course satisfaction in which online learning self-efficacy and online learning climate served as the variables affecting online course satisfaction in the English as a Foreign Language (EFL) context. For this aim, 186 Iranian intermediate EFL learners took part in an online survey. Structural equation modelling was utilized to analyze the structural model of online course satisfaction. The data analysis showed that although both online learning self-efficacy and online learning climate significantly predicted online course satisfaction, online learning climate was a stronger predictor. In addition, it was revealed that the online learning climate had a small substantial influence on online course satisfaction. The outcomes of this study are useful for online EFL practitioners.
Article
Full-text available
Digital technology-based online education is key to promoting high-quality development of higher education. Many studies have analyzed the effects of online education during the COVID-19 pandemic, but analyses based on large-scale data are lacking. This study uses a quasi-natural experiment during the COVID-19 pandemic to explore the short- and long-term relationships between emergency remote education (teaching and learning) and undergraduates’ academic record using multiple comparison analysis of variance (ANOVA) and multiple linear regression. The research data come from the academic record of 123,208 courses of 2622 undergraduates from the classes of 2017–2021 in a Chinese university, across nine semesters. The data do not satisfy the homogeneity of variance hypothesis test; therefore, a non-parametric test is adopted for hypothesis testing. The results show that: (1) In the online education semester, the students’ academic record improved substantially with low fluctuation and greater stability; (2) this improvement is more obvious for sophomores and juniors than for freshmen, and (3) online education during the pandemic period significantly improved the course scores of undergraduates, especially sophomores, in the following one or two semesters.
Chapter
While the growth of blended learning environments in higher education and non-educational settings has continued to increase in recent years, this has not been the case in K-12 settings. Recently, in an effort to explore the viability and effectiveness of K-12 blended learning environments, Florida Virtual School (FLVS) has been piloting blended learning communities in a number of their schools, providing opportunities to explore factors that influence the effectiveness of K-12 blended learning communities. Thus, the purpose of this chapter is to report the results of a study designed to assess conditions that influence the effectiveness of K-12 blended learning communities, and to explore learner, instructor, course, and other factors important to successful blended learning communities. Findings will inform the design, development, and implementation of future K-12 blended teaching and learning environments in an effort to support and strengthen student achievement, the preparation of teachers to facilitate effective blended learning environments.
Article
In-person school research showed that student-student and teacher-student interaction were positively related to student learning, yet unexplored for cyber schools (fully online, primary or secondary schools). The purpose of this study was to explore relationships between synchronous student-student interaction, teacher-student interaction, parents’ interaction concerns, and student achievement for 5,458 students enrolled in 34 U.S. cyber schools located in 28 states. To test hypotheses we performed longitudinal analyses of 2017-2020 data using hierarchical linear modeling (HLM). Results of our analysis of whether positive relationships exist between student-student and teacher-student synchronous interactions and student academic outcomes showed student-student interactions were positively correlated with scores on math and reading assessments, credits earned, and GPA; teacher-student interactions were correlated positively with scores on reading assessments, but negatively with credits earned and GPA. Results of our analysis of whether an inverse relationship exists between student-level interaction concerns and outcomes showed these student-level interaction concerns were negatively correlated with achievement on state assessments in most cases, and with credits earned and GPA in all cases. Results of our analysis of whether an inverse relationship exists between teacher-level interaction concerns and outcomes showed that some teacher-level interaction concerns were negatively correlated with math achievement, credits earned, and GPA in nearly all cases. Unfortunately, cyber school models typically maximize flexibility for students; yet this may come at the expense of student-student and teacher-student interaction, which in turn may be influenced by interaction concerns. Results suggest the need for future research into the mechanisms behind these various relationships.
Article
Results are reported on a preliminary use of the Educational Success Prediction Instrument (ESPRI), a measure designed to discriminate between successful and unsuccessful students in virtual high school (VHS) courses and provide a basis for counseling and support for future VHS students to make them more effective online learners. When used with 135 students in 13 virtual high schools, the instrument was found to discriminate with high accuracy and reliability between groups of successful and unsuccessful students. Suggestions are given for how ESPRI results might be used to help VHS students, and recommendations are discussed for further research in this area.
Article
The purpose of the study reported on in this paper was to explore how teachers and students manifest social presence in the web-based synchronous secondary classroom (WBSSC). Data were collected using structured and unstructured observations of twelve online recordings of web-based synchronous classes in the province of Newfoundland and Labrador, Canada. Structured observations were guided by an instrument developed by Rourke, Anderson, Garrison and Archer (2001) for identifying and measuring social presence in an online context. Findings revealed that teachers and students relied on different tools when providing affective, interactive and cohesive responses related to social presence. Manifestations of social presence by the teachers occurred through use of two-way audio whereas students relied on text-based Direct Messaging. Expressions of social presence by the students and teachers occurred most often in a context of digressions that drew attention away from the delivery of content. In addition, students demonstrated social presence using discourse conventions transferred from informal social contexts of instant messaging such as ICQ and MSN.
Article
This book is really three-books-in-one, dealing with the topic of artifacts in behavioral research. It is about the problems of experimenter effects which have not been solved. Experimenters still differ in the ways in which they see, interpret, and manipulate their data. Experimenters still obtain different responses from research participants (human or infrahuman) as a function of experimenters' states and traits of biosocial, psychosocial, and situational origins. Experimenters' expectations still serve too often as self-fulfilling prophecies, a problem that biomedical researchers have acknowledged and guarded against better than have behavioral researchers; e.g., many biomedical studies would be considered of unpublishable quality had their experimenters not been blind to experimental condition. Problems of participant or subject effects have also not been solved. Researchers usually still draw research samples from a population of volunteers that differ along many dimensions from those not finding their way into our research. Research participants are still often suspicious of experimenters' intent, try to figure out what experimenters are after, and are concerned about what the experimenter thinks of them. That portion of the complexity of human behavior that can be attributed to the social nature of behavioral research can be conceptualized as a set of artifacts to be isolated, measured, considered, and, sometimes, eliminated. This book examines the methodological and substantive implications of sources of artifacts in behavioral research and strategies for improving this situation.
Article
Nineteen on-line graduate courses were analyzed in order to determine how perceived learning varies by course and its relationship to active and passive participation by students in on-line discussions. Study results provided evidence that significant differences existed by course, suggesting that quality assurance is an issue in Internet-based instruction. Moreover, female students felt that they learned more than their male counterparts. Only active interaction, operational-ized by the number of messages posted by students per week, was a significant predictor of perceived learning. Passive interaction, analogous to listening to but not participating in discussions and operationalized by the number of accesses to the discussion boards of the e-learning system each week, was not significant. Résumé Dix-neuf cours en ligne de deuxième cycle ont été analysés afin de déterminer comment la perception de l'apprentissage varie selon le cours et aussi afin de définir le lien avec la participation active et passive des étudiants dans les discus-sions en ligne. Les résultats de l'étude ont fourni des indices sur l'existence de différences importantes selon le cours, suggérant que l'assurance de la qualité est une question importante dans la formation utilisant l'Internet. De plus, les étu-diants de sexe féminin ont le sentiment d'avoir appris davantage que le pensent leurs collègues masculins. L'interaction active, opérationnalisée par le nombre de messages affichés par les étudiants par semaine, était le seul indice significatif de l'apprentissage perçu. L'interaction passive, analogue à écouter mais ne pas parti-ciper aux discussions et opérationnalisée par le nombre d'accès aux forums de discussion du système de e-learning chaque semaine, n'était pas significatif.