Content uploaded by Robin Holding Kay
Author content
All content in this area was uploaded by Robin Holding Kay
Content may be subject to copyright.
Australasian Journal of
Educational Technology
2010, 26(2), 151-163
Assessing laptop use in higher education classrooms:
The Laptop Effectiveness Scale (LES)
Sharon Lauricella and Robin Kay
University of Ontario Institute of Technology
Considerable research has been conducted examining the use of laptops in higher
education, however, a reliable and valid scale to assess in-class use of laptops has yet
to be developed. The purpose of the following study was to develop and evaluate the
Laptop Effectiveness Scale (LES). The scale consisted of four constructs: academic use
and three areas of non-academic use (communication, watching movies, playing
games). Tested on 177 higher education students, the data generated using the LES
showed acceptable internal reliability, construct validity, content and convergent
validity. Feedback from students suggested that both academic and non-academic
constructs assessed by the LES could be expanded to incorporate a wider range of
laptop related behaviour.
Overview
There is extensive debate about the efficacy of laptop computers in the classroom.
Several studies have identified benefits such as keeping students on task and engaged
(Hyden, 2005), or following lectures via PowerPoint or multimedia (Debevec, Shih &
Kashyap, 2006). However, other research indicates that using laptops in class can be a
disadvantage (Fried, 2008). For example, in lecture based classes, students have been
reported using laptops for social activities such as surfing the web and sending emails
(Barkhuus, 2005; Barak, Lipson & Lerman, 2006). To date, no systematic measure of in-
class laptop behaviour has been designed for higher education. The purpose of this
paper is to introduce and evaluate a scale that can be used to assess the in-class
effectiveness of laptops.
Laptop use in higher education
A number of themes have emerged regarding the use of laptops in higher education
including general use, communication, student attitudes toward learning, student
achievement, and distractions. Each of these will be discussed in turn.
General use
Researchers have examined how students use laptops for learning. For example,
Demb, Erickson and Hawkins-Wilding (2004) found that 16% of overall laptop use
involved typing papers and notes. Arend’s (2004) work showed that out of class work
such as writing papers, using software programs, searching the Internet, and
completing group projects constituted the bulk of laptop use. McVay, Snyder and
Graetz (2005) added that students reported using their laptops an average of five hours
per day, with 36% of that time being spent on academic activities.
152 Australasian Journal of Educational Technology, 2010, 26(2)
Communication
Students report that using a laptop to communicate with faculty via email is generally
gratifying (Mitra & Steffensmeier, 2000), and that with email, they communicate with
faculty more often and more freely (Arend, 2004). Traditional office hours are being
replaced with email communication (Dickson & Segars, 1999; Reynolds, 2003), as
students can ask brief questions of faculty without having to meet in person. This
asynchronous communication allows for greater flexibility for students and faculty,
and also provides faculty with an electronic record of student advising and counselling
(Partee, 1996). Brown and Pettito (2003) suggest that email is becoming ubiquitous in
that, “a majority of academic communities are presuming that students and faculty
communicate via email” (p. 26).
Email communication in education is not limited to faculty-student interactions.
Demb, et al. (2004) suggest that laptops offer students the opportunity to engage in
peer-to-peer communication via email, chat rooms, bulletin boards and instant
messaging. This kind of mobile communication, including multimedia messaging, web
access, email and voice/text messaging, provides short learning activities that are
attractive to students and foster collaboration (Shih & Mills, 2007). Gay, Stefanone,
Grace-Martin and Hembrooke (2001) suggest that within a collaborative learning
environment, students working in groups recognise and use social communication for
the exchange of information, and that wireless connections increase the ability for
students to collaborate whenever and wherever they want.
Although email and instant messaging technologies suggest increased communication
among faculty and students, use of these communication tools is potentially
detrimental in the classroom. Grace-Martin and Gay (2001) found that recreational
email and instant messaging are among the primary uses of wireless laptops by
students, while Crook & Barrowcliff (2001) suggest that recreational use of email and
instant messaging in class can be distracting to students. Some attempts at quantifying
the time spent on non-academic communication and social distraction have been
made, such as Barak, et al. (2006), who found that 12% of students used their laptops
for non-learning purposes such as web surfing or social emailing. Crook & Barrowcliff
(2001) also found evidence of recreational laptop activity, arguing that the self-
reported work:play ratio of in student interviews was 30%:70%.
Student attitudes toward learning
Jones (2002) found that a majority of college students own computers and wireless
devices and believe that Internet use has enhanced their learning experience. Similarly,
alumni who participated in an undergraduate laptop program agreed that portable
computers were beneficial in their college careers (Finn & Inman, 2004). Mitra and
Steffensmeier (2000) found that students who used laptops preferred taking classes
where they could use the computer. Despite the affinity for using the computer,
student satisfaction is higher when students report using the laptops for academic
purposes (McVay, Snyder & Graetz, 2005).
A number of studies have reported that students believe that laptops make learning
“easier” (Barak, et al., 2006; Mitra & Steffensmeier, 2000; Hyden, 2006; Weaver &
Nilson, 2005). This may be an extension of the notion that modern students have
grown up with technology, and the progression to having the “convenience of a
personally configured computer [that] students naturally use it as a life tool” (Brown &
Lauricella and Kay 153
Pettito, 2003, p. 27). Caudill (2007) suggests that by carrying a personalised device,
students can quickly and easily access the resources they need.
Student achievement
Zucker (2004) argues that increasing student achievement is the most important goal
for adopting 1:1 computing, and that studies focusing on student learning deserve a
high priority. Results from studies measuring the efficacy of laptops on student
achievement, though, are mixed. Demb, et al. (2004) found that for about two-thirds of
students, the laptop computer made a significant difference in study habits, with
students reporting that the laptops helped with classroom assignments, email,
communication and research. Similarly, Siegle & Foster’s (2001) observations indicated
that students benefited from using PowerPoint presentations and reviewing course
material via the laptop. On the other hand, Wurst, Smarkola and Gaffney (2008)
observed that students felt multitasking was a distraction from academics while in
class. Fried (2008) also found that students who used laptops in class spent
considerable time multitasking and that the level of laptop use was negatively related
to overall course performance.
Distractions
The potential for the laptop to distract students may be the most significant drawback
to its use. Hembrooke and Gay (2003) set out to systematically investigate students
multitasking behaviour, as they found evidence that students were “engaged in
computing activities that were often unrelated to the immediate class lecture and
tasks” (p. 48). Barkhuus (2005) also observed that students with laptops in class were
surfing the internet, emailing or writing assignments during a lecture. The tendency to
engage in multitasking is of particular interest because paying attention “plays a
critical role in issues of motivation, engagement and learning across educational
settings” (Rapp, 2006, p. 604).
It has been argued, though, that students raised in the era of multitasking can
successfully balance Internet use and classroom participation (Young, 2006).
Distraction in the lecture hall is “nothing new”, as doodling, note passing, or
completing other assignments, function as simply low-tech distraction (Hembrooke &
Gay, 2003, p. 47), and games, email or messaging substitutes for traditional doodling
(Bhave, 2002). In contrast to researchers advocating the use of attention-aware systems
which direct users’ attention to the task at hand, Hembroke and Gay (2003) suggest
that multitasking and resulting distraction can be mitigated with self discipline, and
that students who are able to take brief breaks and return to on-task activities will not
suffer the same detriments as students who engage in prolonged web surfing (p. 59).
Methodological issues
There are at least four noteworthy issues with respect to methods used to assess and
evaluate the use of laptops in higher education. First, the predominant method of data
collection is the case study (Weaver, 2005; Weaver & Nilson, 2005; Hyden, 2005;
Birrenkott, Birtrand & Bolt, 2005; Brown, 2005; Granberg & Witte, 2005; Ohland &
Stephan, 2005; Pargas & Weaver, 2005; Stephens, 2005; McVay, Snyder & Graetz, 2005).
While this approach offers guidance and advice on best practices for using laptops, the
reliability and validity of the data have yet to be confirmed for the general population.
154 Australasian Journal of Educational Technology, 2010, 26(2)
Second, some studies have employed quasi-experimental methods to assess the impact
of laptops on student achievement (Hembrooke & Gay, 2003; Siegle & Foster (2001);
Trimmel & Bachmann, 2004). These studies do not offer data on how laptops are
actually used in higher education classrooms.
Third, a number of studies involve students who were not provided equal access to
technology. Motiwalla’s (2007) paper on using handheld devices in class is innovative,
yet it is not truly indicative of the benefits and drawbacks of mobile learning because
many of the students in the sample did not have devices equipped with the technology
essential to participate in the study. Similarly, in with Barkhuus’s (2005) study, not all
students had a laptop.
Finally, the vast majority of studies offered no reliability or validity estimates for the
measures used to assess laptop behaviour (Barak, et al., 2006; Debevec, Shish &
Kashyap, 2006; Kitsantas & Chow, 2008; Shih & Mills, 2007). While the data from these
studies offers useful suggestions regarding laptop use, we argue that a more
systematic approach to measuring laptop use inside the classroom is also needed.
Method
Context
The study took place at University of Ontario Institute of Technology, a small
university (just under 5000 students) located in a large metropolitan area. Every
student in the university was issued a laptop computer and the campus was
completely wireless so students had 24/7 access to the Internet. Students participated
in weekly lectures, but had access to a variety of course materials distributed through
an online learning management system. While the majority of class time was spent
lecturing, a number of laptop based activities were used, including online attitude and
opinion surveys, online quizzes, posting comments in discussion boards, online
research, in-class creation of charts and concept maps, and viewing online videos.
Sample
The sample consisted of 177 higher education students (89 males, 88 females), in their
first (n=74), second (n=59), third (n=30) or fourth year (n=13) of study. Seventy-five
percent (n=132) of the students were born in Canada and 86% (n=153) reported that
English was their first language. Students were enrolled in social science (n=108),
business (n=43), engineering (n=11), science (n=12) or health science (n=3). Twenty-
four of the 177 students did not have English as their first language. The average grade
of first year students before they entered UOIT was 78.9 percent (S.D. =6.3, range 65 to
90). The average grade for second to fourth students was 74.6 percent (S.D. = 7.8, range
59 to 90). Almost 85% (n=149) of the students reported that they were either proficient
(n=94) or very proficient (n=55) in using computers. Average daily use of laptop
computers reported by students was 8.8 hours (S.D. = 4.4, range 2 to 16). All students
leased a laptop that was imaged specifically to their selected program and had wireless
access to the web throughout campus.
Procedure
At the conclusion of the final class meeting in December 2007, students were invited to
participate in an anonymous, online survey by following a link provided by the
Lauricella and Kay 155
instructor. Participation was voluntary and participants could withdraw from the
study at any time. The instructor was unable to determine who chose to participate
and data was not accessed until all marks for the courses were submitted. It took
approximately 10-15 minutes for students to complete the survey. The total number of
students in both courses was 521, resulting in an approximate coverage rate of 34%.
Data sources
Descriptive variables
All students were asked their age, gender, year of study, and average grade at UOIT.
They were also asked how much time they spent in total on non-academic activities
during class and whether, overall, they felt that laptops helped them academically (see
Lauricella & Kay, 2009).
LES survey
The LES scale consisted of four constructs (see Lauricella & Kay, 2009 for a copy of the
scale). These constructs were created based on the literature review which suggested
that the most salient variables were those that focussed on academic and non-academic
use. The first construct consisted of 4 items and assessed the academic use of laptops
during class (Academic - class use). The second construct assessed non-academic use
of laptops during class for the purpose of communicating to others (Communication).
The third construct assessed non-academic use of laptops during class for the purpose
of watching movies (Watching movies). The fourth construct assessed non-academic
use of laptops during class for the purpose of playing games (Playing games).
Descriptive statistics for the LES are presented in Table 1.
Table 1: Description of Laptop Effectiveness Scale (LES)
Scale
Number
of items
Possible
range
Internal
reliability
Academic use
4
0 to 16
r = 0.87
Communication
4
0 to 16
r = 0.85
Watching movies
2
0 to 8
r = 0.85
Non-academic use
Playing games
2
0 to 8
r = 0.70
Student comments
Students were asked two open ended questions about (a) how laptops were helpful
during class time and (b) how laptops were not helpful during class time. Two-
hundred and thirty four comments were made about the laptop being helpful and 108
comments were offered about the laptop not being helpful. These open ended items
were categorised and rated independently by two raters. Comments where categories
were not exactly the same were shared and reviewed a second time by each rater.
Using this approach, an inter-rater reliability of 99% was attained for categories.
Data analysis
A series of analyses were run to assess the reliability and validity of the LES. These
included:
1. internal reliability estimates for the LES constructs (reliability);
2. a principal component factor analysis for the LES (construct validity);
3. correlations among constructs within the LES scale (construct validity);
156 Australasian Journal of Educational Technology, 2010, 26(2)
4. correlation between LES and descriptive variables – average grade, time spent on
non-academic activities, whether laptops helped academics (convergent validity);
5. frequency of laptop behaviours assessed by the LES constructs (content validity);
6. student comments (content validity).
Results
Internal reliability
The internal reliability estimates for the LES constructs based on Cronbach’s alpha
were 0.87 (Academic use), 0.85 (Non-academic: Communication), 0.85 (Non-academic:
Watching movies), and 0.70 (Non-academic: Playing games) – see Table 1. These
moderate to high values are considered acceptable levels for measures in the social
sciences (Kline, 1999; Nunnally, 1978).
Construct validity
Principal component analysis
To date, reliable and valid scales assessing the impact of laptops during class have not
been developed, therefore the use of a confirmatory factor analysis is not warranted.
Instead, a principal components analysis was done to explore whether the four
learning object constructs (academic use, non academic -communication, watching
movies, playing games) in the LES formed four distinct factors. Since all
communalities were above 0.4 (Stevens, 1992), the principal component analysis was
deemed an appropriate exploratory method (Guadagnoli & Velicer, 1988). The results
from the varimax rotation (using Kaiser normalisation) are presented because they
simplify the interpretation of the data (Field, 2005). The Kaiser-Meyer-Olkin measure
of sampling adequacy (0.643) and Bartlett’s test of sphericity (p <.001) indicated that
the sample size was acceptable.
The principal components analysis produced four factors (academic use, non-academic
- communication, watching movies, playing games) which corresponded well with the
proposed laptop effectiveness constructs (Table 2). There was no overlap among
constructs with respect to loadings.
Correlations among LES constructs
The correlations among the LES constructs were modest but significant ranging from
0.15 to 0.41 (Table 3). Academic use of laptops during class was negatively correlated
with all three non-academic use constructs (communication, moves, and games). The
three non-academic constructs were significantly and positively correlated with each
other. Shared variances, ranging from 7% to 17% were small enough to support the
assumption that each construct measured was distinct.
Convergent validity
Academic use
The correlation between academic use of laptops in class and average grade was not
significant (r = 0.14, n.s.). However, academic use was significantly and negatively
correlated with total time spent on non-academic activities (r = - 0.22, p <.01 in course
where survey was taken, r = - 0.30, p <.01 other courses). Finally, academic use was
significantly and positively correlated with student perceptions of whether laptops
helped them academically (r = 0.39, p < .01).
Lauricella and Kay 157
Table 2: Varimax rotated factor loadings on Laptop Effectiveness Scale (LES)
In-class activities
Factor
1
Factor
2
Factor
3
Factor
4
Take notes (other classes)
.875
Use laptop for academic activities (other classes)
.862
Take notes (this class)
.860
Academic –
in class use
Use laptop for academic activities (this classes)
.796
Use email for non-academic (other class)
.875
Instant messaging for non-academic
(this class)
.812
Instant messaging for non-academic
(other class)
.807
Commun-
ication
Use email for non-academic (this class)
.807
Watch movies (this class)
.896
Watching
movies
Watch movies (other class)
.896
Play games (other class)
.904
Non-
academic
Playing
games
Play games (this class)
.749
Factor
Eigenvalue
% of variance
Cumulative %
1
3.79
31.6
31.6
2
2.79
23.3
54.9
3
1.42
11.8
66.7
4
1.14
9.5
76.3
Table 3: Correlations among Laptop Effectiveness Scale constructs
Non-academic
Scale
Academic
use
Commun-
ication
Watching
movies
Playing
games
Academic use
1.00
-0.15
-0.17*
-0.21*
Communication
1.00
-0.41**
-0.26**
Watching movies
1.00
-0.26**
Non-academic
Playing games
1.00
* p < .05; ** p < .01
Non-academic communication
The correlation between non-academic communication use of laptops and average
grade was significant and negative (r = -0.33, p < .01). Non-academic communication
use was significantly and positively correlated with total time spent on non-academic
activities (r = - 0.54, p <.01 in the course where survey was taken, r = - 0.56, p <.01 other
courses). Finally, the correlation between non-academic communication use and
student perceptions of whether laptops helped them academically was not significant
(r = 0.10, n.s.).
Watching movies
The correlation between non-academic movie use of laptops and average grade was
not significant (r = -0.13, n.s.). Non-academic movie use was significantly and
positively correlated with total time spent on non-academic activities (r = - 0.32, p <.01
in course where survey was taken, r = - 0.37, p <.01 other courses). Finally, non
academic movie use was significantly and negatively correlated with student
perceptions of whether laptops helped them academically (r = -0.18, p < .05).
158 Australasian Journal of Educational Technology, 2010, 26(2)
Playing games
The correlation between non-academic games use of laptops and average grade was
significant and negative (r = -0.27, p < .01). Non-academic games use was significantly
and positively correlated with total time spent on non-academic activities (r = - 0.30, p
<.01 in course where survey was taken, r = - 0.45, p <.01 other courses). Finally, the
correlation between non-academic communication use and student perceptions of
whether laptops helped them academically was not significant (r = 0.04, n.s.).
Content validity
Frequency of laptops behaviour observed
The frequency distribution of behaviours assessed by the LES scale is presented in
Table 4. Students estimated the percentage of class time in which they engaged in
academic and non-academic activities. Most students spent over 50% of class time
either taking notes (74%) or engaging in academic activities (68%). Over 70% of
students spent up to 50% of class time sending non-academic email messages. Fifty-six
percent of students exchanged instant messages up to 50% of the time during class,
thirty-one percent of students spent over 50% of class time instant messaging. Just over
one third of students played games up to 50% of the time in-class. Finally, just 10% of
students watched movies up to 50% of in-class time.
Table 4: Frequency of in-class laptop behaviours assessed by the LES Scale (n=177)
Percentage of class time
In-class activities
0%
1-25%
26-50%
51-75%
76-100%
Take notes
2%
8%
15%
34%
40%
Academic
Academic activities
3%
9%
20%
31%
37%
Use email
12%
41%
31%
10%
5%
Instant messaging
8%
32%
24%
17%
14%
Watching movies
89%
6%
4%
0%
1%
Non-academic
Playing games
63%
24%
11%
1%
1%
Student comments on laptops being helpful
Items in the LES for assessing academic use of laptops included note taking and using
the laptop for other academic purposes. Forty-seven percent (n=110) of all student
comments focused on some aspect of notes including taking notes, sharing notes, or
using lecture notes. Twenty-four percent (n=56) of the student comments referred to
an “academic use” of laptops during class (activities, communicating to other students,
using course resources). It is worth noting that 28% (n=65) of the student comments
targeted general aspects of learning that were assisted by the presence of a laptop:
being more organised, increasing efficiency, and improving focus or attention in class.
Finally, a small subset of students with special needs (n=3) commented that the laptop
was helpful in supporting areas of weakness.
Student comments on laptops not being helpful
The LES identified communication, watching movies, and playing games as key
distractions when using a laptop during class. Comments from students confirmed
that these three behaviours were indeed counter productive. Specifically, students
reported communication (22%, n=24 comments), playing games (19%, n=21
comments), and to a lesser extent watching movies (10%, n= 11) as being unhelpful. In
addition, a large group of students (27%, n= 29) noted that laptops were generally
distracting during class. Seven percent of the students added that laptops limited their
Lauricella and Kay 159
focus (7%, n= 8). Finally, surfing the web was acknowledged as being unhelpful by 8
students (7%).
Discussion
The purpose of this study was to develop and evaluate a scale used to assess classroom
use of laptops. Careful attention was paid to addressing methodological concerns
noted in previous laptop studies. Based on a thorough review of the literature, the LES
scale was comprised of four constructs: academic use, non-academic communication,
playing games, and watching movies. The internal reliability, construct validity,
convergent validity, and face validity of the LES were assessed.
Addressing methodological concerns
Four key methodological issues were noted in previous research and addressed in the
current study. First, data collection went beyond the typical case study format and
focused on a large sample of students. Second, both the reliability and validity of
quantitative and qualitative data were provided. Third, all students in this study had
equal access to laptop technology, an oversight that was observed in previous papers.
Overall, the quality of data collected in this study was sound.
Reliability
The internal reliability estimates (0.78 to 0.89) for the learning object constructs in the
LES were good (Kline, 1999; Nunnally, 1978), as was the inter-rater reliability (99%) of
the categories and ratings used to assess student comments. None of the laptop
evaluation studies reviewed for this paper ( e.g. Barak, et al., 2006; Debevec, Shish &
Kashyap, 2006; Kitsantas & Chow, 2008; Shih & Mill, 2007) offered reliability statistics,
yet it is argued that reliability is a fundamental element of any evaluation tool and
should be calculated for future research studies, if the sample size permits.
Validity
Previous research has not reported validity estimates for scales assessing laptop
behaviour during class. Three types of validity were assessed for the LES in this paper:
construct, convergent, and face.
•Construct validity
The principal components analysis revealed four distinct constructs related to the
use of laptops during class: academic use, and non-academic - communication,
watching movies, and playing games. These constructs are consistent with those
proposed by the LES scale and previous research. Furthermore, correlations among
the four LES constructs were significant though relatively small. Shared variances
among constructs were small enough to support the assumption that each construct
measured was distinct.
•Convergent validity
Three variables were used to explore convergent validity: average grade,
perception that laptops help academics, and total time that students estimated they
were using laptops for non-academic purposes in class. We would predict that
academic use of laptops during class would be positively correlated with average
grade and student perceptions of laptop usefulness and negatively correlated with
student estimates of non-academic use of laptops. These predictions were
160 Australasian Journal of Educational Technology, 2010, 26(2)
confirmed and indicate a certain level convergent validity for the academic use
construct.
One would also predict that the three non-academic constructs (communication,
playing games, and watching movies) would be negatively correlated with average
grade and student perceptions of laptop usefulness, and positively correlated with
student estimates on non-academic use of laptops in class. These predications were
supported by the correlation analysis and help establish convergent validity for the
non-academic constructs assessed by the LES.
•Content validity
The LES was designed to assess typical behaviours of students while using a laptop
in class. For the scale to have content validity, we need to determine the extent to
which a measure represents all in-class laptop behaviours displayed by students.
Numerous constructs could be assessed but if the behaviours associated with these
constructs are rarely observed, the content validity of the scale is compromised.
Academic use of laptops, non-academic communication, and playing games were
reported frequently enough to support their inclusion in the LES scale. However,
only 8 to 11 percent of students watched movies. One might question the necessity
of including this construct is the LES scale.
Frequency of student comments about the helpfulness of laptops supported the
inclusion of the note taking and general academic activities items in the LES scale.
However, student comments provided insight into how the academic use construct
might be expanded. First, sharing notes among peers and using posted instructor notes
could be added as these behaviours were noted frequently. Second, communicating to
other students (e.g., through email or instant messaging) for academic purposes and
using course resources might provide more detail and substance to the “general”
academic activities item. Third, a new item reflecting student learning (e.g.,
organisation, increased focus) should probably be added given that almost 30% of the
students commented on these behaviours. Finally, a question about laptops supporting
students with special needs during class may be worth adding.
Student comments about unhelpful behaviour were consistent with the inclusion of
non-academic communication behaviours, playing games, and watching movies in the
LES scale. One item that could be added to the LES scale is a general question about
distracting behaviour. This notion of “general distraction” was the most frequent
observation made by students, yet it was not directly incorporated in the LES scale. A
final item that might be added is surfing the web for non-academic purposes.
In conclusion, the frequency data on laptop behaviour and student comments support
the inclusion of the existing LES constructs, although, new items may need to be
added to establish content validity.
Summary
The purpose of this study was to develop and evaluate a scale used to assess in-class
use of laptops in higher education. The Laptop Evaluation Scale consisted of four
constructs: academic use, non-academic communication, playing games, and watching
movies. All scale items showed acceptable internally reliability. A principal
components factor analysis demonstrated good construct validity. Correlation among
the LES constructs were significant but small enough to support the existence of four
Lauricella and Kay 161
distinct constructs. All four LES constructs were correlated with average grade, time
spent on non-academic activities, and perceptions of laptop helpfulness in class,
thereby proving evidence for convergent validity. Finally, content validity was
supported by frequencies of laptop behaviours reported as well as student comments.
However, student comments indicate that the content of the LES may need to be
expanded to include more specific academic and non-academic behaviours.
Implications for education
A number of higher education institutions have partially or fully integrated laptops
into their programs. To date, a scale has not been developed to assess what students
are doing with laptops during class. This study was a first attempt to develop a metric
to measure in-class laptop behaviour. Reliability and validity estimates indicate that
the LES could be used to assess the academic merits of using laptops in class, as well as
the challenges that students may be experiencing. Identifying the level of academic
activity in laptop based classes is critical to improving the quality of instruction
offered. For example, if the LES indicates that a number of students are distracted by
laptops, playing games, or watching movies, then it might be time to reflect on how
this tool is being used.
In addition, using the LES might be a good start to examining factors that improve the
quality of laptop based education. Comparing classes that score high on academic use
with those that score high on non-academic use could help to provide important
information about designing more effective teaching strategies.
Caveats and future research
Data from the LES appeared to be reliable and valid. Nonetheless, there are several
caveats that should be addressed to guide future research.
First, the sample size, while relatively large, consisted of mostly social science and
business students. The LES needs to be tested on a wider variety of subject areas where
use of laptops could vary widely. Second, evidence based on student comments
indicates that the LES may need to be expanded to include more specific examples of
academic and non-academic use. A more comprehensive scale will provide a more
accurate reflection of in-class laptop behaviour. Third, while qualitative data were
collected in the form of written feedback, it might be prudent to collect focus group
data in order to expand upon the possible laptop behaviours that students might be
expressing. Fourth, the type of laptop behaviours observed is partially dependent on
the “latest” technological trends. Academic and non-academic behaviour may need to
be added or subtracted depending on how the technology changes. Finally, it would
be a good next step to assess the predictive validity of the LES constructs with respect
to student performance in the actual class where the laptop was being used.
References
Arend, B. D. (2004). New patterns of student engagement. About Campus, 9(3), 30-32.
Barak, M., Lipson, A. & Lerman, S. (2006). Wireless laptops as means for promoting active
learning in large lecture halls. Journal of Research on Technology in Education, 38(3), 245-263.
Barkhuus, L. (2005). Bring your own laptop unless you want to follow the lecture: Alternative
communication in the classroom. Proceedings of the 2005 international ACM SIGGROUP
conference on Supporting Group Work, 140-143.
162 Australasian Journal of Educational Technology, 2010, 26(2)
Bhave, M. P. (2002). Classrooms with wi-fi. T.H.E. Journal, 30(14), 17-20. [verified 20 Mar 2010]
http://thejournal.com/articles/2002/11/01/classrooms-with-wifi.aspx
Birrenkott, G., Birtrand, J. A. & Bolt, B. (2005). Incorporating laptop technologies into an animal
sciences curriculum. New Directions for Teaching and Learning, 101, 61-66.
Brown, D. G. (2005). Concluding comments: Laptop learning communities. New Directions for
Teaching and Learning, 101, 89-94.
Brown, D. G. & Petitto, K. R. (2003). The status of ubiquitous computing. Educause Review, 38, 25-
33. http://net.educause.edu/ir/library/pdf/erm0331.pdf
Caudill, J. (2007). The growth of m-learning and the growth of mobile computing: Parallel
developments. International Review of Research in Open and Distance Learning, 8(2), 1-13.
http://www.irrodl.org/index.php/irrodl/article/view/348/873
Crook, C. & Barrowcliff, D. (2001). Ubiquitous computing on campus: Patterns of engagement
by university students. International Journal of Human-Computer Interaction, 13(2), 245-256.
Debevec, K., Shih, M. & Kashyap, V. (2006). Learning strategies and performance in a technology
integrated classroom. Journal of Research on Technology in Education, 38(3), 293-307.
Demb, A., Erickson, D. & Hawkins-Wilding, S. (2004). The laptop alternative: Student reactions
and strategic implications. Computers & Education, 43(4), 383-401.
Dickson, G. W. & Segars, A. (1999). Redefining the high-technology classroom. Journal of
Education for Business, 74(3), 152.
Finn, S. & Inman, J. G. (2004). Digital unity and digital divide: Surveying alumni to study effects
of a campus laptop initiative. Journal of Research on Technology in Education, 36(3), 297-317.
Fried, C. B. (2008). In-class laptop use and its effects on student learning. Computers & Education,
50(3), 906-914.
Gay, G., Stefanone, M., Grace-Martin, M. & Hembrooke, H. (2001). The effects of wireless
computing in collaborative learning environments. International Journal of Human-Computer
Interaction, 13(2), 257-276.
Grace-Martin, M. & Gay, G. (2001). Web browsing, mobile computing and academic
performance. Educational Technology and Society, 4(3), 95-107.
http://www.ifets.info/journals/4_3/grace_martin.html
Granberg, E. & Witte, J. (2005). Teaching with laptops for the first time: Lessons from a social
science classroom. New Directions for Teaching and Learning, 101, 51-59.
Guadagnoli, E. & Velicer, W. (1988). Relation of sample size to the stability of component
patters. Psychological Bulletin, 103, 265-275.
Hembrooke, H. & Gay, G. (2003). The laptop and the lecture: The effects of multitasking in
learning environments. Journal of Computing in Higher Education, 15(1), 46-64.
Hyden, P. (2005). Teaching statistics by taking advantage of the laptop's ubiquity. New Directions
for Teaching and Learning, 101, 37-42.
Kitsantas, A. & Chow, A. (2007). College students' perceived threat and preference for seeking
help in traditional, distributed, and distance learning environments. Computers & Education,
48(3), 383-395.
Kline, P. (1999). The handbook of psychological testing (2nd edition). London: Routledge.
Lauricella, S. & Kay, R. H. (2009). Appendix A - The Laptop Effectiveness Scale.
http://faculty.uoit.ca/kay/papers/les/AppendixA_LES.pdf
McVay, G. J., Snyder, K. D. & Graetz, K. A. (2005). Evolution of a laptop university: A case
study. British Journal of Educational Technology, 36(3), 513-524.
Mitra, A. & Steffensmeier, T. (2000). Changes in student attitudes and student computer use in a
computer-enriched environment. Journal of Research on Computing in Education, 32(3), 417.
Lauricella and Kay 163
Motiwalla, L. F. (2007). Mobile learning: A framework and evaluation. Computers & Education,
49(3), 581-596.
Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.
Ohland, M. W. & Stephan, E. A. (2005). Using laptops in engineering courses for real-time data
collection and analysis. New Directions for Teaching and Learning, 101, 67-79.
Pargas, R. P. & Weaver, K. A. (2005). Laptops in computer science: Creating the "learning
studio". New Directions for Teaching and Learning, 101, 43-51.
Partee, M. H. (1996). Using e-mail, web sites, and newsgroups to enhance traditional instruction.
T.H.E. Journal, 23(11), 79-82. http://thejournal.com/Articles/1996/06/01/Using-EMail-Web-
Sites--Newsgroups-to-Enhance-Traditional-Classroom-Instruction.aspx
Rapp, D. N. The value of attention aware systems in educational settings. Computers in Human
Behavior, 22(4), 603-614.
Reynolds, R. Making IM work for education. [viewed 29 Apr 2009, not found 20 Mar 2010]
http://www.xplanazine.com/2003/04/making-im-work-for-education-part-1
Shih, Y. E. & Mills, D. (2007). Setting the new standard with mobile computing in online
learning. International Review of Research in Open and Distance Learning, 8(2).
http://www.irrodl.org/index.php/irrodl/article/view/361/872
Siegle, D. & Foster, T. (2001). Laptop computers and multimedia and presentation software:
Their effects on student achievement in anatomy and physiology. Journal of Research on
Technology in Education, 34(1), 29-37.
Stephens, B. (2005). Laptops in psychology: Conducting flexible in-class research and writing
laboratories. New Directions for Teaching and Learning, 101, 15-26.
Stevens, J. P. (1992). Applied multivariate statistics for the social science applications (2nd edition).
Hillsdale, NJ: Erlbaum.
Trimmel, M., & Bachmann, J. (2004). Cognitive, social, motivational and health aspects of
students in laptop classrooms. Journal of Computer Assisted Learning, 20(2), 151-158.
Weaver, B. E. (2005). Laptops in the humanities: Classroom walls come tumbling down. New
Directions for Teaching and Learning, 101, 81-88.
Weaver, B. E. & Nilson, L. B. (2005). Laptops in class: What are they good for? What can you do
with them? New Directions for Teaching and Learning, 101, 3-13.
Wurst, C., Smarkola, C. & Gaffney, M. A. (2008). Ubiquitous laptop usage in higher education:
Effects on student achievement, student satisfaction, and constructivist measures in honors
and traditional classrooms. Computers & Education, 51(4), 1766-1783.
Young, J. R. (2006). The fight for classroom attention: Professor vs. laptop. Chronicle of Higher
Education, 52(39), A27-A29.
Zucker, A. (2004). Developing a research agenda for ubiquitous computing in schools. Journal of
Educational Computing Research, 30(4), 371-386.
Dr Sharon Lauricella, Assistant Professor
Faculty of Criminology, Justice and Policy Studies
University of Ontario Institute of Technology.
Email: Sharon.Lauricella@uoit.ca
Web: http://criminologyandjustice.uoit.ca/EN/main/faculty_staff/lauricella.php
Dr Robin Kay (corresponding author), Associate Professor
Faculty of Education, University of Ontario Institute of Technology
11 Simcoe St N, Oshawa ON L1H 7L7, Canada
Email: Robin.Kay@uoit.ca Web: http://faculty.uoit.ca/kay/home/
