Exploring the Impact of ICT Use Patterns on Postgraduate Student Academic Achievement in a Developing Country

Abstract

A common motivation for the use of ICT in teaching and learning is the central belief that it has the potential to exert a positive effect on student academic achievement. Evidence to support this assertion, however, is contradictory. The current study attempts to address known research gaps by examining the impact of ICT use patterns on postgraduate student academic achievement in a developing country. Specifically, potential student bias in reporting educational achievement was removed by using instructor-assessed class marks as the dependent variable. Data collected from 302 students were analysed to test several hypotheses constructed after a literature review of previous studies in the field. Statistical evidence showed constructs of teaching and learning with the aid of ICT, student ICT literacy, and behavioural engagement to correlate strongly with student performance , with challenges of ICT use not influential. The current research findings affirmed the belief that technology-supported education in a developing country can play a significant role in student academic achievement, especially in postgraduate settings.

Full text

Exploring the Impact of ICT Use Patterns on
Postgraduate Student Academic Achievement in a
Developing Country
Thomas M. van der Merwe [] [0000-0002-0710-3366] and Ambrose A. Azeta [0000-0002-3556-933]
School of Computing, University of South Africa Science Campus, 28 Pioneer Ave, Florida
Park, Roodepoort 1709, South Africa
vdmertm@unisa.ac.za, ambrose.azeta@covenantuniversity.edu.ng
Abstract.
A common motivation for the use of ICT in teaching and learning is the central
belief that it has the potential to exert a positive effect on student academic
achievement. Evidence to support this assertion, however, is contradictory. The
current study attempts to address known research gaps by examining the impact
of ICT use patterns on postgraduate student academic achievement in a develop-
ing country. Specifically, potential student bias in reporting educational achieve-
ment was removed by using instructor-assessed class marks as the dependent var-
iable. Data collected from 302 students were analysed to test several hypotheses
constructed after a literature review of previous studies in the field. Statistical
evidence showed constructs of teaching and learning with the aid of ICT, student
ICT literacy, and behavioural engagement to correlate strongly with student per-
formance, with challenges of ICT use not influential. The current research find-
ings affirmed the belief that technology-supported education in a developing
country can play a significant role in student academic achievement, especially
in postgraduate settings.
Keywords: Course Activities using ICT; Student Engagement with ICT; Stu-
dent ICT Literacy; Challenges of ICT Use; Student Academic Achievement, De-
veloping Country, Postgraduate Students
1 Introduction
Over the last few decades, technology investment and usage in education has increased
more than a hundredfold [1]. The primary reason for this spending is the general belief
that information and communication technology (ICT) can increase the effectiveness
of teaching, thereby improving student learning and performance. Not surprisingly, the
provision of ICT in education has become a major focus of many governments, espe-
cially in developing countries. However, research data to support this belief is equivo-
cal. For example, whereas there is research evidence of a positive impact [2-5], other
studies have reported little or no impact [6-9]. One possible reason for the disparity is
141

that the real impact of ICT on learning is not easily understood, and for various reasons
[6]. For example, “Are we referring to ICT infrastructures or their actual use? Is the
intensity of use an important factor? Are we referring to specific skills, competences
and domains, or more holistic concepts of learning?” (p. 28). A further question asks,
how do we define and capture the relationship between ICT use and learning?
Fifteen years ago, Trucano [10] created a series of knowledge maps on behalf of The
World Bank regarding what is and what is not known about key areas of ICT use in
education in both Organisation for Economic Co-operation and Development (OECD)
and developing countries. From these maps, several pressing research questions were
formulated. A decade later, the author noted that many of these research questions are
still relevant, confirming that we still do not have answers to many of the basic ques-
tions asked in 2005. Despite such uncertainties, substantial investments in ICT in edu-
cation continue, and to convince educators that the use of ICT is indeed a worthwhile
pursuit, there remains a pressing need for ongoing research.
The broad focus of this study is on three statements as extracted from the knowledge
base: (a) it is still unknown if some subjects are better suited to ICT integration than
others, (b) many studies that find positive effects of ICT on student learning often rely
(to an unacceptable degree) on self-reporting, which may be open to a variety of posi-
tive biases, and (c) the impact and nature of ICT use on student engagement has not
been well documented. Furthermore, we limit our focus to a developing country, be-
cause what lessons were learned, and best practices developed, have not been packaged
well [10].
The 1st objective of this study is to review and use the available literature on ICT use
in education to identify a suitable research setting, relevant concepts, and research ques-
tions and hypotheses that may assist us in the construction of an appropriate research
instrument. The 2nd objective is to employ the instrument and to use instructor-assessed
class marks in place of student self-reporting as the dependent variable in further sta-
tistical procedures. In reporting the results, the overall purpose of this paper is to con-
tribute to the gaps that have been identified.
2 Literature review
2.1 Identification of research setting
Of the recent literature we reviewed and referenced in this study, and where we could
source relevant factors, a total of 19 studies reported an ICT impact on student achieve-
ment, while 9 did not. Most studies were conducted in developed countries, while sub-
ject areas studied ranged from the traditionally “more challenging” (e.g. medicine and
mathematics) to the arguably “less demanding” (e.g. languages and education). What
stood out from our review, however, was the level of education, which ranged from
primary (5) and secondary (30) to undergraduate (14). Just one study focused on post-
graduate students (at MBA level, in a developed country [12]). On face value, a study
of postgraduate students offers a well-defined group, whose members share some com-
mon characteristics: (1) Being older, but still part of the Millennial Generation, they are
expected to have had more exposure to ICT, thereby removing potential ICT experience
142

bias; (2) postgraduate studies are research-oriented by nature, which necessitates the
use of ICT outside the classroom, such as searching for and exploring online research
resources; and (3) by demonstrating a capacity and dedication for further independent
learning, the use of a wide range of learning strategies, including the use of ICT, is
likely. The latter two notions are in line with research which observed that whereas
undergraduate students use scaffolded learning activities more frequently, postgraduate
students use research-based e-learning activities [13] and are more likely to use ICT for
academic purposes [14]. For these reasons, a post-graduate and research-oriented
course became a focus of our study.
2.2 Formulation of research questions and hypotheses from the
literature
Having identified an appropriate research setting, this section identifies relevant ICT
use concepts from the literature that will guide us in constructing suitable research ques-
tions and hypotheses. At the most rudimentary level, successful use of ICT in education
is reliant on several key constructs, which, in our view and experience, can broadly be
categorised as the extent to which available ICT is employed in course activities by the
instructor; the level of student engagement with such ICT inside and outside the class-
room; students’ ICT knowledge and skills; and ICT challenges encountered. Where
possible, we limit our review to research that focused on the relationship between ICT
use and student achievement.
Course activities using ICT constructs of interest include teaching with the aid of
ICT, presentation and discussion of classwork with ICT, and suitability of classwork
for ICT integration. Whereas positive teaching-with-ICT influences were reported by
many authors [4, 15, 16], students have been found to perform worse when personal
computing technology was available [8]. As for presentation and discussion of class-
work, ICT is seen to support more dialogic and synergistic approaches in both group
and individual activities [17], although its value appears dependent on the context and
situation [18]. We formulate the following broad research question and hypothesis: RQ
1: To what extent do course activities using ICT have an impact on academic achieve-
ment? H1: Course activities using ICT have a positive impact on academic achieve-
ment.
Significant relationships between student engagement and academic achievement
have been reported, with the former defined as the emotional, cognitive, behavioural
and psychological reactions to the learning process [19]. However, no significant dif-
ference in the level of student engagement between a group of students where ICT was
used to teach the subject, and another group, where ICT was not used, has been reported
[20], although relationships between cognitive and behavioural engagement and stu-
dent achievement were found. In both studies, behavioural engagement was a key con-
sideration; hence we focus on three conventional dimensions of student engagement,
namely individual attitude, attentiveness, and preferences and interest. Since student
diligence has also been related to academic achievement [21], it is included as a fourth
construct. We therefore define and use the term “engagement” to imply a symbiotic
143

relationship between student engagement with ICT, general engagement (i.e. attentive-
ness and diligence) and academic performance. RQ 2: To what extent does student
engagement (with ICT) have an impact on academic achievement? H2: Student engage-
ment (with ICT) has a positive impact on academic achievement.
In terms of students’ ICT use, knowledge and skills, quality, rather than quantity of
technology use, have an impact on academic achievement [22]. While some authors
have reported a direct and significant relationship between ICT literacy [12, 20, 23, 24]
and familiarity with ICT [28] and academic success, others found no impact on educa-
tional achievement [25-27]. The constructs we focus on are student use of ICT, ICT
competence and skills, exposure to ICT, and the enhancement of personal skills such
as critical thinking, motivation, self-confidence, and creativity when using ICT [29].
RQ 3: To what extent does student ICT literacy impact on academic achievement? H3:
Student ICT literacy has a positive impact on academic achievement.
Challenges associated with ICT use are numerous. The effects of anxiety on student
academic achievement has been well documented [30 – 32]. Some students may expe-
rience an almost irrational fear and stress brought on either by a lack of experience or
a deep-rooted and adverse reaction to the threat that ICT poses to the user [34]. Effec-
tive education is thought to impossible when there is no provision for ICT facilities
[33], while there is general concern about the impact of undesirable content and internet
overuse on the youth, including internet crime, copyright infringement, and security
and privacy concerns [35]. Barriers to ICT use in developing countries has been re-
ported extensively [e.g. 36 – 38]. Constructs of interest are therefore anxiety over the
use of ICT, privacy, safety and security issues, and barriers to the use of ICT. RQ4: To
what extent do challenges of ICT use have an impact on academic achievement? H4:
Challenges of ICT use have a negative impact on academic achievement.
Having formulated suitable research questions and hypotheses, we now present our
research design.
3 Research design (statement, strategy, methods and
setting)
The main purpose of the current study is to investigate the impact of ICT usage patterns
on postgraduate student academic achievement in a developing country. The philosoph-
ical base adopted is a positivist perspective, the aim to explain and predict objectively.
Given this perspective, the study adopted a quantitative research paradigm [39] that
lends itself to statistical analysis and a deductive approach. Inductive reasoning [40]
was employed to develop the hypotheses previously presented.
Clearance for the study was requested and received from the relevant ethical com-
mittees. Purposive sampling [41] was employed, in terms of which the researchers used
the literature to identify an appropriate subject pool given the objectives of the study.
The master’s class is a largely taught postgraduate three-semester (eighteen months)
program. Research Methodology (RM) is one of the courses offered for all master’s
students from nine departments at a private university in Nigeria. Several instructors
present different aspects of the course to all RM students as a group. Whereas each
144

instructor presents an aspect of the course using his/her choice of ICT, all students are
exposed to the same set of ICTs. In preparation for a final and individual research mini
dissertation to be delivered in the 3rd semester, the 1st and 2nd semesters focus on course
work. The end of 2nd semester, which incorporated formal class time, was used to col-
lect data. The classmark consisted of four instructor-assessed components: a class test
mark, an assignment mark, a research project mark, and a presentation mark.
As research instrument, we selected the survey method. Some survey questions were
sourced from previous studies on ICT use in education. To ensure relevance, we mod-
ified a few questions. Based on our own experiences with, and theoretical research sen-
sitivity about, the use of ICT in education, we constructed a few questions of our own.
The final survey instrument consisted of 77 questions subdivided into five constructs,
each containing several sub-constructs.
Construct A: Personal information, contains eight demographic questions.
Construct B: Course activities using ICT, has three sub-constructs: B1: Teaching
with the aid of ICT (7 items); B2: Presentation and discussion of classwork with ICT
(5 items); B3: Suitability of classwork for ICT integration (7 items, 2 adapted [42, 43]).
Construct C: Student engagement, has four sub-constructs: C1: Individual attitude
(6 items, 1 adapted [20]); C2: Attentiveness (7 items, 6 adapted [20]); C3: Diligence (5
items adapted [20]); C4: Preferences and interest (7 items, 3 adapted [44]).
Construct D: Student ICT literacy, contains four sub-constructs: D1: Exposure to
ICT (4 items, 3 adapted [45]); D2: Student use of ICT (5 items); D3: ICT competence
(6 items adapted [25]); D4: Personal skills (4 items);
Construct E: Challenges of ICT use, has three sub-constructs: E1: Anxiety over the
use of ICT (5 items, 4 adapted [46-48]); E2: Privacy, safety and security issues (4
items); E3: Barriers to ICT use (4 items adapted [47]).
For each item, a five-point Likert scale was employed, where 1 = strongly disagree,
2 = disagree, 3 = indifferent, 4 = agree and 5 = strongly agree.
A unique questionnaire number was made available on both the cover and first page
of each questionnaire. After the administration of the questionnaire, and on the first
page, the course instructor recorded the student’s classmark next to the questionnaire
number. Before returning the completed questionnaires to the researchers, the instructor
removed student information, thereby ensuring the anonymity of data. To explore pos-
sible relationships between the construct items and class marks achieved, multiple lin-
ear regression analysis was conducted using the Statistical Analysis System (SAS) JMP
v.12.
4 Data analysis and results
From a total of 320 questionnaires distributed to students, 302 were returned for a re-
sponse rate of 94%. Exploratory factor analysis was carried out to validate the main
constructs. In Construct B: Course activities using ICT, the initial three sub-constructs
were reduced to a single main construct with the same label. In Construct C: Student
engagement, the four sub-constructs were reduced to three (C1: Individual attitude; C2:
Diligence; C3: Preferences and interests). In Construct D: Student ICT literacy, the
145

four sub-constructs were merged into the main construct, as was the initial three sub-
constructs in Construct E: Challenges of ICT use.
To establish internal consistency, the reliability of the new constructs was measured
using Cronbach’s alpha. The following coefficients were reported: Construct B: Course
activities using ICT (0.969); Construct C: Student engagement (0.934); Construct D:
Student ICT literacy (0.967); and Construct E: Challenges of ICT use (0.963). All the
values are greater than 0.8, which is indicative of excellent results.
Demographics. Fifty-seven percent (57%) of respondents were male and 43% female.
Gender had no significant impact on the mean class mark (male = 61.1%; female =
60.1%). Further analysis revealed other interesting patterns. In the demographic section
of the questionnaire, two questions provided us with a general overview of the extent
to which ICT is used inside and outside the classroom. Here, a significant gender dif-
ference in the mean class mark in favour of males was reported when ICT is used out-
side the classroom for purposes of class work (Table 1). For both genders, the mean
class mark improved significantly the more ICT was used outside the classroom, with
the difference between rarely used and mostly used 10% for males and 8.6% for fe-
males. This result is not only in line with research reporting ICT to increase student
engagement and the amount of time that students spend working outside the classroom
[49], but reinforces a long line of research that points to the existence of a gender gap
in favour of males [42]. Of the 33 students that rarely used ICT outside the classroom
for course work, 52% were male and 48% were female. Of the 79 students that occa-
sionally used ICT, the split is also more or less equal, with 49% being male and 51%
being female. However, of the nearly 70% of students that use ICT most of the time
outside the classroom, only 38% were female. Whereas the gender difference in the
mean class mark was small, the mark increased for both genders the more ICT was used
outside the classroom.
Table 1. Gender difference: To what extent do you use ICT outside the classroom
for purposes of classwork?
Variables
Total
n
Male
n %
Male mean
class mark
Female
n %
Female
mean
class mark
Rarely
33
17
52%
57.2
16 48%
53.6
Occasionally
79
39
49%
54.8
40 51%
59.0
Most of the time
190
117
62%
63.8
73 38%
62.2
* Significant difference (p < 0.05) in mean class marks
As postgraduates, most students fell into the age groups of 21-26 and 27-32. Table 2,
with one exception, shows a marginal increase in the mean class mark across age groups
in favour of the older students. The table also reflects a higher combined mean class
mark (63.3 as opposed to 55.6 and 59.2) when instructors made use of ICT most of the
time in teaching the course. This pattern mirrors student use of ICT outside the class-
room (Table 1), where the mean class mark increased the more ICT was used by all
students for purposes of classwork.
146

Most students, both male and female, had between 1- and 7-years’ experience in
using tablets (78%) or PCs/laptops (57%), with 54% and 98% respectively having up
to and over 10 years’ experience. Overall, more males had more experience (4 years
and more) in the use of tablets and PCs/laptops. Gender differences were not as marked
for the rest of the ICT used, although males and females with no hardware and software
experience returned a lower mean class mark.
Table 2. Other demographics
Construct
Variables
Total n
% of students
Mean classmark
Age
21-26
27-32
33-38
39-44
45-50
51+
136
110
38
10
4
4
45.03%
36.42%
12.58%
3.31%
1.32%
1.32%
60.2%
60.6%
62.5%
60.7%
62.5%
64.0%
My lecturer uses
ICT in presenting
the class work*
Not at all/rarely
Occasionally
Most of the time
55
87
160
18.21%
28.81%
52.98%
55.6%
59.2%
63.3%
* Significant difference (p < 0.05) in mean class marks
Students with more than 10 years’ experience returned the highest mean class mark.
These outliers, however, consisted of only five students each, with class marks for the
rest of the students being relatively constant (59–61%).
Using decision tree analysis and a calculated median of 62% for class marks, two
patterns were readily evident. Eighty-seven percent (87%) of students (n = 110) who
used ICT most of the time outside of the classroom received a mark higher than the
median, while only 13% of students who did not make use of ICT, or made use of ICT
only occasionally, received a mark higher than the median. Similarly, when ICT was
used most of the time in teaching inside the classroom, 97% of students (n = 81) received
a mark higher than the median, while 90% received a mark lower than the median when
ICT was not used, or was used only occasionally, in teaching and learning activities.
ICT usage patterns.
Table 3 presents the correlations reported between the various constructs and class
marks.
Table 3. Construct correlation coefficients
Constructs
B
C1
C2
C3
D
E
Classmark
B: Course activities using ICT
1
C: Student engagement
C1 Individual attitude
C2 Diligence
C3 Prefs & interests
.730*
.694*
.584*
1
.630*
.599*
1
.599*
1
D: Student ICT literacy
.698*
.679*
.633*
.635*
1
E: Challenges of ICT use
-.323*
-.256*
-.222*
-.280*
-.317*
1
Class mark
.827*
.698*
.678*
.632*
.817*
-.251*
1
* Significant at p < 0.05 (Spearman’s correlation coefficients)
147

Except for Construct E: Challenges of ICT use, all constructs returned significant and
positive correlations. Construct B: Course activities using ICT (0.827) returned the
strongest correlation with class marks, which is consistent with the interpretation that
ICT is more effective when it is integrated into a classroom's teaching programme [4].
The next strongest correlations were Construct D: Student ICT literacy (0.817), fol-
lowed by student engagement sub-constructs C1: Individual attitude (0.698) and C2:
Diligence (0.678). The lowest positive correlation was reported for sub-construct C3:
Preferences and interests (0.632). Construct E: Challenges of ICT use, recorded a weak
negative correlation, which suggests that with a decrease in ICT challenges, student
academic achievement will increase marginally. This pattern held true for correlations
between this construct and the other individual constructs. Table 4 ranks the constructs
in terms of their weight.
Table 4. Construct ranking by weight
Constructs and sub constructs
Correlation coefficient
Weight
B: Course activities using ICT
0.8269
Strong
D: Student ICT literacy
C: Student engagement
0.8169
Strong
C1: Individual attitude
0.6982
Medium
C2: Diligence
0.6771
Medium
C3: Preferences and interests
0.6316
Medium
E: Challenges of ICT use
-0.2514
Weak
Regression model assumptions.
The model assumptions satisfied the linearity criteria, except for Construct E: Chal-
lenges of ICT use, which was observed to be nonlinear. To restore linearity, the con-
struct was split into two groups, namely Challenges_E1 (normal) and Challenges_E2
(quadratic), treated as two separate constructs, and included in the regression analysis
as either model 1 or model 2. This resolution resulted in hypotheses 4a and 4b.
H4a: Challenges_E1 (normal) has a negative impact on student academic achieve-
ment. H4b: Challenges_E2 (quadratic) has a negative impact on student academic
achievement.
The multicollinearity criteria were also satisfied when Construct B: Course activities
using ICT, Construct C: Student engagement and Construct D: Student ICT literacy
were treated as equally important, without distinguishing one item from the other.
Regression analysis
Tables 5 and 6 list the multiple regression coefficients reported for model 1 and model
2. Except for Challenges_E1 (normal) (Table 5), all constructs were found to be statis-
tically significant. When Challenges_E1 (normal) and Challenges_E2 (quadratic) were
removed in model 2 (Table 6), the unstandardised and standardised values of the re-
maining three variables were slightly affected, while the significant values remained
relatively unchanged.
148

From these results, and for purposes of prediction, a multiple regression equation [50]
was used in predicting class marks (Y) for model 1: Y = b0 + b1 Course activities using
ICT + b2 Student engagement + b3 Student ICT literacy + b4 Challenges of ICT use
(normal) + b5 Challenges of ICT use (quadratic).
Table 5. Multiple regression coefficients for Model 1
Constructs
Unstandardized
Coefficients
Standardized
Coefficients
t
Ratio
Sig.
B
Std. Error
Beta
(Intercept)
B: Course activities using ICT
C: Student engagement
D: Student ICT literacy
E1: Challenges of ICT use (normal)
E2: Challenges of ICT use (quadratic)
2.565
5.954
1.086
8.030
-1.578
0.389
1.984
0.383
0.436
0.388
1.039
0.186
0
0.430
0.076
0.533
-0.147
0.202
1.29
15.55
2.49
20.71
-1.52
2.10
0.1973
0.0001*
0.0133**
0.0001*
0.1296
0.0370**
* Significant at p < 0.01, ** Significant at p < 0.05. Dependent variable: Class marks
Table 6. Multiple regression coefficients for Model 2
Constructs
Unstandardized
Coefficients
Standardized
Coefficients
t
Ratio
Sig.
B
Std. Error
Beta
(Intercept)
B: Course activities using ICT
C: Student engagement
D: Student ICT literacy
0.906
5.848
1.140
8.201
1.016
0.389
0.446
0.383
0
0.422
0.080
0.545
0.89
15.03
2.56
21.44
0.3736
0.0001*
0.0111**
0.0001*
* Significant at p < 0.01, ** Significant at p < 0.05. Dependent variable: Class marks
Summary of multiple regression analysis
A summary of the multiple regression models 1 and 2 are presented in Table 7.
Table 7. Summary of multiple regression analysis (n = 302)
Construct & fit
Model 1 Unstd/Std β
Model 2 Unstd/Std β
B: Course activities using ICT
5.954 (0.430)*
5.848 (0.422)*
C: Student engagement
1.086 (0.076)**
1.140 (0.080)**
D: Student ICT literacy
8.030 (0.533)*
8.201 (0545)*
E1: Challenges of ICT use (normal)
-1.580 (-0.147)
E2: Challenges of ICT use (quadratic)
.389 (0.202)**
Summary of fit
R Squared
0.930
0.926
R Squared Adjusted
0.928
0.925
* Significant at p < 0.0; ** Significant at p < 0.05
For regression model 1, the results of the four constructs, namely Construct B: Course
activities using ICT (B = 5.954, p < 0.01), Construct C: Student engagement (B = 1.086,
p < 0.05), Construct D: Student ICT literacy (B = 8.030, p < 0.01) and Challenges_E2
(B = 0.389, p < 0.05) were significant. Challenges_E1 (B = -1.578, p > 0.05) was insig-
nificant. As indicated earlier, for regression model 2, Challenges_E1 and Challenges_E2
were removed to test for impact factor changes. Analysis of the remaining three
149

constructs for model 2, namely Construct B: Course activities using ICT (B = 5.848, p
<0.01), Construct C: Student engagement (B = 1.140, p < 0.05) and Construct D: Stu-
dent ICT literacy (B = 8.201, p < 0.01) all showed a significant positive impact on class
marks. In both models 1 and 2, Construct D: Student ICT literacy returned the strongest
positive predictive impact factor on class marks, followed by Construct B: Course ac-
tivities using ICT and, lastly, Construct C: Student engagement. The effect of the re-
moval of Challenges_E1 and Challenges_E2 in model 2 was thus marginal, and model
1 is considered appropriate. R2 was 0.930 for model 1 and 0.926 for model 2, while R2
adjusted was 0.928 for model 1 and 0.925 for model 2. The results of model 1 are sum-
marised in Table 8.
Table 8. Model 1 summary
Model
R2
Adjusted R Square
Root mean square error
1
0.930
0.928
2.938975
Predictors: Construct B: Course activities using ICT, Construct C: Student engagement, Construct D: Stu-
dent ICT literacy, Challenges_E1, Challenges_E2 Dependent Variable: Class marks
Using all the predictors together, 0.930, or 93%, of the variance in class marks for this
particular course can be predicted from Construct B: Course activities using ICT, Con-
struct C: Student engagement, Construct D: Student ICT literacy, Construct E1: Chal-
lenges_E1 (normal) and Construct E2: Challenges_E2 (quadratic).
Analysis of variance (ANOVA)
The ANOVA results presented in Table 9 were used to establish the level of signifi-
cance of model 1.
Table 9. ANOVA (Analysis of Variance)
Degree of freedom (df)
Sum of squares
Mean square
Regression model
5
33240.634
6648.13
Error
291
2513.535
8.64
Total
296
35754.168
F Ratio
769.6750
Significance F
0.0001
Predictors: Construct B: Course activities using ICT, Construct C: Student engagement, Construct D: Stu-
dent ICT literacy, Construct E1: Challenges of ICT use (normal), Construct E2: Challenges of ICT use (quad-
ratic). Dependent variable: Class marks
The result was significant and a good fit of the data with F (5,291) = 769.6750, p <
0.05, R2 = 0.930.
Research question synopsis and confirmation of hypotheses.
The results of our hypotheses testing are presented in Table 10.
RQ1: To what extent do course activities using ICT have an impact on academic
achievement? Construct B: Course activities using ICT, returned a strong positive cor-
relation with and prediction (2nd in weight (β) out of 5 predictors) of student academic
achievement. H1: Course activities using ICT have a positive impact on student
150

academic achievement was supported, since a significant positive relationship was es-
tablished with models 1 and 2 (Unstd β = 5.954 and 5.848, p < 0.01).
Table 10. Variable items and hypothesis testing results for models 1 and 2.
* Significant at p < 0.01, ** Significant at p < 0.05
RQ2: To what extent does student engagement (with ICT) have an impact on aca-
demic achievement? Construct C: Student engagement, had a moderate positive corre-
lation with class marks and returned 3rd in weight (β) out of four predictors of class
marks. H2: Student engagement (with ICT) has a positive impact on academic achieve-
ment, was supported when a significant positive relationship was established for models
1 and 2 (Unstd β = 1.086 and 1.140, p < 0.05).
RQ3: To what extent do Student ICT literacy have an impact on academic achieve-
ment? Construct D: Student ICT literacy, had a strong and significant positive correla-
tion with and prediction (1st in weight (β) out of four predictors) of class marks. Since
a significant positive relationship existed in models 1 and 2 (β = 8.030 and 8.201, p <
0.01), H3: Student ICT literacy has a positive impact on academic achievement, was
supported.
RQ4: To what extent do challenges of ICT use have an impact on academic achieve-
ment? Construct E: Challenges of ICT use, had a weak negative correlation with and
prediction (4th in weight (β) out of four predictors) of instructor-assessed class marks.
H4 were split into H4a:Challenges_E1 (normal) and H4b:Challenges_E2 (quadratic).
Neither were supported when a significant negative relationship was not established
between the two variables in model 1 Challenges_E1 (Unstd β = -1.578, p > 0.05) and
challenges_E2 (Unstd β = 0.389, p < 0.01). H4a and H4b: Challenges of ICT use have
a negative impact on academic achievement, were therefore rejected.
5 Discussion and conclusions
The broad focus of this study was on three statements as they relate to the impact of
ICT use on student academic achievement in a developing country context. A review
of the literature firstly suggested a focus on postgraduate students, and secondly as-
sisted in the development of appropriate research questions, hypotheses and a suitable
questionnaire. By using instructor-assessed class marks in the place of self-reporting,
an appropriate research environment was available. The study was guided by 4 research
questions and hypotheses. To investigate patterns of use, several main and sub-con-
structs were proposed. After data collection and exploratory factor analysis, and to
Hypothesis
Model 1
UnStd/Std β
Model 2
UnStd/Std β
Model 1
(remark)
Model 2
(remark)
H1: Course activities using ICT
5.954 (0 .430)*
5.848 (0.422)*
Supported
Supported
H2: Student engagement
1.086 ( 0.076)**
1.140 (0.080)**
Supported
Supported
H3: Student ICT literacy
8.030 ( 0.533)*
8.201 (0.545)*
Supported
Supported
H4a: Challenges of ICT use (normal)
-1.578 (-0.147)
N/A
Unsupported
N/A
H4b: Challenges of ICT use (quadratic)
.389 (0.202)**
N/A
Unsupported
N/A
151

investigate correlations with instructor-assessed class marks, the five revised main con-
structs were subjected to further analysis
In the specific setting this study reports on, Construct B: Course activities using ICT,
included sub-constructs teaching with the aid of ICT, presentation and discussion of
classwork with ICT and suitability of classwork for ICT integration. The construct re-
turned the highest overall correlation with class marks. This finding is best explained
by the significant and linear correlation between the extent to which instructors made
use of ICT in teaching the course and student achievement, which is in line with results
from several other studies [2, 4, 15, 16, 51 -54].
In terms of student use of ICT, academic achievement improved marginally but sig-
nificantly when ICT was used outside the classroom for purposes of classwork.
Whereas this effect held true for both genders, it was more marked in favour of males.
Although gender research is subjected to a range of factors that make comparisons dif-
ficult, the results nevertheless corroborate various studies that have reported an ICT
gender impact in favour of males [52, 55, 56]. In the current case, the variance that
existed can be explained by the male cohort reporting more years PC/laptop and tablet
experience, the notion that such experience plays an important role in studies at both
tertiary and postgraduate level.
In terms of the impact and nature of student engagement with ICT, results were var-
ied. The behavioural sub-constructs of Construct C: Student engagement with ICT, that
is, individual attitude towards the use of ICT and student preferences for and interests
in ICT, were found to be moderate contributors to academic achievement, with general
attentiveness excluded after exploratory factor analysis. The behavioural construct o f
general student diligence, however, was a necessary co-occurrence, and in line with
previous research results [5].
Student ICT literacy, as investigated with Construct D, considered exposure to ICT,
student use of and competence in ICT, as well as the enhancement of various personal
skills when using ICT for classwork. Combined, these sub-constructs returned the sec-
ond-highest correlation with class marks. This finding is consistent with positive results
from previous studies which investigated the effect of factors ranging from ICT use to
self-efficacy and the use of web-based education on student achievement [12, 23, 57],
but is not in line with other research which reported a negative correlation between
most student ICT activities (e.g. participating in online discussion forums, e-mailing
instructors and others, doing homework on a computer) and student academic achieve-
ment [6, 57]. The latter studies, however, were conducted at lower secondary level.
Construct E: Challenges of ICT use, which considered the sub-constructs anxiety
over the use of ICT, privacy, safety and security issues and barriers to ICT use, some-
what surprisingly revealed known ICT challenges in developing countries to have a
weak and insignificant correlation with academic achievement. At first, the result ap-
pears to be partly in agreement with research that showed internet and campus technol-
ogy to have a significant negative effect on student academic achievement [53]. How-
ever, the current research setting of a private institution, which in general offers better
ICT facilities than its public counterparts, may have allowed successful teaching with
ICT and higher student ICT literacy to negate any negative impact normally associated
with ICT use in developing countries.
152

Overall, despite differences in the factors studied and research settings, the findings
of this study support previous research [2, 4, 15, 19, 30, 32, 53, 57, 59] which found
the use of ICT to have an impact on student academic achievement. However, the find-
ings are also in contrast with several other studies that reported no impact [3, 6, 8, 9].
Have we achieved our goal of contributing to the research gaps identified? We are
confident that we have demonstrated the following: a postgraduate research setting is
ideally suited to ICT integration; the constructs employed in this study have succeeded
in describing the impact and nature of ICT use, student engagement with ICT as well
as general engagement with studies, on student performance; and by employing instruc-
tor-assessed class marks as the dependent variable, we were able to confirm positive
results from previous studies that were conducted at different levels of education, in
different subject areas and where student self-reporting on learning achievement was
utilised. Finally, given that the students in our subject pool were doing a Research
Methodology course, the results have confirmed the potential benefits of integrating
ICT in further postgraduate activities, such as the supervision of students and student
work on a dissertation [59].
References
1. Limi, P. C., Zhao Y., Tondeur J., Chair, C. S., Tsai, C.: Bridging the gap: Technology trends
and use of technology in schools. Educational Technology Society, 16(2), 59-68 (2013).
2. Angeli, C., Tsaggari, A.: Examining the effects of learning in dyads with computer-based
multimedia on third-grade students' performance in history. Computers Education, 92-93,
171-180 (2016).
3. Wainer, J., Vieira, P., Melguizo, T.: The association between having access to computers
and internet and educational achievement for primary students in Brazil. Computers & Ed-
ucation, 80, 68-76 (2015).
4. Bai, Y., Mo, D., Zhang, L., Boswell, M., Rozelle, S.: The impact of integrating ICT with
teaching: Evidence from a randomized controlled trial in rural schools in China. Computers
& Education. An International Journal, 96, 1–14 (2016).
5. Gunuc, S.: The relationships between Student Engagement and their academic achievement.
International Journal on New Trends in Education and Their Implications, 5(4), 216-231
(2014).
6. Biagi, F, Loi, M. Measuring ICT use and learning outcomes: Evidence from recent econo-
metric studies. European Journal of Education, 48 (1), 28-42 (2013).
7. Wainer, J., Dwyer, T., Dutra, R. S., Covic, A., Magalhães, V. B. Ferreira, L.R.R. Too much
computer and internet use is bad for your grades, especially if you are young and poor: Re-
sults from the 2001 Brazilian SAEB. Computers and Education, 5(4), 1417-1429 (2008).
8. Carter, S. P., Greenberg, K., Walker, M. The impact of computer usage on academic perfor-
mance: Evidence from a randomized trial at the United States Military Academy. School
Ineffectiveness and Inequality Initiative (SEII). Discussion Paper #2016.02 May. (2016).
9. Gupta, N., Garg, S., Arora, K.: Pattern of mobile phone usage and its effects on psycholog-
ical health, sleep, and academic performance in students of a medical university. National
Journal of Physiology, 6(2), 132-139 (2016).
10. Trucano, M.: Knowledge maps: ICTs in education. Washington, DC: infoDev / The World
Bank. Retrieved on August 16, 2017 from: www.infodev.org/ en/Publication.8.html.
(2005).
153

11. Gerber, C., Mans-Kemp, N. Schlechter, A.: Investigating the moderating effect of Student
Engagement on academic performance. Acta Academica, 45(4), 256-274 (2013).
12. 13. Erdogan, Y., Bayram, S., Deniz, L.: Factors that influence academic achievement and
attitudes in web-based education. International Journal of Instruction, 1 (1), 31-48 (2008).
13. Brown, C., Czerniewicz, L.: Trends in student use of ICTs in Higher Education in South
Africa. 10th Annual Conference of WWW Applications. Cape Town. 3-6 September.
(2008).
14. Ejechi, E.O.: Information communication technology: Are Nigeria’s university students us-
ing it more for social and leisure activities? International Journal of Humanities and Social
Science, 6(5), 67-73 (2016).
15. Ilomaki, L.: The effects of ICT on school: Teachers’ and students’ perspectives. Doctoral
thesis submitted to the Department of Teacher Education, University of Turku, Finland.
(2008).
16. Abu, Ziden A., Ismail, I., Spian, R., Kumutha, K.: The effects of ICT use in teaching and
learning on students’ achievement in science subject in a primary school in Malaysia. Ma-
laysia Journal of Distance Education, 13(2), 19-32 (2011).
17. Beauchamp, G., Kennewell, S.: Interactivity in the classroom and its impact on learning.
Computers Education, 54, 759-766 (2010).
18. Elsadaani, M.: Teaching staff attitude toward ICT: Is gender a factor? International Women
Online Journal of Distance Education, 1(2), 21-30 (2012).
19. Gunuc, S. Kuzu, A.: Student Engagement scale: Development, reliability and validity. As-
sessment Evaluation in Higher Education, 40(4) (2014).
20. Chenoby, H.: The role of ICT in Student Engagement in learning Mathematics in a prepar-
atory university program. A thesis submitted in fulfilment of the requirements for a master’s
degree of Science (Research). Victoria University. (2014).
21. Bernard, H. Drake, D. D., Paces, J. J., Raynor, H.: Student-centered educational reform: The
impact of parental and educator support of student diligence. School Community Journal,
6(2), 9-25 (1996).
22. Lei, J.: Quantity versus quality: A new approach to examine the relationship between tech-
nology use and student outcomes. British Journal of Educational Technology, 41 (3), 455 -
472 (2010).
23. Mohagheghzadeh, M. S, Mortazavi, S. M. J., Ghasempour, M., Jarideh, S.: The impact of
computer and information communication technology literacy on the academic achievement
of medical and dental students at Shiraz University of Medical Sciences. European Scientific
Journal, 10(9), 273-280 (2014).
24. Boyd, D.: The impact of information communication technology on student achievement.
University of British Columbia. (2012).
25. Wit, K. D., Heerwegh, D., Verhoeven, J. C.: Do ICT competences support educational at-
tainment at university? Journal of Information Technology Education, 1, 1-15 (2012).
26. Hunley, S. A., Evans, J. H., Delgado-Hachey, M., Krise, J., Rich, T., Schell, C.: Adolescent
computer use and academic achievement. Adolescence, 40(158), 307-318 (2005).
27. Geoffrey, O.: Effects of information and communication technology on students’ learning:
A case of Gulu University. A dissertation submitted in partial fulfilment of the requirements
for the degree of Master of Education, Information Communication Technology of Makerere
University. September 2010. (2010).
28. Delen, E., Bulut, O.: The relationship between students' exposure to technology and their
achievement in Science and Math. Turkish Online Journal of Educational Technology,
10(3), 311-317 (2011).
154

29. Fu, J.S.: ICT in education: A critical literature review and its implications. International
Journal of Education and Development using Information and Communication Technology
(IJEDICT), 9(1), 112-125 (2013).
30. Nadeem, M., Ali, A., Maqbool, S., Zaidi, S. U.: Impact of anxiety on the academic achieve-
ment of students having different mental abilities at university level in Bahawalpur (South-
ern Punjab) Pakistan. International Online Journal of Educational Sciences, 4(3), 519-528
(2012).
31. Lyneham, H.: The impact of anxiety on student performance. Centre for Emotional Health,
Macquarie University. (2009).
32. Singh, S.: The impact of anxiety on academic achievement of U.G. students. American
International Journal of Research in Humanities, Arts and Social Sciences, 12(2), 116-119
(2015).
33. Akhihiero, E. T.: Effect of inadequate infrastructural facilities on academic performance of
students of Oredo Local Government Area of Edo State. The Nigerian Academic Forum,
20(1), 1-6 (2011).
34. Brosnan, M., Davidson, M.: Computerphobia - is it a particularly female phenomenon? The
Psychologist, February 1994, 73-78 (1994)
35. OECD.: The economic impact of ICT: Measurement, evidence and implications. Report by
the Organisation for Economic Co-operation and Development (OECD). Retrieved on Sep-
tember 11, 2017 from: www.oced.org. (2010).
36. Khan, S. H., Hasan, M., Clement, C. K.: Barriers to the introduction of ICT into education
in developing countries: The example of Bangladesh. International Journal of Instruction,
5(2), 61-80 (2012).
37. Mndzebele, N.: Challenges faced by schools when introducing ICT in developing countries.
International Journal of Humanities and Social Science Invention, 2(9), 1-4 (2013).
38. Hennessy, S., Onguko, B., Harrison, D., Ang’ondi, E.K., Namalefe, S., Naseem A., Wama-
kote, L.: Developing use of ICT to enhance teaching and learning in East African schools:
A review of the literature. Centre for Commonwealth Education and Aga Khan University
Institute for Educational Development – Eastern Africa Research Report No. 1. (2010).
39. Miles, M., Huberman, A.: Qualitative data analysis: An expanded source book (2nd edition),
Thousand Oakes, Sage. (1994).
40. Dudoskiy, J.: The Ultimate Guide to Writing a Dissertation in Business Studies: A Step-by-
Step Assistance. E-book. Retrieved on May 15, 2017 from: research-methodol-
ogy.net/about-us/ebook/. (2016).
41. Black, K.: Business Statistics: Contemporary Decision Making (6th ed.), John Wiley Sons.
(2010).
42. Joseph, H., Gapsiso, N.D.: Overcoming gender inequalities through technology integration.
IGI Global. (2016).
43. Dhanarajan, G.: Objectives and strategies for effective use of ICTs. Technologies for Edu-
cation: Potentials, Parameters and Prospects. Edited by Wadi D. Haddad Alexandra Draxler.
Prepared for UNESCO by Knowledge Enterprise, Inc. (2002).
44. Kee, C. L., Samsudin, Z.: Mobile devices: Toys or learning tools for the 21st century teen-
agers? The Turkish Online Journal of Educational Technology, 13(3), 85-100 (2014).
45. Passey, D., Rogers, C., Machell, J. McHugh, G.: The motivational effect of ICT on pupils.
Department of Educational Research Lancaster University. Research Report 523 (2004).
46. Akbar, F.: What affects students’ acceptance and use of technology? A test of UTAUT in
the context of a higher-education institution in Qatar. Senior Honors Thesis. Information
Systems, Dietrich College, Carnegie Mellon University. (2013).
155

47. Adomi, E. E. Kpangban, E.: Application of ICTs in Nigerian secondary schools. Library
Philosophy and Practice (e-journal). University of Nebraska – Lincoln. Libraries at Univer-
sity of Nebraska-Lincoln. (2010).
48. Sam, H. K., Othman, A. E. A., Nordin, Z. S.: Computer self-efficacy, computer anxiety, and
attitudes toward the internet: A study among undergraduates in Unimas. Educational Tech-
nology Society, 8(4), 205-219 (2005).
49. Becker, H. J.: Pedagogical motivations for student computer use that leads to Student En-
gagement. Education Technology, 40(5), 5-17 (2000).
50. Stockburger, D.W.: Multivariate Statistics: Concepts, models and applications. WWW Ver-
sion 1.0. 1st Published July 1997. Revised March 1998. Missouri State University. (1998).
51. Comi, S.L., Argentin, G., Gui, M., Origo, F., Pagani, L.: Is it the way they use it? Teachers,
ICT and student achievement. Economics of Education Review, 56, 24-39 (2017).
52. Becta.: How do boys and girls differ in their use of ICTs? Research report: Learning Next
Generation Learning. Retrieved on February 23, 2017 from: dera.ioe.ac.uk/8318/1/ gen-
der_ict_briefing.pdf. (2008).
53. Rugutt, J. K., Chemosit, C.C.: A study of factors that influence college academic achieve-
ment: A structural equation modelling approach. Journal of Educational Research Policy
Studies, 5(1), 66-89 (2005).
54. Dania, P.O.: Effect of gender on students’ academic achievement in secondary school Social
Studies. Journal of Education and Practice, 5 (21), 78-84 (2014).
55. Unity, O., Igbudu, U.: Influence of Gender on Students’ Academic Achievement in Govern-
ment Subject in Public Secondary Schools in Oredo Local Government Area of Edo State,
Nigeria. Journal of Educational and Social Research, 5(2), 101-105 (2015).
56. Rohatgi, A., Scherer, R., Hatlevik, O.E.: The role of ICT self-efficacy for students' ICT use
and their achievement in a computer and information literacy test. Computers Education,
100, 103-116 (2016).
57. Nisar, M. W., Munir, E. U., Shad, S.A.: Usage and impact of ICT in education sector: A
study of Pakistan. Australian Journal of Basic and Applied Sciences, 5(12): 578-583 (2011).
58. Maor, D., Currie, J. K.: The use of technology in postgraduate supervision pedagogy in two
Australian universities. International Journal of Educational Technology in Higher Educa-
tion, 14(1), 1-15 (2017).
156