Information Literacy Skills Assessment of Undergraduate Engineering Students

Abstract

This study assessed Information Literacy (IL) skills of first year undergraduate engineering students of a Pakistani university in order to plan instruction; and to provide the university with reliable data for integration of instruction in the university curricula. Independent samples t-test and ANOVA were applied to check gender and program based differences in IL skills of students. No gender or program-based difference in IL skills of students was found. It was also revealed that students did not possess IL skills good enough to help them in their university level studies. They do not have a fair idea about methods and tools to be used to retrieve required information from various resources. This low level of IL skills shows the need for an IL program properly planned, designed and carried out in a consistent manner.

Full text

S. Kurbanoğlu et al. (Eds.): ECIL 2013, CCIS 397, pp. 471–477, 2013.
© Springer International Publishing Switzerland 2013
Information Literacy Skills Assessment
of Undergraduate Engineering Students
Mamoona Kousar1 and Khalid Mahmood2
1 Air University, Islamabad, Pakistan
librarian@mail.au.edu.pk
2 Department of Library and Information Science,
University of the Punjab, Lahore, Pakistan
khalid.dlis@pu.edu.pk
Abstract. This study assessed Information Literacy (IL) skills of first year
undergraduate engineering students of a Pakistani university in order to plan
instruction; and to provide the university with reliable data for integration of
instruction in the university curricula. Independent samples t-test and ANOVA
were applied to check gender and program based differences in IL skills of
students. No gender or program-based difference in IL skills of students was
found. It was also revealed that students did not possess IL skills good enough
to help them in their university level studies. They do not have a fair idea about
methods and tools to be used to retrieve required information from various
resources. This low level of IL skills shows the need for an IL program properly
planned, designed and carried out in a consistent manner.
Keywords: Information literacy, Pakistan, undergraduate engineering, need
assessment.
1 Introduction
Exponential growth in the production of information and fast paced developments in
information and communication technology (ICT) have highlighted the importance of
efficient use of information especially at the university level. To be efficient users of
information students need to receive proper education for information use, which is
most effectively learned through information literacy instruction (ILI) programs. To
plan any such instruction program for students, first of all, it is useful to assess their
current level of information literacy skills. Such assessments can be used to help to
design and deliver effective ILI programs. In Pakistan any effort to directly assess
information literacy level of undergraduate students has never been done. In this way
it is the first effort to measure current IL skills of Pakistani students.
2 Aims and Objectives
This study was designed to address the following research questions regarding
undergraduate engineering students of the Air University:

472 M. Kousar and K. Mahmood
1. What is the level of these students’ various information literacy skills?
2. What is the difference in students’ information literacy skills based on gender and
program of study?
The following research hypotheses were formulated for conducting statistical tests:
• H1: Most of the students don’t have enough information skills to become good
information users. (It was assumed that the participant students did not have any
formal IL instruction in their secondary schools and intermediate colleges.
Therefore, without a proper training, they were not expected to pass most of the
test items with a 50 percent passing score. They were expected to have poor IL
skills.)
• H2: There is no statistically significant difference between IL skills of students
based on their gender. (The lack of IL instruction was similar in the schools and
colleges for boys and girls. Therefore, no gender based difference in IL skills was
expected.)
• H3: There is no statistically significant difference between IL skills of students
based on their program of study. (The students with the same school background
joined different engineering programs in the university. Therefore, no program
based difference in IL skills was expected.)
3 Literature Review
Almost all over the world the term “Bibliographic Instructions” has been replaced by
“Information Literacy”. The American Library Association (ALA) recognized this
term in 1989 [1] and stated that “to be information literate a person must be able to
recognize when information is needed and have the ability to locate, evaluate and use
effectively the needed information.” In 2000 the Association of College and Research
Libraries (ACRL) adopted this definition in its publication “Information literacy
competency standards for higher education: Standards, performance indicators and
outcomes” [2].
Literature is available on the importance of education of information users and
advocating information literacy instruction. It is widely recognized that the
information explosion of today requires an increasingly critical approach to
information searching. Therefore, information related education is a responsibility
that universities can no longer ignore. Today’s student must be given tools for
tomorrow’s information survival. Library instruction should enable them to acquire
new work procedures allowing them to become more autonomous. The work
environment will undoubtedly be changed very quickly and students will need to
continue learning after they complete their university education. Knowing how to
find, evaluate and use information will be a major asset in their career. Pochet and
Thirion state that library instruction must not be limited to simply obtaining
information but the subsequent steps must be taught such as: how to understand and
critically evaluate the information to ascertain whether it fills the initial need; how to

Information Literacy Skills Assessment of Undergraduate Engineering Students 473
use, manage and integrate information into ones’ knowledge base; and how to
reformulate the information need, taking into account the new knowledge [3].
Despite all this importance given to the education of information and its place in
post-secondary education, information specialists and librarians have long observed
major gaps in the information skills of students. Especially those entering the
university demonstrate very low level of IL skills.
Keeping in view this situation, a study was carried out in the Quebec province of
Canada. The research objective was to determine whether students entering their first
year undergraduate program were equipped with skills necessary to retrieve, process
and evaluate information. The ACRL IL Competency Standards for Higher Education
were used as the basis of this study and research skills considered essential to the
success of information search process were identified. A test consisting of 20
multiple-choice questions was applied to over 3,000 students of 15 universities. The
study demonstrated gaps in IL skills of students [4].
While literature is overwhelmingly focused on the importance and need of
information literacy programs, lists of competencies, programs’ contents and
evaluation, there is paucity of actual assessment tools to measure the competencies of
students. Researchers have developed some tools to measure the students’ knowledge
for any such program but these are either limited to a specific library system or a
database or are supposed to determine effectiveness of a library instruction program
already given. Such tools assess the level of effectiveness of an ongoing IL program.
But at UCLA instead of assessing effectiveness of an existing IL program a study was
carried out to measure IL competencies of students before participating in any such
program. A survey instrument was specifically created for this study and administered
to undergraduate students to measure how much skillful or knowledgeable they
were [5].
In their effort to develop a standardized instrument for institutional and
longitudinal measurement of IL competencies O’Connor, Radcliff and Gedeon [6]
reported eight studies which used “paper and pencil” test to assess IL skills. Other
examples which conducted achievement tests for this purpose include Ali, Abu-
Hassan, Daud, and Jusoff [7] who used an adapted version of Mittermeyer’s
questionnaire to assess IL skills of engineering students in Malaysia. Rehman and
Mohammad evaluated library and information skills of undergraduate students in
Kuwait University [8]. Thirion and Pochet assessed IL skills of university students in
the French speaking community in Belgium [9]. Swoger included multiple choice and
short answer questions in his IL assessment questionnaire at SUNY Geneseo,
USA [10].
In Pakistan, literature has failed to report use of any achievement test to assess IL
skills of students. In a 2010 survey of the university libraries in Pakistan, Bhatti found
that most of them offer ill-planned and informal user education programs without an
assessment of user needs [11]. Based on some studies conducted at the University of
the Punjab on the use of online databases and digital libraries Ameen and Gorman
[12] inferred that the use of these resources was low for a variety of reasons. They
claimed that the community was “unable to use and/or unaware of the databases and
digital services available through the HEC, and this state of information and digital

474 M. Kousar and K. Mahmood
illiteracy is responsible for a significant loss of resources”. According to Bhatti,
inadequate assessment of library users’ needs and information seeking behavior was
among the inhibiting factors for successful implementation of IL instruction programs
in Pakistani universities [13].
4 Method
The Air University offers undergraduate programs in electrical, mechatronics,
mechanical, and computer engineering. When data was collected, the second semester
student population for these disciplines totaled 475.Through cluster sampling 151
(31percent) students, making one entire section from each program, was selected.
Employing a survey method and anonymity principle, a questionnaire was distributed
and completed during class sessions. This technique was selected for a prompt
response, and to eliminate the need for a reminder. There was a100 percent response
rate of all students who were available during class time. For data collection the
instrument developed by Mittermeyer [5] was used with slight variation. A pilot study
was conducted to check deficiencies, omissions, and ideas for improvements, to
calculate the time required to complete the questionnaire, and to identify the level of
difficulty of questionnaire for the students. Minor changes were made accordingly.
Time duration to complete the questionnaire was calculated as 20 minutes. The
response rate was 100 percent.
5 Results and Discussion
The first two demographic questions were about gender and participants’ study
program. Table 1, shows 111 (73.5 percent) were male. The program distribution
shows that maximum participation was from both Electrical and Mechanical
Engineering departments (26.5 percent students from each department).
Table 1. Gender and program of study wise frequency distribution of participants
Variable n %
Gender
Male 111 73.5
Female 40 26.5
Program of study
Bachelor of Computer Engineering 34 22.5
Bachelor of Electrical Engineering 40 26.5
Bachelor of Mechatronics Engineering 37 24.5
Bachelor of Mechanical Engineering 40 26.5
The next 19 questions were directly related to the assessment of IL competencies
of the students. Each correct answer was assigned one mark and only 14 students (9.3
percent) got 10 marks or more. More than 60 percent students got only five or less
marks. This very low level of IL knowledge leads to the acceptance of first hypothesis

Information Literacy Skills Assessment of Undergraduate Engineering Students 475
(H1). This result is similar with that of the previous studies of students’ level of IL
skills conducted in other countries.
The questions were grouped together under five themes. Table 2 shows frequency
of the correct answers. The analysis shows that the percentage of correct answers over
50 occurred only for three questions. The highest frequency for the correct answer
was for a question on choosing the right search strategy to find all documents by a
specific given author in a library catalog. Identification of synonyms or related terms
used to represent a subject is an important component of search strategy. Better ability
in this regard improves retrieval of relevant results. To determine the students’ ability
to correspond to the words used to describe their topic to those employed by the
selected search tool a question was asked. The respondents also showed good
response on it (105, 69.5 percent).
Table 2. Frequency distribution of IL related questions
Rank Question Theme
Frequency
of correct
answer %
1 Search indexes Search strategy 111 73.5
2 Translation into keywords Search strategy 105 69.5
3 Significant words Concept
identification
89
58.9
4 Significant words Concept
identification
68 45.0
5 Meta search engines Search tools 64 42.4
6 Encyclopedias Document type 62 41.1
7 Significant words Concept
identification
57 37.7
8 Reading citations Use of results 56 37.1
9 Bibliographies Use of results 51 33.8
10 Periodicals Document type 46 30.5
11 Search engines Search tools 45 29.8
12 Boolean operator “AND” Search strategy 36 23.8
13 Boolean operator “OR” Search strategy 30 19.9
14 Library catalogs Search tools 24 15.9
15 Scholarly journals Document type 21 13.9
15 Library catalogs Search tools 21 13.9
17 Controlled vocabulary Search strategy 17 11.3
18 Evaluation of information on
Internet
Use of results 16
10.6
19 Ethical use of information Use of results 15 9.9
Three questions (concept identification theme) asked to examine how students
select concepts in their search strategy. Are they able to distinguish between
significant and non-significant terms? Also, are they able to distance themselves from
the formulation used in the statement of the problem or they hold to its wording. Only
one out of these three questions crossed the success rate of 50 percent.
The results show that the lowest frequency for correct answer was for a question on
the ethical use of information (under ‘use of results’ theme). The students had little

476 M. Kousar and K. Mahmood
idea about including a reference to the source of information in their own work. Only
15 (9.9 percent) respondents were able to identify the situations when they should
include a reference to the source of information in their work. This poor knowledge
about the ethical use of information logically results in high rate of plagiarism.
Students were also checked to see if they knew which characteristics should be
used to evaluate the quality of an Internet site. Surprisingly results were not good.
They were unable to identify all the enlisted characteristics necessary to evaluate the
quality of an Internet site. Only 16 (10.6 percent) students provided the correct
answer. Students also had very little idea about controlled vocabulary. Only 17 (11.3
percent) students were able to understand that the thesaurus should be used to identify
terms.
Independent samples t-test was run on scores of the students on each theme and the
total score. Table 3 reveals that second hypothesis was also accepted. It was found
that IL skills of male and female were at the same level.
Table 3. Results of independent samples t-test for gender based IL skills of students
Mean
t Sig.
Themes Male Female
Concept identification 1.32 1.68 -1.90 .060
Search strategy 2.05 1.78 1.43 .155
Document type 0.86 0.83 0.27 .787
Search tools 1.07 0.88 1.46 .141
Use of results 0.88 1.00 -0.65 .519
Total 6.20 6.15 0.11 .910
The third hypothesis was tested through one-way analysis of variance (ANOVA).
The mean scores of the students from four different programs of study were different
but the difference among them was not found significant. Table 4 shows no program
based statistically significant difference between the IL skills of students.
Table 4. Results of one-way ANOVA for program wise IL skills of students
Programs of study Mean SD F Sig.
Bachelor of Computer Engineering 6.79 1.82 2.199 .091
Bachelor of Electrical Engineering 5.48 2.09
Bachelor of Mechatronic Engineering 6.19 2.21
Bachelor of Mechanical Engineering 6.38 2.81
6 Conclusion
The results of this study indicate that the first-year students of undergraduate
engineering programs lack a good understanding of information resources. They do not
have a fair idea about methods and tools to be used to retrieve required information
from these resources. The level of information literacy skills of these students was very
low, and this needs to be improved to help them perform better in their academic
pursuits. In this regard it is required to provide them information literacy instructions,

Information Literacy Skills Assessment of Undergraduate Engineering Students 477
properly planned, designed and carried out in a consistent manner, which is only
possible with the support of university management. Keeping in view the results of this
study, it can be assumed that the students in other Pakistani universities also have
insufficient IL skills. It may be verified by conducting similar studies in other settings.
A national level movement for the institution of IL programs is warranted in Pakistan.
The results of this study have practical implications for many stake holders such as
federal and provincial governments, university authorities, the Higher Education
Commission, the federal Department of Libraries, LIS researchers, LIS schools, library
technology vendors and professional library associations.
References
1. American Library Association. Presidential Committee on Information Literacy: Final
report (1989), http://www.ala.org/ala/acrl/acrlpubs/whitepapers/
presidential.cfm
2. Association of College and Research Libraries: Information literacy Competency
Standards for Higher Education: Standards, Performance Indicators and Outcomes. ALA,
Chicago (2000), http://www.ala.org/ala/acrl/acrlstandards/
informationliteracycompetency.htm
3. Pochet, B., Thirion, P.: Formation Documentaire et Projets Pédagogiques. Bulletin des
Bibliothèques de France 44(1), 16–22 (1999)
4. Mittermeyer, D.: Incoming First Year Undergraduate Students: How Information Literate
are They? Education for Information 23, 203–232 (2005)
5. Caravello, P.S., Borah, E.G., Herschman, J., Mitchell, E.: UCLA Library - Information
Competence at UCLA: Report of a Survey Project (2001),
http://www.library.ucla.edu/infocompetence/
index_noframes.htm
6. O’Connor, L.G., Radcliff, C.J., Gedeon, J.A.: Assessing Information Literacy Skills:
Developing a Standardized Instrument for Institutional and Longitudinal Measurement
(2001), http://www.ala.org/acrl/papers01/oconnor.pdf
7. Ali, R., Abu-Hassan, N., Daud, M.Y.M., Jusoff, K.: Information Literacy Skills of
Engineering Students. International Journal of Research & Reviews in Applied
Sciences 5(3), 264–270 (2010)
8. Rehman, S.U., Mohammad, G.: Relationship of Library Skills with Selected Personal and
Academic Variables: A Study of the Undergraduate Students of Kuwait University. The
International Information & Library Review 34(1), 1–20 (2001)
9. Thirion, P., Pochet, B.: Information Literacy in Students Entering Higher Education in the
French Speaking Community of Belgium: Lessons Learned from an Evaluation. IFLA
Journal 35(2), 152–170 (2009)
10. Swoger, B.J.M.: Closing the Assessment Loop Using Pre- and Post-assessment. Reference
Services Review 39(2), 244–259 (2011)
11. Bhatti, R.: An Evaluation of User-education Programmes in the University Libraries of
Pakistan. Library Philosophy and Practice (2010)
12. Ameen, K., Gorman, G.E.: Information and Digital Literacy: A Stumbling Block to
Development? A Pakistan Perspective. Library Management 30, 99–112 (2009)
13. Bhatti, R.: Information Literacy: Furthering the Cause of Higher Education in Pakistan.
Pakistan Library & Information Science Journal 43(1), 3–11 (2012)