Developing Web literacy in collaborative inquiry activities.

Full text

Page 1

Author's personal copy
Developing Web literacy in collaborative inquiry activities
Els Kuiper*, Monique Volman, Jan Terwel
Vrije Universiteit Amsterdam, Dept. of Theory and Research in Education, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands
a r t i c l ei n f o
Article history:
Received 13 November 2006
Received in revised form 5 November 2008
Accepted 17 November 2008
Keywords:
Media in education
Elementary education
Cooperative/collaborative learning
Pedagogical issues
a b s t r a c t
Although many children are technically skilled in using the Web, their competences to use it in a critical
and meaningful way are usually less well developed. In this article, we report on a multiple case study
focusing on the possibilities and limitations of collaborative inquiry activities as an appropriate context
to acquire Web literacy skills in primary education. Four 5th grade school teachers and their students
worked with collaborative inquiry activities on the subject of ‘healthy food’. The project was aimed at
both the development of Web literacy skills and content knowledge building. Data from a variety of
sources were collected: videotaped and written lesson observations, interviews with teachers and stu-
dents, teacher diaries, student questionnaires, and student assignments. The teachers appeared to be able
to carry out the program to varying degrees. Contextual factors that influenced the realization of the pro-
ject’s goals and results were the adequacy of the research questions formulated by students, students’
inquiry skills, and the teachers’ teaching styles. Students’ learning results show that it is possible to teach
Web literacy skills in the context of collaborative inquiry activities. All classes show knowledge gain with
regard to the subject healthy food and all classes but one show knowledge gain with regard to Web lit-
eracy skills. Although many students show adequate use of particular Web searching, reading and eval-
uating skills after the project, inconsistency, impulsiveness and impatience are also typical of their Web
behaviour. In the context of collaborative inquiry activities teachers are challenged to deal with the par-
adox that they want their students to be active knowledge builders with help of the Web, whereas the
Web seems to invite students to be more or less passive searchers.
? 2008 Elsevier Ltd. All rights reserved.
1. Introduction
Children growing up in current Western societies probably cannot imagine life without the Internet. Most children have the Internet at
their disposal either at school or at home or at other out-of-school locations such as libraries. The Web, in particular, is almost omnipresent
in their lives, and children seem to use it quite naturally for gaming, for instant messaging and downloading their favourite music. They
also become Web authors themselves by constructing their own websites. Children often perceive the Web as a user-friendly resource be-
cause it contains a great deal of information as well as pictures that can be downloaded and relocated for their own purposes. At school,
many students use the Web as an information resource.
The Web certainly has considerable potential as a supplement to the information resources traditionally used in the classroom. How-
ever, it has not been designed for use by children, nor for use in educational settings. Problems with the validity, thoroughness and mean-
ing of knowledge acquired through the Web have been pointed out from the outset (e.g., the ‘butterfly defect’, Salomon, 1998). The Web as
an information resource can only play an adequate role in students’ learning under certain conditions (Kuiper, Volman, & Terwel, 2005).
Research shows that, although children may be confident Web users, they often do not have sufficient Web skills at their disposal. Most
children lack adequate Web searching skills, as well as skills to process and critically evaluate Web information (Fidel et al., 1999; Loren-
zen, 2001; Pritchard & Cartwright, 2004; Schacter, Chung, & Dorr, 1998; Shenton & Dixon, 2003). As a result searching for information usu-
ally leads to insufficient knowledge and understanding. Moreover, studies that focus on older aged (college) students’ Web behaviour also
show a lack of critical Web skills (e.g., Metzger, Flanagin, & Zwarun, 2003). Several authors have observed that the use of the Web requires
the development of new literacies (e.g., Snyder, 2002) as well as Web literacy curricula (Metzger et al., 2003). Pleas have been made for
future research that focuses on the design of learning environments that incorporate the use of the Web into inquiry activities in which
Web literacy skills such as searching for, reading and evaluating information are not ends in themselves but means to answering queries
(e.g., Frechette, 2002).
0360-1315/$ - see front matter ? 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.compedu.2008.11.010
* Corresponding author. Tel.: +31 20 5988924.
E-mail address: ej.kuiper@psy.vu.nl (E. Kuiper).
Computers & Education 52 (2009) 668–680
Contents lists available at ScienceDirect
Computers & Education
journal homepage: www.elsevier.com/locate/compedu

Page 2

Author's personal copy
In this article we present an evaluation of a program in which we collaborated with four 5th grade teachers (students aged 10–11 years)
in order to translate these recommendations into school practices. We developed the outlines of a 10-weeks inquiry unit for students in
which the acquisition of Web literacy skills is targeted in the context of collaborative inquiry activities on the subject ‘healthy food’.1In our
analysis we focus both on the ways teachers and students worked with the program, as well as on the results of the program in terms of
content knowledge and Web literacy skills. By looking closely at the different contexts in which the program was implemented, we wanted
to gain insight into the contextual factors that influence the realization of such a program.
2. Theoretical framework
2.1. Theoretical perspectives
Conditions for the use of the Web as a source of information in educational contexts have been the subject of research in various, mostly
unconnected, bodies of literature, under different theoretical perspectives.
Scholars in critical pedagogy and reading theorists have analysed the Web as a new social tool or new type of text with specific char-
acteristics, requiring new knowledge, skills and attitudes from its users (e.g., Burbules & Callister, 2000; Coiro, 2003; Frechette, 2002; Leu,
Kinzer, Coiro, & Cammack, 2004; Snyder, 1998). The size and topicality of the Web implies that students must have good searching skills in
order to find what they are looking for. They also must be capable of making decisions on their own information needs. The Web’s acces-
sibility implies that children have to become aware of its downsides and learn to cope with such negative aspects. The use of hypertext
makes high demands on critical reading skills. Children have to learn to ask themselves constantly why they are reading a Web text. Assess-
ing the reliability of Web texts plays an important part in this process. Finally, the visual character of the Web makes it necessary for stu-
dents to learn to ‘read’ such visual information and to learn to judge the visual information on functionality and meaning, and how it relates
to the text. All these points highlight the importance of ‘new literacies’ that include ‘‘ [...] the skills, strategies, and dispositions necessary
to successfully use and adapt to the rapidly changing information and communication technologies and contexts that continuously emerge
in our world [...]” (Leu et al., 2004, p. 1572).
Researchers in the field of Library and Information Science (LIS) have studied the Web as a new source of information, in addition to
printed information. This type of empirical research studies the characteristics of children’s Web search behaviour: how do they search
for information on the Web, what problems do they encounter and what are the results of their searches (e.g., Bilal, 2000; Fidel et al.,
1999; Shenton & Dixon, 2003). The results of these studies show a picture of children as technically skilled, but rather naïve Web users.
Most children lack sufficient search skills as well as skills in the effective use of the information they find. Moreover, children rarely look
at the reliability and authority of the information they find. They either do not wonder about the Web’s authority, or do not know how to
evaluate the Web on this point. In terms of this information oriented discipline, all this points to the importance of information literacy
skills: the ability to recognize when information is needed and the ability to locate, evaluate and use the needed information effectively
(American Library Association, 1989).
Finally, educational researchers have explored the use of the Web as a new type of learning tool. They are primarily interested in how
information retrieved through Web search processes can contribute to the construction of knowledge. Most of the research is concentrated
on the science domain and attempts to incorporate the use of the Web information into students’ inquiry activities (e.g., Jones, 2002; Wal-
lace, Kupperman, Krajcik, & Soloway, 2000). The Web offers a valuable supplement to the resources traditionally used in science education,
offering primary, current and interactive material that cannot be found in any textbook. However, almost 10 years ago, Soloway and Wal-
lace (1997) already pointed at the limitations of the use of the Web in science education. They stated that ‘‘[...] given the current tools,
organization, and content on the web, [...] searching the web may well not be an educationally productive activity” (p. 11). They question
the usability of the Web for inquiry activities: the Web invites students to search, which differs from the activity of research, which asks for
repeated searching, and for the construction and synthesis of the information found. Most of the ensuing educational research on Web use
has tried to solve this problem by simplifying the use of the Web for students, for example by supporting them with an interface that offers
some solutions for their lack of search skills (e.g., Hoffman, Wu, Krajcik, & Soloway, 2003; Wallace et al., 2000). This is also noticeable in the
many ‘good practices’ found on the Web that offer teachers selections of websites on a certain domain or subject. Students can focus their
attention on pre-selected websites with useful information and do not have to spend time on endless Web searches.
This solution is understandable when one uses the Web primarily for offering information that can contribute to the construction of
knowledge in a certain subject domain. However, such a solution is neither conducive to, nor appropriate to foster students’ independent
and critical use of the Web (Burbules & Callister, 2000). In this respect Frechette (2002) mentions the importance of empowerment: by
teaching students to deal critically with the form and content of the Web, they will have a greater say in their own knowledge construction.
In light of this, we would advocate an approach whereby the Web is not only seen as a tool in education, but also as a tool that students
must learn to master. In other words, the ability to handle the Web critically should be an explicit educational goal. We will refer to such an
ability as ‘Web literacy’.
2.2. Components of Web literacy
Web literacy may be seen as an umbrella term that comprises a combination of various skills regarding the critical use of the Web for
one’s own purposes. Reading researchers and the above-mentioned LIS researchers, in particular, have worked out the principal compo-
nents of Web literacy (e.g., Burke, 2002; Coiro, 2003; Enochsson, 2005; Sutherland-Smith, 2002). In our view, three principal subcategories
of skills can be distinguished: Web searching skills, Web reading skills and Web evaluating skills. Searching skills, for example, include the
ability to define appropriate key words and locate relevant information as well as knowledge of the different ways to locate Web informa-
tion, and their possibilities and limitations. Reading skills can be understood as the ability to use the results of search engines like Google;
1This program was part of a research project in which we developed and evaluated two different ways of acquiring Web literacy skills. One of these programs used a rather
structured approach and was aimed at schools using traditional teaching methods (Kuiper, Volman, & Terwel, 2008a). The other program is the focus of this article.
E. Kuiper et al./Computers & Education 52 (2009) 668–680
669

Page 3

Author's personal copy
handling all elements of hypertext, e.g. the function of Web page links; assimilating great amounts of diverse information; and knowing
what information to use and explore and what to ignore. Evaluating and processing skills denote the ability to assess the reliability and
authority of Web information; knowing how to relate text and images on Web pages; and using Web information for one’s own informa-
tion need. Although for purposes of research it may both be valuable and necessary to distinguish between such categories of sub skills, we
wish to emphasise that Web literacy is not simply a sum of its parts and that it cannot be reduced to sub skills. Also, it is impossible for the
three categories to be strictly separated; rather, they overlap and are mutually connected. For example, good searching skills invariably
involve some reading and evaluating skills. Moreover, use of the Web implies that searching, reading and evaluating are interwoven
and constantly go together. In fact, the conclusion is warranted that the Web demands flexible use of these skills in particular.
In our design of a curriculum for acquiring Web literacy, we started from a socio-cultural approach. According to socio-cultural theories
of learning, learning is enhanced by students’ participation in ‘real world’ activities that are meaningful and challenging to them (Lave &
Wenger, 1991). On the Web, students can easily and independently investigate questions. From this perspective, actual use of the Web is
also a meaningful way for acquiring Web literacy. It confronts students with the problems involved, such as the abundance of information
and the temptation of endless browsing through hypertexts. However, for learning to take place, reflection on the problems involved in
Web use and support in dealing with these problems are also necessary. It is precisely reflection that education can add over and above
participation in real-life social practices (Ten Dam, Volman, & Wardekker, 2004). In learning environments, designed form a socio-cultural
perspective, this combination of participation and reflection is often organised in communities of inquiry (Wells, 2000). Such communities
are engaged in collaborative group work, inquiry activities based on students’ own research questions and knowledge building through
dialogue between teachers and students. We believe this to be a promising approach to making the development of critical attitudes to-
ward Web information into a meaningful curricular goal. This perspective fits in well with pleas by reading researchers and scholars in
critical pedagogy to embed Web literacy skills in inquiry activities (e.g., Frechette, 2002). Although many Web-based ‘good practices’ exist
which focus on either the acquisition of Web skills or inquiry activities, there is hardly any empirical research that focuses explicitly on the
acquisition of Web skills within inquiry activities, especially with regard to the primary school ages.
Therefore, in this study, we have taken a first step towards putting this approach into practice. We designed, tried out and evaluated a
program/project inwhichthe acquisitionof Webliteracyskillswasembedded inthe contextofcollaborativeinquiryactivitieson the subject
of healthy food. Our overall goal was to gain insight into the possibilities and limitations of collaborative inquiry activities as an appropriate
context in which to acquire Web literacy skills. Our research, being explorative in nature, was guided by the following research questions:
? How do the teachers and students give shape to the program and how do contextual factors influence the realization of the program?
? What are the learning results of the program in the participating classes in terms of both content knowledge and Web literacy skills?
3. Methods
3.1. Participants and setting
Four 5th grade teachers (three male, one female) and in total 93 students from four different schools participated (51 male, 42 female;
mean age 10.6 years). The four participating classes differed in aspects as class size, teacher experience, and the students’ socio-economic
background and their reading ability. All schools had good ict facilities and both teachers and students were used to work with the Web as
well as with other ict-applications. The four participating teachers were familiar with working with collaborative inquiry activities and
with incorporating the Web into these inquiries. They were all convinced of the necessity of teaching students Web searching, reading
and evaluating skills. Although most of their students were used to use the Web both at home and at school for inquiry activities like class
projects and individual papers, the teachers shared the opinion that they did not have enough Web skills at their disposal. All teachers
questioned the knowledge and insight students acquired through using the Web as an information resource, although they were able to
produce good-looking and often quite impressive papers which contained a lot of information.
With regard to the participating classes, there were no significant differences between the classes with respect to Internet connection at
home, the time students spent at the computer at home, self-reported Web skills, their preference for either books or the Web as informa-
tion resource for school assignments, main language spoken at home and the parents’ native country.
Kingsley School2is situated in a newly built suburb of a big city. Its student population is mostly from a white, upper middle class back-
ground. Since its foundation, the school practices socio-cultural views on education. In the upper grades, project work and inquiry activities of
all sorts are very common for both teachers and students. In the school’s policy, ICT plays a significant part: the computer facilities are excel-
lent and the school has a full-time ICT coordinator who assists teachers in the technical and didactical aspects of the use of ICT in their teach-
ing. Thomas, the teacher, is relatively inexperienced in the technical aspects of ICT and in using technical devices such as computer projectors.
The class had 26 students (10 girls and 16 boys). The students’ average reading comprehension score is 51.8 which is above the national aver-
age of 49.0.3There is, however, much variation between students.
Langdon School is located in an old neighbourhood of a big city. Its student population is mixed, but mostly from a white, middle or
upper class background; a minority of the students are from other origins (Surinam, Turkish or Moroccan). The school participates actively
in a network of schools, all working according to socio-cultural views on education. The lower grades are used to working in that way; the
teachers of the upper grades are still looking for ways to combine project work and inquiry activities with the growing pressure to rely on
testing and marking. The school has good ICT facilities which are much used by all teachers. Jim, the teacher of the participating class of 24
students (13 girls, 9 boys), works half-time at the school. The students’ test scores on reading comprehension are 57.4 which is far above
the national average. In spite of the class’ potential, the teacher finds it sometimes difficult to motivate the students and to accomplish a
good class climate.
2Names of schools and people have been changed to ensure anonymity.
3Derived from a standardised Dutch National School Achievement Test (from the CITO, the Dutch National Institute for Test Development).
670
E. Kuiper et al./Computers & Education 52 (2009) 668–680

Page 4

Author's personal copy
Milford School is situated in a town’s suburban area. The student population is mostly from a white, middle class background. Lorna, the
teacher, runs the class together with a colleague, both teachers working half-time. They are familiar with using collaborative inquiry activ-
ities inspired by socio-cultural views on education, although the school does not explicitly share this view. The school has good ict facilities.
Lorna is an experienced Web user, and is familiar with technical devices, for example the use of computer projectors. Her class of 27 stu-
dents (12 girls, 15 boys) has an average reading comprehension score is 46.6, which is slightly below the national average. The teachers see
their students as active (especially the boys), but lacking in reflection; more ‘doers’ than ‘thinkers’.
Norwood School is a small village school. The student population is from a white middle and lower middle class background. The school
often works with inquiry activities of all sorts from several educational, including socio-cultural viewpoints. It has good ICT facilities and a
school principal who stimulates the use of all sorts of ICT applications in the classes. Teacher Jonathan is a very experienced teacher likes to
work with spontaneous, student-directed activities. His class only contains 18 students (7 girls, 11 boys). Their average reading compre-
hension score is not known exactly but characterised by the teacher as being ‘low to average’; there are several dyslectic students in the
class. The teacher has difficulties establishing a good class climate; students are very easily distracted and especially the girls do not
get along very well.
3.2. Materials
3.2.1. Design and content of the program
The project comprised 10 weekly lessons of 1.5–2 h each. Based on the literature, we formulated as starting point to guide the form and
content of the unit: collaborative, inquiry-oriented activities as a means to, and context for, both content knowledge building and acqui-
sition of Web literacy skills.
Students were asked to collaborate in the production of a brochure about ‘healthy food’ intended for the other 5th grade classes at
school. The subject of the unit was chosen because of its authenticity, since it can easily be connected with children’s real life experiences.
Children eat every day, and most of them experience some tension between healthy food habits (stimulated by most parents and teachers)
and their own food preferences. The program was set up in such a way that the development of critical Web skills and content knowledge
building would evolve in tandem during the lessons. Working in pairs and supported by the teacher, students were required to formulate
research questions on the overall theme and on sub themes. These questions had to be appropriate for the acquisition of Web literacy skills,
i.e. be rather concrete and factual, making it possible to find relevant information and to practice those skills. The teacher encouraged them
not only to search for Web content but also to read and evaluate it critically in order to make it meaningful for themselves and their future
readers. Thus, by constantly relating the students’ Web activities to the production of the brochure, the teacher tried to make clear the need
for those skills. For example, getting endless lists of Web pages from a search engine required students to form their own opinions on the
usefulness of those pages for their research questions and for the brochure. During the project, the teachers offered the students tools for
and support in both Web literacy skills (searching, reading and evaluating Web information) and inquiry skills (e.g., constructing adequate
research questions and processing the information found).
Table 1 shows the overall structure of the program as well as the aspects of Web literacy which were addressed by the program.
The first two project lessons were meant to be introductory and consisted of joint activities focusing on collaborative inquiry activities
with shared goals. The teacher and the children discussed several aspects of the project subject, for example based on collages of what
students thought to be healthy and unhealthy food items. By questioning their opinions, the teacher modelled a critical, inquisitive atti-
tude. Suggested home activities related to the project were for example constructing ‘eating diaries’, and collecting packages of food items
and food advertisements. Based on all these activities, the teacher and students constructed research questions on three sub themes. From
the third lesson on, the students used the Web to obtain information for the brochure on the basis of their questions. In these lessons, all
three subcategories of Web literacy skills were successively addressed. Most lessons began with a class discussion about the lesson’s cen-
tral topic, e.g. ‘What are the differences between reading a book and reading a Web text?’ or ‘Do you believe all Web information is true?’.
These class discussions would give the teacher the opportunity to explore the students’ knowledge and opinions on such matters. After the
class discussion, computer work alternated with whole class discussion on the students’ experiences when using the Web. The last two
lessons of the program were intended/meant for the production of the brochure.
The four participating teachers were supported in several ways. They had a manual at their disposal with for each lesson specific learn-
ing goals, suggestions for class activities and class discussions, examples of several ways to support student that particular lesson, and prac-
tical guidelines. The teachers also met each other during a training course in the month preceding the project. In this course, the basic
assumptions of the project and its learning goals were explained and discussed. The teachers also practiced parts of the project’s lessons
together. They were explicitly invited to adapt the project to their own students and circumstances, provided that the project’s main char-
acteristics (as described above) would be preserved.
3.3. Instruments and procedures
An in-depth study of the project required the collection of a variety of data both on the experiences of the teachers and on the learning
processes of students. For this purpose, lesson observations and face-to-face interviews with teachers were combined with student inter-
views, student questionnaires and a final student assignment.
3.3.1. Lesson observations, field notes and interviews with teachers
Every other week a whole lesson was videotaped for each class. In this way, every lesson in the lesson plan was videotaped in two dif-
ferent classes. The focus was primarily on the teachers’ actions and on the dialogue between teachers and students. All videotapes were
transcribed. The same lessons were also observed, using diaries with field notes, and observation checklists focusing on aspects such as
student involvement, class climate, and the way the teacher supported the students. The field notes served as initial data analyses, with
analytic notes providing material for later observations.
In addition, every other week the teachers were interviewed about the lessons and the students’ learning processes. The teachers main-
tained diaries for all lessons. The interviews and diaries focused on the implementation of the program, especially on ways in which
E. Kuiper et al./Computers & Education 52 (2009) 668–680
671

Page 5

Author's personal copy
collaborative inquiry activities and Web literacy skills were brought up. Teachers were explicitly invited to share their opinion of the pro-
gram with the researchers, and to discuss the difficulties they had experienced while working with the lessons. This enabled us to compare
research observations with the teacher experiences.
3.3.2. Student questionnaires, interviews with students, and final assignments
From the total of 93 students, 84 students completed questionnaires before and after the project. Part of these questionnaires focused on
elements of students’ backgrounds and their attitudes towards computers and the Web. Another part consisted of 10 knowledge items on
the subject of healthy food and 15 knowledge items on searching, reading and evaluating Web information (all multiple-choice items). On
both subjects, no reliable tests to determine differences between students of this age are available. Kuhlemeier and Hemker (2005), when
constructing a broad test on internet skills for high school students, deliberately left out items on evaluating Web information because of
the failure to construct reliable and valid test items on such critical Web use. Therefore, we decided to construct a questionnaire that con-
sisted of items on a range of aspects of healthy food and critical Web use, without aiming at constructing a coherent test that could deter-
mine differences between students and between times. This is reflected in rather low reliability scores (a = .55 for items on healthy food;
a = .64 for items on critical Web use). The items on Web use focused on searching/navigating skills and (to a lesser extent) on evaluating
skills. Web reading skills were not explicitly addressed, as a pencil-and-paper questionnaire cannot reflect the complexity of Web reading
in practice. Some elements of reading on the Web were imitated by offering screenshots of Web pages on which the students had to choose
between links (e.g., ‘‘On what part do you click when you want to know the entrance fee of the zoo?”). With regard to healthy food, some
items were directly related to declarative knowledge students might have come across during the program (e.g., ‘‘What are ingredients?”);
others were meant to give an indication of a general way of thinking critically about eating habits (e.g., ‘‘What is the best solution when you
eat not enough vitamins?”).
In each class, three students pairs were selected and more intensively observed, using an observation checklist that focused on aspects
of students’ involvement, collaboration between students and the way students worked with the teaching materials. These students were
also interviewed every other week. In these interviews students were free to discuss their opinions on the lessons, on what they had
learned that week and on the teaching materials. We aimed at diversity between the student pairs, choosing students of mixed gender
and ability. After the program was finished, these 12 student pairs carried out six final assignments that all focused on the application
of critical Web skills. Although Web searching, reading and evaluating skills all play a role in Web activities, in each assignment one or
more of these skills were more prominent than others. The first three assignments focused primarily on Web searching and reading skills.
The assignments 1 and 2 asked students to find a specific answer on the Web, using their own preferred way of searching the Web. Stu-
dents’ spontaneous use of the Web was prominent in these assignments. In assignment 3, the focus was more specifically on Web reading:
students were given a specific website and were asked to search for an answer on that website. The assignments 4, 5 and 6 all focused
specifically on Web evaluating skills. Students were asked to comment on three websites, with a focus on different aspects of Web eval-
uation (such as purpose and usefulness of specific website). In this way, such contemplative use of Web evaluating skills could be com-
pared with the way they evaluated websites during their spontaneous Web searches in the first two assignments. For a more extensive
description of both background and content of the assignments, see Kuiper, Volman, and Terwel (2008b).
All students worked in pairs, as they had done during the program. For each assignment, they got a maximum of 10 min. All sessions
were captured with Camtasia Studio screen recording software, which also recorded the conversations of the student pairs. Due to stu-
dents’ illness, in two schools only two pairs took part in the assignments.
Table 1
Overall structure of the project and aspects of Web literacy addressed in the project.
Web literacy skills and strategiesThematic activities
Weeks 1
and 2
Starting up the project
Drawing up research questions
Weeks 3
and 4
Web searching skills and strategies:
– meaning and proper use of URL’s
– browsing strategies
– formulating key words
– navigating through lists of results from search engines
– evaluating the relevance of these results
– using search strategies in a flexible way
Searching for information on the Web in the context of students’ research questions
Weeks 5
and 6
Web reading and interpreting skills and strategies:
– various ways in which websites may be structured
– practising reading of several types of Web texts
– using navigational aids such as toolbars and menus
– skimming Web texts to identify key words or phrases
– in-depth reading of relevant passages
– purposeful use of links in Web texts
Applying reading skills to be able to select useful Web information for the brochure
Weeks 7
and 8
Web assessment and evaluating skills and strategies:
– assessing one’s own level of understanding of specific Web texts
– getting insight into the relative accuracy and reliability of Web
information
– judging the accuracy and reliability of Web information
– relating the information found on one’s own question or need for
information
– assessing the function and meaning of visual website elements
Applying evaluation skills to be able to assess the selected Web information with regard
to its comprehensibility, reliability, and usefulness for the brochure
Weeks 9
and
10
Joint application of the Web searching, reading and evaluating skills
and strategies, with a view to composing the brochure
Presenting the results of the project: composing the brochure and presenting it to an
audience (students, teachers, parents)
672
E. Kuiper et al./Computers & Education 52 (2009) 668–680

Page 6

Author's personal copy
We decided not to include the brochures made by the students as an outcome measure. Given the character of the program,
the content of these brochures was not really comparable because they reflected the students’ specific research questions. The
brochures also were group products of the whole class, and therefore cannot be reduced to individual students’ learning
results.
3.4. Design of the study and data analysis
We used a multiple case study design for our study (Yin, 2002). Data analysis focused first on within-class descriptions of the specific
way the program had been carried out in each class. Then, cross-class analysis was performed in order to be able to determine which con-
textual factors may affect the realization of the program and students’ learning.
3.4.1. Analysis of lesson observations, field notes and teacher interviews
Video tapes of the lessons were transcribed and analysed in order to characterise the learning environments in each class. Our focal
points were both the way teachers and students gave shape to the collaborative inquiry activities, and the ways in which content knowl-
edge and Web literacy skills were raised within these activities. These focal points firstly included elements that may be considered critical
when working with collaborative inquiry activities:
– how did students’ research questions origin and develop;
– the relation between students’ research questions and the subject and goals of the project;
– the way the teachers supported students in their inquiries.
– how did students collaborate, both in pairs and in the whole class.
Secondly, we considered the ways in which the teachers paid attention to the various Web literacy skills, their embeddedness in the
projects goals and activities, as well as the relation between content knowledge and Web skills.
While transcribing the video tapes, and matching the tapes with the written observations, we looked for ways in which these elements
emerged in the lessons. Lesson segments were described, highlighting key events, after which each classroom was characterised by iden-
tifying what elements were present and the ways in which teachers dealt with those elements. We also looked at the way the teacher
worked with the provided teaching materials. Transcriptions were also made of all teacher interviews. These transcripts were analysed
with a view both to the elements used for the analysis of the observation data and the teachers’ ways of reflecting on their own way of
working, on the project as a whole and on the provided teaching materials. The interviews also provided a means of triangulation, by com-
paring and contrasting them with the observation data.
3.4.2. Analysis of student interviews, observations, questionnaires and final assignments
The transcribed student interviews were analysed with a view to students’ opinions on the project and on the way the teacher worked
with it. We focused specifically on aspects as students’ involvement, their research questions and their success or failure to find relevant
information, and their views on the project. Student observation data primarily served triangulation, by comparing them with the inter-
view data.
Students’ answers on the knowledge items of the questionnaires were scored, both for the total group of 84 students and for the four
classes separately. T-tests were performed to compare the four classes’ mean scores before and after the project.
The Camtasia screen and audio recordings of the final assignments of all 12 student pairs were transcribed. These transcripts pro-
vided insight into students’ ways of working with the Web as well as their collaboration and discussions. First, students’ performances
were analysed with a view to the Web searching, reading and evaluating strategies the program had paid attention to (see Table 1). To
assess the adequacy of students’ Web searching and reading strategies in the first three assignments, we designed a coding procedure in
which each separate reading and searching activity was first coded and then scored as either 0, 0.5 or 1, depending on its adequacy in
the context of that specific task. The reliability of the coding procedure was calculated by coding about 50% of the activities twice, by
two independent researchers (Cohen’s kappa = .87). These scores were added up and then divided by the total amount of reading and
searching activities each pair had conducted in that assignment. In this way, an overall score (laying between 0 and 1) for each student
pair’s adequacy in using Web searching and reading skills was calculated (see Kuiper et al., 2008a, for a more detailed description of the
coding procedure).
In the three assignments that focused primarily on students’ Web evaluating skills, students were asked to discuss the quality, inten-
tions or usefulness of a specific website. To get an overview of all arguments students used as well as their relation with the aspects of
evaluating Web information that were discussed in the lessons, the transcripts of the student pairs’ dialogue and screen behaviour during
the assignments. Jointly, these data provided insight into the Web literacy skills students were able to use, as well as their specific Web
search behaviour during the various assignments.
Finally, data from multiple sources (classroom and student observations, students and teacher interviews, students’ final assignments)
were analysed in relation to each other and between classes.
4. Results
First, we portray the four classes with regard to the way the teachers and students worked with the project. We focus on the way they
gave shape to the collaborative inquiry activities and the acquisition of Web literacy skills, as well as students’ opinions and behaviour and
general issues as the teachers’ appreciation of the project and their comments on the way the project worked out for their specific situation.
Then, in a cross-class comparison we will compare classes with a view to the contextual factors that influence the realization of the pro-
gram as well as students’ learning results.
E. Kuiper et al./Computers & Education 52 (2009) 668–680
673

Page 7

Author's personal copy
4.1. Within-class description of the four classes
4.1.1. Kingsley School
From the beginning of the project, Kingsley School students showed a great deal of knowledge about the subject healthy food. Because of
this, the teacher (Thomas) expected much from students and also placed high standards on the students’ research questions. This partly led
to students formulating rather broad and open research questions. Although the teacher noticed that several questions were not appropri-
ate for Web searching or knowledge building, because of time pressure he rarely questioned them or discussed them with the students. As
a result, part of the students searched the Web with research questions that were either too difficult or too broad, or inappropriate for Web
searching, such as for example, ‘‘How many healthy things are there?” and ‘‘How do vitamins originate?” In those cases, the type of ques-
tions affected both the students’ way of working on the Web, and the way Web literacy skills could be addressed. Students easily became
frustrated or discouraged when their Web searching did not produce the desired results. Moreover, they were confronted with the diffi-
culties of Web use without being able to practice the Web skills discussed in the project on ‘workable’ questions.
The students were used to work in pairs and small groups, and did so when working with the Web during the project. Thomas found it
difficult to support all student pairs, which was partly due to the location of computers in school hall and several corridors. As a result,
some students merely sat together at the computer and did not really work together. In class discussions, the students were not as active
as the teacher expected them to be. He frequently became irritated by their lack of involvement but found it difficult to stimulate and acti-
vate them. The students’ behaviour seemed to reflect their not knowing why they were doing this project. The teacher had explained to
them the need to acquire Web literacy skills, but most students already were quite confident about their own Web skills. The project’s
purpose, the making of a brochure, was mentioned frequently by Thomas, but the students’ did not experience their Web activities as con-
nected with making the brochure: ‘‘It is always yes in the end there will be a brochure. But when will there be one then?”
Part of the students’ comments was a result of the way inquiry and Web activities were commonly regarded in this class. Students were
used to use the Web for their own projects, with subjects of their own choice and looked at the Web as something they could use as they
wanted. They had expected the project to be like that, which suggests a gap between students’ expectations and the project goals and activ-
ities. The teacher tried to motivate and stimulate the students in the subsequent weeks, with moderate success. The brochure was drawn
up after the project by only four students. They assembled some of the information they had found during the project, but did not use any
information provided by the other students. The brochure is instructive, but shows no connection with the Web activities of the previous
weeks. Thomas was rather disappointed in how the project had worked out. Yet, his impression was that the students did learn something
during the project, especially with regard to Web skills.
4.1.2. Langdon School
The Langdon School teacher, Jim, was used to working spontaneously, initiated by the students or himself. This was reflected in the way
he carried out the project. For example, at the start of the project, Jim decided to explain goals and purpose of the project (i.e., making a
brochure for fellow students) not beforehand because he wanted to leave more initiative to the students. Unintentionally, this resulted in
students not knowing why they were doing project activities. Jim’s teaching style was also reflected in his approach of students’ research
questions that would guide their Web activities during the project. Instead of negotiating these questions with students, Jim wanted stu-
dents to discover themselves which questions would ‘work’ and which would not, hoping they would conclude that they had to adapt their
questions.
In doing this, he seemed to overestimate both the students’ motivation and their inquiry skills. In practice, many students did not mind
surfing the Web with all sort of questions, as long as they could use the Web freely. As a result, the research questions that were supposed
to guide their Web activities were often too broad or simply inappropriate for searching the Web, as at Kingsley School. Jim recognized
many students’ need for support and guidance during computer work, but lacked time and overview to fulfil those needs. After a couple
of project weeks, the teacher struggled with the question how to combine the project requirements with his own spontaneous way of
working with the class. The students had grown a bit bored and were losing their motivation.
This process was reinforced by the way students worked together. Although the students were used to collaborative group work, they
did not rely on each other to complete their tasks, neither in pairs nor in class. There was a general class climate in which students did not
really listen to each other during class talk and some students tended to react negatively and childishly to other students’ utterances. In the
teacher’s eyes, his students generally easily lost motivation and ‘gave up’ because they wanted school to be ‘fun’.
Looking back on the project, Jim was not wholly satisfied. The class had not worked in the way he had wanted, and the inquiry activities
did not live up to their promise. He noticed a tension between his own preferred way of teaching and the students’ need for structure and
guidance when using the Web.
However, the students produced a very nice brochure at the end of the project although only a small part of it was connected to the
inquiry activities the students had carried out the weeks before. The quality of the brochure reflected the students’ experience with all sorts
of writing and editing activities. The brochure was presented very nicely as a school paper for all parents and students. The value of the
teacher’s strong inclination to spontaneous activities showed itself in one particular part of the brochure about the pros and cons of eating
insects. By proposing to cook grasshoppers in school because of their nutritiousness, Jim urged students to have a debate about the ethical
aspects of doing this, and to search for information on the Web about grasshoppers’ nutritional value and ways of preparing grasshoppers.
4.1.3. Milford School
From the beginning, Milford School teacher Lorna put much time and effort in the project. She followed the teaching guidelines more or
less, but adapted these if necessary to her own situation. She was enthusiastic about the project and an active and stimulating teacher,
which resulted in students really liking to work with the project. Lorna was able to show students what the project was about without
exercising too much control and she recognized possibilities for learning when they showed up.
Drawing up research questions was a collaborative activity in this class and originated in the students’ own experiences. For example,
when discussing their own eating habits, some students appeared to skip breakfast most days. This resulted in a research question about
the need to have breakfast. One girl, being vegetarian, was challenged by the teacher to formulate research questions about what products
674
E. Kuiper et al./Computers & Education 52 (2009) 668–680

Page 8

Author's personal copy
are necessary to stay healthy when you are vegetarian. Lorna emphasized the connection between the project and the students’ daily lives,
by questioning her own eating habits. Teacher and students discussed these research questions, the teacher being more or less negotiating
in modifying the question if necessary.
The goals of the project (learning Web skills to be able to make a good brochure for fellow students) was the thread that ran through all
project lessons. The brochure was presented as necessary for other students who did not share the same knowledge. For most students, the
brochure became a serious and meaningful activity.
Lorna focused explicitly on collaborative group work and introduced herself as part of that group. She talked with the students about
their ways of working together at the computer and took time to solve problems. She created certain extra conditions to effect this collab-
oration, for example, by arranging the classroom in a way that all students knew what students were working with which research ques-
tions on which part of the brochure. She also made shared folders on the computer in which all students could post their work, so students
knew more or less what fellow students were producing. This was reflected in the way students referred to each others’ questions, and their
helping each other with Web information they came across. This was especially important because the computers were located in several
halls and corridors. The teacher lacked clear overview and had to rely on students helping each other.
Lorna’s frequent use of a computer projector contributed to her being able to connect the students’ inquiry activities with the Web lit-
eracy skills. She took much time and effort discussing the ways in which the students worked on the Web, thereby focusing on the Web
skills that were addressed that lesson. The first two lessons, in which searching skills were at the centre, she practised those skills together
with the students, taking as a starting point the problems some students had finding information for their research question. All students
were invited to think along with each other and to practice possible solutions. In the next lessons that centred around Web reading skills,
she also used the projector in her instruction, discussing with the students the differences between reading a book and reading a website,
and providing examples. At the end, the project resulted in a nice, instructive brochure that reflected very well the project’s activities. All
students contributed to the brochure, albeit in different roles. The brochure was presented by the students in the school’s other upper grade
classes.
On the whole, the project goals were achieved rather well in this class. However, it required a great deal of effort on the part of the
teacher. Lorna also struggled with the fact that on the one hand, the project was too short to pay enough attention to the various Web
literacy skills, while, on the other hand, it was somewhat too long to keep the students motivated. In the end, she was fairly satisfied with
the project and had the impression that students learned quite a lot. She was also pleasantly surprised by the way students cooperated
during the project.
4.1.4. Norwood School
From the beginning, some tension was apparent between the Norwood School teacher Jonathan’s own preferred teaching methods and
the project’s rather strict learning goals. Jonathan appeared to be very good at creating conditions for activating students and for learning.
At the same time, he also tended to give students much freedom in finding out for themselves what they wanted to do and know. For exam-
ple, in one of the first lessons the teacher led an interesting class discussion about healthy and unhealthy eating habits. Interestingly, con-
trary to most children’s opinion that drinking beer and wine is very unhealthy, one child’s mother appeared to have suffered of thrombosis
and her doctor has advised her to drink a glass of red wine every day. The teacher mentioned this as very interesting and hinted at the
opportunities for exploring this further in their research questions. However, because he left everything up to the students after that, noth-
ing happened with it in the next lessons.
The same phenomenon occurred with class discussion subjects such as the wholesomeness of deepfreeze products and the differences
between white and wholemeal bread. This resulted in a loss of potentially motivating research questions and in students searching the
Web with questions that were partly less connected with their everyday life or interests. However, on the whole students in this class also
formulated ‘workable’ questions that stimulated Web searching.
The class’ small size together with the situation of the computers in one room, made it relatively easy to support students in this class.
At many moments during the project, the teacher gave valuable feedback on students’ Web activities. However, his tendency to let students
decide for themselves what they were to do with such feedback proved to be a too open-ended way of teaching in this class. Expecting
students to be active and involved, Jonathan seemed to underestimate his own role. Most of his students needed much support and struc-
ture and got the impression that what they did exactly did not matter very much. Their Web searching often resulted in fragmented bits
and pieces of knowledge without much connection. Students also did not cooperate very well, despite all the teacher’s focus on the rules of
collaborative work.
For the students, the project was mainly about food; the Web skills did not really become integrated into the project. This was partly
due to the fact that the project time was shorter than for the other classes. Because Jonathan experienced the project as being too long and
having too much (theoretical) content, he left out some parts of the curriculum. This resulted in some fragmentation; with regard to the
Web literacy skills, most skills were dealt with in one lesson instead of two lessons. The teacher also did not discuss all Web skills with the
students before letting them practice at the computer. Another important factor was the short attention span of many students, which
caused the teacher to split up the project into short lesson fragments.
Jointly, these factors contributed to the project being most of the time a couple of rather isolated lessons. The students were not par-
ticularly attracted to the idea of making a brochure. The resulting brochure itself was rather short and reflected the reading and writing
problems of some of the students. Some of the texts were taken from websites verbatim. However, in the end all students contributed
to the project. On the whole, Jonathan was not dissatisfied with the project. His impression was that students learned a lot from the project,
especially with regard to evaluating Web information.
4.2. Across-class comparison
4.2.1. Shaping the curriculum: collaborative inquiry and Web literacy
To what extent and in what ways did the teachers differ with respect to their implementing, organising and supporting the collaborative
inquiry activities and their attention to both Web literacy skills and their relation to the project’s theme? We distinguish six aspects, fol-
lowing the focal points that guided our data analysis.
E. Kuiper et al./Computers & Education 52 (2009) 668–680
675

Page 9

Author's personal copy
Development of students’ research questions. In all classes, students constructed their own research questions. In one class, this resulted in
the teacher and students jointly constructing and modifying the research questions. This teacher discussed the usefulness of the questions
with regard to the content of the brochure. Two teachers chose not to intervene in this process a great deal. In their opinion, students would
be more motivated when working with questions of their own and also more challenged to find answers to those questions. Although they
recognized the difficulty or inappropriateness of some questions, they either wanted students to find out for themselves what questions
‘worked’ and what did not or underestimated the consequences. In one class, merely practical circumstances such as time pressure and a
lack of overview of all students’ computer activities resulted in students mostly ‘going their own way’ with regard to making research
questions.
It was assumed in the project that for children of this age, learning to critically search, read and evaluate Web information requires
the formulation of rather concrete and factual research questions that make the finding of relevant information easier. The validity of
this assumption may be illustrated by the consequences of the teachers’ different approaches of the process of constructing research
questions. Students who worked with concrete questions (either completely their own or negotiated with the teacher), experienced suc-
cess when finding relevant information, which in turn resulted in higher motivation. Examples of such questions are ‘Is wholemeal
bread healthier than white bread?’ or ‘What food contains calcium?’. They also had the opportunity to practice the critical Web skills
discussed in class since they found potentially useful information. In contrast, students who worked with either too difficult or inap-
propriate questions (for example ‘Is healthy food always distasteful?’), spent most of their time with unfocused searching for
information.
The relation between students’ questions and the project goals. The project was aimed at the acquisition of Web skills in the context of
inquiry activities on the subject healthy food. One teacher made such connection between students’ questions and project goals from
the start and reminded students repeatedly of the usefulness of critical Web skills when working on the brochure. In this class, the students
knew all the time that practicing those Web skills was necessary for their joint production of an informative and reliable brochure. When
asked, these students related activities such as learning to recognise with what aim websites are made to their own process of writing texts
for the brochure:
‘‘Well, in the brochure, everything must be right somehow. But on the Web you can also find a lot of nonsense. [...] Then you must be
sure you don’t write that nonsense in the brochure, because then people can’t use it.”
The other three teachers tended to focus on the Web skills and to discuss these skills in isolation from the project’s context. As a con-
sequence, their students had difficulty with connecting their Web activities with the making of the brochure and were less motivated to
work with the Web in such a focused way.
The ways the teachers supported the students’ inquiry activities. Some practical circumstances influenced the teachers’ ways of supporting
students’ Web activities. Especially the location of the computers in school proved to be important, since it was difficult for teachers to
support students and to have an overall view of their computer work when computers were located throughout the school building. This
was of course also influenced by class size. Both support of and reflection on students’ Web activities were made easier with help of a com-
puter projector. Web activities tend to be individual activities, as all students work at their own computer. A projector makes it possible to
share students’ Web activities with each other and to practice Web skills together, taking advantage of knowledge and skills students al-
ready possess. However, the ways teachers supported their students also reflected their different teaching styles. The two teachers that
preferred working with spontaneous, student-initiated activities also tended to support students less directly and to leave the initiative
for support with the students. Unintentionally, this often resulted in students not recognizing the necessity of using certain Web skills
or strategies.
The collaboration between students. The classes differed with respect to the ways the students collaborated both in pairs and in the whole
class. The teacher that succeeded best in connecting the acquisition of critical Web skills with the joint production of the brochure, paid
explicit attention to collaboration, focusing especially on the necessity of sharing knowledge with other. She expressed her relying on stu-
dents’ helping each other. This turned out as she intended, since students felt they shared the same goal and realised that only together
they would be able to make a valuable brochure. This was also reflected in the several ways students supported each other during com-
puter work. In contrast, there also were classes in which students did not look at the project as something they needed each other for.
Paying attention to Web literacy skills. With regard to the specific searching, reading and evaluating Web skills, it was striking that the
teachers, as a matter of course, paid attention to searching skills. This aspect of Web literacy was addressed rather frequently at the schools,
during students’ Web activities. The project’s focus on reading and evaluating skills was new to the teachers. Most of them found it more
difficult to connect those skills to the students’ inquiry activities. Although all teachers recognised the necessity of both reading and eval-
uating skills, in the lessons students’ information seeking was sometimes unintentionally narrowed down to searching skills only. Teachers
found it more difficult to address Web reading skills. Students were convicted that searching the Web was easy for them, and disliked the
idea that reading skills were as necessary on the Web as with books. In fact, many students’ preference for the Web as an information re-
source originated in their opinion that ‘‘on the Web you don’t need to read”. The students were most interested in the evaluating skills
which appealed more to them, partly because the activities involved (e.g., discussing the McDonalds website and relating the form and
content of websites of different origin).
Paying attention to content knowledge. During the project, students’ research questions determined their coming across valuable content
with regard to the project’s theme, healthy food. Therefore, both quality and ‘workability’ of those questions were important for their learn-
ing. Students who worked with concrete and factual research questions on aspects of healthy food or healthy eating habits and who were
able to find useful information, clearly learned from their inquiry activities. They had to read and process Web content when composing
texts for the brochure. Because of the connection with the research questions, students came across different aspects of healthy food, vary-
ing from the necessity of eating breakfast to the difference between white and wholemeal bread, and from vitamins in milk to the amount
of vegetables and fruit a child needs every day. The one class in which the students collaborated best, also shared the information they
collected with each other, as was intended. As a result, they jointly built a ‘body of knowledge’ about healthy food which was in turn re-
flected in the brochure for their fellow students. The example of Langdon School students discussing and studying the nutritiousness of
grasshoppers, is another example of shared knowledge through Web-based inquiry activities.
676
E. Kuiper et al./Computers & Education 52 (2009) 668–680

Page 10

Author's personal copy
On the whole, the teachers’ particular teaching style seems to have been determining the way the project worked out in their specific
class. Both the Langdon and the Norwood School teacher were used to work with spontaneous, student-initiated activities. They wanted to
activate and motivate students for example by letting them not only compose their own research questions but also leaving the initiative
for modifying these questions to the students and wanting them to find out for themselves which questions were ‘workable’ and which
were not. Both teachers felt that the project’s lay out and materials required something they did not really want to do, although they were
convinced of the usefulness and necessity of the project’s learning goals. Although one other teacher (from Kingsley School) also
experienced many problems with his students’ passive behaviour, he tended to respond with more control and teacher initiative, instead
of leaving the initiative more to the students. The Milford School teacher seemed to have been most ‘at ease’ during the project; the pro-
ject’s lay out and content fitted her own teaching style.
4.2.2. Students’ learning results: questionnaires
With regard to the learning results of the program in the participating classes, we will look first at the questionnaires. The question-
naires’ knowledge items focused on healthy food habits and Web literacy skills. Table 2 gives an overview of students’ mean scores for
the four classes separately.
With regard to the healthy food items, comparing means has only relative significance, as the students did not follow a fixed curriculum
on this subject. The learning that could take place was necessarily connected to the student’s own research questions. All class means are
higher after the project, both Kingsley and Milford means reaching significance (p < 0.05). With regard to the Milford students, this may be
interpreted in the light of the way the project was carried out in that class. The students from Milford School worked with rather good and
‘workable’ research questions, which were directly related to the subject of healthy food. Moreover, these students also shared the acquired
knowledge with each other (much) more than in the other classes. It could be that in this way the students gained more content knowl-
edge, which is reflected in their answers. However, we are careful with such conclusion because the results of the Kingsley students are less
easy to interpret; it may be that their Web searching has led to knowledge on aspects of healthy food addressed.
The analysis of students’ answers to the items regarding the knowledge of Web literacy aspects shows that the mean scores of three of
the four participating classes are significantly higher after the project (p < 0.05). The Norwood School students’ results stayed the same.
Again, we may only interpret this very carefully. At Norwood school, less attention was on the Web skills during the project, compared
with the attention on healthy food. This may have affected these students’ results on the questionnaires.
4.2.3. Students’ learning results: final assignments
Three student pairs of mixed gender and reading comprehension level from each class performed six final assignments after the project.
The assignments focused on students’ actual Web use. Most student pairs began with searching with Google, and used other strategies only
when they got stuck. Students appeared to be quite able to compose relevant keywords for their searches, although some pairs used spoken
language or typed in the whole question as one search term. Students regularly scanned Web texts, looking for specific headings or clues in
the text, as they had learned during the project. However, they also showed many instances of ‘non-reading’, i.e. overlooking relevant infor-
mation or scanning very quickly without really looking for headings or key words.
Table 3 shows the adequacy scores of the student pairs on the two assignments that focused on both searching for and reading Web
information.
Table 2
Overview of students’ learning results (questionnaires).
Kingsley School N = 18Langdon School N = 23Milford School N = 27Norwood School N = 16
MSDMSDM SDMSD
Food items (N items = 10)
Before
After
7.11
8.11
1.94
1.32
7.26
7.48
1.51
1.93
6.19
7.59
1.84
1.42
6.13
6.69
1.15
1.49
Web skills items (N items = 15)
Before
After
11.89
13.17
2.30
1.04
11.35
12.57
2.27
2.61
11.59
12.56
1.78
1.78
10.19
10.19
3.23
2.83
Table 3
Overview of students pairs’ learning results (final assignments).
Student pairReading levela
Score assignment 1Score assignment 2
Kingsley SchoolKS1 (2 boys)
KS2 (2 girls)
A/A
D/D
0.85
0.61
0.50
0.29
Langdon School LS 1 (2 girls)
LS2 (2 boys)
A/C
B/C
0.94
0.66
0.77
0.52
Milford SchoolMS1 (2 boys)
MS2 (2 girls)
MS3 (2 boys)
B/C
B/D
C/D
0.66
0.81
0.92
0.83
0.48
0.27
Norwood SchoolNS1 (2 boys)
NS2 (2 girls)
NS3 (1 girl, 1 boy)
A/C
A/C
B/D
0.56
0.58
0.60
0.37
0.73
0.72
aOn the basis of the scores on the CITO reading comprehension test, students are classified in one of five categories A, B, C, D or E; A representing the highest and E
representing the lowest achieving student.
E. Kuiper et al./Computers & Education 52 (2009) 668–680
677

Page 11

Author's personal copy
The scores show great variation both between students and between assignments. Better and worse performing students were found in
all schools. Although the assignments varied in difficulty, assignment 1 being easier than assignment 2, this is not reflected in all student
pairs’ scores. There also seems to be no direct relationship between students’ level of reading comprehension and their scores. Sometimes
weaker reading pairs showed less adequate Web behaviour, but there are also examples of surprisingly high scores. Vice versa, strong read-
ers (for example pair KS1, and also NS1 and NS2) did also show inadequate behaviour at one or more assignments. The differences between
students seem to be related to their ability to reflect on their searching, and as a result to adapt their search strategy. The lowest performing
students either continued to use the same, sometimes failing strategy, or (especially) tried many different strategies without much think-
ing. In fact, the highest scores in Table 3 reflect Web behaviour that combined relative good Web skills with both flexibility and patience.
Inconsistency seems to be a prominent feature of many students’ use of Web skills. The variances between their scores on the assign-
ments are remarkably (see for example the pairs MS2 and MS3). Students also showed inconsistency within one assignment, alternating
adequate actions with highly inadequate ones. Thus, although many students were able to use rather sophisticated searching and reading
skills on certain moments during the assignments, they did not do so regularly. For example, students showed that they were able to com-
pose relevant keywords in one assignment but failed to do so in another. This was only partly related to the degree of difficulty and type of
the assignment.
Most students did not seem to plan ahead when searching on the Web. They seemed to act rather impulsively, which turned out
well in some cases, and wrong in others. Many students simply ‘tried out’ one or more search strategies, including strategies they had
learned to be not very useful. Almost all students could lose themselves in inefficient search strategies that seemed to make no sense.
Reading Web information in particular proved to be a bottleneck for students, because reading requires patience. Students had already
expressed their dislike of reading on the Web during the project. Also students’ own habits when using the Web seemed to be very
persistent.
Three assignments focused on various aspects of Web evaluating skills. In the project, students had come across several aspects of Web
evaluating, such as articulating their own opinion about the websites they visited, assessing the intentions of a website (e.g., ‘advertising’,
‘providing information’, ‘giving an opinion on something’, etc.) and assessing the relevance and suitability of websites. When asked to com-
ment freely on a website on animal protection, almost all pairs mentioned the appearance of the site in their comments, either in a positive
or in a negative way. Quantity of the information on the website was much mentioned, again either in a positive way (‘‘it is good that there
is much information”) or in a negative way (‘‘there is too much information”). Students also mentioned the quality of the information and
the easiness (or difficulty) to find information on the website. Most pairs did not navigate deeply into the website as they had learned in the
project and either stayed at the homepage or clicked at links without reading the information.
Almost all pairs could mention at least one intention of a given website (on wetlands protection) adequately; only one pair totally mis-
understood its intentions. Students mentioned mostly some aspect of advertising and of ‘giving an opinion’. Most students mentioned one
label, although in the project they were told that most websites can be characterised by several labels. Two pairs did connect labels with
each other, e.g. ‘They want to raise money so they can better protect the wetlands’.
One assignment asked students to evaluate a specific website with regard to its suitability for children. The website (about safe Internet
use for children) was aimed at parents and teachers, but due to its colours and fonts its appearance was very childlike. Only two pairs ar-
gued that the website was suitable for children. These pairs did not take the information or language into account or did connect that as-
pect of the website with its appearance. Most of the other pairs commented first on the website’s colours and fonts, but also questioned its
suitability for children because there was ‘‘nothing to do for children”. Although an adequate observation, the website’s information or lan-
guage did not really play a role; one pair said that they did not know what the website was about and did not understand its intentions.
Overall, in these assignments most students showed to be able to use at least some Web evaluating skills when asked explicitly to do so.
This was in line with our observations of students Web behaviour during the project lessons. Students appeared to be able to criticize a com-
mercial website’s intentions and reliability. However, when comparing this with students spontaneous use of Web evaluating skills during
the assignments 1 and 2, there appears to be a discrepancy. During these assignments students almost never questioned the purpose,
appearance or reliability of a specific website they visited; at the most they commented on a website’s usefulness. All websites were treated
as equal. This means that paying attention to Web evaluation apart from the context of searching the Web may not be very useful. Students
seemto be acquiring someknowledgeof Webevaluation, withoutbeing able (or willing)to apply thisknowledge during theirWeb searches.
5. Conclusions and discussion
Our research aimed at studying the possibility of collaborative inquiry activities as a context for acquiring Web literacy skills. In a multi-
ple case study design, we studied four 5th grade classes working on the subject ‘healthy food’. Students used the Web to obtain information
on healthy food with a view to making a brochure for fellow students. The teachers supported students’ inquiry activities and stimulated
that they encountered the need for Web searching, reading and evaluating skills when working on the brochure, which they then discussed
and practiced with the students.
The learning results of the students show that thematic inquiry activities can provide a valuable context for teaching and learning Web
literacy skills. Through working collaboratively with research questions with a view to a meaningful goal (i.e. making a brochure), students
were confronted with all difficult aspects involved in Web use. All classes show knowledge gain with regard to the subject healthy food and
all classes but one show knowledge gain with regard to Web literacy skills. With regard to students’ actual Web behaviour, the overall
impression is that the project has contributed to their Web skills. Although many students appear to be inconsistent, impulsive and impa-
tient Web users, they also show many instances of adequate Web searching, reading and evaluating behaviour.
From the teachers’ experiences with the project, we learn that there are several contextual factors that may influence the realization and
results of embedding the acquisition of Web literacy skills within the context of thematic collaborative inquiry activities.
Firstly, teachers and students need explicit support and guidance with regard to formulating ‘workable’ research questions that are
appropriate for the acquisition of Web literacy skills. Students who worked with such questions were able to practice and to improve their
Web skills and found valuable information for the brochure. In contrast, students who worked with questions that were too vague or broad,
did not find the required information, became discouraged and did not have the opportunity to use more advanced Web skills.
678
E. Kuiper et al./Computers & Education 52 (2009) 668–680

Page 12

Author's personal copy
Secondly, the conditions for students working collaboratively deserve attention. Our results confirm the importance of collaborative in-
quiry activities being more than just ‘working together’. Elements as teacher and students sharing the project’s purpose, relying on each
other’s knowledge and skills, and students sharing knowledge require specific attention and effort from the teacher. The class situation in
which such conditions were met, showed students who helped each other, who knew what everyone else was doing and who all shared the
same goals. This resulted in a high motivation and an accumulation of knowledge.
Thirdly, the presence or absence of basic inquiry skills in students proves to be an important factor. These skills were supposed to be
present, because of the teachers’ and students’ experience with inquiry activities. However, many students appeared to lack skills such as
integrating pieces of information and recognizing information as contributing to answering one’s research question. They were often look-
ing for the precise answer to their question and were not (yet) able to recognize the value of bits and pieces of information they could use to
compose the answer themselves. The Web reinforces such ‘looking for the right answer’ by its abundance of information which gives chil-
dren the idea that all answers can be found ‘ready made’.
Finally, the correspondence between the project’s proposed way of working and the teacher’s teaching style deserves attention.
Although all teachers had been selected on the basis of their similar views on education, they appeared to have rather different teaching
styles which were reflected in the ways they carried out the project. Two teachers experienced tension between their own preference for
spontaneous, student-led activities and the structure of the project materials. For example, they did not want to interfere much with the
process of students developing their research questions. They expected the students to be active and motivated knowledge seekers as a
matter of course. In this way, they underestimated the significance of their own contribution to the students’ inquiry activities, and their
role in modelling an inquisitive attitude.
Many students also appeared to need clear guidance and structure. Students liked the project primarily because they liked using the
Web. The literature shows that many students appreciate the Web’s appeared ‘easiness’: lots of information is only ‘one mouse click away’
(e.g., Large & Beheshti, 2000; Watson, 1998). This was reflected in the participating students’ expecting easy results from their Web search-
ing, and their being convinced of their own Web skills. The project asked them to approach their Web use differently, however. They were
supposed to stay focused at their research questions, to search purposefully, and to read thoroughly and critically. This discrepancy be-
tween the students’ expectations and preferred Web use, and the project’s intentions required much structure, support and guidance from
the teacher. Otherwise, students not only became discouraged but also seemed to ‘escape’ and to stay at their own preferred Web routines.
Overall, our results show that the project required complicated teaching skills as well as considerable time, effort and motivation. All
teachers were of the opinion that acquiring Web literacy skills required more time and practice; at the same time, three of them thought
the project was too long to keep students focused and motivated. All teachers suggested to spread out the teaching of Web literacy skills
over more and smaller projects and over a longer period of time, preferably a school year. We agree with this suggestion: Web skills could
then be taught in different projects with different themes, which could also be beneficial to the process of transferring the critical Web
skills learned. Teaching the required inquiry skills may also be more easily incorporated into these smaller projects. One could argue that
acquiring ‘new literacies’ such as Web literacy must be seen as basic skills just like traditional literacy skills as reading, grammar and spell-
ing. As a result, they require the same constant attention.
We also suggest to take better notice of students’ own ways of using the Web. Some students looked at the project as something that
had nothing to do with their own ways of using the Web. By taking them more seriously as participants, in the sense of already possessing
valuable skills, a learning environment could be created in which both teachers and students could share knowledge and skills in collab-
orative activities.
Our research was aimed at studying the acquisition of content knowledge together with critical Web skills. This proved to be difficult for
the teachers, who struggled with keeping a balance between these two learning goals and incorporating acquiring content knowledge in
Web activities. Using the Web in inquiry activities may be difficult for students, teaching to use the Web embedded in inquiry activities is
definitely not easy for teachers. The students’ learning results show a mixed picture of students being able to use rather sophisticated Web
skills, but also showing inconsistent, inflexible and impulsive Web behaviour.
In a parallel study on the acquisition of Web literacy skills, we embedded these skills in a structured educational program in which stu-
dents worked with prescribed exercises instead of collaborative inquiry activities (Kuiper et al., 2008a). This study shows more or less the
same students’ Web behaviour after the project. This suggests that both learning contexts may result in students acquiring valuable Web
skills. What may then be the surplus value of embedding critical Web skills within collaborative inquiry activities? The one class in which
the project’s goals were best achieved, shows that students were able to relate the value and meaning of Web literacy skills to their own
process of finding useful information for the brochure. They experienced the need to use the Web critically, because otherwise they were
not able to put together a reliable and informative brochure. Thus, learning Web literacy skills became a meaningful activity for them. Pre-
cisely this element of contextualisation reflects the surplus value of the inquiry-oriented program and may be the distinguishing factor
between the two programs: differences depended on insights into the necessity and functionality of those skills rather than on the learned
skills themselves.
When observing the way students worked with the project’s inquiry activities, we came across an interesting paradox for which our
study offers no quick and easy answer. Although the Web potentially is a tool that can be used by students for active and meaningful
knowledge construction, it also carries the risk of inviting just the opposite behaviour. The Web entices students to think every answer
can be found ready-made on it, because of the abundance of information it offers. It also invites students to be more or less passive search-
ers, because of the speed with which lots of information appear on the computer screen. Developing valid and meaningful knowledge on
the contrary supposes students to be actively involved in the learning process, to construct knowledge by connecting new information to
already acquired knowledge, and to reflect on this process and its results. This paradox illustrates that the Web not only offers many pos-
sibilities for learning, but also new challenges to teachers and students.
References
American Library Association (1989). Presidential Committee in Information Literacy: Final report. Retrieved January 16, 2006, from <http://www.ala.org/ala/acrl/acrlpubs/
whitepapers/presidential.htm>.
E. Kuiper et al./Computers & Education 52 (2009) 668–680
679

Page 13

Author's personal copy
Bilal, D. (2000). Children’s use of the Yahooligans! search engine. I. Cognitive, physical and affective behaviors on fact-based search tasks. Journal of the American Society for
Information Science, 51(7), 646–665.
Burbules, N. C., & Callister, T. A. (2000). Watch IT. The risks and promises of information technologies for education. Boulder/Oxford: Westview Press.
Burke, J. (2002). The Internet Reader. Educational Leadership, 60(3), 38–42.
Coiro, J. (2003). Reading comprehension on the Internet: Expanding our understanding of reading comprehension to encompass new literacies. The Reading Teacher, 56(6).
Retrieved May 13, 2003, from <http://www.readingonline.org/electronic/elec_index.asp?HREF=/electronic/RT/2-03_column/index.html>.
Enochsson, A. (2005). The development of children’s Web searching skills – A non-linear model. Information Research, 11(1) paper 240. <http://InformationR.net/ir/11-1/
paper240.html>.
Fidel, R., Davies, R. K., Douglass, M. H., Holder, J. K., Hopkins, C. J., Kushner, E. J., et al. (1999). A visit to the information mall: Web searching behaviour of high school students.
Journal of the American Society for Information Science and Technology, 50(1), 24–37.
Frechette, J. D. (2002). Developing media literacy in cyberspace. Pedagogy and critical learning for the twenty-first-century classroom. Westport/London: Praeger.
Hoffman, J. L., Wu, H.-K., Krajcik, J. S., & Soloway, E. (2003). The nature of middle school learners’ science content understandings with the use of on-line resources. Journal of
Research in Science Teaching, 40(3), 323–346.
Jones, B. D. (2002). Recommendations for implementing internet inquiry projects. Journal of Educational Technology Systems, 30(3), 271–291.
Kuhlemeier, H., & Hemker, B. (2005). Computergebruik thuis en internetvaardigheid in het voortgezet onderwijs. [The impact of the internet and computer use at home and
computer skills in the first stage of Dutch secondary education]. Pedagogische Studiën, 82(2), 115–136.
Kuiper, E., Volman, M., & Terwel, J. (2005). The Web as an information resource in K-12 education: Strategies for supporting students in searching and processing information.
Review of Educational Research, 75(3), 285–328.
Kuiper, E., Volman, M., & Terwel, J. (2008a). Integrating critical Web skills and content knowledge: Development and evaluation of a 5th grade educational program.
Computers in Human Behavior, 24, 666–692.
Kuiper, E., Volman, M., & Terwel, J. (2008b). Students’ use of Web literacy skills and strategies: Searching, reading and evaluating Web information. Information Research, 13(3),
paper 351. Available from <http://InformationR.net/ir/13-3/paper351.html>.
Large, A., & Beheshti, J. (2000). The Web as a classroom resource: Reactions from the users. Journal of the American Society for Information Science, 51(12), 1069–1080.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York: Cambridge University Press.
Leu, D. J., Kinzer, C. K., Coiro, J., & Cammack, D. W. (2004). Towards a theory of new literacies emerging from the Internet and other information and communication
technologies. In R. B. Ruddell & N. Unrau (Eds.), Theoretical models and processes of reading (5th ed., pp. 1570–1613). Newark, DE: International Reading Association.
Lorenzen, M. (2001). The land of confusion? High school students and their use of the World Wide Web for research. Research Strategies, 18, 151–163.
Metzger, M. J., Flanagin, A. J., & Zwarun, L. (2003). College student Web use, perceptions of information credibility, and verification behavior. Computers & Education, 41,
271–290.
Pritchard, A., & Cartwright, V. (2004). Transforming that they read: Helping eleven-year-olds engage with Internet information. Literacy, 38(1), 26–31.
Salomon, G. (1998). Novel constructivist learning environments and novel technologies: Some issues to be concerned with. Research Dialogue in Learning and Instruction, 1(1),
3–12.
Schacter, J., Chung, G. K. W. K., & Dorr, A. (1998). Children’s Internet searching on complex problems: Performance and process analysis. Journal of the American Society for
Information Science, 49(9), 840–849.
Shenton, A. K., & Dixon, P. (2003). A comparison of youngsters’ use of CD-ROM and the Internet as information resources. Journal of the American Society for Information Science
and Technology, 54(11), 1029–1049.
Snyder, I. (1998). Beyond the hype: Reassessing hypertext. In I. Snyder (Ed.), Page to screen. Taking literacy in the electronic era. London/New York: Routledge.
Snyder, I. (2002). Literacy education in the digital age: Reframing curriculum and pedagogy. Pedagogisch Tijdschrift, 27(2/3), 145–157.
Soloway, E., & Wallace, R. (1997). Does the Internet support student inquiry? Don’t ask. Communications of the ACM, 40(5), 11–16.
Sutherland-Smith, W. (2002). Weaving the literacy Web: Changes in reading from page to screen. The Reading Teacher, 55(7), 662–669.
Ten Dam, G., Volman, M., & Wardekker, W. (2004). Making sense through participation: Social differences in learning and identity development. In J. van der Linden & P.
Renshaw (Eds.), Dialogic learning (pp. 63–85). Dordrecht: Kluwer.
Wallace, R. M., Kupperman, J., Krajcik, J., & Soloway, E. (2000). Science on the Web: Students on-line in a sixth-grade classroom. The Journal of the Learning Sciences, 9(1),
75–104.
Watson, J. S. (1998). ‘‘If you don’t have it, you can’t find it.” A close look at students’ perceptions of using technology. Journal of the American Society for Information Science,
49(11), 1024–1036.
Wells, G. (2000). Dialogic inquiry in education: Building on the legacy of Vygotsky. In C. D. Lee & P. Smagorinsky (Eds.), Vygotskian perspectives on literacy research: Constructing
meaning through collaborative inquiry. Cambridge: Cambridge University Press.
Yin, R. K. (2002). Case study research (3rd ed.). Design and methods. Thousand Oaks, CA: Sage Publications.
680
E. Kuiper et al./Computers & Education 52 (2009) 668–680