Content uploaded by Bonnie J. F. Meyer
Author content
All content in this area was uploaded by Bonnie J. F. Meyer on May 15, 2014
Content may be subject to copyright.
Wijekumar, K., & Meyer, B. J. F. (2006). Design and pilot of a web-based intelligent tutoring
system to improve reading comprehension in middle school students. International Journal of
Technology in Teaching and Learning, 2(1), 36-49.
________________________________________________________________________
Kay Wijekumar is an Assistant Professor of Information Sciences and Technology at The
Pennsylvania State University Beaver, Bonnie J. F. Meyer is a Professor of Educational
Psychology at The Pennsylvania State University. Please contact Dr. Wijekumar at 212
Administration Building, 100 University Drive, Monaca, PA 15061, E-mail: kxw190@psu.edu.
This research was funded by the US Department of Education, Institute of Education Sciences.
Design and Pilot of a Web-Based Intelligent
Tutoring System to Improve Reading
Comprehension in Middle School Students
Kay Wijekumar
The Pennsylvania State University-Beaver
Bonnie J. F. Meyer
The Pennsylvania State University
Children and adults of all ages have improved their
reading comprehension when trained to use the Structure
Strategy to read, understand, and recall information from
expository text. Training on the structure strategy has
been conducted in small groups or using one-on-one
tutoring and some web-pages with email tutoring, all
showing significant improvements in comprehension.
We are creating a web-based intelligent tutoring system
to present the strategy to 5th-7th grade students. The
advantages of web-based tutors are: consistent tutoring
technique, modeling of good practice, scaffolding,
immediate feedback, and motivation. This paper
presents the foundations of our project Intelligent
Tutoring for the Structure Strategy (ITSS), supporting
research for the design, and preliminary findings from
pilot tests.
Keywords: Intelligent Tutoring, Reading
Comprehension, Structure Strategy
INTRODUCTION
Reading comprehension is a fundamental unit of all human activities. Students in
elementary and middle schools need to read and comprehend information to succeed in
all their future activities. The National Center for Education Statistics annual report on
reading showed that 37% of fourth graders and 26% of eighth graders performed below
basic levels of reading (NCES, 2003). The Structure Strategy (SS) is one approach to
International Journal of Technology in Teaching & Learning 37
improving students’ reading comprehension (Meyer et al., 2002). Extensive research on
the SS has shown significant improvements in reading comprehension for readers of all
ages (Meyer et al., 2002). The SS teaches students that expository texts have six basic
text structures: Comparison, Problem and Solution, Cause and Effect, Description,
Sequence, and Listing. These structures have unique signaling words that should cue the
student to organize the reading and recall using this knowledge. Extensive research has
been conducted with human tutors training students on using the Structure Strategy and
the results have been overwhelmingly positive (Armbruster, Anderson, & Ostertag, 1987;
Bartlett, 1978; Carrell, 1985; Cook & Mayer, 1988; Englert & Hiebert, 1984; Gordon,
1990; Meyer, 1999; Meyer & Poon, 2001; Meyer, Young, & Bartlett, 1989; Paris, Cross,
& Lipson, 1984; Polley, 1994). The methods employed in these studies require trained
human tutors and many tutors to expand the training to a larger audience in schools.
The solution to this problem of needing a large number of trained tutors is the use of
“intelligent” computer tutors to train students on using the Structure Strategy. The
intelligent tutors can provide good modeling, motivation, feedback, consistency, and
access. Based on extensive review of the available technologies and research we have
chosen web-based intelligent tutoring systems with software agents as our approach to
provide this solution. Our US Department of Education – Institute of Education Sciences
funded project is designed to create an Intelligent Tutoring system for the Structure
Strategy (ITSS) and research the effectiveness of the ITSS in 5th-7th grade students.
This paper reports on our goals, approach to designing the ITSS, review of relevant
research, and pilot test.
DESIGNING THE ITSS
The goals of the ITSS are to help children learn to use the Structure Strategy to read
and comprehend expository text. Our design goals for the ITSS are to build a motivating,
easily accessible, and interactive environment (Meyer & Wijekumar, in press).
Motivational components are based on the age of the learners, current topics of interest,
school curriculum, interviews with students, experiences of other researchers working
with children, teacher comments, and a reward system. The use of a web-browser with a
Flash plug-in makes the ITSS easily accessible. Finally, the system is interactive. ITSS
enables us to model Structure Strategy use, collect students’ responses to questions,
assess students’ responses, and provide immediate and consistent feedback on their
performance.
These design elements were synthesized from reviewing the current research and our
experiences. The review of research described next included the Structure Strategy,
multimedia learning, and intelligent tutoring environments.
STRUCTURE STRATEGY RESEARCH
Research on the Structure Strategy showed that the method helps readers to focus on
the text organization, helping them organize their reading accordingly, and showing
significant improvement in recall of expository text (e.g., Meyer, 1999). Meyer, Young,
& Bartlett (1989) found that the strategy helped readers remember more ideas and more
important ideas, as well as remembering the ideas longer.
Some students fail to succeed in tasks such as identifying main ideas from expository
text and giving cohesive and complete accounts of what they read because of how they
read, rather than because they do not read. Meyer, Brandt, and Bluth (1980) showed that
good readers in the ninth grade (as measured by standardized reading achievement tests
and corroborated by teacher appraisals) could use the structure strategy, while most poor
Web-Based Intelligent Tutoring System & Reading Comprehension 38
readers could not.
In a recent pilot study conducted by Dr. Meyer, we randomly sampled 33% of the
fifth-grade students from large, suburban middle school noted for high academic
achievement (Reading Comprehension subtest of the Comprehensive Testing Program
[CTP] [Educational Records Bureau, 1992] = 72 percentile). We found that only 30% of
the students organized recall of information remembered from a short newspaper article
with a problem part and a solution part, giving evidence for using the structure strategy.
Most of the students organized recall as a list of unrelated facts, rather than ideas
interrelated with a problem/solution structure. Clearly, specific difficulty in using the
structure of text in reading for meaning is empirically linked to low levels of academic
achievement in reading.
After the primary grades, children increasingly are expected to learn from expository
text in science, history, social studies, and current events (Gersten, Fuchs, Williams, &
Baker, 2001; Wilson & Rupley, 1997). Reading comprehension from such text is critical
for academic success in school (National Educational Goals Panel, 1999). Understanding
and remembering information from expository text is important throughout the life span
to further develop intellectual abilities (Ackerman, 1998) and maintain functional
independence (e.g., health maintenance and financial management, Meyer, Talbot, Poon,
& Johnson, 2001). As society becomes more technologically advanced, understanding
expository text becomes increasingly important (Alexander & Jetton, 2000; Gersten et al.,
2001; Lapp, Flood, & Ranck-Buhr, 1995).
Instruction about text structures has yielded positive effects for understanding and
remembering information from text with children, young adults, and older adults
(Armbruster et al., 1987; Bartlett, 1978; Carrell, 1985; Cook & Mayer, 1988; Englert &
Hiebert, 1984; Gordon, 1990; Meyer, 1999; Meyer et al., 2002; Meyer & Poon, 2001;
Meyer et al., 1989; Paris et al., 1984; Polley, 1994; Raphael & Kirschner, 1985; Richgels,
McGee, Lomax, & Sheard, 1987; Samuel et al., 1988; Slater, Graves, & Piche, 1985;
Taylor & Beach, 1984; Weisberg & Balajthy, 1989).
MULTIMEDIA & MOTIVATION RESEARCH
Multimedia research is an integral part of any web-based intelligent tutoring system
because the intelligent tutoring systems use the multimedia capabilities to present the
content. Current research on multimedia learning effects suggest that students’
perceptual channels should not be overloaded with too much text or unrelated graphics
(Mayer & Moreno, 2003; Mayer, 2001). Students in multimedia environments attend to
listening and text better than multiple text windows that compete for their attention
(Mayer, 2001).
Research on motivation shows that students’ engagement is enhanced with the use of
more interesting (both situational and personal interest; e.g., Alexander & Jetton, 2000;
Hidi, 1990) authentic text selections, student choices of text passages, immediate and
encouraging feedback, content goals embedded in the lessons, and ways for students to
track individual goal setting related to mastering the strategy (Wijekumar, Meyer,
Ferguson, & Wagoner, 2006).
INTELLIGENT TUTORING RESEARCH
Intelligent Tutoring (IT) research shows that they are very effective in learning
environments. IT is a field of research with great promise in reaching new audiences,
maintaining consistency, motivating, providing immediate feedback, and delivering
effective tutoring (Anderson, Corbett, Koedinger, & Pelletier, 1995). The intelligent
International Journal of Technology in Teaching & Learning 39
tutors range from simple text based interface tutors using key-word searches to find the
correct responses to human-like talking heads that use Latent Semantic Analysis (LSA)
or Bayesian reasoning to interact with the users. Current applications of these agents
include physics tutoring (Graesser et al., 1999), promoting reflective thinking (Baylor,
2001a), teaching biology (Moreno, 2001), and psychological counseling for families
(Johnson, 2001). Research has shown that these agent tutors are believable (Moreno,
2001), improve performance by at least 1 standard deviation (SD) compared to human
tutors 2 SD (Cohen, Kulik, & Kulik, 1982), and are useful to convey feedback through
their facial mannerisms (Graesser, Person, & Magliano, 1995; Link et al., 2001; Person et
al., 2001).
Baylor (2001b) has also shown that students interacting with software agents attend to
the advice, consider the advice and agent useful, and enjoy working with agents. A
tutoring system for reading showed significant improvement in performance of high
school students (Salomon, Globerson, & Guterman, 1989); and the technologies for
creating an intelligent tutoring system have improved a great deal since then.
ITs use a variety of approaches like adapting to learners needs, adjusting the grain size
of learning units (from simple to complex), promoting transfer of skills to different
contexts, allowing students to abstract skills, modeling good problem solving approaches,
providing appropriate feedback (based on students’ input/responses), and adapting to
changing conditions (adapted from Andersen et al., 1995).
Creating ITs usually requires the identification of learning tasks, creation of an ideal
approach to learn the tasks (using task analysis, think-aloud problem solving), creation of
possible tutor-student interactions (using expert tutors who are knowledgeable in the
subject as well as tutoring), and programming the interactions into a computer system
(Anderson et al., 1995). The computer systems that support these ITs include parsers
(that process the student input), meaning making routines (that categorize students’
responses), identification of the current state of the tutoring interactions, rules that govern
the appropriate tutor responses, and the visible interface.
RESEARCH ON WEB-BASED TRAINING FOR THE STRUCTURE STRATEGY
A previous study by Meyer et al. (2002) provided valuable information on students
working in web-based environments. That research study provided instruction on the
Structure Strategy using static web-pages. Students were asked questions and their
responses were emailed to human tutors who were trained in assessing the students’
responses and providing email feedback. Some very important findings from this study
have influenced the design of our ITSS significantly. They are:
• Training human tutors is expensive and human tutors are not always consistent.
• Students require extrinsic motivation and monitoring to complete lessons outside
the traditional classroom environment.
• Students frequently skip ahead and do not read many of the passages and
instructions (This has been reported in previous studies of multi-media learning
(Jacobson, Maouri, Mishra, & Kolar, 1995).
• Students made lots of spelling errors in their typing and also wrote short responses.
• Students with low prior knowledge and poor reading skills (as assessed by the
teachers and test scores) appeared to be discouraged by numerous attempts to
complete their tasks. This was also reported by one of the consultants on our
project who is a 5th grade teacher for many years.
This research study was the primary source of information for our lesson choices,
modeling, interaction design, student responses in Latent Semantic Indexing
Web-Based Intelligent Tutoring System & Reading Comprehension 40
(Intelligence), and feedback.
DESIGN THEMES FOR ITSS
Based on the research reviewed above, we chose the following design themes for the
ITSS environment:
1. The interface must engage student to read and practice using the SS – current
topics of interest, age appropriate questions, and constructive responses.
2. The interface is designed to project a “learning” environment and not a game-like
“play” environment.
3. Students’ visual channels should not be overloaded by multiple windows and
texts. Narration can be one method to encourage students to listen and read
along.
4. Assessment should include multiple constructs like finding signaling words,
writing main ideas, recalling details, and using text structures to organize recall.
5. Feedback should be constructive.
6. Students must be able to interact with the ITSS as a consistent tutoring
environment similar to working with a trained tutor.
7. Students should not be discouraged by too many tries.
8. Students should not have the opportunity to skip over sections especially when
they are not able to use the SS effectively.
The system we created using these design themes, ITSS, has the following features:
1. Infrastructure – the system we designed had to support at least 500 concurrent
users from school districts. The system had to be easily accessible, interactive,
and allow video/audio/animation. To accomplish this goal we chose a .NET
platform with Flash to create the animations and interactions. The system resides
on a Windows 2003 server with a SQL Server database storing all the
interactions and assessment (Dell Dual Processor Server with Microsoft
Windows Server 2003, database – SQLserver 2000, Webserver – IIS, Sun Solaris
Server for Latent Semantic Indexing).
2. Intelligence - The “intelligence” of the system comes from using Latent Semantic
Indexing (LSI) software that learns from each interaction with a student. For
example, the system is loaded with all the responses from students in the pilot
study (The good responses as well as the poor responses). The database
maintains scores for all these responses. When a new student response comes
into the system, it is checked against existing responses and if it is an 80% match
to an existing response then we know how to score the response. If there is no
match in the LSI space, we score the response using keywords and then
incorporate the new response into the LSI space for future use.
3. Interface – Figure 1 shows the interface for ITSS with a “tutor”
affordance/metaphor and the screens looking like a book. Color coded tabs on
the “book” show each structure as the student progresses. Five tabs for each
structure (Comparison, Problem/Solution, Cause/Effect, Description, and Listing)
light up as they start work on that structure. As they complete approximately 10
lessons for each structure, the tab fills up with colors to show student progress in
learning the structure.
4. Individualized - Students will progress through lessons at their own pace (with
unique usernames and passwords) and can start at the lesson where they left off
during the previous session. Additionally, any students completing all the text
structures can transition to the Pennsylvania System of School Assessment
(PSSA) tests for 5th and 8th grade that have been adapted to work in the ITSS
International Journal of Technology in Teaching & Learning 41
system.
5. Interactions – Students are given earphones to listen to the narration of the
lessons and respond to questions by clicking on words or typing answers.
6. Software Agent with Narration- Students interact with a software agent who is a
Caucasian male with a voice narrated by a young adult.
7. Gradually Increasing Complexity - Allow students to apply the SS in text
passages of varying complexity. The system also shows students how text
structures can be embedded within another structure.
8. ITSS Model - Figure 2 shows the model of the intelligent tutoring system. The
system starts operations with the login of the user. The database verifies the
username and password and then controls the system interaction with the user.
Using the human-modeled lessons as the basis for interactions, the system
presents the user with modeling, readings, questions, and responses. The student
interacts with the system by clicking on buttons, words, or typing responses.
When the system receives the student responses, it checks spelling and replaces
words, evaluates the response based on the questions, user’s try (maximum tries
is usually three), keyword matches against the database, and finally a similarity
match against the LSI space. The keyword matches with the database use
recursive procedures to traverse tree structures for each reading passage.
9. Navigation - Navigation buttons to move to the next page do not appear on the
screen until the student has completed listening to the narration.
10. Documentation - User documentation is available for the students and teachers.
This documentation includes a user guide and “Keys” for each text structure to
server as a job aid when the student uses the system or for use during their
regular classes.
11. Diagnostic Reports - The system is capable of producing diagnostic reports for
teachers and students to track their progress as well as comprehension.
Figure 1. ITSS Interface
Web-Based Intelligent Tutoring System & Reading Comprehension 42
Application invocation
using WEB browser
Invoked by .NET
application
Transition
Parse Student Response
Tutee/Student
Via C# & ActiveX
Response
Response
(Database, XML, &
Flash)
Authenticate
Access
<use>
<use>
<use>
Verification of State
<use>
WordNet
Penn Treebank
Figure 2. Interactions in ITSS
STEPS TO IMPLEMENTING THE ITSS
1. Lessons for the ITSS are adapted from Meyer et al. (2002) with new lessons with
current interest added to motivate students. Approximately 10 lessons for six
basic expository text structures (description, sequence, cause/effect,
problem/solution, comparison, and listing) have been developed. Reading
passages for each lesson were chosen based on text structure, good use of
signaling words, varying complexity, and motivation. Passages also varied from
single sentences that allow the student to focus on single signaling words to very
complex prose that have nested text structures that contain hierarchies and many
signaling words.
2. Modeling of Interactions was created by combining data from previous research
study by Dr. Meyer and a 5th grade teacher who modeled the lessons. A detailed
lesson plan with storyboard was created combining passages, explanations,
questions, and answers.
3. Animations and interactions were created based on the storyboard using Flash™
and narration. All web-pages present information and use C# programs to collect
and process responses from students. Narrations are created by a young adult
working with the design team.
4. Mimicking a human-tutor, ITSS uses a human-like agent to make the interface
more realistic. All the human expressions and gestures are created in
Truespace™ and saved as Flash™ files. The tutor is a Caucasian male with three
dimensional modeling of over facial expressions (ranging from negative, re-
assurance, smiling, nodding, and prompting). The tutor also has narrated voice
responses programmed with the facial expressions.
5. Interactions between students and ITSS were adapted using the log files of all the
interactions between the students and tutors the Meyer et al. (2002) study. These
International Journal of Technology in Teaching & Learning 43
interactions were reviewed and coded by trained independent raters. The scores
for main idea, signaling words, text structure, details, and intrusions were coded
into the database and linked to the LSI space.
6. Assessment and scoring of student responses were based on the similarity to
existing responses and/or keyword matches to the reading passages. The
assessment was programmed using a recursive tree to track the important parts of
the passages. For example, Figure 3 shows the tree for the passage on Elephants.
7. Feedback and responses to students were chosen based on their scores. When a
student provides a good response with scores of over 70% on the main idea,
signaling word, details, and text structure, they are shown an animated agent
saying “Good Job”. When one or more of the scores was lower, the agent gave
them detail hints on what they need to improve.
8. Minimizing frustration with the ITSS, we have incorporated a three-try limit to
each question. After each try the learner is given more detailed hints on
correcting their work. If the student fails to improve their response after the third
try they are shown the correct answer and the system transitions them to the next
task.
9. Motivational components currently included in the design are positive comments
by the software agent (“You are really doing well”), referring to the student as
“Intelligent” when they complete their tasks with good scores, and progress
indicators on the tabs of the book interface.
Elephants
Two different kinds of elephants exist today; these two types are the African
elephant and the Indian elephant. These interesting creatures differ
dramatically in ears, backs, and how long they live. African elephants have
very large ears. Their backs arch down in the middle. African elephants live
50 to 60 years.
Indian elephants have small ears. The backs of the Indian elephants arch up
in the middle. They live 70 to 80 years.
Figure 3. Elephant Tree
Comparison
ELEPHANTS
[Signaling =
DIFFERENT,
DIFFER, IN
CONTRAST
equally
favored:
AFRICAN
ELEPHANTS
ears:
LARGE
EARS backs in
middle:
ARCH DOWN
life span:
50-60 YEARS
equally
favored:
INDIAN
ELEPHANTS
ears:
SMALL
EARS
backs in
middle:
ARCH UP
life span:
70-80
YEARS
Web-Based Intelligent Tutoring System & Reading Comprehension 44
PILOT TEST OF THE ITSS
Our ITSS system was pilot tested to evaluate the quality of the interface, interactions,
and learning environment. Another goal of the pilot test was to identify reading
preferences of 5th and 7th grade students. Extended testing of the ITSS is scheduled to
begin in shortly.
Fifteen participants from grades 4 through 8 volunteered to participate. Participants
completed the informed consent forms, demographic survey, a 21-item (five-point Likert
scale) Computer use and opinion questionnaire (Krauss & Hoyer, 1984; Meyer & Poon,
1997), a 23-item self-efficacy questionnaire (4-point Likert scale; reliability = .86)
(Sherer et al., 1982), and a 20-item Motivation to Read profile (Gambrell, Palmer,
Codling, & Mazzoni, 1996). Participants then completed either Form A or Form B
(counterbalanced) researcher designed pre test measure of reading comprehension. The
test had recall, writing a title for a passage, and cloze tasks. After the completion of these
measures students viewed and interacted with the ITSS sample lessons 1-3 (Comparison
Structure). Students were also asked to review the selection of passages and state
whether they would be interested in reading about those topics. At the conclusion of their
interactions students completed the post test measure and were interviewed by the
researcher.
The interview questions were open ended and were directed at their opinion about IT
(the intelligent tutor), sounds, passages, and affordances of the system. The researcher
also maintained a log of time to complete tasks like finding signaling words, typing
responses, and clicking on buttons.
Additionally, two reviews were conducted by consultants and focus groups to evaluate
the usability, effectiveness, motivation, and overall flow of interactions in the ITSS. First,
two subject matter experts, expert tutors, two technical experts, and 10 undergraduate
students reviewed the content, technical features, and layout of the ITSS. The following
elements of the ITSS were reviewed: passage selections, interaction criteria, animated
agent design, technical specifications, modeling, and feedback. Second, one researcher
conducted observations of participants using the ITSS. The observation included a
modified Click Response Time Test and Memory Test (Tullis, et al., 2000). Logged
interactions of the participants were reviewed for consistency as well as learning
outcomes.
The following are sample questions from each research area:
1. Aesthetics, Consistency, and Standards
a. Are the actions, terms, icons, and menus consistent throughout the
lessons?
b. Are colors, layout and fonts consistent and visually pleasing?
c. Is the text size and denseness readable?
d. Is the IT personality friendly and engaging?
e. Was the sound quality good?
f. Were the directions from IT understandable?
g. Was the interaction style of IT – point to text, requesting input, and
feedback useful?
h. Are the selections, user input, feedback, and options consistent and
understandable?
i. Does the level of consistency and standards satisfy the needs of a 5th-7th
grade audience?
j. Does the screen layout focus attention on the important areas?
2. Recognition vs. Recall
a. Does the “Tutor” metaphor get conveyed effectively in the system?
International Journal of Technology in Teaching & Learning 45
b. Does the human-like tutor help you feel at ease with the system?
c. Does the human-like tutor speed up recognition and encourage you to
learn from the system?
3. Visibility of System Status
a. Did you feel that you were in control of the system?
b. Did you know how much of the lesson you had completed?
c. Was the system response time good?
d. What caught your attention on the first page of ITSS you saw today?
4. Providing Informative Feedback
a. Was the feedback from IT informative?
b. Were you able to understand the feedback and act on it?
c. Were you able to listen to the feedback again?
5. User Control and Freedom
a. Were you able to undo any accidental actions?
b. Were you able to navigate through the ITSS easily?
c. Were you feeling confident in your ability to use the ITSS?
6. Error Prevention and Handling
a. Were the error messages useful to you?
b. What caused error messages to appear?
c. Did the system lock-up or stop functioning at any time?
d. Did you know how to recover from the error?
7. Help and Documentation
a. Were the help and documentation sections useful?
b. Did the written guide help in navigating the ITSS?
c. Did you use the “keys”?
d. How useful were the “keys”?
PILOT TEST RESULTS
Participants of all age groups expressed positive attitudes towards IT. Two
participants suggested that IT looked like a “football player”. Four participants said that
they liked working with a “person” suggesting that they related to the software agent as a
human. Participants also said they liked the sounds. Four participants from 7th and 8th
grade stated that they did not like having the passages read to them.
Most importantly, all participants had no prior knowledge of text structure and what a
main idea is. Two participants from 7th grade who have very high standardized reading
scores were surprised that text structure was useful in remembering more information.
They were quick to ask what other text structures were there and what types of signaling
words are used.
All participants found the narration and instructions understandable but some
experienced difficulty in just listening to the instructions (when the instructions did not
appear on the page). An analysis of keystrokes and time for performing tasks found that
there were no major problems with the interface. Students progressed along the lessons
with no problem spots.
All participants took longer times to type their responses than when they had to write
responses on a sheet of paper. In spite of this timing factor, each lesson lasted
approximately 40 minutes. The design team originally estimated times for each lesson to
be between 20 to 40 minutes.
One participant described a scenario in his school where a computer program was
introduced to improve computer usage like browsing the Internet. The participant
reported that a few students in the group class started trying to thwart the system and
Web-Based Intelligent Tutoring System & Reading Comprehension 46
making negative comments. He said that the momentum for the negative comments grew
to the point where almost all students participating in the lessons were not interested in
learning and instead played games with the system trying to break it.
During the ITSS pilot testing participants were shown a list of titles of passages for
lessons with parallel choice lessons. They were asked to choose the passages they would
prefer to read or suggest alternatives. Older participants were more vocal in their lack of
interest in the chosen reading passages like animals and history. Instead, they suggested
passages on sports and music industry figures. All participants suggested that they would
read the passages that were currently available, but preferred to read about sports and
music industry figures.
Based on pilot results we have made the following modifications to the ITSS system:
• New passages have been added to motivate the students to use the structure
strategy. For example, new passages have been added comparing Olympic athletes.
• More on-screen instructions were added to aid the students in completing tasks like
choosing the number of signaling words or writing responses.
• More point and click questions were included to minimize the typing required (for
example, clicking on signaling words instead of typing them).
• Additional motivational elements are being discussed. For example, adding a game
at the end of each text structure as a reward for completing the tasks well.
CONCLUSION
The preliminary phase of our project is underway and has concentrated on the research
and design of the ITSS, pilot study, modifications, and final preparation for the large-
scale study is planned. Fifth and seventh grade students will spend two to three class
sessions of 30 to 40 minutes in duration each week for seven months working on ITSS
during the regular school day. The year spent in development, pilot testing, and final
preparation was critical in preparing a quality system ready for further testing of
components when incorporated into a school district’s curriculum.
REFERENCES
Ackerman, P. (1998). Adult intelligence: Sketch of a theory and applications to learning
and education. In M. Cecil Smith & T. Pourchot (Eds.) Adult learning and
development: Perspectives from educational psychology (pp. 179-200). Mahwah, NJ:
Lawrence Erlbaum.
Alexander, P. A., & Jetton, T. L. (2000). Learning from text: A multidimensional and
developmental perspective. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R.
Barr (Eds.) Handbook of reading research Vol. III (pp. 285-310). Mahwah, NJ:
Lawrence Erlbaum.
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive
tutors: Lessons learned. The Journal of the Learning Sciences, 4(2), 167-207.
Armbruster, B., Anderson, T., & Ostertag, J. (1987). Does text structure/summarization
instruction facilitate learning from expository text? Reading Research Quarterly, 22,
331-346.
Bartlett, B. J. (1978). Top-level structure as an organizational strategy for recall of
classroom text. Unpublished doctoral dissertation, Arizona State University.
Baylor, A. (2001a). Cognitive requirements for agent-based learning environments.
Proceedings of the IEEE International Conference on Advanced Learning
Technologies: Issues, Achievements, and Challenges. (pp. 462-464).
International Journal of Technology in Teaching & Learning 47
Baylor, A. (2001b). Effects of MIMICing instructional theory with MIMIC. Paper
presented at the AECT International Conference, Atlanta, GA.
Carrell, P. L. (1985). Facilitating ESL reading by teaching text structure. TESOL
Quarterly, 19, 727-752.
Cohen, P. A., Kulik, J. A., & Kulik, C. C. (1982). Education outcomes of tutoring: A
meta-analysis of findings. American Educational Research Journal, 19, 237-248.
Cook, L. K., & Mayer, R. E. (1988). Teaching readers about the structure of scientific
text. Journal of Educational Psychology, 80, 448-456.
Educational Records Bureau. (1992). Comprehensive Testing Program III, Levels C-G.
New York: Educational Records Bureau.
Englert, C. S., & Hiebert, E. H. (1984). Children's developing awareness of text
structures in expository materials. Journal of Educational Psychology, 76, 65-74.
Gambrell, L. B., Palmer, B. M., Codling, R. M., Mazzoni, S. A. (1996). Assessing
motivation to read. Reading Teacher, 49, 518-533.
Gersten, R., Fuchs, L. S., Williams, J. P., & Baker, S. (2001). Teaching reading
comprehension strategies to students with learning disabilities: A review of research.
Review of Educational Research, 71, 279-320.
Gordon, C. J. (1990). Contexts for expository text structure use. Reading Research and
Instruction, 29, 55-72.
Graesser, A. C., Person, N. K., & Magliano, J. P. (1995). Collaborative dialogue patterns
in naturalistic one-to-one tutoring. Applied Cognitive Psychology, 9, 1-28.
Graesser, A., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N., & the
Tutoring Research Group. (1999). Using latent semantic analysis to evaluate the
contributions of students in AutoTutor. Interactive Learning Environments, 8(2),
129-147.
Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of
Educational Research, 60, 549-571.
Jacobson, M. J., Maouri, C., Mishra, P., & Kolar, C. (1995). Learning with hypertext
learning environments: Theory, design, and research. Journal of Educational
Multimedia and Hypermedia, 4(4), 321-364.
Johnson, W. (2001) Animated pedagogical agents for education training and
edutainment. Proceedings of the IEEE International Conference on Advanced
Learning Technologies: Issues, Achievements, and Challenges. (p. 501).
Krauss, I. K., & Hoyer, W. J. (1984). High technology and the older person: Age, sex and
experience as moderators of attitudes towards computers. In P K. Robinson, J.
Livingston, and J. E. Birren (Eds.), Aging and Technology (pp. 349-350). New York:
Plenum.
Lapp, D., Flood, J., & Ranck-Buhr, W. (1995). Using multiple text formats to explore
scientific phenomena in middle school classrooms. Reading and Writing Quarterly:
Overcoming Learning Difficulties, 11, 173-186.
Link, K. E., Kreuz, R. J., Graesser, A. C., & the Tutoring Research Group. (2001).
Factors that influence the perception of feedback delivered by a pedagogical agent.
International Journal of Speech Technology, 4, 145-153.
Mayer, R. E. (2001). Multimedia Learning. Cambridge, UK: Cambridge University
Press.
Mayer, R. E. & Moreno, R. (2003) Nine ways to reduce cognitive load in multimedia
learning. Educational Psychologist, 38, 43-52.
Meyer, B. J. F. (1999). The importance of text structure in everyday reading. In A. Ram
& K. Moorman (Eds.), Understanding language understanding: Computational
models of reading. Cambridge, MA: MIT Press.
Web-Based Intelligent Tutoring System & Reading Comprehension 48
Meyer, B. J. F., Brandt, D. M., & Bluth, G. J. (1980). Use of the top-level structure in
text: Key for reading comprehension of ninth-grade students. Reading Research
Quarterly, 16, 72-103.
Meyer, B. J. F., Middlemiss, W., Theodorou, E., S., Brezinski, K. L., McDougall, J., &
Bartlett, B. J. (2002). Older adults tutoring fifth-grade children in the structure
strategy via the Internet. Journal of Educational Psychology, 94(3), 486-519.
Meyer, B. J. F. & Poon, L. W. (1997). Age differences in efficiency of reading
comprehension from printed versus computer-displayed text. Educational
Gerontology, 23, 789-807.
Meyer, B. J. F., & Poon, L. W. (2001). Effects of the structure strategy and signaling on
recall of text. Journal of Educational Psychology, 93, 141-159.
Meyer, B. J. F., Talbot, A. P., Poon, L. W., & Johnson, M. M. (2001). Effects of structure
strategy instruction on text recall of older African American adults. In J. L. Harris, A.
Kamhi, & K. Pollock (Eds.), Literacy in African Americans communities (pp. 233-
263). Mahwah, NJ: Erlbaum.
Meyer, B. J. F., Young, C. J., & Bartlett, B. J. (1989). Memory improved: Reading and
memory enhancement across the life span through strategic text structures. Hillsdale,
NJ: Lawrence Erlbaum.
Meyer, B. J. F., & Wijekumar, K. (in press). Web-based tutoring of the structure strategy:
Theoretical background, design, and findings. In D. S. McNamara (Ed.) Reading
comprehension strategies: Theories, interventions, and technologies, Mahwah, NJ:
Lawrence Erlbaum.
Moreno,R. (2001). Contributions to learning in an agent-based multimedia environment:
A methods-media distinction. Proceedings of the IEEE International Conference on
Advanced Learning Technologies: Issues, Achievements, and Challenges. (pp. 464-
466).
NCES. (2003). National assessment of educational progress: The nation’s report card
reading highlights. National center for education statistics. U.S. Department of
Education Institute of Education Sciences, NCES 2004-452.
National Educational Goals Panel. (1999). Reading achievement state by state, 1999.
Washington, DC: U.S. Government Printing Office.
Paris, S. G., Cross, D. R., & Lipson, M. Y. (1984). Informed strategies for learning: A
program to improve children's reading awareness and comprehension. Journal of
Educational Psychology, 76, 1239-1252.
Person, N. K., Graesser, A. C., Bautista, L., Mathews, E. C., & the Tutoring Research
Group (2001). Evaluating student learning gains in two versions of AutoTutor. In J.
D. Moore, C. L. Redfield, & W. L. Johnson (Eds.) Artificial intelligence in
education: AI-ED in the wired and wireless future (pp. 286-293). Amsterdam: IOS
Press.
Polley, R. R. (1994). Facilitating recall through awareness of text structure.
Unpublished masters thesis, University of Kentucky, Lexington.
Raphael, T. E., & Kirschner, B. W. (1985). The effects of instruction in
comparison/contrast text structure on sixth-grade students’ reading comprehension
and writing products (Research Series No. 161). East Lansing: Michigan State
University, Institute for Research on Teaching.
Richgels, D. J., McGee, L. M., Lomax, R. G., & Sheard, C. (1987). Awareness of four
text structures: Effects in recall of expository text. Reading Research Quarterly, 22,
177-196.
Salomon, G., Globerson, T., & Guterman, E. (1989). The computer as a zone of
proximal development: Internalizing reading-related metacognitions from a reading
partner. Journal of Educational Psychology, 81(4), 620-627.
International Journal of Technology in Teaching & Learning 49
Samuels, S. J., Tennyson, R., Sax, L., Mulcahy, P., Schermer, N., & Hajovy, H. (1988).
Adults' use of text structure in the recall of a scientific journal article. Journal of
Educational Research, 81, 171-174.
Sherer, M., Maddux, J. E., Mercandante, B., Prentice-Dunn, S., Jacobs, B., & Rogers, R.
W. (1982). The self-efficacy scale: Construction and validation. Psychological
Reports, 51, 663-671.
Slater, W. H., Graves, M. F., & Piche, G. L. (1985). Effects of structural organizers on
ninth-grade students' comprehension and recall of four patterns of expository text.
Reading Research Quarterly, 20, 189-201.
Taylor, B. M., & Beach, R. (1984). The effects of text structure instruction on middle
grade students' comprehension and production of expository text. Reading Research
Quarterly, 19, 124-146.
Tullis, T.S., Dixon, E.J., Hersh, H.M. (2000). A “Bag of Tricks” for Web Usability. CHI
2000, April 2000. 302.
Weisberg, R., & Balajthy, E. (1989). Transfer effects of instructing poor readers to
recognize expository text structure. In S. McCormick & J. Zutell (Eds.), Cognitive
and social perspectives for literacy research and instruction: Thirty-eighth yearbook
of the national reading conference (pp.279-285). Chicago: National Reading
Conference.
Wentzel, K. R. (2000). What is it that I'm trying to achieve? Classroom goals from a
content perspective. Contemporary Educational Psychology, 25, 105-115.
Wilson, V. L., & Rupley, W. H. (1997). A structural equation model for reading based on
background, phonemic, and strategy knowledge. Scientific Studies of Reading, 1, 45-
63.
Wijekumar, K., Meyer, B. J. F., Ferguson, L., & Wagoner, D. (2006). Technology
affordances: The “real story” in research with K-12 and undergraduate learners.
British Journal of Educational Technology: Special Issue on Technology Effects, 37,
191-209.
