No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Using Interactive E-Book User Log Variables to Track Reading Processes and Predict Digital Learning Outcomes
Abstract
Stealth assessment has been successfully embedded in educational games to measure students’ learning in an unobtrusive and supportive way. This study explored the possibility of applying stealth assessment in a digital reading platform and sought to identify potential in-system indicators of students’ digital learning outcomes. Utilizing the user log data from third- to fifth-grade students ( n = 573) who read an interactive Word Knowledge E-Book, we examined various user log variables and their associations with word knowledge and strategic reading outcomes. Descriptive analysis provided a depiction of the real-time reading processes and behaviors in which students engaged while digitally reading. Multiple regression analysis with classroom fixed effects demonstrated that user log variables relevant to answering questions and making decisions (i.e., percentage of embedded questions answered correctly; number of attempts to answer the questions; and making implausible decisions) were significantly associated with students’ word knowledge and strategic reading outcomes. Variables indicating reading time and frequency, however, were not significantly associated with these outcomes. This study highlights the potential of interactive e-books as another digital learning environment to establish stealth assessment, which may allow researchers and educators to track students’ reading processes and predict reading outcomes while supporting digital learning.
... Dynamic behavioural data can accurately describe the time and, by proxy, the effort that students dedicate to a target skill. A relevant study to investigate this dimension of engagement was conducted by Yang et al. (2020) who used six interactive e-book user log variables to identify a student's in-the-moment reading processes. They found that log variables, such as time spent reading, frequency of reading the book, and number of attempts to get the correct answer, were associated with word knowledge and strategic reading outcomes. ...
... Our choice of engagement indicators was informed by the literature (e.g., Du et al., 2021;Henrie et al., 2015;Sinatra et al., 2015;Yang et al., 2020). We created six game-based indicators to summarise information across usage, considering both the nature of the game and the data, while avoiding redundancy: all hours spent, variety of games played, all attempts, the number of levels played, days of playing, and proportion of early exit rounds. ...
... Most students demonstrated good engagement with the app and positive progress on the out-of-game assessment, Mind the Gap. In line with a growing body of research into digital educational technology and literacy (see Biancarosa & Griffiths, 2012;Diprossimo et al., 2023;Nizam & Law, 2021;Yang et al., 2020), our findings suggest that incorporating digital reading supplements in classroom-based activities offers promising opportunities for early literacy education. ...
... The WKe-Books were developed iteratively by embedding reading comprehension strategies to support third-fifth grade students to read for understanding (Connor et al. 2019;Umarji et al. 2021;Yang et al. 2021). The WKe-Books were designed to engage young readers through an interactive design using a choose-your-own-adventure and name-your-character format, while also supporting comprehension skills. ...
... ○ Teacher support beyond the technology is still crucial to consider. metacognitive outcomes (Connor et al. 2019;Umarji et al. 2021;Yang et al. 2021). Specifically, Connor et al. (2019) found significant treatment effects of the WKe-Book on third through fifthgrade students' word-related outcomes in their RCT. ...
Background
The affordances of technology, such as e‐books, offer the opportunity to increase engagement and provide personalised feedback to promote students' learning outcomes. E‐books that encourage the use of comprehension monitoring strategies in real time may support stronger outcomes.
Objectives
The purpose of this feasibility study was to examine the potential impact of reading digital interactive e‐books, Word Knowledge E‐Books (WKe‐Books), that taught word‐meaning and deeper text‐meaning strategies to support reading comprehension with third‐fifth grade students.
Methods
Students (N = 435) read two WKe‐Books, that taught word learning and comprehension monitoring strategies in the service of learning vocabulary and targeted science concepts about hurricanes. We investigated whether specific comprehension strategies–(1) word learning and strategies that supported general reading comprehension, (2) summarisation, and (3) question generation–show promise of effectiveness in building vocabulary knowledge and comprehension skills in the WKe‐Books. Students were assigned to read one of three versions of each WKe‐Book. The books employed a choose‐your‐adventure format with embedded comprehension questions that provided students with immediate feedback.
Results and Conclusions
Students demonstrated significant gains in word learning and the targeted hurricane concepts. Hierarchical Linear Modelling (HLM) revealed that no one strategy was associated with larger gains. Performance on the embedded questions in the books was associated with greater posttest outcomes. These findings suggest that the affordances offered in the WKe‐Books can effectively support students' development of reading‐related skills, including strategy use. Further, this work discusses implications for the future development of e‐books that can enhance engagement and improve reading comprehension.
... • In-game item interaction tracking: Capturing detailed information on interactions with in-game items, including time spent, frequency, and sequence, helps identify students who engage meaningfully with content versus those who struggle (Yang et al., 2021). ...
With the guidance of the Evidence-Centered Design (ECD) framework, we systematically implement stealth assessment for the serious game Mission HydroSci (MHS) and empirically evaluate its effectiveness using combinations of features derived from gameplay and their analysis using computational models. We show that adaptive stealth assessment implemented in a game-based learning environment can provide sufficient and reliable information about learning growth, game-related performance, and the associated human behavioral patterns. Stealth assessment unobtrusively collects information about students' in-game behaviors and progress, which teachers and game designers can leverage to sustain engagement and decrease test anxiety. This paper describes the process of implementing and evaluating the stealth assessment step by step, including raw game log generation, feature engineering process, computational model construction, and model outcome interpretation. The model performance indicates that the computational model using Random Forest with the full feature set outperforms other candidates with an accuracy rate of 86\%. This acceptable threshold empirically proves the reasonability of our workflow of stealth assessment implementation. We interpret the model outcome to summarize the findings by making model inferences with the Bayesian Generalized Linear model (BGL), an interpretable "White-box" model that approaches the second-best performance according to the model performance matrix. The discussion section lists promising practical applications inspired by the model outcome and limitations for future studies.
... The criteria used to assess aspects of the material include four aspects: suitability of the material, ease of delivery, quality of the content, and quality of exercises and tests. The aspects assessed by users include 15 aspects, namely, clarity of instructions for using e-learning, clarity of learning objectives, clarity of material descriptions, readability of text or writing, clarity of audio, giving examples, giving exercises, giving evaluations, clarity of instructions for doing tests, relevance of the material to the evaluation/quiz, student feedback on, clarity of language, video display, freedom to choose menus, increasing interest (Saddhono et al., 2020;Saripudin et al., 2022;Yang et al., 2021). ...
Introduction: Nature-based and outdoor learning approaches have shown promising results in enhancing English language skills and student engagement. These methods allow students to explore and interact directly with their environment. Interactive e-books have shown promise in enhancing English language skills across various educational levels. Objective: The research aims to determine the effectiveness of interactive e-books integrating nature-based and outdoor learning to improve students’ English skills and physical engagement. Methodology: This is descriptive quantitative research. The data collection techniques are interviews, questionnaires, and tests. Results: Based on the analysis, the integrated interactive e-book based on nature-based learning and local wisdom is valid and can be used in the learning process. The average value for each aspect is 4.2, which indicates that the e-book is valid. Discussion: The results of hypothesis testing and gain scores show that students in the experimental class who learned using the integrated interactive e-book based on nature-based learning and local wisdom experienced a more significant increase in learning outcomes than students in the control class. The average student score in the experimental class was 70%, a high category, while the average student score in the control class was 50%, a medium category. Conclusions: So, it can be concluded that integrated interactive e-books based on nature-based learning and local wisdom in English learning effectively increase students’ English skills.
... The literacy and learning landscape has been transformed by technological advancements, especially the internet. With these advances, literacy is no longer limited to the ability to read traditional prints (Yang, 2021). Currently, literacy includes the ability to read and understand digital texts, manage data from various sources, and the ability to think critically and creatively (Swanson et al., 2020). ...
The development of the E-Book system in supporting learning literacy is very beneficial for students in the era of globalization. In today’s era of globalization, it has utilized the E-Book system in various schools in the learning system. The use of E-Books in learning has a positive impact on the present and the future. This study aims to understand the concept of utilizing the E-Book system in supporting students’ literacy and independent learning by utilizing the E-Book system in education and to find out the extent of the development of the E-Book system in education. So that teachers and students can use this E-Book system as a means of learning. The method used by the researcher in conducting this study is quantitative, with data obtained through disseminating questionnaires online using Google Forms. The word was processed using the SPSS application by statistical analysis through the ANOVA test; after completing the data test, the researcher presented the data in the form of graphs and tables containing respondents. The results of this study show that the introduction of an offline-based E-Book system will be able to support the literacy and independent learning of students in the world of education. Using the E-Book system in education brings various benefits, such as increasing student participation and supporting the understanding of complex concepts. Learning that utilizes the E-Book system has progressed compared to learning that still uses its own learning method, and cannot compete in the world of education. This study concludes that utilizing the E-Book system, especially independent learning, positively impacts education. The use of the E-Book system is expected to build interactive and effective learning resources. The use of the e-book system in education shows that it can improve the quality of learning and provide advantages in the educational process.
... Digital intervention in educational settings focuses on improving and facilitating learning access, learning processes, and learning outcomes. It may be learning activities, processes, resources, and various environments driven by digital in educational institutions at various levels, including basic education, vocational education, higher education, informal education, and alternative education (Kuosmanen, Fleming, Newell, & Barry, 2017;Haleem, Javaid, Qadri, & Suman, 2022;Sormunen et al., 2020;Yang, Zargar, Adams, Day, & Connor, 2021;O'Connell & Lucić, 2021;Akour & Alenezi, 2022). Examples of digital interventions in educational settings include the use of Internet-based learning, hybrid learning, MOOC platforms, mobile learning apps, student relationship management systems, and virtual classrooms (Rattanasak, 2023;Napaporn et al., 2023;Nittayathammakul et al., 2023;Kohpeisansukwattana, Siriwattananon, & Charoenwanit, 2024;Kaewrattanapat et al., 2023;Raes et al., 2020). ...
This research aims to identify factors predicting pre-service technical teachers’ academic performance in a technical teacher training context. This study employed a predictive correlational design. We developed the conceptual framework by combining previous research, and then created a survey to gather data. Between August 1 and August 30, 2022, 109 undergraduates from the Faculty of Technical Education at Rajamangala University of Technology Krungthep (RMUTK) received an online self-administered questionnaire. The statistical analysis employed the Pearson correlation coefficient and multiple regression. The findings of the research showed that there was a positive correlation between various factors such as gender, motivation and attitude towards learning, study habits, family support, curriculum quality, and teaching quality with pre-service technical teachers' academic performance, with the correlation coefficients ranged from .230 to .292 and were all statistically significant (p < 0.05). The multiple correlation coefficient (R) was .474 showed a significant relationship between the independent and dependent variables at the .05 level. The R-squared value was .225, indicating that these six variables combined explain 22.5% of the variation in academic performance. However, this also suggests that our model fails to explain around 77.5% of the variance. Some aspects of the findings derived from this study are expected to result in the creation of digital interventions to better track students' academic performance, aiming to provide equitable educational experiences that maximize the academic performance of each gender group in the future.
... The challenge arises as to how to use this technology to support learning to be more meaningful. Some research results show student problems when faced with digital learning, including some Students turning pages without reading and guessing answers randomly without thinking carefully (Baker et al., 2008, Yang et al., 2021. Students often have difficulties when faced with situations where they are asked to read a lot of text (Firetto & Van Meter, 2018). ...
Education in the era of digital technology has become a challenge for teachers and lecturers in how this technology can develop comprehensive knowledge and global insight. The purpose of this study is to improve learning achievement through the cognitive process of students in cell metabolism lectures using Interactive Digital Teaching Material (IDTM). The study involved 40 Biology Education students who were taking the third semester at one of the Private Universities in Bandung, West Java. The implementation of lectures and instruments to measure students' cognitive processes is based on the framework of the 1992 Marzano Learning Dimension. The results of the analysis showed that the N-gain value of Acquire and Integrate Knowledge and Extend and Refine Knowledge obtained a percentage of (28%-26%) on low criteria. While Using knowledge with a percentage of N-gain value (32%) on medium criteria. These results illustrate that IDTM facilitates students to train their cognitive processes toward better learning achievement
... Indeed, by making selections on the multiple-choice questions, children received customized audio feedback from the e-book to either further 99 elaborate the story or correct misunderstandings. The findings from the current study added one more piece of evidence to support the benefits of individualized interactions during digital storybook reading to children's learning (Xu et al., 2021;Yang et al., 2021). ...
With authentic English input and multimedia features, English storybook apps have the potential to be used as an effective learning tool for parent-child shared reading practice in families in English as a second or foreign language settings. This cost-effective educational technology may be particularly beneficial for those children from low socio-economic backgrounds in developing countries and language minority families in English speaking countries. However, most of the commercialized English storybook apps on the market are not designed to support parent-child joint engagement, especially for parents with limited English proficiency. Lack of theoretical and empirical evidence can be found to inform app designers and researchers on what design features should be included to facilitate dialogic reading and second language acquisition for this learner population. To this end, this dissertation fills in these research gaps by: 1) designing an interactive English-Chinese storybook app for young children and parents whose first language is Chinese to practice dialogic reading and learn English at home; 2) conducting an evaluation study to examine the learning effectiveness of the innovation and how it interacts with the child and parent characteristics; and 3) conducting a randomized controlled trial to examine whether and to what extent the embedded discussion prompts impact children’s learning and exploring how parents and children responded to these prompts. Using contextual inquiry and participatory design techniques, we identified demands and design challenges of English storybook apps for this learner population and developed an interactive multimedia storybook system with five design features which can effectively facilitate parent-child joint engagement and children’s learning. We found that embedding bilingual discussion prompts in the storybook app significantly promoted children's story comprehension and retelling. Our qualitative analysis of parent-child interactions revealed that the discussion prompts with feedback allow parents to practice dialogic reading strategies and provide scaffolding to their children naturally and effectively without explicit training. With parents translating and elaborating the story, children received more comprehensible input, thus enhancing their comprehension. The design of the questioning avatar further established children’s parasocial relationship with the story character and boosted their motivation. Together, this dissertation contributes to the fields of educational technology, language and literacy, human-computer-interaction by shedding light on the design, affordances, and educational effectiveness of interactive bilingual storybooks for children and their parents from linguistically diverse backgrounds.
... In Martin and Quan-Haase (2013) study, an e-book is accepted as a digital environment. It is essential for e-books to include interactive features, as stated by Yang et al. (2021), in order to successfully deliver digital learning outcomes. It is not believed to be adequate to transfer merely the written content to electronic media. ...
There are several meanings of the term “knowledge” that match to the diverse study areas. A knowledge creation, sharing, dissemination, and presentation environment is required. Digital platforms and technology lead to the development of innovative methods of teaching and learning in a digital environment. It is of intellectual and academic interest to determine if knowledge processes change in digital contexts, which provide more options than conventional environments such as books and newspapers. The study is built on an exhaustive examination of prior studies on knowledge in digital contexts. The databases ERIC and Scopus served as the search grounds for the keyword-based inquiry. The collected documents were evaluated to determine whether or not they contributed to the investigation's objective. The cornerstone of our inquiry was a careful review of the remaining 14 studies. For quantitative investigations, the experimental design was the most desired approach, but the case study method was the most preferred method for qualitative research. The objectives of the research employ the terms “knowledge” and “digital environments” either directly or implicitly. Studies were designated as “creating” “creating with cooperation” “acquiring” “presenting” and “sharing” In the research, data was coded in accordance with the responsibilities assigned in digital worlds. In the study, 10 distinct theories and models were referenced. These models and hypotheses are categorized based such as creating, acquiring, designing, and using their interaction with data. In the investigations, the existence of traditional knowledge creation and application has been investigated. In addition, the process's distinction owing to the capabilities of digital environments has been investigated.
... Using the appropriate media selection, e-books become an effective source of information media in digital learning to deal with learning challenges (Lestari & Adi, 2016). The primary rationale is that it can provide interactive learning while the use of e-books for literacy that is still very rare (Asrowi et al., 2019;López-Escribano et al., 2021;Gohar, 2017), and increasing literacy has an impact on the formation of the cognitive structure and intelligence of learners (Afnita et al., 2021;Zhang et al., 2021;Yang et al., 2021). A prior study on the use of e-books as a learning medium has been provided in increasing the literacy of students in higher education institutions (Suparno, 2018). ...
Having literacies in economics and financial will be beneficial to enhancing individual and family well-being. The purpose of this research aims to produce an e-book on economic literacy. To achieve the purpose, this study adopted a research and development method. Media is developed through student analysis, state objectives, determine goals, choosing methods, media and materials, which require student participation, evaluation and revision. Product development was validated by material experts, media experts, and piloted on a limited scale. The results of this study indicate that the e-book as learning media is effective in contextual economic learning. Also, it is perceived as easy to use for self-study, and it effectively supports blended learning. However, the findings also noted that many students choose traditional classroom learning instead of using blended learning. Further researchers can develop e-book learning media that supports blended learning in other difficult subjects with limited time and access to learning. Keywords: Economic Literacy, Macroeconomics, E-book, Blended Learning, Research Development
... Interactive design psychology modifies mostly the page structure and the rhythm of the selected fragments. Compared with the original version, the revision looks briefer, tidier, and better overall impression, so the likability score has increased slightly compared with the AI-modified version (Yang et al., 2021). Nevertheless, the revision is a far cry from the original text. ...
The advent of the intelligence age has injected new elements into the development of literature. The synergic modification of Anglo-American (AAL) traumatic narrative (TN) literature by artificial intelligence (AI) technology and interactive design (ID) psychology will produce new possibilities in literary creation. First, by studying natural language processing (NLP) technology, this study proposes a modification language model (LM) based on the double-layered recurrent neural network (RNN) algorithm and constructs an intelligent language modification system based on the improved LM model. The results show that the performance of the proposed model is excellent; only about 30% of the respondents like AAL literature; the lack of common cultural background, appreciation difficulties, and language barriers have become the main reasons for the decline of reading willingness of AAL literature. Finally, AI technology and ID psychology are used to modify a famous TN work respectively and synergically, and the modified work is appreciated by respondents to collect their comments. The results corroborate that 62% of the respondents like original articles, but their likability scores have decreased for individually modified work by AI or ID psychology. In comparison, under the synergic modification efforts of AI and ID psychology, the popularity of the modified work has increased slightly, with 65% of the respondents showing a likability to read. Therefore, it is concluded that literary modification by single ID psychology or AI technology will reduce the reading threshold by trading off the literary value of the original work. The core of literary creation depends on human intelligence, and AI might still not be able to generate high-standard literary works independently because human minds and thoughts cannot be controlled and predicted by machines. The research results provide new ideas and improvement directions for the field of AI-assisted writing.
Although the positive effects of the use of technology for students with learning disabilities (LD) have been reported, there is a lack of both theory and practice in terms of integrating technology with the appropriate strategy in accordance with the student, content and purpose. Both teaching materials developed with qualified strategies and the active involvement of the target audience in the process are a need to design effective and sustainable learning materials and processes for LD. This study involved both the production of a material with the common views of stakeholders related to LD (individuals diagnosed with LD, special education teachers, academics working on LD) and the examination of the effectiveness of this material. The focus of the study is primarily on the fact that human being is a social being and learning, language and reading are social phenomena. In this context, within the framework of social constructivist perspective, an e-book design based on context based teaching (CBT) strategy and its effectiveness on reading performance were analysed. The study involved a two-stage process. In the first phase, design-based research was conducted and a CBT-supported e-book (DIJIKIT) was developed. In the second stage, DIJIKIT and an e-book were compared in an adapted alternating implementation design. The participants of the study in the CBT process consisted of researchers, three special education teachers, academicians and two primary school students diagnosed with learning disabilities. In each cycle of the DBR, the participants provided feedback on the material. Semi-structured interviews, focus group discussions, video recordings and a researcher's diary were used as data collection tools in the DBR. As a result of the DBR, DIJIKIT design and content features were determined. In the experimental process, DIJIKIT was used by three primary school students diagnosed with learning disabilities. In the experimental process, efficacy (the informal reading invetory), reliability (treatment integrity, interobserver agreement) and social validity (social validity forms) data were collected. The reading comprehension performances and reading levels of all three students increased. Social validity data supported the experimental process data and clearly demonstrated the need for effective instructional technologies for both parents and teachers. The study offers practical implications and recommendations for future researchers in terms of exemplary design features and greater visibility of the relationship between instructional technologies and special education.
E‐books, powered by multimedia and interactive features, are widely used to support young children's language and literacy development. This study examines the relations of e‐book narration and bilingual parent–child talk to children's learning during shared reading. Data from 121 English learners in China and their parents were analyzed using structural equation modeling. Results revealed that e‐book narration, powered by multimedia features, indirectly predicts children's learning outcomes through parent and child talk in English. Specifically, parent talk in English fully mediated the relationship between e‐book narration and children's English vocabulary gains, while parent and child talk in English mediated the association between e‐book narration and story comprehension and retelling. In contrast, parent–child talk in Chinese did not predict any learning outcomes. Video transcript analysis showed that off‐topic talk was more prevalent in Chinese than in English, particularly among children. These findings have implications for app designers, parents, and researchers, emphasizing the importance of parental support in shared e‐book reading and shedding light on the e‐book design for children reading in a foreign language.
All programs define their program educational objectives (PEOs) that illustrate the knowledge of students for the concepts and their applications after the completion of the degree. PEOs attainment not only gives the evidence for the success of any program but also describes the student skills and knowledge for the various concepts related to the specific program. The chapter explains the planning process, assessment tools, and review panel for the program educational objectives in various educational programs in Arab universities. The same model can be applied for any program for planning, assessment, and review process because the general scenario and relevance of PEOs are the same in the higher educational systems.
Background. The digital era makes education require more sophisticated literacy skills to keep up with the era. The ability to search, access, manage, present, and communicate with digital objects is an ability in digital literacy. Books as reading materials can now be accessed through digital, called E-books. The existence of E-books will bring new changes in the way students learn. The presence of E-books will provide convenience, affordability, and availability. Purpose This study aims to measure the effectiveness of using ebooks in improving students' digital literacy. It also analyzes the factors that influence the effectiveness of using E-books. The presence of E-books as literacy media will be analyzed how influential it is on students' ability to search, access, manage, present and communicate using digital information. Method. The method used in this research is quantitative. The data obtained is the number of responses. The data obtained was taken from the distribution of questionnaires containing questions and statements regarding the effectiveness of E-books in improving students' digital literacy. The statements presented in the questionnaire are a form of google formula called google from. The data will be processed through SPSS which was previously inputted into excel. Results. The results of this study state that the use of E-books as digital reading materials will provide facilities for access, distribution, and consumption of reading materials. The practicality offered will make it easier for educators and students to access it. E-books provide interactive features to enhance the reading experience. In addition, E-books will provide long-distance support in online learning. But of course every advantage has disadvantages or limitations in the use of digital technology. Conclusion This research can be concluded that E-books as reading materials used by teachers and students must be monitored. Analyzing the use of E-books will be useful for identifying inhibiting factors or supporting factors in the world of education. It is also necessary to analyze the effectiveness of using E-books to improve the quality of student literacy. Teachers, parents and formal institutions including students themselves will play an active role in the successful use of E-books in improving digital literacy.
Research fields related to learning (e.g., educational technology and learning sciences) have historically focused on what questions using traditional methods (e.g., comparing different learning tools and methods). New methodologies that are grounded in learning, engagement, and motivational theories are needed to additionally address the how questions. Methodologies that use learners’ process data shed more light on how learners learn and if the learning tools are effective, compared to methodologies that use just outcome data. In this paper, we discuss stealth assessment—an evidence-based methodology that can be used in technology-rich environments (e.g., games) to assess and support hard-to-measure constructs (e.g., creativity) as well as knowledge acquisition (e.g., physics). We also discuss evidence-centered design (ECD), and present specific steps to design, embed in a digital learning environment, and evaluate a stealth assessment. Additionally, we provide two examples of stealth assessment studies in the context of an educational game called Physics Playground: Study 1 illustrates a stealth assessment of creativity and Study 2 describes a stealth assessment of physics understanding and how we used it to make the game adaptive. The purpose of this paper is to provide sufficient detail about stealth assessment to help researchers in the field of educational technology and related fields to adopt this method to assess, foster, and investigate learning processes in various technology-rich environments.
Technologies and teaching practices can provide a rich log data, which enables learning analytics (LA) to bring new insights into the learning process for ultimately enhancing student success. This type of data has been used to discover student online learning patterns, relationships between online learning behaviors and assessment performance. Previous studies have provided empirical evidence that not all log variables were significantly associated with student academic achievement and the relationships varied across courses. Therefore, this study employs a systematic review with meta‐analysis method to provide a comprehensive review of the log variables that have an impact on student academic achievement. We searched six databases and reviewed 88 relevant empirical studies published from 2010 to 2021 for an in‐depth analysis. The results show different types of log variables and the learning contexts investigated in the reviewed studies. We also included four moderating factors to do moderator analyses. A further significance test was performed to test the difference of effect size among different types of log variables. Limitations and future research expectations are provided subsequently.
Practitioner notes
What is already known about this topic Significant relationship between active engagement in online courses and academic achievement was identified in a number of previous studies.
Researchers have reviewed the literature to examine different aspects of applying LA to gain insights for monitoring student learning in digital environments (eg, data sources, data analysis techniques).
What this paper adds Presents a new perspective of the log variables, which provides a reliable quantitative conclusion of log variables in predicting student academic achievement.
Conducted subgroup analysis, examined four potential moderating variables and identified their moderating effect on several log variables such as regularity of study interval, number of online sessions, time‐on‐task, starting late and late submission.
Compared the effect of generic and course‐specific, basic and elaborated log variables, and found significant difference between the basic and elaborated.
Implications for practice and/or policy A depth of understanding of these log variables may enable researchers to build robust prediction models.
It can guide the instructors to timely adjust teaching strategies according to their online learning behaviors.
This randomized controlled trial study examined the effects of bilingual discussion prompts with feedback within a multimedia interactive e-book on parent-child shared reading for young English language learners aged 3-7 in China. Sixty-four parent-child pairs read a multimedia English storybook with bilingual discussion prompts in the treatment condition, and forty-three pairs read the same multimedia storybook without discussion prompts. After reading the storybook twice, children in the discussion-prompt group outperformed the control group on story comprehension and retelling measures. However, children in both groups showed comparable gains in English vocabulary. According to our qualitative analysis of parent-child responses on discussion pages, when parents follow the question-response-evaluate-feedback flow of the discussion sessions, they tend to practice dialogic reading strategies and scaffold children's learning naturally and effectively without explicit training. With the learning facilitation from both the storybook and parents, children received more comprehensible input, produced more English output, and became active storytellers instead of passive learners. Moreover, the e-book with a built-in questioning avatar established children's parasocial relationship with the story characters. These findings suggest an exciting potential for multimedia interactive e-books, powered by bilingual discussion prompts, as an effective educational tool for families from diverse linguistic backgrounds.
Lay Description
What is already known about this topic
Interactive features, such as hotspots on which children can press to receive visual or auditory effects, are commonly incorporated in children's electronic books.
Hotspots may stimulate children's engagement and help children understand the content of the story; hotspots may also increase the demand of working memory, thus resulting in children's cognitive overload.
These interactive features may increase or decrease children's engagement with and comprehension of the story depending on how children use them.
What this paper adds
The present study explored the association of hotspots interaction with children's engagement with and learning from tablet‐based electronic books.
Interaction with hotspots enhanced children's emotional engagement and visual attention as children used an e‐book, but not verbal engagement.
Hotspot interactions benefited children's recall of story elements relevant to the hotspot but did not appear to enhance their holistic comprehension of the story.
Implications of study findings for practitioners
Integrating hotspots that are congruent with the story into electronic books can offer young readers a more enjoyable and focused digital reading experience.
Prompts or instructions should be given to guide children in their hotspot interactions.
Designers may also want to consider incorporating hotspots that may facilitate children's verbal engagement with reading and story comprehension.
Abstract Student clickstream data—time-stamped records of click events in online courses—can provide fine-grained information about student learning. Such data enable researchers and instructors to collect information at scale about how each student navigates through and interacts with online education resources, potentially enabling objective and rich insight into the learning experience beyond self-reports and intermittent assessments. Yet, analyses of these data often require advanced analytic techniques, as they only provide a partial and noisy record of students’ actions. Consequently, these data are not always accessible or useful for course instructors and administrators. In this paper, we provide an overview of the use of clickstream data to define and identify behavioral patterns that are related to student learning outcomes. Through discussions of four studies, we provide examples of the complexities and particular considerations of using these data to examine student self-regulated learning.
Poor reading comprehension may be due to having ineffective comprehension monitoring, the metacognitive process of evaluating and regulating comprehension. When comprehension breaks down due to an inconsistency either at the word-level (e.g., due to an unfamiliar word) or at the sentence-level (e.g., due to contradictory information), readers may identify the misunderstanding and take steps to regulate their comprehension. In the current study, we utilized two eye-movement tasks (one newly developed) to examine comprehension monitoring in third through fifth grade students (n = 123), when confronted with word- and sentence-level inconsistencies, by measuring the amount of time they read (gaze duration) and reread the target inconsistent words. We investigated how this skill may be associated with individual differences in age, reading comprehension ability, and vocabulary knowledge. The results showed that generally, all students detected the word-level inconsistencies, indicated by longer gaze durations, and attempted to regulate their comprehension after detecting both word- and sentence-level inconsistencies, as indicated by more time spent rereading. Students with stronger reading comprehension (when controlling for their vocabulary), and stronger vocabulary knowledge (when controlling for their reading comprehension) were more likely to attempt regulating their comprehension. In general, the difference between the control words and the inconsistent words was smaller for third graders and larger for fourth and fifth graders, which we argue indicates greater levels of comprehension monitoring—specifically employing repair strategies. With eye-tracking technology becoming more accessible, these tasks may be useful in assessing children’s reading processes to better understand at which level of comprehension monitoring they may be struggling, which in return will allow us to develop more individualized instruction for all readers.
In this large-scale study of students from Title 1 schools (N = 14,773), we used multiple-group latent change score (LCS) modeling to investigate the developmental relations between vocabulary knowledge and reading comprehension in students with a school-identified learning disability (LD, n = 627) and typically developing students (n = 14,146). Students were tested for their vocabulary breadth and passage comprehension skills in Kindergarten through fourth grade. For typically developing students, there were bidirectional influences between their vocabulary knowledge and reading comprehension skills. There were no cross-lagged influences across constructs for students with an LD. We find evidence for a developmental delay, such that students with an LD had similar levels and gains in their vocabulary knowledge relative to typically developing students, but these students started much lower in their reading comprehension skills and did not catch up to their typically developing peers. We discuss the implications for children with learning disabilities and the development of their reading comprehension skills.
Early prediction systems have already been applied successfully in various educational contexts. In this study, we investigated developing an early prediction system in the context of eBook-based teaching-learning and used students’ eBook reading data to develop an early warning system for students at-risk of academic failure -students whose academic performance is low. To determine the best performing model and optimum time for possible interventions we created prediction models by using 13 prediction algorithms with the data from different weeks of the course. We also tested effects of data transformation on prediction models. 10-fold cross-validation was used for all prediction models. Accuracy and Kappa metrics were used to compare the performance of the models. Our results revealed that in a sixteen-week long course all models reached their highest performance with the data from the 15th week. On the other hand, starting from the 3rd week, the models classified low and high performing students with an accuracy of over 79%. In terms of algorithms, Random Forest (RF) outperformed other algorithms when raw data were used, however, with the transformed data J48 algorithm performed better. When categorical data were used, Naive Bayes (NB) outperformed other algorithms. Results also indicated that models with transformed data performed lower than the models created using categorical data. However, models with categorical data showed similar performance with models with raw data. The implications of the results presented in this research were also discussed with respect to the field of Learning Analytics.
In a meta-analysis, Takacs, Swart, and Bus (2015) found that when children listen to multimedia electronic storybooks, comprehension is higher than when listening to a traditional oral reading of the story. However, adding interactive features reduced the benefit, and for at-risk children, the interactive features reduced comprehension to below that when listening to the reading. Here we report a contrasting effect. Namely, a type of interactive feature that we call simulation has a large positive benefit on story comprehension in a multimedia environment for dual language learners. In addition, we show that combining simulation with a modicum of support in the native language (in this case, Spanish) produces additional benefits on reading comprehension. That is, when children are poor decoders but have good English-language skills, or when they are good decoders but have poor English-language skills, adding Spanish support to simulation makes up for the deficit and increases performance when reading in English. We trace the differences between our findings and those reported by Takacs et al. to the types of interaction used. In the Takacs et al. findings, the interactive features did not enhance story meaning. In the current research, simulation was specifically designed to highlight meaning. Thus, simulation and Spanish support may be effective for improving comprehension when dual language learners are reading in English.
The development of early literacy skills has been critically linked to a child’s later academic success. In particular, repeated studies have shown that reading aloud to children and providing opportunities for them to discuss the stories that they hear is of utmost importance to later academic success. CloudPrimer is a tablet-based interactive reading primer that aims to foster early literacy skills by supporting parents in shared reading with their children through user-targeted discussion topic suggestions. The tablet application records discussions between parents and children as they read a story and, in combination with a common sense knowledge base, leverages this information to produce suggestions. Because of the unique challenges presented by our application, the suggestion generation method relies on a novel topic modeling method that is based on semantic graph topology. We conducted a user study in which we compared how delivering suggestions generated by our approach compares to expert-crafted suggestions. Our results show that our system can successfully improve engagement and parent–child reading practices in the absence of a literacy expert’s tutoring.
Learning in games has historically been assessed indirectly and/or in a post hoc manner. What’s needed instead is real-time assessment and support of learning based on the dynamic needs of players. We need to be able to experimentally determine the degree to which games can support learning, and how and why they achieve this objective. In this chapter we describe an approach to designing and developing evidence-based diagnostic assessments that may be embedded in a game environment. When embedded assessments are so seamlessly woven into the game that they’re virtually invisible, we call this “stealth assessment.” Embedding assessments within games provides a way to monitor a player’s current level on valued competencies, and then use that information as the basis for support, such as adjusting the difficulty level of challenges or providing a report for the teacher.
Separate mixed model analyses of variance were conducted to examine the effect of textual distance on the accuracy and speed of text consistency judgments among adequate and struggling comprehenders across grades 6–12 (n = 1,203). Multiple regressions examined whether accuracy in text consistency judgments uniquely accounted for variance in comprehension. Results suggest that there is considerable growth across the middle and high school years, particularly for adequate comprehenders in those text integration processes that maintain local coherence. Accuracy in text consistency judgments accounted for significant unique variance for passage-level, but not sentence-level comprehension, particularly for adequate comprehenders.
The following values have no corresponding Zotero field:
ID - 92
Computer-based assessment can provide new insights into behavioral processes of task completion that cannot be uncovered by paper-based instruments. Time presents a major characteristic of the task completion process. Psychologically, time on task has 2 different interpretations, suggesting opposing associations with task outcome: Spending more time may be positively related to the outcome as the task is completed more carefully. However, the relation may be negative if working more fluently, and thus faster, reflects higher skill level. Using a dual processing theory framework, the present study argues that the validity of each assumption is dependent on the relative degree of controlled versus routine cognitive processing required by a task, as well as a person's acquired skill. A total of 1,020 persons ages 16 to 65 years participated in the German field test of the Programme for the International Assessment of Adult Competencies. Test takers completed computer-based reading and problem solving tasks. As revealed by linear mixed models, in problem solving, which required controlled processing, the time on task effect was positive and increased with task difficulty. In reading tasks, which required more routine processing, the time on task effect was negative and the more negative, the easier a task was. In problem solving, the positive time on task effect decreased with increasing skill level. In reading, the negative time on task effect increased with increasing skill level. These heterogeneous effects suggest that time on task has no uniform interpretation but is a function of task difficulty and individual skill.
ABSTRACTS
The purpose of this theoretical essay is to explore the limits of traditional conceptualizations of reader and text and to propose a more general theory based on the concept of a literacy agent. The proposed theoretical perspective subsumes concepts from traditional theory and aims to account for literacy online. The agent‐based literacy theory proposed (a) reframes the concepts of reader and text as roles defined by criteria that allow human or machine agents to be assigned to either role, (b) distinguishes two types of literacy events that more adequately represent the range of interactions possible in online reading environments, and (c) argues that readers' views of literacy events and their success in achieving goals depend on how they assign roles in reading. Several examples of literacy events are presented, illustrating the application of agent‐based literacy theory and exploring its consequences. Examples are followed by an examination of one historically important critique of the concept of machine readers. The essay concludes by identifying empirical predictions of agent‐based literacy theory and reviewing implications of the theory for literacy practice and pedagogy.
El propósito de este ensayo teórico es explorar los límites de las concepciones tradicionales sobre lector y texto y proponer una teoría más general basada en el concepto de agente de alfabetización. La perspectiva teórica propuesta subsume conceptos de la teoría tradicional y apunta a dar cuenta de la alfabetización online. La teoría de la alfabetización basada en los agentes: (a) reformula los conceptos de lector y texto como roles definidos por criterios que permiten que los agentes, sean humanos o máquinas, puedan ser asignados a cualquiera de los roles, (b) distingue entre dos tipos de eventos de alfabetización que representan de manera más precisa el rango posible de interacciones en un contexto de lectura online y (c) argumenta que las ideas de los lectores sobre los eventos de alfabetización y su éxito en alcanzar metas dependen de cómo asignan los roles en lectura. Se presentan varios ejemplos de eventos de alfabetización que ilustran la aplicación de la teoría de la alfabetización basada en los agentes y exploran sus consecuencias. Los ejemplos están seguidos por el examen de una crítica históricamente importante al concepto de lectores máquina. El ensayo concluye con la identificación de predicciones empíricas de la teoría discutida; asimismo se revisan implicancias de la teoría para la práctica y la pedagogía de la alfabetización.
Der Zweck dieser theoretischen Abhandlung ist es, die Grenzen traditioneller Konzeptualisierungen von Leser und Text zu erforschen und eine mehr allgemeine Theorie vorzuschlagen, basierend auf dem Konzept einer Schreibund Lesevermittlung. Die vorgeschlagene theoretische Perspektive faßt Konzepte von der traditionellen Theorie zusammen und setzt sich zum Ziel, der Schreib‐ und Lesebildung durch Online Rechnung zu tragen. Die vermittlungsbasierte Schreib‐ und Lesetheorie schlug vor (a) ein Neuformulieren der Konzepte von Leser und Text in Rollen, definiert durch Kriterien, die es erlauben, daß menschliche oder maschinelle Vermittler der jeweiligen Rolle zugeordnet werden, (b) zu unterscheiden nach zwei Arten von Schreib‐ und Lesebegebenheiten, die angemessener die erreichbare Spanne wechselseitigen Austausches im Lesen in der Online‐Umwelt ermöglichen, und (c) zu argumentieren, daß die Ansichten der Leser von Schreib‐ und Lesebegebenheiten und ihr Erfolg im Erreichen der Ziele davon abhängt, wie sie die Rollen im Lesen zuweisen. Verschiedene Beispiele von Schreib‐ und Lesebegebenheiten werden aufgezeigt, wobei sie die Anwendung der vermittlungsbasierten Schreib‐ und Lesetheorie darstellen und deren Auswirkungen untersuchen. Beispiele werden durch Untersuchung einer historisch wichtigen Kritik des Konzepts von Maschinenlesern ergänzt. Der Aufsatz schließt durch Identifizieren empirischer Vorhersagen der vermittlungsbasierten Theorie und durch ein Überprüfen auf die Auswirkungen auf die Theorie über die Schreib‐ und Lesepraxis und Pädagogik.
Cet essai théorique a pour but d'explorer les limites des conceptualisations traditionnelles du lecteur et du texte et de proposer une théorie plus vaste basée sur le concept d'agent de littérisme. La perspective théorique que nous proposons tientcompte des concepts de la théorie traditionnellle et vise à prendre en compte le littérisme en ligne. La théorie du littérisme basée sur l'agent propose (a) de recadrer les concepts de lecteur et de texte en tant que rôles définis par des critères permettant à des agents humains ou mécaniques d'être assignés à chacun de ces rôles, (b) distingue deux types d'événements de littérisme qui représentent de façon plus satisfaisante l'ensemble des interactions possibles dans un environnement de lecture en ligne, et (c) soutient que les représentations qu'ont les lecteurs des événements de littérisme et de leurs possibilités d'atteindre leurs buts dépend de la façon dont ils attribuent les rôles en lecture. On présente plusieurs exemples d'événements de littérisme pour illustrer l'application de la théorie du lettrisme basée sur l'agent et explorer ses conséquences. Ces exemples sont suivis par l'examen d'une critique historiquement importante du concept de lecteur machine. On conclut cet essai par des prédictions empiriques de la théorie du littérisme basée sur l'agent et par l'examen des implications de cette théorie pour les pratiques de littérisme et la pédagogie.
A meta-analysis of vocabulary interventions in grades pre-K to 12 was conducted with 37 studies to better understand the impact of vocabulary on comprehension. Vocabulary instruction was found to be effective at increasing students' ability to comprehend text with custom measures (d = 0.50), but was less effective for standardized measures (d = 0.10). When considering only custom measures, and controlling for method variables, students with reading difficulties (d = 1.23) benefited more than three times as much as students without reading problems (d = 0.39) on comprehension measures. Gains on vocabulary measures, however, were comparable across reading ability. In addition, the correlation of vocabulary and comprehension effects from studies reporting both outcomes was modest (r = .43).
In educational assessment, we observe what students say, do, or make in a few particular circumstances and attempt to infer what they know, can do, or have accomplished more generally. A web of inference connects the two. Some connections depend on theories and experience concerning the targeted knowledge in the domain, how it is acquired, and the circumstances under which people bring their knowledge to bear. Other connections may depend on statistical models and probability-based reasoning. Still others concern the elements and processes involved in test construction, administration, scoring, and reporting. This article describes a framework for assessment that makes explicit the interrelations among substantive arguments, assessment designs, and operational processes. The work was motivated by the need to develop assessments that incorporate purposes, technologies, and psychological perspectives that are not well served by familiar forms of assessments. However, the framework is equally applicable to analyzing existing assessments or designing new assessments within familiar forms.
ABSTRACTS
A counterbalanced, within‐subjects design was carried out to study the efficacy of electronic books in fostering kindergarten children's emergent story understanding. The study compared effects of children's independent reading of stories electronically with effects of printed books read aloud by adults. Participants were 18 four‐ to five‐year‐old Dutch kindergarten children in the initial stages of developing story comprehension but beyond just responding to pictures. Electronic reading produced experiences and effects similar to adult‐read printed books. Children frequently interacted with the animations often embedded in electronic stories, but there was no evidence that the animations distracted children from listening to the text presented by electronic books, nor that the animations interfered with story understanding. Findings suggested that children at this stage of development profited from electronic books at least when electronic books are read in a context where adults also read books to children.
Se Elaboró un diseño balanceado intra‐sujetos para estudiar la eficacia de los libros electrónicos en la promoción de la comprensión emergente de cuentos en niños de nivel inicial. El estudio comparó los efectos sobre los niños de la lectura electrónica independiente con la lectura oral de libros impresos hecha por adultos. Participaron 18 niños daneses de nivel inicial de cuatro a cinco años que se hallaban en las primeras fases de desarrollo de la comprensión de cuentos, apenas superada la etapa de responder sólo a las imágenes. La lectura electrónica produjo experiencias y efectos similares a los libros impresos leidos por adultos. Los niños interactuaron frecuentemente con las animaciones que acompañan a menudo los cuentos electrónicos, pero no se observó evidencia de que las animaciones distrajeran a los niños de escuchar el texto presentado en los libros electrónicos, ni tampoco de que las animaciones interfirieran con la comprensión de los cuentos. Los hallazgos sugieren que los niños en esta fase del desarrollo aprovechan los libros electrónicos, al menos cuando dichos libros se leen en un contexto en el que los adultos también leen cuentos a los niños.
Es wurde ein ausgewogenes Design innerhalb der Themen ausgeführt, um die Eindrücke von elektronischen Büchern zur Förderung des emporkommenden Verständnisses der Kinder von Erzählungen im Kindergarten zu untersuchen. Die Studie verglich Eindrücke der Kinder beim selbständigen elektronischen Lesen von Erzählungen mit Eindrücken beim lauten Vorlesen von Erwachsenen aus gedruckten Büchern. Die Teilnehmer bestanden aus 18 vier‐ bis fünfjährigen holländischen Kindern im Kindergarten in den Anfangsstufen der Entwicklung des Erzählungsverständnisses, jedoch fortgeschrittener als bloßes Reagieren auf Bilder. Elektronisches Lesen bewirkte Erfahrungen und Effekte ähnlich den durch Erwachsenen gelesenen gedruckten Büchern. Die Kinder gingen häufig gegenseitig auf die in den elektronischen Erzählungen eingebetteten Animationen ein, jedoch gab es keinen Beweis dafür, dass die Animationen Kinder vom Zuhören der von den elektronischen Büchern vermittelten Texte ablenkten, noch dass die Animationen das Erzählungsverständnis beeinträchtigten. Die Erkenntnisse legten nahe, dass Kinder in diesem Entwicklungsstadium von elektronischen Büchern profitierten, zumindest wenn elektronische Bücher in einem Kontext gelesen werden, wo Erwachsene ebenfalls den Kindern aus Büchern vorlesen.
Un plan expérimental contrebalancé intra‐sujets a été construit pour étudier l'efficacité des livres électroniques sur l'amélioration des débuts de la compréhension d'histoires chez des enfants de maternelle. L'étude compare les effets de la lecture autonome d'histoires par des enfants aux effets produits par la lecture de livres lus à haute voix par des adultes. Les participants étaient 18 enfants hollandais de maternelle âgés de quatre à cinq ans, au moment où ils commencent à comprendre une histoire, juste après celui où ils se contentent de réagir à des images. La lecture électronique produit des expériences et des effets semblables à ceux de livres lus par des adultes. Les enfants interagissent fréquemment avec les animations qui sont souvent inclues dans les livres électroniques, mais rien n'indique que ces animations aient distrait les enfants de l'écoute du texte que présentent les livres électroniques, ni qu'elles aient interféré avec la compréhension de l'histoire. Les résultats indiquent que les enfants à ce niveau de développement tirent avantage des livres électroniques, au moins quand ils sont lus dans un contexte où les adultes lisent également des livres aux enfants.
An approach to performance-based assessments that embeds assessments in digital games in order to measure how students are progressing toward targeted goals.
To succeed in today's interconnected and complex world, workers need to be able to think systemically, creatively, and critically. Equipping K-16 students with these twenty-first-century competencies requires new thinking not only about what should be taught in school but also about how to develop valid assessments to measure and support these competencies. In Stealth Assessment, Valerie Shute and Matthew Ventura investigate an approach that embeds performance-based assessments in digital games. They argue that using well-designed games as vehicles to assess and support learning will help combat students' growing disengagement from school, provide dynamic and ongoing measures of learning processes and outcomes, and offer students opportunities to apply such complex competencies as creativity, problem solving, persistence, and collaboration. Embedding assessments within games provides a way to monitor players' progress toward targeted competencies and to use that information to support learning.
Shute and Ventura discuss problems with such traditional assessment methods as multiple-choice questions, review evidence relating to digital games and learning, and illustrate the stealth-assessment approach with a set of assessments they are developing and embedding in the digital game Newton's Playground. These stealth assessments are intended to measure levels of creativity, persistence, and conceptual understanding of Newtonian physics during game play. Finally, they consider future research directions related to stealth assessment in education.
The authors examined the influence of interactive media on children's story memory. First-grade children (6--7-year-olds) experienced a computer-based story in 1 of 4 presentation modes: One group heard only the narration, analogous to radio. A 2nd group saw an audiovisual presentation, analogous to television. A 3rd group viewed the story and interacted with animated areas of the screen. A 4th group was yoked to the interaction group such that they observed but did not control the interaction, The audio-only group consistently recalled and comprehended poorly, but there were no differences among the other media groups. In the interaction groups, there was also no relation between the amount of interaction with the story and subsequent memory. Overall, the results for interactive media were similar to the findings for the television-like presentation. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
The authors report data from a longitudinal study that addresses the relations between working memory capacity and reading comprehension skills in children aged 8, 9, and 11 years. At each time point, the authors assessed children's reading ability, vocabulary and verbal skills, performance on 2 working memory assessments (sentence-span and digit working memory), and component skills of comprehension. At each time point, working memory and component skills of comprehension (inference making, comprehension monitoring, story structure knowledge) predicted unique variance in reading comprehension after word reading ability and vocabulary and verbal ability controls. Further analyses revealed that the relations between reading comprehension and both inference making and comprehension monitoring were not wholly mediated by working memory. Rather, these component skills explained their own unique variance in reading comprehension.
Recent research has indicated that misuse of intelligent tutoring software is correlated with substantially lower learning. Students who frequently engage in behavior termed "gaming the system" (behavior aimed at obtaining correct answers and advancing within the tutoring curriculum by sys- tematically taking advantage of regularities in the software's feedback and help) learn only 2/3 as much as similar students who do not engage in such be- haviors. We present a machine-learned Latent Response Model that can identify if a student is gaming the system in a way that leads to poor learning. We believe this model will be useful both for re-designing tutors to respond appropriately to gaming, and for understanding the phenomenon of gaming better.
The paper makes an attempt to stimulate theoretical thinking and empirical research in the domain of L2 vocabulary learning by introducing a construct of involvement with motivational and cognitive dimensions: Need, Search, and Evaluation. Retention of hitherto unfamiliar words is claimed to be conditional upon the amount of involvement while processing these words. Involvement is operationalised by tasks designed to vary in the degree of need, search, and evaluation. The paper reviews a number of constructs that are currently debated and investigated in the literature on cognitive and motivational aspects of L2 learning. It also re-examines the existing empirical literature on task effect in the light of the proposed construct of task-induced involvement, stresses the need for deepening and broadening the construct, and discusses possibilities it offers for research on vocabulary learning.
Many children fail to comprehend what they read because they do not monitor their understanding, which requires making accurate judgements of what they know and then employing repair strategies when comprehension fails. Relying on research from learning science and cognitive and developmental psychology, we developed the Word Knowledge e-Book (WKe-Book) to improve children's calibration of their word knowledge, strategy use, and word knowledge overall; skills which are associated with reading comprehension. The WKe-Book, which is read on a tablet computer, is a choose-your-own adventure book where choices require choosing between two rare words (e.g., cogitate vs. procrastinate). Depending on the word chosen, the story follows a different plot. There are also embedded comprehension questions where students receive immediate feedback with consequences for incorrect answers, such as being sent back to reread a few pages. In a randomized controlled trial, we tested whether students (N = 603 in 25 third through fifth grade classrooms in Arizona in the US) reading the WKe-Book would demonstrate improved word knowledge, strategy use, and word knowledge calibration. Classrooms were randomly assigned to read the WKe-Book immediately (treatment) or later (delayed-treatment control), and within classrooms, students were randomly assigned to either participate in a 15-min weekly book club (book club treatment) or to read the WKe-Book independently with no book club (no book club control). Results revealed a significant treatment effect of the WKe-Book on students' word knowledge, word knowledge calibration, and strategy use, which predicted student performance on standardized reading comprehension and vocabulary measures. The effects were greater for students who participated in weekly book clubs compared to students in the no book club control. These findings suggest that the affordances offered by technology, which are unavailable in paper-based books, can support students' development of metacognition, including word knowledge calibration, strategy use, and word learning skills.
Measuring the implementation fidelity (IF) or integrity of interventions is extremely important, since without it a positive or negative outcome cannot be interpreted. However, IF is actually measured relatively rarely. Direct and indirect methods of measurement have been used in the past, but tend to over-emphasize teacher behaviour. This paper focuses on student behaviour collated through computers - an interesting alternative. It deals with the reading of real books and reading achievement, for which variables a very large amount of computerised data was available – on 852,295 students in 3243 schools. Reading achievement was measured pre-post with STAR Reading, a computerised item-banked adaptive norm-referenced test of reading comprehension. IF came from the Accelerated Reader (AR), which measures understanding of independent reading of real books the student has chosen by a quiz. Results showed higher IF was related to higher achievement. Neither IF nor reading achievement related to socio-economic status. Primary (elementary) schools had higher IF and achievement than secondary (high) schools. Females had higher IF and achievement than males. Students of higher reading ability implemented AR at a higher level, but did not gain in reading at a higher level. However, this computerised method of measuring IF with book reading showed limited reliability, no greater than methods emphasising teacher behaviour. In future, IF measures emphasising student response and those emphasising teacher behaviour need to be blended, although the latter will never generate the sample size of the former. This may be true of implementation fidelity in areas other than book reading.
The role of working memory in vocabulary acquisition has been well established in the literature. In this study, we proposed and empirically tested the multicomponent view of vocabulary acquisition, which states that multiple language and cognitive skills are involved to facilitate phonological and semantic representations needed for vocabulary acquisition. Working memory and attention were hypothesized to be directly and indirectly related to vocabulary, whereas inference and morphosyntactic knowledge were hypothesized to be directly related to vocabulary (measured by the Picture Vocabulary Test of the Woodcock–Johnson III battery). Results from 262 kindergartners using path analysis revealed that all the multiple cognitive and language skills were directly related to vocabulary after controlling for age, gender, racial/ethnic backgrounds, socioeconomic status (as measured by free or reduced-price lunch eligibility), and each other. Furthermore, working memory and attention also made indirect contributions via inference and morphosyntactic knowledge. Total effects (beta weights), accounting for direct and indirect effects, were .33 for working memory, .23 for attention, .18 for inference, and .18 for morphosyntactic knowledge. These results indicate that although working memory is important, contributions of other language and cognitive skills should be considered in vocabulary acquisition. Theoretical and practical implications are discussed.
In this article, I present a developmental model of how children learn to comprehend what they read, which builds on current models of reading comprehension and integrates findings from instructional research and evidence-based models of development in early and middle childhood. The lattice model holds that children's developing reading comprehension is a function of the interacting, reciprocal, and bootstrapping effects of developing text-specific, linguistic, and social-cognitive processes, which interact with instruction as Child Characteristic × Instruction (C × I) interaction effects. The processes develop over time and in the context of classroom, home, peer, community, and other influences to affect children's development of proficient reading comprehension. First, I describe models of reading comprehension; then, I review the basic processes in the model, the role of instruction, and C × I interactions in the context of the lattice model. Finally, I discuss implications for instruction and research.
We used stealth assessment, embedded in a game called Use Your Brainz (a slightly modified version of Plants vs. Zombies 2), to measure middle-school students’ problem solving skills. We began by developing a problem solving competency model based on a review of the relevant literature. We then identified in-game indicators that would provide evidence about students’ levels on various problem-solving facets. Our problem solving model was then implemented in the game via Bayesian networks. To validate the stealth assessment, we collected data from students who played the game-based assessment for three hours and completed two external problem solving measures (i.e., Raven’s Progressive Matrices and MicroDYN). Results indicated that the problem solving estimates derived from the game significantly correlated with the external measures, which suggests that our stealth assessment is valid. Our next steps include running a larger validation study and developing tools to help educators interpret the results of the assessment, which will subsequently support the development of problem solving skills.
This pilot study explores the impact of online electronic storybooks (e-books) on the reading motivation and listening comprehension of six grade 1 students (aged 7 years) from Ontario, Canada. The researcher measured participants’ perceived enjoyment of the online e-book reading experience using standardized listening comprehension tests, motivation questionnaires, behavioural observation checklists, and take-home e-book reading logs. The researchers used a scoring rubric to assess participants’ oral responses to a set of embedded literal, inferential, and evaluative comprehension questions within each e-book read during the program sessions. The results from this study indicate that all of the participants increased their comprehension scores from pretest to posttest after using the online e-book reading program, enjoyed the e-book reading experience, and frequently read the online e-books at home in their free time.
Based on the analysis of 620 think- aloud verbal protocols from students in grades 7, 9, and 11, we examined students' conscious engagement in inference generation, paraphrasing, verbatim text repetition, and monitoring while reading narrative or informational texts that were either at or above the students' current reading levels. Students were randomly assigned to read informational or narrative text, and each student read two texts in their assigned genre one accessible and one challenging. The research question addressed the combinations of text processes that best differentiated four groups of readers: (1) adequate comprehenders who read narrative and (2) informational text and (3) poor comprehenders who read narrative and (4) informational text. Canonical discriminant analysis (CDA) revealed that the four groups were best differentiated by two latent, underlying functions related to (a) a combination of inference generation in accessible text and paraphrasing in both accessible and difficult text (On- Level Inference/Paraphrasing) and (b) monitoring in both accessible and difficult text (Monitoring). Poor comprehenders who read informational text were significantly lower than the other three groups on On- Level Inference/Paraphrasing. Poor comprehenders in both genres were significantly lower on Monitoring than adequate comprehenders who read informational text. A second CDA further examining the effects of text difficulty identified one latent function primarily explained by inference generation (Inference). Text difficulty had a significant impact on adequate comprehenders' Inference in narrative text. Implications for research and practice are discussed.
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
Dutch advanced students of French read a French short story in one of three text reading conditions: Marginal Glosses (provision of L1 translations of unknown words), Dictionary (opportunity to use a bilingual dictionary), or Control. After reading, students were tested for their recall of 16 words that had appeared either once or three times in the text. Support was found for the hypothesis that frequency of occurrence will foster incidental vocabulary learning more when advanced second language (L2) readers are given the meanings of unknown words through marginal glosses or when they look up meanings in a dictionary than when no external information concerning unknown words' meanings is available. In the former case, reappearance of a word will reinforce the form-meaning connection in the reader's mental lexicon. In the latter case, readers will often ignore unknown words or incorrectly infer their meanings, which will limit the frequency effect. This article ends with recommendations for teachers and researchers.
In an attempt to more clearly understand the erosion of motivation in some readers, a number of researchers (Mohr, 200617.
Mohr , K. 2006. Children's choices for recreational reading: A three-part investigation of selection, preferences, rationales, and processes. Journal of Literacy Research, 38(1): 181–104. View all references; Smith & Wilhelm, 200225.
Smith , M. and
Wilhelm , J. 2002. Reading don't fix no Chevy's: Literacy in the lives of young men, Portsmouth, NH: Heinemann. View all references) and organizations (The Education Alliance, 20076.
Education Alliance. (2007). Gender differences in reading achievement: Policy implications and best practices http://www.educationalliance.org/Downloads/Research/GenderDifferences.pdf (Accessed: 25 July 2007). View all references) have called for the investigation of gender differences in all readers, including young children. Consequently, this study focused on younger, average achieving readers. Specifically, 288 third-grade average readers were studied. Two constructs consistent with expectancy-value theory (Eccles, 19835.
Eccles , J. 1983. “Expectancies, values, and academic behaviors”. In Achievement and achievement motives, Edited by:
Spence , J. T. 75–146. San Francisco, CA: Freeman. View all references), self-concept as a reader and value of reading, were examined. The results suggest that third-grade boys and girls who are average readers are equally self-confident about their reading ability; however, boys value reading less than girls. This finding sheds new light on the complexities of motivation and gender differences.
Silent Sustained Reading (SSR) or some related form of silent, independent reading practice has been incorporated into the daily reading routines of many classrooms and schools across the nation. However, we, like other educators have discovered that simply providing all students the time to self‐select their own books and read silently did not guarantee that they would actually engage in silent reading. In fact, many schools have discontinued using SSR because some students are not engaged in reading during SSR time. This study used a multiple‐baseline across‐subjects research design to investigate the impact of literary discussions on the engagement of three fourth‐grade non‐engaged readers during SSR. While additional studies are required to produce generalizable findings, teachers should be aware that non‐engaged readers benefit from participation in short, teacher or adult‐led literary discussions during SSR time.
Modeling students’ knowledge is a fundamental part of intelligent tutoring systems. One of the most popular methods for estimating
students’ knowledge is Corbett and Anderson’s [6] Bayesian Knowledge Tracing model. The model uses four parameters per skill,
fit using student performance data, to relate performance to learning. Beck [1] showed that existing methods for determining
these parameters are prone to the Identifiability Problem: the same performance data can be fit equally well by different parameters, with different implications on system behavior.
Beck offered a solution based on Dirichlet Priors [1], but, we show this solution is vulnerable to a different problem, Model Degeneracy, where parameter values violate the model’s conceptual meaning (such as a student being more likely to get a correct answer
if he/she does not know a skill than if he/she does).We offer a new method for instantiating Bayesian Knowledge Tracing, using
machine learning to make contextual estimations of the probability that a student has guessed or slipped. This method is no
more prone to problems with Identifiability than Beck’s solution, has less Model Degeneracy than competing approaches, and
fits student performance data better than prior methods. Thus, it allows for more accurate and reliable student modeling in
ITSs that use knowledge tracing.
The best value in formative assessment
- S Chappius
- J Chappius
Chappius, S., & Chappius, J. (2008). The best value in formative
assessment. Educational Leadership, 65(5), 14-19.
Principles of fluency instruction in reading: Relationships with established empirical outcomes
- J K Torgesen
- C A Rashotte
- A Alexander
Torgesen, J. K., Rashotte, C. A., & Alexander, A. (2001).
Principles of fluency instruction in reading: Relationships
with established empirical outcomes. In M. Wolf (Ed.),
Dyslexia, fluency, and the brain (pp. 333-355). York Press.
Predicting Student Test Performance based on Time Series Data of eBook Reader Behavior Using the Cluster-Distance Space Transformation [Conference session
- A Askinadze
- M Liebeck
- S Conrad
Askinadze, A., Liebeck, M., & Conrad, S. (2018, November).
Predicting Student Test Performance based on Time Series
Data of eBook Reader Behavior Using the Cluster-Distance
Space Transformation [Conference session]. International
Conference on Computers in Education (ICCE2018): Learning
Analytics Workshop Joint Activity, (pp. 26-30). Manila,
Philppines.
After the test: Closing the achievement gaps with data. Learning Point
- K W Symonds
Symonds, K. W. (2004). After the test: Closing the achievement
gaps with data. Learning Point.
Technology enhanced innovative assessment: Development, modeling, and scoring from an interdisciplinary perspective
- V J Shute
- G R Moore
Shute, V. J., & Moore, G. R. (2017). Consistency and validity in game-based stealth assessment. In H Jiao, & R.W
Lissitz (Eds.), Technology enhanced innovative assessment:
Development, modeling, and scoring from an interdisciplinary perspective (pp. 31-51). Information Age.
Formative and stealth assessment
- V J Shute
- Y J Kim
Shute, V. J., & Kim, Y. J. (2014). Formative and stealth assessment. In M. Spector, M.D. Merrill, J. Elen, & M. J. Bishop
(Eds.), Handbook of research on educational communications and technology (pp. 311-321). Springer.