Figure - uploaded by Denise Dellarosa Cummins
Content may be subject to copyright.
Source publication
Word problems are notoriously difficult to solve. We suggest that much of the difficulty children experience with word problems can be attributed to difficulty in comprehending abstract or ambiguous language. We tested this hypothesis by (1) requiring children to recall problems either before or after solving them, (2) requiring them to generate fi...
Context in source publication
Citations
... Research on mathematics assessments has indicated that word problems are notorious for their difficulty (Cummins et al., 1988). ...
[ Background ]
Readability metrics provide us with an objective and efficient way to assess the quality of educational texts. We can use the readability measures for finding assessment items that are difficult to read for a given grade level. Hard‐to‐read math word problems can put some students at a disadvantage if they are behind in their literacy learning. Despite their math abilities, these students can perform poorly on difficult‐to‐read word problems because of their poor reading skills. Less readable math tests can create equity issues for students who are relatively new to the language of assessment. Less readable test items can also affect the assessment's construct validity by partially measuring reading comprehension.
[ Objectives ]
This study shows how large language models help us improve the readability of math assessment items.
[ Methods ]
We analyzed 250 test items from grades 3 to 5 of EngageNY, an open‐source curriculum. We used the GPT‐3 AI system to simplify the text of these math word problems. We used text prompts and the few-shot learning method for the simplification task.
[ Results and Conclusions ]
On average, GPT‐3 AI produced output passages that showed improvements in readability metrics, but the outputs had a large amount of noise and were often unrelated to the input. We used thresholds over text similarity metrics and changes in readability measures to filter out the noise. We found meaningful simplifications that can be given to item authors as suggestions for improvement.
[ Takeaways ]
GPT-3 AI is capable of simplifying hard-to-read math word problems. The model generates noisy simplifications using text prompts or few-shot learning methods. The noise can be filtered using text similarity and readability measures. The meaningful simplifications AI produces are sound but not ready to be used as a direct replacement for the original items. To improve test quality, simplifications can be suggested to item authors at the time of digital question authoring.
... Most of the pupils who are struggling in solving mathematical word problems were found to have difficulties in reading the word problem and are unable to comprehend it. This is because they might not fully possess the conceptual knowledge required to solve the problems correctly (Cummins et al., 1988). They tend to misinterpret the given keywords, which leads them to identify the wrong mathematical operation to be used. ...
... Especially in the case of pupils from monolingual backgrounds with little or close to no exposure to the English Language (Jones, 2016), they are struggling and trying to comprehend what the passage meant due to the fact that Mathematics is taught using the English Language. Although Yusof (2003) reported that there is no correlation between comprehension and transformation in word problems with language, Cummins et al. (1988) stated that in order to solve word problems, asides from mathematical computation, other kinds of knowledge, which include linguistic knowledge are required for understanding and comprehending the problems. ...
Various research has been carried out worldwide over the years to identify ideal methods that are helpful to pupils when solving mathematical word problems. This study aims to examine the use of the CUBES Maths Strategy, a mnemonic device, to solve word problems and was conducted in a remote setting. An action research approach using a mixed method research was conducted where all data collected were analysed both quantitatively and qualitatively. The participants involved were pupils from a small local government primary school, aged between 8 and 9. Pupils’ test results from the given pre and post-tests were quantitatively analysed using Wilcoxon Signed-Rank Test, which concluded that there was no significant change in the difference in test scores. Newman’s Error Analysis interview was conducted to investigate the source of errors committed by the pupils, which concluded that the most prominent type of error made is the Comprehension error, followed by the Transformation error. From the observations and reflections, it can be deduced that, as the research was done in a remote setting, the use of the CUBES Maths Strategy was not fully utilised. These results could be based on the interactions between teachers and students during remote online learning.
... Over the past few decades, an emphasis on mathematical problemsolving has intensified in international mathematical education (Niss et al., 2017), including word problems and geometry problems solving. Additionally, numerous studies have investigated the cognitive mechanisms underlying mathematical problem-solving (e.g., Boonen et al., 2013;Cummins et al., 1988;Hegarty & Kozhevnikov, 1999). Furthermore, understanding these mechanisms can help us improve mathematical education. ...
... Mathematical word problems are based on real-world events and relationships, are stated in natural language, and are based on mathematical operations (Bassok, 2001). Therefore, mathematical word problems require a certain level of language understanding (e.g., Cummins et al., 1988;De Corte et al., 1985). Further, it is necessary to apply mathematical knowledge, such as knowledge of number addition and subtraction operations (e.g., Nesher, 2020;Sophian & Vong, 1995). ...
Students' ability to solve mathematical problems is a standard mathematical skill; however, its cognitive correlates are unclear. Thus, this study aimed to examine whether spatial processing (mental rotation, paper folding, and the Corsi blocks test) and logical reasoning (abstract and concrete syllogisms) were correlated with mathematical problem-solving (word problems and geometric proofing) for college students. The regression results showed that after controlling for gender, age, general IQ, language processing, cognitive processing (visual perception, attention, and memory skills), and number sense and arithmetic computation skills, spatial processing skills still predicted mathematical problem-solving and geometry skills in Chinese college students. Contrastingly, logical reasoning measures related to syllogisms did not predict after controlling for these variables. Further, notably, it did not correlate significantly with geometry performance when no control variables were included. Our results suggest that spatial processing is a significant component of math skills involving word and geometry problems (even after controlling for multiple key cognitive factors).
... For example, students may demonstrate poor performance in terms of sentence comprehension but show high proficiency in arithmetic skills. However, previous studies have mostly focused on the effect of sentence comprehension on arithmetic skills (Cummins et al., 1988;Morgan et al., 2017;Nortvedt et al., 2016). For example, some studies have shown that skilled readers regard arithmetic questions as number problem-solving tasks (Cummins et al., 1988;Glenberg et al., 2012;Nortvedt et al., 2016). ...
... However, previous studies have mostly focused on the effect of sentence comprehension on arithmetic skills (Cummins et al., 1988;Morgan et al., 2017;Nortvedt et al., 2016). For example, some studies have shown that skilled readers regard arithmetic questions as number problem-solving tasks (Cummins et al., 1988;Glenberg et al., 2012;Nortvedt et al., 2016). Meanwhile, only few studies have categorised students' arithmetic performance and investigated the effects of different arithmetic performance levels on their sentence comprehension. ...
... These results remain consistent across different quantifier construction form applications and whether the quantifier applied a hyperbole in sentences. These results are also partially consistent with those of previous studies, which demonstrated that students with good arithmetic skills were more sensitive and had faster response to the numeric information process compared with students having poor arithmetic skills (Cummins et al., 1988;Nortvedt et al., 2016;Palm, 2008). Previous studies have also suggested that students with high arithmetic skills demonstrated better performance in processing verbal and analogue nonsymbolic numerical information (Siemann & Petermann, 2018;Wu, 2003;Ye, 2018). ...
This study compared the sentence hyperbole comprehension performance of Chinese poor readers with various levels of arithmetic proficiency. A total of 168 Chinese poor readers in Grade 1 were recruited, and their nonverbal intelligence, verbal working memory, Chinese receptive vocabulary, Chinese grammatical knowledge, character reading and morphological awareness were controlled. The reaction times of these students in correctly answering literal and inferential questions was selected for further comparison. Results of a mixed-effect model analysis show that the participants with good and poor arithmetic proficiency levels demonstrated similar levels of literal information comprehension, regardless of the presentation form of the quantifier construction that was built by the quantifier location of the sentences, the number of numeric characters in a single quantifier and the hyperbole function applied in the sentence quantifier. Students with good arithmetic proficiency also demonstrated faster response in comprehending quantifier inferential information. Primary school students with good arithmetic proficiency outperformed those with poor proficiency in the inferential reading of verbal numeric information. In addition, differences in arithmetic proficiency did not significantly affect the students’ word/character semantic representation cognition and shallow literal information processing. Overall, the results distinguish the interaction and independent numerical information processes at the literal and inferential levels of text quantifier comprehension for young primary school students with poor reading proficiency.
... The basis of promoting innovation intention is that scholars can fully understand and agree with the URES. It is difficult for scholars to understand the evaluation system unless it is carefully formulated according to the method and objectives of the URES [35]. Scholar understanding was limited by the old school of thought that is "scientific research quantity", which decreases their innovation intention [36,37]. ...
Based on the two-dimensional University Research Evaluation System (URES), this paper aimed to develop a comprehensive and scientific measurement scale and to empirically verify the impact of the URES on scholars’ empathy and willingness to innovate. Grounded in theory, this study analyzed the personal information and interview data of 26 university scholars publicly available online. First, through qualitative analysis (using Nvivo 12 software), we developed an initial scale for URES. Second, we tested the reliability and validity of the scale by structural equation modeling (SEM) using Mplus 8.0 software. The results show that the URES includes two dimensions: research process evaluation and investment output evaluation. The URES scale showed good reliability and validity and was confirmed to be positively correlated with scholars’ empathy and willingness to innovate. Therefore, the URES constructed in this study not only fully stimulates scholars’ empathy and innovation willingness, but also promotes the optimal use of scholars’ knowledge resources. Finally, this research helps to reduce unnecessary educational and political investment, which has important implications for the sustainable development of society.
... For example, text features of the everyday register are related to reduce linguistic complexity. This can happen, for example, through contextualization, personalization (pronouns), and formation of active sentences of the text (Abedi & Lord, 2001;Cummins et al., 1988;Gogolin & Lange, 2011). This can lead, as in the case of contextualization, to an increase in the number of certain text features. ...
The study examines language dimensions of mathematical word problems and the classification of mathematical word problems according to these dimensions with unsupervised machine learning (ML) techniques. Previous research suggests that the language dimensions are important for mathematical word problems because it has an influence on the linguistic complexity of word problems. Depending on the linguistic complexity students can have language obstacles to solve mathematical word problems. A lot of research in mathematics education research focus on the analysis on the linguistic complexity based on theoretical build language dimensions. To date, however it has been unclear what empirical relationship between the linguistic features exist for mathematical word problems. To address this issue, we used unsupervised ML techniques to reveal latent linguistic structures of 17 linguistic features for 342 mathematical word problems and classify them. The models showed that three- and five-dimensional linguistic structures have the highest explanatory power. Additionally, the authors consider a four-dimensional solution. Mathematical word problem from the three-dimensional solution can be classify in two groups, three- and five-dimensional solutions in three groups. The findings revealed latent linguistic structures and groups that could have an implication of the linguistic complexity of mathematical word problems and differ from language dimensions, which are considered theoretically. Therefore, the results indicate for new design principles for interventions and materials for language education in mathematics learning and teaching.
... la mémoire de travail : Swanson, 2017), des capacités langagières (e.g. niveau de vocabulaire : Cummins et al., 1988), des difficultés spécifiques aux mathématiques (e.g. fractions ou proportions : Van Dooren et al., 2015) (pour une synthèse : Thevenot, 2017 (Zagar, 1991). ...
... Cette étape de compréhension nécessite en principe de construire une représentation mentale la plus proche possible de celle qui est requise pour la résolution, en imageant et simulant mentalement les relations décrites par le texte (modèle mental : Johnson-Laird, 1983;Thevenot, 2010 ;ou modèle de situation : Brissiaud & Sander, 2010;Van Dijk & Kintsch, 1983). Elle constitue une première source de difficulté (Cummins et al., 1988 & Verschaffel, 1981;De Corte et al., 1990;Zagar, 1991). Leur présence amène souvent les élèves à effectuer les calculs avant même d'avoir élaboré la structure du problème, d'où l'occurrence d'erreurs. ...
... Evidently, effective use of given information is also required in virtually any math problem solving context. For example, errors on math word problems often reflect misunderstanding of or failure to use given information (Cummins, Kintsch, Reusser, & Weimer, 1988;Hegarty, Mayer, & Monk, 1995). ...
... Students who were unsuccessful on the task often omitted given information from their proofs, stated given information inaccurately or imprecisely, or failed to draw some relevant inferences from the given information. Using given information appropriately is critical, and failure to do so effectively is a major source of error, on many other math tasks as well, such as word problem solving in arithmetic (Cummins et al., 1988;Hegarty et al., 1995) and algebra (Nathan, Kintsch, & Young, 1992;Sebrechts, Enright, Bennett, & Martin, 1996). Specific difficulties of this nature include misreading or misinterpreting the problem statement, making assumptions that are not given in the problem statement, and failure accurately to recall information given in the problem statement. ...
Geometric proof has been believed to involve reasoning skills that can be used in other domains of math. However, empirical evidence concerning this belief is scant. The present study tested the hypotheses (1) that there is substantial overlap in the reasoning skills involved in one aspect of geometric proof, termed geometric proof justification, and probabilistic reasoning, and (2) that this overlap includes but is not limited to deductive reasoning. University students completed a scaffolded geometric proof construction task in which they selected justifications for proof steps, and assessments of probabilistic reasoning, conditional inference, and algebra. Accuracy on the geometric proof justification task predicted probabilistic reasoning when controlling for conditional inference or algebra, though not both. Variance in probabilistic reasoning that was uniquely explained by geometric proof justification was related to effective use of given information on the geometric proof justification task. Implications regarding the role of reasoning in geometric proof justification and math more broadly are discussed.
... Wyndhamn and Säljö (1997), as a second example, were interested in students reasoning on how to calculate distances. Reasoning tasks are mostly presented as word problems (Cummins, Kintsch, Reusser, & Weimer, 1988), meaning the problem is embedded in a short text with one or more questions (Verschaffel, Schukajlow, Star, & Van Dooren, 2020). This is often accompanied by illustrations containing further information. ...
Mathematical reasoning is a diffcult activity for students, and although standards have been introduced worldwide, reasoning is seldom practiced in classrooms. Productively supporting students during the process of mathematical reasoning is a challenge for teachers, and applying formative feedback might increase students' reasoning effectiveness. Research has shown that the relationship between formative feedback and achievement is rather indirect, e.g. conveyed through students' self-efficacy beliefs. We examined whether the formative feedback perceived by students, as part of a 10-week student training programme, supported the development of reasoning competence via self-efficacy beliefs among 1261 students in 71 primary classes. We used multi-level modelling to analyse the expected relationships. On the class level, formative feedback predicted reasoning, which was mediated by self-efficacy; on the individual level, formative feedback predicted self-efficacy, but not reasoning. The results only partially confirmed our hypotheses. We discuss explanations for these findings and present implications for teaching mathematical reasoning.
... Similarly, in the context of solving word problems related to numbers and operations or geometry, students should translate words into numbers or figures. Cummins et al. (1988) have emphasized that correct answers to word problems are associated with strong reading comprehension skills in the context of algebra tasks. Although it can be asserted that reading comprehension has a greater impact on algebra than other content standards, more extensive research to confirm this assertion is needed. ...
... To solve algebraic word problems, a student needs to convert words into the language of algebra and to switch between the language of algebra and text language appropriately (Özcan & Doğan, 2018). Cummins et al. (1988) have stated that students' performance in algebraic word problems depends on their mathematical knowledge and their reading comprehension skills. Moreover, they emphasize that errors related to algebraic word problems result from miscomprehending the problem. ...
The literature on the association between reading comprehension and mathematics skills is complicated and conflicting. This study seeks to illuminate the nature of the association between mathematics skills and reading comprehension by incorporating potential moderators, namely components of mathematics skills, domains of content standards in mathematics, age, language status, and developmental issues. The dataset for this study included 49 studies with 91 correlation coefficients representing 37.654 participants. The findings obtained in this study showed that reading comprehension had a significantly strong effect on students' mathematics skills. This association was moderated by components of mathematics skills, domains of content standards in mathematics, age, language status, and developmental issues. Moderation analyses revealed that problem-solving was the strongest moderator of the association between reading comprehension and mathematics skills, whereas spatial skills were the weakest moderator of this relationship. Based on domains of content standards in mathematics, geometry was the weakest moderator of the association between mathematics skills and reading comprehension. Moreover, the effects of reading comprehension on students' mathematics skills significantly differed in favor of elementary students, students with learning disabilities, and second language learners. Therefore, this research can shed light on the literature by synthesizing the effects of reading comprehension on students' mathematics skills.
