Table 6 - uploaded by Caren Diefenderfer
Content may be subject to copyright.
Source publication
In this third paper in a series describing the Quantitative Reasoning in the Contemporary World course, the authors provide an adaptation of the Association of American Colleges and Universities quantitative literacy VALUE rubric. Describing achievement levels in six core competencies (interpretation, representation, calculation, analysis/synthesis...
Context in source publication
Similar publications
This study relates factors in nature of Mathematics and its teaching learning to student difficulties for diverse mathematics tasks. Descriptive survey was done on a sample of 300 high school students in Kerala with a questionnaire on difficulties in learning. Student perception of difficulty on 26 types of tasks, under five heads that students may...
Citations
... References (partial list) Models related to adult numeracy and mathematical literacy Geiger et al. (2015), Kilpatrick (2001), Gal et al. (2005), PIAAC Numeracy Expert Group (2009), Tout and Gal (2015) Models related to broad quantitative reasoning (QR) competencies Boersma et al. (2011), Madison (2014 Heuristics related to critical reading of quantitative statements and other aspects of critical thinking Gal (2002), Ridgway et al. (2016) Models related to statistical literacy, probability literacy, risk literacy, data literacy, and official statistics literacy Callingham and Watson (2017), Gal (2002Gal ( , 2005, Gal and Ograjenšek (2017), Gould (2017), Watson (2013) Models describing cognitive processes in understanding tables and graphs Friel et al. (2001), Kemp and Kisanne (2010), Sharma (2013), Prodromou (2015), Schield (2016) Models and perspectives describing desired behaviours of citizens and citizen engagement, as well as outcomes of statistics education or needed extensions in the twenty-first century, given trends regarding digitization, big data, data science, etc. ...
This chapter presents a comprehensive conceptual framework of 11 facets and tools which together describe the knowledge, skills and dispositions that (young) adults need in order to comprehend, critically evaluate, communicate about, and engage with Civic Statistics regarding ‘burning’ societal issues, and that may enhance citizen empowerment. The framework is organized around three key dimensions involving engagement & action, knowledge, and enabling processes. It identifies knowledge-bases covering meaning for society and policy and critical evaluation and reflection; selected statistical and mathematical constructs and skills; core literacy and mathematical skills; understanding models and modelling, multivariate ideas and textual and rich visual representations; knowledge of research and data production methods and extensions related to official statistics and risk on the societal level; and it emphasises the importance of appropriate dispositions, critical stance, and habits of mind. We offer examples and curriculum tasks that illustrate each of the 11 facets and their interconnectedness. We also describe the use of a ‘radar plot’ tool to support the analysis of how balanced are prospective class activities or test items in terms of covering the 11 facets and tools. The chapter ends with a brief discussion of the implications of the conceptual model and its 11 facets for planning curricula, instruction, and assessments that can promote teaching and learning about Civic Statistics within mathematics education, statistics and data science education, and related disciplines.
... To overcome this limitation, Boersma et al. (2011) adapt the phrasing of the institutional QL VALUE rubric to create the Quantitative Literacy Assessment Rubric (QLAR). The QLAR was originally designed for grading students' written responses to questions in a casebook for a Quantitative Reasoning in the Contemporary World (QRCW) course. ...
... The QLAR was originally designed for grading students' written responses to questions in a casebook for a Quantitative Reasoning in the Contemporary World (QRCW) course. While the QLAR retained the six criteria from the QL VALUE rubric, Boersma et al. (2011) modified the descriptions of the QL VALUE rubric's performance levels because they had problems producing reliable results with the rubric's original phrasing. After mapping a series of casebook questions to the six QL competencies in the QL VALUE rubric, two readers compared their QLAR evaluations of answers to various QRCW casebook questions. ...
... In two separate tests, the coders had strong inter-rater agreement of 88% and 97%. Boersma et al.'s (2011) study highlighted the importance of testing (and, if needed, modifying) institutional QR rubrics to develop tools that function well in a classroom environment. Table 3 compares the purpose of the two main types of rubrics: holistic and analytic. ...
Institutional assessments of quantitative literacy/reasoning (QL/QR) have been extensively tested and reported in the literature. While appropriate for measuring student learning at the programmatic or institutional level, such instruments were not designed for classroom grading. After modifying a widely accepted institutional rubric designed to assess QR in written arguments, the current mixed method study tested the reliability of two QR analytic grading rubrics for written arguments and explored students’ reactions to the grading tools. Undergraduate students enrolled in a business course ( N = 59) participated. A total of 415 QR artifacts from 40 students were assessed; an additional 19 students provided feedback about the grading tools. A new QR writing rubric included three main criteria (numerical evidence, conclusions, and writing), while a second rubric added a fourth criterion for assignments with data visualization. After two coders rated students’ QR assignments, data analysis found both new QR rubrics had good reliability. Cohen’s kappa found the study’s raters had substantial agreement on all rubric criteria (κ = 0.69 to 0.80). Both the QR writing (α = 0.861) and data visualization (α = 0.859) grading rubrics also had good internal consistency. When asked to provide feedback about the new grading tools, 89% of students shared positive comments, reporting the rubrics clarified assignment expectations, improved their performance, and facilitated the writing process. This paper proposes slight modifications to the phrasing of the new rubrics’ writing criterion, discusses best practices for use of rubrics in QR classrooms, and recommends future research.
... Geiger et al., 2015;Kilpatrick, 2001;Gal et al., 2005;PIAAC Numeracy Expert Group, 2009;Tout & Gal, 2015. Models relate to broad quantitative reasoning (QR) competencies Boersma et al., 2011;Madison, 2014. Heuristics related to critical reading of quantitative statements and other aspects of critical thinking Gal, 2002;Ridgway et al., 2016. ...
To appear as Chapter 3 in a book (forthcoming early 2022 from Springer), with 23 peer-reviewed chapters, written in coordination by an international group of experts from ten countries.
___________WHERE TO FIND AND DOWNLOAD A PRE-PRINT (AND SEE THE BOOK'S FULL TABLE OF CONTENTS)____________: Please access my personal webpage at the University of Haifa here: https://hw.haifa.ac.il/en/people/human/igal -- Click on 'Publications' tab, scroll to 'Chapters'. There you can also find links to two other chapters I (co-)wrote for this book: Ch 4 and 13.
_________ABSTRACT__________: This chapter presents a comprehensive conceptual framework of 11 facets and tools which together describe the knowledge, skills and dispositions that (young) adults need in order to comprehend, critically evaluate, communicate about, and engage with Civic Statistics regarding 'burning' societal issues. The framework is organized around three key dimensions involving engagement & action, knowledge, and enabling processes. It identifies knowledge-bases covering selected statistical and mathematical constructs and skills; core literacy and mathematical skills; understanding models and modelling, multivariate ideas and textual and rich visual representations; knowledge of research and data production methods and extensions related to official statistics and risk on the societal level; and it emphasises the importance of appropriate dispositions. We offer examples and curriculum tasks that illustrate each of the 11 facets and their interconnectedness. We also describe the use of a 'radar plot' tool to support the analysis of how balanced are prospective class activities or test items in terms of covering the 11 facets and tools. The chapter ends with a brief discussion of the implications of the conceptual model and its 11 facets for planning curricula, instruction, and assessments related to the promotion of teaching and learning about Civic Statistics.
__________BOOK OVERVIEW_________: The book offers practical approaches to working in a new field of knowledge - Civic Statistics - which sets out to engage with, and overcome well documented and long-standing problems in teaching quantitative skills. This book is one of several products of the ProCivicStat project, which can be found on the IASE webpge here: http://iase-web.org/islp/pcs ._________
Effective citizen engagement with social issues requires active participation and a broad understanding of data and statistics about societal issues. However, many statistics curricula are not designed to teach relevant skills nor to improve learners' statistical literacy. The book aims to support and enhance the work of teachers and lecturers working both at the high school and tertiary (university) levels. It is designed to promote and improve the critical understanding of quantitative evidence relevant to burning social issues – such as epidemics, climate change, poverty, migration, natural disasters, inequality, employment, and racism.
Evidence about social issues is provided to the public via print and digital media, official statistics offices, and other information channels, and a great deal of data is accessible both as aggregated summaries and as individual records. Chapters illustrate the approaches needed to teach and promote the knowledge, skills, dispositions, and enabling processes associated with critical understanding of Civic Statistics presented in many forms. These include statistical analysis of authentic multivariate data, use of dynamic data visualisations, and deconstructing texts about the social and economic well-being of societies and communities. Chapters discuss ideas regarding the development of curricula and educational resources, use of emerging technologies and visualizations, preparation of teachers and teaching approaches and sources for relevant datasets and rich texts about Civic Statistics, and ideas regarding future research, assessment, collaborations between different stakeholders, and other systemic issues.
... In particular, mapping course assessment items (e.g., mathematical tasks on exams or in-class group activities) to specific learning objectives can provide evidence about how the objectives are represented holistically across the course assessment materials. In coursework centered on quantitative literacy, attention to the alignment between learning goals and assessments is of particular importance, given that quantitative literacy is challenging to both operationalize and assess (Shavelson 2008;Boersma et al. 2011). Notwithstanding this challenge, as courses in quantitative literacy grow in scale-both in the form of large course sizes (Tunstall et al. 2016) and the distribution of uniform curricula (e.g., Quantway or Statway) 1 across institutions-there is a marked need for attention to the operationalization and assessment of learning objectives in such courses. ...
In this analysis, we examine how course assessment items were aligned with learning objectives in a quantitative literacy course at Michigan State University. The alignment analysis consisted of mapping assessment items to a list of operationalized learning objectives from the course. Our analysis shows how often the learning objectives are represented in assessment items, how often they are paired with other learning objectives, and how influential they are in contributing to a student’s course grade. In addition, through comparisons across four assessment types (e.g., exams and homework), we show how each learning objective was assessed differently within each assessment type. The most frequently represented learning objectives in the particular course we studied concern the creation and interpretation of graphical representations; these learning objectives were assessed relatively evenly across the assessment types. However, those learning objectives often co-occurred with other objectives in assessment items, and the point values per item associated with these objectives were less than those for other objectives. Our study shows how quantitative literacy learning objectives can vary with assessment type in a course, and also provides numeracy scholars with an analysis technique suitable for use at their respective institutions.
... Nov. 24, 2018). See alsoBoersma et al. 2011. ...
In a previous article, we described our college’s new core curriculum, which included a Quantitative Literacy (QL) component for the first time. We explained how we defined QL in the college catalog, and how we used that definition to choose courses to satisfy the new requirement. We then discussed our early efforts at assessing the effectiveness of the QL program and described our plans for the future. Here we report on our progress towards those goals, including working with faculty from other departments and with our institutional research office to develop a more sophisticated assessment plan, as well as creating and implementing easier-to-use surveys and assessment instruments.
... As developed by an Association of American Colleges and Universities (AAC&U) panel of collegiate faculty and refined by Boersma et al. (2011), six core competencies are required for QR: interpretation, representation, calculation, analysis/synthesis, assumption, and communication. One way to assess course materials for a course in QR is to determine how frequently each of these core competencies is required to complete exercises. ...
Ethan D. Bolker and Maura B. Mast. 2016. Common Sense Mathematics.(Washington DC.: Mathematics Association of America) ISBN-13: 978-1-93951-210-9.
Common Sense Mathematics is an integrative quantitative reasoning (QR) textbook that is built around scores of exercises derived from authentic circumstances from public media and other public sources. The exercises elicit responses from students requiring extensive communication and analyses and distinguish the book from ones typically encountered in a mathematics or science course. Responses to exercises often require one-half page or more of writing and can occupy considerable class time in discussion. The book has material for a one- or two-semester course. Use of the Internet for information is assumed, and the use of spreadsheet technology is incorporated but can be avoided for portions of the latter chapters.
... (4) Finally, I found difficulties similar to those in the literature regarding assessment, and while I found the QLAR rubric of Boersma et al. (2011) to be generally helpful, I did not find it specific enough to highlight key elements in many of my QL assignments. ...
This article reports some of the key findings from a practitioner-action research study that analyzed the impact of co-construction on students’ quantitative literacy (QL) and attitudes towards mathematics. Co-construction is a process where students work alongside their teachers to plan units, lessons, and assessments, and this approach was chosen because of its potential to help students advocate for the specific mathematical contexts that would best develop their QL. This yearlong study took place in a public high school, with forty-five students in two different classes participating. Students formally contributed to the development of the course by completing written questionnaires and participating in large- and small-group discussions, and they contributed informally through their participation in class, performance on assessments, and reflections on various assignments, which I considered in field notes and a research journal. I used the constant comparative method to analyze these data, and I arrived at three key themes: (1) co-construction recast traditional roles in the classroom, while still serving as an important form of instruction in itself; (2) developmentally appropriate assignments triggered students’ situational interest, while individualized co-construction proved somewhat effective at developing individual interest; and (3) the co-construction process challenged the way I used to think about mathematics teaching, as students and I reconsidered the content, technology, and classroom practices that would best develop students’ QL. This study fills an important gap in the QL and co-construction literature, and it also has important implications for mathematics practitioners, and for practitioners who want to give students more of a voice in the classroom.
... My meaning for QL is based on the six core competencies for QL as developed in the AAC&U VALUE rubric for QL as modified by Boersma, et al. (2011). The six core competencies are interpretation, representation, calculation, analysis/synthesis, assumption, and communication. ...
How supportive of quantitative literacy (QL) are the Common Core State Standards in Mathematics (CCSSM)? The answer is tentative and conditional. There are some QL-supportive features including a strong probability and statistics strand in grade 6 through high school; a measurements and data strand in K-5; ratio and proportional reasoning standards in grades 6 and 7; and a comprehensive and coherent approach to algebraic reasoning and logical argument. However, the standards are weak in supporting reasoning and interpretation, and there are indications that the applications in CCSSM – mostly unspecified – will not include many QL contextual situations. Early indicators of assessment items follow a similar path. Except for statistics, most of the high school standards are aimed at development of algebra and precalculus topics, and there will likely be little room for more sophisticated applications of the QL-friendly mathematics of grades 6-8. The experience with CCSSM is limited at this point, leaving several crucial results uncertain, including assessments, emphases on statistics, and kinds of modeling and other applications.
... The design principles that are presented below evolved over the past eight years of teaching QR courses to college students and are rooted in the author's work with Robert Orrill and Lynn Steen in the QR initiative that Orrill led during 2000-2004(Madison and Steen 2008b. Almost all of the principles have been described in three research reports on the QR course (Dingman and Madison 2010;Madison and Dingman 2010;Boersma et al. 2011). These principles were articulated more thoroughly in light of five researchbased and one experience-based sets of criteria on student learning in the analysis of the QR course at the University of Arkansas in a reverse engineering process prompted by an increased need to evaluate, justify and improve the course and its outcomes. ...
... • The six core competencies for QL as articulated in the Association of American Colleges and Universities (AAC&U) QL rubric (AAC&U 2009;Boersma et al. 2011). ...
... The author and colleagues (Boersma et al. 2011) adapted the AAC&U VALUE QL rubric to one to assess individual student work. The result was the Quantitative Literacy Assessment Rubric (QLAR). ...
In the absence of generally accepted content standards and with little evidence on the learning for long-term retrieval and transfer, how does one design or evaluate a course in quantitative reasoning (QR)? This is a report on one way to do so. The subject QR course, which has college algebra as a prerequisite and has been taught for 8 years, is being modified slightly to be offered as an alternative to college algebra. One modification is adding a significant formal writing component. As the modification occurs, the current course and the modified one are judged according to six sets of criteria: the six core competencies of the Association of American Colleges and Universities rubric on quantitative literacy; the five mathematical competencies from the National Research Council (NRC) study report, Adding It Up; the eight practice standards from the Common Core State Standards for Mathematics; the five elements of effective thinking as articulated by Edward Burger and Michael Starbird, the summary research findings on human cognition from the NRC study report, How People Learn; and the ten principles gleaned from applying the science of learning to university teaching. The QR course, as described by ten design principles, is determined to be generally well aligned with most of the overlapping criteria of the six sets, providing cogent evidence of high educational value.
... Across individual dimensions, scores were in complete agreement 63.6% of the time and within one 91.3% of the time. This approach to interrater reliability and our levels of agreement are consistent with a number of earlier studies using rubrics to assess student learning (e.g., Blue et al., 2008; Boersma et al., 2011; Stellmack, Konheim-Kalkstein, Manor, Massey, & Schmitz, 2009; Thaler, Kazemi, Huscher, 2009). Scores for each dimension of the rubric have been combined across both student prompts for ease of exposition. ...
A full-cycle assessment of our efforts to improve quantitative reasoning in an introductory math course is described. Our initial iteration substituted more open-ended performance tasks for the active learning projects than had been used. Using a quasi-experimental design, we compared multiple sections of the same course and found non-significant gains on a pre/post, rubric-scored, measure of quantitative reasoning. Subsequent course modifications included more explicit emphasis on critical thinking as a course goal and extended experience with the rubric used to score the performance tasks. Results of the second iteration yielded stronger evidence for gains in quantitative reasoning and suggest that the impact of open-ended performance tasks is increased when supported by efforts that emphasize their importance. Abstract A full-cycle assessment of our efforts to improve quantitative reasoning in an introductory math course is described. Our initial iteration substituted more open-ended performance tasks for the active learning projects than had been used. Using a quasi-experimental design, we compared multiple sections of the same course and found non-significant gains on a pre/post, rubric-scored, measure of quantitative reasoning. Subsequent course modifications included more explicit emphasis on critical thinking as a course goal and extended experience with the rubric used to score the performance tasks. Results of the second iteration yielded stronger evidence for gains in quantitative reasoning and suggest that the impact of open-ended performance tasks is increased when supported by efforts that emphasize their importance.
