No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
... The area of assessment has received much attention in recent decades worldwide, Israel included (Cai et al., 2020;Davison & Leung, 2009;Kulm, 2013;Zhao et al., 2018). In mathematics, assessment is inseparable from teaching and learning, and teachers must evaluate their students using advanced, decisive, and credible methods, which sometimes leads to difficulties (Cai et al., 2020). ...
... The area of assessment has received much attention in recent decades worldwide, Israel included (Cai et al., 2020;Davison & Leung, 2009;Kulm, 2013;Zhao et al., 2018). In mathematics, assessment is inseparable from teaching and learning, and teachers must evaluate their students using advanced, decisive, and credible methods, which sometimes leads to difficulties (Cai et al., 2020). It should not come as a surprise, therefore, that MTs face concerns about assessing student learning and achievement while having to take into consideration content and curricula goals and objectives, and suitable teaching methods, especially when dealing with heterogeneous classes and students with diverse needs (Cai et al., 2020;Veldhuis & van den Heuvel-Panhuizen, 2014. ...
... In mathematics, assessment is inseparable from teaching and learning, and teachers must evaluate their students using advanced, decisive, and credible methods, which sometimes leads to difficulties (Cai et al., 2020). It should not come as a surprise, therefore, that MTs face concerns about assessing student learning and achievement while having to take into consideration content and curricula goals and objectives, and suitable teaching methods, especially when dealing with heterogeneous classes and students with diverse needs (Cai et al., 2020;Veldhuis & van den Heuvel-Panhuizen, 2014. ...
Evaluation in mathematics is an inherent part of the discipline. In the current study, issues in the assessment of mathematics that concern MTs and S-MTs are studied. The basic assumption for this study is that improving teachers’ ability to deal with the challenges of assessment necessitates examining whether those issues are essential or technical. We first identified assessment issues that concern teachers and student teachers and then defined them as essential or technical difficulties. Sample questions included: What concerns do elementary and middle school MTs and S-MTs have with respect to student assessment, particularly with respect to alternative assessment methods? To what degree do these concerns represent essential or technical difficulties? At what frequency do elementary and middle school MTs and S-MTs encounter these difficulties? We found three main concerns – validity and reliability of exams; heterogeneity of the evaluated students; knowledge and achievements as reflected in the evaluation. The found that teachers seek solutions to practical assessment concerns and aspire to professional and credible evaluations that contribute to the students’ math development.
... Alternative assessment methods in mathematics are diverse: descriptive assessment, which includes open questions or a requirement that students detail the problem-solving process, so that teachers can analyze the way students resolved (and not just the end result), and accordingly -help them understand their learning and improve it (Cai et al., 2020c;Kim & Noh, 2010;National Council of Teachers of Mathematics, 2000). Other alternative assessment methods in math are oral exams and interviews (Kulm, 2013;Watt, 2005;Zhao et al., 2018). ...
... Such data that may inform the teacher about the way students think is particularly helpful in allowing teachers to anticipate students' responses to educational tasks. Thus, help teachers make better decisions in planning teaching units and implementing them, and improving the quality of their teaching (Cai et al., 2020a(Cai et al., , 2020c. In Israel, the internal and external Meitzav tests (school efficiency and growth measures) were instituted in the fifth and eighth grades (Beller, 2012(Beller, , 2013, and they provide data on the level of students, class, and school, which constitute formative evaluation and contribute to improving teaching quality. ...
... On the one hand, the studies indicate that alternative assessment methods help improve the assimilation of learners' learning processes, improve their academic achievement, develop personal learning potential, and improve their positive attitude toward mathematics (Davison & Leung, 2009;Ediger, 2013;Galustyan, 2017;Kulm, 2013;Sahin & Abali Ozturk, 2014;Savickiene, 2011;Veldhuis & van den Heuvel-Panhuizen, 2020;Zhao et al., 2018). On the other hand, despite the increasing recognition of alternative assessment methods as contributing to the quality of learning of learners and promoting their achievements, there are difficulties in their application (Briggset al., 2012;Cai et al., 2020c;Kingston & Nash, 2011, and in educating teachers to use informed considerations to choose valid and reliable assessment methods. ...
As part of efforts to improve the quality of mathematics' teaching and evaluation, we examined the focus of math teachers' considerations in evaluating students’ achievements, as well as the links between these focuses, regarding differences between students and the validity and reliability of assessment methods and examinations. Based on the categorization of issues that concern math teachers while assessing their students, a self-report quantitative-questionnaire was constructed and validated. The questionnaire included three foci of consideration for mathematics evaluation: considerations regarding evaluating learners, considerations regarding the choice of assessment methods, and assessment considerations that are unique to mathematics; positive, high, and significant were found between them. The main conclusion is that math teachers aspire to evaluate their students on the basis of a broad picture, that considers the needs of students studying mathematics as a unique discipline, the available assessment methods and their ability to adapt them to a valid and reliable assessment in mathematics, and the overall need to focus on the challenges and difficulties which are unique to assessing the discipline of mathematics. The findings also strengthen the claim that assessing math learning has unique considerations that are distinguished from assessment considerations in other disciplines.
... 13 How do we, as mathematics teacher educators, identify useful and research-based practices for meeting this goal? In a series of recent editorials, Cai and colleagues discussed several overarching goals for the field of mathematics education that included calls for (a) implementing research pathways that connect theory and practice (Cai et al., 2019) and (b) identifying, valuing, and sharing professional knowledge for teaching (Cai et al., 2020a). In our study, we aimed to translate FA research findings into actionable recommendations for mathematics teacher educators to support secondary mathematics teachers' FA practices. ...
In this Perspective on Practice , we describe an application of recommendations made in “Formative Assessment in Secondary Mathematics: Moving Theory to Recommendations for Evidence-Based Practice” (Kenney et al., 2022) , which was recently published in this journal. The authors made recommendations for mathematics teacher educators about conducting professional development sessions related to formative assessment. After conducting our own workshop for K–5 teachers and considering the recommendations of Kenney and her colleagues, we share our observations about the benefits of a good opening task for jumpstarting a productive session on formative assessment.
... Helping secondary mathematics teachers see the potential of FA and implement it How do we, as mathematics teacher educators, identify useful and research-based practices for meeting this goal? In a series of recent editorials, Cai and colleagues discussed several overarching goals for the field of mathematics education that included calls for (a) implementing research pathways that connect theory and practice (Cai et al., 2019) and (b) identifying, valuing, and sharing professional knowledge for teaching (Cai et al., 2020a). In our study, we aimed to translate FA research findings into actionable recommendations for mathematics teacher educators to support secondary mathematics teachers' FA practices. ...
Mathematics teacher educators play a key role in supporting secondary mathematics teachers’ development of effective, research-based formative assessment (FA) practices. We used qualitative research synthesis as a tool to identify actionable recommendations for mathematics teacher educators as they work with teachers on FA practices in secondary classrooms. These recommendations can strengthen the research-based practices of mathematics teacher educators as they support teachers’ collections and uses of FA data to move student thinking forward in secondary mathematics. We share and discuss recommendations for mathematics teacher educators to connect pedagogical content knowledge of students, teaching, and curriculum to FA practices. We also highlight the usefulness of the qualitative synthesis method, meta-aggregation, for generating research-based connections between theory and practice in mathematics education.
The field of knowledge brokering in education—aiming to better connect research to practice—is currently emerging. Evidence of a community dissonance between researchers and practitioners in education suggests that models of knowledge brokering that consider the perspectives and priorities of both groups are required. It is also a priority to identify what kinds of knowledge brokering products such as research summaries are successful at communicating research to teachers, potentially functioning as boundary objects. We report findings from a comparative judgement study where a group of 28 mathematics teachers and a group of 19 mathematics education researchers ranked twenty research summaries, from different sources, in terms of their success at communicating the research to teachers, and explained which features contributed to this ranking. Overall the findings suggest moderate consensus both within and between the two groups, and some important areas of difference. Both groups agreed that graphic design was the most important element of a research summary; that being summarised, having implications for practice and being easy to read or accessible and well‐structured were key features of a research summary; and that the length of the research summary or the time it might take to read were also important. Whereas teachers mentioned the choice of topic of the research summary and to some extent language more than researchers, researchers highlighted some other features of successful research summaries for them that teachers did not: ideas around methodology and use of evidence, opportunities for critical reflection, and issues of trust and credibility.
A tanítás-tanulás megváltoztatása az adatokon alapuló döntéshozatalon keresztül A mai, adattal, információval teli világban a pedagógiai munka tervezésében is egyre hangsúlyosabb az adatokra épülő döntéshozatal. Oktatási adatnak tekinthetjük többek között a pedagógiai mérések során keletkező adatokat (pl. tantárgyi dolgozatok vagy standardizált tesztek eredményeit), de ide tartozhatnak például a tanulók tantárgyi attitűdjének vizsgálatából nyert adatok és a hiányzások száma is. Mivel az adatok megfelelő felhasználásával növelhető a tanulók eredményessége és fejleszthető a pedagógiai munka, egyre több szakirodalom foglalkozik e problémakörrel. A témában megjelent könyvek között vannak elméleti és gyakorlati fókuszú, valamint a pedagógiai értékeléshez kötődő munkák is. Mandinach és Jackon Transforming Teaching and Learning Through Data-Driven Decision Making című könyve azért hiánypótló, mert egyesíti az elméletet gyakorlati alkalmazással.
In our May editorial (Cai et al., 2018a), we explored how collaborations among teacher-researcher partnerships could harness emerging technological resources to address the problem of isolation in the work of teachers and researchers. In particular, we described a professional knowledge base (Cai et al., 2018b) and a mechanism by which that knowledge base could be continuously populated, updated with data and resources that are useful to teachers and researchers, and shared among partnerships thereby enabling them to work on the same instructional problems. In this editorial, we shift our focus to discuss how data on students' thinking and classroom experiences could be leveraged within such a system to improve instructional practice. We will explore how the knowledge base could serve as a tool to (a) gather, process, and analyze data from individual students; (b) increase our understanding of the effects of students' mathematical learning experiences; and (c) help teacher-researcher partnerships understand and improve students' learning.
The use of data to inform instructional and educational decisions is an increasingly important facet of teachers’ professional practice. However, little is presently known about the best teacher learning mechanisms by which to promote data use. This study (N = 329) examined the nature and distribution of both Illinois public teacher data use practices and teacher learning for data use and how learning opportunities and other factors previously identified in the literature (e.g., leadership and teacher beliefs) relate to data use practices. Our study replicates the importance of specific teacher beliefs for data use practices, and contributes new evidence for the role of course-based learning opportunities.
Teachers who attempt to use inquiry-based, student-centered instructional tasks face challenges that go beyond identifying well-designed tasks and setting them up appropriately in the classroom. Because solution paths are usually not specified for these kinds of tasks, students tend to approach them in unique and sometimes unanticipated ways. Teachers must not only strive to understand how students are making sense of the task but also begin to align students' disparate ideas and approaches with canonical understandings about the nature of mathematics. Research suggests that this is difficult for most teachers (Ball, 1993, 2001; Leinhardt & Steele, 2005; Schoenfeld, 1998; Sherin, 2002). In this article, we present a pedagogical model that specifies five key practices teachers can learn to use student responses to such tasks more effectively in discussions: anticipating, monitoring, selecting, sequencing, and making connections between student responses. We first define each practice, showing how a typical discussion based on a cognitively challenging task could be improved through their use. We then explain how the five practices embody current theory about how to support students' productive disciplinary engagement. Finally, we close by discussing how these practices can make discussion-based pedagogy manageable for more teachers.
In this article we propose that an understanding of students' thinking can provide coherence to teachers' pedagogical content knowledge and their knowledge of subject matter, curriculum, and pedagogy. We describe a research-based model of children's thinking that teachers can use to interpret, transform, and reframe their informal or spontaneous knowledge about students' mathematical thinking. Our major thesis is that children enter school with a great deal of informal or intuitive knowledge of mathematics that can serve as the basis for developing much of the formal mathematics of the primary school curriculum. The development of abstract symbolic procedures is characterized as progressive abstractions of students' attempts to model action and relations depicted in problems. Although we focus on one facet of teachers' pedagogical content knowledge, we argue that understanding students' thinking provides a basis for teachers to reconceptualize their own knowledge more broadly.
Data-driven decision making has become an essential component of educational practice across all levels, from chief state school officers to classroom teachers, and has received unprecedented attention in terms of policy and financial support. It was included as one of the four pillars in the American Recovery and Reinvestment Act (2009), indicating that federal education officials seek to ensure that data and evidence are used to inform policy and practice. This article describes the emergence of data-driven decision making as a topic of interest, some of the challenges to and opportunities for data use, and how the principles of educational psychology can and must be used to inform how educators are using data and the examination of its impact on educational practice.
The purpose of this article is to describe how, within the Dutch approach to mathematics education, called Realistic Mathematics Education (RME), models are used to elicit students' growth in understanding of mathematics. First some background information is given
about the characteristics of RME related to the role of models in this approach. Then the focus is on the use of the bar model
within a longitudinal trajectory on percentage that has been designed for Mathematics in Context, a curriculum for the U.S. middle school. The power of this model is that it develops alongside both the teaching and the
students: from a drawing that represents a context related to percentage to a strip for estimation and reasoning to an abstract
tool that supports the use of percentage as an operator.
Building on the claim that teachers need to know how innovative instructional approaches work to be able to adapt them to their own classrooms, design research is presented as a research method that aims to offer exactly that kind of information. We elaborate design research that aims to develop a local instruction theory-a theory about the process by which students learn a given topic in mathematics and theories about the means of support for that learning process. We will illustrate this with the example of design research on a local instruction theory on addition and subtraction to 100. We further discuss design research that combines the 2 goals of teacher learning and student learning in 1 project as a special case of research on teacher learning. In closing, we briefly look into the relation between design research andteacher research.
This article focuses on a form of instructional design that is deemed fitting for reform mathematics education. Reform mathematics education requires instruction that helps students in developing their current ways of reasoning into more sophisticated ways of mathematical reasoning. This implies that there has to be ample room for teachers to adjust their instruction to the students' thinking. But, the point of departure is that if justice is to be done to the input of the students and their ideas built on, a well-founded plan is needed. Design research on an instructional sequence on addition and subtraction up to 100 is taken as an instance to elucidate how the theory for realistic mathematics education (RME) can be used to develop a local instruction theory that can function as such a plan. Instead of offering an instructional sequence that "works," the objective of design research is to offer teachers an empirically grounded theory on how a certain set of instructional activities can work. The example of addition and subtraction up to 100 is used to clarify how a local instruction theory informs teachers about learning goals, instructional activities, student thinking and learning, and the role of tools and imagery.
In this editorial, we elaborate our vision of the changing roles of researchers and teachers in a future world in which research has a much more direct and meaningful impact on practice (Cai et al., 2017). In previous editorials, we have described characteristics of this future world, including setting research agendas based on instructional problems teachers want to solve (Cai et al., 2017a), developing authentic partnerships between researchers and teachers and connecting multiple partnerships to solve common problems (Cai et al., 2017a, 2018a, 2018b), using new technologies to collect and analyze data on the relationships between students' instructional and learning histories that would enable teachers to plan more effective lessons (Cai et al., 2018a, 2018b), taking advantage of connected partnerships and new data-gathering technologies to build a knowledge base accessible to all teachers facing similar instructional problems (Cai et al., 2018a, 2018b, 2018d), and creating new incentives to appropriately reward researchers and teachers for improving the learning opportunities for all students across classrooms within their school district or state (Cai et al., 2017a). We have alluded to the changing roles this vision would require, including researchers developing hypothetical learning trajectories for concepts that are implicated in teachers' instructional problems (Cai et al., 2017b) and teachers accepting professional responsibilities for contributing to knowledge that improves instruction in all classrooms in their district or state rather than just in their own classroom (Cai et al., 2017a). In this editorial, we create a more complete picture of the new professional roles of researchers and teachers in this future world that intertwines research and practice.
In our March editorial (Cai et al., 2018), we considered the problem of isolation in the work of teachers and researchers. In particular, we proposed ways to take advantage of emerging technological resources, such as online archives of student data linked to instructional activities and indexed by learning goals, to produce a professional knowledge base (Cai et al., 2017b, 2018). This proposal would refashion our conceptions of the nature and collection of data so that teachers, researchers, and teacher-researcher partnerships could benefit from the accumulated learning of ordinarily isolated groups. Although we have discussed the general parameters for such a system in previous editorials, in this editorial, we present a potential mechanism for accumulating learning into a professional knowledge base, a mechanism that involves collaboration between multiple teacher-researcher partnerships. To illustrate our ideas, we return once again to the collaboration between fourth-grade teacher Mr. Lovemath and mathematics education researcher Ms. Research, who are mentioned in our previous editorials(Cai et al., 2017a, 2017b).
Improving teacher evaluation is one of the most pressing but also contested areas of educational policy. Value-added measures have received much of the attention in new evaluation systems, but they can only be used to evaluate a fraction of teachers. Classroom observations are almost universally used to assess teachers, yet their statistical properties have received far less empirical scrutiny, in particular in consequential evaluation systems. In this essay, we highlight some conceptual and empirical challenges that are similar across these different measures of teacher quality. Based on a review of empirical research, we argue that we need much more research focused on observations as performance measures. We conclude by sketching out an agenda for future research in this area.
To improve classroom teaching in a steady, lasting way, the teaching profession needs a knowledge base that grows and improves. In spite of the continuing efforts of researchers, archived research knowledge has had little effect on the improvement of practice in the average classroom. We explore the possibility of building a useful knowledge base for teaching by beginning with practitioners’ knowledge. We outline key features of this knowledge and identify the requirements for this knowledge to be transformed into a professional knowledge base for teaching. By reviewing educational history, we offer an incomplete explanation for why the United States has no countrywide system that meets these requirements. We conclude by wondering if U.S. researchers and teachers can make different choices in the future to enable a system for building and sustaining a professional knowledge base for teaching.
Background/Context
Concerns about instructional quality have unleashed a frenzy of reform efforts, including changes in school governance, teacher recruitment, and curriculum, modification of teaching philosophies and procedures, stricter accountability management, and increased quests for scientific research information and guidance. These efforts have resulted in much hope but modest returns. We have middling success in educating our talented students and fail catastrophically with many others. Science and mathematics instruction are among our most critical weak spots. Urgent demand for technically competent people in the workplace and society calls for early relief from our educational discomforts.
Purpose/Focus
The essay examines how we can take advantage of the colossal scale of the American educational system to improve instruction. Although policy makers have persistently turned away from the fact, the same subject matter is taught in thousands of schools, year after year, and thousands of teachers face very similar instructional tasks. The huge scale of their efforts, coupled with progress in the instructive uses of digital devices, makes it sensible and economically feasible to provide teachers with powerful tools to serve their common needs. We examine requirements and possible configurations for such digital instructional tools and their uses, and we make some cost estimates.
Research Design
This is an analytic essay.
Conclusions/Recommendations
Substantial investment of technical and financial resources is proposed for the development of a pedagogic information support system (APIS) for critical courses or subject matter in which competent teachers are in short supply. These systems should be sufficiently comprehensive to cover all core components that a reasonable teacher might include, as well as instrumentally pedagogic elements. APISs are seen as loosely organized, highly redundant collections, in polymorphic digital forms, of explanations, demonstrations, simulations, interactive exercises, problems, examples, elaborations, integrative expositions, and motivational supports, as well as a deep, searchable information base. An APIS for a particular course would be sufficient to support inexperienced teachers and skillful veterans and could be used both with monitored class groups and supervised individual students. APIS can be expected to provide new opportunities for practical teacher education and to stimulate and coordinate instructional research.
Background/Context
Much has been written about the efficacy of formative assessment. However, relatively little of that research focuses on the design qualities of the assessment instruments themselves that might provide useful information to teachers.
Purpose/Objective/Research Question/Focus of Study
This article decomposes the formative assessment process and reviews the empirical literature on the feedback from an assessment event to the teacher to investigate two research questions. First, what are the design elements of assessments that can enhance their embedded information to help teachers understand how students are thinking about a particular content area? Second, what do teachers need to know to be able to take advantage of more sophisticated assessment designs?
Research Design
The article reviews the literature on test design qualities, focusing on empirical studies that have examined how the qualities of tests inform teachers about student understanding. Overall, 117 articles were synthesized to extract three qualities of test design that enable or constrain their potential to provide insight to teachers about student thinking.
Findings/Results
The literature review identified three crucial test qualities that improve their potential value to teachers. First, tests can convey information about students’ developmental path toward a learning goal. Second, tests can provide information about students’ thought processes. Third, tests can be designed to reveal information about students’ misconceptions within a content area.
Conclusions/Recommendations
The article concludes with a conceptual framework to guide future research on ways to maximize the potential feedback that teachers can receive from assessments. The framework, and hence suggestions for future research, focuses on conducting research on the hypothesized relationships between the qualities of a test itself, the analyses that teachers conduct from the resulting data of student answers, teachers’ instructional responses, and the effect of this process on student learning.
Background
Amid calls for increased data use, there is little research or policy guidance for how to build schools’ capacity to leverage data to improve teaching and learning. Building on previous research highlighting the social nature of data use, we contend that in order to understand how capacity develops, research must focus on relationships and networks that support educators’ practice, conceptualizing capacity as socially embedded.
Purpose
This article explores the development of data use capacity in an elementary school through a social network approach. Our analysis focuses on the structure of data advice networks, the characteristics of perceived experts in the network, and the productiveness of the network in terms of influencing beliefs and practice.
Population
Data come from a sample of 42 educators from an elementary school exemplified by its district as a strong user of data to improve teaching and learning. Participants completed a survey about their data use beliefs, practices, and school context, as well as a social network questionnaire indicating from whom they sought advice on using data.
Research Design
We used the survey data to identify characteristics of the schools’ data use networks using descriptive statistics and social network analysis (SNA). SNA was also used to develop measures of structural location in those networks, which were then used to predict similarities in teachers’ beliefs and practices around data use.
Findings
Findings reveal that data use networks are influenced by the larger professional structure of the school, with data advice being from colleagues who are part of their larger professional network. Network structure reveals few highly central “advice givers” and many “advice seekers” connected by teachers and leaders who serve as brokers of advice. We find that brokers may play an important role in developing shared practices, given that the indirect relationships they support are predictive of shared data use practices.
Conclusions
This research is among the first to explore data use through a social network approach and offers early evidence about how educators’ networks enable schools to build capacity for data use. Our findings have implications for the design of professional development, for professional development for school leaders, and for successful implementation of reforms related to data use.
Background:
Data use has been promoted as a panacea for instructional improvement. However, the field lacks a detailed understanding of how teachers’ actually use assessment data to inform instruction and the factors that shape this process.
Purpose:
This paper provides a review of literature on teachers’ use of assessment data to inform instruction. We draw primarily on empirical studies of data use that have been published in the past decade, most of which have been conducted as data-driven decision making came into more widespread use. The paper reviews research on the types of assessment data teachers use to inform instruction, how teachers analyze data, and how their instruction is impacted.
Research Design:
Review of research.
.
Findings:
In the current accountability context, benchmark assessment data predominate in teachers’ work with data. While teachers are often asked to analyze data in a consistent way, agendas for data use, the nature of the assessments, and teacher beliefs all come into play, leading to variability in how they use data. Instructional changes on the basis of data often focus on struggling students, raising some equity concerns. The general absence of professional development has hampered teachers’ efforts to use data, as well as their confidence in doing so.
Conclusions:
Given that interim benchmark assessment data predominate in teachers’ work with data, we need to think more deeply about the content of those assessments, as well as how we can create conditions for teachers to use assessment to inform instruction. This review of research underscores the need for further research in this area, as well teacher professional development on how to translate assessment data into information that can inform instructional planning.
This 3-year longitudinal study investigated the development of 82 children's understanding of multidigit number concepts and operations in Grades 1-3. Students were individually interviewed 5 times on a variety of tasks involving base-ten number concepts and addition and subtraction problems. The study provides an existence proof that children can invent strategies for adding and subtracting and illustrates both what that invention affords and the role that different concepts may play in that invention. About 90% of the students used invented strategies. Students who used invented strategies before they learned standard algorithms demonstrated better knowledge of base-ten number concepts and were more successful in extending their knowledge to new situations than were students who initially learned standard algorithms.
Background
Increasingly, teachers and other educators are expected to leverage data in making educational decisions. Effective data use is difficult, if not impossible, without computer data systems. Nonetheless, these systems may be underused or even rejected by teachers. One potential explanation for such troubles may relate to how teachers have made sense of such technologies in practice. Recognizing the interpretive flexibility of computer data systems provides an avenue into exploring these issues.
Objective
This study aims to explore the factors affecting teachers’ use of computer data systems. Drawing upon the notion of interpretive flexibility, it highlights the influence of sensemaking processes on the use and implementation of computer data systems.
Research Design
This comparative case study draws upon interview and observational data gathered in three school districts. Matrices were used to compare understandings about data use and about computer data systems within each district by job role (i.e., central office member, campus administrator, and teacher), as well as across districts.
Results
Our findings challenge commonplace assumptions about technologies and their “effects” on teacher work. For example, access to a system or its functions did not determine changes of practice. Paradoxically, we even found that teachers could reject or ignore functions of which they were personally in favor. Although computer data systems can support changes of practice, we found that agency for change rested in people, not in the technologies themselves. Indeed, teachers’ sensemaking about “data” and “data use” shaped whether and how systems were used in practice. Although central offices could be important to sensemak-ing, this role was often underplayed.
Conclusion
We provide recommendations regarding how researchers, school, and district leaders might better conceptualize data and data systems. These recommendations include recognizing implementation as an extended period of social adjustment. Further, we emphasize that it is the unique duty of school and district leaders to share their visions regarding data use, as well as to engage in dialogue with their communities about the natures of schooling and data use.
Background
In the past decade, there has been an increasing national policy push for educators to systematically collect, interpret, and use data for instructional decision making. The assumption by the federal government is that having data systems will be enough to prompt the use of data for a wide range of decision making. These policies rely on inducements to inspire local level changes; however, they leave the processes related to data use largely undefined.
Objective
In this article, I argue that many of the studies on data use either invoke or directly assess network-related concepts and, as such, network theory and analysis provides a useful analytic and complementary framework and methods for examining the social infrastructure in the use of data for educational improvement.
Research Design
This article reports on a literature review of the data use and social network literatures and the utility of intersecting both literatures for studies on data use for educational improvement.
Conclusions
Many data use studies report that the interpretation and use of data takes place both within and between individuals who, through social interaction, are both co-constructing and making sense of data and their use. Given the increasing role of social relationships in data use studies better theorizing and understanding the dynamics of social influence and processes on the interpretation and use of data is needed. Social network theory and analysis offers a useful conceptual framework and accompanying methods for describing and analyzing the structure of a social system in an effort to understand how social relationships support and constrain the interpretation and use of data in educational improvement.
This article presents initial findings from an evaluation research study of the implementation of a Web-based decision support tool, the Quality School Portfolio (QSP), developed at the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) at the University of California, Los Angeles (UCLA). The study focused on users' experiences with the training for and implementation of QSP. Data were collected by telephone interviews. The results show that QSP provided educators with enhanced access to more extensive and broadly founded student data and with the ability to analyze student data to identify at-risk students. Additionally, QSP was found to promote collaboration and shared planning among educators. It is concluded that technology tools, which can facilitate the analysis and reporting of educational data, have opened up the prospect of timely identification of at-risk students and interventions to meet their educational needs. Tools like this also support sound assessment practices, providing opportunities for frequent assessment and other evidence of competency beyond standardized testing.
Accountability mandates such as No Child Left Behind (NCLB) have drawn atten-tion to the practical use of student data for school improvement. Nevertheless, schools may struggle with these mandates because student data are often stored in forms that are difficult to access, manipulate, and interpret. Such access barriers ad-ditionally preclude the use of data at the classroom level to inform and impact in-struction. Fortunately, there are newly available computer technologies that allow ef-ficient organization and access to student data. In addition to allowing easier accountability reporting, these tools allow user-friendly data access at all educational levels, meaning that teachers can use these tools to engage in the informed reflection necessary to improve classroom practice. In this article, I discuss teacher use of these systems, providing insight into the function of these tools and discussing conditions that make these tools of the most service to teachers. The use of data to inform educational decisions has recently drawn increased at-tention, spurred largely by accountability requirements set forth at the state and federal levels. A familiar example is the 2002 No Child Left Behind (NCLB) legis-lation, which mandates a significant increase in the gathering, aggregation, and up-ward reporting of student-level data. NCLB policy carries an implicit assumption that the availability of data will inform and initiate changes in teaching practice, but mechanisms for helping educators turn accountability data into actionable in-JOURNAL OF EDUCATION FOR STUDENTS PLACED AT RISK, 10(3), 295–308 Copyright © 2005, Lawrence Erlbaum Associates, Inc.
Narrative and story in practice and research
- Clandinin
Clandinin, D. J., & Connelly, F. M. (1991). Narrative and story in practice and research. In D. A.
Schon (Ed.), The reflective turn: Case studies in and on educational practice (pp. 258-281).
Teachers College Press.
Using student achievement data to support instructional decision making (Report No. NCEE 2009-4067). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences
- L Hamilton
- R Halverson
- S Jackson
- E Mandinach
- J Supovitz
- J Wayman
Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009). Using
student achievement data to support instructional decision making (Report No. NCEE 2009-4067). National Center for Education Evaluation and Regional Assistance, Institute of Education
Sciences, U.S. Department of Education. http://ies.ed.gov/ncee/wwc/publications/practiceguides/
Herbel-Eisenmann, B. A., & Cirillo, M. (Eds.). (2009). Promoting purposeful discourse: Teacher
research in mathematics classrooms. National Council of Teachers of Mathematics.
Capturing craft knowledge in teaching
- Leinhardt
Leinhardt, G. (1990). Capturing craft knowledge in teaching. Educational Researcher, 19(2), 18-25.
https://doi.org/10.3102/0013189X019002018
A taxonomy of approaches to learning trajectories and progressions
- Lobato
Lobato, J., & Walters, C. D. (2017). A taxonomy of approaches to learning trajectories and
progressions. In J. Cai (Ed.), Compendium for research in mathematics education (pp. 74-101).
National Council of Teachers of Mathematics.
Transforming teaching and learning through data-driven decision making
- E B Mandinach
- S S Jackson
Mandinach, E. B., & Jackson, S. S. (2012). Transforming teaching and learning through data-driven
decision making. Corwin.
Making sense of data-driven decision making in education: Evidence from recent RAND research (Report No. OP-170-EDU)
- J A Marsh
- J F Pane
- L S Hamilton
Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in
education: Evidence from recent RAND research (Report No. OP-170-EDU). RAND Corporation.
Teachers' ability to use data to inform instruction: Challenges and supports
- B Means
- E Chen
- A Debarger
- C Padilla
Means, B., Chen, E., DeBarger, A., & Padilla, C. (2011). Teachers' ability to use data to inform
instruction: Challenges and supports. U.S. Department of Education, Office of Planning,
Evaluation, and Policy Development.
Use of education data at the local level: From accountability to instructional improvement
- B Means
- C Padilla
- L Gallagher
Means, B., Padilla, C., & Gallagher, L. (2010). Use of education data at the local level: From
accountability to instructional improvement. U.S. Department of Education, Office of Planning,
Evaluation, and Policy Development.
A fresh look at formative assessment in mathematics teaching
- E A Silver
- V L Mills
Silver, E. A., & Mills, V. L. (2018). A fresh look at formative assessment in mathematics teaching.
National Council of Teachers of Mathematics.
Evaluating teachers with classroom observations: Lessons learned in four districts
- G J Whitehurst
- M M Chingos
- K M Lindquist
Whitehurst, G. J., Chingos, M. M., & Lindquist, K. M. (2014). Evaluating teachers with classroom
observations: Lessons learned in four districts. Brown Center on Education Policy.
- van den Heuvel-Panhuizen
A knowledge base for the teaching profession: What would it look like and how can we get one?
- Hiebert