Article

What does it mean for teachers to be data literate: Laying out the skills, knowledge, and dispositions

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

This article reports on the evolution of a conceptual framework for a construct called data literacy for teachers. Data use has become an emphasis in education but few educators have received sufficient training or preparation pertaining to data literacy skills. This article lays out the framework, identifying the specific knowledge, skills, and dispositions teachers need to use data effectively and responsibly. It concludes with a call to schools of education and teacher preparation programs to begin to integrate data literacy into curricula and practical experiences.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... However, little is known about the optimum ways in which this integration can or should take place (Reeves & Honig, 2015). Difficulties in determining how to build DIDM into EPP programming may be rooted in the fact that effective data use calls for a complex synthesis of teacher knowledge and skills (DeLuca & Bellara, 2013;Mandinach & Gummer, 2016). Another challenge is that integrating DIDM into EPPs may be contingent upon teacher educators possessing data literacy skills and understanding how to teach data use for instructional 1286798J TEXXX10.1177/00224871241286798Journal of Teacher EducationHock et al. ...
... For example, teachers may focus primarily on using data to measure achievement rather than for the improvement of learning outcomes (Garner et al., 2017). This focus on achievement data-to the exclusion of other data sources that are critical for contextualizing student learning (Mandinach & Gummer, 2016)-may narrow the curricula to make room for additional high-stakes test preparation. Also, data use may affirm teachers' biases about students in ways that maintain the very inequities that DIDM is intended to upend, particularly if data use reinforces low expectations for struggling learners (Datnow & Park, 2018). ...
... According to Beck and Nunnaley (2021), data literacy exists on a continuum, ranging from the novice phase to the expert phase. Given this, researchers have called for EPPs and schools to work together to help teachers develop data literacy (Mandinach & Gummer, 2016), recognizing that datause skills are built incrementally over the course of teachers' careers (Means et al., 2011). ...
Article
Because data-informed decision-making (DIDM) can help teachers meet diverse learners’ needs (van Geel et al., 2016), educator preparation programs (EPPs) must ensure that preservice teachers (PSTs) develop the data literacy skills needed for effective data use. However, little is known about the ways in which EPPs work towards building PSTs’ data literacy, despite licensure and accreditation requirements compelling EPPs to do so. In this study, we analyzed survey, document, and interview data from Virginia EPPs to determine what present practices for DIDM preparation are taking place across the state. Results point to a lack of uniformity among EPPs for how preparation is undertaken, and that PSTS seem to have limited coursework on data use. Additionally, there appears to be minimal collaboration between EPPs and clinical partners, such that PSTs infrequently have opportunities to engage in DIDM during field experiences.
... Data use usually follows a systematic and cyclical process of intentionally using data to inform their instruction. Literature suggests that it involves a process of establishing a goal for data use, collecting data, making sense of the results of data analysis and interpretation to convert data into A conceptual framework for studying the degree of data use practice JPCC usable information, taking action to improve teaching and learning, and finally doing an evaluation (Lai and Schildkamp, 2013;Mandinach and Gummer, 2016a;Schildkamp and Poortman, 2015). Despite this sequential way, teachers sometimes use data in a non-linear way , for example, when they find their hypothesis is incorrect after data interpretation and then go back to the previous step to reformulate the hypothesis . ...
... User-level characteristics generally include teachers' knowledge and skills as well as attitudes toward data (e.g. Datnow and Hubbard, 2016;Hoogland et al., 2016;Mandinach and Gummer, 2013;Mandinach and Gummer, 2016a;Schildkamp and Poortman, 2015;Schildkamp et al., 2017). In the following section, we will discuss the literature on individual-level factors that potentially predict instructional data use. ...
... For example, a data-literate teacher may survey students to gauge their engagement levels in different subjects and interpret the feedback to modify instructional strategies, thereby creating more engaging learning experiences. However, a lack of these competencies can lead to resistance to data use, resulting in poor decision-making and potential misuse of data (Wayman et al., 2007(Wayman et al., , 2010Daly, 2012;Mandinach and Gummer, 2016b). In this study, the terms "knowledge and skills in using data" and "data literacy" are used interchangeably to denote the essential capabilities required for effectively utilising various types of data to inform and enhance instructional practices. ...
Article
Purpose: An evidence-based approach to improving instructional practices and student outcomes in data use. It is a systematic process of evaluating and analysing learning problems, collecting and transforming various types of data into instructional decisions, and implementing informed actions to improve instruction and student learning. Since teachers are the main actors in instructional practices, this article reports on a study aimed at predicting the influence of various teachers’ characteristics on the degree of data use practices for instructional purposes. Design/methodology/approach: In this study, we conducted a survey in a developing country to gather primary data. The collected data were analysed using a supervised machine learning approach, focussing on decision tree analysis, to determine the influential factors. Findings: Our investigation identifies pedagogical knowledge, data literacy, content knowledge, knowledge of English for teaching and attitudes towards data as crucial determinants in predicting the intensity of such data use practices. Notably, pedagogical knowledge emerges as the most potent predictor, emphasising its pivotal role in shaping teachers’ frequency of instructional data use practices. Surprisingly, English proficiency does not exhibit a significant influence in this predictive model. Research limitations/implications: The findings may not be generalisable to a wider context since this study relied on a relatively small teacher self-reported sample collected through surveys, and, as this study used perception data, this may or may not reflect teachers’ actual knowledge and skills. Practical implications: By spotlighting the nuanced interplay between teacher individual characteristics and the practice of data use for instructional improvement, this research contributes to a nuanced understanding of the factors shaping teachers’ engagement with data. Ultimately, it provides a foundation for targeted interventions and strategies aimed at fostering a culture of evidence-based practices to improve instruction and consequently student learning outcomes within educational settings. Social implications: This insight holds significant implications for policymakers, educational practitioners and providers of professional development programmes seeking to facilitate effective data use practices for instructional improvement. Originality/value: By spotlighting the nuanced interplay between teacher individual characteristics and the practice of data use for instructional purposes, this research contributes to a nuanced understanding of the factors shaping teachers’ engagement with data. This study represents an empirical examination of such factors by employing a quantitative approach: a flexible decision tree analysis. This contributes to a growing body of research on factors related to teacher characteristics and much of the research in the field of data use has been done using a qualitative approach.
... Teachers' capacity to engage in DBDM includes a wide set of necessary knowledge, attitudes, and skills, often referred to as "data literacy". Data literacy is defined by Mandinach and Gummer [3] as "the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data to help determine instructional steps. It combines an understanding of data with standards, disciplinary knowledge and practices, curricular knowledge, pedagogical content knowledge, and an understanding of how children learn" (p. ...
... In this paper, when we refer to the DBDM cycle, we refer to five core steps, namely, goal setting, data collection, data analysis and interpretation, improvement actions, and evaluation. Consequently, data literacy includes the knowledge and skills to (1) collect, read, understand, and evaluate quantitative and qualitative data and results of basic quantitative and qualitative analysis and (2) use this information to support DBDM [3,12]. ...
... As DBDM is a complex process, teachers and school teams need to be data literate and possess a range of skills, such as content knowledge, knowledge of the school vision, and knowledge of the curriculum. Eight experts (2,3,4,6,7,8,13,14) reported on implementing PLN as a way for teachers to improve their data literacy (outward brokering) and in turn implement the newly acquired knowledge back into the PLC in their school (inward brokering). Outward brokering of knowledge involves the process of teachers accessing external knowledge and expertise on data literacy in the PLN in order to allow them to successfully complete the DBDM cycle: ...
Article
Full-text available
In the last decade data-based decision making has been promoted to stimulate school improvement and student learning. However, many teachers struggle with one or more elements of data-based decision making, as they are often not data literate. In this exploratory study, professional learning networks are presented as a way to provide access to data literacy that is not available in schools. Through interviews with scientific experts (n = 14), professional learning networks are shown to contribute to data-based decision making in four ways: (1) by regulating motivation and emotions throughout the process, (2) by encouraging cooperation by sharing different perspectives and experiences, (3) increasing collaboration to solve complex educational problems, and (4) encouraging both inward and outward brokering of knowledge. The experts interviewed have varying experiences on whether professional learning networks should have a homogenous and heterogenous composition, the degree of involvement of the school leaders, and which competencies a facilitator needs to facilitate the process of data-based decision making in a professional learning network.
... The other study used students ages 8 to 13, typically Years 3 to 8 (Freeman et al., 2016). Some studies focused on junior high school (n=3), involving Year 9 and 10 middle school students (Cano et al., 2014;Glogger et al., 2012;Malepa-Qhobela & Mosimege, 2022). Only one study focused on senior high school students (Hofstein et al., 2005). ...
... Other studies also collected students' responses in a blog (Freeman et al., 2016). Two case studies obtained data from actual observations, audio recordings, journals, and field notes (Martin et al., 2017;Malepa-Qhobela & Mosimege, 2022). Only one study used a mixedmethod design where data was collected through questionnaires, field notes, observation, and interviews (Baiduri, 2017). ...
... Most of the studies assessed the writing activities of students in their science notebooks (Biyik & Şenel, 2019), learning journals (Glogger et al., 2012), reflection journals (Martin et al., 2017), drawings with discussions of conceptual knowledge (Smith et al., 2019), and student's discussions in their notepads and blog (Freeman et al., 2016). Furthermore, other studies assessed reading comprehension and its relationship to the student's learning achievement (Cano et al., 2014), question answer relationship as observed in the student's reading level of the comprehension passage (Kinniburgh & Baxter, 2012), ability to understand and solve maths problems (Malepa-Qhobela & Mosimege, 2022), and student's reading strategies in reading science trade books (Lai & Chan, 2020). One study assessed the student's speaking skills as manifested in their ability to ask a question and actively participate in the discussions during their peer tutoring sessions (Baiduri, 2017). ...
... Situations such as which data to choose, how to process it, how to use it, etc., become possible with data literacy. Data literacy is defined as "a person's level of understanding of how to find, evaluate, and use data to know how to teach" (Mandinach and Gummer, 2016). According to Qin and D'Ignazio (2010), data literacy focuses on functional skills in data collection, processing, management, evaluation, and use. ...
... and Gummer (AF=1.33). When the two most cited studies are examined, it is seen that the study by Mandinach (2016), published in the Journal of Teaching and Teacher Education, ranked first with 121 total citations and 15.13 total citations per year. The study by Prado (2013) ranked second with 108 total citations and 9.82 total citations per year. ...
... The study by Prado (2013) ranked second with 108 total citations and 9.82 total citations per year. Concerning the content of these studies, the study by Mandinach (2016) addressed a framework that defines the specific knowledge, skills, and attitudes required for teachers to use data in an effective and responsible way (Mandinach & Gummer, 2016). The study by Prado (2013) indicated that it did not aim to contribute to the advancement of data literacy with the proposal of a set of core competencies and contents that can serve as a reference framework for its inclusion in the information literacy programs of libraries (Calzada Prado & Marzal, 2013). ...
Article
Full-text available
Nowadays, large amounts of data are produced in every field, and these data have become a strategic asset for individuals, businesses, states, and societies. Data literacy involves the skills of using these data effectively. The first aim of the current study is to determine in which years and in which periods more research has been conducted on education and data literacy, reveal the trends in this field over time, and understand what is at the forefront in education and data literacy by analyzing changes in research trends. The second aim of this study is to understand which topics have a wider impact in the field by examining the content of the most cited studies, and to evaluate the potential for collaboration by analyzing the collaborative tendencies and main topic focuses of authors and organizations. This study adopted a science mapping method to examine the relationship in the field of education and data literacy and reveal a general condition of the research on this subject. Data for this study were collected through the Web of Science (WoS) database. As a result of the analysis, an apparent increase in research on education and data literacy since 2011 was examined, and a trend that peaked in 2020 but rose again in 2022 was identified. It was shown that technology and big data-oriented studies have gained importance in thematic evolution. This analysis provides a valuable resource for understanding the current state in the field of education and data literacy and identifying strategies for potential areas of future development.
... In recent years, there has been a growing recognition of the importance of teachers' data literacy for educational policy, research, and practice. This trend was ignited in 2009 when Arne Duncan, the former Secretary of Education of the United States, advocated evidence-driven practices in schools to enhance student performance (Mandinach & Gummer, 2016). Since then, there has been an increasing expectation for teachers to engage in data-informed practices to guide teaching and decisionmaking in schools. ...
... We argue that there are several reasons to justify the need for this systematic review. Firstly, we update, complement, and compare our review outcomes and the DLFT framework in Mandinach and Gummer (2016). A systematic review of research studies on teachers' data use was conducted by Mandinach and Gummer (2013b), but the study selection was limited to years between 2001 and 2009. ...
... A systematic review of research studies on teachers' data use was conducted by Mandinach and Gummer (2013b), but the study selection was limited to years between 2001 and 2009. Therefore, one of the aims of the present study is to compare our systematic review outcomes against the dimensions and specific indicators identified in the DLFT framework (Mandinach & Gummer, 2016). The present literature search spans a period from 1990 to 2021. ...
Article
Full-text available
The current study presents a systematic review of teachers’ data literacy, arising from a synthesis of 83 empirical studies published between 1990 to 2021. Our review identified 95 distinct indicators across five dimensions: (a) knowledge about data, (b) skills in using data, (c) dispositions towards data use, (d) data application for various purposes, and (e) data-related behaviors. Our findings indicate that teachers' data literacy goes beyond addressing the needs of supporting student learning and includes elements such as teacher reflection, collaboration, communication, and participation in professional development. Considering these findings, future policies should acknowledge the significance of teacher dispositions and behaviors in relation to data, recognizing that they are as important as knowledge and skills acquisition. Additionally, prioritizing the provision of system-level support to foster teacher collaboration within in-school professional development programs may prove useful in enhancing teachers’ data literacy.
... The complexity of data-based decision-making becomes evident in the process of data use (Mandinach & Schildkamp, 2021). The circular process of data use contains several steps (Keuning et al., 2019;Lai & Schildkamp, 2013;Mandinach & Gummer, 2016;Sampson et al., 2022). We are following the data use for teaching process consisting of five steps: (1) the identification of problems and framing questions, (2) the use of data, (3) the transformation of data into information, (4) the transformation of information into decisions, and (5) the evaluation of outcomes (Mandinach & Gummer, 2016). ...
... The circular process of data use contains several steps (Keuning et al., 2019;Lai & Schildkamp, 2013;Mandinach & Gummer, 2016;Sampson et al., 2022). We are following the data use for teaching process consisting of five steps: (1) the identification of problems and framing questions, (2) the use of data, (3) the transformation of data into information, (4) the transformation of information into decisions, and (5) the evaluation of outcomes (Mandinach & Gummer, 2016). Data should not be collected without a specific goal. ...
... Therefore, the process starts with identifying problems and formulating leading questions to solve the problems by using data (Schildkamp, 2019). In the next step, for example, teachers need to think about which sources they want to use and collect the data (Mandinach & Gummer, 2016;Schildkamp, 2019). The transformation of data into information is part of the data analysis. ...
Article
Full-text available
Data-based decision-making is a well-established field of research in education. In particular, the potential of data use for addressing heterogeneous learning needs is emphasized. With data collected during the learning process of students, teachers gain insight into the performance, strengths, and weaknesses of their students and are potentially able to adjust their teaching accordingly. Digital media are becoming increasingly important for the use of learning data. Students can use digital learning platforms to work on exercises and receive direct feedback, while teachers gain data on the students’ learning processes. Although both data-based decision-making and the use of digital media in schools are already widely studied, there is little evidence on the combination of the two issues. This systematic review aims to answer to what extent the connection between data-based decision-making and the use of digital learning platforms has already been researched in terms of using digital learning data for further instructional design. The analysis of n = 11 studies revealed that the use of data from digital learning platforms for instructional design has so far been researched exploratively. Nevertheless, we gained initial insights into which digital learning platforms teachers use, which data they can obtain from them, and how they further use these data.
... As the research has also clearly demonstrated that effective data usage in education is among the common and key characteristics of high-performing schools with outstanding student results [23,24], one of the newest competencies of an effective teacher is the teacher's data literacy. According to [23] data literacy for teaching is the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data (assessment, school climate, behavioral, snapshot, longitudinal, moment-to-moment, etc.) to help determine instructional steps. ...
... As the research has also clearly demonstrated that effective data usage in education is among the common and key characteristics of high-performing schools with outstanding student results [23,24], one of the newest competencies of an effective teacher is the teacher's data literacy. According to [23] data literacy for teaching is the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data (assessment, school climate, behavioral, snapshot, longitudinal, moment-to-moment, etc.) to help determine instructional steps. ...
... To support teachers to be data-literate in the effective use of data, training is necessary. However, based on our experience and that of others [23,24], offering such training and convincing in-service teachers to participate is quite challenging due to the demanding nature of teaching and the significant workload. ...
Article
Full-text available
The COVID-19 pandemic has underscored the importance of blended learning in contemporary physics and, more generally, STEM education. In this contribution, we summarize current pedagogical models of blended learning, such as rotational and flexible non-rotational models, and customizable configurations of physical and virtual learning spaces. With the inevitable integration of digital technology as one of the pillars of blended learning, teachers find themselves in an unprecedented position to not only obtain data more frequently but also analyze it and adjust instruction accordingly. Consequently, we discuss a crucial element of blended learning effectiveness: data management and usage. In this context, data literacy for teaching emerges as an essential skill for effective blended learning, encompassing the ability to transform various data types into actionable instructional knowledge and practices. In other words, current research in physics education shows that a data-literate science teacher is a more prosperous and effective teacher.
... Access to the site was given prior to the boot camp so participants could get familiar with the contents and had better preparation for the boot camp. Teachers from participating schools were involved in planning the boot camps, considering the critical role of teachers in developing students' data literacy skills [19]. An initial meeting was conducted to discuss the details of the boot camps with the teachers and to gain an understanding of what has currently been taught at school. ...
... Information from teachers was instrumental to understand the starting points to scaffold participants' data literacy skills through the boot camp. Involving teachers in the development of data literacy boot camp programs aimed to ensure that the contents were relevant, aligned with educational agenda, and met the needs of both students and the school [19]. This approach was also aligned with the first fold of the framework of the study to look at current levels of data literacy and skills taught at participating schools [13] in order to enhance students' skills. ...
Article
Full-text available
Data is becoming increasingly ubiquitous today, and data literacy has emerged an essential skill in the workplace. Therefore, it is necessary to equip high school students with data literacy skills in order to prepare them for further learning and future employment. In Indonesia, there is a growing shift towards integrating data literacy in the high school curriculum. As part of a pilot intervention project, academics from two leading Universities organised data literacy boot camps for high school students across various cities in Indonesia. The boot camps aimed at increasing participants’ awareness of the power of analytical and exploration skills, which in turn, would contribute to creating independent and data-literate students. This paper explores student participants’ self-perception of their data literacy as a result of the skills acquired from the boot camps. Qualitative and quantitative data were collected through student surveys and a focus group discussion, and were used to analyse student perception post-intervention. The findings indicate that students became more aware of the usefulness of data literacy and its application in future studies and work after participating in the boot camp. Of the materials delivered at the boot camps, students found the greatest benefit in learning basic statistical concepts and applying them through the use of Microsoft Excel as a tool for basic data analysis. These findings provide valuable policy recommendations that educators and policymakers can use as guidelines for effective data literacy teaching in high schools.
... The extensive use of data provides an opportunity to educate teachers on its effective use within educational contexts (Mandinach & Gummer, 2016). Nevertheless, current studies reveal limited engagement with data among teachers, primarily focusing on accountability rather than school development or instructional improvement (Mandinach & Gummer, 2016). ...
... The extensive use of data provides an opportunity to educate teachers on its effective use within educational contexts (Mandinach & Gummer, 2016). Nevertheless, current studies reveal limited engagement with data among teachers, primarily focusing on accountability rather than school development or instructional improvement (Mandinach & Gummer, 2016). Despite acknowledging the necessity for teachers to work with data, there has been minimal effort to enhance teachers' data literacy through professional development programs or explicit recruitment criteria (Mandinach & Jimerson, 2016). ...
Article
Full-text available
The trend of early retirement among teachers is worrisome as it results in the loss of experienced teachers and contributes to the expanding teacher shortage. This condition is thought to have been caused by the workload and strain connected with technology and data use. Early retirement signifies teachers’ burnout, stress, and lack of job satisfaction, and multiple studies have suggested that they significantly affect professional self-efficacy. This article explores the influence of technology and data use on a teacher's self-efficacy in relation to their profession. A total of 525 school teachers in Malaysia have participated in a research study by completing a questionnaire. The results indicated that teacher professional self-efficacy, technology use, and data use are at a moderate level. The findings demonstrate a favorable correlation among these constructs. However, the connections are not influenced by factors such as age, gender, or school location. The study's implications for the higher education setting are also addressed, suggesting the implementation of enhancement of professional development opportunities, mentorship and peer support, and recognition to teachers. These suggestions aim to better equip teachers with the indispensable skills required in the swiftly evolving realm of data and technology.
... In the context of data use, an educational professional's self-efficacy expresses the extent to which they feel capable of engaging in data use because they possess the necessary competences to do so (Bandura, 1997;. In other words, self-efficacy expresses confidence over one's data literacy, that is, one's knowledge and skills for processing data and formulating responses accordingly (Mandinach & Gummer, 2016). The literature paints a rather pessimistic picture with regard to educators' data literacy (e.g., van der Kleij & Eggen, 2013;Vanhoof et al., 2011) and finds that many still profess to feeling insecure in this respect (Datnow & Hubbard, 2016;Earl & Fullan, 2003). ...
... In this study, we conceptualize self-efficacy as SPF users feeling they are able to understand SPF, interpret it, and translate it into concrete actions. We thus acknowledge that data literacy surpasses mere statistical literacy, but also entails the transformation of information into actionable knowledge (Mandinach & Gummer, 2016). ...
Article
Full-text available
The present study explores predictors of school performance feedback (SPF) use. In total, 470 Flemish educational professionals were surveyed about their use of SPF from school-external, low-stakes standardized assessments. A path analysis was conducted in order to investigate how individual user beliefs impact SPF use at the school level and how those beliefs mediate the effects of school-level features pertaining to school organization, performance, and voluntariness. Findings include that users' cognitive attitude and perceived expectations of others have a small effect on engagement with SPF in schools, and that these predictors mediate the effects of certain organizational characteristics. Whereas performance levels do not impact school-level feedback use, voluntariness in feedback pursuit and particularly an SPF-oriented school culture emerge as drivers. Implications for practice include the need for stimulating ownership in data-based decision making. Suggestions for further research are also discussed.
... For example, socio-demographic changes in the U.S. have led to calls for increased attention to how to work with diverse learners (Livingston & Flores, 2017;Morrier et al., 2007;National Academy of Sciences, 2022). In response to respective expansions to assessment-focused accountability policies, and the influx of digital educational technologies, other domains of teacher professional practice that have received emergent attention include assessment (DeLuca & Bellara, 2013;Livingston & Flores, 2017;Mandinach & Gummer, 2016) and information and communication technology (Livingston & Flores, 2017;Organization for Economic Cooperation and Development, 2005). There has been calls in the U.S. and internationally for more attention to practice, and the "core tasks" of teaching during PSTE (Ball & Forzani, 2009;Boyd et al., 2009;Darling-Hammond et al., 2005;Jenset et al., 2018). ...
... Both of these findings are somewhat curious. A decline in PSTE emphasis on monitoring students' development in learning is unexpected given the general expansion in assessment over this period nationally, especially external assessment (DeLuca & Bellara, 2013), and extensive attention paid to data use in education in the last decade or so (Mandinach & Gummer, 2016). ...
... Hence, to transform data into actionable information, teachers needed, in addition to access to relevant data in a timely and systematic manner, which may promote sustainable processes of decision-making, knowledge regarding how to interpret it, and how to use it effectively (Reeves & Chiang, 2018). However, most teachers lack the knowledge and skills to do this (Mandinach & Gummer, 2016). This may indicate a need for teacher training and professional development, e.g., workshops, that focus on analyzing, interpreting, and using data so that teachers can collect data and analyze it systematically and adequately, i.e., in a manner that is beneficial to their teaching (Schildkamp et al., 2014;Staman et al., 2014). ...
... In addition to accessing relevant data in a timely manner, teachers should be equipped with knowledge regarding how to interpret it and how to use it effectively (Mausethagen et al., 2018;Reeves & Chiang, 2018). However, most teachers lack the knowledge and skills to do so (Mandinach & Gummer, 2016), which points to the need for relevant teacher training and professional development (Schildkamp et al., 2014;Staman et al., 2014). Besides that, the role of school culture is important in establishing a culture of DDDM (Blanc et al., 2010;Farley-Ripple & Buttram, 2014;Harris et al., 2020;Hoogland et al., 2016). ...
Article
Full-text available
Aim/Purpose This study explored teachers’ data-driven decision-making processes during routine and emergency remote teaching, as experienced during the COVID-19 pandemic. Background Decision-making is essential in teaching, with informed decisions promoting student learning and teachers’ professional development most effectively. However, obstacles to the use of data have been identified in many studies. Methodology Using a qualitative methodology (N=20), we studied how teachers make decisions, what data is available, and what data they would like to have to improve their decision-making. We used an inductive approach (bottom-up), utilizing teachers’ statements related to decision-making as the unit of analysis. Contribution Our findings shed an important light on teachers’ Data-Driven Decision-Making (DDDM), highlighting the differences between routine and Emergency Remote Teaching (ERT). Findings Overall, we found that teachers make teaching decisions in three main areas: pedagogy, discipline-related issues, and appearance and behavior. They shift between making decisions based on data and making decisions based on intuition. Academic-related decisions are the most prominent in routine teaching, and during ERT, they were almost the only area in which teachers’ decisions were made. Teachers reported collecting data about students’ academic achievements and emotional state and considered the organizational culture, consultation with colleagues, and parents’ involvement before decision-making. Recommendations for Practitioners Promote a culture of data-driven decision-making across the education system; Make diverse and rich data of different types accessible to teachers; Increase professional and emotional support for teachers. Recommendation for Researchers Researchers have the potential to expand the scope of this study by conducting research using other methodologies and in different countries. Impact on Society This study highlights the importance of teachers’ data-driven decision-making in improving teaching practices and promoting students’ achievement. Future Research Additional research is required to examine data-driven decision-making in diverse circumstances.
... Teachers need to master diverse competencies to utilize SPTQ data to improve their practices, which as a whole are called data literacy (Mandinach and Gummer, 2016). Teachers usually find it difficult to use data to improve their practice (Cowie and Cooper, 2017;Sun et al., 2016). ...
Article
Full-text available
In this study, we examined why a Teacher Professional Development (TPD) program, designed to support teachers in using students’ perceptions of teaching quality (SPTQ) data, faced significant implementation challenges in 17 secondary schools in Chile. Despite voluntary participation and initial interest, 15 of the 17 schools dropped out within 2–3 months of starting the program. Through 12 semi-structured interviews with professional learning community coordinators from nine schools, we investigated four key attributes of the TPD program to understand implementation challenges: its added value, compatibility, clarity, and tolerance. While coordinators valued several aspects of the program (including its structured manual, evidence-based teaching strategies, and integration of SPTQ data) significant implementation barriers emerged. Time constraints, lack of technological infrastructure, and insufficient organizational routines made the implementation of the TPD program too burdensome for most schools. We discuss how compatibility between TPD programs and schools’ existing structures and routines acts as a critical bottleneck that can prevent successful implementation, even when participants see value in the program. This study provides important insights into the conditions necessary for successful TPD implementation in a global south country.
... Data use helps schools reach key goals, such as to enhance instruction, create strong learning communities and to potentially improve student outcomes (Levin, 2010). However, previous research has found that teachers often lack the skills and knowledge about how to use data effectively for school improvement (Schildkamp and Kuiper, 2010;Carroll and Carroll, 2015;Mandinach and Gummer, 2016b;Rääk et al., 2021). Another problem is that data use is not always unified or systematic (Vanlommel, 2018;Rääk et al., 2021;Siemann, 2021). ...
Article
Full-text available
Educational systems worldwide seek sustainable school improvement by fostering collaborative organizational routines that support teachers' practises and students' learning outcomes. This study examines how five Estonian schools perceive evidence-driven school improvement in a 3-year school-university partnership program. In each school, the principal and teachers collaborated with an external mentor. Supported by university experts, the school improvement teams worked on projects aimed at enhancing student learning in their schools and fostering a collaborative, evidence-driven school culture. Data was collected through focus group interviews with the school teams and analyzed using thematic content analysis. The findings reveal that schools view data as connected to accountability and decision-making, with considerably less emphasis on instructional improvement. School organizational and teacher-related factors, together with data overload, hindered systematic data use. Notably, the school improvement program's effectiveness was most evident in the final year, with the sustainability of improvement largely dependent on collaborative routines.
... Over the years this has evolved from a traditional data skills approach to models and definitions that also include critical thinking and data equity. Comparably, Mandinach and Gummer (2016) developed a data literacy framework for teachers, with the primary focus on data use for teaching improvement. However, in later publications, Mandinach and Jimerson (2021, p. 9) expand that data use is "at the heart of data ethics," educators need to carefully consider their actions regarding data usage, the methods they employ in their work, and how they focus their efforts on benefiting-rather than harming-the stakeholders involved. ...
Article
Full-text available
As society increasingly recognises the value of data, proficiency in using and understanding data has become crucial. However, there is no universal consensus on the definition of data literacy. Therefore, this study provides the first extensive, mixed methods scoping review of the topical evolution of data literacy within social and educational sciences from 2011 to 2023. By identifying key themes and research trends, this review offers a comprehensive understanding of the dynamic nature of data literacy. Our sample consists of 210 English-language, peer-reviewed articles from Scopus and Web of Science. The findings reveal a field that is evolving alongside media and digital literacy discussions, with notable growth in publications, particularly in 2019, 2020, and 2023, thus highlighting data literacy’s recognition as a distinct paradigm. Data literacy is shifting beyond traditional frameworks, with increasing attention to issues of equity and accessibility—areas still underexplored in current literature. Notably, the research demonstrates a shift from simply developing individual data skills to fostering a socially aware form of data literacy that empowers citizens to critically engage with data and navigate a datafied society actively and responsibly. This review emphasises the need for a nuanced, context-specific approach to data literacy, much like digital literacy, as different demographics and contexts encounter varying needs and challenges. As a dynamic, ever-evolving concept, future research and programs must address these diverse levels of engagement and expertise, ensuring that data literacy is inclusive, adaptable, and supported by social structures.
... Teacher data-driven decision-making (DDDM) to inform pedagogical practice and improve student outcomes is gaining significant prominence in many educational systems worldwide (Mandinach & Gummer, 2016;Mandinach & Schildkamp, 2021a, b). Teachers' DDDM skills comprise both assessment literacy (Beck & Nunnaley, 2021) and data literacy (Mandinach & Schildkamp, 2021a, 2021b to effectively design and implement effective assessment activities to practice because teachers reject systems and functions which they do not favour or are unfamiliar with (Cho & Wayman, 2014). ...
Article
Full-text available
The use of information and communication technology-based data systems to support teachers in data-driven decision-making (DDDM) remains limited. Despite the growing number of data systems available, their uptake remains limited, and there is a limited understanding of what data system characteristics increase and factors that influence teacher adoption and use. To address this gap, we reviewed the literature using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Synthesis of the 17 articles from three databases revealed six data systems commonly used in schools. Also, there are eight key data system characteristics that teachers find helpful. We have found several factors that influence teacher adoption of data systems related to data features, leadership, individual disposition, and the socio-cultural context. The findings of our review have critical implications for designing and using technology-based data systems for supporting teacher data-driven decision-making.
... Crain-Dorough and Elder (2021) found that educational researchers and teacher practitioner communities differ in how they define and use knowledge. Presenting the work of Mandinach and Gummer (2016), Crain-Dorough and Elder (2021) described seven forms of knowledge often utilized by teacher practitioners: content knowledge; general pedagogical knowledge; curriculum knowledge; pedagogical content knowledge; knowledge of learners and their characteristics; knowledge of contexts; and knowledge of educational ends, purposes, and values. Such forms of knowledge are localized, based on application and outcome, and often not replicable at scale. ...
Article
Full-text available
As Flyvbjerg et al. (2012) explained, critique is a core mechanism within democracy to keep institutions "effective and improving" (p. 294). People must be critical of the structures they live in because the policies and practices designed and implemented within those structures represent their/our values. If outcomes related to those policies misalign with the values of those who live and work by them, something should change. In this article, I analyze the alignment between values and practices as they relate to conceptualizing the "influence" of educational researchers. Do conceptualizations of influence that can be connected to how social media influencers are labeled as such align with the values educational researchers believe are important to embody to be considered influential? Moreover, do the actions incentivized by this conceptualization of influence align with outcomes educational researchers believe are influential? If the answer to either of these questions is no, I argue for a re-concep-tualization of influence to better align the values and practices of educational researchers with the goals and outcomes they seek to obtain.
... Specifically, PjBL serves as a mediator, creating a more effective learning environment aligned with ILOs, thus contributing to the development of graduate competence. (3) In response to Q3, the results revealed that well-designed ASs had a significant positive and direct effect on AGC, as supported by previous studies [23,108]. Clear descriptions and assessment criteria help students understand what competencies they will attain, how to perform an activity, and how they will be assessed, boosting students' comfort and confidence. ...
Article
Full-text available
Addressing the skill gap between labor market requirements and graduate readiness is crucial for the sustainable development of China’s vocational education system. This study investigated how outcome-based education (OBE) influences the attainment of graduate competence in China’s higher vocational education system, using the theory of Constructive Alignment (CA) as its foundation. The OBE model incorporates intended learning outcomes, project-based learning, and assessment strategies to ensure graduate competence aligns with professional sustainability practices. This study assessed the impact of intended learning outcomes, project-based learning, and assessment strategies on graduate competence attainment through surveys administered to 320 Cross-border E-commerce learners in April 2024, resulting in 301 usable responses. Data were analyzed using structural equation modeling (SEM) with SPSS 23.0 and AMOS 24.0. The results indicated that project-based learning and assessment strategies were directly and positively related to graduate competency attainment, while intended learning outcomes were indirectly associated with graduate competence through the mediating roles of project-based learning and assessment strategies. Assessment strategies had the most significant mediating effect, followed by project-based learning and the combined mediating effect. These findings advance the theoretical understanding of OBE and provide methods for promoting sustainability in vocational education. Beneficiaries include educators, policymakers, and accreditation bodies, who can use these insights to implement sustainable educational practices and ensure graduates contribute to sustainable development.
... Studies indicate that many educators face deficiencies in data analysis skills, and struggle to apply the findings of analysis effectively in their teaching practice. Additionally, teachers often encounter difficulties in setting clear objectives, collecting relevant data, and devising interventions aimed at achieving those objectives (Mandinach, Gummer, 2016). ...
Article
This study analyses the charging role of data literacy for general education teachers in the digital age, and highlights the need for educators to be attentive to the feedback data generated by various educational platforms and the emergence of data literacy as a crucial competency. Recognising the importance of data literacy in a variety of domains, it highlights the challenges that teachers face in using data for effective teaching, as teachers play a crucial role in data-driven learning, and make informed pedagogical decisions based on data interpretation and analysis. The study aims to analyse the theoretical foundations, highlighting the implications of data literacy for teaching practice and the quality of education. It concludes that effective and equitable education in the digital era requires the inclusion of teachers’ data literacy skills, and the promotion of a broader public understanding of the interaction between data, education and student outcomes.
... The result revealed that the design of ASs had a significant positive and direct effect on AGC. This finding was supported by previous studies of Mandinach and Gummer (2016) and Alonzo et al. (2023), which highlighted that well-designed ASs could motivate individual students' achievement. When students start an assessment task, the clear description and assessment criteria help them understand what competencies they will attain, how to perform the activity, and how they will be assessed. ...
Preprint
Full-text available
Exploring new approaches to innovating higher vocational education is a worldwide pursuit driven by the increasing demand for graduates equipped with competencies. This study investigated how Outcomes-Based Education (OBE) influences the attainment of graduate competence (AGC) in China’s higher vocational education, using the theory of Constructive Alignment (CA) as its foundation. The OBE framework incorporates intended learning outcomes (ILOs), project-based learning (PjBL), assessment strategies (ASs), and AGC. The study tested the model by assessing the impact of ILOs, PjBL, and ASs on AGC, respectively and sequentially. Surveys were administered to 320 CBEC learners in April 2024, resulting in 301 usable responses. Data were analyzed using structural equation modeling (SEM) with SPSS 23.0 and AMOS 24.0. Results indicated that PjBL and ASs were directly and positively related to graduate competency attainment, while ILOs were indirectly associated with graduate competence through the mediating roles of PjBL and ASs. The most substantial mediating effect between ILOs and graduate competence was ASs, followed by PjBL, and then, the chain mediating effect of PjBL and ASs. These findings enhance the theoretical understanding of OBE and provide detailed procedures for tracking and enhancing sustainability in higher vocational education. Beneficiaries include educators, policymakers, and accreditation bodies.
... This can lead to inconsistencies in data collection, analysis, and reporting, making it diVicult to compare and reproduce results. Another gap is the need for more training and education in data handling skills, particularly for non-technical professionals who may not have a background in statistics or programming (Mandinach & Gummer, 2016). ...
Chapter
Full-text available
The this for episode of the Art of Ignorance I am exploring the following topics: What is research and why should one be writing research? 3 The different research paradigms/philosophies. 6 What is Quantitative Reseach. 10 What is Qualitative Research 13 The qualitative research methods. 14 The skills needed in data handling. 18 Statistical treatments for quantitative research design. 19 Statistical treatments for qualitative research design. 20 References: 21
... Accordingly, early educators' data literacy is a critical aspect of educators' assessment practice. Data literacy includes (a) the recursive practice of identifying specific goals, (b) gathering multiple forms of data to inform the goal, (c) translating data into actionable instructional decisions, and (d) monitoring the influence of the instructional decision on the goal (Mandinach & Gummer, 2016;Mandinach & Jimerson, 2016). Although teachers' assessment literacy expands over time as they integrate new assessment tools and curricular materials, respond to the individual characteristics of the young children and families they serve, and advocate for specific instructional opportunities and resources for young learners, beginning teachers need to enter the field with practical assessment knowledge and skill to support young learners (Love et al., 2019;Mandinach & Jimerson, 2016;Xu & Brown, 2016). ...
Article
Early childhood teacher candidates (TCs) need preparatory experiences using authentic assessments to inform their instructional practices with rural learners and their families. The shift to virtual field placements in response to COVID-19 restrictions pushed faculty in teacher preparation programs to reimagine how to engage TCs in meaningful experiences using authentic assessments with children and families. This study explores how TCs experienced authentic assessment practices using an online assessment management system, Assessment, Evaluation, and Programming System Interactive (AEPS i), during virtual field experiences. Fifty-five undergraduate and graduate TCs from 11 universities completed a 45-item survey about their experience. Teacher candidates articulated knowledge and skills aligned with best practices underpinning authentic assessments. The virtual learning environment created opportunities for faculty to reimagine practices for promoting TCs’ understandings of assessment in early childhood special education contexts. We discuss implications for training TCs in rural areas to use authentic assessment.
... However, while there is some beginning recognition of the role of discipline in shaping data literacy (Mandinach and Gummer, 2016;Pothier, 2019;Ridsdale et al, 2015), educational approaches remain predominantly a 'one size fits all approach'. Providing the same education for different industries and possibly for different roles will not be useful. ...
Article
In the era of digital transformation, data literacy has emerged as a critical competency for organisations, driving a demand for skilled professionals. Despite a shortage of data-literate talent, universities struggle to align their curricula with industry needs, prompting a call for improved data literacy education. Recognising the contextual nuances of this skill set, a one-size-fits-all approach falls short. To address this gap, the authors advocate for a comprehensive exploration of perspectives from key stakeholders such as business advisors, students, teachers, and researchers. Understanding diverse needs and expectations of stakeholders is crucial in identifying deficiencies in data literacy education, paving the way for enhancements in university programmes. The reported study is the preliminary phase of a larger ongoing project in which grounded theory methodology is used to explore the question of ‘how can data literacy education be improved?’. The authors report on a small-scale study (eight interviews) aimed at exploring the perspectives on data literacy definition, competencies, and challenges with two representatives of each of four groups of stakeholders in data literacy education: students, business advisors, educators, and researchers. One common challenge identified among participants is the lack of data literacy and critical thinking skills, as well as a lack of awareness of the importance of data analysis. Although participants were aware that different businesses may need different data literacy skills, they were not able to articulate what those differences might be. The study underscores the need for the development of frameworks to help guide and advance data literacy education.
... It was the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data to help determine instructional steps. [8] Based on the above analysis, this study believed that teacher data literacy was a necessary condition to realize the development from original data to teaching knowledge and then to teaching practice. By ethically accessing, collecting, analyzing, understanding, communicating and using data, we could solve teaching problems in inquiry, creatively establish the connection between research and practice [9], achieve scientific decision-making and promote the development of education. ...
Article
Full-text available
The application of educational big data provides new basis for teachers to improve teaching and promote student development. However, there are still many challenges in using data to solve complex educational problems. Teacher data literacy has become the key to transforming educational data into effective teaching strategies. The study sought to gain a better understanding of the current state of primary and secondary school Teachers self-efficacy of data literacy and put forward some suggestions for teacher data literacy training. This study used a quantitative methodology. 813 primary and secondary school teachers from Guangdong Province, China, completed a questionnaire on self-efficacy of data literacy. (1) Primary and secondary school teachers reported significantly high levels of self-efficacy in data literacy (mean=3.624). Among the dimensions, the data awareness and ethics dimension had the highest self-efficacy (mean=3.738) and the data research dimension had a lower self-efficacy (mean=3.358). (2)There were significant differences in the primary and secondary school teachers’ self-efficacy of data literacy with different ages, educational backgrounds, and school locations. Primary and secondary school teachers generally recognized the importance of data literacy training. Older teachers who worked in rural schools and had low levels of education have the lowest level of self-efficacy for data literacy. For primary and secondary school teachers' low level of data analysis skills and weak data transformation strategy ability, teacher data literacy training adopts an embedded teamwork approach, integrating multiple resources to build a data team, integrating big data technology throughout the entire teaching process, and helping teachers to use new data and technologies to understand students' needs in a timely manner, to select teaching methods, and to solve real-world problems. It makes data literacy the foundation of teachers' teaching and promotes the development of all students.
... Es bedarf ebenfalls b) der Analyse weiterer bedeutsamer Bedingungen der Datennutzung, wie z. B. Unterstützungssystemen und der Data Literacy (Mandinach & Gummer, 2016) von Schulleitungen und Lehrpersonen. Zudem fehlen Längsschnittanalysen zur c) Untersuchung des Zusammenhangs zwischen Datennutzungspraxis sowie organisationalen Merkmalen der Schulen und (Entwicklung der) Unterrichtsqualität sowie SchülerInnenleistung. Ein weiteres bedeutsames Forschungsdesiderat betrifft d) die Möglichkeiten der Integration verschiedener Datenbestände, da sich zeigt, dass zur Ableitung von Entwicklungsaktivitäten mehrere Datenquellen simultan genutzt werden (Wurster, Richter & Lenski, 2017;Wurster & Richter, 2016). ...
Article
Ergebnisse aus Vergleichsarbeiten, zentralen Abschlussprüfungen und internen Evaluationen stehen Lehrkräften als ein Ausgangspunkt zur Unterrichtsentwicklung zur Verfügung. Die Verwendung dieser Daten sowie dafür förderliche und hinderliche Bedingungen wurden auf Basis des IQB-Ländervergleichs 2012 (N = 3099 Lehrkräfte) vergleichend untersucht. Aspekte der Nutzung der drei Verfahren werden im Mittel moderat positiv bewertet und deren Ergebnisse intensiv rezipiert sowie von einem substanziellen Teil der Lehrkräfte zur datengestützten Unterrichtsentwicklung genutzt. Die bedeutsamsten Prädiktoren für die Ergebnisnutzung sind die eingeschätzte Nützlichkeit der Verfahren und der wahrgenommene Veränderungsbedarf sowie teilweise die Besprechung der Ergebnisse im Kollegium.
... The explosion of accessible data, both generalized "big data" and specific "small data," presents tremendous potential for data-informed decision making to enhance outcomes (Marsh & Farrell, 2015). However, substantial research-practice gaps persist, with scarce models successfully leveraging school-level data to drive relevant innovation (Mandinach & Gummer, 2016). Developing frameworks effectively translating data analytics into actions remains critical for progressive education transformation. ...
Article
Full-text available
This paper proposes a continuous, linear data-driven innovation (DDI) model for education transformation grounded in evidence-based tools and data analysis. It advocates prioritizing "big data" and "small data" to inform incremental innovative changes in pedagogical practices. A detailed example of applying this DDI model in a Malaysian primary school is provided, including using exam results and quality standards to select a target school and diagnostic tests to identify issues. An intervention is designed utilizing multiplication tables and evaluated through public exam performance. Subsequent teacher training and student programs emerge. The model catalyzes ongoing localized innovations. RESUMO Este artigo propõe um modelo contínuo e linear de inovação baseada em dados (DDI) para a transformação da educação, fundamentado em ferramentas baseadas em evidências e análise de dados. Defende a priorização de "big data" e "small data" para informar melhorias inovadoras incrementais nas práticas pedagógicas. Fornece um exemplo detalhado da aplicação deste modelo DDI em uma escola primária malaia, incluindo o uso de resultados de exames e padrões de qualidade para selecionar uma escola-alvo e testes de diagnóstico para identificar problemas. Uma intervenção é projetada utilizando tabelas de multiplicação e avaliada por meio do desempenho em exames públicos. Posteriormente, emergem programas de treinamento para professores e alunos. O modelo catalisa inovações localizadas contínuas. Palavras-chave: inovação baseada em dados, transformação da educação, pedagogia, big data, small data.
Article
Full-text available
This research aims to identify the types of special needs that benefit from Data-Based Decision Making (DBDM), the stages of its implementation, and the challenges teachers face in executing DBDM effectively. This research used a qualitative approach, with a systematic literature review method using the PRISMA (Preferred Reporting Items for Systematic Review and Meta-analysis), systematically analyzing relevant articles from the Sage Journal and Wiley Online database scopus indexed published between 2013-2023. A total of 54 articles were initially identified, and through a thorough screening process, 5 articles were included for in-depth review. The data analysis technique for this research used a content analysis approach. The implementation of Data-Based Decision Making (DBDM) supports special needs students facing academic (e.g., reading and writing difficulties) as well as emotional or behavioral challenges. While Curriculum-Based Measurement (CBM) and Mastery Measures with clear decision rules are often used for academic difficulties, DBDM for behavioral issues is more complex due to the diversity of behaviors and required tools. Teachers are encouraged to apply DBDM, provided they develop skills in assessment selection, data processing, and analysis to adjust interventions effectively. Successful DBDM requires strong support from various school stakeholders. The review highlights the need for specialized training for teachers to enhance their competence in applying DBDM for diverse special needs students.
Chapter
This chapter examines the instrumental role effective leadership plays in enhancing student success in higher education. As colleges and universities face mounting pressures to demonstrate outcomes around retention, progression, attainment, career readiness and more, leadership has become mission critical to driving systemic reforms that enable these student achievement aims. The chapter provides higher education administrators, teams and policymakers research-backed principles to translate leadership vision, strategy and capabilities into tangible initiatives and cultural change that help all students thrive.
Article
Full-text available
El objetivo general de la investigación que se informa en este artículo fue conocer las percepciones y prácticas de la jefatura de la Unidad Técnico Pedagógica, del director(a) y de los docentes del primer ciclo de educación básica, asociadas al uso del Diagnóstico Integral de Aprendizajes (DIA) en el área académica. Utilizando un diseño mixto explicativo secuencial, primero se aplicó una encuesta en 20 centros escolares, con la participación de 80 docentes enseñando en el primer ciclo y 40 docentes directivos. En la segunda etapa se realizaron entrevistas en profundidad y observaciones en tres centros escolares seleccionados sobre la base de los resultados de la fase cuantitativa. Los principales resultados muestran un alto nivel de aplicación de las pruebas DIA en el área académica y una alta valoración de la información que estas entregan. Esta prueba se usa para identificar a los y las estudiantes que requieren más apoyo y para planificar el currículo. Aun cuando se observa alto nivel de heterogeneidad en cómo se aborda el trabajo con el DIA, un aspecto generalizado es la ausencia de protocolos para el uso reflexivo de datos. Un factor facilitador del uso de los resultados del DIA es el trabajo colaborativo entre docentes del ciclo y con docentes del Programa de Integración Escolar (PIE), junto a lineamientos que entrega la dirección del establecimiento. El profesorado mayoritariamente señala que no cuenta con suficientes recursos para atender a los y las estudiantes que requieren apoyo adicional, con tiempo para el análisis reflexivo de datos, tampoco se ofrece una formación en alfabetización para el uso de datos. Aunque el tamaño de la muestra es reducido, los resultados entregan pistas sobre posibles modificaciones al DIA para un mejor alineamiento entre las pruebas y el currículo enseñado. Además, se detecta la necesidad de generar oportunidades de aprendizaje profesional para fortalecer la alfabetización para el uso reflexivo de datos.
Article
Many educational systems implemented standardized tests to monitor educational quality and provide schools with performance feedback. However, one of the key characteristics that influence if and how users use such feedback is its perceived relevance. Adding to the current state of the art, this study brings in a situational perspective in the perceived relevance of performance indicators. By means of 19 in-depth interviews, this study provides insights into which feedback indicators are perceived as relevant or irrelevant and why they are. The results show that feedback relevance is not only determined by performance feedback as such, but also by the user that makes sense of it and the context in which users operate. Therefore, feedback designers face the challenge of integrating these different perspectives to facilitate actual feedback use and school improvement.
Book
Full-text available
Il volume esplora la contemporanea intersezione tra i dati e l’educazione, un territorio in cui i big data non solo influenzano, ma trasformano attivamente le modalità di insegnamento e apprendimento. L’opera si apre con un’analisi del ruolo dei big data nella società dell’informazione e dell’influenza che essi esercitano sulle nostre vite individuali e sociali, sollevando questioni di ordine cognitivo, etico e sociale: in che modo sta cambiando il nostro modo di concepire i dati? Quali sono le implicazioni etiche legate alla raccolta e all’utilizzo dei dati in ambito educativo? Quale impatto l’uso dei big data nell’istruzione può avere sull’accesso equo alle opportunità educative? Tali questioni richiedono nuove forme di consapevolezza che vanno promosse tra i cittadini del nuovo millennio per favorire lo sviluppo di data literacy. Il libro offre una panoramica sui significati del concetto di data literacy, soffermandosi sui suoi fondamenti teorici e fornendo un quadro complessivo di cosa significhi essere alfabetizzati ai dati nel XXI secolo...
Article
Presenting a comprehensive knowledge map tailored for researchers within the domain of data literacy, facilitating swift comprehension of pivotal concepts, evolving trends, and emerging research frontiers. Employing advanced bibliometric analysis in conjunction with cutting-edge software tools including CiteSpace, Ucinet, and gCLUTO, the study meticulously examines a corpus of 1250 scholarly documents sourced from both the Web of Science (WoS) and Scopus datasets spanning the period from 2004 to 2023. The resultant scientific knowledge map illuminates the landscape of data literacy research, synthesizing a thorough and methodical literature review that seamlessly integrates qualitative and quantitative methodologies. Findings notably indicate a conspicuous uptick in scholarly interest surrounding data literacy post-2015, with a quantified exponential growth pattern depicted by the function y = 0.8777e 0.3384x . Key research focal points in data literacy foreground domains such as data management and utilization, decision-driven research in data literacy, pedagogical strategies for cultivating data literacy in educators and students, critical data literacy considerations, artificial intelligence, media literacy, among others. Future research trajectories appear poised toward advancing both the training methodologies and practical applications of media data literacy and critical data literacy within the context of information proliferation, particularly emphasizing educational settings.
Chapter
Full-text available
This chapter examines the connections between leadership strategies and enhanced student achievement at higher education institutions. As pressures grow around outcomes, affordability and inclusion, effective leadership has become key to accelerating reforms focused on student success, including increased retention, progression, degree completion and career readiness. Theories explored include transformational, distributed and culturally responsive models. Competencies for contemporary contexts cover cultural competence, change leadership, talent development and data literacy. Examples of successful initiatives promoted include predictive analytics, transition programs, pathway redesigns, early intervention systems and student-centric cultures. Vision, community-building, resource optimization and change management provide frameworks to facilitate progress despite complexity. Findings synthesize scholarship and practice into recommendations translating leadership capabilities to institutional environments enabling all students to thrive.
Research Proposal
Full-text available
In dit project onderzocht Arteveldehogeschool samen met een medewerker van Smartschool Analytics waar er aarzeling zit. Welke randvoorwaarden bevorderen of belemmeren het werken met schooldata? Als we weten welke competenties en omkadering schoolteams nodig hebben om schooldata nuttig in te zetten, kunnen we gericht het datageïnformeerd werken in scholen verhogen en schoolteams ondersteunen in het gebruiken van data om o.a. het leerproces van leerlingen te optimaliseren.
Article
The goal of the European Educability project was to create a specific skills program for educators and information professionals covering six key multiliteracies within the new European framework of digital competencies. The focus of the project is placed on the importance of international collaboration in the development of these programs, using one specific literacy as an example: data literacy. This paper provides the working methodology for the project, based on prior definition of the conceptual elements and, subsequently, on the design of a Delphi study in which 15 experts from the partner countries evaluated and reached a consensus on the importance of the learning objectives and training programs presented. The results for data literacy highlight the need to apply replicable methodologies when designing skills programs. The international cooperation work carried out within the framework of the Educability project contributes to standardization in the approach to these programs, as well as in research on multiliteracies.
Article
Full-text available
School climate surveys are frequently used to collect information about student experiences in school. Less is known about how educators use survey data after survey administration. This paper explores one school district’s critical use of evidence to promote equitable change. We conducted eight semi-structured interviews with district and school leaders to investigate their uses of evidence. Through our qualitative, reflexive thematic analysis, we generated six themes: (1) using evidence to provide a common language; (2) bringing attention to trends to shift staff understanding of problems, (3) making structural changes, (4) planning for professional learning, (5) following up directly with students, and (6) engaging with the community. Findings illustrate how education leaders can apply a critical lens to their generation and use of evidence. We explore how the strategic use of evidence is needed to advance the broader goal of fostering school change and improving school climate for all students.
Article
Full-text available
Primary school teachers face a great degree of heterogeneity in their daily teaching. Identifying the individual learning needs of their pupils and finding suitable solutions can be challenging for them. Digital learning platforms can offer support here: If digital learning platforms are used for practicing in class, teachers have a wide range of data about their students’ learning process at their disposal. With the information that teachers gain from the data, they can derive individualization and differentiation measures. Research on data-based decision making shows that data can be used for decisions in the school and teaching context. There are already approaches for modeling the use of data for decision-making in a circular and process-based manner. For the German-speaking context, however, the processbased use of digital data for instructional design is still widely unexplored. This paper introduces an exploratory study that contributes insights into how teachers use data from digital learning platforms to inform their teaching and how this use can be described along the circle of data-based decision making. To answer the research question, a qualitative research design with a method triangulation of interviews, thinking aloud and observations was used. The results show that the use of data from digital learning platforms by primary school teachers can be described along the five steps of data-based decision-making, but that there are inter-individual differences between the teachers. For example, some teachers only take a look at the data provided by the learning platform in a dashboard without using it further. Other teachers also interpret the data and derive implications for their further lesson planning. The findings from the interviews, the thinking aloud method and the lesson observations are combined, discussed and implications for educational practice and research are derived.
Article
The purpose of this study is to develop a machine learning-based AI convergence class model and class design principles that can foster data literacy in high school students, and to develop detailed guidelines accordingly. We developed a machine learning-based teaching model, design principles, and detailed guidelines through research on prior literature, and applied them to 15 students at a specialized high school in Seoul. As a result of the study, students' data literacy improved statistically significantly (p<.001), so we confirmed that the model of this study has a positive effect on improving learners' data literacy, and it is expected that it will lead to related research in the future.
Article
Full-text available
Despite receiving increased attention from researchers in mathematics education, there is still no comprehensive understanding of the current level of data literacy in the teaching and learning of mathematics. To address this gap, this study pre­sents a review of 247 papers selected from the Sco­pus database between 2009 and 2024. The research aims to explore the following: (i) The overall vol­ume, geographic distribution, and development tra­jectory in the literature on data literacy in mathe­matics. (ii) The researchers and research collabora­tions that have had the greatest influence on the literature on data literacy in mathematics. (iii) The sources that have had the greatest influence on the literature on data literacy in mathematics. (iv) The most important topics in the literature on data literacy in mathematics. It was discovered that the number of publications involving data literacy in mathematics increased from 2016 to 2023. Authors from the Netherlands are the most active in the literature on data literacy in mathematics. The Teacher College Record had the highest number of citations. Lastly, the most important topics addressed in the literature on data literacy in mathematics were data use, data literacy, and data-based decision-making. This study has implications not only for mathematics education researchers but also for other stakeholders in the education sector, including school principals, policymakers, and mathematics teachers.
Article
This study was carried out to visualise the interconnectedness among research works in the field of Data Literacy documents, Sources and Authors, Identify clusters, nodes and key connections to understand the structural patterns within Data Literacy scholarly literature and determine the most influential and highly-cited papers, authors and journals within the bibliographic coupling networks. We specified the syntax ‘DATA LITERACY’. While it is obvious that there may be studies that have addressed this subject using other syntax, ‘Data Literacy’ will yield an appropriately representative sample of keywords in the area. The findings offer valuable insights into the landscape of research on data literacy, particularly in terms of the interconnections and influences among various scholarly works. Several discussions can be carried out based on these findings. The analysis identifies key documents that have significantly contributed to the development of research on data literacy. The analysis of publication years of identified documents offers insights into the temporal evolution of research on data literacy. This helps reveal shifts in research emphasis over time and highlights emerging areas of interest within the field.
Article
Full-text available
Vanaf 2024 zullen centrale toetsen voor Nederlands en wiskunde geïntroduceerd worden in het basis en secundair onderwijs in Vlaanderen. Het voornaamste doel van deze toetsen is om schoolontwikkeling te ondersteunen en de onderwijskwaliteit te verbeteren. De toetsen kunnen dit doel pas bereiken wanneer onderwijsprofessionals betekenisvolle feedback ontvangen en de vertaalslag wordt gemaakt van feedback naar concrete acties om de klas- en schoolpraktijk te verbeteren. Het Steunpunt Centrale Toetsen in Onderwijs zet daarom ten eerste in op het ontwikkelen van een gebruiksvriendelijk feedbackdashboard waarop de onderwijsprofessionals de feedback vlot kunnen raadplegen en ten tweede op het ontwikkelen van een e-course over datageletterdheid en datagebruik. Via educational design research wordt onderzocht hoe het feedbackdashboard en de e-course volgens experten, literatuur en onderwijsprofessionals moet worden vormgegeven. Om de onderzoeksvragen te beantwoorden werden interviews en focusgroepen afgenomen bij experten (n=25), schoolleiders (n=23), leraren (n=19), lerarenopleiders (n=4) en pedagogisch begeleiders (n=24). Uit de resultaten worden vijf designprincipes gedestilleerd die als basis dienen om het feedbackdashboard en de e-course te ontwikkelen, namelijk (1) de inhoud moet als doordacht, relevant, rijk, en betrouwbaar gepercipieerd worden, (2) de manier waarop de feedback weergegeven wordt faciliteert een accurate interpretatie van de feedback die daarna aan de hand van een e-course verder ondersteund wordt, (3) het feedbackdashboard en de e-course bieden onderwijsprofessionals niet enkel data maar ook handvaten voor doelbewust datagebruik in de praktijk, (4) de opbouw is duidelijk en gebruiksvriendelijk en (5) de noden van de onderwijsprofessionals staan centraal en primeren over technologische aspecten.
Article
Full-text available
Background : To assess feasibility of sparing the neural stem cell compartment (NSC), hippocampus, and limbic circuit during partial brain radiotherapy (PBRT) for pediatric intracranial tumors. Methods : Treatment plans were generated for the following pediatric intracranial tumors: low and high grade gliomas, low grade brainstem glioma, optic nerve glioma, hypothalamic glioma, localized ependymoma, skull base sarcoma, central nervous system (CNS) germinoma (involved field radiotherapy [IFRT] and whole ventricular radiotherapy [WVRT]), and craniopharyngioma. For each pathology, standard intensity-modulated radiotherapy (IMRT) plans were generated using helical tomotherapy, as well as IMRT plans which spared limbic circuit, hippocampus, and NSC. Biologically equivalent dose for late effects (BED late effects) was generated for limbic circuit, hippocampus, and NSC. Percent reduction in mean, maximum, and minimum physical dose and BED was calculated between plans. Results : We reduced mean physical dose and BED late effects to these critical structures by 44% and 47.9% respectively (range 5.4-78.8% and 7-80.3%). Greatest benefits in relative dose reduction were seen in high grade hemispheric glioma cases; least relative dose reduction was seen in WVRT cases. Dosimetric coverage of treatment target (PTV) was equivalent in all cases as assessed by D95 and V100 metrics. Integral dose to uninvolved brain was reduced by mean of 7.6% (range-19.3% to +0.3%) in sparing plans. Discussion and Conclusions : It is possible to spare limbic circuit, NSC, and hippocampus during PBRT for primary pediatric intracranial tumors using helical tomotherapy. This approach reduces integral dose delivered to uninvolved normal brain and may reduce late cognitive sequelae of cranial radiotherapy.
Article
Full-text available
The expectation that teachers will use student achievement data to improve their instruction is a major feature of national and local reform agendas. The theory of action behind data-driven decision making is a mostly causal model of professional action, whereby teachers diagnose weaknesses and implement solutions. The purpose of this article is to examine how high school teachers, situated within their policy and work contexts, use data to inform instructional decisions. Using a framework that draws upon sense-making and co-construction theories on reform implementation, we analyze qualitative data gathered in 4 urban public high schools in the United States. Findings reveal that the process of data use by teachers is complex, multilayered, and influenced by teacher interpretations and social interactions. Teachers used a variety of forms of data to inform their decision making and struggled to reconcile policies promoting data-driven decision making with local beliefs and practices. Implications for research and policy are discussed.
Article
Full-text available
Schools are increasingly confronted with the challenges that information about school performance brings with it. It is common for schools' use of performance feedback to be limited. Equally, however, there are documented cases in which school performance feedback is meaningfully used. Purpose: This study looks at how Flemish primary school teachers use school performance feedback and to what extent this use (or lack of use) is determined by school characteristics. Based on evidence from existing research, we focus on four school-related explanatory variables: attitude with regard to school performance feedback, the organisational functioning of the school, performance-orientation and actual pupil performances. The research questions addressed are: (1) 'To what extent do teachers use school performance feedback?' and (2) 'To what extent can the use of school performance feedback by teachers be explained by school characteristics?' Sample: The use of school performance feedback was studied in the context of a Flemish school feedback initiative; 183 primary schools were given school performance feedback at school and pupil level for mathematics, technical reading and spelling, supplemented by data with regard to pupil characteristics. A survey was conducted in this representative sample of Flemish primary schools. In each school, all teachers were asked to complete a survey on their use of the school performance feedback and on their perception of the mentioned school characteristics. The questionnaire was filled out by 2578 respondents from 183 schools. Respondents were regular teachers or teachers that were occupied as pupil welfare co-ordinators. Design and methods: The survey results were analysed statistically. In addition to descriptive analyses, multi-level analyses were carried out to explain variation in the process and the results of school performance feedback use. The school characteristics described in the theoretical framework were included as explaining factors in both models, supplemented by the background variables on the teachers surveyed. Results: Only a limited number of respondents stated that the available school performance feedback had made an actual contribution towards promoting critical reflection with regard to school functioning and/or their own classroom practice. The analyses confirm that the way in which school performance feedback use is approached by teachers is not independent of characteristics of the school. There is a relationship between the process and result of school feedback use and the role of the school principal as culture builder and with the professional relationships between team members. Conclusion: By and large, respondents reported no or only limited results of school performance feedback use. There are, however, appreciable differences between team members within schools. We conclude that the way in which school performance feedback use is implemented by teachers cannot be seen in isolation from the characteristics of the school. Given the research findings, principals with a strong focus on culture building and strong professional relationships between team members offer a slightly better chance of getting teachers to use school performance feedback in a worthwhile and productive way.
Article
Full-text available
The current high-stakes accountability environment has created strong incentives for educators to systematically collect and use data to inform instructional decisions. This article examines the strategies employed by three urban school districts to promote data use for instructional improvement and their effect on administrator, principal, and teacher practice. Several factors are found to affect data use, including accessibility and timeliness of data, perceptions of data validity, training, and support for teachers with regard to data analysis and interpretation, and the alignment of data strategies with other instructional initiatives.
Article
Full-text available
Data-driven decision making has become an essential component of educational practice across all levels, from chief state school officers to classroom teachers, and has received unprecedented attention in terms of policy and financial support. It was included as one of the four pillars in the American Recovery and Reinvestment Act (2009), indicating that federal education officials seek to ensure that data and evidence are used to inform policy and practice. This article describes the emergence of data-driven decision making as a topic of interest, some of the challenges to and opportunities for data use, and how the principles of educational psychology can and must be used to inform how educators are using data and the examination of its impact on educational practice.
Article
Background Data-based decision making can lead to increased student achievement; however, schools struggle with the implementation of data-based decision making. Professional development in the use of data is therefore urgently needed. However, professional development is often ineffective in terms of improving the knowledge, skills, and attitude of the receiver. Purpose We need a more fundamental understanding of how we can increase the effectiveness of data-use-related professional development. This study therefore focuses on the factors influencing a professional development intervention for data-based decision making: the data team procedure. Data teams are teams of teachers and school leaders who collaboratively learn how to use data, following a structured approach and guided by a facilitator from the university. Based on an extensive literature review, we developed a data use framework in which the use of data is influenced by data characteristics, school organization characteristics, and user and team characteristics. Research Design We conducted case studies. Data Collection We focused on observing in depth the factors that influence the work of the data teams and interviewing the data team members about these factors. Four data teams of six schools for upper secondary education were followed over a period of 2 years. We observed and analyzed 34 meetings and analyzed 23 interviews, combined with our field notes. Although this pilot study only permits analytical generalization of the findings, the findings provide more in-depth insight into the factors that enable and hinder interventions, focusing on supporting collaborative data use in schools. Findings The results show that several data characteristics (access and availability of high-quality data), school organizational characteristics (a shared goal, leadership, training and support, involvement of relevant stakeholders), and individual and team characteristics (data literacy, pedagogical content knowledge [PCK], organizational knowledge, attitude, and collaboration) influence the use of data in data teams. The results also show that these influencing factors are interrelated. Conclusions Schools need support in all aspects of the use of data (from formulation of a problem definition to taking action based on the data). This study can form a starting point for larger studies into the factors influencing these types of professional development interventions to ensure effective implementation and sustainability.
Book
Assessment in Science combines professional development and classroom practice in a single volume. The pragmatic nature of the book makes it a valuable resource for administrators and staff developers interested in designing professional development programs, and for science teachers looking for techniques and examples of classroom-based assessments. Unique features of Assessment in Science include: 1) practical strategies and tools for implementing successful professional development programs in science assessment, 2) teacher stories and case studies about classroom-based assessment practice and how these teachers changed their assessment practice, 3) examples of classroom-based assessments and scoring guides, 4) samples of student work with teacher commentary, and 5) examples of how the national reform documents in science education served as tools in professional development programs and in designing classroom-based assessments. Assessment in Science expands the existing literature on science assessment by sharing a model for professional development, and examples of teacher-developed assessments with accompanying student work and teacher commentary. Chapters written by science teachers tell how they assess students and how they have changed their assessment practice, as well as how changing assessment practice has resulted in a change in their science instruction. Assessment in Science is targeted at practising professionals in science education: administrators, staff developers, science teachers, and university science educators. Assessment in Science has applicability to graduate-level courses in science education and in-service courses for science teachers. The teacher chapters are also appropriate for use in undergraduate science methods courses to illustrate classroom-based assessments.
Book
In a context where schools are held more and more accountable for the education they provide, data-based decision making has become increasingly important. This book brings together scholars from several countries to examine data-based decision making. Data-based decision making in this book refers to making decisions based on a broad range of evidence, such as scores on students’ assessments, classroom observations etc. This book supports policy-makers, people working with schools, researchers and school leaders and teachers in the use of data, by bringing together the current research conducted on data use across multiple countries into a single volume. Some of these studies are ‘best practice’ studies, where effective data use has led to improvements in student learning. Others provide insight into challenges in both policy and practice environments. Each of them draws on research and literature in the field.
Article
This paper describes the data use professional development (PD) component of a whole-school intervention that has been replicated in 53 schools over eight years. Quasi-experimental designs were used to test for intervention impact. The intervention improved achievement in reading comprehension, writing and high school qualifications. Effect sizes were generally higher than international comparisons. The data use PD involved collaboratively analyzing data to determine the achievement problems; identifying and testing the causes of the problems using theory evaluation principles; and co-creating solutions. The relative contribution of the data use PD to the intervention and the importance of content knowledge are discussed.
Chapter
The purpose of this book is to present ideas, raise issues, share tools and techniques, and stimulate thinking about assessment in science classrooms. With the aim of enhancing the professional development of teachers in science assessment, the book includes examples of professional development strategies and classroom assessments. The book is pragmatic in that it presents practical tasks, professional development models and activities, and tools and templates for planning and conducting assessment in science. Although the book does not describe or debate assessment policy or large-scale assessments, it does build on the national reform documents in science education: the [NRC] and the [AA AS].
Chapter
This chapter emphasizes two aspects of changing classroom assessment practice: the change process itself and impediments to change. The first portion of this chapter discusses the theoretical aspects of the difficulties teachers encounter as they reflect on and expand their assessment practice. The second portion of the chapter provides insight into the teacher impediments, internal and external, perceived or actual, for changing classroom assessment practice. These impediments are presented as issues to be considered in the professional development of science teachers’ assessment practices.
Chapter
In this chapter, the results of all the studies presented in this book are summarized. What are the lessons learned? Based on the lessons learned, we developed a data use framework. In this framework, data use is influenced by several enablers and barriers (e.g., the school organization context, data and data systems, and user characteristics). Data can be used in a desirable as well as undesirable manner, but what happens a lot in schools as well is that data are not used. Policy may also influence the use of data, as well as its enablers and barriers. Finally, we argue that if data are used in a desirable manner this can lead to school leader, teacher, and student learning (e.g., increased student achievement).
Article
School leaders and teachers are increasingly required to use data as the basis for their decisions. But what does using data for decision-making mean? What counts as “data”? In this chapter, the authors address what is meant by the word “data” and what kinds of data are available and needed. The latter should overlap, but sometimes the available data are not needed and sometimes needed data are not available. In this chapter, we also discuss why teachers and school leaders should use data. Finally, the process of using data and the different ways data can and should be used is described.
Article
Background/Context In recent years, states, districts, schools, and external partners have recognized the need to proactively foster the use of data to guide educational decision-making and practice. Understanding that data alone will not guarantee use, individuals at all levels have invested in interventions to support better access to, interpretation of, and responses to data of all kinds. Despite the emergence of these efforts, there has been little systematic examination of research on such efforts. Purpose/Objective/Research Question/Focus of Study This article synthesizes what we currently know about interventions to support educators’ use of data—ranging from comprehensive, system-level initiatives, such as reforms sponsored by districts or intermediary organizations, to more narrowly focused interventions, such as a workshop. The article summarizes what is what is known across studies about the design and implementation of these interventions, their effects at the individual and organizational levels, and the conditions shown to affect implementation and outcomes. Research Design Literature review. Data Collection and Analysis This review entailed systematic searches of electronic databases and careful sorting to yield a total of 41 books, peer-reviewed journal articles, and reports. Summaries of each publication were coded to identify the study methods (design, framework, sample, time frame, data collection), intervention design (level of schooling, focal data and data user, leverage points, components), and findings on implementation, effects, and conditions. Findings/Results The review uncovers a host of common themes regarding implementation, including promising practices (e.g., making data “usable” and “safe,” targeting multiple leverage points) and persistent challenges (e.g., developing support that is generic but also customized, sustaining sufficient support). The review also finds mixed findings and levels of research evidence on effects of interventions, with relatively more evidence on effects on educators’ knowledge, skills, and practice than on effects on organizations and student achievement. The article also identifies a set of common conditions found to influence intervention implementation and effects, including intervention characteristics (capacity, data properties), broader context (leadership, organizational structure), and individual relationships and characteristics (trust, beliefs and knowledge). Conclusions/Recommendations The review finds that the current research base is limited in quantity and quality. It suggests the need for more methodologically rigorous research and greater attention to the organizational and student-level outcomes of interventions, comparative analyses, interventions that help educators move from knowledge to action, and specific ways in which the quality of data and leadership practices shape the effectiveness of interventions.
Article
Background With the growing emphasis for educators to use data to inform their practice, little has been done to consider the means by which the educators can acquire the requisite data literacy skills. This article provides a context for why schools of education can and must play an important role in preparing teachers to use data. Purpose This article sought to understand if and how schools of education are preparing teacher candidates to use data effectively or responsibly. The study examined the extent to which schools of education teach stand-alone courses on data-driven decision making or integrate data use concepts into existing courses. It also examined state licensure and certification requirements to determine if and how data use is included in documentation. Population A stratified randomized sample of schools of education was drawn with 208 institutions responding, representing a 25.7% response rate. Research Design The survey portion of the study consisted of a stratified randomized sample of all schools or departments of education in the United States. The syllabus review was a voluntary part of the survey. The licensure review was a descriptive analysis of every state's documentation for teacher licensure and certification. Findings/Results The survey results indicated that a vast majority of the schools of education reported that they offered a stand-alone data course, and even more integrated data use into existing courses. The syllabus review provided a deeper dive into the course offerings and indicated that the courses were more about assessment literacy than data literacy. The licensure review yielded a plethora of skills and knowledge related to data that are included in state requirements. However, there was wide variation across states in their requirements. Conclusions Even though schools of education reported that they are teaching about data-driven decision making in their teacher preparation programs, the results indicate that the content is more about assessment literacy than data literacy. This finding is consistent with the often observed conflation of the two constructs. Licensure requirements include both data literacy and assessment literacy, but the emphasis is more on assessment than data. With the increasing emphasis by policy makers on the importance of educators using data, it is essential that schools of education begin to incorporate data concepts into their curricula and that states make explicit the data-related skills and knowledge required for teachers for licensure and certification.
Article
Background The increasing focus on education as an evidence-based practice requires that educators can effectively use data to inform their practice. At the level of classroom instructional decision making, the nature of the specific knowledge and skills teachers need to use data effectively is complex and not well characterized. Being able to characterize this requisite knowledge and skills supports definition and measurement of data literacy. Evolving from empirical analyses, an emergent conceptual framework of knowledge and skills is proposed for the construct, data literacy for teaching. The framework is based on a domain analysis, which is the first step of an evidence-centered design process for data literacy. The framework is contextualized in existing research, with an objective of having it ground future work in the development of instruments to measure data literacy. Purpose This article reports on work to develop a conceptual framework to undergird research, development, and capacity building around data literacy for teaching. The emergent nature of the framework is intended to inform the discussions around data literacy so that continued refinement of operational definitions of the construct will emerge. Without such operational definitions, measurement of progress toward teacher data literacy is not possible. Research Design The conceptual framework is based on a sequence of qualitative studies that sought to determine the nature of knowledge and skills that are required for teachers to be considered data literate. A first study examined the ways that the knowledge and skills around the use of data were characterized in practical guides, books, and manuals on data use, formative assessment, and related topics. These characteristics were integrated with definitions of data literacy elicited from experts. A second study examined the licensure and certification documents required by states for teacher candidates for their treatment of data- and assessment-related knowledge and skills. The synthesis of these studies and their components have yielded an evolving conceptual framework for a new construct: data literacy for teachers. Conclusions The conceptual framework described in this article reflects an evolving effort to understand what it means for teachers to be data literate—that is, what knowledge and skills are required for teachers to use data effectively and responsibly set within an iterative inquiry cycle. The work posits that the construct comprises three interacting domains (data use for teaching, content knowledge, and pedagogical content knowledge), six components of the inquiry cycle (identify problems, frame questions, use data, transform data into information, transform information into a decision, and evaluate outcomes), and, finally, 59 elements of knowledge and skills embedded within those components. However, the complex construct requires additional discussion from policy, research, and practitioners to refine and reorganize it and to expand it beyond a cognitive focus on knowledge and skills to include beliefs/values, identity, and epistemic elements. Next steps will include structuring an ongoing discussion about the nature of the framework and expansion beyond domain analysis through the evidence-centered design process to the development of a suite of instruments to measure the construct.
Article
Background: Data-based decision making can lead to increased student achievement; however, schools struggle with the implementation of data-based decision making. Professional development in the use of data is therefore urgently needed. However, professional development is often ineffective in terms of improving the knowledge, skills, and attitude of the receiver. Purpose: We need a more fundamental understanding of how we can increase the effectiveness of data-use-related professional development. This study therefore focuses on the factors influencing a professional development intervention for data-based decision making: the data team procedure. Data teams are teams of teachers and school leaders who collaboratively learn how to use data, following a structured approach and guided by a facilitator from the university. Based on an extensive literature review, we developed a data use framework in which the use of data is influenced by data characteristics, school organization characteristics, and user and team characteristics. Research Design: We conducted case studies. Data Collection: We focused on observing in depth the factors that influence the work of the data teams and interviewing the data team members about these factors. Four data teams of six schools for upper secondary education were followed over a period of 2 years. We observed and analyzed 34 meetings and analyzed 23 interviews, combined with our field notes. Although this pilot study only permits analytical generalization of the findings, the findings provide more in-depth insight into the factors that enable and hinder interventions, focusing on supporting collaborative data use in schools. Findings: The results show that several data characteristics (access and availability of highquality data), school organizational characteristics (a shared goal, leadership, training and support, involvement of relevant stakeholders), and individual and team characteristics (data literacy, pedagogical content knowledge [PCK], organizational knowledge, attitude, and collaboration) influence the use of data in data teams. The results also show that these influencing factors are interrelated. Conclusions: Schools need support in all aspects of the use of data (from formulation of a problem definition to taking action based on the data). This study can form a starting point for larger studies into the factors influencing these types of professional development interventions to ensure effective implementation and sustainability.
Article
Data use should be a continuous, integrated part of practice, a tool that is used all the time. Good teachers have been doing data-driven decision making all along, it just has not been recognized by that term. But there is more work to be done to ensure that educators know how to continuously, effectively, and ethically use data; that is, to help them to be data literate. Several federal laws protect student data -- most notably FERPA; teachers and other school officials should have a working knowledge of them.
Book
Next Generation Science Standards identifies the science all K-12 students should know. These new standards are based on the National Research Council's A Framework for K-12 Science Education. The National Research Council, the National Science Teachers Association, the American Association for the Advancement of Science, and Achieve have partnered to create standards through a collaborative state-led process. The standards are rich in content and practice and arranged in a coherent manner across disciplines and grades to provide all students an internationally benchmarked science education. The print version of Next Generation Science Standards complements the nextgenscience.org website and: Provides an authoritative offline reference to the standards when creating lesson plans. Arranged by grade level and by core discipline, making information quick and easy to find. Printed in full color with a lay-flat spiral binding. Allows for bookmarking, highlighting, and annotating.
Article
Universal teacher residency would benefit the teaching profession and ultimately the education of our children. We have yet to work out the fine details, but there is nothing more important than developing robust residency schools where young educators go between their undergraduate preparation and their arrival in the classroom as autonomous practitioners. The change won’t happen overnight, but eventually it will redefine the profession.
Article
This work systematically reviews teacher assessment literacy measures within the context of contemporary teacher evaluation policy. In this study, the researchers collected objective tests of assessment knowledge, teacher self-reports, and rubrics to evaluate teachers’ work in assessment literacy studies from 1991 to 2012. Then they evaluated the psychometric work from these measures against a set of claims related to score interpretation and use. Across the 36 measures reviewed, they found support for these claims was weak. This outcome highlights the need for increased work on assessment literacy measures in the educational measurement field. The authors conclude with recommendations and a resource to inform a research agenda focused on assessment literacy measurement to inform policy and practice.
Article
This study aimed at understanding the development of mental models for data use among educators in a small school district located in Texas. Drawing from survey and interview data, the study was guided by three questions: (1) How do educators conceptualize “data” in relation to “evidence” or “information”?; (2) How do teachers and school leaders construe “data” or “data use”?; and (3) What factors affect mental models for data use? Findings indicated that educators approached decision-making from a range of mental models for data use, and that models seemed rooted in ways of thinking about “data” and “data use” that were influenced by formal training, modeling by leaders, social interaction with colleagues, and personal experience.
Article
Data-driven decision making has become increasingly important in education. Policymakers require educators to use data to inform practice. Although the policy emphasis is growing, what has not increased is attention to building human capacity around data use. Educators need to gain data literacy skills to inform practice. Although some professional development opportunities exist for current educators, fewer formal courses and opportunities for data literacy development in schools of education have been developed and implemented. This article explores issues around the growing need for data-driven decision making in programs in schools of education. The issues are complex and the actors needed to bring about change are multiple. A systems perspective to explore course and programmatic implementation is presented.
Article
This two-year project, funded by the Joyce Foundation, was a collaboration of the Wisconsin Center for Education Research (WCER) with UCLA's National Center for Research on Evaluation, Standards, and Student Testing (CRESST) and the Milwaukee Public Schools (MPS). The project was designed to increase the capacity of six MPS schools to use student, classroom, and school data more effectively for decision-making, continuous improvement, and school reform. In addition, WCER staff collaborated with CRESST in providing external feedback on the implementation of the Quality School Portfolio (QSP) software, shared in designing an evaluation, and engaged in joint problem solving and reflection. In working with the six Milwaukee urban schools over the past two years, we have learned that to be effective, data must become an active part of school planning and improvement processes, and it must become infused and accepted into the school culture and organization. Additionally, school staff members must develop the analytical capacity to understand and apply data strategically. Once fully integrated into a school's systems, data can be transformed from mere numbers to useful information, which can then contribute to the staff's knowledge in effective and meaningful ways. The application of data to decision-making presents an array of complex challenges for schools. These challenges must both be addressed initially and attended to continuously if a school is to make successful and effective use of its data. We have identified six challenges schools will need to confront as they build their capacity for using data for decision-making: 1) cultivating the desire to transform data into knowledge; 2) focusing on a process for planned data use; 3) committing to the acquisition and creation of data; 4) organizing data management; 5) developing analytical capacity; and, 6) strategically applying information and results.
Article
Lee S. Shulman builds his foundation for teaching reform on an idea of teaching that emphasizes comprehension and reasoning, transformation and reflection. "This emphasis is justified," he writes, "by the resoluteness with which research and policy have so blatantly ignored those aspects of teaching in the past." To articulate and justify this conception, Shulman responds to four questions: What are the sources of the knowledge base for teaching? In what terms can these sources be conceptualized? What are the processes of pedagogical reasoning and action? and What are the implications for teaching policy and educational reform? The answers — informed by philosophy, psychology, and a growing body of casework based on young and experienced practitioners — go far beyond current reform assumptions and initiatives. The outcome for educational practitioners, scholars, and policymakers is a major redirection in how teaching is to be understood and teachers are to be trained and evaluated. This article was selected for the November 1986 special issue on "Teachers, Teaching, and Teacher Education," but appears here because of the exigencies of publishing.
Article
Teachers spend approximately one third of their time in assessment activities, yet most states do not require a course in tests and measurement for certification. Three hundred ninety-seven teachers completed a survey of their formal measurement training, their beliefs about the ade quacy and importance of their training, influences on their measurement knowledge, and their perceived abilities in measurement. Forty-seven percent of the teachers reported that their mea surement training was somewhat or very inadequate. Most reported that they learned about testing and measurement by trial and error in their classes. Teachers who had taken less than one course in tests and measurement during their undergraduate course work were less likely to acquire measurement skills through graduate courses or inservice work than teachers who took one or more courses. Regardless of the amount of their measurement training, teachers reported that measurement skills were very important and that they considered their own abilities in measurement to be high.
Article
Using evidence from a national survey and earlier literature, Schafer and Lissitz note that the time spent by teachers in assessment activities is not reflected in their evaluation skills nor in their prep aration programs. Prospective teachers receive limited preparation in assess ment procedures despite the existence of well-developed recommendations from professional groups vis-a-vis as sessment objectives for preprofessional training. The authors suggest that cur ricular change be evidenced in teacher preparation programs to ensure that teachers have adequate assessment skills.
Article
Discusses a study of six schools using data-based inquiry and decision-making process to improve instruction. Findings identified two conditions to support successful implementation of the process: administrative support, especially in providing teachers learning time, and teacher leadership to encourage and support colleagues to own the process. (Contains three references.) (PKP)
Article
Ensuring that teachers are rich in data, rich in information, and rich in the skills that enable them to improve student achievement requires focused attention from leaders at all levels, including federal policymakers. For federal policy to best support teachers' use of data to prepare all students for college and careers, there must be a recognition of the challenges for teachers associated with data use, an understanding about which data are more and less helpful to teachers and why, and consensus on what supports and structures need to be in place at the school, district, state, and federal levels to ensure effective data use by educators. This policy brief addresses why using data represents a significant shift for most teachers in how they perform their jobs, explains the importance of using multiple types of data to affect learning, details the infrastructure necessary to encourage teachers' use of data, and provides federal policy recommendations. (Contains 23 endnotes.)
Article
Three national educational organizations articulated seven areas of teacher competency in student assessment. Among 555 teachers surveyed nationwide, the area of best performance was "administering, scoring, and interpreting test results," and the area of worst performance was "communicating test results." Development of inservice training materials is discussed. Includes standards. (SV)
Article
One of the central lessons from research on data use in schools and school districts is that assessments, student tests, and other forms of data are only as good as how they are used. But what influences how they are used? This relatively straightforward question turns out to be fairly complex to answer. Data use implicates a number of processes, conditions, and contexts. It involves interpretive processes, as using data requires that the user interpret the data and construct implications for next steps. It implicates social and organizational conditions, since the data use unfolds in the context of a multileveled organizational system. And, because data can be a source of power, particularly in the current accountability environment, data use also involves power relations. In this article, we put forward a framework for understanding the phenomenon of data use in the context of data use interventions. We draw on existing research and theory to identify key dimensions of data use that we should attend to and offer a way to understand how these dimensions might interact. In so doing, we provide guidance for studying the pathways between data use interventions and various outcomes of value.
Article
Schools around the world are using instruments for performance feedback, but there is no scientific evidence that they have positive effects on education. This paper compares a School Performance Feedback System (SPFS) used in the USA as an accountability instrument to an SPFS used in The Netherlands. The study employs a unique database: one in which 2 separate countries with 2 distinct performance systems are compared using the same instruments. The use and effects of both SPFSs are compared to acquire more knowledge about the utilization and effects of SPFSs in an international context. Also, the variables which influence SPFSs are presented and then utilized to predict the use of the 2 SPFSs in their 2 separate contexts.
Article
The 1990 Standards for Teacher Competence in Educational Assessment of Students (AFT, NCME, & NEA, 1990) made a documentable contribution to the field. However, the Standards have become a bit dated, most notably in two ways: (1) the Standards do not consider current conceptions of formative assessment knowledge and skills, and (2) the Standards do not consider teacher knowledge and skills required to successfully work in the current accountability and “standards-based reform” context. This article briefly reviews the 1990 Standards and their influence, describes some other lists of assessment knowledge and skills that might be considered in updating them, and then proposes educational assessment knowledge and skills for teachers that reflect current teacher assessment needs. This set of competencies should help focus the work of teachers, teacher supervisors, professional developers, teacher educators, and others responsible for teachers’ assessment knowledge and skills.
Article
Schools face a lot of data on the functioning of their school which they can use to make improvements in teaching, learning and the organization. For data use to lead to improvement, it is important to further research the concept data-driven decision making. The results of this explorative study in the Netherlands show that teachers mainly use classroom level data for making instructional decisions at classroom level, and school leaders mainly use school level data for policy development decisions. This article ends with suggestions with regard to enhancing the effectiveness of data-driven decision making, for example by stressing the importance of developing teachers' competence in the use of data.
Chapter
Assessment has a profound impact on the learning that occurs in classrooms and standardized student achievement testing has considerable impact on what goes on in schools (Stake and Theo bold, 1991). Wilson & Corbett (1991) report that increasing the consequences of assessments at the state level results in refocusing educational efforts away from improving curriculum and instruction to improving test scores by emphasizing basic content and skills. According to Ferrara, Willihoft, Seburn, Slaughter and Stevenson (1991), another effect of state assessment systems is the development of local assessment systems that are parallel to that of the state. They describe several benefits and drawbacks to these local assessment systems that use the state assessment frameworks as organizing principles. Benefits include teachers’ and administrators’ perceived belief in assistance with targeting instruction for particular students and groups, in improving classroom assessment practices and in decreasing anxiety about the targets of the tests. However, these benefits come at some cost including a decreased focus on learning objectives outside those of basic skills, loss of instructional time to administration of more assessments, and inappropriate classroom teaching and testing practices. The effect of assessment as testing on the practices of classroom teachers is difficult to measure though O’Sullivan (1991) provides evidence that teachers identify increasingly negative effects as the stakes of the testing increase.
A data coach's guide to improving learning for all students: Unleashing the power of collaborative inquiry
  • N Love
  • K E Stiles
  • S Mundry
  • K Diranna
Love, N., Stiles, K. E., Mundry, S., & DiRanna, K. (2008). A data coach's guide to improving learning for all students: Unleashing the power of collaborative inquiry. Thousand Oaks, CA: Corwin Press.
2013 edTPA field test: Summary report
  • Stanford Center
  • Assessment
  • Equity Learning
Stanford Center for Assessment, Learning, and Equity (SCALE). (2013). 2013 edTPA field test: Summary report. Stanford, CA: Author.
Case studies of schools of education: What we can learn about improving data literacy among educators?
  • Mandinach
Mandinach, E. B., & Friedman, J. M. (2015, April). Case studies of schools of education: What we can learn about improving data literacy among educators?. In Paper presented at the annual meeting of the american educational research association, Chicago, IL.
Data-driven school improvement: Linking data and learning
  • E B Mandinach
  • M Honey
Mandinach, E. B., & Honey, M. (2008). Data-driven school improvement: Linking data and learning. New York: Teachers College Press.
Remarks made at the teachers' use of data to impact teaching and learning conference
  • J Aguerrebere
Aguerrebere, J. (2009, August). Remarks made at the teachers' use of data to impact teaching and learning conference. Washington, DC.