Content uploaded by James P. Spillane
Author content
All content in this area was uploaded by James P. Spillane
Content may be subject to copyright.
A preview of the PDF is not available
Data in Practice: Conceptualizing the Data-Based Decision-Making Phenomena
Abstract
"Data use" and "data-based decision making" are increasingly popular mantras in public policy discourses and texts. Policy makers place tremendous faith in the power of data to transform practice, but the fate of policy makers' efforts will depend in great measure on the very practice they hope to move. In most conversations about data use, however, relations between data and practice have been under conceptualized. In this essay, I identify and discuss some conceptual and analytical tools for studying data in practice by drawing on work from various theoretical traditions. I explore some ways in which we might frame a research agenda in order to investigate data in everyday practice in schools. My account is centered on schoolhouse work practice, but the research apparatus I consider can be applied to practice in other organizations in the education sector and indeed to interorganizational practice, a critical consideration in the education sector.
... The assumption is that teachers and principals will use school performance feedback (SPF), for instance from a standardized assessment, as a mirror to identify strengths and weaknesses, and take action accordingly. In practice, however, distribution of test scores and assessment feedback may bring about no effects at all (Hopster-den Otter, Wools, Eggen, & Veldkamp, 2017;Vanhoof, Verhaeghe, Verhaeghe, Valcke, & Van Petegem, 2011;Verhaeghe, Schildkamp, Luyten, & Valcke, 2015) or result in unintended effects (Spillane, 2012;Visscher & Coe, 2003). Misuse, underuse and unintended uses of SPF sometimes stem from recipients' issues with accurately comprehending the data provided. ...
... Sensemaking describes how people make meaning of something new and/or unexpected by figuring out how it fits in with what they already know and assume (Klein, Phillips, Rall, & Peluso, 2007;Maitlis & Christianson, 2014;Weick, 1995). This entails noticing and bracketing certain elements (Coburn & Turner, 2011;Maitlis & Christianson, 2014;Starbuck & Milliken, 1988;Weick, 1995) and weighing them up to personally and/or organizationally held knowledge and beliefs (Klein et al., 2007;Spillane, 2012). If 'conceptions' are the nodes of knowledge that make up the frames people use to make sense of (new) information, 'misconceptions' can be interpreted as the incorrect assumptions and convictions that seep into these frames and lead to (systematic and persistent) errors (Prinz, Golke, & Wittwer, 2021;Smith, diSessa, & Roschelle, 1994). ...
... In any case, as we argued, in order to promote effective data-based decision making, it is necessary to further investigate data use in practice (Coburn & Turner, 2011;Spillane, 2012). Sensemaking is an act of processing reality, therefore we need to take a closer look at how it takes shape in reality. ...
School performance feedback can be a tool for school improvement. However, when educational professionals do not comprehend the data they are provided with, they will not arrive at valid inferences and correct diagnoses. We interviewed 23 Flemish primary school teachers and principals, asking them to explain authentic feedback from a national assessment. Framework analysis of think-aloud data reveals that participants’ comprehension of typical concepts is clouded by a range of misconceptions. We observed that that visual, verbal and mathematical building blocks in the report can become stumbling blocks. Moreover, misconceptions can be attributed to a certain extent to disconnects between feedback providers’ and feedback users’ frames of reference. These findings have important implications for data providers, considering they have a responsibility to cater to the interpretability of the data they provide.
... 101-102), and is the action itself. The third approach is crucial when studying sustainability, because the variation and interplay between the two aspects can provide insights into what makes initiatives sustainable (Feldman & Pentland, 2003;Spillane, 2012). Studies seem to rarely "specif[y] the concept to the specific routine" (Wolthuis et al., 2021, p. 15). ...
... Leadership practice is constructed in the interaction of leaders, people in their surroundings, and their situation, in the execution of particular leadership tasks (Spillane et al., 2004). It is not restricted to those with formal leadership positions but influence and agency are shared with informal leaders (Spillane, 2012;Woods & Roberts, 2016). Formal leaders are those with "a formally designated leadership position" (Pitts & Spillane, 2009, p. 3), and informal leaders are those who "exert influence over other group members, but do not receive special compensation or rewards for this role" (Pescosolido, 2001, p. 78). ...
... (p. 96) Organizational routines comprise the repeated tasks (i.e., work) that actors perform within an organization and can be a source of stability or change (Feldman & Pentland, 2003) and enable us to understand the work of education and educators in practice as it plays out Spillane, 2012;Stelitano et al., 2020;Tubin, 2015). Organizational routines direct our attention to the content of the routines as well as how they are enacted in practice to understand how educators' work unfolds and changes (Wolthius et al., 2022). ...
This paper investigated principal interns’ leadership work during the COVID-19 pandemic in Texas. We used data from one-on-one semistructured interviews to document the experiences of 21 principal candidates in spring 2021 through spring 2022. Participants’ leadership work changed in four ways during the pandemic: striving, intensifying, expanding, and limiting. We also found that the changes that participants reported varied across the three semesters we studied. Our findings highlight how the pandemic, in some cases, the pandemic created new opportunities for interns, while in other cases, it led to an intensification or limiting of work that narrowed interns’ leadership work.
... Following this milestone in data governance research, other researchers including Spillane (2012) have extended data governance frameworks with scalability as a core theme thus defining core mechanisms (i.e., structural, procedural, relational). These frameworks then extend to specialized capabilities for cloud data storage derived from NIST principles Al-Ruithe ( 2016). ...
Data is considered one of the assets that organizations have to protect through a good corporate governance framework. Organizations must ensure that procedures and practices are in place to assess, direct, monitor, and protect data and related infrastructure, for the value data provides to a corporation and thereby its stakeholders. The objective of a good data governance framework has to be in line with the corporate governance objectives of maintaining and strengthening its contribution to market integrity and the firm’s economic performance. Moreover, data governance influences firms' performance by influencing the quality of data used for financial reporting and corporate takeover markets as an internal governance mechanism
... Following this milestone in data governance research, other researchers including Spillane (2012) have extended data governance frameworks with scalability as a core theme thus defining core mechanisms (i.e., structural, procedural, relational). These frameworks then extend to specialized capabilities for cloud data storage derived from NIST principles Al-Ruithe ( 2016). ...
Data is considered one of the assets that organizations have to protect through a good corporate governance framework. Organizations must ensure that procedures and practices are in place to assess, direct, monitor, and protect data and related infrastructure, for the value data provides to a corporation and thereby its stakeholders. The objective of a good data governance framework has to be in line with the corporate governance objectives of maintaining and strengthening its contribution to market integrity and the firm's economic performance. Moreover, data governance influences firms' performance by influencing the quality of data used for financial reporting and corporate takeover markets as internal governance mechanisms. This paper is part of the doctoral thesis on "Data Governance Influence on Corporate Governance and Firms' Performance", from the Swiss School of Business and Management Geneva.
... With the repeated appeal for the democratic purpose of education in Norwegian education policy, it is a question of whether school leaders are faced with increased competing aims, since local performance monitoring in many municipalities and national regulation remain strong means of governing (Gunnulfsen & Larsen, 2021). Studies have shown that aspects of education that cannot be measured receive less attention (Spillane, 2012). In dealing with conflicting ideals in policy, school leaders will, as usual, have to use their professional discretion to handle daily tasks and challenges and strengthen their argumentation for school priorities. ...
The purpose of this paper is to examine and discuss the role of Nordic school principals in a cross-national context and to highlight the implications for the leadership roles in research and practice. The data builds on the country-wide reports in this book, the findings in the TALIS report (2018), and the perspectives of pedagogical leadership. The ways in which school leadership roles are perceived and dealt with are relatively similar across national school contexts. Nordic school principals seem to experience their practice with shared responsibilities, providing opportunities and educational engagement by acting as pedagogical leaders with a collective approach to their leadership role, even though there are differences in Nordic education policy contexts. Pedagogical leadership in a Nordic context means leadership tasks primarily related to curriculum, teaching activities, and the core values of schooling. Power, trust, control in the form of governance and leadership in the Nordic countries needs high attention.KeywordsPedagogical leadershipTALISShared responsibilityNordic education policyTrustControl
... With the repeated appeal for the democratic purpose of education in Norwegian education policy, it is a question of whether school leaders are faced with increased competing aims, since local performance monitoring in many municipalities and national regulation remain strong means of governing (Gunnulfsen & Larsen, 2021). Studies have shown that aspects of education that cannot be measured receive less attention (Spillane, 2012). In dealing with conflicting ideals in policy, school leaders will, as usual, have to use their professional discretion to handle daily tasks and challenges and strengthen their argumentation for school priorities. ...
This chapter discusses the role and practices of Norwegian school principals in a policy context of increased monitoring and regulation and in a context of dealing with value dilemmas related to upholding a focus on democracy and inclusion. Research has shown that Norwegian school principals work under a loyalty dilemma with increasing pressure to improve student outcomes, which does not always align with other pressing local priorities or the diverse needs of students and student welfare. The chapter (first/initially) describes how the Norwegian education system is built on democratic practices and inclusion and then presents the school principal’s role and position, responsibilities and governance expectations. Moreover, the chapter builds on a scoping review of research on the school principal’s role and practices in a changing policy context. The country report is based on national policy documents and findings from current Norwegian and international research concerning leadership and policy in schools. In the concluding remarks, we state that there is a large room for manoeuvre and discretion, which means opportunities to determine how policy initiatives should be dealt with in the school context. The school principal’s role is closely connected to social influence and relational bargaining as a strategy to promote social justice and student well-being, as well as the school’s academic results. The school principal’s discretionary role in promoting social justice and relational bargaining contributes to the knowledge relevant to the notion of a Nordic model of school leadership.
... Research sug gests that the value of these data is depen dent on their use (Coburn & Turner, 2011), and policymakers have touted data-informed deci sion-mak ing as a means for improv ing school-and stu dent-out comes (Goldring & Berends, 2009). Education research ers point to data use as crit i cal for under stand ing the link between edu ca tional data and school improve ment (Coburn & Turner, 2012;Grissom et al., 2017;Marsh, 2012;Spillane, 2012). ...
Accountability pressures in education have risen steadily over the last two decades and public schools and districts now track school- and student-level data in response to state and federal mandates. Catholic schools and dioceses have not faced the same level of regulation over this period, and less is known about data access and use in the Catholic sector. This descriptive and exploratory research draws on survey and interview data from a national sample of Catholic school superintendents to examine data practices in diocesan central offices as well as barriers faced in the use of data. Findings suggest that although considerable variation exists among dioceses, common data efforts include working toward data centralization, developing tools to monitor the operational health of schools, and finding ways to foster the data culture within dioceses. These practices were present in close to half of all dioceses, but several common challenges related to governance, resources, and data systems hindered progress in and toward these efforts.
Developing learning analytics dashboards (LADs) is a growing research interest as online learning tools have become more accessible in K-12 and higher education settings. This paper reports our multimodal classroom engagement data analysis and dashboard design process and the resulting engagement dashboard. Our work stems from the importance of monitoring classroom engagement, which refers to students' active physical and cognitive involvement in learning that influences their motivation and success in a given course. To monitor this vital facade of learning, we developed an engagement dashboard using an iterative and user-centered process. We first created a multimodal machine learning model that utilizes face and pose features obtained from recent deep learning models. Then, we created a dashboard where users can view their engagement over time and discover their learning/teaching patterns. Finally, we conducted user studies with undergraduate and graduate-level participants to obtain feedback on our dashboard design. Our paper makes three contributions by (1) presenting a student-centric, open-source dashboard, (2) demonstrating a baseline architecture for engagement analysis using our open-access data, and (3) presenting user insights and design takeaways to inspire future LADs. We expect our research to guide the development of tools for novice teacher education, student self-evaluation, and engagement evaluation in crowded classrooms.
Background/Context
Though change is constant in organizations, determining how to successfully implement planned change has been a perennial challenge for both organizational scholars and practitioners. While the empirical knowledge base on planned change in schools and other organizations offers numerous insights, the inattention to activity, or the practice of leading and managing change, remains. Organizational change theory, and specifically organizational routines, offers a useful lens with which to examine planned change in work practice in schools.
Purpose
The purpose of this study is to understand the role organizational routines play in changing school work practice.
Research Design
We employed a longitudinal case study of leadership practice at Adams, a K–8 urban school, over the course of four years. We spent 90 days collecting observations, interviews, and artifacts from a wide range of individuals (e.g., teachers and administrators). We identified one key routine, the Five Week Assessment Routine, and analyzed the ostensive and performative aspects of the routine, identifying similarities and differences in the routine over time. In addition, we identified two different kinds of change, planned and unplanned, as well as the reasons why participants changed the routine.
Conclusions
Organizational routines structure work practice, stabilizing it over time, even in the face of considerable change. They represent one mechanism for sustaining leadership. In addition, routines can be a source of both episodic and continuous change in the work place. New routines can serve as a mechanism to build instructional coherence, internal accountability, and professional community. This account suggests that one way to change norms and culture in an organization is through the design and implementation of new routines. Leaders can create opportunities for change in school practice. By designing and supporting an organizational routine, the leaders at Adams focused the practice of the faculty on improving teaching and learning in ways they had never done before. The case of Adams demonstrates that in the enactment of a routine, school staff can also create change in practice. How they make sense of the routine, and integrate it into their practice, can create shifts in teaching and learning. Change happens in the interplay between individual agency and the structure of the routine. Through a case study of one organizational routine in one school, we analyzed how organizational routines can enable both change and constancy in school practice.
This book brings together contributions from researchers within various social science disciplines who seek to redefine the methods and topics that constitute the study of work. They investigate work activity in ways that do not reduce it to a 'psychology' of individual cognition nor to a 'sociology' of societal structures and communication. A key theme in the material is the relationship between theory and practice. This is not an abstract problem of interest merely to social scientists. Rather, it is discussed as an issue that working people address when they attempt to understand a task and communicate its demands. Mindful practices and communicative interaction are examined as situated issues at work in the reproduction of communities of practice in a variety of settings including: courts of law, computer software design, the piloting of airliners, the coordination of air traffic control, and traffic management in underground railway systems.
The teaching and learning of mathematics in K-12 classrooms is changing. New curricula and methods engage learners in working on real problems. An essential feature of this work involves teacher and students in 'talking mathematics'. How can students learn to do this kind of talking? What can they learn from doing it? First published in 1998, this book addresses these questions by looking at the processes of formulating problems, interpreting contexts in which problems arise, and arguing about the reasonableness of proposed solutions. The studies in this volume seek to retain the complexity of classroom practice rather than looking at it through a particular academic lens.
Humanity and the very notion of the human subject are under threat from postmodernist thinking which has declared not only the 'Death of God' but also the 'Death of Man'. This book is a revindication of the concept of humanity, rejecting contemporary social theory that seeks to diminish human properties and powers. Archer argues that being human depends on an interaction with the real world in which practice takes primacy over language in the emergence of human self-consciousness, thought, emotionality and personal identity - all of which are prior to, and more basic than, our acquisition of a social identity. This original and provocative new book from leading social theorist Margaret S. Archer builds on the themes explored in her previous books Culture and Agency (CUP 1988) and Realist Social Theory (CUP 1995). It will be required reading for academics and students of social theory, cultural theory, political theory, philosophy and theology.
The situative perspective shifts the focus of analysis from individual behavior and cognition to larger systems that include behaving cognitive agents interacting with each other and with other subsystems in the environment. The first section presents a version of the situative perspective that draws on studies of social interaction, philosophical situation theory, and ecological psychology. Framing assumptions and concepts are proposed for a synthesis of the situative and cognitive theoretical perspectives, and a further situative synthesis is suggested that would draw on dynamic-systems theory. The second section discusses relations between the situative, cognitive, and behaviorist theoretical perspectives and principles of educational practice. The third section discusses an approach to research and social practice called interactive research and design, which fits with the situative perspective and provides a productive, albeit syncretic, combination of theory-oriented and instrumental functions of research. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
What is the shoulder-to-the grindstone work of transforming underperforming schools into higher performing schools? What makes this work so difficult? This book sheds light on these questions from the perspective of the Success for All Foundation (SFAF), an organization that has collaborated with thousands of elementary schools to enact a common strategy for comprehensive school reform, all in an effort to improve the reading achievement of millions of students. This story of SFAF spans twenty turbulent years. It begins in 1987, with the strategy of improving reading achievement by improving students' cooperative learning in classrooms. It stretches through 2008, with efforts to influence federal education policy to support that strategy. There is nothing in the story to suggest a quick fix. Rather, the theme that emerges is that the problems and possibilities of effective, large-scale, sustainable education reform lie in the complexity of public education: in interdependencies among underperforming schools, programs of reform, the organizations that advance those programs, and the environments in which all operate. The story ultimately locates the problems of education reform not in schools but, instead, in reformers, themselves. By tracing SFAF's deep push into public education, the purpose of the book is to assist a wide array of reformers in seeing, understanding, and ultimately confronting its complexity.