Figure 2 - uploaded by Thomas Dee
Content may be subject to copyright.
Focus School Status by Student Gap Group (SGG) Score, 2013-14. (a) Fuzzy Regression Discontinuity (RD). (b) Frontier RD.
Source publication
Under waivers to the No Child Left Behind Act, the federal government required states to identify schools where targeted subgroups of students have the lowest achievement and to implement reforms in these “Focus Schools.” In this study, we examine the Focus School reforms in the state of Kentucky. The reforms in this state are uniquely interesting...
Contexts in source publication
Context 1
... begin by illustrating, both graphically and parametrically, the relationship between a school's SGG score and its participation in Focus School reforms. In figure 2a, we show the change in probability of Focus School status for the SGG assignment variable for the Fuzzy RD sample. All schools with an SGG score in the bottom 10 percent are classified as Focus Schools, but because of the secondary assignment mechanism (i.e., TSD score) the probability drops approximately 70 percentage points to the right of the threshold where only 30 percent of schools are Focus Schools. ...
Context 2
... schools with an SGG score in the bottom 10 percent are classified as Focus Schools, but because of the secondary assignment mechanism (i.e., TSD score) the probability drops approximately 70 percentage points to the right of the threshold where only 30 percent of schools are Focus Schools. In figure 2b, the frontier RD changes in probability of treatment from one to nearly zero, a virtually sharp contrast. 26 In table 2 we show the first-stage RD estimates based on the SGG assignment variable for the fuzzy and frontier RD. ...
Context 3
... example, following McCrary (2008), we implement a density test to see whether the density of the SGG values jumps discontinuously at the threshold. We fail to reject the null hypotheses that there is no discontinuity at both thresholds for the Fuzzy and Frontier samples (see the online appendix figure A.2). Second, we also examine the density of the assignment variable visually because "heaping" at certain values may fail to be detected by the McCrary density test (Barreca, Lindo, and Waddell 2016). ...
Context 4
... begin by illustrating, both graphically and parametrically, the relationship between a school's SGG score and its participation in Focus School reforms. In figure 2a, we show the change in probability of Focus School status for the SGG assignment variable for the Fuzzy RD sample. All schools with an SGG score in the bottom 10 percent are classified as Focus Schools, but because of the secondary assignment mechanism (i.e., TSD score) the probability drops approximately 70 percentage points to the right of the threshold where only 30 percent of schools are Focus Schools. ...
Context 5
... schools with an SGG score in the bottom 10 percent are classified as Focus Schools, but because of the secondary assignment mechanism (i.e., TSD score) the probability drops approximately 70 percentage points to the right of the threshold where only 30 percent of schools are Focus Schools. In figure 2b, the frontier RD changes in probability of treatment from one to nearly zero, a virtually sharp contrast. 26 In table 2 we show the first-stage RD estimates based on the SGG assignment variable for the fuzzy and frontier RD. ...
Context 6
... example, following McCrary (2008), we implement a density test to see whether the density of the SGG values jumps discontinuously at the threshold. We fail to reject the null hypotheses that there is no discontinuity at both thresholds for the Fuzzy and Frontier samples (see the online appendix figure A.2). Second, we also examine the density of the assignment variable visually because "heaping" at certain values may fail to be detected by the McCrary density test (Barreca, Lindo, and Waddell 2016). ...
Citations
... More than 20 years later, there is evidence that some but not all of these fears were borne out, as predominantly economically disadvantaged and Black communities, respectively, were disproportionately subject to accountability-driven takeovers, educators reported demoralization arising from the failing label, and economically disadvantaged students in at least one state may have been pushed out of the school system or reclassified by districts seeking to improve their scores (Gregg & Lavertu, 2023;Kitzmiller, 2020;Lipman, 2017;Pearman & Marie Greene, 2022;Strunk et al., 2016). On the other hand, there is evidence that schools receiving low accountability marks received needed resources for improvement (Dee et al., 2013), and in many cases they experienced sometimes sizeable achievement gains (Bonilla & Dee, 2020;Carlson & Lavertu, 2018;Dee & Jacob, 2011;Sun et al., 2017;Sun et al., 2021). Students subject to low pre-NCLB school accountability grades even experienced longer-term benefits in the form of higher educational attainment and lower adult criminal involvement and reliance on social welfare programs (Eren et al., 2023;Mansfield & Slichter, 2021). ...
Many states report school performance grades as a way to inform the public about school quality. However, past research has shown that when these grades drew largely on proficiency-based measures, they served to capture variation in school and community demographics rather than school quality. We extend this literature by examining whether a multidimensional measure of school quality such as those required under the Every Student Succeeds Act is less confounded by out-of-school factors than the proficiency measures that characterized previous generations of accountability. Drawing on school accountability grades from Florida combined with school and community demographic data, we find that more than half the variation in multidimensional measures of school quality can be explained by observable school- and county-level factors outside the school’s locus of control. Together, our findings show that even school grades that draw on multiple measures misattribute the contribution of demographics and socioeconomics to school quality—but subcomponents based on learning gains perform better than those based on proficiency. We conclude with policy implications and recommend that states focus public reporting on school quality measures that driven less by out-of-school factors and more by the school’s true contribution to student outcomes.
... Turnarounds for failing schools are rare, and low performance often persists despite various interventions (Smarick, 2010). While there are some successful school improvement efforts (Bonilla & Dee, 2020;Carlson & Lavertu, 2018;Sun et al., 2017), successful leadership is a critical factor in such endeavors (Leithwood et al., 2019;Ylimaki & Brunderman, 2021;Zhu et al., 2020). While research shows that effective leadership has an even greater impact in struggling schools (Ylimaki & Brunderman, 2021), surprisingly few studies have examined the specific practices used by principals to improve low-performing schools. ...
Recent studies have increasingly highlighted the need for contextual leadership practices within educational settings, however, there remains a notable lack of clarity regarding the specific influence of these practices on crucial organizational outcomes. The present study seeks to address this gap by investigating the relationships between contextual leadership and three key school outcomes: organizational citizenship behavior, group identity, and autonomous motivation for teaching. Additionally, the study examines group cohesion as a potential mediator in these relationships. To achieve this, a cross-sectional survey involving 1617 teachers was conducted, and the data were analyzed using structural equation modeling. The findings indicate positive correlations between contextual leadership and the three targeted outcomes: Organizational citizenship behavior, group identity, and autonomous motivation for teaching. Moreover, the study confirms the mediating role of group cohesion in these relationships. The finding underscores the importance of fostering shared identity and purpose within educational institutions to enhance desired outcomes. This study's implications extend to theoretical and practical domains, offering valuable insights for scholars, practitioners, and policymakers. Ultimately, the study advocates for a strategic focus on cultivating a cohesive and purpose-driven community to optimize educational achievements and teacher engagement.
... Turnarounds for failing schools are rare, and low performance often persists despite various interventions (Smarick, 2010). While there are some successful school improvement efforts (Bonilla & Dee, 2020;Carlson & Lavertu, 2018;Sun et al., 2017), successful leadership is a critical factor in such endeavors (Leithwood et al., 2019;Ylimaki & Brunderman, 2021;Zhu et al., 2020). While research shows that effective leadership has an even greater impact in struggling schools (Ylimaki & Brunderman, 2021), surprisingly few studies have examined the specific practices used by principals to improve low-performing schools. ...
... These waiver-based reforms yielded few positive effects, although the reform models themselves and their implementation again varied widely across states. Of the states with evaluations of waiver reforms, one-Kentucky-produced positive effects on student achievement, which the authors attributed in part to clearly articulated state guidance for the comprehensive school improvement planning process (Bonilla & Dee, 2020). Waiver-based reforms in New York, Michigan, Rhode Island, and Louisiana produced either null or negative effects on student achievement (Atchison, 2020;Dee & Dizon-Ross, 2019;Dougherty & Weiner, 2017;Hemelt & Jacob, 2017. ...
... First, the quality of comprehensive plans can vary, and their content plays an important role in reform implementation (Anfara et al., 2006;Mintrop & MacLellan, 2002;Redding & Searby, 2020;Strunk, Marsh, Bush-Mecenas, & Duque, 2016). This is important because a comprehensive planning process that generates clearly articulated plans has the potential to induce meaningful improvements in low-performing schools and districts-while a pro forma approach may not (Bonilla & Dee, 2020;Huber & Conway, 2015;Player & Katz, 2016;Strunk, Marsh, Hashim, et al., 2016;Sun et al., 2019). ...
The recent Every Student Succeeds Act (ESSA) requires states to identify and turn around their lowest performing schools, but it breaks somewhat from prior policies by granting states significant autonomy over how they identify and turn around these schools. This mixed-methods study, which draws on administrative, qualitative, and survey data, examines the effectiveness of Michigan’s approach to school turnaround under ESSA. We find that students in turnaround schools experienced significant achievement gains in math and to a lesser extent in English language arts (ELA), with effects concentrated among the lowest achieving students. Analyses of qualitative and survey data suggest that these outcomes were influenced by state-level supports, strategic planning, the threat of accountability for continued low performance, and improved leadership quality in turnaround schools.
... Large-scale quantitative evaluations of school turnaround have found mixed results on student achievement. Besides work in Tennessee, rigorous evaluations of school turnaround have provided evidence of positive effects in California (Dee, 2012;Sun et al., 2017), Ohio (Carlson & Lavertu, 2018, Massachusetts (Papay et al., 2021;Schueler et al., 2017), and Kentucky (Bonilla & Dee, 2017) but other studies report null (Dragoset et al., 2017) or even negative effects (Dougherty & Weiner, 2017;Heissel & Ladd, 2017;Henry & Harbatkin, 2018). ...
Mixed results from evaluations of school reform suggest a need for evidence to explain why some models succeed while others fail. Addressing that need, this study uses structural equation modeling to estimate difference-in-differences models that examine mediating mechanisms for positive effects produced by Innovation Zone (iZone) reforms in Memphis, Tennessee. I find that iZone schools increased peer collaboration between teachers, which resulted in improved student achievement. Also, recruiting effective teachers led to a more positive learning environment and ultimately to improved student achievement. These results highlight peer collaboration, a positive learning environment, and the recruitment of effective educators as important practices that will likely facilitate improved school performance under future school reform plans.
... In contrast to these federally sanctioned approaches to turnaround, some states engaged in alternative efforts to improve their lowest performing schools under NCLB waivers (Bonilla & Dee, 2020;Dee & Dizon-Ross, 2017;Dougherty & Weiner, 2019). These state turnaround approaches can be characterized by the use of school accountability systems to identify the schools most in need of technical assistance (Dee & Dizon-Ross, 2017). ...
... These state turnaround approaches can be characterized by the use of school accountability systems to identify the schools most in need of technical assistance (Dee & Dizon-Ross, 2017). The types of supports offered to these schools varied across states, but include comprehensive school improvement planning and reforms aimed at improving school operations (Bonilla & Dee, 2020;Dougherty & Weiner, 2019). Comparing student outcomes between state turnaround efforts and other SIG-promoted turnaround models can demonstrate the extent to which the increased flexibility and reduced resources under ESSA are sufficient for improving the lowest performing schools in a state or if these schools require systematic changes in human capital or management/governance. ...
... Given that different factors shape the decision to close a charter school compared with a traditional public school, we also exclude studies examining the effect of charter school closure on student outcomes (e.g., Carlson & Lavertu, 2016;Chingos & West, 2015). Finally, it should be noted that our definition allows for turnaround to occur in the context of school improvement efforts conducted under NCLB waivers (Bonilla & Dee, 2020;Dee & Dizon-Ross, 2019;Dougherty & Weiner, 2019;Hemelt & Jacob, 2017) as well as state takeover of low-performing schools (Schueler et al., 2017;Zimmer et al., 2017). ...
School turnaround has emerged as a predominant strategy to improve chronically low-performing schools, although the approach remains controversial. This meta-analysis synthesizes results from 35 studies to examine the relationship between school turnaround and various student outcomes. We find that school turnaround is associated with improved attendance, standardized test scores, and graduation rates. When separating the results by the different turnaround models, transformation, turnaround, and restart models are associated with improvements in student test scores. We find no evidence of a significant relationship between school closure or state turnaround conducted under No Child Left Behind (NCLB) waivers and student test scores. We describe how changes in organizational operations, human capital, and the governance and/or management of low-performing schools might have contributed to this observed relationship.
... Many state education agencies worked with stakeholders including educator unions to prepare state ESSA plans. The contrast between implementation of NCLB and ESSA suggests the value in participatory design.NCLB's failure offers many lessons about understanding the strength of local context including history, culture, demographics, politics, and economics(Bonilla and Dee, 2020).Leadership for systems learning begins with examining the outcomes of current practice, inquiry designed to examine links between current practice and outcomes, an assessment of weak links, and the design of new practices developed to forge stronger and more robust links to ...
Purpose
This article focuses on the strategic importance of framing cultural changes in special education through a critical lens. The article explores why cultural responsivity must be understood from a critical perspective that accounts for the historical sedimentation of racism that exists within special education organizational policies and practices. This sedimentation affects current and future organizational features that sustain historical, persistent and pernicious racial and ableist structures, relationships and outcomes.
Design/methodology/approach
By examining the role of power within organizational systems, the authors trace its contribution to reproduction of these systems through special education leadership. Special education leaders along with their peers in general education can frame transformative change through a systemic lens designed to address structural, regulatory and cultural practices that perpetuate raced and ableist outcomes. The pernicious and sustaining structures and practices that have created unequal outcomes in our educational systems need strategic intervention, prevention and re-creation to create equitable supports and services programs.
Findings
By examining the role of power within organizational systems, the authors trace its contribution to reproduction of these systems through special education leadership. Special education leaders along with their peers in general education can frame transformative change through a systemic lens designed to address structural, regulatory and cultural practices that perpetuate raced and ableist outcomes.
Practical implications
With clear outcomes that are responsive to all students, including those identified with dis/abilities, education leaders can make consequential shifts in access, opportunity and the distribution of social and intellectual capital throughout education.
Social implications
The pernicious and sustaining structures and practices that have created unequal outcomes in our educational systems need strategic intervention, prevention and re-creation to create equitable supports and services programs.
Originality/value
The application of DisCrit to educational leadership practices offers an opportunity to frame leadership through a powerful equity lens.
... Whole-school reforms have received substantial policy interest and investment, especially through federal initiatives including Project Follow Through (Egbert, 1981), New American Schools (Berends et al., 2002), Comprehensive School Reform (Aladjem et al., 2010), School Improvement Grants (Dragoset et al., 2017), Race to the Top (Heissel & Ladd, 2018;Henry et al., 2015), and NCLB (No Child Left Behind) waivers under the Obama administration (Bonilla & Dee, 2017;Dee & Dizon-Ross, 2017;Dougherty & Weiner, 2019;Hemelt & Jacob, 2017). Though specific interventions varied, these reform initiatives share the idea that incremental interventions are insufficient and that low-performing schools do not have individual capacity to improve. ...
Recent evaluations of reforms to improve low-performing schools have almost exclusively focused on shorter term effects. In this study, we extend the literature by examining the sustainability and maturation of two turnaround models in Tennessee: the state-led Achievement School District (ASD) and district-led local Innovation Zones (iZones). Using difference-in-differences models, we find overall positive effects on student achievement in iZone schools and null effects in ASD schools. Additional findings suggest a linkage between staff turnover and the effectiveness of reforms. ASD schools experienced high staff turnover in every cohort, and iZone schools faced high turnover in its latest cohort, the only one with negative effects. We discuss how differences in the ASD and iZone interventions may help explain variation in the schools’ ability to recruit and retain effective teachers and principals.
... Heissel and Ladd (2018) found negative effects from the programs in North Carolina, while Zimmer, Henry, and Kho (2015) found some positive effects in Tennessee, particularly among Innovation Zone schools that were governed and managed separately by three school districts. Two companion studies in Kentucky and Louisiana showed opposite findings: Over each of three years, Louisiana's focus school reforms had no measurable impact on school performance (Dee & Dizon-Ross, 2017), while Kentucky's focus school reforms led to substantial improvements in both math and reading achievement (Bonilla & Dee, 2017). ...
Although program evaluations using rigorous quasi-experimental or experimental designs can inform decisions about whether to continue or terminate a given program, they often have limited ability to reveal the mechanisms by which complex interventions achieve their effects. To illuminate these mechanisms, this paper analyzes novel text data from thousands of school improvement planning and implementation reports from Washington State, deploying computer-assisted techniques to extract measures of school improvement processes. Our analysis identified 15 coherent reform strategies that varied greatly across schools and over time. The prevalence of identified reform strategies was largely consistent with school leaders’ own perceptions of reform priorities via interviews. Several reform strategy measures were significantly associated with reductions in student chronic absenteeism and improvements in student achievement. We lastly discuss the opportunities and pitfalls of using novel text data to study reform processes.
... Nonetheless, considering our findings in the context of the results of waiver reforms in other states offers the potential for broader takeaways. Our null findings resemble those found in Michigan (Hemelt & Jacob, 2017) and Rhode Island (Dougherty & Weiner, 2017), but differ from results found in Kentucky (Bonilla & Dee, 2017). Michigan's Priority Schools were similar to Louisiana in that the assignment rules were based on school-level composite scores (though Michigan included a measure of achievement gaps in its composite score) and Priority Schools were the lowest performers based on that score. ...
States that receive federal waivers to the No Child Left Behind Act were required to implement reforms in designated “Focus Schools” that contribute to achievement gaps. We examine the performance effects of such “differentiated accountability” reforms in Louisiana. These Focus School reforms emphasized school-needs assessments and aligned technical assistance. These reforms may have also been uniquely high-powered because they were linked to a letter-based school-rating system. We examine the impact of these reforms in a sharp regression-discontinuity (RD) design. We find that, over each of 3 years, Louisiana’s Focus School reforms had no measurable impact on school performance. We discuss evidence that these findings reflect policy reform fatigue and poor quality of implementation at the state and local level.
