ArticlePDF Available

Abstract and Figures

The estimation of racial test score gap trends plays an important role in monitoring educational equality. Documenting gap trends is complex, however, and estimates can differ depending on the metric, modeling strategy, and psychometric assumptions. The sensitivity of summer learning gap estimates to these factors has been under-examined. Using national data, I find Black-White summer gap trends ranging from a significant relative disadvantage for Black students to a significant relative advantage. Preferred models show no overall gap change the summer after kindergarten, but Black students may make less summer math growth than White students with similar true spring scores. In estimating gap trends, researchers must recognize that different statistical models not only carry unique assumptions but also answer distinct descriptive questions.
Content may be subject to copyright.
A preview of the PDF is not available
... Several studies have found that students' rates of reading growth, on average, slow over summer but rarely-if ever-decline (e.g., Burkam, Ready, Lee, & LoGerfo, 2004;Quinn, 2015;Quinn, Cooc, McIntyre, & Gomez, 2016;von Hippel, Workman, & Downey, 2018). This led one set of researchers to acknowledge that 'near zero learning rates would be better described by a phrase like "summer slowdown" or "summer stagnation" ' (von Hippel, Workman, & Downey, 2018, p. 337). ...
... In comparison to statistical analysis of single-case data, visual analysis has been found to be more conservative (Wolfe, Dickenson, Miller, & McGrath, 2019). Given that strategies for modelling large-group data previously were found to influence interpretations of summer learning gaps (Quinn, 2015), we were interested in the new insights afforded through exploratory visual analysis of multiple time series datasets focusing on the complex problem of what happens to students' reading abilities in the summer. When possible to provide, it is more transparent for readers of different backgrounds and statistical knowledge to observe the direction of trajectories than to interpret complex statistical analyses. ...
... Based on our visual analysis, it remains possible that some students in some circumstances experience a decrement in reading performance over the summer, but this does not a phenomenon make. Despite other studies that have failed to confirm a ubiquitous summer loss (Burkam, Ready, Lee, & LoGerfo, 2004;Quinn, 2015;Quinn, Cooc, McIntyre, & Gomez, 2016;von Hippel, Workman, & Downey, 2018), the confirmation bias in the phenomenon persists. This might be explained by motivated reasoning in which individuals strategically access and evaluate information concordant with current beliefs, while discounting evidence that presents a cognitive dissonance (Kunda, 1990). ...
Article
Historically, there has been concern about students losing reading ability over extended breaks from school, commonly in the summer, but studies of this phenomenon have produced inconsistent results. We applied exploratory visual analysis of multiple datasets to examine whether students in Grades K‐5 appear to lose or improve in various reading abilities over the summer and across consecutive school years. Archival data were obtained on students of different U.S. school districts who did not participate in a formal summer reading programme. Data were disaggregated by groups considered most vulnerable to summer loss: those from economically disadvantaged backgrounds and those identified with disabilities. Given the variety of measure types and scores, we centred scores on each measure's cut point for proficiency in a particular grade level to depict how students' scores deviated from the proficiency classifications before and after a summer break. Overall, students' scores relative to the benchmark appeared on average to have maintained or improved, and there was no observed accumulated decrement in reading performance across years. For anomalous instances of summer loss, we offer possible alternative explanations such as measurement artefacts and unrehearsed learning. Visual analysis of the datasets suggested that summer breaks were not associated with systematic losses of students' reading ability, even among those considered most vulnerable to the phenomenon. However, available assessments and benchmarks are not designed to measure summer learning specifically, and little is known about the kinds of literacy experiences students not in formal programmes might be having. Thus, more research on summer maturation and degeneration is warranted.
... Also, different ways of operationalizing socioeconomic attributes of students result in incomparability of findings (e.g., Sirin 2005). Further, researchers have used absolute and relative metrics to report performance change over the summer, which inherently reflect different perspectives on change; in addition, the handling of measurement error can alter results (Quinn 2015;Quinn et al. 2016). Hippel and Hamrock (2019) found results to be sensitive towards the choice of test score scaling methods and to changes in test forms, i.e. whether the same test form is used for pre-and post-testing. ...
... Hippel and Hamrock (2019) found results to be sensitive towards the choice of test score scaling methods and to changes in test forms, i.e. whether the same test form is used for pre-and post-testing. Moreover, one very crucial point of divergence in the investigation of social disparities in summer setback effects, is the choice of the modeling strategy (see Quinn 2015). Researchers analyzing pretest and post-test designs are confronted with the choice between a difference score approach or a regressor variable approach (Allison 1990;van Breukelen 2013;Gastro-Schilo and Grimm 2018). ...
... Depending on the method chosen, the magnitude and the direction of effects can change. In line with earlier works (e.g., Holland and Rubin 1982), Quinn (2015) argues that rather than pointing to a supposedly paradoxical situation of contradictory results, the statistical models applied answer different research questions. Very recently, Dumont and Ready (2019) outlined how differing modeling strategies of student learning influence our understanding of the role schools play in educational inequality. ...
Article
Full-text available
Summer break study designs are used in educational research to disentangle school from non-school contributions to social performance gaps. The summer breaks provide a natural experimental setting that allows for the measurement of learning progress when school is not in session, which can help to capture the unfolding of social disparities in learning that are the result of non-school influences. Seasonal comparative research has a longer tradition in the U.S. than in Europe, where it is only at its beginning. As such, summer setback studies in Europe lack a common methodological framework, impairing the possibility to draw lines across studies because they differ in their inherent focus on social inequality in learning progress. This paper calls for greater consideration of the parameterization of “unconditional” or “conditional” learning progress in European seasonal comparative research. Different approaches to the modelling of learning progress answer different research questions. Based on real data and constructed examples, this paper outlines in an intuitive fashion the different dynamics in inequality that may be simultaneously present in the survey data and distinctly revealed depending on whether one or the other modeling strategy of learning progress is chosen. An awareness of the parameterization of learning progress is crucial for an accurate interpretation of the findings and their international comparison.
... Em geral, quanto ao nível socioeconômico, os resultados sugerem que não há um aumento na diferença de desempenho e, quando essa ocorre é sutil. Por sua vez, as diferenças no desempenho escolar entre raças podem até diminuir durante as férias (HIPPEL; HAMROCK, 2019; HIPPEL; WORKMAN; DOWNEY, 2018;QUINN, 2015;QUINN;LE, 2018). O que parece ocorrer é uma redução dos ganhos de conhecimento aferidos por testes, aplicados logo após longas paralizações. ...
... Em geral, quanto ao nível socioeconômico, os resultados sugerem que não há um aumento na diferença de desempenho e, quando essa ocorre é sutil. Por sua vez, as diferenças no desempenho escolar entre raças podem até diminuir durante as férias (HIPPEL; HAMROCK, 2019; HIPPEL; WORKMAN; DOWNEY, 2018;QUINN, 2015;QUINN;LE, 2018). O que parece ocorrer é uma redução dos ganhos de conhecimento aferidos por testes, aplicados logo após longas paralizações. ...
Article
Full-text available
Paralisadas pelo pânico decorrente da pandemia provocada pela Covid-19, o fechamento das escolas trará consequências para a aprendizagem dos alunos. Este artigo examina situações de paralisação, uso do tempo e impacto de tecnologias no desempenho escolar. O exame das evidências sugere como pouco promissoras as estratégias de aumento intensivo de tempo ou recurso a tecnologias. Caminhos mais promissores estariam associados a diagnósticos, intervenções estruturadas mais adequadas ao perfil dos professores, melhor aproveitamento do tempo, uso judicioso dos deveres de casa, redução do absenteísmo e tutoria intensiva de alta qualidade focada nos alunos de maior risco. Palavras-chave: Covid-19. Interrupção do Calendário Escolar. Educação Baseada em Evidências. 1 Introdução De acordo com a Organização das Nações Unidas para a Educação, a Ciência e a Cultura (UNESCO, 2020), no início de maio de 2020, 186 países ou regiões fecharam escolas, total ou parcialmente, para conter a disseminação da Covid-19, atingindo cerca de 70% dos alunos. Esse fechamento afetou o calendário escolar, sendo incerto o seu impacto sobre o aprendizado dos alunos. Diferenças no rigor da quarentena, na sua duração e nas estratégias adotadas pelas famílias e escolas são apenas alguns dos fatores que poderão influenciar a trajetória desses alunos.
... We thank the 222 individuals who responded to our survey, Beth Tipton and David Epstein for advice about the survey and help sending it out, and Matthew Kraft, Frank Levy, Justin Reich, and Anna Saavedra for helpful comments on earlier drafts. Bailey is funded by a Jacobs Foundation Early Career Research Fellowship. 1 Changes in gaps can be sensitive to scaling decisions, particularly when achievement levels also change (Bond & Lang, 2013;Quinn, 2015). Because the pandemic may affect both levels and gaps in achievement, no single estimate of observed gaps should be taken as a definitive estimate. ...
Article
A survey targeting education researchers conducted in November 2020 provides forecasts of how much achievement gaps between low- and high-income students in U.S elementary schools will change as a result of COVID-related disruptions to in-class instruction and family life. Relative to a pre-COVID achievement gap of 1.00 SD, respondents’ median forecasts for the jump in the achievement gaps in elementary school by spring 2021 were large–a change from 1.00 to 1.30 and 1.25 SD for math and reading achievement. Forecasts were similar for 2022. Although forecasts were heterogeneous, respondents showed overwhelming consensus that gaps would grow. We discuss implications for strategies to reduce learning gaps exacerbated by the pandemic as well as the mental models researchers appear to employ in making their forecasts.
... Some of these wider exposures may mitigate the relationship between parental and child vocabulary. In particular, it is likely that schooling plays a role (Quinn, 2015a). However, it is of course possible that vocabulary gaps will widen substantially during the cohort members' life course. ...
Article
Full-text available
This paper examines the relationship between parents' and children's language skills for a nationally representative birth cohort born in the United Kingdom-the Millennium Cohort Study (MCS). We investigate both socioeconomic and ethnic differentials in children's vocabulary scores and the role of differences in parents' vocabulary scores in accounting for these. We find large vocabulary gaps between highly educated and less educated parents, and between ethnic groups. Nevertheless, socioeconomic and ethnic gaps in vocabulary scores are far wider among the parents than among their children. Parental vocabulary is a powerful mediator of inequalities in offspring's vocabulary scores at age 14, and also a powerful driver of change in language skills between the ages of five and 14. Once we account for parental vocabulary, no ethnic minority group of young people has a negative "vocabulary gap" compared to whites.
Article
We present results from a meta-analysis of 37 contemporary experimental and quasi-experimental studies of summer programs in mathematics for children in grades pre-K–12, examining what resources and characteristics predict stronger student achievement. Children who participated in summer programs that included mathematics activities experienced significantly better mathematics achievement outcomes compared to their control group counterparts. We find an average weighted impact estimate of +0.10 standard deviations on mathematics achievement outcomes. We find similar effects for programs conducted in higher- and lower-poverty settings. We undertook a secondary analysis exploring the effect of summer programs on noncognitive outcomes and found positive mean impacts. The results indicate that summer programs are a promising tool to strengthen children’s mathematical proficiency outside of school time.
Article
We draw on novel district-level test score data to describe novel approaches for measuring ethnoracial achievement gaps and assessing trends toward achievement equity from 2009 to 2016. Using SEDA data, we estimate gap trends for each grade over time in each district. We measure trends in both within-district gaps—comparing Black or Hispanic to White students in the same district—and national gaps—comparing a district’s Black or Hispanic students to White students nationally. Withindistrict ethnoracial gaps shrunk in one-third to two-thirds of districts, depending on subject and ethnoracial dyad. Across subjects and ethnoracial dyads, national gaps shrunk in more than half of districts, indicating that non-White students gained on White students nationally, but not in their own districts. Our findings add complexity to the achievement gap literature by (1) estimating gaps at the district level; (2) noting considerable variation in the magnitude of gap shrinkage across districts; (3) pointing to the importance of comparison group and imperfect correspondence of within-district and national gap trends in districts; and (4) identifying variation in gap trends across grades and subjects.
Article
Important educational policy decisions, like whether to shorten or extend the school year, often require accurate estimates of how much students learn during the year. Yet, related research relies on a mostly untested assumption: that growth in achievement is linear throughout the entire school year. We examine this assumption using a dataset containing math and reading test scores for over seven million students in kindergarten through 8th grade across the fall, winter, and spring of the 2017–18 school year. Our results indicate that assuming linear within-year growth is often not justified, particularly in reading. Implications for investments in extending the school year, summer learning loss, and the development of racial/ethnic achievement gaps are discussed.
Article
This study examines growth in language and math skills during the summer before kindergarten; considers variation by family income, race/ethnicity, and dual language learner status; and tests whether summer center-based care sustains preschool gains. Growth in skills slowed during summer for all children, but the patterns varied by domain and group. Non-White and dual language learner students showed the largest drop-off in language skills during summer. Lower-income students demonstrated slower summer growth in math skills than their higher-income peers. Students enrolled in summer center-based care had faster growth in math skills than those who did not attend care. Yet lower-income students who attended center-based care showed slower growth in language skills during summer than similar nonattenders. Implications are discussed.
Article
Full-text available
Background/Context Seasonal researchers have developed a theory and hypotheses regarding the importance of neighborhood and school contexts for early childhood learning but have not possessed nationally representative data and precise contextual measures with which to examine their hypotheses. Purpose/Research Questions This empirical study employs a seasonal perspective to assess the degree to which social context and race/ethnic composition—in neighborhoods and schools—affect the reading achievement growth of young children. The authors ask, Were there specific seasons when context and/or composition were particularly salient for reading achievement? Also, did accounting for context and composition challenge established appraisals of the relationship between family factors and achievement? Population Data for a nationally representative sample of students proceeding through kindergarten and first grade came from the Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K). Neighborhood social and race/ethnic measures came from the 2000 Census. Research Design: This quantitative study employs a three-level model that assesses reading achievement at school entry and during three subsequent seasons. The model represents reading achievement as a time-varying process at level 1, conditional upon family socio/demographic factors at level 2, and dependent on social context and race/ethnic composition at level 3. Findings/Results Neighborhood social context mattered substantially for students’ reading achievement levels at school entry and for their reading achievement growth during the summer. The proportion of neighborhood residents from minority race/ethnic groups was not associated with reading achievement at school entry or during the summer season. During the school year, school social context was associated with reading growth during kindergarten, and school social context and race/ethnic composition were associated with reading growth during first grade. Conclusions/Recommendations The magnitude and frequency of contextual effects found in this national sample have considerable implications for achieving educational equality in the United States. The authors recommend that policy makers attend to the quality of neighborhood and school settings as a means of promoting literacy development for young children.
Article
Full-text available
The finding that children of different racial groups and socioeconomic backgrounds begin their formal educational careers with disparities in skills on standardized tests (e.g., Lee and Burkam 2002) and that these inequalities persist through pri-mary and secondary school (Phillips, Crouse, and Ralph 1998) is well established. Although parent-ing practices play an important role in the develop-ment of differences in early childhood achievement (Guo 1998; J. R. Smith, Brooks-Gunn, and Kle-banov 1997), less is known about the role that fam-ilies play in disparities in achievement once chil-dren enter elementary school. In her ethnographic study, Lareau (2002, 2003) reported pronounced so-cial-class differences in the ways parents organize their children's lives around adult-orchestrated lei-sure activities, interact with teachers and the edu-cational system, and verbally and academically en-gage their children. According to Lareau (2003), different ways of organizing their children and their own lives along these dimensions reflect con-trasting parental investments in children's educa-tional growth, which ultimately has important con-sequences for the skills and abilities that children develop. Lareau's research is part of an older tradition that has noted meaningful variation in class-based parenting strategies (e.g., Kohn 1977) and provides an important avenue for the operationalization of parents' patterns of educational investment. In-deed, although Lareau (1989) was critical of the of-ten narrow foci of quantitative research, her most Published in Sociology of Education 81:1 (January 2008), pp. 1–31; doi: 10.1177/003804070808100101 Published by Sage Publications for the American Sociological Association. Used by permission.
Article
Full-text available
Although both economists and psychometricians typically treat them as interval scales, test scores are reported using ordinal scales. Using the Early Childhood Longitudinal Study and the Children of the National Longitudinal Survey, we examine the effect of order-preserving scale transformations on the evolution of the black-white reading test score gap from kindergarten entry through third grade. Plausible transformations reverse the growth of the gap in the CNLSY and greatly mitigate it in the ECLS-K during early school years. All growth from entry through first grade and a nontrivial proportion from first to third grade probably reflects scaling decisions.Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.
Book
Class does make a difference in the lives and futures of American children. Drawing on in-depth observations of black and white middle-class, working-class, and poor families, Unequal Childhoods explores this fact, offering a picture of childhood today. Here are the frenetic families managing their children's hectic schedules of "leisure" activities; and here are families with plenty of time but little economic security. Lareau shows how middle-class parents, whether black or white, engage in a process of "concerted cultivation" designed to draw out children's talents and skills, while working-class and poor families rely on "the accomplishment of natural growth," in which a child's development unfolds spontaneously—as long as basic comfort, food, and shelter are provided. Each of these approaches to childrearing brings its own benefits and its own drawbacks. In identifying and analyzing differences between the two, Lareau demonstrates the power, and limits, of social class in shaping the lives of America's children. The first edition of Unequal Childhoods was an instant classic, portraying in riveting detail the unexpected ways in which social class influences parenting in white and African American families. A decade later, Annette Lareau has revisited the same families and interviewed the original subjects to examine the impact of social class in the transition to adulthood.
Book
A preeminent sociologist of race explains a groundbreaking new framework for understanding racial inequality, challenging both conservative and liberal dogma.In this timely and provocative contribution to the American discourse on race, William Julius Wilson applies an exciting new analytic framework to three politically fraught social problems: the persistence of the inner-city ghetto, the plight of low-skilled black males, and the fragmentation of the African American family. Though the discussion of racial inequality is typically ideologically polarized. Wilson dares to consider both institutional and cultural factors as causes of the persistence of racial inequality. He reaches the controversial conclusion that while structural and cultural forces are inextricably linked, public policy can only change the racial status quo by reforming the institutions that reinforce it.
Article
In a longitudinal study of a random sample of Baltimore youngsters starting first grade, the mathematics achievement level of African-Americans and whites was almost identical. Two years later, African-American students had fallen behind by about half a standard deviation. We use mathematics test score changes over the summer when school is closed to estimate "home" influences, and we investigate three major hypotheses that might account for lower mathematics achievement among African-Americans. The most important source of variation in mathematics achievement is differences in family economic status, followed by school segregation. Two-parent (father-present) vs. one-parent (father-absent) family configurations are probably negligible as a cause when economic status is controlled. Poor children of both races consistently lose ground in the summer but do as well or better than better-off children in winter when school is in session. We discuss the theoretical and policy implications of these findings.
Article
Test scores are commonly reported in a small number of ordered categories. Examples of such reporting include state accountability testing, Advanced Placement tests, and English proficiency tests. This article introduces and evaluates methods for estimating achievement gaps on a familiar standard-deviation-unit metric using data from these ordered categories alone. These methods hold two practical advantages over alternative achievement gap metrics. First, they require only categorical proficiency data, which are often available where means and standard deviations are not. Second, they result in gap estimates that are invariant to score scale transformations, providing a stronger basis for achievement gap comparisons over time and across jurisdictions. The authors find three candidate estimation methods that recover full-distribution gap estimates well when only censored data are available.
Article
Problems of scale typically arise when comparing test score trends, gaps, and gap trends across different tests. To overcome some of these difficulties, test score distributions on the same score scale can be represented by nonparametric graphs or statistics that are invariant under monotone scale transformations. This article motivates and then develops a framework for the comparison of these nonparametric trend, gap, and gap trend representations across tests. The connections between this framework and other nonparametric tools, including probability—probability (PP) plots, the Mann-Whitney U test, and the statistic known as P(Y > X), are highlighted. The author describes the advantages of this framework over scale-dependent trend and gap statistics and demonstrates applications of these nonparametric methods to frequently asked policy questions.