Charles Smith

QTurn LLC

This paper presents steps to (a) identify the real objects we seek to represent with measurement and models (i.e., the parts of an individual’s SEL skill set and the type and amount of skill change that is likely to occur during the program) and (b) produce SEL skill indicators and measures that are feasible and valid for both CQI and impact evalua...

The Management Practices Self-Assessment (MPSA) provides a set of items for management best practices in 21st CCLC administrative and program settings. MPSA provides Project Directors with the opportunity to check-in with themselves and their team by identifying practices and policies that are well-documented and working well, or that could be upda...

This white paper introduces a theoretical framework designed to describe the integrated set of mental and behavioral parts and processes (i.e., schemas, beliefs, awareness, behavior, agency) that are socio-emotional skills and that produce both basic and advanced forms of agency. With improved definitions and understanding of SEL skills, and the ca...

The Q-ODM toolbox addresses practical questions about SEL skills and skill growth, such as: What is high-quality SEL support? How much SEL skill change does our program cause in each cycle? How much program quality does it take for stressed children to fully engage? Does our work create equity effects? The tools are divided into three groups: Desig...

Pattern-Centered treatment using the Quality-Outcomes evaluation design for a quasi-experimental, longitudinal data set on youth SEL skills in English Youth Provision programs.

This report describes the methodology and findings from the Quality-Impact-Equity evaluation design as deployed using data from at home visitation program in an urban region.

Guidance for Out-of-School Time Learning at a Distance (GOLD) is a set of program standards and self-assessment questions for OST program managers and staff who are responsible for delivering OST services to young people in their new individual learning environments. The standards described in the GOLD apply to a diverse range of program delivery m...

Impact evaluation for a high capacity Quality Improvement System in the out-of-school time sector. The overall pattern of impact reflected the hypothesized multi-level cascade of causes and effects.

The fields of psychology and education have over the years developed a science of categorical and ordinal description, pattern-centered and multi-level modeling, and formative measurement. These tools are designed to represent an object of measurement holistically for people and settings, to represent change in correspondence with actual individual...

This report exemplifies use of the suite of SEL measures and benchmarks in the American Youth Circus Organizations network.

Increases in funding for and attention to after-school programs have led to greater scrutiny over both the quality and effectiveness of this programming, putting pressure on programs to measure both setting-level characteristics and the impact of involvement on participants. Yet research on the outcomes of after-school participation has been mixed....

This paper describes validity of QIS performance measures and longitudinal change over four years in Oklahoma 21st CCLC programs.

This paper describes implementation and outcomes for QIS in school-based summer learning programs in multiple cities.

This paper compares outcomes for struggling students in high quality instructional settings to outcomes for similar students in low quality instructional settings.

This technical report describes methodology and findings for (1) best-practice SEL standards, (2) validation of a suite of SEL performance measures for use in QIS, and (3) performance benchmarks for out-of-school time programs focused on building SEL skills with vulnerable children/youth.

The guidebook describes SEL standards and organizational and curriculum features for a set of eight exemplary SEL programs.

This chapter discusses efforts to define and improve the quality of afterschool services, highlighting areas of agreement and identifying leading-edge issues. We conclude that the afterschool field is especially well positioned to deliver high-quality services and demonstrate effectiveness at scale because a strong foundation has been built for con...

The strategy of sharing program decision-making with youth in youth programs, a specific form of youth-adult partnership, is widely recommended in practitioner literature; however, empirical study is relatively limited. We investigated the prevalence and correlates of youth program decision-making practices (e.g., asking youth to help decide what a...

This paper includes: (1) An update to lower-stakes accountability theory and nomenclature with four case studies, (2) empirical evidence about reliability of quality ratings composed to represent a program site using longitudinal data from the Palm Beach county QIS, and (3) guidance about how to align benchmarks and incentives in the lower-stakes a...

This paper uses pattern-centered methods to increase the usefulness of information available from survey-based skill measures, with a focus on using survey-based skills measures to detect skill change over time.

Multi-site randomized controlled trial that identifies a substantively large and statistically significant cross-level cascade of QIS effects from network to organization to point-of-service instruction.

Case study of scaled QIS implementation in Rhode Island with focus on manager skills ad implementation at the organization level.

How-to guide for cities drawing on experience from implementation of QIS in 40+ cities and states.

A unique observational data set was used to explore quality at the point of service in after-school programs. Staff practices in after-school settings were represented on a series of unidimensional scales closely indexed to staff behavior. In order to account for heterogeneity of staff performances, pattern-centered methods were used to construct p...

This paper uses pattern centered methods to describe the association between the quality of instruction available and youth’s level of mental engagement with that instruction.

In the fragmented out-of-school-time sector, defining and measuring quality in terms of staff behaviors at the point of service provides a common framework that can reduce obstacles to cross-sector and cross-program performance improvement efforts and streamline adoption of data-driven accountability policies. This chapter views the point of servic...

This paper uses pattern centered methods to describe the association between four collaborative management practices and the quality of instruction available to youth.

This chapter describes the Youth Program Quality Intervention (YPQI), a setting-level intervention model designed to raise quality in out-of-school time programs. The YPQI takes managers and staff from a network of youth programs through a process of identifying and addressing strengths and areas for improvement, using a standardized assessment too...

Mixed methods study of QIS implementation and effects in afterschool programs in Palm Beach County, Fl.

This paper describes community partnerships in Michigan’s 21st CCLC programs.

The authors provide a research-based construct of youth program quality that differentiates safety, support, interaction, and engagement, four important issues for practitioners working with youth in the middle years. They describe two tools that have helped direct staff to be intentional about creating offerings that are appropriate for early adol...

Design theory and formative evidence to describe setting dynamics at two distinct levels, (1) the organization-level where professional learning communities are formed and (2) point-of-service level (e.g., classrooms) where professional staff and learners meet.

Original validation study for the Youth Program Quality Assessment (Youth PQA). The Youth PQA was developed to assess the quality of instruction in afterschool and other education settings

The Michigan School Readiness Program (MSRP) is the state's early childhood education program for 4-year-olds at risk of school failure. This report details previous findings from the longitudinal study of two cohorts of MSRP children from kindergarten through the primary grades, current findings from the 1998-99 school year, program quality and ri...

For decades the educational system has dedicated a massive quantity of resources to the end of equipping young adults with the necessary skills and values to support a working democratic culture. However, most measures of civic participation and learning suggest that the civic education system has little impact on democratic skills and values. And...