Content uploaded by Diana Owen
Author content
All content in this area was uploaded by Diana Owen on Mar 11, 2024
Content may be subject to copyright.
Diana Owen
Professor
Director, Civic Education Research Lab
Georgetown University
March 2024
PR/Award Number R305A190360
Project Citizen Research Program
Funded by the Institute of Education Sciences, U.S. Department of Education
Civic Education Research Lab
The Civic Education Research Lab (CERL) at Georgetown University is dedicated to
investigating the effectiveness of civic teaching and learning. Founded by Dr. Diana Owen, a
Georgetown political science professor, CERL investigates the effectiveness of civic education
for elementary, secondary, and university students. The CERL team conducts research on civic
education programs and practices, civic engagement, and political communication in the digital
age. To learn more: Civic Education Research Lab (georgetown.edu)
Center for Civic Education
The Center for Civic Education (civiced.org) has led K-12 civic education nationwide since its
founding in 1965. The Center provides professional development, curricular materials,
instructional resources, toolkits, and support for civic educators on a broad scale. Its programs
receive active endorsement from state bar associations, foundations, and educational,
professional, business, and community organizations in every state and the District of Columbia.
Since its inception, the Center’s innovative, evidence-based programs have reached more than 42
million students and 440,000 educators in over 80 countries. Project Citizen provides a practical,
first-hand approach to learning about how government works, how to monitor its actions, and
how to influence public policy. Other Center programs and resources include its flagship We the
People: The Citizen and the Constitution program, Civitas International Programs, and the Civil
Discourse Toolkit. The Center and CERL have received six major grants from the U.S.
Department of Education. These include the James Madison Legacy Project (2015-2020),
Strengthening Democracy through History and Civics (2018- 2021), Project Citizen Research
Program (2019-2024), James Madison Legacy Project Expansion (2021-2025), We the People:
Civics that Empowers All Students (2022-2025), and Project Community: Media Literacy and
Public Policy (2023-2026). To learn more: https://www.civiced.org/
The Project Citizen Research Program was funded by a generous grant from the Institute
of Education Sciences, U.S. Department of Education.
Citation: Owen, Diana. 2024. Project Citizen Research Program: Report. Research Report. Washington,
D.C.: Civic Education Research Lab, Georgetown University.
Disclosure of Potential Conflict of Interest
The research team for this study is based at the Civic Education Research Lab at Georgetown
University in Washington, D.C. Neither the author nor the staff have financial interests that
could be affected by the findings of this study.
ACKNOWLEDGEMENTS
The Institute of Education Sciences (IES) of the U.S. Department of Education is at the forefront
of promoting quality education for all. Their support of this project, especially during the
COVID-19 pandemic, has been extraordinary. I am grateful to Dr. Edward Metz, IES Research
Scientist, and more recently, Dr. Vinita Chhabra, IES Program Officer.
The talent, expertise, and professionalism of the Center for Civic Education’s staff and network
partners of state coordinators and mentor teachers is inspiring. Alissa Irion-Groth has been
exceptional as the Center’s principal investigator for the PCRP. The contributions of the late
Chuck Quigley, founder of the Center for Civic Education, John Hale, Mark Molly, Maria Gallo,
Mark Gage, and other Center staff who shared a vision for Project Citizen and worked on the
successful PCRP grant proposal must be recognized. Dr. Donna Phillips, Vice President and
Chief Program Officer, has supported the work.
I am thankful for the CERL research team members who contributed to this study. Katy Hartzell
deserves special recognition for her dedication to the PCRP as well as to CERL’s research
projects and mission over the long haul. M. Bradlee Sutherland, Molly May, and Arjun Chawla
assisted with the research and made important contributions to the study. I greatly appreciate the
graphic design work of Charlotte Kraenzle Peña that makes the findings of this study more
widely accessible.
I am incredibly lucky to have Jeffrey Owen—and Bella and Gio—in my corner.
Above all, I would like to recognize the teachers and students who took part in this study under
the most trying circumstances. The civic educators who persevered through months of upheaval
and uncertainty while continuing to teach young people about American government and
democracy are heroes.
Table of Contents
......................................................................................................... 1
EXECUTIVE SUMMARY ........................................................................................................................ 3
Key Study Findings ................................................................................................................................. 3
CIVIC KNOWLEDGE ........................................................................................................................... 3
CIVIC DISPOSITIONS ......................................................................................................................... 3
CIVIC SKILLS ....................................................................................................................................... 4
CIVIC ENGAGEMENT ........................................................................................................................ 4
CIVICS-RELATED SEL COMPETENCIES ...................................................................................... 5
STEM IN THE CIVICS CLASSROOM ............................................................................................... 5
PROJECT-BASED LEARNING: ............................................................................................................. 7
THE PROJECT CITIZEN MODEL ......................................................................................................... 7
Project Citizen ......................................................................................................................................... 8
Teacher Professional Development ....................................................................................................... 9
Classroom Curriculum Intervention ................................................................................................... 10
METHODOLOGY ................................................................................................................................... 11
Research Design .................................................................................................................................... 11
Sampling ................................................................................................................................................ 12
Statistical Methodology ........................................................................................................................ 13
THE IMPACT OF THE COVID-19 PANDEMIC ................................................................................ 15
SCHOOL CHARACTERISTICS ............................................................................................................ 17
TEACHER CHARACTERISTICS ......................................................................................................... 19
STUDENT CHARACTERISTICS .......................................................................................................... 21
High-Need Students .............................................................................................................................. 22
CIVIC KNOWLEDGE ............................................................................................................................. 23
Teachers’ Civic Knowledge .................................................................................................................. 23
Measurement ..................................................................................................................................... 24
Analysis .............................................................................................................................................. 25
Teachers’ Instructional Objectives: Civic Knowledge ...................................................................... 25
Measurement ..................................................................................................................................... 26
Analysis .............................................................................................................................................. 26
Teachers’ Efficacy in Conveying Civic Knowledge ........................................................................... 27
Measurement ..................................................................................................................................... 27
Analysis .............................................................................................................................................. 27
Students’ Civic Knowledge .................................................................................................................. 28
Measurement ..................................................................................................................................... 28
Analysis .............................................................................................................................................. 29
CIVIC DISPOSITIONS ........................................................................................................................... 33
Teacher Instructional Objectives: Developing Civic Dispositions .................................................... 33
Measurement ..................................................................................................................................... 34
Analysis .............................................................................................................................................. 34
Student Civic Dispositions .................................................................................................................... 36
Civic Responsibility .............................................................................................................................. 36
Measurement ..................................................................................................................................... 36
Analysis .............................................................................................................................................. 37
Attention to Issues ................................................................................................................................. 38
Measurement ..................................................................................................................................... 38
Analysis .............................................................................................................................................. 39
Duty to Vote ........................................................................................................................................... 41
Measurement ..................................................................................................................................... 41
Analysis .............................................................................................................................................. 41
Future Government Service ................................................................................................................. 44
Measurement ..................................................................................................................................... 44
Analysis .............................................................................................................................................. 45
Trust in Government ............................................................................................................................ 47
Measurement ..................................................................................................................................... 47
Analysis .............................................................................................................................................. 47
Trust in Media ....................................................................................................................................... 49
Measurement ..................................................................................................................................... 49
Analysis .............................................................................................................................................. 50
CIVIC SKILLS ......................................................................................................................................... 51
Teacher Instructional Objectives: Developing Civic Skills ............................................................... 52
Measurement ..................................................................................................................................... 52
Analysis .............................................................................................................................................. 52
PBL Pedagogies for Developing Civic Skills ....................................................................................... 53
Measurement ..................................................................................................................................... 54
Analysis .............................................................................................................................................. 54
Active Pedagogies that Convey Civic Skills ........................................................................................ 55
Measurement ..................................................................................................................................... 56
Analysis .............................................................................................................................................. 56
Students’ Civic Skills ............................................................................................................................ 59
Measurement ..................................................................................................................................... 59
Analysis .............................................................................................................................................. 61
CIVIC ENGAGEMENT .......................................................................................................................... 62
Teacher Efficacy in Getting Students to Engage ................................................................................ 63
Measurement ..................................................................................................................................... 63
Analysis .............................................................................................................................................. 63
Students’ Propensity for Engagement ................................................................................................. 64
Measurement ..................................................................................................................................... 64
Analysis .............................................................................................................................................. 64
CIVICS-RELATED SEL COMPETENCIES ........................................................................................ 69
Teachers’ Efficacy Implementing Civics-Related SEL ..................................................................... 70
Measurement ..................................................................................................................................... 71
Analysis .............................................................................................................................................. 72
Self-Care and Self-Management .......................................................................................................... 73
Relationship Skills ................................................................................................................................. 74
Respectful Discourse ............................................................................................................................. 75
Students’ Social and Emotional Learning .......................................................................................... 76
Problem-Solving Skills .......................................................................................................................... 76
Measurement ..................................................................................................................................... 77
Analysis .............................................................................................................................................. 77
Civic Expression Skills ......................................................................................................................... 79
Measurement ..................................................................................................................................... 79
Analysis .............................................................................................................................................. 80
STEM SKILLS .......................................................................................................................................... 82
Teachers’ Integration of STEM Skills in the Civic Curriculum ....................................................... 83
Measurement ..................................................................................................................................... 83
Analysis .............................................................................................................................................. 84
Student’s Use of STEM ........................................................................................................................ 87
Measurement ..................................................................................................................................... 87
Analysis .............................................................................................................................................. 88
SUMMARY ............................................................................................................................................... 93
REFERENCES .......................................................................................................................................... 95
APPENDIX A .......................................................................................................................................... 106
SAMPLE ATTRITION ...................................................................................................................... 106
Sample Attrition for Schools, Teachers, and Students ........................................................................ 107
APPENDIX B .......................................................................................................................................... 111
ANCOVA TABLES ............................................................................................................................ 111
APPENDIX C .......................................................................................................................................... 139
KNOWLEDGE QUESTION WORDING ........................................................................................ 139
TEACHER CIVIC KNOWLEDGE ITEMS ................................................................................ 140
STUDENT CIVIC KNOWLEDGE ITEMS ................................................................................. 145
1
Project Citizen (PC) was launched by the Center for Civic Education in California in
1992 and was disseminated nationally in 1995. To date, the program has been used in 75
countries (http://civiced.org/programs/project-citizen). PC’s objective is “to motivate and
empower students to use the rights and responsibilities of democratic citizenship by intensively
examining a policy issue in their schools or communities” (Ozturk, 2022). Students work as a
class to identify a problem, research alternative policy-based solutions, develop a policy proposal
to address the problem, and design an action plan to convince public officials to adopt and
implement the policy. Students present their action plans and portfolios to leaders in their
schools and communities. PC is designed to support students’ development of social and
emotional learning (SEL) competencies and science, technology, engineering, and math (STEM)
skills.
The Project Citizen Research Program (PCRP) evaluated the effectiveness of the Center’s
PC teacher professional development (PD) program and curriculum intervention for middle and
high school students. The Civic Education Research Lab (CERL) at Georgetown University
under the direction of Dr. Diana Owen conducted studies of three cohorts of PC teachers and
students from 2020-2023—during the height of the COVID-19 pandemic. The PCRP explored
the impact of the PD program on teachers’ civic content knowledge, instructional objectives, use
of active learning pedagogies, self-efficacy, and integration of STEM into the curriculum.
Student academic learning outcomes after participating in PC related to acquisition of civic
knowledge, dispositions, and skills, the development of civics-related SEL competencies, and the
use of STEM in their classes were assessed. This report marks the culmination of that research.
The PCRP sought to strengthen the research base for teaching and learning in social
studies, especially the core discipline of civics. The program addressed the need for improving
and providing civic education for all students. Delivering accessible quality civic education is
especially urgent in schools with high concentrations of high-need students where attention to
civics is typically overshadowed by a focus on subjects included in high-stakes assessments.
High-need students are disadvantaged in the current system because they have less access to
effective civics instruction tailored to their needs. Students from higher socioeconomic
backgrounds receive more classroom-based civic learning opportunities than other students
(Jamieson, 2013; Educating for American Democracy, 2021). A “civic opportunity gap,” has
been identified where students from high-poverty environments lack access to service learning
and organized activities that would prepare them for civic engagement (Kahne and Middaugh,
2008; Rubin, et al., 2016). Schools can exacerbate the “civic empowerment gap” (Levinson
2010, 2013)—where political power is concentrated among those with the greatest resources and
2
entitlements—by not providing equal civic preparation to students most in need of acquiring the
capacities to participate effectively in political life. The gaps in opportunity, engagement, and
empowerment begin early and grow wider over the life course (Hoggan-Kloubert, et al., 2023).
CERL employed an internal efficacy study to examine PC’s impact on student civic
education outcomes in comparison to the current practice of social studies teachers who employ
traditional lecture and textbook-centric pedagogies in their classrooms. The core quantitative
study design consisted of multi-site, school-level, randomized controlled trials (RCT) based on
pretest/posttest surveys administered to middle and high school teachers and their students. The
RCT compared PC teacher and student outcomes to those of the comparison group. CERL
collected qualitative data to augment, enrich, and provide context for the quantitative analysis
over the course of the three program years.
3
EXECUTIVE SUMMARY
The Project Citizen Research Program (PCRP) evaluated the effectiveness of the Center
for Civic Education’s Project Citizen teacher professional development and curriculum over the
course of three academic years from 2020-23. Teachers attended a summer institute and follow-
up professional development sessions throughout the academic year and instructed the Project
Citizen curriculum. Middle and high school students work as a class to research and develop
proposals for solving a policy problem in their school or community which they present to
stakeholders. The research was conducted by the Civic Education Research Lab at Georgetown
University. The PCPR was funded by a grant from the Institute of Education Sciences, U.S.
Department of Education.
Key Study Findings
CIVIC KNOWLEDGE
Project Citizen teachers’ knowledge of American government, democracy, and the public policy
process increased significantly in each program year.
Project Citizen students’ civic knowledge improved significantly in all three cohorts.
• Middle school students’ civic knowledge increased by 44% in Year 1, 76% in Year 2,
and 66% in Year 3.
• High school students’ civic knowledge increased by 27% in Year 1, 56% in Year 2, and
32% in Year 3.
• Knowledge growth was significantly greater for Project Citizen students than for students
who took a traditional civics, social studies, American government, or American history
class.
CIVIC DISPOSITIONS
Civic dispositions are the public and private traits, attitudes, and ingrained “habits of the heart”
that are consistent with the common good and are central to the functioning of a healthy
democracy.
Project Citizen teachers placed significantly more emphasis on civic dispositions in their classes
after participating in the professional development program.
Project Citizen students embraced the virtues of good democratic citizens.
• Secondary school students became more inclined to keep informed about government and
politics and pay attention to issues in their community.
• Secondary school students had a stronger commitment to vote in elections if given the
opportunity.
4
• Students, like the general public, had low levels of trust in government and the news
media. High school students’ trust in government and media increased after the program.
• High school students became more interested in pursuing a career in government service
and possibly running for office one day.
• Project Citizen students’ civic dispositions improved more than those of the control
group.
CIVIC SKILLS
Civic skills are a range of proficiencies required for democratic engagement. They encompass
behaviors beneficial to the development of personal agency that promotes civic engagement.
Project Citizen teachers were much more likely to emphasize civic skills during their classes.
• The number of teachers who focused a great deal on civic skills increased from 40% to
73% in Year 1, from 57% to 71% in Year 2, and from 22% to 47% in Year 3.
• Teachers’ integration of activities that convey civic skills into their lessons increased by
156% in Year 1, 136% in Year 2, and 99% in Year 3.
Project Citizen prepared students to participate in their communities and public life.
• Secondary school students had a better understanding of policy issues facing the country.
• Students felt that they could help organize people to solve a problem in their community.
• Students could find the official or branch of government responsible for working on
community problems.
• The findings were strongest for high school students.
• Project Citizens students’ civic skills improved significantly more than those of students
in the control group.
CIVIC ENGAGEMENT
Civic engagement is voluntary involvement in community affairs that is put forth in the public
interest. It is active participation that involves community service that is collaborative and works
toward addressing areas of local, national, and global concern.
Project Citizen teachers became more confident in their ability to get students to engage in their
community.
• The percentage of teachers who felt they were greatly effective in encouraging students
to become involved in their community increased from 51% to 70% in Year 1, from 55%
to 71% in Year 2, and from 47% to 54% in Year 3.
Project Citizen students felt better prepared to engage in political and public life after the
program. The findings were strongest for high school students.
5
• The percentage of high school students who were very likely to turn out in elections increased
from 69% to 72% in Year 1, from 63% to 69% in Year 2, and from 66% to 76% in Year 3.
• The percentage of high schoolers who were ready to engage in public life increased by
7% in Year 1, 10% in Year 2, and 6% in Year 3.
• The findings were more apparent for Project Citizen students than for those who took a
traditional civics class.
CIVICS-RELATED SEL COMPETENCIES
Social and emotional learning (SEL) is the process through which people develop the
knowledge, skills, and attitudes conducive to achieving personal and collective goals,
maintaining positive relationships, and making meaningful societal connections. Integrating SEL
and civic learning can give students the opportunity to become caring and engaged community
members.
Teachers felt more capable of promoting students’ self-care and self-management, developing
students’ relationship skills, promoting respectful classroom discourse, and encouraging
students’ civic engagement after participating in Project Citizen.
• 77% of teachers felt that Project Citizen contributed to their students’ acquisition of SEL
competencies.
Project Citizen students gained civics-related SEL skills.
• Middle and high school students achieved significant gains in their problem solving
abilities and civic expression skills.
• The gains in civics-related SEL skills were notably smaller for the control group students.
STEM IN THE CIVICS CLASSROOM
The U.S. Department of education has advocated for the integration of science,
technology, engineering, and mathematics (STEM) education across the curriculum, stating that
“the complexities of today’s world require all people to be equipped with a new set of core
knowledge and skills to solve difficult problems, gather and evaluate evidence, and make sense
of information they receive from varied print, and increasingly, digital media.”
Project Citizen teachers were more inclined to have their students use STEM skills in their
classes after the professional development program.
• 40% of Project Citizen teachers indicated that they were very prepared to incorporate
STEM into the civics curriculum after participating in PCRP compared to 3% pre-
program.
• The percentage of Project Citizen teachers indicating that their students had use STEM
skills in their classes doubled from 25% pre-program to 50% post-program.
6
• The percentage of Project Citizen teachers whose students conducted surveys increased
from 51% to 88%. Control group teachers’ use of surveys was limited and did not
increase.
• Project Citizen teachers were more likely to have their students use technology to engage
in the community after their professional development program.
Secondary school students who participated in Project Citizen applied STEM skills when
conducting research for their projects.
• Project Citizen students were better able to understand how they can use STEM skills to
address problems in their community.
• Middle and high school students were able to make a stronger connection between STEM
and their civics classes after participating in Project Citizen.
• The effects were not apparent for control group students.
7
PROJECT-BASED LEARNING:
THE PROJECT CITIZEN MODEL
PC is project-based learning (PBL) developed for the civics, social studies, and American
government context. PBL is a student-centered, dynamic learning pedagogical approach that
provides students with the opportunity to engage actively and cooperatively with real-world
issues and propose solutions. The Buck Institute for Education defines PBL as follows:
Project-based learning is a teaching method in which students gain knowledge and skills
by working for an extended period of time to investigate and respond to an authentic,
engaging, and complex question, problem, or challenge (Buck Institute for Education,
2024).
PBL allows students to move from the theoretical or hypothetical to the practical.
Material covered in lectures and textbooks can be translated into real world experiences. PBL is
highly adaptable and can be designed to address state and common core standards (Halvorsen
and Duke, 2017). Learning-by-doing has been shown to increase knowledge retention and
enhance the acquisition of dispositions and skills across a range of disciplines (Larmer, 2018).
Students generally have positive views of classes that integrate PBL. They exhibit greater
motivation to learn as they develop a stronger connection to the course content. PBL can initiate
lifelong learning, as students gain soft skills, such as networking and teamwork, analytical skills,
and technical skills, including data analysis (Albert, 2019). It has been shown to increase
students’ literacy skills by enhancing reading and writing comprehension (Wolpert-Gawron,
2018). The affirming impact of PBL on learning outcomes extends to high-need students,
including students of color and students from lower income households (Vontz, Metcalf, and
Patrick, 2000; Halvorsen and Duke, 2017).
PBL can improve student outcomes in civics, social studies, and related fields. Evidence
suggests that students in project-based classes have greater gains in civic knowledge than those
in traditional lecture-style courses (Kingston, 2018). A higher percentage of AP students who are
taught using PBL achieve passing scores on the AP U.S. Government and Politics test than their
peers. AP students also demonstrate a deeper understanding of the content and develop more
sophisticated problem-solving skills (Parker, et al., 2013). PBL’s emphasis on authentic, active,
and collaborative learning contributes to its ability to increase students’ civic orientations and
related SEL competencies (Almulla, 2020). PBL can be a gateway to life-long civic engagement
as it provides students with an invitation to take part in their community. It conveys knowledge
of how the system works, instills an authentic desire to work for the good of society, and imparts
the proficiencies necessary for participation. Students can develop SEL competencies, such as
critical thinking, collaborative, and communication skills, that facilitate high-level civic
engagement. In addition, students’ facility with using STEM proficiencies can be advanced
through PBL, such as when they are used to research and find evidence to support a policy
position (Hanif, Wijaya, and Winaro, 2019).
At the same time, PBL is not without its critics who challenge its efficacy as a genuinely
student-driven approach and argue that PBL requires substantial resources and can be difficult to
implement in the classroom. There are concerns that PBL promotes activities at the cost of
8
covering core content knowledge. At a time when standardized tests and learning objectives have
become the norm, students in classes using PBL may not have the depth of knowledge to
perform well on these required assessments. A recurrent theme is that PBL is difficult to do well,
especially given constraints on teachers’ time and resources. PBL poses challenges to classroom
management, as students are expected to be drivers of the project while teachers act as
facilitators and monitor progress. It can be hard to keep students focused on the project.
Identifying a problem that is relevant, doable, and aligned with real-world circumstances can be
difficult. Teachers may not be able to connect with cooperative community partners to facilitate
the project. Typical issues associated with group work, such as division of labor and
disagreements among participants, may be exacerbated, especially when a project is tough to
manage or controversial (Evenddy, Gailea, and Syafrizal, 2023). Students who lack skills, such
as the ability to work collaboratively or to communicate effectively, will be disadvantaged even
if they have a superior grasp of the subject matter (Aldabbus, 2018). Assessment of projects can
be challenging, as teachers must provide students with continuous feedback while
simultaneously motivating them to do independent work. The criteria for evaluating PBL
outcomes is more complex, time-consuming, and open to subjective judgement than standardized
testing, although the potential to find common ground exists (Miller, 2012).
Project Citizen
Project Citizen encompasses best practices for PBL (see Larmer, 2018) and is designed to
address the criticisms. It is inquiry-based, as students guided by teachers identify an issue or
problem that is societally relevant or personally meaningful, such as a challenging situation
within their school. They devise a question that will frame their project. The PC curriculum is
intended to be implemented over a period of weeks as a form of sustained inquiry. Students have
opportunities for input into how the project will proceed and are encouraged to express their
ideas. They conduct research, propose alternate solutions to the problem they identified, and
work collaboratively and cooperatively to arrive at a consensus about the proposal they will
recommend. Students receive feedback as they present portfolios of their research and their
proposal to community leaders and answer questions. They are encouraged to consider
suggestions that have been offered by their teacher, peers, and stakeholders. The class reflects on
their work and the process, which helps teachers to adapt the curriculum going forward.
The PC curriculum was designed to embrace core objectives of civic education. A
fundamental goal is to provide students with knowledge and understanding of the country’s
foundational principles, democratic values, government institutions and processes, the political,
social, and economic systems, and current affairs. PC also aims to convey civic dispositions and
skills, including media literacy and critical thinking, that will prepare students to be informed
and active participants in the civic life of their communities and country. More recently, goals
associated with increasing awareness of issues of diversity, equity, inclusion, and representation
have been prioritized (EAD, 2021; Winthrop, 2020).
Studies conducted in the U.S. and in other countries have found support for PC’s ability
to convey civic orientations. An early assessment employed case studies in the U.S. that
documented PC students’ “success stories.” Students carried out projects that helped homeless
teens, got a traffic light installed at a dangerous intersection, and developed a method for
9
conducting research on the World Wide Web that was widely adopted (Tolo, 1998). The
curriculum was lauded as a “springboard” that provided American students with an entrée to
community service opportunities by providing them with the requisite dispositions and skills for
engagement (Atherton, 2000). Using a mixed method approach, a study of a middle school PC
class in California found that the program improved students’ civic literacy, increased their sense
of political efficacy, and helped to develop collaborative skills (Morgan, 2016). Research fielded
in Idaho indicated that high school students who participated in PC had higher levels of political
efficacy and a stronger sense of civic responsibility than college students who had not taken part
in the program (Fry and Bentahar, 2013). A large-scale program evaluation found that middle
and high school students who participated in PC made greater improvements in civic
development, including civic knowledge, civic discourse skills, and public policy problem-
solving skills than students in a control group. Teachers who had more experience teaching the
curriculum had better student outcomes than those new to the program. The study found that PC
had similar outcomes regardless of students’ gender, native language, and level of participation
in extra-curricular activities. Non-White students made greater gains in persuasive writing
ability, but racial/ethnic differences on other indicators were negligible (Root and Northup,
2007). Preliminary research on the PCRP has found that PC students made statistically
significant gains in civic knowledge, dispositions, skills, and SEL competencies as well as
acquiring civics-relevant STEM skills (Owen and Irion-Groth, 2020; Owen, 2023).
Evaluators have noted that the PC intervention can be adapted to a wide range of
classroom contexts which explains its widespread international use (Atherton, 2000). A
comparative study of the effectiveness of PC in Indiana (U.S.), Latvia, and Lithuania found that
the curriculum had positive, statistically significant impacts on civic knowledge, dispositions,
and skills in all three countries. Students in Lithuania gained the most civic knowledge. Students
in Lithuania and Latvia showed more interest in politics than their counterparts in Indiana. PC
students in Indiana exhibited the greatest improvement in civic skills (Vontz, Metcalf, and
Patrick, 2000). A study conducted in Taiwan employed a quasi-experimental design where
twelve teachers each instructed one class using PC and another using traditional, discipline-based
civics instruction. The findings demonstrated that PC students significantly outperformed
students in the traditional civics class in terms of their level of political interest, commitment to
the rights and responsibilities of citizenship, and development of core civic skills (Liou, 2004).
Research in Indonesia employed a quasi-experimental design to test whether PC could be used to
promote values-based education mandated by the country’s National Education System. The
study found that PC was an effective model for developing democratic character traits, including
critical thinking, positive interaction, discussion, and collaboration skills, and promoting
decision-making that is in the public interest. The character orientations developed in the context
of citizenship education carried over to the students’ daily lives (Kabatia, Irwan, and Firman,
2021).
Teacher Professional Development
Effectively employing PBL in the classroom requires that teachers have the requisite
active learning pedagogical skills. The need for quality PD for teachers implementing PC in their
classrooms has been documented in the U.S. and abroad (Owen and Irion-Groth, 2020; Owen,
2023; Ozturk, 2022; Ozturk, Rapoport, and Ozturk, 2021; Root and Northup, 2007). Teachers’
grasp of relevant content may exceed what is required for a traditional lecture and textbook
10
heavy approach given the student-driven aspect of PBL. To deal with teacher shortages in recent
years, state policymakers have been relaxing certification requirements (Will, 2022), which
notably impacts civics and social studies education. Increasingly, instructors reach the classroom
without sufficient coursework in the field (Hamilton, et al., 2020).
PC adapts project-based learning to the civics context. Teachers use interactive, student-
centered, cross-disciplinary instructional approaches that integrate independent and group work.
The PC PD program was aimed at developing teachers’ capacity to work with students on all
aspects of the curriculum. Scholars were consulted to enhance teachers’ content knowledge of
the U.S. Constitution, government institutions, and the public policy process. Mentors were
engaged to help teachers develop effective pedagogies for instructing PC and improve their
capacity to engage young people in the PC curriculum through inquiry-based and project-based
learning methods. In addition to more standard knowledge of American government and
institutions, PC teachers must be familiar with how the public policy process, interest groups,
and non-governmental organizations work. They should be able to teach students core research
skills, such as working with primary source documents, using evidence to support arguments,
and evaluating alternative hypotheses and explanations. They also are encouraged to integrate
STEM techniques in their PC classes, such as survey research and basic data analysis, so that
their students can make the connection between STEM and policymaking. At present, students in
STEM classes are far more likely to learn about the connection of science and math to civic
engagement and public policy than students in social studies courses (Ross and Fried, 2022).
PC teachers enrolled in the PCRP received 48 hours of PD sessions in the summer and
the academic year. The PD began with four- or five-day summer institutes at sites across the
country organized by the Center’s staff and their network of state coordinators. The institutes
were held in the summer before teachers would instruct PC with their students. Educators
attended shorter follow-up PD sessions during the academic year which involved follow-up by
mentors. They also participated in a professional learning community consisting of PCRP
participants, Center staff, state coordinators, and mentor teachers. They were prepared to
implement the PC curriculum intervention in their classes during the academic year.
Classroom Curriculum Intervention
The Project Citizen curriculum provides students with hands-on experience as they learn
the fundamentals of the public policymaking process, come to understand the complexities of the
American political system, and gain insights into how to monitor and influence government. A
primary objective of PC is to have students understand why citizen participation is important to
democracy. Students follow a six-step process where they identify and study one problem in
their school or community, recommend a solution in the form of a policy proposal, and present
their research and proposal as a portfolio at a simulated public hearing. Teachers and students
have access to Level 1 and Level 2 textbooks that supplement the curriculum based on the
classes’ civic knowledge base. The Center has extensive online resources, including an
interactive tool for learning how to put together a PC portfolio, scholar videos that convey
information about the policy process, links to relevant web resources, and examples of successful
projects. (See Curriculum - civiced.org.)
11
The student intervention consists of two main components: 1) at least 20 hours of
classroom instruction in PC and 2) development and presentation of a public policy portfolio.
Some of the time developing the portfolio and preparing a presentation was undertaken during
class. The amount of time spent in class was at the discretion of the teacher implementing the
instruction. The underlying process targeted by the student intervention is the acquisition of civic
knowledge, skills, and dispositions through the PC curriculum and its active, inquiry-based,
project-based learning methods. The primary student educational outcomes to be sought include
1) an increase in knowledge of the methods and procedures of governmental institutions, 2) an
increase in civic skills, 3) an increase in the development of positive civic dispositions, 4) an
increase in civics-related SEL competencies, and 4) the improvement of STEM knowledge.
METHODOLOGY
The study was guided by research questions related to the teacher PD program and the
PC student curriculum intervention. The following questions address the effectiveness of the
teacher PD program:
• To what extent does participation in the Project Citizen PD program improve teachers’
knowledge and understanding of the public policy process and civic action?
• To what extent does participation in the Project Citizen PD program improve teachers’
pedagogy?
• Are teachers more likely to incorporate active learning elements in their civics classes
because of the Project Citizen PD program?
The following research questions guided the evaluation of student outcomes:
• To what extent does Project Citizen impact students’ acquisition of civic knowledge?
Specifically, do students gain knowledge of the workings of government, the public
policy process, and civic action due to their participation in Project Citizen?
• To what extent does Project Citizen impact students’ acquisition of civic dispositions?
• To what extent does Project Citizen impact students’ acquisition of civic skills?
• To what extent does Project Citizen impact students’ acquisition of civic-related SEL
competencies?
• How is STEM integrated into the Project Citizen curriculum? To what extent does
participation in Project Citizen improve students’ comprehension of STEMs relationship
to the policymaking process?
Research Design
The study employed multi-site, school-level RCTs based on pretest/posttest surveys to
assess the impact of the PD program on middle school and high school teachers and the PC
curriculum intervention on students. The RCT design provided for randomized assignment of
12
teachers and their students to PC intervention and control groups at the middle school (grades 5-
8) and high school (grades 9-12) levels. Protocols were filed with the Georgetown University
Institutional Review Board (IRB) for the PCRP teacher and student studies. Both studies were
granted exemptions.
1
The first intervention was the teacher PD program. The primary outcomes were increased
teacher knowledge and understanding of the workings of government, public policy, and civic
action as well as improved teaching practices. Pretests were administered to both the intervention
and control group samples prior to the start of the PD program to get a baseline of teacher
knowledge and the pedagogies they typically employ in their civics, social studies, and American
government classes as well as their approach to instruction, teaching objectives, classroom
climate, self-efficacy, and professional engagement. The knowledge tests were administered
separately and were proctored by program coordinators, CERL staff, mentor teachers, and school
administrators. Posttests measuring these concepts were administered to the PC teachers after
they completed the PD program and implemented the Project Citizen curriculum in their classes.
Control group teachers took the posttest after they had finished teaching their traditional civics
class. The tests were administered and proctored online using a secure platform.
The second intervention measured the effectiveness of the Project Citizen classroom
instruction on students’ acquisition of knowledge of the methods and procedures of
governmental institutions related to public policy and their development of civic skills and
dispositions. In addition, the study analyzed the extent to which students were exposed to STEM
in the Project Citizen curriculum. Students took pretests at the outset of the Project Citizen or
their traditional civics class and took posttests at the conclusion of their class. The tests were
administered during class time by their teacher using a secure online platform. Teachers were
instructed to use the methods established by their institution for performing online testing.
The quantitative analysis was supplemented with qualitative data from open-ended
survey items, interviews with study subjects, and observations of the PD sessions and student
showcases. CERL researchers observed PD sessions in both online and in-person formats over
the course of the study. PC’s culminating activity is a showcase where students present their
portfolios to community stakeholders. During the pandemic, many of the showcases were held
virtually, making it possible for CERL team members to attend.
Sampling
The population of interest in the study is middle and high school students who take a
class where the teacher has implemented the Project Citizen curriculum. A hierarchical design
was employed where schools (clusters) were randomly assigned to treatment and control groups.
The school was the appropriate unit of randomization as PC can be implemented as a school-
based project that involves more than one teacher and/or class. The PC program is suitable for
students in most school settings, so the inclusion criteria were broad. The schools in the sample
were public and private middle and high schools serving grades 5 through 12 throughout the
United States that are representative of the schools in their district. All qualified teachers
1
Teacher Study IRB ID: STUDY00002719. Student Study IRB ID: STUDY00002026.
13
instructing civics, social studies, American government, or American history courses within a
school were invited to enroll in the study. All students of teachers in the PC and control groups
were recruited for the research.
Schools and their teachers were recruited for each cohort through the Center’s network of
state coordinator and mentor teachers via personal outreach to their extensive contacts,
advertising in education-related publications, and posting on social media. Qualified applicants
were accredited public or private middle and high school teachers of civics, social studies,
American government, and American history courses in which the PC program could be
incorporated in the ensuing academic year. Schools nationwide were recruited for participation
in the PCRP. All teachers within a school who qualified were invited to take part in the study. In
many cases, a school had only one qualified instructor. Applications were submitted to the
Center where they were reviewed to determine if they met the program and study criteria.
Schools/teachers were then randomly assigned to the PC intervention and control groups. The
students in the study represented diverse populations, including students at risk of failure. While
there was no explicit intention to target teachers from schools serving high-need and
economically disadvantaged students to the program, and this was not a prerequisite for
participation, a majority of the participating teachers served these students.
A total of 70 schools were recruited for each study cohort; 35 schools were randomly
assigned to the PC group and 35 to the control group. A total of 210 schools were enrolled over
the three years and 180 completed the study. A total of 237 teachers enrolled in the study, and
196 stayed. A combined total of 6,521 students enrolled in the study and 5,415 stayed. The
pandemic presented challenges for keeping schools, teachers, and students enrolled in the PRCP.
The common reasons for teachers leaving a study, such as changing positions, new teaching
assignments, switching schools, curriculum shifts, and personal matters, were augmented with
pandemic-related causes. Teachers had difficulty adapting to virtual and hybrid instruction
during the first two cohorts as well as the return to fully in-person instruction in cohort 3. Some
teachers who felt overwhelmed took leave mid-year, and their classes were taught by substitutes
who were not enrolled in the research. Other teachers left the profession entirely. Students
struggled with online learning, and some became completely disengaged. Close to 90% of
students in the sample were from schools that served large numbers of high-need students, a
population that was especially vulnerable during the pandemic. Steps were taken to curb attrition
that had some success, but the pandemic effects on study participation were apparent. (Details on
steps taken to curtail attrition and study attrition for schools, teachers, and students are provided
in Appendix A: Methodology.)
Statistical Methodology
Statistical analyses were conducted on the pretest/posttest teacher and student outcome
data to determine if there were statistically significant changes that were aligned with the PC
intervention. Difference of means tests (paired samples t-tests) were performed to identify within
group shifts in the pretest and posttest outcome measures for the PC and control group teachers
and students for each cohort. The middle and high school student samples for each cohort were
analyzed separately. The pretest and posttest mean scores and standard deviations, the difference
of pretest/posttest means and significance test, the percentage change in pretest/posttest means,
14
the effect size (Hedge’s g), the improvement index, and the pretest/posttest correlation and
significance test were reported. What Works Clearinghouse’s (WWC) improvement index was
computed from Hedge’s g. It represents the average expected change in the percentile rank if an
average comparison group member receives the intervention. In other words, it is the difference
in percentile ranks for an average intervention versus comparison group member.
2
Hierarchical linear models were estimated using analysis of covariance (ANCOVA) to
determine if there were statistically significant differences in the adjusted posttest scores of the
intervention and control group teachers and students. ANCOVA was an appropriate model for
this analysis as it adjusts for non-equivalence in intervention and control group scores at
baseline. For the teacher analysis, the dependent variables were posttest scores. Pretest scores
were entered as a covariate. Intervention/control group was treated as a fixed factor. Separate
ANCOVA models were estimated for middle and high school students. Posttest outcome
measures were the dependent variables. Pretest outcome measures and a variable coded for the
students’ school were entered as covariates. Intervention/control group was a fixed factor. Effect
size for the difference of adjusted posttest means between the PC and control groups tests was
estimated by Hedges’ g. In the ANCOVA models, the adjusted means and unadjusted standard
deviations were used in computing the effect sizes.
3
WWC’s improvement index was calculated
from Hedge’s g.
The interpretation of effect sizes in education research has become a matter of debate. A
common approach adopts Cohen’s (1988) benchmarks which were based on a small number of
controlled lab experiments in social psychology conducted in the 1960s that primarily used
undergraduate subjects. Kraft argues that “effects that are small by Cohen’s standards are large
relative to the impacts of most field-based interventions” (2020: 241). Meta-analyses of more
recent well-designed field experiments in education research have found that effect sizes with
potentially important consequences are interpreted as having no or small effects using Cohen’s
guidelines. Kraft suggests that the magnitude of effect sizes depends on what and how outcomes
are measured. Fixed benchmarks, while easy to use, cannot account for differences in study
features and outcomes (Kraft, 2020). Comparable studies to the present research use
pretest/posttest survey methods to examine student civic learning. Findings for students’ civic
knowledge outcomes that are not overly aligned with the intervention typically have larger
effects than studies of students’ civic dispositions and skills. Thus, it is prudent not to dismiss
small, significant effect sizes related to these outcomes.
The percentage difference between pretest and posttest means was reported for the paired
samples t-tests and for the adjusted posttest means based on the ANCOVA analysis for the PC
and control groups. The percentage change is a useful statistic as it is easily interpreted and
accessible to a wide audience. However, as a ratio it can be misleading, especially if the initial
value is near zero (Curran-Everett and Williams, 2015), which was rarely the case in this study.
The percentage difference supplements other measures of change that are reported.
2
Institute for Education Sciences. 2022. What Works Clearinghouse, Procedures and Standards Handbook, version
5, U.S. Department of Education, pp. 186-187.
3
Institute for Education Sciences, 2022. What Works Clearinghouse, Procedures and Standards Handbook, version
5, pp. 135-36.
15
THE IMPACT OF THE COVID-19 PANDEMIC
The three cohorts of the PCRP spanned the course of the COVID-19 pandemic. The
cohort 1 study was conducted in 2020-21, cohort 2 in 2021-22, and cohort 3 in 2022-23. In each
year, the PD program began with a summer institute and the curriculum was taught during the
following academic year during which teachers attended follow-up PD sessions. The method of
delivering the PD and the curriculum shifted over the course of the study.
As the PCRP was undertaken during the pandemic, the study offers a unique opportunity
to track the implementation, outcomes, and possibilities of a civics program that relies heavily on
active learning during periods of societal upheaval and shifting conditions in the educational
environment. The unique circumstances of implementing PC during the pandemic presented
challenges. Teachers and students had to adapt to virtual and hybrid instruction. Disruptions to
program implementation occurred due to COVID-related absences of teachers and students.
Some teachers left the profession midcourse, requiring classes to be combined or substitutes to
be brought in for the remainder of the school year. Even as teachers and students returned to the
classroom, shifting modes of delivering the curriculum coupled with the lingering deleterious
effects of the pandemic on student learning (Kuhfeld, et al., 2022; Kwakye and Kibort-Crocker,
2021), engagement (Hutchinson and Moore, 2021), and behavior (Lambert, 2022, Shen-Berro,
2023) posed novel challenges to implementing the PC curriculum.
The Center adapted the PCRP to meet the pandemic-created conditions that were
unanticipated when the program was planned. The teacher PD program took place online in
cohort 1 and combined in-person and virtual formats in cohorts 2 and 3. Novel active learning
strategies for implementing PC in the virtual classroom were developed that have lasting value.
Projects were modified so that they addressed problems that could be tackled and were
meaningful in the pandemic environment. Students’ projects focused on issues such as the
proliferation of pet adoptions, increasing substance abuse among young people, food insecurity,
a lack of access to resources for those living in poverty, and mental health problems.
The pandemic seriously disrupted the normal course of secondary school instruction
during the first study cohort (2020-21), as most schools rapidly transitioned to virtual learning.
Despite the challenges, the Center proceeded with the first year of the PCRP in an all-virtual
format. The majority of teachers (69%) implemented the PC curriculum intervention with their
students virtually. Twenty-eight percent of teachers met with their students partially online and
partially in-person or in a hybrid learning environment. Only 2% of teachers implemented the PC
curriculum entirely in-person. Students presented their project portfolios—the culminating PC
activity—primarily online to panels of community and government leaders. Teachers continued
to deal with the impact of the COVID-19 pandemic during cohort 2 (2021-22). The instructional
environment was characterized by a constant state of uncertainty, as teachers were still adapting
to the conditions of the pandemic. Almost all (97%) of the teachers participating in the PCRP
had taught their classes virtually for at least part of the previous academic year. While most
schools started the new academic year with the intention of holding classes in-person, only 36%
of PCRP teachers taught their classes in-person for the duration. Sixty-four percent of teachers
held classes virtually for at least part of the time. By cohort 3 (2022-23), all of the educators in
16
the study were teaching in person, although some experienced short periods of virtual
instructional continuity during heavy outbreaks of the virus.
While often an effective approach for imparting civic knowledge, skills, and dispositions,
PBL can be challenging to execute under the best of circumstances. As the PCRP was
implemented during the height of the COVID-19 pandemic, each cohort of teachers and students
faced a shifting set of challenges. The classroom environment—virtual, in-person, or in
between—was unsettled. Quarantines and school closures kept teachers and students out of the
classroom, as absenteeism rose to astronomical levels. Valuable instructional time was lost.
Teachers had to adjust their lesson plans and pedagogical approaches rapidly. Students faced
disruptions in their schooling and mental health challenges that adversely affected their learning.
The pandemic continues to have substantial, persistent negative impacts on students’ academic
performance and classroom climate to this day (Kuhfeld, et al., 2022).
The PCRP was not immune to these negative circumstances and their effects. Teachers in
the PC and control conditions dropped out of the study, indicating that they were having
difficulty keeping up with even the basics of day-to-day classroom instruction. Some teachers in
the study left the profession. Instructors struggled to implement the curriculum virtually,
especially when students turned off their videos, played on their phones, or got up and walked
away from the computer. Managing students doing group work in virtual breakout rooms was a
challenge. Presenting the project to stakeholders using a video conferencing platform was
difficult for some students, while others were more comfortable expressing themselves in this
environment.
PC teachers described their experience implementing the curriculum during the pandemic
and the ways that they adapted.
This was a hard year for teaching, so trying to implement project-based learning in a
hybrid setting was difficult. We also had limited instructional time, so adding to the
already heavy load seemed impossible. I implemented PC as a club, which worked well,
but I had less students participate because it was outside of school hours.
The pandemic forced all if not most of my days to use Google Meets. At the start of the
year, most students were present. Motivation waned towards the end of the quarter and
semester. Students would sometimes show up, turn the computer screen on but not
participate .... probably "ghosting." Project Citizen helped motivation and enabled the
"doers" to take control of the motivation problem as I asked the students to do...Project
based learning!
Teaching during the pandemic presented numerous challenges this year. Teaching online
all year and hybrid for part of the year made it difficult to conduct parts of the project.
Not having the face-to-face conversations and having different numbers of students per
class in-seat vs. online made discussion difficult. I was forced to move course content to
the forefront of my schedule, which made it difficult to work in some elements of the
project. I find Project Citizen to be very engaging and look forward to trying it next year
under more conventional circumstances.
17
SCHOOL CHARACTERISTICS
School characteristics consisting of public or private school, school type, geographic
location, Title I school designation, and type of high-need students served were recorded. (See
Table 1.) The type of school consisted of alternative, charter, magnet, parochial, religious, and
technical schools. The location of a school in a rural, suburban, or urban geographic area was
recorded. Title I schools serve a high percentage of low-income students who are living below
the national poverty line and receive federal funding (NCES, 2019). Schools serving high-need
students had 30% or more of students who receive free or reduced cost lunch, minority students,
students living in poverty, students far below grade level, English learners (ELL), students with
disabilities (SWD), students unhoused or in foster care, student served by rural local education
agencies (RELA), disconnected or migrant youth, and incarcerated students.
Most of the teachers in all three cohorts taught in public schools. In cohort 1, 96% of PC
and 95% of control teachers worked in public schools. Eighty-nine percent of PC teachers in
cohort 2 were employed in public schools compared to 96% of the control group. In cohort 3,
96% of PC teachers and 91% of control group teachers worked in public schools. A small
percentage of teachers in each cohort worked in alternative, charter, magnet, parochial, religious,
and technical schools.
A higher percentage of PC teachers (25%) in cohort 1 taught in rural schools than control
group teachers (16%). More control group schools (37%) than PC schools (24%) were located in
urban areas. The pattern was reversed in cohort 2, as 27% of PC schools were in rural areas
compared to 33% of control group schools. Twenty-eight percent of PC schools were in urban
areas versus 15% of control group schools. In cohort 3, the trend was similar to cohort 1 with a
higher percentage of PC schools (21%) located in rural areas compared to control group schools
(13%). Thirty-eight percent of PC schools were in urban areas as were 55% of control group
schools. The highest percentage of schools (between 40% and 52%) were in suburban areas with
the exception of control group schools in cohort 3.
A majority of teachers in the study worked in schools that served high-need students.
Half of the PC and control teachers in all three cohorts were employed in Title I schools. In
cohort 3, 61% of the control group teachers served in Title I institutions. Sixty percent or more of
schools had high percentages of students receiving free or reduced cost lunches. The highest
percentage was in cohort 3, with 76% of PC schools and 83% of control group schools serving
this student population. Fifty percent or more of PC and control group schools had large numbers
of minority students across cohorts, except for the control group in cohort 2 (25%). In cohort 1,
36% of PC schools and 30% of control group schools had high numbers of students far below
grade level, compared to 19% for both groups in cohort 2, and 36% of PC schools and 52% of
control group schools in cohort 3. Thirty percent of PC schools and 23% of control group
schools in cohort 1 and 25% of PC schools and 16% of control group schools in cohort 2 had
large numbers of ELLs. The number of schools serving high percentages of ELLs was greatest in
cohort 3 for both PC schools (42%) and control group schools (57%). Small numbers of schools
had high percentages of unhoused students/students in foster care, students served by RLEA, and
disconnected or migrant youth. A control group school in cohort 2 worked with incarcerated
students.
18
Table 1
School Characteristics
Characteristics
Cohort 1
Cohort 2
Cohort 3
Project
Citizen
Control
Project
Citizen
Control
Project
Citizen
Control
Public School
96%
95%
89%
96%
96%
91%
Private School
4%
5%
11%
4%
4%
9%
School Type
Alternative
5%
2%
1%
5%
4%
2%
Charter
8%
12%
4%
0%
8%
0%
Magnet
9%
0%
3%
0%
9%
16%
Parochial
0%
6%
4%
0%
1%
3%
Religious
1%
0%
5%
3%
2%
2%
Technical
3%
4%
1%
0%
1%
0%
Geographic Area
Rural
25%
16%
27%
33%
21%
13%
Suburban
51%
47%
45%
52%
41%
32%
Urban
24%
37%
28%
15%
38%
49%
55 %
Title I School
48%
49%
51%
48%
61%
School Serves High Percentage
Free or Reduced Cost Lunch
Recipients
67%
59%
64%
66%
76%
83%
Minority Students
56%
50%
55%
25%
66%
61%
Students Living in Poverty
52%
34%
44%
44%
60%
52%
Students Far Below Grade
Level
36%
30%
19%
19%
36%
52%
English Learners
30%
23%
25%
16%
42%
57%
Students with Disabilities
25%
23%
23%
13%
30%
26%
Students Unhoused or in
Foster Care
13%
11%
8%
19%
17%
4%
Students Served by RLEA
7%
7%
7%
6%
4%
0%
Disconnected/Migrant Youth
6%
5%
4%
3%
11%
0%
Incarcerated Students
0%
0%
0%
3%
0%
0%
19
TEACHER CHARACTERISTICS
The profiles of teachers in the PC and control groups were similar. (See Table 2.) In
general, teachers in the study had earned advanced degrees and had taught civics courses for ten
years or more. Most were civics instructors while a smaller number primarily taught American
history. (See Table 2.)
A majority of teachers in the study held advanced degrees (master’s, law degree,
doctorate). Twenty-five percent of PC teachers held a bachelor’s degree in cohort 1compared to
32% in the control group. Seventy-five percent of PC teachers had an advanced degree as did
68% of the control group educators. In cohort 2, 20% of the PC teachers and 18% of the control
group teachers had bachelor’s degrees, while 80% or more of both groups had advanced degrees.
A somewhat higher percentage of the cohort 3 control group teachers (91%) had earned
advanced degrees compared to the PC group (74%). A quarter of the PC teachers had a
bachelor’s degree compared to 9% of the control group.
Overall, teachers in the PCRP study had taught civics, social studies, or American
government for at least ten years. In cohort 1, the PC teachers had taught civics for an average of
11.3 years compared to 14.5 years for the control group teachers. The PC group educators taught
civics at their present school for 7.3 years versus 8 years for the control group. The cohort 2 PC
teachers had been civics instructors for an average of 12 years compared to 13.1 years for the
control group teachers. The PC teachers had been at their present school for an average of 7.6
years while the control group teachers were at their present school for 9.6 years. In cohort 3, the
PC teachers had taught civics for an average of 10.1 years versus 10.5 years for the control
group. The control group teachers had been at their present school somewhat longer (7.4 years)
than the PC group teachers (5.3 years). Almost all of the teachers in the study taught middle and
high school students.
Most of the teachers in the study identified their primary area of instruction as civics,
social studies, American government, and American history taught through the lens of civics.
These teachers constituted eighty percent or more of PC teachers in all three cohorts. Eighty
percent of control group teachers in cohort 1, 85% in cohort 2, and 69% in cohort 3 were civics
instructors. A smaller percentage identified primarily as American history teachers.
A small percentage of teachers in each cohort taught special education students, ELLs,
Native American students, and adult education students. Special education teachers comprised
6% of PC teachers and 7% of control teachers in cohort 1, 8% of PC teachers and 3% of control
teachers in cohort 2, and 11% of PC teachers and no control teachers in cohort 3. Three percent
of PC and 2% of control group educators taught Native American students in cohort 1, compared
to 8% of PC and no control group teachers in cohort 2, and 2% of PC and 4% of control group
instructors in cohort 3. Advanced placement (AP) civics, social studies, and American history
teachers were well represented in the sample. However, the classes that they taught for the study
were not AP courses. In cohort 1, 28% of PC teachers and 12% of control group teachers were
AP instructors as were 26% of PC and 41% of control group teachers in cohort 2, and 30% of PC
and 13% of control group educators in cohort 3.
20
Teachers participating in the study primarily taught middle school (grades 5-8) and high
school (grades 9-12). A small percentage of elementary school (grades 1-4) teachers took part in
the program. In Cohort 1, 5% of PC participants taught elementary school, 43% taught middle
school, and 52% taught high school. Two percent of control group teachers were elementary
educators, 51% taught middle school, and 46% taught high school. In Cohort 2, the PC group
was comprised of 7% elementary school teachers, 38% middle school teachers, and 55% high
school teachers. Four percent of the control group taught in elementary school, 41% in middle
school, and 56% in high school. Among the cohort 3 PC teachers, 4% instructed elementary
schoolers, 47% middle schoolers, and 55% high schoolers. Fifty-two percent of control group
teachers instructed middle school students and 48% taught high school students. One teacher’s
PC class consisted of both elementary and middle school students while another PC class was
comprised of middle and high school students.
Table 2
Teacher Characteristics
Characteristics
Cohort 1
Cohort 2
Cohort 3
Project
Citizen
Control
Project
Citizen
Control
Project
Citizen
Control
Highest Degree Earned
Bachelor’s
25%
32%
20%
18%
26%
9%
Master’s
67%
55%
79%
71%
62%
91%
Law
3%
7%
1%
4%
0%
0%
Doctoral
4%
5%
0%
7%
6%
0%
Average Years Teaching
Civics
11.3 yrs
14.5 yrs
12 yrs
13.1 yrs
10.1 yrs
10.5 yrs
Civics at Present School
7.3 yrs
8 yrs
7.6 yrs
9.6 yrs
5.3 yrs
7.4 yrs
Subject
Civics
82%
80%
80%
85%
79%
69%
History
18%
20%
20%
15%
21%
31%
Teacher of
Special Education Students
6%
7%
8%
3%
11%
0%
Native American Students
3%
2%
8%
0%
2%
4%
ELL/ESL
6%
9%
12%
6%
13%
9%
Adult Education
2%
2%
0%
3%
8%
4%
AP Teacher
28%
12%
26%
41%
30%
13%
Grade Level*
Elementary School
5%
2%
7%
4%
4%
0%
Middle School
43%
51%
38%
41%
47%
52%
High School
52%
46%
55%
56%
55%
48%
21
STUDENT CHARACTERISTICS
Students’ demographic characteristics were similar for the PC and control groups across
the three study years. (See Table 3.) In cohort 1, the PC intervention group consisted of 53%
female, 45% male, and 2% gender non-binary students. The control group students included a
higher percentage of males (55%) than females (44%), with 1% identifying as non-binary. In
cohort 2, the PC group consisted of 46% male, 50% female, and 4% non-binary students. The
control group had 49% male, 46% female, and 5% non-binary participants. The PC group in
cohort 3 was composed of 49% female, 47% male, and 4% non-binary students. The control
group consisted of 51% male, 45% female, and 4% non-binary students.
The students participating in the PCRP study were racially and ethnically diverse. (See
Table #.) In cohort 1, the racial/ethnic composition of the PC and control groups was similar.
Eight percent of students identified as Asian American Pacific Islander (AAPI), 14% as
Black/African American, 17% as Latine, 49% as White/Caucasian, and 12.4% as multiracial.
The cohort 2 students in the PC group were 6% AAPI, 11% Black, 23% Latine, 49% White, and
11% multiracial. In the control group, 12% of students identified as AAPI, 5% as Black, 18% as
Latine, 52% as White, and 13% as multiracial. In the cohort 3 PC group, 11% of the students
were AAPIs, 9% were Black, 28% were White, and 15% were multiracial. The control group
included 7% AAPI, 9% Black, 31% Latine, 39% White, and 14% multiracial students.
Table 3
Student Characteristics
Characteristics
Cohort 1
Cohort 2
Cohort 3
Project
Citizen
Control
Project
Citizen
Control
Project
Citizen
Control
Gender
Male
53%
44%
46%
49%
49%
51%
Female
45%
55%
50%
46%
47%
45%
Non-Binary/Other Gender
2%
1%
4%
5%
4%
4%
Race
AAPI
8%
8%
6%
12%
11%
7%
Black/African American
12%
16%
11%
5%
9%
9%
Latine
18%
16%
23%
18%
28%
31%
White/Caucasian
51%
47%
47%
52%
37%
39%
Multiple Races
11%
14%
11%
13%
15%
14%
22
High-Need Students
While the PCRP did not intentionally recruit schools serving high-need students, the
study included a sufficiently large number of these students to allow for subgroup analyses.
High-need students are those at risk of educational failure, underserved, and/or in need of special
assistance and support. These students often receive substandard civic education or are denied
opportunities for civic learning entirely. As the K-12 student population has become increasingly
diverse, disparities in civic education have persisted, even grown (Kahne and Middaugh, 2008;
Fitzgerald, et al., 2021; U.S. Department of Education, 2023). Parts of this report include
analyses of civic outcomes for students of color (SOLs), English language learners (ELLs),
students with disabilities (SWDs), students performing below grade level, and students living in
poverty for the PC group. Students of color are defined as students who identify as Black or
African American, Latine, Chicanx, Asian, South Asian, Pacific Islander (AAPI), Middle
Eastern, Native American, and multiracial (Institute of Education Sciences, 2023). ELLs are
students whose native language is something other than English or who lack proficiency in
English and are eligible to participate in language assistance programs (U.S. Department of
Education, 2016). They are a diverse group of students who have different language abilities and
backgrounds. Among their ranks are refugees, migrants, students with interrupted education,
internationally adopted students, and unaccompanied minors (Colorin Colorado, 2019). Students
with disabilities, as defined under the Individuals with Disabilities Education Act, are students
with “a disability that adversely affects academic performance and are in need of special
education and related services” (IDEA, 2018). Students with disabilities have unique learning
needs and require specially designed instruction. The range of disabilities that can affect
students’ learning ability includes intellectual disabilities, speech or language impairment,
hearing impairment, visual impairment, serious emotional disturbance, traumatic brain injury,
orthopedic impairments, autism spectrum disorder, and developmental delay. Students living in
poverty were identified as those qualifying for free or reduced cost meals. Students performing
far below grade level have not mastered the skills and concepts at their expected level of
difficulty as measured by formal assessments and district or state standards. Classification of
students in the study into these categories was based on teachers’ reports of their classroom
composition. (The number of students in the PCRP in each category by cohort and grade level
appears in Table 4.)
Table 4
Number of High-Need Students in PC Sample
Cohort 1
Cohort 2
Cohort 3
Middle
School
High
School
Middle
School
High
School
Middle
School
High
School
Students of Color
101
463
232
272
392
258
English Learners
93
246
100
132
273
217
Students w/ Disabilities
64
193
80
144
214
94
Below Grade Level
98
319
156
137
335
133
Living in Poverty
131
475
232
385
492
360
23
CIVIC KNOWLEDGE
A focal outcome of the PCRP for teachers and students was the acquisition of civic
content knowledge. Civic knowledge encompasses a vast amount of factual information pertinent
to the principles of democracy, the Constitution and foundational documents, institutions of
government, political processes, public policies, laws, and norms as well as current events.
Knowledgeable citizens understand their role in a democratic polity, know their rights and
responsibilities in society, and are aware of America’s place in the world (Branson and Quigley,
1998; Van Camp and Baugh, 2016). People possessing greater civic knowledge tend to be
supportive of democratic values, such as liberty, equality, and political tolerance, and are more
politically efficacious. They have the confidence and ability to stake a position in the
marketplace of political ideas (Finkel and Ernst, 2005; Galston, 2004; Brody, 1994; Youniss,
2011). Knowledge is fundamental to the development of civic competence. It contributes to the
establishment of a civic identity that disposes individuals toward engagement in political life.
The connection between knowledge and engagement has been used to justify its prominence in
civics instruction. The consistent finding of this association feeds the argument that knowledge is
a building block, if not a necessary precondition, for action (Delli Carpini and Keeter, 1996;
Niemi and Junn, 1998; Galston, 2004; Milner, 2010; Campbell, 2006; Van Camp and Baugh,
2016).
Knowledge has been described as the “bedrock” (Hoffman, 2019) or “basic currency” of
democracy (Delli Carpini and Keeter, 1996). The notion of the “informed citizen” is a central
attribute of a “good citizen.” Students’ ability to access, acquire, and evaluate political
information has become an increasingly pressing issue in the age of digital media. Generational
differences in news consumption suggest that young people are less discerning about the
information they consume and are less likely to retain quality political knowledge than older
citizens (Kleinberg and Lau, 2019). The rise of misinformation has distorted the public’s beliefs
about government and politics (Jerrit and Zhao, 2020). At the same time, people have become
more confident in their civic knowledge, even when it is blatantly erroneous (Graham, 2018).
Civic learning provides students with a strong grounding in core knowledge and has been
suggested as an antidote to these trends (Winthrop, 2020). This requires that teachers have
sufficient command of civic content knowledge.
Teachers’ Civic Knowledge
Teachers reported that their participation in the PC curriculum and PD greatly advanced
their own civic content knowledge. They had moderate to large statistically significant
knowledge gains across all three cohorts. Teachers’ posttest civic knowledge scores were
significantly higher than the scores of the control group. Teachers described PC as an enjoyable
experience that contributed to many areas of personal growth. One PC teacher explains, “I have
gained so much information myself. This was a learning process for me as well as my students. I
look forward to teaching Project Citizen next year.” Teachers highlighted PC’s focus on
government and public policy which advanced their own knowledge. One teacher recalled,
“Learning about public policy was very helpful. I didn't really understand it before. I feel better
able to explain what it is. I enjoyed hearing experts in government.”
24
Measurement
Teachers’ civic knowledge was measured by an index consisting of 46 multiple choice
questions. (See Appendix C.) The items tapped teachers’ general knowledge of the public policy
process, federalism, branches of the U.S. government, government departments involved in the
policy process, interest groups, and nongovernmental organizations. While these content areas
are relevant to the PC curriculum, the items were not overly aligned with the intervention. The
items were derived from questions used in established civic knowledge inventories, such as the
AP test, the National Assessment of Educational Progress (NAEP) Civics Assessment, and the
evaluation of the James Madison Legacy Project (JMLP) (Owen, Hartzell, and Sanchez, 2020).
Each multiple-choice item had four answer choices and the option of responding “I don’t know.”
One point was awarded for a correct answer; no points were given for incorrect or “don’t know”
responses. The 46 items were combined to form an additive civic knowledge index. All items
composing the index have known reliability that meets WWC standards. The pretest and posttest
knowledge index reliabilities (Cronbach’s α) exceeded .870 for all three cohorts. (See Table 5.)
Table 5
Teacher Knowledge Index Range and Reliability
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
0-46
.91
.88
Cohort 2
0-46
.87
.89
Cohort 3
0-46
.87
.91
25
Analysis
Difference of means tests (paired-sample t-tests) were performed for the PC and control
group samples. (See Table 6.) Teachers who received the PC PD had significantly greater gains
in civic knowledge than those in the control group in all three cohorts. The pretest/posttest
improvement in the PC groups’ scores increased with each successive cohort. The percentage
change was 8% for cohort 1, 15% for cohort 2, and 21% for cohort 3. The effect sizes for the PC
group were .44, .95, and 1.07, respectively. The improvement index scores were +17 percentile
points for cohort 1, +32 for cohort 2, and +41 for cohort 3. The difference in pretest/posttest
knowledge scores was small and nonsignificant for the control group across the board.
Table 6
Teachers’ Knowledge by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
27.71
27.56
26.47
28.92
25.02
25.63
Pretest SD
7.48
6.10
8.38
5.90
7.83
8.00
Posttest 𝜒
29.93
27.86
30.41
30.00
30.21
24.79
Posttest SD
5.75
6.18
6.49
5.47
5.53
8.82
𝜒 Difference
2.18
.30
3.94
1.07
5.18
-.85
Sign. Difference
.00
NS
.00
NS
.00
NS
Percentage Change
8%
1%
15%
4%
21%
-3%
Effect Size
.44
.08
.95
.26
1.07
-.14
Improvement Index
+17
+3
+32
+10
+41
-6
Pre/Post Correlation
.74
.82
.87
.74
.80
.78
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
36
27
34
27
39
33
ANCOVA, which adjusts for non-equivalence in intervention and control group
knowledge scores at baseline, was performed to estimate the difference in PC and control group
posttest scores. Posttest knowledge was the dependent variable, PC/control group condition was
a fixed factor, and pretest knowledge was a covariate. The PC teachers’ posttest knowledge
scores were significantly higher than the control groups’ scores for all three cohorts. The
difference in PC/control group scores was greatest for cohort 3 at 19%, compared to 7% for
cohort 1 and 8% for cohort 2. The effect sizes (Hedge’s g) were moderate for the first (.33) and
second (.41) cohorts, and large (.80) for the third cohort. The improvement index scores were
+13, +16, and +29 percentile points. (See Appendix B, Table B1.)
Teachers’ Instructional Objectives: Civic Knowledge
A fundamental goal of the PCRP was for teachers to convey knowledge and
understanding of the country’s foundational principles, democratic values, government
institutions and processes, the political, social, and economic systems, current affairs, and public
26
policymaking to their students. Teachers recorded how much emphasis they placed on six core
dimensions of civic knowledge in their classes. Conveying civic knowledge was a top priority
for teachers from the outset of the study. The slight increases in the PC teachers placed on core
civic knowledge from pretest to posttest were not statistically significant. The one exception was
the significant decline in the control group’s emphasis on these aspects of civic knowledge in
cohort 2.
Measurement
Teachers were asked questions related to their embrace of civic knowledge objectives on
the pretest and posttest. Six items addressed this objective: 1) educating students about core
democratic principles as set forth in the Declaration of Independence and U.S. Constitution, 2)
educating students about government and how it works, 3) teaching students about current
events, 4) teaching students about the electoral process, 5) teaching students about the economic
system, and 6) educating students about the relationship of the United States to other nations and
world affairs. Teachers were asked how much they emphasized these goals in their classroom 1
not at all, 2 very little, 3 somewhat, 4 a great deal. These items were combined to form an
additive civic knowledge instructional objectives index that ranged from 1 (very little emphasis)
to 18 (a great deal of emphasis). The reliability (Cronbach’s α) of the pretest/posttest indexes for
the three cohorts was acceptable. (See Table 7.)
Table 7
Civic Knowledge Objectives Index Range and Reliability
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
1-18
.82
.83
Cohort 2
1-18
.80
.80
Cohort 3
1-18
.88
.90
Analysis
Teachers in the PC and control groups began with average scores indicating that they
placed a strong emphasis on imparting civic knowledge at the outset which remained consistent
over the course of the study. (See Table 8.) For the most part, the mean scores changed little from
pretest to posttest, and the mean differences were not statistically significant. The one exception
was for the control group in cohort 2, where the decline in the average score from pretest to
posttest was -.91 and statistically significant. The emphasis on civic knowledge decreased by
6%, the effect size (Hedge’s g) was -.55, and the improvement index was -20 percentile points.
ANCOVA models comparing the difference between the adjusted PC and control group mean
posttest scores were estimated for each cohort. The differences in the adjusted posttest mean
scores were small and nonsignificant across the board. (See Appendix B, Table B2.)
27
Table 8
Civic Knowledge Objectives Index by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest
14.11
14.33
14.46
15.68
16.18
14.67
Pretest SD
2.90
3.54
2.60
1.80
3.51
4.35
Posttest
14.28
14.57
14.92
14.77
16.43
14.42
Posttest SD
2.77
3.52
3.31
2.02
3.26
4.83
Difference
.16
.24
.46
-.91
.24
-.25
Sign. Difference
NS
NS
NS
.00
NS
NS
Percentage Change
1%
2%
3%
-6%
2%
-2%
Effect Size
.07
.12
.17
-.55
.10
-.06
Improvement Index
+3
+5
+6
-20
+4
-2
Pre/Post Correlation
.60
.85
.59
.66
.75
.65
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
36
27
34
27
39
33
Teachers’ Efficacy in Conveying Civic Knowledge
Teachers’ sense of self-efficacy, their belief and confidence in their ability to effectively
handle the requirements of the position, has been linked to students’ achievement (Kim and Hee,
2018). Civics and social studies teachers who have greater efficacy tend to have better command
over content and use more creative pedagogies to effectively convey knowledge to their classes
(Calking, Yoder, and Wiens, 2021). While PC teachers tended to feel efficacious at the outset of
the study, their sense of being able to successfully provide knowledge to their students increased
after participating in the program for every cohort.
Measurement
Teachers were asked how much they could do to convey content knowledge of American
government to their students on the pretest and posttest. The response categories were 1 very
little, 2 some, and 3 a great deal.
Analysis
Most PC teachers—over two-thirds—felt that they were very effective in conveying
content knowledge to students. (See Table 9.) There were notable increases from pretest to
posttest in cohort 1 (from 76% to 95%) and cohort 3 (from 75% to 85%); the change in cohort 2
was small (from 84% to 87%). The percentage of control group teachers who indicated that they
felt very effective in conveying content knowledge decreased slightly in cohorts 1 and 2. Only
half of the control group teachers in cohort 3 indicated that they had a great deal faith in their
28
ability to convey civic content knowledge. The percentage increased to 62% on the posttest,
which was still well below the norm.
Table 9
Teachers’ Efficacy in Conveying Civic Knowledge
Cohort 1
Cohort 2
Cohort 3
Pretest
Posttest
Pretest
Posttest
Pretest
Posttest
Project Citizen
76%
95%
84%
87%
75%
85%
Control Group
89%
82%
93%
89%
51%
62%
Students’ Civic Knowledge
A primary objective of the PCRP was to advance students’ civic knowledge through
project-based learning. The evidence from this study is consistent with findings of prior research
indicating that students in classes that use project-based approaches have greater gains in civic
knowledge than those who take a traditional civics or social studies course. The mean post-
program civic knowledge scores of middle and high school students who participated in PC were
higher than those of students in standard classes. The results were statistically significant (p≤.01)
across the board. Reflecting on their class, PC students appreciated the opportunity to experience
something useful about government and politics. A middle school student stated, “Project Citizen
has taught me so much about the government! I never realized how important our government is
until I got an in-depth lesson about it and realized how strongly we rely on the government.” A
high school student reflected: “I have thoroughly enjoyed partaking in Project Citizen. I learned
a great deal about the political systems and the inner mechanisms of our government. I can
proudly say, thanks to Project Citizen, I will be an active and knowledgeable participant in my
community.”
Measurement
Students’ civic knowledge was based on twenty multiple choice items relating to general
knowledge of the public policy process, federalism, branches of the U.S. government, agencies
and departments involved in the policy process, interest groups, and nongovernmental
organizations. All of these content areas are addressed by the PC curriculum and tracked with
those used in the teacher civic knowledge analysis. The items were not overly aligned with the
intervention and were based on established measures with known reliability. Each item had four
response categories and a “don’t know” option. One point was given for each correct item; no
points were given for wrong answers or “don’t know” responses. The civic knowledge items
were combined into pretest and posttest indexes. Scores on the pretest and posttest indexes
ranged from 0 to 20 points. The internal consistency reliability of the indexes based on
Cronbach’s α was acceptable for all three cohorts. (See Table 10.)
29
Table 10
Student Civic Knowledge Index Range and Reliability
Middle School
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
0-20
.67
.66
Cohort 2
0-20
.63
.66
Cohort 3
0-20
.68
.65
High School
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
0-20
.74
.76
Cohort 2
0-20
.72
.75
Cohort 3
0-20
.73
.77
Analysis
Middle school students who received the PC intervention gained more civic knowledge
than students in the control condition. (See Table 11.) The trend was apparent across all three
cohorts, although the size of the difference varied. The knowledge scores of the PC students
increased markedly from pretest to posttest. Scores improved 44% in cohort 1, 76% in cohort 2,
and 66% in cohort 3. The improvement index scores were +23, +43, and +24 percentile points,
respectively. In comparison, the control group students’ civic knowledge gains were smaller
across the board. Scores increased 33% from pretest to posttest in cohort 1, 17% in cohort 2, and
52% in cohort 3. The improvement index scores were +18, +11, and +19 percentile points.
Table 11
Middle School Students’ Civic Knowledge by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
4.73
4.35
4.15
5.88
3.68
3.22
Pretest SD
2.91
2.68
2.43
2.96
2.73
2.67
Posttest 𝜒
6.83
5.79
7.37
6.85
6.11
4.88
Posttest SD
3.30
3.02
3.06
3.32
3.00
2.96
𝜒 Difference
2.10
1.44
3.22
1.00
2.43
1.65
Sign. Difference
.00
.00
.00
.00
.00
.00
Percentage Change
44%
33%
76%
17%
66%
52%
Effect Size
.63
.48
1.51
.28
.67
.51
Improvement Index
+23
+18
+43
+11
+24
+19
Pre/Post Correlation
.44
.45
.72
.41
.22
.33
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
382
390
375
461
552
449
30
The knowledge scores of the middle school PC intervention and control groups were not
equivalent at baseline. The ANCOVA model estimates the adjusted posttest mean scores for each
group and the difference of posttest means. Posttest knowledge was the dependent variable in the
model, PC/control group was a fixed factor, and school and pretest knowledge were covariates.
The adjusted mean posttest scores for the students who received the PC intervention were
significantly higher than those of the control group across cohorts. The PC students’ posttest
score was 12% higher than that of the control group in cohort 1. The improvement index,
computed from the effect size based on Hedge’s g, was +10 percentile points. The PC group’s
adjusted posttest mean was 25% greater than the control group’s mean in cohort 2 and the
improvement index was +23 percentile points. In cohort 3, the percentage difference was 45%
and the improvement index was +35 percentile points. In all three models, pretest knowledge
was the strongest predictor of posttest knowledge. PC or control group condition was a
statistically significant predictor in all of the models. The school a student attended was a
significant variable only in cohort 3. (See Appendix B, Table B3.)
Consistent with the findings for middle school, civic knowledge gains were greater for
high school students who participated in PC than for control group students in cohorts 1 and 2.
(See Table 12.) The average scores of the PC students improved by 32% in cohort 1 and 56% in
cohort 2. In contrast, the control group mean scores increased by 17% and 7%, respectively. The
improvement index for the PC group in cohort 1 was +24 percentile points compared to +13
percentile points for the control group. In cohort 2, the improvement index for the PC group was
+44 percentile points versus +4 percentile points for the control group students. The percentage
change for both groups was 32% in cohort 3. However, the pretest/posttest mean difference was
higher for the PC group (2.03) than the control group (1.80). The improvement index for the PC
group was +22 percentile points compared to +16 percentile points for the comparison group
students.
Table 12
High School Student Civic Knowledge by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
7.77
6.61
6.96
7.18
6.35
5.70
Pretest SD
3.62
3.27
3.11
3.46
3.37
3.24
Posttest 𝜒
9.88
7.75
10.86
7.65
8.39
7.50
Posttest SD
3.83
3.65
3.29
4.31
3.79
3.82
𝜒 Difference
2.11
1.14
3.90
.47
2.03
1.80
Sign. Difference
.00
.00
.00
.01
.00
.00
Percentage Change
27%
17%
56%
7%
32%
32%
Effect Size
.66
.34
1.63
.12
.61
.44
Improvement Index
+24
+13
+44
+4
+22
+16
Pre/Post Correlation
.64
.54
.72
.53
.58
.35
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
642
340
462
365
564
376
31
The ANCOVA analysis for high school students revealed statistically significant
differences in adjusted mean posttest scores favoring the PC group. In cohort 1, the PC group’s
mean posttest score was 14% higher than that of the control group. The improvement index was
+14 percentile points. The percentage difference in PC/control group scores in cohort 2 was 31%
in cohort 2, with an improvement index of +35 percentile points. In cohort 3, the PC students’
adjusted posttest mean score was 16% greater than that of the control group, and the
improvement index was +14 percentile points. As was the case for the middle school model,
pretest knowledge had the strongest relationship to posttest knowledge. PC/control group was
statistically significant in all three models. School was a significant predictor in the model for
cohorts 2 and 3. (See Appendix B, Table B4.)
High-need students gained civic knowledge after their participation in PC in every
cohort. Paired samples t-tests were performed to estimate the mean differences between pretest
and posttest knowledge scores for students of color, ELLs, students with disabilities, students
performing below grade level, and students living in poverty. The percentage change in
pretest/posttest mean scores, effect size (Hedge’s g), and the improvement index were reported.
(See Table 13.) The pretest/posttest improvement in civic knowledge was statistically significant
(p≤.05) for high-need middle school students with two exceptions (ELLs in cohort 1 and students
with disabilities in cohort 2). The effect size (Hedge’s g) of the PC curriculum intervention for
high-need middle school students in cohorts 1 and 2 was lower than for all students in the PC
sample. In cohort 3, the effect sizes of the PC curriculum on high-need students were similar to
the effect size for all PC students. The gains in high-need high school students’ civic knowledge
were higher than for middle school students. The knowledge gains were statistically significant
for all categories of high-need students across the board. In cohorts 1 and 3, the effect sizes for
high school students of color, ELLs, students with disabilities, students performing below grade
level, and students living in poverty were higher than for all students in the PC sample. The
findings were mixed in cohort 2. The effect sizes for students of color and living in poverty
exceeded the effect size for the entire sample. The PC curriculum had a smaller effect on
students performing below grade level. The findings for ELLs and students with disabilities were
comparable to those for all students.
32
Table 13
PC Middle and High School Students’ Civic Knowledge
By High-Need Student Categories
Based on Difference of Means Tests
Cohort 1
Cohort 2
Cohort 3
Middle
School
High
School
Middle
School
High
School
Middle
School
High
School
Pre/Post % Change
All Students
44%
27%
76%
56%%
66%
32%
Students of Color
15%
29%
43%
31%
50%
35%
English Learners
10%
37%
38%
28%
51%
35%
Students w/ Disabilities
14%
31%
4%
22%
51%
46%
Below Grade Level
15%
33%
103%
18%
53%
44%
Living in Poverty
18%
31%
57%
32%
60%
36%
Effect Size
All Students
.63
.66
1.51
1.63
.67
.61
Students of Color
.30
.67
.50
.55
.61
.57
English Learners
.15a
.79
.41
.41
.63
.55
Students w/ Disabilities
.23
.61
.05a
.42
.64
.76
Below Grade Level
.23
.78
.67
.31
.62
.59
Living in Poverty
.33
.68
.61
.59
.69
.64
Improvement Index
All Students
+23
+24
+43
+44
+22
+19
Students of Color
+12
+25
+19
+21
+23
+22
English Learners
+6
+29
+16
+16
+24
+21
Students w/ Disabilities
+19
+23
+2
+16
+24
+28
Below Grade Level
+9
+28
+25
+12
+23
+22
Living in Poverty
+13
+15
+23
+22
+25
+24
aNot statistically significant at p≤.05
33
CIVIC DISPOSITIONS
Civic dispositions are the public and private traits, attitudes, and ingrained “habits of the
heart” that are consistent with the common good and are central to the functioning of a healthy
democracy (Tocqueville,1838; Branson, 1998; Crittenden and Levine, 2018). The Campaign for
the Civic Mission of Schools (2011) defines civic dispositions as a concern for others' rights and
welfare, fairness, reasonable levels of trust, and a sense of public duty. These traits are innate to
a sense of civic responsibility. People who evince a strong democratic temperament are willing
to compromise personal interests for the greater good (Stambler, 2011). They embrace their
democratic rights, responsibilities, and duties in a responsible, tolerant, and civil manner. They
have the confidence to engage in civic affairs and to participate actively in political life (Torney-
Purta and Amadeo, 2017). Civic dispositions include respect for the rule of law, a commitment to
justice, equality, and fairness, trust in government, civic duty, attentiveness to political matters,
political efficacy, political tolerance, respect for human rights, concern for the welfare of others,
civility, social responsibility, and community connectedness (Morgan and Streb, 2001; Torney-
Purta and Lopez, 2006; Campaign for the Civic Mission of Schools, 2011; National Council for
the Social Studies, 2013). These dispositions enable people to become independent members of
society who accept the moral and legal obligations of a democracy and take personal
responsibility for their actions (Kahne, et al., 2006). They encourage thoughtful and effective
participation in civic affairs. They require citizens to keep informed about politics and
government, monitor political leaders and public agencies to ensure their actions are principled,
and work through peaceful, legal means to change unjust policies (Branson, 1998).
Teacher Instructional Objectives: Developing Civic Dispositions
Civics instruction in elementary and secondary school can impart lasting democratic
proclivities and prime citizenship orientations that develop over a lifetime (Pasek et al., 2008;
Kahne and Sporte, 2008). Educating students about civic dispositions provides an instructional
opportunity that can be a gateway for making civic knowledge relevant and compelling
(Muetterties, DiGiacomo, and New, 2022). In project-based learning, a deliberate focus on
dispositions can deepen students’ understanding of the reasons behind their civic engagement
and create a more authentic experience (Levinson and Levine, 2013). Despite these possibilities,
civic dispositions often are sidelined in the curriculum. Most teachers desire to have their
students develop into good global citizens who are active and productive members of their
community (Kavanagh and Rich, 2018). However, they are hindered in this pursuit by the
emphasis on standardized testing that prioritizes memorization of content knowledge (Hansen, et
al., 2018; Gewertz, 2019). In addition, civic dispositions are more difficult to convey to students
than factual knowledge (Jamieson, 2013; Hansen, et al., 2018). These trends are reflected in the
fact that less than 20% of civic education studies focus on civic dispositions (Fitzgerald, et al.,
2021).
A central aim of the PCRP was to develop students’ civic dispositions. This objective
was a central theme in the Center’s teacher PD program at the summer institutes and the follow-
up sessions. Consistent with the assumptions of prior research, teachers in the PC and control
34
groups did not place a great deal of emphasis on developing students’ civic dispositions on the
pretest. After participating in the PCRP, teachers were more inclined to accentuate civic
dispositions in their lessons. This change was not found for the control group teachers.
Measurement
Teachers were asked on the pre- and post-program surveys how much emphasis they
placed on developing students’ dispositions to become involved in community affairs. The
response categories were 1 not much, 2 some, and 3 a great deal.
Analysis
PC teachers reported that they emphasized civic dispositions more in their classes after
participating in the program. (See Table 14.) The percentage of PC teachers indicating that they
did not place much emphasis on dispositions declined from pretest to posttest while the
percentage answering “a great deal” increased markedly. In cohort 1, 30% of PC teachers
responded “not much” on the pretest compared to 8% on the posttest. There was a shift from
21% to 46% of PC teachers who placed a great deal of emphasis on dispositions. In cohort 2, the
percentage of PC teachers indicating “not much” changed from 25% on the pretest to 6% on the
posttest. The percentage who placed a great deal of emphasis on civic disposition increased from
21% to 50%. In cohort 3, 48% of PC teachers answered that they did not emphasize dispositions
much on the pretest compared to 28% on the posttest. The number who paid a great deal of
attention to dispositions increased from 17% to 32%. The increase in teachers’ reported emphasis
on civic dispositions found for the PC group was not evident for the control group. The
percentage of control group teachers responding “a great deal” remained constant at 26% in
cohort 1, decreased from 30% to 25% in cohort 2, and went from 15% to 11% in cohort 3. The
percentage of control group teachers who indicated that they did not place much emphasis on
civic dispositions remained constant at 26% in cohort 1. The number who responded “not much”
increased in cohort 2 from 22% to 31% and in cohort 3 from 47% to 68%.
35
Table 14
Teachers’ Emphasis on Civic Dispositions
Project Citizen Teachers
Cohort 1
Cohort 2
Cohort 3
Pretest
Posttest
Pretest
Posttest
Pretest
Posttest
A Great Deal
21%
46%
21%
58%
17%
32%
Some
49%
46%
54%
36%
35%
40%
Not Much
30%
8%
25%
6%
48%
28%
n
36
36
34
34
39
39
Control Group Teachers
Cohort 1
Cohort 2
Cohort 3
Pretest
Posttest
Pretest
Posttest
Pretest
Posttest
A Great Deal
26%
26%
30%
25%
15%
11%
Some
33%
39%
48%
44%
38%
21%
Not Much
41%
35%
22%
31%
47%
68%
n
27
27
27
27
33
33
Paired sample t-tests indicated that PC teachers mean scores on the civic dispositions
indicator increased significantly from pretest to posttest in all study years. (See Table 15.) The
pretest/posttest mean differences were .40, .52, and .28. The percentage change was 20% in
cohort 1, 26% in cohort 2, and 17% in cohort 2. The effect sizes (Hedge’s g) were .38, .74, and
.29 corresponding to improvement index scores of +15, +27, and +11 percentile points. The
pretest/posttest mean difference for the control group was only statistically significant in cohort
3, and the change was negative.
Table 15
Developing Civic Dispositions by Condition
Difference of Means
Developing dispositions
to become involved in
community affairs
Cohort 1
Cohort 2
Cohort 3
PD
Control
PD
Control
PD
Control
Pretest 𝜒
1.96
1.82
2.00
2.18
1.67
1.71
Pretest SD
.67
.83
.68
.73
.74
.71
Posttest 𝜒
2.36
1.91
2.52
1.95
1.95
1.50
Posttest SD
.62
.79
.62
.78
.77
.74
𝜒 Difference
.40
.09
.52
-.23
.28
-.21
Sign. Difference
.00
NS
.00
NS
.04
.05
Percentage Change
20%
5%
26%
-11%
17%
-12%
Effect Size
.38
.09
.74
-.27
.29
-.30
Improvement Index
+15
+3
+27
-10
+11
-12
Pre/Post Correlation
.04
.85
.47
.43
.30
.56
Sign. Correlation
NS
.00
.00
.02
.03
.00
n
36
27
34
27
39
33
36
Student Civic Dispositions
Students’ development of civic dispositions was a focal outcome of the PCRP. Students
who participated in PC became more inclined to keep informed about government and to be
attentive to policy issues. Their sense of their responsibility to vote if given the opportunity
became more robust over time. A middle school PC student noted, “I learned a lot from this class
because I did not know anything about our government and many other things. I think it was
very important to take this class because it will prepare us for the future to participate in voting.”
A high school student stated, “Project Citizen has taught me so much about the government! I
never realized how important our government is until I got an in-depth lesson about it and
realized how strongly we rely on the government. It is important for us to vote and be involved.”
Students exhibited low levels of trust in government and the media commensurate with current
pervasive trends nationally. High school students’ trust increased slightly after PC, especially
their faith in the media.
Civic Responsibility
Civic responsibility is a broad term that encompasses traits associated with good
citizenship, civic duty, and civic engagement. It can be defined as “active participation in the
public life of a community in an informed, committed, and constructive manner, with a focus on
the common good” (Larson-Keagy, 2022: 16). People with a strong sense of civic responsibility
embrace the concept of participatory democracy, understand what constitutes the common good,
and can find a fair balance between rights and responsibilities. They are politically aware,
involved in community decision-making, and act to promote positive change. They recognize the
value and dignity of all people (Gottlieb and Robinson, 2002).
An index measuring students’ civic responsibility encompassed their propensity to keep
informed and actively engage in their community was created. There were no statistically
significant improvements in middle school students’ civic responsibility index scores over the
course of the study for both the PC and control groups. There were modest, statistically
significant increases in civic responsibility for high school students who participated in PC. The
findings were weaker for the control group students.
Measurement
A civic responsibility index was created from two measures that asked students how
much of a responsibility they felt to 1) follow news about government every day and 2) get
actively involved in their community. The response options were 1 not much responsibility, 2 a
great deal of responsibility, and 3) a top priority. The additive index ranged from 1 (not much
responsibility) to 5 (a top priority). The index reliability (Cronbach’s α) was acceptable across
the cohorts. (See Table 16.)
37
Table 16
Civic Responsibility Index Range and Reliability
Middle School
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
1-5
.65
.66
Cohort 2
1-5
.70
.70
Cohort 3
1-5
.64
.65
High School
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
1-5
.66
.72
Cohort 2
1-5
.69
.73
Cohort 3
1-5
.65
.72
Analysis
Middle school students’ sense of their responsibility to keep informed and become
actively engaged in the community did not change over the course of the study. (See Table 17.)
Students’ average scores on the pretest and posttest hovered around 2.00, which is below the
midpoint of the index. The pretest/posttest difference in mean scores on the civic duty index was
not statistically significant for the PC or the control group students for all three cohorts. The
ANCOVA analysis compared the adjusted posttest means of the middle school PC and control
group students. There were virtually no differences in the adjusted posttest mean scores between
the groups for any of the cohorts. (See Appendix B, Table B5.)
Table 17
Middle School Students’ Civic Responsibility by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
1.92
2.06
1.99
2.02
1.92
1.85
Pretest 𝜒
1.09
1.22
1.21
1.22
1.14
1.11
Pretest SD
1.97
1.97
2.03
1.95
1.98
1.94
Posttest 𝜒
1.09
1.17
1.22
1.14
1.13
1.12
Posttest SD
.05
-.09
.04
-.07
.05
.09
𝜒 Difference
NS
NS
NS
NS
NS
NS
Percentage Change
3%
-4%
2%
-3%
3%
4%
Effect Size
.04
-.06
.03
-.05
.04
.06
Improvement Index
+2
-2
+1
-2
+2
+6
Pre/Post Correlation
.31
.32
.36
.31
.31
.25
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
303
346
334
421
471
344
38
Small, statistically significant gains in a sense of civic responsibility were found for high
school students who participated in PC. (See Table 18.) As was the case for middle schoolers, the
mean scores for high school students were positioned below the midpoint of the index. The effect
size for PC students increased for each subsequent cohort, ranging from .07 in cohort 1, .16 in
cohort 2, and .22 in cohort 3. The improvement index scores were +3, +6, and +9 percentile
points. The control group pretest/posttest mean difference was nonsignificant in cohort 1 and was
smaller than for the PC group in cohorts 2 and 3. The results of the ANCOVA analysis showed
that the adjusted posttest difference of means for the high school PC and control group students
were minimal and nonsignificant for all three cohorts. (See Appendix B, Table B6.)
Table 18
High School Students’ Civic Responsibility by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
2.09
2.15
1.86
1.97
1.77
1.80
Pretest SD
1.15
1.27
1.12
1.11
1.03
1.04
Posttest 𝜒
2.18
2.14
2.06
2.10
2.07
2.05
Posttest SD
1.26
1.29
1.19
1.27
1.26
1.18
𝜒 Difference
.09
-.01
.20
.13
.29
.24
Sign. Difference
.05
NS
.00
.04
.00
.00
Percentage Change
4%
<1%
11%
6%
17%
14%
Effect Size
.07
.01
.16
.10
.22
.18
Improvement Index
+3
0
+6
+4
+9
+7
Pre/Post Correlation
.43
.47
.42
.38
.37
.32
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
639
293
426
312
523
328
Attention to Issues
The PC curriculum has students identify a problem to work on in their community or
school which requires that they become aware of issues that need to be addressed. Middle school
students in cohorts 1 and 3 tended to pay significantly greater attention to issues following their
PC experience. A PC middle school student stated, “It’s interesting to know that someone is
concerned about a teen’s views about politics. Personally, I didn’t care about government-related
or political stuff, but in the future, I can see myself becoming more aware.” A high school
student commented, “Project Citizen has honestly made me think more deeply and delve into
politics more and more.” Control group students’ issue attention tended to decline over time.
Measurement
Two items measured students’ attention to issues before and after their PC or traditional
civics class: 1) How much attention do you pay to issues that are affecting your community? and
39
2) How much attention do you pay to issues that are affecting your school? The response
categories were 1 not very much attention, 2 some attention, and 3 a lot of attention. The two
items were added to form an attention index which ranged from 1 (not much attention) to 5 (a lot
of attention). The reliability (Cronbach’s α) was acceptable across the three cohorts. (See Table
19.)
Table 19
Attention Index Range and Reliability
Middle School
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
1-5
.69
.62
Cohort 2
1-5
.61
.68
Cohort 3
1-5
.59
.61
High School
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
1-5
.67
.67
Cohort 2
1-5
.66
.71
Cohort 3
1-5
.65
.68
Analysis
Middle school students’ average levels of issue attentiveness were near the midpoint of
the attention scale. Middle schoolers participating in PC became significantly more attentive to
issues in their community and school in cohorts 1 and 3. (See Table 20.) The gains were small,
with pretest/posttest mean differences of .14 and .10 and corresponding percentage changes of
4% and 3%. The effect sizes (Hedge’s g) were .12 and .08, and the improvement index scores
were +5 and +3. Attention to issues decreased significantly among the control group middle
school students in every cohort. The pretest/posttest mean differences were -.22, -.20, and -.19.
The mean values decreased by 9%, 6%, and 6%. The effect sizes were -.22, -.16, and -.13, which
corresponded to improvement index scores of -9, -6, and -5. The ANCOVA analysis found that
the adjusted posttest means of the PC middle school students were significantly higher than those
of the control group students in cohort 1, but not in cohorts 2 and 3. (See Appendix B, Table B7.)
40
Table 20
Middle School Students’ Attention to Issues by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
3.19
3.24
3.17
3.26
2.98
3.12
Pretest SD
1.11
1.17
1.19
1.20
1.12
1.12
Posttest 𝜒
3.33
2.95
3.13
3.06
3.08
2.93
Posttest SD
1.11
1.19
1.17
1.15
1.11
1.11
𝜒 Difference
.14
-.28
-.04
-.20
.10
-.19
Sign. Difference
.00
.00
NS
.00
.03
.01
Percentage Change
4%
-9%
-1%
-6%
3%
-6%
Effect Size
.12
-.22
-.03
-.16
.08
-.13
Improvement Index
+5
-9
-1
-6
+3
-5
Pre/Post Correlation
.47
.38
.32
.41
.30
.21
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
341
353
357
447
518
393
High school students’ mean scores on issue attentiveness were near the midpoint of the
index. (See Table 21.) PC students’ attention to issues increased significantly in cohorts 1 and 3,
but not in cohort 2. The pretest/posttest mean differences were small at .09 for cohort 1 and .11
for cohort 3. The percentage increase was 3% and 4%, respectively. The effect sizes (Hedge’s g)
were .08 and .09, which corresponded to improvement index scores of +3 and +4 percentile
points. The high school control group pretest/post mean difference was only statistically
significant in cohort 1 where the average attention to issues declined. The mean difference was
-.26, the percentage change was -8%, the effect size was -.21, and the improvement index score
was -.8 percentile points. The ANCOVA analysis established that there were no statistically
significant differences in adjusted mean posttest scores between the PC and control groups for
any cohort. (See Appendix B, Table B8.)
41
Table 21
High School Students’ Attention to Issues by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
3.23
3.37
3.10
3.25
2.90
2.90
Pretest SD
1.16
1.13
1.21
1.14
1.12
1.08
Posttest 𝜒
3.32
3.10
3.15
3.18
3.01
2.98
Posttest SD
1.18
1.18
1.16
1.19
1.14
1.12
𝜒 Difference
.09
-.26
.05
-.07
.11
.07
Sign. Difference
.02
.00
NS
NS
.02
NS
Percentage Change
3%
-8%
2%
-2%
4%
3%
Effect Size
.08
-.21
.03
-.05
.09
.06
Improvement Index
+3
-8
+1
-2
+4
+4.
Pre/Post Correlation
.48
.41
.44
.34
.36
.29
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
648
306
450
335
542
346
Duty to Vote
Middle and high school students generally had a strong sense of duty to vote prior to
taking their PC or traditional civics class. Still, PC students’ sense of duty to vote increased
significantly after participating in the program for all three cohorts. For the most part, the control
group middle and high school students’ sense of duty to vote did not change from pretest to
posttest.
Measurement
Students were asked how responsible they felt to exercise their right to vote in election if
they were eligible. The item was coded 1 not much responsibility, 2 a great deal of responsibility,
and 3 a top priority.
Analysis
Middle school students’ sense of their duty to vote was high from the outset for PC and
control group students. (See Table 22.) The percentage of PC middle schoolers indicating that
their responsibility to vote was a top priority was 28% in all three cohorts. The number increased
to 39% in cohort 1 and 36% in cohorts 2 and 3. The percentage of PC students indicating that
they didn’t have much responsibility to vote declined in each study year. In contrast, there was
little change from pretest to posttest for the control group middle school students.
42
Table 22
Middle School Students’ Duty to Vote
Project Citizen Middle School Students
Duty to Vote
Cohort 1
Cohort 2
Cohort 3
Pretest
Posttest
Pretest
Posttest
Pretest
Posttest
A Top Priority
28%
39%
28%
36%
28%
36%
A Great Deal
39%
38%
38%
35%
37%
35%
Not Much
33%
23%
34%
29%
35%
29%
n
341
341
357
357
518
518
Control Group Middle School Students
Duty to Vote
Cohort 1
Cohort 2
Cohort 3
Pretest
Posttest
Pretest
Posttest
Pretest
Posttest
A Top Priority
29%
31%
30%
30%
30%
32%
A Great Deal
40%
39%
36%
41%
36%
37%
Not Much
31%
31%
34%
29%
34%
31%
n
353
353
447
447
518
518
Difference of means tests indicated that the PC students’ average scores on duty to vote
improved significantly from pretest to posttest across all cohorts. (See Table 23.) The percentage
change for the PC group were 10%, 6%, and 10%, respectively. The effect sizes (Hedge’s g)
were small at .24, .14, and .19. The corresponding improvement index scores were +9, +6, and
+8 percentile points. The pretest/posttest differences in duty to vote for the control group were
negligible and nonsignificant across the board. The ANCOVA model found a statistically
significant difference in the adjusted mean posttest scores between the middle school PC and
control group students in cohort 1. The PC group’s adjusted mean score was higher than that of
the control group. The percentage difference was 7%, the effect size (Hedge’s g) was .19, and the
improvement index was +8 percentile points. The adjusted mean differences between the PC and
control group scores were small and nonsignificant for cohorts 2 and 3. (See Appendix B, Table
B9.)
43
Table 23
Middle School Students’ Duty to Vote by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
1.97
1.98
1.93
1.98
1.73
1.72
Pretest SD
.77
.77
.78
.81
.75
.76
Posttest 𝜒
2.17
2.02
2.05
2.02
1.91
1.76
Posttest SD
.77
.77
.81
.78
.78
.72
𝜒 Difference
.19
.04
.12
.04
.18
.04
Sign. Difference
.00
NS
.00
NS
.00
NS
Percentage Change
10%
2%
6%
2%
10%
2%
Effect Size
.24
.04
.14
.04
.19
.04
Improvement Index
+9
+2
+6
+2
+8
+2
Pre/Post Correlation
.44
.41
.39
.39
.25
.19
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
308
348
349
431
502
374
High school students were more inclined to consider voting a top priority than middle
school students as the prospect of voting was imminent for them. (See Table 24.) The percentage
of PC high school students who prioritized voting increased in each study year. The number of
PC high schoolers responding “a top priority” rose from 45% to 52% in cohort 1 and from 37%
to 39% in cohorts 2 and 3 while the percentage indicating “not much” dropped. The control
group high school students prioritized voting less from pretest to posttest in cohort 1. The
percentages remained relatively stable in cohorts 2 and 3.
Table 24
Middle School Students’ Duty to Vote
Project Citizen High School Students
Cohort 1
Cohort 2
Cohort 3
Pretest
Posttest
Pretest
Posttest
Pretest
Posttest
A Top Priority
45%
52%
37%
39%
37%
39%
A Great Deal
35%
32%
35%
40%
34%
39%
Not Much
20%
16%
28%
21%
29%
22%
n
648
648
450
450
542
542
Control Group High School Students
Cohort 1
Cohort 2
Cohort 3
Pretest
Posttest
Pretest
Posttest
Pretest
Posttest
A Top Priority
47%
43%
34%
35%
34%
35%
A Great Deal
33%
34%
43%
40%
43%
40%
Not Much
20%
23%
23%
25%
23%
25%
n
306
306
335
335
346
346
44
On average, high school students in both the PC and control group felt a great deal of
responsibility to vote prior to their civics classes. (See Table 25.) The PC groups’ mean scores on
duty to vote increased slightly from pretest to posttest. The difference of means was statistically
significant for cohorts 1 and 3; it approached significance for cohort 2. The control group
differences were nonsignificant for cohorts 1 and 2. In cohort 3, the pretest/posttest improvement
in mean scores on duty to vote was slightly higher for the control group than the PC group. The
ANCOVA analysis showed that the difference in adjusted posttest mean scores on duty to vote
was higher for the PC group than the control group. The difference of adjusted means was
statistically significant for cohort 1, approached significance for cohort 2, and was nonsignificant
for cohort 3. While the effect sizes are small, the improvement index scores are +6, +9, and +10
percentile points. (See Appendix B, Table B10.)
Table 25
High School Students’ Duty to Vote by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
2.27
2.27
2.09
2.11
2.06
1.93
Pretest SD
.76
.77
.80
.74
.77
.79
Posttest 𝜒
2.34
2.22
2.15
2.11
2.15
2.04
Posttest SD
.75
.78
.77
.77
.77
.80
𝜒 Difference
.07
-.05
.06
.00
.09
.11
Sign. Difference
.01
NS
.09
NS
.01
.01
Percentage Change
3%
-2%
3%
0%
4%
6%
Effect Size
.09
-.06
.06
.00
.10
.11
Improvement Index
+4
-2
+2
.00
+4
+4
Pre/Post Correlation
.46
.37
.49
.42
.36
.35
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
641
294
441
322
537
343
Future Government Service
In general, middle school students were only modestly interested in a career in
government and politics or running for office one day. Middle school students in both the PC and
control groups were no more likely to desire a career in government service at the conclusion of
the study than they had been at the outset, with one exception. In contrast, high school PC
students became significantly more interested in pursuing a future career in government service
after participating in the program. This finding is not surprising given that government service is
a more compelling option for high school students who may soon be entering the workforce.
Measurement
Students were asked how much they agreed on two items related to their interest in
government service: 1) I am interested in a career in government and politics and 2) I may run
45
for office one day. The response categories were 1 strongly disagree, 2 disagree, 3 neither agree
nor disagree, 4 agree, 5 strongly agree. The items were combined to form a government service
index that ranged from 1 (not interested) to 9 (very interested). The index reliability (Cronbach’s
α) was acceptable for all cohorts. (See Table 26.)
Table 26
Attention Index Range and Reliability
Middle School
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
1-9
.77
.81
Cohort 2
1-9
.80
.79
Cohort 3
1-9
.77
.80
High School
Index Range
Pretest Cronbach’s α
Posttest Cronbach’s α
Cohort 1
1-9
.84
.84
Cohort 2
1-9
.86
.87
Cohort 3
1-9
.82
.84
Analysis
Middle school students’ means scores on the government service index were in the
bottom third of the scale. Few students expressed a strong interest in having a career in
government and politics or one day running for office. (See Table 27.) There were no differences
in the pretest/posttest mean values for either the PC or control group students in cohort 1. PC
students in cohort 2 became more interested in government service after participating in the
program. The difference in pretest/posttest means of .32 was statistically significant and
represented a 9% increase. The effect size (Hedge’s g) was .14 which corresponded to an
improvement index score of +6 percentile points. The control group means did not change from
pretest to posttest. In cohort 3, the small pretest/posttest mean difference was not statistically
significant. However, the mean difference of .27 for the control group was statistically
significant. The percentage change was 8%, the effect size was .11, and the improvement index
was +4 percentile points. The ANCOVA analysis of middle school students’ future government
service found a statistically significant difference between the PC and control group adjusted
posttest means for cohort 2. The adjusted posttest mean was .42 in favor of the PC group. There
was an 11% difference between the group means. The effect size was .20 and the improvement
index score was +8 percentile points. (See Appendix B, Table B11.)
46
Table 27
Middle School Students’ Interest in Government Service by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
3.35
3.33
3.70
3.65
3.74
3.52
Pretest SD
2.10
2.04
2.16
2.01
2.15
2.10
Posttest 𝜒
3.35
3.30
4.02
3.65
3.84
3.80
Posttest SD
2.08
2.01
2.10
2.13
2.22
2
𝜒 Difference
.00
-.03
.32
.00
.10
.27
Sign. Difference
NS
NS
.00
NS
NS
.03
Percentage Change
0
-<1%
9%
0
3%
8%
Effect Size
0
-.02
.14
0
.04
.11
Improvement Index
0
-1
+6
0
+2
+4
Pre/Post Correlation
.46
.45
.47
.47
.39
.27
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
321
343
338
424
493
343
High school students’ interest in future government service was similar to that of middle
school students at the outset. The average scores were in the lower third of the index range across
the three cohorts. (See Table 28.) However, interest in government service increased
significantly for both the PC and control group students over the course of the study. The change
was greater for the PC group than the control group across the board. The difference was
smallest in cohort 1. The pretest/posttest mean difference for the PC group was .26, the
percentage change was 8%, the effect size was .13, and the improvement index score was +5.
The control group mean difference was .20, the percentage change was 6%, the effect size was
.10, and the improvement index score was +4 percentile points. The pretest/posttest mean
differences for the PC groups in cohorts 2 and 3 were .61 and .75, respectively. In cohort 2, the
PC group had a 20% increase in means from pretest to posttest, an effect size of .27, and an
improvement index of +11 percentile points. In cohort 3, there was a 24% increase in
pretest/posttest mean values, the effect size was .31, and the improvement index score was +12
percentile points. The pretest/posttest mean for the control groups increased by 12% in cohort 2
and 9% in cohort 3. The effect sizes were .16 and .12 which corresponded to improvement index
scores of +6 and +5. The ANCOVA analysis confirmed that the small difference in adjusted
posttest means in cohort 1 was not statistically significant. The PC high school students’ adjusted
posttest means were significantly higher than those of the control group in cohorts 2 and 3. (See
Appendix B, Table B12.)
47
Table 28
High School Students’ Future Government Service by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
3.45
3.31
3.07
3.03
3.11
3.23
Pretest SD
2.25
2.11
2.06
2.15
2.03
2.09
Posttest 𝜒
3.72
3.51
3.68
3.38
3.86
3.52
Posttest SD
2.33
2.23
2.32
2.28
2.41
2.23
𝜒 Difference
.26
.20
.61
.34
.75
.28
Sign. Difference
.00
.01
.00
.00
.00
.00
Percentage Change
8%
6%
20%
12%
24%
9%
Effect Size
.13
.10
.27
.16
.31
.12
Improvement Index
+5
+4
+11
+6
+12
+5
Pre/Post Correlation
.61
.62
.46
.53
.39
.43
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
641
294
437
322
528
339
Trust in Government
Public trust in government has been on the decline for decades. Military conflicts,
scandals involving institutions and officials, negative perceptions of the economy, fear of crime,
concerns with domestic issues are some of the factors contributing to plummeting trust (Chanley,
Rudolph, and Rahn, 2000; Gershon, 2016). The media’s negative and sensational coverage of
government and politics contributes to the public’s negative views (Kavanagh, et al., 2020).
Despite a slight uptick during the first two years of the pandemic, less than one-fifth of
Americans felt that they could trust the government in Washington to do what is right most of
the time during the period of this study (Pew Research Center, 2023). Middle school students in
the PC and control groups exhibited low levels of trust in government which persisted from
pretest to posttest. PC high school students’ trust in government increased significantly in
cohorts 1 and 3.
Measurement
Trust in government was measured by the item: I trust government officials to do what is
right most of the time. The response categories were 1 strongly disagree, 2 disagree, 3 neither
agree nor disagree, 4 agree, 5 strongly disagree.
Analysis
Middle school students’ trust in government remained stable over the course of the study
for both the PC and control group students. (See Table 29.) The mean scores hovered around 3,
which was in the middle of the distribution. The ANCOVA analysis found no statistically
significant differences in the adjusted posttest mean scores on trust in government between the
middle school PD and control groups. (See Appendix B, Table B13.)
48
Table 29
Middle School Students’ Trust in Government by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
3.13
3.03
3.16
3.01
3.07
2.96
Pretest SD
1.00
1.02
1.01
.98
1.03
1.02
Posttest 𝜒
3.14
2.99
3.25
2.98
3.14
2.98
Posttest SD
.98
1.03
.97
.96
1.07
1.06
𝜒 Difference
.01
-.04
.09
-.03
.07
.02
Sign. Difference
NS
NS
NS
NS
NS
NS
Percentage Change
0%
-1%
3%
-1%
2%
1%
Effect Size
.00
-.03
.08
-.03
.05
.01
Improvement Index
0
-1
+3
-1
+2
0
Pre/Post Correlation
.32
.40
.34
.35
.25
.20
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
304
342
339
423
496
360
The average levels of trust in government were slightly lower for high school students
than middle school students on the pretest and were below the midpoint of the scale. The mean
levels of trust for high school students who participated in PC increased significantly in cohorts 1
and 3. (See Table 30.) The pretest/posttest mean difference in cohort 1 was .12, with a 5%
improvement in trust. The effect size (Hedge’s g) was .11 and the improvement index score was
+4 percentile points. The increase was larger for PC students in cohort 3, with a pretest/posttest
mean difference of .22 and a percentage change of 8%. The effect size was .18 and the
improvement index was +7. The results of the ANCOVA analysis showed no statistically
significant differences in the adjusted posttest means of the PC and control groups for any
cohort. (See Appendix C, Table B14.)
49
Table 30
High School Students’ Trust in Government by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
2.76
2.87
2.78
2.85
2.71
2.72
Pretest SD
.99
1.17
1.03
1.00
.99
1.02
Posttest 𝜒
2.89
2.83
2.85
2.91
2.94
2.76
Posttest SD
1.00
1.15
1.01
1.04
1.06
1.03
𝜒 Difference
.12
-.04
.06
.06
.22
.04
Sign. Difference
.00
NS
NS
NS
.00
NS
Percentage Change
5%
-1%
3%
2%
8%
1%
Effect Size
.11
-.04
.05
.06
.18
.03
Improvement Index
+4
-2
+2
+2
+7
+1
Pre/Post Correlation
.38
.50
.37
.39
.33
.42
Sign. Correlation
.00
.00
.00
.00
.00
.00
n
634
289
434
319
529
360
Trust in Media
The public’s trust in media reached historic lows in 2020, when the percentage of people
with no trust in the news surpassed that of people with at least some confidence. Trust in the
media remained low throughout the COVID-19 pandemic even as people’s reliance on the news
increased. The public feels that the media are not fulfilling their role in democracy to provide
accurate information that they can use in decision making. People have difficulty sorting the
facts from the glut of misinformation they receive. The perception that reporting is inaccurate,
the news reflects deep partisan biases, and that news organizations’ business priorities
overwhelm their public service imperative is driving down media trust. Half of Americans
believe that the news media deliberately sets out to deceive them (Knight Foundation, 2023).
Young people now get most of their news from social media and trust it more than national news
media (Liedke and Gottfried, 2022).
While students in the study had low levels of trust in government, they trusted the news
media even less. Middle school students’ distrust remained constant over time. High school
students’ trust in the news media increased slightly. The increase was greater than for trust in
government.
Measurement
Trust in media was measured by students’ agreement with the statement: I trust the news
media. The response categories were 1 strongly disagree, 2 disagree, 3 neither agree nor
disagree, 4 agree, 5 strongly disagree.
50
Analysis
PC and control group middle school students’ trust in the media was somewhat lower
than their trust in government. The mean levels of trust remained consistent from pretest to
posttest for the PC group. (See Table 31.) None of the PC groups’ pretest/posttest mean
differences were statistically significant. There was a slight, statistically significant decline in
media trust for the control groups in cohorts 2 and 3. The ANCOVA analysis found no
statistically significant differences in the adjusted posttest means between the middle school PC
and control groups. (See Appendix B, Table B15.)
Table 31
Middle School Students’ Media Trust by Condition
Difference of Means
Cohort 1
Cohort 2
Cohort 3
PC
Control
PC
Control
PC
Control
Pretest 𝜒
2.70
2.77
2.87
2.87
2.82
2.90
Pretest SD
1.04
1.00
1.00
.98
.95
1.00
Posttest 𝜒
2.78
2.74
2.95
2.77
2.86
2.77
Posttest SD
.98
1.00
.97
.88
1.05
.97
𝜒 Difference
.08
-.03
.08
-.10
.04
-.13
Sign. Difference
NS
NS
NS
.05
NS
.03
Percentage Change
3%
-1%
3%
-3%
1%
-4%
Effect Size
.07
-.03
.06
-.08
.03
-.10
Improvement Index
+3
-1
+2
-3
+1
-4
Pre/Post Correlation
.34
.39
.23
.28
.16
.13
Sign. Correlation
.00
.00
.00
.00
.00
.01
n
304
342
339
423
496
360
High school students’ average trust in media was lower than that of middle school
students. However, high school students’ media trust increased from pretest to posttest for both
the PC and control groups. (See Table 32.) The difference in mean pretest/posttest scores for the
PC group was .15 in cohort 1, .13 in cohort 2, and .18 in cohort 3. The percentage change was
6%, 5%, and 8%, respectively. The effect sizes (Hedge’s g) were .14, .12, and .15 corresponding
to improvement index scores of +6, +5, and +6 percentile points. The control group high school
students’ mean scores improved by .13, .24, and .13 across cohorts. The percentage change was
5%, 9%, and 5%. The effect sizes were .10, .22, and .11 with improvement index scores of +4,
+8, and +4 percentile points. There were no significant differences in adjusted posttest mean
scores on media trust based on the ANCOVA analysis. (See Appendix B, Table B16.)
51
Table 32
High School Students’ Media Trust by Condition
Difference of Means